From: Hannes Laimer <h.laimer@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH proxmox v4 02/45] ebpf: policy: add the enforcement subsystem for guest interfaces
Date: Thu, 5 Nov 2026 12:12:41 +0100 [thread overview]
Message-ID: <20261105111324.706922-3-h.laimer@proxmox.com> (raw)
In-Reply-To: <20261105111324.706922-1-h.laimer@proxmox.com>
The subsystem, behind the policy feature, decides which frames reach
an assigned guest NIC. On ingress it stamps the NIC's id into the low
16 bits of the packet mark, the part VXLAN GBP carries to other hosts.
On the egress of the destination NIC both ends are known, so it looks
up the pair of the two ids and drops the frame on a miss or a deny. A
sender without an id counts as id zero. A full pass applies the whole
policy to the host, every pair and every assigned NIC. A plug applies
it to one NIC as it comes up. A pair can carry checks that narrow its
verdict to protocols and ports. The first seven sit in the pair's own
value, the rest go to a ruleset shared by the pairs with the same
remaining checks.
A pair with checks drops a frame whose transport header cannot be
read, since a check could deny what it hides. ARP and the
solicitations and advertisements of neighbor discovery pass in both
directions of a pair that allows anything, otherwise the two ends
could never resolve each other. They also pass when the host sends
them, which is how a gateway on the host or a bridge with ARP
suppression answers. Behind a firewall bridge the host's frames arrive
through the bridge's veth like any other, so a NIC there takes them
from any sender without an id. An unstamped frame that leaves on the
interface it came in on passes, whatever it carries. Only a program
ahead of the policy on that interface can turn a frame around
unstamped, so it is the host's own answer.
A plug cannot install the programs, only a pass can. So a pass loads
the programs as soon as the policy has an assignment, even one that
covers no NIC of this host. A pass for a policy without any assignment
unloads the programs and their maps again. A plug also drops the pins
and entries of interfaces that are gone, since only a pass would
otherwise remove them and a node that starts guests for long without
one would fill the interface map.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
proxmox-ebpf/Cargo.toml | 1 +
proxmox-ebpf/debian/control | 18 +
proxmox-ebpf/src/bpf-shim/bpf/bpf_helpers.h | 1 +
proxmox-ebpf/src/bpf-shim/linux/bpf.h | 1 +
proxmox-ebpf/src/lib.rs | 3 +
proxmox-ebpf/src/policy/bpf/mark.h | 27 +
proxmox-ebpf/src/policy/bpf/policy.bpf.c | 367 +++++++++
proxmox-ebpf/src/policy/bpf/types.h | 72 ++
proxmox-ebpf/src/policy/mod.rs | 807 ++++++++++++++++++++
proxmox-ebpf/src/policy/types.rs | 155 ++++
proxmox-ebpf/src/subsystem.rs | 23 +-
proxmox-ebpf/tests/common/mod.rs | 36 +
proxmox-ebpf/tests/policy.rs | 795 +++++++++++++++++++
13 files changed, 2305 insertions(+), 1 deletion(-)
create mode 100644 proxmox-ebpf/src/policy/bpf/mark.h
create mode 100644 proxmox-ebpf/src/policy/bpf/policy.bpf.c
create mode 100644 proxmox-ebpf/src/policy/bpf/types.h
create mode 100644 proxmox-ebpf/src/policy/mod.rs
create mode 100644 proxmox-ebpf/src/policy/types.rs
create mode 100644 proxmox-ebpf/tests/policy.rs
diff --git a/proxmox-ebpf/Cargo.toml b/proxmox-ebpf/Cargo.toml
index d4607d83e6ff..74755994dfe5 100644
--- a/proxmox-ebpf/Cargo.toml
+++ b/proxmox-ebpf/Cargo.toml
@@ -11,6 +11,7 @@ repository.workspace = true
rust-version.workspace = true
[features]
+policy = ["nix/net"]
# When enabled, BPF programs compile with -DBPF_DEBUG so their DBG() macros
# expand to bpf_printk. Off by default
bpf-debug = []
diff --git a/proxmox-ebpf/debian/control b/proxmox-ebpf/debian/control
index d51e260ef386..76db90f6f813 100644
--- a/proxmox-ebpf/debian/control
+++ b/proxmox-ebpf/debian/control
@@ -34,6 +34,8 @@ Depends:
clang-21,
libbpf-dev,
proxmox-kernel-libc-dev
+Suggests:
+ librust-proxmox-ebpf+policy-dev (= ${binary:Version})
Provides:
librust-proxmox-ebpf+bpf-debug-dev (= ${binary:Version}),
librust-proxmox-ebpf+default-dev (= ${binary:Version}),
@@ -48,3 +50,19 @@ Provides:
librust-proxmox-ebpf-0.1.0+default-dev (= ${binary:Version})
Description: Proxmox VE eBPF - Rust source code
Source code for Debianized Rust crate "proxmox-ebpf"
+
+Package: librust-proxmox-ebpf+policy-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-proxmox-ebpf-dev (= ${binary:Version}),
+ librust-nix-0.29+fs-dev,
+ librust-nix-0.29+net-dev
+Provides:
+ librust-proxmox-ebpf-0+policy-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1+policy-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1.0+policy-dev (= ${binary:Version})
+Description: Proxmox VE eBPF - feature "policy"
+ This metapackage enables feature "policy" for the Rust proxmox-ebpf crate, by
+ pulling in any additional dependencies needed by that feature.
diff --git a/proxmox-ebpf/src/bpf-shim/bpf/bpf_helpers.h b/proxmox-ebpf/src/bpf-shim/bpf/bpf_helpers.h
index b4bcfdf5717f..09b74fe67bb0 100644
--- a/proxmox-ebpf/src/bpf-shim/bpf/bpf_helpers.h
+++ b/proxmox-ebpf/src/bpf-shim/bpf/bpf_helpers.h
@@ -30,5 +30,6 @@ extern long bpf_skb_store_bytes(void *skb, __u32 offset, const void *from, __u32
extern long bpf_skb_pull_data(void *skb, __u32 len);
extern long bpf_skb_change_tail(void *skb, __u32 new_len, __u64 flags);
extern long bpf_redirect(__u32 ifindex, __u64 flags);
+extern long bpf_loop(__u32 nr_loops, void *callback_fn, void *callback_ctx, __u64 flags);
#endif
diff --git a/proxmox-ebpf/src/bpf-shim/linux/bpf.h b/proxmox-ebpf/src/bpf-shim/linux/bpf.h
index ff259c329522..19c9ca217b45 100644
--- a/proxmox-ebpf/src/bpf-shim/linux/bpf.h
+++ b/proxmox-ebpf/src/bpf-shim/linux/bpf.h
@@ -14,6 +14,7 @@ struct __sk_buff {
__u32 len;
__u32 ifindex;
__u32 mark;
+ __u32 ingress_ifindex;
};
// map-definition constants, the values are irrelevant natively
diff --git a/proxmox-ebpf/src/lib.rs b/proxmox-ebpf/src/lib.rs
index ab0763ab3d3d..d035131c6dee 100644
--- a/proxmox-ebpf/src/lib.rs
+++ b/proxmox-ebpf/src/lib.rs
@@ -2,5 +2,8 @@
//! eBPF subsystems, one per concern, each behind a cargo feature of its name. Nothing here runs
//! on its own, consumers pull in only the subsystem they drive.
+#[cfg(feature = "policy")]
+pub mod policy;
+
pub mod subsystem;
pub mod tc;
diff --git a/proxmox-ebpf/src/policy/bpf/mark.h b/proxmox-ebpf/src/policy/bpf/mark.h
new file mode 100644
index 000000000000..1eeee87c6eca
--- /dev/null
+++ b/proxmox-ebpf/src/policy/bpf/mark.h
@@ -0,0 +1,27 @@
+#ifndef PROXMOX_EBPF_POLICY_MARK_H
+#define PROXMOX_EBPF_POLICY_MARK_H
+
+// Include after <linux/bpf.h> and <bpf/bpf_helpers.h>, which provide struct
+// __sk_buff, the __uNN types, __always_inline, and barrier_var().
+//
+// skb->mark is shared with the rest of the host. The policy programs own only
+// the low 16 bits, which hold the wire id and which VXLAN GBP carries between
+// hosts.
+#define POLICY_MARK_MASK 0xffffu
+
+static __always_inline void policy_mark_set(struct __sk_buff *skb, __u16 id) {
+ __u32 mark = skb->mark;
+ mark = (mark & ~POLICY_MARK_MASK) | id;
+ // The verifier takes a write to skb->mark only at its full 32-bit width.
+ // Without the barrier clang turns the read-modify-write into a 16-bit
+ // store of the low half, which the verifier rejects as an invalid context
+ // access.
+ barrier_var(mark);
+ skb->mark = mark;
+}
+
+static __always_inline __u16 policy_mark_get(struct __sk_buff *skb) {
+ return (__u16)skb->mark;
+}
+
+#endif
diff --git a/proxmox-ebpf/src/policy/bpf/policy.bpf.c b/proxmox-ebpf/src/policy/bpf/policy.bpf.c
new file mode 100644
index 000000000000..ed189a1227e1
--- /dev/null
+++ b/proxmox-ebpf/src/policy/bpf/policy.bpf.c
@@ -0,0 +1,367 @@
+#include <linux/bpf.h>
+#include <linux/if_ether.h>
+#include <linux/ip.h>
+#include <linux/ipv6.h>
+#include <linux/pkt_cls.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_endian.h>
+#include "mark.h"
+#include "types.h"
+#include "bpf_debug.h"
+
+char LICENSE[] SEC("license") = "GPL";
+
+#define VLAN_HLEN 4
+#define VLAN_ENCAP_PROTO 2
+
+// the IPv4 fragment offset, below the three flag bits
+#define IPV4_FRAG_OFFSET 0x1fff
+
+#define PROTO_HOPOPTS 0
+#define PROTO_ICMP 1
+#define PROTO_TCP 6
+#define PROTO_UDP 17
+#define PROTO_DCCP 33
+#define PROTO_ROUTING 43
+#define PROTO_FRAGMENT 44
+#define PROTO_AH 51
+#define PROTO_ICMPV6 58
+#define PROTO_DSTOPTS 60
+#define PROTO_SCTP 132
+#define PROTO_UDPLITE 136
+
+#define ICMPV6_NEIGHBOR_SOLICIT 135
+#define ICMPV6_NEIGHBOR_ADVERT 136
+
+// the kernel's own bound on an IPv6 extension header chain, IP6_MAX_EXT_HDRS_CNT
+#define EXT_MAX 12
+
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __type(key, __u32); // ifindex
+ __type(value, struct nic_entry);
+ __uint(max_entries, NICS_MAX);
+ // the bucket array is sized by the ceiling at creation, the elements are allocated on demand
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __uint(pinning, LIBBPF_PIN_BY_NAME);
+} nics SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __type(key, struct pair_key);
+ __type(value, struct pair_value);
+ __uint(max_entries, PAIRS_MAX);
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __uint(pinning, LIBBPF_PIN_BY_NAME);
+} pairs SEC(".maps");
+
+// The checks of a pair past its inline ones, keyed by a hash of their content.
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __type(key, __u64);
+ __type(value, struct ruleset_value);
+ __uint(max_entries, 2 * RULESETS_MAX);
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __uint(pinning, LIBBPF_PIN_BY_NAME);
+} rulesets SEC(".maps");
+
+SEC("classifier")
+int tc_tap_ingress(struct __sk_buff *skb) {
+ __u32 ifindex = skb->ifindex;
+ struct nic_entry *nic = bpf_map_lookup_elem(&nics, &ifindex);
+ if (!nic) return TC_ACT_UNSPEC;
+ DBG("mark: src_id=%u ifindex=%u", nic->id, ifindex);
+ policy_mark_set(skb, nic->id);
+ return TC_ACT_UNSPEC;
+}
+
+// What a check matches on. `matchable` is set once the parse reached the L4 header.
+struct frame {
+ __u16 eth_proto;
+ __u16 sport;
+ __u16 dport;
+ __u8 family;
+ __u8 ip_proto;
+ __u8 icmp_type;
+ __u8 icmp_code;
+ __u8 matchable;
+ __u8 has_ports;
+ __u8 has_icmp;
+ __u8 unreadable;
+ // an authentication header was walked past, so a check for AH matches as well
+ __u8 ah;
+};
+
+static __always_inline int is_ext_header(__u8 proto) {
+ return proto == PROTO_HOPOPTS || proto == PROTO_ROUTING || proto == PROTO_DSTOPTS ||
+ proto == PROTO_FRAGMENT || proto == PROTO_AH;
+}
+
+// Walk the extension header chain to the L4 header. Zero means there is no header to match on.
+static __always_inline int ipv6_l4(struct __sk_buff *skb, struct frame *f, __u32 *off) {
+ for (int i = 0; i < EXT_MAX; i++) {
+ if (!is_ext_header(f->ip_proto)) return 1;
+
+ __u8 hdr[8];
+ if (f->ip_proto == PROTO_FRAGMENT) {
+ if (bpf_skb_load_bytes(skb, *off, hdr, 8) < 0) {
+ f->unreadable = 1;
+ return 0;
+ }
+ // the offset sits in the top 13 bits of the second halfword
+ if (hdr[2] || (hdr[3] & 0xf8)) return 0;
+ *off += 8;
+ } else {
+ if (bpf_skb_load_bytes(skb, *off, hdr, 2) < 0) {
+ f->unreadable = 1;
+ return 0;
+ }
+ if (f->ip_proto == PROTO_AH) {
+ f->ah = 1;
+ // AH counts its length in four byte units, minus two
+ *off += ((__u32)hdr[1] + 2) * 4;
+ } else {
+ // the length field leaves out the first eight bytes, hence the plus one
+ *off += ((__u32)hdr[1] + 1) * 8;
+ }
+ }
+ f->ip_proto = hdr[0];
+ }
+ if (is_ext_header(f->ip_proto)) {
+ f->unreadable = 1;
+ return 0;
+ }
+ return 1;
+}
+
+static __always_inline void parse_l4(struct __sk_buff *skb, struct frame *f, __u32 off) {
+ switch (f->ip_proto) {
+ case PROTO_TCP:
+ case PROTO_UDP:
+ case PROTO_DCCP:
+ case PROTO_SCTP:
+ case PROTO_UDPLITE: {
+ // every one of these carries the two ports in its first four bytes
+ __be16 ports[2];
+ if (bpf_skb_load_bytes(skb, off, ports, sizeof(ports)) < 0) {
+ f->unreadable = 1;
+ return;
+ }
+ f->sport = bpf_ntohs(ports[0]);
+ f->dport = bpf_ntohs(ports[1]);
+ f->has_ports = 1;
+ break;
+ }
+ case PROTO_ICMP:
+ case PROTO_ICMPV6: {
+ __u8 hdr[2];
+ if (bpf_skb_load_bytes(skb, off, hdr, sizeof(hdr)) < 0) {
+ f->unreadable = 1;
+ return;
+ }
+ f->icmp_type = hdr[0];
+ f->icmp_code = hdr[1];
+ f->has_icmp = 1;
+ break;
+ }
+ }
+ f->matchable = 1;
+}
+
+static __always_inline int is_vlan_tagged(__u16 eth_proto) {
+ return eth_proto == ETH_P_8021Q || eth_proto == ETH_P_8021AD;
+}
+
+// Read the frame down to its L4 header. The kernel moves a frame's outer VLAN tag into the skb
+// metadata before either hook, so a tag found here is a second one left in the frame data. A parse
+// that ends early still leaves the ethertype, which the ARP admission needs.
+static __always_inline void parse_frame(struct __sk_buff *skb, struct frame *f) {
+ __builtin_memset(f, 0, sizeof(*f));
+
+ struct ethhdr eth;
+ if (bpf_skb_load_bytes(skb, 0, ð, sizeof(eth)) < 0) return;
+ f->eth_proto = bpf_ntohs(eth.h_proto);
+ __u32 off = sizeof(eth);
+
+ if (is_vlan_tagged(f->eth_proto)) {
+ __be16 inner;
+ if (bpf_skb_load_bytes(skb, off + VLAN_ENCAP_PROTO, &inner, sizeof(inner)) < 0) return;
+ f->eth_proto = bpf_ntohs(inner);
+ off += VLAN_HLEN;
+ // a receiver strips any number of VID 0 tags and reads what is behind them. The parse reads
+ // through one tag only, so a further tag makes the frame unreadable
+ if (is_vlan_tagged(f->eth_proto)) {
+ f->unreadable = 1;
+ return;
+ }
+ }
+
+ if (f->eth_proto == ETH_P_IP) {
+ struct iphdr ip;
+ if (bpf_skb_load_bytes(skb, off, &ip, sizeof(ip)) < 0 || ip.ihl < 5) {
+ f->unreadable = 1;
+ return;
+ }
+ // only the first fragment carries the L4 header
+ if (bpf_ntohs(ip.frag_off) & IPV4_FRAG_OFFSET) return;
+ f->family = FAMILY_V4;
+ f->ip_proto = ip.protocol;
+ off += (__u32)ip.ihl * 4;
+ } else if (f->eth_proto == ETH_P_IPV6) {
+ struct ipv6hdr ip6;
+ if (bpf_skb_load_bytes(skb, off, &ip6, sizeof(ip6)) < 0) {
+ f->unreadable = 1;
+ return;
+ }
+ f->family = FAMILY_V6;
+ f->ip_proto = ip6.nexthdr;
+ off += sizeof(ip6);
+ if (!ipv6_l4(skb, f, &off)) return;
+ } else {
+ return;
+ }
+
+ parse_l4(skb, f, off);
+}
+
+static __always_inline int check_matches(const struct frame *f, const struct check_value *c) {
+ if (!f->matchable) return 0;
+ if (c->family && c->family != f->family) return 0;
+ if (c->ip_proto != f->ip_proto && !(c->ip_proto == PROTO_AH && f->ah)) return 0;
+
+ if (c->match_flags & CHECK_HAS_SPORT) {
+ if (!f->has_ports || f->sport < c->sport_lo || f->sport > c->sport_hi) return 0;
+ }
+ if (c->match_flags & CHECK_HAS_DPORT) {
+ if (!f->has_ports || f->dport < c->dport_lo || f->dport > c->dport_hi) return 0;
+ }
+ if (c->match_flags & CHECK_HAS_ICMP_TYPE) {
+ if (!f->has_icmp || f->icmp_type != c->icmp_type) return 0;
+ }
+ if (c->match_flags & CHECK_HAS_ICMP_CODE) {
+ if (!f->has_icmp || f->icmp_code != c->icmp_code) return 0;
+ }
+ return 1;
+}
+
+// A ruleset is walked through bpf_loop, so the verifier's work does not grow with CHECK_MAX. A
+// plain loop bounded by CHECK_MAX would have it walk every iteration, each with the branches of
+// its match.
+struct ruleset_walk {
+ const struct frame *f;
+ const struct ruleset_value *rs;
+ __u8 decided;
+ __u8 allow;
+};
+
+static long ruleset_check(__u64 i, void *ctx) {
+ struct ruleset_walk *w = ctx;
+ // the count comes from the map, the verifier needs a bound on the index
+ if (i >= CHECK_MAX) return 1;
+ const struct check_value *c = &w->rs->checks[i];
+ if (!check_matches(w->f, c)) return 0;
+ w->decided = 1;
+ w->allow = c->allow;
+ return 1;
+}
+
+static __always_inline int is_arp_or_nd(const struct frame *f) {
+ if (f->eth_proto == ETH_P_ARP) return 1;
+ return f->family == FAMILY_V6 && f->ip_proto == PROTO_ICMPV6 && f->has_icmp &&
+ (f->icmp_type == ICMPV6_NEIGHBOR_SOLICIT || f->icmp_type == ICMPV6_NEIGHBOR_ADVERT);
+}
+
+// ARP and neighbor discovery also pass when the reverse pair allows anything, and when the host
+// sends them. The host's own replies, the suppression replies of a bridge or the answers of a
+// gateway on it, come unstamped and without an ingress interface. Behind a firewall bridge the
+// veth hop gives every frame one, so a tap there is flagged and takes these frames from any
+// unstamped sender.
+static __always_inline int rescue_arp_nd(struct __sk_buff *skb, const struct frame *f,
+ __u16 src_id, const struct nic_entry *dst) {
+ if (!is_arp_or_nd(f)) return 0;
+ if (src_id == ID_UNSTAMPED) return skb->ingress_ifindex == 0 || (dst->flags & NIC_ADMIT_ND);
+ struct pair_key k = { .src_id = dst->id, .dst_id = src_id };
+ struct pair_value *r = bpf_map_lookup_elem(&pairs, &k);
+ return r && (r->flags & PAIR_ADMIT_ND);
+}
+
+static __always_inline int has_checks(const struct pair_value *r) {
+ return r->inline_count || (r->flags & PAIR_HAS_RULESET);
+}
+
+static __always_inline int verdict(__u8 allow) {
+ return allow ? TC_ACT_UNSPEC : TC_ACT_SHOT;
+}
+
+// Enforce on egress of the destination tap, where both ends are known. The destination comes from
+// this tap's NIC entry and the source from skb->mark. A missing pair is a deny, so unstamped
+// frames (src_id 0) need an explicit (0, dst_id) allow. ARP and neighbor discovery as above and a
+// frame turned around on the tap itself are the exceptions.
+SEC("classifier")
+int tc_tap_egress(struct __sk_buff *skb) {
+ __u32 ifindex = skb->ifindex;
+ __u16 src_id = policy_mark_get(skb);
+
+ struct nic_entry *nic = bpf_map_lookup_elem(&nics, &ifindex);
+ // a NIC gets its entry before its links and loses it before them
+ if (!nic) return TC_ACT_UNSPEC;
+
+ // an unstamped frame that came in on this tap was turned around by a program ahead of this
+ // one, the host's own answer to the guest. A guest frame the bridge reflects carries the
+ // stamp.
+ if (src_id == ID_UNSTAMPED && skb->ingress_ifindex == ifindex) return TC_ACT_UNSPEC;
+
+ struct pair_key k = { .src_id = src_id, .dst_id = nic->id };
+ struct pair_value *r = bpf_map_lookup_elem(&pairs, &k);
+ if (r && r->allow && !has_checks(r)) return TC_ACT_UNSPEC;
+
+ struct frame f;
+ parse_frame(skb, &f);
+
+ if (r && has_checks(r)) {
+ // a frame the parse could not read to its L4 header could hide what a check would deny
+ if (f.unreadable) {
+ DBG("deny (L4 header unreadable): src_id=%u dst_id=%u", src_id, nic->id);
+ return TC_ACT_SHOT;
+ }
+
+ for (__u32 i = 0; i < CHECK_INLINE; i++) {
+ if (i >= r->inline_count) break;
+ const struct check_value *c = &r->checks[i];
+ if (check_matches(&f, c)) {
+ DBG("check %u decides: allow=%u", i, c->allow);
+ return verdict(c->allow);
+ }
+ }
+
+ if (r->flags & PAIR_HAS_RULESET) {
+ __u64 key = r->ruleset;
+ struct ruleset_value *rs = bpf_map_lookup_elem(&rulesets, &key);
+ if (rs) {
+ struct ruleset_walk w = { .f = &f, .rs = rs };
+ bpf_loop(rs->count, ruleset_check, &w, 0);
+ if (w.decided) {
+ DBG("ruleset check decides: allow=%u", w.allow);
+ return verdict(w.allow);
+ }
+ } else {
+ // a frame that read its pair before a pass rewrote it can find the old ruleset
+ // removed already. The verdict alone is not what that pair asked for, so the
+ // frame is dropped rather than decided on half the checks.
+ DBG("deny (no ruleset): src_id=%u dst_id=%u", src_id, nic->id);
+ return TC_ACT_SHOT;
+ }
+ }
+
+ if ((r->flags & PAIR_ADMIT_ND) && is_arp_or_nd(&f)) return TC_ACT_UNSPEC;
+ }
+
+ if (r && r->allow) return TC_ACT_UNSPEC;
+ if (rescue_arp_nd(skb, &f, src_id, nic)) return TC_ACT_UNSPEC;
+ if (r) {
+ DBG("deny (explicit): src_id=%u dst_id=%u", src_id, nic->id);
+ } else {
+ DBG("deny (no pair): src_id=%u dst_id=%u", src_id, nic->id);
+ }
+ return TC_ACT_SHOT;
+}
diff --git a/proxmox-ebpf/src/policy/bpf/types.h b/proxmox-ebpf/src/policy/bpf/types.h
new file mode 100644
index 000000000000..dbeb9bea59db
--- /dev/null
+++ b/proxmox-ebpf/src/policy/bpf/types.h
@@ -0,0 +1,72 @@
+#ifndef PROXMOX_EBPF_POLICY_TYPES_H
+#define PROXMOX_EBPF_POLICY_TYPES_H
+
+// Keep in sync with ../types.rs.
+// the __u* types come from linux/types.h
+
+#define CHECK_MAX 512
+// the checks a pair_value holds inline, the rest go to the pair's ruleset
+#define CHECK_INLINE 7
+#define PAIRS_MAX (1 << 20)
+#define RULESETS_MAX (1 << 14)
+#define NICS_MAX (1 << 18)
+
+// what the mark of a frame nobody stamped reads as, never the id of a NIC
+#define ID_UNSTAMPED 0
+
+#define PAIR_ADMIT_ND (1 << 0)
+#define PAIR_HAS_RULESET (1 << 1)
+
+#define NIC_ADMIT_ND (1 << 0)
+
+#define FAMILY_V4 4
+#define FAMILY_V6 6
+
+#define CHECK_HAS_SPORT (1 << 0)
+#define CHECK_HAS_DPORT (1 << 1)
+#define CHECK_HAS_ICMP_TYPE (1 << 2)
+#define CHECK_HAS_ICMP_CODE (1 << 3)
+
+struct nic_entry {
+ __u16 id;
+ __u16 flags;
+ __u8 _pad[4];
+};
+
+struct pair_key {
+ __u16 src_id;
+ __u16 dst_id;
+};
+
+struct check_value {
+ __u8 ip_proto;
+ // 0 matches either family, 4 and 6 pin the match to one
+ __u8 family;
+ __u8 match_flags;
+ __u8 allow;
+ __u16 sport_lo;
+ __u16 sport_hi;
+ __u16 dport_lo;
+ __u16 dport_hi;
+ __u8 icmp_type;
+ __u8 icmp_code;
+ __u8 _pad[2];
+};
+
+struct pair_value {
+ __u8 allow;
+ __u8 flags;
+ __u8 inline_count;
+ __u8 _pad[5];
+ // the rulesets key, read only with PAIR_HAS_RULESET set
+ __u64 ruleset;
+ struct check_value checks[CHECK_INLINE];
+};
+
+struct ruleset_value {
+ __u16 count;
+ __u8 _pad[14];
+ struct check_value checks[CHECK_MAX];
+};
+
+#endif
diff --git a/proxmox-ebpf/src/policy/mod.rs b/proxmox-ebpf/src/policy/mod.rs
new file mode 100644
index 000000000000..4ca65bb9f595
--- /dev/null
+++ b/proxmox-ebpf/src/policy/mod.rs
@@ -0,0 +1,807 @@
+//! Tap-side enforcement. On ingress, a BPF program on the guest tap or veth interface stamps the
+//! source wire id into `skb->mark`. On egress, it looks up the `(src_id, dst_id)` pair and drops
+//! the frame on a miss or a denying pair. A pair can narrow its verdict with checks
+//! ([`CheckEntry`]), run in the order given, and the first check that matches decides. A pair with
+//! checks drops a frame whose L4 header cannot be read. ARP and the solicitations and
+//! advertisements of neighbor discovery pass in both directions between two NICs that a pair admits
+//! something for, and when the host sends them. Behind a firewall bridge the host's frames look
+//! like any other sender's, so a NIC there takes them from every unstamped sender.
+//! An unstamped frame that leaves on the interface it came in on was turned around by a program
+//! ahead of the policy there and passes as the host's answer.
+//!
+//! The checks past the inline ones go to `rulesets`, shared by the pairs with the same checks.
+//!
+//! A pass and a plug take the subsystem's lock before they read their input, the pass exclusively
+//! and the plug shared, so they reach the kernel in the order they read. A plug writes its NIC's
+//! entry over whatever a pass placed, since what it read is at least as new.
+//!
+//! A pass writes the whole policy into fresh maps and moves the links over once they are complete,
+//! so a tap runs on the policy before the pass or the one after it and never on a mix. A NIC the
+//! pass newly assigns is attached last.
+
+pub mod types;
+
+use std::collections::{HashMap, HashSet};
+use std::hash::{DefaultHasher, Hash, Hasher};
+
+use anyhow::{Context, bail};
+use aya::maps::MapError;
+
+use self::types::*;
+use crate::subsystem::{ApplyLock, Exclusive, MapHandle, NotLoaded, Shared, Staged, TcPrograms};
+use crate::tc::Direction;
+use crate::tc_programs;
+
+pub use self::types::{CHECK_MAX, PAIRS_MAX};
+
+/// What a full pass did on the host.
+#[derive(Copy, Clone, Debug, PartialEq, Eq)]
+pub enum Pass {
+ /// The host matches the input.
+ Applied,
+ /// The policy assigns nothing, so nothing is loaded and no link is left.
+ Idle,
+}
+
+/// A guest NIC, the wire id it stamps and whether it sits behind a firewall bridge.
+pub struct Assignment {
+ /// The interface name on this host.
+ pub iface: String,
+ pub id: u16,
+ pub firewall_bridge: bool,
+}
+
+/// One pair of the pairs map. A pair that admits nothing, with no allow of its own and no allowing
+/// check, is valid input and gets no ARP and neighbor discovery pass.
+pub struct Pair {
+ pub src_id: u16,
+ pub dst_id: u16,
+ /// The checks to run, at most [`CHECK_MAX`] of them.
+ pub checks: Vec<CheckEntry>,
+ pub allow: bool,
+}
+
+#[derive(Copy, Clone, Debug, Default, PartialEq, Eq)]
+#[repr(u8)]
+pub enum Family {
+ #[default]
+ Any = 0,
+ V4 = FAMILY_V4,
+ V6 = FAMILY_V6,
+}
+
+/// One protocol match of a narrowed pair. A single port is a range with equal bounds, and a field
+/// left out is not matched on.
+#[derive(Clone, Debug, Default, PartialEq, Eq)]
+pub struct CheckEntry {
+ pub ip_proto: u8,
+ pub family: Family,
+ pub sport: Option<(u16, u16)>,
+ pub dport: Option<(u16, u16)>,
+ pub icmp_type: Option<u8>,
+ pub icmp_code: Option<u8>,
+ pub allow: bool,
+}
+
+impl CheckEntry {
+ fn value(&self) -> CheckValue {
+ let mut match_flags = 0;
+ if self.sport.is_some() {
+ match_flags |= CHECK_HAS_SPORT;
+ }
+ if self.dport.is_some() {
+ match_flags |= CHECK_HAS_DPORT;
+ }
+ if self.icmp_type.is_some() {
+ match_flags |= CHECK_HAS_ICMP_TYPE;
+ }
+ if self.icmp_code.is_some() {
+ match_flags |= CHECK_HAS_ICMP_CODE;
+ }
+ let (sport_lo, sport_hi) = self.sport.unwrap_or_default();
+ let (dport_lo, dport_hi) = self.dport.unwrap_or_default();
+ CheckValue {
+ ip_proto: self.ip_proto,
+ family: self.family as u8,
+ match_flags,
+ allow: self.allow as u8,
+ sport_lo,
+ sport_hi,
+ dport_lo,
+ dport_hi,
+ icmp_type: self.icmp_type.unwrap_or_default(),
+ icmp_code: self.icmp_code.unwrap_or_default(),
+ _pad: [0; 2],
+ }
+ }
+}
+
+const NICS_MAP: &str = "nics";
+const PAIRS_MAP: &str = "pairs";
+const RULESETS_MAP: &str = "rulesets";
+const MAPS: [&str; 3] = [NICS_MAP, PAIRS_MAP, RULESETS_MAP];
+
+// bump when a map layout in types.h, the meaning of a value or a map ceiling changes. An existing
+// pin is reused by name without a check of any of them.
+const SCHEMA_VERSION: u32 = 1;
+
+const PROGRAM_NAMES: [(Direction, &str); 2] = [
+ (Direction::Ingress, "tc_tap_ingress"),
+ (Direction::Egress, "tc_tap_egress"),
+];
+
+type NicsMap<'a> = MapHandle<'a, u32, NicEntry>;
+type PairsMap<'a> = MapHandle<'a, PairKey, PairValue>;
+type RulesetsMap<'a> = MapHandle<'a, u64, RulesetValue>;
+
+static PROGRAMS: TcPrograms = tc_programs! {
+ name: "policy",
+ programs: &PROGRAM_NAMES,
+ maps: &MAPS,
+ schema_version: SCHEMA_VERSION,
+};
+
+/// The lock a plug runs under. Plugs run next to each other and wait only behind a pass.
+pub struct SharedLock {
+ lock: ApplyLock<'static, Shared>,
+}
+
+/// The lock a pass runs under, held until it is through.
+pub struct ExclusiveLock {
+ lock: ApplyLock<'static, Exclusive>,
+}
+
+pub fn lock_shared() -> anyhow::Result<SharedLock> {
+ Ok(SharedLock {
+ lock: PROGRAMS.lock_shared()?,
+ })
+}
+
+pub fn lock_exclusive() -> anyhow::Result<ExclusiveLock> {
+ Ok(ExclusiveLock {
+ lock: PROGRAMS.lock_exclusive()?,
+ })
+}
+
+/// Check `pairs` the way a full pass does, without touching the host.
+pub fn check(pairs: &[Pair]) -> anyhow::Result<()> {
+ count_within(pairs, PAIRS_MAX, RULESETS_MAX)
+}
+
+impl ExclusiveLock {
+ /// Full pass over the host. Afterwards the interface map holds the assigned NICs that exist
+ /// here, each with its id and its links, and nothing else, and the pairs map holds exactly
+ /// `pairs`. `configured` is whether the policy has an assignment at all. Without one the pass
+ /// returns [`Pass::Idle`].
+ pub fn apply(
+ &mut self,
+ configured: bool,
+ assignments: &[Assignment],
+ pairs: &[Pair],
+ ) -> anyhow::Result<Pass> {
+ check_assignments(assignments)?;
+ let encoded = encode_pairs(pairs)?;
+ let lock = &mut self.lock;
+ if !configured {
+ if lock.is_current() || !lock.attached()?.is_empty() {
+ log::info!("policy: the policy assigns nothing, dropping the pinned state");
+ lock.clear()?;
+ }
+ return Ok(Pass::Idle);
+ }
+ let mut local = resolve_local_assignments(assignments)?;
+ carry_plug_flags(lock, &mut local)?;
+ lock.replace_maps(|staged| write_maps(staged, &local, &encoded))?;
+ let desired: HashSet<u32> = local.keys().copied().collect();
+ lock.reconcile(&desired)?;
+ Ok(Pass::Applied)
+ }
+}
+
+// a plug may have flagged a NIC this pass saw before it was put behind its firewall bridge, so
+// only a plug takes the flag away again
+fn carry_plug_flags(
+ lock: &ApplyLock<'_, Exclusive>,
+ desired: &mut HashMap<u32, NicEntry>,
+) -> anyhow::Result<()> {
+ let live: NicsMap = match lock.hash_map(NICS_MAP) {
+ Ok(map) => map,
+ Err(e) if e.downcast_ref::<NotLoaded>().is_some() => return Ok(()),
+ Err(e) => return Err(e),
+ };
+ let live: HashMap<u32, NicEntry> = live
+ .iter()
+ .collect::<Result<_, _>>()
+ .context("read the interface map")?;
+ carry_flags(&live, desired);
+ Ok(())
+}
+
+fn carry_flags(live: &HashMap<u32, NicEntry>, desired: &mut HashMap<u32, NicEntry>) {
+ for (ifindex, entry) in desired {
+ if let Some(placed) = live.get(ifindex) {
+ entry.flags |= placed.flags & NIC_ADMIT_ND;
+ }
+ }
+}
+
+fn write_maps(
+ staged: &Staged<'_>,
+ nics: &HashMap<u32, NicEntry>,
+ encoded: &Encoded,
+) -> anyhow::Result<()> {
+ let mut nics_map: NicsMap = staged.hash_map(NICS_MAP)?;
+ for (&ifindex, &entry) in nics {
+ nics_map
+ .insert(ifindex, entry, 0)
+ .context("write the interface map")?;
+ }
+ let mut rulesets: RulesetsMap = staged.hash_map(RULESETS_MAP)?;
+ for (&key, ruleset) in &encoded.rulesets {
+ rulesets
+ .insert(key, ruleset, 0)
+ .context("write the ruleset map")?;
+ }
+ let mut pairs: PairsMap = staged.hash_map(PAIRS_MAP)?;
+ for (&key, &value) in &encoded.pairs {
+ pairs.insert(key, value, 0).context("write the pairs map")?;
+ }
+ log::info!(
+ "policy: {} NIC entries, {} pairs and {} rulesets written",
+ nics.len(),
+ encoded.pairs.len(),
+ encoded.rulesets.len(),
+ );
+ Ok(())
+}
+
+impl SharedLock {
+ /// Program a single guest NIC present on this host, with the `id` the caller resolved. A
+ /// failure propagates, so the caller can keep the NIC from coming up unenforced. A host
+ /// without current programs fails that way, since only a full pass installs them, so does one
+ /// whose last pass stopped halfway, and so does a full interface map. `firewall_bridge` says
+ /// the NIC is or is about to be put behind a firewall bridge, which a NIC still being plugged
+ /// does not show.
+ pub fn apply_iface(&self, iface: &str, id: u16, firewall_bridge: bool) -> anyhow::Result<()> {
+ let entry = nic_entry(id, firewall_bridge);
+ let Some(ifindex) = ifindex_of(iface)? else {
+ log::info!("policy: {iface} is gone, nothing to do");
+ return Ok(());
+ };
+ prune_gone(&self.lock);
+ apply_one(&self.lock, ifindex, iface, entry)
+ }
+
+ /// Take the entry and the links of a guest NIC away when it comes up again under the same
+ /// index and the config no longer assigns it. Nothing is touched before a new build's first
+ /// pass, that one decides about every NIC.
+ pub fn release_iface(&self, iface: &str) -> anyhow::Result<()> {
+ let Some(ifindex) = ifindex_of(iface)? else {
+ return Ok(());
+ };
+ let lock = &self.lock;
+ if !lock.is_current() {
+ return Ok(());
+ }
+ let mut nics: NicsMap = lock.hash_map(NICS_MAP)?;
+ let removed = match nics.remove(&ifindex) {
+ Ok(()) => true,
+ Err(e) if missing_key(&e) => false,
+ Err(e) => return Err(e).with_context(|| format!("remove the entry of {iface}")),
+ };
+ if removed {
+ log::info!("policy: {iface} (ifindex {ifindex}) is no longer assigned");
+ lock.detach_named(iface, |pin| pin.ifindex != ifindex)?;
+ }
+ Ok(())
+ }
+}
+
+/// Drop what earlier plugs left for interfaces that are gone, so a node that starts guests
+/// for a long time without a full pass does not fill the interface map or the pin directory.
+fn prune_gone(lock: &ApplyLock<'_, Shared>) {
+ let pruned = lock.detach_gone().and_then(|gone| {
+ if gone.is_empty() || !lock.is_current() {
+ return Ok(0);
+ }
+ let mut nics: NicsMap = lock.hash_map(NICS_MAP)?;
+ for ifindex in &gone {
+ match nics.remove(ifindex) {
+ Ok(()) => {}
+ Err(e) if missing_key(&e) => {}
+ Err(e) => {
+ return Err(e).with_context(|| format!("remove the entry of {ifindex}"));
+ }
+ }
+ }
+ Ok(gone.len())
+ });
+ match pruned {
+ Ok(0) => {}
+ Ok(n) => log::debug!("policy: dropped {n} gone interface(s) left by earlier plugs"),
+ Err(e) => log::warn!("policy: prune of gone interfaces: {e:#}"),
+ }
+}
+
+fn apply_one(
+ lock: &ApplyLock<'_, Shared>,
+ ifindex: u32,
+ iface: &str,
+ entry: NicEntry,
+) -> anyhow::Result<()> {
+ if !lock.is_current() {
+ bail!("no current programs are pinned, a full pass has to install them first");
+ }
+ if lock.interrupted() {
+ bail!("the last full pass stopped halfway, the next one has to finish the switch first");
+ }
+ let mut nics: NicsMap = lock.hash_map(NICS_MAP)?;
+ nics.insert(ifindex, entry, 0)
+ .with_context(|| format!("write the entry of {iface}"))?;
+ log::info!("policy: {iface} (ifindex {ifindex}) -> id {}", entry.id);
+ lock.attach_iface(ifindex)?;
+ Ok(())
+}
+
+fn ifindex_of(iface: &str) -> anyhow::Result<Option<u32>> {
+ match nix::net::if_::if_nametoindex(iface) {
+ Ok(ifindex) => Ok(Some(ifindex)),
+ Err(nix::errno::Errno::ENODEV) => Ok(None),
+ Err(e) => Err(e).with_context(|| format!("index of interface {iface}")),
+ }
+}
+
+fn missing_key(err: &MapError) -> bool {
+ matches!(
+ err,
+ MapError::SyscallError(e) if e.io_error.raw_os_error() == Some(nix::libc::ENOENT)
+ )
+}
+
+fn encode(pair: &Pair, key: Option<u64>, tail: &[CheckValue]) -> (PairValue, Option<RulesetValue>) {
+ let admit_nd = pair.allow || pair.checks.iter().any(|check| check.allow);
+ let split = pair.checks.len().min(CHECK_INLINE);
+
+ let checks = check_values(pair.checks[..split].iter().map(CheckEntry::value));
+ let ruleset = (!tail.is_empty()).then(|| RulesetValue {
+ count: tail.len() as u16,
+ _pad: [0; 14],
+ checks: check_values(tail.iter().copied()),
+ });
+
+ let mut flags = 0;
+ if admit_nd {
+ flags |= PAIR_ADMIT_ND;
+ }
+ if ruleset.is_some() {
+ flags |= PAIR_HAS_RULESET;
+ }
+ let value = PairValue {
+ allow: pair.allow as u8,
+ flags,
+ inline_count: split as u8,
+ _pad: [0; 5],
+ ruleset: key.unwrap_or(0),
+ checks,
+ };
+ (value, ruleset)
+}
+
+// the slots past the checks stay zeroed, so an unchanged pair or ruleset compares equal and is not
+// written again
+fn check_values<const N: usize>(values: impl IntoIterator<Item = CheckValue>) -> [CheckValue; N] {
+ let mut slots = [CheckValue::default(); N];
+ for (slot, value) in slots.iter_mut().zip(values) {
+ *slot = value;
+ }
+ slots
+}
+
+// the key only has to be stable within one build. A ruleset pinned under the key of another
+// build is written anew under this one, and the old one is removed as unreferenced
+fn ruleset_key(checks: &[CheckValue]) -> u64 {
+ let mut hasher = DefaultHasher::new();
+ checks.hash(&mut hasher);
+ hasher.finish()
+}
+
+// a NIC given twice would be decided by the input order
+fn check_assignments(assignments: &[Assignment]) -> anyhow::Result<()> {
+ let mut ids = HashMap::new();
+ for a in assignments {
+ if a.id == ID_UNSTAMPED {
+ bail!("NIC {} is given the id of unstamped frames", a.iface);
+ }
+ if let Some(other) = ids.insert(a.iface.as_str(), a.id) {
+ bail!(
+ "NIC {} is given twice, with id {other} and id {}",
+ a.iface,
+ a.id,
+ );
+ }
+ }
+ Ok(())
+}
+
+struct Encoded {
+ pairs: HashMap<PairKey, PairValue>,
+ rulesets: HashMap<u64, RulesetValue>,
+}
+
+fn encode_pairs(pairs: &[Pair]) -> anyhow::Result<Encoded> {
+ encode_within(pairs, PAIRS_MAX, RULESETS_MAX)
+}
+
+fn encode_within(pairs: &[Pair], pairs_max: usize, rulesets_max: usize) -> anyhow::Result<Encoded> {
+ let mut encoded = Encoded {
+ pairs: HashMap::with_capacity(pairs.len()),
+ rulesets: HashMap::new(),
+ };
+ walk_within(
+ pairs,
+ pairs_max,
+ rulesets_max,
+ |pair, pair_key, key, tail| {
+ let (value, ruleset) = encode(pair, key, tail);
+ if let (Some(key), Some(ruleset)) = (key, ruleset)
+ && *encoded.rulesets.entry(key).or_insert(ruleset) != ruleset
+ {
+ bail!("two rulesets share the key {key:016x}");
+ }
+ encoded.pairs.insert(pair_key, value);
+ Ok(())
+ },
+ )?;
+ Ok(encoded)
+}
+
+fn count_within(pairs: &[Pair], pairs_max: usize, rulesets_max: usize) -> anyhow::Result<()> {
+ walk_within(pairs, pairs_max, rulesets_max, |_, _, _, _| Ok(()))
+}
+
+// the input is checked before anything is written. Checks past the limit would be cut off, and a
+// pair given twice would be decided by the input order, and a full map would only be found at the
+// end of the fill. `check` runs the same walk and skips the map image, 128 bytes per pair and 8 KiB
+// per ruleset.
+fn walk_within(
+ pairs: &[Pair],
+ pairs_max: usize,
+ rulesets_max: usize,
+ mut place: impl FnMut(&Pair, PairKey, Option<u64>, &[CheckValue]) -> anyhow::Result<()>,
+) -> anyhow::Result<()> {
+ if pairs.len() > pairs_max {
+ bail!(
+ "{} pairs are given, the pairs map holds at most {pairs_max}",
+ pairs.len(),
+ );
+ }
+ let mut seen = HashSet::with_capacity(pairs.len());
+ let mut rulesets = HashSet::new();
+ for pair in pairs {
+ if pair.checks.len() > CHECK_MAX {
+ bail!(
+ "pair {} -> {} carries {} checks, the program runs at most {CHECK_MAX}",
+ pair.src_id,
+ pair.dst_id,
+ pair.checks.len(),
+ );
+ }
+ let pair_key = PairKey {
+ src_id: pair.src_id,
+ dst_id: pair.dst_id,
+ };
+ if !seen.insert(pair_key) {
+ bail!("pair {} -> {} is given twice", pair.src_id, pair.dst_id);
+ }
+ let tail: Vec<CheckValue> = pair.checks[pair.checks.len().min(CHECK_INLINE)..]
+ .iter()
+ .map(CheckEntry::value)
+ .collect();
+ let key = (!tail.is_empty()).then(|| ruleset_key(&tail));
+ if let Some(key) = key {
+ rulesets.insert(key);
+ if rulesets.len() > rulesets_max {
+ bail!(
+ "{} rulesets are needed, a pass may name at most {rulesets_max}",
+ rulesets.len(),
+ );
+ }
+ }
+ place(pair, pair_key, key, &tail)?;
+ }
+ Ok(())
+}
+
+fn nic_entry(id: u16, firewall_bridge: bool) -> NicEntry {
+ NicEntry {
+ id,
+ flags: if firewall_bridge { NIC_ADMIT_ND } else { 0 },
+ _pad: [0; 4],
+ }
+}
+
+/// The id and firewall bridge flag of every assigned NIC present here, by ifindex.
+fn resolve_local_assignments(assignments: &[Assignment]) -> anyhow::Result<HashMap<u32, NicEntry>> {
+ let ifaces = nix::net::if_::if_nameindex().context("if_nameindex")?;
+ let by_name: HashMap<&str, u32> = ifaces
+ .iter()
+ .filter_map(|i| i.name().to_str().ok().map(|n| (n, i.index())))
+ .collect();
+
+ let mut out = HashMap::new();
+ for a in assignments {
+ if let Some(&ifindex) = by_name.get(a.iface.as_str()) {
+ log::debug!(
+ "policy: resolved {} -> ifindex {ifindex} id {}",
+ a.iface,
+ a.id,
+ );
+ out.insert(ifindex, nic_entry(a.id, a.firewall_bridge));
+ } else {
+ log::trace!("policy: skipping {}: no such interface", a.iface);
+ }
+ }
+ log::debug!("policy: {} managed NICs on this host", out.len());
+ Ok(out)
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ fn entry(port: u16) -> CheckEntry {
+ CheckEntry {
+ ip_proto: 6,
+ family: Family::Any,
+ sport: None,
+ dport: Some((port, port)),
+ icmp_type: None,
+ icmp_code: None,
+ allow: true,
+ }
+ }
+
+ fn pair(allow: bool, checks: Vec<CheckEntry>) -> Pair {
+ Pair {
+ src_id: 3,
+ dst_id: 5,
+ checks,
+ allow,
+ }
+ }
+
+ fn encode_one(pair: &Pair) -> (PairValue, Option<(u64, RulesetValue)>) {
+ let encoded = encode_pairs(std::slice::from_ref(pair)).expect("one pair encodes");
+ let value = encoded
+ .pairs
+ .into_values()
+ .next()
+ .expect("the pair's value");
+ (value, encoded.rulesets.into_iter().next())
+ }
+
+ fn bytes(value: CheckValue) -> [u8; 16] {
+ // SAFETY: CheckValue is 16 bytes of repr(C) with no implicit padding
+ unsafe { std::mem::transmute(value) }
+ }
+
+ #[test]
+ fn check_entry_bytes() {
+ let plain = CheckEntry {
+ ip_proto: 17,
+ family: Family::Any,
+ sport: None,
+ dport: None,
+ icmp_type: None,
+ icmp_code: None,
+ allow: false,
+ };
+ assert_eq!(
+ bytes(plain.value()),
+ [17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
+ );
+
+ let v4 = CheckEntry {
+ family: Family::V4,
+ allow: true,
+ ..plain.clone()
+ };
+ assert_eq!(bytes(v4.value())[..4], [17, 4, 0, 1]);
+ let v6 = CheckEntry {
+ family: Family::V6,
+ ..plain.clone()
+ };
+ assert_eq!(bytes(v6.value())[1], 6);
+
+ let sport = CheckEntry {
+ sport: Some((1000, 2000)),
+ ..plain.clone()
+ };
+ let b = bytes(sport.value());
+ assert_eq!(b[2], CHECK_HAS_SPORT);
+ assert_eq!(b[4..6], 1000u16.to_ne_bytes());
+ assert_eq!(b[6..8], 2000u16.to_ne_bytes());
+ assert_eq!(b[8..], [0; 8]);
+
+ let dport = CheckEntry {
+ dport: Some((22, 23)),
+ ..plain.clone()
+ };
+ let b = bytes(dport.value());
+ assert_eq!(b[2], CHECK_HAS_DPORT);
+ assert_eq!(b[4..8], [0; 4]);
+ assert_eq!(b[8..10], 22u16.to_ne_bytes());
+ assert_eq!(b[10..12], 23u16.to_ne_bytes());
+
+ let icmp = CheckEntry {
+ ip_proto: 1,
+ icmp_type: Some(3),
+ icmp_code: Some(4),
+ ..plain.clone()
+ };
+ let b = bytes(icmp.value());
+ assert_eq!(b[2], CHECK_HAS_ICMP_TYPE | CHECK_HAS_ICMP_CODE);
+ assert_eq!(b[12..], [3, 4, 0, 0]);
+
+ let code_only = CheckEntry {
+ ip_proto: 1,
+ icmp_code: Some(0),
+ ..plain
+ };
+ assert_eq!(bytes(code_only.value())[2], CHECK_HAS_ICMP_CODE);
+ }
+
+ #[test]
+ fn an_admitting_pair_says_so() {
+ let (value, _) = encode_one(&pair(true, vec![]));
+ assert_eq!(value.flags, PAIR_ADMIT_ND);
+ let (value, _) = encode_one(&pair(false, vec![]));
+ assert_eq!(value.flags, 0);
+ }
+
+ #[test]
+ fn a_short_pair_stays_inline() {
+ let checks: Vec<_> = (1..=3).map(entry).collect();
+ let (value, ruleset) = encode_one(&pair(false, checks.clone()));
+ assert!(ruleset.is_none());
+ assert_eq!(value.allow, 0);
+ assert_eq!(value.flags, PAIR_ADMIT_ND);
+ assert_eq!(value.inline_count, 3);
+ assert_eq!(value.ruleset, 0);
+ for (slot, check) in value.checks.iter().zip(&checks) {
+ assert_eq!(*slot, check.value());
+ }
+ assert!(value.checks[3..].iter().all(|c| bytes(*c) == [0; 16]));
+ }
+
+ #[test]
+ fn a_pair_filling_the_inline_slots_needs_no_ruleset() {
+ let checks: Vec<_> = (1..=CHECK_INLINE as u16).map(entry).collect();
+ let (value, ruleset) = encode_one(&pair(true, checks.clone()));
+ assert!(ruleset.is_none());
+ assert_eq!(value.allow, 1);
+ assert_eq!(value.flags, PAIR_ADMIT_ND);
+ assert_eq!(value.inline_count, CHECK_INLINE as u8);
+ assert_eq!(value.ruleset, 0);
+ for (slot, check) in value.checks.iter().zip(&checks) {
+ assert_eq!(*slot, check.value());
+ }
+ }
+
+ #[test]
+ fn the_rest_goes_to_a_ruleset() {
+ let checks: Vec<_> = (1..=10).map(entry).collect();
+ let (value, ruleset) = encode_one(&pair(true, checks.clone()));
+ let (key, _) = ruleset.expect("a ruleset for the checks past the inline ones");
+ assert_eq!(value.flags, PAIR_ADMIT_ND | PAIR_HAS_RULESET);
+ assert_eq!(value.inline_count, CHECK_INLINE as u8);
+ assert_eq!(value.ruleset, key);
+ for (slot, check) in value.checks.iter().zip(&checks[..CHECK_INLINE]) {
+ assert_eq!(*slot, check.value());
+ }
+
+ let mut other_rest = checks.clone();
+ other_rest[CHECK_INLINE] = entry(100);
+ let (_, other_ruleset) = encode_one(&pair(true, other_rest));
+ assert_ne!(other_ruleset.expect("a ruleset").0, key);
+
+ let (other, _) = encode_one(&pair(false, checks));
+ assert_eq!(other.ruleset, key);
+ assert_eq!(other.flags, PAIR_ADMIT_ND | PAIR_HAS_RULESET);
+ }
+
+ #[test]
+ fn ruleset_value() {
+ let checks: Vec<_> = (1..=10).map(entry).collect();
+ let (_, ruleset) = encode_one(&pair(false, checks.clone()));
+ let (_, ruleset) = ruleset.unwrap();
+ assert_eq!(ruleset.count, 3);
+ assert_eq!(ruleset._pad, [0; 14]);
+ for (slot, check) in ruleset.checks.iter().zip(&checks[CHECK_INLINE..]) {
+ assert_eq!(*slot, check.value());
+ }
+ assert!(ruleset.checks[3..].iter().all(|c| bytes(*c) == [0; 16]));
+ }
+
+ fn assignment(iface: &str, id: u16) -> Assignment {
+ Assignment {
+ iface: iface.to_string(),
+ id,
+ firewall_bridge: false,
+ }
+ }
+
+ #[test]
+ fn a_nic_assigned_twice_fails() {
+ let assignments = [assignment("tap100i0", 3), assignment("tap100i1", 3)];
+ assert!(check_assignments(&assignments).is_ok());
+
+ let assignments = [assignment("tap100i0", 3), assignment("tap100i0", 4)];
+ let err = check_assignments(&assignments).unwrap_err().to_string();
+ assert!(err.contains("id 3") && err.contains("id 4"), "{err}");
+ }
+
+ fn placed(id: u16, flags: u16) -> NicEntry {
+ NicEntry {
+ flags,
+ ..nic_entry(id, false)
+ }
+ }
+
+ #[test]
+ fn a_pass_keeps_the_flag_a_plug_set() {
+ let live = HashMap::from([(1, placed(3, NIC_ADMIT_ND))]);
+ let mut desired = HashMap::from([(1, nic_entry(4, false)), (2, nic_entry(5, false))]);
+ carry_flags(&live, &mut desired);
+ assert_eq!(desired[&1], placed(4, NIC_ADMIT_ND));
+ assert_eq!(desired[&2], nic_entry(5, false));
+ }
+
+ #[test]
+ fn a_pair_given_twice_fails() {
+ let pairs = [pair(true, vec![]), pair(false, vec![entry(22)])];
+ assert!(encode_pairs(&pairs).is_err());
+ assert!(encode_pairs(&pairs[..1]).is_ok());
+ }
+
+ #[test]
+ fn a_pair_over_check_max_fails() {
+ let checks: Vec<_> = (0..=CHECK_MAX as u16).map(entry).collect();
+ assert!(encode_pairs(&[pair(true, checks[..CHECK_MAX].to_vec())]).is_ok());
+ assert!(encode_pairs(&[pair(true, checks)]).is_err());
+ }
+
+ fn many_pairs(count: usize, checks: impl Fn(usize) -> Vec<CheckEntry>) -> Vec<Pair> {
+ (0..count)
+ .map(|i| Pair {
+ src_id: (i >> 16) as u16,
+ dst_id: i as u16,
+ checks: checks(i),
+ allow: true,
+ })
+ .collect()
+ }
+
+ // run against small ceilings, the real ones cost a quarter GiB per test for the same check
+ #[test]
+ fn more_pairs_than_the_pairs_map_holds_fail() {
+ let mut pairs = many_pairs(9, |_| vec![]);
+ let err = encode_within(&pairs, 8, 8).err().unwrap().to_string();
+ assert!(err.contains("9 pairs"), "{err}");
+ pairs.pop();
+ assert!(encode_within(&pairs, 8, 8).is_ok());
+ }
+
+ #[test]
+ fn more_rulesets_than_a_pass_may_name_fail() {
+ let own = |i: usize| (0..=CHECK_INLINE).map(|n| entry((i + n) as u16)).collect();
+ let mut pairs = many_pairs(5, own);
+ let err = encode_within(&pairs, 8, 4).err().unwrap().to_string();
+ assert!(err.contains("5 rulesets"), "{err}");
+ pairs.pop();
+ assert!(encode_within(&pairs, 8, 4).is_ok());
+
+ let shared = |_| (0..=CHECK_INLINE).map(|n| entry(n as u16)).collect();
+ assert!(encode_within(&many_pairs(5, shared), 8, 4).is_ok());
+ }
+}
diff --git a/proxmox-ebpf/src/policy/types.rs b/proxmox-ebpf/src/policy/types.rs
new file mode 100644
index 000000000000..e411ba4ae6f2
--- /dev/null
+++ b/proxmox-ebpf/src/policy/types.rs
@@ -0,0 +1,155 @@
+//! Keep in sync with `bpf/types.h`.
+
+/// The checks one pair can carry, which also sizes a ruleset and bounds its walk.
+pub const CHECK_MAX: usize = 512;
+pub const CHECK_INLINE: usize = 7;
+/// The pairs a pass may name, the size of the pairs map.
+pub const PAIRS_MAX: usize = 1 << 20;
+/// The rulesets a pass may name. The map holds twice that, since the ones a pass replaces stay
+/// until its pairs are written.
+pub const RULESETS_MAX: usize = 1 << 14;
+/// The NIC entries a host can hold, the size of the interface map.
+pub const NICS_MAX: usize = 1 << 18;
+
+pub const ID_UNSTAMPED: u16 = 0;
+
+pub const PAIR_ADMIT_ND: u8 = 1 << 0;
+pub const PAIR_HAS_RULESET: u8 = 1 << 1;
+
+pub const NIC_ADMIT_ND: u16 = 1 << 0;
+
+pub const FAMILY_V4: u8 = 4;
+pub const FAMILY_V6: u8 = 6;
+
+pub const CHECK_HAS_SPORT: u8 = 1 << 0;
+pub const CHECK_HAS_DPORT: u8 = 1 << 1;
+pub const CHECK_HAS_ICMP_TYPE: u8 = 1 << 2;
+pub const CHECK_HAS_ICMP_CODE: u8 = 1 << 3;
+
+#[repr(C)]
+#[derive(Copy, Clone, Debug, PartialEq, Eq)]
+pub struct NicEntry {
+ pub id: u16,
+ pub flags: u16,
+ pub _pad: [u8; 4],
+}
+
+#[repr(C)]
+#[derive(Copy, Clone, Debug, Hash, PartialEq, Eq)]
+pub struct PairKey {
+ pub src_id: u16,
+ pub dst_id: u16,
+}
+
+#[repr(C)]
+#[derive(Copy, Clone, Debug, Default, Hash, PartialEq, Eq)]
+pub struct CheckValue {
+ pub ip_proto: u8,
+ pub family: u8,
+ pub match_flags: u8,
+ pub allow: u8,
+ pub sport_lo: u16,
+ pub sport_hi: u16,
+ pub dport_lo: u16,
+ pub dport_hi: u16,
+ pub icmp_type: u8,
+ pub icmp_code: u8,
+ pub _pad: [u8; 2],
+}
+
+#[repr(C)]
+#[derive(Copy, Clone, Debug, PartialEq, Eq)]
+pub struct PairValue {
+ pub allow: u8,
+ pub flags: u8,
+ pub inline_count: u8,
+ pub _pad: [u8; 5],
+ pub ruleset: u64,
+ pub checks: [CheckValue; CHECK_INLINE],
+}
+
+#[repr(C)]
+#[derive(Copy, Clone, Debug, PartialEq, Eq)]
+pub struct RulesetValue {
+ pub count: u16,
+ pub _pad: [u8; 14],
+ pub checks: [CheckValue; CHECK_MAX],
+}
+
+unsafe impl aya::Pod for NicEntry {}
+unsafe impl aya::Pod for PairKey {}
+unsafe impl aya::Pod for PairValue {}
+unsafe impl aya::Pod for RulesetValue {}
+
+#[cfg(test)]
+mod layout {
+ use super::*;
+ use core::mem::{offset_of, size_of};
+
+ #[test]
+ fn sizes() {
+ assert_eq!(size_of::<NicEntry>(), 8);
+ assert_eq!(size_of::<PairKey>(), 4);
+ assert_eq!(size_of::<CheckValue>(), 16);
+ assert_eq!(size_of::<PairValue>(), 128);
+ assert_eq!(size_of::<RulesetValue>(), 16 + CHECK_MAX * 16);
+ }
+
+ #[test]
+ fn offsets() {
+ assert_eq!(offset_of!(PairKey, dst_id), 2);
+ assert_eq!(offset_of!(NicEntry, flags), 2);
+ assert_eq!(offset_of!(CheckValue, sport_lo), 4);
+ assert_eq!(offset_of!(CheckValue, dport_lo), 8);
+ assert_eq!(offset_of!(CheckValue, icmp_type), 12);
+ assert_eq!(offset_of!(PairValue, allow), 0);
+ assert_eq!(offset_of!(PairValue, flags), 1);
+ assert_eq!(offset_of!(PairValue, inline_count), 2);
+ assert_eq!(offset_of!(PairValue, ruleset), 8);
+ assert_eq!(offset_of!(PairValue, checks), 16);
+ assert_eq!(offset_of!(RulesetValue, count), 0);
+ assert_eq!(offset_of!(RulesetValue, checks), 16);
+ }
+
+ #[test]
+ fn limits() {
+ let header = include_str!("bpf/types.h");
+ let defined = |name: &str| -> usize {
+ let prefix = format!("#define {name} ");
+ let value = header
+ .lines()
+ .find_map(|line| line.strip_prefix(prefix.as_str()))
+ .unwrap_or_else(|| panic!("the header defines no {name}"))
+ .trim();
+ let parsed = match value
+ .strip_prefix("(1 << ")
+ .and_then(|v| v.strip_suffix(')'))
+ {
+ Some(shift) => shift.parse().map(|shift: u32| 1 << shift),
+ None => value.parse(),
+ };
+ parsed.unwrap_or_else(|_| panic!("the header defines {name} as {value}, not a number"))
+ };
+ assert_eq!(defined("CHECK_MAX"), CHECK_MAX);
+ assert_eq!(defined("CHECK_INLINE"), CHECK_INLINE);
+ assert_eq!(defined("PAIRS_MAX"), PAIRS_MAX);
+ assert_eq!(defined("RULESETS_MAX"), RULESETS_MAX);
+ assert_eq!(defined("NICS_MAX"), NICS_MAX);
+ assert_eq!(defined("ID_UNSTAMPED"), usize::from(ID_UNSTAMPED));
+ assert_eq!(defined("PAIR_ADMIT_ND"), usize::from(PAIR_ADMIT_ND));
+ assert_eq!(defined("PAIR_HAS_RULESET"), usize::from(PAIR_HAS_RULESET));
+ assert_eq!(defined("NIC_ADMIT_ND"), usize::from(NIC_ADMIT_ND));
+ assert_eq!(defined("FAMILY_V4"), usize::from(FAMILY_V4));
+ assert_eq!(defined("FAMILY_V6"), usize::from(FAMILY_V6));
+ assert_eq!(defined("CHECK_HAS_SPORT"), usize::from(CHECK_HAS_SPORT));
+ assert_eq!(defined("CHECK_HAS_DPORT"), usize::from(CHECK_HAS_DPORT));
+ assert_eq!(
+ defined("CHECK_HAS_ICMP_TYPE"),
+ usize::from(CHECK_HAS_ICMP_TYPE)
+ );
+ assert_eq!(
+ defined("CHECK_HAS_ICMP_CODE"),
+ usize::from(CHECK_HAS_ICMP_CODE)
+ );
+ }
+}
diff --git a/proxmox-ebpf/src/subsystem.rs b/proxmox-ebpf/src/subsystem.rs
index 274c11fdb22a..9a1fec7057f5 100644
--- a/proxmox-ebpf/src/subsystem.rs
+++ b/proxmox-ebpf/src/subsystem.rs
@@ -52,7 +52,7 @@ const ORDER_FILE: &str = "order";
/// missing from the list fails the build. The newest build of the crate on a host decides the
/// order, a new name takes effect with its next full apply, a moved or removed one for the links
/// attached after that, since already attached links stay where they are.
-pub const RUN_ORDER: &[&str] = &[];
+pub const RUN_ORDER: &[&str] = &["policy"];
const fn check_listed(name: &str) {
let mut i = 0;
@@ -874,6 +874,21 @@ impl TcPrograms {
Ok(ifindexes)
}
+ fn detach_gone(&self) -> anyhow::Result<Vec<u32>> {
+ let mut gone = Vec::new();
+ for pin in self.live_links()? {
+ // an index a new netdev took is not gone, the state under it may be the new one's
+ if tc::iface_name(pin.ifindex)?.is_some() {
+ continue;
+ }
+ self.reclaim_link(&pin)?;
+ if !gone.contains(&pin.ifindex) {
+ gone.push(pin.ifindex);
+ }
+ }
+ Ok(gone)
+ }
+
// unpin a link. A pin that cannot be opened stays, a failed open says nothing about the link
// behind it
fn reclaim_link(&self, pin: &tc::LinkPin) -> anyhow::Result<()> {
@@ -1271,6 +1286,12 @@ impl<M> ApplyLock<'_, M> {
pub fn attached(&self) -> anyhow::Result<Vec<tc::LinkPin>> {
self.programs.live_links()
}
+
+ /// Unpin the links of interfaces that are gone and return their indexes, for state keyed by
+ /// them. The kernel dropped such a link with its netdev, only the pin is left.
+ pub fn detach_gone(&self) -> anyhow::Result<Vec<u32>> {
+ self.programs.detach_gone()
+ }
}
impl ApplyLock<'_, Exclusive> {
diff --git a/proxmox-ebpf/tests/common/mod.rs b/proxmox-ebpf/tests/common/mod.rs
index ad03710fb787..5a76710a8344 100644
--- a/proxmox-ebpf/tests/common/mod.rs
+++ b/proxmox-ebpf/tests/common/mod.rs
@@ -16,6 +16,7 @@ use std::ffi::{c_int, c_long, c_void};
pub const TC_ACT_UNSPEC: c_int = -1;
pub const TC_ACT_OK: c_int = 0;
+pub const TC_ACT_SHOT: c_int = 2;
pub const TC_ACT_REDIRECT: c_int = 7;
pub const PKT_CAP: usize = 2048;
@@ -28,6 +29,7 @@ pub struct SkBuff {
pub len: u32,
pub ifindex: u32,
pub mark: u32,
+ pub ingress_ifindex: u32,
}
/// The skb handed to a program plus the buffer behind it. The helpers get the skb pointer and
@@ -50,6 +52,9 @@ impl TestSkb {
len: packet.len() as u32,
ifindex,
mark: 0,
+ // a frame from a guest or the underlay came in on some port, only the host's own
+ // frames have none
+ ingress_ifindex: 1,
},
buf,
};
@@ -67,10 +72,18 @@ impl TestSkb {
&self.buf[..self.skb.len as usize]
}
+ pub fn set_mark(&mut self, mark: u32) {
+ self.skb.mark = mark;
+ }
+
pub fn mark(&self) -> u32 {
self.skb.mark
}
+ pub fn set_ingress_ifindex(&mut self, ifindex: u32) {
+ self.skb.ingress_ifindex = ifindex;
+ }
+
pub fn run(&mut self, prog: unsafe extern "C" fn(*mut SkBuff) -> c_int) -> c_int {
REDIRECTED.with(|r| *r.borrow_mut() = None);
unsafe { prog((self as *mut TestSkb).cast()) }
@@ -108,6 +121,11 @@ pub fn map_insert(map: *const c_void, key: &[u8], value: &[u8]) {
});
}
+pub fn bytes_of<T: aya::Pod>(value: &T) -> &[u8] {
+ // SAFETY: aya's Pod contract is that the type can be converted to a byte slice
+ unsafe { std::slice::from_raw_parts((value as *const T).cast::<u8>(), size_of::<T>()) }
+}
+
/// The ifindex of the redirect the last run issued, if any.
pub fn redirected() -> Option<u32> {
REDIRECTED.with(|r| r.borrow().map(|(ifindex, _)| ifindex))
@@ -210,3 +228,21 @@ pub extern "C" fn bpf_redirect(ifindex: u32, flags: u64) -> c_long {
REDIRECTED.with(|r| *r.borrow_mut() = Some((ifindex, flags)));
TC_ACT_REDIRECT as c_long
}
+
+#[unsafe(no_mangle)]
+pub extern "C" fn bpf_loop(
+ nr_loops: u32,
+ callback_fn: *const c_void,
+ callback_ctx: *mut c_void,
+ _flags: u64,
+) -> c_long {
+ let callback: extern "C" fn(u64, *mut c_void) -> c_long =
+ unsafe { std::mem::transmute(callback_fn) };
+ // like the kernel, a nonzero return stops the loop, and the count includes that call
+ for i in 0..nr_loops {
+ if callback(u64::from(i), callback_ctx) != 0 {
+ return c_long::from(i) + 1;
+ }
+ }
+ c_long::from(nr_loops)
+}
diff --git a/proxmox-ebpf/tests/policy.rs b/proxmox-ebpf/tests/policy.rs
new file mode 100644
index 000000000000..11c513c69945
--- /dev/null
+++ b/proxmox-ebpf/tests/policy.rs
@@ -0,0 +1,795 @@
+//! The policy programs run natively, a crafted frame in and its verdict or mark out. The map
+//! images are built here rather than by the subsystem's encoder, so an encoder bug cannot hide a
+//! program bug.
+
+#![cfg(feature = "policy")]
+
+mod common;
+
+use std::ffi::{c_int, c_void};
+
+use common::*;
+use proxmox_ebpf::policy::types::*;
+
+unsafe extern "C" {
+ fn tc_tap_ingress(skb: *mut SkBuff) -> c_int;
+ fn tc_tap_egress(skb: *mut SkBuff) -> c_int;
+ static nics: u8;
+ static pairs: u8;
+ static rulesets: u8;
+}
+
+const IFINDEX: u32 = 7;
+const SRC_ID: u16 = 3;
+const DST_ID: u16 = 5;
+const RULESET_KEY: u64 = 0x5eed;
+
+const ETHERTYPE_IPV4: u16 = 0x0800;
+const ETHERTYPE_ARP: u16 = 0x0806;
+const ETHERTYPE_IPV6: u16 = 0x86dd;
+
+const PROTO_ICMP: u8 = 1;
+const PROTO_TCP: u8 = 6;
+const PROTO_UDP: u8 = 17;
+const PROTO_FRAGMENT: u8 = 44;
+const PROTO_ESP: u8 = 50;
+const PROTO_AH: u8 = 51;
+const PROTO_ICMPV6: u8 = 58;
+const PROTO_NONE: u8 = 59;
+const PROTO_DSTOPTS: u8 = 60;
+
+fn map(symbol: *const u8) -> *const c_void {
+ symbol.cast()
+}
+
+/// One check of a pair value.
+#[derive(Default)]
+struct Check {
+ ip_proto: u8,
+ family: u8,
+ sport: Option<(u16, u16)>,
+ dport: Option<(u16, u16)>,
+ icmp_type: Option<u8>,
+ icmp_code: Option<u8>,
+ allow: bool,
+}
+
+impl Check {
+ fn value(&self) -> CheckValue {
+ let mut v = CheckValue {
+ ip_proto: self.ip_proto,
+ family: self.family,
+ allow: self.allow as u8,
+ ..Default::default()
+ };
+ if let Some((lo, hi)) = self.sport {
+ v.match_flags |= CHECK_HAS_SPORT;
+ v.sport_lo = lo;
+ v.sport_hi = hi;
+ }
+ if let Some((lo, hi)) = self.dport {
+ v.match_flags |= CHECK_HAS_DPORT;
+ v.dport_lo = lo;
+ v.dport_hi = hi;
+ }
+ if let Some(ty) = self.icmp_type {
+ v.match_flags |= CHECK_HAS_ICMP_TYPE;
+ v.icmp_type = ty;
+ }
+ if let Some(code) = self.icmp_code {
+ v.match_flags |= CHECK_HAS_ICMP_CODE;
+ v.icmp_code = code;
+ }
+ v
+ }
+}
+
+/// The maps as a pass leaves them with this tap's entry and no pair yet.
+fn install_none() {
+ install_tap(0);
+}
+
+fn install_tap(flags: u16) {
+ register_map(map(&raw const nics), 4);
+ let entry = NicEntry {
+ id: DST_ID,
+ flags,
+ _pad: [0; 4],
+ };
+ map_insert(
+ map(&raw const nics),
+ &IFINDEX.to_ne_bytes(),
+ bytes_of(&entry),
+ );
+ register_map(map(&raw const pairs), 4);
+ register_map(map(&raw const rulesets), 8);
+}
+
+/// The maps as a pass leaves them for one pair, its own verdict plus the checks ahead of it.
+fn install(allow: bool, entries: &[Check]) {
+ install_none();
+ insert_pair(SRC_ID, DST_ID, allow, entries);
+}
+
+/// One pair on top of what is installed, nothing is cleared.
+fn insert_pair(src: u16, dst: u16, allow: bool, entries: &[Check]) {
+ let key = PairKey {
+ src_id: src,
+ dst_id: dst,
+ };
+ let (inline, rest) = entries.split_at(entries.len().min(CHECK_INLINE));
+ let mut value = PairValue {
+ allow: allow as u8,
+ flags: 0,
+ inline_count: inline.len() as u8,
+ _pad: [0; 5],
+ ruleset: 0,
+ checks: [CheckValue::default(); CHECK_INLINE],
+ };
+ if allow || entries.iter().any(|check| check.allow) {
+ value.flags |= PAIR_ADMIT_ND;
+ }
+ if !rest.is_empty() {
+ value.flags |= PAIR_HAS_RULESET;
+ value.ruleset = RULESET_KEY;
+ }
+ for (slot, check) in value.checks.iter_mut().zip(inline) {
+ *slot = check.value();
+ }
+ map_insert(map(&raw const pairs), bytes_of(&key), bytes_of(&value));
+
+ if !rest.is_empty() {
+ let mut ruleset = RulesetValue {
+ count: rest.len() as u16,
+ _pad: [0; 14],
+ checks: [CheckValue::default(); CHECK_MAX],
+ };
+ for (slot, check) in ruleset.checks.iter_mut().zip(rest) {
+ *slot = check.value();
+ }
+ map_insert(
+ map(&raw const rulesets),
+ &RULESET_KEY.to_ne_bytes(),
+ bytes_of(&ruleset),
+ );
+ }
+}
+
+fn run(frame: &[u8]) -> c_int {
+ run_from(SRC_ID, frame)
+}
+
+fn run_from(src_id: u16, frame: &[u8]) -> c_int {
+ let mut skb = TestSkb::new(frame, IFINDEX);
+ skb.set_mark(u32::from(src_id));
+ skb.run(tc_tap_egress)
+}
+
+fn ethernet(ethertype: u16, tags: usize, payload: &[u8]) -> Vec<u8> {
+ tagged(&vec![0x8100; tags], ethertype, payload)
+}
+
+fn tagged(tags: &[u16], ethertype: u16, payload: &[u8]) -> Vec<u8> {
+ let mut frame = vec![0x02; 12];
+ for tag in tags {
+ frame.extend_from_slice(&tag.to_be_bytes());
+ frame.extend_from_slice(&100u16.to_be_bytes());
+ }
+ frame.extend_from_slice(ðertype.to_be_bytes());
+ frame.extend_from_slice(payload);
+ frame
+}
+
+fn ipv4(protocol: u8, frag_off: u16, payload: &[u8]) -> Vec<u8> {
+ ipv4_with_options(protocol, frag_off, &[], payload)
+}
+
+/// An IPv4 header carrying `options`, a multiple of four bytes long.
+fn ipv4_with_options(protocol: u8, frag_off: u16, options: &[u8], payload: &[u8]) -> Vec<u8> {
+ let header_len = 20 + options.len();
+ let mut header = vec![0x40 | (header_len / 4) as u8, 0];
+ header.extend_from_slice(&((header_len + payload.len()) as u16).to_be_bytes());
+ header.extend_from_slice(&[0, 0]);
+ header.extend_from_slice(&frag_off.to_be_bytes());
+ header.extend_from_slice(&[64, protocol, 0, 0]);
+ header.extend_from_slice(&[10, 0, 0, 1]);
+ header.extend_from_slice(&[10, 0, 0, 2]);
+ header.extend_from_slice(options);
+ header.extend_from_slice(payload);
+ header
+}
+
+fn ipv6(next_header: u8, payload: &[u8]) -> Vec<u8> {
+ let mut header = vec![0x60, 0, 0, 0];
+ header.extend_from_slice(&(payload.len() as u16).to_be_bytes());
+ header.extend_from_slice(&[next_header, 64]);
+ header.extend_from_slice(&[0; 32]);
+ header.extend_from_slice(payload);
+ header
+}
+
+/// A fragment header for the fragment at `offset`, in units of eight bytes.
+fn fragment_header(next_header: u8, offset: u16) -> Vec<u8> {
+ let mut header = vec![next_header, 0];
+ // the offset sits in the top 13 bits, the low bit says more fragments follow
+ header.extend_from_slice(&((offset << 3) | 1).to_be_bytes());
+ header.extend_from_slice(&[0, 0, 0, 1]);
+ header
+}
+
+/// A destination options header of the minimum length, eight bytes.
+fn ext_header(next_header: u8) -> Vec<u8> {
+ long_ext_header(next_header, 0)
+}
+
+/// A destination options header of `units` eight byte units past the first.
+fn long_ext_header(next_header: u8, units: u8) -> Vec<u8> {
+ let mut header = vec![next_header, units];
+ header.extend_from_slice(&vec![1; 6 + usize::from(units) * 8]);
+ header
+}
+
+/// An authentication header of the minimum length, twelve bytes and no integrity check value.
+fn ah_header(next_header: u8) -> Vec<u8> {
+ // the length counts four byte units minus two
+ let mut header = vec![next_header, 1, 0, 0];
+ header.extend_from_slice(&[0, 0, 0, 1]);
+ header.extend_from_slice(&[0, 0, 0, 1]);
+ header
+}
+
+fn tcp(sport: u16, dport: u16) -> Vec<u8> {
+ let mut header = sport.to_be_bytes().to_vec();
+ header.extend_from_slice(&dport.to_be_bytes());
+ header.extend_from_slice(&[0; 16]);
+ header
+}
+
+fn udp(sport: u16, dport: u16) -> Vec<u8> {
+ let mut header = sport.to_be_bytes().to_vec();
+ header.extend_from_slice(&dport.to_be_bytes());
+ header.extend_from_slice(&[0, 8, 0, 0]);
+ header
+}
+
+fn icmpv6(ty: u8) -> Vec<u8> {
+ let mut header = vec![ty, 0, 0, 0];
+ header.extend_from_slice(&[0; 20]);
+ header
+}
+
+fn icmpv6_frame(ty: u8) -> Vec<u8> {
+ ethernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_ICMPV6, &icmpv6(ty)))
+}
+
+fn solicitation() -> Vec<u8> {
+ icmpv6_frame(135)
+}
+
+fn advertisement() -> Vec<u8> {
+ icmpv6_frame(136)
+}
+
+fn icmp_frame(ty: u8, code: u8) -> Vec<u8> {
+ let mut header = vec![ty, code, 0, 0];
+ header.extend_from_slice(&[0; 4]);
+ ethernet(ETHERTYPE_IPV4, 0, &ipv4(PROTO_ICMP, 0, &header))
+}
+
+fn arp() -> Vec<u8> {
+ ethernet(ETHERTYPE_ARP, 0, &[0; 28])
+}
+
+fn tcp_frame(sport: u16, dport: u16) -> Vec<u8> {
+ ethernet(ETHERTYPE_IPV4, 0, &ipv4(PROTO_TCP, 0, &tcp(sport, dport)))
+}
+
+fn tcp6_frame(sport: u16, dport: u16) -> Vec<u8> {
+ ethernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_TCP, &tcp(sport, dport)))
+}
+
+/// Allowing checks on the destination ports 1 to `count`, which the test frames miss unless they
+/// aim at one. Placed first, `CHECK_INLINE` of them push what follows into the ruleset.
+fn filler(ip_proto: u8, count: u16) -> Vec<Check> {
+ (1..=count)
+ .map(|port| Check {
+ ip_proto,
+ dport: Some((port, port)),
+ allow: true,
+ ..Check::default()
+ })
+ .collect()
+}
+
+fn ssh() -> Check {
+ Check {
+ ip_proto: PROTO_TCP,
+ dport: Some((20, 22)),
+ allow: true,
+ ..Check::default()
+ }
+}
+
+fn deny_ssh() -> Check {
+ Check {
+ allow: false,
+ ..ssh()
+ }
+}
+
+#[test]
+fn a_matching_port_takes_the_checks_verdict() {
+ install(false, &[ssh()]);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_port_outside_the_range_falls_to_the_pair() {
+ install(false, &[ssh()]);
+ assert_eq!(run(&tcp_frame(40000, 80)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_narrowed_deny_drops_only_what_it_matches() {
+ install(true, &[deny_ssh()]);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+ assert_eq!(run(&tcp_frame(40000, 80)), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_denied_pair_admits_arp_and_neighbor_discovery() {
+ install(false, &[ssh()]);
+ let echo = icmpv6_frame(128);
+
+ assert_eq!(run(&arp()), TC_ACT_UNSPEC);
+ assert_eq!(run(&solicitation()), TC_ACT_UNSPEC);
+ assert_eq!(run(&advertisement()), TC_ACT_UNSPEC);
+ assert_eq!(run(&echo), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_narrowed_deny_of_neighbor_discovery_stays_in_charge() {
+ install(
+ false,
+ &[
+ Check {
+ ip_proto: PROTO_ICMPV6,
+ family: 6,
+ icmp_type: Some(135),
+ ..Check::default()
+ },
+ ssh(),
+ ],
+ );
+ assert_eq!(run(&solicitation()), TC_ACT_SHOT);
+ assert_eq!(run(&arp()), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_later_fragment_matches_nothing() {
+ let fragment = ethernet(
+ ETHERTYPE_IPV4,
+ 0,
+ // the frame carries bytes 1480 onward of the datagram, so its first bytes only look
+ // like a TCP header
+ &ipv4(PROTO_TCP, 185, &tcp(40000, 22)),
+ );
+ install(false, &[ssh()]);
+ assert_eq!(run(&fragment), TC_ACT_SHOT);
+
+ install(true, &[deny_ssh()]);
+ assert_eq!(run(&fragment), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn one_vlan_tag_is_parsed_through() {
+ install(
+ false,
+ &[Check {
+ ip_proto: PROTO_UDP,
+ sport: Some((53, 53)),
+ allow: true,
+ ..Check::default()
+ }],
+ );
+ let dns = ipv4(PROTO_UDP, 0, &udp(53, 40000));
+ assert_eq!(run(ðernet(ETHERTYPE_IPV4, 1, &dns)), TC_ACT_UNSPEC);
+ assert_eq!(run(ðernet(ETHERTYPE_IPV4, 2, &dns)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_second_tag_in_the_data_is_unreadable() {
+ install(true, &[deny_ssh()]);
+ let ssh = ipv4(PROTO_TCP, 0, &tcp(40000, 22));
+ let web = ipv4(PROTO_TCP, 0, &tcp(40000, 80));
+ assert_eq!(run(&tagged(&[0x8100], ETHERTYPE_IPV4, &ssh)), TC_ACT_SHOT);
+ assert_eq!(run(&tagged(&[0x88a8], ETHERTYPE_IPV4, &ssh)), TC_ACT_SHOT);
+ assert_eq!(run(&tagged(&[0x88a8], ETHERTYPE_IPV4, &web)), TC_ACT_UNSPEC);
+ let two_tags = tagged(&[0x8100, 0x8100], ETHERTYPE_IPV4, &web);
+ let mixed_tags = tagged(&[0x8100, 0x88a8], ETHERTYPE_IPV4, &web);
+ assert_eq!(run(&two_tags), TC_ACT_SHOT);
+ assert_eq!(run(&mixed_tags), TC_ACT_SHOT);
+}
+
+#[test]
+fn an_extension_header_is_walked_past() {
+ install(
+ false,
+ &[Check {
+ ip_proto: PROTO_TCP,
+ dport: Some((443, 443)),
+ allow: true,
+ ..Check::default()
+ }],
+ );
+ let mut payload = ext_header(PROTO_TCP);
+ payload.extend_from_slice(&tcp(40000, 443));
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_DSTOPTS, &payload))),
+ TC_ACT_UNSPEC
+ );
+}
+
+#[test]
+fn the_icmp_code_narrows_the_match() {
+ install(
+ false,
+ &[Check {
+ ip_proto: PROTO_ICMP,
+ icmp_type: Some(3),
+ icmp_code: Some(4),
+ allow: true,
+ ..Check::default()
+ }],
+ );
+ assert_eq!(run(&icmp_frame(3, 4)), TC_ACT_UNSPEC);
+ assert_eq!(run(&icmp_frame(3, 1)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_family_pinned_check_skips_the_other_family() {
+ install(false, &[Check { family: 6, ..ssh() }]);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+ assert_eq!(run(&tcp6_frame(40000, 22)), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_port_check_skips_a_frame_without_ports() {
+ install(
+ false,
+ &[Check {
+ ip_proto: PROTO_ICMP,
+ dport: Some((0, u16::MAX)),
+ allow: true,
+ ..Check::default()
+ }],
+ );
+ assert_eq!(run(&icmp_frame(8, 0)), TC_ACT_SHOT);
+
+ install(
+ false,
+ &[Check {
+ ip_proto: PROTO_ICMP,
+ allow: true,
+ ..Check::default()
+ }],
+ );
+ assert_eq!(run(&icmp_frame(8, 0)), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_one_way_allow_resolves_addresses_in_reverse() {
+ install_none();
+ insert_pair(DST_ID, SRC_ID, true, &[]);
+ assert_eq!(run(&arp()), TC_ACT_UNSPEC);
+ assert_eq!(run(&advertisement()), TC_ACT_UNSPEC);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+
+ // a narrowed allow in reverse admits as well, a plain deny does not
+ insert_pair(DST_ID, SRC_ID, false, &[ssh()]);
+ assert_eq!(run(&arp()), TC_ACT_UNSPEC);
+ insert_pair(DST_ID, SRC_ID, false, &[]);
+ assert_eq!(run(&arp()), TC_ACT_SHOT);
+
+ // an explicit deny of the forward pair is rescued the same way
+ insert_pair(SRC_ID, DST_ID, false, &[]);
+ insert_pair(DST_ID, SRC_ID, true, &[]);
+ assert_eq!(run(&arp()), TC_ACT_UNSPEC);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_host_made_reply_passes_unstamped() {
+ install_none();
+ let from_host = |frame: &[u8]| {
+ let mut skb = TestSkb::new(frame, IFINDEX);
+ skb.set_ingress_ifindex(0);
+ skb.run(tc_tap_egress)
+ };
+ assert_eq!(from_host(&arp()), TC_ACT_UNSPEC);
+ assert_eq!(from_host(&advertisement()), TC_ACT_UNSPEC);
+ // the host's IP traffic still needs its allow
+ assert_eq!(from_host(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+ // an unstamped reply that came in on a port is a guest's or the underlay's
+ assert_eq!(run_from(0, &arp()), TC_ACT_SHOT);
+ // a stamped frame is never the host's
+ let mut stamped = TestSkb::new(&arp(), IFINDEX);
+ stamped.set_mark(u32::from(SRC_ID));
+ stamped.set_ingress_ifindex(0);
+ assert_eq!(stamped.run(tc_tap_egress), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_frame_the_host_turned_around_passes_unstamped() {
+ install_none();
+ let turned = |src_id: u16, frame: &[u8]| {
+ let mut skb = TestSkb::new(frame, IFINDEX);
+ skb.set_mark(u32::from(src_id));
+ skb.set_ingress_ifindex(IFINDEX);
+ skb.run(tc_tap_egress)
+ };
+ // only a program ahead of the policy on the tap makes such a frame, so all of it passes
+ assert_eq!(turned(0, &tcp_frame(40000, 22)), TC_ACT_UNSPEC);
+ assert_eq!(turned(0, &arp()), TC_ACT_UNSPEC);
+ // a guest frame the bridge reflects carries the stamp and needs its pair
+ assert_eq!(turned(DST_ID, &tcp_frame(40000, 22)), TC_ACT_SHOT);
+ // unstamped from another port is a guest's or the underlay's
+ assert_eq!(run_from(0, &tcp_frame(40000, 22)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_flagged_tap_passes_unstamped_arp_and_nd_from_any_port() {
+ install_tap(NIC_ADMIT_ND);
+ // what the host made came in on the bridge's veth, like a frame of any other sender
+ assert_eq!(run_from(0, &arp()), TC_ACT_UNSPEC);
+ assert_eq!(run_from(0, &advertisement()), TC_ACT_UNSPEC);
+ assert_eq!(run_from(0, &tcp_frame(40000, 22)), TC_ACT_SHOT);
+ // a stamped sender still needs a pair that admits something
+ assert_eq!(run_from(SRC_ID, &arp()), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_missing_pair_denies() {
+ install(true, &[]);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_UNSPEC);
+ assert_eq!(run_from(SRC_ID + 1, &tcp_frame(40000, 22)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_pair_without_checks_takes_its_own_verdict() {
+ install(true, &[]);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_UNSPEC);
+ assert_eq!(run(&arp()), TC_ACT_UNSPEC);
+
+ install(false, &[]);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+ assert_eq!(run(&arp()), TC_ACT_SHOT);
+}
+
+#[test]
+fn an_ipv6_fragment_matches_only_as_the_first() {
+ install(false, &[ssh()]);
+ let mut first = fragment_header(PROTO_TCP, 0);
+ first.extend_from_slice(&tcp(40000, 22));
+ let mut later = fragment_header(PROTO_TCP, 185);
+ later.extend_from_slice(&tcp(40000, 22));
+
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_FRAGMENT, &first))),
+ TC_ACT_UNSPEC
+ );
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_FRAGMENT, &later))),
+ TC_ACT_SHOT
+ );
+}
+
+#[test]
+fn ingress_stamps_the_id_into_the_low_bits_only() {
+ install(true, &[]);
+ let mut skb = TestSkb::new(&tcp_frame(40000, 22), IFINDEX);
+ skb.set_mark(0xabcd_1234);
+ assert_eq!(skb.run(tc_tap_ingress), TC_ACT_UNSPEC);
+ assert_eq!(skb.mark(), 0xabcd_0000 | u32::from(DST_ID));
+
+ let mut unknown = TestSkb::new(&tcp_frame(40000, 22), IFINDEX + 1);
+ unknown.set_mark(0xabcd_1234);
+ assert_eq!(unknown.run(tc_tap_ingress), TC_ACT_UNSPEC);
+ assert_eq!(unknown.mark(), 0xabcd_1234);
+}
+
+#[test]
+fn a_check_past_the_inline_ones_decides_through_the_ruleset() {
+ let mut checks = filler(PROTO_TCP, CHECK_INLINE as u16);
+ checks.push(ssh());
+ install(false, &checks);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_UNSPEC);
+ assert_eq!(run(&tcp_frame(40000, 80)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_ruleset_deny_drops_what_it_matches() {
+ let mut checks = filler(PROTO_TCP, CHECK_INLINE as u16);
+ checks.push(deny_ssh());
+ install(true, &checks);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+ assert_eq!(run(&tcp_frame(40000, 80)), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_missing_ruleset_drops() {
+ let mut checks = filler(PROTO_TCP, CHECK_INLINE as u16);
+ checks.push(deny_ssh());
+ install(true, &checks);
+ // the pair keeps naming the ruleset while the map holds none
+ register_map(map(&raw const rulesets), 8);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+ assert_eq!(run(&tcp_frame(40000, 80)), TC_ACT_SHOT);
+ // a check ahead of the ruleset still decides
+ assert_eq!(run(&tcp_frame(40000, 1)), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_long_extension_header_is_walked_past() {
+ install(
+ false,
+ &[Check {
+ ip_proto: PROTO_TCP,
+ dport: Some((443, 443)),
+ allow: true,
+ ..Check::default()
+ }],
+ );
+ // read with the length ignored, the second unit's bytes would pass for the ports
+ let mut payload = long_ext_header(PROTO_TCP, 1);
+ payload.extend_from_slice(&tcp(40000, 443));
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_DSTOPTS, &payload))),
+ TC_ACT_UNSPEC
+ );
+}
+
+#[test]
+fn a_chain_past_the_bound_is_dropped() {
+ let chain = |headers: usize| {
+ let mut payload = Vec::new();
+ for index in 0..headers {
+ let next = if index + 1 < headers {
+ PROTO_DSTOPTS
+ } else {
+ PROTO_TCP
+ };
+ payload.extend_from_slice(&ext_header(next));
+ }
+ payload.extend_from_slice(&tcp(40000, 22));
+ ethernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_DSTOPTS, &payload))
+ };
+
+ install(false, &[ssh()]);
+ assert_eq!(run(&chain(12)), TC_ACT_UNSPEC);
+ assert_eq!(run(&chain(13)), TC_ACT_SHOT);
+
+ install(true, &[deny_ssh()]);
+ // a chain within the bound reaches L4 and the narrowed check decides, while a longer chain is
+ // dropped even though the pair allows
+ assert_eq!(run(&chain(12)), TC_ACT_SHOT);
+ assert_eq!(run(&chain(13)), TC_ACT_SHOT);
+}
+
+#[test]
+fn an_unreadable_l4_header_is_dropped_where_checks_apply() {
+ let mut short6 = fragment_header(PROTO_TCP, 0);
+ short6.extend_from_slice(&tcp(40000, 22)[..2]);
+ let short6 = ethernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_FRAGMENT, &short6));
+ let short4 = ethernet(
+ ETHERTYPE_IPV4,
+ 0,
+ &ipv4(PROTO_TCP, 0x2000, &tcp(40000, 22)[..2]),
+ );
+
+ let none6 = ethernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_NONE, &[]));
+ let mut whole6 = fragment_header(PROTO_TCP, 0);
+ whole6.extend_from_slice(&tcp(40000, 80));
+ let whole6 = ethernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_FRAGMENT, &whole6));
+
+ install(true, &[deny_ssh()]);
+ assert_eq!(run(&short6), TC_ACT_SHOT);
+ assert_eq!(run(&short4), TC_ACT_SHOT);
+ assert_eq!(run(&none6), TC_ACT_UNSPEC);
+ assert_eq!(run(&whole6), TC_ACT_UNSPEC);
+
+ // without checks the pair's verdict decides as for any other frame
+ install(true, &[]);
+ assert_eq!(run(&short6), TC_ACT_UNSPEC);
+ assert_eq!(run(&short4), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn an_authentication_header_is_walked_past() {
+ install(true, &[deny_ssh()]);
+ let mut payload = ah_header(PROTO_TCP);
+ payload.extend_from_slice(&tcp(40000, 22));
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_AH, &payload))),
+ TC_ACT_SHOT
+ );
+}
+
+#[test]
+fn an_authentication_header_matches_a_check_for_it() {
+ install(
+ false,
+ &[Check {
+ ip_proto: PROTO_AH,
+ allow: true,
+ ..Check::default()
+ }],
+ );
+ // the packet counts as AH for a check naming it and as TCP for every other check, so an
+ // allow for AH admits it as the firewall would while a deny on the TCP behind it still holds
+ let mut payload = ah_header(PROTO_TCP);
+ payload.extend_from_slice(&tcp(40000, 22));
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_AH, &payload))),
+ TC_ACT_UNSPEC
+ );
+}
+
+#[test]
+fn esp_is_not_walked_past() {
+ install(true, &[deny_ssh()]);
+ // the header behind ESP is encrypted, so a TCP check has nothing to match
+ let mut payload = vec![0, 0, 0, 1, 0, 0, 0, 1];
+ payload.extend_from_slice(&tcp(40000, 22));
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_ESP, &payload))),
+ TC_ACT_UNSPEC
+ );
+}
+
+#[test]
+fn ipv4_options_are_skipped() {
+ install(false, &[ssh()]);
+ let frame = ethernet(
+ ETHERTYPE_IPV4,
+ 0,
+ &ipv4_with_options(PROTO_TCP, 0, &[1; 8], &tcp(40000, 22)),
+ );
+ assert_eq!(run(&frame), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn an_interface_without_an_entry_passes() {
+ install(false, &[]);
+ let mut skb = TestSkb::new(&tcp_frame(40000, 22), IFINDEX + 1);
+ skb.set_mark(u32::from(SRC_ID));
+ assert_eq!(skb.run(tc_tap_egress), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn an_inline_check_decides_before_the_ruleset() {
+ let mut checks = filler(PROTO_TCP, CHECK_INLINE as u16);
+ checks[CHECK_INLINE - 1] = deny_ssh();
+ checks.push(ssh());
+ install(true, &checks);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+}
+
+#[test]
+fn the_earlier_ruleset_check_decides() {
+ let mut checks = filler(PROTO_TCP, CHECK_INLINE as u16);
+ checks.push(deny_ssh());
+ checks.push(ssh());
+ install(true, &checks);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+}
+
+#[test]
+fn the_last_check_of_the_longest_pair_decides() {
+ let mut checks = filler(PROTO_TCP, CHECK_INLINE as u16);
+ checks.extend(filler(PROTO_UDP, (CHECK_MAX - CHECK_INLINE - 1) as u16));
+ checks.push(ssh());
+ install(false, &checks);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_UNSPEC);
+ assert_eq!(run(&tcp_frame(40000, 80)), TC_ACT_SHOT);
+}
--
2.47.3
next prev parent reply other threads:[~2026-11-05 11:44 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-05 11:12 [PATCH access-control/cluster/container/docs/manager/network/proxmox{,-ve-rs,-perl-rs}/qemu-server v4 00/45] sdn: add microsegmentation support Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox v4 01/45] ebpf: build: keep the build directory out of the BPF objects Hannes Laimer
2026-11-05 11:12 ` Hannes Laimer [this message]
2026-11-05 11:12 ` [PATCH proxmox v4 03/45] ebpf: policy: add a check for non-first fragments Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-ve-rs v4 04/45] ve-config: firewall: add a constructor for sctp matches Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-ve-rs v4 05/45] ve-config: sdn: microseg: add identity allocation for group sets Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-ve-rs v4 06/45] ve-config: sdn: microseg: add config types Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-ve-rs v4 07/45] ve-config: sdn: microseg: add tag matcher Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-ve-rs v4 08/45] ve-config: sdn: microseg: add name regex matcher Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-ve-rs v4 09/45] ve-config: sdn: microseg: narrow rules by protocol and ports Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-ve-rs v4 10/45] ve-config: sdn: microseg: add group hierarchy Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-ve-rs v4 11/45] ve-config: firewall: expose protocol, icmp type and code numbers Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-ve-rs v4 12/45] ve-config: sdn: microseg: add the desired state Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-ve-rs v4 13/45] microseg: add a crate that applies the policy to a node's guest NICs Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-ve-rs v4 14/45] microseg: add a rule option that matches non-first fragments Hannes Laimer
2026-11-05 11:12 ` [PATCH pve-cluster v4 15/45] cfs: add 'sdn/microseg.cfg' to observed files Hannes Laimer
2026-11-05 11:12 ` [PATCH pve-access-control v4 16/45] permissions: add ACL path for SDN microsegmentation Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-perl-rs v4 17/45] pve-rs: sdn: add microseg config binding Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-perl-rs v4 18/45] pve-rs: sdn: microseg: bind the data plane Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-perl-rs v4 19/45] pve-rs: sdn: microseg: check the data plane limits on render Hannes Laimer
2026-11-05 11:12 ` [PATCH proxmox-perl-rs v4 20/45] pve-rs: debian: fire the pve-microseg-update trigger on upgrade Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-network v4 21/45] sdn: zones: add vxlan-gbp option to vxlan and evpn zones Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-network v4 22/45] sdn: microseg: add config, API and guest inventory Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-network v4 23/45] sdn: dry-run: show pending microseg changes Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-network v4 24/45] sdn: microseg: enforce the policy on guest NICs Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-network v4 25/45] sdn: microseg: add tag matcher Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-network v4 26/45] sdn: microseg: add name regex matcher Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-network v4 27/45] sdn: microseg: add an update for a running NIC Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-network v4 28/45] sdn: microseg: expose protocol, port and macro rule options Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-network v4 29/45] sdn: microseg: allow nesting groups under a parent Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-network v4 30/45] sdn: microseg: expose the fragments rule option Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-manager v4 31/45] network: apply the microseg policy on reload Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-manager v4 32/45] pve-sdn-commit: apply the microseg policy at boot Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-manager v4 33/45] ui: sdn: zones: add vxlan-gbp checkbox to vxlan and evpn Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-manager v4 34/45] ui: sdn: add microsegmentation panel Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-manager v4 35/45] ui: sdn: dry-run: show pending microseg diff Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-manager v4 36/45] ui: sdn: microseg: add tag matcher Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-manager v4 37/45] ui: sdn: microseg: add name regex matcher Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-manager v4 38/45] ui: sdn: microseg: add narrowing to the rule form Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-manager v4 39/45] ui: sdn: microseg: show and edit the group hierarchy Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-manager v4 40/45] ui: sdn: microseg: add fragments option to the rule form Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-manager v4 41/45] ui: sdn: microseg: add a guest tab with its groups and rules Hannes Laimer
2026-11-05 11:13 ` [PATCH qemu-server v4 42/45] network: update the microseg enforcement on a tag or name change Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-container v4 43/45] lxc: update the microseg enforcement on a tag or hostname change Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-docs v4 44/45] sdn: zones: document the VXLAN-GBP option Hannes Laimer
2026-11-05 11:13 ` [PATCH pve-docs v4 45/45] sdn: add microsegmentation section Hannes Laimer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261105111324.706922-3-h.laimer@proxmox.com \
--to=h.laimer@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox