* [PATCH proxmox-ebpf v3 01/45] build: keep the build directory out of the BPF objects
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ebpf v3 02/45] policy: add the enforcement subsystem for guest interfaces Hannes Laimer
` (43 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
A subsystem's pinned state is keyed by the bytes of its object, and
the debug info that BTF needs carries the source paths. The same
program built from another directory therefore got a new fingerprint.
A version bump of the crate alone, built from a path that carries the
version, made every node reinstall the programs on its next pass.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
build.rs | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/build.rs b/build.rs
index 34652bfb6f83..f04467d53cf9 100644
--- a/build.rs
+++ b/build.rs
@@ -133,6 +133,10 @@ fn compile_bpf(
.arg(global_include)
.arg("-I")
.arg(local_include);
+ // the pinned state is keyed by the object's bytes, and -g writes the source paths into it, so
+ // a build from another directory would count as another program
+ let manifest_dir = env::var("CARGO_MANIFEST_DIR").unwrap();
+ cmd.arg(format!("-ffile-prefix-map={manifest_dir}=."));
if debug {
cmd.arg("-DBPF_DEBUG");
}
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-ebpf v3 02/45] policy: add the enforcement subsystem for guest interfaces
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ebpf v3 01/45] build: keep the build directory out of the BPF objects Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ebpf v3 03/45] policy: add a check for non-first fragments Hannes Laimer
` (42 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
The subsystem, behind the policy feature, decides which frames reach
an assigned guest NIC. On ingress it stamps the NIC's id into the low
16 bits of the packet mark, the part VXLAN GBP carries to other hosts.
On the egress of the destination NIC both ends are known, so it looks
up the pair of the two ids and drops the frame on a miss or a deny. A
sender without an id counts as id zero. A full pass applies the whole
policy to the host, every pair and every assigned NIC. A plug applies
it to one NIC as it comes up. A pair can carry checks that narrow its
verdict to protocols and ports. The first seven sit in the pair's own
value, the rest go to a ruleset shared by the pairs with the same
remaining checks.
A pair with checks drops a frame whose transport header cannot be
read, since a check could deny what it hides. ARP and the
solicitations and advertisements of neighbor discovery pass in both
directions of a pair that allows anything, otherwise the two ends
could never resolve each other. They also pass when the host sends
them, which is how a gateway on the host or a bridge with ARP
suppression answers. Behind a firewall bridge the host's frames arrive
through the bridge's veth like any other, so a NIC there takes them
from any sender without an id. An unstamped frame that leaves on the
interface it came in on passes, whatever it carries. Only a program
ahead of the policy on that interface can turn a frame around
unstamped, so it is the host's own answer.
A plug cannot install the programs, only a pass can. So a pass loads
the programs as soon as the policy has an assignment, even one that
covers no NIC of this host. A pass for a policy without any assignment
unloads the programs and their maps again. A plug also drops the pins
and entries of interfaces that are gone, since only a pass would
otherwise remove them and a node that starts guests for long without
one would fill the interface map.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Cargo.toml | 1 +
debian/control | 6 +-
src/bpf-shim/bpf/bpf_helpers.h | 1 +
src/bpf-shim/vmlinux.h | 1 +
src/lib.rs | 3 +
src/policy/bpf/mark.h | 27 +
src/policy/bpf/policy.bpf.c | 371 ++++++++++++
src/policy/bpf/types.h | 72 +++
src/policy/mod.rs | 992 +++++++++++++++++++++++++++++++++
src/policy/types.rs | 155 ++++++
src/subsystem.rs | 35 +-
tests/common/mod.rs | 40 ++
tests/policy.rs | 775 ++++++++++++++++++++++++++
13 files changed, 2473 insertions(+), 6 deletions(-)
create mode 100644 src/policy/bpf/mark.h
create mode 100644 src/policy/bpf/policy.bpf.c
create mode 100644 src/policy/bpf/types.h
create mode 100644 src/policy/mod.rs
create mode 100644 src/policy/types.rs
create mode 100644 tests/policy.rs
diff --git a/Cargo.toml b/Cargo.toml
index 1e8703da8d3a..250c13baf3be 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -11,6 +11,7 @@ homepage = "https://proxmox.com"
exclude = [".cargo", ".gitignore", "Makefile", "build", "debian"]
[features]
+policy = []
# When enabled, BPF programs compile with -DBPF_DEBUG so their DBG() macros
# expand to bpf_printk. Off by default
bpf-debug = []
diff --git a/debian/control b/debian/control
index d3a9cc8baeee..ef31a8c07256 100644
--- a/debian/control
+++ b/debian/control
@@ -37,14 +37,18 @@ Depends:
Provides:
librust-proxmox-ebpf+bpf-debug-dev (= ${binary:Version}),
librust-proxmox-ebpf+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf+policy-dev (= ${binary:Version}),
librust-proxmox-ebpf-0-dev (= ${binary:Version}),
librust-proxmox-ebpf-0+bpf-debug-dev (= ${binary:Version}),
librust-proxmox-ebpf-0+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0+policy-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1+bpf-debug-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1+policy-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1.0-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1.0+bpf-debug-dev (= ${binary:Version}),
- librust-proxmox-ebpf-0.1.0+default-dev (= ${binary:Version})
+ librust-proxmox-ebpf-0.1.0+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1.0+policy-dev (= ${binary:Version})
Description: Proxmox VE eBPF - Rust source code
Source code for Debianized Rust crate "proxmox-ebpf"
diff --git a/src/bpf-shim/bpf/bpf_helpers.h b/src/bpf-shim/bpf/bpf_helpers.h
index de3bd8412d42..bd5752c1a47e 100644
--- a/src/bpf-shim/bpf/bpf_helpers.h
+++ b/src/bpf-shim/bpf/bpf_helpers.h
@@ -30,5 +30,6 @@ extern long bpf_skb_store_bytes(void *skb, __u32 offset, const void *from, __u32
extern long bpf_skb_pull_data(void *skb, __u32 len);
extern long bpf_skb_change_tail(void *skb, __u32 new_len, __u64 flags);
extern long bpf_redirect(__u32 ifindex, __u64 flags);
+extern long bpf_loop(__u32 nr_loops, void *callback_fn, void *callback_ctx, __u64 flags);
#endif
diff --git a/src/bpf-shim/vmlinux.h b/src/bpf-shim/vmlinux.h
index 99a7aaf5e140..1be73c8dc3a0 100644
--- a/src/bpf-shim/vmlinux.h
+++ b/src/bpf-shim/vmlinux.h
@@ -23,6 +23,7 @@ struct __sk_buff {
__u32 len;
__u32 ifindex;
__u32 mark;
+ __u32 ingress_ifindex;
};
struct ethhdr {
diff --git a/src/lib.rs b/src/lib.rs
index 572d8618661d..c05009220a76 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -1,5 +1,8 @@
//! eBPF subsystems, one per concern, each behind a cargo feature of its name. Nothing here runs
//! on its own, consumers pull in only the subsystem they drive.
+#[cfg(feature = "policy")]
+pub mod policy;
+
pub mod subsystem;
pub mod tc;
diff --git a/src/policy/bpf/mark.h b/src/policy/bpf/mark.h
new file mode 100644
index 000000000000..2fafc6d3b4d6
--- /dev/null
+++ b/src/policy/bpf/mark.h
@@ -0,0 +1,27 @@
+#ifndef PROXMOX_EBPF_POLICY_MARK_H
+#define PROXMOX_EBPF_POLICY_MARK_H
+
+// Include after vmlinux.h and <bpf/bpf_helpers.h>, which provide struct
+// __sk_buff, the __uNN types, __always_inline, and barrier_var().
+//
+// skb->mark is shared with the rest of the host. The policy programs own only
+// the low 16 bits, which hold the wire id and which VXLAN GBP carries between
+// hosts.
+#define POLICY_MARK_MASK 0xffffu
+
+static __always_inline void policy_mark_set(struct __sk_buff *skb, __u16 id) {
+ __u32 mark = skb->mark;
+ mark = (mark & ~POLICY_MARK_MASK) | id;
+ // The verifier takes a write to skb->mark only at its full 32-bit width.
+ // Without the barrier clang turns the read-modify-write into a 16-bit
+ // store of the low half, which the verifier rejects as an invalid context
+ // access.
+ barrier_var(mark);
+ skb->mark = mark;
+}
+
+static __always_inline __u16 policy_mark_get(struct __sk_buff *skb) {
+ return (__u16)skb->mark;
+}
+
+#endif
diff --git a/src/policy/bpf/policy.bpf.c b/src/policy/bpf/policy.bpf.c
new file mode 100644
index 000000000000..53acd35cc6df
--- /dev/null
+++ b/src/policy/bpf/policy.bpf.c
@@ -0,0 +1,371 @@
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_endian.h>
+#include "mark.h"
+#include "types.h"
+#include "bpf_debug.h"
+
+char LICENSE[] SEC("license") = "GPL";
+
+#define TC_ACT_UNSPEC (-1)
+#define TC_ACT_SHOT 2
+
+// vmlinux.h carries only some of these as enum constants and the native test build none, so the
+// program brings its own.
+#define ETHERTYPE_IPV4 0x0800
+#define ETHERTYPE_ARP 0x0806
+#define ETHERTYPE_VLAN 0x8100
+#define ETHERTYPE_QINQ 0x88a8
+#define ETHERTYPE_IPV6 0x86dd
+
+#define VLAN_HLEN 4
+#define VLAN_ENCAP_PROTO 2
+
+// the IPv4 fragment offset, below the three flag bits
+#define IPV4_FRAG_OFFSET 0x1fff
+
+#define PROTO_HOPOPTS 0
+#define PROTO_ICMP 1
+#define PROTO_TCP 6
+#define PROTO_UDP 17
+#define PROTO_DCCP 33
+#define PROTO_ROUTING 43
+#define PROTO_FRAGMENT 44
+#define PROTO_AH 51
+#define PROTO_ICMPV6 58
+#define PROTO_DSTOPTS 60
+#define PROTO_SCTP 132
+#define PROTO_UDPLITE 136
+
+#define ICMPV6_NEIGHBOR_SOLICIT 135
+#define ICMPV6_NEIGHBOR_ADVERT 136
+
+// the kernel's own bound on an IPv6 extension header chain, IP6_MAX_EXT_HDRS_CNT
+#define EXT_MAX 12
+
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __type(key, __u32); // ifindex
+ __type(value, struct nic_entry);
+ __uint(max_entries, NICS_MAX);
+ // the bucket array is sized by the ceiling at creation, the elements are allocated on demand
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __uint(pinning, LIBBPF_PIN_BY_NAME);
+} nics SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __type(key, struct pair_key);
+ __type(value, struct pair_value);
+ __uint(max_entries, PAIRS_MAX);
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __uint(pinning, LIBBPF_PIN_BY_NAME);
+} pairs SEC(".maps");
+
+// The checks of a pair past its inline ones, keyed by a hash of their content.
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __type(key, __u64);
+ __type(value, struct ruleset_value);
+ __uint(max_entries, 2 * RULESETS_MAX);
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __uint(pinning, LIBBPF_PIN_BY_NAME);
+} rulesets SEC(".maps");
+
+SEC("classifier")
+int tc_tap_ingress(struct __sk_buff *skb) {
+ __u32 ifindex = skb->ifindex;
+ struct nic_entry *nic = bpf_map_lookup_elem(&nics, &ifindex);
+ if (!nic) return TC_ACT_UNSPEC;
+ DBG("mark: src_id=%u ifindex=%u", nic->id, ifindex);
+ policy_mark_set(skb, nic->id);
+ return TC_ACT_UNSPEC;
+}
+
+// What a check matches on. `matchable` is set once the parse reached the L4 header.
+struct frame {
+ __u16 eth_proto;
+ __u16 sport;
+ __u16 dport;
+ __u8 family;
+ __u8 ip_proto;
+ __u8 icmp_type;
+ __u8 icmp_code;
+ __u8 matchable;
+ __u8 has_ports;
+ __u8 has_icmp;
+ __u8 unreadable;
+};
+
+static __always_inline int is_ext_header(__u8 proto) {
+ return proto == PROTO_HOPOPTS || proto == PROTO_ROUTING || proto == PROTO_DSTOPTS ||
+ proto == PROTO_FRAGMENT || proto == PROTO_AH;
+}
+
+// Walk the extension header chain to the L4 header. Zero means there is no header to match on.
+static __always_inline int ipv6_l4(struct __sk_buff *skb, struct frame *f, __u32 *off) {
+ for (int i = 0; i < EXT_MAX; i++) {
+ if (!is_ext_header(f->ip_proto)) return 1;
+
+ __u8 hdr[8];
+ if (f->ip_proto == PROTO_FRAGMENT) {
+ if (bpf_skb_load_bytes(skb, *off, hdr, 8) < 0) {
+ f->unreadable = 1;
+ return 0;
+ }
+ // the offset sits in the top 13 bits of the second halfword
+ if (hdr[2] || (hdr[3] & 0xf8)) return 0;
+ *off += 8;
+ } else {
+ if (bpf_skb_load_bytes(skb, *off, hdr, 2) < 0) {
+ f->unreadable = 1;
+ return 0;
+ }
+ if (f->ip_proto == PROTO_AH) {
+ // AH counts its length in four byte units, minus two
+ *off += ((__u32)hdr[1] + 2) * 4;
+ } else {
+ // the length field leaves out the first eight bytes, hence the plus one
+ *off += ((__u32)hdr[1] + 1) * 8;
+ }
+ }
+ f->ip_proto = hdr[0];
+ }
+ if (is_ext_header(f->ip_proto)) {
+ f->unreadable = 1;
+ return 0;
+ }
+ return 1;
+}
+
+static __always_inline void parse_l4(struct __sk_buff *skb, struct frame *f, __u32 off) {
+ switch (f->ip_proto) {
+ case PROTO_TCP:
+ case PROTO_UDP:
+ case PROTO_DCCP:
+ case PROTO_SCTP:
+ case PROTO_UDPLITE: {
+ // every one of these carries the two ports in its first four bytes
+ __be16 ports[2];
+ if (bpf_skb_load_bytes(skb, off, ports, sizeof(ports)) < 0) {
+ f->unreadable = 1;
+ return;
+ }
+ f->sport = bpf_ntohs(ports[0]);
+ f->dport = bpf_ntohs(ports[1]);
+ f->has_ports = 1;
+ break;
+ }
+ case PROTO_ICMP:
+ case PROTO_ICMPV6: {
+ __u8 hdr[2];
+ if (bpf_skb_load_bytes(skb, off, hdr, sizeof(hdr)) < 0) {
+ f->unreadable = 1;
+ return;
+ }
+ f->icmp_type = hdr[0];
+ f->icmp_code = hdr[1];
+ f->has_icmp = 1;
+ break;
+ }
+ }
+ f->matchable = 1;
+}
+
+static __always_inline int is_vlan_tagged(__u16 eth_proto) {
+ return eth_proto == ETHERTYPE_VLAN || eth_proto == ETHERTYPE_QINQ;
+}
+
+// Read the frame down to its L4 header. The kernel moves a frame's outer VLAN tag into the skb
+// metadata before either hook, so a tag found here is a second one left in the frame data. A parse
+// that ends early still leaves the ethertype, which the ARP admission needs.
+static __always_inline void parse_frame(struct __sk_buff *skb, struct frame *f) {
+ __builtin_memset(f, 0, sizeof(*f));
+
+ struct ethhdr eth;
+ if (bpf_skb_load_bytes(skb, 0, ð, sizeof(eth)) < 0) return;
+ f->eth_proto = bpf_ntohs(eth.h_proto);
+ __u32 off = sizeof(eth);
+
+ if (is_vlan_tagged(f->eth_proto)) {
+ __be16 inner;
+ if (bpf_skb_load_bytes(skb, off + VLAN_ENCAP_PROTO, &inner, sizeof(inner)) < 0) return;
+ f->eth_proto = bpf_ntohs(inner);
+ off += VLAN_HLEN;
+ // a receiver strips any number of VID 0 tags and reads what is behind them. The parse reads
+ // through one tag only, so a further tag makes the frame unreadable
+ if (is_vlan_tagged(f->eth_proto)) {
+ f->unreadable = 1;
+ return;
+ }
+ }
+
+ if (f->eth_proto == ETHERTYPE_IPV4) {
+ struct iphdr ip;
+ if (bpf_skb_load_bytes(skb, off, &ip, sizeof(ip)) < 0 || ip.ihl < 5) {
+ f->unreadable = 1;
+ return;
+ }
+ // only the first fragment carries the L4 header
+ if (bpf_ntohs(ip.frag_off) & IPV4_FRAG_OFFSET) return;
+ f->family = FAMILY_V4;
+ f->ip_proto = ip.protocol;
+ off += (__u32)ip.ihl * 4;
+ } else if (f->eth_proto == ETHERTYPE_IPV6) {
+ struct ipv6hdr ip6;
+ if (bpf_skb_load_bytes(skb, off, &ip6, sizeof(ip6)) < 0) {
+ f->unreadable = 1;
+ return;
+ }
+ f->family = FAMILY_V6;
+ f->ip_proto = ip6.nexthdr;
+ off += sizeof(ip6);
+ if (!ipv6_l4(skb, f, &off)) return;
+ } else {
+ return;
+ }
+
+ parse_l4(skb, f, off);
+}
+
+static __always_inline int check_matches(const struct frame *f, const struct check_value *c) {
+ if (!f->matchable) return 0;
+ if (c->family && c->family != f->family) return 0;
+ if (c->ip_proto != f->ip_proto) return 0;
+
+ if (c->match_flags & CHECK_HAS_SPORT) {
+ if (!f->has_ports || f->sport < c->sport_lo || f->sport > c->sport_hi) return 0;
+ }
+ if (c->match_flags & CHECK_HAS_DPORT) {
+ if (!f->has_ports || f->dport < c->dport_lo || f->dport > c->dport_hi) return 0;
+ }
+ if (c->match_flags & CHECK_HAS_ICMP_TYPE) {
+ if (!f->has_icmp || f->icmp_type != c->icmp_type) return 0;
+ }
+ if (c->match_flags & CHECK_HAS_ICMP_CODE) {
+ if (!f->has_icmp || f->icmp_code != c->icmp_code) return 0;
+ }
+ return 1;
+}
+
+// A ruleset is walked through bpf_loop, so the verifier's work does not grow with CHECK_MAX. A
+// plain loop bounded by CHECK_MAX would have it walk every iteration, each with the branches of
+// its match.
+struct ruleset_walk {
+ const struct frame *f;
+ const struct ruleset_value *rs;
+ __u8 decided;
+ __u8 allow;
+};
+
+static long ruleset_check(__u64 i, void *ctx) {
+ struct ruleset_walk *w = ctx;
+ // the count comes from the map, the verifier needs a bound on the index
+ if (i >= CHECK_MAX) return 1;
+ const struct check_value *c = &w->rs->checks[i];
+ if (!check_matches(w->f, c)) return 0;
+ w->decided = 1;
+ w->allow = c->allow;
+ return 1;
+}
+
+static __always_inline int is_arp_or_nd(const struct frame *f) {
+ if (f->eth_proto == ETHERTYPE_ARP) return 1;
+ return f->family == FAMILY_V6 && f->ip_proto == PROTO_ICMPV6 && f->has_icmp &&
+ (f->icmp_type == ICMPV6_NEIGHBOR_SOLICIT || f->icmp_type == ICMPV6_NEIGHBOR_ADVERT);
+}
+
+// ARP and neighbor discovery also pass when the reverse pair allows anything, and when the host
+// sends them. The host's own replies, the suppression replies of a bridge or the answers of a
+// gateway on it, come unstamped and without an ingress interface. Behind a firewall bridge the
+// veth hop gives every frame one, so a tap there is flagged and takes these frames from any
+// unstamped sender.
+static __always_inline int rescue_arp_nd(struct __sk_buff *skb, const struct frame *f,
+ __u16 src_id, const struct nic_entry *dst) {
+ if (!is_arp_or_nd(f)) return 0;
+ if (src_id == ID_UNSTAMPED) return skb->ingress_ifindex == 0 || (dst->flags & NIC_ADMIT_ND);
+ struct pair_key k = { .src_id = dst->id, .dst_id = src_id };
+ struct pair_value *r = bpf_map_lookup_elem(&pairs, &k);
+ return r && (r->flags & PAIR_ADMIT_ND);
+}
+
+static __always_inline int has_checks(const struct pair_value *r) {
+ return r->inline_count || (r->flags & PAIR_HAS_RULESET);
+}
+
+static __always_inline int verdict(__u8 allow) {
+ return allow ? TC_ACT_UNSPEC : TC_ACT_SHOT;
+}
+
+// Enforce on egress of the destination tap, where both ends are known. The destination comes from
+// this tap's NIC entry and the source from skb->mark. A missing pair is a deny, so unstamped
+// frames (src_id 0) need an explicit (0, dst_id) allow. ARP and neighbor discovery as above and a
+// frame turned around on the tap itself are the exceptions.
+SEC("classifier")
+int tc_tap_egress(struct __sk_buff *skb) {
+ __u32 ifindex = skb->ifindex;
+ __u16 src_id = policy_mark_get(skb);
+
+ struct nic_entry *nic = bpf_map_lookup_elem(&nics, &ifindex);
+ // a NIC gets its entry before its links and loses it before them
+ if (!nic) return TC_ACT_UNSPEC;
+
+ // an unstamped frame that came in on this tap was turned around by a program ahead of this
+ // one, the host's own answer to the guest. A guest frame the bridge reflects carries the
+ // stamp.
+ if (src_id == ID_UNSTAMPED && skb->ingress_ifindex == ifindex) return TC_ACT_UNSPEC;
+
+ struct pair_key k = { .src_id = src_id, .dst_id = nic->id };
+ struct pair_value *r = bpf_map_lookup_elem(&pairs, &k);
+ if (r && r->allow && !has_checks(r)) return TC_ACT_UNSPEC;
+
+ struct frame f;
+ parse_frame(skb, &f);
+
+ if (r && has_checks(r)) {
+ // a frame the parse could not read to its L4 header could hide what a check would deny
+ if (f.unreadable) {
+ DBG("deny (L4 header unreadable): src_id=%u dst_id=%u", src_id, nic->id);
+ return TC_ACT_SHOT;
+ }
+
+ for (__u32 i = 0; i < CHECK_INLINE; i++) {
+ if (i >= r->inline_count) break;
+ const struct check_value *c = &r->checks[i];
+ if (check_matches(&f, c)) {
+ DBG("check %u decides: allow=%u", i, c->allow);
+ return verdict(c->allow);
+ }
+ }
+
+ if (r->flags & PAIR_HAS_RULESET) {
+ __u64 key = r->ruleset;
+ struct ruleset_value *rs = bpf_map_lookup_elem(&rulesets, &key);
+ if (rs) {
+ struct ruleset_walk w = { .f = &f, .rs = rs };
+ bpf_loop(rs->count, ruleset_check, &w, 0);
+ if (w.decided) {
+ DBG("ruleset check decides: allow=%u", w.allow);
+ return verdict(w.allow);
+ }
+ } else {
+ // a frame that read its pair before a pass rewrote it can find the old ruleset
+ // removed already. The verdict alone is not what that pair asked for, so the
+ // frame is dropped rather than decided on half the checks.
+ DBG("deny (no ruleset): src_id=%u dst_id=%u", src_id, nic->id);
+ return TC_ACT_SHOT;
+ }
+ }
+
+ if ((r->flags & PAIR_ADMIT_ND) && is_arp_or_nd(&f)) return TC_ACT_UNSPEC;
+ }
+
+ if (r && r->allow) return TC_ACT_UNSPEC;
+ if (rescue_arp_nd(skb, &f, src_id, nic)) return TC_ACT_UNSPEC;
+ if (r) {
+ DBG("deny (explicit): src_id=%u dst_id=%u", src_id, nic->id);
+ } else {
+ DBG("deny (no pair): src_id=%u dst_id=%u", src_id, nic->id);
+ }
+ return TC_ACT_SHOT;
+}
diff --git a/src/policy/bpf/types.h b/src/policy/bpf/types.h
new file mode 100644
index 000000000000..a8b1df66fc41
--- /dev/null
+++ b/src/policy/bpf/types.h
@@ -0,0 +1,72 @@
+#ifndef PROXMOX_EBPF_POLICY_TYPES_H
+#define PROXMOX_EBPF_POLICY_TYPES_H
+
+// Keep in sync with ../types.rs.
+// the __u* types come from vmlinux.h, or from the shim in the native build
+
+#define CHECK_MAX 512
+// the checks a pair_value holds inline, the rest go to the pair's ruleset
+#define CHECK_INLINE 7
+#define PAIRS_MAX (1 << 20)
+#define RULESETS_MAX (1 << 14)
+#define NICS_MAX (1 << 18)
+
+// what the mark of a frame nobody stamped reads as, never the id of a NIC
+#define ID_UNSTAMPED 0
+
+#define PAIR_ADMIT_ND (1 << 0)
+#define PAIR_HAS_RULESET (1 << 1)
+
+#define NIC_ADMIT_ND (1 << 0)
+
+#define FAMILY_V4 4
+#define FAMILY_V6 6
+
+#define CHECK_HAS_SPORT (1 << 0)
+#define CHECK_HAS_DPORT (1 << 1)
+#define CHECK_HAS_ICMP_TYPE (1 << 2)
+#define CHECK_HAS_ICMP_CODE (1 << 3)
+
+struct nic_entry {
+ __u16 id;
+ __u16 flags;
+ __u8 _pad[4];
+};
+
+struct pair_key {
+ __u16 src_id;
+ __u16 dst_id;
+};
+
+struct check_value {
+ __u8 ip_proto;
+ // 0 matches either family, 4 and 6 pin the match to one
+ __u8 family;
+ __u8 match_flags;
+ __u8 allow;
+ __u16 sport_lo;
+ __u16 sport_hi;
+ __u16 dport_lo;
+ __u16 dport_hi;
+ __u8 icmp_type;
+ __u8 icmp_code;
+ __u8 _pad[2];
+};
+
+struct pair_value {
+ __u8 allow;
+ __u8 flags;
+ __u8 inline_count;
+ __u8 _pad[5];
+ // the rulesets key, read only with PAIR_HAS_RULESET set
+ __u64 ruleset;
+ struct check_value checks[CHECK_INLINE];
+};
+
+struct ruleset_value {
+ __u16 count;
+ __u8 _pad[14];
+ struct check_value checks[CHECK_MAX];
+};
+
+#endif
diff --git a/src/policy/mod.rs b/src/policy/mod.rs
new file mode 100644
index 000000000000..62ee7576b6fc
--- /dev/null
+++ b/src/policy/mod.rs
@@ -0,0 +1,992 @@
+//! Tap-side enforcement. On ingress, a BPF program on the guest tap or veth interface stamps the
+//! source wire id into `skb->mark`. On egress, it looks up the `(src_id, dst_id)` pair and drops
+//! the frame on a miss or a denying pair. A pair can narrow its verdict with checks
+//! ([`CheckEntry`]), run in the order given, and the first check that matches decides. A pair with
+//! checks drops a frame whose L4 header cannot be read. ARP and the solicitations and
+//! advertisements of neighbor discovery pass in both directions between two NICs that a pair admits
+//! something for, and when the host sends them. Behind a firewall bridge the host's frames look
+//! like any other sender's, so a NIC there takes them from every unstamped sender.
+//! An unstamped frame that leaves on the interface it came in on was turned around by a program
+//! ahead of the policy there and passes as the host's answer.
+//!
+//! The checks past the inline ones go to `rulesets`, shared by the pairs with the same checks. A
+//! ruleset is written before any pair names it and never changes in place, so a pair changes with
+//! a single map write.
+//!
+//! A pass and a plug take the subsystem's lock before they read their input, the pass exclusively
+//! and the plug shared, so they reach the kernel in the order they read. A plug writes its NIC's
+//! entry over whatever a pass placed, since what it read is at least as new.
+//!
+//! Until a pass is through, a tap can get a different verdict from the final one, for example
+//! after a schema change, when its links move onto programs whose maps are still empty. A NIC the
+//! pass newly assigns is attached last, so a first install has no such window.
+
+pub mod types;
+
+use std::collections::{HashMap, HashSet};
+use std::hash::{DefaultHasher, Hash, Hasher};
+
+use anyhow::{Context, bail};
+use aya::maps::MapError;
+
+use self::types::*;
+use crate::subsystem::{ApplyLock, Exclusive, Shared, TcPrograms};
+use crate::tc::Direction;
+use crate::tc_programs;
+
+pub use self::types::{CHECK_MAX, PAIRS_MAX};
+
+/// What a full pass did on the host.
+#[derive(Copy, Clone, Debug, PartialEq, Eq)]
+pub enum Pass {
+ /// The host matches the input.
+ Applied,
+ /// The policy assigns nothing, so nothing is loaded and no link is left.
+ Idle,
+}
+
+/// A guest NIC, the wire id it stamps and whether it sits behind a firewall bridge.
+pub struct Assignment {
+ /// The interface name on this host.
+ pub iface: String,
+ pub id: u16,
+ pub firewall_bridge: bool,
+}
+
+/// One pair of the pairs map. A pair that admits nothing, with no allow of its own and no allowing
+/// check, is valid input and gets no ARP and neighbor discovery pass.
+pub struct Pair {
+ pub src_id: u16,
+ pub dst_id: u16,
+ /// The checks to run, at most [`CHECK_MAX`] of them.
+ pub checks: Vec<CheckEntry>,
+ pub allow: bool,
+}
+
+#[derive(Copy, Clone, Debug, Default, PartialEq, Eq)]
+#[repr(u8)]
+pub enum Family {
+ #[default]
+ Any = 0,
+ V4 = FAMILY_V4,
+ V6 = FAMILY_V6,
+}
+
+/// One protocol match of a narrowed pair. A single port is a range with equal bounds, and a field
+/// left out is not matched on.
+#[derive(Clone, Debug, Default, PartialEq, Eq)]
+pub struct CheckEntry {
+ pub ip_proto: u8,
+ pub family: Family,
+ pub sport: Option<(u16, u16)>,
+ pub dport: Option<(u16, u16)>,
+ pub icmp_type: Option<u8>,
+ pub icmp_code: Option<u8>,
+ pub allow: bool,
+}
+
+impl CheckEntry {
+ fn value(&self) -> CheckValue {
+ let mut match_flags = 0;
+ if self.sport.is_some() {
+ match_flags |= CHECK_HAS_SPORT;
+ }
+ if self.dport.is_some() {
+ match_flags |= CHECK_HAS_DPORT;
+ }
+ if self.icmp_type.is_some() {
+ match_flags |= CHECK_HAS_ICMP_TYPE;
+ }
+ if self.icmp_code.is_some() {
+ match_flags |= CHECK_HAS_ICMP_CODE;
+ }
+ let (sport_lo, sport_hi) = self.sport.unwrap_or_default();
+ let (dport_lo, dport_hi) = self.dport.unwrap_or_default();
+ CheckValue {
+ ip_proto: self.ip_proto,
+ family: self.family as u8,
+ match_flags,
+ allow: self.allow as u8,
+ sport_lo,
+ sport_hi,
+ dport_lo,
+ dport_hi,
+ icmp_type: self.icmp_type.unwrap_or_default(),
+ icmp_code: self.icmp_code.unwrap_or_default(),
+ _pad: [0; 2],
+ }
+ }
+}
+
+const NICS_MAP: &str = "nics";
+const PAIRS_MAP: &str = "pairs";
+const RULESETS_MAP: &str = "rulesets";
+const MAPS: [&str; 3] = [NICS_MAP, PAIRS_MAP, RULESETS_MAP];
+
+// bump when a map layout in types.h, the meaning of a value or a map ceiling changes. An existing
+// pin is reused by name without a check of any of them.
+const SCHEMA_VERSION: u32 = 1;
+
+const PROGRAM_NAMES: [(Direction, &str); 2] = [
+ (Direction::Ingress, "tc_tap_ingress"),
+ (Direction::Egress, "tc_tap_egress"),
+];
+
+type NicsMap = aya::maps::HashMap<aya::maps::MapData, u32, NicEntry>;
+type PairsMap = aya::maps::HashMap<aya::maps::MapData, PairKey, PairValue>;
+type RulesetsMap = aya::maps::HashMap<aya::maps::MapData, u64, RulesetValue>;
+
+static PROGRAMS: TcPrograms = tc_programs! {
+ name: "policy",
+ programs: &PROGRAM_NAMES,
+ maps: &MAPS,
+ schema_version: SCHEMA_VERSION,
+};
+
+/// The lock a plug runs under. Plugs run next to each other and wait only behind a pass.
+pub struct SharedLock {
+ lock: ApplyLock<'static, Shared>,
+}
+
+/// The lock a pass runs under, held until it is through.
+pub struct ExclusiveLock {
+ lock: ApplyLock<'static, Exclusive>,
+}
+
+pub fn lock_shared() -> anyhow::Result<SharedLock> {
+ Ok(SharedLock {
+ lock: PROGRAMS.lock_shared()?,
+ })
+}
+
+pub fn lock_exclusive() -> anyhow::Result<ExclusiveLock> {
+ Ok(ExclusiveLock {
+ lock: PROGRAMS.lock_exclusive()?,
+ })
+}
+
+/// Check `pairs` the way a full pass does, without touching the host.
+pub fn check(pairs: &[Pair]) -> anyhow::Result<()> {
+ count_within(pairs, PAIRS_MAX, RULESETS_MAX)
+}
+
+impl ExclusiveLock {
+ /// Full pass over the host. Afterwards the interface map holds the assigned NICs that exist
+ /// here, each with its id and its links, and nothing else, and the pairs map holds exactly
+ /// `pairs`. `configured` is whether the policy has an assignment at all. Without one the pass
+ /// returns [`Pass::Idle`].
+ pub fn apply(
+ &self,
+ configured: bool,
+ assignments: &[Assignment],
+ pairs: &[Pair],
+ ) -> anyhow::Result<Pass> {
+ check_assignments(assignments)?;
+ let encoded = encode_pairs(pairs)?;
+ let lock = &self.lock;
+ if !configured {
+ if lock.is_current() || any_attached(lock)? {
+ log::info!("policy: the policy assigns nothing, dropping the pinned state");
+ lock.clear()?;
+ }
+ return Ok(Pass::Idle);
+ }
+ let local = resolve_local_assignments(assignments)?;
+ lock.ensure_loaded()?;
+ let mut nics: NicsMap = lock.hash_map(NICS_MAP)?;
+ sync_nics(&mut nics, &local).context("sync the interface map")?;
+ let desired: HashSet<u32> = local.keys().copied().collect();
+ let mut pairs_map: PairsMap = lock.hash_map(PAIRS_MAP)?;
+ let mut rulesets: RulesetsMap = lock.hash_map(RULESETS_MAP)?;
+ let synced = sync_pairs(&mut pairs_map, &mut rulesets, &encoded).context("sync the pairs");
+ // the links are attached even when the pairs failed, since an assigned NIC is better cut
+ // off than left without its programs
+ let attached = lock.reconcile(&desired);
+ synced.and(attached)?;
+ Ok(Pass::Applied)
+ }
+}
+
+fn any_attached(lock: &ApplyLock<'_, Exclusive>) -> anyhow::Result<bool> {
+ Ok(!lock.attached()?.is_empty() || PROGRAMS.links_pinned_elsewhere())
+}
+
+impl SharedLock {
+ /// Program a single guest NIC present on this host, with the `id` the caller resolved. A
+ /// failure propagates, so the caller can keep the NIC from coming up unenforced. A host
+ /// without current programs fails that way, since only a full pass installs them, and so does
+ /// a full interface map. `firewall_bridge` says the NIC is or is about to be put behind a
+ /// firewall bridge, which a NIC still being plugged does not show.
+ pub fn apply_iface(&self, iface: &str, id: u16, firewall_bridge: bool) -> anyhow::Result<()> {
+ let entry = nic_entry(id, firewall_bridge);
+ let Some(ifindex) = ifindex_of(iface)? else {
+ log::info!("policy: {iface} is gone, nothing to do");
+ return Ok(());
+ };
+ prune_gone(&self.lock);
+ apply_one(&self.lock, ifindex, iface, entry)
+ }
+
+ /// Take the entry and the links of a guest NIC away when it comes up again under the same
+ /// index and the config no longer assigns it. Nothing is touched before a new build's first
+ /// pass, that one decides about every NIC.
+ pub fn release_iface(&self, iface: &str) -> anyhow::Result<()> {
+ let Some(ifindex) = ifindex_of(iface)? else {
+ return Ok(());
+ };
+ let lock = &self.lock;
+ if !lock.is_current() {
+ return Ok(());
+ }
+ let mut nics: NicsMap = lock.hash_map(NICS_MAP)?;
+ let removed = match nics.remove(&ifindex) {
+ Ok(()) => true,
+ Err(e) if missing_key(&e) => false,
+ Err(e) => return Err(e).with_context(|| format!("remove the entry of {iface}")),
+ };
+ if removed {
+ log::info!("policy: {iface} (ifindex {ifindex}) is no longer assigned");
+ lock.detach_named(iface, |pin| pin.ifindex != ifindex)?;
+ }
+ Ok(())
+ }
+}
+
+/// Drop what earlier plugs left for interfaces that are gone, so a node that starts guests
+/// for a long time without a full pass does not fill the interface map or the pin directory.
+fn prune_gone(lock: &ApplyLock<'_, Shared>) {
+ let pruned = lock.detach_gone().and_then(|gone| {
+ if gone.is_empty() || !lock.is_current() {
+ return Ok(0);
+ }
+ let mut nics: NicsMap = lock.hash_map(NICS_MAP)?;
+ for ifindex in &gone {
+ match nics.remove(ifindex) {
+ Ok(()) => {}
+ Err(e) if missing_key(&e) => {}
+ Err(e) => {
+ return Err(e).with_context(|| format!("remove the entry of {ifindex}"));
+ }
+ }
+ }
+ Ok(gone.len())
+ });
+ match pruned {
+ Ok(0) => {}
+ Ok(n) => log::debug!("policy: dropped {n} gone interface(s) left by earlier plugs"),
+ Err(e) => log::warn!("policy: prune of gone interfaces: {e:#}"),
+ }
+}
+
+fn apply_one(
+ lock: &ApplyLock<'_, Shared>,
+ ifindex: u32,
+ iface: &str,
+ entry: NicEntry,
+) -> anyhow::Result<()> {
+ if !lock.is_current() {
+ bail!("no current programs are pinned, a full pass has to install them first");
+ }
+ let mut nics: NicsMap = lock.hash_map(NICS_MAP)?;
+ nics.insert(ifindex, entry, 0)
+ .with_context(|| format!("write the entry of {iface}"))?;
+ log::info!("policy: {iface} (ifindex {ifindex}) -> id {}", entry.id);
+ lock.attach_iface(ifindex)?;
+ Ok(())
+}
+
+fn ifindex_of(iface: &str) -> anyhow::Result<Option<u32>> {
+ match nix::net::if_::if_nametoindex(iface) {
+ Ok(ifindex) => Ok(Some(ifindex)),
+ Err(nix::errno::Errno::ENODEV) => Ok(None),
+ Err(e) => Err(e).with_context(|| format!("index of interface {iface}")),
+ }
+}
+
+fn missing_key(err: &MapError) -> bool {
+ matches!(
+ err,
+ MapError::SyscallError(e) if e.io_error.raw_os_error() == Some(nix::libc::ENOENT)
+ )
+}
+
+#[derive(Default)]
+struct NicChanges {
+ write: Vec<(u32, NicEntry)>,
+ remove: Vec<u32>,
+}
+
+fn nic_changes(live: &HashMap<u32, NicEntry>, desired: &HashMap<u32, NicEntry>) -> NicChanges {
+ let mut changes = NicChanges::default();
+ for (&ifindex, &entry) in desired {
+ let placed = live.get(&ifindex);
+ // a plug may have flagged a NIC this pass saw before it was put behind its firewall
+ // bridge, so only a plug takes the flag away again
+ let flags = entry.flags | placed.map_or(0, |placed| placed.flags & NIC_ADMIT_ND);
+ if placed.is_none_or(|placed| placed.id != entry.id || placed.flags != flags) {
+ changes.write.push((ifindex, NicEntry { flags, ..entry }));
+ }
+ }
+ for &ifindex in live.keys() {
+ if !desired.contains_key(&ifindex) {
+ changes.remove.push(ifindex);
+ }
+ }
+ changes
+}
+
+fn sync_nics(map: &mut NicsMap, desired: &HashMap<u32, NicEntry>) -> anyhow::Result<()> {
+ let live: HashMap<u32, NicEntry> = map
+ .iter()
+ .collect::<Result<_, _>>()
+ .context("read the interface map")?;
+ let changes = nic_changes(&live, desired);
+ // a write could hit the map's ceiling while stale entries are still in
+ for ifindex in &changes.remove {
+ log::debug!("policy: drop NIC entry for {ifindex}");
+ map.remove(ifindex)
+ .context("remove from the interface map")?;
+ }
+ for &(ifindex, entry) in &changes.write {
+ map.insert(ifindex, entry, 0)
+ .context("write the interface map")?;
+ }
+ if !changes.write.is_empty() || !changes.remove.is_empty() {
+ log::info!(
+ "policy: NIC entries {} written, {} removed",
+ changes.write.len(),
+ changes.remove.len(),
+ );
+ }
+ Ok(())
+}
+
+fn encode(pair: &Pair, key: Option<u64>, tail: &[CheckValue]) -> (PairValue, Option<RulesetValue>) {
+ let admit_nd = pair.allow || pair.checks.iter().any(|check| check.allow);
+ let split = pair.checks.len().min(CHECK_INLINE);
+
+ let checks = check_values(pair.checks[..split].iter().map(CheckEntry::value));
+ let ruleset = (!tail.is_empty()).then(|| RulesetValue {
+ count: tail.len() as u16,
+ _pad: [0; 14],
+ checks: check_values(tail.iter().copied()),
+ });
+
+ let mut flags = 0;
+ if admit_nd {
+ flags |= PAIR_ADMIT_ND;
+ }
+ if ruleset.is_some() {
+ flags |= PAIR_HAS_RULESET;
+ }
+ let value = PairValue {
+ allow: pair.allow as u8,
+ flags,
+ inline_count: split as u8,
+ _pad: [0; 5],
+ ruleset: key.unwrap_or(0),
+ checks,
+ };
+ (value, ruleset)
+}
+
+// the slots past the checks stay zeroed, so an unchanged pair or ruleset compares equal and is not
+// written again
+fn check_values<const N: usize>(values: impl IntoIterator<Item = CheckValue>) -> [CheckValue; N] {
+ let mut slots = [CheckValue::default(); N];
+ for (slot, value) in slots.iter_mut().zip(values) {
+ *slot = value;
+ }
+ slots
+}
+
+// the key only has to be stable within one build. A ruleset pinned under the key of another
+// build is written anew under this one, and the old one is removed as unreferenced
+fn ruleset_key(checks: &[CheckValue]) -> u64 {
+ let mut hasher = DefaultHasher::new();
+ checks.hash(&mut hasher);
+ hasher.finish()
+}
+
+// a NIC given twice would be decided by the input order
+fn check_assignments(assignments: &[Assignment]) -> anyhow::Result<()> {
+ let mut ids = HashMap::new();
+ for a in assignments {
+ if a.id == ID_UNSTAMPED {
+ bail!("NIC {} is given the id of unstamped frames", a.iface);
+ }
+ if let Some(other) = ids.insert(a.iface.as_str(), a.id) {
+ bail!(
+ "NIC {} is given twice, with id {other} and id {}",
+ a.iface,
+ a.id,
+ );
+ }
+ }
+ Ok(())
+}
+
+struct Encoded {
+ pairs: HashMap<PairKey, PairValue>,
+ rulesets: HashMap<u64, RulesetValue>,
+}
+
+fn encode_pairs(pairs: &[Pair]) -> anyhow::Result<Encoded> {
+ encode_within(pairs, PAIRS_MAX, RULESETS_MAX)
+}
+
+fn encode_within(pairs: &[Pair], pairs_max: usize, rulesets_max: usize) -> anyhow::Result<Encoded> {
+ let mut encoded = Encoded {
+ pairs: HashMap::with_capacity(pairs.len()),
+ rulesets: HashMap::new(),
+ };
+ walk_within(
+ pairs,
+ pairs_max,
+ rulesets_max,
+ |pair, pair_key, key, tail| {
+ let (value, ruleset) = encode(pair, key, tail);
+ if let (Some(key), Some(ruleset)) = (key, ruleset)
+ && *encoded.rulesets.entry(key).or_insert(ruleset) != ruleset
+ {
+ bail!("two rulesets share the key {key:016x}");
+ }
+ encoded.pairs.insert(pair_key, value);
+ Ok(())
+ },
+ )?;
+ Ok(encoded)
+}
+
+fn count_within(pairs: &[Pair], pairs_max: usize, rulesets_max: usize) -> anyhow::Result<()> {
+ walk_within(pairs, pairs_max, rulesets_max, |_, _, _, _| Ok(()))
+}
+
+// the input is checked before anything is written. Checks past the limit would be cut off, and a
+// pair given twice would be decided by the input order. A pass that runs into a full map would
+// leave the NICs on their new ids next to the pairs of the pass before. `check` runs the same walk
+// and skips the map image, 128 bytes per pair and 8 KiB per ruleset.
+fn walk_within(
+ pairs: &[Pair],
+ pairs_max: usize,
+ rulesets_max: usize,
+ mut place: impl FnMut(&Pair, PairKey, Option<u64>, &[CheckValue]) -> anyhow::Result<()>,
+) -> anyhow::Result<()> {
+ if pairs.len() > pairs_max {
+ bail!(
+ "{} pairs are given, the pairs map holds at most {pairs_max}",
+ pairs.len(),
+ );
+ }
+ let mut seen = HashSet::with_capacity(pairs.len());
+ let mut rulesets = HashSet::new();
+ for pair in pairs {
+ if pair.checks.len() > CHECK_MAX {
+ bail!(
+ "pair {} -> {} carries {} checks, the program runs at most {CHECK_MAX}",
+ pair.src_id,
+ pair.dst_id,
+ pair.checks.len(),
+ );
+ }
+ let pair_key = PairKey {
+ src_id: pair.src_id,
+ dst_id: pair.dst_id,
+ };
+ if !seen.insert(pair_key) {
+ bail!("pair {} -> {} is given twice", pair.src_id, pair.dst_id);
+ }
+ let tail: Vec<CheckValue> = pair.checks[pair.checks.len().min(CHECK_INLINE)..]
+ .iter()
+ .map(CheckEntry::value)
+ .collect();
+ let key = (!tail.is_empty()).then(|| ruleset_key(&tail));
+ if let Some(key) = key {
+ rulesets.insert(key);
+ if rulesets.len() > rulesets_max {
+ bail!(
+ "{} rulesets are needed, a pass may name at most {rulesets_max}",
+ rulesets.len(),
+ );
+ }
+ }
+ place(pair, pair_key, key, &tail)?;
+ }
+ Ok(())
+}
+
+#[derive(Default)]
+struct PairChanges<'a> {
+ pairs_remove: Vec<PairKey>,
+ rulesets_orphan: Vec<u64>,
+ rulesets_insert: Vec<(u64, &'a RulesetValue)>,
+ pairs_write: Vec<(PairKey, PairValue)>,
+ /// Pinned rulesets the pass no longer names but a kept pair still does until it is rewritten.
+ rulesets_remove: Vec<u64>,
+}
+
+fn pair_changes<'a>(
+ live: &HashMap<PairKey, PairValue>,
+ live_rulesets: &HashSet<u64>,
+ encoded: &'a Encoded,
+) -> PairChanges<'a> {
+ let mut changes = PairChanges {
+ pairs_remove: live
+ .keys()
+ .filter(|pair_key| !encoded.pairs.contains_key(pair_key))
+ .copied()
+ .collect(),
+ ..PairChanges::default()
+ };
+ let named: HashSet<u64> = live
+ .iter()
+ .filter(|(pair_key, value)| {
+ encoded.pairs.contains_key(pair_key) && value.flags & PAIR_HAS_RULESET != 0
+ })
+ .map(|(_, value)| value.ruleset)
+ .collect();
+ for (&pair_key, &value) in &encoded.pairs {
+ if live.get(&pair_key) != Some(&value) {
+ changes.pairs_write.push((pair_key, value));
+ }
+ }
+ for &ruleset_key in live_rulesets {
+ if !named.contains(&ruleset_key) {
+ changes.rulesets_orphan.push(ruleset_key);
+ } else if !encoded.rulesets.contains_key(&ruleset_key) {
+ changes.rulesets_remove.push(ruleset_key);
+ }
+ }
+ for (&ruleset_key, ruleset) in &encoded.rulesets {
+ if !live_rulesets.contains(&ruleset_key) || !named.contains(&ruleset_key) {
+ changes.rulesets_insert.push((ruleset_key, ruleset));
+ }
+ }
+ changes
+}
+
+fn sync_pairs(
+ map: &mut PairsMap,
+ rulesets: &mut RulesetsMap,
+ encoded: &Encoded,
+) -> anyhow::Result<()> {
+ let live_rulesets: HashSet<u64> = rulesets
+ .keys()
+ .collect::<Result<_, _>>()
+ .context("read the ruleset map")?;
+ let live: HashMap<PairKey, PairValue> = map
+ .iter()
+ .collect::<Result<_, _>>()
+ .context("read the pairs map")?;
+ let changes = pair_changes(&live, &live_rulesets, encoded);
+
+ // a pass that fits the map could hit its ceiling while the old pairs are still in
+ for pair_key in &changes.pairs_remove {
+ map.remove(pair_key).context("remove from the pairs map")?;
+ }
+ // a ruleset no kept pair names goes before the inserts, or the inserts could hit the map's
+ // ceiling. Such rulesets belong to the pairs removed above or were left by a pass that failed
+ // between its inserts and its removals
+ for ruleset_key in &changes.rulesets_orphan {
+ rulesets
+ .remove(ruleset_key)
+ .context("remove from the ruleset map")?;
+ }
+ // a frame may be walking a ruleset a kept pair names, so one under the same key with other
+ // checks fails the pass instead of being overwritten. An orphan under that key is gone by now
+ // and gets written anew.
+ let orphans: HashSet<u64> = changes.rulesets_orphan.iter().copied().collect();
+ for (ruleset_key, ruleset) in &encoded.rulesets {
+ if live_rulesets.contains(ruleset_key)
+ && !orphans.contains(ruleset_key)
+ && rulesets
+ .get(ruleset_key, 0)
+ .context("read the ruleset map")?
+ != *ruleset
+ {
+ bail!("the pinned ruleset {ruleset_key:016x} carries other checks under the same key");
+ }
+ }
+ for &(ruleset_key, ruleset) in &changes.rulesets_insert {
+ rulesets
+ .insert(ruleset_key, ruleset, 0)
+ .context("write the ruleset map")?;
+ }
+ for &(pair_key, value) in &changes.pairs_write {
+ map.insert(pair_key, value, 0)
+ .context("write the pairs map")?;
+ }
+
+ for ruleset_key in &changes.rulesets_remove {
+ rulesets
+ .remove(ruleset_key)
+ .context("remove from the ruleset map")?;
+ }
+
+ if !changes.pairs_write.is_empty() || !changes.pairs_remove.is_empty() {
+ log::info!(
+ "policy: pairs {} written, {} removed",
+ changes.pairs_write.len(),
+ changes.pairs_remove.len(),
+ );
+ } else {
+ log::debug!("policy: {} pairs, no changes", encoded.pairs.len());
+ }
+ let rulesets_removed = changes.rulesets_orphan.len() + changes.rulesets_remove.len();
+ if !changes.rulesets_insert.is_empty() || rulesets_removed > 0 {
+ log::info!(
+ "policy: rulesets {} written, {rulesets_removed} removed",
+ changes.rulesets_insert.len(),
+ );
+ }
+ Ok(())
+}
+
+fn nic_entry(id: u16, firewall_bridge: bool) -> NicEntry {
+ NicEntry {
+ id,
+ flags: if firewall_bridge { NIC_ADMIT_ND } else { 0 },
+ _pad: [0; 4],
+ }
+}
+
+/// The id and firewall bridge flag of every assigned NIC present here, by ifindex.
+fn resolve_local_assignments(assignments: &[Assignment]) -> anyhow::Result<HashMap<u32, NicEntry>> {
+ let ifaces = nix::net::if_::if_nameindex().context("if_nameindex")?;
+ let by_name: HashMap<&str, u32> = ifaces
+ .iter()
+ .filter_map(|i| i.name().to_str().ok().map(|n| (n, i.index())))
+ .collect();
+
+ let mut out = HashMap::new();
+ for a in assignments {
+ if let Some(&ifindex) = by_name.get(a.iface.as_str()) {
+ log::debug!(
+ "policy: resolved {} -> ifindex {ifindex} id {}",
+ a.iface,
+ a.id,
+ );
+ out.insert(ifindex, nic_entry(a.id, a.firewall_bridge));
+ } else {
+ log::trace!("policy: skipping {}: no such interface", a.iface);
+ }
+ }
+ log::debug!("policy: {} managed NICs on this host", out.len());
+ Ok(out)
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ fn entry(port: u16) -> CheckEntry {
+ CheckEntry {
+ ip_proto: 6,
+ family: Family::Any,
+ sport: None,
+ dport: Some((port, port)),
+ icmp_type: None,
+ icmp_code: None,
+ allow: true,
+ }
+ }
+
+ fn pair(allow: bool, checks: Vec<CheckEntry>) -> Pair {
+ Pair {
+ src_id: 3,
+ dst_id: 5,
+ checks,
+ allow,
+ }
+ }
+
+ fn encode_one(pair: &Pair) -> (PairValue, Option<(u64, RulesetValue)>) {
+ let encoded = encode_pairs(std::slice::from_ref(pair)).expect("one pair encodes");
+ let value = encoded
+ .pairs
+ .into_values()
+ .next()
+ .expect("the pair's value");
+ (value, encoded.rulesets.into_iter().next())
+ }
+
+ fn bytes(value: CheckValue) -> [u8; 16] {
+ // SAFETY: CheckValue is 16 bytes of repr(C) with no implicit padding
+ unsafe { std::mem::transmute(value) }
+ }
+
+ #[test]
+ fn check_entry_bytes() {
+ let plain = CheckEntry {
+ ip_proto: 17,
+ family: Family::Any,
+ sport: None,
+ dport: None,
+ icmp_type: None,
+ icmp_code: None,
+ allow: false,
+ };
+ assert_eq!(
+ bytes(plain.value()),
+ [17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
+ );
+
+ let v4 = CheckEntry {
+ family: Family::V4,
+ allow: true,
+ ..plain.clone()
+ };
+ assert_eq!(bytes(v4.value())[..4], [17, 4, 0, 1]);
+ let v6 = CheckEntry {
+ family: Family::V6,
+ ..plain.clone()
+ };
+ assert_eq!(bytes(v6.value())[1], 6);
+
+ let sport = CheckEntry {
+ sport: Some((1000, 2000)),
+ ..plain.clone()
+ };
+ let b = bytes(sport.value());
+ assert_eq!(b[2], CHECK_HAS_SPORT);
+ assert_eq!(b[4..6], 1000u16.to_ne_bytes());
+ assert_eq!(b[6..8], 2000u16.to_ne_bytes());
+ assert_eq!(b[8..], [0; 8]);
+
+ let dport = CheckEntry {
+ dport: Some((22, 23)),
+ ..plain.clone()
+ };
+ let b = bytes(dport.value());
+ assert_eq!(b[2], CHECK_HAS_DPORT);
+ assert_eq!(b[4..8], [0; 4]);
+ assert_eq!(b[8..10], 22u16.to_ne_bytes());
+ assert_eq!(b[10..12], 23u16.to_ne_bytes());
+
+ let icmp = CheckEntry {
+ ip_proto: 1,
+ icmp_type: Some(3),
+ icmp_code: Some(4),
+ ..plain.clone()
+ };
+ let b = bytes(icmp.value());
+ assert_eq!(b[2], CHECK_HAS_ICMP_TYPE | CHECK_HAS_ICMP_CODE);
+ assert_eq!(b[12..], [3, 4, 0, 0]);
+
+ let code_only = CheckEntry {
+ ip_proto: 1,
+ icmp_code: Some(0),
+ ..plain
+ };
+ assert_eq!(bytes(code_only.value())[2], CHECK_HAS_ICMP_CODE);
+ }
+
+ #[test]
+ fn an_admitting_pair_says_so() {
+ let (value, _) = encode_one(&pair(true, vec![]));
+ assert_eq!(value.flags, PAIR_ADMIT_ND);
+ let (value, _) = encode_one(&pair(false, vec![]));
+ assert_eq!(value.flags, 0);
+ }
+
+ #[test]
+ fn a_short_pair_stays_inline() {
+ let checks: Vec<_> = (1..=3).map(entry).collect();
+ let (value, ruleset) = encode_one(&pair(false, checks.clone()));
+ assert!(ruleset.is_none());
+ assert_eq!(value.allow, 0);
+ assert_eq!(value.flags, PAIR_ADMIT_ND);
+ assert_eq!(value.inline_count, 3);
+ assert_eq!(value.ruleset, 0);
+ for (slot, check) in value.checks.iter().zip(&checks) {
+ assert_eq!(*slot, check.value());
+ }
+ assert!(value.checks[3..].iter().all(|c| bytes(*c) == [0; 16]));
+ }
+
+ #[test]
+ fn a_pair_filling_the_inline_slots_needs_no_ruleset() {
+ let checks: Vec<_> = (1..=CHECK_INLINE as u16).map(entry).collect();
+ let (value, ruleset) = encode_one(&pair(true, checks.clone()));
+ assert!(ruleset.is_none());
+ assert_eq!(value.allow, 1);
+ assert_eq!(value.flags, PAIR_ADMIT_ND);
+ assert_eq!(value.inline_count, CHECK_INLINE as u8);
+ assert_eq!(value.ruleset, 0);
+ for (slot, check) in value.checks.iter().zip(&checks) {
+ assert_eq!(*slot, check.value());
+ }
+ }
+
+ #[test]
+ fn the_rest_goes_to_a_ruleset() {
+ let checks: Vec<_> = (1..=10).map(entry).collect();
+ let (value, ruleset) = encode_one(&pair(true, checks.clone()));
+ let (key, _) = ruleset.expect("a ruleset for the checks past the inline ones");
+ assert_eq!(value.flags, PAIR_ADMIT_ND | PAIR_HAS_RULESET);
+ assert_eq!(value.inline_count, CHECK_INLINE as u8);
+ assert_eq!(value.ruleset, key);
+ for (slot, check) in value.checks.iter().zip(&checks[..CHECK_INLINE]) {
+ assert_eq!(*slot, check.value());
+ }
+
+ let mut other_rest = checks.clone();
+ other_rest[CHECK_INLINE] = entry(100);
+ let (_, other_ruleset) = encode_one(&pair(true, other_rest));
+ assert_ne!(other_ruleset.expect("a ruleset").0, key);
+
+ let (other, _) = encode_one(&pair(false, checks));
+ assert_eq!(other.ruleset, key);
+ assert_eq!(other.flags, PAIR_ADMIT_ND | PAIR_HAS_RULESET);
+ }
+
+ #[test]
+ fn ruleset_value() {
+ let checks: Vec<_> = (1..=10).map(entry).collect();
+ let (_, ruleset) = encode_one(&pair(false, checks.clone()));
+ let (_, ruleset) = ruleset.unwrap();
+ assert_eq!(ruleset.count, 3);
+ assert_eq!(ruleset._pad, [0; 14]);
+ for (slot, check) in ruleset.checks.iter().zip(&checks[CHECK_INLINE..]) {
+ assert_eq!(*slot, check.value());
+ }
+ assert!(ruleset.checks[3..].iter().all(|c| bytes(*c) == [0; 16]));
+ }
+
+ fn assignment(iface: &str, id: u16) -> Assignment {
+ Assignment {
+ iface: iface.to_string(),
+ id,
+ firewall_bridge: false,
+ }
+ }
+
+ #[test]
+ fn a_nic_assigned_twice_fails() {
+ let assignments = [assignment("tap100i0", 3), assignment("tap100i1", 3)];
+ assert!(check_assignments(&assignments).is_ok());
+
+ let assignments = [assignment("tap100i0", 3), assignment("tap100i0", 4)];
+ let err = check_assignments(&assignments).unwrap_err().to_string();
+ assert!(err.contains("id 3") && err.contains("id 4"), "{err}");
+ }
+
+ fn placed(id: u16, flags: u16) -> NicEntry {
+ NicEntry {
+ flags,
+ ..nic_entry(id, false)
+ }
+ }
+
+ #[test]
+ fn a_pass_writes_only_what_differs() {
+ let live = HashMap::from([(1, placed(3, 0)), (2, placed(4, 0))]);
+ let desired = HashMap::from([
+ (1, nic_entry(3, false)),
+ (2, nic_entry(6, false)),
+ (3, nic_entry(7, false)),
+ ]);
+ let mut changes = nic_changes(&live, &desired);
+ changes.write.sort_by_key(|(ifindex, _)| *ifindex);
+ assert_eq!(changes.write, [(2, placed(6, 0)), (3, placed(7, 0))]);
+ assert!(changes.remove.is_empty());
+ }
+
+ #[test]
+ fn a_pass_keeps_the_flag_a_plug_set() {
+ let live = HashMap::from([(1, placed(3, NIC_ADMIT_ND))]);
+ let same_id = HashMap::from([(1, nic_entry(3, false))]);
+ assert!(nic_changes(&live, &same_id).write.is_empty());
+ let other_id = HashMap::from([(1, nic_entry(4, false))]);
+ let changes = nic_changes(&live, &other_id);
+ assert_eq!(changes.write, [(1, placed(4, NIC_ADMIT_ND))]);
+ }
+
+ #[test]
+ fn a_pass_writes_and_removes_only_what_changed() {
+ let long = |port: u16| (0..=CHECK_INLINE as u16).map(|n| entry(port + n)).collect();
+ let pair = |src_id, checks| Pair {
+ src_id,
+ ..pair(true, checks)
+ };
+ let before =
+ encode_pairs(&[pair(1, vec![]), pair(2, long(100)), pair(6, long(400))]).unwrap();
+ let after =
+ encode_pairs(&[pair(1, vec![]), pair(3, long(200)), pair(6, long(500))]).unwrap();
+ // a ruleset a failed pass left behind, no pair names it
+ let left = encode_pairs(&[pair(4, long(300))]).unwrap();
+ let live = before.pairs.clone();
+ let live_rulesets: HashSet<_> = before
+ .rulesets
+ .keys()
+ .chain(left.rulesets.keys())
+ .copied()
+ .collect();
+
+ let changes = pair_changes(&live, &live_rulesets, &after);
+ let key = |src_id| PairKey { src_id, dst_id: 5 };
+ let set = |keys: &[u64]| keys.iter().copied().collect::<HashSet<_>>();
+ let ruleset_of = |encoded: &Encoded, src_id| encoded.pairs[&key(src_id)].ruleset;
+ assert_eq!(changes.pairs_remove, [key(2)]);
+ let written: HashSet<_> = changes.pairs_write.iter().map(|(k, _)| *k).collect();
+ assert_eq!(written, HashSet::from([key(3), key(6)]));
+ assert_eq!(
+ set(&changes.rulesets_orphan),
+ HashSet::from([ruleset_of(&before, 2), ruleset_of(&left, 4)])
+ );
+ assert_eq!(
+ set(&changes.rulesets_remove),
+ HashSet::from([ruleset_of(&before, 6)])
+ );
+ let inserted: Vec<u64> = changes.rulesets_insert.iter().map(|(k, _)| *k).collect();
+ assert_eq!(set(&inserted), after.rulesets.keys().copied().collect());
+ }
+
+ #[test]
+ fn a_pair_given_twice_fails() {
+ let pairs = [pair(true, vec![]), pair(false, vec![entry(22)])];
+ assert!(encode_pairs(&pairs).is_err());
+ assert!(encode_pairs(&pairs[..1]).is_ok());
+ }
+
+ #[test]
+ fn a_pair_over_check_max_fails() {
+ let checks: Vec<_> = (0..=CHECK_MAX as u16).map(entry).collect();
+ assert!(encode_pairs(&[pair(true, checks[..CHECK_MAX].to_vec())]).is_ok());
+ assert!(encode_pairs(&[pair(true, checks)]).is_err());
+ }
+
+ fn many_pairs(count: usize, checks: impl Fn(usize) -> Vec<CheckEntry>) -> Vec<Pair> {
+ (0..count)
+ .map(|i| Pair {
+ src_id: (i >> 16) as u16,
+ dst_id: i as u16,
+ checks: checks(i),
+ allow: true,
+ })
+ .collect()
+ }
+
+ // run against small ceilings, the real ones cost a quarter GiB per test for the same check
+ #[test]
+ fn more_pairs_than_the_pairs_map_holds_fail() {
+ let mut pairs = many_pairs(9, |_| vec![]);
+ let err = encode_within(&pairs, 8, 8).err().unwrap().to_string();
+ assert!(err.contains("9 pairs"), "{err}");
+ pairs.pop();
+ assert!(encode_within(&pairs, 8, 8).is_ok());
+ }
+
+ #[test]
+ fn more_rulesets_than_a_pass_may_name_fail() {
+ let own = |i: usize| (0..=CHECK_INLINE).map(|n| entry((i + n) as u16)).collect();
+ let mut pairs = many_pairs(5, own);
+ let err = encode_within(&pairs, 8, 4).err().unwrap().to_string();
+ assert!(err.contains("5 rulesets"), "{err}");
+ pairs.pop();
+ assert!(encode_within(&pairs, 8, 4).is_ok());
+
+ let shared = |_| (0..=CHECK_INLINE).map(|n| entry(n as u16)).collect();
+ assert!(encode_within(&many_pairs(5, shared), 8, 4).is_ok());
+ }
+}
diff --git a/src/policy/types.rs b/src/policy/types.rs
new file mode 100644
index 000000000000..e411ba4ae6f2
--- /dev/null
+++ b/src/policy/types.rs
@@ -0,0 +1,155 @@
+//! Keep in sync with `bpf/types.h`.
+
+/// The checks one pair can carry, which also sizes a ruleset and bounds its walk.
+pub const CHECK_MAX: usize = 512;
+pub const CHECK_INLINE: usize = 7;
+/// The pairs a pass may name, the size of the pairs map.
+pub const PAIRS_MAX: usize = 1 << 20;
+/// The rulesets a pass may name. The map holds twice that, since the ones a pass replaces stay
+/// until its pairs are written.
+pub const RULESETS_MAX: usize = 1 << 14;
+/// The NIC entries a host can hold, the size of the interface map.
+pub const NICS_MAX: usize = 1 << 18;
+
+pub const ID_UNSTAMPED: u16 = 0;
+
+pub const PAIR_ADMIT_ND: u8 = 1 << 0;
+pub const PAIR_HAS_RULESET: u8 = 1 << 1;
+
+pub const NIC_ADMIT_ND: u16 = 1 << 0;
+
+pub const FAMILY_V4: u8 = 4;
+pub const FAMILY_V6: u8 = 6;
+
+pub const CHECK_HAS_SPORT: u8 = 1 << 0;
+pub const CHECK_HAS_DPORT: u8 = 1 << 1;
+pub const CHECK_HAS_ICMP_TYPE: u8 = 1 << 2;
+pub const CHECK_HAS_ICMP_CODE: u8 = 1 << 3;
+
+#[repr(C)]
+#[derive(Copy, Clone, Debug, PartialEq, Eq)]
+pub struct NicEntry {
+ pub id: u16,
+ pub flags: u16,
+ pub _pad: [u8; 4],
+}
+
+#[repr(C)]
+#[derive(Copy, Clone, Debug, Hash, PartialEq, Eq)]
+pub struct PairKey {
+ pub src_id: u16,
+ pub dst_id: u16,
+}
+
+#[repr(C)]
+#[derive(Copy, Clone, Debug, Default, Hash, PartialEq, Eq)]
+pub struct CheckValue {
+ pub ip_proto: u8,
+ pub family: u8,
+ pub match_flags: u8,
+ pub allow: u8,
+ pub sport_lo: u16,
+ pub sport_hi: u16,
+ pub dport_lo: u16,
+ pub dport_hi: u16,
+ pub icmp_type: u8,
+ pub icmp_code: u8,
+ pub _pad: [u8; 2],
+}
+
+#[repr(C)]
+#[derive(Copy, Clone, Debug, PartialEq, Eq)]
+pub struct PairValue {
+ pub allow: u8,
+ pub flags: u8,
+ pub inline_count: u8,
+ pub _pad: [u8; 5],
+ pub ruleset: u64,
+ pub checks: [CheckValue; CHECK_INLINE],
+}
+
+#[repr(C)]
+#[derive(Copy, Clone, Debug, PartialEq, Eq)]
+pub struct RulesetValue {
+ pub count: u16,
+ pub _pad: [u8; 14],
+ pub checks: [CheckValue; CHECK_MAX],
+}
+
+unsafe impl aya::Pod for NicEntry {}
+unsafe impl aya::Pod for PairKey {}
+unsafe impl aya::Pod for PairValue {}
+unsafe impl aya::Pod for RulesetValue {}
+
+#[cfg(test)]
+mod layout {
+ use super::*;
+ use core::mem::{offset_of, size_of};
+
+ #[test]
+ fn sizes() {
+ assert_eq!(size_of::<NicEntry>(), 8);
+ assert_eq!(size_of::<PairKey>(), 4);
+ assert_eq!(size_of::<CheckValue>(), 16);
+ assert_eq!(size_of::<PairValue>(), 128);
+ assert_eq!(size_of::<RulesetValue>(), 16 + CHECK_MAX * 16);
+ }
+
+ #[test]
+ fn offsets() {
+ assert_eq!(offset_of!(PairKey, dst_id), 2);
+ assert_eq!(offset_of!(NicEntry, flags), 2);
+ assert_eq!(offset_of!(CheckValue, sport_lo), 4);
+ assert_eq!(offset_of!(CheckValue, dport_lo), 8);
+ assert_eq!(offset_of!(CheckValue, icmp_type), 12);
+ assert_eq!(offset_of!(PairValue, allow), 0);
+ assert_eq!(offset_of!(PairValue, flags), 1);
+ assert_eq!(offset_of!(PairValue, inline_count), 2);
+ assert_eq!(offset_of!(PairValue, ruleset), 8);
+ assert_eq!(offset_of!(PairValue, checks), 16);
+ assert_eq!(offset_of!(RulesetValue, count), 0);
+ assert_eq!(offset_of!(RulesetValue, checks), 16);
+ }
+
+ #[test]
+ fn limits() {
+ let header = include_str!("bpf/types.h");
+ let defined = |name: &str| -> usize {
+ let prefix = format!("#define {name} ");
+ let value = header
+ .lines()
+ .find_map(|line| line.strip_prefix(prefix.as_str()))
+ .unwrap_or_else(|| panic!("the header defines no {name}"))
+ .trim();
+ let parsed = match value
+ .strip_prefix("(1 << ")
+ .and_then(|v| v.strip_suffix(')'))
+ {
+ Some(shift) => shift.parse().map(|shift: u32| 1 << shift),
+ None => value.parse(),
+ };
+ parsed.unwrap_or_else(|_| panic!("the header defines {name} as {value}, not a number"))
+ };
+ assert_eq!(defined("CHECK_MAX"), CHECK_MAX);
+ assert_eq!(defined("CHECK_INLINE"), CHECK_INLINE);
+ assert_eq!(defined("PAIRS_MAX"), PAIRS_MAX);
+ assert_eq!(defined("RULESETS_MAX"), RULESETS_MAX);
+ assert_eq!(defined("NICS_MAX"), NICS_MAX);
+ assert_eq!(defined("ID_UNSTAMPED"), usize::from(ID_UNSTAMPED));
+ assert_eq!(defined("PAIR_ADMIT_ND"), usize::from(PAIR_ADMIT_ND));
+ assert_eq!(defined("PAIR_HAS_RULESET"), usize::from(PAIR_HAS_RULESET));
+ assert_eq!(defined("NIC_ADMIT_ND"), usize::from(NIC_ADMIT_ND));
+ assert_eq!(defined("FAMILY_V4"), usize::from(FAMILY_V4));
+ assert_eq!(defined("FAMILY_V6"), usize::from(FAMILY_V6));
+ assert_eq!(defined("CHECK_HAS_SPORT"), usize::from(CHECK_HAS_SPORT));
+ assert_eq!(defined("CHECK_HAS_DPORT"), usize::from(CHECK_HAS_DPORT));
+ assert_eq!(
+ defined("CHECK_HAS_ICMP_TYPE"),
+ usize::from(CHECK_HAS_ICMP_TYPE)
+ );
+ assert_eq!(
+ defined("CHECK_HAS_ICMP_CODE"),
+ usize::from(CHECK_HAS_ICMP_CODE)
+ );
+ }
+}
diff --git a/src/subsystem.rs b/src/subsystem.rs
index 3bf163437044..8940c58c8500 100644
--- a/src/subsystem.rs
+++ b/src/subsystem.rs
@@ -42,7 +42,7 @@ const RUN_ROOT: &str = "/run/proxmox-ebpf";
/// The subsystems in the order their programs run on an interface, the first runs first. A name
/// missing from the list fails the build.
-pub const RUN_ORDER: &[&str] = &[];
+pub const RUN_ORDER: &[&str] = &["policy"];
const fn rank_of(name: &str) -> usize {
let mut i = 0;
@@ -360,15 +360,19 @@ impl TcPrograms {
/// a pin arriving after it is another attach's. A failed listing counts as a hit. So do links
/// still pinned under another schema, an install may be moving them over.
pub fn link_pinned(&self, name: &str) -> bool {
- let elsewhere = self
- .other_schemas()
- .map_or(true, |others| links_pinned_under(&others));
- elsewhere
+ self.links_pinned_elsewhere()
|| self
.live_links()
.map_or(true, |links| links.iter().any(|pin| pin.name == name))
}
+ /// Whether links are pinned under another schema, read without the lock. A failed listing
+ /// counts as a hit.
+ pub fn links_pinned_elsewhere(&self) -> bool {
+ self.other_schemas()
+ .map_or(true, |others| links_pinned_under(&others))
+ }
+
/// Take the apply lock shared, for a single-interface step.
pub fn lock_shared(&self) -> anyhow::Result<ApplyLock<'_, Shared>> {
self.lock(false)
@@ -676,6 +680,21 @@ impl TcPrograms {
Ok(ifindexes)
}
+ fn detach_gone(&self) -> anyhow::Result<Vec<u32>> {
+ let mut gone = Vec::new();
+ for pin in self.live_links()? {
+ // an index a new netdev took is not gone, the state under it may be the new one's
+ if tc::iface_name(pin.ifindex)?.is_some() {
+ continue;
+ }
+ self.reclaim_link(&pin);
+ if !gone.contains(&pin.ifindex) {
+ gone.push(pin.ifindex);
+ }
+ }
+ Ok(gone)
+ }
+
/// Unpin a link. The pin file goes even when the kernel object behind it cannot be opened
/// anymore.
fn reclaim_link(&self, pin: &tc::LinkPin) {
@@ -959,6 +978,12 @@ impl<M> ApplyLock<'_, M> {
pub fn attached(&self) -> anyhow::Result<Vec<tc::LinkPin>> {
self.programs.live_links()
}
+
+ /// Unpin the links of interfaces that are gone and return their indexes, for state keyed by
+ /// them. The kernel dropped such a link with its netdev, only the pin is left.
+ pub fn detach_gone(&self) -> anyhow::Result<Vec<u32>> {
+ self.programs.detach_gone()
+ }
}
impl ApplyLock<'_, Exclusive> {
diff --git a/tests/common/mod.rs b/tests/common/mod.rs
index 67fac33e3e2c..0d080d63882d 100644
--- a/tests/common/mod.rs
+++ b/tests/common/mod.rs
@@ -15,6 +15,7 @@ use std::ffi::{c_int, c_long, c_void};
pub const TC_ACT_UNSPEC: c_int = -1;
pub const TC_ACT_OK: c_int = 0;
+pub const TC_ACT_SHOT: c_int = 2;
pub const TC_ACT_REDIRECT: c_int = 7;
pub const PKT_CAP: usize = 2048;
@@ -27,6 +28,7 @@ pub struct SkBuff {
pub len: u32,
pub ifindex: u32,
pub mark: u32,
+ pub ingress_ifindex: u32,
}
/// The skb handed to a program plus the buffer behind it. The helpers get the skb pointer and
@@ -49,6 +51,9 @@ impl TestSkb {
len: packet.len() as u32,
ifindex,
mark: 0,
+ // a frame from a guest or the underlay came in on some port, only the host's own
+ // frames have none
+ ingress_ifindex: 1,
},
buf,
};
@@ -66,6 +71,18 @@ impl TestSkb {
&self.buf[..self.skb.len as usize]
}
+ pub fn set_mark(&mut self, mark: u32) {
+ self.skb.mark = mark;
+ }
+
+ pub fn mark(&self) -> u32 {
+ self.skb.mark
+ }
+
+ pub fn set_ingress_ifindex(&mut self, ifindex: u32) {
+ self.skb.ingress_ifindex = ifindex;
+ }
+
pub fn run(&mut self, prog: unsafe extern "C" fn(*mut SkBuff) -> c_int) -> c_int {
REDIRECTED.with(|r| *r.borrow_mut() = None);
unsafe { prog((self as *mut TestSkb).cast()) }
@@ -103,6 +120,11 @@ pub fn map_insert(map: *const c_void, key: &[u8], value: &[u8]) {
});
}
+pub fn bytes_of<T: aya::Pod>(value: &T) -> &[u8] {
+ // SAFETY: aya's Pod contract is that the type can be converted to a byte slice
+ unsafe { std::slice::from_raw_parts((value as *const T).cast::<u8>(), size_of::<T>()) }
+}
+
/// The ifindex of the redirect the last run issued, if any.
pub fn redirected() -> Option<u32> {
REDIRECTED.with(|r| r.borrow().map(|(ifindex, _)| ifindex))
@@ -205,3 +227,21 @@ pub extern "C" fn bpf_redirect(ifindex: u32, flags: u64) -> c_long {
REDIRECTED.with(|r| *r.borrow_mut() = Some((ifindex, flags)));
TC_ACT_REDIRECT as c_long
}
+
+#[unsafe(no_mangle)]
+pub extern "C" fn bpf_loop(
+ nr_loops: u32,
+ callback_fn: *const c_void,
+ callback_ctx: *mut c_void,
+ _flags: u64,
+) -> c_long {
+ let callback: extern "C" fn(u64, *mut c_void) -> c_long =
+ unsafe { std::mem::transmute(callback_fn) };
+ // like the kernel, a nonzero return stops the loop, and the count includes that call
+ for i in 0..nr_loops {
+ if callback(u64::from(i), callback_ctx) != 0 {
+ return c_long::from(i) + 1;
+ }
+ }
+ c_long::from(nr_loops)
+}
diff --git a/tests/policy.rs b/tests/policy.rs
new file mode 100644
index 000000000000..c755c2cc5a60
--- /dev/null
+++ b/tests/policy.rs
@@ -0,0 +1,775 @@
+//! The policy programs run natively, a crafted frame in and its verdict or mark out. The map
+//! images are built here rather than by the subsystem's encoder, so an encoder bug cannot hide a
+//! program bug.
+
+#![cfg(feature = "policy")]
+
+mod common;
+
+use std::ffi::{c_int, c_void};
+
+use common::*;
+use proxmox_ebpf::policy::types::*;
+
+unsafe extern "C" {
+ fn tc_tap_ingress(skb: *mut SkBuff) -> c_int;
+ fn tc_tap_egress(skb: *mut SkBuff) -> c_int;
+ static nics: u8;
+ static pairs: u8;
+ static rulesets: u8;
+}
+
+const IFINDEX: u32 = 7;
+const SRC_ID: u16 = 3;
+const DST_ID: u16 = 5;
+const RULESET_KEY: u64 = 0x5eed;
+
+const ETHERTYPE_IPV4: u16 = 0x0800;
+const ETHERTYPE_ARP: u16 = 0x0806;
+const ETHERTYPE_IPV6: u16 = 0x86dd;
+
+const PROTO_ICMP: u8 = 1;
+const PROTO_TCP: u8 = 6;
+const PROTO_UDP: u8 = 17;
+const PROTO_FRAGMENT: u8 = 44;
+const PROTO_ESP: u8 = 50;
+const PROTO_AH: u8 = 51;
+const PROTO_ICMPV6: u8 = 58;
+const PROTO_NONE: u8 = 59;
+const PROTO_DSTOPTS: u8 = 60;
+
+fn map(symbol: *const u8) -> *const c_void {
+ symbol.cast()
+}
+
+/// One check of a pair value.
+#[derive(Default)]
+struct Check {
+ ip_proto: u8,
+ family: u8,
+ sport: Option<(u16, u16)>,
+ dport: Option<(u16, u16)>,
+ icmp_type: Option<u8>,
+ icmp_code: Option<u8>,
+ allow: bool,
+}
+
+impl Check {
+ fn value(&self) -> CheckValue {
+ let mut v = CheckValue {
+ ip_proto: self.ip_proto,
+ family: self.family,
+ allow: self.allow as u8,
+ ..Default::default()
+ };
+ if let Some((lo, hi)) = self.sport {
+ v.match_flags |= CHECK_HAS_SPORT;
+ v.sport_lo = lo;
+ v.sport_hi = hi;
+ }
+ if let Some((lo, hi)) = self.dport {
+ v.match_flags |= CHECK_HAS_DPORT;
+ v.dport_lo = lo;
+ v.dport_hi = hi;
+ }
+ if let Some(ty) = self.icmp_type {
+ v.match_flags |= CHECK_HAS_ICMP_TYPE;
+ v.icmp_type = ty;
+ }
+ if let Some(code) = self.icmp_code {
+ v.match_flags |= CHECK_HAS_ICMP_CODE;
+ v.icmp_code = code;
+ }
+ v
+ }
+}
+
+/// The maps as a pass leaves them with this tap's entry and no pair yet.
+fn install_none() {
+ install_tap(0);
+}
+
+fn install_tap(flags: u16) {
+ register_map(map(&raw const nics), 4);
+ let entry = NicEntry {
+ id: DST_ID,
+ flags,
+ _pad: [0; 4],
+ };
+ map_insert(
+ map(&raw const nics),
+ &IFINDEX.to_ne_bytes(),
+ bytes_of(&entry),
+ );
+ register_map(map(&raw const pairs), 4);
+ register_map(map(&raw const rulesets), 8);
+}
+
+/// The maps as a pass leaves them for one pair, its own verdict plus the checks ahead of it.
+fn install(allow: bool, entries: &[Check]) {
+ install_none();
+ insert_pair(SRC_ID, DST_ID, allow, entries);
+}
+
+/// One pair on top of what is installed, nothing is cleared.
+fn insert_pair(src: u16, dst: u16, allow: bool, entries: &[Check]) {
+ let key = PairKey {
+ src_id: src,
+ dst_id: dst,
+ };
+ let (inline, rest) = entries.split_at(entries.len().min(CHECK_INLINE));
+ let mut value = PairValue {
+ allow: allow as u8,
+ flags: 0,
+ inline_count: inline.len() as u8,
+ _pad: [0; 5],
+ ruleset: 0,
+ checks: [CheckValue::default(); CHECK_INLINE],
+ };
+ if allow || entries.iter().any(|check| check.allow) {
+ value.flags |= PAIR_ADMIT_ND;
+ }
+ if !rest.is_empty() {
+ value.flags |= PAIR_HAS_RULESET;
+ value.ruleset = RULESET_KEY;
+ }
+ for (slot, check) in value.checks.iter_mut().zip(inline) {
+ *slot = check.value();
+ }
+ map_insert(map(&raw const pairs), bytes_of(&key), bytes_of(&value));
+
+ if !rest.is_empty() {
+ let mut ruleset = RulesetValue {
+ count: rest.len() as u16,
+ _pad: [0; 14],
+ checks: [CheckValue::default(); CHECK_MAX],
+ };
+ for (slot, check) in ruleset.checks.iter_mut().zip(rest) {
+ *slot = check.value();
+ }
+ map_insert(
+ map(&raw const rulesets),
+ &RULESET_KEY.to_ne_bytes(),
+ bytes_of(&ruleset),
+ );
+ }
+}
+
+fn run(frame: &[u8]) -> c_int {
+ run_from(SRC_ID, frame)
+}
+
+fn run_from(src_id: u16, frame: &[u8]) -> c_int {
+ let mut skb = TestSkb::new(frame, IFINDEX);
+ skb.set_mark(u32::from(src_id));
+ skb.run(tc_tap_egress)
+}
+
+fn ethernet(ethertype: u16, tags: usize, payload: &[u8]) -> Vec<u8> {
+ tagged(&vec![0x8100; tags], ethertype, payload)
+}
+
+fn tagged(tags: &[u16], ethertype: u16, payload: &[u8]) -> Vec<u8> {
+ let mut frame = vec![0x02; 12];
+ for tag in tags {
+ frame.extend_from_slice(&tag.to_be_bytes());
+ frame.extend_from_slice(&100u16.to_be_bytes());
+ }
+ frame.extend_from_slice(ðertype.to_be_bytes());
+ frame.extend_from_slice(payload);
+ frame
+}
+
+fn ipv4(protocol: u8, frag_off: u16, payload: &[u8]) -> Vec<u8> {
+ ipv4_with_options(protocol, frag_off, &[], payload)
+}
+
+/// An IPv4 header carrying `options`, a multiple of four bytes long.
+fn ipv4_with_options(protocol: u8, frag_off: u16, options: &[u8], payload: &[u8]) -> Vec<u8> {
+ let header_len = 20 + options.len();
+ let mut header = vec![0x40 | (header_len / 4) as u8, 0];
+ header.extend_from_slice(&((header_len + payload.len()) as u16).to_be_bytes());
+ header.extend_from_slice(&[0, 0]);
+ header.extend_from_slice(&frag_off.to_be_bytes());
+ header.extend_from_slice(&[64, protocol, 0, 0]);
+ header.extend_from_slice(&[10, 0, 0, 1]);
+ header.extend_from_slice(&[10, 0, 0, 2]);
+ header.extend_from_slice(options);
+ header.extend_from_slice(payload);
+ header
+}
+
+fn ipv6(next_header: u8, payload: &[u8]) -> Vec<u8> {
+ let mut header = vec![0x60, 0, 0, 0];
+ header.extend_from_slice(&(payload.len() as u16).to_be_bytes());
+ header.extend_from_slice(&[next_header, 64]);
+ header.extend_from_slice(&[0; 32]);
+ header.extend_from_slice(payload);
+ header
+}
+
+/// A fragment header for the fragment at `offset`, in units of eight bytes.
+fn fragment_header(next_header: u8, offset: u16) -> Vec<u8> {
+ let mut header = vec![next_header, 0];
+ // the offset sits in the top 13 bits, the low bit says more fragments follow
+ header.extend_from_slice(&((offset << 3) | 1).to_be_bytes());
+ header.extend_from_slice(&[0, 0, 0, 1]);
+ header
+}
+
+/// A destination options header of the minimum length, eight bytes.
+fn ext_header(next_header: u8) -> Vec<u8> {
+ long_ext_header(next_header, 0)
+}
+
+/// A destination options header of `units` eight byte units past the first.
+fn long_ext_header(next_header: u8, units: u8) -> Vec<u8> {
+ let mut header = vec![next_header, units];
+ header.extend_from_slice(&vec![1; 6 + usize::from(units) * 8]);
+ header
+}
+
+/// An authentication header of the minimum length, twelve bytes and no integrity check value.
+fn ah_header(next_header: u8) -> Vec<u8> {
+ // the length counts four byte units minus two
+ let mut header = vec![next_header, 1, 0, 0];
+ header.extend_from_slice(&[0, 0, 0, 1]);
+ header.extend_from_slice(&[0, 0, 0, 1]);
+ header
+}
+
+fn tcp(sport: u16, dport: u16) -> Vec<u8> {
+ let mut header = sport.to_be_bytes().to_vec();
+ header.extend_from_slice(&dport.to_be_bytes());
+ header.extend_from_slice(&[0; 16]);
+ header
+}
+
+fn udp(sport: u16, dport: u16) -> Vec<u8> {
+ let mut header = sport.to_be_bytes().to_vec();
+ header.extend_from_slice(&dport.to_be_bytes());
+ header.extend_from_slice(&[0, 8, 0, 0]);
+ header
+}
+
+fn icmpv6(ty: u8) -> Vec<u8> {
+ let mut header = vec![ty, 0, 0, 0];
+ header.extend_from_slice(&[0; 20]);
+ header
+}
+
+fn icmpv6_frame(ty: u8) -> Vec<u8> {
+ ethernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_ICMPV6, &icmpv6(ty)))
+}
+
+fn solicitation() -> Vec<u8> {
+ icmpv6_frame(135)
+}
+
+fn advertisement() -> Vec<u8> {
+ icmpv6_frame(136)
+}
+
+fn icmp_frame(ty: u8, code: u8) -> Vec<u8> {
+ let mut header = vec![ty, code, 0, 0];
+ header.extend_from_slice(&[0; 4]);
+ ethernet(ETHERTYPE_IPV4, 0, &ipv4(PROTO_ICMP, 0, &header))
+}
+
+fn arp() -> Vec<u8> {
+ ethernet(ETHERTYPE_ARP, 0, &[0; 28])
+}
+
+fn tcp_frame(sport: u16, dport: u16) -> Vec<u8> {
+ ethernet(ETHERTYPE_IPV4, 0, &ipv4(PROTO_TCP, 0, &tcp(sport, dport)))
+}
+
+fn tcp6_frame(sport: u16, dport: u16) -> Vec<u8> {
+ ethernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_TCP, &tcp(sport, dport)))
+}
+
+/// Allowing checks on the destination ports 1 to `count`, which the test frames miss unless they
+/// aim at one. Placed first, `CHECK_INLINE` of them push what follows into the ruleset.
+fn filler(ip_proto: u8, count: u16) -> Vec<Check> {
+ (1..=count)
+ .map(|port| Check {
+ ip_proto,
+ dport: Some((port, port)),
+ allow: true,
+ ..Check::default()
+ })
+ .collect()
+}
+
+fn ssh() -> Check {
+ Check {
+ ip_proto: PROTO_TCP,
+ dport: Some((20, 22)),
+ allow: true,
+ ..Check::default()
+ }
+}
+
+fn deny_ssh() -> Check {
+ Check {
+ allow: false,
+ ..ssh()
+ }
+}
+
+#[test]
+fn a_matching_port_takes_the_checks_verdict() {
+ install(false, &[ssh()]);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_port_outside_the_range_falls_to_the_pair() {
+ install(false, &[ssh()]);
+ assert_eq!(run(&tcp_frame(40000, 80)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_narrowed_deny_drops_only_what_it_matches() {
+ install(true, &[deny_ssh()]);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+ assert_eq!(run(&tcp_frame(40000, 80)), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_denied_pair_admits_arp_and_neighbor_discovery() {
+ install(false, &[ssh()]);
+ let echo = icmpv6_frame(128);
+
+ assert_eq!(run(&arp()), TC_ACT_UNSPEC);
+ assert_eq!(run(&solicitation()), TC_ACT_UNSPEC);
+ assert_eq!(run(&advertisement()), TC_ACT_UNSPEC);
+ assert_eq!(run(&echo), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_narrowed_deny_of_neighbor_discovery_stays_in_charge() {
+ install(
+ false,
+ &[
+ Check {
+ ip_proto: PROTO_ICMPV6,
+ family: 6,
+ icmp_type: Some(135),
+ ..Check::default()
+ },
+ ssh(),
+ ],
+ );
+ assert_eq!(run(&solicitation()), TC_ACT_SHOT);
+ assert_eq!(run(&arp()), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_later_fragment_matches_nothing() {
+ let fragment = ethernet(
+ ETHERTYPE_IPV4,
+ 0,
+ // the frame carries bytes 1480 onward of the datagram, so its first bytes only look
+ // like a TCP header
+ &ipv4(PROTO_TCP, 185, &tcp(40000, 22)),
+ );
+ install(false, &[ssh()]);
+ assert_eq!(run(&fragment), TC_ACT_SHOT);
+
+ install(true, &[deny_ssh()]);
+ assert_eq!(run(&fragment), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn one_vlan_tag_is_parsed_through() {
+ install(
+ false,
+ &[Check {
+ ip_proto: PROTO_UDP,
+ sport: Some((53, 53)),
+ allow: true,
+ ..Check::default()
+ }],
+ );
+ let dns = ipv4(PROTO_UDP, 0, &udp(53, 40000));
+ assert_eq!(run(ðernet(ETHERTYPE_IPV4, 1, &dns)), TC_ACT_UNSPEC);
+ assert_eq!(run(ðernet(ETHERTYPE_IPV4, 2, &dns)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_second_tag_in_the_data_is_unreadable() {
+ install(true, &[deny_ssh()]);
+ let ssh = ipv4(PROTO_TCP, 0, &tcp(40000, 22));
+ let web = ipv4(PROTO_TCP, 0, &tcp(40000, 80));
+ assert_eq!(run(&tagged(&[0x8100], ETHERTYPE_IPV4, &ssh)), TC_ACT_SHOT);
+ assert_eq!(run(&tagged(&[0x88a8], ETHERTYPE_IPV4, &ssh)), TC_ACT_SHOT);
+ assert_eq!(run(&tagged(&[0x88a8], ETHERTYPE_IPV4, &web)), TC_ACT_UNSPEC);
+ let two_tags = tagged(&[0x8100, 0x8100], ETHERTYPE_IPV4, &web);
+ let mixed_tags = tagged(&[0x8100, 0x88a8], ETHERTYPE_IPV4, &web);
+ assert_eq!(run(&two_tags), TC_ACT_SHOT);
+ assert_eq!(run(&mixed_tags), TC_ACT_SHOT);
+}
+
+#[test]
+fn an_extension_header_is_walked_past() {
+ install(
+ false,
+ &[Check {
+ ip_proto: PROTO_TCP,
+ dport: Some((443, 443)),
+ allow: true,
+ ..Check::default()
+ }],
+ );
+ let mut payload = ext_header(PROTO_TCP);
+ payload.extend_from_slice(&tcp(40000, 443));
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_DSTOPTS, &payload))),
+ TC_ACT_UNSPEC
+ );
+}
+
+#[test]
+fn the_icmp_code_narrows_the_match() {
+ install(
+ false,
+ &[Check {
+ ip_proto: PROTO_ICMP,
+ icmp_type: Some(3),
+ icmp_code: Some(4),
+ allow: true,
+ ..Check::default()
+ }],
+ );
+ assert_eq!(run(&icmp_frame(3, 4)), TC_ACT_UNSPEC);
+ assert_eq!(run(&icmp_frame(3, 1)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_family_pinned_check_skips_the_other_family() {
+ install(false, &[Check { family: 6, ..ssh() }]);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+ assert_eq!(run(&tcp6_frame(40000, 22)), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_port_check_skips_a_frame_without_ports() {
+ install(
+ false,
+ &[Check {
+ ip_proto: PROTO_ICMP,
+ dport: Some((0, u16::MAX)),
+ allow: true,
+ ..Check::default()
+ }],
+ );
+ assert_eq!(run(&icmp_frame(8, 0)), TC_ACT_SHOT);
+
+ install(
+ false,
+ &[Check {
+ ip_proto: PROTO_ICMP,
+ allow: true,
+ ..Check::default()
+ }],
+ );
+ assert_eq!(run(&icmp_frame(8, 0)), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_one_way_allow_resolves_addresses_in_reverse() {
+ install_none();
+ insert_pair(DST_ID, SRC_ID, true, &[]);
+ assert_eq!(run(&arp()), TC_ACT_UNSPEC);
+ assert_eq!(run(&advertisement()), TC_ACT_UNSPEC);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+
+ // a narrowed allow in reverse admits as well, a plain deny does not
+ insert_pair(DST_ID, SRC_ID, false, &[ssh()]);
+ assert_eq!(run(&arp()), TC_ACT_UNSPEC);
+ insert_pair(DST_ID, SRC_ID, false, &[]);
+ assert_eq!(run(&arp()), TC_ACT_SHOT);
+
+ // an explicit deny of the forward pair is rescued the same way
+ insert_pair(SRC_ID, DST_ID, false, &[]);
+ insert_pair(DST_ID, SRC_ID, true, &[]);
+ assert_eq!(run(&arp()), TC_ACT_UNSPEC);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_host_made_reply_passes_unstamped() {
+ install_none();
+ let from_host = |frame: &[u8]| {
+ let mut skb = TestSkb::new(frame, IFINDEX);
+ skb.set_ingress_ifindex(0);
+ skb.run(tc_tap_egress)
+ };
+ assert_eq!(from_host(&arp()), TC_ACT_UNSPEC);
+ assert_eq!(from_host(&advertisement()), TC_ACT_UNSPEC);
+ // the host's IP traffic still needs its allow
+ assert_eq!(from_host(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+ // an unstamped reply that came in on a port is a guest's or the underlay's
+ assert_eq!(run_from(0, &arp()), TC_ACT_SHOT);
+ // a stamped frame is never the host's
+ let mut stamped = TestSkb::new(&arp(), IFINDEX);
+ stamped.set_mark(u32::from(SRC_ID));
+ stamped.set_ingress_ifindex(0);
+ assert_eq!(stamped.run(tc_tap_egress), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_frame_the_host_turned_around_passes_unstamped() {
+ install_none();
+ let turned = |src_id: u16, frame: &[u8]| {
+ let mut skb = TestSkb::new(frame, IFINDEX);
+ skb.set_mark(u32::from(src_id));
+ skb.set_ingress_ifindex(IFINDEX);
+ skb.run(tc_tap_egress)
+ };
+ // only a program ahead of the policy on the tap makes such a frame, so all of it passes
+ assert_eq!(turned(0, &tcp_frame(40000, 22)), TC_ACT_UNSPEC);
+ assert_eq!(turned(0, &arp()), TC_ACT_UNSPEC);
+ // a guest frame the bridge reflects carries the stamp and needs its pair
+ assert_eq!(turned(DST_ID, &tcp_frame(40000, 22)), TC_ACT_SHOT);
+ // unstamped from another port is a guest's or the underlay's
+ assert_eq!(run_from(0, &tcp_frame(40000, 22)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_flagged_tap_passes_unstamped_arp_and_nd_from_any_port() {
+ install_tap(NIC_ADMIT_ND);
+ // what the host made came in on the bridge's veth, like a frame of any other sender
+ assert_eq!(run_from(0, &arp()), TC_ACT_UNSPEC);
+ assert_eq!(run_from(0, &advertisement()), TC_ACT_UNSPEC);
+ assert_eq!(run_from(0, &tcp_frame(40000, 22)), TC_ACT_SHOT);
+ // a stamped sender still needs a pair that admits something
+ assert_eq!(run_from(SRC_ID, &arp()), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_missing_pair_denies() {
+ install(true, &[]);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_UNSPEC);
+ assert_eq!(run_from(SRC_ID + 1, &tcp_frame(40000, 22)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_pair_without_checks_takes_its_own_verdict() {
+ install(true, &[]);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_UNSPEC);
+ assert_eq!(run(&arp()), TC_ACT_UNSPEC);
+
+ install(false, &[]);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+ assert_eq!(run(&arp()), TC_ACT_SHOT);
+}
+
+#[test]
+fn an_ipv6_fragment_matches_only_as_the_first() {
+ install(false, &[ssh()]);
+ let mut first = fragment_header(PROTO_TCP, 0);
+ first.extend_from_slice(&tcp(40000, 22));
+ let mut later = fragment_header(PROTO_TCP, 185);
+ later.extend_from_slice(&tcp(40000, 22));
+
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_FRAGMENT, &first))),
+ TC_ACT_UNSPEC
+ );
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_FRAGMENT, &later))),
+ TC_ACT_SHOT
+ );
+}
+
+#[test]
+fn ingress_stamps_the_id_into_the_low_bits_only() {
+ install(true, &[]);
+ let mut skb = TestSkb::new(&tcp_frame(40000, 22), IFINDEX);
+ skb.set_mark(0xabcd_1234);
+ assert_eq!(skb.run(tc_tap_ingress), TC_ACT_UNSPEC);
+ assert_eq!(skb.mark(), 0xabcd_0000 | u32::from(DST_ID));
+
+ let mut unknown = TestSkb::new(&tcp_frame(40000, 22), IFINDEX + 1);
+ unknown.set_mark(0xabcd_1234);
+ assert_eq!(unknown.run(tc_tap_ingress), TC_ACT_UNSPEC);
+ assert_eq!(unknown.mark(), 0xabcd_1234);
+}
+
+#[test]
+fn a_check_past_the_inline_ones_decides_through_the_ruleset() {
+ let mut checks = filler(PROTO_TCP, CHECK_INLINE as u16);
+ checks.push(ssh());
+ install(false, &checks);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_UNSPEC);
+ assert_eq!(run(&tcp_frame(40000, 80)), TC_ACT_SHOT);
+}
+
+#[test]
+fn a_ruleset_deny_drops_what_it_matches() {
+ let mut checks = filler(PROTO_TCP, CHECK_INLINE as u16);
+ checks.push(deny_ssh());
+ install(true, &checks);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+ assert_eq!(run(&tcp_frame(40000, 80)), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_missing_ruleset_drops() {
+ let mut checks = filler(PROTO_TCP, CHECK_INLINE as u16);
+ checks.push(deny_ssh());
+ install(true, &checks);
+ // the pair keeps naming the ruleset while the map holds none
+ register_map(map(&raw const rulesets), 8);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+ assert_eq!(run(&tcp_frame(40000, 80)), TC_ACT_SHOT);
+ // a check ahead of the ruleset still decides
+ assert_eq!(run(&tcp_frame(40000, 1)), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn a_long_extension_header_is_walked_past() {
+ install(
+ false,
+ &[Check {
+ ip_proto: PROTO_TCP,
+ dport: Some((443, 443)),
+ allow: true,
+ ..Check::default()
+ }],
+ );
+ // read with the length ignored, the second unit's bytes would pass for the ports
+ let mut payload = long_ext_header(PROTO_TCP, 1);
+ payload.extend_from_slice(&tcp(40000, 443));
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_DSTOPTS, &payload))),
+ TC_ACT_UNSPEC
+ );
+}
+
+#[test]
+fn a_chain_past_the_bound_is_dropped() {
+ let chain = |headers: usize| {
+ let mut payload = Vec::new();
+ for index in 0..headers {
+ let next = if index + 1 < headers {
+ PROTO_DSTOPTS
+ } else {
+ PROTO_TCP
+ };
+ payload.extend_from_slice(&ext_header(next));
+ }
+ payload.extend_from_slice(&tcp(40000, 22));
+ ethernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_DSTOPTS, &payload))
+ };
+
+ install(false, &[ssh()]);
+ assert_eq!(run(&chain(12)), TC_ACT_UNSPEC);
+ assert_eq!(run(&chain(13)), TC_ACT_SHOT);
+
+ install(true, &[deny_ssh()]);
+ // a chain within the bound reaches L4 and the narrowed check decides, while a longer chain is
+ // dropped even though the pair allows
+ assert_eq!(run(&chain(12)), TC_ACT_SHOT);
+ assert_eq!(run(&chain(13)), TC_ACT_SHOT);
+}
+
+#[test]
+fn an_unreadable_l4_header_is_dropped_where_checks_apply() {
+ let mut short6 = fragment_header(PROTO_TCP, 0);
+ short6.extend_from_slice(&tcp(40000, 22)[..2]);
+ let short6 = ethernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_FRAGMENT, &short6));
+ let short4 = ethernet(
+ ETHERTYPE_IPV4,
+ 0,
+ &ipv4(PROTO_TCP, 0x2000, &tcp(40000, 22)[..2]),
+ );
+
+ let none6 = ethernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_NONE, &[]));
+ let mut whole6 = fragment_header(PROTO_TCP, 0);
+ whole6.extend_from_slice(&tcp(40000, 80));
+ let whole6 = ethernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_FRAGMENT, &whole6));
+
+ install(true, &[deny_ssh()]);
+ assert_eq!(run(&short6), TC_ACT_SHOT);
+ assert_eq!(run(&short4), TC_ACT_SHOT);
+ assert_eq!(run(&none6), TC_ACT_UNSPEC);
+ assert_eq!(run(&whole6), TC_ACT_UNSPEC);
+
+ // without checks the pair's verdict decides as for any other frame
+ install(true, &[]);
+ assert_eq!(run(&short6), TC_ACT_UNSPEC);
+ assert_eq!(run(&short4), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn an_authentication_header_is_walked_past() {
+ install(true, &[deny_ssh()]);
+ let mut payload = ah_header(PROTO_TCP);
+ payload.extend_from_slice(&tcp(40000, 22));
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_AH, &payload))),
+ TC_ACT_SHOT
+ );
+}
+
+#[test]
+fn esp_is_not_walked_past() {
+ install(true, &[deny_ssh()]);
+ // the header behind ESP is encrypted, so a TCP check has nothing to match
+ let mut payload = vec![0, 0, 0, 1, 0, 0, 0, 1];
+ payload.extend_from_slice(&tcp(40000, 22));
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_ESP, &payload))),
+ TC_ACT_UNSPEC
+ );
+}
+
+#[test]
+fn ipv4_options_are_skipped() {
+ install(false, &[ssh()]);
+ let frame = ethernet(
+ ETHERTYPE_IPV4,
+ 0,
+ &ipv4_with_options(PROTO_TCP, 0, &[1; 8], &tcp(40000, 22)),
+ );
+ assert_eq!(run(&frame), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn an_interface_without_an_entry_passes() {
+ install(false, &[]);
+ let mut skb = TestSkb::new(&tcp_frame(40000, 22), IFINDEX + 1);
+ skb.set_mark(u32::from(SRC_ID));
+ assert_eq!(skb.run(tc_tap_egress), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn an_inline_check_decides_before_the_ruleset() {
+ let mut checks = filler(PROTO_TCP, CHECK_INLINE as u16);
+ checks[CHECK_INLINE - 1] = deny_ssh();
+ checks.push(ssh());
+ install(true, &checks);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+}
+
+#[test]
+fn the_earlier_ruleset_check_decides() {
+ let mut checks = filler(PROTO_TCP, CHECK_INLINE as u16);
+ checks.push(deny_ssh());
+ checks.push(ssh());
+ install(true, &checks);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+}
+
+#[test]
+fn the_last_check_of_the_longest_pair_decides() {
+ let mut checks = filler(PROTO_TCP, CHECK_INLINE as u16);
+ checks.extend(filler(PROTO_UDP, (CHECK_MAX - CHECK_INLINE - 1) as u16));
+ checks.push(ssh());
+ install(false, &checks);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_UNSPEC);
+ assert_eq!(run(&tcp_frame(40000, 80)), TC_ACT_SHOT);
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-ebpf v3 03/45] policy: add a check for non-first fragments
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ebpf v3 01/45] build: keep the build directory out of the BPF objects Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ebpf v3 02/45] policy: add the enforcement subsystem for guest interfaces Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 04/45] ve-config: firewall: add a constructor for sctp matches Hannes Laimer
` (41 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
A non-first fragment has no transport header, so no check matched it,
and a pair allowed only through narrowed rules dropped every later
fragment. Add a check that matches non-first fragments of a specific
(or any) protocol.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/policy/bpf/policy.bpf.c | 26 +++++++++++----
src/policy/bpf/types.h | 3 ++
src/policy/mod.rs | 63 ++++++++++++++++++++++---------------
src/policy/types.rs | 4 +++
tests/policy.rs | 62 ++++++++++++++++++++++++++++++++++++
5 files changed, 127 insertions(+), 31 deletions(-)
diff --git a/src/policy/bpf/policy.bpf.c b/src/policy/bpf/policy.bpf.c
index 53acd35cc6df..fcf513bb13de 100644
--- a/src/policy/bpf/policy.bpf.c
+++ b/src/policy/bpf/policy.bpf.c
@@ -82,7 +82,10 @@ int tc_tap_ingress(struct __sk_buff *skb) {
return TC_ACT_UNSPEC;
}
-// What a check matches on. `matchable` is set once the parse reached the L4 header.
+// What a check matches on. `matchable` is set once the parse reached the L4 header. `fragment`
+// marks a non-first fragment, which has a protocol and no L4 header. `unreadable` marks a frame
+// the parse could not read to its L4 header, such as one cut short or behind a second tag in the
+// frame data.
struct frame {
__u16 eth_proto;
__u16 sport;
@@ -95,6 +98,7 @@ struct frame {
__u8 has_ports;
__u8 has_icmp;
__u8 unreadable;
+ __u8 fragment;
};
static __always_inline int is_ext_header(__u8 proto) {
@@ -114,7 +118,11 @@ static __always_inline int ipv6_l4(struct __sk_buff *skb, struct frame *f, __u32
return 0;
}
// the offset sits in the top 13 bits of the second halfword
- if (hdr[2] || (hdr[3] & 0xf8)) return 0;
+ if (hdr[2] || (hdr[3] & 0xf8)) {
+ f->fragment = 1;
+ f->ip_proto = hdr[0];
+ return 0;
+ }
*off += 8;
} else {
if (bpf_skb_load_bytes(skb, *off, hdr, 2) < 0) {
@@ -206,10 +214,12 @@ static __always_inline void parse_frame(struct __sk_buff *skb, struct frame *f)
f->unreadable = 1;
return;
}
- // only the first fragment carries the L4 header
- if (bpf_ntohs(ip.frag_off) & IPV4_FRAG_OFFSET) return;
f->family = FAMILY_V4;
f->ip_proto = ip.protocol;
+ if (bpf_ntohs(ip.frag_off) & IPV4_FRAG_OFFSET) {
+ f->fragment = 1;
+ return;
+ }
off += (__u32)ip.ihl * 4;
} else if (f->eth_proto == ETHERTYPE_IPV6) {
struct ipv6hdr ip6;
@@ -229,9 +239,13 @@ static __always_inline void parse_frame(struct __sk_buff *skb, struct frame *f)
}
static __always_inline int check_matches(const struct frame *f, const struct check_value *c) {
- if (!f->matchable) return 0;
+ if (c->match_flags & CHECK_FRAGMENT) {
+ if (!f->fragment) return 0;
+ } else if (!f->matchable) {
+ return 0;
+ }
if (c->family && c->family != f->family) return 0;
- if (c->ip_proto != f->ip_proto) return 0;
+ if ((c->match_flags & CHECK_HAS_PROTO) && c->ip_proto != f->ip_proto) return 0;
if (c->match_flags & CHECK_HAS_SPORT) {
if (!f->has_ports || f->sport < c->sport_lo || f->sport > c->sport_hi) return 0;
diff --git a/src/policy/bpf/types.h b/src/policy/bpf/types.h
index a8b1df66fc41..0bfe1d3b9038 100644
--- a/src/policy/bpf/types.h
+++ b/src/policy/bpf/types.h
@@ -26,6 +26,9 @@
#define CHECK_HAS_DPORT (1 << 1)
#define CHECK_HAS_ICMP_TYPE (1 << 2)
#define CHECK_HAS_ICMP_CODE (1 << 3)
+// the check is for non-first fragments, which no other check matches
+#define CHECK_FRAGMENT (1 << 4)
+#define CHECK_HAS_PROTO (1 << 5)
struct nic_entry {
__u16 id;
diff --git a/src/policy/mod.rs b/src/policy/mod.rs
index 62ee7576b6fc..e3d3e580f7f3 100644
--- a/src/policy/mod.rs
+++ b/src/policy/mod.rs
@@ -76,8 +76,11 @@ pub enum Family {
/// left out is not matched on.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct CheckEntry {
- pub ip_proto: u8,
+ pub ip_proto: Option<u8>,
pub family: Family,
+ /// Match non-first fragments instead of frames with an L4 header. A fragment has no L4 header,
+ /// so a check that names a port or an ICMP field matches nothing.
+ pub fragment: bool,
pub sport: Option<(u16, u16)>,
pub dport: Option<(u16, u16)>,
pub icmp_type: Option<u8>,
@@ -88,6 +91,12 @@ pub struct CheckEntry {
impl CheckEntry {
fn value(&self) -> CheckValue {
let mut match_flags = 0;
+ if self.fragment {
+ match_flags |= CHECK_FRAGMENT;
+ }
+ if self.ip_proto.is_some() {
+ match_flags |= CHECK_HAS_PROTO;
+ }
if self.sport.is_some() {
match_flags |= CHECK_HAS_SPORT;
}
@@ -103,7 +112,7 @@ impl CheckEntry {
let (sport_lo, sport_hi) = self.sport.unwrap_or_default();
let (dport_lo, dport_hi) = self.dport.unwrap_or_default();
CheckValue {
- ip_proto: self.ip_proto,
+ ip_proto: self.ip_proto.unwrap_or_default(),
family: self.family as u8,
match_flags,
allow: self.allow as u8,
@@ -681,13 +690,10 @@ mod tests {
fn entry(port: u16) -> CheckEntry {
CheckEntry {
- ip_proto: 6,
- family: Family::Any,
- sport: None,
+ ip_proto: Some(6),
dport: Some((port, port)),
- icmp_type: None,
- icmp_code: None,
allow: true,
+ ..Default::default()
}
}
@@ -718,25 +724,19 @@ mod tests {
#[test]
fn check_entry_bytes() {
let plain = CheckEntry {
- ip_proto: 17,
- family: Family::Any,
- sport: None,
- dport: None,
- icmp_type: None,
- icmp_code: None,
- allow: false,
+ ip_proto: Some(17),
+ ..Default::default()
};
- assert_eq!(
- bytes(plain.value()),
- [17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
- );
+ let b = bytes(plain.value());
+ assert_eq!(b[..4], [17, 0, CHECK_HAS_PROTO, 0]);
+ assert_eq!(b[4..], [0; 12]);
let v4 = CheckEntry {
family: Family::V4,
allow: true,
..plain.clone()
};
- assert_eq!(bytes(v4.value())[..4], [17, 4, 0, 1]);
+ assert_eq!(bytes(v4.value())[..4], [17, 4, CHECK_HAS_PROTO, 1]);
let v6 = CheckEntry {
family: Family::V6,
..plain.clone()
@@ -748,7 +748,7 @@ mod tests {
..plain.clone()
};
let b = bytes(sport.value());
- assert_eq!(b[2], CHECK_HAS_SPORT);
+ assert_eq!(b[2], CHECK_HAS_PROTO | CHECK_HAS_SPORT);
assert_eq!(b[4..6], 1000u16.to_ne_bytes());
assert_eq!(b[6..8], 2000u16.to_ne_bytes());
assert_eq!(b[8..], [0; 8]);
@@ -758,27 +758,40 @@ mod tests {
..plain.clone()
};
let b = bytes(dport.value());
- assert_eq!(b[2], CHECK_HAS_DPORT);
+ assert_eq!(b[2], CHECK_HAS_PROTO | CHECK_HAS_DPORT);
assert_eq!(b[4..8], [0; 4]);
assert_eq!(b[8..10], 22u16.to_ne_bytes());
assert_eq!(b[10..12], 23u16.to_ne_bytes());
let icmp = CheckEntry {
- ip_proto: 1,
+ ip_proto: Some(1),
icmp_type: Some(3),
icmp_code: Some(4),
..plain.clone()
};
let b = bytes(icmp.value());
- assert_eq!(b[2], CHECK_HAS_ICMP_TYPE | CHECK_HAS_ICMP_CODE);
+ assert_eq!(
+ b[2],
+ CHECK_HAS_PROTO | CHECK_HAS_ICMP_TYPE | CHECK_HAS_ICMP_CODE
+ );
assert_eq!(b[12..], [3, 4, 0, 0]);
let code_only = CheckEntry {
- ip_proto: 1,
+ ip_proto: Some(1),
icmp_code: Some(0),
+ ..plain.clone()
+ };
+ assert_eq!(
+ bytes(code_only.value())[2],
+ CHECK_HAS_PROTO | CHECK_HAS_ICMP_CODE
+ );
+
+ let fragment = CheckEntry {
+ ip_proto: None,
+ fragment: true,
..plain
};
- assert_eq!(bytes(code_only.value())[2], CHECK_HAS_ICMP_CODE);
+ assert_eq!(bytes(fragment.value())[..3], [0, 0, CHECK_FRAGMENT]);
}
#[test]
diff --git a/src/policy/types.rs b/src/policy/types.rs
index e411ba4ae6f2..c8e97c177625 100644
--- a/src/policy/types.rs
+++ b/src/policy/types.rs
@@ -25,6 +25,8 @@ pub const CHECK_HAS_SPORT: u8 = 1 << 0;
pub const CHECK_HAS_DPORT: u8 = 1 << 1;
pub const CHECK_HAS_ICMP_TYPE: u8 = 1 << 2;
pub const CHECK_HAS_ICMP_CODE: u8 = 1 << 3;
+pub const CHECK_FRAGMENT: u8 = 1 << 4;
+pub const CHECK_HAS_PROTO: u8 = 1 << 5;
#[repr(C)]
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
@@ -151,5 +153,7 @@ mod layout {
defined("CHECK_HAS_ICMP_CODE"),
usize::from(CHECK_HAS_ICMP_CODE)
);
+ assert_eq!(defined("CHECK_FRAGMENT"), usize::from(CHECK_FRAGMENT));
+ assert_eq!(defined("CHECK_HAS_PROTO"), usize::from(CHECK_HAS_PROTO));
}
}
diff --git a/tests/policy.rs b/tests/policy.rs
index c755c2cc5a60..5ca034da5f68 100644
--- a/tests/policy.rs
+++ b/tests/policy.rs
@@ -51,6 +51,8 @@ struct Check {
dport: Option<(u16, u16)>,
icmp_type: Option<u8>,
icmp_code: Option<u8>,
+ fragment: bool,
+ any_proto: bool,
allow: bool,
}
@@ -80,6 +82,12 @@ impl Check {
v.match_flags |= CHECK_HAS_ICMP_CODE;
v.icmp_code = code;
}
+ if self.fragment {
+ v.match_flags |= CHECK_FRAGMENT;
+ }
+ if !self.any_proto {
+ v.match_flags |= CHECK_HAS_PROTO;
+ }
v
}
}
@@ -381,6 +389,60 @@ fn a_later_fragment_matches_nothing() {
assert_eq!(run(&fragment), TC_ACT_UNSPEC);
}
+#[test]
+fn a_fragment_check_matches_later_fragments_only() {
+ let fragments = Check {
+ fragment: true,
+ any_proto: true,
+ allow: true,
+ ..Default::default()
+ };
+ install(false, &[fragments]);
+
+ let later = ipv4(PROTO_TCP, 185, &tcp(40000, 22));
+ assert_eq!(run(ðernet(ETHERTYPE_IPV4, 0, &later)), TC_ACT_UNSPEC);
+ let mut later = fragment_header(PROTO_UDP, 185);
+ later.extend_from_slice(&udp(40000, 53));
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_FRAGMENT, &later))),
+ TC_ACT_UNSPEC
+ );
+
+ // the first fragment carries the L4 header, more fragments following or not
+ let first = ipv4(PROTO_TCP, 0x2000, &tcp(40000, 22));
+ assert_eq!(run(ðernet(ETHERTYPE_IPV4, 0, &first)), TC_ACT_SHOT);
+ assert_eq!(run(&tcp_frame(40000, 22)), TC_ACT_SHOT);
+ let mut first = fragment_header(PROTO_UDP, 0);
+ first.extend_from_slice(&udp(40000, 53));
+ assert_eq!(
+ run(ðernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_FRAGMENT, &first))),
+ TC_ACT_SHOT
+ );
+}
+
+#[test]
+fn a_fragment_check_can_name_the_protocol() {
+ let udp_fragments = Check {
+ ip_proto: PROTO_UDP,
+ fragment: true,
+ allow: true,
+ ..Default::default()
+ };
+ install(false, &[udp_fragments]);
+
+ let later = |protocol| ethernet(ETHERTYPE_IPV4, 0, &ipv4(protocol, 185, &[0; 8]));
+ assert_eq!(run(&later(PROTO_UDP)), TC_ACT_UNSPEC);
+ assert_eq!(run(&later(PROTO_TCP)), TC_ACT_SHOT);
+
+ let later = |protocol| {
+ let mut payload = fragment_header(protocol, 185);
+ payload.extend_from_slice(&[0; 8]);
+ ethernet(ETHERTYPE_IPV6, 0, &ipv6(PROTO_FRAGMENT, &payload))
+ };
+ assert_eq!(run(&later(PROTO_UDP)), TC_ACT_UNSPEC);
+ assert_eq!(run(&later(PROTO_TCP)), TC_ACT_SHOT);
+}
+
#[test]
fn one_vlan_tag_is_parsed_through() {
install(
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-ve-rs v3 04/45] ve-config: firewall: add a constructor for sctp matches
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (2 preceding siblings ...)
2026-10-27 8:54 ` [PATCH proxmox-ebpf v3 03/45] policy: add a check for non-first fragments Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 05/45] ve-config: sdn: microseg: add identity allocation for group sets Hannes Laimer
` (40 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
The tcp and udp matches can already be built from source and
destination ports, while an sctp match with ports can only be built by
parsing a rule. Add the missing constructor so a match on any protocol
that carries ports can be built without parsing.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
proxmox-ve-config/src/firewall/types/rule_match.rs | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/proxmox-ve-config/src/firewall/types/rule_match.rs b/proxmox-ve-config/src/firewall/types/rule_match.rs
index 8e142f74e4e6..8e9c0c69efe8 100644
--- a/proxmox-ve-config/src/firewall/types/rule_match.rs
+++ b/proxmox-ve-config/src/firewall/types/rule_match.rs
@@ -437,6 +437,10 @@ pub struct Sctp {
}
impl Sctp {
+ pub fn new(ports: Ports) -> Self {
+ Self { ports }
+ }
+
fn from_options(options: &RuleOptions) -> Result<Self, Error> {
Ok(Self {
ports: Ports::from_options(options)?,
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-ve-rs v3 05/45] ve-config: sdn: microseg: add identity allocation for group sets
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (3 preceding siblings ...)
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 04/45] ve-config: firewall: add a constructor for sctp matches Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 06/45] ve-config: sdn: microseg: add config types Hannes Laimer
` (39 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
A guest interface carries a set of groups, but the wire has room for a
single 16-bit id. Group sets that no rule can tell apart form one
class with one id. So the ids in use grow with what the rules
distinguish, not with the number of distinct group sets.
An id is stamped on one host and may be looked up on another after a
config change. Reusing it for an unrelated class could give a packet
still in flight the wrong verdict. So a class keeps its id while it
exists. A new class takes an id never issued before. Once every id has
been issued, an id unused for an hour can be issued again. Without one
the allocation fails.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
.../src/sdn/microseg/identity.rs | 459 ++++++++++++++++++
proxmox-ve-config/src/sdn/microseg/mod.rs | 9 +
proxmox-ve-config/src/sdn/mod.rs | 1 +
3 files changed, 469 insertions(+)
create mode 100644 proxmox-ve-config/src/sdn/microseg/identity.rs
create mode 100644 proxmox-ve-config/src/sdn/microseg/mod.rs
diff --git a/proxmox-ve-config/src/sdn/microseg/identity.rs b/proxmox-ve-config/src/sdn/microseg/identity.rs
new file mode 100644
index 000000000000..24e91402070e
--- /dev/null
+++ b/proxmox-ve-config/src/sdn/microseg/identity.rs
@@ -0,0 +1,459 @@
+//! Signature-based identity allocation for microseg.
+//!
+//! An *identity* is not one group set but a class of group sets that no policy can tell apart. Two
+//! sets share an id when every policy's src and dst predicate fires the same way on both. The class
+//! carries an *id* that is stamped on the wire, and a *representative*, the smallest member set in
+//! sort order. A later split or merge therefore keeps the same survivor, and the ids stay stable.
+//!
+//! Allocation is **fresh-first**. A packet stamped with a number can still be in flight when the
+//! config changes, and rebinding the number to an unrelated class could give it the wrong verdict.
+//! Ids therefore come from a counter that only increases, and a split issues fresh ones. An id no
+//! class carries any more, after a merge or once its sets left the config, retires into a
+//! time-based quarantine (`RETIRE_QUARANTINE_SECS`). Once the counter has issued every number,
+//! allocation reclaims the lowest number whose quarantine has passed. It fails only with all 65535
+//! numbers live or quarantined at the same time.
+
+use std::collections::{BTreeMap, BTreeSet, HashMap};
+
+use serde::{Deserialize, Deserializer, Serialize};
+
+use proxmox_schema::api;
+
+/// The group set assigned to an interface. `BTreeSet`'s `Ord` compares the sorted elements in
+/// order, which is exactly the representative order.
+pub(crate) type GroupSet = BTreeSet<String>;
+
+/// The reserved name of the built-in *untagged* group, the no-group identity. A source predicate
+/// names it to match unstamped traffic, such as gateway, DHCP and ARP replies or cross-node frames
+/// without a GBP tag. It can stand next to real groups, as in `any{untagged, web}`. It is not a
+/// stored group and cannot be used in an assignment.
+pub(crate) const UNTAGGED_GROUP: &str = "untagged";
+
+/// An unstamped frame reads as this id. It is never issued to a class, so a real group set
+/// (even one no policy matches) never collides with it and never inherits its `untagged`
+/// allows. Its representative in the rule fold is the built-in `untagged` group.
+pub(crate) const UNTAGGED_ID: u16 = 0;
+
+/// Reusing a number is only safe once no packet stamped under its old meaning can still exist. The
+/// generous margin also covers nodes that apply a render late. A node cut off from pmxcfs for
+/// longer than this while still passing traffic keeps stamping ids that may by then name another
+/// class. A cluster in that state is broken in worse ways.
+const RETIRE_QUARANTINE_SECS: u64 = 60 * 60;
+
+#[api()]
+#[derive(Debug, Clone, Copy, Default, Serialize, Deserialize, PartialEq, Eq)]
+#[serde(rename_all = "lowercase")]
+/// How a match compares the listed items with the present ones.
+pub enum PredicateMatch {
+ /// Fires when at least one listed item is present.
+ #[default]
+ Any,
+ /// Fires when every listed item is present.
+ All,
+ /// Fires only when the present items are exactly the listed ones.
+ Exact,
+ /// Fires on every set holding a group, nothing is listed.
+ Every,
+}
+
+/// A predicate over group names, the src or dst half of a policy.
+/// Validation refuses an empty `groups` except for `Every`, which lists nothing and never fires
+/// on the untagged identity.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct Predicate {
+ pub(crate) kind: PredicateMatch,
+ pub(crate) groups: GroupSet,
+}
+
+impl Predicate {
+ pub(crate) fn fires(&self, set: &GroupSet) -> bool {
+ match self.kind {
+ PredicateMatch::Any => self.groups.intersection(set).next().is_some(),
+ PredicateMatch::All => self.groups.is_subset(set),
+ PredicateMatch::Exact => &self.groups == set,
+ PredicateMatch::Every => set.iter().any(|group| group != UNTAGGED_GROUP),
+ }
+ }
+}
+
+/// The verdict is not part of identity, it lives in the fold.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct Policy {
+ pub(crate) src: Predicate,
+ pub(crate) dst: Predicate,
+}
+
+/// How a set fires against each policy in order, as `[p0.src, p0.dst, p1.src, ...]`. Only equality
+/// matters, sets with equal signatures get the same verdicts. Signatures compare only under one
+/// policy order.
+#[derive(Debug, Clone, PartialEq, Eq, Hash)]
+pub(crate) struct Signature(Vec<bool>);
+
+pub(crate) fn signature(set: &GroupSet, policies: &[Policy]) -> Signature {
+ let mut bits = Vec::with_capacity(policies.len() * 2);
+ for policy in policies {
+ bits.push(policy.src.fires(set));
+ bits.push(policy.dst.fires(set));
+ }
+ Signature(bits)
+}
+
+/// How much of the id space a registry uses. A retired id whose quarantine has passed is counted
+/// until the next allocation drops it.
+#[derive(Debug, Serialize)]
+pub struct IdSpace {
+ pub used: usize,
+ pub retired: usize,
+ /// Every id but the one untagged traffic carries.
+ pub total: usize,
+}
+
+/// Each live id with its representative set, plus the id counter. Only which id each class
+/// carries is kept, the classes themselves come from the policies. Serialized into the running
+/// config so every host reads the same id-to-class naming.
+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
+pub struct Registry {
+ next: u16,
+ #[serde(deserialize_with = "int_keys")]
+ classes: BTreeMap<u16, GroupSet>,
+ /// Retired id to unix time of retirement. Entries whose quarantine has passed are dropped by
+ /// the next [`allocate`].
+ #[serde(
+ default,
+ deserialize_with = "int_keys",
+ skip_serializing_if = "BTreeMap::is_empty"
+ )]
+ retired: BTreeMap<u16, u64>,
+}
+
+/// A round trip through perlmod hands integer map keys back as strings, which serde does not
+/// accept for a u16 key.
+fn int_keys<'de, D: Deserializer<'de>, V: Deserialize<'de>>(
+ deserializer: D,
+) -> Result<BTreeMap<u16, V>, D::Error> {
+ BTreeMap::<String, V>::deserialize(deserializer)?
+ .into_iter()
+ .map(|(key, value)| {
+ key.parse::<u16>()
+ .map(|key| (key, value))
+ .map_err(serde::de::Error::custom)
+ })
+ .collect()
+}
+
+impl Default for Registry {
+ fn default() -> Self {
+ Self::new()
+ }
+}
+
+impl Registry {
+ /// Ids start at 1, since 0 is the untagged id.
+ pub(crate) fn new() -> Self {
+ Self {
+ next: 1,
+ classes: BTreeMap::new(),
+ retired: BTreeMap::new(),
+ }
+ }
+
+ pub fn classes(&self) -> &BTreeMap<u16, GroupSet> {
+ &self.classes
+ }
+
+ pub fn id_space(&self) -> IdSpace {
+ IdSpace {
+ used: self.classes.len(),
+ retired: self.retired.len(),
+ total: u16::MAX as usize,
+ }
+ }
+}
+
+/// Recompute the registry from the sets the assignments name and the policies, fresh-first against
+/// `prev`.
+///
+/// Sets with the same signature form a class. Each class keeps the id of the previous class whose
+/// representative still lands in it. A merge keeps the id of the smallest representative and
+/// retires the rest. The sets no policy matches share the all-false signature and form one class
+/// with its own id, distinct from `UNTAGGED_ID`. `now` is the render time (unix seconds) the
+/// quarantine runs on.
+pub fn allocate(
+ prev: &Registry,
+ sets: &BTreeSet<GroupSet>,
+ policies: &[Policy],
+ now: u64,
+) -> anyhow::Result<Registry> {
+ // The sets arrive in ascending order, so the first set seen with a signature is the class's
+ // representative and the classes come out in representative order. That order decides which
+ // class gets which fresh number, so the same input issues the same numbers.
+ let mut class_of: HashMap<Signature, usize> = HashMap::new();
+ let mut reps: Vec<&GroupSet> = Vec::new();
+ for set in sets {
+ class_of.entry(signature(set, policies)).or_insert_with(|| {
+ reps.push(set);
+ reps.len() - 1
+ });
+ }
+
+ let mut claims: Vec<Option<(u16, &GroupSet)>> = vec![None; reps.len()];
+ for (&id, rep) in &prev.classes {
+ let Some(&class) = class_of.get(&signature(rep, policies)) else {
+ continue;
+ };
+ if claims[class].is_none_or(|(_, best)| rep < best) {
+ claims[class] = Some((id, rep));
+ }
+ }
+
+ let mut retired: BTreeMap<u16, u64> = prev
+ .retired
+ .iter()
+ .filter(|&(_, &at)| now.saturating_sub(at) < RETIRE_QUARANTINE_SECS)
+ .map(|(&id, &at)| (id, at))
+ .collect();
+
+ let mut next = prev.next;
+ let mut classes = BTreeMap::new();
+ for (rep, claim) in reps.into_iter().zip(claims) {
+ let id = match claim {
+ Some((id, _)) => id,
+ None => {
+ // the counter wrapped to 0, so the fresh space is used up
+ if next == UNTAGGED_ID {
+ reclaim(prev, &retired, &classes)?
+ } else {
+ let id = next;
+ next = next.wrapping_add(1);
+ id
+ }
+ }
+ };
+ classes.insert(id, rep.clone());
+ }
+
+ for &id in prev.classes.keys() {
+ if !classes.contains_key(&id) {
+ retired.insert(id, now);
+ }
+ }
+
+ Ok(Registry {
+ next,
+ classes,
+ retired,
+ })
+}
+
+/// Called with the fresh space used up, so a number free here was retired and has left its
+/// quarantine. An id that was live until this pass may still be on the wire, so it is kept like one
+/// in quarantine.
+fn reclaim(
+ prev: &Registry,
+ quarantined: &BTreeMap<u16, u64>,
+ taken: &BTreeMap<u16, GroupSet>,
+) -> anyhow::Result<u16> {
+ (1..=u16::MAX)
+ .find(|id| {
+ !prev.classes.contains_key(id)
+ && !quarantined.contains_key(id)
+ && !taken.contains_key(id)
+ })
+ .ok_or_else(|| {
+ anyhow::anyhow!(
+ "microseg identity id space exhausted \
+ (all 65535 ids are live or recently retired)"
+ )
+ })
+}
+
+#[cfg(test)]
+mod tests {
+ use super::PredicateMatch::*;
+ use super::*;
+
+ fn set(xs: &[&str]) -> GroupSet {
+ xs.iter().map(|x| x.to_string()).collect()
+ }
+
+ fn pred(kind: PredicateMatch, xs: &[&str]) -> Predicate {
+ Predicate {
+ kind,
+ groups: set(xs),
+ }
+ }
+
+ fn id_of(reg: &Registry, set: &GroupSet, policies: &[Policy]) -> Option<u16> {
+ let sig = signature(set, policies);
+ reg.classes
+ .iter()
+ .find(|(_, rep)| signature(rep, policies) == sig)
+ .map(|(&id, _)| id)
+ }
+
+ fn sets(xs: &[&[&str]]) -> BTreeSet<GroupSet> {
+ xs.iter().map(|s| set(s)).collect()
+ }
+
+ #[test]
+ fn unmatched_set_gets_its_own_id_not_the_untagged_id() {
+ let policies = [Policy {
+ src: pred(Any, &["untagged"]),
+ dst: pred(Any, &["b"]),
+ }];
+ let reg = allocate(&Registry::new(), &sets(&[&["e"]]), &policies, 0).unwrap();
+
+ let id = id_of(®, &set(&["e"]), &policies).expect("an unmatched set still has a class");
+ assert_ne!(
+ id, UNTAGGED_ID,
+ "an unmatched set must not alias onto the untagged id"
+ );
+
+ let reg = allocate(&Registry::new(), &sets(&[&["e"], &["f"]]), &policies, 0).unwrap();
+ assert_eq!(
+ reg.classes().len(),
+ 1,
+ "all unmatched sets collapse to one class"
+ );
+ assert_eq!(
+ id_of(®, &set(&["e"]), &policies),
+ id_of(®, &set(&["f"]), &policies)
+ );
+ }
+
+ #[test]
+ fn split_keeps_old_rep_subclass_id_and_issues_the_other() {
+ let p1 = [Policy {
+ src: pred(Any, &["b"]),
+ dst: pred(Any, &["i"]),
+ }];
+ let r1 = allocate(&Registry::new(), &sets(&[&["b", "c"], &["b", "d"]]), &p1, 0).unwrap();
+ assert_eq!(id_of(&r1, &set(&["b", "c"]), &p1), Some(1));
+ assert_eq!(id_of(&r1, &set(&["b", "d"]), &p1), Some(1));
+
+ // then EXACT{b,c} is added, which fires only on {b,c}, so the class splits
+ let p2 = [
+ Policy {
+ src: pred(Any, &["b"]),
+ dst: pred(Any, &["i"]),
+ },
+ Policy {
+ src: pred(Exact, &["b", "c"]),
+ dst: pred(Any, &["i"]),
+ },
+ ];
+ let r2 = allocate(&r1, &sets(&[&["b", "c"], &["b", "d"]]), &p2, 0).unwrap();
+
+ assert_eq!(
+ id_of(&r2, &set(&["b", "c"]), &p2),
+ Some(1),
+ "old-rep subclass keeps the id"
+ );
+ assert_eq!(
+ id_of(&r2, &set(&["b", "d"]), &p2),
+ Some(2),
+ "the other subclass issues fresh"
+ );
+ assert_eq!(r2.next, 3);
+ }
+
+ #[test]
+ fn merge_retires_the_fused_id_and_prefers_fresh() {
+ let p_split = [
+ Policy {
+ src: pred(Any, &["b"]),
+ dst: pred(Any, &["i"]),
+ },
+ Policy {
+ src: pred(Exact, &["b", "c"]),
+ dst: pred(Any, &["i"]),
+ },
+ ];
+ let r_split = allocate(
+ &Registry::new(),
+ &sets(&[&["b", "c"], &["b", "d"]]),
+ &p_split,
+ 0,
+ )
+ .unwrap();
+ assert_eq!(id_of(&r_split, &set(&["b", "c"]), &p_split), Some(1));
+ assert_eq!(id_of(&r_split, &set(&["b", "d"]), &p_split), Some(2));
+ assert_eq!(r_split.next, 3);
+
+ // Drop the distinguisher, so {b,c} and {b,d} merge.
+ let p_merge = [Policy {
+ src: pred(Any, &["b"]),
+ dst: pred(Any, &["i"]),
+ }];
+ let r_merge = allocate(&r_split, &sets(&[&["b", "c"], &["b", "d"]]), &p_merge, 0).unwrap();
+ assert_eq!(id_of(&r_merge, &set(&["b", "c"]), &p_merge), Some(1));
+ assert_eq!(
+ id_of(&r_merge, &set(&["b", "d"]), &p_merge),
+ Some(1),
+ "merged into the survivor"
+ );
+ assert!(!r_merge.classes().contains_key(&2), "id 2 is retired");
+ assert_eq!(r_merge.retired.get(&2), Some(&0), "id 2 is quarantined");
+ assert_eq!(r_merge.next, 3, "the counter never rewinds");
+
+ let r_next = allocate(
+ &r_merge,
+ &sets(&[&["b", "c"], &["b", "d"], &["i"]]),
+ &p_merge,
+ 0,
+ )
+ .unwrap();
+ assert_eq!(
+ id_of(&r_next, &set(&["i"]), &p_merge),
+ Some(3),
+ "new class skips the retired id"
+ );
+ }
+
+ fn exhausted_registry(retired_id: u16, retired_at: u64, free_id: u16) -> Registry {
+ let classes: BTreeMap<u16, GroupSet> = (1..=u16::MAX)
+ .filter(|&id| id != retired_id && id != free_id)
+ .map(|id| (id, set(&[&id.to_string()])))
+ .collect();
+ Registry {
+ next: 0,
+ classes,
+ retired: BTreeMap::from([(retired_id, retired_at)]),
+ }
+ }
+
+ #[test]
+ fn reclaims_a_retired_id_only_after_its_quarantine() {
+ // {b,e} fires both src predicates, no singleton does, so no previous class claims it and it
+ // needs an id with no fresh space left.
+ let policies = [
+ Policy {
+ src: pred(Any, &["b"]),
+ dst: pred(Any, &["i"]),
+ },
+ Policy {
+ src: pred(Any, &["e"]),
+ dst: pred(Any, &["i"]),
+ },
+ ];
+ let retired_at = 1_000;
+ let prev = exhausted_registry(7, retired_at, 11);
+
+ // one number is free outright, so the first issued id takes it regardless of the quarantine
+ let reg = allocate(&prev, &sets(&[&["b", "e"]]), &policies, retired_at + 1).unwrap();
+ assert_eq!(id_of(®, &set(&["b", "e"]), &policies), Some(11));
+
+ // with 11 also live, the only candidate is the quarantined 7. Inside the quarantine
+ // issuing must fail rather than rebind a number packets may still carry
+ let mut prev = exhausted_registry(7, retired_at, 11);
+ prev.classes.insert(11, set(&["11"]));
+ let before = retired_at + RETIRE_QUARANTINE_SECS - 1;
+ let err = allocate(&prev, &sets(&[&["b", "e"]]), &policies, before).unwrap_err();
+ assert!(err.to_string().contains("exhausted"), "got: {err:#}");
+
+ let after = retired_at + RETIRE_QUARANTINE_SECS;
+ let reg = allocate(&prev, &sets(&[&["b", "e"]]), &policies, after).unwrap();
+ assert_eq!(id_of(®, &set(&["b", "e"]), &policies), Some(7));
+ assert!(!reg.retired.contains_key(&7));
+ }
+}
diff --git a/proxmox-ve-config/src/sdn/microseg/mod.rs b/proxmox-ve-config/src/sdn/microseg/mod.rs
new file mode 100644
index 000000000000..37c4f92ff0c8
--- /dev/null
+++ b/proxmox-ve-config/src/sdn/microseg/mod.rs
@@ -0,0 +1,9 @@
+//! Microseg subsystem of the SDN config.
+//!
+//! Microseg is an identity-based stateless firewall. Each managed guest NIC is assigned a *set* of
+//! *groups*. The render allocates a wire identity for each set (see [`identity`]), and every host
+//! gets the same ids. Each host looks the identities up and compiles the policy *rules* into a
+//! `(src_id, dst_id) -> allow|deny` map. A rule matches traffic by a `src` and a `dst`
+//! *predicate*, each a match kind over a list of groups.
+
+pub mod identity;
diff --git a/proxmox-ve-config/src/sdn/mod.rs b/proxmox-ve-config/src/sdn/mod.rs
index 2133396145bd..f647e872c636 100644
--- a/proxmox-ve-config/src/sdn/mod.rs
+++ b/proxmox-ve-config/src/sdn/mod.rs
@@ -1,6 +1,7 @@
pub mod config;
pub mod fabric;
pub mod ipam;
+pub mod microseg;
pub mod prefix_list;
pub mod route_map;
pub mod wireguard;
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-ve-rs v3 06/45] ve-config: sdn: microseg: add config types
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (4 preceding siblings ...)
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 05/45] ve-config: sdn: microseg: add identity allocation for group sets Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 07/45] ve-config: sdn: microseg: add tag matcher Hannes Laimer
` (38 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
Add the microseg section config together with the operations that
change it and its render into the SDN running config. An assignment
puts one or all NICs of a guest into a set of groups, and a rule
allows or denies traffic between NICs by the groups they are in. Among
the matching rules the highest priority decides, and at the same
priority a deny wins. Traffic no rule matches is denied.
A NIC gets the groups of the first assignment in the match order that
covers it, and nothing is merged across assignments. The render
allocates an id for the set of every assignment without looking at
the guests, so the running config already holds the id of a NIC that
appears later.
Rule and assignment ids are derived from what they match. An object
deleted and created again keeps its id, and the same object cannot be
created twice. A group cannot take a name shaped like a derived id,
since all entries share one id space. The place of an assignment in
the match order is stored as a number with gaps, so moving one usually
changes only its own entry. A new assignment goes first unless a
position is given.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
proxmox-ve-config/src/sdn/microseg/api.rs | 528 ++++++++++++++
.../src/sdn/microseg/identity.rs | 19 +-
proxmox-ve-config/src/sdn/microseg/mod.rs | 669 +++++++++++++++++-
proxmox-ve-config/src/sdn/microseg/types.rs | 329 +++++++++
4 files changed, 1536 insertions(+), 9 deletions(-)
create mode 100644 proxmox-ve-config/src/sdn/microseg/api.rs
create mode 100644 proxmox-ve-config/src/sdn/microseg/types.rs
diff --git a/proxmox-ve-config/src/sdn/microseg/api.rs b/proxmox-ve-config/src/sdn/microseg/api.rs
new file mode 100644
index 000000000000..0c89ba27c21e
--- /dev/null
+++ b/proxmox-ve-config/src/sdn/microseg/api.rs
@@ -0,0 +1,528 @@
+//! The API side of the config, its parameter types, the operations that change entries and derive
+//! their ids, and the texts it shows. Parameters arrive as an untyped raw hash and deserialize into
+//! [`MicrosegCreate`] or [`MicrosegUpdate`].
+
+use std::collections::BTreeMap;
+
+use anyhow::{Error, bail};
+use proxmox_section_config::typed::ApiSectionDataEntry;
+use serde::Deserialize;
+
+use super::types::{
+ Binding, GroupSection, GuestAssignmentSection, GuestInfo, MicrosegEntry, MicrosegId,
+ MicrosegRunningConfig, RuleSection,
+};
+use super::{PredicateMatch, assigned_nics, assignments, sorted_rules, validate, validate_entry};
+
+#[derive(Debug, Clone, Deserialize)]
+#[serde(tag = "type", rename_all = "kebab-case")]
+pub enum MicrosegCreate {
+ Group(GroupSection),
+ Rule(RuleCreate),
+ GuestAssignment(GuestAssignmentCreate),
+}
+
+impl MicrosegCreate {
+ fn binding(&self) -> Option<&BindingCreate> {
+ match self {
+ Self::GuestAssignment(a) => Some(&a.binding),
+ Self::Group(_) | Self::Rule(_) => None,
+ }
+ }
+}
+
+#[derive(Debug, Clone, Deserialize)]
+pub struct RuleCreate {
+ #[serde(default)]
+ pub src: Vec<MicrosegId>,
+ #[serde(default, rename = "src-match")]
+ pub src_match: PredicateMatch,
+ #[serde(default)]
+ pub dst: Vec<MicrosegId>,
+ #[serde(default, rename = "dst-match")]
+ pub dst_match: PredicateMatch,
+ #[serde(default, deserialize_with = "proxmox_serde::perl::deserialize_bool")]
+ pub bidirectional: bool,
+ #[serde(default, deserialize_with = "proxmox_serde::perl::deserialize_u16")]
+ pub prio: u16,
+ #[serde(deserialize_with = "proxmox_serde::perl::deserialize_bool")]
+ pub allow: bool,
+ #[serde(default)]
+ pub comment: Option<String>,
+}
+
+/// What a new assignment binds, with the `position` it asks for in the match order.
+#[derive(Debug, Clone, Deserialize)]
+pub struct BindingCreate {
+ #[serde(default, deserialize_with = "proxmox_serde::perl::deserialize_u32")]
+ pub iface: Option<u32>,
+ #[serde(default)]
+ pub groups: Vec<MicrosegId>,
+ #[serde(default, deserialize_with = "proxmox_serde::perl::deserialize_u32")]
+ pub position: Option<u32>,
+ #[serde(default)]
+ pub comment: Option<String>,
+}
+
+impl From<BindingCreate> for Binding {
+ fn from(create: BindingCreate) -> Self {
+ Binding {
+ iface: create.iface,
+ groups: unique_groups(create.groups),
+ order: None,
+ comment: create.comment,
+ }
+ }
+}
+
+#[derive(Debug, Clone, Deserialize)]
+pub struct GuestAssignmentCreate {
+ #[serde(deserialize_with = "proxmox_serde::perl::deserialize_u32")]
+ pub vmid: u32,
+ #[serde(flatten)]
+ pub binding: BindingCreate,
+}
+
+#[derive(Debug, Clone, Deserialize)]
+#[serde(tag = "type", rename_all = "kebab-case")]
+pub enum MicrosegUpdate {
+ Group(GroupUpdate),
+ Rule(RuleUpdate),
+ GuestAssignment(AssignmentUpdate),
+}
+
+#[derive(Debug, Clone, Deserialize)]
+pub struct GroupUpdate {
+ #[serde(default)]
+ pub comment: Option<String>,
+ #[serde(default)]
+ pub delete: Vec<GroupDeletableProperties>,
+}
+
+#[derive(Debug, Clone, Copy, Deserialize)]
+#[serde(rename_all = "lowercase")]
+pub enum GroupDeletableProperties {
+ Comment,
+}
+
+#[derive(Debug, Clone, Deserialize)]
+pub struct RuleUpdate {
+ #[serde(default, deserialize_with = "proxmox_serde::perl::deserialize_u16")]
+ pub prio: Option<u16>,
+ #[serde(default, deserialize_with = "proxmox_serde::perl::deserialize_bool")]
+ pub allow: Option<bool>,
+ #[serde(default)]
+ pub comment: Option<String>,
+ #[serde(default)]
+ pub delete: Vec<RuleDeletableProperties>,
+}
+
+#[derive(Debug, Clone, Copy, Deserialize)]
+#[serde(rename_all = "lowercase")]
+pub enum RuleDeletableProperties {
+ Comment,
+}
+
+#[derive(Debug, Clone, Default, Deserialize)]
+pub struct AssignmentUpdate {
+ #[serde(default)]
+ pub groups: Option<Vec<MicrosegId>>,
+ #[serde(default, deserialize_with = "proxmox_serde::perl::deserialize_u32")]
+ pub position: Option<u32>,
+ #[serde(default)]
+ pub comment: Option<String>,
+ #[serde(default)]
+ pub delete: Vec<AssignmentDeletableProperties>,
+}
+
+#[derive(Debug, Clone, Copy, Deserialize)]
+#[serde(rename_all = "lowercase")]
+pub enum AssignmentDeletableProperties {
+ Comment,
+}
+
+fn unique_groups(groups: Vec<MicrosegId>) -> Vec<MicrosegId> {
+ let mut unique: Vec<MicrosegId> = Vec::with_capacity(groups.len());
+ for group in groups {
+ if !unique.contains(&group) {
+ unique.push(group);
+ }
+ }
+ unique
+}
+
+fn sorted_groups<'a>(
+ names: &'a [MicrosegId],
+ entries: &BTreeMap<String, MicrosegEntry>,
+) -> Result<Vec<&'a str>, Error> {
+ for name in names {
+ super::known_group(entries, name.as_ref())?;
+ }
+ let mut groups: Vec<&str> = names.iter().map(|name| name.as_ref()).collect();
+ groups.sort_unstable();
+ groups.dedup();
+ Ok(groups)
+}
+
+/// FNV-1a over what the rule matches. Priority, verdict and comment stay out, so they can
+/// change under the same id.
+fn rule_id(
+ src_match: PredicateMatch,
+ src_groups: &[&str],
+ dst_match: PredicateMatch,
+ dst_groups: &[&str],
+ bidirectional: bool,
+) -> Result<MicrosegId, Error> {
+ let mut hash = Fnv1a::new();
+ hash.match_kind(src_match);
+ hash.groups(src_groups);
+ hash.match_kind(dst_match);
+ hash.groups(dst_groups);
+ hash.flag(bidirectional);
+ format!("r{:016x}", hash.0).parse()
+}
+
+/// Every input is self-delimiting, so a value cannot shift into the field after it.
+struct Fnv1a(u64);
+
+impl Fnv1a {
+ fn new() -> Self {
+ Self(0xcbf2_9ce4_8422_2325)
+ }
+
+ fn byte(&mut self, byte: u8) {
+ self.0 ^= u64::from(byte);
+ self.0 = self.0.wrapping_mul(0x0000_0100_0000_01b3);
+ }
+
+ fn bytes(&mut self, bytes: &[u8]) {
+ for byte in bytes {
+ self.byte(*byte);
+ }
+ }
+
+ fn text(&mut self, text: &str) {
+ self.bytes(text.as_bytes());
+ self.byte(0);
+ }
+
+ fn flag(&mut self, flag: bool) {
+ self.byte(u8::from(flag));
+ }
+
+ /// By name rather than by discriminant, so reordering the variants keeps the ids.
+ fn match_kind(&mut self, match_kind: PredicateMatch) {
+ self.text(match_kind.as_str());
+ }
+
+ fn groups(&mut self, groups: &[&str]) {
+ self.bytes(&(groups.len() as u16).to_be_bytes());
+ for group in groups {
+ self.text(group);
+ }
+ }
+}
+
+fn build_entry(
+ payload: MicrosegCreate,
+ entries: &BTreeMap<String, MicrosegEntry>,
+) -> Result<MicrosegEntry, Error> {
+ Ok(match payload {
+ MicrosegCreate::Group(group) => MicrosegEntry::Group(group),
+ MicrosegCreate::Rule(rule) => {
+ let src_groups = sorted_groups(&rule.src, entries)?;
+ let dst_groups = sorted_groups(&rule.dst, entries)?;
+ let id = rule_id(
+ rule.src_match,
+ &src_groups,
+ rule.dst_match,
+ &dst_groups,
+ rule.bidirectional,
+ )?;
+ MicrosegEntry::Rule(RuleSection {
+ id,
+ src: unique_groups(rule.src),
+ src_match: rule.src_match,
+ dst: unique_groups(rule.dst),
+ dst_match: rule.dst_match,
+ bidirectional: rule.bidirectional,
+ prio: rule.prio,
+ allow: rule.allow,
+ comment: rule.comment,
+ })
+ }
+ MicrosegCreate::GuestAssignment(assignment) => {
+ let vmid = assignment.vmid;
+ let id = match assignment.binding.iface {
+ Some(iface) => format!("vm{vmid}i{iface}").parse()?,
+ None => format!("vm{vmid}").parse()?,
+ };
+ MicrosegEntry::GuestAssignment(GuestAssignmentSection {
+ id,
+ vmid,
+ binding: assignment.binding.into(),
+ })
+ }
+ })
+}
+
+/// The `order` of the first assignment.
+const ORDER_START: u32 = 1_000_000;
+/// How far from its neighbor a new first or last assignment goes, which leaves free
+/// numbers in front of the list and between two assignments.
+const ORDER_GAP: u32 = 1_000;
+
+/// Move assignment `id` to `position` in the match order, counted without it. Only its own
+/// `order` changes while a number is free at that place, otherwise every assignment gets a
+/// new one.
+fn place_assignment(entries: &mut BTreeMap<String, MicrosegEntry>, id: &str, position: u32) {
+ let mut others: Vec<(String, Option<u32>)> = Vec::new();
+ let mut current = None;
+ for assignment in assignments(entries) {
+ if assignment.id == id {
+ current = Some((others.len(), assignment.order));
+ } else {
+ others.push((assignment.id.to_string(), assignment.order));
+ }
+ }
+ let Some((current, order)) = current else {
+ return;
+ };
+ let position = (position as usize).min(others.len());
+ // a new assignment doesn't have an order yet, so never return early
+ if position == current && order.is_some() {
+ return;
+ }
+
+ let below = position.checked_sub(1).map(|index| others[index].1);
+ let above = others.get(position).map(|other| other.1);
+ let free = match (below, above) {
+ (None, None | Some(None)) => Some(ORDER_START),
+ (None, Some(Some(above))) => above.checked_sub(ORDER_GAP),
+ (Some(Some(below)), None | Some(None)) => below.checked_add(ORDER_GAP),
+ (Some(Some(below)), Some(Some(above))) => {
+ Some(below + (above - below) / 2).filter(|number| *number > below)
+ }
+ // nothing sorts after an assignment without a number
+ (Some(None), _) => None,
+ };
+
+ let mut set_order = |id: &str, number: u32| {
+ if let Some(binding) = entries.get_mut(id).and_then(MicrosegEntry::binding_mut) {
+ binding.order = Some(number);
+ }
+ };
+ match free {
+ Some(number) => set_order(id, number),
+ None => {
+ others.insert(position, (id.to_string(), None));
+ // the size limit of the config file keeps the list far shorter than the numbers
+ // reach
+ for (index, (id, _)) in others.iter().enumerate() {
+ set_order(id, ORDER_START + index as u32 * ORDER_GAP);
+ }
+ }
+ }
+}
+
+/// Add the entry a create payload describes and return its id. An assignment goes to the
+/// `position` asked for, the top when there is none.
+pub fn create(
+ entries: &mut BTreeMap<String, MicrosegEntry>,
+ payload: MicrosegCreate,
+) -> Result<String, Error> {
+ let position = payload
+ .binding()
+ .map(|binding| binding.position.unwrap_or(0));
+ let entry = build_entry(payload, entries)?;
+ let id = entry.id().to_string();
+
+ if let Some(existing) = entries.get(&id) {
+ if matches!(entry, MicrosegEntry::Group(_)) {
+ bail!("microseg object '{id}' already exists");
+ }
+ // a group cannot be named like a generated id, so the entry under it was generated from the
+ // same fields
+ bail!(
+ "a {} with the same identifying fields already exists as '{id}'",
+ existing.section_type()
+ );
+ }
+
+ // named by its contents, since the user has not seen the generated id yet
+ let subject = describe_referrer(&entry);
+ entries.insert(id.clone(), entry);
+ validate_entry(entries, &subject, &entries[&id])?;
+ if let Some(position) = position {
+ place_assignment(entries, &id, position);
+ }
+ validate(entries)?;
+
+ Ok(id)
+}
+
+fn apply_assignment_update(entry: &mut MicrosegEntry, update: AssignmentUpdate) -> Option<u32> {
+ let binding = entry.binding_mut()?;
+ if let Some(groups) = update.groups {
+ binding.groups = unique_groups(groups);
+ }
+ if update.comment.is_some() {
+ binding.comment = update.comment;
+ }
+ for property in update.delete {
+ match property {
+ AssignmentDeletableProperties::Comment => binding.comment = None,
+ }
+ }
+ update.position
+}
+
+/// Apply a partial update to entry `id`, which has to be of the update's type. An assignment
+/// given a `position` moves there.
+pub fn update(
+ entries: &mut BTreeMap<String, MicrosegEntry>,
+ id: &str,
+ update: MicrosegUpdate,
+) -> Result<(), Error> {
+ let Some(entry) = entries.get_mut(id) else {
+ bail!("microseg object '{id}' does not exist");
+ };
+ let position = match (entry, update) {
+ (MicrosegEntry::Group(group), MicrosegUpdate::Group(update)) => {
+ if update.comment.is_some() {
+ group.comment = update.comment;
+ }
+ for property in update.delete {
+ match property {
+ GroupDeletableProperties::Comment => group.comment = None,
+ }
+ }
+ None
+ }
+ (MicrosegEntry::Rule(rule), MicrosegUpdate::Rule(update)) => {
+ if let Some(prio) = update.prio {
+ rule.prio = prio;
+ }
+ if let Some(allow) = update.allow {
+ rule.allow = allow;
+ }
+ if update.comment.is_some() {
+ rule.comment = update.comment;
+ }
+ for property in update.delete {
+ match property {
+ RuleDeletableProperties::Comment => rule.comment = None,
+ }
+ }
+ None
+ }
+ (entry @ MicrosegEntry::GuestAssignment(_), MicrosegUpdate::GuestAssignment(update)) => {
+ apply_assignment_update(entry, update)
+ }
+ _ => bail!("update type does not match the existing object's type"),
+ };
+ if let Some(position) = position {
+ place_assignment(entries, id, position);
+ }
+ validate(entries)
+}
+
+/// Remove entry `id`. Returns Err for a group another entry still references.
+pub fn delete(entries: &mut BTreeMap<String, MicrosegEntry>, id: &str) -> Result<(), Error> {
+ let Some(entry) = entries.get(id) else {
+ bail!("microseg object '{id}' does not exist");
+ };
+
+ if matches!(entry, MicrosegEntry::Group(_)) {
+ let referrers = group_referenced_by(entries, id);
+ if !referrers.is_empty() {
+ bail!(
+ "group '{id}' is still referenced by {}",
+ referrers.join(", ")
+ );
+ }
+ }
+
+ entries.remove(id);
+ validate(entries)
+}
+
+fn predicate_str(match_kind: PredicateMatch, groups: &[impl AsRef<str>]) -> String {
+ let kind = match_kind.as_str();
+ let list = groups
+ .iter()
+ .map(|g| g.as_ref())
+ .collect::<Vec<_>>()
+ .join(", ");
+ format!("{kind}{{{list}}}")
+}
+
+fn describe_rule(rule: &RuleSection) -> String {
+ let arrow = if rule.bidirectional { "<->" } else { "->" };
+ format!(
+ "rule {} {arrow} {}",
+ predicate_str(rule.src_match, &rule.src),
+ predicate_str(rule.dst_match, &rule.dst),
+ )
+}
+
+/// A human-readable description of an entry for diagnostics, by what a user would recognize rather
+/// than by its id.
+fn describe_referrer(entry: &MicrosegEntry) -> String {
+ let (desc, comment) = match entry {
+ MicrosegEntry::Rule(rule) => (describe_rule(rule), rule.comment.as_deref()),
+ MicrosegEntry::GuestAssignment(assignment) => (
+ match assignment.binding.iface {
+ Some(iface) => format!("guest assignment vm{} net{iface}", assignment.vmid),
+ None => format!("guest assignment vm{}", assignment.vmid),
+ },
+ assignment.binding.comment.as_deref(),
+ ),
+ MicrosegEntry::Group(_) => ("object".to_string(), None),
+ };
+ match comment {
+ Some(comment) => format!("{desc} (comment: \"{comment}\")"),
+ None => desc,
+ }
+}
+
+/// The microseg state as text in a stable order, so two states can be compared as a diff.
+pub fn summary(cfg: &MicrosegRunningConfig, inventory: &[GuestInfo]) -> String {
+ let mut lines = vec!["identities:".to_string()];
+ for &id in cfg.identities().classes().keys() {
+ lines.push(format!(" {}", cfg.describe_class(id)));
+ }
+ lines.push("interfaces:".to_string());
+ for nic in assigned_nics(cfg.ids(), inventory) {
+ let mut groups: Vec<&str> = nic.groups.iter().map(|group| group.as_ref()).collect();
+ groups.sort_unstable();
+ let groups = groups.join(", ");
+ lines.push(format!(" vm{} net{} -> {groups}", nic.vmid, nic.iface));
+ }
+ lines.push("rules:".to_string());
+ for rule in sorted_rules(cfg.ids()) {
+ let verdict = if rule.allow { "allow" } else { "deny" };
+ let desc = describe_rule(rule);
+ lines.push(format!(" {desc}: {verdict} prio {}", rule.prio));
+ }
+ lines.join("\n") + "\n"
+}
+
+fn group_referenced_by(entries: &BTreeMap<String, MicrosegEntry>, name: &str) -> Vec<String> {
+ let mut referrers: Vec<String> = entries
+ .values()
+ .filter(|entry| match entry {
+ MicrosegEntry::Rule(rule) => {
+ rule.src.iter().any(|g| g.as_ref() == name)
+ || rule.dst.iter().any(|g| g.as_ref() == name)
+ }
+ MicrosegEntry::Group(_) => false,
+ _ => entry
+ .assignment_groups()
+ .is_some_and(|groups| groups.iter().any(|g| g.as_ref() == name)),
+ })
+ .map(describe_referrer)
+ .collect();
+ referrers.sort();
+ referrers
+}
diff --git a/proxmox-ve-config/src/sdn/microseg/identity.rs b/proxmox-ve-config/src/sdn/microseg/identity.rs
index 24e91402070e..eb8317076b44 100644
--- a/proxmox-ve-config/src/sdn/microseg/identity.rs
+++ b/proxmox-ve-config/src/sdn/microseg/identity.rs
@@ -56,9 +56,20 @@ pub enum PredicateMatch {
Every,
}
-/// A predicate over group names, the src or dst half of a policy.
-/// Validation refuses an empty `groups` except for `Every`, which lists nothing and never fires
-/// on the untagged identity.
+impl PredicateMatch {
+ pub(crate) fn as_str(self) -> &'static str {
+ match self {
+ Self::Any => "any",
+ Self::All => "all",
+ Self::Exact => "exact",
+ Self::Every => "every",
+ }
+ }
+}
+
+/// A predicate over group names, the src or dst half of a policy. Validation refuses an
+/// empty `groups` except for `Every`, which lists nothing and never fires on the untagged
+/// identity.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Predicate {
pub(crate) kind: PredicateMatch,
@@ -157,7 +168,7 @@ impl Registry {
}
}
- pub fn classes(&self) -> &BTreeMap<u16, GroupSet> {
+ pub(crate) fn classes(&self) -> &BTreeMap<u16, GroupSet> {
&self.classes
}
diff --git a/proxmox-ve-config/src/sdn/microseg/mod.rs b/proxmox-ve-config/src/sdn/microseg/mod.rs
index 37c4f92ff0c8..0a1b1b69225c 100644
--- a/proxmox-ve-config/src/sdn/microseg/mod.rs
+++ b/proxmox-ve-config/src/sdn/microseg/mod.rs
@@ -1,9 +1,668 @@
//! Microseg subsystem of the SDN config.
//!
-//! Microseg is an identity-based stateless firewall. Each managed guest NIC is assigned a *set* of
-//! *groups*. The render allocates a wire identity for each set (see [`identity`]), and every host
-//! gets the same ids. Each host looks the identities up and compiles the policy *rules* into a
-//! `(src_id, dst_id) -> allow|deny` map. A rule matches traffic by a `src` and a `dst`
-//! *predicate*, each a match kind over a list of groups.
+//! Microseg is an identity-based stateless firewall. Each managed guest NIC gets the *set* of
+//! *groups* of the first *assignment* in admin order that covers it. The render allocates a wire
+//! identity for the set of every assignment (see [`identity`]), and the running config carries it
+//! to every host, so a NIC can only land in a class that already exists. Each host looks the
+//! identities up and compiles the policy *rules* into a `(src_id, dst_id) -> allow|deny` map. A
+//! rule matches traffic by a `src` and a `dst` *predicate*, each a match kind over a list of
+//! groups.
+pub mod api;
pub mod identity;
+mod types;
+
+pub use identity::PredicateMatch;
+pub use types::{
+ Binding, GroupName, GroupSection, GuestAssignmentSection, GuestInfo, MicrosegEntry, MicrosegId,
+ MicrosegRunningConfig, RuleSection,
+};
+
+use std::collections::{BTreeMap, BTreeSet, HashMap};
+
+use anyhow::{Context, Error, bail};
+use identity::UNTAGGED_GROUP;
+use serde::Serialize;
+
+/// Validate a microseg config before anything renders from it, so resolving group names and
+/// expanding rules cannot fail further down.
+pub fn validate(entries: &BTreeMap<String, MicrosegEntry>) -> Result<(), Error> {
+ // the first invalid entry ends the walk, so walking by id reports the same one every run
+ for (name, entry) in entries {
+ let kind = match entry {
+ MicrosegEntry::Rule(_) => "rule",
+ MicrosegEntry::GuestAssignment(_) => "assignment",
+ MicrosegEntry::Group(_) => "group",
+ };
+ validate_entry(entries, &format!("{kind} '{name}'"), entry)?;
+ }
+
+ Ok(())
+}
+
+fn validate_entry(
+ entries: &BTreeMap<String, MicrosegEntry>,
+ subject: &str,
+ entry: &MicrosegEntry,
+) -> Result<(), Error> {
+ match entry {
+ MicrosegEntry::Rule(rule) => validate_rule(entries, subject, rule)?,
+ MicrosegEntry::GuestAssignment(_) | MicrosegEntry::Group(_) => {}
+ }
+ if let Some(groups) = entry.assignment_groups() {
+ if groups.is_empty() {
+ bail!("{subject} is not assigned to any group");
+ }
+ for group in groups {
+ if group.as_ref() == UNTAGGED_GROUP {
+ bail!("{subject} cannot use the reserved '{UNTAGGED_GROUP}' group");
+ }
+ known_group(entries, group.as_ref()).with_context(|| subject.to_string())?;
+ }
+ }
+
+ Ok(())
+}
+
+fn validate_rule(
+ entries: &BTreeMap<String, MicrosegEntry>,
+ subject: &str,
+ rule: &RuleSection,
+) -> Result<(), Error> {
+ for (side, groups, kind) in [
+ ("source", &rule.src, rule.src_match),
+ ("destination", &rule.dst, rule.dst_match),
+ ] {
+ if kind == PredicateMatch::Every && !groups.is_empty() {
+ bail!("{subject} lists groups in an 'every' {side} predicate, which takes none");
+ }
+ if kind != PredicateMatch::Every && groups.is_empty() {
+ bail!("{subject} has an empty {side} predicate");
+ }
+ }
+ let src_untagged = rule.src.iter().any(|g| g.as_ref() == UNTAGGED_GROUP);
+ let src_has_groups = rule.src.iter().any(|g| g.as_ref() != UNTAGGED_GROUP);
+ if src_untagged && src_has_groups && rule.src_match != PredicateMatch::Any {
+ bail!(
+ "{subject} names '{UNTAGGED_GROUP}' next to real groups in an 'all' or \
+ 'exact' source, it is accepted there only on its own"
+ );
+ }
+ let dst_untagged = rule.dst.iter().any(|g| g.as_ref() == UNTAGGED_GROUP);
+ let dst_has_groups = rule.dst.iter().any(|g| g.as_ref() != UNTAGGED_GROUP);
+ if dst_untagged && rule.dst_match != PredicateMatch::Any {
+ bail!(
+ "{subject} names '{UNTAGGED_GROUP}' in an 'all' or 'exact' destination, it \
+ is accepted only in an 'any' destination and ignored there"
+ );
+ }
+ if dst_untagged && !dst_has_groups {
+ bail!(
+ "{subject} names only '{UNTAGGED_GROUP}' in its destination, no enforced set \
+ carries it so the rule can never fire"
+ );
+ }
+ if rule.bidirectional && (src_untagged || dst_untagged) {
+ bail!(
+ "{subject} is bidirectional and names '{UNTAGGED_GROUP}', which does not \
+ mean the same in a source as in a destination"
+ );
+ }
+ for group in rule.src.iter().chain(&rule.dst) {
+ known_group(entries, group.as_ref()).with_context(|| subject.to_string())?;
+ }
+ Ok(())
+}
+
+fn is_group(entries: &BTreeMap<String, MicrosegEntry>, name: &str) -> bool {
+ matches!(entries.get(name), Some(MicrosegEntry::Group(_)))
+}
+
+fn known_group(entries: &BTreeMap<String, MicrosegEntry>, name: &str) -> Result<(), Error> {
+ if name != UNTAGGED_GROUP && !is_group(entries, name) {
+ bail!("unknown group '{name}'");
+ }
+ Ok(())
+}
+
+fn group_set<'a>(
+ entries: &BTreeMap<String, MicrosegEntry>,
+ groups: impl IntoIterator<Item = &'a MicrosegId>,
+) -> Result<identity::GroupSet, Error> {
+ groups
+ .into_iter()
+ .map(|group| {
+ known_group(entries, group.as_ref())?;
+ Ok(group.to_string())
+ })
+ .collect()
+}
+
+fn predicate(
+ entries: &BTreeMap<String, MicrosegEntry>,
+ match_kind: PredicateMatch,
+ groups: &[MicrosegId],
+) -> Result<identity::Predicate, Error> {
+ let set = group_set(entries, groups)?;
+ Ok(identity::Predicate {
+ kind: match_kind,
+ groups: set,
+ })
+}
+
+fn rule_entries(entries: &BTreeMap<String, MicrosegEntry>) -> impl Iterator<Item = &RuleSection> {
+ entries.values().filter_map(|entry| match entry {
+ MicrosegEntry::Rule(rule) => Some(rule),
+ _ => None,
+ })
+}
+
+fn sorted_rules(entries: &BTreeMap<String, MicrosegEntry>) -> Vec<&RuleSection> {
+ rule_entries(entries).collect()
+}
+
+/// Signatures only compare under one policy order, so the order is fixed by id. A bidirectional
+/// rule yields one policy, since its mirror would only swap the two signature bits the forward
+/// orientation already contributes.
+fn sorted_policies(
+ entries: &BTreeMap<String, MicrosegEntry>,
+) -> Result<Vec<(&RuleSection, identity::Policy)>, Error> {
+ let mut policies = Vec::new();
+ for rule in sorted_rules(entries) {
+ let context = || format!("rule '{}'", rule.id);
+ let src = predicate(entries, rule.src_match, &rule.src).with_context(context)?;
+ let dst = predicate(entries, rule.dst_match, &rule.dst).with_context(context)?;
+ policies.push((rule, identity::Policy { src, dst }));
+ }
+ Ok(policies)
+}
+
+fn build_policies(
+ entries: &BTreeMap<String, MicrosegEntry>,
+) -> Result<Vec<identity::Policy>, Error> {
+ Ok(sorted_policies(entries)?
+ .into_iter()
+ .map(|(_, policy)| policy)
+ .collect())
+}
+
+enum Matcher {
+ Guest(u32),
+}
+
+struct Assignment<'a> {
+ id: &'a str,
+ order: Option<u32>,
+ iface: Option<u32>,
+ groups: &'a [MicrosegId],
+ matcher: Matcher,
+}
+
+impl Assignment<'_> {
+ fn covers(&self, guest: &GuestInfo, iface: u32) -> bool {
+ let on_guest = match &self.matcher {
+ Matcher::Guest(vmid) => *vmid == guest.vmid,
+ };
+ on_guest && self.iface.is_none_or(|scoped| scoped == iface)
+ }
+}
+
+/// What the match order sorts by. A lower `order` comes first, an unset one after every set one,
+/// and the id breaks a tie so the walk is stable.
+fn match_order(order: Option<u32>, id: &str) -> (bool, Option<u32>, &str) {
+ (order.is_none(), order, id)
+}
+
+fn assignments(entries: &BTreeMap<String, MicrosegEntry>) -> Vec<Assignment<'_>> {
+ let mut assignments: Vec<Assignment<'_>> = entries
+ .iter()
+ .filter_map(|(id, entry)| {
+ let (binding, matcher) = match entry {
+ MicrosegEntry::GuestAssignment(a) => (&a.binding, Matcher::Guest(a.vmid)),
+ MicrosegEntry::Group(_) | MicrosegEntry::Rule(_) => return None,
+ };
+ Some(Assignment {
+ id,
+ order: binding.order,
+ iface: binding.iface,
+ groups: &binding.groups,
+ matcher,
+ })
+ })
+ .collect();
+ assignments.sort_by_key(|assignment| match_order(assignment.order, assignment.id));
+ assignments
+}
+
+/// One NIC of the inventory with the assignment that covers it first and the groups that
+/// assignment binds.
+#[derive(Debug)]
+pub struct AssignedNic<'a> {
+ pub vmid: u32,
+ pub iface: u32,
+ pub assignment: &'a str,
+ pub groups: &'a [MicrosegId],
+}
+
+/// Every NIC of the inventory some assignment covers, ascending by guest and interface.
+pub fn assigned_nics<'a>(
+ entries: &'a BTreeMap<String, MicrosegEntry>,
+ inventory: &[GuestInfo],
+) -> Vec<AssignedNic<'a>> {
+ let assignments = assignments(entries);
+ let mut nics = Vec::new();
+ for guest in inventory {
+ for &iface in &guest.nics {
+ if let Some(assignment) = assignments
+ .iter()
+ .find(|assignment| assignment.covers(guest, iface))
+ {
+ nics.push(AssignedNic {
+ vmid: guest.vmid,
+ iface,
+ assignment: assignment.id,
+ groups: assignment.groups,
+ });
+ }
+ }
+ }
+ nics.sort_unstable_by_key(|nic| (nic.vmid, nic.iface));
+ nics
+}
+
+/// A group set some assignment names and the rules that fire on it. It evaluates the predicates
+/// with the code the allocation uses, so the two cannot disagree.
+#[derive(Debug, Serialize)]
+pub struct GroupSetRules {
+ /// The groups, sorted.
+ pub groups: Vec<MicrosegId>,
+ /// Ids of the rules that fire on this group set as a source.
+ pub outbound: Vec<MicrosegId>,
+ /// Ids of the rules that fire on this group set as a destination.
+ pub inbound: Vec<MicrosegId>,
+}
+
+pub fn rules_by_group_set(
+ entries: &BTreeMap<String, MicrosegEntry>,
+) -> Result<Vec<GroupSetRules>, Error> {
+ let rules = sorted_policies(entries)?;
+
+ let classes: BTreeSet<BTreeSet<MicrosegId>> = entries
+ .values()
+ .filter_map(MicrosegEntry::assignment_groups)
+ .map(|groups| groups.iter().cloned().collect())
+ .collect();
+
+ classes
+ .into_iter()
+ .map(|class| {
+ let set = group_set(entries, class.iter())?;
+ let mut outbound = Vec::new();
+ let mut inbound = Vec::new();
+ for (rule, policy) in &rules {
+ let src = policy.src.fires(&set);
+ let dst = policy.dst.fires(&set);
+ if src || (rule.bidirectional && dst) {
+ outbound.push(rule.id.clone());
+ }
+ if dst || (rule.bidirectional && src) {
+ inbound.push(rule.id.clone());
+ }
+ }
+ Ok(GroupSetRules {
+ groups: class.into_iter().collect(),
+ outbound,
+ inbound,
+ })
+ })
+ .collect()
+}
+
+/// The distinct group sets the assignments name, the input the identity allocator partitions.
+fn assignment_group_sets(
+ entries: &BTreeMap<String, MicrosegEntry>,
+) -> Result<BTreeSet<identity::GroupSet>, Error> {
+ entries
+ .iter()
+ .filter_map(|(id, entry)| Some((id, entry.assignment_groups()?)))
+ .map(|(id, groups)| {
+ group_set(entries, groups).with_context(|| format!("assignment '{id}'"))
+ })
+ .collect()
+}
+
+/// Render the running config. Identities are allocated for the set of every assignment rather
+/// than for what the guests match right now, so a NIC that appears later finds its class in
+/// place. `prev` is the previous running config's registry, a fresh one renumbers the whole
+/// cluster. `now` is the render node's clock in unix seconds, which the quarantine of retired
+/// ids is measured against.
+pub fn render(
+ prev: &identity::Registry,
+ entries: BTreeMap<String, MicrosegEntry>,
+ now: u64,
+) -> Result<MicrosegRunningConfig, Error> {
+ validate(&entries)?;
+ let policies = build_policies(&entries)?;
+ let sets = assignment_group_sets(&entries)?;
+ let identities = identity::allocate(prev, &sets, &policies, now)?;
+ Ok(MicrosegRunningConfig {
+ ids: entries,
+ identities,
+ })
+}
+
+/// Each assignment's place in the match order, by id. The first has the lowest number.
+pub fn assignment_order(entries: &BTreeMap<String, MicrosegEntry>) -> HashMap<String, usize> {
+ assignments(entries)
+ .iter()
+ .enumerate()
+ .map(|(place, assignment)| (assignment.id.to_string(), place))
+ .collect()
+}
+
+#[derive(Debug)]
+pub struct ResolvedRule {
+ src: identity::Predicate,
+ dst: identity::Predicate,
+ prio: u16,
+ allow: bool,
+}
+
+pub fn resolved_rules(
+ entries: &BTreeMap<String, MicrosegEntry>,
+) -> Result<Vec<ResolvedRule>, Error> {
+ let mut resolved = Vec::new();
+ for rule in rule_entries(entries) {
+ let context = || format!("rule '{}'", rule.id);
+ let forward = ResolvedRule {
+ src: predicate(entries, rule.src_match, &rule.src).with_context(context)?,
+ dst: predicate(entries, rule.dst_match, &rule.dst).with_context(context)?,
+ prio: rule.prio,
+ allow: rule.allow,
+ };
+ let mirrored = rule.bidirectional.then(|| ResolvedRule {
+ src: forward.dst.clone(),
+ dst: forward.src.clone(),
+ prio: forward.prio,
+ allow: forward.allow,
+ });
+ resolved.push(forward);
+ resolved.extend(mirrored);
+ }
+ Ok(resolved)
+}
+
+/// Fold the rules that fire on one `(src_class, dst_class)` pair into its verdict. The highest
+/// priority wins, and a single deny among the rules tied there overrides the allows. Without a
+/// firing rule the pair denies.
+pub fn fold<'a>(firing: impl IntoIterator<Item = &'a ResolvedRule>) -> bool {
+ let mut best: Option<(u16, bool)> = None;
+ for rule in firing {
+ best = Some(match best {
+ Some((prio, _)) if rule.prio > prio => (rule.prio, rule.allow),
+ Some((prio, allow)) if rule.prio == prio => (prio, allow && rule.allow),
+ Some(current) => current,
+ None => (rule.prio, rule.allow),
+ });
+ }
+ best.is_some_and(|(_, allow)| allow)
+}
+
+#[cfg(test)]
+mod tests {
+
+ use super::*;
+
+ pub(super) fn firing_rules<'a>(
+ rules: &'a [ResolvedRule],
+ src_rep: &identity::GroupSet,
+ dst_rep: &identity::GroupSet,
+ ) -> Vec<&'a ResolvedRule> {
+ rules
+ .iter()
+ .filter(|rule| rule.src.fires(src_rep) && rule.dst.fires(dst_rep))
+ .collect()
+ }
+
+ fn id(s: &str) -> MicrosegId {
+ s.parse().expect("valid microseg id")
+ }
+
+ fn ids(names: &[&str]) -> Vec<MicrosegId> {
+ names.iter().map(|n| id(n)).collect()
+ }
+
+ fn group(name: &str) -> (String, MicrosegEntry) {
+ (
+ name.to_string(),
+ MicrosegEntry::Group(GroupSection {
+ id: name.parse().expect("valid group name"),
+ comment: None,
+ }),
+ )
+ }
+
+ fn rule(name: &str, src: &[&str], dst: &[&str], allow: bool) -> (String, MicrosegEntry) {
+ rule_full(
+ name,
+ src,
+ PredicateMatch::Any,
+ dst,
+ PredicateMatch::Any,
+ 0,
+ allow,
+ )
+ }
+
+ fn with_rule(
+ (id_str, entry): (String, MicrosegEntry),
+ modify: impl FnOnce(&mut RuleSection),
+ ) -> (String, MicrosegEntry) {
+ let MicrosegEntry::Rule(mut rule) = entry else {
+ unreachable!()
+ };
+ modify(&mut rule);
+ (id_str, MicrosegEntry::Rule(rule))
+ }
+
+ fn bidirectional(entry: (String, MicrosegEntry)) -> (String, MicrosegEntry) {
+ with_rule(entry, |rule| rule.bidirectional = true)
+ }
+
+ fn resolved(entries: &BTreeMap<String, MicrosegEntry>) -> Vec<ResolvedRule> {
+ resolved_rules(entries).expect("resolve")
+ }
+
+ #[test]
+ fn bidirectional_rule_mirrors_the_pair() {
+ let entries: BTreeMap<String, MicrosegEntry> = BTreeMap::from([
+ group("web"),
+ group("db"),
+ bidirectional(rule("both", &["web"], &["db"], true)),
+ ]);
+ validate(&entries).expect("valid");
+
+ let rules = resolved(&entries);
+ assert!(verdict(&rules, &set(&["web"]), &set(&["db"])));
+ assert!(
+ verdict(&rules, &set(&["db"]), &set(&["web"])),
+ "the mirror decides the reverse pair"
+ );
+
+ let one_way: BTreeMap<String, MicrosegEntry> = BTreeMap::from([
+ group("web"),
+ group("db"),
+ rule("oneway", &["web"], &["db"], true),
+ ]);
+ let rules = resolved(&one_way);
+ assert!(verdict(&rules, &set(&["web"]), &set(&["db"])));
+ assert!(!verdict(&rules, &set(&["db"]), &set(&["web"])));
+ }
+
+ fn assignment(id_str: &str, vmid: u32, iface: u32, groups: &[&str]) -> (String, MicrosegEntry) {
+ (
+ id_str.to_string(),
+ MicrosegEntry::GuestAssignment(GuestAssignmentSection {
+ id: id(id_str),
+ vmid,
+ binding: Binding {
+ iface: Some(iface),
+ groups: ids(groups),
+ order: None,
+ comment: None,
+ },
+ }),
+ )
+ }
+
+ #[test]
+ fn placing_an_assignment_leaves_the_other_numbers() {
+ let match_order =
+ |entries: &BTreeMap<String, MicrosegEntry>| -> Vec<(String, Option<u32>)> {
+ assignments(entries)
+ .iter()
+ .map(|assignment| (assignment.id.to_string(), assignment.order))
+ .collect()
+ };
+ let mut entries = BTreeMap::from([group("web")]);
+ for vmid in [100, 101, 102] {
+ let create = api::MicrosegCreate::GuestAssignment(api::GuestAssignmentCreate {
+ vmid,
+ binding: api::BindingCreate {
+ iface: None,
+ groups: ids(&["web"]),
+ position: None,
+ comment: None,
+ },
+ });
+ api::create(&mut entries, create).expect("create");
+ }
+ // a new assignment goes first
+ let created = match_order(&entries);
+ let created_ids: Vec<&str> = created.iter().map(|(id, _)| id.as_str()).collect();
+ assert_eq!(created_ids, ["vm102", "vm101", "vm100"]);
+
+ let to_the_middle = api::MicrosegUpdate::GuestAssignment(api::AssignmentUpdate {
+ position: Some(1),
+ ..Default::default()
+ });
+ api::update(&mut entries, "vm100", to_the_middle).expect("move");
+ let moved = match_order(&entries);
+ assert_eq!(moved[0], created[0]);
+ assert_eq!(moved[1].0, "vm100");
+ assert_eq!(moved[2], created[1]);
+
+ api::delete(&mut entries, "vm100").expect("delete");
+ assert_eq!(match_order(&entries), created[..2]);
+ }
+
+ #[test]
+ fn replaced_group_does_not_inherit_the_class_id() {
+ // each group gets a rule of its own. Without one, web and db would both land in the class
+ // no rule matches, and that class keeps its id
+ let first = render(
+ &identity::Registry::new(),
+ BTreeMap::from([
+ group("web"),
+ rule("r1", &["web"], &["web"], true),
+ assignment("vm100i0", 100, 0, &["web"]),
+ ]),
+ 0,
+ )
+ .expect("first render");
+ let web_id = *first
+ .identities()
+ .classes()
+ .keys()
+ .next()
+ .expect("web has a class");
+
+ let second = render(
+ first.identities(),
+ BTreeMap::from([
+ group("db"),
+ rule("r1", &["db"], &["db"], true),
+ assignment("vm100i0", 100, 0, &["db"]),
+ ]),
+ 0,
+ )
+ .expect("second render");
+ let classes = second.identities().classes();
+ let db_id = classes
+ .iter()
+ .find(|(_, rep)| **rep == set(&["db"]))
+ .map(|(&id, _)| id)
+ .expect("db has a class");
+ assert_ne!(db_id, web_id, "db does not take over the id of web");
+ assert!(!classes.contains_key(&web_id), "the id of web is not live");
+ }
+
+ fn set(names: &[&str]) -> identity::GroupSet {
+ names.iter().map(|name| name.to_string()).collect()
+ }
+
+ fn verdict(rules: &[ResolvedRule], src: &identity::GroupSet, dst: &identity::GroupSet) -> bool {
+ fold(firing_rules(rules, src, dst))
+ }
+
+ fn rule_full(
+ name: &str,
+ src: &[&str],
+ src_match: PredicateMatch,
+ dst: &[&str],
+ dst_match: PredicateMatch,
+ prio: u16,
+ allow: bool,
+ ) -> (String, MicrosegEntry) {
+ (
+ name.to_string(),
+ MicrosegEntry::Rule(RuleSection {
+ id: id(name),
+ src: ids(src),
+ src_match,
+ dst: ids(dst),
+ dst_match,
+ bidirectional: false,
+ prio,
+ allow,
+ comment: None,
+ }),
+ )
+ }
+
+ #[test]
+ fn fold_priority_overrides_and_ties_deny() {
+ use PredicateMatch::{Any, Exact};
+ let entries: BTreeMap<String, MicrosegEntry> = BTreeMap::from([
+ group("web"),
+ group("db"),
+ group("app"),
+ rule_full("broad", &["web", "app"], Any, &["db"], Any, 0, false),
+ rule_full("spec", &["web"], Exact, &["db"], Any, 10, true),
+ ]);
+ let rules = resolved(&entries);
+
+ // both fire on web -> db, the prio-10 allow wins over the prio-0 deny
+ assert!(verdict(&rules, &set(&["web"]), &set(&["db"])));
+ // only the broad deny fires on app -> db
+ assert!(!verdict(&rules, &set(&["app"]), &set(&["db"])));
+ // no rule fires on web -> web, so the default deny holds
+ assert!(!verdict(&rules, &set(&["web"]), &set(&["web"])));
+
+ // A deny overrides an allow at the same priority.
+ let tied: BTreeMap<String, MicrosegEntry> = BTreeMap::from([
+ group("web"),
+ group("db"),
+ group("app"),
+ rule_full("ra", &["web"], Any, &["db"], Any, 5, true),
+ rule_full("rb", &["app"], Any, &["db"], Any, 5, false),
+ ]);
+ let tied_rules = resolved(&tied);
+ assert!(
+ !verdict(&tied_rules, &set(&["web", "app"]), &set(&["db"])),
+ "allow and deny tied at the top priority deny"
+ );
+ // from an interface in web alone only the allow fires
+ assert!(verdict(&tied_rules, &set(&["web"]), &set(&["db"])));
+ }
+}
diff --git a/proxmox-ve-config/src/sdn/microseg/types.rs b/proxmox-ve-config/src/sdn/microseg/types.rs
new file mode 100644
index 000000000000..5e20ee6ca2df
--- /dev/null
+++ b/proxmox-ve-config/src/sdn/microseg/types.rs
@@ -0,0 +1,329 @@
+//! The section types of the config and the running config.
+
+use std::collections::BTreeMap;
+
+use anyhow::{Error, bail};
+use const_format::concatcp;
+use serde::{Deserialize, Serialize};
+
+use proxmox_schema::{ApiStringFormat, ApiType, api, api_string_type, const_regex};
+
+use super::PredicateMatch;
+use super::identity;
+use super::identity::UNTAGGED_GROUP;
+
+const MICROSEG_ID_REGEX_STR: &str = r"(?:[a-zA-Z0-9](?:[a-zA-Z0-9\-_]){0,30}[a-zA-Z0-9])";
+
+const_regex! {
+ pub MICROSEG_ID_REGEX = concatcp!(r"^", MICROSEG_ID_REGEX_STR, r"$");
+}
+
+pub const MICROSEG_ID_FORMAT: ApiStringFormat = ApiStringFormat::Pattern(&MICROSEG_ID_REGEX);
+
+api_string_type! {
+ /// Identifier of a microseg object, used as the section id and to reference groups.
+ #[api(format: &MICROSEG_ID_FORMAT)]
+ #[derive(Debug, Deserialize, Serialize, Clone, Hash, PartialEq, Eq, PartialOrd, Ord)]
+ pub struct MicrosegId(String);
+}
+
+const GENERATED_ID_REGEX_STR: &str = r"(?:vm[0-9]+(?:i[0-9]+)?|r[0-9a-f]{16})";
+
+const_regex! {
+ GENERATED_ID_REGEX = concatcp!(r"^", GENERATED_ID_REGEX_STR, r"$");
+}
+
+// sections are keyed by their id alone, so a group named like a generated id would take the id of
+// the rule or assignment that derives it
+fn verify_group_name(name: &str) -> Result<(), Error> {
+ MicrosegId::API_SCHEMA
+ .unwrap_string_schema()
+ .check_constraints(name)?;
+ if name == UNTAGGED_GROUP || GENERATED_ID_REGEX.is_match(name) {
+ bail!("'{name}' is a reserved group name");
+ }
+ Ok(())
+}
+
+pub const MICROSEG_GROUP_NAME_FORMAT: ApiStringFormat =
+ ApiStringFormat::VerifyFn(verify_group_name);
+
+api_string_type! {
+ /// Name of a microseg group.
+ #[api(format: &MICROSEG_GROUP_NAME_FORMAT)]
+ #[derive(Debug, Deserialize, Serialize, Clone, Hash, PartialEq, Eq, PartialOrd, Ord)]
+ #[serde(try_from = "String")]
+ pub struct GroupName(String);
+}
+
+#[api(
+ properties: {
+ comment: {
+ type: String,
+ optional: true,
+ max_length: 256,
+ description: "Free-form comment.",
+ },
+ },
+)]
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
+/// A microseg group. An interface is assigned a set of groups, which resolves to a wire identity.
+pub struct GroupSection {
+ pub(crate) id: GroupName,
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub(crate) comment: Option<String>,
+}
+
+#[api(
+ properties: {
+ src: {
+ type: Array,
+ items: { type: MicrosegId },
+ optional: true,
+ },
+ "src-match": {
+ type: PredicateMatch,
+ optional: true,
+ },
+ dst: {
+ type: Array,
+ items: { type: MicrosegId },
+ optional: true,
+ },
+ "dst-match": {
+ type: PredicateMatch,
+ optional: true,
+ },
+ bidirectional: {
+ type: Boolean,
+ optional: true,
+ },
+ prio: {
+ type: Integer,
+ optional: true,
+ minimum: 0,
+ maximum: 65535,
+ },
+ comment: {
+ type: String,
+ optional: true,
+ max_length: 256,
+ description: "Free-form comment.",
+ },
+ },
+)]
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
+/// A policy rule. It fires on traffic from an interface whose group set matches `(src_match, src)`
+/// to one whose set matches `(dst_match, dst)`, and decides it with `allow`. A rule ahead of it in
+/// the evaluation order decides first. Traffic no rule decides is denied.
+pub struct RuleSection {
+ pub(crate) id: MicrosegId,
+ /// Groups the source predicate lists.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ pub(crate) src: Vec<MicrosegId>,
+ /// How the source predicate matches its groups.
+ #[serde(default, rename = "src-match")]
+ pub(crate) src_match: PredicateMatch,
+ /// Groups the destination predicate lists.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ pub(crate) dst: Vec<MicrosegId>,
+ /// How the destination predicate matches its groups.
+ #[serde(default, rename = "dst-match")]
+ pub(crate) dst_match: PredicateMatch,
+ /// Whether the rule also decides the reverse direction, with the predicates swapped.
+ #[serde(
+ default,
+ deserialize_with = "proxmox_serde::perl::deserialize_bool",
+ skip_serializing_if = "is_false"
+ )]
+ pub(crate) bidirectional: bool,
+ /// Rules of a higher priority win.
+ #[serde(default, deserialize_with = "proxmox_serde::perl::deserialize_u16")]
+ pub(crate) prio: u16,
+ /// Whether traffic matching both predicates is allowed.
+ #[serde(deserialize_with = "proxmox_serde::perl::deserialize_bool")]
+ pub(crate) allow: bool,
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub(crate) comment: Option<String>,
+}
+
+fn is_false(value: &bool) -> bool {
+ !*value
+}
+
+#[api(
+ properties: {
+ iface: {
+ type: Integer,
+ optional: true,
+ minimum: 0,
+ maximum: 31,
+ description: "The guest's netN interface. If unset, all NICs of each matching guest.",
+ },
+ groups: {
+ type: Array,
+ description: "Groups the matching NICs belong to.",
+ items: { type: MicrosegId },
+ },
+ order: {
+ type: Integer,
+ optional: true,
+ minimum: 0,
+ description: "Place in the match order, the lowest number first. A NIC belongs to the first assignment that covers it.",
+ },
+ comment: {
+ type: String,
+ optional: true,
+ max_length: 256,
+ description: "Free-form comment.",
+ },
+ },
+)]
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
+/// What an assignment binds, apart from the matcher that picks its guests.
+pub struct Binding {
+ #[serde(
+ default,
+ deserialize_with = "proxmox_serde::perl::deserialize_u32",
+ skip_serializing_if = "Option::is_none"
+ )]
+ pub(crate) iface: Option<u32>,
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ pub(crate) groups: Vec<MicrosegId>,
+ #[serde(
+ default,
+ deserialize_with = "proxmox_serde::perl::deserialize_u32",
+ skip_serializing_if = "Option::is_none"
+ )]
+ pub(crate) order: Option<u32>,
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub(crate) comment: Option<String>,
+}
+
+#[api(
+ properties: {
+ vmid: {
+ type: Integer,
+ minimum: 1,
+ maximum: 999999999,
+ description: "Guest (VM or CT) id whose NIC(s) are bound.",
+ },
+ binding: { flatten: true, type: Binding },
+ },
+)]
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
+/// Binds a specific guest's NIC(s) to a set of groups. Stored as the `guest-assignment` section
+/// type.
+pub struct GuestAssignmentSection {
+ pub(crate) id: MicrosegId,
+ #[serde(deserialize_with = "proxmox_serde::perl::deserialize_u32")]
+ pub(crate) vmid: u32,
+ #[serde(flatten)]
+ pub(crate) binding: Binding,
+}
+
+#[api(
+ "id-property": "id",
+ "id-schema": {
+ type: String,
+ description: "Microseg object identifier.",
+ format: &MICROSEG_ID_FORMAT,
+ },
+ "type-key": "type",
+)]
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
+#[serde(rename_all = "kebab-case", tag = "type")]
+/// One entry in the microseg section config.
+pub enum MicrosegEntry {
+ /// A group.
+ Group(GroupSection),
+ /// A rule.
+ Rule(RuleSection),
+ /// An assignment by guest id.
+ GuestAssignment(GuestAssignmentSection),
+}
+
+impl MicrosegEntry {
+ /// The section id this entry is keyed by.
+ pub fn id(&self) -> &str {
+ match self {
+ Self::Group(g) => &g.id,
+ Self::Rule(r) => &r.id,
+ Self::GuestAssignment(a) => &a.id,
+ }
+ }
+
+ pub(super) fn binding(&self) -> Option<&Binding> {
+ match self {
+ Self::GuestAssignment(a) => Some(&a.binding),
+ Self::Group(_) | Self::Rule(_) => None,
+ }
+ }
+
+ pub(super) fn binding_mut(&mut self) -> Option<&mut Binding> {
+ match self {
+ Self::GuestAssignment(a) => Some(&mut a.binding),
+ Self::Group(_) | Self::Rule(_) => None,
+ }
+ }
+
+ pub(super) fn assignment_groups(&self) -> Option<&[MicrosegId]> {
+ self.binding().map(|binding| binding.groups.as_slice())
+ }
+}
+
+/// A Perl caller hands over untyped scalars, so an index may come as a string as well as a number.
+fn deserialize_perl_nics<'de, D>(deserializer: D) -> Result<Vec<u32>, D::Error>
+where
+ D: serde::Deserializer<'de>,
+{
+ #[derive(Deserialize)]
+ struct Nic(#[serde(deserialize_with = "proxmox_serde::perl::deserialize_u32")] u32);
+ Ok(Vec::<Nic>::deserialize(deserializer)?
+ .into_iter()
+ .map(|Nic(index)| index)
+ .collect())
+}
+
+/// One guest of the inventory the assignments are resolved against.
+#[derive(Clone, Debug, Default, Deserialize)]
+pub struct GuestInfo {
+ #[serde(deserialize_with = "proxmox_serde::perl::deserialize_u32")]
+ pub vmid: u32,
+ /// The present netN interface indices.
+ #[serde(default, deserialize_with = "deserialize_perl_nics")]
+ pub nics: Vec<u32>,
+}
+
+/// The microseg block of the SDN running config (`running.microseg`), the entries keyed by section
+/// id plus what the render derives from them.
+#[derive(Clone, Debug, Default, Serialize, Deserialize, PartialEq, Eq)]
+pub struct MicrosegRunningConfig {
+ #[serde(default)]
+ pub(super) ids: BTreeMap<String, MicrosegEntry>,
+ #[serde(default)]
+ pub(super) identities: identity::Registry,
+}
+
+impl MicrosegRunningConfig {
+ pub fn ids(&self) -> &BTreeMap<String, MicrosegEntry> {
+ &self.ids
+ }
+
+ pub fn identities(&self) -> &identity::Registry {
+ &self.identities
+ }
+
+ pub fn describe_class(&self, id: u16) -> String {
+ if id == identity::UNTAGGED_ID {
+ return format!("{id} ({UNTAGGED_GROUP})");
+ }
+ match self.identities.classes().get(&id) {
+ Some(rep) => {
+ let groups: Vec<&str> = rep.iter().map(String::as_str).collect();
+ format!("{id} ({})", groups.join(", "))
+ }
+ None => id.to_string(),
+ }
+ }
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-ve-rs v3 07/45] ve-config: sdn: microseg: add tag matcher
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (5 preceding siblings ...)
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 06/45] ve-config: sdn: microseg: add config types Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 08/45] ve-config: sdn: microseg: add name regex matcher Hannes Laimer
` (37 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
Bind the NICs of every guest matching a set of tags to a set of
groups. The tags are read from the guest inventory whenever NICs are
resolved, so a tag change needs no new render of the running config.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
proxmox-ve-config/src/sdn/microseg/api.rs | 70 ++++++++++++++++++++-
proxmox-ve-config/src/sdn/microseg/mod.rs | 36 +++++++++--
proxmox-ve-config/src/sdn/microseg/types.rs | 51 ++++++++++++++-
3 files changed, 150 insertions(+), 7 deletions(-)
diff --git a/proxmox-ve-config/src/sdn/microseg/api.rs b/proxmox-ve-config/src/sdn/microseg/api.rs
index 0c89ba27c21e..4f47c1a3ed80 100644
--- a/proxmox-ve-config/src/sdn/microseg/api.rs
+++ b/proxmox-ve-config/src/sdn/microseg/api.rs
@@ -10,7 +10,7 @@ use serde::Deserialize;
use super::types::{
Binding, GroupSection, GuestAssignmentSection, GuestInfo, MicrosegEntry, MicrosegId,
- MicrosegRunningConfig, RuleSection,
+ MicrosegRunningConfig, RuleSection, Tag, TagAssignmentSection,
};
use super::{PredicateMatch, assigned_nics, assignments, sorted_rules, validate, validate_entry};
@@ -20,12 +20,14 @@ pub enum MicrosegCreate {
Group(GroupSection),
Rule(RuleCreate),
GuestAssignment(GuestAssignmentCreate),
+ TagAssignment(TagAssignmentCreate),
}
impl MicrosegCreate {
fn binding(&self) -> Option<&BindingCreate> {
match self {
Self::GuestAssignment(a) => Some(&a.binding),
+ Self::TagAssignment(a) => Some(&a.binding),
Self::Group(_) | Self::Rule(_) => None,
}
}
@@ -83,12 +85,23 @@ pub struct GuestAssignmentCreate {
pub binding: BindingCreate,
}
+#[derive(Debug, Clone, Deserialize)]
+pub struct TagAssignmentCreate {
+ #[serde(default)]
+ pub tags: Vec<Tag>,
+ #[serde(default, rename = "tag-match")]
+ pub tag_match: PredicateMatch,
+ #[serde(flatten)]
+ pub binding: BindingCreate,
+}
+
#[derive(Debug, Clone, Deserialize)]
#[serde(tag = "type", rename_all = "kebab-case")]
pub enum MicrosegUpdate {
Group(GroupUpdate),
Rule(RuleUpdate),
GuestAssignment(AssignmentUpdate),
+ TagAssignment(AssignmentUpdate),
}
#[derive(Debug, Clone, Deserialize)]
@@ -221,6 +234,32 @@ impl Fnv1a {
self.text(group);
}
}
+
+ fn iface(&mut self, iface: Option<u32>) {
+ match iface {
+ Some(iface) => {
+ self.byte(1);
+ self.bytes(&iface.to_be_bytes());
+ }
+ None => self.byte(0),
+ }
+ }
+}
+
+/// FNV-1a over the matcher and the interface, so the bound groups can change under the same id.
+fn tag_assignment_id(
+ tags: &[Tag],
+ tag_match: PredicateMatch,
+ iface: Option<u32>,
+) -> Result<MicrosegId, Error> {
+ let mut sorted: Vec<&str> = tags.iter().map(AsRef::as_ref).collect();
+ sorted.sort_unstable();
+ sorted.dedup();
+ let mut hash = Fnv1a::new();
+ hash.match_kind(tag_match);
+ hash.iface(iface);
+ hash.groups(&sorted);
+ format!("tag{:016x}", hash.0).parse()
}
fn build_entry(
@@ -263,6 +302,19 @@ fn build_entry(
binding: assignment.binding.into(),
})
}
+ MicrosegCreate::TagAssignment(assignment) => {
+ let id = tag_assignment_id(
+ &assignment.tags,
+ assignment.tag_match,
+ assignment.binding.iface,
+ )?;
+ MicrosegEntry::TagAssignment(TagAssignmentSection {
+ id,
+ tags: assignment.tags,
+ tag_match: assignment.tag_match,
+ binding: assignment.binding.into(),
+ })
+ }
})
}
@@ -416,7 +468,8 @@ pub fn update(
}
None
}
- (entry @ MicrosegEntry::GuestAssignment(_), MicrosegUpdate::GuestAssignment(update)) => {
+ (entry @ MicrosegEntry::GuestAssignment(_), MicrosegUpdate::GuestAssignment(update))
+ | (entry @ MicrosegEntry::TagAssignment(_), MicrosegUpdate::TagAssignment(update)) => {
apply_assignment_update(entry, update)
}
_ => bail!("update type does not match the existing object's type"),
@@ -469,6 +522,11 @@ fn describe_rule(rule: &RuleSection) -> String {
/// A human-readable description of an entry for diagnostics, by what a user would recognize rather
/// than by its id.
fn describe_referrer(entry: &MicrosegEntry) -> String {
+ let on_net = |iface: Option<u32>| {
+ iface
+ .map(|iface| format!(" on net{iface}"))
+ .unwrap_or_default()
+ };
let (desc, comment) = match entry {
MicrosegEntry::Rule(rule) => (describe_rule(rule), rule.comment.as_deref()),
MicrosegEntry::GuestAssignment(assignment) => (
@@ -478,6 +536,14 @@ fn describe_referrer(entry: &MicrosegEntry) -> String {
},
assignment.binding.comment.as_deref(),
),
+ MicrosegEntry::TagAssignment(assignment) => (
+ format!(
+ "tag assignment {}{}",
+ predicate_str(assignment.tag_match, &assignment.tags),
+ on_net(assignment.binding.iface),
+ ),
+ assignment.binding.comment.as_deref(),
+ ),
MicrosegEntry::Group(_) => ("object".to_string(), None),
};
match comment {
diff --git a/proxmox-ve-config/src/sdn/microseg/mod.rs b/proxmox-ve-config/src/sdn/microseg/mod.rs
index 0a1b1b69225c..0688d4e58180 100644
--- a/proxmox-ve-config/src/sdn/microseg/mod.rs
+++ b/proxmox-ve-config/src/sdn/microseg/mod.rs
@@ -15,7 +15,7 @@ mod types;
pub use identity::PredicateMatch;
pub use types::{
Binding, GroupName, GroupSection, GuestAssignmentSection, GuestInfo, MicrosegEntry, MicrosegId,
- MicrosegRunningConfig, RuleSection,
+ MicrosegRunningConfig, RuleSection, Tag, TagAssignmentSection,
};
use std::collections::{BTreeMap, BTreeSet, HashMap};
@@ -31,7 +31,7 @@ pub fn validate(entries: &BTreeMap<String, MicrosegEntry>) -> Result<(), Error>
for (name, entry) in entries {
let kind = match entry {
MicrosegEntry::Rule(_) => "rule",
- MicrosegEntry::GuestAssignment(_) => "assignment",
+ MicrosegEntry::GuestAssignment(_) | MicrosegEntry::TagAssignment(_) => "assignment",
MicrosegEntry::Group(_) => "group",
};
validate_entry(entries, &format!("{kind} '{name}'"), entry)?;
@@ -47,6 +47,14 @@ fn validate_entry(
) -> Result<(), Error> {
match entry {
MicrosegEntry::Rule(rule) => validate_rule(entries, subject, rule)?,
+ MicrosegEntry::TagAssignment(assignment) => {
+ if assignment.tags.is_empty() {
+ bail!("{subject} has an empty tag matcher");
+ }
+ if assignment.tag_match == PredicateMatch::Every {
+ bail!("{subject} matches tags with 'every', which only a rule predicate takes");
+ }
+ }
MicrosegEntry::GuestAssignment(_) | MicrosegEntry::Group(_) => {}
}
if let Some(groups) = entry.assignment_groups() {
@@ -186,8 +194,24 @@ fn build_policies(
.collect())
}
-enum Matcher {
+fn tags_fire(tags: &[Tag], tag_match: PredicateMatch, guest: &GuestInfo) -> bool {
+ let has = |tag: &Tag| guest.tags.contains(tag);
+ match tag_match {
+ PredicateMatch::Any => tags.iter().any(has),
+ PredicateMatch::All => tags.iter().all(has),
+ PredicateMatch::Exact => {
+ let want: BTreeSet<&Tag> = tags.iter().collect();
+ let have: BTreeSet<&Tag> = guest.tags.iter().collect();
+ want == have
+ }
+ // validation rejects it on an assignment
+ PredicateMatch::Every => false,
+ }
+}
+
+enum Matcher<'a> {
Guest(u32),
+ Tags(&'a [Tag], PredicateMatch),
}
struct Assignment<'a> {
@@ -195,13 +219,14 @@ struct Assignment<'a> {
order: Option<u32>,
iface: Option<u32>,
groups: &'a [MicrosegId],
- matcher: Matcher,
+ matcher: Matcher<'a>,
}
impl Assignment<'_> {
fn covers(&self, guest: &GuestInfo, iface: u32) -> bool {
let on_guest = match &self.matcher {
Matcher::Guest(vmid) => *vmid == guest.vmid,
+ Matcher::Tags(tags, tag_match) => tags_fire(tags, *tag_match, guest),
};
on_guest && self.iface.is_none_or(|scoped| scoped == iface)
}
@@ -219,6 +244,9 @@ fn assignments(entries: &BTreeMap<String, MicrosegEntry>) -> Vec<Assignment<'_>>
.filter_map(|(id, entry)| {
let (binding, matcher) = match entry {
MicrosegEntry::GuestAssignment(a) => (&a.binding, Matcher::Guest(a.vmid)),
+ MicrosegEntry::TagAssignment(a) => {
+ (&a.binding, Matcher::Tags(&a.tags, a.tag_match))
+ }
MicrosegEntry::Group(_) | MicrosegEntry::Rule(_) => return None,
};
Some(Assignment {
diff --git a/proxmox-ve-config/src/sdn/microseg/types.rs b/proxmox-ve-config/src/sdn/microseg/types.rs
index 5e20ee6ca2df..cf6886267e95 100644
--- a/proxmox-ve-config/src/sdn/microseg/types.rs
+++ b/proxmox-ve-config/src/sdn/microseg/types.rs
@@ -27,7 +27,7 @@ api_string_type! {
pub struct MicrosegId(String);
}
-const GENERATED_ID_REGEX_STR: &str = r"(?:vm[0-9]+(?:i[0-9]+)?|r[0-9a-f]{16})";
+const GENERATED_ID_REGEX_STR: &str = r"(?:vm[0-9]+(?:i[0-9]+)?|(?:r|tag)[0-9a-f]{16})";
const_regex! {
GENERATED_ID_REGEX = concatcp!(r"^", GENERATED_ID_REGEX_STR, r"$");
@@ -56,6 +56,21 @@ api_string_type! {
pub struct GroupName(String);
}
+const MICROSEG_TAG_REGEX_STR: &str = r"[a-zA-Z0-9_][a-zA-Z0-9_+.\-]*";
+
+const_regex! {
+ pub MICROSEG_TAG_REGEX = concatcp!(r"^", MICROSEG_TAG_REGEX_STR, r"$");
+}
+
+pub const MICROSEG_TAG_FORMAT: ApiStringFormat = ApiStringFormat::Pattern(&MICROSEG_TAG_REGEX);
+
+api_string_type! {
+ /// A guest tag a tag assignment matches on.
+ #[api(format: &MICROSEG_TAG_FORMAT)]
+ #[derive(Debug, Deserialize, Serialize, Clone, Hash, PartialEq, Eq, PartialOrd, Ord)]
+ pub struct Tag(String);
+}
+
#[api(
properties: {
comment: {
@@ -222,6 +237,33 @@ pub struct GuestAssignmentSection {
pub(crate) binding: Binding,
}
+#[api(
+ properties: {
+ tags: {
+ type: Array,
+ description: "Guest tags to match on, compared exactly as the guest carries them.",
+ items: { type: Tag },
+ },
+ "tag-match": {
+ type: PredicateMatch,
+ optional: true,
+ },
+ binding: { flatten: true, type: Binding },
+ },
+)]
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
+/// Binds the NICs of every guest matching `tags` (by `tag_match`) to a set of groups. Stored as the
+/// `tag-assignment` section type.
+pub struct TagAssignmentSection {
+ pub(crate) id: MicrosegId,
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ pub(crate) tags: Vec<Tag>,
+ #[serde(default, rename = "tag-match")]
+ pub(crate) tag_match: PredicateMatch,
+ #[serde(flatten)]
+ pub(crate) binding: Binding,
+}
+
#[api(
"id-property": "id",
"id-schema": {
@@ -241,6 +283,8 @@ pub enum MicrosegEntry {
Rule(RuleSection),
/// An assignment by guest id.
GuestAssignment(GuestAssignmentSection),
+ /// An assignment by guest tags.
+ TagAssignment(TagAssignmentSection),
}
impl MicrosegEntry {
@@ -250,12 +294,14 @@ impl MicrosegEntry {
Self::Group(g) => &g.id,
Self::Rule(r) => &r.id,
Self::GuestAssignment(a) => &a.id,
+ Self::TagAssignment(a) => &a.id,
}
}
pub(super) fn binding(&self) -> Option<&Binding> {
match self {
Self::GuestAssignment(a) => Some(&a.binding),
+ Self::TagAssignment(a) => Some(&a.binding),
Self::Group(_) | Self::Rule(_) => None,
}
}
@@ -263,6 +309,7 @@ impl MicrosegEntry {
pub(super) fn binding_mut(&mut self) -> Option<&mut Binding> {
match self {
Self::GuestAssignment(a) => Some(&mut a.binding),
+ Self::TagAssignment(a) => Some(&mut a.binding),
Self::Group(_) | Self::Rule(_) => None,
}
}
@@ -290,6 +337,8 @@ where
pub struct GuestInfo {
#[serde(deserialize_with = "proxmox_serde::perl::deserialize_u32")]
pub vmid: u32,
+ #[serde(default)]
+ pub tags: Vec<Tag>,
/// The present netN interface indices.
#[serde(default, deserialize_with = "deserialize_perl_nics")]
pub nics: Vec<u32>,
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-ve-rs v3 08/45] ve-config: sdn: microseg: add name regex matcher
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (6 preceding siblings ...)
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 07/45] ve-config: sdn: microseg: add tag matcher Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 09/45] ve-config: sdn: microseg: narrow rules by protocol and ports Hannes Laimer
` (36 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
Assign groups to the NICs of every guest whose name matches a regular
expression. Like guest tags, the names come from the guest inventory
and never enter the running config. A rename takes effect the next
time the NICs are resolved. The expression has to be ASCII, like the
names it is matched against.
A query tells whether renaming a guest changes which expressions match
its name. It uses the same regex engine as the matcher, so a caller
does not have to match the expressions in a dialect of its own.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
proxmox-ve-config/src/sdn/microseg/api.rs | 42 +++++++++++++++-
proxmox-ve-config/src/sdn/microseg/mod.rs | 53 ++++++++++++++++++++-
proxmox-ve-config/src/sdn/microseg/types.rs | 30 +++++++++++-
3 files changed, 120 insertions(+), 5 deletions(-)
diff --git a/proxmox-ve-config/src/sdn/microseg/api.rs b/proxmox-ve-config/src/sdn/microseg/api.rs
index 4f47c1a3ed80..08b5ee1f4653 100644
--- a/proxmox-ve-config/src/sdn/microseg/api.rs
+++ b/proxmox-ve-config/src/sdn/microseg/api.rs
@@ -10,7 +10,7 @@ use serde::Deserialize;
use super::types::{
Binding, GroupSection, GuestAssignmentSection, GuestInfo, MicrosegEntry, MicrosegId,
- MicrosegRunningConfig, RuleSection, Tag, TagAssignmentSection,
+ MicrosegRunningConfig, RegexAssignmentSection, RuleSection, Tag, TagAssignmentSection,
};
use super::{PredicateMatch, assigned_nics, assignments, sorted_rules, validate, validate_entry};
@@ -21,6 +21,7 @@ pub enum MicrosegCreate {
Rule(RuleCreate),
GuestAssignment(GuestAssignmentCreate),
TagAssignment(TagAssignmentCreate),
+ RegexAssignment(RegexAssignmentCreate),
}
impl MicrosegCreate {
@@ -28,6 +29,7 @@ impl MicrosegCreate {
match self {
Self::GuestAssignment(a) => Some(&a.binding),
Self::TagAssignment(a) => Some(&a.binding),
+ Self::RegexAssignment(a) => Some(&a.binding),
Self::Group(_) | Self::Rule(_) => None,
}
}
@@ -95,6 +97,14 @@ pub struct TagAssignmentCreate {
pub binding: BindingCreate,
}
+#[derive(Debug, Clone, Deserialize)]
+pub struct RegexAssignmentCreate {
+ #[serde(default, rename = "name-regex")]
+ pub name_regex: String,
+ #[serde(flatten)]
+ pub binding: BindingCreate,
+}
+
#[derive(Debug, Clone, Deserialize)]
#[serde(tag = "type", rename_all = "kebab-case")]
pub enum MicrosegUpdate {
@@ -102,6 +112,7 @@ pub enum MicrosegUpdate {
Rule(RuleUpdate),
GuestAssignment(AssignmentUpdate),
TagAssignment(AssignmentUpdate),
+ RegexAssignment(AssignmentUpdate),
}
#[derive(Debug, Clone, Deserialize)]
@@ -262,6 +273,13 @@ fn tag_assignment_id(
format!("tag{:016x}", hash.0).parse()
}
+fn regex_assignment_id(name_regex: &str, iface: Option<u32>) -> Result<MicrosegId, Error> {
+ let mut hash = Fnv1a::new();
+ hash.iface(iface);
+ hash.text(name_regex);
+ format!("re{:016x}", hash.0).parse()
+}
+
fn build_entry(
payload: MicrosegCreate,
entries: &BTreeMap<String, MicrosegEntry>,
@@ -315,6 +333,17 @@ fn build_entry(
binding: assignment.binding.into(),
})
}
+ MicrosegCreate::RegexAssignment(assignment) => {
+ // section-config trims every property value on read, so an untrimmed regex would
+ // come back under an id derived from text the entry no longer holds
+ let name_regex = assignment.name_regex.trim().to_string();
+ let id = regex_assignment_id(&name_regex, assignment.binding.iface)?;
+ MicrosegEntry::RegexAssignment(RegexAssignmentSection {
+ id,
+ name_regex,
+ binding: assignment.binding.into(),
+ })
+ }
})
}
@@ -469,7 +498,8 @@ pub fn update(
None
}
(entry @ MicrosegEntry::GuestAssignment(_), MicrosegUpdate::GuestAssignment(update))
- | (entry @ MicrosegEntry::TagAssignment(_), MicrosegUpdate::TagAssignment(update)) => {
+ | (entry @ MicrosegEntry::TagAssignment(_), MicrosegUpdate::TagAssignment(update))
+ | (entry @ MicrosegEntry::RegexAssignment(_), MicrosegUpdate::RegexAssignment(update)) => {
apply_assignment_update(entry, update)
}
_ => bail!("update type does not match the existing object's type"),
@@ -544,6 +574,14 @@ fn describe_referrer(entry: &MicrosegEntry) -> String {
),
assignment.binding.comment.as_deref(),
),
+ MicrosegEntry::RegexAssignment(assignment) => (
+ format!(
+ "regex assignment /{}/{}",
+ assignment.name_regex,
+ on_net(assignment.binding.iface),
+ ),
+ assignment.binding.comment.as_deref(),
+ ),
MicrosegEntry::Group(_) => ("object".to_string(), None),
};
match comment {
diff --git a/proxmox-ve-config/src/sdn/microseg/mod.rs b/proxmox-ve-config/src/sdn/microseg/mod.rs
index 0688d4e58180..620f342ad2b0 100644
--- a/proxmox-ve-config/src/sdn/microseg/mod.rs
+++ b/proxmox-ve-config/src/sdn/microseg/mod.rs
@@ -15,13 +15,14 @@ mod types;
pub use identity::PredicateMatch;
pub use types::{
Binding, GroupName, GroupSection, GuestAssignmentSection, GuestInfo, MicrosegEntry, MicrosegId,
- MicrosegRunningConfig, RuleSection, Tag, TagAssignmentSection,
+ MicrosegRunningConfig, RegexAssignmentSection, RuleSection, Tag, TagAssignmentSection,
};
use std::collections::{BTreeMap, BTreeSet, HashMap};
use anyhow::{Context, Error, bail};
use identity::UNTAGGED_GROUP;
+use regex::Regex;
use serde::Serialize;
/// Validate a microseg config before anything renders from it, so resolving group names and
@@ -31,7 +32,9 @@ pub fn validate(entries: &BTreeMap<String, MicrosegEntry>) -> Result<(), Error>
for (name, entry) in entries {
let kind = match entry {
MicrosegEntry::Rule(_) => "rule",
- MicrosegEntry::GuestAssignment(_) | MicrosegEntry::TagAssignment(_) => "assignment",
+ MicrosegEntry::GuestAssignment(_)
+ | MicrosegEntry::TagAssignment(_)
+ | MicrosegEntry::RegexAssignment(_) => "assignment",
MicrosegEntry::Group(_) => "group",
};
validate_entry(entries, &format!("{kind} '{name}'"), entry)?;
@@ -55,6 +58,17 @@ fn validate_entry(
bail!("{subject} matches tags with 'every', which only a rule predicate takes");
}
}
+ MicrosegEntry::RegexAssignment(assignment) => {
+ if assignment.name_regex.is_empty() {
+ bail!("{subject} has an empty name regex");
+ }
+ if !assignment.name_regex.is_ascii() {
+ bail!("{subject} has a name regex with non-ASCII characters");
+ }
+ if let Err(err) = Regex::new(&assignment.name_regex) {
+ bail!("{subject} has an invalid name regex: {err}");
+ }
+ }
MicrosegEntry::GuestAssignment(_) | MicrosegEntry::Group(_) => {}
}
if let Some(groups) = entry.assignment_groups() {
@@ -212,6 +226,9 @@ fn tags_fire(tags: &[Tag], tag_match: PredicateMatch, guest: &GuestInfo) -> bool
enum Matcher<'a> {
Guest(u32),
Tags(&'a [Tag], PredicateMatch),
+ /// `None` for a regex that does not compile, which validation rejects, so it fires on nothing
+ /// rather than failing the resolve.
+ Name(Option<Regex>),
}
struct Assignment<'a> {
@@ -227,6 +244,12 @@ impl Assignment<'_> {
let on_guest = match &self.matcher {
Matcher::Guest(vmid) => *vmid == guest.vmid,
Matcher::Tags(tags, tag_match) => tags_fire(tags, *tag_match, guest),
+ Matcher::Name(regex) => regex.as_ref().is_some_and(|regex| {
+ guest
+ .name
+ .as_deref()
+ .is_some_and(|name| regex.is_match(name))
+ }),
};
on_guest && self.iface.is_none_or(|scoped| scoped == iface)
}
@@ -247,6 +270,9 @@ fn assignments(entries: &BTreeMap<String, MicrosegEntry>) -> Vec<Assignment<'_>>
MicrosegEntry::TagAssignment(a) => {
(&a.binding, Matcher::Tags(&a.tags, a.tag_match))
}
+ MicrosegEntry::RegexAssignment(a) => {
+ (&a.binding, Matcher::Name(Regex::new(&a.name_regex).ok()))
+ }
MicrosegEntry::Group(_) | MicrosegEntry::Rule(_) => return None,
};
Some(Assignment {
@@ -379,6 +405,29 @@ pub fn render(
})
}
+/// Whether a guest renamed from `old` to `new` enters or leaves the names a regex assignment
+/// matches, decided by the same expressions that resolve the NICs.
+pub fn name_match_changed(
+ entries: &BTreeMap<String, MicrosegEntry>,
+ old: Option<&str>,
+ new: Option<&str>,
+) -> bool {
+ assignments(entries)
+ .iter()
+ .any(|assignment| match &assignment.matcher {
+ Matcher::Name(regex) => {
+ let matches = |name: Option<&str>| {
+ regex
+ .as_ref()
+ .zip(name)
+ .is_some_and(|(regex, name)| regex.is_match(name))
+ };
+ matches(old) != matches(new)
+ }
+ _ => false,
+ })
+}
+
/// Each assignment's place in the match order, by id. The first has the lowest number.
pub fn assignment_order(entries: &BTreeMap<String, MicrosegEntry>) -> HashMap<String, usize> {
assignments(entries)
diff --git a/proxmox-ve-config/src/sdn/microseg/types.rs b/proxmox-ve-config/src/sdn/microseg/types.rs
index cf6886267e95..18367490e076 100644
--- a/proxmox-ve-config/src/sdn/microseg/types.rs
+++ b/proxmox-ve-config/src/sdn/microseg/types.rs
@@ -27,7 +27,7 @@ api_string_type! {
pub struct MicrosegId(String);
}
-const GENERATED_ID_REGEX_STR: &str = r"(?:vm[0-9]+(?:i[0-9]+)?|(?:r|tag)[0-9a-f]{16})";
+const GENERATED_ID_REGEX_STR: &str = r"(?:vm[0-9]+(?:i[0-9]+)?|(?:r|tag|re)[0-9a-f]{16})";
const_regex! {
GENERATED_ID_REGEX = concatcp!(r"^", GENERATED_ID_REGEX_STR, r"$");
@@ -264,6 +264,27 @@ pub struct TagAssignmentSection {
pub(crate) binding: Binding,
}
+#[api(
+ properties: {
+ "name-regex": {
+ type: String,
+ max_length: 1024,
+ description: "Regular expression matched against each guest's name, ASCII only. It matches anywhere in the name, so anchor it with '^' and '$' to match the whole name. A guest without a name is never matched.",
+ },
+ binding: { flatten: true, type: Binding },
+ },
+)]
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
+/// Binds the NICs of every guest whose name matches `name_regex` to a set of groups. Stored as the
+/// `regex-assignment` section type.
+pub struct RegexAssignmentSection {
+ pub(crate) id: MicrosegId,
+ #[serde(default, rename = "name-regex")]
+ pub(crate) name_regex: String,
+ #[serde(flatten)]
+ pub(crate) binding: Binding,
+}
+
#[api(
"id-property": "id",
"id-schema": {
@@ -285,6 +306,8 @@ pub enum MicrosegEntry {
GuestAssignment(GuestAssignmentSection),
/// An assignment by guest tags.
TagAssignment(TagAssignmentSection),
+ /// An assignment by guest name regex.
+ RegexAssignment(RegexAssignmentSection),
}
impl MicrosegEntry {
@@ -295,6 +318,7 @@ impl MicrosegEntry {
Self::Rule(r) => &r.id,
Self::GuestAssignment(a) => &a.id,
Self::TagAssignment(a) => &a.id,
+ Self::RegexAssignment(a) => &a.id,
}
}
@@ -302,6 +326,7 @@ impl MicrosegEntry {
match self {
Self::GuestAssignment(a) => Some(&a.binding),
Self::TagAssignment(a) => Some(&a.binding),
+ Self::RegexAssignment(a) => Some(&a.binding),
Self::Group(_) | Self::Rule(_) => None,
}
}
@@ -310,6 +335,7 @@ impl MicrosegEntry {
match self {
Self::GuestAssignment(a) => Some(&mut a.binding),
Self::TagAssignment(a) => Some(&mut a.binding),
+ Self::RegexAssignment(a) => Some(&mut a.binding),
Self::Group(_) | Self::Rule(_) => None,
}
}
@@ -339,6 +365,8 @@ pub struct GuestInfo {
pub vmid: u32,
#[serde(default)]
pub tags: Vec<Tag>,
+ #[serde(default)]
+ pub name: Option<String>,
/// The present netN interface indices.
#[serde(default, deserialize_with = "deserialize_perl_nics")]
pub nics: Vec<u32>,
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-ve-rs v3 09/45] ve-config: sdn: microseg: narrow rules by protocol and ports
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (7 preceding siblings ...)
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 08/45] ve-config: sdn: microseg: add name regex matcher Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 10/45] ve-config: sdn: microseg: add group hierarchy Hannes Laimer
` (35 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
A rule can be narrowed to a protocol, optionally with ports or an
ICMP type, or to a firewall macro standing in for them. It then
decides only the traffic it matches.
For a pair, the highest priority with a rule for the whole pair gives
the verdict. Narrowed rules down to that priority are checked first,
in priority order. So a narrowed rule makes an exception to a
whole-pair rule of the same priority. The model also reports each
rule's place in the evaluation order, so a caller that lists the
rules does not have to work the order out again.
What a rule narrows to is part of its id, so two rules on the same
predicates can coexist. Like the predicates, it cannot be changed
after creation. A both-ways rule matches the reverse direction with
the ports swapped. It cannot be narrowed to an ICMP type, since the
reverse type cannot be derived in general.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
proxmox-ve-config/src/firewall/fw_macros.rs | 7 +
proxmox-ve-config/src/firewall/ports.rs | 65 +++++
proxmox-ve-config/src/sdn/microseg/api.rs | 64 ++++-
proxmox-ve-config/src/sdn/microseg/mod.rs | 266 ++++++++++++++++++--
proxmox-ve-config/src/sdn/microseg/types.rs | 116 ++++++++-
5 files changed, 496 insertions(+), 22 deletions(-)
diff --git a/proxmox-ve-config/src/firewall/fw_macros.rs b/proxmox-ve-config/src/firewall/fw_macros.rs
index 5fa8dab1ca26..e0bdf365635b 100644
--- a/proxmox-ve-config/src/firewall/fw_macros.rs
+++ b/proxmox-ve-config/src/firewall/fw_macros.rs
@@ -67,3 +67,10 @@ fn macros() -> &'static HashMap<String, FwMacro> {
pub fn get_macro(name: &str) -> Option<&'static FwMacro> {
macros().get(name)
}
+
+pub fn macro_name(name: &str) -> Option<&'static str> {
+ macros()
+ .keys()
+ .find(|known| known.eq_ignore_ascii_case(name))
+ .map(String::as_str)
+}
diff --git a/proxmox-ve-config/src/firewall/ports.rs b/proxmox-ve-config/src/firewall/ports.rs
index f88a442adab4..62d3bacda228 100644
--- a/proxmox-ve-config/src/firewall/ports.rs
+++ b/proxmox-ve-config/src/firewall/ports.rs
@@ -78,3 +78,68 @@ pub fn parse_named_port(name: &str) -> Result<u16, Error> {
.find(name)
.ok_or_else(|| format_err!("unknown port name {name:?}"))
}
+
+#[derive(Default)]
+struct NamedProtocols {
+ protocols: std::collections::HashMap<String, u8>,
+}
+
+impl NamedProtocols {
+ fn new() -> Self {
+ use std::io::BufRead;
+
+ log::trace!("loading /etc/protocols");
+
+ let mut this = Self::default();
+
+ let file = match std::fs::File::open("/etc/protocols") {
+ Ok(file) => file,
+ Err(_) => return this,
+ };
+
+ for line in std::io::BufReader::new(file).lines() {
+ let line = match line {
+ Ok(line) => line,
+ Err(_) => break,
+ };
+
+ let line = line.split_once('#').map_or(line.as_str(), |(text, _)| text);
+
+ let mut parts = line.split_ascii_whitespace();
+
+ let (name, number) = match (parts.next(), parts.next()) {
+ (Some(name), Some(number)) => (name, number),
+ _ => continue,
+ };
+
+ let number: u8 = match number.parse() {
+ Ok(number) => number,
+ Err(_) => continue,
+ };
+
+ // the first line naming a protocol wins, as for getprotobyname
+ for name in std::iter::once(name).chain(parts) {
+ this.protocols.entry(name.to_string()).or_insert(number);
+ }
+ }
+
+ this
+ }
+
+ fn find(&self, name: &str) -> Option<u8> {
+ self.protocols.get(name).copied()
+ }
+}
+
+fn named_protocols() -> &'static NamedProtocols {
+ static NAMED_PROTOCOLS: OnceLock<NamedProtocols> = OnceLock::new();
+
+ NAMED_PROTOCOLS.get_or_init(NamedProtocols::new)
+}
+
+/// Parse a named protocol with the help of `/etc/protocols`.
+pub fn parse_named_protocol(name: &str) -> Result<u8, Error> {
+ named_protocols()
+ .find(name)
+ .ok_or_else(|| format_err!("unknown protocol name {name:?}"))
+}
diff --git a/proxmox-ve-config/src/sdn/microseg/api.rs b/proxmox-ve-config/src/sdn/microseg/api.rs
index 08b5ee1f4653..fe125c4d4c1b 100644
--- a/proxmox-ve-config/src/sdn/microseg/api.rs
+++ b/proxmox-ve-config/src/sdn/microseg/api.rs
@@ -8,6 +8,8 @@ use anyhow::{Error, bail};
use proxmox_section_config::typed::ApiSectionDataEntry;
use serde::Deserialize;
+use crate::firewall::fw_macros;
+
use super::types::{
Binding, GroupSection, GuestAssignmentSection, GuestInfo, MicrosegEntry, MicrosegId,
MicrosegRunningConfig, RegexAssignmentSection, RuleSection, Tag, TagAssignmentSection,
@@ -52,6 +54,16 @@ pub struct RuleCreate {
#[serde(deserialize_with = "proxmox_serde::perl::deserialize_bool")]
pub allow: bool,
#[serde(default)]
+ pub proto: Option<String>,
+ #[serde(default)]
+ pub sport: Option<String>,
+ #[serde(default)]
+ pub dport: Option<String>,
+ #[serde(default, rename = "icmp-type")]
+ pub icmp_type: Option<String>,
+ #[serde(default, rename = "macro")]
+ pub fw_macro: Option<String>,
+ #[serde(default)]
pub comment: Option<String>,
}
@@ -195,6 +207,7 @@ fn rule_id(
src_groups: &[&str],
dst_match: PredicateMatch,
dst_groups: &[&str],
+ narrowing: &[Option<&str>],
bidirectional: bool,
) -> Result<MicrosegId, Error> {
let mut hash = Fnv1a::new();
@@ -202,6 +215,9 @@ fn rule_id(
hash.groups(src_groups);
hash.match_kind(dst_match);
hash.groups(dst_groups);
+ for field in narrowing {
+ hash.text(field.unwrap_or_default());
+ }
hash.flag(bidirectional);
format!("r{:016x}", hash.0).parse()
}
@@ -289,11 +305,30 @@ fn build_entry(
MicrosegCreate::Rule(rule) => {
let src_groups = sorted_groups(&rule.src, entries)?;
let dst_groups = sorted_groups(&rule.dst, entries)?;
+ let trimmed = |field: Option<String>| field.map(|value| value.trim().to_string());
+ let proto = trimmed(rule.proto);
+ let sport = trimmed(rule.sport);
+ let dport = trimmed(rule.dport);
+ let icmp_type = trimmed(rule.icmp_type);
+ // the expansion looks a macro up by its exact name, so the table's spelling is
+ // stored and an unknown name stays as given for validation to refuse
+ let fw_macro = trimmed(rule.fw_macro).map(|name| match fw_macros::macro_name(&name) {
+ Some(known) => known.to_string(),
+ None => name,
+ });
+ let narrowing = [
+ proto.as_deref(),
+ sport.as_deref(),
+ dport.as_deref(),
+ icmp_type.as_deref(),
+ fw_macro.as_deref(),
+ ];
let id = rule_id(
rule.src_match,
&src_groups,
rule.dst_match,
&dst_groups,
+ &narrowing,
rule.bidirectional,
)?;
MicrosegEntry::Rule(RuleSection {
@@ -305,6 +340,11 @@ fn build_entry(
bidirectional: rule.bidirectional,
prio: rule.prio,
allow: rule.allow,
+ proto,
+ sport,
+ dport,
+ icmp_type,
+ fw_macro,
comment: rule.comment,
})
}
@@ -540,13 +580,33 @@ fn predicate_str(match_kind: PredicateMatch, groups: &[impl AsRef<str>]) -> Stri
format!("{kind}{{{list}}}")
}
+/// Rendered from the stored fields, since expanding a macro can fail and its name is what a user
+/// recognizes.
+fn narrowing_str(rule: &RuleSection) -> Option<String> {
+ if let Some(name) = &rule.fw_macro {
+ return Some(format!("macro {name}"));
+ }
+ let parts = [
+ rule.proto.clone(),
+ rule.sport.as_ref().map(|ports| format!("sport {ports}")),
+ rule.dport.as_ref().map(|ports| format!("dport {ports}")),
+ rule.icmp_type.as_ref().map(|kind| format!("type {kind}")),
+ ];
+ let narrowing = parts.into_iter().flatten().collect::<Vec<_>>().join(" ");
+ (!narrowing.is_empty()).then_some(narrowing)
+}
+
fn describe_rule(rule: &RuleSection) -> String {
let arrow = if rule.bidirectional { "<->" } else { "->" };
- format!(
+ let base = format!(
"rule {} {arrow} {}",
predicate_str(rule.src_match, &rule.src),
predicate_str(rule.dst_match, &rule.dst),
- )
+ );
+ match narrowing_str(rule) {
+ Some(narrowing) => format!("{base} ({narrowing})"),
+ None => base,
+ }
}
/// A human-readable description of an entry for diagnostics, by what a user would recognize rather
diff --git a/proxmox-ve-config/src/sdn/microseg/mod.rs b/proxmox-ve-config/src/sdn/microseg/mod.rs
index 620f342ad2b0..3aaf299f8169 100644
--- a/proxmox-ve-config/src/sdn/microseg/mod.rs
+++ b/proxmox-ve-config/src/sdn/microseg/mod.rs
@@ -1,12 +1,12 @@
//! Microseg subsystem of the SDN config.
//!
//! Microseg is an identity-based stateless firewall. Each managed guest NIC gets the *set* of
-//! *groups* of the first *assignment* in admin order that covers it. The render allocates a wire
+//! *groups* of the first *assignment* in match order that covers it. The render allocates a wire
//! identity for the set of every assignment (see [`identity`]), and the running config carries it
//! to every host, so a NIC can only land in a class that already exists. Each host looks the
-//! identities up and compiles the policy *rules* into a `(src_id, dst_id) -> allow|deny` map. A
-//! rule matches traffic by a `src` and a `dst` *predicate*, each a match kind over a list of
-//! groups.
+//! identities up and folds the policy *rules* per `(src_id, dst_id)` pair. The fold yields the
+//! narrowed rules to check and the verdict for the traffic none of them matches. A rule matches
+//! traffic by a `src` and a `dst` *predicate*, each a match kind over a list of groups.
pub mod api;
pub mod identity;
@@ -18,6 +18,7 @@ pub use types::{
MicrosegRunningConfig, RegexAssignmentSection, RuleSection, Tag, TagAssignmentSection,
};
+use std::cmp::Ordering;
use std::collections::{BTreeMap, BTreeSet, HashMap};
use anyhow::{Context, Error, bail};
@@ -25,6 +26,8 @@ use identity::UNTAGGED_GROUP;
use regex::Regex;
use serde::Serialize;
+use crate::firewall::types::rule_match::{IcmpType, Icmpv6Type, Ports, Protocol, Sctp, Tcp, Udp};
+
/// Validate a microseg config before anything renders from it, so resolving group names and
/// expanding rules cannot fail further down.
pub fn validate(entries: &BTreeMap<String, MicrosegEntry>) -> Result<(), Error> {
@@ -86,6 +89,14 @@ fn validate_entry(
Ok(())
}
+fn has_icmp_type(protocol: &Protocol) -> bool {
+ match protocol {
+ Protocol::Icmp(icmp) => !matches!(icmp.ty(), None | Some(IcmpType::Any)),
+ Protocol::Icmpv6(icmp) => !matches!(icmp.ty(), None | Some(Icmpv6Type::Any)),
+ _ => false,
+ }
+}
+
fn validate_rule(
entries: &BTreeMap<String, MicrosegEntry>,
subject: &str,
@@ -133,6 +144,18 @@ fn validate_rule(
for group in rule.src.iter().chain(&rule.dst) {
known_group(entries, group.as_ref()).with_context(|| subject.to_string())?;
}
+ let protocols = match rule.protocols() {
+ Ok(protocols) => protocols,
+ Err(err) => bail!("{subject}: {err}"),
+ };
+ // a macro can expand to an entry with an ICMP type, so the check reads the expanded
+ // matches and not the rule's own field
+ if rule.bidirectional && protocols.iter().any(has_icmp_type) {
+ bail!(
+ "{subject} is bidirectional and names an ICMP type, the mirror does not \
+ derive a reverse type"
+ );
+ }
Ok(())
}
@@ -405,6 +428,31 @@ pub fn render(
})
}
+fn evaluation_order(a: &ResolvedRule, b: &ResolvedRule) -> Ordering {
+ b.prio
+ .cmp(&a.prio)
+ .then(b.is_narrowed().cmp(&a.is_narrowed()))
+ .then(a.allow.cmp(&b.allow))
+ // the id keeps the order stable, and the orientation after it, since both orientations of
+ // a bidirectional rule carry the same id
+ .then(a.id.cmp(&b.id))
+ .then(a.reverse.cmp(&b.reverse))
+}
+
+/// Each rule's place in the evaluation order, by rule id. The first has the lowest number.
+pub fn rule_order(
+ entries: &BTreeMap<String, MicrosegEntry>,
+) -> Result<HashMap<MicrosegId, usize>, Error> {
+ let mut rules = resolved_rules(entries)?;
+ rules.sort_by(evaluation_order);
+ let mut order = HashMap::new();
+ for rule in rules {
+ let place = order.len();
+ order.entry(rule.id).or_insert(place);
+ }
+ Ok(order)
+}
+
/// Whether a guest renamed from `old` to `new` enters or leaves the names a regex assignment
/// matches, decided by the same expressions that resolve the NICs.
pub fn name_match_changed(
@@ -439,10 +487,35 @@ pub fn assignment_order(entries: &BTreeMap<String, MicrosegEntry>) -> HashMap<St
#[derive(Debug)]
pub struct ResolvedRule {
+ id: MicrosegId,
src: identity::Predicate,
dst: identity::Predicate,
prio: u16,
allow: bool,
+ /// The protocol matches narrowing the rule. Empty for a rule deciding the whole pair.
+ protocols: Vec<Protocol>,
+ /// Set on the mirrored orientation of a bidirectional rule.
+ reverse: bool,
+}
+
+impl ResolvedRule {
+ fn is_narrowed(&self) -> bool {
+ !self.protocols.is_empty()
+ }
+}
+
+fn reversed_protocol(protocol: &Protocol) -> Protocol {
+ let swapped = |ports: &Ports| Ports::new(ports.dport().cloned(), ports.sport().cloned());
+ match protocol {
+ Protocol::Dccp(ports) => Protocol::Dccp(swapped(ports)),
+ Protocol::Sctp(sctp) => Protocol::Sctp(Sctp::new(swapped(sctp.ports()))),
+ Protocol::Tcp(tcp) => Protocol::Tcp(Tcp::new(swapped(tcp.ports()))),
+ Protocol::Udp(udp) => Protocol::Udp(Udp::new(swapped(udp.ports()))),
+ Protocol::UdpLite(ports) => Protocol::UdpLite(swapped(ports)),
+ Protocol::Icmp(_) | Protocol::Icmpv6(_) | Protocol::Named(_) | Protocol::Numeric(_) => {
+ protocol.clone()
+ }
+ }
}
pub fn resolved_rules(
@@ -452,16 +525,22 @@ pub fn resolved_rules(
for rule in rule_entries(entries) {
let context = || format!("rule '{}'", rule.id);
let forward = ResolvedRule {
+ id: rule.id.clone(),
src: predicate(entries, rule.src_match, &rule.src).with_context(context)?,
dst: predicate(entries, rule.dst_match, &rule.dst).with_context(context)?,
prio: rule.prio,
allow: rule.allow,
+ protocols: rule.protocols().with_context(context)?,
+ reverse: false,
};
let mirrored = rule.bidirectional.then(|| ResolvedRule {
+ id: forward.id.clone(),
src: forward.dst.clone(),
dst: forward.src.clone(),
prio: forward.prio,
allow: forward.allow,
+ protocols: forward.protocols.iter().map(reversed_protocol).collect(),
+ reverse: true,
});
resolved.push(forward);
resolved.extend(mirrored);
@@ -469,26 +548,39 @@ pub fn resolved_rules(
Ok(resolved)
}
-/// Fold the rules that fire on one `(src_class, dst_class)` pair into its verdict. The highest
-/// priority wins, and a single deny among the rules tied there overrides the allows. Without a
-/// firing rule the pair denies.
-pub fn fold<'a>(firing: impl IntoIterator<Item = &'a ResolvedRule>) -> bool {
- let mut best: Option<(u16, bool)> = None;
- for rule in firing {
- best = Some(match best {
- Some((prio, _)) if rule.prio > prio => (rule.prio, rule.allow),
- Some((prio, allow)) if rule.prio == prio => (prio, allow && rule.allow),
- Some(current) => current,
- None => (rule.prio, rule.allow),
- });
+#[derive(Debug)]
+pub struct Folded<'a> {
+ /// Narrowed rules in evaluation order. The first one whose protocol match fires decides.
+ pub checks: Vec<&'a ResolvedRule>,
+ pub allow: bool,
+}
+
+/// Fold the rules that fire on one `(src_class, dst_class)` pair into its verdict and checks. The
+/// verdict comes from the highest priority with a rule that decides the whole pair, and a single
+/// deny among those rules wins. Without such a rule the pair denies. The checks are the narrowed
+/// rules from the top priority down to that one, denies ahead of allows within a priority.
+pub fn fold<'a>(firing: impl IntoIterator<Item = &'a ResolvedRule>) -> Folded<'a> {
+ let mut firing: Vec<&ResolvedRule> = firing.into_iter().collect();
+ firing.sort_by(|a, b| evaluation_order(a, b));
+
+ let mut checks = Vec::new();
+ let mut allow = false;
+ for level in firing.chunk_by(|a, b| a.prio == b.prio) {
+ checks.extend(level.iter().filter(|rule| rule.is_narrowed()));
+ let mut deciding = level.iter().filter(|rule| !rule.is_narrowed()).peekable();
+ if deciding.peek().is_some() {
+ allow = deciding.all(|rule| rule.allow);
+ break;
+ }
}
- best.is_some_and(|(_, allow)| allow)
+ Folded { checks, allow }
}
#[cfg(test)]
mod tests {
use super::*;
+ use crate::firewall::types::rule_match::RuleOptions;
pub(super) fn firing_rules<'a>(
rules: &'a [ResolvedRule],
@@ -679,7 +771,9 @@ mod tests {
}
fn verdict(rules: &[ResolvedRule], src: &identity::GroupSet, dst: &identity::GroupSet) -> bool {
- fold(firing_rules(rules, src, dst))
+ let folded = fold(firing_rules(rules, src, dst));
+ assert!(folded.checks.is_empty(), "no narrowed rules expected");
+ folded.allow
}
fn rule_full(
@@ -702,6 +796,11 @@ mod tests {
bidirectional: false,
prio,
allow,
+ proto: None,
+ sport: None,
+ dport: None,
+ icmp_type: None,
+ fw_macro: None,
comment: None,
}),
)
@@ -742,4 +841,135 @@ mod tests {
// from an interface in web alone only the allow fires
assert!(verdict(&tied_rules, &set(&["web"]), &set(&["db"])));
}
+
+ fn narrowed(
+ name: &str,
+ src: &[&str],
+ dst: &[&str],
+ prio: u16,
+ allow: bool,
+ proto: Option<&str>,
+ dport: Option<&str>,
+ ) -> (String, MicrosegEntry) {
+ let entry = rule_full(
+ name,
+ src,
+ PredicateMatch::Any,
+ dst,
+ PredicateMatch::Any,
+ prio,
+ allow,
+ );
+ with_rule(entry, |rule| {
+ rule.proto = proto.map(str::to_string);
+ rule.dport = dport.map(str::to_string);
+ })
+ }
+
+ #[test]
+ fn cell_orders_checks_and_decides_the_rest() {
+ let entries: BTreeMap<String, MicrosegEntry> = BTreeMap::from([
+ group("web"),
+ group("db"),
+ rule_full(
+ "whole",
+ &["web"],
+ PredicateMatch::Any,
+ &["db"],
+ PredicateMatch::Any,
+ 0,
+ true,
+ ),
+ narrowed("ssh", &["web"], &["db"], 10, false, Some("tcp"), Some("22")),
+ narrowed("dns", &["web"], &["db"], 10, true, Some("udp"), Some("53")),
+ narrowed("http", &["web"], &["db"], 20, true, Some("tcp"), Some("80")),
+ ]);
+ let rules = resolved(&entries);
+ let folded = fold(firing_rules(&rules, &set(&["web"]), &set(&["db"])));
+ let order: Vec<&str> = folded.checks.iter().map(|r| r.id.as_ref()).collect();
+ assert_eq!(
+ order,
+ ["http", "ssh", "dns"],
+ "prio first, then deny before allow"
+ );
+ assert!(folded.allow, "the whole-pair allow decides the rest");
+
+ // A whole-pair deny at prio 15 ends the walk before the narrowed rules below it.
+ let mut cut = entries.clone();
+ cut.extend([rule_full(
+ "cut",
+ &["web"],
+ PredicateMatch::Any,
+ &["db"],
+ PredicateMatch::Any,
+ 15,
+ false,
+ )]);
+ let rules = resolved(&cut);
+ let folded = fold(firing_rules(&rules, &set(&["web"]), &set(&["db"])));
+ let order: Vec<&str> = folded.checks.iter().map(|r| r.id.as_ref()).collect();
+ assert_eq!(order, ["http"]);
+ assert!(!folded.allow);
+
+ // Only narrowed rules, so the rest is denied.
+ let mut only = entries.clone();
+ only.remove("whole");
+ let rules = resolved(&only);
+ let folded = fold(firing_rules(&rules, &set(&["web"]), &set(&["db"])));
+ assert_eq!(folded.checks.len(), 3);
+ assert!(!folded.allow);
+ }
+
+ #[test]
+ fn bidirectional_narrowing_swaps_the_ports() {
+ let tcp = |sport: Option<&str>, dport: Option<&str>| {
+ Protocol::from_options(&RuleOptions {
+ proto: Some("tcp".to_string()),
+ sport: sport.map(str::to_string),
+ dport: dport.map(str::to_string),
+ ..Default::default()
+ })
+ .expect("valid options")
+ .expect("a protocol")
+ };
+ let check = |rules: &[ResolvedRule], src: &str, dst: &str| {
+ let folded = fold(firing_rules(rules, &set(&[src]), &set(&[dst])));
+ assert_eq!(folded.checks.len(), 1, "one check per direction");
+ folded.checks[0].protocols.clone()
+ };
+
+ let explicit = BTreeMap::from([
+ group("web"),
+ group("db"),
+ bidirectional(narrowed(
+ "bsql",
+ &["web"],
+ &["db"],
+ 0,
+ true,
+ Some("tcp"),
+ Some("5432"),
+ )),
+ ]);
+ validate(&explicit).expect("valid");
+ let rules = resolved(&explicit);
+ assert_eq!(check(&rules, "web", "db"), [tcp(None, Some("5432"))]);
+ assert_eq!(
+ check(&rules, "db", "web"),
+ [tcp(Some("5432"), None)],
+ "the reverse direction matches the port as a source port"
+ );
+
+ let from_macro = BTreeMap::from([
+ group("web"),
+ group("db"),
+ bidirectional(with_rule(rule("bhttps", &["web"], &["db"], true), |rule| {
+ rule.fw_macro = Some("HTTPS".into())
+ })),
+ ]);
+ validate(&from_macro).expect("valid");
+ let rules = resolved(&from_macro);
+ assert_eq!(check(&rules, "web", "db"), [tcp(None, Some("443"))]);
+ assert_eq!(check(&rules, "db", "web"), [tcp(Some("443"), None)]);
+ }
}
diff --git a/proxmox-ve-config/src/sdn/microseg/types.rs b/proxmox-ve-config/src/sdn/microseg/types.rs
index 18367490e076..ee0a03beec51 100644
--- a/proxmox-ve-config/src/sdn/microseg/types.rs
+++ b/proxmox-ve-config/src/sdn/microseg/types.rs
@@ -2,12 +2,16 @@
use std::collections::BTreeMap;
-use anyhow::{Error, bail};
+use anyhow::{Error, anyhow, bail};
use const_format::concatcp;
use serde::{Deserialize, Serialize};
use proxmox_schema::{ApiStringFormat, ApiType, api, api_string_type, const_regex};
+use crate::firewall::fw_macros;
+use crate::firewall::ports::parse_named_protocol;
+use crate::firewall::types::rule_match::{Protocol, RuleOptions};
+
use super::PredicateMatch;
use super::identity;
use super::identity::UNTAGGED_GROUP;
@@ -119,6 +123,36 @@ pub struct GroupSection {
minimum: 0,
maximum: 65535,
},
+ proto: {
+ type: String,
+ optional: true,
+ max_length: 32,
+ description: "IP protocol, a name from /etc/protocols or a number.",
+ },
+ sport: {
+ type: String,
+ optional: true,
+ max_length: 128,
+ description: "Source port list, only with a port-carrying protocol.",
+ },
+ dport: {
+ type: String,
+ optional: true,
+ max_length: 128,
+ description: "Destination port list, only with a port-carrying protocol.",
+ },
+ "icmp-type": {
+ type: String,
+ optional: true,
+ max_length: 64,
+ description: "ICMP type, only with an ICMP protocol.",
+ },
+ "macro": {
+ type: String,
+ optional: true,
+ max_length: 128,
+ description: "Firewall macro standing in for the protocol and ports.",
+ },
comment: {
type: String,
optional: true,
@@ -131,6 +165,9 @@ pub struct GroupSection {
/// A policy rule. It fires on traffic from an interface whose group set matches `(src_match, src)`
/// to one whose set matches `(dst_match, dst)`, and decides it with `allow`. A rule ahead of it in
/// the evaluation order decides first. Traffic no rule decides is denied.
+///
+/// A rule may be narrowed to a protocol with ports or an ICMP type, or to a firewall macro that
+/// stands in for them.
pub struct RuleSection {
pub(crate) id: MicrosegId,
/// Groups the source predicate lists.
@@ -145,7 +182,8 @@ pub struct RuleSection {
/// How the destination predicate matches its groups.
#[serde(default, rename = "dst-match")]
pub(crate) dst_match: PredicateMatch,
- /// Whether the rule also decides the reverse direction, with the predicates swapped.
+ /// Whether the rule also decides the reverse direction, with the predicates and the ports
+ /// swapped.
#[serde(
default,
deserialize_with = "proxmox_serde::perl::deserialize_bool",
@@ -159,9 +197,83 @@ pub struct RuleSection {
#[serde(deserialize_with = "proxmox_serde::perl::deserialize_bool")]
pub(crate) allow: bool,
#[serde(default, skip_serializing_if = "Option::is_none")]
+ pub(crate) proto: Option<String>,
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub(crate) sport: Option<String>,
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub(crate) dport: Option<String>,
+ #[serde(default, rename = "icmp-type", skip_serializing_if = "Option::is_none")]
+ pub(crate) icmp_type: Option<String>,
+ #[serde(default, rename = "macro", skip_serializing_if = "Option::is_none")]
+ pub(crate) fw_macro: Option<String>,
+ #[serde(default, skip_serializing_if = "Option::is_none")]
pub(crate) comment: Option<String>,
}
+impl RuleSection {
+ /// The protocol matches narrowing this rule, empty for a rule that decides the whole pair.
+ /// Ports are only accepted with a port-carrying protocol and an ICMP type only with an ICMP
+ /// protocol, since either would otherwise be silently ignored.
+ pub(super) fn protocols(&self) -> Result<Vec<Protocol>, Error> {
+ let explicit = [&self.proto, &self.sport, &self.dport, &self.icmp_type]
+ .iter()
+ .any(|field| field.is_some());
+ if let Some(name) = &self.fw_macro {
+ if explicit {
+ bail!("a macro cannot be combined with a protocol, ports or an ICMP type");
+ }
+ return match fw_macros::get_macro(name) {
+ Some(fw_macro) => Ok(fw_macro.code.clone()),
+ None => bail!("unknown macro '{name}'"),
+ };
+ }
+ let Some(proto) = &self.proto else {
+ if explicit {
+ bail!("ports and an ICMP type need a protocol");
+ }
+ return Ok(Vec::new());
+ };
+ let proto = proto.trim();
+ let options = RuleOptions {
+ proto: (!proto.is_empty()).then(|| proto.to_string()),
+ sport: self.sport.clone(),
+ dport: self.dport.clone(),
+ icmp_type: self.icmp_type.clone(),
+ ..Default::default()
+ };
+ let mut protocol = Protocol::from_options(&options)?
+ .ok_or_else(|| anyhow!("empty protocol specification"))?;
+ // the data plane matches the number, so an unknown name fails validation rather than the
+ // apply
+ if let Protocol::Named(name) = &protocol {
+ let number = parse_named_protocol(name)?.to_string();
+ let options = RuleOptions {
+ proto: Some(number),
+ ..options
+ };
+ protocol = Protocol::from_options(&options)?
+ .ok_or_else(|| anyhow!("empty protocol specification"))?;
+ }
+ let ports = self.sport.is_some() || self.dport.is_some();
+ let carries_ports = matches!(
+ protocol,
+ Protocol::Tcp(_)
+ | Protocol::Udp(_)
+ | Protocol::Sctp(_)
+ | Protocol::Dccp(_)
+ | Protocol::UdpLite(_)
+ );
+ if ports && !carries_ports {
+ bail!("protocol '{proto}' carries no ports");
+ }
+ let icmp = matches!(protocol, Protocol::Icmp(_) | Protocol::Icmpv6(_));
+ if self.icmp_type.is_some() && !icmp {
+ bail!("an ICMP type needs an ICMP protocol, not '{proto}'");
+ }
+ Ok(vec![protocol])
+ }
+}
+
fn is_false(value: &bool) -> bool {
!*value
}
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-ve-rs v3 10/45] ve-config: sdn: microseg: add group hierarchy
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (8 preceding siblings ...)
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 09/45] ve-config: sdn: microseg: narrow rules by protocol and ports Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 11/45] ve-config: firewall: expose protocol, icmp type and code numbers Hannes Laimer
` (34 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
A group may name a parent. An interface assigned to a subgroup also
belongs to every group above it. So a rule on a group covers its
subgroups, and a rule on a subgroup covers only that subgroup.
At the same priority the rules with the most specific all or exact
predicates decide, so a rule on a subgroup outranks the same rule on
its parent. Equally specific rules still deny if one of them denies.
Identity allocation is unchanged and gets each assignment's groups
together with their ancestors.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
proxmox-ve-config/src/sdn/microseg/api.rs | 17 +-
proxmox-ve-config/src/sdn/microseg/mod.rs | 393 +++++++++++++++++++-
proxmox-ve-config/src/sdn/microseg/types.rs | 12 +
3 files changed, 405 insertions(+), 17 deletions(-)
diff --git a/proxmox-ve-config/src/sdn/microseg/api.rs b/proxmox-ve-config/src/sdn/microseg/api.rs
index fe125c4d4c1b..d9cd22344cb5 100644
--- a/proxmox-ve-config/src/sdn/microseg/api.rs
+++ b/proxmox-ve-config/src/sdn/microseg/api.rs
@@ -11,7 +11,7 @@ use serde::Deserialize;
use crate::firewall::fw_macros;
use super::types::{
- Binding, GroupSection, GuestAssignmentSection, GuestInfo, MicrosegEntry, MicrosegId,
+ Binding, GroupName, GroupSection, GuestAssignmentSection, GuestInfo, MicrosegEntry, MicrosegId,
MicrosegRunningConfig, RegexAssignmentSection, RuleSection, Tag, TagAssignmentSection,
};
use super::{PredicateMatch, assigned_nics, assignments, sorted_rules, validate, validate_entry};
@@ -129,6 +129,8 @@ pub enum MicrosegUpdate {
#[derive(Debug, Clone, Deserialize)]
pub struct GroupUpdate {
+ #[serde(default)]
+ pub parent: Option<GroupName>,
#[serde(default)]
pub comment: Option<String>,
#[serde(default)]
@@ -138,6 +140,7 @@ pub struct GroupUpdate {
#[derive(Debug, Clone, Copy, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum GroupDeletableProperties {
+ Parent,
Comment,
}
@@ -510,11 +513,15 @@ pub fn update(
};
let position = match (entry, update) {
(MicrosegEntry::Group(group), MicrosegUpdate::Group(update)) => {
+ if update.parent.is_some() {
+ group.parent = update.parent;
+ }
if update.comment.is_some() {
group.comment = update.comment;
}
for property in update.delete {
match property {
+ GroupDeletableProperties::Parent => group.parent = None,
GroupDeletableProperties::Comment => group.comment = None,
}
}
@@ -642,7 +649,9 @@ fn describe_referrer(entry: &MicrosegEntry) -> String {
),
assignment.binding.comment.as_deref(),
),
- MicrosegEntry::Group(_) => ("object".to_string(), None),
+ MicrosegEntry::Group(group) => {
+ (format!("subgroup '{}'", group.id), group.comment.as_deref())
+ }
};
match comment {
Some(comment) => format!("{desc} (comment: \"{comment}\")"),
@@ -680,7 +689,9 @@ fn group_referenced_by(entries: &BTreeMap<String, MicrosegEntry>, name: &str) ->
rule.src.iter().any(|g| g.as_ref() == name)
|| rule.dst.iter().any(|g| g.as_ref() == name)
}
- MicrosegEntry::Group(_) => false,
+ MicrosegEntry::Group(group) => {
+ group.parent.as_ref().is_some_and(|p| p.as_ref() == name)
+ }
_ => entry
.assignment_groups()
.is_some_and(|groups| groups.iter().any(|g| g.as_ref() == name)),
diff --git a/proxmox-ve-config/src/sdn/microseg/mod.rs b/proxmox-ve-config/src/sdn/microseg/mod.rs
index 3aaf299f8169..1744e3e5fc02 100644
--- a/proxmox-ve-config/src/sdn/microseg/mod.rs
+++ b/proxmox-ve-config/src/sdn/microseg/mod.rs
@@ -19,7 +19,7 @@ pub use types::{
};
use std::cmp::Ordering;
-use std::collections::{BTreeMap, BTreeSet, HashMap};
+use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet};
use anyhow::{Context, Error, bail};
use identity::UNTAGGED_GROUP;
@@ -72,7 +72,25 @@ fn validate_entry(
bail!("{subject} has an invalid name regex: {err}");
}
}
- MicrosegEntry::GuestAssignment(_) | MicrosegEntry::Group(_) => {}
+ MicrosegEntry::Group(group) => {
+ if let Some(parent) = &group.parent {
+ known_group(entries, parent.as_ref()).with_context(|| subject.to_string())?;
+ // the walk may start off the cycle, so the error names the group whose parent
+ // link closes the cycle
+ let name = group.id.as_str();
+ let mut seen: HashSet<&str> = HashSet::from([name]);
+ let mut closing = name;
+ let mut current = Some(parent.as_str());
+ while let Some(group) = current {
+ if !seen.insert(group) {
+ bail!("group '{closing}' is its own ancestor through '{group}'");
+ }
+ closing = group;
+ current = parent_of(entries, group);
+ }
+ }
+ }
+ MicrosegEntry::GuestAssignment(_) => {}
}
if let Some(groups) = entry.assignment_groups() {
if groups.is_empty() {
@@ -163,6 +181,13 @@ fn is_group(entries: &BTreeMap<String, MicrosegEntry>, name: &str) -> bool {
matches!(entries.get(name), Some(MicrosegEntry::Group(_)))
}
+fn parent_of<'a>(entries: &'a BTreeMap<String, MicrosegEntry>, name: &str) -> Option<&'a str> {
+ match entries.get(name)? {
+ MicrosegEntry::Group(group) => Some(group.parent.as_ref()?.as_ref()),
+ _ => None,
+ }
+}
+
fn known_group(entries: &BTreeMap<String, MicrosegEntry>, name: &str) -> Result<(), Error> {
if name != UNTAGGED_GROUP && !is_group(entries, name) {
bail!("unknown group '{name}'");
@@ -183,18 +208,65 @@ fn group_set<'a>(
.collect()
}
+/// The groups with all their ancestors. Membership in a subgroup implies membership in every group
+/// above it.
+fn closed_groups<'a>(
+ entries: &BTreeMap<String, MicrosegEntry>,
+ groups: impl IntoIterator<Item = &'a MicrosegId>,
+) -> Result<identity::GroupSet, Error> {
+ let mut set = identity::GroupSet::new();
+ for group in groups {
+ let mut current = group.as_ref();
+ loop {
+ known_group(entries, current)?;
+ // a group already in the set ends the walk, so a parent cycle that validation rejects
+ // cannot spin here
+ if !set.insert(current.to_string()) {
+ break;
+ }
+ match parent_of(entries, current) {
+ Some(parent) => current = parent,
+ None => break,
+ }
+ }
+ }
+ Ok(set)
+}
+
+/// An `all` or `exact` predicate gets the ancestors of its groups added, like the sets it is
+/// matched against. An `all` fires on the same sets either way, and an `exact` naming a subgroup
+/// could not fire at all without them. An `any` stays as written, since with the ancestors added it
+/// would also fire on sets holding only an ancestor of a group it names.
fn predicate(
entries: &BTreeMap<String, MicrosegEntry>,
match_kind: PredicateMatch,
groups: &[MicrosegId],
) -> Result<identity::Predicate, Error> {
- let set = group_set(entries, groups)?;
+ let set = match match_kind {
+ PredicateMatch::Any => group_set(entries, groups)?,
+ PredicateMatch::All | PredicateMatch::Exact => closed_groups(entries, groups)?,
+ PredicateMatch::Every => identity::GroupSet::new(),
+ };
Ok(identity::Predicate {
kind: match_kind,
groups: set,
})
}
+/// How much a predicate pins down, the size of its closed group set for `all` and `exact`, where
+/// every group named is a further condition. An `any` gets weaker the more it names, so it
+/// counts nothing.
+fn predicate_specificity(predicate: &identity::Predicate) -> usize {
+ match predicate.kind {
+ PredicateMatch::Any | PredicateMatch::Every => 0,
+ PredicateMatch::All | PredicateMatch::Exact => predicate.groups.len(),
+ }
+}
+
+fn specificity(src: &identity::Predicate, dst: &identity::Predicate) -> usize {
+ predicate_specificity(src) + predicate_specificity(dst)
+}
+
fn rule_entries(entries: &BTreeMap<String, MicrosegEntry>) -> impl Iterator<Item = &RuleSection> {
entries.values().filter_map(|entry| match entry {
MicrosegEntry::Rule(rule) => Some(rule),
@@ -373,7 +445,8 @@ pub fn rules_by_group_set(
classes
.into_iter()
.map(|class| {
- let set = group_set(entries, class.iter())?;
+ // the class keeps its written groups, only the matching sees the closure
+ let set = closed_groups(entries, class.iter())?;
let mut outbound = Vec::new();
let mut inbound = Vec::new();
for (rule, policy) in &rules {
@@ -395,7 +468,15 @@ pub fn rules_by_group_set(
.collect()
}
-/// The distinct group sets the assignments name, the input the identity allocator partitions.
+pub fn rule_specificity(
+ entries: &BTreeMap<String, MicrosegEntry>,
+) -> Result<HashMap<MicrosegId, usize>, Error> {
+ Ok(sorted_policies(entries)?
+ .into_iter()
+ .map(|(rule, policy)| (rule.id.clone(), specificity(&policy.src, &policy.dst)))
+ .collect())
+}
+
fn assignment_group_sets(
entries: &BTreeMap<String, MicrosegEntry>,
) -> Result<BTreeSet<identity::GroupSet>, Error> {
@@ -403,7 +484,7 @@ fn assignment_group_sets(
.iter()
.filter_map(|(id, entry)| Some((id, entry.assignment_groups()?)))
.map(|(id, groups)| {
- group_set(entries, groups).with_context(|| format!("assignment '{id}'"))
+ closed_groups(entries, groups).with_context(|| format!("assignment '{id}'"))
})
.collect()
}
@@ -432,6 +513,7 @@ fn evaluation_order(a: &ResolvedRule, b: &ResolvedRule) -> Ordering {
b.prio
.cmp(&a.prio)
.then(b.is_narrowed().cmp(&a.is_narrowed()))
+ .then(b.specificity().cmp(&a.specificity()))
.then(a.allow.cmp(&b.allow))
// the id keeps the order stable, and the orientation after it, since both orientations of
// a bidirectional rule carry the same id
@@ -502,6 +584,10 @@ impl ResolvedRule {
fn is_narrowed(&self) -> bool {
!self.protocols.is_empty()
}
+
+ fn specificity(&self) -> usize {
+ specificity(&self.src, &self.dst)
+ }
}
fn reversed_protocol(protocol: &Protocol) -> Protocol {
@@ -555,10 +641,11 @@ pub struct Folded<'a> {
pub allow: bool,
}
-/// Fold the rules that fire on one `(src_class, dst_class)` pair into its verdict and checks. The
-/// verdict comes from the highest priority with a rule that decides the whole pair, and a single
-/// deny among those rules wins. Without such a rule the pair denies. The checks are the narrowed
-/// rules from the top priority down to that one, denies ahead of allows within a priority.
+/// Fold the rules that fire on one `(src_id, dst_id)` pair into its verdict and checks. The
+/// verdict comes from the highest priority with a rule that decides the whole pair. The most
+/// specific of those rules decide, and a single deny among them wins. Without such a rule the pair
+/// denies. The checks are the narrowed rules from the top priority down to that one, the more
+/// specific first and denies ahead of allows within a priority.
pub fn fold<'a>(firing: impl IntoIterator<Item = &'a ResolvedRule>) -> Folded<'a> {
let mut firing: Vec<&ResolvedRule> = firing.into_iter().collect();
firing.sort_by(|a, b| evaluation_order(a, b));
@@ -567,9 +654,11 @@ pub fn fold<'a>(firing: impl IntoIterator<Item = &'a ResolvedRule>) -> Folded<'a
let mut allow = false;
for level in firing.chunk_by(|a, b| a.prio == b.prio) {
checks.extend(level.iter().filter(|rule| rule.is_narrowed()));
- let mut deciding = level.iter().filter(|rule| !rule.is_narrowed()).peekable();
- if deciding.peek().is_some() {
- allow = deciding.all(|rule| rule.allow);
+ let deciding = || level.iter().copied().filter(|rule| !rule.is_narrowed());
+ if let Some(most) = deciding().map(|rule| rule.specificity()).max() {
+ allow = deciding()
+ .filter(|rule| rule.specificity() == most)
+ .all(|rule| rule.allow);
break;
}
}
@@ -601,16 +690,21 @@ mod tests {
names.iter().map(|n| id(n)).collect()
}
- fn group(name: &str) -> (String, MicrosegEntry) {
+ fn group_entry(name: &str, parent: Option<&str>) -> (String, MicrosegEntry) {
(
name.to_string(),
MicrosegEntry::Group(GroupSection {
id: name.parse().expect("valid group name"),
+ parent: parent.map(|parent| parent.parse().expect("valid group name")),
comment: None,
}),
)
}
+ fn group(name: &str) -> (String, MicrosegEntry) {
+ group_entry(name, None)
+ }
+
fn rule(name: &str, src: &[&str], dst: &[&str], allow: bool) -> (String, MicrosegEntry) {
rule_full(
name,
@@ -972,4 +1066,275 @@ mod tests {
assert_eq!(check(&rules, "web", "db"), [tcp(None, Some("443"))]);
assert_eq!(check(&rules, "db", "web"), [tcp(Some("443"), None)]);
}
+
+ fn subgroup(name: &str, parent: &str) -> (String, MicrosegEntry) {
+ group_entry(name, Some(parent))
+ }
+
+ #[test]
+ fn subgroup_membership_implies_the_parents() {
+ use PredicateMatch::{All, Any, Exact};
+ let entries: BTreeMap<String, MicrosegEntry> = BTreeMap::from([
+ group("ga"),
+ subgroup("ga-b", "ga"),
+ subgroup("ga-b-c", "ga-b"),
+ group("gx"),
+ rule_full("on-a", &["ga"], All, &["gx"], Any, 0, true),
+ ]);
+ validate(&entries).expect("valid");
+ assert_eq!(
+ closed_groups(&entries, &ids(&["ga-b"])).unwrap(),
+ set(&["ga", "ga-b"])
+ );
+ assert_eq!(
+ closed_groups(&entries, &ids(&["ga-b-c"])).unwrap(),
+ set(&["ga", "ga-b", "ga-b-c"])
+ );
+ let rules = resolved(&entries);
+ assert!(
+ verdict(&rules, &set(&["ga", "ga-b"]), &set(&["gx"])),
+ "a parent rule covers the subgroup"
+ );
+ assert!(verdict(&rules, &set(&["ga"]), &set(&["gx"])));
+
+ let mut sub = entries.clone();
+ sub.extend([rule_full("on-a-b", &["ga-b"], All, &["gx"], Any, 0, false)]);
+ let rules = resolved(&sub);
+ let on_a_b = rules.iter().find(|r| r.id.as_ref() == "on-a-b").unwrap();
+ assert_eq!(on_a_b.src.groups, set(&["ga", "ga-b"]));
+ assert_eq!(on_a_b.specificity(), 2);
+ assert!(
+ !verdict(&rules, &set(&["ga", "ga-b"]), &set(&["gx"])),
+ "the deny on a/b outranks the allow on a"
+ );
+ assert!(
+ verdict(&rules, &set(&["ga"]), &set(&["gx"])),
+ "plain a members are untouched"
+ );
+
+ // exact compares the closed set
+ let exact: BTreeMap<String, MicrosegEntry> = BTreeMap::from([
+ group("ga"),
+ subgroup("ga-b", "ga"),
+ group("gx"),
+ rule_full("just-a", &["ga"], Exact, &["gx"], Any, 0, true),
+ rule_full("just-a-b", &["ga-b"], Exact, &["gx"], Any, 0, true),
+ ]);
+ let rules = resolved(&exact);
+ let just_a = rules.iter().find(|r| r.id.as_ref() == "just-a").unwrap();
+ let just_a_b = rules.iter().find(|r| r.id.as_ref() == "just-a-b").unwrap();
+ assert!(just_a.src.fires(&set(&["ga"])) && !just_a.src.fires(&set(&["ga", "ga-b"])));
+ assert!(just_a_b.src.fires(&set(&["ga", "ga-b"])) && !just_a_b.src.fires(&set(&["ga"])));
+ }
+
+ #[test]
+ fn any_predicates_stay_as_written() {
+ use PredicateMatch::Any;
+ let entries: BTreeMap<String, MicrosegEntry> = BTreeMap::from([
+ group("ga"),
+ subgroup("ga-b", "ga"),
+ group("gx"),
+ rule_full("any-a", &["ga"], Any, &["gx"], Any, 0, true),
+ ]);
+ let rules = resolved(&entries);
+ assert!(
+ verdict(&rules, &set(&["ga", "ga-b"]), &set(&["gx"])),
+ "an any on the parent covers the subgroup"
+ );
+ assert!(verdict(&rules, &set(&["ga"]), &set(&["gx"])));
+
+ let mut sub = entries.clone();
+ sub.remove("any-a");
+ sub.extend([rule_full("any-a-b", &["ga-b"], Any, &["gx"], Any, 0, true)]);
+ let rules = resolved(&sub);
+ assert_eq!(rules[0].src.groups, set(&["ga-b"]));
+ assert_eq!(rules[0].specificity(), 0);
+ assert!(verdict(&rules, &set(&["ga", "ga-b"]), &set(&["gx"])));
+ assert!(
+ !verdict(&rules, &set(&["ga"]), &set(&["gx"])),
+ "plain a members are not covered"
+ );
+
+ let mut with_untagged = entries.clone();
+ with_untagged.extend([rule_full(
+ "any-u",
+ &[UNTAGGED_GROUP, "ga-b"],
+ Any,
+ &["gx"],
+ Any,
+ 0,
+ true,
+ )]);
+ let rules = resolved(&with_untagged);
+ let any_u = rules.iter().find(|r| r.id.as_ref() == "any-u").unwrap();
+ assert_eq!(any_u.src.groups, set(&[UNTAGGED_GROUP, "ga-b"]));
+ }
+
+ #[test]
+ fn every_covers_each_group_set_but_untagged() {
+ use PredicateMatch::{Any, Every};
+ let entries: BTreeMap<String, MicrosegEntry> = BTreeMap::from([
+ group("ga"),
+ group("gx"),
+ rule_full("to-x", &[], Every, &["gx"], Any, 0, true),
+ ]);
+ validate(&entries).expect("valid");
+ let rules = resolved(&entries);
+ assert_eq!(rules[0].specificity(), 0);
+ assert!(verdict(&rules, &set(&["ga"]), &set(&["gx"])));
+ assert!(verdict(&rules, &set(&["gx"]), &set(&["gx"])));
+ assert!(
+ !verdict(&rules, &set(&[UNTAGGED_GROUP]), &set(&["gx"])),
+ "unstamped traffic still needs a rule naming untagged"
+ );
+
+ let mut listed = entries.clone();
+ listed.extend([rule_full("bad", &["ga"], Every, &["gx"], Any, 0, true)]);
+ assert!(validate(&listed).is_err(), "every takes no groups");
+ }
+
+ #[test]
+ fn specificity_breaks_priority_ties() {
+ use PredicateMatch::{All, Any};
+ let base: BTreeMap<String, MicrosegEntry> = BTreeMap::from([
+ group("ga"),
+ subgroup("ga-b", "ga"),
+ group("gv"),
+ group("gu"),
+ group("go"),
+ group("gs"),
+ group("gx"),
+ rule_full("deny-vu", &["gv", "gu"], All, &["gs"], All, 5, false),
+ rule_full("allow-ao", &["ga", "go"], All, &["gx"], All, 5, true),
+ ]);
+ let dst = set(&["gs", "gx"]);
+ let plain = set(&["ga", "gv", "gu", "go"]);
+ let sub = set(&["ga", "ga-b", "gv", "gu", "go"]);
+
+ // two unrelated rules of equal specificity tie and deny, for ga and ga-b members alike
+ let rules = resolved(&base);
+ assert!(!verdict(&rules, &plain, &dst));
+ assert!(!verdict(&rules, &sub, &dst));
+
+ // a rule written for ga-b is longer once closed and wins the tie, for ga-b members only
+ let mut more = base.clone();
+ more.extend([rule_full(
+ "allow-abo",
+ &["ga-b", "go"],
+ All,
+ &["gx"],
+ All,
+ 5,
+ true,
+ )]);
+ let rules = resolved(&more);
+ assert!(verdict(&rules, &sub, &dst));
+ assert!(!verdict(&rules, &plain, &dst));
+
+ // equally specific rules still AND their verdicts
+ more.extend([rule_full(
+ "deny-abo",
+ &["ga-b", "go"],
+ All,
+ &["gx"],
+ All,
+ 5,
+ false,
+ )]);
+ let rules = resolved(&more);
+ assert!(!verdict(&rules, &sub, &dst));
+
+ // an any counts nothing, so an all carve-out beats a broad any deny at the same priority
+ let carve: BTreeMap<String, MicrosegEntry> = BTreeMap::from([
+ group("web"),
+ group("app"),
+ group("prod"),
+ rule_full("broad", &["web"], Any, &["app"], Any, 5, false),
+ rule_full(
+ "carve",
+ &["prod", "web"],
+ All,
+ &["prod", "app"],
+ All,
+ 5,
+ true,
+ ),
+ ]);
+ let rules = resolved(&carve);
+ assert!(verdict(
+ &rules,
+ &set(&["web", "prod"]),
+ &set(&["app", "prod"])
+ ));
+ assert!(!verdict(&rules, &set(&["web"]), &set(&["app"])));
+
+ // priority still comes first
+ let mut prio = carve.clone();
+ prio.extend([rule_full("top", &["web"], Any, &["app"], Any, 6, false)]);
+ let rules = resolved(&prio);
+ assert!(!verdict(
+ &rules,
+ &set(&["web", "prod"]),
+ &set(&["app", "prod"])
+ ));
+ }
+
+ #[test]
+ fn a_narrowed_rule_runs_ahead_of_a_more_specific_whole_pair_rule() {
+ use PredicateMatch::All;
+ let mut entries: BTreeMap<String, MicrosegEntry> = BTreeMap::from([
+ group("web"),
+ group("db"),
+ group("prod"),
+ rule_full("whole", &["prod", "web"], All, &["db"], All, 5, false),
+ narrowed("pg", &["web"], &["db"], 5, true, Some("tcp"), Some("5432")),
+ ]);
+ let src = set(&["prod", "web"]);
+ let dst = set(&["db"]);
+
+ // specificity orders within a kind only, so the narrowed allow is checked although the
+ // deny of the same priority names more groups
+ let rules = resolved(&entries);
+ let folded = fold(firing_rules(&rules, &src, &dst));
+ let order: Vec<&str> = folded.checks.iter().map(|r| r.id.as_ref()).collect();
+ assert_eq!(order, ["pg"]);
+ assert!(!folded.allow);
+
+ // a narrowed rule one priority below the whole-pair rule is never reached
+ entries.extend([narrowed(
+ "pg",
+ &["web"],
+ &["db"],
+ 4,
+ true,
+ Some("tcp"),
+ Some("5432"),
+ )]);
+ let rules = resolved(&entries);
+ let folded = fold(firing_rules(&rules, &src, &dst));
+ assert!(folded.checks.is_empty());
+ assert!(!folded.allow);
+ }
+
+ #[test]
+ fn validate_rejects_bad_parents() {
+ let with = |entries: Vec<(String, MicrosegEntry)>| validate(&entries.into_iter().collect());
+ with(vec![group("ga"), subgroup("gb", "nope")]).expect_err("an unknown parent");
+ with(vec![subgroup("ga", "ga")]).expect_err("a group as its own parent");
+ with(vec![subgroup("ga", "gb"), subgroup("gb", "ga")]).expect_err("a cycle");
+ with(vec![
+ subgroup("ga", "gb"),
+ subgroup("gb", "gc"),
+ subgroup("gc", "ga"),
+ ])
+ .expect_err("a longer cycle");
+ serde_json::from_str::<GroupSection>(r#"{"id":"ga","parent":"untagged"}"#)
+ .expect_err("untagged as a parent");
+ with(vec![
+ group("ga"),
+ subgroup("gb", "ga"),
+ subgroup("gc", "gb"),
+ ])
+ .expect("a chain");
+ }
}
diff --git a/proxmox-ve-config/src/sdn/microseg/types.rs b/proxmox-ve-config/src/sdn/microseg/types.rs
index ee0a03beec51..39f80764e5e0 100644
--- a/proxmox-ve-config/src/sdn/microseg/types.rs
+++ b/proxmox-ve-config/src/sdn/microseg/types.rs
@@ -77,6 +77,12 @@ api_string_type! {
#[api(
properties: {
+ parent: {
+ type: String,
+ optional: true,
+ format: &MICROSEG_GROUP_NAME_FORMAT,
+ description: "The group this one is a subgroup of. Membership in a subgroup implies membership in every group above it.",
+ },
comment: {
type: String,
optional: true,
@@ -87,9 +93,15 @@ api_string_type! {
)]
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
/// A microseg group. An interface is assigned a set of groups, which resolves to a wire identity.
+///
+/// Groups may form a hierarchy through `parent`. A member of a subgroup is a member of every
+/// group above it, so an `any` or `all` rule on a group covers its subgroups. An `exact`
+/// rule compares the whole set and does not.
pub struct GroupSection {
pub(crate) id: GroupName,
#[serde(default, skip_serializing_if = "Option::is_none")]
+ pub(crate) parent: Option<GroupName>,
+ #[serde(default, skip_serializing_if = "Option::is_none")]
pub(crate) comment: Option<String>,
}
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-ve-rs v3 11/45] ve-config: firewall: expose protocol, icmp type and code numbers
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (9 preceding siblings ...)
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 10/45] ve-config: sdn: microseg: add group hierarchy Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 12/45] ve-config: sdn: microseg: add the desired state Hannes Laimer
` (33 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
A packet carries its protocol, ICMP type and code as numbers, while an
ICMP type or code may be given by name. The tables that map those
names to numbers are private. Expose the numbers, so a caller does not
need its own copy of the tables.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
.../src/firewall/types/rule_match.rs | 59 +++++++++++++++++++
1 file changed, 59 insertions(+)
diff --git a/proxmox-ve-config/src/firewall/types/rule_match.rs b/proxmox-ve-config/src/firewall/types/rule_match.rs
index 8e9c0c69efe8..e408b68a3b1a 100644
--- a/proxmox-ve-config/src/firewall/types/rule_match.rs
+++ b/proxmox-ve-config/src/firewall/types/rule_match.rs
@@ -327,6 +327,20 @@ impl Protocol {
}))
}
+ pub fn number(&self) -> Option<u8> {
+ match self {
+ Self::Dccp(_) => Some(33),
+ Self::Sctp(_) => Some(132),
+ Self::Tcp(_) => Some(6),
+ Self::Udp(_) => Some(17),
+ Self::UdpLite(_) => Some(136),
+ Self::Icmp(_) => Some(1),
+ Self::Icmpv6(_) => Some(58),
+ Self::Named(_) => None,
+ Self::Numeric(number) => Some(*number),
+ }
+ }
+
pub fn family(&self) -> Option<Family> {
match self {
Self::Icmp(_) => Some(Family::V4),
@@ -580,6 +594,23 @@ const ICMP_TYPES: [(&str, u8); 15] = sorted!([
("timestamp-request", 13),
]);
+fn table_number(table: &[(&str, u8)], name: &str) -> u8 {
+ let index = table
+ .binary_search_by(|v| v.0.cmp(name))
+ .expect("every named type and code is built from a name in its table");
+ table[index].1
+}
+
+impl IcmpType {
+ pub fn number(&self) -> Option<u8> {
+ match self {
+ IcmpType::Numeric(ty) => Some(*ty),
+ IcmpType::Named(name) => Some(table_number(&ICMP_TYPES, name)),
+ IcmpType::Any => None,
+ }
+ }
+}
+
impl std::str::FromStr for IcmpType {
type Err = Error;
@@ -628,6 +659,15 @@ const ICMP_CODES: [(&str, u8); 7] = sorted!([
("prot-unreachable", 2),
]);
+impl IcmpCode {
+ pub fn number(&self) -> u8 {
+ match self {
+ IcmpCode::Numeric(code) => *code,
+ IcmpCode::Named(name) => table_number(&ICMP_CODES, name),
+ }
+ }
+}
+
impl std::str::FromStr for IcmpCode {
type Err = Error;
@@ -800,6 +840,16 @@ const ICMPV6_TYPES: [(&str, u8); 19] = sorted!([
("time-exceeded", 3),
]);
+impl Icmpv6Type {
+ pub fn number(&self) -> Option<u8> {
+ match self {
+ Icmpv6Type::Numeric(ty) => Some(*ty),
+ Icmpv6Type::Named(name) => Some(table_number(&ICMPV6_TYPES, name)),
+ Icmpv6Type::Any => None,
+ }
+ }
+}
+
impl std::str::FromStr for Icmpv6Type {
type Err = Error;
@@ -847,6 +897,15 @@ const ICMPV6_CODES: [(&str, u8); 6] = sorted!([
("reject-route", 6),
]);
+impl Icmpv6Code {
+ pub fn number(&self) -> u8 {
+ match self {
+ Icmpv6Code::Numeric(code) => *code,
+ Icmpv6Code::Named(name) => table_number(&ICMPV6_CODES, name),
+ }
+ }
+}
+
impl std::str::FromStr for Icmpv6Code {
type Err = Error;
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-ve-rs v3 12/45] ve-config: sdn: microseg: add the desired state
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (10 preceding siblings ...)
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 11/45] ve-config: firewall: expose protocol, icmp type and code numbers Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 13/45] microseg: add a crate that applies the policy to a node's guest NICs Hannes Laimer
` (32 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
The data plane needs the policy as the id each assigned guest NIC
stamps and, for every pair of ids, the verdict and the narrowed rules
to check. That form comes from the running config and the guest
inventory alone and does not depend on how a host enforces it. So it
is built here, next to the model.
The pairs cover every allocated id, whether a NIC uses it or not. So
a NIC that comes up later needs only its own id, which resolves
without folding the pairs.
Denies alone need no pair, a missing pair denies. The exception is a
narrowed deny of ICMPv6 where the reverse pair allows or the source is
untagged, since the data plane passes neighbor discovery there.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
.../src/sdn/microseg/identity.rs | 15 +-
proxmox-ve-config/src/sdn/microseg/mod.rs | 44 +-
proxmox-ve-config/src/sdn/microseg/state.rs | 612 ++++++++++++++++++
3 files changed, 657 insertions(+), 14 deletions(-)
create mode 100644 proxmox-ve-config/src/sdn/microseg/state.rs
diff --git a/proxmox-ve-config/src/sdn/microseg/identity.rs b/proxmox-ve-config/src/sdn/microseg/identity.rs
index eb8317076b44..ef1ea10f4260 100644
--- a/proxmox-ve-config/src/sdn/microseg/identity.rs
+++ b/proxmox-ve-config/src/sdn/microseg/identity.rs
@@ -179,6 +179,13 @@ impl Registry {
total: u16::MAX as usize,
}
}
+
+ pub(crate) fn ids_by_signature(&self, policies: &[Policy]) -> HashMap<Signature, u16> {
+ self.classes
+ .iter()
+ .map(|(&id, rep)| (signature(rep, policies), id))
+ .collect()
+ }
}
/// Recompute the registry from the sets the assignments name and the policies, fresh-first against
@@ -295,11 +302,9 @@ mod tests {
}
fn id_of(reg: &Registry, set: &GroupSet, policies: &[Policy]) -> Option<u16> {
- let sig = signature(set, policies);
- reg.classes
- .iter()
- .find(|(_, rep)| signature(rep, policies) == sig)
- .map(|(&id, _)| id)
+ reg.ids_by_signature(policies)
+ .get(&signature(set, policies))
+ .copied()
}
fn sets(xs: &[&[&str]]) -> BTreeSet<GroupSet> {
diff --git a/proxmox-ve-config/src/sdn/microseg/mod.rs b/proxmox-ve-config/src/sdn/microseg/mod.rs
index 1744e3e5fc02..be48b890206f 100644
--- a/proxmox-ve-config/src/sdn/microseg/mod.rs
+++ b/proxmox-ve-config/src/sdn/microseg/mod.rs
@@ -10,6 +10,7 @@
pub mod api;
pub mod identity;
+pub mod state;
mod types;
pub use identity::PredicateMatch;
@@ -383,6 +384,19 @@ fn assignments(entries: &BTreeMap<String, MicrosegEntry>) -> Vec<Assignment<'_>>
assignments
}
+/// The NIC does not have to be among the guest's `nics`, so a NIC the inventory does not list yet
+/// resolves too.
+fn assigned_groups<'a>(
+ entries: &'a BTreeMap<String, MicrosegEntry>,
+ guest: &GuestInfo,
+ iface: u32,
+) -> Option<&'a [MicrosegId]> {
+ assignments(entries)
+ .into_iter()
+ .find(|assignment| assignment.covers(guest, iface))
+ .map(|assignment| assignment.groups)
+}
+
/// One NIC of the inventory with the assignment that covers it first and the groups that
/// assignment binds.
#[derive(Debug)]
@@ -568,7 +582,7 @@ pub fn assignment_order(entries: &BTreeMap<String, MicrosegEntry>) -> HashMap<St
}
#[derive(Debug)]
-pub struct ResolvedRule {
+struct ResolvedRule {
id: MicrosegId,
src: identity::Predicate,
dst: identity::Predicate,
@@ -585,6 +599,21 @@ impl ResolvedRule {
!self.protocols.is_empty()
}
+ /// A narrowed deny that can match neighbor discovery.
+ fn can_deny_nd(&self) -> bool {
+ !self.allow
+ && self.protocols.iter().any(|protocol| match protocol {
+ Protocol::Icmpv6(icmp) => matches!(
+ icmp.ty(),
+ None | Some(Icmpv6Type::Any | Icmpv6Type::Numeric(135 | 136))
+ | Some(Icmpv6Type::Named(
+ "nd-neighbor-solicit" | "nd-neighbor-advert"
+ ))
+ ),
+ _ => false,
+ })
+ }
+
fn specificity(&self) -> usize {
specificity(&self.src, &self.dst)
}
@@ -604,9 +633,7 @@ fn reversed_protocol(protocol: &Protocol) -> Protocol {
}
}
-pub fn resolved_rules(
- entries: &BTreeMap<String, MicrosegEntry>,
-) -> Result<Vec<ResolvedRule>, Error> {
+fn resolved_rules(entries: &BTreeMap<String, MicrosegEntry>) -> Result<Vec<ResolvedRule>, Error> {
let mut resolved = Vec::new();
for rule in rule_entries(entries) {
let context = || format!("rule '{}'", rule.id);
@@ -635,10 +662,9 @@ pub fn resolved_rules(
}
#[derive(Debug)]
-pub struct Folded<'a> {
- /// Narrowed rules in evaluation order. The first one whose protocol match fires decides.
- pub checks: Vec<&'a ResolvedRule>,
- pub allow: bool,
+struct Folded<'a> {
+ checks: Vec<&'a ResolvedRule>,
+ allow: bool,
}
/// Fold the rules that fire on one `(src_id, dst_id)` pair into its verdict and checks. The
@@ -646,7 +672,7 @@ pub struct Folded<'a> {
/// specific of those rules decide, and a single deny among them wins. Without such a rule the pair
/// denies. The checks are the narrowed rules from the top priority down to that one, the more
/// specific first and denies ahead of allows within a priority.
-pub fn fold<'a>(firing: impl IntoIterator<Item = &'a ResolvedRule>) -> Folded<'a> {
+fn fold<'a>(firing: impl IntoIterator<Item = &'a ResolvedRule>) -> Folded<'a> {
let mut firing: Vec<&ResolvedRule> = firing.into_iter().collect();
firing.sort_by(|a, b| evaluation_order(a, b));
diff --git a/proxmox-ve-config/src/sdn/microseg/state.rs b/proxmox-ve-config/src/sdn/microseg/state.rs
new file mode 100644
index 000000000000..1392860f5062
--- /dev/null
+++ b/proxmox-ve-config/src/sdn/microseg/state.rs
@@ -0,0 +1,612 @@
+//! The mechanism-independent form of the microseg policy, built from the running config.
+//!
+//! It holds the wire id each managed guest NIC stamps and the folded `(src_id, dst_id)` pairs.
+
+use std::collections::{BTreeMap, BTreeSet, HashMap};
+
+use anyhow::{Context, bail};
+
+use super::identity;
+use super::identity::UNTAGGED_GROUP;
+use super::types::{GuestInfo, MicrosegEntry, MicrosegId, MicrosegRunningConfig};
+use super::{
+ ResolvedRule, assigned_groups, assigned_nics, build_policies, closed_groups, fold,
+ resolved_rules,
+};
+use crate::firewall::types::rule_match::Protocol;
+
+#[derive(Debug)]
+pub struct DesiredState {
+ /// The policy has an assignment, so there is something to enforce even on a node without an
+ /// assigned NIC.
+ pub configured: bool,
+ /// The pairs an allow fires on or where a deny of neighbor discovery has to hold, left out
+ /// when their fold neither admits nor checks anything. A missing pair is a deny, the data
+ /// plane's default.
+ pub pairs: Vec<Pair>,
+ /// The narrowed rules the pairs refer to by index.
+ pub checks: Vec<Check>,
+ /// The wire id each managed guest NIC should stamp.
+ pub nics: Vec<ResolvedNic>,
+}
+
+#[derive(Debug)]
+pub struct Pair {
+ pub src_id: u16,
+ pub dst_id: u16,
+ /// The indexes of the narrowed rules to run in order. The first whose match fires decides.
+ pub checks: Vec<usize>,
+ pub allow: bool,
+}
+
+/// A narrowed rule.
+#[derive(Debug)]
+pub struct Check {
+ pub protocols: Vec<Protocol>,
+ pub allow: bool,
+}
+
+#[derive(Debug)]
+pub struct ResolvedNic {
+ pub vmid: u32,
+ pub iface: u32,
+ /// The interface's wire id (the identity its group set resolves to).
+ pub id: u16,
+}
+
+/// Building the policies and the class signatures walks the whole config, so do it once per build
+/// instead of once per NIC.
+struct Resolver<'a> {
+ entries: &'a BTreeMap<String, MicrosegEntry>,
+ policies: Vec<identity::Policy>,
+ class_ids: HashMap<identity::Signature, u16>,
+}
+
+impl<'a> Resolver<'a> {
+ fn new(cfg: &'a MicrosegRunningConfig) -> anyhow::Result<Self> {
+ let entries = cfg.ids();
+ let policies = build_policies(entries).context("build policies")?;
+ let class_ids = cfg.identities().ids_by_signature(&policies);
+ Ok(Self {
+ entries,
+ policies,
+ class_ids,
+ })
+ }
+
+ /// The render allocates a class for every set an assignment names, so a set no class carries is
+ /// a broken registry rather than a NIC to leave untagged.
+ fn id_of(&self, vmid: u32, iface: u32, groups: &[MicrosegId]) -> anyhow::Result<u16> {
+ let set = closed_groups(self.entries, groups)
+ .with_context(|| format!("NIC vm{vmid} net{iface}"))?;
+ let Some(&id) = self
+ .class_ids
+ .get(&identity::signature(&set, &self.policies))
+ else {
+ bail!("NIC vm{vmid} net{iface} has no allocated identity");
+ };
+ Ok(id)
+ }
+
+ fn nics(&self, inventory: &[GuestInfo]) -> anyhow::Result<Vec<ResolvedNic>> {
+ assigned_nics(self.entries, inventory)
+ .into_iter()
+ .map(|nic| {
+ Ok(ResolvedNic {
+ vmid: nic.vmid,
+ iface: nic.iface,
+ id: self.id_of(nic.vmid, nic.iface, nic.groups)?,
+ })
+ })
+ .collect()
+ }
+}
+
+/// The wire id a guest NIC stamps, `None` when no assignment covers it. Skips the pair fold, which
+/// one NIC does not need.
+pub fn resolve_iface(
+ cfg: &MicrosegRunningConfig,
+ guest: &GuestInfo,
+ iface: u32,
+) -> anyhow::Result<Option<u16>> {
+ let Some(groups) = assigned_groups(cfg.ids(), guest, iface) else {
+ return Ok(None);
+ };
+ Resolver::new(cfg)?
+ .id_of(guest.vmid, iface, groups)
+ .map(Some)
+}
+
+impl DesiredState {
+ /// `pairs_max` is the size of the data plane's pairs map. A policy that needs more pairs
+ /// returns Err before any pair is folded.
+ pub fn build(
+ cfg: &MicrosegRunningConfig,
+ inventory: &[GuestInfo],
+ pairs_max: usize,
+ ) -> anyhow::Result<Self> {
+ let resolver = Resolver::new(cfg)?;
+ let rules = resolved_rules(resolver.entries).context("resolve rules")?;
+ let nics = resolver.nics(inventory)?;
+ let firing = firing_pairs(cfg.identities().classes(), &rules, pairs_max)?;
+ let (pairs, checks) = fold_pairs(&firing);
+ Ok(Self {
+ // a class exists for every group set an assignment names
+ configured: !cfg.identities().classes().is_empty(),
+ pairs,
+ checks,
+ nics,
+ })
+ }
+}
+
+/// The classes a rule fires on, ascending for the binary search.
+struct Reach {
+ sources: Vec<u16>,
+ destinations: Vec<u16>,
+}
+
+fn rule_reaches(classes: &BTreeMap<u16, identity::GroupSet>, rules: &[ResolvedRule]) -> Vec<Reach> {
+ // Every allocated class is folded, whether or not a NIC carries it right now, since a NIC
+ // that comes up later can only land in one of them. Only the sources include the untagged
+ // id, so an `untagged -> X` rule admits unstamped traffic. Enforcement runs at assigned
+ // NICs, whose sets never carry the untagged group, so no untagged destination is folded.
+ let untagged_rep = identity::GroupSet::from([UNTAGGED_GROUP.to_string()]);
+ let sources: Vec<(u16, &identity::GroupSet)> =
+ std::iter::once((identity::UNTAGGED_ID, &untagged_rep))
+ .chain(classes.iter().map(|(&id, rep)| (id, rep)))
+ .collect();
+ rules
+ .iter()
+ .map(|rule| Reach {
+ sources: sources
+ .iter()
+ .filter(|(_, rep)| rule.src.fires(rep))
+ .map(|&(id, _)| id)
+ .collect(),
+ destinations: classes
+ .iter()
+ .filter(|(_, rep)| rule.dst.fires(rep))
+ .map(|(&id, _)| id)
+ .collect(),
+ })
+ .collect()
+}
+
+/// Each candidate pair with the rules that fire on it, as `(src_id, dst_id, rule)`, grouped by
+/// pair in pair order. A candidate is a pair an allow fires on, or one where a narrowed deny
+/// of neighbor discovery has to hold. More candidates than `pairs_max` fail the build.
+fn firing_pairs<'r>(
+ classes: &BTreeMap<u16, identity::GroupSet>,
+ rules: &'r [ResolvedRule],
+ pairs_max: usize,
+) -> anyhow::Result<Vec<(u16, u16, &'r ResolvedRule)>> {
+ let reaches = rule_reaches(classes, rules);
+ let mut candidates = BTreeSet::new();
+ for (rule, reach) in rules.iter().zip(&reaches) {
+ if !rule.allow {
+ continue;
+ }
+ for &src_id in &reach.sources {
+ for &dst_id in &reach.destinations {
+ add_candidate(&mut candidates, (src_id, dst_id), pairs_max)?;
+ }
+ }
+ }
+ // Denies alone need no pair, since a missing pair denies. The exception is a narrowed deny
+ // that can match neighbor discovery. The data plane passes neighbor discovery when the
+ // reverse pair admits anything, and from an unstamped sender when the host sent it or the
+ // NIC sits behind a firewall bridge. So such a deny gets a pair wherever that pass applies.
+ for (rule, reach) in rules.iter().zip(&reaches) {
+ if !rule.can_deny_nd() {
+ continue;
+ }
+ for &src_id in &reach.sources {
+ for &dst_id in &reach.destinations {
+ // the reverse test can read the set while it grows, an added exception only
+ // matches the test of the allow candidate it was added for, which is in the
+ // set already
+ if src_id == identity::UNTAGGED_ID || candidates.contains(&(dst_id, src_id)) {
+ add_candidate(&mut candidates, (src_id, dst_id), pairs_max)?;
+ }
+ }
+ }
+ }
+ // a rule reaching more pairs than there are candidates is tested per candidate instead of
+ // expanded, so a deny over everything costs one pass
+ let mut firing = Vec::new();
+ for (rule, reach) in rules.iter().zip(&reaches) {
+ if reach.sources.len() * reach.destinations.len() <= candidates.len() {
+ for &src_id in &reach.sources {
+ for &dst_id in &reach.destinations {
+ if candidates.contains(&(src_id, dst_id)) {
+ firing.push((src_id, dst_id, rule));
+ }
+ }
+ }
+ } else {
+ for &(src_id, dst_id) in &candidates {
+ if reach.sources.binary_search(&src_id).is_ok()
+ && reach.destinations.binary_search(&dst_id).is_ok()
+ {
+ firing.push((src_id, dst_id, rule));
+ }
+ }
+ }
+ }
+ firing.sort_by_key(|&(src_id, dst_id, _)| (src_id, dst_id));
+ Ok(firing)
+}
+
+fn add_candidate(
+ candidates: &mut BTreeSet<(u16, u16)>,
+ pair: (u16, u16),
+ pairs_max: usize,
+) -> anyhow::Result<()> {
+ candidates.insert(pair);
+ if candidates.len() > pairs_max {
+ bail!(
+ "the policy needs more than {pairs_max} pairs, the pairs map holds at most {pairs_max}"
+ );
+ }
+ Ok(())
+}
+
+fn fold_pairs(firing: &[(u16, u16, &ResolvedRule)]) -> (Vec<Pair>, Vec<Check>) {
+ let mut pairs = Vec::new();
+ let mut checks = Vec::new();
+ // The two orientations of a bidirectional rule carry the rule's id and match opposite ports,
+ // so each of them is a check of its own.
+ let mut check_index: HashMap<(&MicrosegId, bool), usize> = HashMap::new();
+ for pair in firing.chunk_by(|a, b| (a.0, a.1) == (b.0, b.1)) {
+ let (src_id, dst_id, _) = pair[0];
+ let folded = fold(pair.iter().map(|&(_, _, rule)| rule));
+ if !folded.allow && folded.checks.is_empty() {
+ continue;
+ }
+ let cell_checks = folded
+ .checks
+ .iter()
+ .map(|rule| {
+ *check_index
+ .entry((&rule.id, rule.reverse))
+ .or_insert_with(|| {
+ checks.push(Check {
+ protocols: rule.protocols.clone(),
+ allow: rule.allow,
+ });
+ checks.len() - 1
+ })
+ })
+ .collect();
+ pairs.push(Pair {
+ src_id,
+ dst_id,
+ checks: cell_checks,
+ allow: folded.allow,
+ });
+ }
+ (pairs, checks)
+}
+
+#[cfg(test)]
+mod tests {
+ use super::super::tests::firing_rules;
+ use super::super::{Folded, render};
+ use super::*;
+
+ const WEB_AND_DB: &str = r#"{"next":3,"classes":{"1":["web"],"2":["db"]}}"#;
+
+ fn web_to_db(assignments: &str, identities: &str) -> MicrosegRunningConfig {
+ let json = format!(
+ r#"{{
+ "ids": {{
+ "web": {{"type":"group","id":"web"}},
+ "db": {{"type":"group","id":"db"}},
+ "r1": {{"type":"rule","id":"r1","src":["web"],"dst":["db"],"allow":1}},
+ {assignments}
+ }},
+ "identities": {identities}
+ }}"#
+ );
+ serde_json::from_str(&json).expect("parse running config")
+ }
+
+ fn vm100_in(group: &str) -> String {
+ format!(
+ r#""vm100i0": {{"type":"guest-assignment","id":"vm100i0","vmid":100,"iface":0,"groups":["{group}"]}}"#
+ )
+ }
+
+ fn guest(vmid: u32, tags: &[&str], nics: &[u32]) -> GuestInfo {
+ GuestInfo {
+ vmid,
+ tags: tags
+ .iter()
+ .map(|tag| tag.parse().expect("valid tag"))
+ .collect(),
+ name: None,
+ nics: nics.to_vec(),
+ }
+ }
+
+ fn stamped(state: &DesiredState) -> Vec<(u32, u32, u16)> {
+ state.nics.iter().map(|a| (a.vmid, a.iface, a.id)).collect()
+ }
+
+ /// Two tag assignments that both match a guest tagged `a` and `b`. By id `tdb` sorts ahead of
+ /// `tweb`.
+ fn two_tags(tweb: &str, tdb: &str) -> MicrosegRunningConfig {
+ let assignments = format!(
+ r#""tdb": {{"type":"tag-assignment","id":"tdb","tags":["b"],"groups":["db"]{tdb}}},
+ "tweb": {{"type":"tag-assignment","id":"tweb","tags":["a"],"groups":["web"]{tweb}}}"#
+ );
+ web_to_db(&assignments, WEB_AND_DB)
+ }
+
+ #[test]
+ fn resolve_iface_answers_one_nic() {
+ let cfg = web_to_db(&vm100_in("web"), WEB_AND_DB);
+ let vm100 = guest(100, &[], &[0]);
+ assert_eq!(resolve_iface(&cfg, &vm100, 0).expect("resolve"), Some(1));
+
+ assert_eq!(resolve_iface(&cfg, &vm100, 1).expect("resolve"), None);
+ assert_eq!(
+ resolve_iface(&cfg, &guest(404, &[], &[0]), 0).expect("resolve"),
+ None
+ );
+
+ let cfg = web_to_db(&vm100_in("db"), r#"{"next":2,"classes":{"1":["web"]}}"#);
+ assert!(resolve_iface(&cfg, &vm100, 0).is_err());
+ assert!(
+ DesiredState::build(&cfg, &[vm100], usize::MAX).is_err(),
+ "the full build agrees"
+ );
+ }
+
+ #[test]
+ fn first_match_takes_one_assignment_not_a_union() {
+ let cfg = two_tags(r#","order":1"#, r#","order":2"#);
+ let tagged = guest(100, &["a", "b"], &[0]);
+ let state = DesiredState::build(&cfg, std::slice::from_ref(&tagged), usize::MAX)
+ .expect("build state");
+
+ // the earlier tweb decides alone, a union {web, db} would have no class at all
+ assert_eq!(stamped(&state), [(100, 0, 1)]);
+ assert_eq!(resolve_iface(&cfg, &tagged, 0).expect("resolve"), Some(1));
+ }
+
+ #[test]
+ fn unmatched_assignment_still_gets_a_class() {
+ let cfg = two_tags("", "");
+ let rendered = render(&identity::Registry::new(), cfg.ids().clone(), 0).expect("render");
+ let state = DesiredState::build(&rendered, &[], usize::MAX).expect("build state");
+
+ assert!(stamped(&state).is_empty());
+ assert_eq!(rendered.describe_class(1), "1 (db)");
+ assert_eq!(rendered.describe_class(2), "2 (web)");
+ assert_eq!(
+ state.pairs.len(),
+ 1,
+ "web -> db is folded for the empty classes"
+ );
+ assert_eq!(
+ resolve_iface(&rendered, &guest(100, &["b"], &[]), 0).expect("resolve"),
+ Some(1)
+ );
+ }
+
+ /// Renders a config of the groups, each with its parent, the rules, each given as its JSON
+ /// fields without the id, and one guest assignment per group list.
+ fn policy(
+ groups: &[(String, Option<String>)],
+ rules: &[String],
+ assigned: &[Vec<String>],
+ ) -> MicrosegRunningConfig {
+ let mut ids = Vec::new();
+ for (name, parent) in groups {
+ let parent = parent
+ .as_ref()
+ .map(|parent| format!(r#","parent":"{parent}""#))
+ .unwrap_or_default();
+ ids.push(format!(
+ r#""{name}":{{"type":"group","id":"{name}"{parent}}}"#
+ ));
+ }
+ for (n, fields) in rules.iter().enumerate() {
+ ids.push(format!(r#""r{n}":{{"type":"rule","id":"r{n}",{fields}}}"#));
+ }
+ for (n, groups) in assigned.iter().enumerate() {
+ let vmid = 100 + n;
+ ids.push(format!(
+ r#""vm{vmid}":{{"type":"guest-assignment","id":"vm{vmid}","vmid":{vmid},"groups":{groups:?}}}"#
+ ));
+ }
+ let json = format!(r#"{{"ids":{{{}}}}}"#, ids.join(","));
+ let cfg: MicrosegRunningConfig = serde_json::from_str(&json).expect("parse");
+ render(&identity::Registry::new(), cfg.ids().clone(), 0).expect("render")
+ }
+
+ fn tenants(count: usize) -> MicrosegRunningConfig {
+ let mut groups = vec![("mon".to_string(), None)];
+ let mut rules = Vec::new();
+ let mut assigned = vec![vec!["mon".to_string()]];
+ for t in 0..count {
+ groups.push((format!("t{t}"), None));
+ for tier in ["web", "app", "db"] {
+ groups.push((format!("t{t}-{tier}"), Some(format!("t{t}"))));
+ assigned.push(vec![format!("t{t}-{tier}")]);
+ }
+ rules.push(format!(
+ r#""src":["t{t}-web"],"dst":["t{t}-app"],"allow":1,"bidirectional":1,"prio":10"#
+ ));
+ rules.push(format!(
+ r#""src":["t{t}-app"],"dst":["t{t}-db"],"allow":1,"bidirectional":1,"prio":10"#
+ ));
+ rules.push(format!(
+ r#""src":["mon"],"dst":["t{t}"],"allow":1,"bidirectional":1,"prio":20,"proto":"tcp","dport":"9100""#
+ ));
+ }
+ policy(&groups, &rules, &assigned)
+ }
+
+ fn mesh(count: usize) -> MicrosegRunningConfig {
+ mesh_under(count, None)
+ }
+
+ /// The mesh with a deny over all of it at `deny_prio`, between the allow and the narrowed
+ /// denies, so every pair stays a candidate and only the diagonal is carried.
+ fn mesh_under(count: usize, deny_prio: Option<u16>) -> MicrosegRunningConfig {
+ let mut groups = vec![("mesh".to_string(), None)];
+ let mut rules = vec![r#""src":["mesh"],"dst":["mesh"],"allow":1,"prio":10"#.to_string()];
+ if let Some(prio) = deny_prio {
+ rules.push(format!(
+ r#""src":["mesh"],"dst":["mesh"],"allow":0,"prio":{prio}"#
+ ));
+ }
+ let mut assigned = Vec::new();
+ for k in 0..count {
+ groups.push((format!("g{k}"), None));
+ rules.push(format!(
+ r#""src":["g{k}"],"dst":["g{k}"],"allow":0,"prio":20,"proto":"tcp","dport":"22""#
+ ));
+ assigned.push(vec![format!("g{k}"), "mesh".to_string()]);
+ }
+ policy(&groups, &rules, &assigned)
+ }
+
+ fn mixed() -> MicrosegRunningConfig {
+ let groups: Vec<_> = ["prod", "staging", "web", "app", "db", "mon"]
+ .map(|name| (name.to_string(), None))
+ .into();
+ let rules = [
+ r#""src":["web","prod"],"src-match":"all","dst":["app","prod"],"dst-match":"all","allow":1,"prio":10"#,
+ r#""src":["web"],"dst":["app"],"allow":0,"prio":20,"proto":"tcp","dport":"8080""#,
+ r#""src":["web"],"dst":["db"],"allow":0,"prio":5"#,
+ r#""src":["web","staging"],"src-match":"all","dst":["db"],"allow":1,"prio":5"#,
+ r#""src":["db"],"src-match":"exact","dst":["app"],"allow":1,"prio":7"#,
+ r#""src":["mon"],"dst":["web","app","db"],"allow":1,"bidirectional":1,"prio":3,"proto":"icmp""#,
+ r#""src":["untagged"],"dst":["web"],"allow":1,"prio":1"#,
+ r#""src":["app"],"dst":["web"],"allow":1,"prio":30"#,
+ r#""src":["app"],"dst":["web"],"allow":0,"prio":30"#,
+ ]
+ .map(str::to_string);
+ let assigned = [
+ vec!["web", "prod"],
+ vec!["app", "prod"],
+ vec!["db", "prod"],
+ vec!["web", "staging"],
+ vec!["app", "staging"],
+ vec!["db"],
+ vec!["mon"],
+ ]
+ .map(|groups| groups.into_iter().map(str::to_string).collect());
+ policy(&groups, &rules, &assigned)
+ }
+
+ /// A pair as `(src_id, dst_id, checks, allow)`, each check by its protocols and verdict.
+ type CellContent = (u16, u16, Vec<(Vec<Protocol>, bool)>, bool);
+
+ fn admits(folded: &Folded) -> bool {
+ folded.allow || folded.checks.iter().any(|rule| rule.allow)
+ }
+
+ /// What `build` has to carry, folded pair by pair instead of rule by rule.
+ fn every_pair(cfg: &MicrosegRunningConfig) -> Vec<CellContent> {
+ let rules = resolved_rules(cfg.ids()).expect("resolve rules");
+ let classes = cfg.identities().classes();
+ let untagged_rep = identity::GroupSet::from([UNTAGGED_GROUP.to_string()]);
+ let sources = std::iter::once((identity::UNTAGGED_ID, &untagged_rep))
+ .chain(classes.iter().map(|(&id, rep)| (id, rep)));
+ let mut pairs = Vec::new();
+ for (src_id, src_rep) in sources {
+ for (&dst_id, dst_rep) in classes {
+ let folded = fold(firing_rules(&rules, src_rep, dst_rep));
+ let rescued = || {
+ src_id == identity::UNTAGGED_ID
+ || admits(&fold(firing_rules(&rules, dst_rep, src_rep)))
+ };
+ let denies_nd = folded.checks.iter().any(|rule| rule.can_deny_nd());
+ if admits(&folded) || (denies_nd && rescued()) {
+ let checks = folded
+ .checks
+ .iter()
+ .map(|rule| (rule.protocols.clone(), rule.allow))
+ .collect();
+ pairs.push((src_id, dst_id, checks, folded.allow));
+ }
+ }
+ }
+ pairs
+ }
+
+ #[test]
+ fn build_folds_the_cells_every_pair_would() {
+ for (name, cfg, count) in [
+ ("tenants", tenants(6), 6 * 10),
+ ("mesh", mesh(9), 9 * 9),
+ ("mixed", mixed(), 1 + 2 + 2 + 6 + 6 + 2),
+ ] {
+ let state = DesiredState::build(&cfg, &[], usize::MAX).expect("build state");
+ let built: Vec<_> = state
+ .pairs
+ .iter()
+ .map(|pair| {
+ let checks = pair
+ .checks
+ .iter()
+ .map(|&index| {
+ let check = &state.checks[index];
+ (check.protocols.clone(), check.allow)
+ })
+ .collect();
+ (pair.src_id, pair.dst_id, checks, pair.allow)
+ })
+ .collect();
+ assert_eq!(built, every_pair(&cfg), "{name}");
+ assert_eq!(built.len(), count, "{name}");
+ }
+
+ // the bound counts the pairs an allow fires on, shadowed or not
+ let shadowed = mesh_under(9, Some(15));
+ let state = DesiredState::build(&shadowed, &[], usize::MAX).expect("build state");
+ assert_eq!(state.pairs.len(), 9, "only the diagonal is carried");
+ assert!(
+ DesiredState::build(&shadowed, &[], 80).is_err(),
+ "81 candidates are refused at a ceiling of 80"
+ );
+ }
+
+ #[test]
+ fn denies_alone_need_no_pair() {
+ let groups = [("web".to_string(), None), ("db".to_string(), None)];
+ let assigned = [vec!["web".to_string()], vec!["db".to_string()]];
+ let allow = r#""src":["web"],"dst":["db"],"allow":1,"prio":10"#.to_string();
+ let every =
+ r#""src":[],"src-match":"every","dst":[],"dst-match":"every","allow":0,"prio":20"#;
+ let pairs = |rules: &[String], pairs_max| {
+ let state = DesiredState::build(&policy(&groups, rules, &assigned), &[], pairs_max)
+ .expect("build state");
+ state
+ .pairs
+ .iter()
+ .map(|pair| (pair.src_id, pair.dst_id))
+ .collect::<Vec<_>>()
+ };
+
+ // a deny over every pair makes none of them a candidate on its own
+ let ssh = format!(r#"{every},"proto":"tcp","dport":"22""#);
+ assert_eq!(pairs(&[allow.clone(), ssh], 1), [(2, 1)]);
+
+ // a deny of ICMPv6 is kept where the reverse pair allows, db -> web next to web -> db
+ let nd = format!(r#"{every},"proto":"ipv6-icmp""#);
+ assert_eq!(pairs(&[allow.clone(), nd], 2), [(1, 2), (2, 1)]);
+ let alias = format!(r#"{every},"proto":"IPv6-ICMP""#);
+ assert_eq!(pairs(&[allow, alias], 2), [(1, 2), (2, 1)]);
+
+ // and from untagged, whose neighbor discovery passes without a reverse pair
+ let untagged =
+ r#""src":["untagged"],"dst":["web"],"allow":0,"prio":20,"proto":"ipv6-icmp""#;
+ assert_eq!(pairs(&[untagged.to_string()], 1), [(0, 2)]);
+ }
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-ve-rs v3 13/45] microseg: add a crate that applies the policy to a node's guest NICs
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (11 preceding siblings ...)
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 12/45] ve-config: sdn: microseg: add the desired state Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 14/45] microseg: add a rule option that matches non-first fragments Hannes Laimer
` (31 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
The crate hands the microseg policy of the SDN running config to the
policy subsystem of proxmox-ebpf. A full pass applies it to every
guest NIC on the node, a plug to a single NIC that is about to come
up. A validation needs only the config and tells whether the data
plane can hold it.
Only the microseg block of the running config is parsed. A plug runs
for every guest NIC, so a zone or DHCP type that this node's
proxmox-ve-config does not know yet must not keep guests from
starting. The SDN apply on such a node fails either way.
A guest's NICs come from the interfaces on the node and not from its
config, since the config names a hot-plugged NIC only after the
hotplug.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Cargo.toml | 2 +
proxmox-microseg/Cargo.toml | 20 +++
proxmox-microseg/debian/changelog | 5 +
proxmox-microseg/debian/control | 46 +++++
proxmox-microseg/debian/copyright | 18 ++
proxmox-microseg/debian/debcargo.toml | 7 +
proxmox-microseg/debian/source/format | 1 +
proxmox-microseg/src/ebpf.rs | 235 +++++++++++++++++++++++++
proxmox-microseg/src/lib.rs | 66 +++++++
proxmox-microseg/src/nic.rs | 120 +++++++++++++
proxmox-microseg/src/running_config.rs | 26 +++
11 files changed, 546 insertions(+)
create mode 100644 proxmox-microseg/Cargo.toml
create mode 100644 proxmox-microseg/debian/changelog
create mode 100644 proxmox-microseg/debian/control
create mode 100644 proxmox-microseg/debian/copyright
create mode 100644 proxmox-microseg/debian/debcargo.toml
create mode 100644 proxmox-microseg/debian/source/format
create mode 100644 proxmox-microseg/src/ebpf.rs
create mode 100644 proxmox-microseg/src/lib.rs
create mode 100644 proxmox-microseg/src/nic.rs
create mode 100644 proxmox-microseg/src/running_config.rs
diff --git a/Cargo.toml b/Cargo.toml
index 4655b178b47a..d38e48897216 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -3,6 +3,7 @@ members = [
"proxmox-ve-config",
"proxmox-frr",
"proxmox-sdn-types",
+ "proxmox-microseg",
]
exclude = [
"build",
@@ -29,3 +30,4 @@ proxmox-network-types = { version = "1.1.2" }
proxmox-schema = { version = "5" }
proxmox-sdn-types = { version = "0.2", path = "proxmox-sdn-types" }
proxmox-serde = { version = "1.0.0" }
+proxmox-ve-config = { version = "0.10", path = "proxmox-ve-config" }
diff --git a/proxmox-microseg/Cargo.toml b/proxmox-microseg/Cargo.toml
new file mode 100644
index 000000000000..067b0f593990
--- /dev/null
+++ b/proxmox-microseg/Cargo.toml
@@ -0,0 +1,20 @@
+[package]
+name = "proxmox-microseg"
+description = "Proxmox VE SDN microsegmentation data plane"
+version = "0.1.0"
+authors.workspace = true
+edition.workspace = true
+license.workspace = true
+homepage.workspace = true
+exclude.workspace = true
+# proxmox-ebpf needs it
+rust-version = "1.88"
+
+[dependencies]
+anyhow = { workspace = true }
+log = "0.4"
+serde = { workspace = true, features = [ "derive" ] }
+serde_json = "1"
+
+proxmox-ebpf = { version = "0.1", features = [ "policy" ] }
+proxmox-ve-config = { workspace = true }
diff --git a/proxmox-microseg/debian/changelog b/proxmox-microseg/debian/changelog
new file mode 100644
index 000000000000..e8f8ee14dcf9
--- /dev/null
+++ b/proxmox-microseg/debian/changelog
@@ -0,0 +1,5 @@
+rust-proxmox-microseg (0.1.0) trixie; urgency=medium
+
+ * Initial release.
+
+ -- Proxmox Support Team <support@proxmox.com> Tue, 22 Sep 2026 16:02:32 +0200
diff --git a/proxmox-microseg/debian/control b/proxmox-microseg/debian/control
new file mode 100644
index 000000000000..35487025c2b7
--- /dev/null
+++ b/proxmox-microseg/debian/control
@@ -0,0 +1,46 @@
+Source: rust-proxmox-microseg
+Section: rust
+Priority: optional
+Build-Depends: debhelper-compat (= 13),
+ dh-sequence-cargo
+Build-Depends-Arch: cargo:native <!nocheck>,
+ rustc:native (>= 1.88) <!nocheck>,
+ libstd-rust-dev <!nocheck>,
+ librust-anyhow-1+default-dev <!nocheck>,
+ librust-log-0.4+default-dev <!nocheck>,
+ librust-proxmox-ebpf-0.1+default-dev <!nocheck>,
+ librust-proxmox-ebpf-0.1+policy-dev <!nocheck>,
+ librust-proxmox-ve-config-0.10+default-dev <!nocheck>,
+ librust-serde-1+default-dev <!nocheck>,
+ librust-serde-1+derive-dev <!nocheck>,
+ librust-serde-json-1+default-dev <!nocheck>
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Standards-Version: 4.7.2
+Vcs-Git: git://git.proxmox.com/git/proxmox-ve-rs.git
+Vcs-Browser: https://git.proxmox.com/?p=proxmox-ve-rs.git
+Homepage: https://proxmox.com
+X-Cargo-Crate: proxmox-microseg
+
+Package: librust-proxmox-microseg-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-anyhow-1+default-dev,
+ librust-log-0.4+default-dev,
+ librust-proxmox-ebpf-0.1+default-dev,
+ librust-proxmox-ebpf-0.1+policy-dev,
+ librust-proxmox-ve-config-0.10+default-dev,
+ librust-serde-1+default-dev,
+ librust-serde-1+derive-dev,
+ librust-serde-json-1+default-dev
+Provides:
+ librust-proxmox-microseg+default-dev (= ${binary:Version}),
+ librust-proxmox-microseg-0-dev (= ${binary:Version}),
+ librust-proxmox-microseg-0+default-dev (= ${binary:Version}),
+ librust-proxmox-microseg-0.1-dev (= ${binary:Version}),
+ librust-proxmox-microseg-0.1+default-dev (= ${binary:Version}),
+ librust-proxmox-microseg-0.1.0-dev (= ${binary:Version}),
+ librust-proxmox-microseg-0.1.0+default-dev (= ${binary:Version})
+Description: Proxmox VE SDN microsegmentation data plane - Rust source code
+ Source code for Debianized Rust crate "proxmox-microseg"
diff --git a/proxmox-microseg/debian/copyright b/proxmox-microseg/debian/copyright
new file mode 100644
index 000000000000..77952eba571e
--- /dev/null
+++ b/proxmox-microseg/debian/copyright
@@ -0,0 +1,18 @@
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+
+Files:
+ *
+Copyright: 2019 - 2026 Proxmox Server Solutions GmbH <support@proxmox.com>
+License: AGPL-3.0-or-later
+ This program is free software: you can redistribute it and/or modify it under
+ the terms of the GNU Affero General Public License as published by the Free
+ Software Foundation, either version 3 of the License, or (at your option) any
+ later version.
+ .
+ This program is distributed in the hope that it will be useful, but WITHOUT
+ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+ details.
+ .
+ You should have received a copy of the GNU Affero General Public License along
+ with this program. If not, see <https://www.gnu.org/licenses/>.
diff --git a/proxmox-microseg/debian/debcargo.toml b/proxmox-microseg/debian/debcargo.toml
new file mode 100644
index 000000000000..87a787e6d03e
--- /dev/null
+++ b/proxmox-microseg/debian/debcargo.toml
@@ -0,0 +1,7 @@
+overlay = "."
+crate_src_path = ".."
+maintainer = "Proxmox Support Team <support@proxmox.com>"
+
+[source]
+vcs_git = "git://git.proxmox.com/git/proxmox-ve-rs.git"
+vcs_browser = "https://git.proxmox.com/?p=proxmox-ve-rs.git"
diff --git a/proxmox-microseg/debian/source/format b/proxmox-microseg/debian/source/format
new file mode 100644
index 000000000000..89ae9db8f88b
--- /dev/null
+++ b/proxmox-microseg/debian/source/format
@@ -0,0 +1 @@
+3.0 (native)
diff --git a/proxmox-microseg/src/ebpf.rs b/proxmox-microseg/src/ebpf.rs
new file mode 100644
index 000000000000..02ea16d5a2ef
--- /dev/null
+++ b/proxmox-microseg/src/ebpf.rs
@@ -0,0 +1,235 @@
+use anyhow::{Context, bail};
+use proxmox_ebpf::policy::{self, Assignment, CHECK_MAX, CheckEntry, Family, Pair, Pass};
+use proxmox_ve_config::firewall::types::port::{PortEntry, PortList};
+use proxmox_ve_config::firewall::types::rule_match::{
+ IcmpCode, IcmpType, Icmpv6Code, Icmpv6Type, Ports, Protocol,
+};
+use proxmox_ve_config::sdn::microseg::MicrosegRunningConfig;
+use proxmox_ve_config::sdn::microseg::state::{self, DesiredState};
+
+use crate::nic::{self, LocalNics};
+
+pub use proxmox_ebpf::policy::{ExclusiveLock, PAIRS_MAX, SharedLock, lock_exclusive, lock_shared};
+
+fn port_bounds(list: Option<&PortList>) -> Vec<Option<(u16, u16)>> {
+ let Some(list) = list else {
+ // without a port list every port matches
+ return vec![None];
+ };
+ list.iter()
+ .map(|entry| {
+ Some(match entry {
+ PortEntry::Port(port) => (*port, *port),
+ PortEntry::Range(from, to) => (*from, *to),
+ })
+ })
+ .collect()
+}
+
+/// One entry per combination of source and destination range, since an entry carries a single range
+/// of each and both have to match.
+fn port_entries(ip_proto: u8, ports: &Ports, allow: bool) -> Vec<CheckEntry> {
+ let dports = port_bounds(ports.dport());
+ let mut entries = Vec::new();
+ for sport in port_bounds(ports.sport()) {
+ for &dport in &dports {
+ entries.push(CheckEntry {
+ ip_proto: Some(ip_proto),
+ sport,
+ dport,
+ allow,
+ ..Default::default()
+ });
+ }
+ }
+ entries
+}
+
+fn protocol_entries(protocol: &Protocol, allow: bool) -> anyhow::Result<Vec<CheckEntry>> {
+ let Some(ip_proto) = protocol.number() else {
+ bail!("protocol {protocol:?} was not resolved to a number");
+ };
+ Ok(match protocol {
+ Protocol::Tcp(tcp) => port_entries(ip_proto, tcp.ports(), allow),
+ Protocol::Udp(udp) => port_entries(ip_proto, udp.ports(), allow),
+ Protocol::Sctp(sctp) => port_entries(ip_proto, sctp.ports(), allow),
+ Protocol::Dccp(ports) | Protocol::UdpLite(ports) => port_entries(ip_proto, ports, allow),
+ // an icmp match covers IPv4 frames and an icmpv6 match IPv6 frames, which the protocol
+ // number alone does not say
+ Protocol::Icmp(icmp) => vec![CheckEntry {
+ ip_proto: Some(ip_proto),
+ family: Family::V4,
+ icmp_type: icmp.ty().and_then(IcmpType::number),
+ icmp_code: icmp.code().map(IcmpCode::number),
+ allow,
+ ..Default::default()
+ }],
+ Protocol::Icmpv6(icmpv6) => vec![CheckEntry {
+ ip_proto: Some(ip_proto),
+ family: Family::V6,
+ icmp_type: icmpv6.ty().and_then(Icmpv6Type::number),
+ icmp_code: icmpv6.code().map(Icmpv6Code::number),
+ allow,
+ ..Default::default()
+ }],
+ Protocol::Named(_) | Protocol::Numeric(_) => vec![CheckEntry {
+ ip_proto: Some(ip_proto),
+ allow,
+ ..Default::default()
+ }],
+ })
+}
+
+/// Pairs share checks, so translate each check once up front instead of once per pair.
+fn check_entries(state: &DesiredState) -> anyhow::Result<Vec<Vec<CheckEntry>>> {
+ let mut checks = Vec::with_capacity(state.checks.len());
+ for check in &state.checks {
+ let mut entries = Vec::new();
+ for protocol in &check.protocols {
+ entries.extend(protocol_entries(protocol, check.allow)?);
+ }
+ checks.push(entries);
+ }
+ Ok(checks)
+}
+
+fn pair_entries(
+ checks: &[Vec<CheckEntry>],
+ pair: &state::Pair,
+ cfg: &MicrosegRunningConfig,
+) -> anyhow::Result<Vec<CheckEntry>> {
+ let mut entries = Vec::new();
+ for &check in &pair.checks {
+ entries.extend_from_slice(&checks[check]);
+ }
+ // proxmox-ebpf refuses such a pair as well, but cannot name the classes by their groups
+ if entries.len() > CHECK_MAX {
+ bail!(
+ "pair {} -> {} needs {} checks, a pair holds at most {CHECK_MAX}",
+ cfg.describe_class(pair.src_id),
+ cfg.describe_class(pair.dst_id),
+ entries.len(),
+ );
+ }
+ Ok(entries)
+}
+
+fn pairs(state: &DesiredState, cfg: &MicrosegRunningConfig) -> anyhow::Result<Vec<Pair>> {
+ let checks = check_entries(state)?;
+ let mut pairs = Vec::with_capacity(state.pairs.len());
+ for pair in &state.pairs {
+ pairs.push(Pair {
+ src_id: pair.src_id,
+ dst_id: pair.dst_id,
+ checks: pair_entries(&checks, pair, cfg)?,
+ allow: pair.allow,
+ });
+ }
+ Ok(pairs)
+}
+
+pub struct FullPass {
+ configured: bool,
+ assignments: Vec<Assignment>,
+ pairs: Vec<Pair>,
+}
+
+pub fn full_pass(
+ nics: &LocalNics,
+ state: &DesiredState,
+ cfg: &MicrosegRunningConfig,
+) -> anyhow::Result<FullPass> {
+ let assignments = state
+ .nics
+ .iter()
+ .map(|resolved| {
+ let Some(name) = nics.name(resolved.vmid, resolved.iface) else {
+ bail!(
+ "NIC vm{} net{} has no interface on this node",
+ resolved.vmid,
+ resolved.iface
+ );
+ };
+ Ok(Assignment {
+ iface: name.to_owned(),
+ id: resolved.id,
+ firewall_bridge: nic::behind_firewall_bridge(name),
+ })
+ })
+ .collect::<anyhow::Result<Vec<_>>>()?;
+ Ok(FullPass {
+ configured: state.configured,
+ assignments,
+ pairs: pairs(state, cfg)?,
+ })
+}
+
+pub fn check(state: &DesiredState, cfg: &MicrosegRunningConfig) -> anyhow::Result<()> {
+ policy::check(&pairs(state, cfg)?).context("the data plane cannot take the microseg policy")
+}
+
+pub fn apply(lock: &ExclusiveLock, pass: &FullPass) -> anyhow::Result<()> {
+ let applied = lock.apply(pass.configured, &pass.assignments, &pass.pairs)?;
+ if applied != Pass::Applied {
+ return Ok(());
+ }
+ let level = if pass.assignments.is_empty() {
+ log::Level::Debug
+ } else {
+ log::Level::Info
+ };
+ log::log!(
+ level,
+ "microseg: applied {} assigned NICs on this node, {} pairs",
+ pass.assignments.len(),
+ pass.pairs.len()
+ );
+ Ok(())
+}
+
+#[cfg(test)]
+mod tests {
+ use proxmox_ve_config::firewall::types::rule_match::Tcp;
+ use proxmox_ve_config::sdn::microseg::state::Check;
+
+ use super::*;
+
+ fn ports(list: &str) -> PortList {
+ list.parse().expect("valid port list")
+ }
+
+ /// The entries depend on nothing but the one check.
+ fn state(protocols: Vec<Protocol>) -> DesiredState {
+ DesiredState {
+ configured: true,
+ pairs: vec![state::Pair {
+ src_id: 1,
+ dst_id: 2,
+ checks: vec![0],
+ allow: false,
+ }],
+ checks: vec![Check {
+ protocols,
+ allow: true,
+ }],
+ nics: Vec::new(),
+ }
+ }
+
+ #[test]
+ fn check_rejects_what_the_apply_would() {
+ let elements: Vec<String> = (1..=CHECK_MAX + 1).map(|port| port.to_string()).collect();
+ let tcp = Tcp::new(Ports::new(None, ports(&elements.join(","))));
+ let over = state(vec![Protocol::Tcp(tcp)]);
+ let cfg: MicrosegRunningConfig = serde_json::from_str(
+ r#"{"ids":{},"identities":{"next":3,"classes":{"1":["web"],"2":["db"]}}}"#,
+ )
+ .expect("parse running config");
+ let error = check(&over, &cfg).expect_err("past the bound");
+ let message = format!("{error:#}");
+ assert!(message.contains("pair 1 (web) -> 2 (db)"), "{message}");
+
+ let tcp = Tcp::new(Ports::new(None, ports("22")));
+ check(&state(vec![Protocol::Tcp(tcp)]), &cfg).expect("an ordinary pair");
+ }
+}
diff --git a/proxmox-microseg/src/lib.rs b/proxmox-microseg/src/lib.rs
new file mode 100644
index 000000000000..4646d70982d1
--- /dev/null
+++ b/proxmox-microseg/src/lib.rs
@@ -0,0 +1,66 @@
+mod ebpf;
+mod nic;
+mod running_config;
+
+use anyhow::{Context, bail};
+use proxmox_ve_config::sdn::microseg::state::{DesiredState, resolve_iface};
+use proxmox_ve_config::sdn::microseg::{GuestInfo, MicrosegRunningConfig};
+
+pub use crate::ebpf::{ExclusiveLock, SharedLock, lock_exclusive, lock_shared};
+pub use crate::running_config::parse as parse_running_config;
+
+fn full_pass(
+ cfg: &MicrosegRunningConfig,
+ inventory: &[GuestInfo],
+) -> anyhow::Result<ebpf::FullPass> {
+ let nics = nic::LocalNics::list()?;
+ let state = DesiredState::build(cfg, &nics.guests(inventory), ebpf::PAIRS_MAX)
+ .context("desired state")?;
+ ebpf::full_pass(&nics, &state, cfg)
+}
+
+pub fn apply(
+ lock: &ExclusiveLock,
+ cfg: &MicrosegRunningConfig,
+ inventory: &[GuestInfo],
+) -> anyhow::Result<()> {
+ ebpf::apply(lock, &full_pass(cfg, inventory)?)
+}
+
+/// Fails for a config the data plane could not install. The pairs alone decide that, and they
+/// cover every allocated class whatever the guests match, so no inventory is needed.
+pub fn check(cfg: &MicrosegRunningConfig) -> anyhow::Result<()> {
+ let state = DesiredState::build(cfg, &[], ebpf::PAIRS_MAX).context("desired state")?;
+ ebpf::check(&state, cfg)
+}
+
+/// Enforce the policy on a guest NIC that is about to come up. An assigned NIC gets its class. A
+/// NIC no assignment covers loses what an earlier plug or pass placed for its index, so one that
+/// comes up again in place drops its old identity. A failure in either case is an error rather
+/// than a skip, since the NIC would come up unenforced or as a class it left. `guest` is the
+/// plugged guest as the inventory lists it. `firewall_bridge` says the NIC is about to be put
+/// behind a firewall bridge.
+pub fn apply_iface(
+ lock: &SharedLock,
+ cfg: &MicrosegRunningConfig,
+ guest: &GuestInfo,
+ ifname: &str,
+ firewall_bridge: bool,
+) -> anyhow::Result<()> {
+ let Some((vmid, iface)) = nic::parse(ifname) else {
+ log::warn!("microseg: {ifname} is not a guest NIC name, skipping");
+ return Ok(());
+ };
+ if vmid != guest.vmid {
+ bail!(
+ "{ifname} is a NIC of guest {vmid}, not of guest {}",
+ guest.vmid
+ );
+ }
+ let Some(id) = resolve_iface(cfg, guest, iface)? else {
+ log::debug!("microseg: {ifname} is unassigned");
+ lock.release_iface(ifname)?;
+ return Ok(());
+ };
+ lock.apply_iface(ifname, id, firewall_bridge)
+}
diff --git a/proxmox-microseg/src/nic.rs b/proxmox-microseg/src/nic.rs
new file mode 100644
index 000000000000..6a0b974b0f49
--- /dev/null
+++ b/proxmox-microseg/src/nic.rs
@@ -0,0 +1,120 @@
+use std::collections::{BTreeMap, BTreeSet, HashMap};
+
+use anyhow::Context;
+use proxmox_ve_config::sdn::microseg::GuestInfo;
+
+/// The guest and the NIC index a host interface name stands for.
+pub fn parse(name: &str) -> Option<(u32, u32)> {
+ let rest = name
+ .strip_prefix("tap")
+ .or_else(|| name.strip_prefix("veth"))?;
+ let (vmid, iface) = rest.split_once('i')?;
+ // the integer parser alone would also take a sign
+ let number = |digits: &str| -> Option<u32> {
+ digits
+ .bytes()
+ .all(|byte| byte.is_ascii_digit())
+ .then(|| digits.parse().ok())
+ .flatten()
+ };
+ Some((number(vmid)?, number(iface)?))
+}
+
+/// A guest without a name or tags may be missing from the inventory. An assignment by VMID covers
+/// it all the same, so it is known by its VMID alone.
+fn guest(known: Option<&GuestInfo>, vmid: u32) -> GuestInfo {
+ known.cloned().unwrap_or(GuestInfo {
+ vmid,
+ ..Default::default()
+ })
+}
+
+pub fn behind_firewall_bridge(name: &str) -> bool {
+ let Some((vmid, iface)) = parse(name) else {
+ return false;
+ };
+ std::fs::read_link(format!("/sys/class/net/{name}/master")).is_ok_and(|bridge| {
+ bridge
+ .file_name()
+ .is_some_and(|master| master.to_string_lossy() == format!("fwbr{vmid}i{iface}"))
+ })
+}
+
+/// The node says which NICs a guest has, since its config does not name a hot-plugged NIC until the
+/// plug is through.
+pub struct LocalNics(BTreeMap<(u32, u32), String>);
+
+impl LocalNics {
+ pub fn list() -> anyhow::Result<Self> {
+ // a failure must not read as a node without guest NICs, a full pass would then take
+ // enforcement off every one of them
+ let names = std::fs::read_dir("/sys/class/net")
+ .and_then(|entries| {
+ entries
+ .map(|entry| Ok(entry?.file_name().to_string_lossy().into_owned()))
+ .collect::<std::io::Result<Vec<_>>>()
+ })
+ .context("list the interfaces of this node")?;
+ Ok(Self::from_names(names))
+ }
+
+ fn from_names(names: impl IntoIterator<Item = String>) -> Self {
+ let nics = names
+ .into_iter()
+ .filter_map(|name| Some((parse(&name)?, name)));
+ Self(nics.collect())
+ }
+
+ pub fn name(&self, vmid: u32, iface: u32) -> Option<&str> {
+ self.0.get(&(vmid, iface)).map(String::as_str)
+ }
+
+ pub fn guests(&self, inventory: &[GuestInfo]) -> Vec<GuestInfo> {
+ let mut nics: BTreeMap<u32, BTreeSet<u32>> = BTreeMap::new();
+ for &(vmid, iface) in self.0.keys() {
+ nics.entry(vmid).or_default().insert(iface);
+ }
+ let known: HashMap<u32, &GuestInfo> = inventory.iter().map(|g| (g.vmid, g)).collect();
+ nics.into_iter()
+ .map(|(vmid, nics)| GuestInfo {
+ nics: nics.into_iter().collect(),
+ ..guest(known.get(&vmid).copied(), vmid)
+ })
+ .collect()
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn a_guest_has_the_nics_the_node_has() {
+ let inventory = [
+ GuestInfo {
+ vmid: 100,
+ name: Some("web".to_string()),
+ nics: vec![0, 5],
+ ..Default::default()
+ },
+ GuestInfo {
+ vmid: 300,
+ nics: vec![0],
+ ..Default::default()
+ },
+ ];
+ let names = ["lo", "tap100i1", "fwln100i0", "veth200i0", "tap100i0"];
+ let nics = LocalNics::from_names(names.map(str::to_string));
+ let guests = nics.guests(&inventory);
+ assert_eq!(nics.name(200, 0), Some("veth200i0"));
+
+ // net1 is not in the config of vm100 yet and net5 is not plugged, vm300 runs elsewhere
+ assert_eq!(guests.len(), 2);
+ assert_eq!(guests[0].vmid, 100);
+ assert_eq!(guests[0].name.as_deref(), Some("web"));
+ assert_eq!(guests[0].nics, [0, 1]);
+ assert_eq!(guests[1].vmid, 200);
+ assert_eq!(guests[1].name, None);
+ assert_eq!(guests[1].nics, [0]);
+ }
+}
diff --git a/proxmox-microseg/src/running_config.rs b/proxmox-microseg/src/running_config.rs
new file mode 100644
index 000000000000..dbf8160de196
--- /dev/null
+++ b/proxmox-microseg/src/running_config.rs
@@ -0,0 +1,26 @@
+use proxmox_ve_config::sdn::microseg::MicrosegRunningConfig;
+use serde::Deserialize;
+
+#[derive(Deserialize)]
+struct Sections {
+ microseg: Option<MicrosegRunningConfig>,
+}
+
+/// `None` for a running config without a microseg block. Every apply writes the block, even an
+/// empty one, so only a node without microseg support leaves it out. Such a document says nothing
+/// about the policy, it does not declare an empty one.
+pub fn parse(raw: &str) -> anyhow::Result<Option<MicrosegRunningConfig>> {
+ let sections: Sections = serde_json::from_str(raw)?;
+ Ok(sections.microseg)
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn a_missing_block_is_absent() {
+ let cfg = parse(r#"{"zones": {"ids": {}}}"#).expect("parse running config");
+ assert_eq!(cfg, None);
+ }
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-ve-rs v3 14/45] microseg: add a rule option that matches non-first fragments
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (12 preceding siblings ...)
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 13/45] microseg: add a crate that applies the policy to a node's guest NICs Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH pve-cluster v3 15/45] cfs: add 'sdn/microseg.cfg' to observed files Hannes Laimer
` (30 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
A narrowed rule matches only frames with a transport header. Only the
first fragment of a datagram carries one. So a pair allowed only
through narrowed rules dropped every later fragment.
A rule can now be narrowed to non-first fragments, of one protocol or
of every protocol. Like any narrowed rule it decides only what it
matches. The only other narrowing it accepts is a protocol, since a
fragment carries no ports and no ICMP type.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
proxmox-microseg/src/ebpf.rs | 12 +++++
proxmox-ve-config/src/sdn/microseg/api.rs | 7 +++
proxmox-ve-config/src/sdn/microseg/mod.rs | 50 ++++++++++++++++-----
proxmox-ve-config/src/sdn/microseg/state.rs | 4 ++
proxmox-ve-config/src/sdn/microseg/types.rs | 27 +++++++++--
5 files changed, 85 insertions(+), 15 deletions(-)
diff --git a/proxmox-microseg/src/ebpf.rs b/proxmox-microseg/src/ebpf.rs
index 02ea16d5a2ef..717e02fe3ff6 100644
--- a/proxmox-microseg/src/ebpf.rs
+++ b/proxmox-microseg/src/ebpf.rs
@@ -88,6 +88,17 @@ fn check_entries(state: &DesiredState) -> anyhow::Result<Vec<Vec<CheckEntry>>> {
for protocol in &check.protocols {
entries.extend(protocol_entries(protocol, check.allow)?);
}
+ if check.fragments {
+ if entries.is_empty() {
+ entries.push(CheckEntry {
+ allow: check.allow,
+ ..Default::default()
+ });
+ }
+ for fragment_entry in &mut entries {
+ fragment_entry.fragment = true;
+ }
+ }
checks.push(entries);
}
Ok(checks)
@@ -210,6 +221,7 @@ mod tests {
}],
checks: vec![Check {
protocols,
+ fragments: false,
allow: true,
}],
nics: Vec::new(),
diff --git a/proxmox-ve-config/src/sdn/microseg/api.rs b/proxmox-ve-config/src/sdn/microseg/api.rs
index d9cd22344cb5..650658fe0f13 100644
--- a/proxmox-ve-config/src/sdn/microseg/api.rs
+++ b/proxmox-ve-config/src/sdn/microseg/api.rs
@@ -63,6 +63,8 @@ pub struct RuleCreate {
pub icmp_type: Option<String>,
#[serde(default, rename = "macro")]
pub fw_macro: Option<String>,
+ #[serde(default, deserialize_with = "proxmox_serde::perl::deserialize_bool")]
+ pub fragments: bool,
#[serde(default)]
pub comment: Option<String>,
}
@@ -211,6 +213,7 @@ fn rule_id(
dst_match: PredicateMatch,
dst_groups: &[&str],
narrowing: &[Option<&str>],
+ fragments: bool,
bidirectional: bool,
) -> Result<MicrosegId, Error> {
let mut hash = Fnv1a::new();
@@ -221,6 +224,7 @@ fn rule_id(
for field in narrowing {
hash.text(field.unwrap_or_default());
}
+ hash.flag(fragments);
hash.flag(bidirectional);
format!("r{:016x}", hash.0).parse()
}
@@ -332,6 +336,7 @@ fn build_entry(
rule.dst_match,
&dst_groups,
&narrowing,
+ rule.fragments,
rule.bidirectional,
)?;
MicrosegEntry::Rule(RuleSection {
@@ -348,6 +353,7 @@ fn build_entry(
dport,
icmp_type,
fw_macro,
+ fragments: rule.fragments,
comment: rule.comment,
})
}
@@ -598,6 +604,7 @@ fn narrowing_str(rule: &RuleSection) -> Option<String> {
rule.sport.as_ref().map(|ports| format!("sport {ports}")),
rule.dport.as_ref().map(|ports| format!("dport {ports}")),
rule.icmp_type.as_ref().map(|kind| format!("type {kind}")),
+ rule.fragments.then(|| "fragments".to_string()),
];
let narrowing = parts.into_iter().flatten().collect::<Vec<_>>().join(" ");
(!narrowing.is_empty()).then_some(narrowing)
diff --git a/proxmox-ve-config/src/sdn/microseg/mod.rs b/proxmox-ve-config/src/sdn/microseg/mod.rs
index be48b890206f..c3f776d1226a 100644
--- a/proxmox-ve-config/src/sdn/microseg/mod.rs
+++ b/proxmox-ve-config/src/sdn/microseg/mod.rs
@@ -588,28 +588,26 @@ struct ResolvedRule {
dst: identity::Predicate,
prio: u16,
allow: bool,
- /// The protocol matches narrowing the rule. Empty for a rule deciding the whole pair.
protocols: Vec<Protocol>,
- /// Set on the mirrored orientation of a bidirectional rule.
+ fragments: bool,
reverse: bool,
}
impl ResolvedRule {
fn is_narrowed(&self) -> bool {
- !self.protocols.is_empty()
+ self.fragments || !self.protocols.is_empty()
}
- /// A narrowed deny that can match neighbor discovery.
+ /// A narrowed deny that can match neighbor discovery. A non-first fragment carries no ICMPv6
+ /// header, so a fragments rule cannot.
fn can_deny_nd(&self) -> bool {
!self.allow
+ && !self.fragments
&& self.protocols.iter().any(|protocol| match protocol {
- Protocol::Icmpv6(icmp) => matches!(
- icmp.ty(),
- None | Some(Icmpv6Type::Any | Icmpv6Type::Numeric(135 | 136))
- | Some(Icmpv6Type::Named(
- "nd-neighbor-solicit" | "nd-neighbor-advert"
- ))
- ),
+ Protocol::Icmpv6(icmp) => icmp
+ .ty()
+ .and_then(Icmpv6Type::number)
+ .is_none_or(|ty| ty == 135 || ty == 136),
_ => false,
})
}
@@ -644,6 +642,7 @@ fn resolved_rules(entries: &BTreeMap<String, MicrosegEntry>) -> Result<Vec<Resol
prio: rule.prio,
allow: rule.allow,
protocols: rule.protocols().with_context(context)?,
+ fragments: rule.fragments,
reverse: false,
};
let mirrored = rule.bidirectional.then(|| ResolvedRule {
@@ -653,6 +652,7 @@ fn resolved_rules(entries: &BTreeMap<String, MicrosegEntry>) -> Result<Vec<Resol
prio: forward.prio,
allow: forward.allow,
protocols: forward.protocols.iter().map(reversed_protocol).collect(),
+ fragments: forward.fragments,
reverse: true,
});
resolved.push(forward);
@@ -921,6 +921,7 @@ mod tests {
dport: None,
icmp_type: None,
fw_macro: None,
+ fragments: false,
comment: None,
}),
)
@@ -1040,6 +1041,33 @@ mod tests {
assert!(!folded.allow);
}
+ #[test]
+ fn a_fragments_rule_decides_the_fragments_only() {
+ let with = |modify: fn(&mut RuleSection)| {
+ BTreeMap::from([
+ group("web"),
+ group("db"),
+ with_rule(rule("frag", &["web"], &["db"], true), modify),
+ ])
+ };
+ let entries = with(|rule| rule.fragments = true);
+ validate(&entries).expect("valid");
+ let rules = resolved(&entries);
+ let folded = fold(firing_rules(&rules, &set(&["web"]), &set(&["db"])));
+ assert!(folded.checks[0].fragments);
+ assert!(
+ !folded.allow,
+ "the allow is for the fragments, not the pair"
+ );
+
+ let with_ports = with(|rule| {
+ rule.fragments = true;
+ rule.proto = Some("tcp".to_string());
+ rule.dport = Some("22".to_string());
+ });
+ validate(&with_ports).expect_err("a fragment carries no ports");
+ }
+
#[test]
fn bidirectional_narrowing_swaps_the_ports() {
let tcp = |sport: Option<&str>, dport: Option<&str>| {
diff --git a/proxmox-ve-config/src/sdn/microseg/state.rs b/proxmox-ve-config/src/sdn/microseg/state.rs
index 1392860f5062..ce733de9b37d 100644
--- a/proxmox-ve-config/src/sdn/microseg/state.rs
+++ b/proxmox-ve-config/src/sdn/microseg/state.rs
@@ -43,6 +43,9 @@ pub struct Pair {
#[derive(Debug)]
pub struct Check {
pub protocols: Vec<Protocol>,
+ /// The check matches non-first fragments, those of `protocols` or of every protocol when there
+ /// is none.
+ pub fragments: bool,
pub allow: bool,
}
@@ -273,6 +276,7 @@ fn fold_pairs(firing: &[(u16, u16, &ResolvedRule)]) -> (Vec<Pair>, Vec<Check>) {
.or_insert_with(|| {
checks.push(Check {
protocols: rule.protocols.clone(),
+ fragments: rule.fragments,
allow: rule.allow,
});
checks.len() - 1
diff --git a/proxmox-ve-config/src/sdn/microseg/types.rs b/proxmox-ve-config/src/sdn/microseg/types.rs
index 39f80764e5e0..a8c3fcb0557b 100644
--- a/proxmox-ve-config/src/sdn/microseg/types.rs
+++ b/proxmox-ve-config/src/sdn/microseg/types.rs
@@ -165,6 +165,10 @@ pub struct GroupSection {
max_length: 128,
description: "Firewall macro standing in for the protocol and ports.",
},
+ fragments: {
+ type: Boolean,
+ optional: true,
+ },
comment: {
type: String,
optional: true,
@@ -179,7 +183,8 @@ pub struct GroupSection {
/// the evaluation order decides first. Traffic no rule decides is denied.
///
/// A rule may be narrowed to a protocol with ports or an ICMP type, or to a firewall macro that
-/// stands in for them.
+/// stands in for them. A non-first fragment carries no transport header, so among the
+/// narrowed rules only one narrowed to fragments matches it.
pub struct RuleSection {
pub(crate) id: MicrosegId,
/// Groups the source predicate lists.
@@ -218,18 +223,32 @@ pub struct RuleSection {
pub(crate) icmp_type: Option<String>,
#[serde(default, rename = "macro", skip_serializing_if = "Option::is_none")]
pub(crate) fw_macro: Option<String>,
+ /// Whether the rule matches non-first fragments, those of `proto` or of every protocol.
+ #[serde(
+ default,
+ deserialize_with = "proxmox_serde::perl::deserialize_bool",
+ skip_serializing_if = "is_false"
+ )]
+ pub(crate) fragments: bool,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub(crate) comment: Option<String>,
}
impl RuleSection {
- /// The protocol matches narrowing this rule, empty for a rule that decides the whole pair.
- /// Ports are only accepted with a port-carrying protocol and an ICMP type only with an ICMP
- /// protocol, since either would otherwise be silently ignored.
+ /// The protocol matches narrowing this rule, empty for a rule that decides the whole pair or
+ /// matches the fragments of every protocol. Ports without a port-carrying protocol and an
+ /// ICMP type without an ICMP protocol return Err, since the protocol match would ignore
+ /// them.
pub(super) fn protocols(&self) -> Result<Vec<Protocol>, Error> {
let explicit = [&self.proto, &self.sport, &self.dport, &self.icmp_type]
.iter()
.any(|field| field.is_some());
+ let beyond_proto = [&self.sport, &self.dport, &self.icmp_type, &self.fw_macro]
+ .iter()
+ .any(|field| field.is_some());
+ if self.fragments && beyond_proto {
+ bail!("a fragment carries no ports and no ICMP type, so only a protocol can narrow it");
+ }
if let Some(name) = &self.fw_macro {
if explicit {
bail!("a macro cannot be combined with a protocol, ports or an ICMP type");
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-cluster v3 15/45] cfs: add 'sdn/microseg.cfg' to observed files
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (13 preceding siblings ...)
2026-10-27 8:54 ` [PATCH proxmox-ve-rs v3 14/45] microseg: add a rule option that matches non-first fragments Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH pve-access-control v3 16/45] permissions: add ACL path for SDN microsegmentation Hannes Laimer
` (29 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
Stores the SDN microsegmentation groups, rules and NIC assignments.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/Cluster.pm | 1 +
src/pmxcfs/status.c | 1 +
2 files changed, 2 insertions(+)
diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
index 034b78cd2d95..cef0b1a9c483 100644
--- a/src/PVE/Cluster.pm
+++ b/src/PVE/Cluster.pm
@@ -84,6 +84,7 @@ my $observed = {
'sdn/mac-cache.json' => 1,
'sdn/dns.cfg' => 1,
'sdn/fabrics.cfg' => 1,
+ 'sdn/microseg.cfg' => 1,
'sdn/route-maps.cfg' => 1,
'sdn/prefix-lists.cfg' => 1,
'sdn/.running-config' => 1,
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index e1ab438bbc17..80aecfc378b6 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -112,6 +112,7 @@ static memdb_change_t memdb_change_array[] = {
{.path = "sdn/pve-ipam-state.json"},
{.path = "sdn/dns.cfg"},
{.path = "sdn/fabrics.cfg"},
+ {.path = "sdn/microseg.cfg"},
{.path = "sdn/route-maps.cfg"},
{.path = "sdn/prefix-lists.cfg"},
{.path = "sdn/.running-config"},
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-access-control v3 16/45] permissions: add ACL path for SDN microsegmentation
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (14 preceding siblings ...)
2026-10-27 8:54 ` [PATCH pve-cluster v3 15/45] cfs: add 'sdn/microseg.cfg' to observed files Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:54 ` [PATCH proxmox-perl-rs v3 17/45] pve-rs: sdn: add microseg config binding Hannes Laimer
` (28 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
Accept /sdn/microseg as an ACL path, so a user can get permissions on
SDN microsegmentation without a role on all of /sdn. The path has no
per-object children, so permissions are set on the policy as a whole.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/AccessControl.pm | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/PVE/AccessControl.pm b/src/PVE/AccessControl.pm
index 8879ab8c3ab6..e709e3d1c6bd 100644
--- a/src/PVE/AccessControl.pm
+++ b/src/PVE/AccessControl.pm
@@ -1300,6 +1300,7 @@ sub check_path {
|/sdn/fabrics/[[:alnum:]]+
|/sdn/ipams
|/sdn/ipams/[[:alnum:]]+
+ |/sdn/microseg
|/sdn/prefix-lists
|/sdn/prefix-lists/[[:alnum:]]+
|/sdn/route-maps
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-perl-rs v3 17/45] pve-rs: sdn: add microseg config binding
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (15 preceding siblings ...)
2026-10-27 8:54 ` [PATCH pve-access-control v3 16/45] permissions: add ACL path for SDN microsegmentation Hannes Laimer
@ 2026-10-27 8:54 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH proxmox-perl-rs v3 18/45] pve-rs: sdn: microseg: bind the data plane Hannes Laimer
` (27 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:54 UTC (permalink / raw)
To: pve-devel
Expose the microseg config to Perl, shaped like the other SDN config
bindings. Which NICs an assignment wins is not stored, so the model
answers it for a guest inventory the caller passes in. Creating an
object returns its id, since rule and assignment ids are generated.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
pve-rs/Makefile | 1 +
pve-rs/src/bindings/sdn/microseg.rs | 243 ++++++++++++++++++++++++++++
pve-rs/src/bindings/sdn/mod.rs | 1 +
3 files changed, 245 insertions(+)
create mode 100644 pve-rs/src/bindings/sdn/microseg.rs
diff --git a/pve-rs/Makefile b/pve-rs/Makefile
index bb1cd2d5f73a..fc199ae5c9b8 100644
--- a/pve-rs/Makefile
+++ b/pve-rs/Makefile
@@ -33,6 +33,7 @@ PERLMOD_PACKAGES := \
PVE::RS::ResourceScheduling::Static \
PVE::RS::ResourceScheduling::Dynamic \
PVE::RS::SDN::Fabrics \
+ PVE::RS::SDN::Microseg \
PVE::RS::SDN::PrefixLists \
PVE::RS::SDN::RouteMaps \
PVE::RS::SDN::WireGuard::PrivateKeys \
diff --git a/pve-rs/src/bindings/sdn/microseg.rs b/pve-rs/src/bindings/sdn/microseg.rs
new file mode 100644
index 000000000000..5deaf5189aec
--- /dev/null
+++ b/pve-rs/src/bindings/sdn/microseg.rs
@@ -0,0 +1,243 @@
+#[perlmod::package(name = "PVE::RS::SDN::Microseg", lib = "pve_rs")]
+pub mod pve_rs_sdn_microseg {
+ //! The `PVE::RS::SDN::Microseg` package.
+ //!
+ //! This provides the configuration for SDN microseg, as well as helper methods for reading
+ //! / writing the configuration.
+
+ use std::collections::{BTreeMap, HashMap};
+ use std::ops::Deref;
+ use std::sync::Mutex;
+
+ use anyhow::{Context, Error};
+ use openssl::hash::{MessageDigest, hash};
+
+ use perlmod::Value;
+ use serde::Serialize;
+
+ use proxmox_section_config::typed::{ApiSectionDataEntry, SectionConfigData};
+ use proxmox_ve_config::sdn::microseg::api::{MicrosegCreate, MicrosegUpdate};
+ use proxmox_ve_config::sdn::microseg::identity::{IdSpace, Registry};
+ use proxmox_ve_config::sdn::microseg::{
+ self, GroupSetRules, GuestInfo, MicrosegEntry, MicrosegId, MicrosegRunningConfig,
+ };
+
+ /// A SDN Microseg config instance.
+ pub struct PerlMicrosegConfig {
+ pub microseg: Mutex<BTreeMap<String, MicrosegEntry>>,
+ }
+
+ perlmod::declare_magic!(Box<PerlMicrosegConfig> : &PerlMicrosegConfig as "PVE::RS::SDN::Microseg::Config");
+
+ /// Class method: Parse the raw configuration from `/etc/pve/sdn/microseg.cfg`.
+ #[export]
+ pub fn config(#[raw] class: Value, raw_config: &[u8]) -> Result<perlmod::Value, Error> {
+ let raw_config = std::str::from_utf8(raw_config)?;
+ let config = MicrosegEntry::parse_section_config("microseg.cfg", raw_config)?;
+ let entries: BTreeMap<String, MicrosegEntry> = config.into_iter().collect();
+
+ microseg::validate(&entries).context("microseg config")?;
+
+ Ok(
+ perlmod::instantiate_magic!(&class, MAGIC => Box::new(PerlMicrosegConfig {
+ microseg: Mutex::new(entries),
+ })),
+ )
+ }
+
+ /// Class method: Parse the configuration from `/etc/pve/sdn/.running-config`.
+ #[export]
+ pub fn running_config(
+ #[raw] class: Value,
+ entries: BTreeMap<String, MicrosegEntry>,
+ ) -> Result<perlmod::Value, Error> {
+ microseg::validate(&entries)?;
+
+ Ok(
+ perlmod::instantiate_magic!(&class, MAGIC => Box::new(PerlMicrosegConfig {
+ microseg: Mutex::new(entries),
+ })),
+ )
+ }
+
+ /// Method: Render the running config from this config, keeping the ids of `previous`, the
+ /// `identities` of the last one (`undef` renumbers everything).
+ #[export]
+ pub fn render(
+ #[try_from_ref] this: &PerlMicrosegConfig,
+ previous: Option<Registry>,
+ ) -> Result<MicrosegRunningConfig, Error> {
+ let entries = this.microseg.lock().unwrap().deref().clone();
+ let previous = previous.unwrap_or_default();
+ // the quarantine is only as reliable as the clocks of the nodes that render
+ let now = std::time::SystemTime::now()
+ .duration_since(std::time::UNIX_EPOCH)
+ .context("the system clock is set before 1970")?
+ .as_secs();
+ microseg::render(&previous, entries, now)
+ }
+
+ #[derive(Serialize)]
+ pub struct AssignmentMember {
+ vmid: u32,
+ nics: Vec<u32>,
+ }
+
+ /// Method: Returns the guests and NICs each assignment wins in `inventory`, keyed by assignment
+ /// id.
+ #[export]
+ pub fn assigned_members(
+ #[try_from_ref] this: &PerlMicrosegConfig,
+ inventory: Vec<GuestInfo>,
+ ) -> HashMap<String, Vec<AssignmentMember>> {
+ let entries = this.microseg.lock().unwrap();
+ let mut members: HashMap<String, Vec<AssignmentMember>> = HashMap::new();
+ // sorted by guest, so a guest that already has an entry is the last one in the list
+ for nic in microseg::assigned_nics(entries.deref(), &inventory) {
+ let guests = members.entry(nic.assignment.to_string()).or_default();
+ match guests.last_mut() {
+ Some(member) if member.vmid == nic.vmid => member.nics.push(nic.iface),
+ _ => guests.push(AssignmentMember {
+ vmid: nic.vmid,
+ nics: vec![nic.iface],
+ }),
+ }
+ }
+ members
+ }
+
+ /// Method: Returns, for each group set an assignment names, the rules that fire on it.
+ #[export]
+ pub fn rules_by_group_set(
+ #[try_from_ref] this: &PerlMicrosegConfig,
+ ) -> Result<Vec<GroupSetRules>, Error> {
+ let entries = this.microseg.lock().unwrap();
+ microseg::rules_by_group_set(entries.deref())
+ }
+
+ /// Method: Returns each rule's place in the evaluation order, by rule id.
+ #[export]
+ pub fn rule_order(
+ #[try_from_ref] this: &PerlMicrosegConfig,
+ ) -> Result<HashMap<MicrosegId, usize>, Error> {
+ let entries = this.microseg.lock().unwrap();
+ microseg::rule_order(entries.deref())
+ }
+
+ /// Method: Returns each assignment's place in the match order, by id.
+ #[export]
+ pub fn assignment_order(#[try_from_ref] this: &PerlMicrosegConfig) -> HashMap<String, usize> {
+ let entries = this.microseg.lock().unwrap();
+ microseg::assignment_order(entries.deref())
+ }
+
+ /// Method: Whether a guest renamed from `old` to `new` enters or leaves the names a regex
+ /// assignment matches.
+ #[export]
+ pub fn name_match_changed(
+ #[try_from_ref] this: &PerlMicrosegConfig,
+ old: Option<&str>,
+ new: Option<&str>,
+ ) -> bool {
+ let entries = this.microseg.lock().unwrap();
+ microseg::name_match_changed(entries.deref(), old, new)
+ }
+
+ /// Method: Returns how specific each rule is, by rule id.
+ #[export]
+ pub fn rule_specificity(
+ #[try_from_ref] this: &PerlMicrosegConfig,
+ ) -> Result<HashMap<MicrosegId, usize>, Error> {
+ let entries = this.microseg.lock().unwrap();
+ microseg::rule_specificity(entries.deref())
+ }
+
+ /// How much of the id space the `identities` of a running config use.
+ #[export]
+ pub fn id_space(identities: Option<Registry>) -> IdSpace {
+ identities.unwrap_or_default().id_space()
+ }
+
+ /// The microseg block of a running config as text.
+ #[export]
+ pub fn summary(microseg: MicrosegRunningConfig, inventory: Vec<GuestInfo>) -> String {
+ microseg::api::summary(µseg, &inventory)
+ }
+
+ /// Method: Returns all microseg objects, keyed by id.
+ #[export]
+ pub fn to_sections(
+ #[try_from_ref] this: &PerlMicrosegConfig,
+ ) -> BTreeMap<String, MicrosegEntry> {
+ this.microseg.lock().unwrap().clone()
+ }
+
+ /// Method: Convert the configuration into the section config string.
+ ///
+ /// Used for writing `/etc/pve/sdn/microseg.cfg`
+ #[export]
+ pub fn to_raw(#[try_from_ref] this: &PerlMicrosegConfig) -> Result<String, Error> {
+ let config = this.microseg.lock().unwrap();
+ // id order keeps the output and its digest stable across calls
+ let data: SectionConfigData<MicrosegEntry> = config
+ .iter()
+ .map(|(id, entry)| (id.clone(), entry.clone()))
+ .collect();
+
+ MicrosegEntry::write_section_config("microseg.cfg", &data)
+ }
+
+ /// Method: Generate a digest for the whole configuration.
+ #[export]
+ pub fn digest(#[try_from_ref] this: &PerlMicrosegConfig) -> Result<String, Error> {
+ let raw = to_raw(this)?;
+ let digest = hash(MessageDigest::sha256(), raw.as_bytes())?;
+
+ Ok(hex::encode(digest))
+ }
+
+ /// Method: Returns a single microseg object.
+ #[export]
+ pub fn get(#[try_from_ref] this: &PerlMicrosegConfig, id: &str) -> Option<MicrosegEntry> {
+ this.microseg.lock().unwrap().get(id).cloned()
+ }
+
+ /// The model's edits validate after they change the map, so a failed one must not reach the
+ /// stored config.
+ fn change<T>(
+ this: &PerlMicrosegConfig,
+ change: impl FnOnce(&mut BTreeMap<String, MicrosegEntry>) -> Result<T, Error>,
+ ) -> Result<T, Error> {
+ let mut entries = this.microseg.lock().unwrap();
+ let mut candidate = entries.clone();
+ let result = change(&mut candidate)?;
+ *entries = candidate;
+
+ Ok(result)
+ }
+
+ /// Method: Create a new microseg object and return its id.
+ #[export]
+ pub fn create(
+ #[try_from_ref] this: &PerlMicrosegConfig,
+ create: MicrosegCreate,
+ ) -> Result<String, Error> {
+ change(this, |entries| microseg::api::create(entries, create))
+ }
+
+ /// Method: Update an existing microseg object.
+ #[export]
+ pub fn update(
+ #[try_from_ref] this: &PerlMicrosegConfig,
+ id: &str,
+ update: MicrosegUpdate,
+ ) -> Result<(), Error> {
+ change(this, |entries| microseg::api::update(entries, id, update))
+ }
+
+ /// Method: Delete a microseg object.
+ #[export]
+ pub fn delete(#[try_from_ref] this: &PerlMicrosegConfig, id: &str) -> Result<(), Error> {
+ change(this, |entries| microseg::api::delete(entries, id))
+ }
+}
diff --git a/pve-rs/src/bindings/sdn/mod.rs b/pve-rs/src/bindings/sdn/mod.rs
index dcae046a7e46..1d4c23fb4258 100644
--- a/pve-rs/src/bindings/sdn/mod.rs
+++ b/pve-rs/src/bindings/sdn/mod.rs
@@ -1,4 +1,5 @@
pub(crate) mod fabrics;
+pub(crate) mod microseg;
pub(crate) mod prefix_lists;
pub(crate) mod route_maps;
pub(crate) mod wireguard;
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-perl-rs v3 18/45] pve-rs: sdn: microseg: bind the data plane
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (16 preceding siblings ...)
2026-10-27 8:54 ` [PATCH proxmox-perl-rs v3 17/45] pve-rs: sdn: add microseg config binding Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH proxmox-perl-rs v3 19/45] pve-rs: sdn: microseg: check the data plane limits on render Hannes Laimer
` (26 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
Expose the policy lock to Perl, with the full apply of the running
config and the apply for one guest NIC at plug time as its methods.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
pve-rs/Cargo.toml | 2 +
pve-rs/Makefile | 2 +
pve-rs/debian/control | 1 +
pve-rs/src/bindings/sdn/microseg.rs | 105 ++++++++++++++++++++++++++++
4 files changed, 110 insertions(+)
diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
index 5ae908249b8c..b2b0a983981a 100644
--- a/pve-rs/Cargo.toml
+++ b/pve-rs/Cargo.toml
@@ -38,6 +38,7 @@ proxmox-frr = { version = "0.5.1" }
proxmox-http = { version = "1.0.2", features = ["client-sync", "client-trait"] }
proxmox-http-error = "1"
proxmox-log = "1"
+proxmox-microseg = "0.1"
proxmox-network-types = "1.1.2"
proxmox-notify = { version = "1", features = ["pve-context"] }
proxmox-oci = "0.2.1"
@@ -79,6 +80,7 @@ proxmox-wireguard = { version = "0.1.2" }
# proxmox-log = { path = "../../proxmox/proxmox-log" }
# proxmox-login = { path = "../../proxmox/proxmox-login" }
# proxmox-metrics = { path = "../../proxmox/proxmox-metrics" }
+# proxmox-microseg = { path = "../../proxmox-ve-rs/proxmox-microseg" }
# proxmox-network-api = { path = "../../proxmox/proxmox-network-api" }
# proxmox-network-types = { path = "../../proxmox/proxmox-network-types" }
# proxmox-notify = { path = "../../proxmox/proxmox-notify" }
diff --git a/pve-rs/Makefile b/pve-rs/Makefile
index fc199ae5c9b8..ea0f4570198c 100644
--- a/pve-rs/Makefile
+++ b/pve-rs/Makefile
@@ -34,6 +34,8 @@ PERLMOD_PACKAGES := \
PVE::RS::ResourceScheduling::Dynamic \
PVE::RS::SDN::Fabrics \
PVE::RS::SDN::Microseg \
+ PVE::RS::SDN::Microseg::ExclusiveLock \
+ PVE::RS::SDN::Microseg::SharedLock \
PVE::RS::SDN::PrefixLists \
PVE::RS::SDN::RouteMaps \
PVE::RS::SDN::WireGuard::PrivateKeys \
diff --git a/pve-rs/debian/control b/pve-rs/debian/control
index 70367dc54504..5600e01fcbd6 100644
--- a/pve-rs/debian/control
+++ b/pve-rs/debian/control
@@ -26,6 +26,7 @@ Build-Depends: debhelper-compat (= 13),
librust-proxmox-http-1+default-dev (>= 1.0.2-~~),
librust-proxmox-http-error-1+default-dev,
librust-proxmox-log-1+default-dev,
+ librust-proxmox-microseg-0.1+default-dev,
librust-proxmox-network-types-1+default-dev (>= 1.1.2-~~),
librust-proxmox-notify-1+default-dev,
librust-proxmox-notify-1+pve-context-dev,
diff --git a/pve-rs/src/bindings/sdn/microseg.rs b/pve-rs/src/bindings/sdn/microseg.rs
index 5deaf5189aec..7300376460e5 100644
--- a/pve-rs/src/bindings/sdn/microseg.rs
+++ b/pve-rs/src/bindings/sdn/microseg.rs
@@ -240,4 +240,109 @@ pub mod pve_rs_sdn_microseg {
pub fn delete(#[try_from_ref] this: &PerlMicrosegConfig, id: &str) -> Result<(), Error> {
change(this, |entries| microseg::api::delete(entries, id))
}
+
+ pub(super) fn running_config_from(
+ raw: Option<&[u8]>,
+ ) -> Result<Option<MicrosegRunningConfig>, Error> {
+ match raw {
+ // pmxcfs yields the empty string for a zero-length file and the Perl reader of the
+ // same file treats that as an empty config.
+ Some(raw) if !raw.is_empty() => std::str::from_utf8(raw)
+ .map_err(Error::from)
+ .and_then(proxmox_microseg::parse_running_config)
+ .context("parse /etc/pve/sdn/.running-config"),
+ _ => Ok(Some(MicrosegRunningConfig::default())),
+ }
+ }
+}
+
+#[perlmod::package(name = "PVE::RS::SDN::Microseg::SharedLock", lib = "pve_rs")]
+pub mod pve_rs_sdn_microseg_shared_lock {
+ //! The `PVE::RS::SDN::Microseg::SharedLock` package.
+ //!
+ //! The policy lock in shared mode. A plug takes it before reading the running config and the
+ //! guest, and holds it until the NIC is placed. Released when the object goes away.
+
+ use anyhow::Error;
+ use perlmod::Value;
+ use proxmox_microseg::SharedLock;
+ use proxmox_ve_config::sdn::microseg::GuestInfo;
+
+ use super::pve_rs_sdn_microseg::running_config_from;
+
+ pub struct PerlSharedLock {
+ lock: SharedLock,
+ }
+
+ perlmod::declare_magic!(Box<PerlSharedLock> : &PerlSharedLock as "PVE::RS::SDN::Microseg::SharedLock");
+
+ /// Class method: Take the lock. Fails when a full pass holds it past the lock timeout.
+ #[export]
+ pub fn new(#[raw] class: Value) -> Result<Value, Error> {
+ let lock = proxmox_microseg::lock_shared()?;
+ Ok(perlmod::instantiate_magic!(&class, MAGIC => Box::new(PerlSharedLock { lock })))
+ }
+
+ /// Method: Enforce the microseg policy on the guest NIC `iface`, `raw` as for the full pass.
+ /// Returns false for a running config without a microseg block. Fails where only a full pass
+ /// can place the NIC.
+ #[export]
+ pub fn apply_iface(
+ #[try_from_ref] this: &PerlSharedLock,
+ raw: Option<&[u8]>,
+ guest: GuestInfo,
+ iface: &str,
+ firewall_bridge: bool,
+ ) -> Result<bool, Error> {
+ let Some(cfg) = running_config_from(raw)? else {
+ return Ok(false);
+ };
+ proxmox_microseg::apply_iface(&this.lock, &cfg, &guest, iface, firewall_bridge)?;
+ Ok(true)
+ }
+}
+
+#[perlmod::package(name = "PVE::RS::SDN::Microseg::ExclusiveLock", lib = "pve_rs")]
+pub mod pve_rs_sdn_microseg_exclusive_lock {
+ //! The `PVE::RS::SDN::Microseg::ExclusiveLock` package.
+ //!
+ //! The policy lock in exclusive mode. A full pass takes it before reading the running config
+ //! and the inventory, and holds it until the node is synced. Released when the object goes
+ //! away.
+
+ use anyhow::Error;
+ use perlmod::Value;
+ use proxmox_microseg::ExclusiveLock;
+ use proxmox_ve_config::sdn::microseg::GuestInfo;
+
+ use super::pve_rs_sdn_microseg::running_config_from;
+
+ pub struct PerlExclusiveLock {
+ lock: ExclusiveLock,
+ }
+
+ perlmod::declare_magic!(Box<PerlExclusiveLock> : &PerlExclusiveLock as "PVE::RS::SDN::Microseg::ExclusiveLock");
+
+ /// Class method: Take the lock. Fails when plugs or a pass hold it past the lock timeout.
+ #[export]
+ pub fn new(#[raw] class: Value) -> Result<Value, Error> {
+ let lock = proxmox_microseg::lock_exclusive()?;
+ Ok(perlmod::instantiate_magic!(&class, MAGIC => Box::new(PerlExclusiveLock { lock })))
+ }
+
+ /// Method: Enforce the microseg policy on this whole node. `raw` is the content of
+ /// `/etc/pve/sdn/.running-config`, `undef` or empty before the first commit. Returns false
+ /// and leaves the node untouched for a running config without a microseg block.
+ #[export]
+ pub fn apply(
+ #[try_from_ref] this: &PerlExclusiveLock,
+ raw: Option<&[u8]>,
+ inventory: Vec<GuestInfo>,
+ ) -> Result<bool, Error> {
+ let Some(cfg) = running_config_from(raw)? else {
+ return Ok(false);
+ };
+ proxmox_microseg::apply(&this.lock, &cfg, &inventory)?;
+ Ok(true)
+ }
}
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-perl-rs v3 19/45] pve-rs: sdn: microseg: check the data plane limits on render
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (17 preceding siblings ...)
2026-10-27 8:55 ` [PATCH proxmox-perl-rs v3 18/45] pve-rs: sdn: microseg: bind the data plane Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH proxmox-perl-rs v3 20/45] pve-rs: debian: fire the pve-microseg-update trigger on upgrade Hannes Laimer
` (25 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
A policy with more pairs or checks than the data plane holds fails
the render, and with it the SDN apply, instead of the reload of every
node.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
pve-rs/src/bindings/sdn/microseg.rs | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/pve-rs/src/bindings/sdn/microseg.rs b/pve-rs/src/bindings/sdn/microseg.rs
index 7300376460e5..dc6849928228 100644
--- a/pve-rs/src/bindings/sdn/microseg.rs
+++ b/pve-rs/src/bindings/sdn/microseg.rs
@@ -74,7 +74,9 @@ pub mod pve_rs_sdn_microseg {
.duration_since(std::time::UNIX_EPOCH)
.context("the system clock is set before 1970")?
.as_secs();
- microseg::render(&previous, entries, now)
+ let running = microseg::render(&previous, entries, now).context("render policy")?;
+ proxmox_microseg::check(&running)?;
+ Ok(running)
}
#[derive(Serialize)]
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH proxmox-perl-rs v3 20/45] pve-rs: debian: fire the pve-microseg-update trigger on upgrade
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (18 preceding siblings ...)
2026-10-27 8:55 ` [PATCH proxmox-perl-rs v3 19/45] pve-rs: sdn: microseg: check the data plane limits on render Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-network v3 21/45] sdn: zones: add vxlan-gbp option to vxlan and evpn zones Hannes Laimer
` (24 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
The BPF programs are compiled into the library, and only a full apply
puts a new build's programs into the kernel.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
pve-rs/debian/triggers | 1 +
1 file changed, 1 insertion(+)
diff --git a/pve-rs/debian/triggers b/pve-rs/debian/triggers
index 59dd6885a5f9..e9a60dbfaa48 100644
--- a/pve-rs/debian/triggers
+++ b/pve-rs/debian/triggers
@@ -1 +1,2 @@
activate-noawait pve-api-updates
+activate-noawait pve-microseg-update
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-network v3 21/45] sdn: zones: add vxlan-gbp option to vxlan and evpn zones
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (19 preceding siblings ...)
2026-10-27 8:55 ` [PATCH proxmox-perl-rs v3 20/45] pve-rs: debian: fire the pve-microseg-update trigger on upgrade Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-network v3 22/45] sdn: microseg: add config, API and guest inventory Hannes Laimer
` (23 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
The option enables Group Based Policy (GBP) on the zone's VXLAN
interfaces. The low 16 bits of the packet mark then travel as the
group ID in the VXLAN header, and the receiving node puts them back
into the mark. A VXLAN interface without GBP drops frames with a
group ID, so every VTEP of the zone needs GBP.
Zones that share a VXLAN port must agree on the option, so an SDN
apply refuses a config where they do not. The kernel shares a port's
socket only between interfaces with the same GBP setting, and the
interfaces of the zone that comes second would stay down without any
error.
The VXLAN interfaces of an EVPN zone with the option also get VXLAN
learning off at creation. Since their bridge learning is off,
ifupdown2 otherwise changes them after creation to turn VXLAN
learning off as well. A change cannot carry the GBP flag, so the
kernel then treats the GBP bits as reserved again and drops incoming
GBP frames as malformed.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/Network/SDN/Zones.pm | 8 ++
src/PVE/Network/SDN.pm | 1 +
src/PVE/Network/SDN/Zones.pm | 18 +++++
src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 8 ++
src/PVE/Network/SDN/Zones/VxlanPlugin.pm | 12 +++
.../zones/evpn/gbp/expected_controller_config | 77 +++++++++++++++++++
.../zones/evpn/gbp/expected_sdn_interfaces | 45 +++++++++++
src/test/zones/evpn/gbp/interfaces | 7 ++
src/test/zones/evpn/gbp/sdn_config | 26 +++++++
.../zones/vxlan/gbp/expected_sdn_interfaces | 17 ++++
src/test/zones/vxlan/gbp/interfaces | 7 ++
src/test/zones/vxlan/gbp/sdn_config | 11 +++
12 files changed, 237 insertions(+)
create mode 100644 src/test/zones/evpn/gbp/expected_controller_config
create mode 100644 src/test/zones/evpn/gbp/expected_sdn_interfaces
create mode 100644 src/test/zones/evpn/gbp/interfaces
create mode 100644 src/test/zones/evpn/gbp/sdn_config
create mode 100644 src/test/zones/vxlan/gbp/expected_sdn_interfaces
create mode 100644 src/test/zones/vxlan/gbp/interfaces
create mode 100644 src/test/zones/vxlan/gbp/sdn_config
diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm
index b897cbdff973..b05b8970d82d 100644
--- a/src/PVE/API2/Network/SDN/Zones.pm
+++ b/src/PVE/API2/Network/SDN/Zones.pm
@@ -192,6 +192,14 @@ my $ZONE_PROPERTIES = {
optional => 1,
default => 4789,
},
+ 'vxlan-gbp' => {
+ type => 'boolean',
+ description => "Enable VXLAN Group Based Policy (GBP) on the zone's VXLAN interfaces."
+ . " Carries the 16-bit group ID of the packet mark across hosts. Every VTEP"
+ . " in the zone must have it enabled. Changing it on a zone in use recreates its"
+ . " VXLAN interfaces on the next apply. VXLAN and EVPN zone only.",
+ optional => 1,
+ },
'bridge' => {
type => 'string',
description => 'the bridge for which VLANs should be managed. VLAN & QinQ zone only.',
diff --git a/src/PVE/Network/SDN.pm b/src/PVE/Network/SDN.pm
index 42af00dbf63a..2dc4d9cda704 100644
--- a/src/PVE/Network/SDN.pm
+++ b/src/PVE/Network/SDN.pm
@@ -209,6 +209,7 @@ sub compile_running_cfg {
my $vnets_cfg = PVE::Network::SDN::Vnets::config();
my $zones_cfg = PVE::Network::SDN::Zones::config();
+ PVE::Network::SDN::Zones::check_vxlan_gbp_agrees($zones_cfg);
my $controllers_cfg = PVE::Network::SDN::Controllers::config();
my $subnets_cfg = PVE::Network::SDN::Subnets::config();
my $fabrics_cfg = PVE::Network::SDN::Fabrics::config();
diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm
index ff2097a2ccaa..efcbeb3c1444 100644
--- a/src/PVE/Network/SDN/Zones.pm
+++ b/src/PVE/Network/SDN/Zones.pm
@@ -52,6 +52,24 @@ sub config {
return cfs_read_file("sdn/zones.cfg");
}
+# the kernel shares a VXLAN port's socket only between interfaces with the same GBP setting, so
+# the interfaces of the zone that comes up second stay down and nothing reports it
+sub check_vxlan_gbp_agrees {
+ my ($zones_cfg) = @_;
+
+ my $first_by_port = {};
+ for my $zoneid (sort keys $zones_cfg->{ids}->%*) {
+ my $zone = $zones_cfg->{ids}->{$zoneid};
+ next if $zone->{type} ne 'vxlan' && $zone->{type} ne 'evpn';
+ my $port = $zone->{'vxlan-port'} // 4789;
+ my $gbp = $zone->{'vxlan-gbp'} ? 1 : 0;
+ my $first = $first_by_port->{$port} //= { id => $zoneid, gbp => $gbp };
+ die "zones '$first->{id}' and '$zoneid' share VXLAN port $port"
+ . " and disagree on vxlan-gbp\n"
+ if $first->{gbp} != $gbp;
+ }
+}
+
sub get_plugin_config {
my ($vnet) = @_;
my $zoneid = $vnet->{zone};
diff --git a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
index 0e79707c3746..261525b5a3ed 100644
--- a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
@@ -119,6 +119,7 @@ sub options {
'bridge-disable-mac-learning' => { optional => 1 },
'rt-import' => { optional => 1 },
'vxlan-port' => { optional => 1 },
+ 'vxlan-gbp' => { optional => 1 },
mtu => { optional => 1 },
mac => { optional => 1 },
dns => { optional => 1 },
@@ -151,6 +152,11 @@ sub generate_sdn_config {
my $vrf_iface = "vrf_$zoneid";
my $vrfvxlan = $plugin_config->{'vrf-vxlan'};
my $local_node = PVE::INotify::nodename();
+ # ifupdown2 would otherwise turn VXLAN learning off to match bridge learning in a later link
+ # change, and the kernel then treats the GBP bits as reserved again
+ # TODO: drop once the kernel keeps the reserved bits across a changelink, see
+ # https://lore.kernel.org/all/20261023-b4-vxlan-changelink-gbp-v1-0-f0d13951c09b@proxmox.com/
+ my @gbp = $plugin_config->{'vxlan-gbp'} ? ("vxlan-gbp on", "vxlan-learning off") : ();
die "missing vxlan tag" if !$tag;
die "missing controller" if !$controller;
@@ -224,6 +230,7 @@ sub generate_sdn_config {
push @iface_config, "vxlan-id $tag";
push @iface_config, "vxlan-local-tunnelip $ifaceip" if $ifaceip;
push @iface_config, "vxlan-port $vxlanport" if $vxlanport;
+ push @iface_config, @gbp;
push @iface_config, "bridge-learning off";
push @iface_config, "bridge-arp-nd-suppress on"
if !$plugin_config->{'disable-arp-nd-suppression'};
@@ -328,6 +335,7 @@ sub generate_sdn_config {
push @iface_config, "vxlan-id $vrfvxlan";
push @iface_config, "vxlan-local-tunnelip $ifaceip" if $ifaceip;
push @iface_config, "vxlan-port $vxlanport" if $vxlanport;
+ push @iface_config, @gbp;
push @iface_config, "bridge-learning off";
push @iface_config, "bridge-arp-nd-suppress on"
if !$plugin_config->{'disable-arp-nd-suppression'};
diff --git a/src/PVE/Network/SDN/Zones/VxlanPlugin.pm b/src/PVE/Network/SDN/Zones/VxlanPlugin.pm
index 8eb7e1c1f911..d6f0046478a6 100644
--- a/src/PVE/Network/SDN/Zones/VxlanPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/VxlanPlugin.pm
@@ -44,6 +44,16 @@ sub properties {
type => 'string',
format => 'pve-sdn-fabric-id',
},
+ 'vxlan-gbp' => {
+ description => "Enable VXLAN Group Based Policy (GBP) on the zone's VXLAN"
+ . " interfaces. Carries the 16-bit group ID of the packet mark"
+ . " across hosts. Every VTEP in the zone must have it enabled. Zones that"
+ . " share a VXLAN port must agree on it, an SDN apply refuses a config where"
+ . " they do not. Changing it on a zone in use recreates its VXLAN interfaces"
+ . " on the next apply.",
+ type => 'boolean',
+ optional => 1,
+ },
};
}
@@ -58,6 +68,7 @@ sub options {
dnszone => { optional => 1 },
ipam => { optional => 1 },
fabric => { optional => 1 },
+ 'vxlan-gbp' => { optional => 1 },
};
}
@@ -133,6 +144,7 @@ sub generate_sdn_config {
push @iface_config, "vxlan_remoteip $address";
}
push @iface_config, "vxlan-port $vxlanport" if $vxlanport;
+ push @iface_config, "vxlan-gbp on" if $plugin_config->{'vxlan-gbp'};
push @iface_config, "mtu $mtu" if $mtu;
push(@{ $config->{$vxlan_iface} }, @iface_config) if !$config->{$vxlan_iface};
diff --git a/src/test/zones/evpn/gbp/expected_controller_config b/src/test/zones/evpn/gbp/expected_controller_config
new file mode 100644
index 000000000000..e7923027f01e
--- /dev/null
+++ b/src/test/zones/evpn/gbp/expected_controller_config
@@ -0,0 +1,77 @@
+frr version 10.6.1
+frr defaults datacenter
+hostname localhost
+log syslog informational
+service integrated-vtysh-config
+!
+vrf vrf_myzone
+ vni 1000
+exit-vrf
+!
+router bgp 65000
+ bgp router-id 192.168.0.1
+ no bgp hard-administrative-reset
+ no bgp default ipv4-unicast
+ coalesce-time 1000
+ no bgp graceful-restart notification
+ neighbor VTEP peer-group
+ neighbor VTEP remote-as 65000
+ neighbor VTEP bfd
+ neighbor 192.168.0.2 peer-group VTEP
+ neighbor 192.168.0.3 peer-group VTEP
+ !
+ address-family ipv4 unicast
+ import vrf vrf_myzone
+ exit-address-family
+ !
+ address-family ipv6 unicast
+ import vrf vrf_myzone
+ exit-address-family
+ !
+ address-family l2vpn evpn
+ neighbor VTEP activate
+ neighbor VTEP route-map MAP_VTEP_IN in
+ neighbor VTEP route-map MAP_VTEP_OUT out
+ advertise-all-vni
+ exit-address-family
+exit
+!
+router bgp 65000 vrf vrf_myzone
+ bgp router-id 192.168.0.1
+ no bgp hard-administrative-reset
+ no bgp graceful-restart notification
+ !
+ address-family ipv4 unicast
+ redistribute connected
+ exit-address-family
+ !
+ address-family ipv6 unicast
+ redistribute connected
+ exit-address-family
+ !
+ address-family l2vpn evpn
+ default-originate ipv4
+ default-originate ipv6
+ exit-address-family
+exit
+!
+ip prefix-list only_default seq 1 permit 0.0.0.0/0
+!
+ipv6 prefix-list only_default_v6 seq 1 permit ::/0
+!
+route-map MAP_VTEP_IN deny 1
+ match ip address prefix-list only_default
+exit
+!
+route-map MAP_VTEP_IN deny 2
+ match ipv6 address prefix-list only_default_v6
+exit
+!
+route-map MAP_VTEP_IN permit 3
+exit
+!
+route-map MAP_VTEP_OUT permit 1
+exit
+!
+line vty
+!
diff --git a/src/test/zones/evpn/gbp/expected_sdn_interfaces b/src/test/zones/evpn/gbp/expected_sdn_interfaces
new file mode 100644
index 000000000000..5480b2a1b225
--- /dev/null
+++ b/src/test/zones/evpn/gbp/expected_sdn_interfaces
@@ -0,0 +1,45 @@
+#version:1
+
+auto myvnet
+iface myvnet
+ address 10.0.0.1/24
+ bridge_ports vxlan_myvnet
+ bridge_stp off
+ bridge_fd 0
+ mtu 1450
+ ip-forward on
+ arp-accept on
+ vrf vrf_myzone
+
+auto vrf_myzone
+iface vrf_myzone
+ vrf-table auto
+ post-up ip route del vrf vrf_myzone unreachable default metric 4278198272
+
+auto vrfbr_myzone
+iface vrfbr_myzone
+ bridge-ports vrfvx_myzone
+ bridge_stp off
+ bridge_fd 0
+ mtu 1450
+ vrf vrf_myzone
+
+auto vrfvx_myzone
+iface vrfvx_myzone
+ vxlan-id 1000
+ vxlan-local-tunnelip 192.168.0.1
+ vxlan-gbp on
+ vxlan-learning off
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ mtu 1450
+
+auto vxlan_myvnet
+iface vxlan_myvnet
+ vxlan-id 100
+ vxlan-local-tunnelip 192.168.0.1
+ vxlan-gbp on
+ vxlan-learning off
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ mtu 1450
diff --git a/src/test/zones/evpn/gbp/interfaces b/src/test/zones/evpn/gbp/interfaces
new file mode 100644
index 000000000000..66bb826a44b3
--- /dev/null
+++ b/src/test/zones/evpn/gbp/interfaces
@@ -0,0 +1,7 @@
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1/24
+ gateway 192.168.0.254
+ bridge-ports eth0
+ bridge-stp off
+ bridge-fd 0
diff --git a/src/test/zones/evpn/gbp/sdn_config b/src/test/zones/evpn/gbp/sdn_config
new file mode 100644
index 000000000000..9b3cbb6f893d
--- /dev/null
+++ b/src/test/zones/evpn/gbp/sdn_config
@@ -0,0 +1,26 @@
+{
+ version => 1,
+ vnets => {
+ ids => {
+ myvnet => { tag => "100", type => "vnet", zone => "myzone" },
+ },
+ },
+
+ zones => {
+ ids => { myzone => { ipam => "pve", type => "evpn", controller => "evpnctl", 'vrf-vxlan' => 1000, 'vxlan-gbp' => 1, exitnodes => { 'localhost' => 1 } } },
+ },
+ controllers => {
+ ids => { evpnctl => { type => "evpn", 'peers' => '192.168.0.1,192.168.0.2,192.168.0.3', asn => "65000" } },
+ },
+
+ subnets => {
+ ids => { 'myzone-10.0.0.0-24' => {
+ 'type' => 'subnet',
+ 'vnet' => 'myvnet',
+ 'gateway' => '10.0.0.1',
+ }
+ }
+ }
+}
+
+
diff --git a/src/test/zones/vxlan/gbp/expected_sdn_interfaces b/src/test/zones/vxlan/gbp/expected_sdn_interfaces
new file mode 100644
index 000000000000..c66cff3f0c85
--- /dev/null
+++ b/src/test/zones/vxlan/gbp/expected_sdn_interfaces
@@ -0,0 +1,17 @@
+#version:1
+
+auto myvnet
+iface myvnet
+ bridge_ports vxlan_myvnet
+ bridge_stp off
+ bridge_fd 0
+ mtu 1450
+
+auto vxlan_myvnet
+iface vxlan_myvnet
+ vxlan-id 100
+ vxlan-local-tunnelip 192.168.0.1
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+ vxlan-gbp on
+ mtu 1450
diff --git a/src/test/zones/vxlan/gbp/interfaces b/src/test/zones/vxlan/gbp/interfaces
new file mode 100644
index 000000000000..66bb826a44b3
--- /dev/null
+++ b/src/test/zones/vxlan/gbp/interfaces
@@ -0,0 +1,7 @@
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1/24
+ gateway 192.168.0.254
+ bridge-ports eth0
+ bridge-stp off
+ bridge-fd 0
diff --git a/src/test/zones/vxlan/gbp/sdn_config b/src/test/zones/vxlan/gbp/sdn_config
new file mode 100644
index 000000000000..c61a7de5f371
--- /dev/null
+++ b/src/test/zones/vxlan/gbp/sdn_config
@@ -0,0 +1,11 @@
+{
+ version => 1,
+ vnets => {
+ ids => {
+ myvnet => { tag => 100, type => "vnet", zone => "myzone" },
+ },
+ },
+ zones => {
+ ids => { myzone => { ipam => "pve", type => "vxlan", 'vxlan-gbp' => 1, peers => "192.168.0.1,192.168.0.2,192.168.0.3" } },
+ },
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-network v3 22/45] sdn: microseg: add config, API and guest inventory
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (20 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-network v3 21/45] sdn: zones: add vxlan-gbp option to vxlan and evpn zones Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-network v3 23/45] sdn: dry-run: show pending microseg changes Hannes Laimer
` (22 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
The microseg config holds groups, allow and deny rules between
groups, and assignments of guest NICs to groups. It lives in
sdn/microseg.cfg.
An SDN apply copies the entries into the running config, together
with a 16-bit identity for each class of assigned group sets. Group
sets that no rule tells apart form one class. The apply starts from
the identities of the previous running config, so a class that stays
keeps its identity.
Nothing stores which NICs belong to an assignment, since guests and
their NICs change without an SDN apply. The overview at
/cluster/sdn/microseg/all lists them per assignment, read from the
guest configs of the whole cluster. It shows a guest's NICs only to a
caller with VM.Audit on that guest, as /cluster/resources does.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/Network/SDN.pm | 11 +
src/PVE/API2/Network/SDN/Makefile | 2 +
src/PVE/API2/Network/SDN/Microseg.pm | 215 ++++++++++
.../API2/Network/SDN/Microseg/Assignment.pm | 23 ++
src/PVE/API2/Network/SDN/Microseg/Base.pm | 377 ++++++++++++++++++
src/PVE/API2/Network/SDN/Microseg/Group.pm | 17 +
src/PVE/API2/Network/SDN/Microseg/Makefile | 11 +
src/PVE/API2/Network/SDN/Microseg/Rule.pm | 22 +
src/PVE/Network/SDN.pm | 23 ++
src/PVE/Network/SDN/Makefile | 1 +
src/PVE/Network/SDN/Microseg.pm | 282 +++++++++++++
11 files changed, 984 insertions(+)
create mode 100644 src/PVE/API2/Network/SDN/Microseg.pm
create mode 100644 src/PVE/API2/Network/SDN/Microseg/Assignment.pm
create mode 100644 src/PVE/API2/Network/SDN/Microseg/Base.pm
create mode 100644 src/PVE/API2/Network/SDN/Microseg/Group.pm
create mode 100644 src/PVE/API2/Network/SDN/Microseg/Makefile
create mode 100644 src/PVE/API2/Network/SDN/Microseg/Rule.pm
create mode 100644 src/PVE/Network/SDN/Microseg.pm
diff --git a/src/PVE/API2/Network/SDN.pm b/src/PVE/API2/Network/SDN.pm
index e3c8d9ddecdf..8640607a6b5c 100644
--- a/src/PVE/API2/Network/SDN.pm
+++ b/src/PVE/API2/Network/SDN.pm
@@ -22,6 +22,7 @@ use PVE::API2::Network::SDN::Zones;
use PVE::API2::Network::SDN::Ipams;
use PVE::API2::Network::SDN::Dns;
use PVE::API2::Network::SDN::Fabrics;
+use PVE::API2::Network::SDN::Microseg;
use PVE::API2::Network::SDN::PrefixLists;
use PVE::API2::Network::SDN::RouteMaps;
@@ -57,6 +58,11 @@ __PACKAGE__->register_method({
path => 'fabrics',
});
+__PACKAGE__->register_method({
+ subclass => "PVE::API2::Network::SDN::Microseg",
+ path => 'microseg',
+});
+
__PACKAGE__->register_method({
subclass => "PVE::API2::Network::SDN::PrefixLists",
path => 'prefix-lists',
@@ -99,6 +105,7 @@ __PACKAGE__->register_method({
{ id => 'ipams' },
{ id => 'dns' },
{ id => 'fabrics' },
+ { id => 'microseg' },
{ id => 'prefix-lists' },
{ id => 'route-maps' },
];
@@ -271,6 +278,10 @@ __PACKAGE__->register_method({
PVE::RS::SDN::PrefixLists->running_config($prefix_list_config);
PVE::Network::SDN::PrefixLists::write_config($parsed_prefix_list_config);
+ my $parsed_microseg_config =
+ PVE::Network::SDN::Microseg::running_policy($running_config);
+ PVE::Network::SDN::Microseg::write_config($parsed_microseg_config);
+
PVE::Network::SDN::delete_global_lock() if $lock_token && $release_lock;
};
diff --git a/src/PVE/API2/Network/SDN/Makefile b/src/PVE/API2/Network/SDN/Makefile
index 6b91f8cc0127..3ae11b02eac6 100644
--- a/src/PVE/API2/Network/SDN/Makefile
+++ b/src/PVE/API2/Network/SDN/Makefile
@@ -6,6 +6,7 @@ SOURCES=Vnets.pm\
Dns.pm\
Ips.pm\
Fabrics.pm\
+ Microseg.pm\
PrefixLists.pm\
RouteMaps.pm
@@ -18,4 +19,5 @@ install:
make -C Nodes install
make -C RouteMaps install
make -C PrefixLists install
+ make -C Microseg install
diff --git a/src/PVE/API2/Network/SDN/Microseg.pm b/src/PVE/API2/Network/SDN/Microseg.pm
new file mode 100644
index 000000000000..525fa429ae50
--- /dev/null
+++ b/src/PVE/API2/Network/SDN/Microseg.pm
@@ -0,0 +1,215 @@
+package PVE::API2::Network::SDN::Microseg;
+
+use strict;
+use warnings;
+
+use PVE::JSONSchema qw(get_standard_option);
+
+use PVE::Network::SDN;
+use PVE::Network::SDN::Microseg;
+use PVE::RS::SDN::Microseg;
+
+use PVE::API2::Network::SDN::Microseg::Base;
+use PVE::API2::Network::SDN::Microseg::Group;
+use PVE::API2::Network::SDN::Microseg::Rule;
+use PVE::API2::Network::SDN::Microseg::Assignment;
+
+use PVE::RESTHandler;
+use base qw(PVE::RESTHandler);
+
+__PACKAGE__->register_method({
+ subclass => "PVE::API2::Network::SDN::Microseg::Group",
+ path => 'group',
+});
+
+__PACKAGE__->register_method({
+ subclass => "PVE::API2::Network::SDN::Microseg::Rule",
+ path => 'rule',
+});
+
+__PACKAGE__->register_method({
+ subclass => "PVE::API2::Network::SDN::Microseg::Assignment",
+ path => 'assignment',
+});
+
+__PACKAGE__->register_method({
+ name => 'index',
+ path => '',
+ method => 'GET',
+ description => "SDN microseg index.",
+ permissions => {
+ check => ['perm', '/sdn/microseg', ['SDN.Audit']],
+ },
+ parameters => {
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => "object",
+ properties => {
+ subdir => { type => 'string' },
+ },
+ },
+ links => [{ rel => 'child', href => "{subdir}" }],
+ },
+ code => sub {
+ return [
+ { subdir => 'group' },
+ { subdir => 'rule' },
+ { subdir => 'assignment' },
+ { subdir => 'all' },
+ ];
+ },
+});
+
+my $OBJECT_PROPERTIES = {
+ map { PVE::Network::SDN::Microseg::optional_properties($_)->%* } (
+ PVE::Network::SDN::Microseg::group_properties(0),
+ PVE::Network::SDN::Microseg::rule_properties(0),
+ PVE::Network::SDN::Microseg::assignment_object_properties(),
+ ),
+};
+delete $OBJECT_PROPERTIES->{type};
+
+__PACKAGE__->register_method({
+ name => 'all',
+ path => 'all',
+ method => 'GET',
+ description =>
+ "Every microseg object with its assigned members, plus the group sets and the"
+ . " id space usage.",
+ permissions => {
+ check => ['perm', '/sdn/microseg', ['SDN.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => PVE::API2::Network::SDN::Microseg::Base::list_parameters(),
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ objects => {
+ type => 'array',
+ description => 'Every microseg object across all types.',
+ items => {
+ type => "object",
+ properties => {
+ PVE::API2::Network::SDN::Microseg::Base::list_item_properties()->%*,
+ type => {
+ type => 'string',
+ enum => [
+ 'group',
+ 'rule',
+ $PVE::Network::SDN::Microseg::ASSIGNMENT_TYPES->@*,
+ ],
+ },
+ members => {
+ type => 'array',
+ optional => 1,
+ description =>
+ 'The guests and NICs that belong to the assignment, resolved'
+ . ' against the current guest inventory, without the guests the'
+ . ' caller may not audit. Assignment objects only.',
+ items => {
+ type => 'object',
+ properties => {
+ vmid => get_standard_option('pve-vmid'),
+ nics => {
+ type => 'array',
+ items => { type => 'integer' },
+ description =>
+ 'Indices of the covered netN interfaces.',
+ },
+ },
+ },
+ },
+ $OBJECT_PROPERTIES->%*,
+ },
+ },
+ },
+ 'id-space' => {
+ type => 'object',
+ description => 'Usage of the 16-bit wire identity space in the applied config.',
+ properties => {
+ used => {
+ type => 'integer',
+ description =>
+ 'Ids in use, one per class of group sets the rules treat alike.',
+ },
+ retired => {
+ type => 'integer',
+ description => 'Retired ids still in quarantine. One whose quarantine'
+ . ' has passed is counted until the next allocation drops it.',
+ },
+ total => {
+ type => 'integer',
+ description =>
+ 'Allocatable ids, id 0 stays reserved for untagged traffic.',
+ },
+ },
+ },
+ 'group-sets' => {
+ type => 'array',
+ description =>
+ 'The rules that govern each group set an assignment binds, as source'
+ . ' (outbound) and as destination (inbound).',
+ items => {
+ type => "object",
+ properties => {
+ groups => {
+ type => 'array',
+ items => { type => 'string' },
+ description => 'The group set as an assignment names it, sorted.',
+ },
+ outbound => {
+ type => 'array',
+ items => { type => 'string' },
+ description =>
+ 'Ids of the rules that apply with this group set as source, a'
+ . ' bidirectional rule in either direction.',
+ },
+ inbound => {
+ type => 'array',
+ items => { type => 'string' },
+ description =>
+ 'Ids of the rules that apply with this group set as destination,'
+ . ' a bidirectional rule in either direction.',
+ },
+ },
+ },
+ },
+ },
+ },
+ code => sub {
+ my ($param) = @_;
+
+ my $running_cfg = PVE::Network::SDN::running_config();
+ my $running = $param->{running} && !$param->{pending};
+ my $config =
+ $running
+ ? PVE::Network::SDN::Microseg::running_policy($running_cfg)
+ : PVE::Network::SDN::Microseg::config();
+ my $objects = PVE::API2::Network::SDN::Microseg::Base::list_objects(
+ undef, $param, $config, $running_cfg,
+ );
+
+ my $members = $config->assigned_members(
+ PVE::Network::SDN::Microseg::audited_guest_inventory(),
+ );
+ for my $object (@$objects) {
+ my $id = $object->{id};
+ $object->{members} = $members->{$id} if defined($members->{$id});
+ }
+
+ my $registry = ($running_cfg->{microseg} // {})->{identities};
+
+ return {
+ objects => $objects,
+ 'group-sets' => $config->rules_by_group_set(),
+ 'id-space' => PVE::RS::SDN::Microseg::id_space($registry),
+ };
+ },
+});
+
+1;
diff --git a/src/PVE/API2/Network/SDN/Microseg/Assignment.pm b/src/PVE/API2/Network/SDN/Microseg/Assignment.pm
new file mode 100644
index 000000000000..0cc3f5baf1f1
--- /dev/null
+++ b/src/PVE/API2/Network/SDN/Microseg/Assignment.pm
@@ -0,0 +1,23 @@
+package PVE::API2::Network::SDN::Microseg::Assignment;
+
+use strict;
+use warnings;
+
+use PVE::Network::SDN::Microseg;
+
+use PVE::API2::Network::SDN::Microseg::Base;
+use base qw(PVE::API2::Network::SDN::Microseg::Base);
+
+__PACKAGE__->register_handlers({
+ types => $PVE::Network::SDN::Microseg::ASSIGNMENT_TYPES,
+ name => 'assignment',
+ plural => 'NIC-to-group assignments',
+ properties => \&PVE::Network::SDN::Microseg::assignment_properties,
+ object_properties => \&PVE::Network::SDN::Microseg::assignment_object_properties,
+ create_details => "The id is derived from its matcher and interface, so creating the same"
+ . " assignment again is refused as a duplicate.",
+ generated_id => 1,
+ generated_id_description => "The generated assignment id.",
+});
+
+1;
diff --git a/src/PVE/API2/Network/SDN/Microseg/Base.pm b/src/PVE/API2/Network/SDN/Microseg/Base.pm
new file mode 100644
index 000000000000..dde88d10c7b1
--- /dev/null
+++ b/src/PVE/API2/Network/SDN/Microseg/Base.pm
@@ -0,0 +1,377 @@
+package PVE::API2::Network::SDN::Microseg::Base;
+
+use strict;
+use warnings;
+
+use PVE::Cluster;
+use PVE::Exception qw(raise_param_exc);
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::Tools qw(extract_param);
+
+use PVE::Network::SDN;
+use PVE::Network::SDN::Microseg;
+
+use PVE::RESTHandler;
+use base qw(PVE::RESTHandler);
+
+sub list_parameters {
+ return {
+ running => {
+ type => 'boolean',
+ optional => 1,
+ description => "Display running config.",
+ },
+ pending => {
+ type => 'boolean',
+ optional => 1,
+ description => "Display pending config.",
+ },
+ };
+}
+
+sub list_item_properties {
+ return {
+ id => get_standard_option('pve-sdn-microseg-id'),
+ digest => {
+ type => 'string',
+ description => 'Digest of the microseg configuration.',
+ optional => 1,
+ },
+ state => get_standard_option('pve-sdn-config-state'),
+ pending => {
+ type => 'object',
+ optional => 1,
+ description => 'Changes not yet applied to the running configuration.',
+ },
+ };
+}
+
+my sub typed_entry {
+ my ($config, $types, $id) = @_;
+
+ my $entry = $config->get($id);
+ raise_param_exc({ id => "sdn microseg object '$id' does not exist" })
+ if !$entry || !PVE::Network::SDN::Microseg::type_matches($entry->{type}, $types);
+
+ return $entry;
+}
+
+my sub get_object {
+ my ($types, $id) = @_;
+
+ my $config = PVE::Network::SDN::Microseg::config();
+ my $entry = typed_entry($config, $types, $id);
+
+ $entry->{digest} = $config->digest();
+
+ return $entry;
+}
+
+my sub assert_assignment_kind {
+ my ($type, $param) = @_;
+
+ my $properties = PVE::Network::SDN::Microseg::assignment_properties(0);
+ my $errors = {};
+ for my $property (keys %$properties) {
+ my $kinds = $properties->{$property}->{'instance-types'} or next;
+ $errors->{$property} = "not allowed for type '$type'"
+ if defined($param->{$property}) && !grep { $_ eq $type } @$kinds;
+ }
+ raise_param_exc($errors) if %$errors;
+}
+
+# the config's errors for an empty list do not name the parameter
+my sub assert_non_empty_lists {
+ my ($param) = @_;
+
+ my $errors = {};
+ for my $property (qw(src dst groups)) {
+ my $value = $param->{$property};
+ $errors->{$property} = "must not be empty" if $value && !@$value;
+ }
+ raise_param_exc($errors) if %$errors;
+}
+
+my sub create_object {
+ my ($types, $param) = @_;
+
+ my $type = @$types > 1 ? $param->{type} : $types->[0];
+
+ # the schema does not reject a property of another matcher kind
+ assert_assignment_kind($type, $param)
+ if PVE::Network::SDN::Microseg::type_matches(
+ $type,
+ $PVE::Network::SDN::Microseg::ASSIGNMENT_TYPES,
+ );
+ assert_non_empty_lists($param);
+
+ my $lock_token = extract_param($param, 'lock-token');
+ $param->{type} = $type;
+
+ PVE::Cluster::check_cfs_quorum();
+ mkdir("/etc/pve/sdn");
+
+ return PVE::Network::SDN::lock_sdn_config(
+ sub {
+ my $config = PVE::Network::SDN::Microseg::config();
+ my $id = $config->create($param);
+ PVE::Network::SDN::Microseg::write_config($config);
+ return $id;
+ },
+ "create sdn microseg object failed",
+ $lock_token,
+ );
+}
+
+my sub update_object {
+ my ($types, $id, $param) = @_;
+
+ assert_non_empty_lists($param);
+
+ my $digest = extract_param($param, 'digest');
+ my $lock_token = extract_param($param, 'lock-token');
+
+ PVE::Network::SDN::lock_sdn_config(
+ sub {
+ my $config = PVE::Network::SDN::Microseg::config();
+ my $entry = typed_entry($config, $types, $id);
+
+ PVE::Tools::assert_if_modified($config->digest(), $digest) if $digest;
+
+ $param->{type} = $entry->{type};
+
+ $config->update($id, $param);
+ PVE::Network::SDN::Microseg::write_config($config);
+ },
+ "update sdn microseg object failed",
+ $lock_token,
+ );
+}
+
+my sub delete_object {
+ my ($types, $id, $param) = @_;
+
+ my $lock_token = extract_param($param, 'lock-token');
+
+ PVE::Network::SDN::lock_sdn_config(
+ sub {
+ my $config = PVE::Network::SDN::Microseg::config();
+ typed_entry($config, $types, $id);
+
+ $config->delete($id);
+ PVE::Network::SDN::Microseg::write_config($config);
+ },
+ "delete sdn microseg object failed",
+ $lock_token,
+ );
+}
+
+sub list_objects {
+ my ($types, $param, $config, $running_cfg) = @_;
+
+ my $ids;
+ my $digest;
+ if ($param->{pending}) {
+ $running_cfg //= PVE::Network::SDN::running_config();
+ $config //= PVE::Network::SDN::Microseg::config();
+ my $sections = { ids => $config->to_sections() };
+ my $pending = PVE::Network::SDN::pending_config($running_cfg, $sections, 'microseg');
+ $ids = $pending->{ids};
+ $digest = $config->digest();
+ } elsif ($param->{running}) {
+ $running_cfg //= PVE::Network::SDN::running_config();
+ $ids = ($running_cfg->{microseg} // {})->{ids} // {};
+ } else {
+ $config //= PVE::Network::SDN::Microseg::config();
+ $ids = $config->to_sections();
+ $digest = $config->digest();
+ }
+
+ my $res = [];
+ for my $id (sort keys %$ids) {
+ my $scfg = $ids->{$id};
+ next if defined $types && !PVE::Network::SDN::Microseg::type_matches($scfg->{type}, $types);
+ $scfg->{id} = $id;
+ $scfg->{digest} = $digest if defined $digest;
+ push @$res, $scfg;
+ }
+
+ return $res;
+}
+
+sub register_handlers {
+ my ($class, $def) = @_;
+
+ my ($types, $name, $properties) = $def->@{qw(types name properties)};
+ my $plural = $def->{plural} // "${name}s";
+ my $generated_id = $def->{generated_id};
+
+ my $create_description = "Create a microseg $name.";
+ $create_description .= " $def->{create_details}" if defined($def->{create_details});
+
+ my @create_id = (id => get_standard_option('pve-sdn-microseg-id'));
+ my $create_returns = { type => 'null' };
+ if ($generated_id) {
+ @create_id = ();
+ $create_returns = { type => 'string', description => $def->{generated_id_description} };
+ }
+
+ # with several types the properties declare the type themselves
+ my @type_property;
+ if (@$types == 1) {
+ @type_property = (
+ type => {
+ type => 'string',
+ description => 'Type of the object.',
+ enum => [@$types],
+ },
+ );
+ }
+
+ my $object_properties = ($def->{object_properties} // sub { $properties->(0) })->();
+ my $index_properties = PVE::Network::SDN::Microseg::optional_properties($object_properties);
+
+ $class->register_method({
+ name => 'index',
+ path => '',
+ method => 'GET',
+ description => "List microseg $plural.",
+ permissions => {
+ check => ['perm', '/sdn/microseg', ['SDN.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => list_parameters(),
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => "object",
+ properties => {
+ list_item_properties()->%*, @type_property, $index_properties->%*,
+ },
+ },
+ links => [{ rel => 'child', href => "{id}" }],
+ },
+ code => sub {
+ my ($param) = @_;
+
+ return list_objects($types, $param);
+ },
+ });
+
+ $class->register_method({
+ name => 'read',
+ path => '{id}',
+ method => 'GET',
+ description => "Read one microseg $name.",
+ permissions => {
+ check => ['perm', '/sdn/microseg', ['SDN.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ id => get_standard_option('pve-sdn-microseg-id'),
+ },
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ id => get_standard_option('pve-sdn-microseg-id'),
+ digest => get_standard_option('pve-config-digest'),
+ @type_property,
+ $object_properties->%*,
+ },
+ },
+ code => sub {
+ my ($param) = @_;
+
+ return get_object($types, extract_param($param, 'id'));
+ },
+ });
+
+ $class->register_method({
+ name => 'create',
+ protected => 1,
+ path => '',
+ method => 'POST',
+ description => $create_description,
+ permissions => {
+ check => ['perm', '/sdn/microseg', ['SDN.Allocate']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ @create_id,
+ $properties->(0)->%*,
+ 'lock-token' => get_standard_option('pve-sdn-lock-token'),
+ },
+ },
+ returns => $create_returns,
+ code => sub {
+ my ($param) = @_;
+
+ my $id = create_object($types, $param);
+
+ return $generated_id ? $id : undef;
+ },
+ });
+
+ $class->register_method({
+ name => 'update',
+ protected => 1,
+ path => '{id}',
+ method => 'PUT',
+ description => "Update a microseg $name.",
+ permissions => {
+ check => ['perm', '/sdn/microseg', ['SDN.Allocate']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ id => get_standard_option('pve-sdn-microseg-id'),
+ $properties->(1)->%*,
+ digest => get_standard_option('pve-config-digest'),
+ 'lock-token' => get_standard_option('pve-sdn-lock-token'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $id = extract_param($param, 'id');
+ update_object($types, $id, $param);
+
+ return undef;
+ },
+ });
+
+ $class->register_method({
+ name => 'delete',
+ protected => 1,
+ path => '{id}',
+ method => 'DELETE',
+ description => "Delete a microseg $name.",
+ permissions => {
+ check => ['perm', '/sdn/microseg', ['SDN.Allocate']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ id => get_standard_option('pve-sdn-microseg-id'),
+ 'lock-token' => get_standard_option('pve-sdn-lock-token'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $id = extract_param($param, 'id');
+ delete_object($types, $id, $param);
+
+ return undef;
+ },
+ });
+}
+
+1;
diff --git a/src/PVE/API2/Network/SDN/Microseg/Group.pm b/src/PVE/API2/Network/SDN/Microseg/Group.pm
new file mode 100644
index 000000000000..e1c6a85c5863
--- /dev/null
+++ b/src/PVE/API2/Network/SDN/Microseg/Group.pm
@@ -0,0 +1,17 @@
+package PVE::API2::Network::SDN::Microseg::Group;
+
+use strict;
+use warnings;
+
+use PVE::Network::SDN::Microseg;
+
+use PVE::API2::Network::SDN::Microseg::Base;
+use base qw(PVE::API2::Network::SDN::Microseg::Base);
+
+__PACKAGE__->register_handlers({
+ types => ['group'],
+ name => 'group',
+ properties => \&PVE::Network::SDN::Microseg::group_properties,
+});
+
+1;
diff --git a/src/PVE/API2/Network/SDN/Microseg/Makefile b/src/PVE/API2/Network/SDN/Microseg/Makefile
new file mode 100644
index 000000000000..910265f95c57
--- /dev/null
+++ b/src/PVE/API2/Network/SDN/Microseg/Makefile
@@ -0,0 +1,11 @@
+SOURCES=Base.pm\
+ Group.pm\
+ Rule.pm\
+ Assignment.pm
+
+
+PERL5DIR=${DESTDIR}/usr/share/perl5
+
+.PHONY: install
+install:
+ for i in ${SOURCES}; do install -D -m 0644 $$i ${PERL5DIR}/PVE/API2/Network/SDN/Microseg/$$i; done
diff --git a/src/PVE/API2/Network/SDN/Microseg/Rule.pm b/src/PVE/API2/Network/SDN/Microseg/Rule.pm
new file mode 100644
index 000000000000..575ca1286ed9
--- /dev/null
+++ b/src/PVE/API2/Network/SDN/Microseg/Rule.pm
@@ -0,0 +1,22 @@
+package PVE::API2::Network::SDN::Microseg::Rule;
+
+use strict;
+use warnings;
+
+use PVE::Network::SDN::Microseg;
+
+use PVE::API2::Network::SDN::Microseg::Base;
+use base qw(PVE::API2::Network::SDN::Microseg::Base);
+
+__PACKAGE__->register_handlers({
+ types => ['rule'],
+ name => 'rule',
+ properties => \&PVE::Network::SDN::Microseg::rule_properties,
+ create_details => "The id is derived from the src and dst predicates and from the"
+ . " direction. Creating the same rule again is refused as a duplicate. Change an existing"
+ . " rule's priority or verdict by updating it rather than creating a second rule.",
+ generated_id => 1,
+ generated_id_description => "The generated rule id.",
+});
+
+1;
diff --git a/src/PVE/Network/SDN.pm b/src/PVE/Network/SDN.pm
index 2dc4d9cda704..107095e3ca71 100644
--- a/src/PVE/Network/SDN.pm
+++ b/src/PVE/Network/SDN.pm
@@ -25,6 +25,7 @@ use PVE::Network::SDN::Subnets;
use PVE::Network::SDN::Dhcp;
use PVE::Network::SDN::Frr;
use PVE::Network::SDN::Fabrics;
+use PVE::Network::SDN::Microseg;
use PVE::Network::SDN::RouteMaps;
use PVE::Network::SDN::PrefixLists;
@@ -40,12 +41,26 @@ my $parse_running_cfg = sub {
eval { $cfg = from_json($raw); };
return {} if $@;
+ for my $entry (values((($cfg->{microseg} // {})->{ids} // {})->%*)) {
+ utf8::decode($entry->{comment}) if defined($entry->{comment});
+ }
+
return $cfg;
};
my $write_running_cfg = sub {
my ($filename, $cfg) = @_;
+ # a microseg comment may carry non-ASCII text
+ if (my $ids = ($cfg->{microseg} // {})->{ids}) {
+ my $encoded = {};
+ for my $id (keys %$ids) {
+ $encoded->{$id} = { $ids->{$id}->%* };
+ utf8::encode($encoded->{$id}->{comment}) if defined($encoded->{$id}->{comment});
+ }
+ $cfg = { %$cfg, microseg => { $cfg->{microseg}->%*, ids => $encoded } };
+ }
+
my $json = to_json($cfg);
return $json;
@@ -213,6 +228,7 @@ sub compile_running_cfg {
my $controllers_cfg = PVE::Network::SDN::Controllers::config();
my $subnets_cfg = PVE::Network::SDN::Subnets::config();
my $fabrics_cfg = PVE::Network::SDN::Fabrics::config();
+ my $microseg_cfg = PVE::Network::SDN::Microseg::config();
my $route_maps_cfg = PVE::Network::SDN::RouteMaps::config();
my $prefix_lists_cfg = PVE::Network::SDN::PrefixLists::config();
@@ -221,6 +237,8 @@ sub compile_running_cfg {
my $controllers = { ids => $controllers_cfg->{ids} };
my $subnets = { ids => $subnets_cfg->{ids} };
my $fabrics = { ids => $fabrics_cfg->to_sections() };
+ # the previous identities keep the ids stable
+ my $microseg = $microseg_cfg->render(($cfg->{microseg} // {})->{identities});
my $route_maps = { ids => $route_maps_cfg->to_sections() };
my $prefix_lists = { ids => $prefix_lists_cfg->to_sections() };
@@ -231,6 +249,7 @@ sub compile_running_cfg {
controllers => $controllers,
subnets => $subnets,
fabrics => $fabrics,
+ microseg => $microseg,
'route-maps' => $route_maps,
'prefix-lists' => $prefix_lists,
};
@@ -254,6 +273,7 @@ sub has_pending_changes {
subnets => PVE::Network::SDN::Subnets::config(),
controllers => PVE::Network::SDN::Controllers::config(),
fabrics => { ids => PVE::Network::SDN::Fabrics::config()->to_sections() },
+ microseg => { ids => PVE::Network::SDN::Microseg::config()->to_sections() },
'route-maps' => { ids => PVE::Network::SDN::RouteMaps::config()->to_sections() },
'prefix-lists' => { ids => PVE::Network::SDN::PrefixLists::config()->to_sections() },
};
@@ -522,6 +542,9 @@ sub encode_value {
|| $key eq 'allowed_ips'
|| $key eq 'secondary-controllers'
|| $key eq 'redistribute'
+ || $key eq 'groups'
+ || $key eq 'src'
+ || $key eq 'dst'
) {
if (ref($value) eq 'HASH') {
return join(',', sort keys(%$value));
diff --git a/src/PVE/Network/SDN/Makefile b/src/PVE/Network/SDN/Makefile
index d0b4bce2d067..3042baba6513 100644
--- a/src/PVE/Network/SDN/Makefile
+++ b/src/PVE/Network/SDN/Makefile
@@ -9,6 +9,7 @@ SOURCES=Vnets.pm\
Dhcp.pm\
Fabrics.pm\
Frr.pm\
+ Microseg.pm\
PrefixLists.pm\
RouteMaps.pm\
WireGuard.pm
diff --git a/src/PVE/Network/SDN/Microseg.pm b/src/PVE/Network/SDN/Microseg.pm
new file mode 100644
index 000000000000..05c432d4eec7
--- /dev/null
+++ b/src/PVE/Network/SDN/Microseg.pm
@@ -0,0 +1,282 @@
+package PVE::Network::SDN::Microseg;
+
+use strict;
+use warnings;
+
+use PVE::Cluster qw(cfs_register_file cfs_read_file cfs_write_file);
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::RPCEnvironment;
+use PVE::Tools;
+
+use PVE::RS::SDN::Microseg;
+
+PVE::JSONSchema::register_format(
+ 'pve-sdn-microseg-id',
+ sub {
+ my ($id) = @_;
+ die "microseg id '$id' contains illegal characters or has the wrong length\n"
+ if $id !~ m/^[a-zA-Z0-9][a-zA-Z0-9_-]{0,30}[a-zA-Z0-9]$/;
+ return $id;
+ },
+);
+
+PVE::JSONSchema::register_standard_option(
+ 'pve-sdn-microseg-id',
+ {
+ description => "The microseg object identifier.",
+ type => 'string',
+ format => 'pve-sdn-microseg-id',
+ },
+);
+
+cfs_register_file('sdn/microseg.cfg', \&parse_microseg_config, \&write_microseg_config);
+
+our $ASSIGNMENT_TYPES = ['guest-assignment'];
+
+sub parse_microseg_config {
+ my ($filename, $raw) = @_;
+ return $raw // '';
+}
+
+sub write_microseg_config {
+ my ($filename, $config) = @_;
+ return $config // '';
+}
+
+sub config {
+ my $microseg_config = cfs_read_file("sdn/microseg.cfg");
+ return PVE::RS::SDN::Microseg->config($microseg_config);
+}
+
+sub running_policy {
+ my ($running_cfg) = @_;
+ my $microseg_config = ($running_cfg->{microseg} // {})->{ids} // {};
+ return PVE::RS::SDN::Microseg->running_config($microseg_config);
+}
+
+sub write_config {
+ my ($config) = @_;
+ cfs_write_file("sdn/microseg.cfg", $config->to_raw(), 1);
+}
+
+my $MAX_NIC_INDEX = 31;
+
+my $COMMENT_PROPERTY = {
+ type => 'string',
+ optional => 1,
+ maxLength => 256,
+ description => "Free-form comment.",
+};
+
+my @MATCH_KINDS = ('any', 'all', 'exact');
+my @RULE_MATCH_KINDS = (@MATCH_KINDS, 'every');
+
+my sub delete_property {
+ my (@keys) = @_;
+
+ return {
+ type => 'array',
+ optional => 1,
+ items => {
+ type => 'string',
+ enum => [@keys],
+ },
+ };
+}
+
+sub group_properties {
+ my ($update) = @_;
+
+ my $properties = {
+ comment => $COMMENT_PROPERTY,
+ };
+
+ if ($update) {
+ $properties->{delete} = delete_property('comment');
+ }
+
+ return $properties;
+}
+
+sub rule_properties {
+ my ($update) = @_;
+
+ my $properties = {
+ allow => {
+ type => 'boolean',
+ optional => $update,
+ description =>
+ "0 = deny, 1 = allow. Traffic no rule matches is denied, and there is"
+ . " no connection tracking.",
+ },
+ prio => {
+ type => 'integer',
+ minimum => 0,
+ maximum => 65535,
+ optional => 1,
+ ($update ? () : (default => 0)),
+ description => "Priority. The highest-priority matching rule wins."
+ . " At a tie, a deny overrides an allow.",
+ },
+ comment => $COMMENT_PROPERTY,
+ };
+
+ # the id is derived from these
+ if (!$update) {
+ $properties->{src} = {
+ type => 'array',
+ items => get_standard_option('pve-sdn-microseg-id'),
+ optional => 1,
+ description => "Groups the source predicate is evaluated against, left out only"
+ . " with 'every'. The built-in 'untagged' group matches traffic that carries no"
+ . " group, for example gateway, DHCP, and ARP replies. Alone it works with each"
+ . " match mode that lists groups, together with other groups only with 'any'.",
+ };
+ $properties->{'src-match'} = {
+ type => 'string',
+ enum => [@RULE_MATCH_KINDS],
+ optional => 1,
+ default => 'any',
+ description => "How the source predicate matches. 'any' shares a listed group,"
+ . " 'all' has every listed group, 'exact' equals the listed set, 'every' lists"
+ . " none and fires on each interface that has a group.",
+ };
+ $properties->{dst} = {
+ type => 'array',
+ items => get_standard_option('pve-sdn-microseg-id'),
+ optional => 1,
+ description =>
+ "Groups the destination predicate is evaluated against, left out only with"
+ . " 'every'. Every enforced interface has a group, so the built-in 'untagged'"
+ . " group never matches a destination. It is accepted only with 'any' next to"
+ . " other groups, where it is ignored.",
+ };
+ $properties->{'dst-match'} = {
+ type => 'string',
+ enum => [@RULE_MATCH_KINDS],
+ optional => 1,
+ default => 'any',
+ description => "How the destination predicate matches, as for the source.",
+ };
+ $properties->{bidirectional} = {
+ type => 'boolean',
+ optional => 1,
+ default => 0,
+ description => "Also apply the rule with the source and destination predicates"
+ . " swapped. It is exactly that pair of rules and adds no connection tracking."
+ . " Not with the 'untagged' group.",
+ };
+ } else {
+ $properties->{delete} = delete_property('comment');
+ }
+
+ return $properties;
+}
+
+sub assignment_properties {
+ my ($update) = @_;
+
+ my $properties = {
+ groups => {
+ type => 'array',
+ items => get_standard_option('pve-sdn-microseg-id'),
+ optional => $update,
+ description => "Groups the matching NICs belong to.",
+ },
+ position => {
+ type => 'integer',
+ minimum => 0,
+ optional => 1,
+ description => "Where to put the assignment in the match order, 0 being the first"
+ . " place. A position past the end puts it last. A NIC belongs to the first"
+ . " assignment that covers it."
+ . ($update ? '' : ' A new assignment without a position goes first.'),
+ },
+ comment => $COMMENT_PROPERTY,
+ };
+
+ if (!$update) {
+ $properties->{type} = {
+ type => 'string',
+ enum => $ASSIGNMENT_TYPES,
+ description => "The matcher kind. 'guest-assignment' binds a specific guest.",
+ };
+ $properties->{vmid} = get_standard_option(
+ 'pve-vmid',
+ {
+ 'type-property' => 'type',
+ 'instance-types' => ['guest-assignment'],
+ description => "The guest whose NICs to bind.",
+ },
+ );
+ $properties->{iface} = {
+ type => 'integer',
+ minimum => 0,
+ maximum => $MAX_NIC_INDEX,
+ optional => 1,
+ description => "The guest's netN interface, or all of the guest's NICs if unset.",
+ };
+ } else {
+ $properties->{delete} = delete_property('comment');
+ }
+
+ return $properties;
+}
+
+sub assignment_object_properties {
+ my $properties = assignment_properties(0);
+
+ delete $properties->{position};
+ $properties->{order} = {
+ type => 'integer',
+ minimum => 0,
+ optional => 1,
+ description => 'Place in the match order, the lowest number first.',
+ };
+
+ return $properties;
+}
+
+# with pending=1 a new object has only its type, state and pending changes
+sub optional_properties {
+ my ($properties) = @_;
+
+ return { map { $_ => { $properties->{$_}->%*, optional => 1 } } keys $properties->%* };
+}
+
+sub guest_inventory {
+ my @props = map { "net$_" } 0 .. $MAX_NIC_INDEX;
+
+ my $by_vmid = PVE::Cluster::get_guest_config_properties(\@props);
+
+ my $inventory = [];
+ for my $vmid (sort { $a <=> $b } keys %$by_vmid) {
+ my $guest = $by_vmid->{$vmid};
+
+ my @nics = grep { defined $guest->{"net$_"} } 0 .. $MAX_NIC_INDEX;
+
+ push @$inventory,
+ {
+ vmid => $vmid,
+ nics => \@nics,
+ };
+ }
+
+ return $inventory;
+}
+
+sub audited_guest_inventory {
+ my $rpcenv = PVE::RPCEnvironment::get();
+ my $authuser = $rpcenv->get_user();
+
+ return [grep { $rpcenv->check($authuser, "/vms/$_->{vmid}", ['VM.Audit'], 1) }
+ guest_inventory()->@*];
+}
+
+sub type_matches {
+ my ($got, $want) = @_;
+ return 0 if !defined $got;
+ return (grep { $_ eq $got } @$want) ? 1 : 0;
+}
+
+1;
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-network v3 23/45] sdn: dry-run: show pending microseg changes
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (21 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-network v3 22/45] sdn: microseg: add config, API and guest inventory Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-network v3 24/45] sdn: microseg: enforce the policy on guest NICs Hannes Laimer
` (21 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
Microseg has no per-node generated file to diff, so the dry-run
summarizes the running and the pending config as text and returns the
diff of the two. Both summaries resolve memberships against the guests
as they are now, so the diff shows only what the pending config
changes.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/Network/SDN.pm | 36 ++++++++++++++++++++++++++++++++++--
1 file changed, 34 insertions(+), 2 deletions(-)
diff --git a/src/PVE/API2/Network/SDN.pm b/src/PVE/API2/Network/SDN.pm
index 8640607a6b5c..8a28b2f58e85 100644
--- a/src/PVE/API2/Network/SDN.pm
+++ b/src/PVE/API2/Network/SDN.pm
@@ -367,11 +367,14 @@ __PACKAGE__->register_method({
});
sub get_diff {
- my ($filename_one, $filename_two) = @_;
+ my ($filename_one, $filename_two, $label_one, $label_two) = @_;
my $diff = '';
- my $cmd = ['/usr/bin/diff', '-b', '-N', '-u', $filename_one, $filename_two];
+ my $cmd = ['/usr/bin/diff', '-b', '-N', '-u'];
+ push @$cmd, '--label', $label_one, '--label', $label_two
+ if defined($label_one) && defined($label_two);
+ push @$cmd, $filename_one, $filename_two;
PVE::Tools::run_command(
$cmd,
noerr => 1,
@@ -419,6 +422,15 @@ __PACKAGE__->register_method({
description =>
'The difference between the current and pending /etc/network/interfaces.d/sdn configuration.',
},
+ "microseg-diff" => {
+ type => 'string',
+ optional => 1,
+ description =>
+ 'The difference between the current and pending microsegmentation state. The'
+ . ' state covers the allocated identities, the per-NIC group membership, and'
+ . ' the rules, without the interfaces of guests the caller may not'
+ . ' audit. It is left out without SDN.Audit on /sdn/microseg.',
+ },
},
},
code => sub {
@@ -449,6 +461,26 @@ __PACKAGE__->register_method({
$return_value->{"interfaces-diff"} =
get_diff('/etc/network/interfaces.d/sdn', $interfaces_tmp_filename);
+ my $rpcenv = PVE::RPCEnvironment::get();
+ my $authuser = $rpcenv->get_user();
+
+ if ($rpcenv->check($authuser, '/sdn/microseg', ['SDN.Audit'], 1)) {
+ my $current_cfg = PVE::Network::SDN::running_config();
+ my $inventory = PVE::Network::SDN::Microseg::audited_guest_inventory();
+ my $microseg_old =
+ PVE::RS::SDN::Microseg::summary($current_cfg->{microseg} // {}, $inventory);
+ my $microseg_new =
+ PVE::RS::SDN::Microseg::summary($running_cfg->{microseg} // {}, $inventory);
+ my ($microseg_old_filename, $microseg_old_fh) =
+ PVE::File::tempfile_contents($microseg_old, 0700);
+ my ($microseg_new_filename, $microseg_new_fh) =
+ PVE::File::tempfile_contents($microseg_new, 0700);
+ $return_value->{"microseg-diff"} =
+ get_diff($microseg_old_filename, $microseg_new_filename, 'current', 'pending');
+ close($microseg_old_fh);
+ close($microseg_new_fh);
+ }
+
close($frr_tmp_fh);
close($interfaces_tmp_fh);
return $return_value;
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-network v3 24/45] sdn: microseg: enforce the policy on guest NICs
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (22 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-network v3 23/45] sdn: dry-run: show pending microseg changes Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-network v3 25/45] sdn: microseg: add tag matcher Hannes Laimer
` (20 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
A guest NIC is enforced when it is plugged, and if attaching the
enforcement fails, so does the guest start. A full pass enforces
every guest NIC of the node. The package runs one when it is
configured and when libpve-rs-perl is upgraded, since the BPF
programs are compiled into that library.
A running config without a microseg block, as a version without
microseg support writes it, says nothing about the policy. A pass and
a plug leave enforcement as it is and warn where a policy is
configured.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
debian/libpve-network-perl.postinst | 21 ++++++++++
debian/triggers | 1 +
src/PVE/Network/SDN/Microseg.pm | 65 ++++++++++++++++++++++++++++-
src/PVE/Network/SDN/Zones.pm | 5 +++
4 files changed, 91 insertions(+), 1 deletion(-)
diff --git a/debian/libpve-network-perl.postinst b/debian/libpve-network-perl.postinst
index 99faedf48f56..88e9675b6928 100644
--- a/debian/libpve-network-perl.postinst
+++ b/debian/libpve-network-perl.postinst
@@ -22,6 +22,18 @@ migrate_mac_cache() {
fi
}
+# The BPF programs are compiled into libpve-rs-perl, whose upgrade fires the
+# pve-microseg-update trigger so a full pass installs the new build. When both
+# packages are upgraded in one run, this package is only unpacked when the
+# library is set up, so dpkg drops the trigger and the configure runs the pass.
+apply_microseg() {
+ test -f /etc/pve/local/pve-ssl.pem || return 0
+ test -e /proxmox_install_mode && return 0
+
+ perl -MPVE::Network::SDN::Microseg -e 'PVE::Network::SDN::Microseg::apply_all()' \
+ || echo "applying the microsegmentation policy failed, run an SDN apply or a network reload"
+}
+
case "$1" in
configure)
# TODO: remove with PVE 9+
@@ -29,6 +41,15 @@ case "$1" in
migrate_ipam_db
migrate_mac_cache
fi
+ apply_microseg
+ ;;
+
+ triggered)
+ case " $2 " in
+ *" pve-microseg-update "*)
+ apply_microseg
+ ;;
+ esac
;;
esac
diff --git a/debian/triggers b/debian/triggers
index 59dd6885a5f9..30113fe73944 100644
--- a/debian/triggers
+++ b/debian/triggers
@@ -1 +1,2 @@
activate-noawait pve-api-updates
+interest-noawait pve-microseg-update
diff --git a/src/PVE/Network/SDN/Microseg.pm b/src/PVE/Network/SDN/Microseg.pm
index 05c432d4eec7..3cd9c3a8ed35 100644
--- a/src/PVE/Network/SDN/Microseg.pm
+++ b/src/PVE/Network/SDN/Microseg.pm
@@ -5,10 +5,13 @@ use warnings;
use PVE::Cluster qw(cfs_register_file cfs_read_file cfs_write_file);
use PVE::JSONSchema qw(get_standard_option);
+use PVE::RESTEnvironment qw(log_warn);
use PVE::RPCEnvironment;
use PVE::Tools;
use PVE::RS::SDN::Microseg;
+use PVE::RS::SDN::Microseg::ExclusiveLock;
+use PVE::RS::SDN::Microseg::SharedLock;
PVE::JSONSchema::register_format(
'pve-sdn-microseg-id',
@@ -245,9 +248,11 @@ sub optional_properties {
}
sub guest_inventory {
+ my (%opts) = @_;
+
my @props = map { "net$_" } 0 .. $MAX_NIC_INDEX;
- my $by_vmid = PVE::Cluster::get_guest_config_properties(\@props);
+ my $by_vmid = PVE::Cluster::get_guest_config_properties(\@props, $opts{vmid});
my $inventory = [];
for my $vmid (sort { $a <=> $b } keys %$by_vmid) {
@@ -279,4 +284,62 @@ sub type_matches {
return (grep { $_ eq $got } @$want) ? 1 : 0;
}
+my sub parse_guest_nic {
+ my ($iface) = @_;
+
+ return $iface =~ m/^(?:tap|veth)(\d+)i(\d+)$/;
+}
+
+# the guest as the inventory lists it, with nothing but its vmid when it has no NICs
+my sub guest_info {
+ my ($vmid) = @_;
+
+ my ($guest) = guest_inventory(vmid => $vmid)->@*;
+
+ return $guest // { vmid => int($vmid), nics => [] };
+}
+
+# an unmounted pmxcfs reads like a config that was never committed, and enforcement would be dropped
+my sub read_running_config {
+ PVE::Cluster::check_cfs_is_mounted();
+ return PVE::Cluster::get_config('sdn/.running-config');
+}
+
+# a config that cannot be read may still hold a policy
+my sub is_configured {
+ my $sections = eval { config()->to_sections() };
+ return !defined($sections) || scalar(%$sections);
+}
+
+sub apply_interface {
+ my ($iface, $firewall_bridge) = @_;
+
+ my ($vmid) = parse_guest_nic($iface);
+ return if !defined($vmid);
+
+ $firewall_bridge = $firewall_bridge ? 1 : 0;
+
+ my $lock = PVE::RS::SDN::Microseg::SharedLock->new();
+ my $raw = read_running_config();
+ my $guest = guest_info($vmid);
+ if (!$lock->apply_iface($raw, $guest, $iface, $firewall_bridge)) {
+ log_warn("microseg: the running config carries no microseg block,"
+ . " '$iface' stays unenforced. looks like a node is running an older version")
+ if is_configured();
+ }
+}
+
+sub apply_all {
+ eval {
+ my $lock = PVE::RS::SDN::Microseg::ExclusiveLock->new();
+ my $raw = read_running_config();
+ $lock->apply($raw, guest_inventory())
+ or !is_configured()
+ or log_warn("microseg: the running config carries no microseg block, as written"
+ . " before the upgrade or by a node without microsegmentation support, leaving"
+ . " enforcement unchanged");
+ };
+ die "microseg: failed to apply running config: $@" if $@;
+}
+
1;
diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm
index efcbeb3c1444..da24ef16dddb 100644
--- a/src/PVE/Network/SDN/Zones.pm
+++ b/src/PVE/Network/SDN/Zones.pm
@@ -11,6 +11,7 @@ use PVE::IPRoute2;
use PVE::Network;
use PVE::Network::SDN::Vnets;
+use PVE::Network::SDN::Microseg;
use PVE::Network::SDN::Zones::VlanPlugin;
use PVE::Network::SDN::Zones::QinQPlugin;
use PVE::Network::SDN::Zones::VxlanPlugin;
@@ -364,6 +365,10 @@ sub veth_create {
sub tap_plug {
my ($iface, $bridge, $tag, $firewall, $trunks, $rate) = @_;
+ # attach enforcement before bridging, so an assigned NIC never passes traffic unenforced
+ eval { PVE::Network::SDN::Microseg::apply_interface($iface, $firewall) };
+ die "microseg: refusing to bring up '$iface' unenforced: $@" if $@;
+
my $vnet = PVE::Network::SDN::Vnets::get_vnet($bridge, 1);
if (!$vnet) { # fallback for classic bridge
my $interfaces_config = PVE::INotify::read_file('interfaces');
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-network v3 25/45] sdn: microseg: add tag matcher
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (23 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-network v3 24/45] sdn: microseg: enforce the policy on guest NICs Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-network v3 26/45] sdn: microseg: add name regex matcher Hannes Laimer
` (19 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
A tag matcher binds every guest with the given tags to a set of
groups, guests created later included.
A tag the matchers name moves a guest between groups, an SDN change
like putting a NIC on a vnet. So a new permission check requires
SDN.Use on /sdn/microseg when a guest gains or loses such a tag.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
debian/control | 1 +
src/PVE/API2/Network/SDN/Microseg/Base.pm | 7 ++-
src/PVE/Network/SDN.pm | 1 +
src/PVE/Network/SDN/Microseg.pm | 74 +++++++++++++++++++----
4 files changed, 71 insertions(+), 12 deletions(-)
diff --git a/debian/control b/debian/control
index 86a6694d3a5f..f25e2ccfdf2a 100644
--- a/debian/control
+++ b/debian/control
@@ -41,6 +41,7 @@ Description: Proxmox VE's SDN (Software Defined Network) stack
Package: libpve-network-api-perl
Architecture: all
Depends: libpve-common-perl (>= 9.0.2),
+ libpve-guest-common-perl,
pve-firewall (>= 5.1.0~),
pve-cluster (>= 9.0.1),
libpve-network-perl (= ${binary:Version}),
diff --git a/src/PVE/API2/Network/SDN/Microseg/Base.pm b/src/PVE/API2/Network/SDN/Microseg/Base.pm
index dde88d10c7b1..58ec75c6fb12 100644
--- a/src/PVE/API2/Network/SDN/Microseg/Base.pm
+++ b/src/PVE/API2/Network/SDN/Microseg/Base.pm
@@ -5,6 +5,7 @@ use warnings;
use PVE::Cluster;
use PVE::Exception qw(raise_param_exc);
+use PVE::GuestHelpers;
use PVE::JSONSchema qw(get_standard_option);
use PVE::Tools qw(extract_param);
@@ -85,7 +86,7 @@ my sub assert_non_empty_lists {
my ($param) = @_;
my $errors = {};
- for my $property (qw(src dst groups)) {
+ for my $property (qw(src dst groups tags)) {
my $value = $param->{$property};
$errors->{$property} = "must not be empty" if $value && !@$value;
}
@@ -108,6 +109,10 @@ my sub create_object {
my $lock_token = extract_param($param, 'lock-token');
$param->{type} = $type;
+ if ($param->{tags}) {
+ $param->{tags} = PVE::GuestHelpers::get_unique_tags($param->{tags}, 1);
+ }
+
PVE::Cluster::check_cfs_quorum();
mkdir("/etc/pve/sdn");
diff --git a/src/PVE/Network/SDN.pm b/src/PVE/Network/SDN.pm
index 107095e3ca71..0aef192c2815 100644
--- a/src/PVE/Network/SDN.pm
+++ b/src/PVE/Network/SDN.pm
@@ -543,6 +543,7 @@ sub encode_value {
|| $key eq 'secondary-controllers'
|| $key eq 'redistribute'
|| $key eq 'groups'
+ || $key eq 'tags'
|| $key eq 'src'
|| $key eq 'dst'
) {
diff --git a/src/PVE/Network/SDN/Microseg.pm b/src/PVE/Network/SDN/Microseg.pm
index 3cd9c3a8ed35..8090043268a7 100644
--- a/src/PVE/Network/SDN/Microseg.pm
+++ b/src/PVE/Network/SDN/Microseg.pm
@@ -34,7 +34,7 @@ PVE::JSONSchema::register_standard_option(
cfs_register_file('sdn/microseg.cfg', \&parse_microseg_config, \&write_microseg_config);
-our $ASSIGNMENT_TYPES = ['guest-assignment'];
+our $ASSIGNMENT_TYPES = ['guest-assignment', 'tag-assignment'];
sub parse_microseg_config {
my ($filename, $raw) = @_;
@@ -202,7 +202,8 @@ sub assignment_properties {
$properties->{type} = {
type => 'string',
enum => $ASSIGNMENT_TYPES,
- description => "The matcher kind. 'guest-assignment' binds a specific guest.",
+ description => "The matcher kind. 'guest-assignment' binds a specific guest,"
+ . " 'tag-assignment' every guest carrying the matched tags.",
};
$properties->{vmid} = get_standard_option(
'pve-vmid',
@@ -212,6 +213,27 @@ sub assignment_properties {
description => "The guest whose NICs to bind.",
},
);
+ $properties->{tags} = {
+ type => 'array',
+ items => {
+ type => 'string',
+ format => 'pve-tag',
+ },
+ 'type-property' => 'type',
+ 'instance-types' => ['tag-assignment'],
+ description => "The guest tags to match. They are normalized like the tags on a"
+ . " guest, so case only matters with a case-sensitive datacenter tag style.",
+ };
+ $properties->{'tag-match'} = {
+ type => 'string',
+ enum => [@MATCH_KINDS],
+ optional => 1,
+ default => 'any',
+ 'type-property' => 'type',
+ 'instance-types' => ['tag-assignment'],
+ description => "How tags match. 'any' shares a listed tag, 'all' carries every"
+ . " listed tag, 'exact' equals the listed set.",
+ };
$properties->{iface} = {
type => 'integer',
minimum => 0,
@@ -247,10 +269,36 @@ sub optional_properties {
return { map { $_ => { $properties->{$_}->%*, optional => 1 } } keys $properties->%* };
}
+sub policy_tags {
+ my $tags = {};
+ for my $entry (values config()->to_sections()->%*) {
+ next if ($entry->{type} // '') ne 'tag-assignment';
+ $tags->{$_} = 1 for ($entry->{tags} // [])->@*;
+ }
+ return $tags;
+}
+
+# adding or removing a tag the assignments name can move the guest between groups, an SDN action
+# like putting a NIC on a vnet, so it needs SDN.Use on the policy. Both lists are tags as stored
+# in the guest config.
+sub check_tag_access {
+ my ($rpcenv, $authuser, $old, $new) = @_;
+
+ my %old = map { $_ => 1 } PVE::Tools::split_list($old // '');
+ my %new = map { $_ => 1 } PVE::Tools::split_list($new // '');
+ my $policy_tags = policy_tags();
+ for my $tag (keys %old, keys %new) {
+ next if ($old{$tag} // 0) == ($new{$tag} // 0) || !$policy_tags->{$tag};
+ $rpcenv->check($authuser, '/sdn/microseg', ['SDN.Use']);
+ return;
+ }
+}
+
sub guest_inventory {
my (%opts) = @_;
- my @props = map { "net$_" } 0 .. $MAX_NIC_INDEX;
+ # the data plane takes a guest's NICs from the node, so an apply leaves them out
+ my @props = ('tags', $opts{skip_nics} ? () : map { "net$_" } 0 .. $MAX_NIC_INDEX);
my $by_vmid = PVE::Cluster::get_guest_config_properties(\@props, $opts{vmid});
@@ -258,11 +306,14 @@ sub guest_inventory {
for my $vmid (sort { $a <=> $b } keys %$by_vmid) {
my $guest = $by_vmid->{$vmid};
+ my @tags = PVE::Tools::split_list($guest->{tags});
+
my @nics = grep { defined $guest->{"net$_"} } 0 .. $MAX_NIC_INDEX;
push @$inventory,
{
vmid => $vmid,
+ tags => \@tags,
nics => \@nics,
};
}
@@ -290,13 +341,13 @@ my sub parse_guest_nic {
return $iface =~ m/^(?:tap|veth)(\d+)i(\d+)$/;
}
-# the guest as the inventory lists it, with nothing but its vmid when it has no NICs
+# the guest as the inventory lists it, with nothing but its vmid when it has no tags
my sub guest_info {
my ($vmid) = @_;
- my ($guest) = guest_inventory(vmid => $vmid)->@*;
+ my ($guest) = guest_inventory(vmid => $vmid, skip_nics => 1)->@*;
- return $guest // { vmid => int($vmid), nics => [] };
+ return $guest // { vmid => int($vmid), tags => [], nics => [] };
}
# an unmounted pmxcfs reads like a config that was never committed, and enforcement would be dropped
@@ -333,11 +384,12 @@ sub apply_all {
eval {
my $lock = PVE::RS::SDN::Microseg::ExclusiveLock->new();
my $raw = read_running_config();
- $lock->apply($raw, guest_inventory())
- or !is_configured()
- or log_warn("microseg: the running config carries no microseg block, as written"
- . " before the upgrade or by a node without microsegmentation support, leaving"
- . " enforcement unchanged");
+ if (!$lock->apply($raw, guest_inventory(skip_nics => 1))) {
+ log_warn("microseg: the running config carries no microseg block, as written"
+ . " before the upgrade or by a node without microsegmentation support, leaving"
+ . " enforcement unchanged")
+ if is_configured();
+ }
};
die "microseg: failed to apply running config: $@" if $@;
}
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-network v3 26/45] sdn: microseg: add name regex matcher
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (24 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-network v3 25/45] sdn: microseg: add tag matcher Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-network v3 27/45] sdn: microseg: add an update for a running NIC Hannes Laimer
` (18 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
A name regex matcher binds every guest whose name matches a regular
expression. For a container the name is its hostname.
A rename into or out of the names a matcher covers moves the guest
between groups. So a check for renames asks for SDN.Use on
/sdn/microseg, like the one for tag changes.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/Network/SDN/Microseg.pm | 43 +++++++++++++++++++++++++++++----
1 file changed, 38 insertions(+), 5 deletions(-)
diff --git a/src/PVE/Network/SDN/Microseg.pm b/src/PVE/Network/SDN/Microseg.pm
index 8090043268a7..bdfde447b320 100644
--- a/src/PVE/Network/SDN/Microseg.pm
+++ b/src/PVE/Network/SDN/Microseg.pm
@@ -32,9 +32,18 @@ PVE::JSONSchema::register_standard_option(
},
);
+PVE::JSONSchema::register_format(
+ 'pve-sdn-microseg-name-regex',
+ sub {
+ my ($regex) = @_;
+ die "name regex '$regex' contains non-ASCII characters\n" if $regex =~ m/[^\x00-\x7f]/;
+ return $regex;
+ },
+);
+
cfs_register_file('sdn/microseg.cfg', \&parse_microseg_config, \&write_microseg_config);
-our $ASSIGNMENT_TYPES = ['guest-assignment', 'tag-assignment'];
+our $ASSIGNMENT_TYPES = ['guest-assignment', 'tag-assignment', 'regex-assignment'];
sub parse_microseg_config {
my ($filename, $raw) = @_;
@@ -203,7 +212,8 @@ sub assignment_properties {
type => 'string',
enum => $ASSIGNMENT_TYPES,
description => "The matcher kind. 'guest-assignment' binds a specific guest,"
- . " 'tag-assignment' every guest carrying the matched tags.",
+ . " 'tag-assignment' every guest carrying the matched tags, 'regex-assignment'"
+ . " every guest whose name matches a regex.",
};
$properties->{vmid} = get_standard_option(
'pve-vmid',
@@ -234,6 +244,17 @@ sub assignment_properties {
description => "How tags match. 'any' shares a listed tag, 'all' carries every"
. " listed tag, 'exact' equals the listed set.",
};
+ $properties->{'name-regex'} = {
+ type => 'string',
+ format => 'pve-sdn-microseg-name-regex',
+ minLength => 1,
+ maxLength => 1024,
+ 'type-property' => 'type',
+ 'instance-types' => ['regex-assignment'],
+ description => "A regular expression matched against each guest's name, ASCII only."
+ . " It matches anywhere in the name, so anchor it with '^' and '\$' for a"
+ . " whole-name match. A guest without a name is never matched.",
+ };
$properties->{iface} = {
type => 'integer',
minimum => 0,
@@ -294,11 +315,22 @@ sub check_tag_access {
}
}
+# a rename that changes which regex assignments match can move the guest between groups like a
+# tag change, so it needs SDN.Use on the policy. The match runs in the config, so it uses the
+# regex engine that resolves the NICs.
+sub check_name_access {
+ my ($rpcenv, $authuser, $old, $new) = @_;
+
+ return if !config()->name_match_changed($old, $new);
+ $rpcenv->check($authuser, '/sdn/microseg', ['SDN.Use']);
+}
+
sub guest_inventory {
my (%opts) = @_;
- # the data plane takes a guest's NICs from the node, so an apply leaves them out
- my @props = ('tags', $opts{skip_nics} ? () : map { "net$_" } 0 .. $MAX_NIC_INDEX);
+ # the data plane takes a guest's NICs from the node
+ my @props =
+ ('tags', 'name', 'hostname', $opts{skip_nics} ? () : map { "net$_" } 0 .. $MAX_NIC_INDEX);
my $by_vmid = PVE::Cluster::get_guest_config_properties(\@props, $opts{vmid});
@@ -313,6 +345,7 @@ sub guest_inventory {
push @$inventory,
{
vmid => $vmid,
+ name => $guest->{name} // $guest->{hostname},
tags => \@tags,
nics => \@nics,
};
@@ -341,7 +374,7 @@ my sub parse_guest_nic {
return $iface =~ m/^(?:tap|veth)(\d+)i(\d+)$/;
}
-# the guest as the inventory lists it, with nothing but its vmid when it has no tags
+# the inventory leaves out a guest without tags and name
my sub guest_info {
my ($vmid) = @_;
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-network v3 27/45] sdn: microseg: add an update for a running NIC
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (25 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-network v3 26/45] sdn: microseg: add name regex matcher Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-network v3 28/45] sdn: microseg: expose protocol, port and macro rule options Hannes Laimer
` (17 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
A tag or name change of a running guest moves its NICs between
groups, and they would stay on the old ones until the next pass or
plug. The update plugs a NIC again right away.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/Network/SDN/Zones.pm | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm
index da24ef16dddb..8aa72c518ac6 100644
--- a/src/PVE/Network/SDN/Zones.pm
+++ b/src/PVE/Network/SDN/Zones.pm
@@ -9,6 +9,7 @@ use PVE::Tools qw(extract_param dir_glob_regex run_command);
use PVE::Cluster qw(cfs_read_file cfs_write_file cfs_lock_file);
use PVE::IPRoute2;
use PVE::Network;
+use PVE::RESTEnvironment qw(log_warn);
use PVE::Network::SDN::Vnets;
use PVE::Network::SDN::Microseg;
@@ -390,6 +391,14 @@ sub tap_plug {
$plugin->tap_plug($plugin_config, $vnet, $tag, $iface, $bridge, $firewall, $trunks, $rate);
}
+# the next pass or plug catches up after a failure
+sub tap_update {
+ my ($iface, $firewall) = @_;
+
+ eval { PVE::Network::SDN::Microseg::apply_interface($iface, $firewall) };
+ log_warn("microseg: '$iface' keeps its enforcement: $@") if $@;
+}
+
sub add_bridge_fdb {
my ($iface, $macaddr, $bridge) = @_;
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-network v3 28/45] sdn: microseg: expose protocol, port and macro rule options
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (26 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-network v3 27/45] sdn: microseg: add an update for a running NIC Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-network v3 29/45] sdn: microseg: allow nesting groups under a parent Hannes Laimer
` (16 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
The options take the values of the firewall rule options of the same
name. The config library checks them with its firewall parsers, so the
schema here only limits their length.
The options are part of what a rule's id is derived from, so the same
groups can carry one rule per narrowing. As with the groups, changing
them means replacing the rule.
At the same priority a rule with a protocol or macro is checked ahead
of one without, so the priorities alone do not show the evaluation
order. The overview reports each rule's place in it and each
assignment's place in the match order.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/Network/SDN/Microseg.pm | 10 +++++
src/PVE/API2/Network/SDN/Microseg/Rule.pm | 6 +--
src/PVE/Network/SDN/Microseg.pm | 51 +++++++++++++++++++++--
3 files changed, 60 insertions(+), 7 deletions(-)
diff --git a/src/PVE/API2/Network/SDN/Microseg.pm b/src/PVE/API2/Network/SDN/Microseg.pm
index 525fa429ae50..5d4062f7db30 100644
--- a/src/PVE/API2/Network/SDN/Microseg.pm
+++ b/src/PVE/API2/Network/SDN/Microseg.pm
@@ -124,6 +124,14 @@ __PACKAGE__->register_method({
},
},
},
+ rank => {
+ type => 'integer',
+ optional => 1,
+ description =>
+ "The object's place in the order its kind is looked at, rules"
+ . ' in evaluation order and assignments in match order. The'
+ . ' first has the lowest number.',
+ },
$OBJECT_PROPERTIES->%*,
},
},
@@ -197,9 +205,11 @@ __PACKAGE__->register_method({
my $members = $config->assigned_members(
PVE::Network::SDN::Microseg::audited_guest_inventory(),
);
+ my $rank = { $config->rule_order()->%*, $config->assignment_order()->%* };
for my $object (@$objects) {
my $id = $object->{id};
$object->{members} = $members->{$id} if defined($members->{$id});
+ $object->{rank} = $rank->{$id} if defined($rank->{$id});
}
my $registry = ($running_cfg->{microseg} // {})->{identities};
diff --git a/src/PVE/API2/Network/SDN/Microseg/Rule.pm b/src/PVE/API2/Network/SDN/Microseg/Rule.pm
index 575ca1286ed9..ed34d83b2c5c 100644
--- a/src/PVE/API2/Network/SDN/Microseg/Rule.pm
+++ b/src/PVE/API2/Network/SDN/Microseg/Rule.pm
@@ -12,9 +12,9 @@ __PACKAGE__->register_handlers({
types => ['rule'],
name => 'rule',
properties => \&PVE::Network::SDN::Microseg::rule_properties,
- create_details => "The id is derived from the src and dst predicates and from the"
- . " direction. Creating the same rule again is refused as a duplicate. Change an existing"
- . " rule's priority or verdict by updating it rather than creating a second rule.",
+ create_details => "The id is derived from what the rule matches, so creating the same rule"
+ . " again is refused as a duplicate. Change an existing rule's priority or verdict by"
+ . " updating it rather than creating a second rule.",
generated_id => 1,
generated_id_description => "The generated rule id.",
});
diff --git a/src/PVE/Network/SDN/Microseg.pm b/src/PVE/Network/SDN/Microseg.pm
index bdfde447b320..c5023c17b1d9 100644
--- a/src/PVE/Network/SDN/Microseg.pm
+++ b/src/PVE/Network/SDN/Microseg.pm
@@ -128,13 +128,55 @@ sub rule_properties {
optional => 1,
($update ? () : (default => 0)),
description => "Priority. The highest-priority matching rule wins."
- . " At a tie, a deny overrides an allow.",
+ . " At a tie, a deny overrides an allow."
+ . " A narrowed rule is checked ahead of an unnarrowed rule at the same priority.",
},
comment => $COMMENT_PROPERTY,
};
# the id is derived from these
if (!$update) {
+ my $port_descr =
+ "A service name or number (0-65535) as defined in '/etc/services', a range such"
+ . " as '80:85', or a comma-separated list of those. Requires a protocol with ports.";
+ $properties->{proto} = {
+ type => 'string',
+ minLength => 1,
+ maxLength => 32,
+ optional => 1,
+ description => "Restrict the rule to an IP protocol, a name or number from"
+ . " '/etc/protocols'. Without a protocol or macro the rule decides all traffic"
+ . " between the matched interfaces.",
+ };
+ $properties->{sport} = {
+ type => 'string',
+ minLength => 1,
+ maxLength => 128,
+ optional => 1,
+ description => "Restrict the source port. $port_descr",
+ };
+ $properties->{dport} = {
+ type => 'string',
+ minLength => 1,
+ maxLength => 128,
+ optional => 1,
+ description => "Restrict the destination port. $port_descr",
+ };
+ $properties->{'icmp-type'} = {
+ type => 'string',
+ minLength => 1,
+ maxLength => 64,
+ optional => 1,
+ description => "Restrict the ICMP type. Only with an ICMP protocol.",
+ };
+ $properties->{macro} = {
+ type => 'string',
+ minLength => 1,
+ maxLength => 128,
+ optional => 1,
+ description => "A firewall macro standing in for the protocol, ports and ICMP type."
+ . " Cannot be combined with them.",
+ };
$properties->{src} = {
type => 'array',
items => get_standard_option('pve-sdn-microseg-id'),
@@ -174,9 +216,10 @@ sub rule_properties {
type => 'boolean',
optional => 1,
default => 0,
- description => "Also apply the rule with the source and destination predicates"
- . " swapped. It is exactly that pair of rules and adds no connection tracking."
- . " Not with the 'untagged' group.",
+ description => "Also apply the rule with the source and destination predicates and"
+ . " ports swapped. It is exactly that pair of rules and adds no connection"
+ . " tracking. Not with the 'untagged' group or an ICMP type, including one a macro"
+ . " expands to.",
};
} else {
$properties->{delete} = delete_property('comment');
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-network v3 29/45] sdn: microseg: allow nesting groups under a parent
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (27 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-network v3 28/45] sdn: microseg: expose protocol, port and macro rule options Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-network v3 30/45] sdn: microseg: expose the fragments rule option Hannes Laimer
` (15 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
A group can name a parent group. An interface in a subgroup then
belongs to every group above it as well. The parent can be changed or
removed on an existing group, since a referenced group cannot be
deleted and created again.
At a priority tie an all or exact rule on a subgroup outranks the same
rule on its parent. The specificity of such a rule counts the parents
of its groups, so the rule alone does not show it. The listing of all
microseg objects reports it for each rule.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/Network/SDN/Microseg.pm | 11 +++++++++++
src/PVE/Network/SDN/Microseg.pm | 13 +++++++++++--
2 files changed, 22 insertions(+), 2 deletions(-)
diff --git a/src/PVE/API2/Network/SDN/Microseg.pm b/src/PVE/API2/Network/SDN/Microseg.pm
index 5d4062f7db30..e01989d9a923 100644
--- a/src/PVE/API2/Network/SDN/Microseg.pm
+++ b/src/PVE/API2/Network/SDN/Microseg.pm
@@ -132,6 +132,15 @@ __PACKAGE__->register_method({
. ' in evaluation order and assignments in match order. The'
. ' first has the lowest number.',
},
+ specificity => {
+ type => 'integer',
+ optional => 1,
+ description =>
+ "The number of groups the rule's 'all' and 'exact' predicates"
+ . ' require, the parents of a named group included. At a priority'
+ . ' tie the rules with the highest number decide. Rule objects'
+ . ' only.',
+ },
$OBJECT_PROPERTIES->%*,
},
},
@@ -206,10 +215,12 @@ __PACKAGE__->register_method({
PVE::Network::SDN::Microseg::audited_guest_inventory(),
);
my $rank = { $config->rule_order()->%*, $config->assignment_order()->%* };
+ my $specificity = $config->rule_specificity();
for my $object (@$objects) {
my $id = $object->{id};
$object->{members} = $members->{$id} if defined($members->{$id});
$object->{rank} = $rank->{$id} if defined($rank->{$id});
+ $object->{specificity} = $specificity->{$id} if defined($specificity->{$id});
}
my $registry = ($running_cfg->{microseg} // {})->{identities};
diff --git a/src/PVE/Network/SDN/Microseg.pm b/src/PVE/Network/SDN/Microseg.pm
index c5023c17b1d9..0732577dc42d 100644
--- a/src/PVE/Network/SDN/Microseg.pm
+++ b/src/PVE/Network/SDN/Microseg.pm
@@ -100,11 +100,19 @@ sub group_properties {
my ($update) = @_;
my $properties = {
+ parent => {
+ type => 'string',
+ format => 'pve-sdn-microseg-id',
+ optional => 1,
+ description => "The group this one is a subgroup of. An interface in a subgroup is"
+ . " also in every group above it, so 'any' and 'all' rules on a group cover"
+ . " its subgroups, while rules on the subgroup cover only the subgroup.",
+ },
comment => $COMMENT_PROPERTY,
};
if ($update) {
- $properties->{delete} = delete_property('comment');
+ $properties->{delete} = delete_property('parent', 'comment');
}
return $properties;
@@ -128,7 +136,8 @@ sub rule_properties {
optional => 1,
($update ? () : (default => 0)),
description => "Priority. The highest-priority matching rule wins."
- . " At a tie, a deny overrides an allow."
+ . " At a tie the most specific rules decide, and among equally specific ones"
+ . " a deny overrides an allow."
. " A narrowed rule is checked ahead of an unnarrowed rule at the same priority.",
},
comment => $COMMENT_PROPERTY,
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-network v3 30/45] sdn: microseg: expose the fragments rule option
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (28 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-network v3 29/45] sdn: microseg: allow nesting groups under a parent Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-manager v3 31/45] network: apply the microseg policy on reload Hannes Laimer
` (14 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
A rule narrowed to a protocol or a macro never matches the later
fragments of a datagram. Where only such rules allow traffic between
a sender and a receiver, a fragmented datagram does not arrive whole.
Expose the option that narrows a rule to those fragments.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/Network/SDN/Microseg.pm | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/src/PVE/Network/SDN/Microseg.pm b/src/PVE/Network/SDN/Microseg.pm
index 0732577dc42d..f10b2e2ea36c 100644
--- a/src/PVE/Network/SDN/Microseg.pm
+++ b/src/PVE/Network/SDN/Microseg.pm
@@ -186,6 +186,18 @@ sub rule_properties {
description => "A firewall macro standing in for the protocol, ports and ICMP type."
. " Cannot be combined with them.",
};
+ $properties->{fragments} = {
+ type => 'boolean',
+ optional => 1,
+ default => 0,
+ description =>
+ "Restrict the rule to non-first IP fragments, of the given protocol or"
+ . " of any. They carry no ports and no ICMP type, so only a protocol can narrow"
+ . " them, and no other narrowed rule matches them. An IPv6 fragment counts under"
+ . " the protocol its fragment header names. Where that is an extension header, a"
+ . " rule narrowed to the transport protocol does not match it, one without a"
+ . " protocol does.",
+ };
$properties->{src} = {
type => 'array',
items => get_standard_option('pve-sdn-microseg-id'),
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-manager v3 31/45] network: apply the microseg policy on reload
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (29 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-network v3 30/45] sdn: microseg: expose the fragments rule option Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-manager v3 32/45] pve-sdn-commit: apply the microseg policy at boot Hannes Laimer
` (13 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
A network reload is the per-node step of an SDN apply, so the reload
applies the microseg policy of the running config after ifreload. A
failed ifreload or FRR step does not skip the apply, since a node with
one broken interface stanza would otherwise stay on its old policy. A
policy that cannot be applied fails the reload task.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
PVE/API2/Network.pm | 31 ++++++++++++++++++++-----------
1 file changed, 20 insertions(+), 11 deletions(-)
diff --git a/PVE/API2/Network.pm b/PVE/API2/Network.pm
index cc8aa8cacb19..316ef1a70a02 100644
--- a/PVE/API2/Network.pm
+++ b/PVE/API2/Network.pm
@@ -933,18 +933,27 @@ __PACKAGE__->register_method({
print "$2 : $line \n";
}
};
- PVE::Tools::run_command(['ifreload', '-a'], errfunc => $err);
-
- if (defined($regenerate_frr)) {
- # SDN (or a manual invocation) is requesting a
- # specific behavior, so we abide by the parameter set.
- PVE::Network::SDN::generate_frr_config(1) if $regenerate_frr;
- } elsif (PVE::Network::SDN::running_config_has_frr()) {
- # Reload is coming from the Web UI (or a manual
- # invocation requesting no specific behavior), so we
- # reload if there are FRR entities in the SDN config.
- PVE::Network::SDN::generate_frr_config(1);
+ eval {
+ PVE::Tools::run_command(['ifreload', '-a'], errfunc => $err);
+
+ if (defined($regenerate_frr)) {
+ # SDN (or a manual invocation) is requesting a
+ # specific behavior, so we abide by the parameter set.
+ PVE::Network::SDN::generate_frr_config(1) if $regenerate_frr;
+ } elsif (PVE::Network::SDN::running_config_has_frr()) {
+ # Reload is coming from the Web UI (or a manual
+ # invocation requesting no specific behavior), so we
+ # reload if there are FRR entities in the SDN config.
+ PVE::Network::SDN::generate_frr_config(1);
+ }
+ };
+ my $errors = $@;
+
+ if ($have_sdn) {
+ eval { PVE::Network::SDN::Microseg::apply_all() };
+ $errors .= $@;
}
+ die $errors if $errors;
};
return $rpcenv->fork_worker('srvreload', 'networking', $authuser, $worker);
},
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-manager v3 32/45] pve-sdn-commit: apply the microseg policy at boot
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (30 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-manager v3 31/45] network: apply the microseg policy on reload Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-manager v3 33/45] ui: sdn: zones: add vxlan-gbp checkbox to vxlan and evpn Hannes Laimer
` (12 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
The microseg BPF programs and maps do not survive a reboot, so the
helper applies the policy of the running config on every boot. It is
ordered before the guests that start on boot, since an assigned guest
interface cannot come up without the programs. A failed ifreload or
FRR step does not skip the apply, and the helper still exits with
that failure afterwards.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
bin/pve-sdn-commit | 47 ++++++++++++++++++++-------------
services/pve-sdn-commit.service | 1 +
2 files changed, 30 insertions(+), 18 deletions(-)
diff --git a/bin/pve-sdn-commit b/bin/pve-sdn-commit
index aa7e9b29056a..29b867780dc8 100644
--- a/bin/pve-sdn-commit
+++ b/bin/pve-sdn-commit
@@ -13,6 +13,7 @@ use PVE::Network::SDN::Vnets;
use PVE::Network::SDN::Subnets;
use PVE::Network::SDN::Controllers;
use PVE::Network::SDN::Fabrics;
+use PVE::Network::SDN::Microseg;
use PVE::Tools;
for (my $i = 0; !PVE::Cluster::check_cfs_quorum(1); $i++) {
@@ -96,29 +97,39 @@ sub sdn_changed {
return fabrics_changed();
}
-if (!sdn_changed()) {
- print "No changes to SDN configuration detected, skipping reload\n";
- exit 0;
-}
+my $errors = '';
-my $previous_config_has_frr = PVE::Network::SDN::running_config_has_frr();
-PVE::Network::SDN::commit_config();
+if (sdn_changed()) {
+ my $previous_config_has_frr = PVE::Network::SDN::running_config_has_frr();
+ PVE::Network::SDN::commit_config();
-my $new_config_has_frr = PVE::Network::SDN::running_config_has_frr();
-my $regenerate_frr = ($previous_config_has_frr || $new_config_has_frr);
+ my $new_config_has_frr = PVE::Network::SDN::running_config_has_frr();
+ my $regenerate_frr = ($previous_config_has_frr || $new_config_has_frr);
-PVE::Network::SDN::generate_etc_network_config();
-PVE::Network::SDN::generate_dhcp_config();
+ PVE::Network::SDN::generate_etc_network_config();
+ PVE::Network::SDN::generate_dhcp_config();
-my $err = sub {
- my $line = shift;
- if ($line =~ /(warning|error): (\S+):/) {
- print "$2 : $line \n";
- }
-};
+ my $err = sub {
+ my $line = shift;
+ if ($line =~ /(warning|error): (\S+):/) {
+ print "$2 : $line \n";
+ }
+ };
-PVE::Tools::run_command(['ifreload', '-a'], errfunc => $err);
+ eval {
+ PVE::Tools::run_command(['ifreload', '-a'], errfunc => $err);
+
+ PVE::Network::SDN::generate_frr_config(1) if $regenerate_frr;
+ };
+ $errors = $@;
+} else {
+ print "No changes to SDN configuration detected, skipping reload\n";
+}
-PVE::Network::SDN::generate_frr_config(1) if $regenerate_frr;
+# the pinned policy does not survive a reboot, so it is applied here, before the guests that
+# start on boot
+eval { PVE::Network::SDN::Microseg::apply_all() };
+$errors .= $@;
+die $errors if $errors;
exit 0;
diff --git a/services/pve-sdn-commit.service b/services/pve-sdn-commit.service
index ff723725ddcd..d091abd2bd84 100644
--- a/services/pve-sdn-commit.service
+++ b/services/pve-sdn-commit.service
@@ -3,6 +3,7 @@ Description=Commit Proxmox VE SDN changes
DefaultDependencies=no
Wants=pve-cluster.service network.target
After=frr.service network.target corosync.service
+Before=pve-ha-lrm.service pve-guests.service
[Service]
ExecStart=/usr/share/pve-manager/helpers/pve-sdn-commit
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-manager v3 33/45] ui: sdn: zones: add vxlan-gbp checkbox to vxlan and evpn
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (31 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-manager v3 32/45] pve-sdn-commit: apply the microseg policy at boot Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-manager v3 34/45] ui: sdn: add microsegmentation panel Hannes Laimer
` (11 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
www/manager6/sdn/zones/EvpnEdit.js | 8 ++++++++
www/manager6/sdn/zones/VxlanEdit.js | 11 +++++++++++
2 files changed, 19 insertions(+)
diff --git a/www/manager6/sdn/zones/EvpnEdit.js b/www/manager6/sdn/zones/EvpnEdit.js
index 7229fc140200..f1de2e761263 100644
--- a/www/manager6/sdn/zones/EvpnEdit.js
+++ b/www/manager6/sdn/zones/EvpnEdit.js
@@ -100,6 +100,14 @@ Ext.define('PVE.sdn.zones.EvpnInputPanel', {
skipEmptyText: true,
deleteEmpty: !me.isCreate,
},
+ {
+ xtype: 'proxmoxcheckbox',
+ name: 'vxlan-gbp',
+ uncheckedValue: null,
+ checked: false,
+ fieldLabel: 'VXLAN-GBP',
+ deleteEmpty: !me.isCreate,
+ },
];
me.callParent();
diff --git a/www/manager6/sdn/zones/VxlanEdit.js b/www/manager6/sdn/zones/VxlanEdit.js
index 4b53c2ffae86..b39b4c48c21d 100644
--- a/www/manager6/sdn/zones/VxlanEdit.js
+++ b/www/manager6/sdn/zones/VxlanEdit.js
@@ -78,6 +78,17 @@ Ext.define('PVE.sdn.zones.VxlanInputPanel', {
},
];
+ me.advancedItems = [
+ {
+ xtype: 'proxmoxcheckbox',
+ name: 'vxlan-gbp',
+ uncheckedValue: null,
+ checked: false,
+ fieldLabel: 'VXLAN-GBP',
+ deleteEmpty: !me.isCreate,
+ },
+ ];
+
me.callParent();
},
});
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-manager v3 34/45] ui: sdn: add microsegmentation panel
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (32 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-manager v3 33/45] ui: sdn: zones: add vxlan-gbp checkbox to vxlan and evpn Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-manager v3 35/45] ui: sdn: dry-run: show pending microseg diff Hannes Laimer
` (10 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
Add a Microsegmentation panel under Datacenter > SDN to manage groups,
rules and guest assignments, with a tree of the groups and their
guests and NICs. Rules are grouped by priority, highest first.
A NIC belongs to the first assignment that matches it, so the
assignment list keeps the match order and can be reordered by drag and
drop or a position field. An assignment that wins no NIC is dimmed.
Selecting a group shows the rules that list it. Selecting a guest or
NIC shows the rules the API reports for its groups, so the panel does
not evaluate rules itself.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
www/css/ext6-pve.css | 23 ++
www/manager6/Makefile | 9 +
www/manager6/Utils.js | 74 ++++++
www/manager6/data/PermPathStore.js | 1 +
www/manager6/dc/Config.js | 8 +
www/manager6/form/SDNMicrosegGroupSelector.js | 83 ++++++
www/manager6/sdn/MicrosegView.js | 92 +++++++
www/manager6/sdn/microseg/AssignmentEdit.js | 80 ++++++
www/manager6/sdn/microseg/AssignmentView.js | 193 ++++++++++++++
www/manager6/sdn/microseg/Base.js | 215 +++++++++++++++
www/manager6/sdn/microseg/GroupEdit.js | 30 +++
www/manager6/sdn/microseg/RuleEdit.js | 158 +++++++++++
www/manager6/sdn/microseg/RuleView.js | 242 +++++++++++++++++
www/manager6/sdn/microseg/Tree.js | 249 ++++++++++++++++++
14 files changed, 1457 insertions(+)
create mode 100644 www/manager6/form/SDNMicrosegGroupSelector.js
create mode 100644 www/manager6/sdn/MicrosegView.js
create mode 100644 www/manager6/sdn/microseg/AssignmentEdit.js
create mode 100644 www/manager6/sdn/microseg/AssignmentView.js
create mode 100644 www/manager6/sdn/microseg/Base.js
create mode 100644 www/manager6/sdn/microseg/GroupEdit.js
create mode 100644 www/manager6/sdn/microseg/RuleEdit.js
create mode 100644 www/manager6/sdn/microseg/RuleView.js
create mode 100644 www/manager6/sdn/microseg/Tree.js
diff --git a/www/css/ext6-pve.css b/www/css/ext6-pve.css
index b1e0b4f81002..07f2c65d33ef 100644
--- a/www/css/ext6-pve.css
+++ b/www/css/ext6-pve.css
@@ -708,3 +708,26 @@ table.osds td:first-of-type {
.pmx-column-wrapped > div {
white-space: pre-wrap;
}
+
+.pve-microseg-group {
+ display: inline-block;
+ border: 1px solid #999;
+ border-radius: 3px;
+ padding: 0px 5px;
+ margin: 0px 1px;
+}
+
+.pve-microseg-muted {
+ opacity: 0.5;
+}
+
+.pve-microseg-id-space {
+ font-size: smaller;
+ font-weight: normal;
+}
+
+.pve-microseg-deleted td:not(.pve-microseg-state) .x-grid-cell-inner,
+.pve-microseg-deleted td:not(.pve-microseg-state) .pve-microseg-group,
+.pve-microseg-deleted td:not(.pve-microseg-state) .fa {
+ text-decoration: line-through;
+}
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index d2ea786be00b..7c48b0622aa7 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -71,6 +71,7 @@ JSSRC= \
form/SDNControllerSelector.js \
form/SDNZoneSelector.js \
form/SDNVnetSelector.js \
+ form/SDNMicrosegGroupSelector.js \
form/SDNIpamSelector.js \
form/SDNDnsSelector.js \
form/ScsiHwSelector.js \
@@ -357,6 +358,14 @@ JSSRC= \
sdn/fabrics/bgp/InterfacePanel.js \
sdn/fabrics/bgp/NodeEdit.js \
sdn/fabrics/bgp/FabricEdit.js \
+ sdn/MicrosegView.js \
+ sdn/microseg/Base.js \
+ sdn/microseg/Tree.js \
+ sdn/microseg/GroupEdit.js \
+ sdn/microseg/RuleEdit.js \
+ sdn/microseg/AssignmentEdit.js \
+ sdn/microseg/RuleView.js \
+ sdn/microseg/AssignmentView.js \
storage/ContentView.js \
storage/BackupView.js \
storage/Base.js \
diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index 8b99371d3447..8eb4b68c51b4 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -231,6 +231,22 @@ Ext.define('PVE.Utils', {
return `<span data-qtip="${Ext.htmlEncode(tip)}">${icon}${Ext.htmlEncode(value)}</span>`;
},
+ sdnmicroseg_row_cls: function (rec) {
+ return rec.data.state === 'deleted' ? 'pve-microseg-deleted' : '';
+ },
+
+ render_sdnmicroseg_state: function (value) {
+ return value
+ ? '<i class="fa fa-fw fa-refresh warning" data-qtip="' +
+ Ext.String.htmlEncode(value) +
+ '"></i>'
+ : '';
+ },
+
+ render_sdnmicroseg_group: function (group) {
+ return `<span class="pve-microseg-group">${Ext.String.htmlEncode(group)}</span>`;
+ },
+
render_ceph_health: function (healthObj) {
var state = {
iconCls: PVE.Utils.get_health_icon(),
@@ -966,6 +982,64 @@ Ext.define('PVE.Utils', {
},
},
+ sdnmicrosegSchema: {
+ group: {
+ name: gettext('Group'),
+ ipanel: 'GroupInputPanel',
+ width: 400,
+ },
+ rule: {
+ name: gettext('Rule'),
+ ipanel: 'RuleInputPanel',
+ },
+ 'guest-assignment': {
+ name: gettext('Guest Assignment'),
+ ipanel: 'GuestAssignmentInputPanel',
+ width: 400,
+ faIcon: 'link',
+ assignment: true,
+ },
+ },
+
+ sdnmicroseg_is_assignment: function (type) {
+ return !!PVE.Utils.sdnmicrosegSchema[type]?.assignment;
+ },
+
+ format_sdnmicroseg_type: function (value) {
+ let schema = PVE.Utils.sdnmicrosegSchema[value];
+ if (schema) {
+ return schema.name;
+ }
+ return Proxmox.Utils.unknownText;
+ },
+
+ sdnmicroseg_merge_pending: function (obj) {
+ let merged = { ...obj };
+ for (const [key, value] of Object.entries(obj.pending ?? {})) {
+ if (value === 'deleted') {
+ delete merged[key];
+ } else if (value !== undefined && value !== null) {
+ merged[key] = value;
+ }
+ }
+ return merged;
+ },
+
+ format_sdnmicroseg_predicate: function (match, groups) {
+ if (match === 'every') {
+ return match;
+ }
+ return `${match || 'any'}{${(groups || []).join(', ')}}`;
+ },
+
+ format_sdnmicroseg_matcher: function (assignment) {
+ if (!PVE.Utils.sdnmicrosegSchema[assignment.type]) {
+ return Proxmox.Utils.unknownText;
+ }
+ let iface = (assignment.iface ?? '') === '' ? '' : ` net${assignment.iface}`;
+ return `vm${assignment.vmid}${iface}`;
+ },
+
sdnipamSchema: {
ipam: {
name: 'ipam',
diff --git a/www/manager6/data/PermPathStore.js b/www/manager6/data/PermPathStore.js
index 7210e0cd48af..65c8d17c850f 100644
--- a/www/manager6/data/PermPathStore.js
+++ b/www/manager6/data/PermPathStore.js
@@ -17,6 +17,7 @@ Ext.define('PVE.data.PermPathStore', {
{ value: '/nodes' },
{ value: '/pool' },
{ value: '/sdn/fabrics' },
+ { value: '/sdn/microseg' },
{ value: '/sdn/zones' },
{ value: '/storage' },
{ value: '/vms' },
diff --git a/www/manager6/dc/Config.js b/www/manager6/dc/Config.js
index 83c48c3d633c..fce20937f3ba 100644
--- a/www/manager6/dc/Config.js
+++ b/www/manager6/dc/Config.js
@@ -309,6 +309,14 @@ Ext.define('PVE.dc.Config', {
iconCls: 'fa fa-road',
itemId: 'sdnfabrics',
},
+ {
+ xtype: 'pveSDNMicroseg',
+ groups: ['sdn'],
+ title: gettext('Microsegmentation'),
+ hidden: true,
+ iconCls: 'fa fa-tags',
+ itemId: 'sdnmicroseg',
+ },
{
xtype: 'pveSDNRouteMaps',
groups: ['sdn'],
diff --git a/www/manager6/form/SDNMicrosegGroupSelector.js b/www/manager6/form/SDNMicrosegGroupSelector.js
new file mode 100644
index 000000000000..1b0c39a1cfba
--- /dev/null
+++ b/www/manager6/form/SDNMicrosegGroupSelector.js
@@ -0,0 +1,83 @@
+Ext.define(
+ 'PVE.form.SDNMicrosegGroupSelector',
+ {
+ extend: 'Proxmox.form.ComboGrid',
+ alias: ['widget.pveSDNMicrosegGroupSelector'],
+
+ valueField: 'id',
+ displayField: 'id',
+
+ syncUntagged: function () {
+ let me = this;
+ let store = me.getStore();
+ let record = store.getById('untagged');
+ if (me.includeUntagged && !record) {
+ store.add({
+ id: 'untagged',
+ comment: gettext('No group (untagged)'),
+ });
+ } else if (!me.includeUntagged && record) {
+ store.remove(record);
+ }
+ },
+
+ setIncludeUntagged: function (include) {
+ let me = this;
+ me.includeUntagged = include;
+ let value = me.getValue();
+ if (!include && Ext.isArray(value)) {
+ me.setValue(value.filter((id) => id !== 'untagged'));
+ }
+ me.syncUntagged();
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ let store = new Ext.data.Store({
+ model: 'pve-sdn-microseg-group',
+ sorters: {
+ property: 'id',
+ direction: 'ASC',
+ },
+ });
+
+ me.mon(store, 'load', () => me.syncUntagged());
+
+ Ext.apply(me, {
+ store: store,
+ autoSelect: false,
+ listConfig: {
+ columns: [
+ {
+ header: gettext('Group'),
+ sortable: true,
+ dataIndex: 'id',
+ flex: 1,
+ },
+ {
+ header: gettext('Comment'),
+ dataIndex: 'comment',
+ flex: 2,
+ renderer: Ext.String.htmlEncode,
+ },
+ ],
+ },
+ });
+
+ me.callParent();
+
+ store.load();
+ },
+ },
+ function () {
+ Ext.define('pve-sdn-microseg-group', {
+ extend: 'Ext.data.Model',
+ fields: ['id', 'comment'],
+ proxy: {
+ type: 'proxmox',
+ url: '/api2/json/cluster/sdn/microseg/group',
+ },
+ });
+ },
+);
diff --git a/www/manager6/sdn/MicrosegView.js b/www/manager6/sdn/MicrosegView.js
new file mode 100644
index 000000000000..805b5b7ebdd9
--- /dev/null
+++ b/www/manager6/sdn/MicrosegView.js
@@ -0,0 +1,92 @@
+Ext.define('PVE.sdn.MicrosegView', {
+ extend: 'Ext.panel.Panel',
+ alias: 'widget.pveSDNMicroseg',
+
+ onlineHelp: 'pvesdn_config_microseg',
+
+ layout: 'border',
+
+ controller: {
+ xclass: 'Ext.app.ViewController',
+
+ reload: function () {
+ let me = this;
+ PVE.sdn.microseg.Membership.load(me.getView(), (parts) => {
+ me.lookup('rules').setData(parts.rules);
+ me.lookup('assignments').setData(parts.assignments);
+ me.lookup('tree').setData(
+ parts.groups,
+ parts.assignments,
+ parts.groupSets,
+ parts.idSpace,
+ );
+ });
+ },
+
+ onTreeSelect: function (tree, selection) {
+ let me = this;
+ let sel = selection[0]?.data.microsegSelection ?? null;
+ me.lookup('rules').setTreeSelection(sel);
+ me.lookup('assignments').setTreeSelection(sel);
+ },
+
+ selectedGroup: function () {
+ let node = this.lookup('tree').getSelection()[0];
+ return node && PVE.sdn.microseg.Tree.editable(node) ? node.data.microsegId : undefined;
+ },
+
+ addGroup: function () {
+ this.lookup('tree').openEdit('group');
+ },
+
+ editGroup: function () {
+ let me = this;
+ let group = me.selectedGroup();
+ if (group) {
+ me.lookup('tree').openEdit('group', group);
+ }
+ },
+ },
+
+ items: [
+ {
+ xtype: 'pveSDNMicrosegTree',
+ reference: 'tree',
+ region: 'west',
+ title: gettext('Groups'),
+ width: 400,
+ minWidth: 250,
+ split: true,
+ listeners: {
+ microsegreload: 'reload',
+ selectionchange: 'onTreeSelect',
+ itemdblclick: 'editGroup',
+ },
+ },
+ {
+ xtype: 'tabpanel',
+ region: 'center',
+ border: false,
+ items: [
+ {
+ xtype: 'pveSDNMicrosegRules',
+ reference: 'rules',
+ listeners: {
+ microsegreload: 'reload',
+ },
+ },
+ {
+ xtype: 'pveSDNMicrosegAssignments',
+ reference: 'assignments',
+ listeners: {
+ microsegreload: 'reload',
+ },
+ },
+ ],
+ },
+ ],
+
+ listeners: {
+ activate: 'reload',
+ },
+});
diff --git a/www/manager6/sdn/microseg/AssignmentEdit.js b/www/manager6/sdn/microseg/AssignmentEdit.js
new file mode 100644
index 000000000000..71cabaeeea52
--- /dev/null
+++ b/www/manager6/sdn/microseg/AssignmentEdit.js
@@ -0,0 +1,80 @@
+Ext.define('PVE.sdn.microseg.AssignmentInputPanel', {
+ extend: 'Proxmox.panel.InputPanel',
+
+ onlineHelp: 'pvesdn_microseg_assignment',
+
+ onGetValues: function (values) {
+ let me = this;
+ if (me.isCreate) {
+ values.type = me.type;
+ }
+ return values;
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ me.items = [
+ ...me.matcherItems,
+ {
+ xtype: 'proxmoxintegerfield',
+ name: 'iface',
+ fieldLabel: gettext('Network Interface'),
+ minValue: 0,
+ maxValue: 31,
+ allowBlank: true,
+ emptyText: gettext('All NICs'),
+ disabled: !me.isCreate,
+ },
+ {
+ xtype: 'pveSDNMicrosegGroupSelector',
+ name: 'groups',
+ fieldLabel: gettext('Groups'),
+ multiSelect: true,
+ allowBlank: false,
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'comment',
+ fieldLabel: gettext('Comment'),
+ allowBlank: true,
+ maxLength: 256,
+ deleteEmpty: !me.isCreate,
+ },
+ ];
+
+ me.advancedItems = [
+ {
+ xtype: 'proxmoxintegerfield',
+ name: 'position',
+ fieldLabel: gettext('Position'),
+ value: me.position,
+ minValue: 0,
+ allowBlank: true,
+ emptyText: me.isCreate ? '0' : gettext('Unchanged'),
+ },
+ ];
+
+ me.callParent();
+ },
+});
+
+Ext.define('PVE.sdn.microseg.GuestAssignmentInputPanel', {
+ extend: 'PVE.sdn.microseg.AssignmentInputPanel',
+
+ initComponent: function () {
+ let me = this;
+
+ me.matcherItems = [
+ {
+ xtype: 'vmComboSelector',
+ name: 'vmid',
+ fieldLabel: gettext('Guest'),
+ allowBlank: false,
+ disabled: !me.isCreate,
+ },
+ ];
+
+ me.callParent();
+ },
+});
diff --git a/www/manager6/sdn/microseg/AssignmentView.js b/www/manager6/sdn/microseg/AssignmentView.js
new file mode 100644
index 000000000000..4744380461a7
--- /dev/null
+++ b/www/manager6/sdn/microseg/AssignmentView.js
@@ -0,0 +1,193 @@
+Ext.define(
+ 'PVE.sdn.microseg.AssignmentView',
+ {
+ extend: 'PVE.sdn.microseg.BaseGrid',
+ alias: 'widget.pveSDNMicrosegAssignments',
+
+ title: gettext('Assignments'),
+
+ baseurl: '/cluster/sdn/microseg/assignment/',
+
+ getRecordName: (rec) => PVE.Utils.format_sdnmicroseg_matcher(rec.data),
+
+ editPanelConfig: (rec) => ({ position: rec.data.rank }),
+
+ setData: function (assignments) {
+ let me = this;
+ let rows = assignments.map((a) => ({
+ ...a,
+ rank: me.rankOf(a),
+ memberCount: (a.members || []).reduce((n, m) => n + m.nics.length, 0),
+ }));
+ me.getStore().loadData(rows);
+ },
+
+ setTreeSelection: function (sel) {
+ let me = this;
+ me.treeSelection = sel;
+ me.getView().refresh();
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ let store = Ext.create('Ext.data.Store', {
+ model: 'pve-sdn-microseg-assignment',
+ sorters: [
+ { property: 'rank', direction: 'ASC' },
+ { property: 'id', direction: 'ASC' },
+ ],
+ });
+
+ let move = (rec, index) =>
+ Proxmox.Utils.API2Request({
+ url: `${me.baseurl}${rec.data.id}`,
+ method: 'PUT',
+ params: { position: index },
+ waitMsgTarget: me,
+ failure: (response) =>
+ Ext.Msg.alert(Proxmox.Utils.errorText, response.htmlStatus),
+ callback: () => me.reload(),
+ });
+
+ let movable = (rec) => rec.data.state !== 'deleted';
+
+ let winsNic = (rec, vmid, nic) =>
+ (rec.data.members || []).some(
+ (m) => m.vmid === vmid && (nic === undefined || m.nics.includes(nic)),
+ );
+
+ let dimmed = (rec) => {
+ if (!rec.data.memberCount) {
+ return true;
+ }
+ let sel = me.treeSelection;
+ if (!sel) {
+ return false;
+ }
+ if (sel.vmid !== undefined) {
+ return !winsNic(rec, sel.vmid, sel.nic);
+ }
+ let group = sel.mode === 'membership' ? sel.groups[0] : undefined;
+ if (group && group !== 'untagged') {
+ return !(rec.data.groups ?? []).includes(group);
+ }
+ return false;
+ };
+
+ let renderMembers = (value, rec) => {
+ if (!value) {
+ return '<span class="pve-microseg-muted">0</span>';
+ }
+ let nics = [];
+ for (const member of rec.data.members || []) {
+ for (const nic of member.nics) {
+ nics.push(`vm${member.vmid} net${nic}`);
+ }
+ }
+ return `<span data-qtip="${Ext.String.htmlEncode(nics.join(', '))}">${value}</span>`;
+ };
+
+ me.columns = [
+ {
+ text: '#',
+ dataIndex: 'rank',
+ width: 50,
+ renderer: (v, m, rec) => (rec.data.state === 'deleted' ? '' : v),
+ },
+ {
+ text: gettext('Matcher'),
+ dataIndex: 'type',
+ flex: 2,
+ renderer: (v, m, rec) => {
+ let text = Ext.String.htmlEncode(
+ PVE.Utils.format_sdnmicroseg_matcher(rec.data),
+ );
+ let icon = PVE.Utils.sdnmicrosegSchema[v]?.faIcon;
+ return icon ? `<i class="fa fa-fw fa-${icon}"></i> ${text}` : text;
+ },
+ },
+ {
+ text: gettext('Groups'),
+ dataIndex: 'groups',
+ flex: 2,
+ renderer: (v) => (v ?? []).map(PVE.Utils.render_sdnmicroseg_group).join(''),
+ },
+ {
+ text: gettext('NICs'),
+ dataIndex: 'memberCount',
+ width: 90,
+ renderer: (v, m, rec) => renderMembers(v, rec),
+ },
+ { xtype: 'pveSDNMicrosegStateColumn' },
+ {
+ text: gettext('Comment'),
+ dataIndex: 'comment',
+ flex: 1,
+ renderer: Ext.String.htmlEncode,
+ },
+ ];
+
+ Ext.apply(me, {
+ store: store,
+ viewConfig: {
+ plugins: [
+ {
+ ptype: 'gridviewdragdrop',
+ dragGroup: 'pveSDNMicrosegAssignmentDD',
+ dropGroup: 'pveSDNMicrosegAssignmentDD',
+ },
+ ],
+ getRowClass: (rec) =>
+ [
+ PVE.Utils.sdnmicroseg_row_cls(rec),
+ dimmed(rec) ? 'pve-microseg-muted' : '',
+ ].join(' '),
+ listeners: {
+ beforedrop: (node, data, dropRec, dropPosition) => {
+ let rec = data.records[0];
+ if (!dropRec || rec === dropRec || !movable(rec) || !movable(dropRec)) {
+ return false;
+ }
+ let list = store
+ .getRange()
+ .filter((r) => r !== rec && r.data.state !== 'deleted');
+ move(rec, list.indexOf(dropRec) + (dropPosition === 'after' ? 1 : 0));
+ return false;
+ },
+ },
+ },
+ addButton: {
+ text: gettext('Add'),
+ menu: {
+ items: Object.keys(PVE.Utils.sdnmicrosegSchema)
+ .filter((type) => PVE.Utils.sdnmicroseg_is_assignment(type))
+ .map((type) => ({
+ text: PVE.Utils.format_sdnmicroseg_type(type),
+ handler: () => me.openEdit(type),
+ })),
+ },
+ },
+ });
+
+ me.callParent();
+ },
+ },
+ function () {
+ Ext.define('pve-sdn-microseg-assignment', {
+ extend: 'Ext.data.Model',
+ fields: [
+ 'id',
+ 'type',
+ 'vmid',
+ 'iface',
+ 'groups',
+ 'comment',
+ 'state',
+ 'members',
+ 'rank',
+ 'memberCount',
+ ],
+ });
+ },
+);
diff --git a/www/manager6/sdn/microseg/Base.js b/www/manager6/sdn/microseg/Base.js
new file mode 100644
index 000000000000..6751da1c7e88
--- /dev/null
+++ b/www/manager6/sdn/microseg/Base.js
@@ -0,0 +1,215 @@
+Ext.define('PVE.sdn.microseg.BaseEdit', {
+ extend: 'Proxmox.window.Edit',
+
+ initComponent: function () {
+ let me = this;
+
+ me.isCreate = !me.microsegId;
+
+ let schema = PVE.Utils.sdnmicrosegSchema[me.type];
+ if (!schema || !schema.ipanel) {
+ throw 'no editor registered for microseg type: ' + me.type;
+ }
+
+ if (me.isCreate) {
+ me.url = '/api2/extjs' + me.baseurl;
+ me.method = 'POST';
+ } else {
+ me.url = '/api2/extjs' + me.baseurl + me.microsegId;
+ me.method = 'PUT';
+ }
+
+ let ipanel = Ext.create(
+ 'PVE.sdn.microseg.' + schema.ipanel,
+ Ext.apply(
+ {
+ type: me.type,
+ isCreate: me.isCreate,
+ microsegId: me.microsegId,
+ },
+ me.panelConfig,
+ ),
+ );
+
+ Ext.apply(me, {
+ subject: PVE.Utils.format_sdnmicroseg_type(me.type),
+ isAdd: true,
+ width: schema.width,
+ items: [ipanel],
+ });
+
+ me.callParent();
+
+ if (!me.isCreate) {
+ me.load();
+ }
+ },
+});
+
+Ext.define('PVE.sdn.microseg.Membership', {
+ singleton: true,
+
+ load: function (view, callback) {
+ Proxmox.Utils.API2Request({
+ url: '/cluster/sdn/microseg/all?pending=1',
+ method: 'GET',
+ failure: (response) => Ext.Msg.alert(Proxmox.Utils.errorText, response.htmlStatus),
+ success: (response) => {
+ if (view.destroyed) {
+ return;
+ }
+ let data = response.result.data;
+ let objects = data.objects.map(PVE.Utils.sdnmicroseg_merge_pending);
+ callback({
+ groups: objects.filter((o) => o.type === 'group'),
+ rules: objects.filter((o) => o.type === 'rule'),
+ assignments: objects.filter((o) => PVE.Utils.sdnmicroseg_is_assignment(o.type)),
+ groupSets: data['group-sets'],
+ idSpace: data['id-space'],
+ });
+ },
+ });
+ },
+
+ rulesFor: function (groupSets) {
+ let classRules = {};
+ groupSets.forEach((cls) => {
+ classRules[cls.groups.join(',')] = cls;
+ });
+ return (sets) => {
+ let outbound = new Set();
+ let inbound = new Set();
+ sets.forEach((groupSet) => {
+ let cls = classRules[groupSet.join(',')];
+ cls?.outbound.forEach((r) => outbound.add(r));
+ cls?.inbound.forEach((r) => inbound.add(r));
+ });
+ return { outbound: outbound, inbound: inbound };
+ };
+ },
+
+ nicGroupSets: function (assignments, rulesFor) {
+ let nicGroupSet = {};
+ assignments.forEach((a) => {
+ let sortedGroups = (a.groups || []).slice().sort();
+ let groupSet = {
+ groups: sortedGroups,
+ rules: rulesFor([sortedGroups]),
+ note: PVE.Utils.format_sdnmicroseg_matcher(a),
+ };
+ (a.members || []).forEach((m) => {
+ m.nics.forEach((nic) => {
+ nicGroupSet[`${m.vmid}/${nic}`] = groupSet;
+ });
+ });
+ });
+ return nicGroupSet;
+ },
+
+ guestSelection: function (vmid, nicGroupSet, rulesFor) {
+ let keys = Object.keys(nicGroupSet).filter((key) => key.startsWith(`${vmid}/`));
+ let rules = rulesFor(keys.map((key) => nicGroupSet[key].groups));
+ return {
+ scope: `vm${vmid}`,
+ mode: 'group-set',
+ outbound: rules.outbound,
+ inbound: rules.inbound,
+ vmid: vmid,
+ };
+ },
+});
+
+Ext.define('PVE.sdn.microseg.StateColumn', {
+ extend: 'Ext.grid.column.Column',
+ xtype: 'pveSDNMicrosegStateColumn',
+
+ text: '',
+ dataIndex: 'state',
+ width: 40,
+ resizable: false,
+ tdCls: 'pve-microseg-state',
+ renderer: PVE.Utils.render_sdnmicroseg_state,
+});
+
+Ext.define('PVE.sdn.microseg.ViewMixin', {
+ extend: 'Ext.Mixin',
+
+ reload: function () {
+ this.fireEvent('microsegreload');
+ },
+
+ rankOf: function (data) {
+ return data.rank ?? Number.MAX_SAFE_INTEGER;
+ },
+
+ openEdit: function (type, microsegId, panelConfig) {
+ let me = this;
+ Ext.create('PVE.sdn.microseg.BaseEdit', {
+ type: type,
+ baseurl: me.baseurl,
+ microsegId: microsegId,
+ panelConfig: panelConfig,
+ autoShow: true,
+ listeners: { destroy: () => me.reload() },
+ });
+ },
+});
+
+Ext.define('PVE.sdn.microseg.BaseGrid', {
+ extend: 'Ext.grid.GridPanel',
+ mixins: ['PVE.sdn.microseg.ViewMixin'],
+
+ border: false,
+ sortableColumns: false,
+
+ initComponent: function () {
+ let me = this;
+
+ if (!me.baseurl) {
+ throw 'no baseurl given';
+ }
+
+ let sm = Ext.create('Ext.selection.RowModel', {});
+
+ let editable = (rec) => rec.data.state !== 'deleted';
+
+ let runEditor = () => {
+ let rec = sm.getSelection()[0];
+ if (!rec || !editable(rec)) {
+ return;
+ }
+ me.openEdit(rec.data.type, rec.data.id, me.editPanelConfig?.(rec));
+ };
+
+ Ext.apply(me, {
+ selModel: sm,
+ tbar: [
+ me.addButton,
+ new Proxmox.button.Button({
+ text: gettext('Edit'),
+ disabled: true,
+ selModel: sm,
+ enableFn: editable,
+ handler: runEditor,
+ }),
+ Ext.create('Proxmox.button.StdRemoveButton', {
+ selModel: sm,
+ enableFn: editable,
+ baseurl: me.baseurl,
+ getRecordName: me.getRecordName,
+ callback: () => me.reload(),
+ }),
+ {
+ text: gettext('Reload'),
+ handler: () => me.reload(),
+ },
+ ...(me.tbar ?? []),
+ ],
+ listeners: {
+ itemdblclick: runEditor,
+ },
+ });
+
+ me.callParent();
+ },
+});
diff --git a/www/manager6/sdn/microseg/GroupEdit.js b/www/manager6/sdn/microseg/GroupEdit.js
new file mode 100644
index 000000000000..82639c344264
--- /dev/null
+++ b/www/manager6/sdn/microseg/GroupEdit.js
@@ -0,0 +1,30 @@
+Ext.define('PVE.sdn.microseg.GroupInputPanel', {
+ extend: 'Proxmox.panel.InputPanel',
+
+ onlineHelp: 'pvesdn_microseg_group',
+
+ initComponent: function () {
+ let me = this;
+
+ me.items = [
+ {
+ xtype: me.isCreate ? 'textfield' : 'displayfield',
+ name: 'id',
+ value: me.microsegId || '',
+ fieldLabel: gettext('Name'),
+ allowBlank: false,
+ maxLength: 32,
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'comment',
+ fieldLabel: gettext('Comment'),
+ allowBlank: true,
+ maxLength: 256,
+ deleteEmpty: !me.isCreate,
+ },
+ ];
+
+ me.callParent();
+ },
+});
diff --git a/www/manager6/sdn/microseg/RuleEdit.js b/www/manager6/sdn/microseg/RuleEdit.js
new file mode 100644
index 000000000000..4ec5467f61be
--- /dev/null
+++ b/www/manager6/sdn/microseg/RuleEdit.js
@@ -0,0 +1,158 @@
+Ext.define('PVE.sdn.microseg.RuleInputPanel', {
+ extend: 'Proxmox.panel.InputPanel',
+
+ onlineHelp: 'pvesdn_microseg_rule',
+
+ initComponent: function () {
+ let me = this;
+
+ let matchItems = [
+ ['any', gettext('Any (shares a group)')],
+ ['all', gettext('All (has every group)')],
+ ['exact', gettext('Exact (same groups)')],
+ ['every', gettext('Every (has a group)')],
+ ];
+ let setEvery = (selector, every) => {
+ selector.setDisabled(every || !me.isCreate);
+ if (every) {
+ selector.setValue([]);
+ }
+ };
+
+ me.srcSelector = Ext.create({
+ xtype: 'pveSDNMicrosegGroupSelector',
+ name: 'src',
+ fieldLabel: gettext('Source Groups'),
+ multiSelect: true,
+ allowBlank: false,
+ includeUntagged: true,
+ disabled: !me.isCreate,
+ validator: () => {
+ let groups = me.srcSelector.getValue() || [];
+ let match = me.down('field[name=src-match]')?.getValue() ?? 'any';
+ if (
+ match !== 'any' &&
+ groups.includes('untagged') &&
+ groups.some((group) => group !== 'untagged')
+ ) {
+ return gettext(
+ "With 'all' or 'exact', 'untagged' cannot be combined with other groups",
+ );
+ }
+ return true;
+ },
+ });
+
+ me.dstSelector = Ext.create({
+ xtype: 'pveSDNMicrosegGroupSelector',
+ name: 'dst',
+ fieldLabel: gettext('Dest. Groups'),
+ multiSelect: true,
+ allowBlank: false,
+ includeUntagged: true,
+ disabled: !me.isCreate,
+ validator: () => {
+ let groups = me.dstSelector.getValue() || [];
+ if (groups.length === 1 && groups[0] === 'untagged') {
+ return gettext("'untagged' cannot be the only destination group");
+ }
+ return true;
+ },
+ });
+
+ me.column1 = [
+ me.srcSelector,
+ {
+ xtype: 'proxmoxKVComboBox',
+ name: 'src-match',
+ fieldLabel: gettext('Source Match'),
+ value: 'any',
+ disabled: !me.isCreate,
+ comboItems: matchItems,
+ listeners: {
+ change: function (f, value) {
+ setEvery(me.srcSelector, value === 'every');
+ me.srcSelector.validate();
+ },
+ },
+ },
+ {
+ xtype: 'proxmoxcheckbox',
+ name: 'bidirectional',
+ fieldLabel: gettext('Both Directions'),
+ uncheckedValue: null,
+ disabled: !me.isCreate,
+ autoEl: {
+ tag: 'div',
+ 'data-qtip': gettext(
+ 'Also apply the rule with source and destination swapped.',
+ ),
+ },
+ },
+ ];
+
+ me.column2 = [
+ me.dstSelector,
+ {
+ xtype: 'proxmoxKVComboBox',
+ name: 'dst-match',
+ fieldLabel: gettext('Dest. Match'),
+ value: 'any',
+ disabled: !me.isCreate,
+ comboItems: matchItems,
+ listeners: {
+ change: function (f, value) {
+ setEvery(me.dstSelector, value === 'every');
+ me.dstSelector.setIncludeUntagged(value === 'any');
+ me.dstSelector.validate();
+ },
+ },
+ },
+ {
+ xtype: 'proxmoxintegerfield',
+ name: 'prio',
+ fieldLabel: gettext('Priority'),
+ minValue: 0,
+ maxValue: 65535,
+ value: 0,
+ allowBlank: false,
+ autoEl: {
+ tag: 'div',
+ 'data-qtip': gettext(
+ 'The highest priority wins. At a tie, the most specific rules decide and a deny wins among equals.',
+ ),
+ },
+ },
+ {
+ xtype: 'proxmoxKVComboBox',
+ name: 'allow',
+ fieldLabel: gettext('Action'),
+ value: 1,
+ comboItems: [
+ [1, gettext('Allow')],
+ [0, gettext('Deny')],
+ ],
+ },
+ ];
+
+ me.columnB = [
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'comment',
+ fieldLabel: gettext('Comment'),
+ allowBlank: true,
+ maxLength: 256,
+ deleteEmpty: !me.isCreate,
+ },
+ {
+ xtype: 'displayfield',
+ userCls: 'pmx-hint',
+ value: gettext(
+ 'Rules are stateless. A one-way rule needs a second rule for the replies.',
+ ),
+ },
+ ];
+
+ me.callParent();
+ },
+});
diff --git a/www/manager6/sdn/microseg/RuleView.js b/www/manager6/sdn/microseg/RuleView.js
new file mode 100644
index 000000000000..57ca0eddc1c9
--- /dev/null
+++ b/www/manager6/sdn/microseg/RuleView.js
@@ -0,0 +1,242 @@
+Ext.define(
+ 'PVE.sdn.microseg.RuleView',
+ {
+ extend: 'PVE.sdn.microseg.BaseGrid',
+ alias: 'widget.pveSDNMicrosegRules',
+
+ title: gettext('Rules'),
+
+ baseurl: '/cluster/sdn/microseg/rule/',
+
+ features: [
+ {
+ ftype: 'grouping',
+ groupHeaderTpl: Ext.String.format(gettext('Priority {0}'), '{name:htmlEncode}'),
+ enableGroupingMenu: false,
+ },
+ ],
+
+ deriveRow: function (data) {
+ let me = this;
+ let sel = me.treeSelection;
+ let src = data.src || [];
+ let dst = data.dst || [];
+ let out = false;
+ let inbound = false;
+ if (sel?.mode === 'membership') {
+ let every = (match) => match === 'every' && !sel.untagged;
+ out = every(data['src-match']) || src.some((g) => sel.groups.has(g));
+ inbound = every(data['dst-match']) || dst.some((g) => sel.groups.has(g));
+ if (data.bidirectional && (out || inbound)) {
+ out = true;
+ inbound = true;
+ }
+ } else if (sel?.mode === 'group-set') {
+ out = sel.outbound.has(data.id);
+ inbound = sel.inbound.has(data.id);
+ }
+ return {
+ direction: out && inbound ? 'both' : out ? 'out' : inbound ? 'in' : '',
+ };
+ },
+
+ loadRows: function () {
+ let me = this;
+ let rows = (me.rules || []).map((data) => ({ ...data, ...me.deriveRow(data) }));
+ if (me.treeSelection) {
+ rows = rows.filter((row) => row.direction !== '');
+ }
+ me.getStore().loadData(rows);
+ },
+
+ setData: function (rules) {
+ let me = this;
+ me.rules = rules;
+ me.loadRows();
+ },
+
+ setTreeSelection: function (sel) {
+ let me = this;
+
+ let mode = sel?.mode;
+ me.treeSelection = mode
+ ? {
+ mode: mode,
+ untagged: sel.untagged,
+ groups: new Set(sel.groups),
+ outbound: sel.outbound,
+ inbound: sel.inbound,
+ }
+ : null;
+
+ let text = '';
+ if (sel?.scope) {
+ text = Ext.String.format(
+ gettext('Rules for {0}'),
+ Ext.String.htmlEncode(sel.scope),
+ );
+ }
+ me.down('#microsegScopeText').setText(text);
+
+ me.loadRows();
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ let store = Ext.create('Ext.data.Store', {
+ model: 'pve-sdn-microseg-rule',
+ grouper: {
+ property: 'prio',
+ sortProperty: 'prio',
+ direction: 'DESC',
+ },
+ sorters: [{ property: 'id', direction: 'ASC' }],
+ });
+
+ let matchWords = {
+ any: gettext('any of'),
+ all: gettext('all of'),
+ exact: gettext('exactly'),
+ every: gettext('every'),
+ };
+
+ let renderPredicate = (match, groups) => {
+ if (match === 'every') {
+ return matchWords.every;
+ }
+ let list = groups || [];
+ let chips = list.map(PVE.Utils.render_sdnmicroseg_group).join('');
+ let kind = match || 'any';
+ if (kind !== 'any' || list.length > 1) {
+ return `${matchWords[kind]} ${chips}`;
+ }
+ return chips;
+ };
+
+ let allowIcon =
+ '<i class="fa fa-fw fa-check good" data-qtip="' + gettext('Allow') + '"></i>';
+ let denyIcon =
+ '<i class="fa fa-fw fa-times critical" data-qtip="' + gettext('Deny') + '"></i>';
+
+ let outIcon =
+ '<i class="fa fa-fw fa-sign-out" data-qtip="' +
+ gettext('Outbound, the selection is the source') +
+ '"></i>';
+ let inIcon =
+ '<i class="fa fa-fw fa-sign-in" data-qtip="' +
+ gettext('Inbound, the selection is the destination') +
+ '"></i>';
+ let bothIcon =
+ '<i class="fa fa-fw fa-exchange" data-qtip="' +
+ gettext('Both, the selection is the source and the destination') +
+ '"></i>';
+ let arrowIcon = '<i class="fa fa-fw fa-long-arrow-right"></i>';
+ let bothWaysIcon =
+ '<i class="fa fa-fw fa-arrows-h" data-qtip="' +
+ gettext('Both directions') +
+ '"></i>';
+ let renderDirection = (value, rec) => {
+ if (value === 'both') {
+ return bothIcon;
+ }
+ if (value === 'out') {
+ return outIcon;
+ }
+ if (value === 'in') {
+ return inIcon;
+ }
+ return rec.data.bidirectional ? bothWaysIcon : arrowIcon;
+ };
+
+ let ruleName = (rec) => {
+ let arrow = rec.data.bidirectional ? '<->' : '->';
+ let src = PVE.Utils.format_sdnmicroseg_predicate(
+ rec.data['src-match'],
+ rec.data.src,
+ );
+ let dst = PVE.Utils.format_sdnmicroseg_predicate(
+ rec.data['dst-match'],
+ rec.data.dst,
+ );
+ return `${src} ${arrow} ${dst}`;
+ };
+
+ me.columns = [
+ {
+ text: '',
+ dataIndex: 'allow',
+ width: 40,
+ resizable: false,
+ renderer: (v) => (v ? allowIcon : denyIcon),
+ },
+ {
+ text: gettext('Source'),
+ dataIndex: 'src',
+ flex: 2,
+ renderer: (v, m, rec) => renderPredicate(rec.data['src-match'], v),
+ },
+ {
+ text: '',
+ dataIndex: 'direction',
+ width: 40,
+ resizable: false,
+ renderer: (v, m, rec) => renderDirection(v, rec),
+ },
+ {
+ text: gettext('Destination'),
+ dataIndex: 'dst',
+ flex: 2,
+ renderer: (v, m, rec) => renderPredicate(rec.data['dst-match'], v),
+ },
+ { xtype: 'pveSDNMicrosegStateColumn' },
+ {
+ text: gettext('Comment'),
+ dataIndex: 'comment',
+ flex: 1,
+ renderer: Ext.String.htmlEncode,
+ },
+ ];
+
+ Ext.apply(me, {
+ store: store,
+ viewConfig: {
+ getRowClass: PVE.Utils.sdnmicroseg_row_cls,
+ },
+ getRecordName: ruleName,
+ addButton: {
+ text: gettext('Add Rule'),
+ handler: () => me.openEdit('rule'),
+ },
+ tbar: [
+ '->',
+ {
+ xtype: 'tbtext',
+ itemId: 'microsegScopeText',
+ },
+ ],
+ });
+
+ me.callParent();
+ },
+ },
+ function () {
+ Ext.define('pve-sdn-microseg-rule', {
+ extend: 'Ext.data.Model',
+ fields: [
+ 'id',
+ 'type',
+ 'comment',
+ 'state',
+ 'src',
+ 'src-match',
+ 'dst',
+ 'dst-match',
+ 'prio',
+ 'allow',
+ 'bidirectional',
+ 'direction',
+ ],
+ });
+ },
+);
diff --git a/www/manager6/sdn/microseg/Tree.js b/www/manager6/sdn/microseg/Tree.js
new file mode 100644
index 000000000000..e261e4944e81
--- /dev/null
+++ b/www/manager6/sdn/microseg/Tree.js
@@ -0,0 +1,249 @@
+Ext.define(
+ 'PVE.sdn.microseg.Tree',
+ {
+ extend: 'Ext.tree.Panel',
+ alias: 'widget.pveSDNMicrosegTree',
+ mixins: ['PVE.sdn.microseg.ViewMixin'],
+
+ baseurl: '/cluster/sdn/microseg/group/',
+
+ rootVisible: false,
+ animate: false,
+ border: false,
+
+ buildNodes: function (groups, assignments, groupSets) {
+ let guestDisplay = (vmid) => {
+ let store = PVE.data.ResourceStore;
+ let rec = store.getById(`qemu/${vmid}`) || store.getById(`lxc/${vmid}`);
+ if (!rec) {
+ return { text: `vm${vmid}`, iconCls: 'fa fa-fw fa-desktop' };
+ }
+ return {
+ text: PVE.Utils.getFormattedGuestIdentifier(vmid, rec.data.name),
+ iconCls: `fa-fw ${PVE.Utils.get_object_icon_class(rec.data.type, rec.data)}`,
+ };
+ };
+
+ groups.sort((a, b) => a.id.localeCompare(b.id));
+
+ let membership = PVE.sdn.microseg.Membership;
+ let rulesFor = membership.rulesFor(groupSets);
+ let nicGroupSet = membership.nicGroupSets(assignments, rulesFor);
+
+ let groupNics = {};
+ for (const [key, groupSet] of Object.entries(nicGroupSet)) {
+ let [vmid, nic] = key.split('/').map(Number);
+ groupSet.groups.forEach((g) => {
+ (groupNics[g] = groupNics[g] || []).push({ vmid: vmid, nic: nic });
+ });
+ }
+
+ let guests = {};
+ let guestInfo = (vmid) => {
+ if (!guests[vmid]) {
+ guests[vmid] = {
+ ...guestDisplay(vmid),
+ selection: membership.guestSelection(vmid, nicGroupSet, rulesFor),
+ };
+ }
+ return guests[vmid];
+ };
+
+ let nodes = [
+ {
+ id: 'all',
+ text: gettext('All Rules'),
+ iconCls: 'fa fa-fw fa-list',
+ leaf: true,
+ },
+ {
+ id: 'untagged',
+ text: 'untagged',
+ iconCls: 'fa fa-fw fa-ban',
+ microsegSelection: {
+ scope: 'untagged',
+ mode: 'membership',
+ untagged: true,
+ groups: ['untagged'],
+ },
+ leaf: true,
+ },
+ ];
+
+ for (const group of groups) {
+ let groupId = `g/${group.id}`;
+ let byVmid = {};
+ for (const { vmid, nic } of groupNics[group.id] || []) {
+ (byVmid[vmid] = byVmid[vmid] || []).push(nic);
+ }
+ let guestNodes = Object.keys(byVmid)
+ .map(Number)
+ .sort((a, b) => a - b)
+ .map((vmid) => {
+ let guestId = `${groupId}/m/${vmid}`;
+ let nics = byVmid[vmid].sort((a, b) => a - b);
+ let guest = guestInfo(vmid);
+ return {
+ id: guestId,
+ text: guest.text,
+ iconCls: guest.iconCls,
+ microsegSelection: guest.selection,
+ children: nics.map((nic) => {
+ let groupSet = nicGroupSet[`${vmid}/${nic}`];
+ return {
+ id: `${guestId}/n/${nic}`,
+ text: `net${nic}`,
+ microsegNote: groupSet.note,
+ iconCls: 'fa fa-fw fa-plug',
+ microsegSelection: {
+ scope: `vm${vmid}/net${nic} {${groupSet.groups.join(', ')}}`,
+ mode: 'group-set',
+ outbound: groupSet.rules.outbound,
+ inbound: groupSet.rules.inbound,
+ vmid: vmid,
+ nic: nic,
+ },
+ leaf: true,
+ };
+ }),
+ };
+ });
+
+ let count = (groupNics[group.id] || []).length;
+ nodes.push({
+ id: groupId,
+ text: group.id,
+ microsegNote: count ? `(${count})` : undefined,
+ iconCls: 'fa fa-fw fa-object-group',
+ microsegId: group.id,
+ microsegSelection: {
+ scope: group.id,
+ mode: 'membership',
+ groups: [group.id],
+ },
+ state: group.state,
+ leaf: !guestNodes.length,
+ children: guestNodes,
+ });
+ }
+ return nodes;
+ },
+
+ updateIdSpace: function (ids) {
+ let me = this;
+ let text = Ext.String.format(gettext('{0} of {1} IDs in use'), ids.used, ids.total);
+ let tip = ids.retired ? Ext.String.format(gettext('{0} retired IDs'), ids.retired) : '';
+ me.down('#microsegIdSpace').update(
+ `<span data-qtip="${Ext.String.htmlEncode(tip)}">${Ext.String.htmlEncode(text)}</span>`,
+ );
+ },
+
+ setData: function (groups, assignments, groupSets, idSpace) {
+ let me = this;
+ let expanded = [];
+ let oldRoot = me.getRootNode();
+ if (oldRoot) {
+ oldRoot.cascadeBy((node) => {
+ if (!node.isRoot() && node.isExpanded()) {
+ expanded.push(node.getId());
+ }
+ });
+ }
+ let selectedId = me.getSelection()[0]?.getId();
+
+ me.updateIdSpace(idSpace);
+ me.setRootNode({
+ expanded: true,
+ children: me.buildNodes(groups, assignments, groupSets),
+ });
+ let store = me.getStore();
+ for (const id of expanded) {
+ store.getNodeById(id)?.expand();
+ }
+ if (selectedId) {
+ let node = store.getNodeById(selectedId);
+ if (node) {
+ me.setSelection(node);
+ }
+ }
+ },
+
+ statics: {
+ editable: (node) => !!node.data.microsegId && node.data.state !== 'deleted',
+ },
+
+ header: {
+ items: [
+ {
+ xtype: 'component',
+ itemId: 'microsegIdSpace',
+ cls: 'pve-microseg-muted pve-microseg-id-space',
+ },
+ ],
+ },
+
+ viewConfig: {
+ getRowClass: PVE.Utils.sdnmicroseg_row_cls,
+ toggleOnDblClick: false,
+ },
+
+ store: {
+ type: 'tree',
+ model: 'pve-sdn-microseg-tree',
+ root: { expanded: true, children: [] },
+ },
+
+ columns: [
+ {
+ xtype: 'treecolumn',
+ text: gettext('Name'),
+ dataIndex: 'text',
+ flex: 1,
+ renderer: (v, m, rec) => {
+ let html = Ext.String.htmlEncode(v);
+ if (rec.data.microsegNote) {
+ html += ` <span class="pve-microseg-muted">${Ext.String.htmlEncode(rec.data.microsegNote)}</span>`;
+ }
+ return html;
+ },
+ },
+ { xtype: 'pveSDNMicrosegStateColumn' },
+ ],
+
+ tbar: [
+ {
+ text: gettext('Add Group'),
+ handler: 'addGroup',
+ },
+ '-',
+ {
+ xtype: 'proxmoxButton',
+ text: gettext('Edit'),
+ parentXType: 'treepanel',
+ disabled: true,
+ enableFn: (node) => PVE.sdn.microseg.Tree.editable(node),
+ handler: 'editGroup',
+ },
+ {
+ xtype: 'proxmoxStdRemoveButton',
+ parentXType: 'treepanel',
+ enableFn: (node) => PVE.sdn.microseg.Tree.editable(node),
+ getUrl: function (node) {
+ return this.up('treepanel').baseurl + node.data.microsegId;
+ },
+ getRecordName: (node) => node.data.microsegId,
+ callback: 'reload',
+ },
+ {
+ text: gettext('Reload'),
+ handler: 'reload',
+ },
+ ],
+ },
+ function () {
+ Ext.define('pve-sdn-microseg-tree', {
+ extend: 'Ext.data.TreeModel',
+ fields: ['microsegId', 'microsegNote', 'microsegSelection', 'state'],
+ });
+ },
+);
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-manager v3 35/45] ui: sdn: dry-run: show pending microseg diff
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (33 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-manager v3 34/45] ui: sdn: add microsegmentation panel Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-manager v3 36/45] ui: sdn: microseg: add tag matcher Hannes Laimer
` (9 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
www/manager6/sdn/SdnDiffView.js | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/www/manager6/sdn/SdnDiffView.js b/www/manager6/sdn/SdnDiffView.js
index 2bc52049585f..b835c8503044 100644
--- a/www/manager6/sdn/SdnDiffView.js
+++ b/www/manager6/sdn/SdnDiffView.js
@@ -17,6 +17,8 @@ Ext.define('PVE.sdn.SdnDiffView', {
node: undefined,
frr_diff: '',
interfaces_diff: '',
+ microseg_diff: '',
+ microseg_withheld: false,
},
},
@@ -44,6 +46,11 @@ Ext.define('PVE.sdn.SdnDiffView', {
'interfaces_diff',
Ext.htmlEncode(diff['interfaces-diff'] ?? gettext('No changes')),
);
+ vm.set(
+ 'microseg_diff',
+ Ext.htmlEncode(diff['microseg-diff'] ?? gettext('No changes')),
+ );
+ vm.set('microseg_withheld', diff['microseg-diff'] === undefined);
})
.catch(Proxmox.Utils.alertResponseFailure)
.finally(() => {
@@ -102,6 +109,28 @@ Ext.define('PVE.sdn.SdnDiffView', {
},
],
},
+ {
+ xtype: 'panel',
+ title: gettext('Microsegmentation'),
+ flex: 1,
+ scrollable: true,
+ bind: {
+ hidden: '{microseg_withheld}',
+ },
+ items: [
+ {
+ xtype: 'component',
+ padding: 5,
+ style: {
+ 'white-space': 'pre',
+ 'font-family': 'monospace',
+ },
+ bind: {
+ html: '{microseg_diff}',
+ },
+ },
+ ],
+ },
],
buttons: [
{
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-manager v3 36/45] ui: sdn: microseg: add tag matcher
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (34 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-manager v3 35/45] ui: sdn: dry-run: show pending microseg diff Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-manager v3 37/45] ui: sdn: microseg: add name regex matcher Hannes Laimer
` (8 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
www/manager6/Utils.js | 14 +++++++
www/manager6/sdn/microseg/AssignmentEdit.js | 46 +++++++++++++++++++++
www/manager6/sdn/microseg/AssignmentView.js | 2 +
3 files changed, 62 insertions(+)
diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index 8eb4b68c51b4..71c010bc91cc 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -999,6 +999,13 @@ Ext.define('PVE.Utils', {
faIcon: 'link',
assignment: true,
},
+ 'tag-assignment': {
+ name: gettext('Tag Assignment'),
+ ipanel: 'TagAssignmentInputPanel',
+ width: 400,
+ faIcon: 'tag',
+ assignment: true,
+ },
},
sdnmicroseg_is_assignment: function (type) {
@@ -1037,6 +1044,13 @@ Ext.define('PVE.Utils', {
return Proxmox.Utils.unknownText;
}
let iface = (assignment.iface ?? '') === '' ? '' : ` net${assignment.iface}`;
+ if (assignment.type === 'tag-assignment') {
+ let tags = PVE.Utils.format_sdnmicroseg_predicate(
+ assignment['tag-match'],
+ assignment.tags,
+ );
+ return `${tags}${iface}`;
+ }
return `vm${assignment.vmid}${iface}`;
},
diff --git a/www/manager6/sdn/microseg/AssignmentEdit.js b/www/manager6/sdn/microseg/AssignmentEdit.js
index 71cabaeeea52..470ca15a271d 100644
--- a/www/manager6/sdn/microseg/AssignmentEdit.js
+++ b/www/manager6/sdn/microseg/AssignmentEdit.js
@@ -78,3 +78,49 @@ Ext.define('PVE.sdn.microseg.GuestAssignmentInputPanel', {
me.callParent();
},
});
+
+Ext.define('PVE.sdn.microseg.TagAssignmentInputPanel', {
+ extend: 'PVE.sdn.microseg.AssignmentInputPanel',
+
+ onGetValues: function (values) {
+ let me = this;
+ if (typeof values.tags === 'string') {
+ values.tags = values.tags.split(';').filter((tag) => tag !== '');
+ }
+ return me.callParent([values]);
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ me.matcherItems = [
+ {
+ xtype: 'pveTagFieldSet',
+ name: 'tags',
+ maxHeight: 150,
+ disabled: !me.isCreate,
+ getErrors: function (value) {
+ let errors = PVE.form.TagFieldSet.prototype.getErrors.call(this, value);
+ if (this.getValue() === '') {
+ errors.push(gettext('At least one tag is required'));
+ }
+ return errors;
+ },
+ },
+ {
+ xtype: 'proxmoxKVComboBox',
+ name: 'tag-match',
+ fieldLabel: gettext('Tag Match'),
+ value: 'any',
+ disabled: !me.isCreate,
+ comboItems: [
+ ['any', gettext('Any (shares a tag)')],
+ ['all', gettext('All (has every tag)')],
+ ['exact', gettext('Exact (same tags)')],
+ ],
+ },
+ ];
+
+ me.callParent();
+ },
+});
diff --git a/www/manager6/sdn/microseg/AssignmentView.js b/www/manager6/sdn/microseg/AssignmentView.js
index 4744380461a7..0dc96dc831eb 100644
--- a/www/manager6/sdn/microseg/AssignmentView.js
+++ b/www/manager6/sdn/microseg/AssignmentView.js
@@ -181,6 +181,8 @@ Ext.define(
'type',
'vmid',
'iface',
+ 'tags',
+ 'tag-match',
'groups',
'comment',
'state',
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-manager v3 37/45] ui: sdn: microseg: add name regex matcher
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (35 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-manager v3 36/45] ui: sdn: microseg: add tag matcher Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-manager v3 38/45] ui: sdn: microseg: add narrowing to the rule form Hannes Laimer
` (7 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
Only the backend checks the pattern's syntax, since it matches the
pattern as a Rust regex.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
www/manager6/Utils.js | 10 ++++++++++
www/manager6/sdn/microseg/AssignmentEdit.js | 22 +++++++++++++++++++++
www/manager6/sdn/microseg/AssignmentView.js | 1 +
3 files changed, 33 insertions(+)
diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index 71c010bc91cc..8dbecabe7a75 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -1006,6 +1006,13 @@ Ext.define('PVE.Utils', {
faIcon: 'tag',
assignment: true,
},
+ 'regex-assignment': {
+ name: gettext('Regex Assignment'),
+ ipanel: 'RegexAssignmentInputPanel',
+ width: 400,
+ faIcon: 'code',
+ assignment: true,
+ },
},
sdnmicroseg_is_assignment: function (type) {
@@ -1051,6 +1058,9 @@ Ext.define('PVE.Utils', {
);
return `${tags}${iface}`;
}
+ if (assignment.type === 'regex-assignment') {
+ return `/${assignment['name-regex']}/${iface}`;
+ }
return `vm${assignment.vmid}${iface}`;
},
diff --git a/www/manager6/sdn/microseg/AssignmentEdit.js b/www/manager6/sdn/microseg/AssignmentEdit.js
index 470ca15a271d..a36f0f45648b 100644
--- a/www/manager6/sdn/microseg/AssignmentEdit.js
+++ b/www/manager6/sdn/microseg/AssignmentEdit.js
@@ -79,6 +79,28 @@ Ext.define('PVE.sdn.microseg.GuestAssignmentInputPanel', {
},
});
+Ext.define('PVE.sdn.microseg.RegexAssignmentInputPanel', {
+ extend: 'PVE.sdn.microseg.AssignmentInputPanel',
+
+ initComponent: function () {
+ let me = this;
+
+ me.matcherItems = [
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'name-regex',
+ fieldLabel: gettext('Name Regex'),
+ allowBlank: false,
+ maxLength: 1024,
+ emptyText: '^web-',
+ disabled: !me.isCreate,
+ },
+ ];
+
+ me.callParent();
+ },
+});
+
Ext.define('PVE.sdn.microseg.TagAssignmentInputPanel', {
extend: 'PVE.sdn.microseg.AssignmentInputPanel',
diff --git a/www/manager6/sdn/microseg/AssignmentView.js b/www/manager6/sdn/microseg/AssignmentView.js
index 0dc96dc831eb..ea6c447bf710 100644
--- a/www/manager6/sdn/microseg/AssignmentView.js
+++ b/www/manager6/sdn/microseg/AssignmentView.js
@@ -183,6 +183,7 @@ Ext.define(
'iface',
'tags',
'tag-match',
+ 'name-regex',
'groups',
'comment',
'state',
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-manager v3 38/45] ui: sdn: microseg: add narrowing to the rule form
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (36 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-manager v3 37/45] ui: sdn: microseg: add name regex matcher Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-manager v3 39/45] ui: sdn: microseg: show and edit the group hierarchy Hannes Laimer
` (6 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
Reuse the firewall's macro, protocol and ICMP type selectors, so a
narrowed rule is entered the same way as a firewall rule.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
www/manager6/grid/FirewallRules.js | 2 +
www/manager6/sdn/microseg/RuleEdit.js | 111 +++++++++++++++++++++++++-
www/manager6/sdn/microseg/RuleView.js | 42 +++++++++-
3 files changed, 151 insertions(+), 4 deletions(-)
diff --git a/www/manager6/grid/FirewallRules.js b/www/manager6/grid/FirewallRules.js
index 0db817ebce0e..d408bd00b7f0 100644
--- a/www/manager6/grid/FirewallRules.js
+++ b/www/manager6/grid/FirewallRules.js
@@ -81,6 +81,7 @@ Ext.define('PVE.form.ICMPTypeSelector', {
});
let ICMP_TYPE_NAMES_STORE = Ext.create('Ext.data.Store', {
+ storeId: 'pve-icmp-type-names',
field: ['type', 'name'],
data: [
{ type: 'any', name: 'any' },
@@ -123,6 +124,7 @@ let ICMP_TYPE_NAMES_STORE = Ext.create('Ext.data.Store', {
],
});
let ICMPV6_TYPE_NAMES_STORE = Ext.create('Ext.data.Store', {
+ storeId: 'pve-icmpv6-type-names',
field: ['type', 'name'],
data: [
{ type: '1', name: 'destination-unreachable' },
diff --git a/www/manager6/sdn/microseg/RuleEdit.js b/www/manager6/sdn/microseg/RuleEdit.js
index 4ec5467f61be..3ee9c3b36e46 100644
--- a/www/manager6/sdn/microseg/RuleEdit.js
+++ b/www/manager6/sdn/microseg/RuleEdit.js
@@ -60,6 +60,37 @@ Ext.define('PVE.sdn.microseg.RuleInputPanel', {
},
});
+ let setNarrowing = (field, enabled, clear) => {
+ field.setDisabled(!enabled);
+ if (clear) {
+ field.setValue('');
+ }
+ };
+ let updateNarrowing = () => {
+ let macro = me.down('field[name=macro]');
+ let proto = me.down('field[name=proto]');
+ let byMacro = !Ext.isEmpty(macro.getValue());
+ if (me.isCreate) {
+ setNarrowing(proto, !byMacro, byMacro);
+ }
+ let editable = me.isCreate && !byMacro;
+ let protocol = proto.getValue();
+ let icmpv4 = protocol === 'icmp';
+ let icmpv6 = protocol === 'icmpv6' || protocol === 'ipv6-icmp';
+ let v4 = me.down('#microsegIcmpv4Type');
+ let v6 = me.down('#microsegIcmpv6Type');
+ v4.setHidden(!icmpv4);
+ setNarrowing(v4, icmpv4 && editable);
+ v6.setHidden(!icmpv6);
+ setNarrowing(v6, icmpv6 && editable);
+ let ports = !icmpv4 && !icmpv6;
+ for (const name of ['sport', 'dport']) {
+ let field = me.down(`field[name=${name}]`);
+ field.setHidden(!ports);
+ setNarrowing(field, ports && editable, me.isCreate && !editable);
+ }
+ };
+
me.column1 = [
me.srcSelector,
{
@@ -76,6 +107,69 @@ Ext.define('PVE.sdn.microseg.RuleInputPanel', {
},
},
},
+ {
+ xtype: 'displayfield',
+ fieldLabel: '',
+ height: 7,
+ value: '',
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'sport',
+ fieldLabel: gettext('Source port'),
+ disabled: !me.isCreate,
+ },
+ {
+ xtype: 'pveICMPTypeSelector',
+ name: 'icmp-type',
+ itemId: 'microsegIcmpv4Type',
+ autoSelect: false,
+ editable: true,
+ hidden: true,
+ disabled: true,
+ value: '',
+ skipEmptyText: true,
+ fieldLabel: gettext('ICMP type'),
+ store: 'pve-icmp-type-names',
+ },
+ {
+ xtype: 'pveICMPTypeSelector',
+ name: 'icmp-type',
+ itemId: 'microsegIcmpv6Type',
+ autoSelect: false,
+ editable: true,
+ hidden: true,
+ disabled: true,
+ value: '',
+ skipEmptyText: true,
+ fieldLabel: gettext('ICMP type'),
+ store: 'pve-icmpv6-type-names',
+ },
+ {
+ xtype: 'pveFWMacroSelector',
+ name: 'macro',
+ fieldLabel: gettext('Macro'),
+ editable: true,
+ allowBlank: true,
+ skipEmptyText: true,
+ disabled: !me.isCreate,
+ listeners: {
+ change: updateNarrowing,
+ },
+ },
+ {
+ xtype: 'pveIPProtocolSelector',
+ name: 'proto',
+ autoSelect: false,
+ editable: true,
+ value: '',
+ skipEmptyText: true,
+ fieldLabel: gettext('Protocol'),
+ disabled: !me.isCreate,
+ listeners: {
+ change: updateNarrowing,
+ },
+ },
{
xtype: 'proxmoxcheckbox',
name: 'bidirectional',
@@ -85,7 +179,7 @@ Ext.define('PVE.sdn.microseg.RuleInputPanel', {
autoEl: {
tag: 'div',
'data-qtip': gettext(
- 'Also apply the rule with source and destination swapped.',
+ 'Also apply the rule with source and destination swapped, ports included.',
),
},
},
@@ -108,6 +202,19 @@ Ext.define('PVE.sdn.microseg.RuleInputPanel', {
},
},
},
+ {
+ xtype: 'displayfield',
+ fieldLabel: '',
+ height: 7,
+ value: '',
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'dport',
+ fieldLabel: gettext('Dest. port'),
+ hideMode: 'visibility',
+ disabled: !me.isCreate,
+ },
{
xtype: 'proxmoxintegerfield',
name: 'prio',
@@ -119,7 +226,7 @@ Ext.define('PVE.sdn.microseg.RuleInputPanel', {
autoEl: {
tag: 'div',
'data-qtip': gettext(
- 'The highest priority wins. At a tie, the most specific rules decide and a deny wins among equals.',
+ 'The highest priority wins. At a tie, the most specific rules decide and a deny wins among equals. A narrowed rule is checked ahead of one without any narrowing at the same priority.',
),
},
},
diff --git a/www/manager6/sdn/microseg/RuleView.js b/www/manager6/sdn/microseg/RuleView.js
index 57ca0eddc1c9..016e1dc4455e 100644
--- a/www/manager6/sdn/microseg/RuleView.js
+++ b/www/manager6/sdn/microseg/RuleView.js
@@ -36,6 +36,7 @@ Ext.define(
inbound = sel.inbound.has(data.id);
}
return {
+ rank: me.rankOf(data),
direction: out && inbound ? 'both' : out ? 'out' : inbound ? 'in' : '',
};
},
@@ -91,7 +92,10 @@ Ext.define(
sortProperty: 'prio',
direction: 'DESC',
},
- sorters: [{ property: 'id', direction: 'ASC' }],
+ sorters: [
+ { property: 'rank', direction: 'ASC' },
+ { property: 'id', direction: 'ASC' },
+ ],
});
let matchWords = {
@@ -149,6 +153,26 @@ Ext.define(
return rec.data.bidirectional ? bothWaysIcon : arrowIcon;
};
+ let matchText = (rec) => {
+ let data = rec.data;
+ if (data.macro) {
+ return data.macro;
+ }
+ let proto = data.proto;
+ if (!proto) {
+ return '';
+ }
+ let text = proto;
+ if (data['icmp-type']) {
+ text += ` ${data['icmp-type']}`;
+ } else if (data.sport) {
+ text += ` ${data.sport} -> ${data.dport || '*'}`;
+ } else if (data.dport) {
+ text += `/${data.dport}`;
+ }
+ return text;
+ };
+
let ruleName = (rec) => {
let arrow = rec.data.bidirectional ? '<->' : '->';
let src = PVE.Utils.format_sdnmicroseg_predicate(
@@ -159,7 +183,9 @@ Ext.define(
rec.data['dst-match'],
rec.data.dst,
);
- return `${src} ${arrow} ${dst}`;
+ let name = `${src} ${arrow} ${dst}`;
+ let match = matchText(rec);
+ return match ? `${name} (${match})` : name;
};
me.columns = [
@@ -189,6 +215,12 @@ Ext.define(
flex: 2,
renderer: (v, m, rec) => renderPredicate(rec.data['dst-match'], v),
},
+ {
+ text: gettext('Match'),
+ dataIndex: 'proto',
+ flex: 1,
+ renderer: (v, m, rec) => Ext.String.htmlEncode(matchText(rec)),
+ },
{ xtype: 'pveSDNMicrosegStateColumn' },
{
text: gettext('Comment'),
@@ -235,7 +267,13 @@ Ext.define(
'prio',
'allow',
'bidirectional',
+ 'proto',
+ 'sport',
+ 'dport',
+ 'icmp-type',
+ 'macro',
'direction',
+ 'rank',
],
});
},
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-manager v3 39/45] ui: sdn: microseg: show and edit the group hierarchy
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (37 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-manager v3 38/45] ui: sdn: microseg: add narrowing to the rule form Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-manager v3 40/45] ui: sdn: microseg: add fragments option to the rule form Hannes Laimer
` (5 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
Selecting a group filters the rules to those naming it or a group
above it, so no rule its members inherit is left out.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
www/manager6/form/SDNMicrosegGroupSelector.js | 26 +++++++++-
www/manager6/sdn/MicrosegView.js | 8 ++-
www/manager6/sdn/microseg/AssignmentView.js | 5 +-
www/manager6/sdn/microseg/Base.js | 48 ++++++++++++++++--
www/manager6/sdn/microseg/GroupEdit.js | 10 ++++
www/manager6/sdn/microseg/RuleView.js | 26 ++++++++++
www/manager6/sdn/microseg/Tree.js | 49 +++++++++++++++----
7 files changed, 153 insertions(+), 19 deletions(-)
diff --git a/www/manager6/form/SDNMicrosegGroupSelector.js b/www/manager6/form/SDNMicrosegGroupSelector.js
index 1b0c39a1cfba..e7a4077e97df 100644
--- a/www/manager6/form/SDNMicrosegGroupSelector.js
+++ b/www/manager6/form/SDNMicrosegGroupSelector.js
@@ -34,12 +34,28 @@ Ext.define(
initComponent: function () {
let me = this;
- let store = new Ext.data.Store({
+ let store;
+ let isExcluded = (rec) => {
+ let data = store.getData();
+ let all = data.getSource() || data;
+ let seen = new Set();
+ while (rec && !seen.has(rec.getId())) {
+ if (rec.getId() === me.excludeId) {
+ return true;
+ }
+ seen.add(rec.getId());
+ rec = all.get(rec.get('parent'));
+ }
+ return false;
+ };
+
+ store = new Ext.data.Store({
model: 'pve-sdn-microseg-group',
sorters: {
property: 'id',
direction: 'ASC',
},
+ filters: me.excludeId ? [(rec) => !isExcluded(rec)] : [],
});
me.mon(store, 'load', () => me.syncUntagged());
@@ -55,6 +71,12 @@ Ext.define(
dataIndex: 'id',
flex: 1,
},
+ {
+ header: gettext('Parent'),
+ sortable: true,
+ dataIndex: 'parent',
+ flex: 1,
+ },
{
header: gettext('Comment'),
dataIndex: 'comment',
@@ -73,7 +95,7 @@ Ext.define(
function () {
Ext.define('pve-sdn-microseg-group', {
extend: 'Ext.data.Model',
- fields: ['id', 'comment'],
+ fields: ['id', 'parent', 'comment'],
proxy: {
type: 'proxmox',
url: '/api2/json/cluster/sdn/microseg/group',
diff --git a/www/manager6/sdn/MicrosegView.js b/www/manager6/sdn/MicrosegView.js
index 805b5b7ebdd9..028253920081 100644
--- a/www/manager6/sdn/MicrosegView.js
+++ b/www/manager6/sdn/MicrosegView.js
@@ -36,7 +36,13 @@ Ext.define('PVE.sdn.MicrosegView', {
},
addGroup: function () {
- this.lookup('tree').openEdit('group');
+ let me = this;
+ let parent = me.selectedGroup();
+ me.lookup('tree').openEdit(
+ 'group',
+ undefined,
+ parent ? { defaultParent: parent } : undefined,
+ );
},
editGroup: function () {
diff --git a/www/manager6/sdn/microseg/AssignmentView.js b/www/manager6/sdn/microseg/AssignmentView.js
index ea6c447bf710..373efe02eccd 100644
--- a/www/manager6/sdn/microseg/AssignmentView.js
+++ b/www/manager6/sdn/microseg/AssignmentView.js
@@ -68,9 +68,8 @@ Ext.define(
if (sel.vmid !== undefined) {
return !winsNic(rec, sel.vmid, sel.nic);
}
- let group = sel.mode === 'membership' ? sel.groups[0] : undefined;
- if (group && group !== 'untagged') {
- return !(rec.data.groups ?? []).includes(group);
+ if (sel.mode === 'membership' && !sel.untagged) {
+ return !(rec.data.groups ?? []).some((g) => sel.subtree.includes(g));
}
return false;
};
diff --git a/www/manager6/sdn/microseg/Base.js b/www/manager6/sdn/microseg/Base.js
index 6751da1c7e88..02c176b04f5c 100644
--- a/www/manager6/sdn/microseg/Base.js
+++ b/www/manager6/sdn/microseg/Base.js
@@ -71,6 +71,43 @@ Ext.define('PVE.sdn.microseg.Membership', {
});
},
+ hierarchy: function (groups) {
+ let byId = {};
+ for (const group of groups) {
+ byId[group.id] = group;
+ }
+ let childrenOf = {};
+ for (const group of groups) {
+ let parent = group.parent;
+ let key = parent && byId[parent] ? parent : '';
+ (childrenOf[key] = childrenOf[key] || []).push(group);
+ }
+ let ancestors = (group) => {
+ let out = [];
+ let seen = new Set();
+ let parent = group.parent;
+ while (parent && byId[parent] && !seen.has(parent)) {
+ seen.add(parent);
+ out.push(parent);
+ parent = byId[parent].parent;
+ }
+ return out;
+ };
+ let close = (list) => {
+ let out = new Set();
+ for (const g of list) {
+ out.add(g);
+ if (byId[g]) {
+ for (const a of ancestors(byId[g])) {
+ out.add(a);
+ }
+ }
+ }
+ return [...out].sort();
+ };
+ return { byId, childrenOf, ancestors, close };
+ },
+
rulesFor: function (groupSets) {
let classRules = {};
groupSets.forEach((cls) => {
@@ -88,12 +125,13 @@ Ext.define('PVE.sdn.microseg.Membership', {
};
},
- nicGroupSets: function (assignments, rulesFor) {
+ nicGroupSets: function (assignments, hierarchy, rulesFor) {
let nicGroupSet = {};
assignments.forEach((a) => {
let sortedGroups = (a.groups || []).slice().sort();
let groupSet = {
groups: sortedGroups,
+ closedGroups: hierarchy.close(sortedGroups),
rules: rulesFor([sortedGroups]),
note: PVE.Utils.format_sdnmicroseg_matcher(a),
};
@@ -106,12 +144,16 @@ Ext.define('PVE.sdn.microseg.Membership', {
return nicGroupSet;
},
- guestSelection: function (vmid, nicGroupSet, rulesFor) {
+ guestSelection: function (vmid, nicGroupSet, hierarchy, rulesFor) {
let keys = Object.keys(nicGroupSet).filter((key) => key.startsWith(`${vmid}/`));
- let rules = rulesFor(keys.map((key) => nicGroupSet[key].groups));
+ let sets = keys.map((key) => nicGroupSet[key].groups);
+ let ownGroups = sets.flat();
+ let rules = rulesFor(sets);
return {
scope: `vm${vmid}`,
mode: 'group-set',
+ groups: hierarchy.close(ownGroups),
+ ownGroups: ownGroups,
outbound: rules.outbound,
inbound: rules.inbound,
vmid: vmid,
diff --git a/www/manager6/sdn/microseg/GroupEdit.js b/www/manager6/sdn/microseg/GroupEdit.js
index 82639c344264..94f108ad46f8 100644
--- a/www/manager6/sdn/microseg/GroupEdit.js
+++ b/www/manager6/sdn/microseg/GroupEdit.js
@@ -15,6 +15,16 @@ Ext.define('PVE.sdn.microseg.GroupInputPanel', {
allowBlank: false,
maxLength: 32,
},
+ {
+ xtype: 'pveSDNMicrosegGroupSelector',
+ name: 'parent',
+ fieldLabel: gettext('Parent'),
+ allowBlank: true,
+ skipEmptyText: true,
+ deleteEmpty: !me.isCreate,
+ excludeId: me.microsegId,
+ value: me.defaultParent,
+ },
{
xtype: 'proxmoxtextfield',
name: 'comment',
diff --git a/www/manager6/sdn/microseg/RuleView.js b/www/manager6/sdn/microseg/RuleView.js
index 016e1dc4455e..58d81221402c 100644
--- a/www/manager6/sdn/microseg/RuleView.js
+++ b/www/manager6/sdn/microseg/RuleView.js
@@ -35,8 +35,15 @@ Ext.define(
out = sel.outbound.has(data.id);
inbound = sel.inbound.has(data.id);
}
+ let matched = sel
+ ? [...new Set([...src, ...dst])].filter((g) => sel.groups.has(g))
+ : [];
return {
rank: me.rankOf(data),
+ specificity: data.specificity ?? 0,
+ matched: matched
+ .sort()
+ .map((g) => (sel.ownGroups.has(g) ? g : `${g} (${gettext('inherited')})`)),
direction: out && inbound ? 'both' : out ? 'out' : inbound ? 'in' : '',
};
},
@@ -65,6 +72,7 @@ Ext.define(
mode: mode,
untagged: sel.untagged,
groups: new Set(sel.groups),
+ ownGroups: new Set(sel.ownGroups),
outbound: sel.outbound,
inbound: sel.inbound,
}
@@ -79,6 +87,8 @@ Ext.define(
}
me.down('#microsegScopeText').setText(text);
+ me.down('#microsegMatched').setVisible(!!mode);
+
me.loadRows();
},
@@ -221,6 +231,20 @@ Ext.define(
flex: 1,
renderer: (v, m, rec) => Ext.String.htmlEncode(matchText(rec)),
},
+ {
+ text: gettext('Specificity'),
+ dataIndex: 'specificity',
+ width: 100,
+ hidden: true,
+ },
+ {
+ text: gettext('Matched By'),
+ itemId: 'microsegMatched',
+ dataIndex: 'matched',
+ flex: 1,
+ hidden: true,
+ renderer: (v) => Ext.String.htmlEncode((v || []).join(', ')),
+ },
{ xtype: 'pveSDNMicrosegStateColumn' },
{
text: gettext('Comment'),
@@ -274,6 +298,8 @@ Ext.define(
'macro',
'direction',
'rank',
+ 'specificity',
+ 'matched',
],
});
},
diff --git a/www/manager6/sdn/microseg/Tree.js b/www/manager6/sdn/microseg/Tree.js
index e261e4944e81..1a3c0905d0aa 100644
--- a/www/manager6/sdn/microseg/Tree.js
+++ b/www/manager6/sdn/microseg/Tree.js
@@ -27,8 +27,11 @@ Ext.define(
groups.sort((a, b) => a.id.localeCompare(b.id));
let membership = PVE.sdn.microseg.Membership;
+ let hierarchy = membership.hierarchy(groups);
+ let childrenOf = hierarchy.childrenOf;
+ let ancestorsOf = hierarchy.ancestors;
let rulesFor = membership.rulesFor(groupSets);
- let nicGroupSet = membership.nicGroupSets(assignments, rulesFor);
+ let nicGroupSet = membership.nicGroupSets(assignments, hierarchy, rulesFor);
let groupNics = {};
for (const [key, groupSet] of Object.entries(nicGroupSet)) {
@@ -43,7 +46,12 @@ Ext.define(
if (!guests[vmid]) {
guests[vmid] = {
...guestDisplay(vmid),
- selection: membership.guestSelection(vmid, nicGroupSet, rulesFor),
+ selection: membership.guestSelection(
+ vmid,
+ nicGroupSet,
+ hierarchy,
+ rulesFor,
+ ),
};
}
return guests[vmid];
@@ -65,16 +73,20 @@ Ext.define(
mode: 'membership',
untagged: true,
groups: ['untagged'],
+ ownGroups: ['untagged'],
+ subtree: ['untagged'],
},
leaf: true,
},
];
- for (const group of groups) {
+ let groupNode = (group) => {
let groupId = `g/${group.id}`;
let byVmid = {};
+ let nicKeys = new Set();
for (const { vmid, nic } of groupNics[group.id] || []) {
(byVmid[vmid] = byVmid[vmid] || []).push(nic);
+ nicKeys.add(`${vmid}/${nic}`);
}
let guestNodes = Object.keys(byVmid)
.map(Number)
@@ -98,6 +110,8 @@ Ext.define(
microsegSelection: {
scope: `vm${vmid}/net${nic} {${groupSet.groups.join(', ')}}`,
mode: 'group-set',
+ groups: groupSet.closedGroups,
+ ownGroups: groupSet.groups,
outbound: groupSet.rules.outbound,
inbound: groupSet.rules.inbound,
vmid: vmid,
@@ -109,22 +123,37 @@ Ext.define(
};
});
- let count = (groupNics[group.id] || []).length;
- nodes.push({
+ let subgroupNodes = [];
+ let subtree = [group.id];
+ for (const sub of childrenOf[group.id] || []) {
+ let subgroup = groupNode(sub);
+ subgroupNodes.push(subgroup.node);
+ subgroup.nicKeys.forEach((key) => nicKeys.add(key));
+ subtree.push(...subgroup.subtree);
+ }
+ let children = [...subgroupNodes, ...guestNodes];
+ let node = {
id: groupId,
text: group.id,
- microsegNote: count ? `(${count})` : undefined,
+ microsegNote: nicKeys.size ? `(${nicKeys.size})` : undefined,
iconCls: 'fa fa-fw fa-object-group',
microsegId: group.id,
microsegSelection: {
scope: group.id,
mode: 'membership',
- groups: [group.id],
+ groups: [group.id, ...ancestorsOf(group)],
+ ownGroups: [group.id],
+ subtree: subtree,
},
state: group.state,
- leaf: !guestNodes.length,
- children: guestNodes,
- });
+ leaf: !children.length,
+ children: children,
+ };
+ return { node: node, nicKeys: nicKeys, subtree: subtree };
+ };
+
+ for (const group of childrenOf[''] || []) {
+ nodes.push(groupNode(group).node);
}
return nodes;
},
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-manager v3 40/45] ui: sdn: microseg: add fragments option to the rule form
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (38 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-manager v3 39/45] ui: sdn: microseg: show and edit the group hierarchy Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-manager v3 41/45] ui: sdn: microseg: add a guest tab with its groups and rules Hannes Laimer
` (4 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
www/manager6/sdn/microseg/RuleEdit.js | 26 +++++++++++++++++++++++---
www/manager6/sdn/microseg/RuleView.js | 4 ++++
2 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/www/manager6/sdn/microseg/RuleEdit.js b/www/manager6/sdn/microseg/RuleEdit.js
index 3ee9c3b36e46..95620050fad4 100644
--- a/www/manager6/sdn/microseg/RuleEdit.js
+++ b/www/manager6/sdn/microseg/RuleEdit.js
@@ -69,20 +69,24 @@ Ext.define('PVE.sdn.microseg.RuleInputPanel', {
let updateNarrowing = () => {
let macro = me.down('field[name=macro]');
let proto = me.down('field[name=proto]');
+ let fragments = me.isCreate && !!me.down('field[name=fragments]').getValue();
+ if (me.isCreate) {
+ setNarrowing(macro, !fragments, fragments);
+ }
let byMacro = !Ext.isEmpty(macro.getValue());
if (me.isCreate) {
setNarrowing(proto, !byMacro, byMacro);
}
- let editable = me.isCreate && !byMacro;
+ let editable = me.isCreate && !fragments && !byMacro;
let protocol = proto.getValue();
let icmpv4 = protocol === 'icmp';
let icmpv6 = protocol === 'icmpv6' || protocol === 'ipv6-icmp';
let v4 = me.down('#microsegIcmpv4Type');
let v6 = me.down('#microsegIcmpv6Type');
v4.setHidden(!icmpv4);
- setNarrowing(v4, icmpv4 && editable);
+ setNarrowing(v4, icmpv4 && editable, fragments);
v6.setHidden(!icmpv6);
- setNarrowing(v6, icmpv6 && editable);
+ setNarrowing(v6, icmpv6 && editable, fragments);
let ports = !icmpv4 && !icmpv6;
for (const name of ['sport', 'dport']) {
let field = me.down(`field[name=${name}]`);
@@ -240,6 +244,22 @@ Ext.define('PVE.sdn.microseg.RuleInputPanel', {
[0, gettext('Deny')],
],
},
+ {
+ xtype: 'proxmoxcheckbox',
+ name: 'fragments',
+ fieldLabel: gettext('Fragments Only'),
+ uncheckedValue: null,
+ disabled: !me.isCreate,
+ autoEl: {
+ tag: 'div',
+ 'data-qtip': gettext(
+ 'Match the IP fragments after the first. They carry no ports and no ICMP type.',
+ ),
+ },
+ listeners: {
+ change: updateNarrowing,
+ },
+ },
];
me.columnB = [
diff --git a/www/manager6/sdn/microseg/RuleView.js b/www/manager6/sdn/microseg/RuleView.js
index 58d81221402c..95c08343e725 100644
--- a/www/manager6/sdn/microseg/RuleView.js
+++ b/www/manager6/sdn/microseg/RuleView.js
@@ -169,6 +169,9 @@ Ext.define(
return data.macro;
}
let proto = data.proto;
+ if (data.fragments) {
+ return proto ? `${proto} ${gettext('fragments')}` : gettext('fragments');
+ }
if (!proto) {
return '';
}
@@ -296,6 +299,7 @@ Ext.define(
'dport',
'icmp-type',
'macro',
+ 'fragments',
'direction',
'rank',
'specificity',
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-manager v3 41/45] ui: sdn: microseg: add a guest tab with its groups and rules
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (39 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-manager v3 40/45] ui: sdn: microseg: add fragments option to the rule form Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH qemu-server v3 42/45] network: update the microseg enforcement on a tag or name change Hannes Laimer
` (3 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
The tab lists each assigned NIC with its assignment and groups, and
below that the rules that apply to the guest. They are grouped by
direction instead of by priority as in the SDN panel, since for a
single guest the direction matters more. Editing stays in the SDN
panel.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
www/manager6/Makefile | 1 +
www/manager6/lxc/Config.js | 10 ++
www/manager6/qemu/Config.js | 10 ++
www/manager6/sdn/microseg/Base.js | 41 +++++----
www/manager6/sdn/microseg/GuestPanel.js | 117 ++++++++++++++++++++++++
www/manager6/sdn/microseg/RuleView.js | 40 ++++++--
6 files changed, 195 insertions(+), 24 deletions(-)
create mode 100644 www/manager6/sdn/microseg/GuestPanel.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 7c48b0622aa7..1f429fb20bdf 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -366,6 +366,7 @@ JSSRC= \
sdn/microseg/AssignmentEdit.js \
sdn/microseg/RuleView.js \
sdn/microseg/AssignmentView.js \
+ sdn/microseg/GuestPanel.js \
storage/ContentView.js \
storage/BackupView.js \
storage/Base.js \
diff --git a/www/manager6/lxc/Config.js b/www/manager6/lxc/Config.js
index 3a99d6a71dec..d14bfa244958 100644
--- a/www/manager6/lxc/Config.js
+++ b/www/manager6/lxc/Config.js
@@ -387,6 +387,16 @@ Ext.define('PVE.lxc.Config', {
});
}
+ if (caps.vms['VM.Audit'] && caps.sdn['SDN.Audit']) {
+ me.items.push({
+ xtype: 'pveSDNMicrosegGuest',
+ title: gettext('Microsegmentation'),
+ iconCls: 'fa fa-tags',
+ itemId: 'microseg',
+ vmid: vmid,
+ });
+ }
+
if (caps.vms['Permissions.Modify']) {
me.items.push({
xtype: 'pveACLView',
diff --git a/www/manager6/qemu/Config.js b/www/manager6/qemu/Config.js
index 842d35de1202..7d7b17bfa870 100644
--- a/www/manager6/qemu/Config.js
+++ b/www/manager6/qemu/Config.js
@@ -440,6 +440,16 @@ Ext.define('PVE.qemu.Config', {
});
}
+ if (caps.vms['VM.Audit'] && caps.sdn['SDN.Audit']) {
+ me.items.push({
+ xtype: 'pveSDNMicrosegGuest',
+ title: gettext('Microsegmentation'),
+ iconCls: 'fa fa-tags',
+ itemId: 'microseg',
+ vmid: vmid,
+ });
+ }
+
if (caps.vms['Permissions.Modify']) {
me.items.push({
xtype: 'pveACLView',
diff --git a/www/manager6/sdn/microseg/Base.js b/www/manager6/sdn/microseg/Base.js
index 02c176b04f5c..327a49b49aad 100644
--- a/www/manager6/sdn/microseg/Base.js
+++ b/www/manager6/sdn/microseg/Base.js
@@ -134,6 +134,7 @@ Ext.define('PVE.sdn.microseg.Membership', {
closedGroups: hierarchy.close(sortedGroups),
rules: rulesFor([sortedGroups]),
note: PVE.Utils.format_sdnmicroseg_matcher(a),
+ assignment: a,
};
(a.members || []).forEach((m) => {
m.nics.forEach((nic) => {
@@ -223,33 +224,37 @@ Ext.define('PVE.sdn.microseg.BaseGrid', {
me.openEdit(rec.data.type, rec.data.id, me.editPanelConfig?.(rec));
};
+ let editButtons = me.readOnly
+ ? []
+ : [
+ me.addButton,
+ new Proxmox.button.Button({
+ text: gettext('Edit'),
+ disabled: true,
+ selModel: sm,
+ enableFn: editable,
+ handler: runEditor,
+ }),
+ Ext.create('Proxmox.button.StdRemoveButton', {
+ selModel: sm,
+ enableFn: editable,
+ baseurl: me.baseurl,
+ getRecordName: me.getRecordName,
+ callback: () => me.reload(),
+ }),
+ ];
+
Ext.apply(me, {
selModel: sm,
tbar: [
- me.addButton,
- new Proxmox.button.Button({
- text: gettext('Edit'),
- disabled: true,
- selModel: sm,
- enableFn: editable,
- handler: runEditor,
- }),
- Ext.create('Proxmox.button.StdRemoveButton', {
- selModel: sm,
- enableFn: editable,
- baseurl: me.baseurl,
- getRecordName: me.getRecordName,
- callback: () => me.reload(),
- }),
+ ...editButtons,
{
text: gettext('Reload'),
handler: () => me.reload(),
},
...(me.tbar ?? []),
],
- listeners: {
- itemdblclick: runEditor,
- },
+ listeners: me.readOnly ? {} : { itemdblclick: runEditor },
});
me.callParent();
diff --git a/www/manager6/sdn/microseg/GuestPanel.js b/www/manager6/sdn/microseg/GuestPanel.js
new file mode 100644
index 000000000000..657fa4d1fbc7
--- /dev/null
+++ b/www/manager6/sdn/microseg/GuestPanel.js
@@ -0,0 +1,117 @@
+Ext.define('PVE.sdn.microseg.GuestPanel', {
+ extend: 'Ext.panel.Panel',
+ alias: 'widget.pveSDNMicrosegGuest',
+
+ onlineHelp: 'pvesdn_config_microseg',
+
+ layout: 'border',
+
+ controller: {
+ xclass: 'Ext.app.ViewController',
+
+ reload: function () {
+ let me = this;
+ let view = me.getView();
+ let membership = PVE.sdn.microseg.Membership;
+ membership.load(view, (parts) => {
+ let hierarchy = membership.hierarchy(parts.groups);
+ let rulesFor = membership.rulesFor(parts.groupSets);
+ let nicGroupSet = membership.nicGroupSets(parts.assignments, hierarchy, rulesFor);
+ let prefix = `${view.vmid}/`;
+ let nics = Object.entries(nicGroupSet)
+ .filter(([key]) => key.startsWith(prefix))
+ .map(([key, groupSet]) => ({
+ nic: Number(key.slice(prefix.length)),
+ groups: groupSet.groups,
+ matcher: groupSet.note,
+ comment: groupSet.assignment.comment,
+ state: groupSet.assignment.state,
+ }))
+ .sort((a, b) => a.nic - b.nic);
+ me.lookup('nics').getStore().loadData(nics);
+
+ let rulesView = me.lookup('rules');
+ rulesView.setData(parts.rules);
+ rulesView.setTreeSelection(
+ membership.guestSelection(view.vmid, nicGroupSet, hierarchy, rulesFor),
+ );
+ });
+ },
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ if (!me.vmid) {
+ throw 'no vmid given';
+ }
+
+ me.items = [
+ {
+ xtype: 'grid',
+ reference: 'nics',
+ region: 'north',
+ height: 150,
+ split: true,
+ border: false,
+ store: {
+ fields: ['nic', 'groups', 'matcher', 'comment', 'state'],
+ sorters: [{ property: 'nic', direction: 'ASC' }],
+ },
+ viewConfig: {
+ getRowClass: PVE.Utils.sdnmicroseg_row_cls,
+ },
+ emptyText: gettext('No assignment covers this guest'),
+ tbar: [
+ {
+ text: gettext('Reload'),
+ handler: 'reload',
+ },
+ ],
+ columns: [
+ {
+ text: gettext('Interface'),
+ dataIndex: 'nic',
+ width: 100,
+ renderer: (v) => `net${v}`,
+ },
+ {
+ text: gettext('Groups'),
+ dataIndex: 'groups',
+ flex: 2,
+ renderer: (v) => (v || []).map(PVE.Utils.render_sdnmicroseg_group).join(''),
+ },
+ {
+ text: gettext('Assignment'),
+ dataIndex: 'matcher',
+ flex: 1,
+ renderer: Ext.String.htmlEncode,
+ },
+ { xtype: 'pveSDNMicrosegStateColumn' },
+ {
+ text: gettext('Comment'),
+ dataIndex: 'comment',
+ flex: 1,
+ renderer: Ext.String.htmlEncode,
+ },
+ ],
+ },
+ {
+ xtype: 'pveSDNMicrosegRules',
+ reference: 'rules',
+ region: 'center',
+ readOnly: true,
+ groupByDirection: true,
+ listeners: {
+ microsegreload: 'reload',
+ },
+ },
+ ];
+
+ me.callParent();
+ },
+
+ listeners: {
+ activate: 'reload',
+ },
+});
diff --git a/www/manager6/sdn/microseg/RuleView.js b/www/manager6/sdn/microseg/RuleView.js
index 95c08343e725..8c982a7bc5aa 100644
--- a/www/manager6/sdn/microseg/RuleView.js
+++ b/www/manager6/sdn/microseg/RuleView.js
@@ -38,13 +38,15 @@ Ext.define(
let matched = sel
? [...new Set([...src, ...dst])].filter((g) => sel.groups.has(g))
: [];
+ let direction = out && inbound ? 'both' : out ? 'out' : inbound ? 'in' : '';
return {
rank: me.rankOf(data),
specificity: data.specificity ?? 0,
matched: matched
.sort()
.map((g) => (sel.ownGroups.has(g) ? g : `${g} (${gettext('inherited')})`)),
- direction: out && inbound ? 'both' : out ? 'out' : inbound ? 'in' : '',
+ direction: direction,
+ directionRank: { in: 0, out: 1, both: 2 }[direction] ?? 3,
};
},
@@ -95,13 +97,32 @@ Ext.define(
initComponent: function () {
let me = this;
+ if (me.groupByDirection) {
+ let directionNames = {
+ in: gettext('Inbound'),
+ out: gettext('Outbound'),
+ both: gettext('Both'),
+ };
+ me.features = [
+ {
+ ftype: 'grouping',
+ groupHeaderTpl: [
+ '{groupValue:this.directionName}',
+ {
+ directionName: (name) =>
+ Ext.String.htmlEncode(directionNames[name] ?? name),
+ },
+ ],
+ enableGroupingMenu: false,
+ },
+ ];
+ }
+
let store = Ext.create('Ext.data.Store', {
model: 'pve-sdn-microseg-rule',
- grouper: {
- property: 'prio',
- sortProperty: 'prio',
- direction: 'DESC',
- },
+ grouper: me.groupByDirection
+ ? { property: 'direction', sortProperty: 'directionRank', direction: 'ASC' }
+ : { property: 'prio', sortProperty: 'prio', direction: 'DESC' },
sorters: [
{ property: 'rank', direction: 'ASC' },
{ property: 'id', direction: 'ASC' },
@@ -234,6 +255,12 @@ Ext.define(
flex: 1,
renderer: (v, m, rec) => Ext.String.htmlEncode(matchText(rec)),
},
+ {
+ text: gettext('Priority'),
+ dataIndex: 'prio',
+ width: 80,
+ hidden: !me.groupByDirection,
+ },
{
text: gettext('Specificity'),
dataIndex: 'specificity',
@@ -301,6 +328,7 @@ Ext.define(
'macro',
'fragments',
'direction',
+ 'directionRank',
'rank',
'specificity',
'matched',
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH qemu-server v3 42/45] network: update the microseg enforcement on a tag or name change
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (40 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-manager v3 41/45] ui: sdn: microseg: add a guest tab with its groups and rules Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-container v3 43/45] lxc: update the microseg enforcement on a tag or hostname change Hannes Laimer
` (2 subsequent siblings)
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
The groups of a VM's NICs depend on its tags and name, but a change of
those took effect at the next start of the VM or the next network
reload of the node. Update the NICs of a running VM once the new
config is written, so the change applies right away. The update reads
the config under the same lock as a network reload, so a reload
running at the same time cannot leave the old groups in place. A
failed update only warns, and the next reload or start catches up.
Moving a VM between groups is an SDN change like putting a NIC on a
vnet, which needs SDN.Use on the vnet. So a tag or name change that
can move the VM also needs SDN.Use on /sdn/microseg, and so does
creating a VM with such a tag or name.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/Qemu.pm | 26 ++++++++++++++++++++++++++
src/PVE/QemuServer.pm | 12 +++++++++++-
src/PVE/QemuServer/Network.pm | 14 ++++++++++++++
3 files changed, 51 insertions(+), 1 deletion(-)
diff --git a/src/PVE/API2/Qemu.pm b/src/PVE/API2/Qemu.pm
index f4dbd2a13396..bd1046784d66 100644
--- a/src/PVE/API2/Qemu.pm
+++ b/src/PVE/API2/Qemu.pm
@@ -26,6 +26,7 @@ use PVE::JSONSchema qw(get_standard_option);
use PVE::RESTHandler;
use PVE::ReplicationConfig;
use PVE::GuestHelpers qw(assert_tag_permissions);
+use PVE::Network::SDN::Microseg;
use PVE::GuestImport;
use PVE::QemuConfig;
use PVE::QemuServer;
@@ -1544,6 +1545,10 @@ __PACKAGE__->register_method({
my $vga = PVE::QemuServer::parse_vga($conf->{vga});
PVE::QemuServer::assert_clipboard_config($vga);
+ PVE::Network::SDN::Microseg::check_name_access(
+ $rpcenv, $authuser, undef, $conf->{name},
+ ) if defined($conf->{name});
+
PVE::QemuServer::Drive::warn_about_virtio_win_issues_in_config($conf);
# auto generate uuid if user did not specify smbios1 option
@@ -1571,6 +1576,9 @@ __PACKAGE__->register_method({
$vmid, {}, $conf->{tags}, $rpcenv, $authuser,
);
$conf->{tags} = PVE::GuestHelpers::get_unique_tags($conf->{tags});
+ PVE::Network::SDN::Microseg::check_tag_access(
+ $rpcenv, $authuser, '', $conf->{tags},
+ );
}
PVE::QemuConfig->write_config($vmid, $conf);
@@ -2409,8 +2417,11 @@ my $update_vm_api = sub {
PVE::QemuConfig->write_config($vmid, $conf);
} elsif ($opt eq 'tags') {
assert_tag_permissions($vmid, $val, '', $rpcenv, $authuser);
+ PVE::Network::SDN::Microseg::check_tag_access($rpcenv, $authuser, $val, '');
delete $conf->{$opt};
PVE::QemuConfig->write_config($vmid, $conf);
+ PVE::QemuServer::Network::update_nets_enforcement($conf, $vmid)
+ if PVE::QemuServer::check_running($vmid);
} elsif ($opt =~ m/^net\d+$/) {
if ($conf->{$opt}) {
PVE::QemuServer::check_bridge_access(
@@ -2421,6 +2432,10 @@ my $update_vm_api = sub {
}
PVE::QemuConfig->add_to_pending_delete($conf, $opt, $force);
PVE::QemuConfig->write_config($vmid, $conf);
+ } elsif ($opt eq 'name') {
+ PVE::Network::SDN::Microseg::check_name_access($rpcenv, $authuser, $val, undef);
+ PVE::QemuConfig->add_to_pending_delete($conf, $opt, $force);
+ PVE::QemuConfig->write_config($vmid, $conf);
} else {
PVE::QemuConfig->add_to_pending_delete($conf, $opt, $force);
PVE::QemuConfig->write_config($vmid, $conf);
@@ -2538,6 +2553,12 @@ my $update_vm_api = sub {
);
$conf->{pending}->{$opt} =
PVE::GuestHelpers::get_unique_tags($param->{$opt});
+ PVE::Network::SDN::Microseg::check_tag_access(
+ $rpcenv,
+ $authuser,
+ $conf->{$opt},
+ $conf->{pending}->{$opt},
+ );
} elsif ($opt =~ m/^net\d+$/) {
if ($conf->{$opt}) {
PVE::QemuServer::check_bridge_access(
@@ -2578,6 +2599,11 @@ my $update_vm_api = sub {
if $param->{cipassword} !~ /^\$(?:[156]|2[ay])(\$.+){2}/;
}
$conf->{cipassword} = $param->{cipassword};
+ } elsif ($opt eq 'name') {
+ PVE::Network::SDN::Microseg::check_name_access(
+ $rpcenv, $authuser, $conf->{$opt}, $param->{$opt},
+ );
+ $conf->{pending}->{$opt} = $param->{$opt};
} else {
$conf->{pending}->{$opt} = $param->{$opt};
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 1efeeb66e0e3..8595131e9e6b 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -4412,6 +4412,11 @@ sub foreach_volid {
}
}
+my $microseg_option = {
+ 'name' => 1,
+ 'tags' => 1,
+};
+
my $fast_plug_option = {
'description' => 1,
'hookscript' => 1,
@@ -4524,12 +4529,14 @@ sub vmconfig_hotplug_pending {
};
my $changes = 0;
+ my $microseg_relevant = 0;
foreach my $opt (keys %{ $conf->{pending} }) { # add/change
if ($fast_plug_option->{$opt}) {
my $new = delete $conf->{pending}->{$opt};
$cloudinit_record_changed->($conf, $opt, $conf->{$opt}, $new);
$conf->{$opt} = $new;
$changes = 1;
+ $microseg_relevant = 1 if $microseg_option->{$opt};
}
}
@@ -4589,7 +4596,7 @@ sub vmconfig_hotplug_pending {
my $balloon = get_current_memory($conf->{memory});
mon_cmd($vmid, "balloon", value => $balloon * 1024 * 1024);
} elsif ($fast_plug_option->{$opt}) {
- # do nothing
+ $microseg_relevant = 1 if $microseg_option->{$opt};
} elsif ($opt =~ m/^net(\d+)$/) {
die "skip\n" if !$hotplug_features->{network};
vm_deviceunplug($vmid, $conf, $opt);
@@ -4765,6 +4772,9 @@ sub vmconfig_hotplug_pending {
PVE::QemuConfig->write_config($vmid, $conf);
+ # reads the tags and name from pmxcfs, so after the write
+ PVE::QemuServer::Network::update_nets_enforcement($conf, $vmid) if $microseg_relevant;
+
if ($hotplug_features->{cloudinit} && PVE::QemuServer::Cloudinit::has_changes($conf)) {
PVE::QemuServer::vmconfig_update_cloudinit_drive($storecfg, $conf, $vmid);
}
diff --git a/src/PVE/QemuServer/Network.pm b/src/PVE/QemuServer/Network.pm
index b4893c015646..ebde9621fbf8 100644
--- a/src/PVE/QemuServer/Network.pm
+++ b/src/PVE/QemuServer/Network.pm
@@ -355,6 +355,20 @@ sub tap_plug {
PVE::Network::SDN::Zones::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate);
}
+# the groups of a NIC depend on the VM's tags and name
+sub update_nets_enforcement {
+ my ($conf, $vmid) = @_;
+
+ for my $opt (keys %$conf) {
+ next if $opt !~ m/^net(\d+)$/;
+ my $iface = "tap${vmid}i$1";
+ next if !-d "/sys/class/net/$iface";
+ my $net = parse_net($conf->{$opt}) or next;
+ my $firewall = $net->{firewall} && PVE::Firewall::Helpers::needs_fwbr($net->{bridge});
+ PVE::Network::SDN::Zones::tap_update($iface, $firewall);
+ }
+}
+
sub get_nets_host_mtu {
my ($vmid, $conf) = @_;
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-container v3 43/45] lxc: update the microseg enforcement on a tag or hostname change
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (41 preceding siblings ...)
2026-10-27 8:55 ` [PATCH qemu-server v3 42/45] network: update the microseg enforcement on a tag or name change Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-docs v3 44/45] sdn: zones: document the VXLAN-GBP option Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-docs v3 45/45] sdn: add microsegmentation section Hannes Laimer
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
The groups of a container's NICs depend on its tags and hostname, but
a change of those took effect at the next start of the container or
the next network reload of the node. Update the NICs of a running
container once the new config is written, so the change applies right
away. The update reads the config under the same lock as a network
reload, so a reload running at the same time cannot leave the old
groups in place. A failed update only warns, and the next reload or
start catches up.
Moving a container between groups is an SDN change like putting a NIC
on a vnet, which needs SDN.Use on the vnet. So a tag or hostname
change that can move the container also needs SDN.Use on
/sdn/microseg, and so does creating a container with such a tag or
hostname.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/LXC/Config.pm | 3 +++
src/PVE/LXC.pm | 28 ++++++++++++++++++++++++++++
2 files changed, 31 insertions(+)
diff --git a/src/PVE/API2/LXC/Config.pm b/src/PVE/API2/LXC/Config.pm
index 805eac0eac38..1ba8c5d865de 100644
--- a/src/PVE/API2/LXC/Config.pm
+++ b/src/PVE/API2/LXC/Config.pm
@@ -262,6 +262,9 @@ __PACKAGE__->register_method({
$conf = PVE::LXC::Config->load_config($vmid);
PVE::LXC::update_lxc_config($vmid, $conf);
+ # reads the tags and hostname from pmxcfs, so after the write
+ PVE::LXC::update_nets_enforcement($conf, $vmid)
+ if $running && grep { $_ eq 'tags' || $_ eq 'hostname' } (keys %$param, @delete);
raise_param_exc($errors) if scalar(keys %$errors);
};
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 44ae4fa69bef..0472d1b16bf5 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -25,6 +25,7 @@ use PVE::GuestHelpers qw(check_vnet_access safe_string_ne safe_num_ne safe_boole
use PVE::INotify;
use PVE::JSONSchema qw(get_standard_option);
use PVE::Network;
+use PVE::Network::SDN::Microseg;
use PVE::Network::SDN::Vnets;
use PVE::Network::SDN::Zones;
use PVE::ProcFSTools;
@@ -1122,6 +1123,21 @@ sub net_tap_plug : prototype($$) {
PVE::Tools::run_command(['/sbin/ip', 'link', 'set', 'dev', $iface, 'up']);
}
+# the groups of a NIC depend on the container's tags and hostname
+sub update_nets_enforcement {
+ my ($conf, $vmid) = @_;
+
+ for my $opt (keys %$conf) {
+ next if $opt !~ m/^net(\d+)$/;
+ my $iface = "veth${vmid}i$1";
+ next if !-d "/sys/class/net/$iface";
+ my $net = PVE::LXC::Config->parse_lxc_network($conf->{$opt});
+ next if ($net->{type} // '') ne 'veth';
+ my $firewall = $net->{firewall} && PVE::Firewall::Helpers::needs_fwbr($net->{bridge});
+ PVE::Network::SDN::Zones::tap_update($iface, $firewall);
+ }
+}
+
sub update_net {
my ($vmid, $conf, $opt, $newnet, $netid, $rootdir) = @_;
@@ -1691,6 +1707,12 @@ sub check_ct_modify_config_perm {
raise_perm_exc("configuring device passthrough is only allowed for root\@pam");
} elsif ($opt eq 'nameserver' || $opt eq 'searchdomain' || $opt eq 'hostname') {
$rpcenv->check_vm_perm($authuser, $vmid, $pool, ['VM.Config.Network']);
+ PVE::Network::SDN::Microseg::check_name_access(
+ $rpcenv,
+ $authuser,
+ $oldconf->{$opt},
+ $delete ? undef : $newconf->{$opt},
+ ) if $opt eq 'hostname';
} elsif ($opt eq 'features') {
raise_perm_exc(
"changing feature flags for privileged container is only allowed for root\@pam")
@@ -1744,6 +1766,12 @@ sub check_ct_modify_config_perm {
my $old = $oldconf->{$opt};
my $new = $delete ? '' : $newconf->{$opt};
PVE::GuestHelpers::assert_tag_permissions($vmid, $old, $new, $rpcenv, $authuser);
+ PVE::Network::SDN::Microseg::check_tag_access(
+ $rpcenv,
+ $authuser,
+ $old,
+ PVE::GuestHelpers::get_unique_tags($new),
+ );
} else {
$rpcenv->check_vm_perm($authuser, $vmid, $pool, ['VM.Config.Options']);
}
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-docs v3 44/45] sdn: zones: document the VXLAN-GBP option
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (42 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-container v3 43/45] lxc: update the microseg enforcement on a tag or hostname change Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
2026-10-27 8:55 ` [PATCH pve-docs v3 45/45] sdn: add microsegmentation section Hannes Laimer
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
EVPN zones run on VXLAN interfaces as well, so the EVPN section gets
the same entry as the VXLAN one.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
pvesdn.adoc | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/pvesdn.adoc b/pvesdn.adoc
index 3fd3533ad07f..7a674597d470 100644
--- a/pvesdn.adoc
+++ b/pvesdn.adoc
@@ -310,6 +310,18 @@ SDN Fabric:: Instead of manually defining all the peers, use a
MTU:: Because VXLAN encapsulation uses 50 bytes, the MTU needs to be 50 bytes
lower than the outgoing physical interface.
+VXLAN-GBP:: Enable VXLAN Group Based Policy (GBP) on the zone's VXLAN
+ interfaces. GBP carries a 16-bit group ID in the VXLAN header, taken from
+ the packet mark on the sending node and restored on the receiving one.
+ Every VTEP in the zone must have it enabled, since one without it drops
+ GBP-tagged traffic. All zones that use the same VXLAN port must agree on
+ this option, an SDN apply refuses a configuration where they do not.
+ Optional.
+
+NOTE: The kernel sets GBP only when it creates a VXLAN interface. Changing the
+option on an existing zone therefore recreates the zone's VXLAN interfaces on
+the next apply, which briefly interrupts traffic.
+
[[pvesdn_zone_plugin_evpn]]
EVPN Zones
@@ -371,6 +383,18 @@ MTU:: Because VXLAN encapsulation uses 50 bytes, the MTU needs to be 50 bytes
less than the maximal MTU of the outgoing physical interface. Optional,
defaults to 1450.
+VXLAN-GBP:: Enable VXLAN Group Based Policy (GBP) on the zone's VXLAN
+ interfaces. GBP carries a 16-bit group ID in the VXLAN header, taken from
+ the packet mark on the sending node and restored on the receiving one.
+ Every VTEP in the zone must have it enabled, since one without it drops
+ GBP-tagged traffic. All zones that use the same VXLAN port must agree on
+ this option, an SDN apply refuses a configuration where they do not.
+ Optional.
+
+NOTE: The kernel sets GBP only when it creates a VXLAN interface. Changing the
+option on an existing zone therefore recreates the zone's VXLAN interfaces on
+the next apply, which briefly interrupts traffic.
+
[[pvesdn_config_vnet]]
VNets
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread* [PATCH pve-docs v3 45/45] sdn: add microsegmentation section
2026-10-27 8:54 [PATCH access-control/cluster/container/docs/manager/network/proxmox{-ebpf,-ve-rs,-perl-rs}/qemu-server v3 00/45] sdn: add microsegmentation support Hannes Laimer
` (43 preceding siblings ...)
2026-10-27 8:55 ` [PATCH pve-docs v3 44/45] sdn: zones: document the VXLAN-GBP option Hannes Laimer
@ 2026-10-27 8:55 ` Hannes Laimer
44 siblings, 0 replies; 46+ messages in thread
From: Hannes Laimer @ 2026-10-27 8:55 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
pvesdn.adoc | 403 ++++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 391 insertions(+), 12 deletions(-)
diff --git a/pvesdn.adoc b/pvesdn.adoc
index 7a674597d470..84f44ab73592 100644
--- a/pvesdn.adoc
+++ b/pvesdn.adoc
@@ -147,6 +147,9 @@ following sections:
* xref:pvesdn_config_vnet[VNets]: Create virtual network bridges and
manage subnets
+* xref:pvesdn_config_microseg[Microsegmentation]: Segment traffic between
+ guests with group-based policies
+
The Options category allows adding and managing additional services to be used
in your SDN setup.
@@ -189,6 +192,83 @@ files located in '/etc/pve/sdn'.
// TODO: extend implementation and technology details.
+[[pvesdn_microseg_background]]
+Microsegmentation Enforcement
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The node enforces the xref:pvesdn_config_microseg[microsegmentation] policy in
+the kernel, at the network interface of the receiving guest. It checks each
+packet on its own, without connection tracking, and drops a denied packet
+without an answer. Nothing runs in the guest, so a guest cannot turn it off or
+get around it.
+
+Each set of groups that an assignment lists gets a 16-bit number, its group
+identity, and sets that every rule treats alike share one. The node marks every
+packet an assigned interface sends with the identity of that interface. When a
+packet reaches an assigned interface, the node checks the identity of the
+sender against the identity of the receiver. Addresses play no part in this.
+
+A guest cannot forge its identity, since the node sets it. The network between
+the nodes is trusted to carry the identity unchanged, as it is trusted with
+VLAN tags.
+
+On the node, the identity lives in the low 16 bits of the packet mark and stays
+there when the node routes a packet. Other uses of the mark, such as policy
+routing by fwmark or firewall mark rules, must leave these bits alone. A rule
+that matches on the mark masks them out, for example
+`fwmark 0x10000/0xffff0000`. A packet the node itself sends with a value in
+these bits counts as that identity.
+
+Between nodes, only VXLAN and EVPN zones with the *VXLAN-GBP* option carry the
+identity, in the VXLAN header (see xref:pvesdn_zone_plugin_vxlan[VXLAN Zones]
+and xref:pvesdn_zone_plugin_evpn[EVPN Zones]). On every other zone, traffic
+from another node arrives without an identity and counts as untagged. The
+policy is bound to the guest interface, not to the zone. An interface on a
+bridge outside the SDN, such as `vmbr0`, is enforced like one on a VNet.
+
+For each pair of a sending and a receiving identity that a rule allows, the
+node keeps a verdict and the checks of the narrowed rules. A narrowed rule
+needs one check per protocol and per combination of a source and a destination
+port range, so a destination port list of `80,443` needs two checks. There is
+room for 65535 identities, 1048576 pairs and 512 checks per pair. A pair with
+more than 7 checks keeps the rest in a list shared with the pairs that have the
+same ones, and there is room for 16384 such lists.
+
+An SDN apply refuses a policy over one of these limits and names the limit, so
+such a policy never reaches a node. An identity that is no longer needed stays
+reserved for an hour, and the overview counts it as retired.
+
+A guest starts on a node only if all of its assigned interfaces can be enforced
+there, and a migration fails on the target node where a start would fail. If a
+node cannot read the applied SDN configuration, no guest with a network
+interface starts on it, since the node cannot tell which interfaces are
+assigned.
+
+If a node cannot apply the policy during an SDN apply, its network reload task
+fails and the node keeps its previous enforcement. On a node without loaded
+programs, for example after a failed apply at boot, no guest with an assigned
+interface starts until a reload succeeds. If the node cannot load a new
+package's programs, the previous package's programs stay in place.
+
+NOTE: Nodes with and without microsegmentation support can run in the same
+cluster during an upgrade, but the SDN apply should only be run on the nodes
+that support it. An apply from a node without support leaves the assigned
+guests that start after it unenforced.
+
+The guest firewall still applies, a packet has to pass both. A packet that
+microsegmentation drops gets no answer, even if the firewall is set to reject.
+
+With the guest firewall enabled, an interface usually sits behind its own
+firewall bridge, only the nftables firewall on a Linux bridge needs none. Behind
+a firewall bridge, the node cannot tell its own ARP and neighbor discovery
+replies from those of other senders. Such an interface therefore accepts both
+from every sender without an identity. Other interfaces accept them without an
+identity only from the node itself, or where a rule allows untagged traffic.
+
+WARNING: Behind a firewall bridge, an unassigned guest on the same VNet can
+answer ARP and neighbor discovery for addresses that are not its own. Only that
+guest's own firewall, filtering its IP and MAC addresses, prevents it.
+
[[pvesdn_config_zone]]
Zones
-----
@@ -311,12 +391,13 @@ MTU:: Because VXLAN encapsulation uses 50 bytes, the MTU needs to be 50 bytes
lower than the outgoing physical interface.
VXLAN-GBP:: Enable VXLAN Group Based Policy (GBP) on the zone's VXLAN
- interfaces. GBP carries a 16-bit group ID in the VXLAN header, taken from
- the packet mark on the sending node and restored on the receiving one.
- Every VTEP in the zone must have it enabled, since one without it drops
- GBP-tagged traffic. All zones that use the same VXLAN port must agree on
- this option, an SDN apply refuses a configuration where they do not.
- Optional.
+ interfaces. xref:pvesdn_config_microseg[Microsegmentation] uses it to carry
+ an interface's group identity between nodes. GBP carries a 16-bit group ID
+ in the VXLAN header, taken from the packet mark on the sending node and
+ restored on the receiving one. Every VTEP in the zone must have it enabled,
+ since one without it drops GBP-tagged traffic. All zones that use the same
+ VXLAN port must agree on this option, an SDN apply refuses a configuration
+ where they do not. Optional.
NOTE: The kernel sets GBP only when it creates a VXLAN interface. Changing the
option on an existing zone therefore recreates the zone's VXLAN interfaces on
@@ -384,12 +465,13 @@ MTU:: Because VXLAN encapsulation uses 50 bytes, the MTU needs to be 50 bytes
defaults to 1450.
VXLAN-GBP:: Enable VXLAN Group Based Policy (GBP) on the zone's VXLAN
- interfaces. GBP carries a 16-bit group ID in the VXLAN header, taken from
- the packet mark on the sending node and restored on the receiving one.
- Every VTEP in the zone must have it enabled, since one without it drops
- GBP-tagged traffic. All zones that use the same VXLAN port must agree on
- this option, an SDN apply refuses a configuration where they do not.
- Optional.
+ interfaces. xref:pvesdn_config_microseg[Microsegmentation] uses it to carry
+ an interface's group identity between nodes. GBP carries a 16-bit group ID
+ in the VXLAN header, taken from the packet mark on the sending node and
+ restored on the receiving one. Every VTEP in the zone must have it enabled,
+ since one without it drops GBP-tagged traffic. All zones that use the same
+ VXLAN port must agree on this option, an SDN apply refuses a configuration
+ where they do not. Optional.
NOTE: The kernel sets GBP only when it creates a VXLAN interface. Changing the
option on an existing zone therefore recreates the zone's VXLAN interfaces on
@@ -467,6 +549,303 @@ DNS Zone Prefix:: Add a prefix to the domain registration, like
<hostname>.prefix.<domain> Optional.
+[[pvesdn_config_microseg]]
+Microsegmentation
+-----------------
+
+Microsegmentation allows or denies traffic between groups of guests at their
+network interfaces, independent of their addresses. The node enforces it in the
+kernel, so a guest cannot turn it off, and a guest keeps its rules when it
+moves to another node.
+
+The policy consists of three kinds of objects. *Groups* are labels.
+*Assignments* put guest network interfaces into groups, by guest, by tag or by
+name. *Rules* allow or deny traffic from one set of groups to another. An
+interface that an assignment covers receives only what a rule allows.
+
+Microsegmentation is managed under `Datacenter -> SDN -> Microsegmentation`.
+The *Microsegmentation* tab of a guest shows the groups of its interfaces and
+the rules that apply to them. Changes to the policy take effect with the next
+SDN apply.
+
+Rules are stateless and apply to one direction, so a connection needs a rule
+for each direction, or one rule with *Both Directions* set. Traffic that does
+not come from a guest in a group, such as the replies of a gateway or a DHCP
+server, is untagged. It needs a rule with the built-in `untagged` group as the
+source.
+
+Between nodes, only VXLAN and EVPN zones with the *VXLAN-GBP* option carry the
+groups of a guest. On every other zone, traffic from another node arrives
+untagged. The guest firewall still applies, a packet has to pass both. See
+xref:pvesdn_microseg_background[Microsegmentation Enforcement] for how the
+enforcement works and for its limits.
+
+Viewing the configuration needs `SDN.Audit` on the `/sdn/microseg` ACL path,
+changing it needs `SDN.Allocate` there. The guests an assignment covers are
+shown only where you have `VM.Audit` on the guest, as in the resource tree.
+
+[[pvesdn_microseg_group]]
+Groups
+~~~~~~
+
+A group is a label for guest interfaces. An interface gets its groups from the
+one assignment that covers it (see
+xref:pvesdn_microseg_assignment[Assignments]). Rules match the groups of an
+interface, never the interface itself.
+
+Groups can be nested. An interface in a subgroup is also in every group above
+it. With `database` a subgroup of `production`, an interface assigned
+`database` has both groups. An `any` or `all` match over `production` covers
+it, an `exact` match over `production` does not, and rules on `database` cover
+databases only.
+
+The groups that an `all` or `exact` match lists include the groups above them,
+so an `exact` match over `database` means `production` and `database`. A rule
+with such a match on a subgroup is therefore more specific than the same rule
+on the parent. Specificity decides a tie between rules of the same priority
+(see xref:pvesdn_microseg_rule[Rules]). An `any` match counts no groups.
+
+Group configuration options:
+
+Name:: 2 to 32 letters, digits, `-` and `_`, starting and ending with a letter
+ or digit. The name `untagged` and names shaped like a generated ID, such as
+ `vm100`, are reserved.
+
+Parent:: The group this one is a subgroup of. Optional.
+
+Comment:: A descriptive comment. Optional.
+
+NOTE: A group cannot be deleted while a rule names it, an assignment lists it,
+or it has subgroups.
+
+[[pvesdn_microseg_rule]]
+Rules
+~~~~~
+
+A rule allows or denies traffic from a source to a destination. Each side lists
+groups and a match kind, checked against the groups of the sending and of the
+receiving interface. The default is deny, so rules allow traffic, or deny part
+of what a broader rule allows. Traffic within one group needs a rule like any
+other.
+
+A rule applies to one direction. A TCP connection from `web` to `app` needs an
+allow from `web` to `app`, and one from `app` to `web` for the replies. A rule
+with *Both Directions* set is exactly that pair, the rule and its mirror with
+source and destination swapped.
+
+Two guests can only talk once they have resolved each other's addresses. ARP
+and the neighbor solicitations and advertisements of IPv6 therefore pass
+between a sender and a receiver in both directions as soon as a rule allows
+any traffic between them in one direction.
+
+They also pass when the node sends them itself. That is how the ARP replies of
+a gateway on the node and the suppression replies of an EVPN zone reach a
+guest. Nothing else passes without a rule.
+
+A rule can be narrowed to a protocol, ports or an ICMP type, like a firewall
+rule. A narrowed rule decides only the IP traffic it names. Anything that is
+not IP, ARP, LLDP or spanning tree for example, gets the verdict of the rules
+without narrowing.
+
+Narrowed rules are checked from the highest priority down to the priority of
+the highest rule without narrowing. At the same priority the more specific
+rule comes first, and of two equally specific rules the deny. The first
+narrowed rule that matches a packet decides it. Traffic that no narrowed rule
+matches gets the verdict of the rules without narrowing, or is denied if there
+are none.
+
+To allow SSH from `web` to `app`, allow `tcp` with destination port `22` from
+`web` to `app`, and `tcp` with source port `22` from `app` to `web` for the
+replies. One rule with *Both Directions* set and destination port `22` does the
+same, since the mirror swaps the ports. Either way the reply rule allows every
+segment from port `22`, not only replies, so `app` can also open connections to
+`web` from that port.
+
+NOTE: Neighbor discovery is ICMPv6, so a narrowed deny of ICMPv6 blocks it even
+where the reverse direction allows traffic.
+
+IP fragments after the first one carry no transport header. A rule narrowed by
+ports or an ICMP type never matches them, and neither does a rule narrowed to
+just a protocol. If only narrowed allows let traffic pass between a sender and a
+receiver, the later fragments are dropped and fragmented packets never arrive.
+
+An allow with *Fragments Only* set lets them pass. It also passes the later
+fragments of other datagrams, but those cannot be reassembled without their
+first fragment, which the other rules decide.
+
+An IPv6 fragment counts as the protocol its fragment header names. If that is
+an extension header instead of the transport protocol, a rule narrowed to the
+transport protocol does not match it, but a rule without a protocol does.
+
+Where narrowed rules apply to a sender and a receiver, a packet whose transport
+header cannot be read is dropped, since the rules cannot be checked against it.
+Examples are a first fragment that ends before the transport header, an IPv6
+packet with more than 12 extension headers, and a packet with more than two
+VLAN tags. Where no narrowed rule applies, such a packet gets the same verdict
+as any other.
+
+The built-in `untagged` group stands for traffic without a group. It comes from
+an interface that no assignment covers, from a host outside the policy, or from
+another node over a zone without the *VXLAN-GBP* option. To allow such traffic,
+list `untagged` in the source of a rule, on its own or next to other groups
+with the `any` match. This is how the replies of a gateway, of a DHCP server or
+of any other host outside the policy reach a guest.
+
+NOTE: In a source with the `all` or `exact` match, `untagged` must be the only
+listed group, and an `every` source never includes it. A destination never
+matches `untagged`, since every enforced interface has a group. It can list
+`untagged` only next to other groups with the `any` match, where it has no
+effect. Assignments cannot use it.
+
+Rule configuration options:
+
+Source Groups/Dest. Groups:: The groups the sending or the receiving interface
+ is checked against. Not set with the `every` match.
+
+Source Match/Dest. Match:: How the listed groups compare with the groups of an
+ interface. `any` matches an interface that has at least one of them, `all`
+ one that has all of them, and `exact` one that has exactly the listed
+ groups. For `all` and `exact`, the listed groups include the groups above
+ them. `every` lists no groups and matches every assigned interface.
+ Optional, defaults to `any`.
+
+Priority:: A number from 0 to 65535. Among the rules without narrowing that
+ match a sender and a receiver, the highest priority decides, and among those
+ the most specific. The specificity of a rule is the number of groups its
+ `all` and `exact` matches list, the groups above them included, and an `any`
+ or `every` match counts none. Equally specific rules deny if one of them
+ denies. Optional, defaults to 0.
+
+Action:: Allow or deny.
+
+Protocol/Source port/Dest. port/ICMP type:: Narrow the rule, with the syntax of
+ firewall rules. Ports need a protocol that has ports, and an ICMP type needs
+ an ICMP protocol. Optional.
+
+Macro:: A firewall macro, for example `HTTPS` or `Ceph`, in place of the
+ protocol, ports and ICMP type. Not combined with them. Optional.
+
+Fragments Only:: Narrow the rule to the IP fragments after the first one, of
+ the given protocol or of every protocol. Not combined with ports, an ICMP
+ type or a macro. Optional.
+
+Both Directions:: Also apply the rule with source and destination swapped,
+ ports included. Not with the `untagged` group or an ICMP type, whether set
+ directly or through a macro such as `Ping`. Optional.
+
+Comment:: A descriptive comment. Optional.
+
+The source, the destination, the direction and the narrowing identify a rule,
+so they cannot be changed on an existing rule. Delete the rule and create it
+again instead. The priority, the action and the comment can be changed.
+
+[[pvesdn_microseg_assignment]]
+Assignments
+~~~~~~~~~~~
+
+An assignment puts guest network interfaces into a set of groups. Its matcher
+selects one guest, the guests with certain tags, or the guests whose name
+matches a regular expression. You choose the matcher when you add the
+assignment, with *Guest Assignment*, *Tag Assignment* or *Regex Assignment* in
+the *Add* menu.
+
+Assignments are an ordered list, checked from first to last. An interface gets
+exactly the groups of the first assignment that covers it, and later ones have
+no effect on it. A new assignment goes to the top unless you give it a
+position, and you can move an existing one. Place an assignment for a single
+guest above the tag or regex assignments that also cover that guest.
+
+An interface that no assignment covers is not enforced. All traffic reaches
+it, and its own traffic reaches enforced guests as untagged.
+
+A combination of groups needs one assignment that lists all of them, since an
+interface gets the groups of a single assignment. For guests tagged both `prod`
+and `web`, add a tag assignment with the `all` match over both tags that lists
+both groups. Place it above the assignments that match only one of the tags.
+
+Tag and regex matchers use the current tags and name of a guest. They are
+matched each time an interface of the guest comes up, so a new guest or a
+hot-plugged interface is enforced from the start. A change of a running guest's
+tags or name made with `qm set` or `pct set`, through the API or the web
+interface, takes effect immediately.
+
+Changes to the assignments themselves, their order included, are pending until
+the next SDN apply.
+
+NOTE: Moving an assignment usually changes only its own place, so only that
+assignment shows up as a pending change. After many moves to the same spot, all
+assignments are renumbered once and show up together.
+
+A tag or a name can put a guest into a group. Setting one that an assignment
+matches, at creation or later, therefore needs `SDN.Use` on `/sdn/microseg`
+besides the right to change the option. This applies to a tag that a tag
+assignment matches, and to a rename into or out of the names a regex
+assignment matches.
+
+Assignment configuration options:
+
+Guest:: (guest matcher) The VM or container.
+
+Tags:: (tag matcher) The guest tags to match. They are normalized like guest
+ tags, so their case matters only with a case-sensitive datacenter tag style.
+
+Tag Match:: (tag matcher) `any` matches a guest with at least one of the
+ listed tags, `all` one with all of them, and `exact` one with exactly these
+ tags. Optional, defaults to `any`.
+
+Name Regex:: (regex matcher) A regular expression in Rust syntax, ASCII only,
+ without backreferences or lookaround. It is matched anywhere in the VM name
+ or the container hostname, so anchor it with `^` and `$` for the whole name.
+ A VM without a name is never matched, not even by `.*`, since the `VM 100`
+ shown for it is not its name. A container created without a hostname gets
+ one like `CT100`, which is matched like any other name.
+
+Network Interface:: The number of a guest interface, from 0 to 31, `0` for
+ `net0`. Left empty, the assignment covers all interfaces of the guest.
+ Optional.
+
+Groups:: The groups that the covered interfaces get.
+
+Position:: The place in the list, 0 being the first. A position past the end
+ puts the assignment last. Left empty on an existing assignment, it keeps its
+ place. Optional, defaults to 0 for a new assignment.
+
+Comment:: A descriptive comment. Optional.
+
+The matcher fields and the network interface identify an assignment, so they
+cannot be changed on an existing one. Delete the assignment and create it
+again instead. The groups, the position and the comment can be changed.
+
+[[pvesdn_microseg_example]]
+Example
+~~~~~~~
+
+Web, application and database guests share one VNet and are tagged `web`,
+`app` and `db`. The web guests should reach the application guests, and those
+the databases, and nothing else.
+
+. Create the groups `web`, `app` and `db`.
+. For each tag, add a tag assignment that puts the guests with that tag into
+ the group of the same name.
+. Add a rule that allows `web` to `app` with *Both Directions* set, and one
+ that allows `app` to `db` the same way.
+. Add a rule that allows `untagged` to `web`, `app` and `db`, for the replies
+ of the gateway and the DHCP server.
+. Apply the SDN configuration.
+
+Web guests now reach the application guests, and those reach the databases.
+Traffic from web to database and traffic within a tier is dropped.
+
+NOTE: The rule for `untagged` also admits every guest on the VNet that no
+assignment covers. Either assign every guest, or narrow that rule to the
+protocols the outside traffic needs.
+
+To keep a staging environment apart from production, tag each guest with its
+environment as well and add the groups `prod` and `staging`. Then add one tag
+assignment per combination, such as `all` over `prod` and `web`, that lists
+both groups. In the rules, use the `all` match over the tier and the
+environment, for example `all` over `web` and `prod`.
+
[[pvesdn_config_controllers]]
Controllers
-----------
--
2.47.3
^ permalink raw reply related [flat|nested] 46+ messages in thread