From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id B9E941FF0AA for ; Tue, 22 Sep 2026 12:10:00 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id B3F77215AB; Tue, 22 Sep 2026 12:09:57 +0200 (CEST) From: Hannes Laimer To: pve-devel@lists.proxmox.com Subject: [PATCH access-control/cluster/docs/manager/network/proxmox{-ve-rs,-firewall,-perl-rs} v3 00/38] sdn: add microsegmentation support Date: Tue, 22 Sep 2026 12:09:04 +0200 Message-ID: <20260922100943.1151484-1-h.laimer@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1790071784733 X-SPAM-LEVEL: Spam detection results: 1 AWL -2.177 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_CSS_A 0.1 Contains URL's A record listed in the Spamhaus CSS blocklist [23.95.107.30] URIBL_DBL_SPAM 5 Contains a spam URL listed in the Spamhaus DBL blocklist [sdn.pm] Message-ID-Hash: LFBFJZCK3N6OMXMPPCMQ6QJNGRWKVQRC X-Message-ID-Hash: LFBFJZCK3N6OMXMPPCMQ6QJNGRWKVQRC X-MailFrom: h.laimer@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Guests on a vnet are separated only by IP addressing and the guest firewall. Its address rules have to follow every move and renumbering. No identity crosses nodes with a frame, and tags and names carry no policy. This series adds a group-based policy with a datacenter panel over /etc/pve/sdn/microseg.cfg and /cluster/sdn/microseg. A rule has a priority and a verdict and matches a source and a destination predicate over groups, any, all or exact. It can be narrowed to a protocol and ports, an ICMP type or a firewall macro. Assignments bind guest interfaces to groups by guest, tag or name regex, and a subgroup member is in every group above it. VXLAN and EVPN zones gain a vxlan-gbp option that carries the identity across nodes. The host stamps it where a frame enters the bridge, so a guest cannot forge its own. Default is deny. Rules are stateless and directional, so a reply needs its own allow, or the rule is marked both ways, which is exactly that pair with the ports swapped. The highest priority wins, at a tie the more specific all and exact predicates decide, and deny wins among equals. Identities stay stable across config changes, and a retired one is reused only after a quarantine. A full apply swaps a node's policy in one transaction, and a guest interface is enforced before it is bridged. A running config a node cannot read fails every guest start there, one it cannot parse or build fails the start of guests with assigned interfaces, as does an interface headed for an Open vSwitch port without a firewall bridge. One already on such a port is named in the node's network reload task and stays unenforced until the firewall option is set. A guest matched after the last apply stays unmanaged until the next one. proxmox-ve-config carries the types, the identity engine and the desired state, the policy in a form no enforcing mechanism depends on. pve-cluster adds the observed file, pve-access-control the ACL path. pve-network holds the config, the API, the zone option and the plug trigger. proxmox-microseg, a library crate in the proxmox-firewall workspace, renders the desired state into a bridge family nftables table, and pve-rs binds it next to the config, so Perl applies it on an SDN apply and on a NIC plug. pve-manager adds the panel and applies on network reload. pve-docs adds the section. Connection tracking, rule logging, the SRv6 carrier and an eBPF plane are left out on purpose. libpve-network-perl loads only with the pve-cluster of this series and needs its libpve-rs-perl, or every guest start on the node fails at the plug. It gains dependencies on nftables and libpve-guest-common-perl. The API package needs the libpve-access-control of this series, or no ACL can be granted on /sdn/microseg and its endpoints only see what /sdn passes down. pve-manager builds only with the pve-docs of this series, whose anchors its help buttons name, and at runtime needs the new libpve-network-perl, since its network reload applies the microseg policy through it. pve-rs builds against the proxmox-microseg crate, which pulls in proxmox-nftables. pve-rs and the crate both need a proxmox-ve-config with the new module, and so does the proxmox-firewall source package that carries the crate. None of these floors is bumped here. The ifupdown2 vxlan-gbp attribute and four proxmox-nftables patches (building without the config extension, mark-keyed verdict maps, shared protocol rendering, crate packaging) are sent separately. # changelog v3: - data plane moved from eBPF programs to an nftables bridge-family table, proxmox-ebpf is gone and the data plane is a library crate in the proxmox-firewall workspace that pve-rs binds, without an agent binary or a boot-time unit since the first plug builds the table, the desired state it renders lives in proxmox-ve-config, and libpve-network-perl depends on nftables - rules can be narrowed to a protocol, ports, an ICMP type or a firewall macro, the protocol and port rendering is shared with the firewall - a rule can be marked both ways, which applies it with the predicates and the ports swapped as well, under one id, so the reply rule of a stateless pair no longer has to be written by hand - groups can form a hierarchy, membership in a subgroup implies the groups above it, and for all and exact predicates specificity breaks priority ties so a subgroup rule outranks its parent's - a plug fails when the running config cannot be read, or cannot be parsed or built and the NIC is assigned, or an assigned NIC would become an Open vSwitch port with no firewall bridge, an unreachable pmxcfs fails the plug instead of reading as an empty config, and a realized set without an identity fails the apply instead of passing as untagged - the microseg endpoints check their privileges on a new /sdn/microseg ACL path, added by a pve-access-control patch in the series - the overview endpoint reports identity id space usage, shown in the tree header - the pve-cluster patch also registers the file with the version array pmxcfs keeps for observed files - validation rejects mark 0 for a group, `untagged` in an all or exact destination, and `untagged` next to real groups in an all or exact source - an assignment naming an interface index is realized whether or not the guest has that interface yet, and tag matcher tags are normalized like guest tags - the running config stores the realized assignments per group set with per-guest interface lists, so the file stays small at scale - the network reload warns about an assigned interface that is already up on an Open vSwitch port without a firewall bridge, since only a plug can refuse one - two-column rule editor reusing the firewall's selectors, the tree shows subgroups - the SRv6 carrier is out of the series, it belongs with an SRv6 zone - the ifupdown2 `vxlan-gbp` attribute and the proxmox-nftables additions with the crate packaging are sent as standalone patches, the zone option stays in the series and the evpn vxlan-learning fix is folded into it - docs updated, cover letter rewritten v2: - core model reworked: each interface now holds a *set* of groups compiled into a per-NIC wire identity, replacing the single-group parent/child tree - rules gained explicit `src`/`dst` predicates (any/all/exact) and a `prio`, replacing the implicit tree-distance ordering - `untagged` is now a nameable built-in group (mark 0) - assignments are additive: a NIC's set is the union of all that resolve to it - added tag-matcher and name-regex assignments - rule and dynamic-assignment ids are now generated - identity allocation is fresh-first with quarantined reclamation, so churn cannot exhaust the id space - split the SRv6 carrier support into droppable tail patches - updated docs - updated cover-letter proxmox-ve-rs: Hannes Laimer (8): ve-config: firewall: add a constructor for sctp matches ve-config: sdn: add microseg signature-identity engine ve-config: sdn: add microseg config types ve-config: sdn: microseg: add tag matcher ve-config: sdn: microseg: add name regex matcher ve-config: sdn: microseg: narrow rules by protocol and ports ve-config: sdn: microseg: add group hierarchy ve-config: sdn: microseg: add the desired state .../src/firewall/types/rule_match.rs | 4 + .../src/sdn/microseg/identity.rs | 625 +++ proxmox-ve-config/src/sdn/microseg/mod.rs | 3819 +++++++++++++++++ proxmox-ve-config/src/sdn/microseg/state.rs | 316 ++ proxmox-ve-config/src/sdn/mod.rs | 1 + 5 files changed, 4765 insertions(+) create mode 100644 proxmox-ve-config/src/sdn/microseg/identity.rs create mode 100644 proxmox-ve-config/src/sdn/microseg/mod.rs create mode 100644 proxmox-ve-config/src/sdn/microseg/state.rs proxmox-firewall: Hannes Laimer (5): nftables: build without the config extension nftables: add support for mark-keyed verdict maps nftables: move protocol match rendering into the lib nftables: package the crate for debcargo microseg: add the SDN microsegmentation data plane .gitignore | 4 + Cargo.toml | 4 + Makefile | 12 +- build.sh | 46 ++ proxmox-firewall/src/rule.rs | 166 +--- proxmox-microseg/Cargo.toml | 22 + proxmox-microseg/debian/changelog | 6 + proxmox-microseg/debian/control | 46 ++ proxmox-microseg/debian/copyright | 18 + proxmox-microseg/debian/debcargo.toml | 7 + proxmox-microseg/debian/source/format | 1 + proxmox-microseg/src/dataplane.rs | 12 + proxmox-microseg/src/lib.rs | 41 + proxmox-microseg/src/nft.rs | 458 +++++++++++ proxmox-microseg/src/nic.rs | 54 ++ proxmox-microseg/src/running_config.rs | 28 + ..._tests__empty_state_removes_the_table.snap | 24 + ..._tests__full_apply_replaces_the_table.snap | 726 ++++++++++++++++++ ...t__tests__nic_apply_adds_its_elements.snap | 46 ++ proxmox-nftables/Cargo.toml | 7 +- proxmox-nftables/debian/changelog | 6 + proxmox-nftables/debian/control | 60 ++ proxmox-nftables/debian/copyright | 18 + proxmox-nftables/debian/debcargo.toml | 7 + proxmox-nftables/debian/source/format | 1 + proxmox-nftables/src/expression.rs | 3 +- proxmox-nftables/src/lib.rs | 2 + proxmox-nftables/src/protocol.rs | 70 ++ proxmox-nftables/src/statement.rs | 10 + proxmox-nftables/src/types.rs | 1 + 30 files changed, 1748 insertions(+), 158 deletions(-) create mode 100755 build.sh create mode 100644 proxmox-microseg/Cargo.toml create mode 100644 proxmox-microseg/debian/changelog create mode 100644 proxmox-microseg/debian/control create mode 100644 proxmox-microseg/debian/copyright create mode 100644 proxmox-microseg/debian/debcargo.toml create mode 100644 proxmox-microseg/debian/source/format create mode 100644 proxmox-microseg/src/dataplane.rs create mode 100644 proxmox-microseg/src/lib.rs create mode 100644 proxmox-microseg/src/nft.rs create mode 100644 proxmox-microseg/src/nic.rs create mode 100644 proxmox-microseg/src/running_config.rs create mode 100644 proxmox-microseg/src/snapshots/proxmox_microseg__nft__tests__empty_state_removes_the_table.snap create mode 100644 proxmox-microseg/src/snapshots/proxmox_microseg__nft__tests__full_apply_replaces_the_table.snap create mode 100644 proxmox-microseg/src/snapshots/proxmox_microseg__nft__tests__nic_apply_adds_its_elements.snap create mode 100644 proxmox-nftables/debian/changelog create mode 100644 proxmox-nftables/debian/control create mode 100644 proxmox-nftables/debian/copyright create mode 100644 proxmox-nftables/debian/debcargo.toml create mode 100644 proxmox-nftables/debian/source/format create mode 100644 proxmox-nftables/src/protocol.rs pve-cluster: Hannes Laimer (1): cfs: add 'sdn/microseg.cfg' to observed files src/PVE/Cluster.pm | 1 + src/pmxcfs/status.c | 1 + 2 files changed, 2 insertions(+) pve-access-control: Hannes Laimer (1): permissions: add ACL path for SDN microsegmentation src/PVE/AccessControl.pm | 1 + 1 file changed, 1 insertion(+) proxmox-perl-rs: Hannes Laimer (2): pve-rs: sdn: add microseg config binding pve-rs: sdn: microseg: bind the data plane pve-rs/Cargo.toml | 2 + pve-rs/Makefile | 1 + pve-rs/debian/control | 1 + pve-rs/src/bindings/sdn/microseg.rs | 278 ++++++++++++++++++++++++++++ pve-rs/src/bindings/sdn/mod.rs | 1 + 5 files changed, 283 insertions(+) create mode 100644 pve-rs/src/bindings/sdn/microseg.rs pve-network: Hannes Laimer (8): sdn: zones: add vxlan-gbp option to vxlan and evpn zones sdn: microseg: add config, API and guest inventory sdn: dry-run: show pending microseg changes sdn: zones: trigger microseg apply on tap_plug sdn: microseg: add tag matcher sdn: microseg: add name regex matcher sdn: microseg: expose protocol, port and macro rule options sdn: microseg: expose the group parent debian/control | 2 + src/PVE/API2/Network/SDN.pm | 38 +- src/PVE/API2/Network/SDN/Makefile | 2 + src/PVE/API2/Network/SDN/Microseg.pm | 277 +++++++ .../API2/Network/SDN/Microseg/Assignment.pm | 192 +++++ src/PVE/API2/Network/SDN/Microseg/Group.pm | 191 +++++ src/PVE/API2/Network/SDN/Microseg/Makefile | 10 + src/PVE/API2/Network/SDN/Microseg/Rule.pm | 194 +++++ src/PVE/API2/Network/SDN/Zones.pm | 8 + src/PVE/Network/SDN.pm | 36 + src/PVE/Network/SDN/Makefile | 1 + src/PVE/Network/SDN/Microseg.pm | 774 ++++++++++++++++++ src/PVE/Network/SDN/Zones.pm | 5 + src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 5 + src/PVE/Network/SDN/Zones/VxlanPlugin.pm | 10 + .../zones/evpn/gbp/expected_controller_config | 77 ++ .../zones/evpn/gbp/expected_sdn_interfaces | 45 + src/test/zones/evpn/gbp/interfaces | 7 + src/test/zones/evpn/gbp/sdn_config | 26 + .../zones/vxlan/gbp/expected_sdn_interfaces | 16 + src/test/zones/vxlan/gbp/interfaces | 7 + src/test/zones/vxlan/gbp/sdn_config | 11 + 22 files changed, 1932 insertions(+), 2 deletions(-) create mode 100644 src/PVE/API2/Network/SDN/Microseg.pm create mode 100644 src/PVE/API2/Network/SDN/Microseg/Assignment.pm create mode 100644 src/PVE/API2/Network/SDN/Microseg/Group.pm create mode 100644 src/PVE/API2/Network/SDN/Microseg/Makefile create mode 100644 src/PVE/API2/Network/SDN/Microseg/Rule.pm create mode 100644 src/PVE/Network/SDN/Microseg.pm create mode 100644 src/test/zones/evpn/gbp/expected_controller_config create mode 100644 src/test/zones/evpn/gbp/expected_sdn_interfaces create mode 100644 src/test/zones/evpn/gbp/interfaces create mode 100644 src/test/zones/evpn/gbp/sdn_config create mode 100644 src/test/zones/vxlan/gbp/expected_sdn_interfaces create mode 100644 src/test/zones/vxlan/gbp/interfaces create mode 100644 src/test/zones/vxlan/gbp/sdn_config pve-manager: Hannes Laimer (9): network: apply microseg state on reload ui: firewall: register the ICMP type stores by id ui: sdn: zones: add vxlan-gbp checkbox to vxlan and evpn ui: sdn: add microsegmentation panel ui: sdn: dry-run: show pending microseg diff ui: sdn: microseg: add tag matcher ui: sdn: microseg: add name regex matcher ui: sdn: microseg: add narrowing to the rule form ui: sdn: microseg: show and edit the group hierarchy PVE/API2/Network.pm | 4 + www/css/ext6-pve.css | 8 + www/manager6/Makefile | 8 + www/manager6/Utils.js | 34 ++ www/manager6/data/PermPathStore.js | 1 + www/manager6/dc/Config.js | 8 + www/manager6/form/MicrosegGroupSelector.js | 95 ++++ www/manager6/grid/FirewallRules.js | 2 + www/manager6/sdn/MicrosegView.js | 43 ++ www/manager6/sdn/SdnDiffView.js | 24 + www/manager6/sdn/microseg/AssignmentEdit.js | 238 ++++++++++ www/manager6/sdn/microseg/Base.js | 123 ++++++ www/manager6/sdn/microseg/GroupEdit.js | 47 ++ www/manager6/sdn/microseg/PolicyView.js | 401 +++++++++++++++++ www/manager6/sdn/microseg/RuleEdit.js | 285 ++++++++++++ www/manager6/sdn/microseg/Tree.js | 460 ++++++++++++++++++++ www/manager6/sdn/zones/EvpnEdit.js | 8 + www/manager6/sdn/zones/VxlanEdit.js | 11 + 18 files changed, 1800 insertions(+) create mode 100644 www/manager6/form/MicrosegGroupSelector.js create mode 100644 www/manager6/sdn/MicrosegView.js create mode 100644 www/manager6/sdn/microseg/AssignmentEdit.js create mode 100644 www/manager6/sdn/microseg/Base.js create mode 100644 www/manager6/sdn/microseg/GroupEdit.js create mode 100644 www/manager6/sdn/microseg/PolicyView.js create mode 100644 www/manager6/sdn/microseg/RuleEdit.js create mode 100644 www/manager6/sdn/microseg/Tree.js pve-docs: Hannes Laimer (4): sdn: add VXLAN-GBP flag to evpn/vxlan zone sections sdn: add microsegmentation section sdn: microseg: document narrowed rules sdn: microseg: document group hierarchy and the specificity tie-break pvesdn.adoc | 264 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 264 insertions(+) Summary over all repositories: 84 files changed, 10795 insertions(+), 160 deletions(-) -- Generated by murpp 0.12.0