From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 3E7CA1FF0B3 for ; Wed, 09 Sep 2026 12:41:59 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id E4381215A3; Wed, 09 Sep 2026 12:41:56 +0200 (CEST) From: Hannes Laimer To: pve-devel@lists.proxmox.com Subject: [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Date: Wed, 9 Sep 2026 12:41:28 +0200 Message-ID: <20260909104144.1110031-1-h.laimer@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1788950501005 X-SPAM-LEVEL: Spam detection results: 0 AWL -1.094 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLACK 3 Contains an URL listed in the URIBL blacklist [types.rs] Message-ID-Hash: VWST6ZKANRO2QKSHBBQWYMTUHQ5UTHX5 X-Message-ID-Hash: VWST6ZKANRO2QKSHBBQWYMTUHQ5UTHX5 X-MailFrom: h.laimer@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Adds a second DHCP backend, `ebpf`, next to dnsmasq, selectable per zone. It aims to replace dnsmasq eventually and keeping both for now makes a migration simple. Every zone type can enable DHCP and the UI offers the selector on the zone types it edits. `dnsmasq` stays limited to simple zones. The responder lives in proxmox-ebpf [1]. The two patches for it apply on top of the v3 of that series, which follows separately. pve-rs gains a build dependency on the librust-proxmox-ebpf-dev it provides. pve-network needs this series' pve-rs at build time and at run time. qemu-server needs this series' pve-network as a Depends. pve-container works without the package and needs a Breaks on the older versions. Both pve-manager patches need it as well. # On the tap An eBPF program on the ingress of every guest tap on a vnet of an ebpf zone parses DHCP requests. It looks the interface up in an interface -> ip+options map and turns the request into the reply in place, back out of the tap. The exchange never reaches the bridge. Everything else passes untouched to the bridge and through the firewall as before. The answer is the interface's and only a request carrying its NIC's MAC gets it. Another MAC is passed on unanswered. So no guest can read another one's record by asking with its MAC, and a nested client is not handed the outer guest's address. Equal subnets in two zones do not mix either, each interface is answered from its own vnet's subnet. # The records and who writes them IPAM is the source and the map holds the answer of every guest interface on the node. A record change writes the records of the MAC's interfaces here. A tap plug attaches its interface with its record and an unplug drops it. All of that runs under a shared lock. The full pass builds the desired state from the configs, every guest NIC in the cluster with the vnet it sits on and the answer its MAC gets there from the MAC cache. The responder gives every listed interface present here its record, and a link where it is served. An interface the configs serve without a mapping gets a marker, so a plug that read the cache before the mapping went loses to it. It leaves a linked interface the configs do not name yet alone, since a hotplug plugs before it writes the config. It drops what a crashed guest left. A link on an interface without a record is harmless, the program passes everything there. So nothing is lost by attaching first and answering once the record is in. A pass failing midway warns in its task, and what was served keeps being served meanwhile. A change drawn before it is dropped, the next apply carries its input. - a guest start, NIC hotplug or migration allocates the record before the interface is plugged. Then a new tap plug hook of the dhcp plugins attaches the program, and a stop or unplug detaches it - a mapping created, updated or deleted through the API is written by the editing node. That node pokes the node running the guest through a new node endpoint (POST /nodes/{node}/sdn/dhcp-mapping) with a detached pvesh call, proxied over ssh like pvesh does. An unreachable node catches up on its next apply - an SDN apply runs the full pass and refreshes the programs, so a schema rebuild is refilled and a zone switching its backend takes effect for its running guests that hold a mapping as well - after a boot nothing is pinned, so the SDN commit brings the responder back before the guests start. A change that still finds nothing loaded runs the full pass itself # Ordering Nothing forces the collection of that state into one order, so every change carries a generation, a per-node counter the responder hands out. There is one rule. A change draws after its input is written and a reader before it reads. So two changes are ordered by the input they saw and not by when they reach the kernel. Who draws when: - record change after the IPAM record and the cache are written - full pass before it reads the configs and the cache - tap plug before it reads the cache. Its number orders the cache read and says nothing about the guest config, which is not written yet What the number decides: - record keeps the generation that wrote it, an older change never overwrites it. A removal by a record change leaves a marker with its generation for the same reason - full pass records its generation before it runs, so an older pass is dropped. It leaves alone every record a newer change wrote since it read its input Links carry no generation. The configs decide about the interfaces they name, a linked interface they do not name is left alone. So a hotplug's plug survives a pass that read the config before the write. A generation cannot order a pass against a hotplug that moves a tap between bridges. The plug runs before the guest config names the new bridge, so a pass in between reads the old one. That is why every record also names the vnet its writer saw the tap on. What the vnet decides: - pass, record change leave an entry alone that names another vnet than their config does, their config is stale for that tap. The pass lets the link follow the entry - tap plug replaces such an entry. Plugs of one tap are serialized by the guest lock, so the plug is the latest word on where it sits. On its own vnet it compares by generation like any other change. A plug finding no entry writes, a pass that stamped meanwhile never saw its interface - plug elsewhere a tap moved onto a plain bridge or a zone of another backend trades its entry for a marker naming that place. So a pass still reading the old vnet leaves it alone, and one reading the new place keeps it as it is. A tap without an entry gets none - tap unplug drops the entry with the link, the tap is gone or about to be. A tap with neither a link nor an interface left leaves its entry to the next pass The counter lives under /run and restarts with the node like everything it orders. # What a guest sees A changed mapping reaches a running guest at its next renewal, so the lease time bounds how long an old answer survives. A deleted one is no longer answered, the guest keeps its address until the lease runs out. Subnets get a `dhcp-lease-time` property, used by both backends. Without it dnsmasq keeps handing out infinite leases and the responder defaults to ten minutes. Since the request is answered on the tap, neither the firewall nor the rate limit of the interface sees it. So neither the guest's rules nor its DHCP firewall option nor the rate limit apply to an answered request. The reply is put on the tap's egress, so the rate limit shapes it and the firewall never sees it. The responder identifies itself with the subnet gateway. A subnet without one is served under a link-local identifier and without a default route, its guests renew by broadcast at rebinding time. It hands out an IPv4 DNS server only, an IPv6 one on a v4 subnet is left out. A guest tagging its own requests is not answered either, so a trunk port keeps working as before. Only DHCPv4 for now, but adding v6 is very possible once we are happy with the design. Changes made directly on an external IPAM service are not detectable and the answers are cached per MAC. So an apply does not pick them up either, exactly like with dnsmasq today. note on the earlier versions: The main problem was that a tap plug ran on information not in the config yet, only pending. This allowed a race between the tap plug hook updating the kernel state and a fast enough SDN apply undoing it again, and a few other subtleties. This version should address all of these properly. Sorry for the noise with the other two versions, both the result of me spotting the problem too late and being too fast in assuming I had solved it. [1] https://lore.proxmox.com/pve-devel/20260909103952.1108084-1-h.laimer@proxmox.com/T/#t proxmox-ebpf: Hannes Laimer (2): dhcp: add per-tap responder BPF program dhcp: add responder subsystem Cargo.toml | 5 + debian/control | 6 +- src/bpf-shim/bpf/bpf_helpers.h | 3 + src/bpf-shim/vmlinux.h | 1 + src/dhcp/bpf/dhcp.bpf.c | 405 ++++++++++++ src/dhcp/bpf/types.h | 25 + src/dhcp/mod.rs | 1113 ++++++++++++++++++++++++++++++++ src/dhcp/types.rs | 80 +++ src/lib.rs | 3 + tests/common/mod.rs | 63 +- tests/dhcp.rs | 827 ++++++++++++++++++++++++ 11 files changed, 2526 insertions(+), 5 deletions(-) create mode 100644 src/dhcp/bpf/dhcp.bpf.c create mode 100644 src/dhcp/bpf/types.h create mode 100644 src/dhcp/mod.rs create mode 100644 src/dhcp/types.rs create mode 100644 tests/dhcp.rs proxmox-perl-rs: Hannes Laimer (1): pve-rs: sdn: add dhcp responder bindings pve-rs/Cargo.toml | 2 + pve-rs/Makefile | 1 + pve-rs/debian/control | 2 + pve-rs/src/bindings/sdn/dhcp.rs | 162 ++++++++++++++++++++++++++++++++ pve-rs/src/bindings/sdn/mod.rs | 1 + 5 files changed, 168 insertions(+) create mode 100644 pve-rs/src/bindings/sdn/dhcp.rs pve-network: Hannes Laimer (8): sdn: push mapping changes from the ipam API to the dhcp backend sdn: ipam: do not cache negative per-MAC answers, lock the write sdn: subnets: add dhcp-lease-time property sdn: dhcp: only assert a backend's availability for zones using it sdn: dhcp: add ebpf plugin sdn: zones: attach the dhcp responder on tap plug, detach on unplug sdn: dhcp: apply mapping edits on the node serving the guest sdn: zones: offer dhcp on all zone types, keep dnsmasq simple-only debian/libpve-network-perl.prerm | 13 + src/PVE/API2/Network/SDN/Ips.pm | 10 + src/PVE/API2/Network/SDN/Nodes/Status.pm | 37 + src/PVE/API2/Network/SDN/Zones.pm | 11 +- src/PVE/Network/SDN/Dhcp.pm | 202 ++- src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 104 +- src/PVE/Network/SDN/Dhcp/Ebpf.pm | 355 ++++++ src/PVE/Network/SDN/Dhcp/Makefile | 2 +- src/PVE/Network/SDN/Dhcp/Plugin.pm | 25 +- src/PVE/Network/SDN/Ipams.pm | 61 +- src/PVE/Network/SDN/SubnetPlugin.pm | 11 + src/PVE/Network/SDN/Vnets.pm | 5 +- src/PVE/Network/SDN/Zones.pm | 31 +- src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 32 + src/PVE/Network/SDN/Zones/FaucetPlugin.pm | 1 + src/PVE/Network/SDN/Zones/QinQPlugin.pm | 7 + src/PVE/Network/SDN/Zones/SimplePlugin.pm | 2 +- src/PVE/Network/SDN/Zones/VlanPlugin.pm | 7 + src/PVE/Network/SDN/Zones/VxlanPlugin.pm | 25 + src/test/run_test_vnets_blackbox.pl | 1361 ++++++++++++++++++++- 20 files changed, 2223 insertions(+), 79 deletions(-) create mode 100755 debian/libpve-network-perl.prerm create mode 100644 src/PVE/Network/SDN/Dhcp/Ebpf.pm qemu-server: Hannes Laimer (1): network: report NIC plug and unplug to SDN with the MAC src/PVE/CLI/qm.pm | 4 ++-- src/PVE/QemuServer.pm | 4 ++++ src/PVE/QemuServer/Network.pm | 4 ++-- src/usr/pve-bridge | 8 +++++++- src/usr/pve-bridgedown | 4 ++-- 5 files changed, 17 insertions(+), 7 deletions(-) pve-container: Hannes Laimer (1): net: report veth plug and unplug to SDN with the hwaddr src/PVE/LXC.pm | 24 ++++++++++++++++++++++-- src/PVE/LXC/Config.pm | 2 +- src/lxc-pve-poststop-hook | 5 ++--- 3 files changed, 25 insertions(+), 6 deletions(-) pve-manager: Hannes Laimer (2): ui: sdn: dhcp backend selector on all zones, expose dhcp options sdn: bring the dhcp backends up at boot before the guests start bin/pve-sdn-commit | 3 +++ services/pve-sdn-commit.service | 3 ++- www/manager6/sdn/SubnetEdit.js | 23 +++++++++++++++++++++++ www/manager6/sdn/zones/Base.js | 17 +++++++++++++++++ www/manager6/sdn/zones/SimpleEdit.js | 11 ----------- 5 files changed, 45 insertions(+), 12 deletions(-) pve-docs: Hannes Laimer (1): sdn: dhcp: document the ebpf backend pvesdn.adoc | 113 +++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 95 insertions(+), 18 deletions(-) Summary over all repositories: 50 files changed, 5099 insertions(+), 127 deletions(-) -- Generated by murpp 0.12.0