From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 9BD131FF0AA for ; Tue, 20 Oct 2026 10:03:57 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id F0F612183B; Tue, 20 Oct 2026 10:01:40 +0200 (CEST) Message-ID: Date: Tue, 20 Oct 2026 10:01:08 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: superseded: [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF To: pve-devel@lists.proxmox.com References: <20260909104144.1110031-1-h.laimer@proxmox.com> From: Hannes Laimer Content-Language: en-US In-Reply-To: <20260909104144.1110031-1-h.laimer@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1792483268869 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.247 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: YOJPCZJXU7EZQSQQKKT3SSA4IBCFRDUX X-Message-ID-Hash: YOJPCZJXU7EZQSQQKKT3SSA4IBCFRDUX X-MailFrom: h.laimer@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: superseded-by: https://lore.proxmox.com/pve-devel/20261020080005.470757-1-h.laimer@proxmox.com/ On 2026-09-09 12:42, Hannes Laimer wrote: > Adds a second DHCP backend, `ebpf`, next to dnsmasq, selectable per > zone. It aims to replace dnsmasq eventually and keeping both for now > makes a migration simple. Every zone type can enable DHCP and the UI > offers the selector on the zone types it edits. `dnsmasq` stays limited > to simple zones. > > The responder lives in proxmox-ebpf [1]. The two patches for it apply > on top of the v3 of that series, which follows separately. pve-rs gains > a build dependency on the librust-proxmox-ebpf-dev it provides. > pve-network needs this series' pve-rs at build time and at run time. > qemu-server needs this series' pve-network as a Depends. pve-container > works without the package and needs a Breaks on the older versions. > Both pve-manager patches need it as well. > > # On the tap > > An eBPF program on the ingress of every guest tap on a vnet of an ebpf > zone parses DHCP requests. It looks the interface up in an > interface -> ip+options map and turns the request into the reply in > place, back out of the tap. The exchange never reaches the bridge. > Everything else passes untouched to the bridge and through the firewall > as before. The answer is the interface's and only a request carrying > its NIC's MAC gets it. Another MAC is passed on unanswered. So no guest > can read another one's record by asking with its MAC, and a nested > client is not handed the outer guest's address. Equal subnets in two > zones do not mix either, each interface is answered from its own vnet's > subnet. > > # The records and who writes them > > IPAM is the source and the map holds the answer of every guest > interface on the node. A record change writes the records of the MAC's > interfaces here. A tap plug attaches its interface with its record and > an unplug drops it. All of that runs under a shared lock. The full pass > builds the desired state from the configs, every guest NIC in the > cluster with the vnet it sits on and the answer its MAC gets there from > the MAC cache. The responder gives every listed interface present here > its record, and a link where it is served. An interface the configs > serve without a mapping gets a marker, so a plug that read the cache > before the mapping went loses to it. It leaves a linked interface the > configs do not name yet alone, since a hotplug plugs before it writes > the config. It drops what a crashed guest left. A link on an interface > without a record is harmless, the program passes everything there. So > nothing is lost by attaching first and answering once the record is in. > A pass failing midway warns in its task, and what was served keeps > being served meanwhile. A change drawn before it is dropped, the next > apply carries its input. > > - a guest start, NIC hotplug or migration allocates the record before > the interface is plugged. Then a new tap plug hook of the dhcp > plugins attaches the program, and a stop or unplug detaches it > - a mapping created, updated or deleted through the API is written by > the editing node. That node pokes the node running the guest through > a new node endpoint (POST /nodes/{node}/sdn/dhcp-mapping) with a > detached pvesh call, proxied over ssh like pvesh does. An > unreachable node catches up on its next apply > - an SDN apply runs the full pass and refreshes the programs, so a > schema rebuild is refilled and a zone switching its backend takes > effect for its running guests that hold a mapping as well > - after a boot nothing is pinned, so the SDN commit brings the > responder back before the guests start. A change that still finds > nothing loaded runs the full pass itself > > # Ordering > > Nothing forces the collection of that state into one order, so every > change carries a generation, a per-node counter the responder hands > out. There is one rule. A change draws after its input is written and > a reader before it reads. So two changes are ordered by the input they > saw and not by when they reach the kernel. > > Who draws when: > - record change after the IPAM record and the cache are written > - full pass before it reads the configs and the cache > - tap plug before it reads the cache. Its number orders the > cache read and says nothing about the guest config, > which is not written yet > > What the number decides: > - record keeps the generation that wrote it, an older change > never overwrites it. A removal by a record change > leaves a marker with its generation for the same > reason > - full pass records its generation before it runs, so an older > pass is dropped. It leaves alone every record a > newer change wrote since it read its input > > Links carry no generation. The configs decide about the interfaces they > name, a linked interface they do not name is left alone. So a hotplug's > plug survives a pass that read the config before the write. > > A generation cannot order a pass against a hotplug that moves a tap > between bridges. The plug runs before the guest config names the new > bridge, so a pass in between reads the old one. That is why every > record also names the vnet its writer saw the tap on. > > What the vnet decides: > - pass, record change leave an entry alone that names another vnet > than their config does, their config is stale > for that tap. The pass lets the link follow > the entry > - tap plug replaces such an entry. Plugs of one tap are > serialized by the guest lock, so the plug is > the latest word on where it sits. On its own > vnet it compares by generation like any other > change. A plug finding no entry writes, a pass > that stamped meanwhile never saw its interface > - plug elsewhere a tap moved onto a plain bridge or a zone of > another backend trades its entry for a marker > naming that place. So a pass still reading > the old vnet leaves it alone, and one reading > the new place keeps it as it is. A tap without > an entry gets none > - tap unplug drops the entry with the link, the tap is > gone or about to be. A tap with neither a > link nor an interface left leaves its entry > to the next pass > > The counter lives under /run and restarts with the node like > everything it orders. > > # What a guest sees > > A changed mapping reaches a running guest at its next renewal, so the > lease time bounds how long an old answer survives. A deleted one is no > longer answered, the guest keeps its address until the lease runs out. > Subnets get a `dhcp-lease-time` property, used by both backends. > Without it dnsmasq keeps handing out infinite leases and the responder > defaults to ten minutes. > > Since the request is answered on the tap, neither the firewall nor the > rate limit of the interface sees it. So neither the guest's rules nor > its DHCP firewall option nor the rate limit apply to an answered > request. The reply is put on the tap's egress, so the rate limit shapes > it and the firewall never sees it. > > The responder identifies itself with the subnet gateway. A subnet > without one is served under a link-local identifier and without a > default route, its guests renew by broadcast at rebinding time. It > hands out an IPv4 DNS server only, an IPv6 one on a v4 subnet is left > out. A guest tagging its own requests is not answered either, so a > trunk port keeps working as before. Only DHCPv4 for now, but adding v6 > is very possible once we are happy with the design. > > Changes made directly on an external IPAM service are not detectable > and the answers are cached per MAC. So an apply does not pick them up > either, exactly like with dnsmasq today. > > note on the earlier versions: The main problem was that a tap plug ran > on information not in the config yet, only pending. This allowed a race > between the tap plug hook updating the kernel state and a fast enough > SDN apply undoing it again, and a few other subtleties. This version > should address all of these properly. Sorry for the noise with the other > two versions, both the result of me spotting the problem too late and > being too fast in assuming I had solved it. > > > [1] https://lore.proxmox.com/pve-devel/20260909103952.1108084-1-h.laimer@proxmox.com/T/#t > > > proxmox-ebpf: > > Hannes Laimer (2): > dhcp: add per-tap responder BPF program > dhcp: add responder subsystem > > Cargo.toml | 5 + > debian/control | 6 +- > src/bpf-shim/bpf/bpf_helpers.h | 3 + > src/bpf-shim/vmlinux.h | 1 + > src/dhcp/bpf/dhcp.bpf.c | 405 ++++++++++++ > src/dhcp/bpf/types.h | 25 + > src/dhcp/mod.rs | 1113 ++++++++++++++++++++++++++++++++ > src/dhcp/types.rs | 80 +++ > src/lib.rs | 3 + > tests/common/mod.rs | 63 +- > tests/dhcp.rs | 827 ++++++++++++++++++++++++ > 11 files changed, 2526 insertions(+), 5 deletions(-) > create mode 100644 src/dhcp/bpf/dhcp.bpf.c > create mode 100644 src/dhcp/bpf/types.h > create mode 100644 src/dhcp/mod.rs > create mode 100644 src/dhcp/types.rs > create mode 100644 tests/dhcp.rs > > > proxmox-perl-rs: > > Hannes Laimer (1): > pve-rs: sdn: add dhcp responder bindings > > pve-rs/Cargo.toml | 2 + > pve-rs/Makefile | 1 + > pve-rs/debian/control | 2 + > pve-rs/src/bindings/sdn/dhcp.rs | 162 ++++++++++++++++++++++++++++++++ > pve-rs/src/bindings/sdn/mod.rs | 1 + > 5 files changed, 168 insertions(+) > create mode 100644 pve-rs/src/bindings/sdn/dhcp.rs > > > pve-network: > > Hannes Laimer (8): > sdn: push mapping changes from the ipam API to the dhcp backend > sdn: ipam: do not cache negative per-MAC answers, lock the write > sdn: subnets: add dhcp-lease-time property > sdn: dhcp: only assert a backend's availability for zones using it > sdn: dhcp: add ebpf plugin > sdn: zones: attach the dhcp responder on tap plug, detach on unplug > sdn: dhcp: apply mapping edits on the node serving the guest > sdn: zones: offer dhcp on all zone types, keep dnsmasq simple-only > > debian/libpve-network-perl.prerm | 13 + > src/PVE/API2/Network/SDN/Ips.pm | 10 + > src/PVE/API2/Network/SDN/Nodes/Status.pm | 37 + > src/PVE/API2/Network/SDN/Zones.pm | 11 +- > src/PVE/Network/SDN/Dhcp.pm | 202 ++- > src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 104 +- > src/PVE/Network/SDN/Dhcp/Ebpf.pm | 355 ++++++ > src/PVE/Network/SDN/Dhcp/Makefile | 2 +- > src/PVE/Network/SDN/Dhcp/Plugin.pm | 25 +- > src/PVE/Network/SDN/Ipams.pm | 61 +- > src/PVE/Network/SDN/SubnetPlugin.pm | 11 + > src/PVE/Network/SDN/Vnets.pm | 5 +- > src/PVE/Network/SDN/Zones.pm | 31 +- > src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 32 + > src/PVE/Network/SDN/Zones/FaucetPlugin.pm | 1 + > src/PVE/Network/SDN/Zones/QinQPlugin.pm | 7 + > src/PVE/Network/SDN/Zones/SimplePlugin.pm | 2 +- > src/PVE/Network/SDN/Zones/VlanPlugin.pm | 7 + > src/PVE/Network/SDN/Zones/VxlanPlugin.pm | 25 + > src/test/run_test_vnets_blackbox.pl | 1361 ++++++++++++++++++++- > 20 files changed, 2223 insertions(+), 79 deletions(-) > create mode 100755 debian/libpve-network-perl.prerm > create mode 100644 src/PVE/Network/SDN/Dhcp/Ebpf.pm > > > qemu-server: > > Hannes Laimer (1): > network: report NIC plug and unplug to SDN with the MAC > > src/PVE/CLI/qm.pm | 4 ++-- > src/PVE/QemuServer.pm | 4 ++++ > src/PVE/QemuServer/Network.pm | 4 ++-- > src/usr/pve-bridge | 8 +++++++- > src/usr/pve-bridgedown | 4 ++-- > 5 files changed, 17 insertions(+), 7 deletions(-) > > > pve-container: > > Hannes Laimer (1): > net: report veth plug and unplug to SDN with the hwaddr > > src/PVE/LXC.pm | 24 ++++++++++++++++++++++-- > src/PVE/LXC/Config.pm | 2 +- > src/lxc-pve-poststop-hook | 5 ++--- > 3 files changed, 25 insertions(+), 6 deletions(-) > > > pve-manager: > > Hannes Laimer (2): > ui: sdn: dhcp backend selector on all zones, expose dhcp options > sdn: bring the dhcp backends up at boot before the guests start > > bin/pve-sdn-commit | 3 +++ > services/pve-sdn-commit.service | 3 ++- > www/manager6/sdn/SubnetEdit.js | 23 +++++++++++++++++++++++ > www/manager6/sdn/zones/Base.js | 17 +++++++++++++++++ > www/manager6/sdn/zones/SimpleEdit.js | 11 ----------- > 5 files changed, 45 insertions(+), 12 deletions(-) > > > pve-docs: > > Hannes Laimer (1): > sdn: dhcp: document the ebpf backend > > pvesdn.adoc | 113 +++++++++++++++++++++++++++++++++++++++++++--------- > 1 file changed, 95 insertions(+), 18 deletions(-) > > > Summary over all repositories: > 50 files changed, 5099 insertions(+), 127 deletions(-) >