superseded: [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF

[Hannes Laimer <h.laimer@proxmox.com>]

superseded-by:
https://lore.proxmox.com/pve-devel/20261020080005.470757-1-h.laimer@proxmox.com/

On 2026-09-09 12:42, Hannes Laimer wrote:
> Adds a second DHCP backend, `ebpf`, next to dnsmasq, selectable per
> zone. It aims to replace dnsmasq eventually and keeping both for now
> makes a migration simple. Every zone type can enable DHCP and the UI
> offers the selector on the zone types it edits. `dnsmasq` stays limited
> to simple zones.
> 
> The responder lives in proxmox-ebpf [1]. The two patches for it apply
> on top of the v3 of that series, which follows separately. pve-rs gains
> a build dependency on the librust-proxmox-ebpf-dev it provides.
> pve-network needs this series' pve-rs at build time and at run time.
> qemu-server needs this series' pve-network as a Depends. pve-container
> works without the package and needs a Breaks on the older versions.
> Both pve-manager patches need it as well.
> 
> # On the tap
> 
> An eBPF program on the ingress of every guest tap on a vnet of an ebpf
> zone parses DHCP requests. It looks the interface up in an
> interface -> ip+options map and turns the request into the reply in
> place, back out of the tap. The exchange never reaches the bridge.
> Everything else passes untouched to the bridge and through the firewall
> as before. The answer is the interface's and only a request carrying
> its NIC's MAC gets it. Another MAC is passed on unanswered. So no guest
> can read another one's record by asking with its MAC, and a nested
> client is not handed the outer guest's address. Equal subnets in two
> zones do not mix either, each interface is answered from its own vnet's
> subnet.
> 
> # The records and who writes them
> 
> IPAM is the source and the map holds the answer of every guest
> interface on the node. A record change writes the records of the MAC's
> interfaces here. A tap plug attaches its interface with its record and
> an unplug drops it. All of that runs under a shared lock. The full pass
> builds the desired state from the configs, every guest NIC in the
> cluster with the vnet it sits on and the answer its MAC gets there from
> the MAC cache. The responder gives every listed interface present here
> its record, and a link where it is served. An interface the configs
> serve without a mapping gets a marker, so a plug that read the cache
> before the mapping went loses to it. It leaves a linked interface the
> configs do not name yet alone, since a hotplug plugs before it writes
> the config. It drops what a crashed guest left. A link on an interface
> without a record is harmless, the program passes everything there. So
> nothing is lost by attaching first and answering once the record is in.
> A pass failing midway warns in its task, and what was served keeps
> being served meanwhile. A change drawn before it is dropped, the next
> apply carries its input.
> 
>  - a guest start, NIC hotplug or migration allocates the record before
>    the interface is plugged. Then a new tap plug hook of the dhcp
>    plugins attaches the program, and a stop or unplug detaches it
>  - a mapping created, updated or deleted through the API is written by
>    the editing node. That node pokes the node running the guest through
>    a new node endpoint (POST /nodes/{node}/sdn/dhcp-mapping) with a
>    detached pvesh call, proxied over ssh like pvesh does. An
>    unreachable node catches up on its next apply
>  - an SDN apply runs the full pass and refreshes the programs, so a
>    schema rebuild is refilled and a zone switching its backend takes
>    effect for its running guests that hold a mapping as well
>  - after a boot nothing is pinned, so the SDN commit brings the
>    responder back before the guests start. A change that still finds
>    nothing loaded runs the full pass itself
> 
> # Ordering
> 
> Nothing forces the collection of that state into one order, so every
> change carries a generation, a per-node counter the responder hands
> out. There is one rule. A change draws after its input is written and
> a reader before it reads. So two changes are ordered by the input they
> saw and not by when they reach the kernel.
> 
> Who draws when:
>  - record change  after the IPAM record and the cache are written
>  - full pass      before it reads the configs and the cache
>  - tap plug       before it reads the cache. Its number orders the
>                   cache read and says nothing about the guest config,
>                   which is not written yet
> 
> What the number decides:
>  - record         keeps the generation that wrote it, an older change
>                   never overwrites it. A removal by a record change
>                   leaves a marker with its generation for the same
>                   reason
>  - full pass      records its generation before it runs, so an older
>                   pass is dropped. It leaves alone every record a
>                   newer change wrote since it read its input
> 
> Links carry no generation. The configs decide about the interfaces they
> name, a linked interface they do not name is left alone. So a hotplug's
> plug survives a pass that read the config before the write.
> 
> A generation cannot order a pass against a hotplug that moves a tap
> between bridges. The plug runs before the guest config names the new
> bridge, so a pass in between reads the old one. That is why every
> record also names the vnet its writer saw the tap on.
> 
> What the vnet decides:
>  - pass, record change   leave an entry alone that names another vnet
>                          than their config does, their config is stale
>                          for that tap. The pass lets the link follow
>                          the entry
>  - tap plug              replaces such an entry. Plugs of one tap are
>                          serialized by the guest lock, so the plug is
>                          the latest word on where it sits. On its own
>                          vnet it compares by generation like any other
>                          change. A plug finding no entry writes, a pass
>                          that stamped meanwhile never saw its interface
>  - plug elsewhere        a tap moved onto a plain bridge or a zone of
>                          another backend trades its entry for a marker
>                          naming that place. So a pass still reading
>                          the old vnet leaves it alone, and one reading
>                          the new place keeps it as it is. A tap without
>                          an entry gets none
>  - tap unplug            drops the entry with the link, the tap is
>                          gone or about to be. A tap with neither a
>                          link nor an interface left leaves its entry
>                          to the next pass
> 
> The counter lives under /run and restarts with the node like
> everything it orders.
> 
> # What a guest sees
> 
> A changed mapping reaches a running guest at its next renewal, so the
> lease time bounds how long an old answer survives. A deleted one is no
> longer answered, the guest keeps its address until the lease runs out.
> Subnets get a `dhcp-lease-time` property, used by both backends.
> Without it dnsmasq keeps handing out infinite leases and the responder
> defaults to ten minutes.
> 
> Since the request is answered on the tap, neither the firewall nor the
> rate limit of the interface sees it. So neither the guest's rules nor
> its DHCP firewall option nor the rate limit apply to an answered
> request. The reply is put on the tap's egress, so the rate limit shapes
> it and the firewall never sees it.
> 
> The responder identifies itself with the subnet gateway. A subnet
> without one is served under a link-local identifier and without a
> default route, its guests renew by broadcast at rebinding time. It
> hands out an IPv4 DNS server only, an IPv6 one on a v4 subnet is left
> out. A guest tagging its own requests is not answered either, so a
> trunk port keeps working as before. Only DHCPv4 for now, but adding v6
> is very possible once we are happy with the design.
> 
> Changes made directly on an external IPAM service are not detectable
> and the answers are cached per MAC. So an apply does not pick them up
> either, exactly like with dnsmasq today.
> 
> note on the earlier versions: The main problem was that a tap plug ran
> on information not in the config yet, only pending. This allowed a race
> between the tap plug hook updating the kernel state and a fast enough
> SDN apply undoing it again, and a few other subtleties. This version
> should address all of these properly. Sorry for the noise with the other
> two versions, both the result of me spotting the problem too late and
> being too fast in assuming I had solved it.
> 
> 
> [1] https://lore.proxmox.com/pve-devel/20260909103952.1108084-1-h.laimer@proxmox.com/T/#t
> 
> 
> proxmox-ebpf:
> 
> Hannes Laimer (2):
>   dhcp: add per-tap responder BPF program
>   dhcp: add responder subsystem
> 
>  Cargo.toml                     |    5 +
>  debian/control                 |    6 +-
>  src/bpf-shim/bpf/bpf_helpers.h |    3 +
>  src/bpf-shim/vmlinux.h         |    1 +
>  src/dhcp/bpf/dhcp.bpf.c        |  405 ++++++++++++
>  src/dhcp/bpf/types.h           |   25 +
>  src/dhcp/mod.rs                | 1113 ++++++++++++++++++++++++++++++++
>  src/dhcp/types.rs              |   80 +++
>  src/lib.rs                     |    3 +
>  tests/common/mod.rs            |   63 +-
>  tests/dhcp.rs                  |  827 ++++++++++++++++++++++++
>  11 files changed, 2526 insertions(+), 5 deletions(-)
>  create mode 100644 src/dhcp/bpf/dhcp.bpf.c
>  create mode 100644 src/dhcp/bpf/types.h
>  create mode 100644 src/dhcp/mod.rs
>  create mode 100644 src/dhcp/types.rs
>  create mode 100644 tests/dhcp.rs
> 
> 
> proxmox-perl-rs:
> 
> Hannes Laimer (1):
>   pve-rs: sdn: add dhcp responder bindings
> 
>  pve-rs/Cargo.toml               |   2 +
>  pve-rs/Makefile                 |   1 +
>  pve-rs/debian/control           |   2 +
>  pve-rs/src/bindings/sdn/dhcp.rs | 162 ++++++++++++++++++++++++++++++++
>  pve-rs/src/bindings/sdn/mod.rs  |   1 +
>  5 files changed, 168 insertions(+)
>  create mode 100644 pve-rs/src/bindings/sdn/dhcp.rs
> 
> 
> pve-network:
> 
> Hannes Laimer (8):
>   sdn: push mapping changes from the ipam API to the dhcp backend
>   sdn: ipam: do not cache negative per-MAC answers, lock the write
>   sdn: subnets: add dhcp-lease-time property
>   sdn: dhcp: only assert a backend's availability for zones using it
>   sdn: dhcp: add ebpf plugin
>   sdn: zones: attach the dhcp responder on tap plug, detach on unplug
>   sdn: dhcp: apply mapping edits on the node serving the guest
>   sdn: zones: offer dhcp on all zone types, keep dnsmasq simple-only
> 
>  debian/libpve-network-perl.prerm          |   13 +
>  src/PVE/API2/Network/SDN/Ips.pm           |   10 +
>  src/PVE/API2/Network/SDN/Nodes/Status.pm  |   37 +
>  src/PVE/API2/Network/SDN/Zones.pm         |   11 +-
>  src/PVE/Network/SDN/Dhcp.pm               |  202 ++-
>  src/PVE/Network/SDN/Dhcp/Dnsmasq.pm       |  104 +-
>  src/PVE/Network/SDN/Dhcp/Ebpf.pm          |  355 ++++++
>  src/PVE/Network/SDN/Dhcp/Makefile         |    2 +-
>  src/PVE/Network/SDN/Dhcp/Plugin.pm        |   25 +-
>  src/PVE/Network/SDN/Ipams.pm              |   61 +-
>  src/PVE/Network/SDN/SubnetPlugin.pm       |   11 +
>  src/PVE/Network/SDN/Vnets.pm              |    5 +-
>  src/PVE/Network/SDN/Zones.pm              |   31 +-
>  src/PVE/Network/SDN/Zones/EvpnPlugin.pm   |   32 +
>  src/PVE/Network/SDN/Zones/FaucetPlugin.pm |    1 +
>  src/PVE/Network/SDN/Zones/QinQPlugin.pm   |    7 +
>  src/PVE/Network/SDN/Zones/SimplePlugin.pm |    2 +-
>  src/PVE/Network/SDN/Zones/VlanPlugin.pm   |    7 +
>  src/PVE/Network/SDN/Zones/VxlanPlugin.pm  |   25 +
>  src/test/run_test_vnets_blackbox.pl       | 1361 ++++++++++++++++++++-
>  20 files changed, 2223 insertions(+), 79 deletions(-)
>  create mode 100755 debian/libpve-network-perl.prerm
>  create mode 100644 src/PVE/Network/SDN/Dhcp/Ebpf.pm
> 
> 
> qemu-server:
> 
> Hannes Laimer (1):
>   network: report NIC plug and unplug to SDN with the MAC
> 
>  src/PVE/CLI/qm.pm             | 4 ++--
>  src/PVE/QemuServer.pm         | 4 ++++
>  src/PVE/QemuServer/Network.pm | 4 ++--
>  src/usr/pve-bridge            | 8 +++++++-
>  src/usr/pve-bridgedown        | 4 ++--
>  5 files changed, 17 insertions(+), 7 deletions(-)
> 
> 
> pve-container:
> 
> Hannes Laimer (1):
>   net: report veth plug and unplug to SDN with the hwaddr
> 
>  src/PVE/LXC.pm            | 24 ++++++++++++++++++++++--
>  src/PVE/LXC/Config.pm     |  2 +-
>  src/lxc-pve-poststop-hook |  5 ++---
>  3 files changed, 25 insertions(+), 6 deletions(-)
> 
> 
> pve-manager:
> 
> Hannes Laimer (2):
>   ui: sdn: dhcp backend selector on all zones, expose dhcp options
>   sdn: bring the dhcp backends up at boot before the guests start
> 
>  bin/pve-sdn-commit                   |  3 +++
>  services/pve-sdn-commit.service      |  3 ++-
>  www/manager6/sdn/SubnetEdit.js       | 23 +++++++++++++++++++++++
>  www/manager6/sdn/zones/Base.js       | 17 +++++++++++++++++
>  www/manager6/sdn/zones/SimpleEdit.js | 11 -----------
>  5 files changed, 45 insertions(+), 12 deletions(-)
> 
> 
> pve-docs:
> 
> Hannes Laimer (1):
>   sdn: dhcp: document the ebpf backend
> 
>  pvesdn.adoc | 113 +++++++++++++++++++++++++++++++++++++++++++---------
>  1 file changed, 95 insertions(+), 18 deletions(-)
> 
> 
> Summary over all repositories:
>   50 files changed, 5099 insertions(+), 127 deletions(-)
>