* [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF
@ 2026-10-20 7:59 Hannes Laimer
2026-10-20 7:59 ` [PATCH proxmox-ebpf v3 01/17] dhcp: add the per-tap DHCPv4 responder program Hannes Laimer
` (18 more replies)
0 siblings, 19 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 7:59 UTC (permalink / raw)
To: pve-devel
This series adds a second DHCP backend, ebpf, next to dnsmasq. A zone
picks its backend. dnsmasq stays limited to simple zones, while ebpf
answers on the guest's own interface and works on every zone type. A
guest on a zone with the ebpf backend gets the address its NIC has in
the IPAM. There is no daemon per zone and no lease file. The answers
are held in the kernel and do not depend on the IPAM being reachable.
The responder is a BPF program on the ingress of each guest interface
of a zone with the ebpf backend. It answers a request from the record
of that interface, and the exchange never reaches the bridge. A
request whose client MAC is not the interface's own passes on
unanswered. So a guest cannot read another guest's record, and a
nested client does not get the outer guest's address.
An edit through /cluster/sdn/vnets/{vnet}/ips reaches the guest's node
by a detached call to a new node endpoint, POST
/nodes/{node}/sdn/dhcp-mapping. The guest sees the change at its next
renewal, so the lease time bounds how long an old answer survives.
Subnets get a dhcp-lease-time property for both backends. Without it
dnsmasq hands out infinite leases and the responder ten minutes. A
failed push is logged in the journal, and the responder on the guest's
node catches up at the node's next apply or the guest's next start. An
apply also attaches the responder to running guests, so a zone
switched to ebpf serves them without a restart.
Changes reach the kernel in the order they read their input. So no
change overwrites a newer one with older input. A hotplug moves the
interface before the guest config names the new bridge. An apply or an
edit that read the older config in between leaves the moved interface
as its plug left it. A change that fails only warns, and what was
served stays served. Nothing of the responder survives a reboot, so it
is loaded again at boot before the guests start.
Only DHCPv4 is answered. A request the guest tagged itself passes
unanswered as well, so a trunk port keeps working as before. The
request is answered on the guest interface, so the guest's firewall
rules, its DHCP firewall option and its rate limit do not apply to it.
The reply leaves through the interface's egress, where the rate limit
shapes it. A request the responder does not answer passes on to the
bridge and through the firewall as before. Mappings changed directly
in an external IPAM are not picked up, as with dnsmasq.
In patch order, the series adds
- proxmox-ebpf, the dhcp subsystem, a tc program answering on the
guest interface and the per-interface map it answers from
- pve-rs, the bindings
- pve-network, the ebpf backend, the hooks that keep a backend current
when a guest interface or a mapping changes, dhcp on every zone
type, and fixes to the dnsmasq reservations and the MAC cache
- qemu-server and pve-container, the reports of a guest NIC coming,
going or moving, and for VMs the push at a migration's resume
- pve-manager, the backend and subnet options in the UI, and the
backends brought up at boot ahead of the guests
- pve-docs, documentation
- proxmox-ebpf applies on top of the v4 of its own series [1]
- pve-rs needs proxmox-ebpf with the dhcp feature
- pve-network needs pve-rs, at build and run time
- qemu-server, pve-container and pve-manager need pve-network
An older pve-network fails its apply on the unknown backend, so every
node needs the new one before a zone gets ebpf.
Changes since v2:
* drop the generation counter. Every change takes the responder's
lock before it reads its input instead. An edit takes it
exclusively
* the resume at the end of a live migration pushes the guest's
mappings again on the target, so an edit made during the migration
reaches it
* new first pve-network commit, a dnsmasq fix that replaces a stale
reservation of another MAC instead of dropping it
* no MTU option for a zone without an MTU, where v2 handed out 1500
* a mapping edit without a MAC no longer reaches the backends
* rewrite commit messages, cover letter and parts of the docs
* rebase onto current master
(pre-build packages on sani)
[1] https://lore.proxmox.com/pve-devel/20261019142413.2255644-1-h.laimer@proxmox.com/
proxmox-ebpf:
Hannes Laimer (2):
dhcp: add the per-tap DHCPv4 responder program
dhcp: add the subsystem that loads the responder and keeps its records
Cargo.toml | 5 +
debian/control | 6 +-
src/bpf-shim/bpf/bpf_helpers.h | 3 +
src/bpf-shim/vmlinux.h | 1 +
src/dhcp/bpf/dhcp.bpf.c | 401 +++++++++++++
src/dhcp/bpf/types.h | 24 +
src/dhcp/mod.rs | 1031 ++++++++++++++++++++++++++++++++
src/dhcp/types.rs | 64 ++
src/lib.rs | 3 +
src/subsystem.rs | 2 +-
tests/common/mod.rs | 63 +-
tests/dhcp.rs | 825 +++++++++++++++++++++++++
12 files changed, 2422 insertions(+), 6 deletions(-)
create mode 100644 src/dhcp/bpf/dhcp.bpf.c
create mode 100644 src/dhcp/bpf/types.h
create mode 100644 src/dhcp/mod.rs
create mode 100644 src/dhcp/types.rs
create mode 100644 tests/dhcp.rs
proxmox-perl-rs:
Hannes Laimer (1):
pve-rs: sdn: add bindings for the eBPF DHCP responder
pve-rs/Cargo.toml | 2 +
pve-rs/Makefile | 3 +
pve-rs/debian/control | 2 +
pve-rs/src/bindings/sdn/dhcp.rs | 229 ++++++++++++++++++++++++++++++++
pve-rs/src/bindings/sdn/mod.rs | 1 +
5 files changed, 237 insertions(+)
create mode 100644 pve-rs/src/bindings/sdn/dhcp.rs
pve-network:
Hannes Laimer (9):
sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it
sdn: dhcp: update reservations when ip mappings change
sdn: ipam: key the MAC cache by lower-case MAC, stop caching misses
sdn: dhcp: make the lease time configurable per subnet
sdn: dhcp: require a backend only if a zone on this node uses it
sdn: dhcp: add an eBPF backend answering on each guest interface
sdn: zones: report interface plugs and unplugs to the DHCP backends
sdn: dhcp: push IP mapping edits to the guest's node
sdn: zones: allow the ebpf DHCP backend on every zone type
debian/libpve-network-perl.prerm | 13 +
src/PVE/API2/Network/SDN/Ips.pm | 10 +
src/PVE/API2/Network/SDN/Nodes/Status.pm | 37 +
src/PVE/API2/Network/SDN/Zones.pm | 11 +-
src/PVE/Network/SDN/Dhcp.pm | 200 ++-
src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 114 +-
src/PVE/Network/SDN/Dhcp/Ebpf.pm | 308 +++++
src/PVE/Network/SDN/Dhcp/Makefile | 2 +-
src/PVE/Network/SDN/Dhcp/Plugin.pm | 26 +-
src/PVE/Network/SDN/Ipams.pm | 61 +-
src/PVE/Network/SDN/SubnetPlugin.pm | 11 +
src/PVE/Network/SDN/Vnets.pm | 5 +-
src/PVE/Network/SDN/Zones.pm | 29 +-
src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 32 +
src/PVE/Network/SDN/Zones/FaucetPlugin.pm | 1 +
src/PVE/Network/SDN/Zones/QinQPlugin.pm | 7 +
src/PVE/Network/SDN/Zones/SimplePlugin.pm | 2 +-
src/PVE/Network/SDN/Zones/VlanPlugin.pm | 7 +
src/PVE/Network/SDN/Zones/VxlanPlugin.pm | 25 +
src/test/run_test_vnets_blackbox.pl | 1442 ++++++++++++++++++++-
20 files changed, 2263 insertions(+), 80 deletions(-)
create mode 100755 debian/libpve-network-perl.prerm
create mode 100644 src/PVE/Network/SDN/Dhcp/Ebpf.pm
qemu-server:
Hannes Laimer (1):
network: sdn: keep the DHCP backends current on guest NICs
src/PVE/CLI/qm.pm | 4 ++--
src/PVE/QemuServer.pm | 4 ++++
src/PVE/QemuServer/Network.pm | 19 +++++++++++++++++--
src/PVE/QemuServer/RunState.pm | 2 ++
src/usr/pve-bridge | 8 +++++++-
src/usr/pve-bridgedown | 4 ++--
6 files changed, 34 insertions(+), 7 deletions(-)
pve-container:
Hannes Laimer (1):
net: report NIC plugs and removals to the SDN DHCP backends
src/PVE/LXC.pm | 13 +++++++++++--
src/PVE/LXC/Config.pm | 3 ++-
src/lxc-pve-poststop-hook | 6 +++---
3 files changed, 16 insertions(+), 6 deletions(-)
pve-manager:
Hannes Laimer (2):
ui: sdn: add zone DHCP backend selector and subnet DHCP options
sdn: bring up the DHCP backends at boot before the guests start
bin/pve-sdn-commit | 3 +++
services/pve-sdn-commit.service | 1 +
www/manager6/sdn/SubnetEdit.js | 23 +++++++++++++++++++++++
www/manager6/sdn/zones/Base.js | 17 +++++++++++++++++
www/manager6/sdn/zones/SimpleEdit.js | 11 -----------
5 files changed, 44 insertions(+), 11 deletions(-)
pve-docs:
Hannes Laimer (1):
sdn: dhcp: add eBPF backend section
pvesdn.adoc | 101 ++++++++++++++++++++++++++++++++++++++++++----------
1 file changed, 83 insertions(+), 18 deletions(-)
Summary over all repositories:
52 files changed, 5099 insertions(+), 128 deletions(-)
--
Generated by murpp 0.12.0
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH proxmox-ebpf v3 01/17] dhcp: add the per-tap DHCPv4 responder program
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
@ 2026-10-20 7:59 ` Hannes Laimer
2026-10-20 7:59 ` [PATCH proxmox-ebpf v3 02/17] dhcp: add the subsystem that loads the responder and keeps its records Hannes Laimer
` (17 subsequent siblings)
18 siblings, 0 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 7:59 UTC (permalink / raw)
To: pve-devel
The program answers a guest's DISCOVER and REQUEST on the guest's own
tap from the record stored for that interface, so the exchange never
reaches the bridge. It keeps no lease state and needs no address of
the host in the guest's subnet. Only the NIC's own MAC gets the
record's address. A REQUEST for another address gets a NAK, which
ends a stale lease right away. Anything else passes on unchanged,
such as a REQUEST that selects another server or one from a network
the guest runs itself.
A tc program cannot create a packet, so the reply overwrites the
request and is sent out as that packet. The kernel then treats it as
it would have treated the request, and completes the UDP checksum on
the way out only if the guest sent with checksum offload. The field
has to hold the pseudo-header sum in that case and a valid checksum
otherwise, and the program cannot read which. It finds out by adding
one to a zeroed field through the kernel's checksum helper, whose
result differs between the two.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Cargo.toml | 5 +
debian/control | 6 +-
src/bpf-shim/bpf/bpf_helpers.h | 3 +
src/bpf-shim/vmlinux.h | 1 +
src/dhcp/bpf/dhcp.bpf.c | 401 ++++++++++++++++
src/dhcp/bpf/types.h | 24 +
tests/common/mod.rs | 63 ++-
tests/dhcp.rs | 825 +++++++++++++++++++++++++++++++++
8 files changed, 1323 insertions(+), 5 deletions(-)
create mode 100644 src/dhcp/bpf/dhcp.bpf.c
create mode 100644 src/dhcp/bpf/types.h
create mode 100644 tests/dhcp.rs
diff --git a/Cargo.toml b/Cargo.toml
index 1e8703d..172042e 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -11,6 +11,7 @@ homepage = "https://proxmox.com"
exclude = [".cargo", ".gitignore", "Makefile", "build", "debian"]
[features]
+dhcp = []
# When enabled, BPF programs compile with -DBPF_DEBUG so their DBG() macros
# expand to bpf_printk. Off by default
bpf-debug = []
@@ -20,3 +21,7 @@ aya = "0.13"
anyhow = "1"
log = "0.4"
nix = { version = "0.29", features = ["fs", "net"] }
+
+[[test]]
+name = "dhcp"
+required-features = ["dhcp"]
diff --git a/debian/control b/debian/control
index d3a9cc8..2dc841a 100644
--- a/debian/control
+++ b/debian/control
@@ -37,14 +37,18 @@ Depends:
Provides:
librust-proxmox-ebpf+bpf-debug-dev (= ${binary:Version}),
librust-proxmox-ebpf+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf+dhcp-dev (= ${binary:Version}),
librust-proxmox-ebpf-0-dev (= ${binary:Version}),
librust-proxmox-ebpf-0+bpf-debug-dev (= ${binary:Version}),
librust-proxmox-ebpf-0+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0+dhcp-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1+bpf-debug-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1+dhcp-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1.0-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1.0+bpf-debug-dev (= ${binary:Version}),
- librust-proxmox-ebpf-0.1.0+default-dev (= ${binary:Version})
+ librust-proxmox-ebpf-0.1.0+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1.0+dhcp-dev (= ${binary:Version})
Description: Proxmox VE eBPF - Rust source code
Source code for Debianized Rust crate "proxmox-ebpf"
diff --git a/src/bpf-shim/bpf/bpf_helpers.h b/src/bpf-shim/bpf/bpf_helpers.h
index de3bd84..3118de2 100644
--- a/src/bpf-shim/bpf/bpf_helpers.h
+++ b/src/bpf-shim/bpf/bpf_helpers.h
@@ -29,6 +29,9 @@ extern long bpf_skb_load_bytes(const void *skb, __u32 offset, void *to, __u32 le
extern long bpf_skb_store_bytes(void *skb, __u32 offset, const void *from, __u32 len, __u64 flags);
extern long bpf_skb_pull_data(void *skb, __u32 len);
extern long bpf_skb_change_tail(void *skb, __u32 new_len, __u64 flags);
+extern long bpf_l4_csum_replace(void *skb, __u32 offset, __u64 from, __u64 to, __u64 flags);
extern long bpf_redirect(__u32 ifindex, __u64 flags);
+#define BPF_F_PSEUDO_HDR 0x10
+
#endif
diff --git a/src/bpf-shim/vmlinux.h b/src/bpf-shim/vmlinux.h
index 99a7aaf..677d2ee 100644
--- a/src/bpf-shim/vmlinux.h
+++ b/src/bpf-shim/vmlinux.h
@@ -23,6 +23,7 @@ struct __sk_buff {
__u32 len;
__u32 ifindex;
__u32 mark;
+ __u32 vlan_present;
};
struct ethhdr {
diff --git a/src/dhcp/bpf/dhcp.bpf.c b/src/dhcp/bpf/dhcp.bpf.c
new file mode 100644
index 0000000..ab21ffb
--- /dev/null
+++ b/src/dhcp/bpf/dhcp.bpf.c
@@ -0,0 +1,401 @@
+// Per-tap DHCPv4 responder. Attached on tap ingress, it answers DISCOVER and
+// REQUEST on interfaces with a record by rewriting the request into the reply
+// in place and redirecting it back out the same interface. The redirect
+// consumes the request, so nothing upstream on the bridge sees it. Everything
+// else, including requests on interfaces without a record, is handed on
+// untouched to whatever runs after this program on the interface.
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_endian.h>
+#include "types.h"
+#include "bpf_debug.h"
+
+char LICENSE[] SEC("license") = "GPL";
+
+// what is left alone gets TC_ACT_UNSPEC, since TC_ACT_OK would end the
+// chain before the qdisc filters, where the rate limiter sits
+#define TC_ACT_UNSPEC (-1)
+#define TC_ACT_SHOT 2
+
+#define ETH_HLEN 14
+#define ETH_P_IP 0x0800
+#define IP_PROTO_UDP 17
+
+#define BOOTP_LEN 236
+// replies are padded up to the minimal BOOTP message size (RFC 1542), since
+// some old client stacks drop anything shorter
+#define BOOTP_MIN_LEN 300
+#define DHCP_MAGIC 0x63825363
+#define BOOTP_FLAG_BCAST 0x8000
+
+#define BOOTREQUEST 1
+#define BOOTREPLY 2
+
+#define DHCP_DISCOVER 1
+#define DHCP_OFFER 2
+#define DHCP_REQUEST 3
+#define DHCP_ACK 5
+#define DHCP_NAK 6
+
+#define OPT_PAD 0
+#define OPT_NETMASK 1
+#define OPT_ROUTER 3
+#define OPT_DNS 6
+#define OPT_MTU 26
+#define OPT_REQUESTED_IP 50
+#define OPT_LEASE 51
+#define OPT_MSGTYPE 53
+#define OPT_SERVER_ID 54
+#define OPT_WPAD 252
+#define OPT_END 255
+
+// the answer of every attached interface, keyed by ifindex
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __type(key, __u32);
+ __type(value, struct dhcp_record);
+ __uint(max_entries, 16384);
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __uint(pinning, LIBBPF_PIN_BY_NAME);
+} dhcp_iface_records SEC(".maps");
+
+struct bootp {
+ __u8 op;
+ __u8 htype;
+ __u8 hlen;
+ __u8 hops;
+ __u32 xid;
+ __u16 secs;
+ __u16 flags;
+ __u32 ciaddr;
+ __u32 yiaddr;
+ __u32 siaddr;
+ __u32 giaddr;
+ __u8 chaddr[16];
+ __u8 sname[64];
+ __u8 file[128];
+};
+
+struct reply_pkt {
+ struct iphdr ip;
+ struct udphdr udp;
+ struct bootp bp;
+ __u8 cookie[4];
+ __u8 opts[BOOTP_MIN_LEN - BOOTP_LEN - 4];
+} __attribute__((packed));
+
+// the responder has no address on the tap, so replies carry a fixed locally
+// administered source MAC that nothing ever learns
+static const __u8 SRC_MAC[6] = { 0x02, 0x00, 0x00, 0x00, 0x00, 0x01 };
+static const __u8 BCAST_MAC[6] = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff };
+
+static __always_inline __u16 csum_fold(__u32 sum)
+{
+ sum = (sum & 0xffff) + (sum >> 16);
+ sum = (sum & 0xffff) + (sum >> 16);
+ return sum;
+}
+
+static __always_inline __u16 ip_csum(const __u8 *hdr)
+{
+ __u32 sum = 0;
+#pragma unroll
+ for (int i = 0; i < 10; i++) {
+ __u16 word;
+ __builtin_memcpy(&word, hdr + 2 * i, 2);
+ sum += word;
+ }
+ return ~csum_fold(sum);
+}
+
+// the pseudo-header sum, the starting value the kernel expects in a
+// checksum it completes on the way out
+static __always_inline __u16 udp_pseudo_csum(__u32 saddr, __u32 daddr, __u16 len)
+{
+ __u32 sum = (saddr & 0xffff) + (saddr >> 16) + (daddr & 0xffff) + (daddr >> 16);
+ sum += bpf_htons(IP_PROTO_UDP) + len;
+ return csum_fold(sum);
+}
+
+#define UDP_CHECK_OFF (ETH_HLEN + sizeof(struct iphdr) + 6)
+
+SEC("classifier")
+int tc_dhcp_ingress(struct __sk_buff *skb)
+{
+ __u32 min_len = ETH_HLEN + sizeof(struct iphdr) + sizeof(struct udphdr) + BOOTP_LEN + 4;
+
+ // the kernel strips a VLAN tag before this hook. A request the guest
+ // tagged itself belongs to a network of the guest's own and passes
+ if (skb->vlan_present)
+ return TC_ACT_UNSPEC;
+
+ // everything shorter is not a DHCP request
+ if (skb->len < min_len)
+ return TC_ACT_UNSPEC;
+
+ // the headers are read in place. A large packet from the guest is mostly
+ // outside the linear part, and pulling it in would cost every one of them
+ struct {
+ struct ethhdr eth;
+ struct iphdr ip;
+ struct udphdr udp;
+ } __attribute__((packed)) hdr;
+ if (bpf_skb_load_bytes(skb, 0, &hdr, sizeof(hdr)))
+ return TC_ACT_UNSPEC;
+ if (hdr.eth.h_proto != bpf_htons(ETH_P_IP))
+ return TC_ACT_UNSPEC;
+ if (hdr.ip.version != 4 || hdr.ip.ihl != 5 || hdr.ip.protocol != IP_PROTO_UDP)
+ return TC_ACT_UNSPEC;
+ // a fragment is never a whole request, and past the first one the bytes at
+ // the UDP header offset are arbitrary payload
+ if (hdr.ip.frag_off & bpf_htons(0x3fff))
+ return TC_ACT_UNSPEC;
+ if (hdr.udp.dest != bpf_htons(67))
+ return TC_ACT_UNSPEC;
+
+ void *data = (void *)(long)skb->data;
+ void *data_end = (void *)(long)skb->data_end;
+
+ if (data + min_len > data_end) {
+ // a request may sit in a non-linear skb
+ if (bpf_skb_pull_data(skb, min_len))
+ return TC_ACT_UNSPEC;
+ data = (void *)(long)skb->data;
+ data_end = (void *)(long)skb->data_end;
+ if (data + min_len > data_end)
+ return TC_ACT_UNSPEC;
+ }
+
+ struct iphdr *ip = data + ETH_HLEN;
+
+ struct bootp *req = data + ETH_HLEN + sizeof(struct iphdr) + sizeof(struct udphdr);
+ if (req->op != BOOTREQUEST || req->htype != 1 || req->hlen != 6)
+ return TC_ACT_UNSPEC;
+
+ // a reply to a relayed request must go unicast to the relay instead of the
+ // client. Those are left to a server that speaks relay
+ if (req->giaddr) {
+ DBG("dhcp: request relayed via %pI4 passed", &req->giaddr);
+ return TC_ACT_UNSPEC;
+ }
+
+ __u32 *magic = (void *)req + BOOTP_LEN;
+ if (*magic != bpf_htonl(DHCP_MAGIC))
+ return TC_ACT_UNSPEC;
+
+ // only the NIC's own MAC gets the interface's answer. The client identifier
+ // option is not honored either, so the reply goes back to the hardware
+ // address the request came with. That is manual allocation in RFC 2131
+ // terms
+ __u8 chaddr[6];
+ __builtin_memcpy(chaddr, req->chaddr, 6);
+ __u32 ifindex = skb->ifindex;
+ struct dhcp_record *rec = bpf_map_lookup_elem(&dhcp_iface_records, &ifindex);
+ // a marker only names its vnet and serves nothing
+ if (rec && !rec->ip)
+ rec = (void *)0;
+ if (!rec) {
+ DBG("dhcp: no record for ifindex %d", ifindex);
+ return TC_ACT_UNSPEC;
+ }
+ for (int i = 0; i < 6; i++) {
+ if (chaddr[i] != rec->mac[i]) {
+ DBG("dhcp: chaddr is not the NIC's on ifindex %d", ifindex);
+ return TC_ACT_UNSPEC;
+ }
+ }
+
+ __u8 msgtype = 0;
+ __u32 requested_ip = 0;
+ __u32 server_id = 0;
+ __u32 off = min_len;
+ // the options end with the UDP datagram. Whatever the IP datagram or the
+ // frame carries beyond it is padding or a trailer, not the guest's request
+ __u32 end = ETH_HLEN + sizeof(struct iphdr) + bpf_ntohs(hdr.udp.len);
+ if (end > ETH_HLEN + bpf_ntohs(ip->tot_len))
+ end = ETH_HLEN + bpf_ntohs(ip->tot_len);
+ if (end > skb->len)
+ end = skb->len;
+
+ for (int i = 0; i < 128; i++) {
+ __u8 code, len;
+ if (off >= end || bpf_skb_load_bytes(skb, off, &code, 1))
+ break;
+ if (code == OPT_END)
+ break;
+ if (code == OPT_PAD) {
+ off += 1;
+ continue;
+ }
+ if (off + 2 > end || bpf_skb_load_bytes(skb, off + 1, &len, 1))
+ break;
+ if (off + 2 + len > end)
+ break;
+ if (code == OPT_MSGTYPE && len == 1)
+ bpf_skb_load_bytes(skb, off + 2, &msgtype, 1);
+ else if (code == OPT_REQUESTED_IP && len == 4)
+ bpf_skb_load_bytes(skb, off + 2, &requested_ip, 4);
+ else if (code == OPT_SERVER_ID && len == 4)
+ bpf_skb_load_bytes(skb, off + 2, &server_id, 4);
+ off += 2 + len;
+ }
+
+ __u8 reply;
+ if (msgtype == DHCP_DISCOVER) {
+ reply = DHCP_OFFER;
+ } else if (msgtype == DHCP_REQUEST) {
+ // a REQUEST naming another server declines our offer and is
+ // answered, if at all, by that server
+ if (server_id && server_id != rec->server_id) {
+ DBG("dhcp: request selected server %pI4, passed", &server_id);
+ return TC_ACT_UNSPEC;
+ }
+ // a NAK on a stale address makes the client re-DISCOVER instead of
+ // keeping the old address until its lease ends
+ __u32 want = requested_ip ? requested_ip : req->ciaddr;
+ reply = want == rec->ip ? DHCP_ACK : DHCP_NAK;
+ if (reply == DHCP_NAK)
+ DBG("dhcp: request for %pI4 against record %pI4, nak", &want, &rec->ip);
+ } else {
+ DBG("dhcp: message type %d passed", msgtype);
+ return TC_ACT_UNSPEC;
+ }
+
+ __u32 xid = req->xid;
+ __u16 flags = req->flags;
+ __u32 ciaddr = req->ciaddr;
+
+ struct reply_pkt buf = {};
+
+ buf.bp.op = BOOTREPLY;
+ buf.bp.htype = 1;
+ buf.bp.hlen = 6;
+ buf.bp.xid = xid;
+ buf.bp.flags = flags;
+ __builtin_memcpy(buf.bp.chaddr, chaddr, 6);
+
+ buf.cookie[0] = 0x63;
+ buf.cookie[1] = 0x82;
+ buf.cookie[2] = 0x53;
+ buf.cookie[3] = 0x63;
+
+ __u32 o = 0;
+ buf.opts[o++] = OPT_MSGTYPE;
+ buf.opts[o++] = 1;
+ buf.opts[o++] = reply;
+ buf.opts[o++] = OPT_SERVER_ID;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->server_id, 4);
+ o += 4;
+
+ if (reply != DHCP_NAK) {
+ buf.bp.yiaddr = rec->ip;
+
+ buf.opts[o++] = OPT_LEASE;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->lease, 4);
+ o += 4;
+ buf.opts[o++] = OPT_NETMASK;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->netmask, 4);
+ o += 4;
+ if (rec->router) {
+ buf.opts[o++] = OPT_ROUTER;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->router, 4);
+ o += 4;
+ }
+ if (rec->dns) {
+ buf.opts[o++] = OPT_DNS;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->dns, 4);
+ o += 4;
+ }
+ if (rec->mtu) {
+ buf.opts[o++] = OPT_MTU;
+ buf.opts[o++] = 2;
+ __builtin_memcpy(&buf.opts[o], &rec->mtu, 2);
+ o += 2;
+ }
+ // an empty WPAD answer, without it some Windows versions
+ // probe for a proxy config indefinitely
+ buf.opts[o++] = OPT_WPAD;
+ buf.opts[o++] = 1;
+ buf.opts[o++] = '\n';
+ }
+ buf.opts[o++] = OPT_END;
+
+ // the zeroed tail of opts pads short replies out to the minimum, and on
+ // the wire those zeros read as pad options after the end option
+ __u32 bootp_out = BOOTP_LEN + 4 + o;
+ if (bootp_out < BOOTP_MIN_LEN)
+ bootp_out = BOOTP_MIN_LEN;
+
+ __u32 payload = sizeof(struct udphdr) + bootp_out;
+
+ buf.ip.version = 4;
+ buf.ip.ihl = 5;
+ buf.ip.ttl = 64;
+ buf.ip.protocol = IP_PROTO_UDP;
+ buf.ip.tot_len = bpf_htons(sizeof(struct iphdr) + payload);
+ buf.ip.saddr = rec->server_id;
+ // a renewing client has an address and gets a unicast answer.
+ // Everything else cannot be addressed yet
+ buf.ip.daddr = (reply != DHCP_NAK && ciaddr) ? ciaddr : 0xffffffff;
+ buf.ip.check = ip_csum((const __u8 *)&buf.ip);
+
+ buf.udp.source = bpf_htons(67);
+ buf.udp.dest = bpf_htons(68);
+ buf.udp.len = bpf_htons(payload);
+ buf.udp.check = 0;
+
+ __u32 wlen = sizeof(struct iphdr) + payload;
+ if (wlen > sizeof(buf))
+ return TC_ACT_UNSPEC;
+
+ if (bpf_skb_change_tail(skb, ETH_HLEN + wlen, 0))
+ return TC_ACT_UNSPEC;
+ // the request is gone from here on, so a failed write drops the frame
+ // rather than forwarding what is left of it
+
+ // NAKs are always broadcast, and a client without an address that set the
+ // BROADCAST flag asked for a broadcast reply
+ __u8 eth_hdr[ETH_HLEN];
+ if (reply == DHCP_NAK || (!ciaddr && (flags & bpf_htons(BOOTP_FLAG_BCAST)))) {
+ __builtin_memcpy(eth_hdr, BCAST_MAC, 6);
+ } else {
+ __builtin_memcpy(eth_hdr, chaddr, 6);
+ }
+ __builtin_memcpy(eth_hdr + 6, SRC_MAC, 6);
+ eth_hdr[12] = 0x08;
+ eth_hdr[13] = 0x00;
+
+ if (bpf_skb_store_bytes(skb, 0, eth_hdr, ETH_HLEN, 0))
+ return TC_ACT_SHOT;
+ if (bpf_skb_store_bytes(skb, ETH_HLEN, &buf, wlen, 0))
+ return TC_ACT_SHOT;
+
+ // The reply reuses the request's skb, whose checksum state the program
+ // cannot see. A request sent with checksum offload is completed by the kernel
+ // on the way out, and it expects the pseudo-header sum in the field. Any
+ // other request needs the field valid as it is, and zero is valid. The
+ // helper below folds a checksum field once more when the kernel still
+ // has to complete it, so adding one to the zeroed field ends up as 1 in
+ // that state and as its complement otherwise. The low flag bits give
+ // the width of the field.
+ if (bpf_l4_csum_replace(skb, UDP_CHECK_OFF, 0, 1, BPF_F_PSEUDO_HDR | 2))
+ return TC_ACT_SHOT;
+ __u16 probe = 0;
+ if (bpf_skb_load_bytes(skb, UDP_CHECK_OFF, &probe, 2))
+ return TC_ACT_SHOT;
+ __u16 check = probe == 1 ? udp_pseudo_csum(buf.ip.saddr, buf.ip.daddr, buf.udp.len) : 0;
+ if (bpf_skb_store_bytes(skb, UDP_CHECK_OFF, &check, 2, 0))
+ return TC_ACT_SHOT;
+
+#ifdef BPF_DEBUG
+ __u32 yiaddr = buf.bp.yiaddr;
+ DBG("dhcp: reply type %d with %pI4 to ifindex %d", reply, &yiaddr, skb->ifindex);
+#endif
+ return bpf_redirect(skb->ifindex, 0);
+}
diff --git a/src/dhcp/bpf/types.h b/src/dhcp/bpf/types.h
new file mode 100644
index 0000000..7da6969
--- /dev/null
+++ b/src/dhcp/bpf/types.h
@@ -0,0 +1,24 @@
+#ifndef PROXMOX_EBPF_DHCP_TYPES_H
+#define PROXMOX_EBPF_DHCP_TYPES_H
+
+// Keep in sync with ../types.rs
+// __u8/__u16/__u32 come from vmlinux.h
+
+// addresses, lease and mtu are stored in network byte order, so a
+// reply copies them into the packet as-is
+struct dhcp_record {
+ __u32 ip; // 0 = removed, the entry is a marker naming its vnet
+ __u32 netmask;
+ __u32 router; // 0 = not served
+ __u32 dns; // 0 = not served
+ __u32 server_id;
+ __u32 lease;
+ __u16 mtu; // 0 = not served
+ __u8 vnet[8]; // the vnet the writer saw the interface on, zero for none, never read here
+ __u8 mac[6]; // the NIC's, a request carrying another is passed on
+};
+
+_Static_assert(sizeof(struct dhcp_record) == 40, "dhcp_record layout");
+_Static_assert(__builtin_offsetof(struct dhcp_record, mac) == 34, "dhcp_record layout");
+
+#endif
diff --git a/tests/common/mod.rs b/tests/common/mod.rs
index 67fac33..c8fd285 100644
--- a/tests/common/mod.rs
+++ b/tests/common/mod.rs
@@ -27,13 +27,16 @@ pub struct SkBuff {
pub len: u32,
pub ifindex: u32,
pub mark: u32,
+ pub vlan_present: u32,
}
/// The skb handed to a program plus the buffer behind it. The helpers get the skb pointer and
-/// cast back to this, so the skb must stay the first field.
+/// cast back to this, so the skb must stay the first field. `partial` models a checksum the
+/// kernel still has to complete, the state a request sent with checksum offload carries.
#[repr(C)]
pub struct TestSkb {
skb: SkBuff,
+ pub partial: bool,
buf: Box<[u8; PKT_CAP]>,
}
@@ -49,7 +52,9 @@ impl TestSkb {
len: packet.len() as u32,
ifindex,
mark: 0,
+ vlan_present: 0,
},
+ partial: false,
buf,
};
t.sync();
@@ -62,6 +67,11 @@ impl TestSkb {
self.skb.data_end = base + self.skb.len as usize;
}
+ /// The frame arrived tagged, and the kernel keeps the tag beside the data.
+ pub fn tag_vlan(&mut self) {
+ self.skb.vlan_present = 1;
+ }
+
pub fn packet(&self) -> &[u8] {
&self.buf[..self.skb.len as usize]
}
@@ -184,9 +194,10 @@ pub extern "C" fn bpf_skb_pull_data(skb: *mut c_void, len: u32) -> c_long {
#[unsafe(no_mangle)]
pub extern "C" fn bpf_skb_change_tail(skb: *mut c_void, new_len: u32, _flags: u64) -> c_long {
let t = unsafe { testskb(skb) };
- // the kernel refuses to cut into the headers it knows, which is the link header here.
- // Growing stops at the harness buffer
- let min_len = 14;
+ // the kernel refuses to cut into the headers it knows. tun sets the transport header on
+ // every frame it hands in, so an IPv4 UDP frame keeps 34 bytes. One with checksum offload
+ // keeps its checksum field as well. Growing stops at the harness buffer
+ let min_len = if t.partial { 42 } else { 34 };
if new_len < min_len || new_len as usize > PKT_CAP {
return -EINVAL;
}
@@ -200,6 +211,50 @@ pub extern "C" fn bpf_skb_change_tail(skb: *mut c_void, new_len: u32, _flags: u6
0
}
+// the kernel's ones' complement primitives, csum_fold complements its result
+fn csum_add(a: u32, b: u32) -> u32 {
+ let (r, carry) = a.overflowing_add(b);
+ r + carry as u32
+}
+fn csum_fold(sum: u32) -> u16 {
+ let mut s = (sum & 0xffff) + (sum >> 16);
+ s = (s & 0xffff) + (s >> 16);
+ !(s as u16)
+}
+
+/// The kernel's field update, inet_proto_csum_replace4 for a 16-bit field. A partial checksum
+/// gets the difference folded in, and any other is complemented around the update. So the same
+/// call ends in different values.
+#[unsafe(no_mangle)]
+pub extern "C" fn bpf_l4_csum_replace(
+ skb: *mut c_void,
+ offset: u32,
+ from: u64,
+ to: u64,
+ flags: u64,
+) -> c_long {
+ let t = unsafe { testskb(skb) };
+ let offset = offset as usize;
+ assert_eq!(flags & 0xf, 2, "the responder updates the 16-bit field");
+ assert!(
+ flags & 0x10 != 0,
+ "the responder passes the pseudo-header flag"
+ );
+ if offset + 2 > t.skb.len as usize {
+ return -EFAULT;
+ }
+ let field = u16::from_ne_bytes([t.buf[offset], t.buf[offset + 1]]) as u32;
+ // a 16-bit field takes 16-bit values, so the kernel narrows both before the update
+ let (from, to) = (from as u16 as u32, to as u16 as u32);
+ let new = if t.partial {
+ !csum_fold(csum_add(csum_add(field, !from), to))
+ } else {
+ csum_fold(csum_add(csum_add(!field, !from), to))
+ };
+ t.buf[offset..offset + 2].copy_from_slice(&new.to_ne_bytes());
+ 0
+}
+
#[unsafe(no_mangle)]
pub extern "C" fn bpf_redirect(ifindex: u32, flags: u64) -> c_long {
REDIRECTED.with(|r| *r.borrow_mut() = Some((ifindex, flags)));
diff --git a/tests/dhcp.rs b/tests/dhcp.rs
new file mode 100644
index 0000000..8a00392
--- /dev/null
+++ b/tests/dhcp.rs
@@ -0,0 +1,825 @@
+//! The dhcp responder program run natively, crafted requests in and the rewritten reply out.
+
+mod common;
+
+use std::ffi::{c_int, c_void};
+
+use common::*;
+
+unsafe extern "C" {
+ fn tc_dhcp_ingress(skb: *mut SkBuff) -> c_int;
+ static dhcp_iface_records: u8;
+}
+
+const MAC: [u8; 6] = [0xda, 0x65, 0x8f, 0x18, 0x9b, 0x6f];
+const SRC_MAC: [u8; 6] = [0x02, 0x00, 0x00, 0x00, 0x00, 0x01];
+const BCAST: [u8; 6] = [0xff; 6];
+const IFINDEX: u32 = 42;
+
+const IP: [u8; 4] = [10, 0, 0, 100];
+const MASK: [u8; 4] = [255, 255, 255, 0];
+const GW: [u8; 4] = [10, 0, 0, 1];
+const DNS: [u8; 4] = [1, 1, 1, 1];
+const LEASE: u32 = 300;
+const MTU: u16 = 1500;
+
+const OPTS_OFF: usize = 14 + 20 + 8 + 236 + 4;
+
+const DISCOVER: &[u8] = &[53, 1, 1];
+const REQUEST: &[u8] = &[53, 1, 3];
+
+fn records_map() -> *const c_void {
+ (unsafe { &dhcp_iface_records } as *const u8).cast()
+}
+
+// the record map of a node, still empty
+fn setup_maps() {
+ register_map(records_map(), 4);
+}
+
+// the record map is keyed by the interface the request arrives on
+fn key() -> [u8; 4] {
+ IFINDEX.to_ne_bytes()
+}
+
+fn record(
+ ip: [u8; 4],
+ mask: [u8; 4],
+ router: [u8; 4],
+ dns: [u8; 4],
+ server: [u8; 4],
+ lease: u32,
+ mtu: u16,
+) -> [u8; 40] {
+ let mut r = [0u8; 40];
+ r[0..4].copy_from_slice(&ip);
+ r[4..8].copy_from_slice(&mask);
+ r[8..12].copy_from_slice(&router);
+ r[12..16].copy_from_slice(&dns);
+ r[16..20].copy_from_slice(&server);
+ r[20..24].copy_from_slice(&lease.to_be_bytes());
+ r[24..26].copy_from_slice(&mtu.to_be_bytes());
+ r[26..34].copy_from_slice(b"testvnet");
+ r[34..40].copy_from_slice(&MAC);
+ r
+}
+
+fn setup_full_record() {
+ setup_maps();
+ map_insert(
+ records_map(),
+ &key(),
+ &record(IP, MASK, GW, DNS, GW, LEASE, MTU),
+ );
+}
+
+struct Request {
+ mac: [u8; 6],
+ xid: [u8; 4],
+ flags: [u8; 2],
+ ciaddr: [u8; 4],
+ giaddr: [u8; 4],
+ options: Vec<u8>,
+ dport: u16,
+ ethertype: [u8; 2],
+ ipproto: u8,
+ frag_off: [u8; 2],
+ ip_options: Vec<u8>,
+ op: u8,
+ htype: u8,
+ hlen: u8,
+ end: bool,
+ trailer: Vec<u8>,
+ // claimed on top of the real IP total length
+ tot_len_excess: u16,
+}
+
+impl Default for Request {
+ fn default() -> Self {
+ Request {
+ mac: MAC,
+ xid: [0xde, 0xad, 0xbe, 0xef],
+ flags: [0, 0],
+ ciaddr: [0; 4],
+ giaddr: [0; 4],
+ options: DISCOVER.to_vec(),
+ dport: 67,
+ ethertype: [0x08, 0x00],
+ ipproto: 17,
+ frag_off: [0, 0],
+ ip_options: Vec::new(),
+ op: 1,
+ htype: 1,
+ hlen: 6,
+ end: true,
+ trailer: Vec::new(),
+ tot_len_excess: 0,
+ }
+ }
+}
+
+impl Request {
+ fn build(&self) -> Vec<u8> {
+ assert_eq!(self.ip_options.len() % 4, 0);
+ let ihl = 5 + self.ip_options.len() / 4;
+ let payload = 8 + 236 + 4 + self.options.len() + usize::from(self.end);
+ let mut p = Vec::new();
+ // ethernet
+ p.extend_from_slice(&BCAST);
+ p.extend_from_slice(&self.mac);
+ p.extend_from_slice(&self.ethertype);
+ // ipv4
+ p.extend_from_slice(&[0x40 | ihl as u8, 0]);
+ p.extend_from_slice(&((ihl * 4 + payload) as u16 + self.tot_len_excess).to_be_bytes());
+ p.extend_from_slice(&[0, 0]); // id
+ p.extend_from_slice(&self.frag_off);
+ p.extend_from_slice(&[64, self.ipproto, 0, 0]);
+ p.extend_from_slice(&[0, 0, 0, 0]); // saddr
+ p.extend_from_slice(&[255, 255, 255, 255]); // daddr
+ p.extend_from_slice(&self.ip_options);
+ // udp
+ p.extend_from_slice(&68u16.to_be_bytes());
+ p.extend_from_slice(&self.dport.to_be_bytes());
+ p.extend_from_slice(&(payload as u16).to_be_bytes());
+ p.extend_from_slice(&[0, 0]);
+ // bootp
+ p.extend_from_slice(&[self.op, self.htype, self.hlen, 0]);
+ p.extend_from_slice(&self.xid);
+ p.extend_from_slice(&[0, 0]); // secs
+ p.extend_from_slice(&self.flags);
+ p.extend_from_slice(&self.ciaddr);
+ p.extend_from_slice(&[0; 8]); // yiaddr, siaddr
+ p.extend_from_slice(&self.giaddr);
+ p.extend_from_slice(&self.mac);
+ p.extend_from_slice(&[0; 10]); // chaddr padding
+ p.extend_from_slice(&[0; 192]); // sname, file
+ // cookie and options
+ p.extend_from_slice(&[0x63, 0x82, 0x53, 0x63]);
+ p.extend_from_slice(&self.options);
+ if self.end {
+ p.push(255);
+ }
+ // bytes past the datagram, padding or a trailer the frame carries
+ p.extend_from_slice(&self.trailer);
+ p
+ }
+
+ fn run(&self) -> (c_int, TestSkb) {
+ self.run_on(IFINDEX)
+ }
+
+ fn run_on(&self, ifindex: u32) -> (c_int, TestSkb) {
+ let mut skb = TestSkb::new(&self.build(), ifindex);
+ let verdict = skb.run(tc_dhcp_ingress);
+ (verdict, skb)
+ }
+}
+
+fn parse_opts(pkt: &[u8]) -> Vec<(u8, Vec<u8>)> {
+ let mut opts = Vec::new();
+ let mut off = OPTS_OFF;
+ loop {
+ let code = pkt[off];
+ if code == 255 {
+ break;
+ }
+ if code == 0 {
+ off += 1;
+ continue;
+ }
+ let len = pkt[off + 1] as usize;
+ opts.push((code, pkt[off + 2..off + 2 + len].to_vec()));
+ off += 2 + len;
+ }
+ opts
+}
+
+fn ip_csum_ok(pkt: &[u8]) -> bool {
+ let mut sum = 0u32;
+ for w in pkt[14..34].chunks(2) {
+ sum += u32::from(u16::from_be_bytes([w[0], w[1]]));
+ }
+ while sum > 0xffff {
+ sum = (sum & 0xffff) + (sum >> 16);
+ }
+ sum == 0xffff
+}
+
+/// The offset just past the end option of a reply.
+fn opts_end(pkt: &[u8]) -> usize {
+ let mut off = OPTS_OFF;
+ loop {
+ match pkt[off] {
+ 255 => return off + 1,
+ 0 => off += 1,
+ _ => off += 2 + pkt[off + 1] as usize,
+ }
+ }
+}
+
+/// What every reply has to look like whatever the request was. The addressing and the answer
+/// itself are the caller's to check.
+fn assert_reply_invariants(pkt: &[u8]) {
+ assert_eq!(&pkt[6..12], &SRC_MAC, "eth src");
+ assert_eq!(&pkt[12..14], &[0x08, 0x00], "ethertype");
+
+ assert_eq!(pkt[14], 0x45, "ihl/version");
+ assert_eq!(pkt[22], 64, "ttl");
+ assert_eq!(pkt[23], 17, "protocol");
+ assert_eq!(&pkt[26..30], &GW, "ip src is the server id");
+ assert!(ip_csum_ok(pkt), "ip checksum");
+ let tot_len = u16::from_be_bytes([pkt[16], pkt[17]]) as usize;
+ assert_eq!(tot_len, pkt.len() - 14, "ip total length");
+
+ assert_eq!(&pkt[34..36], &67u16.to_be_bytes(), "udp sport");
+ assert_eq!(&pkt[36..38], &68u16.to_be_bytes(), "udp dport");
+ let udp_len = u16::from_be_bytes([pkt[38], pkt[39]]) as usize;
+ assert_eq!(udp_len, pkt.len() - 34, "udp length");
+ assert_eq!(&pkt[40..42], &[0, 0], "udp checksum zero, none to complete");
+ assert!(pkt.len() - 42 >= 300, "bootp minimum message size");
+
+ assert_eq!(&pkt[42..45], &[2, 1, 6], "bootp reply op, htype, hlen");
+ assert_eq!(&pkt[46..50], &[0xde, 0xad, 0xbe, 0xef], "xid preserved");
+ assert_eq!(&pkt[70..76], &MAC, "chaddr");
+ assert_eq!(&pkt[278..282], &[0x63, 0x82, 0x53, 0x63], "cookie");
+ assert!(
+ matches!(parse_opts(pkt).first(), Some((53, t)) if t.len() == 1 && [2, 5, 6].contains(&t[0])),
+ "message type leads the options"
+ );
+ let end = opts_end(pkt);
+ assert!(
+ pkt[end..].iter().all(|&b| b == 0),
+ "padding after the end option is zero"
+ );
+}
+
+fn assert_reply_headers(pkt: &[u8], eth_dst: [u8; 6], ip_dst: [u8; 4], yiaddr: [u8; 4]) {
+ assert_reply_invariants(pkt);
+ assert_eq!(&pkt[0..6], ð_dst, "eth dst");
+ assert_eq!(&pkt[30..34], &ip_dst, "ip dst");
+ assert_eq!(&pkt[58..62], &yiaddr, "yiaddr");
+}
+
+#[test]
+fn discover_gets_full_offer() {
+ setup_full_record();
+ let (verdict, skb) = Request::default().run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_eq!(redirected(), Some(IFINDEX));
+ assert_eq!(
+ redirect_flags(),
+ Some(0),
+ "the reply leaves through the tap"
+ );
+
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, MAC, [255, 255, 255, 255], IP);
+ assert_eq!(
+ parse_opts(pkt),
+ vec![
+ (53, vec![2]),
+ (54, GW.to_vec()),
+ (51, LEASE.to_be_bytes().to_vec()),
+ (1, MASK.to_vec()),
+ (3, GW.to_vec()),
+ (6, DNS.to_vec()),
+ (26, MTU.to_be_bytes().to_vec()),
+ (252, vec![b'\n']),
+ ],
+ );
+}
+
+#[test]
+fn offer_skips_unset_options() {
+ setup_maps();
+ map_insert(
+ records_map(),
+ &key(),
+ &record(IP, MASK, [0; 4], [0; 4], GW, LEASE, 0),
+ );
+ let (verdict, skb) = Request::default().run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let codes: Vec<u8> = parse_opts(skb.packet()).iter().map(|(c, _)| *c).collect();
+ assert_eq!(codes, vec![53, 54, 51, 1, 252]);
+}
+
+#[test]
+fn matching_request_gets_ack() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[50, 4]);
+ options.extend_from_slice(&IP);
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, MAC, [255, 255, 255, 255], IP);
+ assert_eq!(parse_opts(pkt)[0], (53, vec![5]));
+}
+
+// the pseudo-header sum, the starting value the kernel expects in a checksum it completes on
+// the way out
+fn udp_pseudo(pkt: &[u8]) -> [u8; 2] {
+ let words = [
+ u16::from_be_bytes([pkt[26], pkt[27]]),
+ u16::from_be_bytes([pkt[28], pkt[29]]),
+ u16::from_be_bytes([pkt[30], pkt[31]]),
+ u16::from_be_bytes([pkt[32], pkt[33]]),
+ 17,
+ u16::from_be_bytes([pkt[38], pkt[39]]),
+ ];
+ let mut s: u32 = words.iter().map(|&w| w as u32).sum();
+ while s >> 16 != 0 {
+ s = (s & 0xffff) + (s >> 16);
+ }
+ (s as u16).to_be_bytes()
+}
+
+#[test]
+fn offloaded_request_gets_the_pseudo_header_sum() {
+ setup_full_record();
+ let mut skb = TestSkb::new(&Request::default().build(), IFINDEX);
+ skb.partial = true;
+ assert_eq!(skb.run(tc_dhcp_ingress), TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_eq!(
+ &pkt[40..42],
+ &udp_pseudo(pkt),
+ "udp checksum holds the pseudo-header sum"
+ );
+ assert_ne!(&pkt[40..42], &[0, 0]);
+}
+
+#[test]
+fn removed_record_is_not_answered() {
+ setup_maps();
+ map_insert(records_map(), &key(), &[0u8; 40]);
+ let (verdict, _) = Request::default().run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+}
+
+#[test]
+fn tagged_request_passes() {
+ setup_full_record();
+ let mut skb = TestSkb::new(&Request::default().build(), IFINDEX);
+ skb.tag_vlan();
+ assert_eq!(skb.run(tc_dhcp_ingress), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn trailer_past_the_datagram_is_not_read_as_options() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[50, 4]);
+ options.extend_from_slice(&IP);
+ // no END option, and the frame goes on with bytes shaped like a server
+ // identifier naming another server
+ let (verdict, skb) = Request {
+ options,
+ end: false,
+ trailer: vec![54, 4, 10, 9, 9, 9],
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_eq!(parse_opts(skb.packet())[0], (53, vec![5]));
+}
+
+#[test]
+fn bytes_past_the_udp_datagram_are_not_read_as_options() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[50, 4]);
+ options.extend_from_slice(&IP);
+ // the IP length claims the trailer and the UDP length does not. The
+ // trailer is shaped like a server identifier naming another server
+ let (verdict, skb) = Request {
+ options,
+ end: false,
+ trailer: vec![54, 4, 10, 9, 9, 9],
+ tot_len_excess: 6,
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_eq!(parse_opts(skb.packet())[0], (53, vec![5]));
+}
+
+#[test]
+fn claimed_length_past_the_frame_is_clamped() {
+ // the message type is followed by a second one cut off by the frame. A walk trusting the
+ // claimed length would read past the frame and take the zeroed result as the type
+ setup_full_record();
+ let mut options = DISCOVER.to_vec();
+ options.extend_from_slice(&[53, 1]);
+ let req = Request {
+ options,
+ end: false,
+ tot_len_excess: 1,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_reply_invariants(skb.packet());
+}
+
+#[test]
+fn request_without_options_passes_untouched() {
+ setup_full_record();
+ let req = Request {
+ options: Vec::new(),
+ end: false,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(skb.packet(), &req.build()[..]);
+}
+
+#[test]
+fn stale_request_gets_nak() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[50, 4, 10, 0, 0, 77]);
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, BCAST, [255, 255, 255, 255], [0; 4]);
+ assert_eq!(parse_opts(pkt), vec![(53, vec![6]), (54, GW.to_vec())]);
+}
+
+#[test]
+fn renewal_is_answered_unicast() {
+ setup_full_record();
+ let (verdict, skb) = Request {
+ options: REQUEST.to_vec(),
+ ciaddr: IP,
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, MAC, IP, IP);
+ assert_eq!(parse_opts(pkt)[0], (53, vec![5]));
+}
+
+#[test]
+fn request_selecting_another_server_passes_untouched() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[54, 4, 192, 168, 7, 7]);
+ options.extend_from_slice(&[50, 4, 10, 0, 0, 77]);
+ let req = Request {
+ options,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+}
+
+#[test]
+fn relayed_request_passes_untouched() {
+ setup_full_record();
+ let req = Request {
+ giaddr: [10, 0, 0, 254],
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+}
+
+#[test]
+fn fragments_pass_untouched() {
+ setup_full_record();
+ for frag_off in [[0x20, 0x00], [0x00, 0x01]] {
+ let req = Request {
+ frag_off,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(skb.packet(), &req.build()[..]);
+ }
+
+ let (verdict, _) = Request {
+ frag_off: [0x40, 0x00],
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(verdict, TC_ACT_REDIRECT, "DF alone is not a fragment");
+}
+
+#[test]
+fn broadcast_flag_gets_l2_broadcast_reply() {
+ setup_full_record();
+ let (verdict, skb) = Request {
+ flags: [0x80, 0],
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, BCAST, [255, 255, 255, 255], IP);
+ assert_eq!(&pkt[52..54], &[0x80, 0], "flags echoed");
+
+ // ciaddr wins over the flag, since a renewing client is addressable
+ let (verdict, skb) = Request {
+ flags: [0x80, 0],
+ ciaddr: IP,
+ options: REQUEST.to_vec(),
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_reply_headers(skb.packet(), MAC, IP, IP);
+}
+
+#[test]
+fn interface_without_a_record_passes_untouched() {
+ setup_maps();
+ let req = Request::default();
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+
+ // the record is one interface's, so a request arriving on any other gets nothing from it
+ setup_full_record();
+ let (verdict, skb) = req.run_on(IFINDEX + 1);
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+}
+
+#[test]
+fn another_mac_on_the_interface_passes_untouched() {
+ // a request carrying another MAC than the NIC's is left to the bridge
+ setup_full_record();
+ let other = [0x02, 0x11, 0x22, 0x33, 0x44, 0x55];
+ let req = Request {
+ mac: other,
+ ..Request::default()
+ };
+ let before = req.build();
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(skb.packet(), &before[..], "untouched");
+}
+
+#[test]
+fn non_dhcp_traffic_passes_untouched() {
+ setup_full_record();
+ let variants = [
+ Request {
+ dport: 53,
+ ..Request::default()
+ },
+ Request {
+ ethertype: [0x86, 0xdd],
+ ..Request::default()
+ },
+ Request {
+ ipproto: 6,
+ ..Request::default()
+ },
+ ];
+ for req in variants {
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(skb.packet(), &req.build()[..]);
+ }
+
+ let frame = Request::default().build();
+ let mut skb = TestSkb::new(&frame[..100], IFINDEX);
+ assert_eq!(skb.run(tc_dhcp_ingress), TC_ACT_UNSPEC);
+
+ let mut corrupted = frame.clone();
+ corrupted[278] = 0;
+ let mut skb = TestSkb::new(&corrupted, IFINDEX);
+ assert_eq!(skb.run(tc_dhcp_ingress), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn options_walk_handles_padding_and_early_end() {
+ setup_full_record();
+
+ let mut options = vec![0, 0, 0];
+ options.extend_from_slice(DISCOVER);
+ let (verdict, _) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+
+ let mut options = vec![255];
+ options.extend_from_slice(DISCOVER);
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(
+ verdict, TC_ACT_UNSPEC,
+ "message type after end is not parsed"
+ );
+ assert_eq!(redirected(), None);
+ drop(skb);
+}
+
+#[test]
+fn stale_renewal_gets_broadcast_nak() {
+ setup_full_record();
+ let (verdict, skb) = Request {
+ options: REQUEST.to_vec(),
+ ciaddr: [10, 0, 0, 77],
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, BCAST, [255, 255, 255, 255], [0; 4]);
+ assert_eq!(parse_opts(pkt), vec![(53, vec![6]), (54, GW.to_vec())]);
+}
+
+#[test]
+fn request_without_address_gets_nak() {
+ setup_full_record();
+ let (verdict, skb) = Request {
+ options: REQUEST.to_vec(),
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, BCAST, [255, 255, 255, 255], [0; 4]);
+ assert_eq!(parse_opts(pkt)[0], (53, vec![6]));
+}
+
+#[test]
+fn realistic_request_is_answered_and_shrunk() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ // a client identifier naming something else is ignored and the chaddr decides
+ options.extend_from_slice(&[61, 7, 1, 0xde, 0xad, 0xbe, 0xef, 0x00, 0x01]);
+ options.extend_from_slice(&[50, 4]);
+ options.extend_from_slice(&IP);
+ options.extend_from_slice(&[54, 4]);
+ options.extend_from_slice(&GW);
+ options.extend_from_slice(&[12, 14]);
+ options.extend_from_slice(b"dhcp-client-01");
+ options.extend_from_slice(&[60, 32]);
+ options.extend_from_slice(b"PXEClient:Arch:00007:UNDI:003016");
+ options.extend_from_slice(&[55, 20]);
+ options.extend_from_slice(&[
+ 1, 2, 3, 6, 12, 15, 17, 26, 28, 40, 41, 42, 51, 54, 58, 59, 66, 67, 119, 121,
+ ]);
+ options.extend_from_slice(&[57, 2, 0x05, 0xdc]);
+ options.extend_from_slice(&[93, 2, 0, 7]);
+ options.extend_from_slice(&[94, 3, 1, 3, 16]);
+ options.extend_from_slice(&[97, 17, 0]);
+ options.extend_from_slice(&[0x11; 16]);
+ let req = Request {
+ options,
+ ..Request::default()
+ };
+ let request_len = req.build().len();
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert!(pkt.len() < request_len, "reply is shorter than the request");
+ assert_eq!(
+ pkt.len(),
+ 14 + 20 + 8 + 300,
+ "trimmed to the padded minimum"
+ );
+ assert_reply_headers(pkt, MAC, [255, 255, 255, 255], IP);
+ assert_eq!(parse_opts(pkt)[0], (53, vec![5]));
+}
+
+#[test]
+fn other_message_types_pass_untouched() {
+ setup_full_record();
+ for msgtype in [4, 7, 8] {
+ let req = Request {
+ options: vec![53, 1, msgtype],
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC, "message type {msgtype}");
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+ }
+}
+
+#[test]
+fn foreign_bootp_passes_untouched() {
+ setup_full_record();
+ let variants = [
+ // a reply coming in from the guest side, a guest running its own server
+ Request {
+ op: 2,
+ ..Request::default()
+ },
+ Request {
+ htype: 6,
+ ..Request::default()
+ },
+ Request {
+ hlen: 16,
+ ..Request::default()
+ },
+ Request {
+ ip_options: vec![1, 1, 1, 1],
+ ..Request::default()
+ },
+ ];
+ for req in variants {
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+ }
+}
+
+#[test]
+fn truncated_option_ends_the_walk() {
+ setup_full_record();
+
+ // a length running past the packet before the message type leaves nothing to answer
+ let mut options = vec![12, 200];
+ options.extend_from_slice(DISCOVER);
+ let req = Request {
+ options,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+
+ // after the message type it only cuts the walk short, and what was read still counts
+ let mut options = DISCOVER.to_vec();
+ options.extend_from_slice(&[12, 200]);
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_reply_headers(skb.packet(), MAC, [255, 255, 255, 255], IP);
+}
+
+#[test]
+fn arbitrary_option_bytes_never_break_the_reply() {
+ setup_full_record();
+ let mut lcg = 0x2545f4914f6cdd1du64;
+ let mut rand = move || {
+ lcg = lcg
+ .wrapping_mul(6364136223846793005)
+ .wrapping_add(1442695040888963407);
+ (lcg >> 33) as u8
+ };
+
+ for _ in 0..500 {
+ let len = (rand() % 60) as usize;
+ let options: Vec<u8> = (0..len).map(|_| rand()).collect();
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+ assert!(verdict == TC_ACT_UNSPEC || verdict == TC_ACT_REDIRECT);
+ if verdict == TC_ACT_REDIRECT {
+ assert_reply_invariants(skb.packet());
+ }
+ }
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH proxmox-ebpf v3 02/17] dhcp: add the subsystem that loads the responder and keeps its records
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
2026-10-20 7:59 ` [PATCH proxmox-ebpf v3 01/17] dhcp: add the per-tap DHCPv4 responder program Hannes Laimer
@ 2026-10-20 7:59 ` Hannes Laimer
2026-10-20 13:09 ` Stefan Hanreich
2026-10-20 7:59 ` [PATCH proxmox-perl-rs v3 03/17] pve-rs: sdn: add bindings for the eBPF DHCP responder Hannes Laimer
` (16 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 7:59 UTC (permalink / raw)
To: pve-devel
The subsystem loads the program, pins a map with one entry per guest
interface and attaches the program to the interfaces it serves.
Callers run a full pass over every guest interface, plug or unplug one
interface, or change the records of a guest's interfaces.
A load can leave the map empty, and only a full pass is handed every
guest interface. So the full pass is the only step that loads. A plug
or a record change that finds nothing loaded reports that a full pass
has to run first.
A hotplug moves a tap before the guest config names the new bridge,
and no lock closes that gap. So every entry names the vnet its writer
saw the interface on. A pass or the record change of a mapping edit
leaves an entry naming another vnet alone. A plug writes outright, so
plugs share the lock. The edit's record change reads the entry before
it writes, so it takes the lock exclusively.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/dhcp/mod.rs | 1031 +++++++++++++++++++++++++++++++++++++++++++++
src/dhcp/types.rs | 64 +++
src/lib.rs | 3 +
src/subsystem.rs | 2 +-
4 files changed, 1099 insertions(+), 1 deletion(-)
create mode 100644 src/dhcp/mod.rs
create mode 100644 src/dhcp/types.rs
diff --git a/src/dhcp/mod.rs b/src/dhcp/mod.rs
new file mode 100644
index 0000000..d0c4148
--- /dev/null
+++ b/src/dhcp/mod.rs
@@ -0,0 +1,1031 @@
+//! The dhcp subsystem, a per-tap DHCPv4 responder answering from a pinned map of one entry per
+//! interface.
+//!
+//! The map holds nothing of its own. The SDN side hands in every guest interface with the answer
+//! it gets, if any. A full pass makes the kernel match that for the interfaces it names. A record
+//! change or a plug writes its own interfaces only. A linked interface the pass does not name
+//! keeps its link and its entry. A plug runs under the shared lock and writes its entry whatever
+//! is there. A pass runs under the exclusive lock, and so does the record change an edit pushes,
+//! since it reads the entry's vnet before it writes. A guest's own start holds the guest lock, so
+//! nothing else plugs its interfaces meanwhile and it writes its records under the shared lock
+//! without a read.
+//!
+//! A hotplug plugs the tap before the guest config names the new bridge, and no lock orders that.
+//! So every entry names the vnet its writer saw the interface on. A pass or an edit's record change
+//! leaves an entry alone that names another vnet than the configs do, and the link follows such an
+//! entry. A plug replaces it. Plugs of one interface are serialized by the guest lock, so the plug
+//! is the latest word on where its interface sits. An unplug drops the interface's entry. An
+//! interface that went away without a link keeps its entry until the next pass. One plugged onto a
+//! bridge this subsystem does not serve trades its entry for a marker naming that place.
+mod types;
+
+use std::collections::{HashMap, HashSet};
+use std::io::ErrorKind;
+use std::net::Ipv4Addr;
+
+use anyhow::{Context, bail};
+use aya::maps::MapError;
+
+use self::types::*;
+use crate::subsystem::{ApplyLock, Exclusive, NotLoaded, Shared, TcPrograms};
+use crate::tc::{Direction, LinkPin};
+use crate::tc_programs;
+
+/// One guest interface as the configs know it. `serve` says whether a pass links it. A plug
+/// links its interface regardless. `record` is the answer it gets there, and without one the
+/// interface gets no answer. `vnet` is the vnet it sits on, none for a plain bridge.
+pub struct Iface {
+ pub name: String,
+ pub vnet: Option<String>,
+ pub serve: bool,
+ pub record: Option<Record>,
+}
+
+/// One interface's answer, the address and every option the reply carries.
+pub struct Record {
+ pub ip: Ipv4Addr,
+ pub prefixlen: u8,
+ pub server_id: Ipv4Addr,
+ pub lease: u32,
+ pub router: Option<Ipv4Addr>,
+ pub dns: Option<Ipv4Addr>,
+ pub mtu: Option<u16>,
+ pub mac: [u8; 6],
+}
+
+/// What became of an unplugged interface.
+pub enum Unplugged {
+ /// Gone, or going with the unplug.
+ Gone,
+ /// Plugged onto a bridge this subsystem does not serve, the vnet it sits on now, none for a
+ /// plain bridge.
+ MovedTo(Option<String>),
+}
+
+/// What a single-entry step did.
+#[derive(Debug, PartialEq, Eq)]
+pub enum Outcome {
+ /// Nothing is left to do. The entry is in place, or its interface is not here.
+ Done,
+ /// Nothing is loaded to write into. Only a full pass loads, so the caller runs one and retries.
+ NeedsFullPass,
+}
+
+const RECORDS_MAP: &str = "dhcp_iface_records";
+const MAPS: [&str; 1] = [RECORDS_MAP];
+
+// bump with any change to the key or value layout of a map in dhcp.bpf.c
+const SCHEMA_VERSION: u32 = 1;
+
+// requests only arrive from the guest side and replies leave through a redirect, so only ingress
+// carries a program
+const PROGRAM_NAMES: [(Direction, &str); 1] = [(Direction::Ingress, "tc_dhcp_ingress")];
+
+type RecordsMap = aya::maps::HashMap<aya::maps::MapData, u32, DhcpRecord>;
+
+static PROGRAMS: TcPrograms = tc_programs! {
+ name: "dhcp",
+ programs: &PROGRAM_NAMES,
+ maps: &MAPS,
+ schema_version: SCHEMA_VERSION,
+};
+
+pub struct DhcpSubsystem {
+ programs: &'static TcPrograms,
+}
+
+impl Default for DhcpSubsystem {
+ fn default() -> Self {
+ Self::new()
+ }
+}
+
+impl DhcpSubsystem {
+ pub fn new() -> Self {
+ Self {
+ programs: &PROGRAMS,
+ }
+ }
+
+ /// The lock a plug runs under, held until [`attach`](SharedLock::attach) returns. Plugs run
+ /// next to each other and wait only behind an exclusive holder.
+ pub fn lock_shared(&self) -> anyhow::Result<SharedLock> {
+ Ok(SharedLock {
+ lock: self.programs.lock_shared()?,
+ })
+ }
+
+ /// The lock a full pass or a record change runs under, held until the change is written.
+ pub fn lock_exclusive(&self) -> anyhow::Result<ExclusiveLock> {
+ Ok(ExclusiveLock {
+ lock: self.programs.lock_exclusive()?,
+ })
+ }
+
+ /// A tap unplug. The interface may be gone by then, so its link is found by the name the pin
+ /// carries. [`Unplugged`] decides what becomes of the entry.
+ pub fn detach(&self, iface: &str, unplugged: Unplugged) -> anyhow::Result<()> {
+ // resolved before the lock wait, an interface recreated under the name meanwhile is not
+ // this unplug's and keeps its link
+ let now = ifindex_of(iface)?;
+ // every guest unplug on the host lands here. Most never had a link and must not wait
+ // behind a pass. One without a link whose interface is gone leaves its entry to the
+ // next pass
+ if !self.programs.link_pinned(iface) && now.is_none() {
+ return Ok(());
+ }
+ let lock = self.programs.lock_shared()?;
+
+ if !lock.is_current() {
+ log::debug!("dhcp detach: nothing current, {iface} is left to the next full pass");
+ return Ok(());
+ }
+ // the map is opened and the marker built before the links go, so a failure there
+ // leaves them attached
+ let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?;
+ let marker = match &unplugged {
+ Unplugged::Gone => None,
+ Unplugged::MovedTo(vnet) => Some(DhcpRecord::removed(place(vnet.as_deref())?)),
+ };
+ // an interface without a link has an entry as well, a marker where it is not served
+ let mut ifindexes = lock.detach_named(iface, |pin| {
+ Some(pin.ifindex) != now && pin_alive(pin).unwrap_or(false)
+ })?;
+ if let Some(now) = now.filter(|now| !ifindexes.contains(now)) {
+ ifindexes.push(now);
+ }
+ // the marker is the moved interface's. Older incarnations under the name are gone
+ unplug_entries(&mut map, &ifindexes, now.zip(marker))
+ }
+}
+
+/// The shared lock of the responder, what a plug holds, and the record change of a guest's own
+/// start.
+pub struct SharedLock {
+ lock: ApplyLock<'static, Shared>,
+}
+
+impl SharedLock {
+ /// A tap plug, the entry goes in and the program on. The plug loads nothing, so with nothing
+ /// pinned the caller runs a full pass first.
+ pub fn attach(&self, iface: &Iface) -> anyhow::Result<Outcome> {
+ if !self.lock.is_current() {
+ return Ok(Outcome::NeedsFullPass);
+ }
+ let Some(ifindex) = ifindex_of(iface.name.as_str())? else {
+ log::info!("dhcp attach: {} is gone, nothing to do", iface.name);
+ return Ok(Outcome::Done);
+ };
+ // the entry goes in before the program runs, a marker when there is no record or one
+ // the program cannot carry. A guest start must not fail on a bad record
+ let mut map: RecordsMap = self.lock.hash_map(RECORDS_MAP)?;
+ let entry = match iface.entry() {
+ Ok(entry) => entry,
+ Err(e) => {
+ log::warn!(
+ "dhcp: {} attaches with nothing to answer: {e:#}",
+ iface.name
+ );
+ DhcpRecord::removed(iface.place()?)
+ }
+ };
+ write_entry(&mut map, ifindex, &entry, Elsewhere::Override)?;
+ match self.lock.attach_iface(ifindex) {
+ Ok(()) => Ok(Outcome::Done),
+ Err(e) if e.downcast_ref::<NotLoaded>().is_some() => Ok(Outcome::NeedsFullPass),
+ Err(e) => Err(e).with_context(|| format!("dhcp responder for {}", iface.name)),
+ }
+ }
+
+ /// A record change for a caller that holds the guest lock. Nothing else plugs the guest's
+ /// interfaces under that lock, so each entry is written outright, one atomic write, without
+ /// reading what it holds.
+ pub fn update(&self, ifaces: &[Iface]) -> anyhow::Result<Outcome> {
+ update_entries(&self.lock, ifaces, Elsewhere::Override)
+ }
+}
+
+/// The exclusive lock of the responder, what a full pass and a record change hold.
+pub struct ExclusiveLock {
+ lock: ApplyLock<'static, Exclusive>,
+}
+
+impl ExclusiveLock {
+ /// The full pass. Afterward every interface the configs name that exists here has its entry,
+ /// a record where it is served with one and a marker otherwise, and a link is on every
+ /// served interface and on every interface whose entry names another vnet. Entries of
+ /// interfaces that are gone are dropped, as are entries of live interfaces the configs do
+ /// not name and that have no link. Bad input leaves its interface as it is, with a warning.
+ pub fn apply(&self, ifaces: &[Iface]) -> anyhow::Result<()> {
+ self.lock.ensure_loaded()?;
+ let present = present_ifindexes(ifaces)?;
+ // a link the configs do not name stays while its interface lives under the pin's
+ // name. A reused index is not that interface
+ let mut unknown_alive = HashSet::new();
+ for pin in self.lock.attached()? {
+ if !present.contains_key(&pin.ifindex) && pin_alive(&pin)? {
+ unknown_alive.insert(pin.ifindex);
+ }
+ }
+ let mut map: RecordsMap = self.lock.hash_map(RECORDS_MAP)?;
+ let (elsewhere, failed) = sync_records(&mut map, &present, &unknown_alive)?;
+ // a failed write or removal leaves its entry as it is. The links are still put right
+ // and every failure is reported once the rest is done
+ let links = self
+ .lock
+ .reconcile(&wanted_links(&present, &elsewhere, &unknown_alive));
+ match (failed, links) {
+ (0, links) => links,
+ (failed, Ok(())) => bail!("{failed} entries could not be changed"),
+ (failed, Err(e)) => {
+ Err(e).with_context(|| format!("{failed} entries could not be changed"))
+ }
+ }
+ }
+
+ /// A record change. Interfaces not found here are skipped. The rest are written whether they
+ /// have a link yet or not. A record the reply cannot carry leaves its interface as it is.
+ /// With nothing loaded the caller runs a full pass first, like a plug.
+ pub fn update(&self, ifaces: &[Iface]) -> anyhow::Result<Outcome> {
+ update_entries(&self.lock, ifaces, Elsewhere::Skip)
+ }
+
+ /// Detach everywhere and drop the pinned state, for the last zone leaving the backend.
+ pub fn clear(&self) -> anyhow::Result<()> {
+ self.lock.clear()
+ }
+}
+
+fn update_entries<M>(
+ lock: &ApplyLock<'static, M>,
+ ifaces: &[Iface],
+ elsewhere: Elsewhere,
+) -> anyhow::Result<Outcome> {
+ if !lock.is_current() {
+ return Ok(Outcome::NeedsFullPass);
+ }
+ let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?;
+ let mut failed = 0usize;
+ for iface in ifaces {
+ let ifindex = match ifindex_of(iface.name.as_str()) {
+ Ok(Some(ifindex)) => ifindex,
+ Ok(None) => continue,
+ Err(e) => {
+ log::warn!("dhcp: {e:#}");
+ failed += 1;
+ continue;
+ }
+ };
+ let entry = match iface.entry() {
+ Ok(entry) => entry,
+ Err(e) => {
+ log::warn!("dhcp: {} is left as it is: {e:#}", iface.name);
+ continue;
+ }
+ };
+ if let Err(e) = write_entry(&mut map, ifindex, &entry, elsewhere) {
+ log::warn!("dhcp: writing the entry of {}: {e:#}", iface.name);
+ failed += 1;
+ }
+ }
+ if failed > 0 {
+ bail!("{failed} interfaces could not be updated");
+ }
+ Ok(Outcome::Done)
+}
+
+const NO_VNET: [u8; 8] = [0; 8];
+
+// the map update flag that refuses a key the map does not hold, from the kernel's uapi bpf.h
+const BPF_EXIST: u64 = 2;
+
+fn unplug_entries(
+ map: &mut impl RecordStore,
+ ifindexes: &[u32],
+ moved: Option<(u32, DhcpRecord)>,
+) -> anyhow::Result<()> {
+ for &ifindex in ifindexes {
+ match moved {
+ // only an existing entry becomes a marker, so an interface without one gets none
+ Some((current, marker)) if current == ifindex => map.mark(ifindex, marker)?,
+ _ => map.remove(ifindex)?,
+ }
+ }
+ Ok(())
+}
+
+// the vnet id padded with zeros, all zero for none
+fn place(vnet: Option<&str>) -> anyhow::Result<[u8; 8]> {
+ let Some(vnet) = vnet else {
+ return Ok(NO_VNET);
+ };
+ let bytes = vnet.as_bytes();
+ if bytes.is_empty() || bytes.len() > NO_VNET.len() {
+ bail!("vnet id {vnet:?} does not fit a record");
+ }
+ let mut place = NO_VNET;
+ place[..bytes.len()].copy_from_slice(bytes);
+ Ok(place)
+}
+
+// the record map as the writers see it, so their decisions run against a plain map in tests
+trait RecordStore {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>>;
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()>;
+ // replace an entry that exists and do nothing for one that does not, in one step, so it is
+ // safe under the shared lock
+ fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()>;
+ // a missing entry is removed already
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()>;
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>>;
+}
+
+impl RecordStore for RecordsMap {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
+ match aya::maps::HashMap::get(self, &ifindex, 0) {
+ Ok(live) => Ok(Some(live)),
+ Err(MapError::KeyNotFound) => Ok(None),
+ Err(e) => Err(e.into()),
+ }
+ }
+
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ aya::maps::HashMap::insert(self, ifindex, rec, 0)?;
+ Ok(())
+ }
+
+ fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ match aya::maps::HashMap::insert(self, ifindex, rec, BPF_EXIST) {
+ Ok(()) => Ok(()),
+ // the kernel answers a missing key with ENOENT, and aya passes that through as is
+ Err(MapError::SyscallError(e)) if e.io_error.kind() == ErrorKind::NotFound => Ok(()),
+ Err(e) => Err(e.into()),
+ }
+ }
+
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
+ match aya::maps::HashMap::remove(self, &ifindex) {
+ Ok(()) => Ok(()),
+ // the kernel answers a missing key with ENOENT, and aya passes that through as is
+ Err(MapError::SyscallError(e)) if e.io_error.kind() == ErrorKind::NotFound => Ok(()),
+ Err(e) => Err(e.into()),
+ }
+ }
+
+ // a record the listing cannot read is one the pass cannot compare against, so the pass
+ // fails rather than writes past it
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
+ self.iter()
+ .collect::<Result<Vec<_>, _>>()
+ .context("listing the records")
+ }
+}
+
+// what a write does with an entry naming another vnet than its own
+#[derive(Clone, Copy)]
+enum Elsewhere {
+ // a record change, its configs are stale for the interface
+ Skip,
+ // a plug, or a record change under the guest lock, both see where the interface sits now
+ Override,
+}
+
+fn write_entry(
+ map: &mut impl RecordStore,
+ ifindex: u32,
+ rec: &DhcpRecord,
+ elsewhere: Elsewhere,
+) -> anyhow::Result<()> {
+ if matches!(elsewhere, Elsewhere::Skip)
+ && let Some(live) = map.get(ifindex)?
+ && live.vnet != rec.vnet
+ {
+ log::debug!("dhcp: ifindex {ifindex} sits elsewhere than the configs say, leaving it");
+ return Ok(());
+ }
+ map.insert(ifindex, *rec)
+}
+
+// entries of live links the pass does not know stay, since their plug wrote them. The other
+// unknown ones go with their interface. Returns the interfaces whose entry names another vnet
+// than the configs, and how many writes failed
+fn sync_records(
+ map: &mut impl RecordStore,
+ present: &HashMap<u32, &Iface>,
+ unknown_alive: &HashSet<u32>,
+) -> anyhow::Result<(HashSet<u32>, usize)> {
+ let live: HashMap<u32, DhcpRecord> = map.entries()?.into_iter().collect();
+ let dead: Vec<u32> = live
+ .keys()
+ .filter(|ifindex| !present.contains_key(ifindex) && !unknown_alive.contains(ifindex))
+ .copied()
+ .collect();
+ let mut failed = 0usize;
+ for &ifindex in &dead {
+ if let Err(e) = map.remove(ifindex) {
+ log::warn!("dhcp: removing the record of ifindex {ifindex}: {e:#}");
+ failed += 1;
+ }
+ }
+ let mut written = 0usize;
+ let mut elsewhere = HashSet::new();
+
+ for (ifindex, iface) in present {
+ // bad input leaves its interface as it is and the pass goes on
+ let desired = match iface.entry() {
+ Ok(desired) => desired,
+ Err(e) => {
+ log::warn!("dhcp: {} is left as it is: {e:#}", iface.name);
+ continue;
+ }
+ };
+ match live.get(ifindex) {
+ Some(entry) if entry.vnet != desired.vnet => {
+ elsewhere.insert(*ifindex);
+ }
+
+ Some(entry) if *entry == desired => {}
+ _ => match map.insert(*ifindex, desired) {
+ Ok(()) => written += 1,
+ Err(e) => {
+ log::warn!("dhcp: writing the entry of {}: {e:#}", iface.name);
+ failed += 1;
+ }
+ },
+ }
+ }
+
+ if written + dead.len() + elsewhere.len() > 0 {
+ log::info!(
+ "dhcp: {written} entries written, {} removed, {} left where their plug put them",
+ dead.len(),
+ elsewhere.len()
+ );
+ } else {
+ log::debug!("dhcp: {} records, no changes", live.len());
+ }
+ Ok((elsewhere, failed))
+}
+
+// the interface's index here, none when it does not exist. Any other failure says nothing about
+// the interface, and a step taking it for gone would detach what it could not resolve
+fn ifindex_of(name: &str) -> anyhow::Result<Option<u32>> {
+ match nix::net::if_::if_nametoindex(name) {
+ Ok(ifindex) => Ok(Some(ifindex)),
+ Err(nix::errno::Errno::ENODEV) => Ok(None),
+ Err(e) => Err(e).with_context(|| format!("resolve interface {name}")),
+ }
+}
+
+// the named interfaces that exist here by index, the names come from configs cluster-wide and
+// most belong to other nodes
+fn present_ifindexes(ifaces: &[Iface]) -> anyhow::Result<HashMap<u32, &Iface>> {
+ let mut present = HashMap::new();
+ for iface in ifaces {
+ if let Some(ifindex) = ifindex_of(iface.name.as_str())? {
+ present.insert(ifindex, iface);
+ }
+ }
+ Ok(present)
+}
+
+// the link follows the configs where they agree with the entry. An entry naming another vnet
+// keeps its link, and a served interface without a mapping gets one. So a record change filling
+// a marker attaches nothing itself. A live link the configs do not know stays
+fn wanted_links(
+ present: &HashMap<u32, &Iface>,
+ elsewhere: &HashSet<u32>,
+ unknown_alive: &HashSet<u32>,
+) -> HashSet<u32> {
+ let mut wanted: HashSet<u32> = present
+ .iter()
+ .filter(|(ifindex, iface)| elsewhere.contains(ifindex) || iface.serve)
+ .map(|(ifindex, _)| *ifindex)
+ .collect();
+
+ wanted.extend(unknown_alive);
+ wanted
+}
+
+// whether the link's interface still exists under the name and index its pin carries
+fn pin_alive(pin: &LinkPin) -> anyhow::Result<bool> {
+ Ok(ifindex_of(pin.name.as_str())? == Some(pin.ifindex))
+}
+
+impl Iface {
+ // a marker where nothing is served
+ fn entry(&self) -> anyhow::Result<DhcpRecord> {
+ let vnet = self.place()?;
+ match &self.record {
+ Some(record) if self.serve => record
+ .entry(vnet)
+ .with_context(|| format!("record for {}", self.name)),
+ _ => Ok(DhcpRecord::removed(vnet)),
+ }
+ }
+
+ fn place(&self) -> anyhow::Result<[u8; 8]> {
+ place(self.vnet.as_deref()).with_context(|| format!("interface {}", self.name))
+ }
+}
+
+impl Record {
+ // refused here is what the reply cannot carry. Zero stands for absent in the entry, so an
+ // address or an MTU of zero cannot be told from none
+ fn entry(&self, vnet: [u8; 8]) -> anyhow::Result<DhcpRecord> {
+ if self.prefixlen == 0 || self.prefixlen > 32 {
+ bail!("prefixlen {} out of range", self.prefixlen);
+ }
+ if self.ip.is_unspecified() {
+ bail!("no address");
+ }
+ if self.server_id.is_unspecified() {
+ bail!("no server address");
+ }
+ if self.lease == 0 {
+ bail!("lease of zero seconds");
+ }
+ if self.mtu.is_some_and(|mtu| mtu < 68) {
+ bail!("mtu below the IPv4 minimum of 68");
+ }
+ if self.mac == [0; 6] {
+ bail!("no MAC");
+ }
+ let netmask = u32::MAX << (32 - self.prefixlen);
+ Ok(DhcpRecord {
+ ip: u32::from(self.ip).to_be(),
+ netmask: netmask.to_be(),
+ router: u32::from(self.router.unwrap_or(Ipv4Addr::UNSPECIFIED)).to_be(),
+ dns: u32::from(self.dns.unwrap_or(Ipv4Addr::UNSPECIFIED)).to_be(),
+ server_id: u32::from(self.server_id).to_be(),
+ lease: self.lease.to_be(),
+ mtu: self.mtu.unwrap_or(0).to_be(),
+ vnet,
+ mac: self.mac,
+ })
+ }
+}
+
+#[cfg(test)]
+mod test {
+ use std::collections::BTreeMap;
+
+ use super::*;
+
+ fn record() -> Record {
+ Record {
+ ip: Ipv4Addr::new(10, 0, 0, 100),
+ prefixlen: 24,
+ server_id: Ipv4Addr::new(10, 0, 0, 1),
+ lease: 300,
+ router: Some(Ipv4Addr::new(10, 0, 0, 1)),
+ dns: Some(Ipv4Addr::new(1, 1, 1, 1)),
+ mtu: Some(1500),
+ mac: MAC,
+ }
+ }
+
+ const MAC: [u8; 6] = [0xda, 0x65, 0x8f, 0x18, 0x9b, 0x6f];
+
+ const VNET: [u8; 8] = *b"vnet1\0\0\0";
+
+ #[test]
+ fn converts_full_record() {
+ let rec = record().entry(VNET).unwrap();
+ assert_eq!(
+ u32::from_be(rec.ip),
+ u32::from(Ipv4Addr::new(10, 0, 0, 100))
+ );
+ assert_eq!(u32::from_be(rec.netmask), 0xffffff00);
+ assert_eq!(
+ u32::from_be(rec.router),
+ u32::from(Ipv4Addr::new(10, 0, 0, 1))
+ );
+ assert_eq!(u32::from_be(rec.dns), u32::from(Ipv4Addr::new(1, 1, 1, 1)));
+ assert_eq!(
+ u32::from_be(rec.server_id),
+ u32::from(Ipv4Addr::new(10, 0, 0, 1))
+ );
+ assert_eq!(u32::from_be(rec.lease), 300);
+ assert_eq!(u16::from_be(rec.mtu), 1500);
+ assert_eq!(rec.vnet, VNET);
+ assert_eq!(rec.mac, MAC);
+
+ let rec = Record {
+ router: None,
+ dns: None,
+ mtu: None,
+ ..record()
+ }
+ .entry(VNET)
+ .unwrap();
+ assert_eq!(rec.router, 0);
+ assert_eq!(rec.dns, 0);
+ assert_eq!(rec.mtu, 0);
+ }
+
+ #[test]
+ fn entries_name_the_vnet_of_the_interface() {
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("testvnet".into()),
+ serve: true,
+ record: Some(record()),
+ };
+ assert_eq!(&iface.entry().unwrap().vnet, b"testvnet");
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: None,
+ serve: false,
+ record: None,
+ };
+ assert_eq!(iface.entry().unwrap().vnet, NO_VNET);
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("ninechars".into()),
+ serve: true,
+ record: Some(record()),
+ };
+ assert!(iface.entry().is_err());
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some(String::new()),
+ serve: true,
+ record: Some(record()),
+ };
+ assert!(iface.entry().is_err());
+ }
+
+ #[test]
+ fn unserved_interfaces_get_a_marker() {
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("vnet1".into()),
+ serve: false,
+ record: Some(record()),
+ };
+ assert_eq!(iface.entry().unwrap().ip, 0);
+ assert_eq!(iface.entry().unwrap().vnet, VNET);
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("vnet1".into()),
+ serve: true,
+ record: None,
+ };
+ assert_eq!(iface.entry().unwrap().ip, 0);
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("vnet1".into()),
+ serve: true,
+ record: Some(record()),
+ };
+ assert_ne!(iface.entry().unwrap().ip, 0);
+ }
+
+ #[test]
+ fn refuses_what_the_reply_cannot_carry() {
+ for rec in [
+ Record {
+ prefixlen: 33,
+ ..record()
+ },
+ Record {
+ prefixlen: 0,
+ ..record()
+ },
+ Record {
+ server_id: Ipv4Addr::UNSPECIFIED,
+ ..record()
+ },
+ Record {
+ lease: 0,
+ ..record()
+ },
+ Record {
+ ip: Ipv4Addr::UNSPECIFIED,
+ ..record()
+ },
+ Record {
+ mtu: Some(67),
+ ..record()
+ },
+ Record {
+ mac: [0; 6],
+ ..record()
+ },
+ ] {
+ assert!(rec.entry(VNET).is_err());
+ }
+ let rec = Record {
+ mtu: Some(68),
+ ..record()
+ };
+ assert_eq!(u16::from_be(rec.entry(VNET).unwrap().mtu), 68);
+
+ let rec = Record {
+ prefixlen: 32,
+ ..record()
+ };
+ assert_eq!(u32::from_be(rec.entry(VNET).unwrap().netmask), u32::MAX);
+ let rec = Record {
+ prefixlen: 1,
+ ..rec
+ };
+ assert_eq!(u32::from_be(rec.entry(VNET).unwrap().netmask), 1 << 31);
+ }
+
+ impl RecordStore for BTreeMap<u32, DhcpRecord> {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
+ Ok(BTreeMap::get(self, &ifindex).copied())
+ }
+
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ BTreeMap::insert(self, ifindex, rec);
+ Ok(())
+ }
+
+ fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ if let Some(entry) = BTreeMap::get_mut(self, &ifindex) {
+ *entry = rec;
+ }
+ Ok(())
+ }
+
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
+ BTreeMap::remove(self, &ifindex);
+ Ok(())
+ }
+
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
+ Ok(self.iter().map(|(k, v)| (*k, *v)).collect())
+ }
+ }
+
+ const OTHER: [u8; 8] = *b"vnet2\0\0\0";
+
+ fn entry(vnet: [u8; 8]) -> DhcpRecord {
+ record().entry(vnet).unwrap()
+ }
+
+ fn other_answer(vnet: [u8; 8]) -> DhcpRecord {
+ Record {
+ ip: Ipv4Addr::new(10, 0, 0, 200),
+ ..record()
+ }
+ .entry(vnet)
+ .unwrap()
+ }
+
+ #[test]
+ fn a_record_change_writes_on_its_own_vnet() {
+ let mut map = BTreeMap::new();
+ write_entry(&mut map, 1, &entry(VNET), Elsewhere::Skip).unwrap();
+ assert_eq!(map[&1], entry(VNET));
+ write_entry(&mut map, 1, &other_answer(VNET), Elsewhere::Skip).unwrap();
+ assert_eq!(map[&1], other_answer(VNET));
+ // an entry naming another vnet was placed by a plug and stays
+ map.insert(1, entry(OTHER));
+ write_entry(&mut map, 1, &other_answer(VNET), Elsewhere::Skip).unwrap();
+ assert_eq!(map[&1], entry(OTHER));
+ }
+
+ #[test]
+ fn a_plug_writes_whatever_is_there() {
+ let mut map = BTreeMap::new();
+ write_entry(&mut map, 1, &entry(VNET), Elsewhere::Override).unwrap();
+ assert_eq!(map[&1], entry(VNET));
+ map.insert(1, entry(OTHER));
+ write_entry(&mut map, 1, &other_answer(VNET), Elsewhere::Override).unwrap();
+ assert_eq!(map[&1], other_answer(VNET));
+ }
+
+ #[test]
+ fn an_unplug_removes_or_marks_the_entries() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(VNET));
+ map.insert(2, entry(VNET));
+ map.insert(3, entry(VNET));
+ unplug_entries(&mut map, &[1, 2], None).unwrap();
+ assert_eq!(map.keys().copied().collect::<Vec<_>>(), [3]);
+ // a missing entry is removed already
+ unplug_entries(&mut map, &[1], None).unwrap();
+ // a move marks the interface that moved, and an older incarnation under its name is gone
+ map.insert(4, entry(VNET));
+ let marker = DhcpRecord::removed(NO_VNET);
+ unplug_entries(&mut map, &[3, 4], Some((4, marker))).unwrap();
+ assert_eq!(map.keys().copied().collect::<Vec<_>>(), [4]);
+ assert_eq!(map[&4].ip, 0);
+ assert_eq!(map[&4].vnet, NO_VNET);
+ // an interface without an entry gets no marker
+ unplug_entries(&mut map, &[5], Some((5, marker))).unwrap();
+ assert!(!map.contains_key(&5));
+ }
+
+ fn iface(name: &str, vnet: Option<&str>, serve: bool, record: Option<Record>) -> Iface {
+ Iface {
+ name: name.into(),
+ vnet: vnet.map(String::from),
+ serve,
+ record,
+ }
+ }
+
+ #[test]
+ fn the_pass_syncs_the_records_it_knows() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(VNET)); // same answer as the config
+ map.insert(2, other_answer(VNET)); // a changed answer
+ map.insert(4, entry(VNET)); // not served anymore
+ map.insert(6, entry(VNET)); // its interface is gone
+ map.insert(7, entry(VNET)); // a live link the configs do not name
+ map.insert(8, entry(VNET)); // its mapping is gone
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(record())),
+ iface("b", Some("vnet1"), true, Some(record())),
+ iface("d", Some("vnet1"), false, None),
+ iface("e", Some("vnet1"), true, Some(record())), // no entry yet
+ iface("f", Some("vnet1"), true, None), // no entry, served, no mapping
+ iface("g", Some("vnet1"), true, None),
+ ];
+ let present: HashMap<u32, &Iface> =
+ [1, 2, 4, 5, 9, 8].into_iter().zip(ifaces.iter()).collect();
+ let unknown_alive = HashSet::from([7]);
+ let (elsewhere, failed) = sync_records(&mut map, &present, &unknown_alive).unwrap();
+
+ assert!(elsewhere.is_empty());
+ assert_eq!(failed, 0);
+ assert_eq!(map[&1], entry(VNET));
+ assert_eq!(map[&2], entry(VNET), "a changed answer is rewritten");
+ assert_eq!(
+ map[&4].ip, 0,
+ "an interface not served anymore keeps a marker"
+ );
+ assert_eq!(map[&5], entry(VNET), "a new interface gets its entry");
+ assert!(!map.contains_key(&6), "a gone interface loses its entry");
+ assert_eq!(
+ map[&9].ip, 0,
+ "a served interface without a mapping gets a marker"
+ );
+ assert_eq!(map[&8].ip, 0, "a mapping that is gone leaves a marker");
+ assert!(
+ map.contains_key(&7),
+ "a live link the configs do not name keeps its entry"
+ );
+ }
+
+ #[test]
+ fn the_links_follow_the_configs_or_the_entry() {
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(record())), // served, entry agrees
+ iface("b", Some("vnet1"), false, None), // not served
+ iface("c", Some("vnet1"), false, None), // config stale, entry has a record
+ iface("d", Some("vnet1"), true, Some(record())), // config stale, entry is a marker
+ ];
+ let present: HashMap<u32, &Iface> = (1..=4).zip(ifaces.iter()).collect();
+ let elsewhere = HashSet::from([3, 4]);
+ let unknown_alive = HashSet::from([9]);
+ assert_eq!(
+ wanted_links(&present, &elsewhere, &unknown_alive),
+ HashSet::from([1, 3, 4, 9])
+ );
+ }
+
+ // a store that refuses one interface's writes
+ struct Flaky(BTreeMap<u32, DhcpRecord>, u32);
+
+ impl RecordStore for Flaky {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
+ RecordStore::get(&self.0, ifindex)
+ }
+
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ if ifindex == self.1 {
+ bail!("map full");
+ }
+ RecordStore::insert(&mut self.0, ifindex, rec)
+ }
+
+ fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ RecordStore::mark(&mut self.0, ifindex, rec)
+ }
+
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
+ if ifindex == self.1 {
+ bail!("map busy");
+ }
+ RecordStore::remove(&mut self.0, ifindex)
+ }
+
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
+ RecordStore::entries(&self.0)
+ }
+ }
+
+ #[test]
+ fn bad_input_leaves_its_interface_alone() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(VNET));
+ let bad = Record {
+ lease: 0,
+ ..record()
+ };
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(bad)),
+ iface("b", Some("vnet1"), true, Some(record())),
+ ];
+ let present: HashMap<u32, &Iface> = (1..=2).zip(ifaces.iter()).collect();
+ let (elsewhere, failed) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
+ assert!(elsewhere.is_empty());
+ assert_eq!(failed, 0);
+ assert_eq!(map[&1], entry(VNET), "the old entry stays");
+ assert_eq!(map[&2], entry(VNET), "the rest is written");
+ }
+
+ #[test]
+ fn a_failed_write_does_not_stop_the_pass() {
+ let mut map = Flaky(BTreeMap::new(), 2);
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(record())),
+ iface("b", Some("vnet1"), true, Some(record())),
+ iface("c", Some("vnet1"), true, Some(record())),
+ ];
+ let present: HashMap<u32, &Iface> = (1..=3).zip(ifaces.iter()).collect();
+ let (elsewhere, failed) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
+ assert!(elsewhere.is_empty());
+ assert_eq!(failed, 1);
+ assert!(map.0.contains_key(&1));
+ assert!(!map.0.contains_key(&2));
+ assert!(map.0.contains_key(&3));
+ }
+
+ // a store whose listing fails
+ struct Unlistable(BTreeMap<u32, DhcpRecord>);
+
+ impl RecordStore for Unlistable {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
+ RecordStore::get(&self.0, ifindex)
+ }
+
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ RecordStore::insert(&mut self.0, ifindex, rec)
+ }
+
+ fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ RecordStore::mark(&mut self.0, ifindex, rec)
+ }
+
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
+ RecordStore::remove(&mut self.0, ifindex)
+ }
+
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
+ bail!("map gone")
+ }
+ }
+
+ #[test]
+ fn a_failed_listing_fails_the_pass_before_it_writes() {
+ let mut map = Unlistable(BTreeMap::new());
+ map.0.insert(1, entry(OTHER));
+ let ifaces = [iface("a", Some("vnet1"), true, Some(record()))];
+ let present: HashMap<u32, &Iface> = [(1, &ifaces[0])].into_iter().collect();
+ assert!(sync_records(&mut map, &present, &HashSet::new()).is_err());
+ assert_eq!(map.0[&1].vnet, OTHER);
+ }
+
+ #[test]
+ fn a_failed_removal_counts_like_a_failed_write() {
+ let mut map = Flaky(BTreeMap::new(), 2);
+ map.0.insert(2, entry(VNET));
+ let ifaces = [iface("a", Some("vnet1"), true, Some(record()))];
+ let present: HashMap<u32, &Iface> = [(1, &ifaces[0])].into_iter().collect();
+ let (_, failed) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
+ assert_eq!(failed, 1);
+ assert!(map.0.contains_key(&1));
+ assert!(map.0.contains_key(&2));
+ }
+
+ #[test]
+ fn the_pass_leaves_entries_of_another_vnet_to_their_link() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(OTHER)); // a plug the configs trail, with a record
+ map.insert(2, DhcpRecord::removed(NO_VNET)); // moved to a plain bridge
+ map.insert(3, DhcpRecord::removed(NO_VNET)); // the configs caught up with that
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(record())),
+ iface("b", Some("vnet1"), true, Some(record())),
+ iface("c", None, false, None),
+ ];
+ let present: HashMap<u32, &Iface> = (1..=3).zip(ifaces.iter()).collect();
+ let (elsewhere, _) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
+
+ assert_eq!(elsewhere, HashSet::from([1, 2]));
+
+ assert_eq!(map[&1].vnet, OTHER);
+ assert_eq!(map[&2].vnet, NO_VNET);
+ assert_eq!(
+ map[&3].ip, 0,
+ "a marker the configs agree with stays as it is"
+ );
+ }
+}
diff --git a/src/dhcp/types.rs b/src/dhcp/types.rs
new file mode 100644
index 0000000..71a1b94
--- /dev/null
+++ b/src/dhcp/types.rs
@@ -0,0 +1,64 @@
+//! Keep in sync with 'bpf/types.h'.
+
+/// Addresses, lease and mtu are stored in network byte order, so a reply copies them into the
+/// packet as-is.
+#[repr(C)]
+#[derive(Copy, Clone, Debug, PartialEq, Eq)]
+pub struct DhcpRecord {
+ /// 0 = removed, the entry is a marker naming its vnet
+ pub ip: u32,
+ pub netmask: u32,
+ /// 0 = not served
+ pub router: u32,
+ /// 0 = not served
+ pub dns: u32,
+ pub server_id: u32,
+ pub lease: u32,
+ /// 0 = not served
+ pub mtu: u16,
+ /// The vnet the writer saw the interface on, the id padded with zeros, all zero for none.
+ /// Never read by the program.
+ pub vnet: [u8; 8],
+ /// The NIC's MAC, a request carrying another is passed on. Zero in a marker.
+ pub mac: [u8; 6],
+}
+
+impl DhcpRecord {
+ /// The marker a removal leaves behind, on the vnet the writer saw the interface on.
+ pub fn removed(vnet: [u8; 8]) -> Self {
+ Self {
+ ip: 0,
+ netmask: 0,
+ router: 0,
+ dns: 0,
+ server_id: 0,
+ lease: 0,
+ mtu: 0,
+ vnet,
+ mac: [0; 6],
+ }
+ }
+}
+
+unsafe impl aya::Pod for DhcpRecord {}
+
+#[cfg(test)]
+mod layout {
+ use core::mem::{offset_of, size_of};
+
+ use super::*;
+
+ #[test]
+ fn matches_bpf_abi() {
+ assert_eq!(size_of::<DhcpRecord>(), 40);
+ assert_eq!(offset_of!(DhcpRecord, ip), 0);
+ assert_eq!(offset_of!(DhcpRecord, netmask), 4);
+ assert_eq!(offset_of!(DhcpRecord, router), 8);
+ assert_eq!(offset_of!(DhcpRecord, dns), 12);
+ assert_eq!(offset_of!(DhcpRecord, server_id), 16);
+ assert_eq!(offset_of!(DhcpRecord, lease), 20);
+ assert_eq!(offset_of!(DhcpRecord, mtu), 24);
+ assert_eq!(offset_of!(DhcpRecord, vnet), 26);
+ assert_eq!(offset_of!(DhcpRecord, mac), 34);
+ }
+}
diff --git a/src/lib.rs b/src/lib.rs
index 572d861..a8c94ba 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -1,5 +1,8 @@
//! eBPF subsystems, one per concern, each behind a cargo feature of its name. Nothing here runs
//! on its own, consumers pull in only the subsystem they drive.
+#[cfg(feature = "dhcp")]
+pub mod dhcp;
+
pub mod subsystem;
pub mod tc;
diff --git a/src/subsystem.rs b/src/subsystem.rs
index 3bf1634..6014dbe 100644
--- a/src/subsystem.rs
+++ b/src/subsystem.rs
@@ -42,7 +42,7 @@ const RUN_ROOT: &str = "/run/proxmox-ebpf";
/// The subsystems in the order their programs run on an interface, the first runs first. A name
/// missing from the list fails the build.
-pub const RUN_ORDER: &[&str] = &[];
+pub const RUN_ORDER: &[&str] = &["dhcp"];
const fn rank_of(name: &str) -> usize {
let mut i = 0;
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH proxmox-perl-rs v3 03/17] pve-rs: sdn: add bindings for the eBPF DHCP responder
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
2026-10-20 7:59 ` [PATCH proxmox-ebpf v3 01/17] dhcp: add the per-tap DHCPv4 responder program Hannes Laimer
2026-10-20 7:59 ` [PATCH proxmox-ebpf v3 02/17] dhcp: add the subsystem that loads the responder and keeps its records Hannes Laimer
@ 2026-10-20 7:59 ` Hannes Laimer
2026-10-20 13:10 ` Stefan Hanreich
2026-10-20 7:59 ` [PATCH pve-network v3 04/17] sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it Hannes Laimer
` (15 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 7:59 UTC (permalink / raw)
To: pve-devel
The responder answers with the records the caller hands in. The
caller has to build them under the responder's lock, or a full pass
could write older records over those a plug or record change built
later. So the lock is an object the caller takes before it builds the
records, and every call that hands in records is a method of that
lock.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
pve-rs/Cargo.toml | 2 +
pve-rs/Makefile | 3 +
pve-rs/debian/control | 2 +
pve-rs/src/bindings/sdn/dhcp.rs | 229 ++++++++++++++++++++++++++++++++
pve-rs/src/bindings/sdn/mod.rs | 1 +
5 files changed, 237 insertions(+)
create mode 100644 pve-rs/src/bindings/sdn/dhcp.rs
diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
index 5ae9082..0adf00b 100644
--- a/pve-rs/Cargo.toml
+++ b/pve-rs/Cargo.toml
@@ -34,6 +34,7 @@ proxmox-apt = { version = "1.0.1", features = ["cache"] }
proxmox-apt-api-types = "3"
proxmox-base64 = "1"
proxmox-config-digest = "1"
+proxmox-ebpf = { version = "0.1", features = ["dhcp"] }
proxmox-frr = { version = "0.5.1" }
proxmox-http = { version = "1.0.2", features = ["client-sync", "client-trait"] }
proxmox-http-error = "1"
@@ -70,6 +71,7 @@ proxmox-wireguard = { version = "0.1.2" }
# proxmox-config-digest = { path = "../../proxmox/proxmox-config-digest" }
# proxmox-daemon = { path = "../../proxmox/proxmox-daemon" }
# proxmox-dns-api = { path = "../../proxmox/proxmox-dns-api" }
+# proxmox-ebpf = { path = "../../proxmox-ebpf" }
# proxmox-http-error = { path = "../../proxmox/proxmox-http-error" }
# proxmox-http = { path = "../../proxmox/proxmox-http" }
# proxmox-human-byte = { path = "../../proxmox/proxmox-human-byte" }
diff --git a/pve-rs/Makefile b/pve-rs/Makefile
index bb1cd2d..ebe8bd3 100644
--- a/pve-rs/Makefile
+++ b/pve-rs/Makefile
@@ -32,6 +32,9 @@ PERLMOD_PACKAGES := \
PVE::RS::OpenId \
PVE::RS::ResourceScheduling::Static \
PVE::RS::ResourceScheduling::Dynamic \
+ PVE::RS::SDN::Dhcp \
+ PVE::RS::SDN::Dhcp::ExclusiveLock \
+ PVE::RS::SDN::Dhcp::SharedLock \
PVE::RS::SDN::Fabrics \
PVE::RS::SDN::PrefixLists \
PVE::RS::SDN::RouteMaps \
diff --git a/pve-rs/debian/control b/pve-rs/debian/control
index 70367dc..521d966 100644
--- a/pve-rs/debian/control
+++ b/pve-rs/debian/control
@@ -20,6 +20,8 @@ Build-Depends: debhelper-compat (= 13),
librust-proxmox-apt-api-types-3+default-dev (>= 3.0.0-~~),
librust-proxmox-base64-1+default-dev,
librust-proxmox-config-digest-1+default-dev,
+ librust-proxmox-ebpf-0.1+default-dev,
+ librust-proxmox-ebpf-0.1+dhcp-dev,
librust-proxmox-frr-0.5+default-dev (>= 0.5.1-~~),
librust-proxmox-http-1+client-sync-dev (>= 1.0.2-~~),
librust-proxmox-http-1+client-trait-dev (>= 1.0.2-~~),
diff --git a/pve-rs/src/bindings/sdn/dhcp.rs b/pve-rs/src/bindings/sdn/dhcp.rs
new file mode 100644
index 0000000..a6cdc95
--- /dev/null
+++ b/pve-rs/src/bindings/sdn/dhcp.rs
@@ -0,0 +1,229 @@
+#[perlmod::package(name = "PVE::RS::SDN::Dhcp", lib = "pve_rs")]
+pub mod pve_rs_sdn_dhcp {
+ //! The `PVE::RS::SDN::Dhcp` package.
+ //!
+ //! Bindings for the per-tap eBPF DHCP responder. Interfaces and the records they answer with
+ //! are handed in here and served directly from the kernel. The calls that change the responder
+ //! are methods of a lock object, so nothing writes to it without holding the lock in the mode
+ //! the change needs.
+
+ use std::net::Ipv4Addr;
+
+ use anyhow::Error;
+ use serde::{Deserialize, Deserializer};
+
+ use proxmox_ebpf::dhcp::{DhcpSubsystem, Iface, Record, Unplugged};
+
+ /// One interface's DHCP answer, the address and every option the reply carries.
+ #[derive(Deserialize)]
+ pub struct DhcpRecord {
+ ip: Ipv4Addr,
+ prefixlen: u8,
+ server_id: Ipv4Addr,
+ lease: u32,
+ #[serde(default)]
+ router: Option<Ipv4Addr>,
+ #[serde(default)]
+ dns: Option<Ipv4Addr>,
+ #[serde(default)]
+ mtu: Option<u16>,
+ #[serde(deserialize_with = "mac_from_str")]
+ mac: [u8; 6],
+ }
+
+ // a MAC as six hex pairs
+ fn mac_from_str<'de, D: Deserializer<'de>>(d: D) -> Result<[u8; 6], D::Error> {
+ let text = String::deserialize(d)?;
+ let mut mac = [0u8; 6];
+ let mut parts = text.split(':');
+ for byte in &mut mac {
+ let part = parts
+ .next()
+ .ok_or_else(|| serde::de::Error::custom(format!("MAC {text} is too short")))?;
+ if part.len() != 2 || !part.bytes().all(|b| b.is_ascii_hexdigit()) {
+ return Err(serde::de::Error::custom(format!("MAC {text} is malformed")));
+ }
+ *byte = u8::from_str_radix(part, 16)
+ .map_err(|_| serde::de::Error::custom(format!("MAC {text} is malformed")))?;
+ }
+ if parts.next().is_some() {
+ return Err(serde::de::Error::custom(format!("MAC {text} is too long")));
+ }
+ Ok(mac)
+ }
+
+ /// One guest interface, the vnet it sits on if any, whether the responder runs on it and what
+ /// it answers there.
+ #[derive(Deserialize)]
+ pub struct DhcpIface {
+ name: String,
+ #[serde(default)]
+ vnet: Option<String>,
+ serve: bool,
+ #[serde(default)]
+ record: Option<DhcpRecord>,
+ }
+
+ /// Where an unplugged interface went, the vnet it sits on now, none for a plain bridge.
+ #[derive(Deserialize)]
+ pub struct DhcpTarget {
+ #[serde(default)]
+ vnet: Option<String>,
+ }
+
+ impl From<DhcpRecord> for Record {
+ fn from(r: DhcpRecord) -> Self {
+ Record {
+ ip: r.ip,
+ prefixlen: r.prefixlen,
+ server_id: r.server_id,
+ lease: r.lease,
+ router: r.router,
+ dns: r.dns,
+ mtu: r.mtu,
+ mac: r.mac,
+ }
+ }
+ }
+
+ impl From<DhcpIface> for Iface {
+ fn from(i: DhcpIface) -> Self {
+ Iface {
+ name: i.name,
+ vnet: i.vnet,
+ serve: i.serve,
+ record: i.record.map(Record::from),
+ }
+ }
+ }
+
+ pub(super) fn to_ifaces(ifaces: Vec<DhcpIface>) -> Vec<Iface> {
+ ifaces.into_iter().map(Iface::from).collect()
+ }
+
+ /// A tap unplug. Without a target the interface is gone or going and takes its record with
+ /// it. With one it was plugged onto a bridge the responder does not serve and keeps a marker
+ /// naming that place.
+ #[export]
+ pub fn detach(iface: &str, target: Option<DhcpTarget>) -> Result<(), Error> {
+ let unplugged = match target {
+ None => Unplugged::Gone,
+ Some(target) => Unplugged::MovedTo(target.vnet),
+ };
+ DhcpSubsystem::new().detach(iface, unplugged)
+ }
+}
+
+#[perlmod::package(name = "PVE::RS::SDN::Dhcp::SharedLock", lib = "pve_rs")]
+pub mod pve_rs_sdn_dhcp_shared_lock {
+ //! The `PVE::RS::SDN::Dhcp::SharedLock` package.
+ //!
+ //! The responder's lock taken shared, what a tap plug and a guest's own record change hold.
+ //! Released when the object goes away.
+
+ use anyhow::Error;
+ use perlmod::Value;
+
+ use proxmox_ebpf::dhcp::{DhcpSubsystem, Iface, Outcome, Record, SharedLock};
+
+ use super::pve_rs_sdn_dhcp::{DhcpIface, DhcpRecord, to_ifaces};
+
+ /// The lock as Perl holds it.
+ pub struct PerlSharedLock {
+ lock: SharedLock,
+ }
+
+ perlmod::declare_magic!(Box<PerlSharedLock> : &PerlSharedLock as "PVE::RS::SDN::Dhcp::SharedLock");
+
+ /// Class method: take the lock. Plugs hold it next to each other and wait only behind a full
+ /// pass or a record change.
+ #[export]
+ pub fn new(#[raw] class: Value) -> Result<Value, Error> {
+ let lock = DhcpSubsystem::new().lock_shared()?;
+ Ok(perlmod::instantiate_magic!(&class, MAGIC => Box::new(PerlSharedLock { lock })))
+ }
+
+ /// Method: a tap plug onto a vnet with the record it answers. Without a record the interface
+ /// gets no answer. Returns false when nothing is loaded yet. An interface gone already counts
+ /// as done.
+ #[export]
+ pub fn attach(
+ #[try_from_ref] this: &PerlSharedLock,
+ iface: String,
+ vnet: String,
+ record: Option<DhcpRecord>,
+ ) -> Result<bool, Error> {
+ let iface = Iface {
+ name: iface,
+ vnet: Some(vnet),
+ serve: true,
+ record: record.map(Record::from),
+ };
+ Ok(this.lock.attach(&iface)? == Outcome::Done)
+ }
+
+ /// Method: a record change for the listed interfaces by a caller holding the guest lock.
+ /// Returns false when nothing is loaded to write into, so a full pass is due.
+ #[export]
+ pub fn update(
+ #[try_from_ref] this: &PerlSharedLock,
+ ifaces: Vec<DhcpIface>,
+ ) -> Result<bool, Error> {
+ Ok(this.lock.update(&to_ifaces(ifaces))? == Outcome::Done)
+ }
+}
+
+#[perlmod::package(name = "PVE::RS::SDN::Dhcp::ExclusiveLock", lib = "pve_rs")]
+pub mod pve_rs_sdn_dhcp_exclusive_lock {
+ //! The `PVE::RS::SDN::Dhcp::ExclusiveLock` package.
+ //!
+ //! The responder's lock taken exclusively, what a full pass and a record change hold. Released
+ //! when the object goes away.
+
+ use anyhow::Error;
+ use perlmod::Value;
+
+ use proxmox_ebpf::dhcp::{DhcpSubsystem, ExclusiveLock, Outcome};
+
+ use super::pve_rs_sdn_dhcp::{DhcpIface, to_ifaces};
+
+ /// The lock as Perl holds it.
+ pub struct PerlExclusiveLock {
+ lock: ExclusiveLock,
+ }
+
+ perlmod::declare_magic!(Box<PerlExclusiveLock> : &PerlExclusiveLock as "PVE::RS::SDN::Dhcp::ExclusiveLock");
+
+ /// Class method: take the lock, waiting out every plug in flight.
+ #[export]
+ pub fn new(#[raw] class: Value) -> Result<Value, Error> {
+ let lock = DhcpSubsystem::new().lock_exclusive()?;
+ Ok(perlmod::instantiate_magic!(&class, MAGIC => Box::new(PerlExclusiveLock { lock })))
+ }
+
+ /// Method: the full pass over every guest interface of the cluster. Those found here get
+ /// their entry and a link where served.
+ #[export]
+ pub fn apply(
+ #[try_from_ref] this: &PerlExclusiveLock,
+ ifaces: Vec<DhcpIface>,
+ ) -> Result<(), Error> {
+ this.lock.apply(&to_ifaces(ifaces))
+ }
+
+ /// Method: a record change for the listed interfaces. Those not found here are skipped.
+ /// Returns false when nothing is loaded to write into, so a full pass is due.
+ #[export]
+ pub fn update(
+ #[try_from_ref] this: &PerlExclusiveLock,
+ ifaces: Vec<DhcpIface>,
+ ) -> Result<bool, Error> {
+ Ok(this.lock.update(&to_ifaces(ifaces))? == Outcome::Done)
+ }
+
+ /// Method: detach the responder everywhere and drop its pinned state.
+ #[export]
+ pub fn clear(#[try_from_ref] this: &PerlExclusiveLock) -> Result<(), Error> {
+ this.lock.clear()
+ }
+}
diff --git a/pve-rs/src/bindings/sdn/mod.rs b/pve-rs/src/bindings/sdn/mod.rs
index dcae046..4b99b8f 100644
--- a/pve-rs/src/bindings/sdn/mod.rs
+++ b/pve-rs/src/bindings/sdn/mod.rs
@@ -1,3 +1,4 @@
+pub(crate) mod dhcp;
pub(crate) mod fabrics;
pub(crate) mod prefix_lists;
pub(crate) mod route_maps;
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH pve-network v3 04/17] sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (2 preceding siblings ...)
2026-10-20 7:59 ` [PATCH proxmox-perl-rs v3 03/17] pve-rs: sdn: add bindings for the eBPF DHCP responder Hannes Laimer
@ 2026-10-20 7:59 ` Hannes Laimer
2026-10-20 13:15 ` Stefan Hanreich
2026-10-20 7:59 ` [PATCH pve-network v3 05/17] sdn: dhcp: update reservations when ip mappings change Hannes Laimer
` (14 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 7:59 UTC (permalink / raw)
To: pve-devel
A reservation rewrite dropped every line that no longer matched the
MAC cache and wrote nothing in its place. The guest behind such a
line was unknown to dnsmasq until its next start, so its requests
were ignored. The line is now replaced by the cache's current one.
Only a MAC the cache holds no address for is dropped.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 11 ++++++++++-
src/test/run_test_vnets_blackbox.pl | 28 ++++++++++++++++++++++++++++
2 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
index 477b700f..5c9496c0 100644
--- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
+++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
@@ -88,7 +88,9 @@ sub add_ip_mapping {
}
$parsed_ip6 = $1 if $parsed_ip6 && $parsed_ip6 =~ m/\[(\S+)\]/;
- #delete changed
+ # a line that no longer matches the cache is replaced by the cache's
+ # current one. Dropping it left that guest unknown to dnsmasq until
+ # its next start
if (
!defined($macdb->{macs}->{$parsed_mac})
|| ($parsed_ip4
@@ -99,6 +101,13 @@ sub add_ip_mapping {
&& $macdb->{macs}->{$parsed_mac}->{'ip6'} ne $parsed_ip6)
) {
$reload = 1;
+ # the MAC being added gets its line below, like a missing one
+ my $current = $macdb->{macs}->{$parsed_mac};
+ next if $parsed_mac eq $mac || !$current;
+ my $replacement = $parsed_mac;
+ $replacement .= ",$current->{ip4}" if $current->{ip4};
+ $replacement .= ",[$current->{ip6}]" if $current->{ip6};
+ print $out "$replacement\n" if $replacement ne $parsed_mac;
next;
}
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 9f4c424f..36225a92 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -963,4 +963,32 @@ run_test(
2,
);
+sub test_dnsmasq_stale_reservation_replaced {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+
+ my $stale = "da:65:8f:18:9b:01";
+ my $gone = "da:65:8f:18:9b:02";
+ my $added = "da:65:8f:18:9b:03";
+ $test_state->{macdb} = {
+ macs => {
+ $stale => { ip4 => '10.0.1.11' },
+ $added => { ip4 => '10.0.1.13' },
+ },
+ };
+ PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "$stale,10.0.1.10\n$gone,10.0.1.12\n");
+
+ PVE::Network::SDN::Dhcp::Dnsmasq->add_ip_mapping(
+ 'TESTZONE', $test_state->{macdb}, $added, '10.0.1.13', undef,
+ );
+
+ my @lines = sort grep { /\S/ } split(/\n/, PVE::Tools::file_get_contents($TMP_ETHERS_FILE));
+ eq_or_diff(
+ \@lines,
+ ["$stale,10.0.1.11", "$added,10.0.1.13"],
+ "$test_name: the stale line is replaced, the unknown one dropped",
+ );
+}
+
+run_test(\&test_dnsmasq_stale_reservation_replaced);
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH pve-network v3 05/17] sdn: dhcp: update reservations when ip mappings change
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (3 preceding siblings ...)
2026-10-20 7:59 ` [PATCH pve-network v3 04/17] sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it Hannes Laimer
@ 2026-10-20 7:59 ` Hannes Laimer
2026-10-20 13:42 ` Stefan Hanreich
2026-10-20 7:59 ` [PATCH pve-network v3 06/17] sdn: ipam: key the MAC cache by lower-case MAC, stop caching misses Hannes Laimer
` (13 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 7:59 UTC (permalink / raw)
To: pve-devel
Creating, updating or deleting an IP mapping through the API changed
the IPAM and the MAC cache, but never the zone's dnsmasq reservation
file. dnsmasq kept serving the old reservation, or none for a new
mapping, until the guest's next start. Deleting a guest NIC's
addresses left its reservation in the file as well.
A guest start, an API edit and the deletion of a NIC's addresses now
all rewrite the MAC's reservation from the cache, on the node the
change is made on. The other nodes catch up when the guest starts
there. The zone and its backend come from the applied config, and a
zone confined to other nodes is skipped. A failed update only warns,
since the IPAM records are already written.
The rewrite reads the cache only once it holds the file's lock. A
read from before could undo a rewrite that ran in between. The lock
is on a separate file, since a lock on the reservation file would not
survive the rename that replaces it. The MAC's line is written in full
from the cache, so a guest gaining an IPv6 address next to its IPv4
one gets it reserved too. MACs and addresses are compared in lower
case, since the cache keeps each as its writer spelled it. A cache
entry without any address no longer keeps a stale line.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/Network/SDN/Ips.pm | 7 +
src/PVE/Network/SDN/Dhcp.pm | 48 +++---
src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 110 ++++++++----
src/PVE/Network/SDN/Dhcp/Plugin.pm | 7 +-
src/PVE/Network/SDN/Vnets.pm | 5 +-
src/test/run_test_vnets_blackbox.pl | 250 +++++++++++++++++++++++++++-
6 files changed, 352 insertions(+), 75 deletions(-)
diff --git a/src/PVE/API2/Network/SDN/Ips.pm b/src/PVE/API2/Network/SDN/Ips.pm
index 5ff05e75..d7b682dd 100644
--- a/src/PVE/API2/Network/SDN/Ips.pm
+++ b/src/PVE/API2/Network/SDN/Ips.pm
@@ -46,6 +46,8 @@ __PACKAGE__->register_method({
eval { PVE::Network::SDN::Vnets::del_ip($vnet, $ip, '', $mac); };
die "$@\n" if $@;
+ PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+
return undef;
},
});
@@ -82,6 +84,8 @@ __PACKAGE__->register_method({
PVE::Network::SDN::Vnets::add_ip($vnet, $ip, '', $mac, undef);
+ PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+
return undef;
},
});
@@ -132,6 +136,9 @@ __PACKAGE__->register_method({
}
die "$error\n" if $error;
+
+ PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+
return undef;
},
});
diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
index 65e40d40..ab5d6b5b 100644
--- a/src/PVE/Network/SDN/Dhcp.pm
+++ b/src/PVE/Network/SDN/Dhcp.pm
@@ -6,7 +6,6 @@ use warnings;
use PVE::Cluster;
use PVE::Network::SDN;
-use PVE::Network::SDN::Ipams;
use PVE::Network::SDN::Subnets;
use PVE::Network::SDN::Dhcp::Plugin;
use PVE::Network::SDN::Dhcp::Dnsmasq;
@@ -22,37 +21,32 @@ sub plugin_types {
return PVE::Network::SDN::Dhcp::Plugin->lookup_types();
}
-sub add_mapping {
- my ($vnetid, $mac, $ip4, $ip6) = @_;
+# re-apply a MAC's mapping on this node from the current records, best effort
+# since the record write stays authoritative. A caller that knows the guest
+# passes its vmid, so only that guest's config is read
+sub update_mapping {
+ my ($vnetid, $mac, $vmid) = @_;
- my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid);
- return if !$vnet;
+ # a record without a MAC has no guest NIC behind it, so no backend serves it
+ return if !$mac;
- my $zoneid = $vnet->{zone};
- my $zone = PVE::Network::SDN::Zones::get_zone($zoneid);
+ eval {
+ # the backend to push to is the applied one, since a zone edited but
+ # not applied has nothing running yet
+ my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid, 1);
+ return if !$vnet;
- return if !$zone->{ipam} || !$zone->{dhcp};
+ my $zoneid = $vnet->{zone};
+ my $zone = PVE::Network::SDN::Zones::get_zone($zoneid, 1);
+ return if !$zone || !$zone->{ipam} || !$zone->{dhcp};
- my $dhcptype = $zone->{dhcp};
+ # a zone confined to other nodes has no backend running here
+ return if defined($zone->{nodes}) && !$zone->{nodes}->{ PVE::INotify::nodename() };
- my $macdb = PVE::Network::SDN::Ipams::read_macdb();
- my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($dhcptype);
- $dhcp_plugin->add_ip_mapping($zoneid, $macdb, $mac, $ip4, $ip6);
-}
-
-sub remove_mapping {
- my ($vnetid, $mac) = @_;
-
- my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid);
- return if !$vnet;
-
- my $zoneid = $vnet->{zone};
- my $zone = PVE::Network::SDN::Zones::get_zone($zoneid);
-
- return if !$zone->{ipam} || !$zone->{dhcp};
-
- my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($zone->{dhcp});
- $dhcp_plugin->del_ip_mapping($zoneid, $mac);
+ PVE::Network::SDN::Dhcp::Plugin->lookup($zone->{dhcp})
+ ->update_ip_mapping($zoneid, $mac, $vmid);
+ };
+ warn "could not update dhcp mapping for $mac: $@" if $@;
}
sub regenerate_config {
diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
index 5c9496c0..57ce1f73 100644
--- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
+++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
@@ -12,6 +12,7 @@ use File::Copy;
use Net::DBus;
use PVE::RESTEnvironment qw(log_warn);
+use PVE::Network::SDN::Ipams;
my $DNSMASQ_CONFIG_ROOT = '/etc/dnsmasq.d';
my $DNSMASQ_DEFAULT_ROOT = '/etc/default';
@@ -51,29 +52,65 @@ sub update_lease {
$manager->AddDhcpLease($ip4, $mac, \@hostname, undef, 0, 0, 0) if $ip4;
}
-sub add_ip_mapping {
- my ($class, $dhcpid, $macdb, $mac, $ip4, $ip6) = @_;
+sub update_ip_mapping {
+ my ($class, $dhcpid, $mac) = @_;
my $ethers_file = ethers_file($dhcpid);
my $ethers_tmp_file = "$ethers_file.tmp";
- my $reload = undef;
+ my ($ip4, $ip6);
+ my $reservation = undef;
+ my $changed = undef;
+
+ my $rewriteFn = sub {
+ # read under the ethers lock. A snapshot from before it would sweep a
+ # line another push wrote meanwhile, and an older push would put its
+ # addresses back
+
+ my $macdb = PVE::Network::SDN::Ipams::read_macdb();
+ # the cache spells a MAC and an address as their writer did, while the
+ # file holds one spelling of both. An entry without an address says
+ # nothing about its MAC, so a line of it is stale
+ my %macs;
+ for my $key (sort keys $macdb->{macs}->%*) {
+ my $entry = $macdb->{macs}->{$key};
+ my %addresses =
+ map { $_ => lc($entry->{$_}) } grep { defined($entry->{$_}) } qw(ip4 ip6);
+ next if !%addresses;
+ $macs{ lc($key) } = { %{ $macs{ lc($key) } // {} }, %addresses };
+ }
+
+ my $entry = $macs{ lc($mac) };
+ ($ip4, $ip6) = ($entry->{ip4}, $entry->{ip6}) if $entry;
+ if ($ip4 || $ip6) {
+ $reservation = $mac;
+ $reservation .= ",$ip4" if $ip4;
+ $reservation .= ",[$ip6]" if $ip6;
+ # an external IPAM spells addresses its own way, while the file holds one spelling
+ $reservation = lc($reservation);
+ }
- my $appendFn = sub {
+ # the first reservation of a zone finds no file yet
+ file_set_contents($ethers_file, '') if !-e $ethers_file;
open(my $in, '<', $ethers_file) or die "Could not open file '$ethers_file' $!\n";
open(my $out, '>', $ethers_tmp_file)
or die "Could not open file '$ethers_tmp_file' $!\n";
- my $match = undef;
+ my $kept = undef;
- my $line_no = 0;
while (my $line = <$in>) {
- $line_no++;
chomp($line);
+ next if $line !~ m/\S/;
my ($parsed_mac, $parsed_ip1, $parsed_ip2) = split(/,/, $line);
- if (!defined($parsed_mac)) {
- warn "failed to parse MAC from $dhcpid ethers file on line $line_no: '$line'\n";
+ # the MAC's own reservation is rewritten. An unchanged one stays in place
+ if (lc($parsed_mac) eq lc($mac)) {
+ if (defined($reservation) && !$kept && lc($line) eq $reservation) {
+ $kept = 1;
+ print $out "$line\n";
+ } else {
+ $changed = 1;
+ }
next;
}
@@ -81,50 +118,43 @@ sub add_ip_mapping {
if ($parsed_ip2) {
$parsed_ip4 = $parsed_ip1;
$parsed_ip6 = $parsed_ip2;
- } elsif (Net::IP::ip_is_ipv4($parsed_ip1)) {
+ } elsif (defined($parsed_ip1) && Net::IP::ip_is_ipv4($parsed_ip1)) {
$parsed_ip4 = $parsed_ip1;
- } else {
+ } elsif (defined($parsed_ip1)) {
$parsed_ip6 = $parsed_ip1;
}
$parsed_ip6 = $1 if $parsed_ip6 && $parsed_ip6 =~ m/\[(\S+)\]/;
+ # a line from before addresses were lower-cased may spell one in upper case
+ $parsed_ip6 = lc($parsed_ip6) if $parsed_ip6;
# a line that no longer matches the cache is replaced by the cache's
# current one. Dropping it left that guest unknown to dnsmasq until
# its next start
if (
- !defined($macdb->{macs}->{$parsed_mac})
+ !defined($macs{ lc($parsed_mac) })
|| ($parsed_ip4
- && $macdb->{macs}->{$parsed_mac}->{'ip4'}
- && $macdb->{macs}->{$parsed_mac}->{'ip4'} ne $parsed_ip4)
+ && $macs{ lc($parsed_mac) }->{'ip4'}
+ && $macs{ lc($parsed_mac) }->{'ip4'} ne $parsed_ip4)
|| ($parsed_ip6
- && $macdb->{macs}->{$parsed_mac}->{'ip6'}
- && $macdb->{macs}->{$parsed_mac}->{'ip6'} ne $parsed_ip6)
+ && $macs{ lc($parsed_mac) }->{'ip6'}
+ && $macs{ lc($parsed_mac) }->{'ip6'} ne $parsed_ip6)
) {
- $reload = 1;
- # the MAC being added gets its line below, like a missing one
- my $current = $macdb->{macs}->{$parsed_mac};
- next if $parsed_mac eq $mac || !$current;
- my $replacement = $parsed_mac;
+ $changed = 1;
+ my $current = $macs{ lc($parsed_mac) };
+ next if !$current;
+ my $replacement = lc($parsed_mac);
$replacement .= ",$current->{ip4}" if $current->{ip4};
$replacement .= ",[$current->{ip6}]" if $current->{ip6};
- print $out "$replacement\n" if $replacement ne $parsed_mac;
+ print $out "$replacement\n";
next;
}
- if ($parsed_mac eq $mac) {
- $match = 1 if $ip4 && $parsed_ip4 && $ip4;
- $match = 1 if $ip6 && $parsed_ip6 && $ip6;
- }
-
print $out "$line\n";
}
- if (!$match) {
- my $reservation = $mac;
- $reservation .= ",$ip4" if $ip4;
- $reservation .= ",[$ip6]" if $ip6;
+ if (defined($reservation) && !$kept) {
print $out "$reservation\n";
- $reload = 1;
+ $changed = 1;
}
close $in;
@@ -133,16 +163,22 @@ sub add_ip_mapping {
chmod 0644, $ethers_file;
};
- PVE::Tools::lock_file($ethers_file, 10, $appendFn);
+ # the rewrite replaces the reservation file, so a lock on the file itself
+ # would not outlive the rename
+ PVE::Tools::lock_file("$ethers_file.lock", 10, $rewriteFn);
if ($@) {
- warn "Unable to add $mac to the dnsmasq configuration: $@\n";
+ warn "Unable to update $mac in the dnsmasq configuration: $@\n";
return;
}
- my $service_name = "dnsmasq\@$dhcpid";
- systemctl_service('reload', $service_name) if $reload;
- update_lease($dhcpid, $ip4, $mac);
+ systemctl_service('reload', "dnsmasq\@$dhcpid") if $changed;
+
+ # the reservation is written, so a lease refresh that fails only warns
+ if ($ip4) {
+ eval { update_lease($dhcpid, $ip4, $mac) };
+ log_warn("could not update the dnsmasq lease of $mac: $@") if $@;
+ }
}
sub configure_subnet {
diff --git a/src/PVE/Network/SDN/Dhcp/Plugin.pm b/src/PVE/Network/SDN/Dhcp/Plugin.pm
index b5d32fa1..2fa8683c 100644
--- a/src/PVE/Network/SDN/Dhcp/Plugin.pm
+++ b/src/PVE/Network/SDN/Dhcp/Plugin.pm
@@ -22,8 +22,11 @@ sub private {
return $defaultData;
}
-sub add_ip_mapping {
- my ($class, $dhcpid, $macdb, $mac, $ip4, $ip6) = @_;
+# the MAC's records changed, so make the backend serve the current ones. With no
+# record left its reservation goes. The vmid names the guest when the caller
+# knows it, which an API edit does not
+sub update_ip_mapping {
+ my ($class, $dhcpid, $mac, $vmid) = @_;
die 'implement in sub class';
}
diff --git a/src/PVE/Network/SDN/Vnets.pm b/src/PVE/Network/SDN/Vnets.pm
index c327a4be..84b08037 100644
--- a/src/PVE/Network/SDN/Vnets.pm
+++ b/src/PVE/Network/SDN/Vnets.pm
@@ -211,6 +211,8 @@ sub del_ips_from_mac {
PVE::Network::SDN::Vnets::del_ip($vnetid, $ip4, $hostname, $mac) if $ip4;
PVE::Network::SDN::Vnets::del_ip($vnetid, $ip6, $hostname, $mac) if $ip6;
+ PVE::Network::SDN::Dhcp::update_mapping($vnetid, $mac) if $ip4 || $ip6;
+
return ($ip4, $ip6);
}
@@ -225,11 +227,12 @@ sub add_dhcp_mapping {
return if !$zone->{ipam} || !$zone->{dhcp};
my ($ip4, $ip6) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ # one push at the end covers the allocated and the existing records alike
add_next_free_cidr($vnetid, $name, $mac, "$vmid", undef, 1, 4) if !$ip4;
add_next_free_cidr($vnetid, $name, $mac, "$vmid", undef, 1, 6) if !$ip6;
($ip4, $ip6) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
- PVE::Network::SDN::Dhcp::add_mapping($vnetid, $mac, $ip4, $ip6) if $ip4 || $ip6;
+ PVE::Network::SDN::Dhcp::update_mapping($vnetid, $mac, $vmid) if $ip4 || $ip6;
}
1;
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 36225a92..d324538e 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -46,6 +46,7 @@ sub clear_test_state {
vnets_config => {},
macdb => {},
ipamdb => {},
+ dnsmasq_calls => [],
ipam_config => {
'ids' => {
'pve' => {
@@ -220,7 +221,9 @@ $mocked_sdn_dhcp_dnsmasq->mock(
assert_dnsmasq_installed => sub { return 1; },
before_configure => sub { },
ethers_file => sub { return "/tmp/ethers"; },
- systemctl_service => sub { },
+ systemctl_service => sub {
+ push $test_state->{dnsmasq_calls}->@*, $_[0];
+ },
update_lease => sub { },
);
@@ -239,6 +242,11 @@ $mocked_rpc_env_obj->mock(
check_any => sub { return 1; },
);
+my $mocked_pve_inotify = Test::MockModule->new('PVE::INotify');
+$mocked_pve_inotify->mock(
+ nodename => sub { return 'localnode'; },
+);
+
my $mocked_pve_cluster_obj = Test::MockModule->new('PVE::Cluster');
$mocked_pve_cluster_obj->mock(
check_cfs_quorum => sub { return 1; },
@@ -288,6 +296,11 @@ sub create_zone {
return $zone;
}
+sub update_zone {
+ my ($zoneid, $params) = @_;
+ PVE::API2::Network::SDN::Zones->update({ zone => $zoneid, %$params });
+}
+
sub get_vnet {
my ($id) = @_;
return eval { PVE::API2::Network::SDN::Vnets->read({ vnet => $id }); };
@@ -330,6 +343,16 @@ sub create_ip {
return PVE::API2::Network::SDN::Ips->ipcreate($param);
}
+sub update_ip {
+ my ($param) = @_;
+ return PVE::API2::Network::SDN::Ips->ipupdate($param);
+}
+
+sub delete_ip {
+ my ($param) = @_;
+ return PVE::API2::Network::SDN::Ips->ipdelete($param);
+}
+
sub run_test {
my $test = shift;
clear_test_state();
@@ -977,18 +1000,229 @@ sub test_dnsmasq_stale_reservation_replaced {
};
PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "$stale,10.0.1.10\n$gone,10.0.1.12\n");
- PVE::Network::SDN::Dhcp::Dnsmasq->add_ip_mapping(
- 'TESTZONE', $test_state->{macdb}, $added, '10.0.1.13', undef,
- );
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping('TESTZONE', $added);
- my @lines = sort grep { /\S/ } split(/\n/, PVE::Tools::file_get_contents($TMP_ETHERS_FILE));
- eq_or_diff(
- \@lines,
- ["$stale,10.0.1.11", "$added,10.0.1.13"],
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$stale,10.0.1.11\n$added,10.0.1.13\n",
"$test_name: the stale line is replaced, the unknown one dropped",
);
}
run_test(\&test_dnsmasq_stale_reservation_replaced);
+# -------------- dnsmasq mapping pushes
+
+sub test_dnsmasq_mapping_push {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+
+ # the first push of a zone finds no ethers file yet
+ unlink($TMP_ETHERS_FILE);
+
+ my $ethers = sub { PVE::Tools::file_get_contents($TMP_ETHERS_FILE) };
+ my $reloads = sub {
+ my $count = scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*);
+ $test_state->{dnsmasq_calls} = [];
+ return $count;
+ };
+
+ eval { nic_start($vnetid, $mac, "999", "testhostname"); };
+ if ($@) {
+ fail("$test_name: nic_start: $@");
+ return;
+ }
+ is($ethers->(), "$mac,10.0.0.100\n", "$test_name: guest start reserves the allocation");
+ is($reloads->(), 1, "$test_name: guest start reloads once");
+
+ # the records exist already, so the restart pushes them without a change
+ nic_start($vnetid, $mac, "999", "testhostname");
+ is($ethers->(), "$mac,10.0.0.100\n", "$test_name: restart keeps the reservation");
+ is($reloads->(), 0, "$test_name: restart without a change does not reload");
+
+ update_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ is($ethers->(), "$mac,10.0.0.150\n", "$test_name: mapping edit replaces the reservation");
+ is($reloads->(), 1, "$test_name: mapping edit reloads once");
+
+ # a push writes what the cache holds, so a repeated one rewrites nothing
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
+ is($ethers->(), "$mac,10.0.0.150\n", "$test_name: a repeated push keeps the reservation");
+ is($reloads->(), 0, "$test_name: a repeated push changes nothing");
+
+ # the cache and the file may spell a MAC differently, but it is one MAC
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, uc($mac));
+ is(
+ $ethers->(),
+ "$mac,10.0.0.150\n",
+ "$test_name: a push spelled differently keeps the line as it is",
+ );
+ is($reloads->(), 0, "$test_name: and reloads nothing");
+
+ delete_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ is($ethers->(), "", "$test_name: mapping delete drops the reservation");
+ is($reloads->(), 1, "$test_name: mapping delete reloads once");
+
+ # a record without a MAC has no NIC to serve, so nothing is pushed for it
+ my @warnings;
+ {
+ local $SIG{__WARN__} = sub { push @warnings, @_; };
+ create_ip({ zone => $zoneid, vnet => $vnetid, ip => "10.0.0.160" });
+ delete_ip({ zone => $zoneid, vnet => $vnetid, ip => "10.0.0.160" });
+ }
+ is($reloads->(), 0, "$test_name: a record without a MAC is not pushed");
+ eq_or_diff(\@warnings, [], "$test_name: and warns nothing");
+
+ # a zone confined to other nodes runs no dnsmasq here, so an edit is not
+ # pushed on this node
+ update_zone($zoneid, { nodes => 'other' });
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ is($ethers->(), "", "$test_name: a zone confined to other nodes gets no push here");
+ is($reloads->(), 0, "$test_name: nothing to reload for it");
+}
+
+run_test(\&test_dnsmasq_mapping_push);
+
+sub test_dnsmasq_dual_stack_and_sweep {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "8888::/64",
+ gateway => "8888::1",
+ 'dhcp-range' => ["start-address=8888::100,end-address=8888::200"],
+ });
+
+ # a reservation of a MAC the cache does not know is swept by the next
+ # rewrite, and both addresses of the guest go on one line
+ PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "aa:bb:cc:dd:ee:ff,10.0.0.77\n");
+ $test_state->{dnsmasq_calls} = [];
+
+ eval { nic_start($vnetid, $mac, "999", "testhostname"); };
+ if ($@) {
+ fail("$test_name: nic_start: $@");
+ return;
+ }
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$mac,10.0.0.100,[8888::100]\n",
+ "$test_name: dual-stack reservation on one line, the unknown MAC swept",
+ );
+ is(
+ scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
+ 1,
+ "$test_name: one reload for the rewrite",
+ );
+
+ # an external IPAM may spell an address in upper case, but the file holds
+ # one spelling
+ $test_state->{macdb}->{macs}->{$mac}->{ip6} = '8888::AB';
+ $test_state->{dnsmasq_calls} = [];
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$mac,10.0.0.100,[8888::ab]\n",
+ "$test_name: the reservation is written in one spelling",
+ );
+ is(
+ scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
+ 1,
+ "$test_name: one reload for the changed address",
+ );
+
+ # an entry without an address, as a miss once left behind, says nothing
+ # about its MAC, so a line of it is swept
+ my $stale = 'da:65:8f:18:9b:70';
+ $test_state->{macdb}->{macs}->{$stale} = { ip4 => undef, ip6 => undef };
+ PVE::Tools::file_set_contents(
+ $TMP_ETHERS_FILE,
+ "$mac,10.0.0.100,[8888::ab]\n$stale,10.0.0.101\n",
+ );
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$mac,10.0.0.100,[8888::ab]\n",
+ "$test_name: a line of a MAC cached without an address is swept",
+ );
+
+ # a line with an upper-case IPv6 address from before they were lower-cased is kept
+ my $other = 'da:65:8f:18:9b:71';
+ $test_state->{macdb}->{macs}->{$other} = { ip4 => '10.0.0.102', ip6 => '8888::AB' };
+ PVE::Tools::file_set_contents(
+ $TMP_ETHERS_FILE,
+ "$mac,10.0.0.100,[8888::ab]\n$other,10.0.0.102,[8888::AB]\n",
+ );
+ $test_state->{dnsmasq_calls} = [];
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$mac,10.0.0.100,[8888::ab]\n$other,10.0.0.102,[8888::AB]\n",
+ "$test_name: a line of another MAC spelled by the old code is kept",
+ );
+ is(
+ scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
+ 0,
+ "$test_name: without a reload",
+ );
+}
+
+run_test(\&test_dnsmasq_dual_stack_and_sweep);
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH pve-network v3 06/17] sdn: ipam: key the MAC cache by lower-case MAC, stop caching misses
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (4 preceding siblings ...)
2026-10-20 7:59 ` [PATCH pve-network v3 05/17] sdn: dhcp: update reservations when ip mappings change Hannes Laimer
@ 2026-10-20 7:59 ` Hannes Laimer
2026-10-20 13:10 ` Stefan Hanreich
2026-10-20 7:59 ` [PATCH pve-network v3 07/17] sdn: dhcp: make the lease time configurable per subnet Hannes Laimer
` (12 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 7:59 UTC (permalink / raw)
To: pve-devel
Generated guest MACs are upper case, but the API accepts either
spelling. The cache was keyed by the MAC as its writer spelled it. A
deletion in the other spelling missed the entry, so the cache kept
answering with the deleted address. Entries cached before this
change keep their spelling. A lookup and a deletion still find them,
and the next write for their MAC moves them to the lower-case key.
A lookup that missed the cache stored the IPAM's empty answer, and
any entry counted as a hit. So a record created later directly in the
IPAM stayed hidden, and a guest start allocated another address. An
empty answer is no longer stored, and an entry without an address
counts as a miss.
The lookup also wrote the whole cache back without the cluster lock
the other writers take. A change made between its read and its write
was lost. The lookup now takes the lock and reads the cache again
before it stores its answer.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/Network/SDN/Ipams.pm | 61 +++++++++++---
src/test/run_test_vnets_blackbox.pl | 118 ++++++++++++++++++++++++++++
2 files changed, 168 insertions(+), 11 deletions(-)
diff --git a/src/PVE/Network/SDN/Ipams.pm b/src/PVE/Network/SDN/Ipams.pm
index 179bdf76..d8051d8d 100644
--- a/src/PVE/Network/SDN/Ipams.pm
+++ b/src/PVE/Network/SDN/Ipams.pm
@@ -47,14 +47,27 @@ sub write_macdb {
cfs_write_file($macdb_filename, $data);
}
+# an entry under another spelling moves to the lower-case key
+my sub fold_spellings {
+ my ($db, $mac) = @_;
+
+ for my $key (grep { $_ ne $mac && lc($_) eq $mac } reverse sort keys $db->{macs}->%*) {
+ my $old = delete $db->{macs}->{$key};
+ $db->{macs}->{$mac}->{$_} //= $old->{$_} for grep { defined($old->{$_}) } qw(ip4 ip6);
+ }
+}
+
+# the cache is keyed by the MAC in lower case, whatever spelling its writers use
sub add_cache_mac_ip {
my ($mac, $ip) = @_;
+ $mac = lc($mac);
cfs_lock_file(
$macdb_filename,
undef,
sub {
my $db = read_macdb();
+ fold_spellings($db, $mac);
if (Net::IP::ip_is_ipv4($ip)) {
$db->{macs}->{$mac}->{ip4} = $ip;
} else {
@@ -74,13 +87,17 @@ sub del_cache_mac_ip {
undef,
sub {
my $db = read_macdb();
- if (Net::IP::ip_is_ipv4($ip)) {
- delete $db->{macs}->{$mac}->{ip4};
- } else {
- delete $db->{macs}->{$mac}->{ip6};
+ # entries from before the cache was keyed in lower case carry their writer's spelling
+ for my $key (grep { lc($_) eq lc($mac) } keys $db->{macs}->%*) {
+ if (Net::IP::ip_is_ipv4($ip)) {
+ delete $db->{macs}->{$key}->{ip4};
+ } else {
+ delete $db->{macs}->{$key}->{ip6};
+ }
+ delete $db->{macs}->{$key}
+ if !defined($db->{macs}->{$key}->{ip4})
+ && !defined($db->{macs}->{$key}->{ip6});
}
- delete $db->{macs}->{$mac}
- if !defined($db->{macs}->{$mac}->{ip4}) && !defined($db->{macs}->{$mac}->{ip6});
write_macdb($db);
},
);
@@ -136,16 +153,38 @@ sub get_ips_from_mac {
my ($mac, $zoneid, $zone) = @_;
my $macdb = read_macdb();
- return ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6}) if $macdb->{macs}->{$mac};
+ # entries from before the cache was keyed in lower case carry their writer's
+ # spelling, and the lower-case one wins where two hold the same family
+ my %cached;
+ for my $key (grep { lc($_) eq lc($mac) } sort keys $macdb->{macs}->%*) {
+ my $entry = $macdb->{macs}->{$key};
+ $cached{$_} = $entry->{$_} for grep { defined($entry->{$_}) } qw(ip4 ip6);
+ }
+
+ # an entry holding no address says nothing about the MAC, so ask the IPAM
+ return ($cached{ip4}, $cached{ip6}) if defined($cached{ip4}) || defined($cached{ip6});
my $plugin_config = get_plugin_config($zone);
my $plugin = PVE::Network::SDN::Ipams::Plugin->lookup($plugin_config->{type});
- ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6}) =
- $plugin->get_ips_from_mac($plugin_config, $mac, $zoneid);
+ my ($ip4, $ip6) = $plugin->get_ips_from_mac($plugin_config, $mac, $zoneid);
+
+ # an empty answer is not cached, since the record may simply not exist yet
+ return if !defined($ip4) && !defined($ip6);
- write_macdb($macdb);
+ cfs_lock_file(
+ $macdb_filename,
+ undef,
+ sub {
+ my $db = read_macdb();
+ fold_spellings($db, lc($mac));
+ $db->{macs}->{ lc($mac) }->{ip4} = $ip4 if defined($ip4);
+ $db->{macs}->{ lc($mac) }->{ip6} = $ip6 if defined($ip6);
+ write_macdb($db);
+ },
+ );
+ warn "$@" if $@;
- return ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6});
+ return ($ip4, $ip6);
}
1;
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index d324538e..476e7f8d 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -1118,6 +1118,124 @@ sub test_dnsmasq_mapping_push {
run_test(\&test_dnsmasq_mapping_push);
+sub test_ipam_cache_misses {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+
+ my @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ is(scalar(grep { defined } @ips), 0, "$test_name: an unknown MAC has no answer");
+ ok(!exists $test_state->{macdb}->{macs}->{$mac}, "$test_name: the miss is not cached");
+
+ # an entry holding no address, left behind by an earlier miss, does not
+ # hide a record created since
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.50",
+ });
+ $test_state->{macdb}->{macs}->{$mac} = { ip4 => undef, ip6 => undef };
+ @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ is($ips[0], "10.0.0.50", "$test_name: an address-less entry counts as a miss");
+ is(
+ $test_state->{macdb}->{macs}->{$mac}->{ip4},
+ "10.0.0.50",
+ "$test_name: the answer is cached",
+ );
+}
+
+run_test(\&test_ipam_cache_misses);
+
+sub test_ipam_cache_case {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # an entry written under the config's spelling before the cache was keyed in
+ # lower case is read and released like any other
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ $test_state->{macdb}->{macs}->{ uc($mac) } = { ip4 => '10.0.0.150' };
+ my ($ip4) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ is($ip4, '10.0.0.150', "$test_name: an old entry is found however it is spelled");
+ PVE::Network::SDN::Ipams::del_cache_mac_ip($mac, '10.0.0.150');
+ ok(
+ !(grep { lc($_) eq $mac } keys $test_state->{macdb}->{macs}->%*),
+ "$test_name: and released",
+ );
+
+ # a write spelled either way moves such an entry to the lower-case key
+ # and keeps what it held
+ $test_state->{macdb}->{macs}->{ uc($mac) } = { ip4 => '10.0.0.150' };
+ PVE::Network::SDN::Ipams::add_cache_mac_ip(uc($mac), 'fd00::150');
+ is_deeply(
+ [grep { lc($_) eq $mac } keys $test_state->{macdb}->{macs}->%*],
+ [$mac],
+ "$test_name: a write leaves one entry for the MAC",
+ );
+ is_deeply(
+ $test_state->{macdb}->{macs}->{$mac},
+ { ip4 => '10.0.0.150', ip6 => 'fd00::150' },
+ "$test_name: holding both addresses",
+ );
+
+ # the lookup's write moves an address-less entry under another spelling as well
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.160",
+ });
+ $test_state->{macdb}->{macs} = { uc($mac) => { ip4 => undef, ip6 => undef } };
+ my @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ is($ips[0], '10.0.0.160', "$test_name: a lookup past an address-less entry asks the IPAM");
+ is_deeply(
+ [sort keys $test_state->{macdb}->{macs}->%*],
+ [$mac],
+ "$test_name: and its write leaves one entry for the MAC",
+ );
+}
+
+run_test(\&test_ipam_cache_case);
+
sub test_dnsmasq_dual_stack_and_sweep {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH pve-network v3 07/17] sdn: dhcp: make the lease time configurable per subnet
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (5 preceding siblings ...)
2026-10-20 7:59 ` [PATCH pve-network v3 06/17] sdn: ipam: key the MAC cache by lower-case MAC, stop caching misses Hannes Laimer
@ 2026-10-20 7:59 ` Hannes Laimer
2026-10-20 13:17 ` Stefan Hanreich
2026-10-20 7:59 ` [PATCH pve-network v3 08/17] sdn: dhcp: require a backend only if a zone on this node uses it Hannes Laimer
` (11 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 7:59 UTC (permalink / raw)
To: pve-devel
dnsmasq hands out infinite leases, so a guest asks for its address
again only at its next reboot or network restart. A finite lease makes
it renew within the lease time. An address changed in the IPAM goes to
dnsmasq on the node the change is made on, so a guest there picks up
the new address at its next renewal. Guests on other nodes get it only
at their next start there.
A subnet without a lease time keeps infinite leases.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 3 +-
src/PVE/Network/SDN/SubnetPlugin.pm | 11 +++++++
src/test/run_test_vnets_blackbox.pl | 51 +++++++++++++++++++++++++++++
3 files changed, 64 insertions(+), 1 deletion(-)
diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
index 57ce1f73..c6c9ce1d 100644
--- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
+++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
@@ -200,7 +200,8 @@ sub configure_subnet {
$mask = join('.', unpack("C4", pack("N", $mask)));
}
- push @{$config}, "dhcp-range=set:$tag,$network,static,$mask,infinite";
+ my $lease = $subnet_config->{'dhcp-lease-time'} // 'infinite';
+ push @{$config}, "dhcp-range=set:$tag,$network,static,$mask,$lease";
my $option_string;
if (ip_is_ipv6($subnet_config->{network})) {
diff --git a/src/PVE/Network/SDN/SubnetPlugin.pm b/src/PVE/Network/SDN/SubnetPlugin.pm
index e2a0e50d..29edf7b6 100644
--- a/src/PVE/Network/SDN/SubnetPlugin.pm
+++ b/src/PVE/Network/SDN/SubnetPlugin.pm
@@ -177,6 +177,16 @@ sub properties {
description => 'IP address for the DNS server',
optional => 1,
},
+ 'dhcp-lease-time' => {
+ type => 'integer',
+ minimum => 60,
+ maximum => 4294967295,
+ description =>
+ 'Lease time in seconds for DHCP answers. Without it dnsmasq hands out'
+ . ' infinite leases, and it raises anything below two minutes to two'
+ . ' minutes.',
+ optional => 1,
+ },
};
}
@@ -189,6 +199,7 @@ sub options {
dnszoneprefix => { optional => 1 },
'dhcp-range' => { optional => 1 },
'dhcp-dns-server' => { optional => 1 },
+ 'dhcp-lease-time' => { optional => 1 },
};
}
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 476e7f8d..dc54bba3 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -334,6 +334,11 @@ sub create_subnet {
PVE::API2::Network::SDN::Subnets->create($params);
}
+sub update_subnet {
+ my ($params) = @_;
+ PVE::API2::Network::SDN::Subnets->update($params);
+}
+
sub get_ipam_entries {
return PVE::API2::Network::SDN::Ipams->ipamindex({ ipam => "pve" });
}
@@ -1118,6 +1123,52 @@ sub test_dnsmasq_mapping_push {
run_test(\&test_dnsmasq_mapping_push);
+sub test_dnsmasq_lease_time {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+
+ # the subnet's lease time reaches the range, and infinite stays the default
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ my $range = sub {
+ my $subnets = PVE::Network::SDN::Vnets::get_subnets($vnetid, 1);
+ my $config = [];
+ for my $id (sort keys %$subnets) {
+ PVE::Network::SDN::Dhcp::Dnsmasq->configure_subnet(
+ $config, $zoneid, $vnetid, $subnets->{$id},
+ );
+ }
+ my ($line) = grep { m/^dhcp-range=/ } @$config;
+ return $line;
+ };
+ like($range->(), qr/,infinite$/, "$test_name: no lease time means infinite leases");
+ update_subnet({
+ vnet => $vnetid,
+ subnet => "$zoneid-10.0.0.0-24",
+ 'dhcp-lease-time' => 300,
+ });
+ like($range->(), qr/,300$/, "$test_name: the lease time reaches the range");
+}
+
+run_test(\&test_dnsmasq_lease_time);
+
sub test_ipam_cache_misses {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH pve-network v3 08/17] sdn: dhcp: require a backend only if a zone on this node uses it
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (6 preceding siblings ...)
2026-10-20 7:59 ` [PATCH pve-network v3 07/17] sdn: dhcp: make the lease time configurable per subnet Hannes Laimer
@ 2026-10-20 7:59 ` Hannes Laimer
2026-10-20 13:17 ` Stefan Hanreich
2026-10-20 7:59 ` [PATCH pve-network v3 09/17] sdn: dhcp: add an eBPF backend answering on each guest interface Hannes Laimer
` (10 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 7:59 UTC (permalink / raw)
To: pve-devel
A network reload failed on a node without the dnsmasq package as soon
as any zone had DHCP, even a zone confined to other nodes. On a node
with dnsmasq, the reload wrote such a zone's configuration and started
an instance for it, although none of the zone's vnets exist there.
A reload now skips the zones confined to other nodes, so a backend is
required and started only for the zones on the reloading node.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/Network/SDN/Dhcp.pm | 13 +++++++--
src/test/run_test_vnets_blackbox.pl | 43 +++++++++++++++++++++++++++++
2 files changed, 53 insertions(+), 3 deletions(-)
diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
index ab5d6b5b..2a023534 100644
--- a/src/PVE/Network/SDN/Dhcp.pm
+++ b/src/PVE/Network/SDN/Dhcp.pm
@@ -63,17 +63,24 @@ sub regenerate_config {
my $plugins = PVE::Network::SDN::Dhcp::Plugin->lookup_types();
- my $any_zone_needs_dhcp = grep { $_->{dhcp} } values $zone_cfg->{ids}->%*;
+ my %plugin_needed = ();
+ for my $zone (values $zone_cfg->{ids}->%*) {
+ next if !$zone->{dhcp};
+ # a zone confined to other nodes runs no backend here
+ next if defined($zone->{nodes}) && !$zone->{nodes}->{$nodename};
+ $plugin_needed{ $zone->{dhcp} } = 1;
+ }
foreach my $plugin_name (@$plugins) {
my $plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($plugin_name);
- eval { $plugin->before_regenerate(!$any_zone_needs_dhcp) };
+ eval { $plugin->before_regenerate(!$plugin_needed{$plugin_name}) };
die "Could not run before_regenerate for DHCP plugin $plugin_name $@\n" if $@;
}
foreach my $zoneid (sort keys %{ $zone_cfg->{ids} }) {
my $zone = $zone_cfg->{ids}->{$zoneid};
next if !$zone->{dhcp};
+ next if defined($zone->{nodes}) && !$zone->{nodes}->{$nodename};
my $dhcp_plugin_name = $zone->{dhcp};
my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($dhcp_plugin_name);
@@ -114,7 +121,7 @@ sub regenerate_config {
warn "Could not configure vnet $vnetid: $@\n" if $@;
}
- eval { $dhcp_plugin->after_configure($zoneid, !$any_zone_needs_dhcp) };
+ eval { $dhcp_plugin->after_configure($zoneid, !$plugin_needed{$dhcp_plugin_name}) };
warn "Could not run after_configure for DHCP server $zoneid $@\n" if $@;
}
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index dc54bba3..cda4b7ef 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -104,6 +104,13 @@ my $mocked_pve_sdn;
$mocked_pve_sdn = Test::MockModule->new('PVE::Network::SDN');
$mocked_pve_sdn->mock(
cfs_lock_file => $mocked_cfs_lock_file,
+ running_config => sub {
+ return {
+ zones => $test_state->{zones_config},
+ vnets => $test_state->{vnets_config},
+ subnets => $test_state->{subnets_config},
+ };
+ },
);
my $mocked_pve_tools = Test::MockModule->new('PVE::Tools');
@@ -1169,6 +1176,42 @@ sub test_dnsmasq_lease_time {
run_test(\&test_dnsmasq_lease_time);
+sub test_dhcp_backend_needed_per_node {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+
+ # a backend is told whether a zone on this node uses it. A zone confined
+ # to other nodes runs nothing here
+ my $asked = [];
+ my $configured = [];
+ $mocked_sdn_dhcp_dnsmasq->mock(
+ before_regenerate => sub { push @$asked, $_[1] ? 'optional' : 'needed'; },
+ after_configure => sub { push @$configured, $_[1]; },
+ );
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => "DNSZONE",
+ nodes => 'other',
+ });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ $asked,
+ ['optional'],
+ "$test_name: a dnsmasq zone confined elsewhere needs no dnsmasq here",
+ );
+ eq_or_diff($configured, [], "$test_name: and is not configured here");
+
+ @$asked = ();
+ update_zone("DNSZONE", { delete => 'nodes' });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff($asked, ['needed'], "$test_name: a dnsmasq zone on this node needs it");
+ eq_or_diff($configured, ['DNSZONE'], "$test_name: and is configured");
+ $mocked_sdn_dhcp_dnsmasq->unmock($_) for qw(before_regenerate after_configure);
+}
+
+run_test(\&test_dhcp_backend_needed_per_node);
+
sub test_ipam_cache_misses {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH pve-network v3 09/17] sdn: dhcp: add an eBPF backend answering on each guest interface
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (7 preceding siblings ...)
2026-10-20 7:59 ` [PATCH pve-network v3 08/17] sdn: dhcp: require a backend only if a zone on this node uses it Hannes Laimer
@ 2026-10-20 7:59 ` Hannes Laimer
2026-10-20 13:10 ` Stefan Hanreich
2026-10-20 7:59 ` [PATCH pve-network v3 10/17] sdn: zones: report interface plugs and unplugs to the DHCP backends Hannes Laimer
` (9 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 7:59 UTC (permalink / raw)
To: pve-devel
The ebpf DHCP backend answers DHCPv4 from a program in the kernel on
every guest interface of its zones, with the address the SDN MAC cache
holds for the interface's MAC. No server runs per zone, and IPv6
subnets are not served. A subnet without a gateway is served under a
link-local server identifier no host owns, so its guests renew by
broadcast at their rebinding time.
An apply runs a full pass on each node, and on a node that no zone
with the ebpf backend includes, the pass removes the responder
altogether. A record change rewrites the answers for its MAC on the
node the change is made on, and the other nodes catch up at their
next apply.
The responder is pinned in bpffs and does not survive a reboot, so a
boot hook runs a full pass. Removing the package deletes those pins,
since nothing would detach the responder afterward.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
debian/libpve-network-perl.prerm | 13 +
src/PVE/API2/Network/SDN/Zones.pm | 3 +-
src/PVE/Network/SDN/Dhcp.pm | 79 ++++
src/PVE/Network/SDN/Dhcp/Ebpf.pm | 308 +++++++++++++
src/PVE/Network/SDN/Dhcp/Makefile | 2 +-
src/PVE/Network/SDN/Dhcp/Plugin.pm | 19 +
src/PVE/Network/SDN/SubnetPlugin.pm | 4 +-
src/test/run_test_vnets_blackbox.pl | 642 ++++++++++++++++++++++++++++
8 files changed, 1066 insertions(+), 4 deletions(-)
create mode 100755 debian/libpve-network-perl.prerm
create mode 100644 src/PVE/Network/SDN/Dhcp/Ebpf.pm
diff --git a/debian/libpve-network-perl.prerm b/debian/libpve-network-perl.prerm
new file mode 100755
index 00000000..d28c74fe
--- /dev/null
+++ b/debian/libpve-network-perl.prerm
@@ -0,0 +1,13 @@
+#!/bin/sh
+
+set -e
+
+case "$1" in
+ remove)
+ # the eBPF DHCP responder stays attached through its pins in bpffs, and with
+ # the package gone nothing would ever update or detach it
+ rm -rf /sys/fs/bpf/proxmox-ebpf/dhcp
+ ;;
+esac
+
+exit 0
diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm
index b897cbdf..0e907260 100644
--- a/src/PVE/API2/Network/SDN/Zones.pm
+++ b/src/PVE/API2/Network/SDN/Zones.pm
@@ -16,6 +16,7 @@ use PVE::Network::SDN::Dns;
use PVE::Network::SDN::Subnets;
use PVE::Network::SDN::Vnets;
use PVE::Network::SDN;
+use PVE::Network::SDN::Dhcp;
use PVE::Network::SDN::Zones::EvpnPlugin;
use PVE::Network::SDN::Zones::FaucetPlugin;
@@ -90,7 +91,7 @@ my $ZONE_PROPERTIES = {
},
dhcp => {
type => 'string',
- enum => ['dnsmasq'],
+ enum => PVE::Network::SDN::Dhcp->plugin_types(),
optional => 1,
description => 'Name of DHCP server backend for this zone.',
},
diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
index 2a023534..cf805a03 100644
--- a/src/PVE/Network/SDN/Dhcp.pm
+++ b/src/PVE/Network/SDN/Dhcp.pm
@@ -9,6 +9,7 @@ use PVE::Network::SDN;
use PVE::Network::SDN::Subnets;
use PVE::Network::SDN::Dhcp::Plugin;
use PVE::Network::SDN::Dhcp::Dnsmasq;
+use PVE::Network::SDN::Dhcp::Ebpf;
use PVE::INotify;
@@ -17,6 +18,9 @@ PVE::Network::SDN::Dhcp::Plugin->init();
PVE::Network::SDN::Dhcp::Dnsmasq->register();
PVE::Network::SDN::Dhcp::Dnsmasq->init();
+PVE::Network::SDN::Dhcp::Ebpf->register();
+PVE::Network::SDN::Dhcp::Ebpf->init();
+
sub plugin_types {
return PVE::Network::SDN::Dhcp::Plugin->lookup_types();
}
@@ -49,6 +53,81 @@ sub update_mapping {
warn "could not update dhcp mapping for $mac: $@" if $@;
}
+# the NIC lines come from pmxcfs in one go, so no guest config gets parsed for
+# this. With a vmid only that guest's lines are fetched
+sub guest_nics {
+ my ($vmid) = @_;
+
+ # a hook script starts with the cluster cache empty
+ PVE::Cluster::cfs_update();
+ my $vmlist = PVE::Cluster::get_vmlist();
+ my $nets = PVE::Cluster::get_guest_config_properties([map { "net$_" } 0 .. 31], $vmid);
+ my $nics = [];
+ for my $vmid (sort keys %$nets) {
+ my $guest = $vmlist->{ids}->{$vmid} // next;
+ my $prefix = ($guest->{type} // '') eq 'lxc' ? 'veth' : 'tap';
+ for my $key (sort keys %{ $nets->{$vmid} }) {
+ my ($index) = $key =~ m/^net(\d+)$/ or next;
+ my $net = $nets->{$vmid}->{$key};
+ my ($bridge) = $net =~ m/(?:^|,)bridge=([^,]+)/ or next;
+ # a VM names the MAC as the value of its model, a container as hwaddr
+ my ($mac) = $net =~ m/(?:^|,)[a-z0-9_-]+=([0-9a-f]{2}(?::[0-9a-f]{2}){5})(?:,|$)/i;
+ push @$nics,
+ {
+ vmid => $vmid,
+ node => $guest->{node},
+ iface => "$prefix${vmid}i$index",
+ bridge => $bridge,
+ mac => $mac,
+ };
+ }
+ }
+
+ return $nics;
+}
+
+sub tap_plug {
+ my ($bridge, $iface, $mac) = @_;
+
+ my $vnet = PVE::Network::SDN::Vnets::get_vnet($bridge, 1);
+ my $zone = $vnet ? PVE::Network::SDN::Zones::get_zone($vnet->{zone}, 1) : undef;
+ my $backend = $zone ? $zone->{dhcp} : undef;
+
+ # every other backend hears where the interface sits now. A plain bridge
+ # is no vnet
+ my $target = { vnet => $vnet ? $bridge : undef };
+ for my $type (PVE::Network::SDN::Dhcp::Plugin->lookup_types()->@*) {
+ next if defined($backend) && $type eq $backend;
+ eval { PVE::Network::SDN::Dhcp::Plugin->lookup($type)->tap_unplug($iface, $target) };
+ warn "could not unplug $iface from the $type dhcp backend: $@" if $@;
+ }
+ return if !$backend;
+
+ eval {
+ PVE::Network::SDN::Dhcp::Plugin->lookup($backend)
+ ->tap_plug($vnet->{zone}, $bridge, $iface, $mac);
+ };
+ warn "could not plug $iface into the dhcp backend of zone $vnet->{zone}: $@" if $@;
+}
+
+sub boot {
+ for my $type (PVE::Network::SDN::Dhcp::Plugin->lookup_types()->@*) {
+ eval { PVE::Network::SDN::Dhcp::Plugin->lookup($type)->boot() };
+ warn "could not bring up the $type dhcp backend: $@" if $@;
+ }
+}
+
+# the interface is usually gone already and with it any way to its zone, so
+# every backend is told
+sub tap_unplug {
+ my ($iface) = @_;
+
+ for my $type (PVE::Network::SDN::Dhcp::Plugin->lookup_types()->@*) {
+ eval { PVE::Network::SDN::Dhcp::Plugin->lookup($type)->tap_unplug($iface) };
+ warn "could not unplug $iface from the $type dhcp backend: $@" if $@;
+ }
+}
+
sub regenerate_config {
my ($reload) = @_;
diff --git a/src/PVE/Network/SDN/Dhcp/Ebpf.pm b/src/PVE/Network/SDN/Dhcp/Ebpf.pm
new file mode 100644
index 00000000..fc4af1e0
--- /dev/null
+++ b/src/PVE/Network/SDN/Dhcp/Ebpf.pm
@@ -0,0 +1,308 @@
+package PVE::Network::SDN::Dhcp::Ebpf;
+
+use strict;
+use warnings;
+
+use base qw(PVE::Network::SDN::Dhcp::Plugin);
+
+use Net::IP;
+use Net::Subnet qw(subnet_matcher);
+
+use PVE::Cluster;
+use PVE::INotify;
+use PVE::Network::SDN::Ipams;
+use PVE::RESTEnvironment qw(log_warn);
+
+use PVE::RS::SDN::Dhcp;
+use PVE::RS::SDN::Dhcp::ExclusiveLock;
+use PVE::RS::SDN::Dhcp::SharedLock;
+
+my $DEFAULT_LEASE_TIME = 600;
+
+# a subnet without a gateway is served under a link-local server identifier,
+# which no host on the subnet owns. A guest cannot renew by unicast then and
+# broadcasts at its rebinding time instead
+my $SERVER_ID_NO_GATEWAY = '169.254.0.1';
+
+sub type {
+ return 'ebpf';
+}
+
+# the MTU option is 16 bits wide with 68 the smallest MTU IPv4 allows, so a
+# zone MTU outside that is reported on an apply and not handed out. A zone
+# without one hands out none and the guest keeps its own setting
+my sub zone_mtu {
+ my ($zoneid, $zone, $report) = @_;
+
+ my $mtu = PVE::Network::SDN::Zones::Plugin->lookup($zone->{type})->get_mtu($zone);
+ return undef if !defined($mtu);
+ return $mtu if $mtu >= 68 && $mtu <= 65535;
+ log_warn("zone $zoneid has MTU $mtu, not handing it out over DHCP") if $report;
+ return undef;
+}
+
+my sub dhcp_record {
+ my ($ip4, $subnet, $mtu, $mac) = @_;
+
+ my $gateway = $subnet->{gateway};
+
+ # a DNS server has to be reachable over the address family served
+ my $dns = $subnet->{'dhcp-dns-server'};
+ $dns = undef if defined($dns) && !Net::IP::ip_is_ipv4($dns);
+
+ # the config hands its numbers over as strings, while the bindings take integers only
+ return {
+ ip => $ip4,
+ prefixlen => int($subnet->{mask}),
+ server_id => $gateway // $SERVER_ID_NO_GATEWAY,
+ lease => int($subnet->{'dhcp-lease-time'} // $DEFAULT_LEASE_TIME),
+ router => $gateway,
+ dns => $dns,
+ mac => lc($mac),
+ mtu => defined($mtu) ? int($mtu) : undef,
+ };
+}
+
+# the IPv4 subnets of a vnet, warning about a DNS server the reply cannot
+# carry when asked to. Only the apply asks, so a guest start does not repeat
+# the warning
+my sub served_subnets {
+ my ($vnetid, $report) = @_;
+
+ my $subnets = PVE::Network::SDN::Vnets::get_subnets($vnetid, 1) // {};
+ my $served = [];
+ for my $subnetid (sort keys %$subnets) {
+ my $subnet = $subnets->{$subnetid};
+ next if !Net::IP::ip_is_ipv4($subnet->{network});
+ log_warn("subnet $subnetid has an IPv6 DNS server, not handing it out over IPv4")
+ if $report
+ && defined($subnet->{'dhcp-dns-server'})
+ && !Net::IP::ip_is_ipv4($subnet->{'dhcp-dns-server'});
+ push @$served, $subnet;
+ }
+
+ return $served;
+}
+
+# entries from before the cache was keyed in lower case carry their writer's
+# spelling
+my sub cached_macs {
+ my ($macdb) = @_;
+
+ my $macs = {};
+ for my $mac (sort keys $macdb->{macs}->%*) {
+ my $entry = $macdb->{macs}->{$mac};
+ $macs->{ lc($mac) }->{$_} = $entry->{$_} for grep { defined($entry->{$_}) } qw(ip4 ip6);
+ }
+ return $macs;
+}
+
+my $matchers = {};
+
+# the vnet's subnet holding the cached address decides the answer
+my sub vnet_record {
+ my ($subnets, $mtu, $macs, $mac) = @_;
+
+ my $ip4 = $macs->{ lc($mac) }->{ip4};
+ return undef if !$ip4;
+ # the most specific of overlapping subnets is the one to answer from
+ for my $subnet (sort { $b->{mask} <=> $a->{mask} } @$subnets) {
+ my $matcher = $matchers->{ $subnet->{cidr} } //= subnet_matcher($subnet->{cidr});
+ next if !$matcher->($ip4);
+ # the vnet's own gateway address is cached too and never a lease
+ return undef if defined($subnet->{gateway}) && $ip4 eq $subnet->{gateway};
+ return dhcp_record($ip4, $subnet, $mtu, $mac);
+ }
+
+ return undef;
+}
+
+my sub full_state {
+ my ($report) = @_;
+
+ # the configs and the cache are read after the cluster state is refreshed,
+ # since a copy from before the lock was taken would hand the pass older input
+ PVE::Cluster::cfs_update();
+ my $cfg = PVE::Network::SDN::running_config();
+ my $zones = $cfg->{zones}->{ids} // {};
+ my $vnets = $cfg->{vnets}->{ids} // {};
+ # an ebpf zone confined to other nodes runs no responder here
+ my $nodename = PVE::INotify::nodename();
+ my %ebpf = map { $_ => 1 } grep {
+ ($zones->{$_}->{dhcp} // '') eq 'ebpf'
+ && (!defined($zones->{$_}->{nodes}) || $zones->{$_}->{nodes}->{$nodename})
+ } keys %$zones;
+ return (0, []) if !%ebpf;
+
+ my $macs = cached_macs(PVE::Network::SDN::Ipams::read_macdb());
+ my (%mtu, %subnets);
+ # a subnet or a zone MTU the responder cannot serve is reported per apply,
+ # guests or not
+ if ($report) {
+ for my $vnetid (sort keys %$vnets) {
+ my $zoneid = $vnets->{$vnetid}->{zone};
+ next if !$zoneid || !$ebpf{$zoneid};
+ $subnets{$vnetid} //= served_subnets($vnetid, 1);
+ $mtu{$zoneid} = zone_mtu($zoneid, $zones->{$zoneid}, 1) if !exists $mtu{$zoneid};
+ }
+ }
+ my $ifaces = [];
+ for my $nic (PVE::Network::SDN::Dhcp::guest_nics()->@*) {
+ my $vnetid = $nic->{bridge};
+ my $zoneid = $vnets->{$vnetid} ? $vnets->{$vnetid}->{zone} : undef;
+ my $serve = $zoneid && $ebpf{$zoneid} ? 1 : 0;
+ my $record;
+ if ($serve) {
+ $mtu{$zoneid} = zone_mtu($zoneid, $zones->{$zoneid}, $report) if !exists $mtu{$zoneid};
+ $subnets{$vnetid} //= served_subnets($vnetid, $report);
+ $record = vnet_record($subnets{$vnetid}, $mtu{$zoneid}, $macs, $nic->{mac})
+ if $nic->{mac};
+ }
+ push @$ifaces,
+ {
+ name => $nic->{iface},
+ vnet => $vnets->{$vnetid} ? $vnetid : undef,
+ serve => $serve,
+ record => $record,
+ };
+ }
+
+ return (scalar(keys %ebpf), $ifaces);
+}
+
+my sub full_pass {
+ my ($report) = @_;
+
+ my $lock = eval { PVE::RS::SDN::Dhcp::ExclusiveLock->new() };
+ if (!$lock) {
+ log_warn("could not lock the DHCP responder: $@");
+ return;
+ }
+
+ my ($zones, $ifaces) = eval { full_state($report) };
+ if ($@) {
+ log_warn("could not collect the DHCP responder state: $@");
+ return;
+ }
+
+ eval { $zones ? $lock->apply($ifaces) : $lock->clear() };
+ log_warn("could not apply the DHCP responder state: $@") if $@;
+}
+
+my sub mac_ifaces {
+ my ($mac, $macs, $vmid) = @_;
+
+ my $cfg = PVE::Network::SDN::running_config();
+ my $zones = $cfg->{zones}->{ids} // {};
+ my $vnets = $cfg->{vnets}->{ids} // {};
+ my $ifaces = [];
+ for my $nic (PVE::Network::SDN::Dhcp::guest_nics($vmid)->@*) {
+ next if lc($nic->{mac} // '') ne lc($mac);
+ my $vnet = $vnets->{ $nic->{bridge} } // next;
+ my $zone = $zones->{ $vnet->{zone} } // next;
+ next if ($zone->{dhcp} // '') ne 'ebpf';
+ my $record = vnet_record(
+ served_subnets($nic->{bridge}, 0),
+ zone_mtu($vnet->{zone}, $zone, 0),
+ $macs,
+ $mac,
+ );
+ push @$ifaces,
+ { name => $nic->{iface}, vnet => $nic->{bridge}, serve => 1, record => $record };
+ }
+
+ return $ifaces;
+}
+
+# under the guest lock nothing else writes the guest's entries, so a caller
+# that names the guest takes the shared lock. An API edit names no guest and
+# takes the exclusive one
+sub update_ip_mapping {
+ my ($class, $dhcpid, $mac, $vmid) = @_;
+
+ my $update = sub {
+ my $lock =
+ defined($vmid)
+ ? PVE::RS::SDN::Dhcp::SharedLock->new()
+ : PVE::RS::SDN::Dhcp::ExclusiveLock->new();
+ PVE::Cluster::cfs_update();
+ my $ifaces =
+ mac_ifaces($mac, cached_macs(PVE::Network::SDN::Ipams::read_macdb()), $vmid);
+ return 1 if !@$ifaces;
+ return $lock->update($ifaces);
+ };
+
+ my $done = eval { $update->() };
+ if ($@) {
+ log_warn("could not update the DHCP responder records of $mac: $@");
+ return;
+ }
+ return if $done;
+
+ # nothing loaded, the pass takes its own lock once the update's is gone
+ full_pass(0);
+}
+
+# the dispatcher's per-zone walk is not needed, since the full pass collects
+# the state itself
+sub before_regenerate { }
+sub before_configure { }
+sub configure_subnet { }
+sub configure_range { }
+sub configure_vnet { }
+sub after_configure { }
+
+sub after_regenerate {
+ my ($class) = @_;
+
+ full_pass(1);
+}
+
+sub boot {
+ my ($class) = @_;
+
+ full_pass(1);
+}
+
+sub tap_unplug {
+ my ($class, $iface, $target) = @_;
+
+ eval { PVE::RS::SDN::Dhcp::detach($iface, $target) };
+ log_warn("could not detach the DHCP responder from $iface: $@") if $@;
+}
+
+sub tap_plug {
+ my ($class, $dhcpid, $vnetid, $iface, $mac) = @_;
+
+ my $attach = sub {
+ my $lock = PVE::RS::SDN::Dhcp::SharedLock->new();
+ my $record;
+ if ($mac) {
+ PVE::Cluster::cfs_update();
+ my $zone = PVE::Network::SDN::Zones::get_zone($dhcpid, 1);
+ $record = vnet_record(
+ served_subnets($vnetid, 0),
+ zone_mtu($dhcpid, $zone, 0),
+ cached_macs(PVE::Network::SDN::Ipams::read_macdb()),
+ $mac,
+ );
+ }
+ return $lock->attach($iface, $vnetid, $record);
+ };
+
+ my $done = eval { $attach->() };
+ if ($@) {
+ log_warn("could not attach DHCP responder to $iface: $@");
+ return;
+ }
+ return if $done;
+
+ # nothing is loaded, after a rebuild or a boot where the SDN commit loaded
+ # nothing. The plug's lock is gone by here, so the pass can take its own
+ full_pass(0);
+ $done = eval { $attach->() };
+ log_warn("could not attach DHCP responder to $iface: " . ($@ || "nothing is loaded"))
+ if $@ || !$done;
+}
+
+1;
diff --git a/src/PVE/Network/SDN/Dhcp/Makefile b/src/PVE/Network/SDN/Dhcp/Makefile
index 65465135..ce86aaef 100644
--- a/src/PVE/Network/SDN/Dhcp/Makefile
+++ b/src/PVE/Network/SDN/Dhcp/Makefile
@@ -1,4 +1,4 @@
-SOURCES=Plugin.pm Dnsmasq.pm
+SOURCES=Plugin.pm Dnsmasq.pm Ebpf.pm
PERL5DIR=${DESTDIR}/usr/share/perl5
diff --git a/src/PVE/Network/SDN/Dhcp/Plugin.pm b/src/PVE/Network/SDN/Dhcp/Plugin.pm
index 2fa8683c..149873e6 100644
--- a/src/PVE/Network/SDN/Dhcp/Plugin.pm
+++ b/src/PVE/Network/SDN/Dhcp/Plugin.pm
@@ -60,9 +60,28 @@ sub before_regenerate {
die 'implement in sub class';
}
+# the node booted. A backend whose state does not survive a reboot brings it
+# back here, before the guests start
+sub boot {
+ my ($class) = @_;
+}
+
sub after_regenerate {
my ($class) = @_;
die 'implement in sub class';
}
+# a guest interface was plugged into a vnet of a zone using this backend.
+# Backends serving the bridge rather than the interface have nothing to do
+sub tap_plug {
+ my ($class, $dhcpid, $vnetid, $iface, $mac) = @_;
+}
+
+# a guest interface left its vnet or went away, and by then nothing says which
+# zone it was on. The target is the vnet it sits on now, none for a plain
+# bridge, and no target means it is gone
+sub tap_unplug {
+ my ($class, $iface, $target) = @_;
+}
+
1;
diff --git a/src/PVE/Network/SDN/SubnetPlugin.pm b/src/PVE/Network/SDN/SubnetPlugin.pm
index 29edf7b6..38579f21 100644
--- a/src/PVE/Network/SDN/SubnetPlugin.pm
+++ b/src/PVE/Network/SDN/SubnetPlugin.pm
@@ -183,8 +183,8 @@ sub properties {
maximum => 4294967295,
description =>
'Lease time in seconds for DHCP answers. Without it dnsmasq hands out'
- . ' infinite leases, and it raises anything below two minutes to two'
- . ' minutes.',
+ . ' infinite leases and the ebpf responder ten minutes. dnsmasq raises'
+ . ' anything below two minutes to two minutes.',
optional => 1,
},
};
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index cda4b7ef..db6fc472 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -47,6 +47,9 @@ sub clear_test_state {
macdb => {},
ipamdb => {},
dnsmasq_calls => [],
+ ebpf_calls => [],
+ order => [],
+ warnings => [],
ipam_config => {
'ids' => {
'pve' => {
@@ -54,6 +57,7 @@ sub clear_test_state {
},
},
},
+ ebpf_answer => undef,
};
PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "\n");
}
@@ -234,6 +238,43 @@ $mocked_sdn_dhcp_dnsmasq->mock(
update_lease => sub { },
);
+my $mocked_pve_rs_dhcp = Test::MockModule->new('PVE::RS::SDN::Dhcp');
+$mocked_pve_rs_dhcp->mock(
+ detach => sub {
+ push $test_state->{ebpf_calls}->@*, { method => 'detach', args => [@_] };
+ return $test_state->{ebpf_answer}->{detach} // 1;
+ },
+);
+
+# taking a lock is the ordered step, and the responder calls are its methods
+my %mocked_pve_rs_dhcp_locks;
+for my $class (qw(SharedLock ExclusiveLock)) {
+ my $package = "PVE::RS::SDN::Dhcp::$class";
+ my $mock = Test::MockModule->new($package, no_auto => 1);
+ $mock->mock(
+ new => sub {
+ push $test_state->{order}->@*, 'lock';
+ push $test_state->{lock_modes}->@*, $class eq 'SharedLock' ? 'shared' : 'exclusive';
+ return bless {}, $package;
+ },
+ map {
+ my $method = $_;
+ $method => sub {
+ my ($self, @args) = @_;
+ push $test_state->{order}->@*, 'write';
+ push $test_state->{ebpf_calls}->@*, { method => $method, args => [@args] };
+ return $test_state->{ebpf_answer}->{$method} // 1;
+ };
+ } ($class eq 'SharedLock' ? qw(attach update) : qw(apply update clear)),
+ );
+ $mocked_pve_rs_dhcp_locks{$class} = $mock;
+}
+
+my $mocked_sdn_dhcp_ebpf = Test::MockModule->new('PVE::Network::SDN::Dhcp::Ebpf');
+$mocked_sdn_dhcp_ebpf->mock(
+ log_warn => sub { push $test_state->{warnings}->@*, $_[0]; },
+);
+
my $mocked_api_zones = Test::MockModule->new('PVE::API2::Network::SDN::Zones');
$mocked_api_zones->mock(
create_etc_interfaces_sdn_dir => sub { },
@@ -258,6 +299,15 @@ my $mocked_pve_cluster_obj = Test::MockModule->new('PVE::Cluster');
$mocked_pve_cluster_obj->mock(
check_cfs_quorum => sub { return 1; },
cfs_lock_domain => $mocked_cfs_lock_domain,
+ cfs_update => sub { push $test_state->{order}->@*, 'refresh'; },
+ get_vmlist => sub { return $test_state->{vmlist} // { ids => {} }; },
+ get_guest_config_properties => sub {
+ my ($properties, $vmid) = @_;
+ my $nets = $test_state->{guest_nets} // {};
+ push $test_state->{nic_listings}->@*, $vmid;
+ return $nets if !defined($vmid);
+ return exists $nets->{$vmid} ? { $vmid => $nets->{$vmid} } : {};
+ },
);
# ------- TEST FUNCTIONS --------------
@@ -365,6 +415,24 @@ sub delete_ip {
return PVE::API2::Network::SDN::Ips->ipdelete($param);
}
+sub take_ebpf_calls {
+ my $calls = $test_state->{ebpf_calls};
+ $test_state->{ebpf_calls} = [];
+ return $calls;
+}
+
+# the lock takes, cluster refreshes and responder writes in the order they
+# ran, a repeated step collapsed into one
+sub take_order {
+ my $order = $test_state->{order};
+ $test_state->{order} = [];
+ my @shape;
+ for my $step (@$order) {
+ push @shape, $step if !@shape || $shape[-1] ne $step;
+ }
+ return \@shape;
+}
+
sub run_test {
my $test = shift;
clear_test_state();
@@ -1437,4 +1505,578 @@ sub test_dnsmasq_dual_stack_and_sweep {
run_test(\&test_dnsmasq_dual_stack_and_sweep);
+# -------------- ebpf dhcp backend
+my $ebpf_record = sub {
+ my ($ip, %over) = @_;
+ return {
+ ip => $ip,
+ prefixlen => 24,
+ server_id => '10.0.0.1',
+ lease => 600,
+ router => '10.0.0.1',
+ dns => undef,
+ mtu => undef,
+ mac => 'da:65:8f:18:9b:6f',
+ %over,
+ };
+};
+
+sub test_ebpf_backend {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+
+ # the guest behind the MAC runs here, so its interface gets the record
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ take_ebpf_calls();
+ $test_state->{nic_listings} = [];
+ $test_state->{lock_modes} = [];
+ eval { nic_start($vnetid, $mac, "999", "testhostname"); };
+ if ($@) {
+ fail("$test_name: nic_start: $@");
+ return;
+ }
+ is(
+ $test_state->{nic_listings}->[0],
+ 999,
+ "$test_name: the start reads its own guest's config only",
+ );
+ eq_or_diff(
+ $test_state->{lock_modes},
+ ['shared'],
+ "$test_name: and writes its record under the shared lock",
+ );
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.100"),
+ }]],
+ }],
+ "$test_name: a guest start writes the record of its interface",
+ );
+
+ # a mapping edit rewrites it, and a delete leaves the interface without one
+ $test_state->{lock_modes} = [];
+ update_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ eq_or_diff(
+ $test_state->{lock_modes},
+ ['exclusive'],
+ "$test_name: an edit names no guest and writes under the exclusive lock",
+ );
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.150"),
+ }]],
+ }],
+ "$test_name: a mapping edit rewrites the record",
+ );
+ delete_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{ name => 'tap999i0', vnet => $vnetid, serve => 1, record => undef }]],
+ }],
+ "$test_name: a mapping delete drops the record",
+ );
+
+ # a guest running elsewhere has no interface here to write to
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'other' } } };
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.150"),
+ }]],
+ }],
+ "$test_name: a guest elsewhere is handed over too, the responder skips what is not here",
+ );
+
+ # the full pass names every guest NIC in the cluster, whether the
+ # responder runs on it and what it answers. The responder keeps those
+ # present here. A guest the vmlist does not know is skipped, and a NIC
+ # without a MAC is served nothing
+ $test_state->{vmlist} = {
+ ids => {
+ 100 => { type => 'qemu', node => 'other' },
+ 101 => { type => 'lxc', node => 'other' },
+ 103 => { type => 'qemu', node => 'other' },
+ },
+ };
+ $test_state->{guest_nets} = {
+ 100 => {
+ net0 => "virtio=$mac,bridge=$vnetid,firewall=1",
+ net1 => "virtio=00:11:22:33:44:55,bridge=vmbr0",
+ },
+ 101 => { net2 => "name=eth0,bridge=$vnetid,hwaddr=00:11:22:33:44:66" },
+ 102 => { net0 => "virtio=00:11:22:33:44:77,bridge=$vnetid" },
+ 103 => { net0 => "virtio,bridge=$vnetid" },
+ };
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'apply',
+ args => [[
+ {
+ name => 'tap100i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.150"),
+ },
+ { name => 'tap100i1', vnet => undef, serve => 0, record => undef },
+ { name => 'veth101i2', vnet => $vnetid, serve => 1, record => undef },
+ { name => 'tap103i0', vnet => $vnetid, serve => 1, record => undef },
+ ]],
+ }],
+ "$test_name: the full pass names every guest NIC with its answer",
+ );
+
+ # a responder with nothing loaded cannot take a single record, so the
+ # full pass loads it and fills everything
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ $test_state->{ebpf_answer} = { update => 0 };
+ $test_state->{lock_modes} = [];
+ nic_start($vnetid, $mac, "999", "testhostname");
+ $test_state->{ebpf_answer} = undef;
+ eq_or_diff(
+ [map { $_->{method} } take_ebpf_calls()->@*],
+ ['update', 'apply'],
+ "$test_name: an unloaded responder gets the full pass",
+ );
+ eq_or_diff(
+ $test_state->{lock_modes},
+ ['shared', 'exclusive'],
+ "$test_name: which takes its own lock after the record change let go",
+ );
+
+ # a plug attaches the interface with the answer its MAC gets on the
+ # vnet. With nothing loaded the full pass runs first and the attach is
+ # retried
+ PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0', $mac);
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'attach', args => ['tap999i0', $vnetid, $ebpf_record->("10.0.0.150")] }],
+ "$test_name: a plug attaches its interface with the record",
+ );
+ $test_state->{ebpf_answer} = { attach => 0 };
+ PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0', $mac);
+ $test_state->{ebpf_answer} = undef;
+ eq_or_diff(
+ [map { $_->{method} } take_ebpf_calls()->@*],
+ ['attach', 'apply', 'attach'],
+ "$test_name: a plug with nothing loaded runs the full pass first",
+ );
+ PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0');
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'attach', args => ['tap999i0', $vnetid, undef] }],
+ "$test_name: a plug without a MAC attaches with nothing to answer",
+ );
+}
+
+sub test_ebpf_backend_edge_cases {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a regenerate without any ebpf zone tears the responder state down
+ create_zone({
+ type => "simple",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ take_ebpf_calls();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'clear', args => [] }],
+ "$test_name: regenerate without an ebpf zone clears the responder",
+ );
+
+ # an ebpf zone confined to other nodes runs no responder here either
+ update_zone($zoneid, { dhcp => "ebpf", nodes => 'other' });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'clear', args => [] }],
+ "$test_name: an ebpf zone confined to other nodes clears the responder here",
+ );
+ update_zone($zoneid, { delete => 'nodes' });
+
+ # the SDN commit brings the responder back at boot, one full pass
+ PVE::Network::SDN::Dhcp::boot();
+ eq_or_diff(
+ [map { $_->{method} } take_ebpf_calls()->@*],
+ ['apply'],
+ "$test_name: the boot runs the full pass",
+ );
+
+ # the zone switches to ebpf, but its subnets lack what a record needs
+ update_zone($zoneid, { dhcp => "ebpf" });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "fd00::/64",
+ gateway => "fd00::1",
+ 'dhcp-range' => ["start-address=fd00::100,end-address=fd00::200"],
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+
+ take_ebpf_calls();
+ eval { nic_start($vnetid, $mac, "999", "testhostname"); };
+ if ($@) {
+ fail("$test_name: nic_start: $@");
+ return;
+ }
+
+ # a gateway-less v4 subnet is served under a link-local identifier and
+ # without a router. The v6 subnet yields nothing
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->(
+ "10.0.0.100",
+ server_id => '169.254.0.1',
+ router => undef,
+ ),
+ }]],
+ }],
+ "$test_name: a subnet without a gateway is served without a router",
+ );
+ $test_state->{warnings} = [];
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ take_ebpf_calls();
+ is(scalar($test_state->{warnings}->@*), 0, "$test_name: and the full pass has nothing to say");
+
+ # a v6 resolver on a v4 subnet is dropped from the record instead of
+ # failing the whole set
+ update_subnet({
+ vnet => $vnetid,
+ subnet => "$zoneid-10.0.0.0-24",
+ gateway => "10.0.0.1",
+ 'dhcp-dns-server' => "fd00::53",
+ });
+ $test_state->{warnings} = [];
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ my $calls = take_ebpf_calls();
+ is(scalar(@$calls), 1, "$test_name: regenerate applies once");
+ my $iface = $calls->[0]->{args}->[0]->[0];
+ is($iface->{name}, 'tap999i0', "$test_name: the guest interface is listed");
+ is($iface->{record}->{ip}, '10.0.0.100', "$test_name: the v4 subnet now yields the record");
+ is($iface->{record}->{dns}, undef, "$test_name: the IPv6 resolver is not handed out");
+ is(
+ scalar(grep { m/IPv6 DNS server/ } $test_state->{warnings}->@*),
+ 1,
+ "$test_name: and the full pass says so",
+ );
+}
+
+run_test(\&test_ebpf_backend);
+run_test(\&test_ebpf_backend_edge_cases);
+
+sub test_ebpf_mac_case {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a mapping typed in lower case for a NIC whose config spells the MAC in
+ # upper case is the same mapping
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=" . uc($mac) . ",bridge=$vnetid" } };
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.100",
+ });
+ take_ebpf_calls();
+ take_order();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'apply',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.100"),
+ }]],
+ }],
+ "$test_name: the full pass finds the mapping however the config spells the MAC",
+ );
+ # the lock is taken before the refresh and the reads
+ is_deeply(
+ take_order(),
+ ['lock', 'refresh', 'write'],
+ "$test_name: the pass locks, refreshes, then writes",
+ );
+ PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0', uc($mac));
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'attach', args => ['tap999i0', $vnetid, $ebpf_record->("10.0.0.100")] }],
+ "$test_name: and so does a plug",
+ );
+
+ # an entry split across spellings is read as one
+ $test_state->{macdb} = {
+ macs => { uc($mac) => { ip4 => '10.0.0.100' }, $mac => { ip6 => '8888::100' } },
+ };
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{ip},
+ '10.0.0.100',
+ "$test_name: a cache entry split across spellings is read as one",
+ );
+}
+
+run_test(\&test_ebpf_mac_case);
+
+sub test_ebpf_equal_subnets {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+
+ # two ebpf zones carry the same subnet. Each interface is answered from
+ # the subnet of the vnet it sits on, whatever address the cache holds
+ # for its MAC
+ for my $net (['ZONEA', 'vneta', '10.0.0.1'], ['ZONEB', 'vnetb', '10.0.0.2']) {
+ my ($zoneid, $vnetid, $gateway) = @$net;
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => $gateway,
+ });
+ }
+ $test_state->{vmlist} = {
+ ids => {
+ 100 => { type => 'qemu', node => 'localnode' },
+ 101 => { type => 'qemu', node => 'localnode' },
+ },
+ };
+ $test_state->{guest_nets} = {
+ 100 => { net0 => "virtio=da:65:8f:18:9b:6f,bridge=vneta" },
+ 101 => { net0 => "virtio=da:65:8f:18:9b:70,bridge=vnetb" },
+ };
+ $test_state->{macdb} = {
+ macs => {
+ 'da:65:8f:18:9b:6f' => { ip4 => '10.0.0.150' },
+ 'da:65:8f:18:9b:70' => { ip4 => '10.0.0.150' },
+ },
+ };
+
+ take_ebpf_calls();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ my $ifaces = take_ebpf_calls()->[0]->{args}->[0];
+ eq_or_diff(
+ [map { [$_->{name}, $_->{record}->{router}] } @$ifaces],
+ [['tap100i0', '10.0.0.1'], ['tap101i0', '10.0.0.2']],
+ "$test_name: each interface is answered from its own vnet",
+ );
+}
+
+run_test(\&test_ebpf_equal_subnets);
+
+sub test_ebpf_zone_mtu {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a zone MTU the DHCP option cannot carry is reported on the apply and
+ # left out of the answer
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ mtu => 10,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ $test_state->{macdb} = { macs => { $mac => { ip4 => '10.0.0.100' } } };
+
+ take_ebpf_calls();
+ $test_state->{warnings} = [];
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ scalar(grep { m/MTU 10/ } $test_state->{warnings}->@*),
+ 1,
+ "$test_name: the zone MTU is reported once",
+ );
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{mtu},
+ undef,
+ "$test_name: and not handed out",
+ );
+
+ update_zone($zoneid, { mtu => 9000 });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{mtu},
+ 9000,
+ "$test_name: a zone MTU the option carries is handed out",
+ );
+}
+
+run_test(\&test_ebpf_zone_mtu);
+
+sub test_ebpf_overlapping_subnets {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a vnet with overlapping subnets answers from the most specific one
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/16",
+ gateway => "10.0.255.1",
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ $test_state->{macdb} = { macs => { $mac => { ip4 => '10.0.0.100' } } };
+
+ take_ebpf_calls();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ my $record = take_ebpf_calls()->[0]->{args}->[0]->[0]->{record};
+ is_deeply(
+ [$record->@{qw(prefixlen router)}],
+ [24, '10.0.0.1'],
+ "$test_name: the most specific subnet answers",
+ );
+}
+
+run_test(\&test_ebpf_overlapping_subnets);
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH pve-network v3 10/17] sdn: zones: report interface plugs and unplugs to the DHCP backends
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (8 preceding siblings ...)
2026-10-20 7:59 ` [PATCH pve-network v3 09/17] sdn: dhcp: add an eBPF backend answering on each guest interface Hannes Laimer
@ 2026-10-20 7:59 ` Hannes Laimer
2026-10-20 13:17 ` Stefan Hanreich
2026-10-20 7:59 ` [PATCH pve-network v3 11/17] sdn: dhcp: push IP mapping edits to the guest's node Hannes Laimer
` (8 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 7:59 UTC (permalink / raw)
To: pve-devel
The ebpf DHCP backend answers on each guest interface rather than on
the bridge, so the zone layer now reports every plug to the DHCP
backends. The plug takes the guest's MAC along, since the address to
hand out is recorded per MAC. A plug onto a plain bridge is reported
too, since a running guest's interface can move there from a vnet.
A failing backend only warns and the plug goes ahead.
The zone layer also gains an unplug and a veth removal, which report
the interface to the backends as gone.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/Network/SDN/Zones.pm | 29 ++++++-
src/test/run_test_vnets_blackbox.pl | 129 ++++++++++++++++++++++++++++
2 files changed, 154 insertions(+), 4 deletions(-)
diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm
index 4229feda..10fc0731 100644
--- a/src/PVE/Network/SDN/Zones.pm
+++ b/src/PVE/Network/SDN/Zones.pm
@@ -326,16 +326,18 @@ sub veth_create {
}
sub tap_plug {
- my ($iface, $bridge, $tag, $firewall, $trunks, $rate) = @_;
+ my ($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts) = @_;
my $vnet = PVE::Network::SDN::Vnets::get_vnet($bridge, 1);
if (!$vnet) { # fallback for classic bridge
my $interfaces_config = PVE::INotify::read_file('interfaces');
- my $opts = {};
- $opts->{learning} = 0
+ my $bridge_opts = {};
+ $bridge_opts->{learning} = 0
if $interfaces_config->{ifaces}->{$bridge}
&& $interfaces_config->{ifaces}->{$bridge}->{'bridge-disable-mac-learning'};
- PVE::Network::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts);
+ # the dhcp backends hear of an interface moving onto a plain bridge as well
+ PVE::Network::SDN::Dhcp::tap_plug($bridge, $iface, $opts->{mac});
+ PVE::Network::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate, $bridge_opts);
return;
}
@@ -346,9 +348,28 @@ sub tap_plug {
if $plugin_config->{nodes} && !defined($plugin_config->{nodes}->{$nodename});
my $plugin = PVE::Network::SDN::Zones::Plugin->lookup($plugin_config->{type});
+
+ # the dhcp backends act before the interface joins the bridge, so no request slips past
+ # them. Dhcp.pm comes in through Vnets.pm, since a use of it here would create a load-order
+ # cycle
+ PVE::Network::SDN::Dhcp::tap_plug($bridge, $iface, $opts->{mac});
$plugin->tap_plug($plugin_config, $vnet, $tag, $iface, $bridge, $firewall, $trunks, $rate);
}
+sub tap_unplug {
+ my ($iface) = @_;
+
+ PVE::Network::SDN::Dhcp::tap_unplug($iface);
+ PVE::Network::tap_unplug($iface);
+}
+
+sub veth_delete {
+ my ($veth) = @_;
+
+ PVE::Network::SDN::Dhcp::tap_unplug($veth);
+ PVE::Network::veth_delete($veth);
+}
+
sub add_bridge_fdb {
my ($iface, $macaddr, $bridge) = @_;
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index db6fc472..783d0d47 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -1844,6 +1844,135 @@ sub test_ebpf_backend_edge_cases {
}
run_test(\&test_ebpf_backend);
+
+sub test_zone_tap_plug {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # the zone plug hands the MAC on to the dhcp backends, and a plug onto a
+ # plain bridge tells them where the interface went
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.100",
+ });
+ my $bridged = [];
+ my $unplugged = [];
+ $mocked_sdn_zones_super_plugin->mock(
+ tap_plug => sub {
+ push @$bridged, $_[5];
+ push $test_state->{order}->@*, 'bridge';
+ },
+ );
+ my $mocked_pve_network = Test::MockModule->new('PVE::Network');
+ $mocked_pve_network->mock(
+ tap_plug => sub {
+ push @$bridged, $_[1];
+ push $test_state->{order}->@*, 'bridge';
+ },
+ tap_unplug => sub { push @$unplugged, $_[0]; },
+ veth_delete => sub { push @$unplugged, $_[0]; },
+ );
+ $mocked_pve_inotify->mock(read_file => sub { return { ifaces => {} }; });
+ take_ebpf_calls();
+ take_order();
+
+ PVE::Network::SDN::Zones::tap_plug('tap999i0', $vnetid, undef, 0, undef, undef,
+ { mac => $mac });
+ eq_or_diff($bridged, [$vnetid], "$test_name: the zone plugin plugs the interface");
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'attach', args => ['tap999i0', $vnetid, $ebpf_record->("10.0.0.100")] }],
+ "$test_name: and the dhcp backend attaches it with the MAC's record",
+ );
+ is_deeply(
+ take_order(),
+ ['lock', 'refresh', 'write', 'bridge'],
+ "$test_name: the responder attaches before the interface joins the bridge",
+ );
+
+ @$bridged = ();
+ PVE::Network::SDN::Zones::tap_plug('tap999i0', 'vmbr0', undef, 0, undef, undef,
+ { mac => $mac });
+ eq_or_diff($bridged, ['vmbr0'], "$test_name: a plain bridge is plugged the classic way");
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'detach', args => ['tap999i0', { vnet => undef }] }],
+ "$test_name: and the dhcp backends hear where the interface went",
+ );
+
+ # a vnet of another backend is a place the ebpf backend hears of as well
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => "DNSZONE",
+ });
+ create_vnet({
+ type => "vnet",
+ zone => "DNSZONE",
+ vnet => "dnsvnet",
+ });
+ @$bridged = ();
+ PVE::Network::SDN::Zones::tap_plug(
+ 'tap999i0',
+ 'dnsvnet',
+ undef,
+ 0,
+ undef,
+ undef,
+ { mac => $mac },
+ );
+ eq_or_diff(
+ $bridged,
+ ['dnsvnet'],
+ "$test_name: a vnet of another backend is plugged by its zone",
+ );
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'detach', args => ['tap999i0', { vnet => 'dnsvnet' }] }],
+ "$test_name: and the ebpf backend hears which vnet the interface went to",
+ );
+
+ # the zone unplug and the veth delete report the interface gone before
+ # the classic ones run
+ PVE::Network::SDN::Zones::tap_unplug('tap999i0');
+ PVE::Network::SDN::Zones::veth_delete('veth999i0');
+ eq_or_diff($unplugged, ['tap999i0', 'veth999i0'], "$test_name: the classic unplugs run");
+ eq_or_diff(
+ take_ebpf_calls(),
+ [
+ { method => 'detach', args => ['tap999i0', undef] },
+ { method => 'detach', args => ['veth999i0', undef] },
+ ],
+ "$test_name: and the dhcp backends hear the interfaces are gone",
+ );
+ $mocked_sdn_zones_super_plugin->unmock('tap_plug');
+ $mocked_pve_inotify->unmock('read_file');
+}
+
+run_test(\&test_zone_tap_plug);
run_test(\&test_ebpf_backend_edge_cases);
sub test_ebpf_mac_case {
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH pve-network v3 11/17] sdn: dhcp: push IP mapping edits to the guest's node
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (9 preceding siblings ...)
2026-10-20 7:59 ` [PATCH pve-network v3 10/17] sdn: zones: report interface plugs and unplugs to the DHCP backends Hannes Laimer
@ 2026-10-20 7:59 ` Hannes Laimer
2026-10-20 13:09 ` Stefan Hanreich
2026-10-20 8:00 ` [PATCH pve-network v3 12/17] sdn: zones: allow the ebpf DHCP backend on every zone type Hannes Laimer
` (7 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 7:59 UTC (permalink / raw)
To: pve-devel
IP mappings live in the cluster-wide IPAM and can be edited through
the API on any node. The DHCP backends hold their copy on each node,
dnsmasq in a reservation file and the ebpf responder in a kernel map.
An edit only updated the backend of the node it was made on, so a
guest on another node kept getting the old mapping.
The new endpoint POST /nodes/{node}/sdn/dhcp-mapping updates the
mapping of one MAC on that node from the current records. An edit
made on a node other than the guest's calls it on the guest's node.
Only that node answers the guest's DHCP requests, so no other node is
called. The others catch up when the guest starts on one of them.
The call runs in the background, so the request returns without
waiting on the other node. A call that fails logs its error to the
journal of the node the edit was made on. One that gets no answer is
stopped after a minute. The record stays as written either way, and
the guest's node picks it up at the latest when the guest next starts
there.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/Network/SDN/Ips.pm | 3 +
src/PVE/API2/Network/SDN/Nodes/Status.pm | 37 ++++++++++
src/PVE/Network/SDN/Dhcp.pm | 74 +++++++++++++++++++
src/test/run_test_vnets_blackbox.pl | 90 ++++++++++++++++++++++++
4 files changed, 204 insertions(+)
diff --git a/src/PVE/API2/Network/SDN/Ips.pm b/src/PVE/API2/Network/SDN/Ips.pm
index d7b682dd..5b45de7c 100644
--- a/src/PVE/API2/Network/SDN/Ips.pm
+++ b/src/PVE/API2/Network/SDN/Ips.pm
@@ -47,6 +47,7 @@ __PACKAGE__->register_method({
die "$@\n" if $@;
PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+ PVE::Network::SDN::Dhcp::notify_guest_node($vnet, $mac);
return undef;
},
@@ -85,6 +86,7 @@ __PACKAGE__->register_method({
PVE::Network::SDN::Vnets::add_ip($vnet, $ip, '', $mac, undef);
PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+ PVE::Network::SDN::Dhcp::notify_guest_node($vnet, $mac);
return undef;
},
@@ -138,6 +140,7 @@ __PACKAGE__->register_method({
die "$error\n" if $error;
PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+ PVE::Network::SDN::Dhcp::notify_guest_node($vnet, $mac);
return undef;
},
diff --git a/src/PVE/API2/Network/SDN/Nodes/Status.pm b/src/PVE/API2/Network/SDN/Nodes/Status.pm
index 7977e0c8..fe750a85 100644
--- a/src/PVE/API2/Network/SDN/Nodes/Status.pm
+++ b/src/PVE/API2/Network/SDN/Nodes/Status.pm
@@ -9,6 +9,9 @@ use PVE::API2::Network::SDN::Nodes::Vnets;
use PVE::JSONSchema qw(get_standard_option);
+use PVE::Network::SDN::Vnets;
+use PVE::Network::SDN::Dhcp;
+
use PVE::RESTHandler;
use base qw(PVE::RESTHandler);
@@ -27,6 +30,40 @@ __PACKAGE__->register_method({
path => 'vnets',
});
+__PACKAGE__->register_method({
+ name => 'dhcp_mapping',
+ path => 'dhcp-mapping',
+ method => 'POST',
+ description =>
+ 'Re-apply the DHCP mapping of a MAC address on this node from the current records.',
+ permissions => {
+ check => ['perm', '/sdn/zones/{zone}/{vnet}', ['SDN.Allocate']],
+ },
+ protected => 1,
+ proxyto => 'node',
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ zone => get_standard_option('pve-sdn-zone-id'),
+ vnet => get_standard_option('pve-sdn-vnet-id'),
+ mac => get_standard_option('mac-addr'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $vnet = PVE::Network::SDN::Vnets::get_vnet($param->{vnet}, 1);
+ die "vnet '$param->{vnet}' does not exist in zone '$param->{zone}'\n"
+ if !$vnet || $vnet->{zone} ne $param->{zone};
+
+ PVE::Network::SDN::Dhcp::update_mapping($param->{vnet}, $param->{mac});
+
+ return undef;
+ },
+});
+
__PACKAGE__->register_method({
name => 'sdnindex',
path => '',
diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
index cf805a03..1f43745f 100644
--- a/src/PVE/Network/SDN/Dhcp.pm
+++ b/src/PVE/Network/SDN/Dhcp.pm
@@ -3,6 +3,8 @@ package PVE::Network::SDN::Dhcp;
use strict;
use warnings;
+use POSIX qw();
+
use PVE::Cluster;
use PVE::Network::SDN;
@@ -86,6 +88,78 @@ sub guest_nics {
return $nics;
}
+sub guest_node_by_mac {
+ my ($vnetid, $mac) = @_;
+
+ for my $nic (guest_nics()->@*) {
+ return $nic->{node}
+ if lc($nic->{mac} // '') eq lc($mac) && $nic->{bridge} eq $vnetid;
+ }
+
+ return undef;
+}
+
+sub notify_guest_node {
+ my ($vnetid, $mac) = @_;
+
+ return if !$mac;
+
+ # best effort like the push itself, since the record is written by now
+ my ($zoneid, $node) = eval {
+ my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid, 1);
+ return if !$vnet;
+
+ my $zone = PVE::Network::SDN::Zones::get_zone($vnet->{zone}, 1);
+ return if !$zone || !$zone->{ipam} || !$zone->{dhcp};
+
+ return ($vnet->{zone}, guest_node_by_mac($vnetid, $mac));
+ };
+ if ($@) {
+ warn "could not find the node to push the dhcp mapping of $mac to: $@";
+ return;
+ }
+ return if !$node || $node eq PVE::INotify::nodename();
+
+ # double fork, the grandchild is reparented to init so nothing on
+ # the request path ever waits on it
+ my $pid = fork();
+ if (!defined($pid)) {
+ warn "could not fork for the dhcp mapping push to $node: $!\n";
+ return;
+ }
+ if ($pid) {
+ waitpid($pid, 0);
+ return;
+ }
+
+ POSIX::setsid();
+ my $child = fork();
+ POSIX::_exit(1) if !defined($child);
+ POSIX::_exit(0) if $child;
+
+ # nothing of the request stays with the push, and since the daemon's
+ # stderr is closed a failed push reports through the journal
+ open(STDIN, '<', '/dev/null') or POSIX::_exit(1);
+ open(STDOUT, '>', '/dev/null') or POSIX::_exit(1);
+ open(STDERR, '|-', 'logger', '-t', 'pve-sdn', '-p', 'daemon.warning')
+ or POSIX::_exit(1);
+ # a node that takes the connection but never answers must not keep the
+ # push around
+ exec(
+ 'timeout',
+ '60',
+ 'pvesh',
+ 'create',
+ "/nodes/$node/sdn/dhcp-mapping",
+ '--zone',
+ $zoneid,
+ '--vnet',
+ $vnetid,
+ '--mac',
+ $mac,
+ ) or POSIX::_exit(1);
+}
+
sub tap_plug {
my ($bridge, $iface, $mac) = @_;
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 783d0d47..9a4dbb9c 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -49,6 +49,7 @@ sub clear_test_state {
dnsmasq_calls => [],
ebpf_calls => [],
order => [],
+ notify_calls => [],
warnings => [],
ipam_config => {
'ids' => {
@@ -238,6 +239,11 @@ $mocked_sdn_dhcp_dnsmasq->mock(
update_lease => sub { },
);
+my $mocked_sdn_dhcp = Test::MockModule->new('PVE::Network::SDN::Dhcp');
+$mocked_sdn_dhcp->mock(
+ notify_guest_node => sub { push $test_state->{notify_calls}->@*, [@_]; },
+);
+
my $mocked_pve_rs_dhcp = Test::MockModule->new('PVE::RS::SDN::Dhcp');
$mocked_pve_rs_dhcp->mock(
detach => sub {
@@ -1280,6 +1286,90 @@ sub test_dhcp_backend_needed_per_node {
run_test(\&test_dhcp_backend_needed_per_node);
+sub test_mapping_push_target {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ });
+
+ # the guest behind a MAC is found by its NIC on the vnet, and the config
+ # writes the MAC in upper case
+ $test_state->{vmlist} = {
+ ids => {
+ 100 => { type => 'qemu', node => 'other' },
+ 101 => { type => 'lxc', node => 'third' },
+ },
+ };
+ $test_state->{guest_nets} = {
+ 1 => { net0 => "virtio=$mac,bridge=$vnetid" },
+ 100 => { net0 => "virtio=" . uc($mac) . ",bridge=$vnetid" },
+ 101 => { net0 => "name=eth0,bridge=vmbr0,hwaddr=00:11:22:33:44:55" },
+ };
+ is(
+ PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, $mac),
+ 'other',
+ "$test_name: the MAC resolves to the node of its guest, a guest the list lacks is skipped",
+ );
+ is(
+ PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, '00:11:22:33:44:55'),
+ undef,
+ "$test_name: a MAC on another bridge does not resolve",
+ );
+ is(
+ PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, '00:11:22:33:44:66'),
+ undef,
+ "$test_name: an unknown MAC does not resolve",
+ );
+
+ # every mapping edit through the API pokes the guest's node once
+ $test_state->{notify_calls} = [];
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.50",
+ });
+ update_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.51",
+ });
+ delete_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.51",
+ });
+ eq_or_diff(
+ $test_state->{notify_calls},
+ [[$vnetid, $mac], [$vnetid, $mac], [$vnetid, $mac]],
+ "$test_name: create, edit and delete each poke the guest's node",
+ );
+ delete $test_state->{vmlist};
+ delete $test_state->{guest_nets};
+}
+
+run_test(\&test_mapping_push_target);
+
sub test_ipam_cache_misses {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH pve-network v3 12/17] sdn: zones: allow the ebpf DHCP backend on every zone type
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (10 preceding siblings ...)
2026-10-20 7:59 ` [PATCH pve-network v3 11/17] sdn: dhcp: push IP mapping edits to the guest's node Hannes Laimer
@ 2026-10-20 8:00 ` Hannes Laimer
2026-10-20 8:00 ` [PATCH qemu-server v3 13/17] network: sdn: keep the DHCP backends current on guest NICs Hannes Laimer
` (6 subsequent siblings)
18 siblings, 0 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 8:00 UTC (permalink / raw)
To: pve-devel
The ebpf backend answers each guest on the guest's own tap or veth
device, so the zone type makes no difference to it. dnsmasq stays
limited to simple zones, since every node runs its own instance on
the vnet bridge and only a simple zone's bridge is cut off from the
other nodes. Choosing it elsewhere fails on create and on update.
Every zone type now reports the MTU its guests should use. The ebpf
backend hands it out, and until now only simple zones could answer.
A VXLAN or EVPN zone without an MTU setting takes it from the
interface that reaches its peers.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/Network/SDN/Zones.pm | 8 +-
src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 32 +++++++
src/PVE/Network/SDN/Zones/FaucetPlugin.pm | 1 +
src/PVE/Network/SDN/Zones/QinQPlugin.pm | 7 ++
src/PVE/Network/SDN/Zones/SimplePlugin.pm | 2 +-
src/PVE/Network/SDN/Zones/VlanPlugin.pm | 7 ++
src/PVE/Network/SDN/Zones/VxlanPlugin.pm | 25 +++++
src/test/run_test_vnets_blackbox.pl | 111 +++++++++++++++++++++-
8 files changed, 188 insertions(+), 5 deletions(-)
diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm
index 0e907260..181d2872 100644
--- a/src/PVE/API2/Network/SDN/Zones.pm
+++ b/src/PVE/API2/Network/SDN/Zones.pm
@@ -93,7 +93,7 @@ my $ZONE_PROPERTIES = {
type => 'string',
enum => PVE::Network::SDN::Dhcp->plugin_types(),
optional => 1,
- description => 'Name of DHCP server backend for this zone.',
+ description => 'DHCP backend serving the zone. dnsmasq works on simple zones only.',
},
'rt-import' => {
type => 'string',
@@ -448,6 +448,9 @@ __PACKAGE__->register_method({
raise_param_exc({ ipam => "$ipam not existing" })
if $ipam && !$ipam_cfg->{ids}->{$ipam};
+ raise_param_exc({ dhcp => "the dnsmasq backend only supports simple zones" })
+ if ($opts->{dhcp} // '') eq 'dnsmasq' && $plugin->type() ne 'simple';
+
$zone_cfg->{ids}->{$id} = $opts;
$plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
@@ -544,6 +547,9 @@ __PACKAGE__->register_method({
raise_param_exc({ ipam => "$ipam not existing" })
if $ipam && !$ipam_cfg->{ids}->{$ipam};
+ raise_param_exc({ dhcp => "the dnsmasq backend only supports simple zones" })
+ if ($scfg->{dhcp} // '') eq 'dnsmasq' && $plugin->type() ne 'simple';
+
$plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
PVE::Network::SDN::Zones::write_config($zone_cfg);
diff --git a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
index 0e79707c..60fe788f 100644
--- a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
@@ -125,10 +125,42 @@ sub options {
reversedns => { optional => 1 },
dnszone => { optional => 1 },
ipam => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
# Plugin implementation
+# the underlay interface is the one the controller's peers name, found the
+# way the bridge generation finds it
+sub get_mtu {
+ my ($class, $plugin_config) = @_;
+
+ return $plugin_config->{mtu} if $plugin_config->{mtu};
+
+ my $iface;
+ my $controller_cfg = PVE::Network::SDN::running_config()->{controllers};
+ my $controller = $controller_cfg->{ids}->{ $plugin_config->{controller} // '' };
+ if ($controller && $controller->{peers}) {
+ my @peers = PVE::Tools::split_list($controller->{peers});
+ my $local_node = PVE::INotify::nodename();
+ my $bgprouter = PVE::Network::SDN::Controllers::EvpnPlugin::find_bgp_controller(
+ $local_node, $controller_cfg,
+ );
+ my $isisrouter = PVE::Network::SDN::Controllers::EvpnPlugin::find_isis_controller(
+ $local_node, $controller_cfg,
+ );
+ my $loopback = $bgprouter->{loopback} // $isisrouter->{loopback};
+ (undef, $iface) = eval {
+ PVE::Network::SDN::Zones::Plugin::find_local_ip_interface_peers(\@peers, $loopback);
+ };
+ }
+ return PVE::Network::SDN::Zones::VxlanPlugin::vxlan_mtu(
+ $plugin_config,
+ $iface,
+ PVE::INotify::read_file('interfaces'),
+ );
+}
+
sub generate_sdn_config {
my (
$class,
diff --git a/src/PVE/Network/SDN/Zones/FaucetPlugin.pm b/src/PVE/Network/SDN/Zones/FaucetPlugin.pm
index 5f069aea..4e3f5eff 100644
--- a/src/PVE/Network/SDN/Zones/FaucetPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/FaucetPlugin.pm
@@ -30,6 +30,7 @@ sub options {
reversedns => { optional => 1 },
dnszone => { optional => 1 },
ipam => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
diff --git a/src/PVE/Network/SDN/Zones/QinQPlugin.pm b/src/PVE/Network/SDN/Zones/QinQPlugin.pm
index 642420e9..85d9ce98 100644
--- a/src/PVE/Network/SDN/Zones/QinQPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/QinQPlugin.pm
@@ -36,6 +36,12 @@ sub properties {
};
}
+sub get_mtu {
+ my ($class, $plugin_config) = @_;
+
+ return $plugin_config->{mtu};
+}
+
sub options {
return {
nodes => { optional => 1 },
@@ -48,6 +54,7 @@ sub options {
reversedns => { optional => 1 },
dnszone => { optional => 1 },
ipam => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
diff --git a/src/PVE/Network/SDN/Zones/SimplePlugin.pm b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
index 347eee9a..655f7e41 100644
--- a/src/PVE/Network/SDN/Zones/SimplePlugin.pm
+++ b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
@@ -31,7 +31,7 @@ sub properties {
description => "dns domain zone ex: mydomain.com",
},
dhcp => {
- description => 'Type of the DHCP backend for this zone',
+ description => 'DHCP backend serving the zone. dnsmasq works on simple zones only.',
type => 'string',
enum => PVE::Network::SDN::Dhcp->plugin_types(),
},
diff --git a/src/PVE/Network/SDN/Zones/VlanPlugin.pm b/src/PVE/Network/SDN/Zones/VlanPlugin.pm
index 271b3fab..a5d0fb0d 100644
--- a/src/PVE/Network/SDN/Zones/VlanPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/VlanPlugin.pm
@@ -36,6 +36,12 @@ sub properties {
};
}
+sub get_mtu {
+ my ($class, $plugin_config) = @_;
+
+ return $plugin_config->{mtu};
+}
+
sub options {
return {
@@ -47,6 +53,7 @@ sub options {
reversedns => { optional => 1 },
dnszone => { optional => 1 },
ipam => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
diff --git a/src/PVE/Network/SDN/Zones/VxlanPlugin.pm b/src/PVE/Network/SDN/Zones/VxlanPlugin.pm
index a4082615..0dab5d95 100644
--- a/src/PVE/Network/SDN/Zones/VxlanPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/VxlanPlugin.pm
@@ -47,6 +47,30 @@ sub properties {
};
}
+# the vnet bridge takes the underlay interface's MTU minus the vxlan
+# encapsulation where the peers name a local interface, the default
+# otherwise. An explicit zone MTU wins
+sub vxlan_mtu {
+ my ($plugin_config, $iface, $interfaces_config) = @_;
+
+ return $plugin_config->{mtu} if $plugin_config->{mtu};
+ return $interfaces_config->{ifaces}->{$iface}->{mtu} - 50
+ if $iface && $interfaces_config->{ifaces}->{$iface}->{mtu};
+ return 1450;
+}
+
+sub get_mtu {
+ my ($class, $plugin_config) = @_;
+
+ my $iface;
+ if ($plugin_config->{peers}) {
+ my @peers = PVE::Tools::split_list($plugin_config->{peers});
+ (undef, $iface) =
+ eval { PVE::Network::SDN::Zones::Plugin::find_local_ip_interface_peers(\@peers) };
+ }
+ return vxlan_mtu($plugin_config, $iface, PVE::INotify::read_file('interfaces'));
+}
+
sub options {
return {
nodes => { optional => 1 },
@@ -58,6 +82,7 @@ sub options {
dnszone => { optional => 1 },
ipam => { optional => 1 },
fabric => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 9a4dbb9c..816f8382 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -1488,6 +1488,55 @@ sub test_ipam_cache_case {
run_test(\&test_ipam_cache_case);
+sub test_zone_dhcp_backends {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+
+ # dnsmasq answers on the vnet bridge, so it only fits zones confined to
+ # one node. The ebpf backend answers on the guest's tap and fits any
+ my $zone = {
+ type => "vlan",
+ bridge => "vmbr0",
+ ipam => "pve",
+ zone => $zoneid,
+ };
+ eval { PVE::API2::Network::SDN::Zones->create({ %$zone, dhcp => "dnsmasq" }); };
+ like(
+ $@,
+ qr/dnsmasq backend only supports simple zones/,
+ "$test_name: dnsmasq is refused on a vlan zone",
+ );
+
+ create_zone({ %$zone, dhcp => "ebpf" });
+ is(get_zone($zoneid)->{dhcp}, "ebpf", "$test_name: ebpf is accepted on a vlan zone");
+
+ eval { update_zone($zoneid, { dhcp => "dnsmasq" }); };
+ like(
+ $@,
+ qr/dnsmasq backend only supports simple zones/,
+ "$test_name: switching a vlan zone to dnsmasq is refused",
+ );
+
+ # every other zone type offers the backend as well
+ $test_state->{controller_config} = {
+ ids => { ctrl => { type => 'evpn', asn => 65000, peers => '10.0.0.1' } },
+ };
+ my $params = {
+ qinq => { bridge => 'vmbr0', tag => 100 },
+ vxlan => { peers => '10.0.0.1' },
+ evpn => { controller => 'ctrl', 'vrf-vxlan' => 1000 },
+ faucet => { 'dp-id' => 1, controller => 'ctrl' },
+ };
+ for my $type (sort keys %$params) {
+ my $id = uc($type) . "Z";
+ my $zone = { type => $type, ipam => "pve", zone => $id, %{ $params->{$type} } };
+ create_zone({ %$zone, dhcp => "ebpf" });
+ is(get_zone($id)->{dhcp}, "ebpf", "$test_name: ebpf is accepted on a $type zone");
+ }
+}
+
+run_test(\&test_zone_dhcp_backends);
+
sub test_dnsmasq_dual_stack_and_sweep {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
@@ -1584,7 +1633,7 @@ sub test_dnsmasq_dual_stack_and_sweep {
is(
PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
"$mac,10.0.0.100,[8888::ab]\n$other,10.0.0.102,[8888::AB]\n",
- "$test_name: a line of another MAC spelled by the old code is kept",
+ "$test_name: a line of another MAC in its earlier spelling is kept",
);
is(
scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
@@ -1818,6 +1867,8 @@ sub test_ebpf_backend {
);
}
+run_test(\&test_ebpf_backend);
+
sub test_ebpf_backend_edge_cases {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
@@ -1933,7 +1984,7 @@ sub test_ebpf_backend_edge_cases {
);
}
-run_test(\&test_ebpf_backend);
+run_test(\&test_ebpf_backend_edge_cases);
sub test_zone_tap_plug {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
@@ -2063,7 +2114,6 @@ sub test_zone_tap_plug {
}
run_test(\&test_zone_tap_plug);
-run_test(\&test_ebpf_backend_edge_cases);
sub test_ebpf_mac_case {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
@@ -2252,6 +2302,61 @@ sub test_ebpf_zone_mtu {
run_test(\&test_ebpf_zone_mtu);
+sub test_ebpf_vxlan_mtu {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "VXZONE";
+ my $vnetid = "vxvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a vxlan zone without an MTU of its own answers with the one its bridge
+ # takes from the underlay, the encapsulation overhead taken off
+ my $mocked_zone_plugin = Test::MockModule->new('PVE::Network::SDN::Zones::Plugin');
+ $mocked_zone_plugin->mock(
+ find_local_ip_interface_peers => sub { return ('10.10.10.1', 'eno1'); });
+ $mocked_pve_inotify->mock(
+ read_file => sub { return { ifaces => { eno1 => { mtu => 1400 } } }; });
+ create_zone({
+ type => "vxlan",
+ peers => "10.10.10.2",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ tag => 100,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ $test_state->{macdb} = { macs => { $mac => { ip4 => '10.0.0.100' } } };
+
+ take_ebpf_calls();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{mtu},
+ 1350,
+ "$test_name: the underlay's MTU minus the encapsulation is handed out",
+ );
+ update_zone($zoneid, { mtu => 1500 });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{mtu},
+ 1500,
+ "$test_name: an explicit zone MTU wins",
+ );
+ $mocked_pve_inotify->unmock('read_file');
+}
+
+run_test(\&test_ebpf_vxlan_mtu);
+
sub test_ebpf_overlapping_subnets {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH qemu-server v3 13/17] network: sdn: keep the DHCP backends current on guest NICs
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (11 preceding siblings ...)
2026-10-20 8:00 ` [PATCH pve-network v3 12/17] sdn: zones: allow the ebpf DHCP backend on every zone type Hannes Laimer
@ 2026-10-20 8:00 ` Hannes Laimer
2026-10-20 13:38 ` Stefan Hanreich
2026-10-20 8:00 ` [PATCH pve-container v3 14/17] net: report NIC plugs and removals to the SDN DHCP backends Hannes Laimer
` (5 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 8:00 UTC (permalink / raw)
To: pve-devel
A DHCP backend that serves the guest's tap itself has to hear of
every plug and removal of that tap. The plug now carries the NIC's
MAC, so the backend can look up the NIC's record. Removing a tap now
goes through the SDN layer as well, so the backend drops its entry
for that tap.
A bridge change allocated the NIC's address in the new vnet, but
unlike a start or a hotplug it never pushed the mapping to the
backend of that vnet's zone. It now does.
During a live migration the cluster lists the guest on the source
until its config moves. A mapping edited in that window is sent to
the source, while the target keeps the record it plugged the NIC
with. When the target resumes the guest after the migration, it
first pushes the current mapping of every NIC.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/CLI/qm.pm | 4 ++--
src/PVE/QemuServer.pm | 4 ++++
src/PVE/QemuServer/Network.pm | 19 +++++++++++++++++--
src/PVE/QemuServer/RunState.pm | 2 ++
src/usr/pve-bridge | 8 +++++++-
src/usr/pve-bridgedown | 4 ++--
6 files changed, 34 insertions(+), 7 deletions(-)
diff --git a/src/PVE/CLI/qm.pm b/src/PVE/CLI/qm.pm
index b903c1f1..6c53581a 100755
--- a/src/PVE/CLI/qm.pm
+++ b/src/PVE/CLI/qm.pm
@@ -22,7 +22,7 @@ use PVE::GuestHelpers;
use PVE::GuestImport::OVF;
use PVE::INotify;
use PVE::JSONSchema qw(get_standard_option);
-use PVE::Network;
+use PVE::Network::SDN::Zones;
use PVE::RPCEnvironment;
use PVE::SafeSyslog;
use PVE::Tools qw(extract_param file_get_contents);
@@ -1143,7 +1143,7 @@ __PACKAGE__->register_method({
foreach my $opt (keys %$conf) {
next if $opt !~ m/^net(\d+)$/;
my $interface = $1;
- PVE::Network::tap_unplug("tap${vmid}i${interface}");
+ PVE::Network::SDN::Zones::tap_unplug("tap${vmid}i${interface}");
}
}
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 135c993f..39bdfc8b 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -4975,6 +4975,9 @@ sub vmconfig_update_net {
PVE::Network::SDN::Vnets::add_next_free_cidr(
$newnet->{bridge}, $conf->{name}, $newnet->{macaddr}, $vmid, undef, 1,
);
+ PVE::Network::SDN::Vnets::add_dhcp_mapping(
+ $newnet->{bridge}, $newnet->{macaddr}, $vmid, $conf->{name},
+ );
}
PVE::QemuServer::Network::tap_plug(
@@ -4984,6 +4987,7 @@ sub vmconfig_update_net {
$newnet->{firewall},
$newnet->{trunks},
$newnet->{rate},
+ { mac => $newnet->{macaddr} },
);
} elsif (safe_num_ne($oldnet->{rate}, $newnet->{rate})) {
diff --git a/src/PVE/QemuServer/Network.pm b/src/PVE/QemuServer/Network.pm
index b4893c01..e95fa62d 100644
--- a/src/PVE/QemuServer/Network.pm
+++ b/src/PVE/QemuServer/Network.pm
@@ -8,6 +8,7 @@ use PVE::Firewall::Helpers;
use PVE::JSONSchema qw(get_standard_option parse_property_string);
use PVE::Network::SDN::Vnets;
use PVE::Network::SDN::Zones;
+use PVE::Network::SDN::Dhcp;
use PVE::RESTEnvironment qw(log_warn);
use PVE::Tools qw($IPV6RE file_read_firstline);
@@ -272,6 +273,20 @@ sub add_random_macs {
}
}
+# an edit made while the VM migrated went to the source, which the vmlist
+# still named. The target pushes every NIC's current mapping once it runs here
+# for good
+sub add_nets_dhcp_mapping {
+ my ($conf, $vmid) = @_;
+
+ for my $opt (keys %$conf) {
+ next if $opt !~ m/^net\d+$/;
+ my $net = parse_net($conf->{$opt}, 1) or next;
+ next if !$net->{bridge} || !$net->{macaddr};
+ PVE::Network::SDN::Dhcp::update_mapping($net->{bridge}, $net->{macaddr}, $vmid);
+ }
+}
+
sub add_nets_bridge_fdb {
my ($conf, $vmid) = @_;
@@ -349,10 +364,10 @@ sub delete_ifaces_ipams_ips {
}
sub tap_plug {
- my ($iface, $bridge, $tag, $firewall, $trunks, $rate) = @_;
+ my ($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts) = @_;
$firewall = $firewall && PVE::Firewall::Helpers::needs_fwbr($bridge);
- PVE::Network::SDN::Zones::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate);
+ PVE::Network::SDN::Zones::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts);
}
sub get_nets_host_mtu {
diff --git a/src/PVE/QemuServer/RunState.pm b/src/PVE/QemuServer/RunState.pm
index 889ddd28..0fdd8807 100644
--- a/src/PVE/QemuServer/RunState.pm
+++ b/src/PVE/QemuServer/RunState.pm
@@ -181,6 +181,8 @@ sub vm_resume($vmid, $skiplock = undef, $nocheck = undef) {
PVE::QemuServer::Network::add_nets_bridge_fdb($conf, $vmid)
if $resume_cmd eq 'cont';
+ PVE::QemuServer::Network::add_nets_dhcp_mapping($conf, $vmid)
+ if $nocheck && $resume_cmd eq 'cont';
mon_cmd($vmid, $resume_cmd);
},
diff --git a/src/usr/pve-bridge b/src/usr/pve-bridge
index c7260c6b..20724271 100755
--- a/src/usr/pve-bridge
+++ b/src/usr/pve-bridge
@@ -43,7 +43,13 @@ die "unable to parse network config '$netid'\n" if !$net;
PVE::Network::SDN::Vnets::add_dhcp_mapping($net->{bridge}, $net->{macaddr}, $vmid, $conf->{name});
PVE::Network::SDN::Zones::tap_create($iface, $net->{bridge});
PVE::QemuServer::Network::tap_plug(
- $iface, $net->{bridge}, $net->{tag}, $net->{firewall}, $net->{trunks}, $net->{rate},
+ $iface,
+ $net->{bridge},
+ $net->{tag},
+ $net->{firewall},
+ $net->{trunks},
+ $net->{rate},
+ { mac => $net->{macaddr} },
);
exit 0;
diff --git a/src/usr/pve-bridgedown b/src/usr/pve-bridgedown
index e867b640..79a3722b 100755
--- a/src/usr/pve-bridgedown
+++ b/src/usr/pve-bridgedown
@@ -2,7 +2,7 @@
use strict;
use warnings;
-use PVE::Network;
+use PVE::Network::SDN::Zones;
my $iface = shift;
@@ -11,6 +11,6 @@ die "no interface specified\n" if !$iface;
die "got strange interface name '$iface'\n"
if $iface !~ m/^tap(\d+)i(\d+)$/;
-PVE::Network::tap_unplug($iface);
+PVE::Network::SDN::Zones::tap_unplug($iface);
exit 0;
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH pve-container v3 14/17] net: report NIC plugs and removals to the SDN DHCP backends
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (12 preceding siblings ...)
2026-10-20 8:00 ` [PATCH qemu-server v3 13/17] network: sdn: keep the DHCP backends current on guest NICs Hannes Laimer
@ 2026-10-20 8:00 ` Hannes Laimer
2026-10-20 13:36 ` Stefan Hanreich
2026-10-20 8:00 ` [PATCH pve-manager v3 15/17] ui: sdn: add zone DHCP backend selector and subnet DHCP options Hannes Laimer
` (4 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 8:00 UTC (permalink / raw)
To: pve-devel
An SDN DHCP backend answers a guest NIC with the address IPAM holds
for its MAC. A NIC joining its bridge now hands its MAC to the SDN
plug, so the backend of its vnet can answer its first request.
Deleting a veth also reports it as gone to the backends, since their
state for an interface can outlive it.
On a running container, a bridge change now pushes the DHCP mapping
for the new vnet, as adding a NIC or changing its MAC already does.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/LXC.pm | 13 +++++++++++--
src/PVE/LXC/Config.pm | 3 ++-
src/lxc-pve-poststop-hook | 6 +++---
3 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 1f11490..c3cf9e7 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -1111,7 +1111,13 @@ sub net_tap_plug : prototype($$) {
my $create_firewall_bridges = $net->{firewall} && PVE::Firewall::Helpers::needs_fwbr($bridge);
PVE::Network::SDN::Zones::tap_plug(
- $iface, $bridge, $tag, $create_firewall_bridges, $trunks, $rate,
+ $iface,
+ $bridge,
+ $tag,
+ $create_firewall_bridges,
+ $trunks,
+ $rate,
+ { mac => $hwaddr },
);
PVE::Network::SDN::Zones::add_bridge_fdb($iface, $hwaddr, $bridge);
@@ -1137,7 +1143,7 @@ sub update_net {
|| safe_string_ne($oldnet->{name}, $newnet->{name})
) {
- PVE::Network::veth_delete($veth);
+ PVE::Network::SDN::Zones::veth_delete($veth);
if (safe_string_ne($oldnet->{hwaddr}, $newnet->{hwaddr})) {
eval {
@@ -1192,6 +1198,9 @@ sub update_net {
PVE::Network::SDN::Vnets::add_next_free_cidr(
$newnet->{bridge}, $conf->{hostname}, $newnet->{hwaddr}, $vmid, undef, 1,
);
+ PVE::Network::SDN::Vnets::add_dhcp_mapping(
+ $newnet->{bridge}, $newnet->{hwaddr}, $vmid, $conf->{hostname},
+ );
}
eval { PVE::LXC::net_tap_plug($veth, $newnet) };
if (my $err = $@) {
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 9823d00..b522413 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -12,6 +12,7 @@ use PVE::GuestHelpers;
use PVE::INotify;
use PVE::JSONSchema qw(get_standard_option);
use PVE::Network::SDN::Vnets;
+use PVE::Network::SDN::Zones;
use PVE::RESTEnvironment;
use PVE::Tools;
@@ -1696,7 +1697,7 @@ sub vmconfig_hotplug_pending {
my $net = parse_lxc_network($conf->{$opt});
PVE::LXC::kill_dhclients($vmid, $net->{name}) if $net->{'host-managed'};
- PVE::Network::veth_delete("veth${vmid}i$netid");
+ PVE::Network::SDN::Zones::veth_delete("veth${vmid}i$netid");
print "delete ips from $opt\n";
eval {
PVE::Network::SDN::Vnets::del_ips_from_mac(
diff --git a/src/lxc-pve-poststop-hook b/src/lxc-pve-poststop-hook
index c8bbb79..c5d1059 100755
--- a/src/lxc-pve-poststop-hook
+++ b/src/lxc-pve-poststop-hook
@@ -11,7 +11,7 @@ use PVE::GuestHelpers;
use PVE::LXC::Config;
use PVE::LXC::Tools;
use PVE::LXC;
-use PVE::Network;
+use PVE::Network::SDN::Zones;
use PVE::RESTEnvironment;
use PVE::Storage;
use PVE::Tools;
@@ -49,8 +49,8 @@ PVE::LXC::Tools::lxc_hook(
my $ind = $1;
my $net = PVE::LXC::Config->parse_lxc_network($conf->{$k});
next if $net->{type} ne 'veth';
- # veth_delete tests with '-d /sys/class/net/$name' before running the command
- PVE::Network::veth_delete("veth${vmid}i$ind");
+ # reports the unplug and deletes the veth only where it still exists
+ PVE::Network::SDN::Zones::veth_delete("veth${vmid}i$ind");
}
my $config_updated = 0;
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH pve-manager v3 15/17] ui: sdn: add zone DHCP backend selector and subnet DHCP options
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (13 preceding siblings ...)
2026-10-20 8:00 ` [PATCH pve-container v3 14/17] net: report NIC plugs and removals to the SDN DHCP backends Hannes Laimer
@ 2026-10-20 8:00 ` Hannes Laimer
2026-10-20 13:19 ` Stefan Hanreich
2026-10-20 8:00 ` [PATCH pve-manager v3 16/17] sdn: bring up the DHCP backends at boot before the guests start Hannes Laimer
` (3 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 8:00 UTC (permalink / raw)
To: pve-devel
Zones of every type can now get DHCP from the eBPF backend, while
dnsmasq stays limited to simple zones. The checkbox that enabled
dnsmasq on simple zones becomes a backend selector in the advanced
options of every zone dialog.
The subnet dialog gains fields for the new DHCP lease time and for the
DHCP DNS server, which could be set only through the API until now.
The eBPF backend hands out a DNS server only when the subnet sets one.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
www/manager6/sdn/SubnetEdit.js | 23 +++++++++++++++++++++++
www/manager6/sdn/zones/Base.js | 17 +++++++++++++++++
www/manager6/sdn/zones/SimpleEdit.js | 11 -----------
3 files changed, 40 insertions(+), 11 deletions(-)
diff --git a/www/manager6/sdn/SubnetEdit.js b/www/manager6/sdn/SubnetEdit.js
index a3608428..a1d36eeb 100644
--- a/www/manager6/sdn/SubnetEdit.js
+++ b/www/manager6/sdn/SubnetEdit.js
@@ -56,6 +56,29 @@ Ext.define('PVE.sdn.SubnetInputPanel', {
deleteEmpty: '{!isCreate}',
},
},
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'dhcp-dns-server',
+ vtype: 'IP64Address',
+ fieldLabel: gettext('DHCP DNS Server'),
+ allowBlank: true,
+ skipEmptyText: true,
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
+ {
+ xtype: 'proxmoxintegerfield',
+ name: 'dhcp-lease-time',
+ fieldLabel: gettext('DHCP Lease Time') + ' (s)',
+ emptyText: Proxmox.Utils.defaultText,
+ minValue: 60,
+ maxValue: 4294967295,
+ allowBlank: true,
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
],
});
diff --git a/www/manager6/sdn/zones/Base.js b/www/manager6/sdn/zones/Base.js
index 66f93de0..7696136d 100644
--- a/www/manager6/sdn/zones/Base.js
+++ b/www/manager6/sdn/zones/Base.js
@@ -82,6 +82,23 @@ Ext.define('PVE.panel.SDNZoneBase', {
},
);
+ let dhcpBackends = [
+ ['__default__', Proxmox.Utils.NoneText],
+ ['ebpf', 'eBPF'],
+ ];
+ if (me.type === 'simple') {
+ dhcpBackends.splice(1, 0, ['dnsmasq', 'dnsmasq']);
+ }
+
+ me.advancedItems.push({
+ xtype: 'proxmoxKVComboBox',
+ name: 'dhcp',
+ fieldLabel: gettext('DHCP Backend'),
+ comboItems: dhcpBackends,
+ value: '__default__',
+ deleteEmpty: !me.isCreate,
+ });
+
me.callParent();
},
});
diff --git a/www/manager6/sdn/zones/SimpleEdit.js b/www/manager6/sdn/zones/SimpleEdit.js
index ba10bb36..aeb076ae 100644
--- a/www/manager6/sdn/zones/SimpleEdit.js
+++ b/www/manager6/sdn/zones/SimpleEdit.js
@@ -19,17 +19,6 @@ Ext.define('PVE.sdn.zones.SimpleInputPanel', {
var me = this;
me.items = [];
- me.advancedItems = [
- {
- xtype: 'proxmoxcheckbox',
- name: 'dhcp',
- inputValue: 'dnsmasq',
- uncheckedValue: null,
- checked: false,
- fieldLabel: gettext('Automatic DHCP'),
- deleteEmpty: !me.isCreate,
- },
- ];
me.callParent();
},
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH pve-manager v3 16/17] sdn: bring up the DHCP backends at boot before the guests start
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (14 preceding siblings ...)
2026-10-20 8:00 ` [PATCH pve-manager v3 15/17] ui: sdn: add zone DHCP backend selector and subnet DHCP options Hannes Laimer
@ 2026-10-20 8:00 ` Hannes Laimer
2026-10-20 13:21 ` Stefan Hanreich
2026-10-20 8:00 ` [PATCH pve-docs v3 17/17] sdn: dhcp: add eBPF backend section Hannes Laimer
` (2 subsequent siblings)
18 siblings, 1 reply; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 8:00 UTC (permalink / raw)
To: pve-devel
The eBPF DHCP responder keeps its programs and maps pinned in bpffs,
so they are gone after a reboot. The SDN commit helper now brings the
DHCP backends back up on every boot, not only when it has changes to
commit.
The helper runs before the node and HA start the guests at boot.
Otherwise the first guest interface plug loads the responder with a
full pass, which holds the responder's lock exclusively. A guest
starting meanwhile waits for that lock. One that waits too long comes
up without the responder on its interface.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
bin/pve-sdn-commit | 3 +++
services/pve-sdn-commit.service | 1 +
2 files changed, 4 insertions(+)
diff --git a/bin/pve-sdn-commit b/bin/pve-sdn-commit
index aa7e9b29..ed2d6935 100644
--- a/bin/pve-sdn-commit
+++ b/bin/pve-sdn-commit
@@ -96,6 +96,9 @@ sub sdn_changed {
return fabrics_changed();
}
+# the dhcp backends bring back what a reboot took, pending changes or not
+PVE::Network::SDN::Dhcp::boot();
+
if (!sdn_changed()) {
print "No changes to SDN configuration detected, skipping reload\n";
exit 0;
diff --git a/services/pve-sdn-commit.service b/services/pve-sdn-commit.service
index ff723725..825926be 100644
--- a/services/pve-sdn-commit.service
+++ b/services/pve-sdn-commit.service
@@ -3,6 +3,7 @@ Description=Commit Proxmox VE SDN changes
DefaultDependencies=no
Wants=pve-cluster.service network.target
After=frr.service network.target corosync.service
+Before=pve-guests.service pve-ha-lrm.service
[Service]
ExecStart=/usr/share/pve-manager/helpers/pve-sdn-commit
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH pve-docs v3 17/17] sdn: dhcp: add eBPF backend section
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (15 preceding siblings ...)
2026-10-20 8:00 ` [PATCH pve-manager v3 16/17] sdn: bring up the DHCP backends at boot before the guests start Hannes Laimer
@ 2026-10-20 8:00 ` Hannes Laimer
2026-10-20 13:11 ` [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Stefan Hanreich
2026-10-21 8:46 ` superseded: " Hannes Laimer
18 siblings, 0 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-20 8:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
pvesdn.adoc | 101 ++++++++++++++++++++++++++++++++++++++++++----------
1 file changed, 83 insertions(+), 18 deletions(-)
diff --git a/pvesdn.adoc b/pvesdn.adoc
index 3fd3533..ec5c500 100644
--- a/pvesdn.adoc
+++ b/pvesdn.adoc
@@ -98,10 +98,12 @@ source /etc/network/interfaces.d/*
DHCP IPAM
~~~~~~~~~
-The DHCP integration into the built-in 'PVE' IP Address Management stack
-currently uses `dnsmasq` for giving out DHCP leases. This is currently opt-in.
+The DHCP integration into the built-in 'PVE' IP Address Management stack gives
+out leases either through `dnsmasq` or through an eBPF backend built into
+{pve}. Each zone selects its backend, and DHCP is opt-in per zone.
-To use that feature you need to install the `dnsmasq` package on every node:
+The eBPF backend needs no additional package. To use the `dnsmasq` backend you
+need to install the `dnsmasq` package on every node:
----
apt update
@@ -153,9 +155,8 @@ in your SDN setup.
* xref:pvesdn_config_controllers[Controllers]: For controlling layer 3 routing
in complex setups
-* xref:pvesdn_config_dhcp[DHCP]: Define a DHCP server for a zone that
- automatically allocates IPs for guests in the IPAM and leases them to the
- guests via DHCP.
+* xref:pvesdn_config_dhcp[DHCP]: Let a zone automatically allocate IPs for
+ guests in the IPAM and lease them to the guests via DHCP.
* xref:pvesdn_config_ipam[IPAM]: Enables external for IP address management for
guests
@@ -229,6 +230,11 @@ DNSZone:: DNS domain name. Used to register hostnames, such as
`<hostname>.<domain>`. The DNS zone must already exist on the DNS server.
Optional.
+DHCP Backend:: Hand out leases to the zone's guests through this backend, see
+ xref:pvesdn_config_dhcp[DHCP]. The `dnsmasq` backend is only available for
+ Simple Zones. The `ebpf` backend works with every zone type. Optional,
+ defaults to none, which disables DHCP.
+
[[pvesdn_zone_plugin_simple]]
Simple Zones
@@ -442,6 +448,15 @@ SNAT:: Enable Source NAT which allows VMs from inside a
DNS Zone Prefix:: Add a prefix to the domain registration, like
<hostname>.prefix.<domain> Optional.
+DHCP DNS Server:: The DNS server handed out to guests with their lease.
+ Optional.
+
+DHCP Lease Time:: The lease time in seconds, at least 60. A guest picks up a
+ changed mapping when it renews its lease, so a shorter lease time applies
+ changes sooner. Without it `dnsmasq` hands out infinite leases and the `ebpf`
+ backend ten-minute leases. `dnsmasq` raises anything below two minutes to
+ two minutes. Optional.
+
[[pvesdn_config_controllers]]
Controllers
@@ -1343,8 +1358,9 @@ DHCP
The DHCP plugin in {pve} SDN can be used to automatically deploy a DHCP server
for a Zone. It provides DHCP for all Subnets in a Zone that have a DHCP range
-configured. Currently the only available backend plugin for DHCP is the dnsmasq
-plugin.
+configured. Each zone selects one of two backend plugins, `dnsmasq` or `ebpf`.
+Both answer from the MAC address and IP mappings in the IPAM. The `ebpf`
+backend serves IPv4 subnets only.
The DHCP plugin works by allocating an IP in the IPAM plugin configured in the
Zone when adding a new network interface to a VM/CT. You can find more
@@ -1362,10 +1378,12 @@ available when using the xref:pvesdn_ipam_plugin_pveipam[PVE IPAM plugin].
Configuration
~~~~~~~~~~~~~
-You can enable automatic DHCP for a zone in the Web UI via the Zones panel and
-enabling DHCP in the advanced options of a zone.
+You can enable automatic DHCP for a zone in the web interface via the Zones
+panel by selecting a backend in the *DHCP Backend* field of the zone's advanced
+options.
-NOTE: Currently only Simple Zones have support for automatic DHCP
+NOTE: The `dnsmasq` backend is only available for Simple Zones. The `ebpf`
+backend works with every zone type.
After automatic DHCP has been enabled for a Zone, DHCP Ranges need to be
configured for the subnets in a Zone. In order to that, go to the Vnets panel and
@@ -1379,22 +1397,24 @@ pvesh set /cluster/sdn/vnets/<vnet>/subnets/<subnet>
-dhcp-range start-address=10.0.2.100,end-address=10.0.2.200
----
-You also need to have a gateway configured for the subnet - otherwise
-automatic DHCP will not work.
-
The DHCP plugin will then allocate IPs in the IPAM only in the configured
ranges.
-Do not forget to follow the installation steps for the
-xref:pvesdn_install_dhcp_ipam[dnsmasq DHCP plugin] as well.
+The `dnsmasq` backend needs a gateway on the subnet. The `ebpf` backend also
+serves a subnet without one, but then hands out no default route.
+
+The subnet options *DHCP DNS Server* and *DHCP Lease Time* set the DNS server
+and the lease time handed out, see xref:pvesdn_config_subnet[Subnets].
+
+When using the `dnsmasq` backend, do not forget to follow the installation
+steps for the xref:pvesdn_install_dhcp_ipam[dnsmasq DHCP plugin] as well.
Plugins
~~~~~~~
Dnsmasq Plugin
^^^^^^^^^^^^^^
-Currently this is the only DHCP plugin and therefore the plugin that gets used
-when you enable DHCP for a zone.
+This plugin serves zones with `dnsmasq` as their DHCP backend.
.Installation
For installation see the xref:pvesdn_install_dhcp_ipam[DHCP IPAM] section.
@@ -1437,6 +1457,51 @@ For more information please consult the documentation of
xref:pvesdn_ipam_plugin_pveipam[the PVE IPAM plugin]. Changing DHCP leases is
currently not supported for the other IPAM plugins.
+[[pvesdn_dhcp_plugin_ebpf]]
+eBPF Plugin
+^^^^^^^^^^^
+This plugin serves zones with `ebpf` as their DHCP backend.
+
+.Installation
+The backend needs no additional package. It requires host kernel version 6.6
+or newer.
+
+Instead of a DHCP server on the bridge, the backend attaches a small eBPF
+program to the network interface of every guest on the zone's VNets. The
+program answers the guest's DHCP requests from the MAC address and IP mapping
+in the IPAM, so these requests never reach the bridge. It only answers
+requests that carry the MAC address of its own network interface. Requests on
+a network interface without a mapping pass through untouched, so such a guest
+can still reach another DHCP server on the network.
+
+The backend attaches the program when a guest starts or a network interface
+is hot-plugged. Applying the SDN configuration attaches it to running guests
+as well, so a zone switched to `ebpf` serves them without a restart.
+
+A mapping changed through {pve} reaches a running guest when it next renews
+its lease. After a mapping is deleted, the backend stops answering the guest,
+which keeps its address until the lease runs out.
+
+The answer carries the address, netmask, lease time, and server address. The
+zone's MTU, the subnet's gateway, and its IPv4 DNS server are included where
+the zone or subnet has one. Options `dnsmasq` can add through its custom
+configuration, such as a domain name or PXE boot parameters, are not
+available.
+
+.Notes
+The backend serves IPv4 only. Only untagged requests are answered, so a guest
+tagging its own traffic, for example on a trunk port, is not served. A guest
+with a mapping cannot use another DHCP server on the network, since its
+requests never leave its network interface. Clients behind a guest, such as
+containers bridged inside a VM, get no answer, and neither does a guest that
+changed its MAC address in the guest OS.
+
+NOTE: Requests this backend answers never reach the firewall of the guest's
+network interface, so the guest's firewall rules and the DHCP option of its
+firewall have no effect on them. The rate limit applies to the reply only.
+Requests the backend does not answer pass the firewall and the rate limit as
+usual.
+
[[pvesdn_firewall_integration]]
Firewall Integration
--------------------
--
2.47.3
^ permalink raw reply related [flat|nested] 43+ messages in thread
* Re: [PATCH proxmox-ebpf v3 02/17] dhcp: add the subsystem that loads the responder and keeps its records
2026-10-20 7:59 ` [PATCH proxmox-ebpf v3 02/17] dhcp: add the subsystem that loads the responder and keeps its records Hannes Laimer
@ 2026-10-20 13:09 ` Stefan Hanreich
2026-10-21 7:09 ` Hannes Laimer
0 siblings, 1 reply; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:09 UTC (permalink / raw)
To: pve-devel
On 10/20/26 10:03 AM, Hannes Laimer wrote:
> The subsystem loads the program, pins a map with one entry per guest
> interface and attaches the program to the interfaces it serves.
> Callers run a full pass over every guest interface, plug or unplug one
> interface, or change the records of a guest's interfaces.
>
> A load can leave the map empty, and only a full pass is handed every
> guest interface. So the full pass is the only step that loads. A plug
> or a record change that finds nothing loaded reports that a full pass
> has to run first.
>
> A hotplug moves a tap before the guest config names the new bridge,
> and no lock closes that gap. So every entry names the vnet its writer
> saw the interface on. A pass or the record change of a mapping edit
> leaves an entry naming another vnet alone. A plug writes outright, so
> plugs share the lock. The edit's record change reads the entry before
> it writes, so it takes the lock exclusively.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> src/dhcp/mod.rs | 1031 +++++++++++++++++++++++++++++++++++++++++++++
> src/dhcp/types.rs | 64 +++
> src/lib.rs | 3 +
> src/subsystem.rs | 2 +-
> 4 files changed, 1099 insertions(+), 1 deletion(-)
> create mode 100644 src/dhcp/mod.rs
> create mode 100644 src/dhcp/types.rs
>
> diff --git a/src/dhcp/mod.rs b/src/dhcp/mod.rs
> new file mode 100644
> index 0000000..d0c4148
> --- /dev/null
> +++ b/src/dhcp/mod.rs
> @@ -0,0 +1,1031 @@
> +//! The dhcp subsystem, a per-tap DHCPv4 responder answering from a pinned map of one entry per
> +//! interface.
> +//!
> +//! The map holds nothing of its own. The SDN side hands in every guest interface with the answer
> +//! it gets, if any. A full pass makes the kernel match that for the interfaces it names. A record
> +//! change or a plug writes its own interfaces only. A linked interface the pass does not name
> +//! keeps its link and its entry. A plug runs under the shared lock and writes its entry whatever
> +//! is there. A pass runs under the exclusive lock, and so does the record change an edit pushes,
> +//! since it reads the entry's vnet before it writes. A guest's own start holds the guest lock, so
> +//! nothing else plugs its interfaces meanwhile and it writes its records under the shared lock
> +//! without a read.
> +//!
> +//! A hotplug plugs the tap before the guest config names the new bridge, and no lock orders that.
> +//! So every entry names the vnet its writer saw the interface on. A pass or an edit's record change
> +//! leaves an entry alone that names another vnet than the configs do, and the link follows such an
> +//! entry. A plug replaces it. Plugs of one interface are serialized by the guest lock, so the plug
> +//! is the latest word on where its interface sits. An unplug drops the interface's entry. An
> +//! interface that went away without a link keeps its entry until the next pass. One plugged onto a
> +//! bridge this subsystem does not serve trades its entry for a marker naming that place.
> +mod types;
> +
> +use std::collections::{HashMap, HashSet};
> +use std::io::ErrorKind;
> +use std::net::Ipv4Addr;
> +
> +use anyhow::{Context, bail};
> +use aya::maps::MapError;
> +
> +use self::types::*;
> +use crate::subsystem::{ApplyLock, Exclusive, NotLoaded, Shared, TcPrograms};
> +use crate::tc::{Direction, LinkPin};
> +use crate::tc_programs;
> +
> +/// One guest interface as the configs know it. `serve` says whether a pass links it. A plug
> +/// links its interface regardless. `record` is the answer it gets there, and without one the
> +/// interface gets no answer. `vnet` is the vnet it sits on, none for a plain bridge.
> +pub struct Iface {
> + pub name: String,
> + pub vnet: Option<String>,
> + pub serve: bool,
> + pub record: Option<Record>,
> +}
> +
> +/// One interface's answer, the address and every option the reply carries.
> +pub struct Record {
> + pub ip: Ipv4Addr,
> + pub prefixlen: u8,
> + pub server_id: Ipv4Addr,
> + pub lease: u32,
> + pub router: Option<Ipv4Addr>,
> + pub dns: Option<Ipv4Addr>,
> + pub mtu: Option<u16>,
> + pub mac: [u8; 6],
proxmox-network-types has a MAC address type that would fit here and in other structs
> +}
> +
> +/// What became of an unplugged interface.
> +pub enum Unplugged {
> + /// Gone, or going with the unplug.
> + Gone,
> + /// Plugged onto a bridge this subsystem does not serve, the vnet it sits on now, none for a
> + /// plain bridge.
> + MovedTo(Option<String>),
> +}
> +
> +/// What a single-entry step did.
> +#[derive(Debug, PartialEq, Eq)]
> +pub enum Outcome {
> + /// Nothing is left to do. The entry is in place, or its interface is not here.
> + Done,
> + /// Nothing is loaded to write into. Only a full pass loads, so the caller runs one and retries.
> + NeedsFullPass,
> +}
> +
> +const RECORDS_MAP: &str = "dhcp_iface_records";
> +const MAPS: [&str; 1] = [RECORDS_MAP];
> +
> +// bump with any change to the key or value layout of a map in dhcp.bpf.c
> +const SCHEMA_VERSION: u32 = 1;
> +
> +// requests only arrive from the guest side and replies leave through a redirect, so only ingress
> +// carries a program
> +const PROGRAM_NAMES: [(Direction, &str); 1] = [(Direction::Ingress, "tc_dhcp_ingress")];
> +
> +type RecordsMap = aya::maps::HashMap<aya::maps::MapData, u32, DhcpRecord>;
> +
> +static PROGRAMS: TcPrograms = tc_programs! {
> + name: "dhcp",
> + programs: &PROGRAM_NAMES,
> + maps: &MAPS,
> + schema_version: SCHEMA_VERSION,
> +};
> +
> +pub struct DhcpSubsystem {
> + programs: &'static TcPrograms,
> +}
> +
> +impl Default for DhcpSubsystem {
> + fn default() -> Self {
> + Self::new()
> + }
> +}
> +
> +impl DhcpSubsystem {
> + pub fn new() -> Self {
> + Self {
> + programs: &PROGRAMS,
> + }
> + }
> +
> + /// The lock a plug runs under, held until [`attach`](SharedLock::attach) returns. Plugs run
> + /// next to each other and wait only behind an exclusive holder.
> + pub fn lock_shared(&self) -> anyhow::Result<SharedLock> {
> + Ok(SharedLock {
> + lock: self.programs.lock_shared()?,
> + })
> + }
> +
> + /// The lock a full pass or a record change runs under, held until the change is written.
> + pub fn lock_exclusive(&self) -> anyhow::Result<ExclusiveLock> {
> + Ok(ExclusiveLock {
> + lock: self.programs.lock_exclusive()?,
> + })
> + }
> +
> + /// A tap unplug. The interface may be gone by then, so its link is found by the name the pin
> + /// carries. [`Unplugged`] decides what becomes of the entry.
> + pub fn detach(&self, iface: &str, unplugged: Unplugged) -> anyhow::Result<()> {
> + // resolved before the lock wait, an interface recreated under the name meanwhile is not
> + // this unplug's and keeps its link
> + let now = ifindex_of(iface)?;
> + // every guest unplug on the host lands here. Most never had a link and must not wait
> + // behind a pass. One without a link whose interface is gone leaves its entry to the
> + // next pass
> + if !self.programs.link_pinned(iface) && now.is_none() {
> + return Ok(());
> + }
> + let lock = self.programs.lock_shared()?;
> +
> + if !lock.is_current() {
> + log::debug!("dhcp detach: nothing current, {iface} is left to the next full pass");
> + return Ok(());
> + }
> + // the map is opened and the marker built before the links go, so a failure there
> + // leaves them attached
> + let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?;
> + let marker = match &unplugged {
> + Unplugged::Gone => None,
> + Unplugged::MovedTo(vnet) => Some(DhcpRecord::removed(place(vnet.as_deref())?)),
> + };
> + // an interface without a link has an entry as well, a marker where it is not served
> + let mut ifindexes = lock.detach_named(iface, |pin| {
the name confused me a bit here, maybe detach_named_if or something would be more fitting?
> + Some(pin.ifindex) != now && pin_alive(pin).unwrap_or(false)
> + })?;
> + if let Some(now) = now.filter(|now| !ifindexes.contains(now)) {
> + ifindexes.push(now);
> + }
> + // the marker is the moved interface's. Older incarnations under the name are gone
> + unplug_entries(&mut map, &ifindexes, now.zip(marker))
> + }
> +}
> +
> +/// The shared lock of the responder, what a plug holds, and the record change of a guest's own
> +/// start.
> +pub struct SharedLock {
> + lock: ApplyLock<'static, Shared>,
> +}
> +
> +impl SharedLock {
> + /// A tap plug, the entry goes in and the program on. The plug loads nothing, so with nothing
> + /// pinned the caller runs a full pass first.
> + pub fn attach(&self, iface: &Iface) -> anyhow::Result<Outcome> {
> + if !self.lock.is_current() {
> + return Ok(Outcome::NeedsFullPass);
> + }
> + let Some(ifindex) = ifindex_of(iface.name.as_str())? else {
> + log::info!("dhcp attach: {} is gone, nothing to do", iface.name);
> + return Ok(Outcome::Done);
> + };
> + // the entry goes in before the program runs, a marker when there is no record or one
> + // the program cannot carry. A guest start must not fail on a bad record
> + let mut map: RecordsMap = self.lock.hash_map(RECORDS_MAP)?;
> + let entry = match iface.entry() {
> + Ok(entry) => entry,
> + Err(e) => {
> + log::warn!(
> + "dhcp: {} attaches with nothing to answer: {e:#}",
> + iface.name
> + );
> + DhcpRecord::removed(iface.place()?)
> + }
> + };
> + write_entry(&mut map, ifindex, &entry, Elsewhere::Override)?;
> + match self.lock.attach_iface(ifindex) {
> + Ok(()) => Ok(Outcome::Done),
> + Err(e) if e.downcast_ref::<NotLoaded>().is_some() => Ok(Outcome::NeedsFullPass),
> + Err(e) => Err(e).with_context(|| format!("dhcp responder for {}", iface.name)),
> + }
> + }
> +
> + /// A record change for a caller that holds the guest lock. Nothing else plugs the guest's
> + /// interfaces under that lock, so each entry is written outright, one atomic write, without
> + /// reading what it holds.
> + pub fn update(&self, ifaces: &[Iface]) -> anyhow::Result<Outcome> {
> + update_entries(&self.lock, ifaces, Elsewhere::Override)
> + }
> +}
> +
> +/// The exclusive lock of the responder, what a full pass and a record change hold.
> +pub struct ExclusiveLock {
> + lock: ApplyLock<'static, Exclusive>,
> +}
> +
> +impl ExclusiveLock {
> + /// The full pass. Afterward every interface the configs name that exists here has its entry,
> + /// a record where it is served with one and a marker otherwise, and a link is on every
> + /// served interface and on every interface whose entry names another vnet. Entries of
> + /// interfaces that are gone are dropped, as are entries of live interfaces the configs do
> + /// not name and that have no link. Bad input leaves its interface as it is, with a warning.
> + pub fn apply(&self, ifaces: &[Iface]) -> anyhow::Result<()> {
> + self.lock.ensure_loaded()?;
> + let present = present_ifindexes(ifaces)?;
> + // a link the configs do not name stays while its interface lives under the pin's
> + // name. A reused index is not that interface
> + let mut unknown_alive = HashSet::new();
> + for pin in self.lock.attached()? {
> + if !present.contains_key(&pin.ifindex) && pin_alive(&pin)? {
> + unknown_alive.insert(pin.ifindex);
> + }
> + }
> + let mut map: RecordsMap = self.lock.hash_map(RECORDS_MAP)?;
> + let (elsewhere, failed) = sync_records(&mut map, &present, &unknown_alive)?;
> + // a failed write or removal leaves its entry as it is. The links are still put right
> + // and every failure is reported once the rest is done
> + let links = self
> + .lock
> + .reconcile(&wanted_links(&present, &elsewhere, &unknown_alive));
> + match (failed, links) {
> + (0, links) => links,
> + (failed, Ok(())) => bail!("{failed} entries could not be changed"),
> + (failed, Err(e)) => {
> + Err(e).with_context(|| format!("{failed} entries could not be changed"))
> + }
> + }
> + }
> +
> + /// A record change. Interfaces not found here are skipped. The rest are written whether they
> + /// have a link yet or not. A record the reply cannot carry leaves its interface as it is.
> + /// With nothing loaded the caller runs a full pass first, like a plug.
> + pub fn update(&self, ifaces: &[Iface]) -> anyhow::Result<Outcome> {
> + update_entries(&self.lock, ifaces, Elsewhere::Skip)
> + }
> +
> + /// Detach everywhere and drop the pinned state, for the last zone leaving the backend.
> + pub fn clear(&self) -> anyhow::Result<()> {
> + self.lock.clear()
> + }
> +}
> +
> +fn update_entries<M>(
> + lock: &ApplyLock<'static, M>,
> + ifaces: &[Iface],
> + elsewhere: Elsewhere,
> +) -> anyhow::Result<Outcome> {
> + if !lock.is_current() {
> + return Ok(Outcome::NeedsFullPass);
> + }
> + let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?;
> + let mut failed = 0usize;
> + for iface in ifaces {
> + let ifindex = match ifindex_of(iface.name.as_str()) {
> + Ok(Some(ifindex)) => ifindex,
> + Ok(None) => continue,
> + Err(e) => {
> + log::warn!("dhcp: {e:#}");
> + failed += 1;
> + continue;
> + }
> + };
> + let entry = match iface.entry() {
> + Ok(entry) => entry,
> + Err(e) => {
> + log::warn!("dhcp: {} is left as it is: {e:#}", iface.name);
> + continue;
> + }
> + };
> + if let Err(e) = write_entry(&mut map, ifindex, &entry, elsewhere) {
> + log::warn!("dhcp: writing the entry of {}: {e:#}", iface.name);
> + failed += 1;
> + }
> + }
> + if failed > 0 {
> + bail!("{failed} interfaces could not be updated");
> + }
> + Ok(Outcome::Done)
> +}
> +
> +const NO_VNET: [u8; 8] = [0; 8];
> +
> +// the map update flag that refuses a key the map does not hold, from the kernel's uapi bpf.h
> +const BPF_EXIST: u64 = 2;
> +
> +fn unplug_entries(
> + map: &mut impl RecordStore,
> + ifindexes: &[u32],
> + moved: Option<(u32, DhcpRecord)>,
> +) -> anyhow::Result<()> {
> + for &ifindex in ifindexes {
> + match moved {
> + // only an existing entry becomes a marker, so an interface without one gets none
> + Some((current, marker)) if current == ifindex => map.mark(ifindex, marker)?,
> + _ => map.remove(ifindex)?,
> + }
> + }
> + Ok(())
> +}
> +
> +// the vnet id padded with zeros, all zero for none
> +fn place(vnet: Option<&str>) -> anyhow::Result<[u8; 8]> {
> + let Some(vnet) = vnet else {
> + return Ok(NO_VNET);
> + };
> + let bytes = vnet.as_bytes();
> + if bytes.is_empty() || bytes.len() > NO_VNET.len() {
> + bail!("vnet id {vnet:?} does not fit a record");
> + }
> + let mut place = NO_VNET;
> + place[..bytes.len()].copy_from_slice(bytes);
> + Ok(place)
> +}
> +
> +// the record map as the writers see it, so their decisions run against a plain map in tests
> +trait RecordStore {
> + fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>>;
> + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()>;
> + // replace an entry that exists and do nothing for one that does not, in one step, so it is
> + // safe under the shared lock
> + fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()>;
> + // a missing entry is removed already
> + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()>;
> + fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>>;
> +}
> +
> +impl RecordStore for RecordsMap {
> + fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
> + match aya::maps::HashMap::get(self, &ifindex, 0) {
> + Ok(live) => Ok(Some(live)),
> + Err(MapError::KeyNotFound) => Ok(None),
> + Err(e) => Err(e.into()),
> + }
> + }
> +
> + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
> + aya::maps::HashMap::insert(self, ifindex, rec, 0)?;
> + Ok(())
> + }
> +
> + fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
> + match aya::maps::HashMap::insert(self, ifindex, rec, BPF_EXIST) {
> + Ok(()) => Ok(()),
> + // the kernel answers a missing key with ENOENT, and aya passes that through as is
> + Err(MapError::SyscallError(e)) if e.io_error.kind() == ErrorKind::NotFound => Ok(()),
> + Err(e) => Err(e.into()),
> + }
> + }
> +
> + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
> + match aya::maps::HashMap::remove(self, &ifindex) {
> + Ok(()) => Ok(()),
> + // the kernel answers a missing key with ENOENT, and aya passes that through as is
> + Err(MapError::SyscallError(e)) if e.io_error.kind() == ErrorKind::NotFound => Ok(()),
> + Err(e) => Err(e.into()),
> + }
> + }
> +
> + // a record the listing cannot read is one the pass cannot compare against, so the pass
> + // fails rather than writes past it
> + fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
> + self.iter()
> + .collect::<Result<Vec<_>, _>>()
> + .context("listing the records")
> + }
> +}
> +
> +// what a write does with an entry naming another vnet than its own
> +#[derive(Clone, Copy)]
> +enum Elsewhere {
> + // a record change, its configs are stale for the interface
> + Skip,
> + // a plug, or a record change under the guest lock, both see where the interface sits now
> + Override,
> +}
> +
> +fn write_entry(
> + map: &mut impl RecordStore,
> + ifindex: u32,
> + rec: &DhcpRecord,
> + elsewhere: Elsewhere,
> +) -> anyhow::Result<()> {
> + if matches!(elsewhere, Elsewhere::Skip)
> + && let Some(live) = map.get(ifindex)?
> + && live.vnet != rec.vnet
> + {
> + log::debug!("dhcp: ifindex {ifindex} sits elsewhere than the configs say, leaving it");
> + return Ok(());
> + }
> + map.insert(ifindex, *rec)
> +}
> +
> +// entries of live links the pass does not know stay, since their plug wrote them. The other
> +// unknown ones go with their interface. Returns the interfaces whose entry names another vnet
> +// than the configs, and how many writes failed
> +fn sync_records(
> + map: &mut impl RecordStore,
> + present: &HashMap<u32, &Iface>,
> + unknown_alive: &HashSet<u32>,
> +) -> anyhow::Result<(HashSet<u32>, usize)> {
> + let live: HashMap<u32, DhcpRecord> = map.entries()?.into_iter().collect();
> + let dead: Vec<u32> = live
> + .keys()
> + .filter(|ifindex| !present.contains_key(ifindex) && !unknown_alive.contains(ifindex))
> + .copied()
> + .collect();
> + let mut failed = 0usize;
> + for &ifindex in &dead {
> + if let Err(e) = map.remove(ifindex) {
> + log::warn!("dhcp: removing the record of ifindex {ifindex}: {e:#}");
error message is missing an error or some other prefix?
> + failed += 1;
> + }
> + }
> + let mut written = 0usize;
> + let mut elsewhere = HashSet::new();
> +
> + for (ifindex, iface) in present {
> + // bad input leaves its interface as it is and the pass goes on
> + let desired = match iface.entry() {
> + Ok(desired) => desired,
> + Err(e) => {
> + log::warn!("dhcp: {} is left as it is: {e:#}", iface.name);
> + continue;
> + }
> + };
> + match live.get(ifindex) {
> + Some(entry) if entry.vnet != desired.vnet => {
> + elsewhere.insert(*ifindex);
> + }
> +
> + Some(entry) if *entry == desired => {}
> + _ => match map.insert(*ifindex, desired) {
> + Ok(()) => written += 1,
> + Err(e) => {
> + log::warn!("dhcp: writing the entry of {}: {e:#}", iface.name);
> + failed += 1;
> + }
> + },
> + }
> + }
> +
> + if written + dead.len() + elsewhere.len() > 0 {
> + log::info!(
> + "dhcp: {written} entries written, {} removed, {} left where their plug put them",
> + dead.len(),
> + elsewhere.len()
> + );
> + } else {
> + log::debug!("dhcp: {} records, no changes", live.len());
> + }
> + Ok((elsewhere, failed))
> +}
> +
> +// the interface's index here, none when it does not exist. Any other failure says nothing about
> +// the interface, and a step taking it for gone would detach what it could not resolve
> +fn ifindex_of(name: &str) -> anyhow::Result<Option<u32>> {
> + match nix::net::if_::if_nametoindex(name) {
> + Ok(ifindex) => Ok(Some(ifindex)),
> + Err(nix::errno::Errno::ENODEV) => Ok(None),
> + Err(e) => Err(e).with_context(|| format!("resolve interface {name}")),
> + }
> +}
> +
> +// the named interfaces that exist here by index, the names come from configs cluster-wide and
> +// most belong to other nodes
> +fn present_ifindexes(ifaces: &[Iface]) -> anyhow::Result<HashMap<u32, &Iface>> {
> + let mut present = HashMap::new();
> + for iface in ifaces {
> + if let Some(ifindex) = ifindex_of(iface.name.as_str())? {
> + present.insert(ifindex, iface);
> + }
> + }
> + Ok(present)
> +}
Could we use if_nameindex here instead of querying if_nametoindex for every interface?
> +// the link follows the configs where they agree with the entry. An entry naming another vnet
> +// keeps its link, and a served interface without a mapping gets one. So a record change filling
> +// a marker attaches nothing itself. A live link the configs do not know stays
> +fn wanted_links(
> + present: &HashMap<u32, &Iface>,
> + elsewhere: &HashSet<u32>,
> + unknown_alive: &HashSet<u32>,
> +) -> HashSet<u32> {
> + let mut wanted: HashSet<u32> = present
> + .iter()
> + .filter(|(ifindex, iface)| elsewhere.contains(ifindex) || iface.serve)
> + .map(|(ifindex, _)| *ifindex)
> + .collect();
> +
> + wanted.extend(unknown_alive);
> + wanted
> +}
> +
> +// whether the link's interface still exists under the name and index its pin carries
> +fn pin_alive(pin: &LinkPin) -> anyhow::Result<bool> {
> + Ok(ifindex_of(pin.name.as_str())? == Some(pin.ifindex))
> +}
> +
> +impl Iface {
> + // a marker where nothing is served
> + fn entry(&self) -> anyhow::Result<DhcpRecord> {
> + let vnet = self.place()?;
> + match &self.record {
> + Some(record) if self.serve => record
> + .entry(vnet)
> + .with_context(|| format!("record for {}", self.name)),
> + _ => Ok(DhcpRecord::removed(vnet)),
> + }
> + }
> +
> + fn place(&self) -> anyhow::Result<[u8; 8]> {
> + place(self.vnet.as_deref()).with_context(|| format!("interface {}", self.name))
> + }
> +}
> +
> +impl Record {
> + // refused here is what the reply cannot carry. Zero stands for absent in the entry, so an
> + // address or an MTU of zero cannot be told from none
> + fn entry(&self, vnet: [u8; 8]) -> anyhow::Result<DhcpRecord> {
> + if self.prefixlen == 0 || self.prefixlen > 32 {
> + bail!("prefixlen {} out of range", self.prefixlen);
> + }
> + if self.ip.is_unspecified() {
> + bail!("no address");
> + }
> + if self.server_id.is_unspecified() {
> + bail!("no server address");
> + }
> + if self.lease == 0 {
> + bail!("lease of zero seconds");
> + }
> + if self.mtu.is_some_and(|mtu| mtu < 68) {
> + bail!("mtu below the IPv4 minimum of 68");
> + }
> + if self.mac == [0; 6] {
> + bail!("no MAC");
> + }
> + let netmask = u32::MAX << (32 - self.prefixlen);
> + Ok(DhcpRecord {
> + ip: u32::from(self.ip).to_be(),
> + netmask: netmask.to_be(),
> + router: u32::from(self.router.unwrap_or(Ipv4Addr::UNSPECIFIED)).to_be(),
> + dns: u32::from(self.dns.unwrap_or(Ipv4Addr::UNSPECIFIED)).to_be(),
> + server_id: u32::from(self.server_id).to_be(),
> + lease: self.lease.to_be(),
> + mtu: self.mtu.unwrap_or(0).to_be(),
> + vnet,
> + mac: self.mac,
> + })
> + }
> +}
> +
> +#[cfg(test)]
> +mod test {
> + use std::collections::BTreeMap;
> +
> + use super::*;
> +
> + fn record() -> Record {
> + Record {
> + ip: Ipv4Addr::new(10, 0, 0, 100),
> + prefixlen: 24,
> + server_id: Ipv4Addr::new(10, 0, 0, 1),
> + lease: 300,
> + router: Some(Ipv4Addr::new(10, 0, 0, 1)),
> + dns: Some(Ipv4Addr::new(1, 1, 1, 1)),
> + mtu: Some(1500),
> + mac: MAC,
> + }
> + }
> +
> + const MAC: [u8; 6] = [0xda, 0x65, 0x8f, 0x18, 0x9b, 0x6f];
> +
> + const VNET: [u8; 8] = *b"vnet1\0\0\0";
> +
> + #[test]
> + fn converts_full_record() {
> + let rec = record().entry(VNET).unwrap();
> + assert_eq!(
> + u32::from_be(rec.ip),
> + u32::from(Ipv4Addr::new(10, 0, 0, 100))
> + );
> + assert_eq!(u32::from_be(rec.netmask), 0xffffff00);
> + assert_eq!(
> + u32::from_be(rec.router),
> + u32::from(Ipv4Addr::new(10, 0, 0, 1))
> + );
> + assert_eq!(u32::from_be(rec.dns), u32::from(Ipv4Addr::new(1, 1, 1, 1)));
> + assert_eq!(
> + u32::from_be(rec.server_id),
> + u32::from(Ipv4Addr::new(10, 0, 0, 1))
> + );
> + assert_eq!(u32::from_be(rec.lease), 300);
> + assert_eq!(u16::from_be(rec.mtu), 1500);
> + assert_eq!(rec.vnet, VNET);
> + assert_eq!(rec.mac, MAC);
> +
> + let rec = Record {
> + router: None,
> + dns: None,
> + mtu: None,
> + ..record()
> + }
> + .entry(VNET)
> + .unwrap();
> + assert_eq!(rec.router, 0);
> + assert_eq!(rec.dns, 0);
> + assert_eq!(rec.mtu, 0);
> + }
> +
> + #[test]
> + fn entries_name_the_vnet_of_the_interface() {
> + let iface = Iface {
> + name: "tap100i0".into(),
> + vnet: Some("testvnet".into()),
> + serve: true,
> + record: Some(record()),
> + };
> + assert_eq!(&iface.entry().unwrap().vnet, b"testvnet");
> + let iface = Iface {
> + name: "tap100i0".into(),
> + vnet: None,
> + serve: false,
> + record: None,
> + };
> + assert_eq!(iface.entry().unwrap().vnet, NO_VNET);
> + let iface = Iface {
> + name: "tap100i0".into(),
> + vnet: Some("ninechars".into()),
> + serve: true,
> + record: Some(record()),
> + };
> + assert!(iface.entry().is_err());
> + let iface = Iface {
> + name: "tap100i0".into(),
> + vnet: Some(String::new()),
> + serve: true,
> + record: Some(record()),
> + };
> + assert!(iface.entry().is_err());
> + }
> +
> + #[test]
> + fn unserved_interfaces_get_a_marker() {
> + let iface = Iface {
> + name: "tap100i0".into(),
> + vnet: Some("vnet1".into()),
> + serve: false,
> + record: Some(record()),
> + };
> + assert_eq!(iface.entry().unwrap().ip, 0);
> + assert_eq!(iface.entry().unwrap().vnet, VNET);
> + let iface = Iface {
> + name: "tap100i0".into(),
> + vnet: Some("vnet1".into()),
> + serve: true,
> + record: None,
> + };
> + assert_eq!(iface.entry().unwrap().ip, 0);
> + let iface = Iface {
> + name: "tap100i0".into(),
> + vnet: Some("vnet1".into()),
> + serve: true,
> + record: Some(record()),
> + };
> + assert_ne!(iface.entry().unwrap().ip, 0);
> + }
> +
> + #[test]
> + fn refuses_what_the_reply_cannot_carry() {
> + for rec in [
> + Record {
> + prefixlen: 33,
> + ..record()
> + },
> + Record {
> + prefixlen: 0,
> + ..record()
> + },
> + Record {
> + server_id: Ipv4Addr::UNSPECIFIED,
> + ..record()
> + },
> + Record {
> + lease: 0,
> + ..record()
> + },
> + Record {
> + ip: Ipv4Addr::UNSPECIFIED,
> + ..record()
> + },
> + Record {
> + mtu: Some(67),
> + ..record()
> + },
> + Record {
> + mac: [0; 6],
> + ..record()
> + },
> + ] {
> + assert!(rec.entry(VNET).is_err());
> + }
> + let rec = Record {
> + mtu: Some(68),
> + ..record()
> + };
> + assert_eq!(u16::from_be(rec.entry(VNET).unwrap().mtu), 68);
> +
> + let rec = Record {
> + prefixlen: 32,
> + ..record()
> + };
> + assert_eq!(u32::from_be(rec.entry(VNET).unwrap().netmask), u32::MAX);
> + let rec = Record {
> + prefixlen: 1,
> + ..rec
> + };
> + assert_eq!(u32::from_be(rec.entry(VNET).unwrap().netmask), 1 << 31);
> + }
> +
> + impl RecordStore for BTreeMap<u32, DhcpRecord> {
> + fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
> + Ok(BTreeMap::get(self, &ifindex).copied())
> + }
> +
> + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
> + BTreeMap::insert(self, ifindex, rec);
> + Ok(())
> + }
> +
> + fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
> + if let Some(entry) = BTreeMap::get_mut(self, &ifindex) {
> + *entry = rec;
> + }
> + Ok(())
> + }
> +
> + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
> + BTreeMap::remove(self, &ifindex);
> + Ok(())
> + }
> +
> + fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
> + Ok(self.iter().map(|(k, v)| (*k, *v)).collect())
> + }
> + }
> +
> + const OTHER: [u8; 8] = *b"vnet2\0\0\0";
> +
> + fn entry(vnet: [u8; 8]) -> DhcpRecord {
> + record().entry(vnet).unwrap()
> + }
> +
> + fn other_answer(vnet: [u8; 8]) -> DhcpRecord {
> + Record {
> + ip: Ipv4Addr::new(10, 0, 0, 200),
> + ..record()
> + }
> + .entry(vnet)
> + .unwrap()
> + }
> +
> + #[test]
> + fn a_record_change_writes_on_its_own_vnet() {
> + let mut map = BTreeMap::new();
> + write_entry(&mut map, 1, &entry(VNET), Elsewhere::Skip).unwrap();
> + assert_eq!(map[&1], entry(VNET));
> + write_entry(&mut map, 1, &other_answer(VNET), Elsewhere::Skip).unwrap();
> + assert_eq!(map[&1], other_answer(VNET));
> + // an entry naming another vnet was placed by a plug and stays
> + map.insert(1, entry(OTHER));
> + write_entry(&mut map, 1, &other_answer(VNET), Elsewhere::Skip).unwrap();
> + assert_eq!(map[&1], entry(OTHER));
> + }
> +
> + #[test]
> + fn a_plug_writes_whatever_is_there() {
> + let mut map = BTreeMap::new();
> + write_entry(&mut map, 1, &entry(VNET), Elsewhere::Override).unwrap();
> + assert_eq!(map[&1], entry(VNET));
> + map.insert(1, entry(OTHER));
> + write_entry(&mut map, 1, &other_answer(VNET), Elsewhere::Override).unwrap();
> + assert_eq!(map[&1], other_answer(VNET));
> + }
> +
> + #[test]
> + fn an_unplug_removes_or_marks_the_entries() {
> + let mut map = BTreeMap::new();
> + map.insert(1, entry(VNET));
> + map.insert(2, entry(VNET));
> + map.insert(3, entry(VNET));
> + unplug_entries(&mut map, &[1, 2], None).unwrap();
> + assert_eq!(map.keys().copied().collect::<Vec<_>>(), [3]);
> + // a missing entry is removed already
> + unplug_entries(&mut map, &[1], None).unwrap();
> + // a move marks the interface that moved, and an older incarnation under its name is gone
> + map.insert(4, entry(VNET));
> + let marker = DhcpRecord::removed(NO_VNET);
> + unplug_entries(&mut map, &[3, 4], Some((4, marker))).unwrap();
> + assert_eq!(map.keys().copied().collect::<Vec<_>>(), [4]);
> + assert_eq!(map[&4].ip, 0);
> + assert_eq!(map[&4].vnet, NO_VNET);
> + // an interface without an entry gets no marker
> + unplug_entries(&mut map, &[5], Some((5, marker))).unwrap();
> + assert!(!map.contains_key(&5));
> + }
> +
> + fn iface(name: &str, vnet: Option<&str>, serve: bool, record: Option<Record>) -> Iface {
> + Iface {
> + name: name.into(),
> + vnet: vnet.map(String::from),
> + serve,
> + record,
> + }
> + }
> +
> + #[test]
> + fn the_pass_syncs_the_records_it_knows() {
> + let mut map = BTreeMap::new();
> + map.insert(1, entry(VNET)); // same answer as the config
> + map.insert(2, other_answer(VNET)); // a changed answer
> + map.insert(4, entry(VNET)); // not served anymore
> + map.insert(6, entry(VNET)); // its interface is gone
> + map.insert(7, entry(VNET)); // a live link the configs do not name
> + map.insert(8, entry(VNET)); // its mapping is gone
> + let ifaces = [
> + iface("a", Some("vnet1"), true, Some(record())),
> + iface("b", Some("vnet1"), true, Some(record())),
> + iface("d", Some("vnet1"), false, None),
> + iface("e", Some("vnet1"), true, Some(record())), // no entry yet
> + iface("f", Some("vnet1"), true, None), // no entry, served, no mapping
> + iface("g", Some("vnet1"), true, None),
> + ];
> + let present: HashMap<u32, &Iface> =
> + [1, 2, 4, 5, 9, 8].into_iter().zip(ifaces.iter()).collect();
> + let unknown_alive = HashSet::from([7]);
> + let (elsewhere, failed) = sync_records(&mut map, &present, &unknown_alive).unwrap();
> +
> + assert!(elsewhere.is_empty());
> + assert_eq!(failed, 0);
> + assert_eq!(map[&1], entry(VNET));
> + assert_eq!(map[&2], entry(VNET), "a changed answer is rewritten");
> + assert_eq!(
> + map[&4].ip, 0,
> + "an interface not served anymore keeps a marker"
> + );
> + assert_eq!(map[&5], entry(VNET), "a new interface gets its entry");
> + assert!(!map.contains_key(&6), "a gone interface loses its entry");
> + assert_eq!(
> + map[&9].ip, 0,
> + "a served interface without a mapping gets a marker"
> + );
> + assert_eq!(map[&8].ip, 0, "a mapping that is gone leaves a marker");
> + assert!(
> + map.contains_key(&7),
> + "a live link the configs do not name keeps its entry"
> + );
> + }
> +
> + #[test]
> + fn the_links_follow_the_configs_or_the_entry() {
> + let ifaces = [
> + iface("a", Some("vnet1"), true, Some(record())), // served, entry agrees
> + iface("b", Some("vnet1"), false, None), // not served
> + iface("c", Some("vnet1"), false, None), // config stale, entry has a record
> + iface("d", Some("vnet1"), true, Some(record())), // config stale, entry is a marker
> + ];
> + let present: HashMap<u32, &Iface> = (1..=4).zip(ifaces.iter()).collect();
> + let elsewhere = HashSet::from([3, 4]);
> + let unknown_alive = HashSet::from([9]);
> + assert_eq!(
> + wanted_links(&present, &elsewhere, &unknown_alive),
> + HashSet::from([1, 3, 4, 9])
> + );
> + }
> +
> + // a store that refuses one interface's writes
> + struct Flaky(BTreeMap<u32, DhcpRecord>, u32);
> +
> + impl RecordStore for Flaky {
> + fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
> + RecordStore::get(&self.0, ifindex)
> + }
> +
> + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
> + if ifindex == self.1 {
> + bail!("map full");
> + }
> + RecordStore::insert(&mut self.0, ifindex, rec)
> + }
> +
> + fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
> + RecordStore::mark(&mut self.0, ifindex, rec)
> + }
> +
> + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
> + if ifindex == self.1 {
> + bail!("map busy");
> + }
> + RecordStore::remove(&mut self.0, ifindex)
> + }
> +
> + fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
> + RecordStore::entries(&self.0)
> + }
> + }
> +
> + #[test]
> + fn bad_input_leaves_its_interface_alone() {
> + let mut map = BTreeMap::new();
> + map.insert(1, entry(VNET));
> + let bad = Record {
> + lease: 0,
> + ..record()
> + };
> + let ifaces = [
> + iface("a", Some("vnet1"), true, Some(bad)),
> + iface("b", Some("vnet1"), true, Some(record())),
> + ];
> + let present: HashMap<u32, &Iface> = (1..=2).zip(ifaces.iter()).collect();
> + let (elsewhere, failed) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
> + assert!(elsewhere.is_empty());
> + assert_eq!(failed, 0);
> + assert_eq!(map[&1], entry(VNET), "the old entry stays");
> + assert_eq!(map[&2], entry(VNET), "the rest is written");
> + }
> +
> + #[test]
> + fn a_failed_write_does_not_stop_the_pass() {
> + let mut map = Flaky(BTreeMap::new(), 2);
> + let ifaces = [
> + iface("a", Some("vnet1"), true, Some(record())),
> + iface("b", Some("vnet1"), true, Some(record())),
> + iface("c", Some("vnet1"), true, Some(record())),
> + ];
> + let present: HashMap<u32, &Iface> = (1..=3).zip(ifaces.iter()).collect();
> + let (elsewhere, failed) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
> + assert!(elsewhere.is_empty());
> + assert_eq!(failed, 1);
> + assert!(map.0.contains_key(&1));
> + assert!(!map.0.contains_key(&2));
> + assert!(map.0.contains_key(&3));
> + }
> +
> + // a store whose listing fails
> + struct Unlistable(BTreeMap<u32, DhcpRecord>);
> +
> + impl RecordStore for Unlistable {
> + fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
> + RecordStore::get(&self.0, ifindex)
> + }
> +
> + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
> + RecordStore::insert(&mut self.0, ifindex, rec)
> + }
> +
> + fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
> + RecordStore::mark(&mut self.0, ifindex, rec)
> + }
> +
> + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
> + RecordStore::remove(&mut self.0, ifindex)
> + }
> +
> + fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
> + bail!("map gone")
> + }
> + }
> +
> + #[test]
> + fn a_failed_listing_fails_the_pass_before_it_writes() {
> + let mut map = Unlistable(BTreeMap::new());
> + map.0.insert(1, entry(OTHER));
> + let ifaces = [iface("a", Some("vnet1"), true, Some(record()))];
> + let present: HashMap<u32, &Iface> = [(1, &ifaces[0])].into_iter().collect();
> + assert!(sync_records(&mut map, &present, &HashSet::new()).is_err());
> + assert_eq!(map.0[&1].vnet, OTHER);
> + }
> +
> + #[test]
> + fn a_failed_removal_counts_like_a_failed_write() {
> + let mut map = Flaky(BTreeMap::new(), 2);
> + map.0.insert(2, entry(VNET));
> + let ifaces = [iface("a", Some("vnet1"), true, Some(record()))];
> + let present: HashMap<u32, &Iface> = [(1, &ifaces[0])].into_iter().collect();
> + let (_, failed) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
> + assert_eq!(failed, 1);
> + assert!(map.0.contains_key(&1));
> + assert!(map.0.contains_key(&2));
> + }
> +
> + #[test]
> + fn the_pass_leaves_entries_of_another_vnet_to_their_link() {
> + let mut map = BTreeMap::new();
> + map.insert(1, entry(OTHER)); // a plug the configs trail, with a record
> + map.insert(2, DhcpRecord::removed(NO_VNET)); // moved to a plain bridge
> + map.insert(3, DhcpRecord::removed(NO_VNET)); // the configs caught up with that
> + let ifaces = [
> + iface("a", Some("vnet1"), true, Some(record())),
> + iface("b", Some("vnet1"), true, Some(record())),
> + iface("c", None, false, None),
> + ];
> + let present: HashMap<u32, &Iface> = (1..=3).zip(ifaces.iter()).collect();
> + let (elsewhere, _) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
> +
> + assert_eq!(elsewhere, HashSet::from([1, 2]));
> +
> + assert_eq!(map[&1].vnet, OTHER);
> + assert_eq!(map[&2].vnet, NO_VNET);
> + assert_eq!(
> + map[&3].ip, 0,
> + "a marker the configs agree with stays as it is"
> + );
> + }
> +}
> diff --git a/src/dhcp/types.rs b/src/dhcp/types.rs
> new file mode 100644
> index 0000000..71a1b94
> --- /dev/null
> +++ b/src/dhcp/types.rs
> @@ -0,0 +1,64 @@
> +//! Keep in sync with 'bpf/types.h'.
> +
> +/// Addresses, lease and mtu are stored in network byte order, so a reply copies them into the
> +/// packet as-is.
> +#[repr(C)]
> +#[derive(Copy, Clone, Debug, PartialEq, Eq)]
> +pub struct DhcpRecord {
> + /// 0 = removed, the entry is a marker naming its vnet
> + pub ip: u32,
> + pub netmask: u32,
> + /// 0 = not served
> + pub router: u32,
> + /// 0 = not served
> + pub dns: u32,
> + pub server_id: u32,
> + pub lease: u32,
> + /// 0 = not served
> + pub mtu: u16,
> + /// The vnet the writer saw the interface on, the id padded with zeros, all zero for none.
> + /// Never read by the program.
> + pub vnet: [u8; 8],
> + /// The NIC's MAC, a request carrying another is passed on. Zero in a marker.
> + pub mac: [u8; 6],
> +}
> +
> +impl DhcpRecord {
> + /// The marker a removal leaves behind, on the vnet the writer saw the interface on.
> + pub fn removed(vnet: [u8; 8]) -> Self {
> + Self {
> + ip: 0,
> + netmask: 0,
> + router: 0,
> + dns: 0,
> + server_id: 0,
> + lease: 0,
> + mtu: 0,
> + vnet,
> + mac: [0; 6],
> + }
> + }
> +}
> +
> +unsafe impl aya::Pod for DhcpRecord {}
> +
> +#[cfg(test)]
> +mod layout {
> + use core::mem::{offset_of, size_of};
> +
> + use super::*;
> +
> + #[test]
> + fn matches_bpf_abi() {
> + assert_eq!(size_of::<DhcpRecord>(), 40);
> + assert_eq!(offset_of!(DhcpRecord, ip), 0);
> + assert_eq!(offset_of!(DhcpRecord, netmask), 4);
> + assert_eq!(offset_of!(DhcpRecord, router), 8);
> + assert_eq!(offset_of!(DhcpRecord, dns), 12);
> + assert_eq!(offset_of!(DhcpRecord, server_id), 16);
> + assert_eq!(offset_of!(DhcpRecord, lease), 20);
> + assert_eq!(offset_of!(DhcpRecord, mtu), 24);
> + assert_eq!(offset_of!(DhcpRecord, vnet), 26);
> + assert_eq!(offset_of!(DhcpRecord, mac), 34);
> + }
> +}
> diff --git a/src/lib.rs b/src/lib.rs
> index 572d861..a8c94ba 100644
> --- a/src/lib.rs
> +++ b/src/lib.rs
> @@ -1,5 +1,8 @@
> //! eBPF subsystems, one per concern, each behind a cargo feature of its name. Nothing here runs
> //! on its own, consumers pull in only the subsystem they drive.
>
> +#[cfg(feature = "dhcp")]
> +pub mod dhcp;
> +
> pub mod subsystem;
> pub mod tc;
> diff --git a/src/subsystem.rs b/src/subsystem.rs
> index 3bf1634..6014dbe 100644
> --- a/src/subsystem.rs
> +++ b/src/subsystem.rs
> @@ -42,7 +42,7 @@ const RUN_ROOT: &str = "/run/proxmox-ebpf";
>
> /// The subsystems in the order their programs run on an interface, the first runs first. A name
> /// missing from the list fails the build.
> -pub const RUN_ORDER: &[&str] = &[];
> +pub const RUN_ORDER: &[&str] = &["dhcp"];
>
> const fn rank_of(name: &str) -> usize {
> let mut i = 0;
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-network v3 11/17] sdn: dhcp: push IP mapping edits to the guest's node
2026-10-20 7:59 ` [PATCH pve-network v3 11/17] sdn: dhcp: push IP mapping edits to the guest's node Hannes Laimer
@ 2026-10-20 13:09 ` Stefan Hanreich
2026-10-21 6:44 ` Hannes Laimer
0 siblings, 1 reply; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:09 UTC (permalink / raw)
To: pve-devel
On 10/20/26 10:02 AM, Hannes Laimer wrote:
> IP mappings live in the cluster-wide IPAM and can be edited through
> the API on any node. The DHCP backends hold their copy on each node,
> dnsmasq in a reservation file and the ebpf responder in a kernel map.
> An edit only updated the backend of the node it was made on, so a
> guest on another node kept getting the old mapping.
>
> The new endpoint POST /nodes/{node}/sdn/dhcp-mapping updates the
> mapping of one MAC on that node from the current records. An edit
> made on a node other than the guest's calls it on the guest's node.
> Only that node answers the guest's DHCP requests, so no other node is
> called. The others catch up when the guest starts on one of them.
>
> The call runs in the background, so the request returns without
> waiting on the other node. A call that fails logs its error to the
> journal of the node the edit was made on. One that gets no answer is
> stopped after a minute. The record stays as written either way, and
> the guest's node picks it up at the latest when the guest next starts
> there.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> src/PVE/API2/Network/SDN/Ips.pm | 3 +
> src/PVE/API2/Network/SDN/Nodes/Status.pm | 37 ++++++++++
> src/PVE/Network/SDN/Dhcp.pm | 74 +++++++++++++++++++
> src/test/run_test_vnets_blackbox.pl | 90 ++++++++++++++++++++++++
> 4 files changed, 204 insertions(+)
>
> diff --git a/src/PVE/API2/Network/SDN/Ips.pm b/src/PVE/API2/Network/SDN/Ips.pm
> index d7b682dd..5b45de7c 100644
> --- a/src/PVE/API2/Network/SDN/Ips.pm
> +++ b/src/PVE/API2/Network/SDN/Ips.pm
> @@ -47,6 +47,7 @@ __PACKAGE__->register_method({
> die "$@\n" if $@;
>
> PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
> + PVE::Network::SDN::Dhcp::notify_guest_node($vnet, $mac);
>
> return undef;
> },
> @@ -85,6 +86,7 @@ __PACKAGE__->register_method({
> PVE::Network::SDN::Vnets::add_ip($vnet, $ip, '', $mac, undef);
>
> PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
> + PVE::Network::SDN::Dhcp::notify_guest_node($vnet, $mac);
>
> return undef;
> },
> @@ -138,6 +140,7 @@ __PACKAGE__->register_method({
> die "$error\n" if $error;
>
> PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
> + PVE::Network::SDN::Dhcp::notify_guest_node($vnet, $mac);
>
> return undef;
> },
> diff --git a/src/PVE/API2/Network/SDN/Nodes/Status.pm b/src/PVE/API2/Network/SDN/Nodes/Status.pm
> index 7977e0c8..fe750a85 100644
> --- a/src/PVE/API2/Network/SDN/Nodes/Status.pm
> +++ b/src/PVE/API2/Network/SDN/Nodes/Status.pm
> @@ -9,6 +9,9 @@ use PVE::API2::Network::SDN::Nodes::Vnets;
>
> use PVE::JSONSchema qw(get_standard_option);
>
> +use PVE::Network::SDN::Vnets;
> +use PVE::Network::SDN::Dhcp;
> +
> use PVE::RESTHandler;
> use base qw(PVE::RESTHandler);
>
> @@ -27,6 +30,40 @@ __PACKAGE__->register_method({
> path => 'vnets',
> });
>
> +__PACKAGE__->register_method({
> + name => 'dhcp_mapping',
> + path => 'dhcp-mapping',
> + method => 'POST',
> + description =>
> + 'Re-apply the DHCP mapping of a MAC address on this node from the current records.',
> + permissions => {
> + check => ['perm', '/sdn/zones/{zone}/{vnet}', ['SDN.Allocate']],
> + },
> + protected => 1,
> + proxyto => 'node',
> + parameters => {
> + additionalProperties => 0,
> + properties => {
> + node => get_standard_option('pve-node'),
> + zone => get_standard_option('pve-sdn-zone-id'),
> + vnet => get_standard_option('pve-sdn-vnet-id'),
> + mac => get_standard_option('mac-addr'),
> + },
> + },
> + returns => { type => 'null' },
> + code => sub {
> + my ($param) = @_;
> +
> + my $vnet = PVE::Network::SDN::Vnets::get_vnet($param->{vnet}, 1);
> + die "vnet '$param->{vnet}' does not exist in zone '$param->{zone}'\n"
> + if !$vnet || $vnet->{zone} ne $param->{zone};
> +
> + PVE::Network::SDN::Dhcp::update_mapping($param->{vnet}, $param->{mac});
> +
> + return undef;
> + },
> +});
> +
> __PACKAGE__->register_method({
> name => 'sdnindex',
> path => '',
> diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
> index cf805a03..1f43745f 100644
> --- a/src/PVE/Network/SDN/Dhcp.pm
> +++ b/src/PVE/Network/SDN/Dhcp.pm
> @@ -3,6 +3,8 @@ package PVE::Network::SDN::Dhcp;
> use strict;
> use warnings;
>
> +use POSIX qw();
> +
> use PVE::Cluster;
>
> use PVE::Network::SDN;
> @@ -86,6 +88,78 @@ sub guest_nics {
> return $nics;
> }
>
> +sub guest_node_by_mac {
> + my ($vnetid, $mac) = @_;
> +
> + for my $nic (guest_nics()->@*) {
> + return $nic->{node}
> + if lc($nic->{mac} // '') eq lc($mac) && $nic->{bridge} eq $vnetid;
> + }
> +
> + return undef;
> +}
seems like we're always (or at least most of the time) querying by MAC
so would assembling a hash keyed by MAC addresses in the first place be sensible?
> +sub notify_guest_node {
> + my ($vnetid, $mac) = @_;
> +
> + return if !$mac;
> +
> + # best effort like the push itself, since the record is written by now
> + my ($zoneid, $node) = eval {
> + my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid, 1);
> + return if !$vnet;
> +
> + my $zone = PVE::Network::SDN::Zones::get_zone($vnet->{zone}, 1);
> + return if !$zone || !$zone->{ipam} || !$zone->{dhcp};
> +
> + return ($vnet->{zone}, guest_node_by_mac($vnetid, $mac));
> + };
> + if ($@) {
> + warn "could not find the node to push the dhcp mapping of $mac to: $@";
> + return;
> + }
> + return if !$node || $node eq PVE::INotify::nodename();
> +
> + # double fork, the grandchild is reparented to init so nothing on
> + # the request path ever waits on it
> + my $pid = fork();
> + if (!defined($pid)) {
> + warn "could not fork for the dhcp mapping push to $node: $!\n";
> + return;
> + }
> + if ($pid) {
> + waitpid($pid, 0);
> + return;
> + }
> +
> + POSIX::setsid();
> + my $child = fork();
> + POSIX::_exit(1) if !defined($child);
> + POSIX::_exit(0) if $child;
> +
> + # nothing of the request stays with the push, and since the daemon's
> + # stderr is closed a failed push reports through the journal
> + open(STDIN, '<', '/dev/null') or POSIX::_exit(1);
> + open(STDOUT, '>', '/dev/null') or POSIX::_exit(1);
> + open(STDERR, '|-', 'logger', '-t', 'pve-sdn', '-p', 'daemon.warning')
> + or POSIX::_exit(1);
> + # a node that takes the connection but never answers must not keep the
> + # push around
> + exec(
> + 'timeout',
> + '60',
> + 'pvesh',
> + 'create',
> + "/nodes/$node/sdn/dhcp-mapping",
> + '--zone',
> + $zoneid,
> + '--vnet',
> + $vnetid,
> + '--mac',
> + $mac,
> + ) or POSIX::_exit(1);
> +}
> +
There've been efforts of moving away from pvesh - see my other patch
series introducing the api client instead of calling pvesh here:
https://lore.proxmox.com/all/20260709140625.275618-1-s.hanreich@proxmox.com/
Might be the better option here as well?
Also, there is proxyto_callback which we could use to proxy to the correct
node in the first place?
> sub tap_plug {
> my ($bridge, $iface, $mac) = @_;
>
> diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
> index 783d0d47..9a4dbb9c 100755
> --- a/src/test/run_test_vnets_blackbox.pl
> +++ b/src/test/run_test_vnets_blackbox.pl
> @@ -49,6 +49,7 @@ sub clear_test_state {
> dnsmasq_calls => [],
> ebpf_calls => [],
> order => [],
> + notify_calls => [],
> warnings => [],
> ipam_config => {
> 'ids' => {
> @@ -238,6 +239,11 @@ $mocked_sdn_dhcp_dnsmasq->mock(
> update_lease => sub { },
> );
>
> +my $mocked_sdn_dhcp = Test::MockModule->new('PVE::Network::SDN::Dhcp');
> +$mocked_sdn_dhcp->mock(
> + notify_guest_node => sub { push $test_state->{notify_calls}->@*, [@_]; },
> +);
> +
> my $mocked_pve_rs_dhcp = Test::MockModule->new('PVE::RS::SDN::Dhcp');
> $mocked_pve_rs_dhcp->mock(
> detach => sub {
> @@ -1280,6 +1286,90 @@ sub test_dhcp_backend_needed_per_node {
>
> run_test(\&test_dhcp_backend_needed_per_node);
>
> +sub test_mapping_push_target {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> + my $zoneid = "TESTZONE";
> + my $vnetid = "testvnet";
> + my $mac = "da:65:8f:18:9b:6f";
> +
> + create_zone({
> + type => "simple",
> + dhcp => "dnsmasq",
> + ipam => "pve",
> + zone => $zoneid,
> + });
> + create_vnet({
> + type => "vnet",
> + zone => $zoneid,
> + vnet => $vnetid,
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/24",
> + gateway => "10.0.0.1",
> + });
> +
> + # the guest behind a MAC is found by its NIC on the vnet, and the config
> + # writes the MAC in upper case
> + $test_state->{vmlist} = {
> + ids => {
> + 100 => { type => 'qemu', node => 'other' },
> + 101 => { type => 'lxc', node => 'third' },
> + },
> + };
> + $test_state->{guest_nets} = {
> + 1 => { net0 => "virtio=$mac,bridge=$vnetid" },
> + 100 => { net0 => "virtio=" . uc($mac) . ",bridge=$vnetid" },
> + 101 => { net0 => "name=eth0,bridge=vmbr0,hwaddr=00:11:22:33:44:55" },
> + };
> + is(
> + PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, $mac),
> + 'other',
> + "$test_name: the MAC resolves to the node of its guest, a guest the list lacks is skipped",
> + );
> + is(
> + PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, '00:11:22:33:44:55'),
> + undef,
> + "$test_name: a MAC on another bridge does not resolve",
> + );
> + is(
> + PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, '00:11:22:33:44:66'),
> + undef,
> + "$test_name: an unknown MAC does not resolve",
> + );
> +
> + # every mapping edit through the API pokes the guest's node once
> + $test_state->{notify_calls} = [];
> + create_ip({
> + zone => $zoneid,
> + vnet => $vnetid,
> + mac => $mac,
> + ip => "10.0.0.50",
> + });
> + update_ip({
> + zone => $zoneid,
> + vnet => $vnetid,
> + mac => $mac,
> + ip => "10.0.0.51",
> + });
> + delete_ip({
> + zone => $zoneid,
> + vnet => $vnetid,
> + mac => $mac,
> + ip => "10.0.0.51",
> + });
> + eq_or_diff(
> + $test_state->{notify_calls},
> + [[$vnetid, $mac], [$vnetid, $mac], [$vnetid, $mac]],
> + "$test_name: create, edit and delete each poke the guest's node",
> + );
> + delete $test_state->{vmlist};
> + delete $test_state->{guest_nets};
> +}
> +
> +run_test(\&test_mapping_push_target);
> +
> sub test_ipam_cache_misses {
> my $test_name = (split(/::/, (caller(0))[3]))[-1];
> my $zoneid = "TESTZONE";
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-network v3 06/17] sdn: ipam: key the MAC cache by lower-case MAC, stop caching misses
2026-10-20 7:59 ` [PATCH pve-network v3 06/17] sdn: ipam: key the MAC cache by lower-case MAC, stop caching misses Hannes Laimer
@ 2026-10-20 13:10 ` Stefan Hanreich
2026-10-21 6:17 ` Hannes Laimer
0 siblings, 1 reply; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:10 UTC (permalink / raw)
To: pve-devel
On 10/20/26 10:01 AM, Hannes Laimer wrote:
> Generated guest MACs are upper case, but the API accepts either
> spelling. The cache was keyed by the MAC as its writer spelled it. A
> deletion in the other spelling missed the entry, so the cache kept
> answering with the deleted address. Entries cached before this
> change keep their spelling. A lookup and a deletion still find them,
> and the next write for their MAC moves them to the lower-case key.
>
> A lookup that missed the cache stored the IPAM's empty answer, and
> any entry counted as a hit. So a record created later directly in the
> IPAM stayed hidden, and a guest start allocated another address. An
> empty answer is no longer stored, and an entry without an address
> counts as a miss.
>
> The lookup also wrote the whole cache back without the cluster lock
> the other writers take. A change made between its read and its write
> was lost. The lookup now takes the lock and reads the cache again
> before it stores its answer.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> src/PVE/Network/SDN/Ipams.pm | 61 +++++++++++---
> src/test/run_test_vnets_blackbox.pl | 118 ++++++++++++++++++++++++++++
> 2 files changed, 168 insertions(+), 11 deletions(-)
>
> diff --git a/src/PVE/Network/SDN/Ipams.pm b/src/PVE/Network/SDN/Ipams.pm
> index 179bdf76..d8051d8d 100644
> --- a/src/PVE/Network/SDN/Ipams.pm
> +++ b/src/PVE/Network/SDN/Ipams.pm
> @@ -47,14 +47,27 @@ sub write_macdb {
> cfs_write_file($macdb_filename, $data);
> }
>
> +# an entry under another spelling moves to the lower-case key
> +my sub fold_spellings {
> + my ($db, $mac) = @_;
> +
> + for my $key (grep { $_ ne $mac && lc($_) eq $mac } reverse sort keys $db->{macs}->%*) {
> + my $old = delete $db->{macs}->{$key};
> + $db->{macs}->{$mac}->{$_} //= $old->{$_} for grep { defined($old->{$_}) } qw(ip4 ip6);
> + }
> +}
> +
> +# the cache is keyed by the MAC in lower case, whatever spelling its writers use
> sub add_cache_mac_ip {
> my ($mac, $ip) = @_;
> + $mac = lc($mac);
>
> cfs_lock_file(
> $macdb_filename,
> undef,
> sub {
> my $db = read_macdb();
> + fold_spellings($db, $mac);
> if (Net::IP::ip_is_ipv4($ip)) {
> $db->{macs}->{$mac}->{ip4} = $ip;
> } else {
> @@ -74,13 +87,17 @@ sub del_cache_mac_ip {
> undef,
> sub {
> my $db = read_macdb();
> - if (Net::IP::ip_is_ipv4($ip)) {
> - delete $db->{macs}->{$mac}->{ip4};
> - } else {
> - delete $db->{macs}->{$mac}->{ip6};
> + # entries from before the cache was keyed in lower case carry their writer's spelling
> + for my $key (grep { lc($_) eq lc($mac) } keys $db->{macs}->%*) {
> + if (Net::IP::ip_is_ipv4($ip)) {
> + delete $db->{macs}->{$key}->{ip4};
> + } else {
> + delete $db->{macs}->{$key}->{ip6};
> + }
> + delete $db->{macs}->{$key}
> + if !defined($db->{macs}->{$key}->{ip4})
> + && !defined($db->{macs}->{$key}->{ip6});
> }
> - delete $db->{macs}->{$mac}
> - if !defined($db->{macs}->{$mac}->{ip4}) && !defined($db->{macs}->{$mac}->{ip6});
> write_macdb($db);
> },
> );
> @@ -136,16 +153,38 @@ sub get_ips_from_mac {
> my ($mac, $zoneid, $zone) = @_;
>
> my $macdb = read_macdb();
> - return ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6}) if $macdb->{macs}->{$mac};
> + # entries from before the cache was keyed in lower case carry their writer's
> + # spelling, and the lower-case one wins where two hold the same family
> + my %cached;
> + for my $key (grep { lc($_) eq lc($mac) } sort keys $macdb->{macs}->%*) {
> + my $entry = $macdb->{macs}->{$key};
> + $cached{$_} = $entry->{$_} for grep { defined($entry->{$_}) } qw(ip4 ip6);
> + }
> +
> + # an entry holding no address says nothing about the MAC, so ask the IPAM
> + return ($cached{ip4}, $cached{ip6}) if defined($cached{ip4}) || defined($cached{ip6});
>
> my $plugin_config = get_plugin_config($zone);
> my $plugin = PVE::Network::SDN::Ipams::Plugin->lookup($plugin_config->{type});
> - ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6}) =
> - $plugin->get_ips_from_mac($plugin_config, $mac, $zoneid);
> + my ($ip4, $ip6) = $plugin->get_ips_from_mac($plugin_config, $mac, $zoneid);
> +
> + # an empty answer is not cached, since the record may simply not exist yet
> + return if !defined($ip4) && !defined($ip6);
>
> - write_macdb($macdb);
> + cfs_lock_file(
> + $macdb_filename,
> + undef,
> + sub {
> + my $db = read_macdb();
> + fold_spellings($db, lc($mac));
> + $db->{macs}->{ lc($mac) }->{ip4} = $ip4 if defined($ip4);
> + $db->{macs}->{ lc($mac) }->{ip6} = $ip6 if defined($ip6);
> + write_macdb($db);
> + },
> + );
> + warn "$@" if $@;
>
> - return ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6});
> + return ($ip4, $ip6);
> }> 1;
would it make sense to move all of this case handling into read_macdb itself? Currently
we're doing it at every call site of read_macdb in Ipams.pm
> diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
> index d324538e..476e7f8d 100755
> --- a/src/test/run_test_vnets_blackbox.pl
> +++ b/src/test/run_test_vnets_blackbox.pl
> @@ -1118,6 +1118,124 @@ sub test_dnsmasq_mapping_push {
>
> run_test(\&test_dnsmasq_mapping_push);
>
> +sub test_ipam_cache_misses {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> + my $zoneid = "TESTZONE";
> + my $vnetid = "testvnet";
> + my $mac = "da:65:8f:18:9b:6f";
> +
> + create_zone({
> + type => "simple",
> + dhcp => "dnsmasq",
> + ipam => "pve",
> + zone => $zoneid,
> + });
> + create_vnet({
> + type => "vnet",
> + zone => $zoneid,
> + vnet => $vnetid,
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/24",
> + gateway => "10.0.0.1",
> + 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
> + });
> +
> + my @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
> + is(scalar(grep { defined } @ips), 0, "$test_name: an unknown MAC has no answer");
> + ok(!exists $test_state->{macdb}->{macs}->{$mac}, "$test_name: the miss is not cached");
> +
> + # an entry holding no address, left behind by an earlier miss, does not
> + # hide a record created since
> + create_ip({
> + zone => $zoneid,
> + vnet => $vnetid,
> + mac => $mac,
> + ip => "10.0.0.50",
> + });
> + $test_state->{macdb}->{macs}->{$mac} = { ip4 => undef, ip6 => undef };
> + @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
> + is($ips[0], "10.0.0.50", "$test_name: an address-less entry counts as a miss");
> + is(
> + $test_state->{macdb}->{macs}->{$mac}->{ip4},
> + "10.0.0.50",
> + "$test_name: the answer is cached",
> + );
> +}
> +
> +run_test(\&test_ipam_cache_misses);
> +
> +sub test_ipam_cache_case {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> + my $zoneid = "TESTZONE";
> + my $vnetid = "testvnet";
> + my $mac = "da:65:8f:18:9b:6f";
> +
> + # an entry written under the config's spelling before the cache was keyed in
> + # lower case is read and released like any other
> + create_zone({
> + type => "simple",
> + dhcp => "dnsmasq",
> + ipam => "pve",
> + zone => $zoneid,
> + });
> + create_vnet({
> + type => "vnet",
> + zone => $zoneid,
> + vnet => $vnetid,
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/24",
> + gateway => "10.0.0.1",
> + 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
> + });
> + $test_state->{macdb}->{macs}->{ uc($mac) } = { ip4 => '10.0.0.150' };
> + my ($ip4) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
> + is($ip4, '10.0.0.150', "$test_name: an old entry is found however it is spelled");
> + PVE::Network::SDN::Ipams::del_cache_mac_ip($mac, '10.0.0.150');
> + ok(
> + !(grep { lc($_) eq $mac } keys $test_state->{macdb}->{macs}->%*),
> + "$test_name: and released",
> + );
> +
> + # a write spelled either way moves such an entry to the lower-case key
> + # and keeps what it held
> + $test_state->{macdb}->{macs}->{ uc($mac) } = { ip4 => '10.0.0.150' };
> + PVE::Network::SDN::Ipams::add_cache_mac_ip(uc($mac), 'fd00::150');
> + is_deeply(
> + [grep { lc($_) eq $mac } keys $test_state->{macdb}->{macs}->%*],
> + [$mac],
> + "$test_name: a write leaves one entry for the MAC",
> + );
> + is_deeply(
> + $test_state->{macdb}->{macs}->{$mac},
> + { ip4 => '10.0.0.150', ip6 => 'fd00::150' },
> + "$test_name: holding both addresses",
> + );
> +
> + # the lookup's write moves an address-less entry under another spelling as well
> + create_ip({
> + zone => $zoneid,
> + vnet => $vnetid,
> + mac => $mac,
> + ip => "10.0.0.160",
> + });
> + $test_state->{macdb}->{macs} = { uc($mac) => { ip4 => undef, ip6 => undef } };
> + my @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
> + is($ips[0], '10.0.0.160', "$test_name: a lookup past an address-less entry asks the IPAM");
> + is_deeply(
> + [sort keys $test_state->{macdb}->{macs}->%*],
> + [$mac],
> + "$test_name: and its write leaves one entry for the MAC",
> + );
> +}
> +
> +run_test(\&test_ipam_cache_case);
> +
> sub test_dnsmasq_dual_stack_and_sweep {
> my $test_name = (split(/::/, (caller(0))[3]))[-1];
> my $zoneid = "TESTZONE";
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH proxmox-perl-rs v3 03/17] pve-rs: sdn: add bindings for the eBPF DHCP responder
2026-10-20 7:59 ` [PATCH proxmox-perl-rs v3 03/17] pve-rs: sdn: add bindings for the eBPF DHCP responder Hannes Laimer
@ 2026-10-20 13:10 ` Stefan Hanreich
2026-10-21 7:32 ` Hannes Laimer
0 siblings, 1 reply; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:10 UTC (permalink / raw)
To: pve-devel
On 10/20/26 10:00 AM, Hannes Laimer wrote:
> The responder answers with the records the caller hands in. The
> caller has to build them under the responder's lock, or a full pass
> could write older records over those a plug or record change built
> later. So the lock is an object the caller takes before it builds the
> records, and every call that hands in records is a method of that
> lock.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> pve-rs/Cargo.toml | 2 +
> pve-rs/Makefile | 3 +
> pve-rs/debian/control | 2 +
> pve-rs/src/bindings/sdn/dhcp.rs | 229 ++++++++++++++++++++++++++++++++
> pve-rs/src/bindings/sdn/mod.rs | 1 +
> 5 files changed, 237 insertions(+)
> create mode 100644 pve-rs/src/bindings/sdn/dhcp.rs
>
> diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
> index 5ae9082..0adf00b 100644
> --- a/pve-rs/Cargo.toml
> +++ b/pve-rs/Cargo.toml
> @@ -34,6 +34,7 @@ proxmox-apt = { version = "1.0.1", features = ["cache"] }
> proxmox-apt-api-types = "3"
> proxmox-base64 = "1"
> proxmox-config-digest = "1"
> +proxmox-ebpf = { version = "0.1", features = ["dhcp"] }
> proxmox-frr = { version = "0.5.1" }
> proxmox-http = { version = "1.0.2", features = ["client-sync", "client-trait"] }
> proxmox-http-error = "1"
> @@ -70,6 +71,7 @@ proxmox-wireguard = { version = "0.1.2" }
> # proxmox-config-digest = { path = "../../proxmox/proxmox-config-digest" }
> # proxmox-daemon = { path = "../../proxmox/proxmox-daemon" }
> # proxmox-dns-api = { path = "../../proxmox/proxmox-dns-api" }
> +# proxmox-ebpf = { path = "../../proxmox-ebpf" }
> # proxmox-http-error = { path = "../../proxmox/proxmox-http-error" }
> # proxmox-http = { path = "../../proxmox/proxmox-http" }
> # proxmox-human-byte = { path = "../../proxmox/proxmox-human-byte" }
> diff --git a/pve-rs/Makefile b/pve-rs/Makefile
> index bb1cd2d..ebe8bd3 100644
> --- a/pve-rs/Makefile
> +++ b/pve-rs/Makefile
> @@ -32,6 +32,9 @@ PERLMOD_PACKAGES := \
> PVE::RS::OpenId \
> PVE::RS::ResourceScheduling::Static \
> PVE::RS::ResourceScheduling::Dynamic \
> + PVE::RS::SDN::Dhcp \
> + PVE::RS::SDN::Dhcp::ExclusiveLock \
> + PVE::RS::SDN::Dhcp::SharedLock \
> PVE::RS::SDN::Fabrics \
> PVE::RS::SDN::PrefixLists \
> PVE::RS::SDN::RouteMaps \
> diff --git a/pve-rs/debian/control b/pve-rs/debian/control
> index 70367dc..521d966 100644
> --- a/pve-rs/debian/control
> +++ b/pve-rs/debian/control
> @@ -20,6 +20,8 @@ Build-Depends: debhelper-compat (= 13),
> librust-proxmox-apt-api-types-3+default-dev (>= 3.0.0-~~),
> librust-proxmox-base64-1+default-dev,
> librust-proxmox-config-digest-1+default-dev,
> + librust-proxmox-ebpf-0.1+default-dev,
> + librust-proxmox-ebpf-0.1+dhcp-dev,
> librust-proxmox-frr-0.5+default-dev (>= 0.5.1-~~),
> librust-proxmox-http-1+client-sync-dev (>= 1.0.2-~~),
> librust-proxmox-http-1+client-trait-dev (>= 1.0.2-~~),
> diff --git a/pve-rs/src/bindings/sdn/dhcp.rs b/pve-rs/src/bindings/sdn/dhcp.rs
> new file mode 100644
> index 0000000..a6cdc95
> --- /dev/null
> +++ b/pve-rs/src/bindings/sdn/dhcp.rs
> @@ -0,0 +1,229 @@
> +#[perlmod::package(name = "PVE::RS::SDN::Dhcp", lib = "pve_rs")]
> +pub mod pve_rs_sdn_dhcp {
> + //! The `PVE::RS::SDN::Dhcp` package.
> + //!
> + //! Bindings for the per-tap eBPF DHCP responder. Interfaces and the records they answer with
> + //! are handed in here and served directly from the kernel. The calls that change the responder
> + //! are methods of a lock object, so nothing writes to it without holding the lock in the mode
> + //! the change needs.
> +
> + use std::net::Ipv4Addr;
> +
> + use anyhow::Error;
> + use serde::{Deserialize, Deserializer};
> +
> + use proxmox_ebpf::dhcp::{DhcpSubsystem, Iface, Record, Unplugged};
> +
> + /// One interface's DHCP answer, the address and every option the reply carries.
> + #[derive(Deserialize)]
> + pub struct DhcpRecord {
> + ip: Ipv4Addr,
> + prefixlen: u8,
> + server_id: Ipv4Addr,
> + lease: u32,
> + #[serde(default)]
> + router: Option<Ipv4Addr>,
> + #[serde(default)]
> + dns: Option<Ipv4Addr>,
> + #[serde(default)]
> + mtu: Option<u16>,
> + #[serde(deserialize_with = "mac_from_str")]
> + mac: [u8; 6],
proxmox-network-types has a MAC address type that would fit here if we add an as_slice() method
> + }
> +
> + // a MAC as six hex pairs
> + fn mac_from_str<'de, D: Deserializer<'de>>(d: D) -> Result<[u8; 6], D::Error> {
> + let text = String::deserialize(d)?;
> + let mut mac = [0u8; 6];
> + let mut parts = text.split(':');
> + for byte in &mut mac {
> + let part = parts
> + .next()
> + .ok_or_else(|| serde::de::Error::custom(format!("MAC {text} is too short")))?;
> + if part.len() != 2 || !part.bytes().all(|b| b.is_ascii_hexdigit()) {
> + return Err(serde::de::Error::custom(format!("MAC {text} is malformed")));
> + }
> + *byte = u8::from_str_radix(part, 16)
> + .map_err(|_| serde::de::Error::custom(format!("MAC {text} is malformed")))?;
> + }
> + if parts.next().is_some() {
> + return Err(serde::de::Error::custom(format!("MAC {text} is too long")));
> + }
> + Ok(mac)
> + }
would become obsolete if using the MacAddr type
> + /// One guest interface, the vnet it sits on if any, whether the responder runs on it and what
> + /// it answers there.
> + #[derive(Deserialize)]
> + pub struct DhcpIface {
> + name: String,
> + #[serde(default)]
> + vnet: Option<String>,
> + serve: bool,
> + #[serde(default)]
> + record: Option<DhcpRecord>,
> + }
> +
> + /// Where an unplugged interface went, the vnet it sits on now, none for a plain bridge.
> + #[derive(Deserialize)]
> + pub struct DhcpTarget {
> + #[serde(default)]
> + vnet: Option<String>,
> + }
> +
> + impl From<DhcpRecord> for Record {
> + fn from(r: DhcpRecord) -> Self {
> + Record {
> + ip: r.ip,
> + prefixlen: r.prefixlen,
> + server_id: r.server_id,
> + lease: r.lease,
> + router: r.router,
> + dns: r.dns,
> + mtu: r.mtu,
> + mac: r.mac,
> + }
> + }
> + }
> +
> + impl From<DhcpIface> for Iface {
> + fn from(i: DhcpIface) -> Self {
> + Iface {
> + name: i.name,
> + vnet: i.vnet,
> + serve: i.serve,
> + record: i.record.map(Record::from),
> + }
> + }
> + }
> +
> + pub(super) fn to_ifaces(ifaces: Vec<DhcpIface>) -> Vec<Iface> {
> + ifaces.into_iter().map(Iface::from).collect()
> + }
> +
> + /// A tap unplug. Without a target the interface is gone or going and takes its record with
> + /// it. With one it was plugged onto a bridge the responder does not serve and keeps a marker
> + /// naming that place.
> + #[export]
> + pub fn detach(iface: &str, target: Option<DhcpTarget>) -> Result<(), Error> {
> + let unplugged = match target {
> + None => Unplugged::Gone,
> + Some(target) => Unplugged::MovedTo(target.vnet),
> + };
> + DhcpSubsystem::new().detach(iface, unplugged)
> + }
> +}
> +
> +#[perlmod::package(name = "PVE::RS::SDN::Dhcp::SharedLock", lib = "pve_rs")]
> +pub mod pve_rs_sdn_dhcp_shared_lock {
> + //! The `PVE::RS::SDN::Dhcp::SharedLock` package.
> + //!
> + //! The responder's lock taken shared, what a tap plug and a guest's own record change hold.
> + //! Released when the object goes away.
> +
> + use anyhow::Error;
> + use perlmod::Value;
> +
> + use proxmox_ebpf::dhcp::{DhcpSubsystem, Iface, Outcome, Record, SharedLock};
> +
> + use super::pve_rs_sdn_dhcp::{DhcpIface, DhcpRecord, to_ifaces};
> +
> + /// The lock as Perl holds it.
> + pub struct PerlSharedLock {
> + lock: SharedLock,
> + }
> +
> + perlmod::declare_magic!(Box<PerlSharedLock> : &PerlSharedLock as "PVE::RS::SDN::Dhcp::SharedLock");
> +
> + /// Class method: take the lock. Plugs hold it next to each other and wait only behind a full
> + /// pass or a record change.
> + #[export]
> + pub fn new(#[raw] class: Value) -> Result<Value, Error> {
> + let lock = DhcpSubsystem::new().lock_shared()?;
> + Ok(perlmod::instantiate_magic!(&class, MAGIC => Box::new(PerlSharedLock { lock })))
> + }
> +
> + /// Method: a tap plug onto a vnet with the record it answers. Without a record the interface
> + /// gets no answer. Returns false when nothing is loaded yet. An interface gone already counts
> + /// as done.
> + #[export]
> + pub fn attach(
> + #[try_from_ref] this: &PerlSharedLock,
> + iface: String,
> + vnet: String,
> + record: Option<DhcpRecord>,
> + ) -> Result<bool, Error> {
> + let iface = Iface {
> + name: iface,
> + vnet: Some(vnet),
> + serve: true,
> + record: record.map(Record::from),
> + };
> + Ok(this.lock.attach(&iface)? == Outcome::Done)
> + }
> +
> + /// Method: a record change for the listed interfaces by a caller holding the guest lock.
> + /// Returns false when nothing is loaded to write into, so a full pass is due.
> + #[export]
> + pub fn update(
> + #[try_from_ref] this: &PerlSharedLock,
> + ifaces: Vec<DhcpIface>,
> + ) -> Result<bool, Error> {
> + Ok(this.lock.update(&to_ifaces(ifaces))? == Outcome::Done)
> + }
> +}
> +
> +#[perlmod::package(name = "PVE::RS::SDN::Dhcp::ExclusiveLock", lib = "pve_rs")]
> +pub mod pve_rs_sdn_dhcp_exclusive_lock {
> + //! The `PVE::RS::SDN::Dhcp::ExclusiveLock` package.
> + //!
> + //! The responder's lock taken exclusively, what a full pass and a record change hold. Released
> + //! when the object goes away.
> +
> + use anyhow::Error;
> + use perlmod::Value;
> +
> + use proxmox_ebpf::dhcp::{DhcpSubsystem, ExclusiveLock, Outcome};
> +
> + use super::pve_rs_sdn_dhcp::{DhcpIface, to_ifaces};
> +
> + /// The lock as Perl holds it.
> + pub struct PerlExclusiveLock {
> + lock: ExclusiveLock,
> + }
> +
> + perlmod::declare_magic!(Box<PerlExclusiveLock> : &PerlExclusiveLock as "PVE::RS::SDN::Dhcp::ExclusiveLock");
> +
> + /// Class method: take the lock, waiting out every plug in flight.
> + #[export]
> + pub fn new(#[raw] class: Value) -> Result<Value, Error> {
> + let lock = DhcpSubsystem::new().lock_exclusive()?;
> + Ok(perlmod::instantiate_magic!(&class, MAGIC => Box::new(PerlExclusiveLock { lock })))
> + }
> +
> + /// Method: the full pass over every guest interface of the cluster. Those found here get
> + /// their entry and a link where served.
> + #[export]
> + pub fn apply(
> + #[try_from_ref] this: &PerlExclusiveLock,
> + ifaces: Vec<DhcpIface>,
> + ) -> Result<(), Error> {
> + this.lock.apply(&to_ifaces(ifaces))
> + }
> +
> + /// Method: a record change for the listed interfaces. Those not found here are skipped.
> + /// Returns false when nothing is loaded to write into, so a full pass is due.
> + #[export]
> + pub fn update(
> + #[try_from_ref] this: &PerlExclusiveLock,
> + ifaces: Vec<DhcpIface>,
> + ) -> Result<bool, Error> {
> + Ok(this.lock.update(&to_ifaces(ifaces))? == Outcome::Done)
> + }
> +
> + /// Method: detach the responder everywhere and drop its pinned state.
> + #[export]
> + pub fn clear(#[try_from_ref] this: &PerlExclusiveLock) -> Result<(), Error> {
> + this.lock.clear()
> + }
> +}
> diff --git a/pve-rs/src/bindings/sdn/mod.rs b/pve-rs/src/bindings/sdn/mod.rs
> index dcae046..4b99b8f 100644
> --- a/pve-rs/src/bindings/sdn/mod.rs
> +++ b/pve-rs/src/bindings/sdn/mod.rs
> @@ -1,3 +1,4 @@
> +pub(crate) mod dhcp;
> pub(crate) mod fabrics;
> pub(crate) mod prefix_lists;
> pub(crate) mod route_maps;
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-network v3 09/17] sdn: dhcp: add an eBPF backend answering on each guest interface
2026-10-20 7:59 ` [PATCH pve-network v3 09/17] sdn: dhcp: add an eBPF backend answering on each guest interface Hannes Laimer
@ 2026-10-20 13:10 ` Stefan Hanreich
2026-10-21 6:16 ` Hannes Laimer
0 siblings, 1 reply; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:10 UTC (permalink / raw)
To: pve-devel
On 10/20/26 10:04 AM, Hannes Laimer wrote:
> The ebpf DHCP backend answers DHCPv4 from a program in the kernel on
> every guest interface of its zones, with the address the SDN MAC cache
> holds for the interface's MAC. No server runs per zone, and IPv6
> subnets are not served. A subnet without a gateway is served under a
> link-local server identifier no host owns, so its guests renew by
> broadcast at their rebinding time.
>
> An apply runs a full pass on each node, and on a node that no zone
> with the ebpf backend includes, the pass removes the responder
> altogether. A record change rewrites the answers for its MAC on the
> node the change is made on, and the other nodes catch up at their
> next apply.
>
> The responder is pinned in bpffs and does not survive a reboot, so a
> boot hook runs a full pass. Removing the package deletes those pins,
> since nothing would detach the responder afterward.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> debian/libpve-network-perl.prerm | 13 +
> src/PVE/API2/Network/SDN/Zones.pm | 3 +-
> src/PVE/Network/SDN/Dhcp.pm | 79 ++++
> src/PVE/Network/SDN/Dhcp/Ebpf.pm | 308 +++++++++++++
> src/PVE/Network/SDN/Dhcp/Makefile | 2 +-
> src/PVE/Network/SDN/Dhcp/Plugin.pm | 19 +
> src/PVE/Network/SDN/SubnetPlugin.pm | 4 +-
> src/test/run_test_vnets_blackbox.pl | 642 ++++++++++++++++++++++++++++
> 8 files changed, 1066 insertions(+), 4 deletions(-)
> create mode 100755 debian/libpve-network-perl.prerm
> create mode 100644 src/PVE/Network/SDN/Dhcp/Ebpf.pm
>
> diff --git a/debian/libpve-network-perl.prerm b/debian/libpve-network-perl.prerm
> new file mode 100755
> index 00000000..d28c74fe
> --- /dev/null
> +++ b/debian/libpve-network-perl.prerm
> @@ -0,0 +1,13 @@
> +#!/bin/sh
> +
> +set -e
> +
> +case "$1" in
> + remove)
> + # the eBPF DHCP responder stays attached through its pins in bpffs, and with
> + # the package gone nothing would ever update or detach it
> + rm -rf /sys/fs/bpf/proxmox-ebpf/dhcp
> + ;;
> +esac
> +
> +exit 0
> diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm
> index b897cbdf..0e907260 100644
> --- a/src/PVE/API2/Network/SDN/Zones.pm
> +++ b/src/PVE/API2/Network/SDN/Zones.pm
> @@ -16,6 +16,7 @@ use PVE::Network::SDN::Dns;
> use PVE::Network::SDN::Subnets;
> use PVE::Network::SDN::Vnets;
> use PVE::Network::SDN;
> +use PVE::Network::SDN::Dhcp;
>
> use PVE::Network::SDN::Zones::EvpnPlugin;
> use PVE::Network::SDN::Zones::FaucetPlugin;
> @@ -90,7 +91,7 @@ my $ZONE_PROPERTIES = {
> },
> dhcp => {
> type => 'string',
> - enum => ['dnsmasq'],
> + enum => PVE::Network::SDN::Dhcp->plugin_types(),
> optional => 1,
> description => 'Name of DHCP server backend for this zone.',
> },
> diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
> index 2a023534..cf805a03 100644
> --- a/src/PVE/Network/SDN/Dhcp.pm
> +++ b/src/PVE/Network/SDN/Dhcp.pm
> @@ -9,6 +9,7 @@ use PVE::Network::SDN;
> use PVE::Network::SDN::Subnets;
> use PVE::Network::SDN::Dhcp::Plugin;
> use PVE::Network::SDN::Dhcp::Dnsmasq;
> +use PVE::Network::SDN::Dhcp::Ebpf;
>
> use PVE::INotify;
>
> @@ -17,6 +18,9 @@ PVE::Network::SDN::Dhcp::Plugin->init();
> PVE::Network::SDN::Dhcp::Dnsmasq->register();
> PVE::Network::SDN::Dhcp::Dnsmasq->init();
>
> +PVE::Network::SDN::Dhcp::Ebpf->register();
> +PVE::Network::SDN::Dhcp::Ebpf->init();
> +
> sub plugin_types {
> return PVE::Network::SDN::Dhcp::Plugin->lookup_types();
> }
> @@ -49,6 +53,81 @@ sub update_mapping {
> warn "could not update dhcp mapping for $mac: $@" if $@;
> }
>
> +# the NIC lines come from pmxcfs in one go, so no guest config gets parsed for
> +# this. With a vmid only that guest's lines are fetched
> +sub guest_nics {
> + my ($vmid) = @_;
> +
> + # a hook script starts with the cluster cache empty
> + PVE::Cluster::cfs_update();
> + my $vmlist = PVE::Cluster::get_vmlist();
> + my $nets = PVE::Cluster::get_guest_config_properties([map { "net$_" } 0 .. 31], $vmid);
> + my $nics = [];
> + for my $vmid (sort keys %$nets) {
> + my $guest = $vmlist->{ids}->{$vmid} // next;
> + my $prefix = ($guest->{type} // '') eq 'lxc' ? 'veth' : 'tap';
> + for my $key (sort keys %{ $nets->{$vmid} }) {
> + my ($index) = $key =~ m/^net(\d+)$/ or next;
> + my $net = $nets->{$vmid}->{$key};
> + my ($bridge) = $net =~ m/(?:^|,)bridge=([^,]+)/ or next;
> + # a VM names the MAC as the value of its model, a container as hwaddr
> + my ($mac) = $net =~ m/(?:^|,)[a-z0-9_-]+=([0-9a-f]{2}(?::[0-9a-f]{2}){5})(?:,|$)/i;
Is there a reason for not using the pre-existing parse_net /
parse_lxc_network functions in qemu-server / pve-container here?
They'd provide a standalone way of parsing the property string only,
without re-implementing the logic here.
Circular dependencies potentially?
> + push @$nics,
> + {
> + vmid => $vmid,
> + node => $guest->{node},
> + iface => "$prefix${vmid}i$index",
> + bridge => $bridge,
> + mac => $mac,
> + };
> + }
> + }
> +
> + return $nics;
> +}
> +
> +sub tap_plug {
> + my ($bridge, $iface, $mac) = @_;
> +
> + my $vnet = PVE::Network::SDN::Vnets::get_vnet($bridge, 1);
> + my $zone = $vnet ? PVE::Network::SDN::Zones::get_zone($vnet->{zone}, 1) : undef;
> + my $backend = $zone ? $zone->{dhcp} : undef;
> +
> + # every other backend hears where the interface sits now. A plain bridge
> + # is no vnet
> + my $target = { vnet => $vnet ? $bridge : undef };
> + for my $type (PVE::Network::SDN::Dhcp::Plugin->lookup_types()->@*) {
> + next if defined($backend) && $type eq $backend;
> + eval { PVE::Network::SDN::Dhcp::Plugin->lookup($type)->tap_unplug($iface, $target) };
> + warn "could not unplug $iface from the $type dhcp backend: $@" if $@;
> + }
> + return if !$backend;
> +
> + eval {
> + PVE::Network::SDN::Dhcp::Plugin->lookup($backend)
> + ->tap_plug($vnet->{zone}, $bridge, $iface, $mac);
> + };
> + warn "could not plug $iface into the dhcp backend of zone $vnet->{zone}: $@" if $@;
> +}
> +
> +sub boot {
> + for my $type (PVE::Network::SDN::Dhcp::Plugin->lookup_types()->@*) {
> + eval { PVE::Network::SDN::Dhcp::Plugin->lookup($type)->boot() };
> + warn "could not bring up the $type dhcp backend: $@" if $@;
> + }
> +}
> +
> +# the interface is usually gone already and with it any way to its zone, so
> +# every backend is told
> +sub tap_unplug {
> + my ($iface) = @_;
> +
> + for my $type (PVE::Network::SDN::Dhcp::Plugin->lookup_types()->@*) {
> + eval { PVE::Network::SDN::Dhcp::Plugin->lookup($type)->tap_unplug($iface) };
> + warn "could not unplug $iface from the $type dhcp backend: $@" if $@;
> + }
> +}
> +
> sub regenerate_config {
> my ($reload) = @_;
>
> diff --git a/src/PVE/Network/SDN/Dhcp/Ebpf.pm b/src/PVE/Network/SDN/Dhcp/Ebpf.pm
> new file mode 100644
> index 00000000..fc4af1e0
> --- /dev/null
> +++ b/src/PVE/Network/SDN/Dhcp/Ebpf.pm
> @@ -0,0 +1,308 @@
> +package PVE::Network::SDN::Dhcp::Ebpf;
> +
> +use strict;
> +use warnings;
> +
> +use base qw(PVE::Network::SDN::Dhcp::Plugin);
> +
> +use Net::IP;
> +use Net::Subnet qw(subnet_matcher);
> +
> +use PVE::Cluster;
> +use PVE::INotify;
> +use PVE::Network::SDN::Ipams;
> +use PVE::RESTEnvironment qw(log_warn);
> +
> +use PVE::RS::SDN::Dhcp;
> +use PVE::RS::SDN::Dhcp::ExclusiveLock;
> +use PVE::RS::SDN::Dhcp::SharedLock;
> +
> +my $DEFAULT_LEASE_TIME = 600;
> +
> +# a subnet without a gateway is served under a link-local server identifier,
> +# which no host on the subnet owns. A guest cannot renew by unicast then and
> +# broadcasts at its rebinding time instead
> +my $SERVER_ID_NO_GATEWAY = '169.254.0.1';
> +
> +sub type {
> + return 'ebpf';
> +}
> +
> +# the MTU option is 16 bits wide with 68 the smallest MTU IPv4 allows, so a
> +# zone MTU outside that is reported on an apply and not handed out. A zone
> +# without one hands out none and the guest keeps its own setting
> +my sub zone_mtu {
> + my ($zoneid, $zone, $report) = @_;
> +
> + my $mtu = PVE::Network::SDN::Zones::Plugin->lookup($zone->{type})->get_mtu($zone);
> + return undef if !defined($mtu);
> + return $mtu if $mtu >= 68 && $mtu <= 65535;
> + log_warn("zone $zoneid has MTU $mtu, not handing it out over DHCP") if $report;
> + return undef;
> +}
> +
> +my sub dhcp_record {
> + my ($ip4, $subnet, $mtu, $mac) = @_;
> +
> + my $gateway = $subnet->{gateway};
> +
> + # a DNS server has to be reachable over the address family served
> + my $dns = $subnet->{'dhcp-dns-server'};
> + $dns = undef if defined($dns) && !Net::IP::ip_is_ipv4($dns);
> +
> + # the config hands its numbers over as strings, while the bindings take integers only
> + return {
> + ip => $ip4,
> + prefixlen => int($subnet->{mask}),
> + server_id => $gateway // $SERVER_ID_NO_GATEWAY,
> + lease => int($subnet->{'dhcp-lease-time'} // $DEFAULT_LEASE_TIME),
> + router => $gateway,
> + dns => $dns,
> + mac => lc($mac),
> + mtu => defined($mtu) ? int($mtu) : undef,
> + };
> +}
> +
> +# the IPv4 subnets of a vnet, warning about a DNS server the reply cannot
> +# carry when asked to. Only the apply asks, so a guest start does not repeat
> +# the warning
> +my sub served_subnets {
> + my ($vnetid, $report) = @_;
> +
> + my $subnets = PVE::Network::SDN::Vnets::get_subnets($vnetid, 1) // {};
> + my $served = [];
> + for my $subnetid (sort keys %$subnets) {
> + my $subnet = $subnets->{$subnetid};
> + next if !Net::IP::ip_is_ipv4($subnet->{network});
> + log_warn("subnet $subnetid has an IPv6 DNS server, not handing it out over IPv4")
> + if $report
> + && defined($subnet->{'dhcp-dns-server'})
> + && !Net::IP::ip_is_ipv4($subnet->{'dhcp-dns-server'});
> + push @$served, $subnet;
> + }
> +
> + return $served;
> +}
> +
> +# entries from before the cache was keyed in lower case carry their writer's
> +# spelling
> +my sub cached_macs {
> + my ($macdb) = @_;
> +
> + my $macs = {};
> + for my $mac (sort keys $macdb->{macs}->%*) {
> + my $entry = $macdb->{macs}->{$mac};
> + $macs->{ lc($mac) }->{$_} = $entry->{$_} for grep { defined($entry->{$_}) } qw(ip4 ip6);
> + }
> + return $macs;
> +}
> +
> +my $matchers = {};
> +
> +# the vnet's subnet holding the cached address decides the answer
> +my sub vnet_record {
> + my ($subnets, $mtu, $macs, $mac) = @_;
> +
> + my $ip4 = $macs->{ lc($mac) }->{ip4};
> + return undef if !$ip4;
> + # the most specific of overlapping subnets is the one to answer from
> + for my $subnet (sort { $b->{mask} <=> $a->{mask} } @$subnets) {
> + my $matcher = $matchers->{ $subnet->{cidr} } //= subnet_matcher($subnet->{cidr});
> + next if !$matcher->($ip4);
> + # the vnet's own gateway address is cached too and never a lease
> + return undef if defined($subnet->{gateway}) && $ip4 eq $subnet->{gateway};
> + return dhcp_record($ip4, $subnet, $mtu, $mac);
> + }
> +
> + return undef;
> +}
> +
> +my sub full_state {
> + my ($report) = @_;
> +
> + # the configs and the cache are read after the cluster state is refreshed,
> + # since a copy from before the lock was taken would hand the pass older input
> + PVE::Cluster::cfs_update();
> + my $cfg = PVE::Network::SDN::running_config();
> + my $zones = $cfg->{zones}->{ids} // {};
> + my $vnets = $cfg->{vnets}->{ids} // {};
> + # an ebpf zone confined to other nodes runs no responder here
> + my $nodename = PVE::INotify::nodename();
> + my %ebpf = map { $_ => 1 } grep {
> + ($zones->{$_}->{dhcp} // '') eq 'ebpf'
> + && (!defined($zones->{$_}->{nodes}) || $zones->{$_}->{nodes}->{$nodename})
> + } keys %$zones;
> + return (0, []) if !%ebpf;
> +
> + my $macs = cached_macs(PVE::Network::SDN::Ipams::read_macdb());
> + my (%mtu, %subnets);
> + # a subnet or a zone MTU the responder cannot serve is reported per apply,
> + # guests or not
> + if ($report) {
> + for my $vnetid (sort keys %$vnets) {
> + my $zoneid = $vnets->{$vnetid}->{zone};
> + next if !$zoneid || !$ebpf{$zoneid};
> + $subnets{$vnetid} //= served_subnets($vnetid, 1);
> + $mtu{$zoneid} = zone_mtu($zoneid, $zones->{$zoneid}, 1) if !exists $mtu{$zoneid};
> + }
> + }
> + my $ifaces = [];
> + for my $nic (PVE::Network::SDN::Dhcp::guest_nics()->@*) {
> + my $vnetid = $nic->{bridge};
> + my $zoneid = $vnets->{$vnetid} ? $vnets->{$vnetid}->{zone} : undef;
> + my $serve = $zoneid && $ebpf{$zoneid} ? 1 : 0;
> + my $record;
> + if ($serve) {
> + $mtu{$zoneid} = zone_mtu($zoneid, $zones->{$zoneid}, $report) if !exists $mtu{$zoneid};
> + $subnets{$vnetid} //= served_subnets($vnetid, $report);
> + $record = vnet_record($subnets{$vnetid}, $mtu{$zoneid}, $macs, $nic->{mac})
> + if $nic->{mac};
> + }
> + push @$ifaces,
> + {
> + name => $nic->{iface},
> + vnet => $vnets->{$vnetid} ? $vnetid : undef,
> + serve => $serve,
> + record => $record,
> + };
> + }
> +
> + return (scalar(keys %ebpf), $ifaces);
> +}
> +
> +my sub full_pass {
> + my ($report) = @_;
> +
> + my $lock = eval { PVE::RS::SDN::Dhcp::ExclusiveLock->new() };
> + if (!$lock) {
> + log_warn("could not lock the DHCP responder: $@");
> + return;
> + }
> +
> + my ($zones, $ifaces) = eval { full_state($report) };
> + if ($@) {
> + log_warn("could not collect the DHCP responder state: $@");
> + return;
> + }
> +
> + eval { $zones ? $lock->apply($ifaces) : $lock->clear() };
> + log_warn("could not apply the DHCP responder state: $@") if $@;
> +}
> +
> +my sub mac_ifaces {
> + my ($mac, $macs, $vmid) = @_;
> +
> + my $cfg = PVE::Network::SDN::running_config();
> + my $zones = $cfg->{zones}->{ids} // {};
> + my $vnets = $cfg->{vnets}->{ids} // {};
> + my $ifaces = [];
> + for my $nic (PVE::Network::SDN::Dhcp::guest_nics($vmid)->@*) {
> + next if lc($nic->{mac} // '') ne lc($mac);
> + my $vnet = $vnets->{ $nic->{bridge} } // next;
> + my $zone = $zones->{ $vnet->{zone} } // next;
> + next if ($zone->{dhcp} // '') ne 'ebpf';
> + my $record = vnet_record(
> + served_subnets($nic->{bridge}, 0),
> + zone_mtu($vnet->{zone}, $zone, 0),
> + $macs,
> + $mac,
> + );
> + push @$ifaces,
> + { name => $nic->{iface}, vnet => $nic->{bridge}, serve => 1, record => $record };
> + }
> +
> + return $ifaces;
> +}
> +
> +# under the guest lock nothing else writes the guest's entries, so a caller
> +# that names the guest takes the shared lock. An API edit names no guest and
> +# takes the exclusive one
> +sub update_ip_mapping {
> + my ($class, $dhcpid, $mac, $vmid) = @_;
> +
> + my $update = sub {
> + my $lock =
> + defined($vmid)
> + ? PVE::RS::SDN::Dhcp::SharedLock->new()
> + : PVE::RS::SDN::Dhcp::ExclusiveLock->new();
> + PVE::Cluster::cfs_update();
> + my $ifaces =
> + mac_ifaces($mac, cached_macs(PVE::Network::SDN::Ipams::read_macdb()), $vmid);
> + return 1 if !@$ifaces;
> + return $lock->update($ifaces);
> + };
> +
> + my $done = eval { $update->() };
> + if ($@) {
> + log_warn("could not update the DHCP responder records of $mac: $@");
> + return;
> + }
> + return if $done;
> +
> + # nothing loaded, the pass takes its own lock once the update's is gone
> + full_pass(0);
> +}
> +
> +# the dispatcher's per-zone walk is not needed, since the full pass collects
> +# the state itself
> +sub before_regenerate { }
> +sub before_configure { }
> +sub configure_subnet { }
> +sub configure_range { }
> +sub configure_vnet { }
> +sub after_configure { }
> +
> +sub after_regenerate {
> + my ($class) = @_;
> +
> + full_pass(1);
> +}
> +
> +sub boot {
> + my ($class) = @_;
> +
> + full_pass(1);
> +}
> +
> +sub tap_unplug {
> + my ($class, $iface, $target) = @_;
> +
> + eval { PVE::RS::SDN::Dhcp::detach($iface, $target) };
> + log_warn("could not detach the DHCP responder from $iface: $@") if $@;
> +}
> +
> +sub tap_plug {
> + my ($class, $dhcpid, $vnetid, $iface, $mac) = @_;
> +
> + my $attach = sub {
> + my $lock = PVE::RS::SDN::Dhcp::SharedLock->new();
> + my $record;
> + if ($mac) {
> + PVE::Cluster::cfs_update();
> + my $zone = PVE::Network::SDN::Zones::get_zone($dhcpid, 1);
> + $record = vnet_record(
> + served_subnets($vnetid, 0),
> + zone_mtu($dhcpid, $zone, 0),
> + cached_macs(PVE::Network::SDN::Ipams::read_macdb()),
> + $mac,
> + );
> + }
> + return $lock->attach($iface, $vnetid, $record);
> + };
> +
> + my $done = eval { $attach->() };
> + if ($@) {
> + log_warn("could not attach DHCP responder to $iface: $@");
> + return;
> + }
> + return if $done;
> +
> + # nothing is loaded, after a rebuild or a boot where the SDN commit loaded
> + # nothing. The plug's lock is gone by here, so the pass can take its own
> + full_pass(0);
> + $done = eval { $attach->() };
> + log_warn("could not attach DHCP responder to $iface: " . ($@ || "nothing is loaded"))
> + if $@ || !$done;
> +}
> +
> +1;
> diff --git a/src/PVE/Network/SDN/Dhcp/Makefile b/src/PVE/Network/SDN/Dhcp/Makefile
> index 65465135..ce86aaef 100644
> --- a/src/PVE/Network/SDN/Dhcp/Makefile
> +++ b/src/PVE/Network/SDN/Dhcp/Makefile
> @@ -1,4 +1,4 @@
> -SOURCES=Plugin.pm Dnsmasq.pm
> +SOURCES=Plugin.pm Dnsmasq.pm Ebpf.pm
>
>
> PERL5DIR=${DESTDIR}/usr/share/perl5
> diff --git a/src/PVE/Network/SDN/Dhcp/Plugin.pm b/src/PVE/Network/SDN/Dhcp/Plugin.pm
> index 2fa8683c..149873e6 100644
> --- a/src/PVE/Network/SDN/Dhcp/Plugin.pm
> +++ b/src/PVE/Network/SDN/Dhcp/Plugin.pm
> @@ -60,9 +60,28 @@ sub before_regenerate {
> die 'implement in sub class';
> }
>
> +# the node booted. A backend whose state does not survive a reboot brings it
> +# back here, before the guests start
> +sub boot {
> + my ($class) = @_;
> +}
> +
> sub after_regenerate {
> my ($class) = @_;
> die 'implement in sub class';
> }
>
> +# a guest interface was plugged into a vnet of a zone using this backend.
> +# Backends serving the bridge rather than the interface have nothing to do
> +sub tap_plug {
> + my ($class, $dhcpid, $vnetid, $iface, $mac) = @_;
> +}
> +
> +# a guest interface left its vnet or went away, and by then nothing says which
> +# zone it was on. The target is the vnet it sits on now, none for a plain
> +# bridge, and no target means it is gone
> +sub tap_unplug {
> + my ($class, $iface, $target) = @_;
> +}
> +
> 1;
> diff --git a/src/PVE/Network/SDN/SubnetPlugin.pm b/src/PVE/Network/SDN/SubnetPlugin.pm
> index 29edf7b6..38579f21 100644
> --- a/src/PVE/Network/SDN/SubnetPlugin.pm
> +++ b/src/PVE/Network/SDN/SubnetPlugin.pm
> @@ -183,8 +183,8 @@ sub properties {
> maximum => 4294967295,
> description =>
> 'Lease time in seconds for DHCP answers. Without it dnsmasq hands out'
> - . ' infinite leases, and it raises anything below two minutes to two'
> - . ' minutes.',
> + . ' infinite leases and the ebpf responder ten minutes. dnsmasq raises'
> + . ' anything below two minutes to two minutes.',
> optional => 1,
> },
> };
> diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
> index cda4b7ef..db6fc472 100755
> --- a/src/test/run_test_vnets_blackbox.pl
> +++ b/src/test/run_test_vnets_blackbox.pl
> @@ -47,6 +47,9 @@ sub clear_test_state {
> macdb => {},
> ipamdb => {},
> dnsmasq_calls => [],
> + ebpf_calls => [],
> + order => [],
> + warnings => [],
> ipam_config => {
> 'ids' => {
> 'pve' => {
> @@ -54,6 +57,7 @@ sub clear_test_state {
> },
> },
> },
> + ebpf_answer => undef,
> };
> PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "\n");
> }
> @@ -234,6 +238,43 @@ $mocked_sdn_dhcp_dnsmasq->mock(
> update_lease => sub { },
> );
>
> +my $mocked_pve_rs_dhcp = Test::MockModule->new('PVE::RS::SDN::Dhcp');
> +$mocked_pve_rs_dhcp->mock(
> + detach => sub {
> + push $test_state->{ebpf_calls}->@*, { method => 'detach', args => [@_] };
> + return $test_state->{ebpf_answer}->{detach} // 1;
> + },
> +);
> +
> +# taking a lock is the ordered step, and the responder calls are its methods
> +my %mocked_pve_rs_dhcp_locks;
> +for my $class (qw(SharedLock ExclusiveLock)) {
> + my $package = "PVE::RS::SDN::Dhcp::$class";
> + my $mock = Test::MockModule->new($package, no_auto => 1);
> + $mock->mock(
> + new => sub {
> + push $test_state->{order}->@*, 'lock';
> + push $test_state->{lock_modes}->@*, $class eq 'SharedLock' ? 'shared' : 'exclusive';
> + return bless {}, $package;
> + },
> + map {
> + my $method = $_;
> + $method => sub {
> + my ($self, @args) = @_;
> + push $test_state->{order}->@*, 'write';
> + push $test_state->{ebpf_calls}->@*, { method => $method, args => [@args] };
> + return $test_state->{ebpf_answer}->{$method} // 1;
> + };
> + } ($class eq 'SharedLock' ? qw(attach update) : qw(apply update clear)),
> + );
> + $mocked_pve_rs_dhcp_locks{$class} = $mock;
> +}
> +
> +my $mocked_sdn_dhcp_ebpf = Test::MockModule->new('PVE::Network::SDN::Dhcp::Ebpf');
> +$mocked_sdn_dhcp_ebpf->mock(
> + log_warn => sub { push $test_state->{warnings}->@*, $_[0]; },
> +);
> +
> my $mocked_api_zones = Test::MockModule->new('PVE::API2::Network::SDN::Zones');
> $mocked_api_zones->mock(
> create_etc_interfaces_sdn_dir => sub { },
> @@ -258,6 +299,15 @@ my $mocked_pve_cluster_obj = Test::MockModule->new('PVE::Cluster');
> $mocked_pve_cluster_obj->mock(
> check_cfs_quorum => sub { return 1; },
> cfs_lock_domain => $mocked_cfs_lock_domain,
> + cfs_update => sub { push $test_state->{order}->@*, 'refresh'; },
> + get_vmlist => sub { return $test_state->{vmlist} // { ids => {} }; },
> + get_guest_config_properties => sub {
> + my ($properties, $vmid) = @_;
> + my $nets = $test_state->{guest_nets} // {};
> + push $test_state->{nic_listings}->@*, $vmid;
> + return $nets if !defined($vmid);
> + return exists $nets->{$vmid} ? { $vmid => $nets->{$vmid} } : {};
> + },
> );
>
> # ------- TEST FUNCTIONS --------------
> @@ -365,6 +415,24 @@ sub delete_ip {
> return PVE::API2::Network::SDN::Ips->ipdelete($param);
> }
>
> +sub take_ebpf_calls {
> + my $calls = $test_state->{ebpf_calls};
> + $test_state->{ebpf_calls} = [];
> + return $calls;
> +}
> +
> +# the lock takes, cluster refreshes and responder writes in the order they
> +# ran, a repeated step collapsed into one
> +sub take_order {
> + my $order = $test_state->{order};
> + $test_state->{order} = [];
> + my @shape;
> + for my $step (@$order) {
> + push @shape, $step if !@shape || $shape[-1] ne $step;
> + }
> + return \@shape;
> +}
> +
> sub run_test {
> my $test = shift;
> clear_test_state();
> @@ -1437,4 +1505,578 @@ sub test_dnsmasq_dual_stack_and_sweep {
>
> run_test(\&test_dnsmasq_dual_stack_and_sweep);
>
> +# -------------- ebpf dhcp backend
> +my $ebpf_record = sub {
> + my ($ip, %over) = @_;
> + return {
> + ip => $ip,
> + prefixlen => 24,
> + server_id => '10.0.0.1',
> + lease => 600,
> + router => '10.0.0.1',
> + dns => undef,
> + mtu => undef,
> + mac => 'da:65:8f:18:9b:6f',
> + %over,
> + };
> +};
> +
> +sub test_ebpf_backend {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> + my $zoneid = "TESTZONE";
> + my $vnetid = "testvnet";
> + my $mac = "da:65:8f:18:9b:6f";
> +
> + create_zone({
> + type => "simple",
> + dhcp => "ebpf",
> + ipam => "pve",
> + zone => $zoneid,
> + });
> + create_vnet({
> + type => "vnet",
> + zone => $zoneid,
> + vnet => $vnetid,
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/24",
> + gateway => "10.0.0.1",
> + 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
> + });
> +
> + # the guest behind the MAC runs here, so its interface gets the record
> + $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
> + $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
> + take_ebpf_calls();
> + $test_state->{nic_listings} = [];
> + $test_state->{lock_modes} = [];
> + eval { nic_start($vnetid, $mac, "999", "testhostname"); };
> + if ($@) {
> + fail("$test_name: nic_start: $@");
> + return;
> + }
> + is(
> + $test_state->{nic_listings}->[0],
> + 999,
> + "$test_name: the start reads its own guest's config only",
> + );
> + eq_or_diff(
> + $test_state->{lock_modes},
> + ['shared'],
> + "$test_name: and writes its record under the shared lock",
> + );
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{
> + method => 'update',
> + args => [[{
> + name => 'tap999i0',
> + vnet => $vnetid,
> + serve => 1,
> + record => $ebpf_record->("10.0.0.100"),
> + }]],
> + }],
> + "$test_name: a guest start writes the record of its interface",
> + );
> +
> + # a mapping edit rewrites it, and a delete leaves the interface without one
> + $test_state->{lock_modes} = [];
> + update_ip({
> + zone => $zoneid,
> + vnet => $vnetid,
> + mac => $mac,
> + ip => "10.0.0.150",
> + });
> + eq_or_diff(
> + $test_state->{lock_modes},
> + ['exclusive'],
> + "$test_name: an edit names no guest and writes under the exclusive lock",
> + );
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{
> + method => 'update',
> + args => [[{
> + name => 'tap999i0',
> + vnet => $vnetid,
> + serve => 1,
> + record => $ebpf_record->("10.0.0.150"),
> + }]],
> + }],
> + "$test_name: a mapping edit rewrites the record",
> + );
> + delete_ip({
> + zone => $zoneid,
> + vnet => $vnetid,
> + mac => $mac,
> + ip => "10.0.0.150",
> + });
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{
> + method => 'update',
> + args => [[{ name => 'tap999i0', vnet => $vnetid, serve => 1, record => undef }]],
> + }],
> + "$test_name: a mapping delete drops the record",
> + );
> +
> + # a guest running elsewhere has no interface here to write to
> + $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'other' } } };
> + create_ip({
> + zone => $zoneid,
> + vnet => $vnetid,
> + mac => $mac,
> + ip => "10.0.0.150",
> + });
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{
> + method => 'update',
> + args => [[{
> + name => 'tap999i0',
> + vnet => $vnetid,
> + serve => 1,
> + record => $ebpf_record->("10.0.0.150"),
> + }]],
> + }],
> + "$test_name: a guest elsewhere is handed over too, the responder skips what is not here",
> + );
> +
> + # the full pass names every guest NIC in the cluster, whether the
> + # responder runs on it and what it answers. The responder keeps those
> + # present here. A guest the vmlist does not know is skipped, and a NIC
> + # without a MAC is served nothing
> + $test_state->{vmlist} = {
> + ids => {
> + 100 => { type => 'qemu', node => 'other' },
> + 101 => { type => 'lxc', node => 'other' },
> + 103 => { type => 'qemu', node => 'other' },
> + },
> + };
> + $test_state->{guest_nets} = {
> + 100 => {
> + net0 => "virtio=$mac,bridge=$vnetid,firewall=1",
> + net1 => "virtio=00:11:22:33:44:55,bridge=vmbr0",
> + },
> + 101 => { net2 => "name=eth0,bridge=$vnetid,hwaddr=00:11:22:33:44:66" },
> + 102 => { net0 => "virtio=00:11:22:33:44:77,bridge=$vnetid" },
> + 103 => { net0 => "virtio,bridge=$vnetid" },
> + };
> + PVE::Network::SDN::Dhcp::regenerate_config();
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{
> + method => 'apply',
> + args => [[
> + {
> + name => 'tap100i0',
> + vnet => $vnetid,
> + serve => 1,
> + record => $ebpf_record->("10.0.0.150"),
> + },
> + { name => 'tap100i1', vnet => undef, serve => 0, record => undef },
> + { name => 'veth101i2', vnet => $vnetid, serve => 1, record => undef },
> + { name => 'tap103i0', vnet => $vnetid, serve => 1, record => undef },
> + ]],
> + }],
> + "$test_name: the full pass names every guest NIC with its answer",
> + );
> +
> + # a responder with nothing loaded cannot take a single record, so the
> + # full pass loads it and fills everything
> + $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
> + $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
> + $test_state->{ebpf_answer} = { update => 0 };
> + $test_state->{lock_modes} = [];
> + nic_start($vnetid, $mac, "999", "testhostname");
> + $test_state->{ebpf_answer} = undef;
> + eq_or_diff(
> + [map { $_->{method} } take_ebpf_calls()->@*],
> + ['update', 'apply'],
> + "$test_name: an unloaded responder gets the full pass",
> + );
> + eq_or_diff(
> + $test_state->{lock_modes},
> + ['shared', 'exclusive'],
> + "$test_name: which takes its own lock after the record change let go",
> + );
> +
> + # a plug attaches the interface with the answer its MAC gets on the
> + # vnet. With nothing loaded the full pass runs first and the attach is
> + # retried
> + PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0', $mac);
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{ method => 'attach', args => ['tap999i0', $vnetid, $ebpf_record->("10.0.0.150")] }],
> + "$test_name: a plug attaches its interface with the record",
> + );
> + $test_state->{ebpf_answer} = { attach => 0 };
> + PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0', $mac);
> + $test_state->{ebpf_answer} = undef;
> + eq_or_diff(
> + [map { $_->{method} } take_ebpf_calls()->@*],
> + ['attach', 'apply', 'attach'],
> + "$test_name: a plug with nothing loaded runs the full pass first",
> + );
> + PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0');
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{ method => 'attach', args => ['tap999i0', $vnetid, undef] }],
> + "$test_name: a plug without a MAC attaches with nothing to answer",
> + );
> +}
> +
> +sub test_ebpf_backend_edge_cases {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> + my $zoneid = "TESTZONE";
> + my $vnetid = "testvnet";
> + my $mac = "da:65:8f:18:9b:6f";
> +
> + # a regenerate without any ebpf zone tears the responder state down
> + create_zone({
> + type => "simple",
> + ipam => "pve",
> + zone => $zoneid,
> + });
> + take_ebpf_calls();
> + PVE::Network::SDN::Dhcp::regenerate_config();
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{ method => 'clear', args => [] }],
> + "$test_name: regenerate without an ebpf zone clears the responder",
> + );
> +
> + # an ebpf zone confined to other nodes runs no responder here either
> + update_zone($zoneid, { dhcp => "ebpf", nodes => 'other' });
> + PVE::Network::SDN::Dhcp::regenerate_config();
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{ method => 'clear', args => [] }],
> + "$test_name: an ebpf zone confined to other nodes clears the responder here",
> + );
> + update_zone($zoneid, { delete => 'nodes' });
> +
> + # the SDN commit brings the responder back at boot, one full pass
> + PVE::Network::SDN::Dhcp::boot();
> + eq_or_diff(
> + [map { $_->{method} } take_ebpf_calls()->@*],
> + ['apply'],
> + "$test_name: the boot runs the full pass",
> + );
> +
> + # the zone switches to ebpf, but its subnets lack what a record needs
> + update_zone($zoneid, { dhcp => "ebpf" });
> + create_vnet({
> + type => "vnet",
> + zone => $zoneid,
> + vnet => $vnetid,
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/24",
> + 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "fd00::/64",
> + gateway => "fd00::1",
> + 'dhcp-range' => ["start-address=fd00::100,end-address=fd00::200"],
> + });
> + $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
> + $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
> +
> + take_ebpf_calls();
> + eval { nic_start($vnetid, $mac, "999", "testhostname"); };
> + if ($@) {
> + fail("$test_name: nic_start: $@");
> + return;
> + }
> +
> + # a gateway-less v4 subnet is served under a link-local identifier and
> + # without a router. The v6 subnet yields nothing
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{
> + method => 'update',
> + args => [[{
> + name => 'tap999i0',
> + vnet => $vnetid,
> + serve => 1,
> + record => $ebpf_record->(
> + "10.0.0.100",
> + server_id => '169.254.0.1',
> + router => undef,
> + ),
> + }]],
> + }],
> + "$test_name: a subnet without a gateway is served without a router",
> + );
> + $test_state->{warnings} = [];
> + PVE::Network::SDN::Dhcp::regenerate_config();
> + take_ebpf_calls();
> + is(scalar($test_state->{warnings}->@*), 0, "$test_name: and the full pass has nothing to say");
> +
> + # a v6 resolver on a v4 subnet is dropped from the record instead of
> + # failing the whole set
> + update_subnet({
> + vnet => $vnetid,
> + subnet => "$zoneid-10.0.0.0-24",
> + gateway => "10.0.0.1",
> + 'dhcp-dns-server' => "fd00::53",
> + });
> + $test_state->{warnings} = [];
> + PVE::Network::SDN::Dhcp::regenerate_config();
> + my $calls = take_ebpf_calls();
> + is(scalar(@$calls), 1, "$test_name: regenerate applies once");
> + my $iface = $calls->[0]->{args}->[0]->[0];
> + is($iface->{name}, 'tap999i0', "$test_name: the guest interface is listed");
> + is($iface->{record}->{ip}, '10.0.0.100', "$test_name: the v4 subnet now yields the record");
> + is($iface->{record}->{dns}, undef, "$test_name: the IPv6 resolver is not handed out");
> + is(
> + scalar(grep { m/IPv6 DNS server/ } $test_state->{warnings}->@*),
> + 1,
> + "$test_name: and the full pass says so",
> + );
> +}
> +
> +run_test(\&test_ebpf_backend);
> +run_test(\&test_ebpf_backend_edge_cases);
> +
> +sub test_ebpf_mac_case {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> + my $zoneid = "TESTZONE";
> + my $vnetid = "testvnet";
> + my $mac = "da:65:8f:18:9b:6f";
> +
> + # a mapping typed in lower case for a NIC whose config spells the MAC in
> + # upper case is the same mapping
> + create_zone({
> + type => "simple",
> + dhcp => "ebpf",
> + ipam => "pve",
> + zone => $zoneid,
> + });
> + create_vnet({
> + type => "vnet",
> + zone => $zoneid,
> + vnet => $vnetid,
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/24",
> + gateway => "10.0.0.1",
> + 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
> + });
> + $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
> + $test_state->{guest_nets} = { 999 => { net0 => "virtio=" . uc($mac) . ",bridge=$vnetid" } };
> + create_ip({
> + zone => $zoneid,
> + vnet => $vnetid,
> + mac => $mac,
> + ip => "10.0.0.100",
> + });
> + take_ebpf_calls();
> + take_order();
> + PVE::Network::SDN::Dhcp::regenerate_config();
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{
> + method => 'apply',
> + args => [[{
> + name => 'tap999i0',
> + vnet => $vnetid,
> + serve => 1,
> + record => $ebpf_record->("10.0.0.100"),
> + }]],
> + }],
> + "$test_name: the full pass finds the mapping however the config spells the MAC",
> + );
> + # the lock is taken before the refresh and the reads
> + is_deeply(
> + take_order(),
> + ['lock', 'refresh', 'write'],
> + "$test_name: the pass locks, refreshes, then writes",
> + );
> + PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0', uc($mac));
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{ method => 'attach', args => ['tap999i0', $vnetid, $ebpf_record->("10.0.0.100")] }],
> + "$test_name: and so does a plug",
> + );
> +
> + # an entry split across spellings is read as one
> + $test_state->{macdb} = {
> + macs => { uc($mac) => { ip4 => '10.0.0.100' }, $mac => { ip6 => '8888::100' } },
> + };
> + PVE::Network::SDN::Dhcp::regenerate_config();
> + is(
> + take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{ip},
> + '10.0.0.100',
> + "$test_name: a cache entry split across spellings is read as one",
> + );
> +}
> +
> +run_test(\&test_ebpf_mac_case);
> +
> +sub test_ebpf_equal_subnets {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> +
> + # two ebpf zones carry the same subnet. Each interface is answered from
> + # the subnet of the vnet it sits on, whatever address the cache holds
> + # for its MAC
> + for my $net (['ZONEA', 'vneta', '10.0.0.1'], ['ZONEB', 'vnetb', '10.0.0.2']) {
> + my ($zoneid, $vnetid, $gateway) = @$net;
> + create_zone({
> + type => "simple",
> + dhcp => "ebpf",
> + ipam => "pve",
> + zone => $zoneid,
> + });
> + create_vnet({
> + type => "vnet",
> + zone => $zoneid,
> + vnet => $vnetid,
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/24",
> + gateway => $gateway,
> + });
> + }
> + $test_state->{vmlist} = {
> + ids => {
> + 100 => { type => 'qemu', node => 'localnode' },
> + 101 => { type => 'qemu', node => 'localnode' },
> + },
> + };
> + $test_state->{guest_nets} = {
> + 100 => { net0 => "virtio=da:65:8f:18:9b:6f,bridge=vneta" },
> + 101 => { net0 => "virtio=da:65:8f:18:9b:70,bridge=vnetb" },
> + };
> + $test_state->{macdb} = {
> + macs => {
> + 'da:65:8f:18:9b:6f' => { ip4 => '10.0.0.150' },
> + 'da:65:8f:18:9b:70' => { ip4 => '10.0.0.150' },
> + },
> + };
> +
> + take_ebpf_calls();
> + PVE::Network::SDN::Dhcp::regenerate_config();
> + my $ifaces = take_ebpf_calls()->[0]->{args}->[0];
> + eq_or_diff(
> + [map { [$_->{name}, $_->{record}->{router}] } @$ifaces],
> + [['tap100i0', '10.0.0.1'], ['tap101i0', '10.0.0.2']],
> + "$test_name: each interface is answered from its own vnet",
> + );
> +}
> +
> +run_test(\&test_ebpf_equal_subnets);
> +
> +sub test_ebpf_zone_mtu {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> + my $zoneid = "TESTZONE";
> + my $vnetid = "testvnet";
> + my $mac = "da:65:8f:18:9b:6f";
> +
> + # a zone MTU the DHCP option cannot carry is reported on the apply and
> + # left out of the answer
> + create_zone({
> + type => "simple",
> + dhcp => "ebpf",
> + ipam => "pve",
> + zone => $zoneid,
> + mtu => 10,
> + });
> + create_vnet({
> + type => "vnet",
> + zone => $zoneid,
> + vnet => $vnetid,
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/24",
> + gateway => "10.0.0.1",
> + });
> + $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
> + $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
> + $test_state->{macdb} = { macs => { $mac => { ip4 => '10.0.0.100' } } };
> +
> + take_ebpf_calls();
> + $test_state->{warnings} = [];
> + PVE::Network::SDN::Dhcp::regenerate_config();
> + is(
> + scalar(grep { m/MTU 10/ } $test_state->{warnings}->@*),
> + 1,
> + "$test_name: the zone MTU is reported once",
> + );
> + is(
> + take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{mtu},
> + undef,
> + "$test_name: and not handed out",
> + );
> +
> + update_zone($zoneid, { mtu => 9000 });
> + PVE::Network::SDN::Dhcp::regenerate_config();
> + is(
> + take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{mtu},
> + 9000,
> + "$test_name: a zone MTU the option carries is handed out",
> + );
> +}
> +
> +run_test(\&test_ebpf_zone_mtu);
> +
> +sub test_ebpf_overlapping_subnets {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> + my $zoneid = "TESTZONE";
> + my $vnetid = "testvnet";
> + my $mac = "da:65:8f:18:9b:6f";
> +
> + # a vnet with overlapping subnets answers from the most specific one
> + create_zone({
> + type => "simple",
> + dhcp => "ebpf",
> + ipam => "pve",
> + zone => $zoneid,
> + });
> + create_vnet({
> + type => "vnet",
> + zone => $zoneid,
> + vnet => $vnetid,
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/16",
> + gateway => "10.0.255.1",
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/24",
> + gateway => "10.0.0.1",
> + });
> + $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
> + $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
> + $test_state->{macdb} = { macs => { $mac => { ip4 => '10.0.0.100' } } };
> +
> + take_ebpf_calls();
> + PVE::Network::SDN::Dhcp::regenerate_config();
> + my $record = take_ebpf_calls()->[0]->{args}->[0]->[0]->{record};
> + is_deeply(
> + [$record->@{qw(prefixlen router)}],
> + [24, '10.0.0.1'],
> + "$test_name: the most specific subnet answers",
> + );
> +}
> +
> +run_test(\&test_ebpf_overlapping_subnets);
> +
> done_testing();
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (16 preceding siblings ...)
2026-10-20 8:00 ` [PATCH pve-docs v3 17/17] sdn: dhcp: add eBPF backend section Hannes Laimer
@ 2026-10-20 13:11 ` Stefan Hanreich
2026-10-21 7:35 ` Hannes Laimer
2026-10-21 8:46 ` superseded: " Hannes Laimer
18 siblings, 1 reply; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:11 UTC (permalink / raw)
To: pve-devel
Did some testing:
* PVE IPAM / All zones / eBPF backend
* PVE IPAM / Simple zone / dnsmasq backend
* Netbox IPAM / All zones / eBPF backend
* Netbox IPAM / Simple zone / dnsmasq backend
* switching from dnsmasq -> eBPF and vice-versa
-> pre-existing: dnsmasq doesn't clean its config folder
* changing bridge on network devices
* changing subnets on the vnet and then reacquiring a lease
Everything worked really well, also the pre-existing dnsmasq zone. This
revision seems a lot nicer already without the generation counter, which
seemed a bit clunky. This is a more elegant solution imo.
One thing I found:
The new DHCP backend type needs to be added to proxmox-ve-rs as well,
otherwise the config fails to parse (firewall). We should add fallback
variants there...
for code-related stuff see the respective patches.
On 10/20/26 10:00 AM, Hannes Laimer wrote:
> This series adds a second DHCP backend, ebpf, next to dnsmasq. A zone
> picks its backend. dnsmasq stays limited to simple zones, while ebpf
> answers on the guest's own interface and works on every zone type. A
> guest on a zone with the ebpf backend gets the address its NIC has in
> the IPAM. There is no daemon per zone and no lease file. The answers
> are held in the kernel and do not depend on the IPAM being reachable.
>
> The responder is a BPF program on the ingress of each guest interface
> of a zone with the ebpf backend. It answers a request from the record
> of that interface, and the exchange never reaches the bridge. A
> request whose client MAC is not the interface's own passes on
> unanswered. So a guest cannot read another guest's record, and a
> nested client does not get the outer guest's address.
>
> An edit through /cluster/sdn/vnets/{vnet}/ips reaches the guest's node
> by a detached call to a new node endpoint, POST
> /nodes/{node}/sdn/dhcp-mapping. The guest sees the change at its next
> renewal, so the lease time bounds how long an old answer survives.
> Subnets get a dhcp-lease-time property for both backends. Without it
> dnsmasq hands out infinite leases and the responder ten minutes. A
> failed push is logged in the journal, and the responder on the guest's
> node catches up at the node's next apply or the guest's next start. An
> apply also attaches the responder to running guests, so a zone
> switched to ebpf serves them without a restart.
>
> Changes reach the kernel in the order they read their input. So no
> change overwrites a newer one with older input. A hotplug moves the
> interface before the guest config names the new bridge. An apply or an
> edit that read the older config in between leaves the moved interface
> as its plug left it. A change that fails only warns, and what was
> served stays served. Nothing of the responder survives a reboot, so it
> is loaded again at boot before the guests start.
>
> Only DHCPv4 is answered. A request the guest tagged itself passes
> unanswered as well, so a trunk port keeps working as before. The
> request is answered on the guest interface, so the guest's firewall
> rules, its DHCP firewall option and its rate limit do not apply to it.
> The reply leaves through the interface's egress, where the rate limit
> shapes it. A request the responder does not answer passes on to the
> bridge and through the firewall as before. Mappings changed directly
> in an external IPAM are not picked up, as with dnsmasq.
>
> In patch order, the series adds
> - proxmox-ebpf, the dhcp subsystem, a tc program answering on the
> guest interface and the per-interface map it answers from
> - pve-rs, the bindings
> - pve-network, the ebpf backend, the hooks that keep a backend current
> when a guest interface or a mapping changes, dhcp on every zone
> type, and fixes to the dnsmasq reservations and the MAC cache
> - qemu-server and pve-container, the reports of a guest NIC coming,
> going or moving, and for VMs the push at a migration's resume
> - pve-manager, the backend and subnet options in the UI, and the
> backends brought up at boot ahead of the guests
> - pve-docs, documentation
>
> - proxmox-ebpf applies on top of the v4 of its own series [1]
> - pve-rs needs proxmox-ebpf with the dhcp feature
> - pve-network needs pve-rs, at build and run time
> - qemu-server, pve-container and pve-manager need pve-network
> An older pve-network fails its apply on the unknown backend, so every
> node needs the new one before a zone gets ebpf.
>
> Changes since v2:
> * drop the generation counter. Every change takes the responder's
> lock before it reads its input instead. An edit takes it
> exclusively
> * the resume at the end of a live migration pushes the guest's
> mappings again on the target, so an edit made during the migration
> reaches it
> * new first pve-network commit, a dnsmasq fix that replaces a stale
> reservation of another MAC instead of dropping it
> * no MTU option for a zone without an MTU, where v2 handed out 1500
> * a mapping edit without a MAC no longer reaches the backends
> * rewrite commit messages, cover letter and parts of the docs
> * rebase onto current master
>
> (pre-build packages on sani)
>
> [1] https://lore.proxmox.com/pve-devel/20261019142413.2255644-1-h.laimer@proxmox.com/
>
>
> proxmox-ebpf:
>
> Hannes Laimer (2):
> dhcp: add the per-tap DHCPv4 responder program
> dhcp: add the subsystem that loads the responder and keeps its records
>
> Cargo.toml | 5 +
> debian/control | 6 +-
> src/bpf-shim/bpf/bpf_helpers.h | 3 +
> src/bpf-shim/vmlinux.h | 1 +
> src/dhcp/bpf/dhcp.bpf.c | 401 +++++++++++++
> src/dhcp/bpf/types.h | 24 +
> src/dhcp/mod.rs | 1031 ++++++++++++++++++++++++++++++++
> src/dhcp/types.rs | 64 ++
> src/lib.rs | 3 +
> src/subsystem.rs | 2 +-
> tests/common/mod.rs | 63 +-
> tests/dhcp.rs | 825 +++++++++++++++++++++++++
> 12 files changed, 2422 insertions(+), 6 deletions(-)
> create mode 100644 src/dhcp/bpf/dhcp.bpf.c
> create mode 100644 src/dhcp/bpf/types.h
> create mode 100644 src/dhcp/mod.rs
> create mode 100644 src/dhcp/types.rs
> create mode 100644 tests/dhcp.rs
>
>
> proxmox-perl-rs:
>
> Hannes Laimer (1):
> pve-rs: sdn: add bindings for the eBPF DHCP responder
>
> pve-rs/Cargo.toml | 2 +
> pve-rs/Makefile | 3 +
> pve-rs/debian/control | 2 +
> pve-rs/src/bindings/sdn/dhcp.rs | 229 ++++++++++++++++++++++++++++++++
> pve-rs/src/bindings/sdn/mod.rs | 1 +
> 5 files changed, 237 insertions(+)
> create mode 100644 pve-rs/src/bindings/sdn/dhcp.rs
>
>
> pve-network:
>
> Hannes Laimer (9):
> sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it
> sdn: dhcp: update reservations when ip mappings change
> sdn: ipam: key the MAC cache by lower-case MAC, stop caching misses
> sdn: dhcp: make the lease time configurable per subnet
> sdn: dhcp: require a backend only if a zone on this node uses it
> sdn: dhcp: add an eBPF backend answering on each guest interface
> sdn: zones: report interface plugs and unplugs to the DHCP backends
> sdn: dhcp: push IP mapping edits to the guest's node
> sdn: zones: allow the ebpf DHCP backend on every zone type
>
> debian/libpve-network-perl.prerm | 13 +
> src/PVE/API2/Network/SDN/Ips.pm | 10 +
> src/PVE/API2/Network/SDN/Nodes/Status.pm | 37 +
> src/PVE/API2/Network/SDN/Zones.pm | 11 +-
> src/PVE/Network/SDN/Dhcp.pm | 200 ++-
> src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 114 +-
> src/PVE/Network/SDN/Dhcp/Ebpf.pm | 308 +++++
> src/PVE/Network/SDN/Dhcp/Makefile | 2 +-
> src/PVE/Network/SDN/Dhcp/Plugin.pm | 26 +-
> src/PVE/Network/SDN/Ipams.pm | 61 +-
> src/PVE/Network/SDN/SubnetPlugin.pm | 11 +
> src/PVE/Network/SDN/Vnets.pm | 5 +-
> src/PVE/Network/SDN/Zones.pm | 29 +-
> src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 32 +
> src/PVE/Network/SDN/Zones/FaucetPlugin.pm | 1 +
> src/PVE/Network/SDN/Zones/QinQPlugin.pm | 7 +
> src/PVE/Network/SDN/Zones/SimplePlugin.pm | 2 +-
> src/PVE/Network/SDN/Zones/VlanPlugin.pm | 7 +
> src/PVE/Network/SDN/Zones/VxlanPlugin.pm | 25 +
> src/test/run_test_vnets_blackbox.pl | 1442 ++++++++++++++++++++-
> 20 files changed, 2263 insertions(+), 80 deletions(-)
> create mode 100755 debian/libpve-network-perl.prerm
> create mode 100644 src/PVE/Network/SDN/Dhcp/Ebpf.pm
>
>
> qemu-server:
>
> Hannes Laimer (1):
> network: sdn: keep the DHCP backends current on guest NICs
>
> src/PVE/CLI/qm.pm | 4 ++--
> src/PVE/QemuServer.pm | 4 ++++
> src/PVE/QemuServer/Network.pm | 19 +++++++++++++++++--
> src/PVE/QemuServer/RunState.pm | 2 ++
> src/usr/pve-bridge | 8 +++++++-
> src/usr/pve-bridgedown | 4 ++--
> 6 files changed, 34 insertions(+), 7 deletions(-)
>
>
> pve-container:
>
> Hannes Laimer (1):
> net: report NIC plugs and removals to the SDN DHCP backends
>
> src/PVE/LXC.pm | 13 +++++++++++--
> src/PVE/LXC/Config.pm | 3 ++-
> src/lxc-pve-poststop-hook | 6 +++---
> 3 files changed, 16 insertions(+), 6 deletions(-)
>
>
> pve-manager:
>
> Hannes Laimer (2):
> ui: sdn: add zone DHCP backend selector and subnet DHCP options
> sdn: bring up the DHCP backends at boot before the guests start
>
> bin/pve-sdn-commit | 3 +++
> services/pve-sdn-commit.service | 1 +
> www/manager6/sdn/SubnetEdit.js | 23 +++++++++++++++++++++++
> www/manager6/sdn/zones/Base.js | 17 +++++++++++++++++
> www/manager6/sdn/zones/SimpleEdit.js | 11 -----------
> 5 files changed, 44 insertions(+), 11 deletions(-)
>
>
> pve-docs:
>
> Hannes Laimer (1):
> sdn: dhcp: add eBPF backend section
>
> pvesdn.adoc | 101 ++++++++++++++++++++++++++++++++++++++++++----------
> 1 file changed, 83 insertions(+), 18 deletions(-)
>
>
> Summary over all repositories:
> 52 files changed, 5099 insertions(+), 128 deletions(-)
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-network v3 04/17] sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it
2026-10-20 7:59 ` [PATCH pve-network v3 04/17] sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it Hannes Laimer
@ 2026-10-20 13:15 ` Stefan Hanreich
0 siblings, 0 replies; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:15 UTC (permalink / raw)
To: pve-devel
Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com>
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
On 10/20/26 10:00 AM, Hannes Laimer wrote:
> A reservation rewrite dropped every line that no longer matched the
> MAC cache and wrote nothing in its place. The guest behind such a
> line was unknown to dnsmasq until its next start, so its requests
> were ignored. The line is now replaced by the cache's current one.
> Only a MAC the cache holds no address for is dropped.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 11 ++++++++++-
> src/test/run_test_vnets_blackbox.pl | 28 ++++++++++++++++++++++++++++
> 2 files changed, 38 insertions(+), 1 deletion(-)
>
> diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
> index 477b700f..5c9496c0 100644
> --- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
> +++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
> @@ -88,7 +88,9 @@ sub add_ip_mapping {
> }
> $parsed_ip6 = $1 if $parsed_ip6 && $parsed_ip6 =~ m/\[(\S+)\]/;
>
> - #delete changed
> + # a line that no longer matches the cache is replaced by the cache's
> + # current one. Dropping it left that guest unknown to dnsmasq until
> + # its next start
> if (
> !defined($macdb->{macs}->{$parsed_mac})
> || ($parsed_ip4
> @@ -99,6 +101,13 @@ sub add_ip_mapping {
> && $macdb->{macs}->{$parsed_mac}->{'ip6'} ne $parsed_ip6)
> ) {
> $reload = 1;
> + # the MAC being added gets its line below, like a missing one
> + my $current = $macdb->{macs}->{$parsed_mac};
> + next if $parsed_mac eq $mac || !$current;
> + my $replacement = $parsed_mac;
> + $replacement .= ",$current->{ip4}" if $current->{ip4};
> + $replacement .= ",[$current->{ip6}]" if $current->{ip6};
> + print $out "$replacement\n" if $replacement ne $parsed_mac;
> next;
> }
>
> diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
> index 9f4c424f..36225a92 100755
> --- a/src/test/run_test_vnets_blackbox.pl
> +++ b/src/test/run_test_vnets_blackbox.pl
> @@ -963,4 +963,32 @@ run_test(
> 2,
> );
>
> +sub test_dnsmasq_stale_reservation_replaced {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> +
> + my $stale = "da:65:8f:18:9b:01";
> + my $gone = "da:65:8f:18:9b:02";
> + my $added = "da:65:8f:18:9b:03";
> + $test_state->{macdb} = {
> + macs => {
> + $stale => { ip4 => '10.0.1.11' },
> + $added => { ip4 => '10.0.1.13' },
> + },
> + };
> + PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "$stale,10.0.1.10\n$gone,10.0.1.12\n");
> +
> + PVE::Network::SDN::Dhcp::Dnsmasq->add_ip_mapping(
> + 'TESTZONE', $test_state->{macdb}, $added, '10.0.1.13', undef,
> + );
> +
> + my @lines = sort grep { /\S/ } split(/\n/, PVE::Tools::file_get_contents($TMP_ETHERS_FILE));
> + eq_or_diff(
> + \@lines,
> + ["$stale,10.0.1.11", "$added,10.0.1.13"],
> + "$test_name: the stale line is replaced, the unknown one dropped",
> + );
> +}
> +
> +run_test(\&test_dnsmasq_stale_reservation_replaced);
> +
> done_testing();
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-network v3 07/17] sdn: dhcp: make the lease time configurable per subnet
2026-10-20 7:59 ` [PATCH pve-network v3 07/17] sdn: dhcp: make the lease time configurable per subnet Hannes Laimer
@ 2026-10-20 13:17 ` Stefan Hanreich
0 siblings, 0 replies; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:17 UTC (permalink / raw)
To: pve-devel
Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com>
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
On 10/20/26 10:00 AM, Hannes Laimer wrote:
> dnsmasq hands out infinite leases, so a guest asks for its address
> again only at its next reboot or network restart. A finite lease makes
> it renew within the lease time. An address changed in the IPAM goes to
> dnsmasq on the node the change is made on, so a guest there picks up
> the new address at its next renewal. Guests on other nodes get it only
> at their next start there.
>
> A subnet without a lease time keeps infinite leases.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 3 +-
> src/PVE/Network/SDN/SubnetPlugin.pm | 11 +++++++
> src/test/run_test_vnets_blackbox.pl | 51 +++++++++++++++++++++++++++++
> 3 files changed, 64 insertions(+), 1 deletion(-)
>
> diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
> index 57ce1f73..c6c9ce1d 100644
> --- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
> +++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
> @@ -200,7 +200,8 @@ sub configure_subnet {
> $mask = join('.', unpack("C4", pack("N", $mask)));
> }
>
> - push @{$config}, "dhcp-range=set:$tag,$network,static,$mask,infinite";
> + my $lease = $subnet_config->{'dhcp-lease-time'} // 'infinite';
> + push @{$config}, "dhcp-range=set:$tag,$network,static,$mask,$lease";
>
> my $option_string;
> if (ip_is_ipv6($subnet_config->{network})) {
> diff --git a/src/PVE/Network/SDN/SubnetPlugin.pm b/src/PVE/Network/SDN/SubnetPlugin.pm
> index e2a0e50d..29edf7b6 100644
> --- a/src/PVE/Network/SDN/SubnetPlugin.pm
> +++ b/src/PVE/Network/SDN/SubnetPlugin.pm
> @@ -177,6 +177,16 @@ sub properties {
> description => 'IP address for the DNS server',
> optional => 1,
> },
> + 'dhcp-lease-time' => {
> + type => 'integer',
> + minimum => 60,
> + maximum => 4294967295,
> + description =>
> + 'Lease time in seconds for DHCP answers. Without it dnsmasq hands out'
> + . ' infinite leases, and it raises anything below two minutes to two'
> + . ' minutes.',
> + optional => 1,
> + },
> };
> }
>
> @@ -189,6 +199,7 @@ sub options {
> dnszoneprefix => { optional => 1 },
> 'dhcp-range' => { optional => 1 },
> 'dhcp-dns-server' => { optional => 1 },
> + 'dhcp-lease-time' => { optional => 1 },
> };
> }
>
> diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
> index 476e7f8d..dc54bba3 100755
> --- a/src/test/run_test_vnets_blackbox.pl
> +++ b/src/test/run_test_vnets_blackbox.pl
> @@ -334,6 +334,11 @@ sub create_subnet {
> PVE::API2::Network::SDN::Subnets->create($params);
> }
>
> +sub update_subnet {
> + my ($params) = @_;
> + PVE::API2::Network::SDN::Subnets->update($params);
> +}
> +
> sub get_ipam_entries {
> return PVE::API2::Network::SDN::Ipams->ipamindex({ ipam => "pve" });
> }
> @@ -1118,6 +1123,52 @@ sub test_dnsmasq_mapping_push {
>
> run_test(\&test_dnsmasq_mapping_push);
>
> +sub test_dnsmasq_lease_time {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> + my $zoneid = "TESTZONE";
> + my $vnetid = "testvnet";
> +
> + # the subnet's lease time reaches the range, and infinite stays the default
> + create_zone({
> + type => "simple",
> + dhcp => "dnsmasq",
> + ipam => "pve",
> + zone => $zoneid,
> + });
> + create_vnet({
> + type => "vnet",
> + zone => $zoneid,
> + vnet => $vnetid,
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/24",
> + gateway => "10.0.0.1",
> + 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
> + });
> + my $range = sub {
> + my $subnets = PVE::Network::SDN::Vnets::get_subnets($vnetid, 1);
> + my $config = [];
> + for my $id (sort keys %$subnets) {
> + PVE::Network::SDN::Dhcp::Dnsmasq->configure_subnet(
> + $config, $zoneid, $vnetid, $subnets->{$id},
> + );
> + }
> + my ($line) = grep { m/^dhcp-range=/ } @$config;
> + return $line;
> + };
> + like($range->(), qr/,infinite$/, "$test_name: no lease time means infinite leases");
> + update_subnet({
> + vnet => $vnetid,
> + subnet => "$zoneid-10.0.0.0-24",
> + 'dhcp-lease-time' => 300,
> + });
> + like($range->(), qr/,300$/, "$test_name: the lease time reaches the range");
> +}
> +
> +run_test(\&test_dnsmasq_lease_time);
> +
> sub test_ipam_cache_misses {
> my $test_name = (split(/::/, (caller(0))[3]))[-1];
> my $zoneid = "TESTZONE";
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-network v3 08/17] sdn: dhcp: require a backend only if a zone on this node uses it
2026-10-20 7:59 ` [PATCH pve-network v3 08/17] sdn: dhcp: require a backend only if a zone on this node uses it Hannes Laimer
@ 2026-10-20 13:17 ` Stefan Hanreich
0 siblings, 0 replies; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:17 UTC (permalink / raw)
To: pve-devel
Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com>
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
On 10/20/26 10:01 AM, Hannes Laimer wrote:
> A network reload failed on a node without the dnsmasq package as soon
> as any zone had DHCP, even a zone confined to other nodes. On a node
> with dnsmasq, the reload wrote such a zone's configuration and started
> an instance for it, although none of the zone's vnets exist there.
>
> A reload now skips the zones confined to other nodes, so a backend is
> required and started only for the zones on the reloading node.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> src/PVE/Network/SDN/Dhcp.pm | 13 +++++++--
> src/test/run_test_vnets_blackbox.pl | 43 +++++++++++++++++++++++++++++
> 2 files changed, 53 insertions(+), 3 deletions(-)
>
> diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
> index ab5d6b5b..2a023534 100644
> --- a/src/PVE/Network/SDN/Dhcp.pm
> +++ b/src/PVE/Network/SDN/Dhcp.pm
> @@ -63,17 +63,24 @@ sub regenerate_config {
>
> my $plugins = PVE::Network::SDN::Dhcp::Plugin->lookup_types();
>
> - my $any_zone_needs_dhcp = grep { $_->{dhcp} } values $zone_cfg->{ids}->%*;
> + my %plugin_needed = ();
> + for my $zone (values $zone_cfg->{ids}->%*) {
> + next if !$zone->{dhcp};
> + # a zone confined to other nodes runs no backend here
> + next if defined($zone->{nodes}) && !$zone->{nodes}->{$nodename};
> + $plugin_needed{ $zone->{dhcp} } = 1;
> + }
>
> foreach my $plugin_name (@$plugins) {
> my $plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($plugin_name);
> - eval { $plugin->before_regenerate(!$any_zone_needs_dhcp) };
> + eval { $plugin->before_regenerate(!$plugin_needed{$plugin_name}) };
> die "Could not run before_regenerate for DHCP plugin $plugin_name $@\n" if $@;
> }
>
> foreach my $zoneid (sort keys %{ $zone_cfg->{ids} }) {
> my $zone = $zone_cfg->{ids}->{$zoneid};
> next if !$zone->{dhcp};
> + next if defined($zone->{nodes}) && !$zone->{nodes}->{$nodename};
>
> my $dhcp_plugin_name = $zone->{dhcp};
> my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($dhcp_plugin_name);
> @@ -114,7 +121,7 @@ sub regenerate_config {
> warn "Could not configure vnet $vnetid: $@\n" if $@;
> }
>
> - eval { $dhcp_plugin->after_configure($zoneid, !$any_zone_needs_dhcp) };
> + eval { $dhcp_plugin->after_configure($zoneid, !$plugin_needed{$dhcp_plugin_name}) };
> warn "Could not run after_configure for DHCP server $zoneid $@\n" if $@;
>
> }
> diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
> index dc54bba3..cda4b7ef 100755
> --- a/src/test/run_test_vnets_blackbox.pl
> +++ b/src/test/run_test_vnets_blackbox.pl
> @@ -104,6 +104,13 @@ my $mocked_pve_sdn;
> $mocked_pve_sdn = Test::MockModule->new('PVE::Network::SDN');
> $mocked_pve_sdn->mock(
> cfs_lock_file => $mocked_cfs_lock_file,
> + running_config => sub {
> + return {
> + zones => $test_state->{zones_config},
> + vnets => $test_state->{vnets_config},
> + subnets => $test_state->{subnets_config},
> + };
> + },
> );
>
> my $mocked_pve_tools = Test::MockModule->new('PVE::Tools');
> @@ -1169,6 +1176,42 @@ sub test_dnsmasq_lease_time {
>
> run_test(\&test_dnsmasq_lease_time);
>
> +sub test_dhcp_backend_needed_per_node {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> +
> + # a backend is told whether a zone on this node uses it. A zone confined
> + # to other nodes runs nothing here
> + my $asked = [];
> + my $configured = [];
> + $mocked_sdn_dhcp_dnsmasq->mock(
> + before_regenerate => sub { push @$asked, $_[1] ? 'optional' : 'needed'; },
> + after_configure => sub { push @$configured, $_[1]; },
> + );
> + create_zone({
> + type => "simple",
> + dhcp => "dnsmasq",
> + ipam => "pve",
> + zone => "DNSZONE",
> + nodes => 'other',
> + });
> + PVE::Network::SDN::Dhcp::regenerate_config();
> + eq_or_diff(
> + $asked,
> + ['optional'],
> + "$test_name: a dnsmasq zone confined elsewhere needs no dnsmasq here",
> + );
> + eq_or_diff($configured, [], "$test_name: and is not configured here");
> +
> + @$asked = ();
> + update_zone("DNSZONE", { delete => 'nodes' });
> + PVE::Network::SDN::Dhcp::regenerate_config();
> + eq_or_diff($asked, ['needed'], "$test_name: a dnsmasq zone on this node needs it");
> + eq_or_diff($configured, ['DNSZONE'], "$test_name: and is configured");
> + $mocked_sdn_dhcp_dnsmasq->unmock($_) for qw(before_regenerate after_configure);
> +}
> +
> +run_test(\&test_dhcp_backend_needed_per_node);
> +
> sub test_ipam_cache_misses {
> my $test_name = (split(/::/, (caller(0))[3]))[-1];
> my $zoneid = "TESTZONE";
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-network v3 10/17] sdn: zones: report interface plugs and unplugs to the DHCP backends
2026-10-20 7:59 ` [PATCH pve-network v3 10/17] sdn: zones: report interface plugs and unplugs to the DHCP backends Hannes Laimer
@ 2026-10-20 13:17 ` Stefan Hanreich
0 siblings, 0 replies; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:17 UTC (permalink / raw)
To: pve-devel
Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com>
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
On 10/20/26 10:01 AM, Hannes Laimer wrote:
> The ebpf DHCP backend answers on each guest interface rather than on
> the bridge, so the zone layer now reports every plug to the DHCP
> backends. The plug takes the guest's MAC along, since the address to
> hand out is recorded per MAC. A plug onto a plain bridge is reported
> too, since a running guest's interface can move there from a vnet.
> A failing backend only warns and the plug goes ahead.
>
> The zone layer also gains an unplug and a veth removal, which report
> the interface to the backends as gone.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> src/PVE/Network/SDN/Zones.pm | 29 ++++++-
> src/test/run_test_vnets_blackbox.pl | 129 ++++++++++++++++++++++++++++
> 2 files changed, 154 insertions(+), 4 deletions(-)
>
> diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm
> index 4229feda..10fc0731 100644
> --- a/src/PVE/Network/SDN/Zones.pm
> +++ b/src/PVE/Network/SDN/Zones.pm
> @@ -326,16 +326,18 @@ sub veth_create {
> }
>
> sub tap_plug {
> - my ($iface, $bridge, $tag, $firewall, $trunks, $rate) = @_;
> + my ($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts) = @_;
>
> my $vnet = PVE::Network::SDN::Vnets::get_vnet($bridge, 1);
> if (!$vnet) { # fallback for classic bridge
> my $interfaces_config = PVE::INotify::read_file('interfaces');
> - my $opts = {};
> - $opts->{learning} = 0
> + my $bridge_opts = {};
> + $bridge_opts->{learning} = 0
> if $interfaces_config->{ifaces}->{$bridge}
> && $interfaces_config->{ifaces}->{$bridge}->{'bridge-disable-mac-learning'};
> - PVE::Network::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts);
> + # the dhcp backends hear of an interface moving onto a plain bridge as well
> + PVE::Network::SDN::Dhcp::tap_plug($bridge, $iface, $opts->{mac});
> + PVE::Network::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate, $bridge_opts);
> return;
> }
>
> @@ -346,9 +348,28 @@ sub tap_plug {
> if $plugin_config->{nodes} && !defined($plugin_config->{nodes}->{$nodename});
>
> my $plugin = PVE::Network::SDN::Zones::Plugin->lookup($plugin_config->{type});
> +
> + # the dhcp backends act before the interface joins the bridge, so no request slips past
> + # them. Dhcp.pm comes in through Vnets.pm, since a use of it here would create a load-order
> + # cycle
> + PVE::Network::SDN::Dhcp::tap_plug($bridge, $iface, $opts->{mac});
> $plugin->tap_plug($plugin_config, $vnet, $tag, $iface, $bridge, $firewall, $trunks, $rate);
> }
>
> +sub tap_unplug {
> + my ($iface) = @_;
> +
> + PVE::Network::SDN::Dhcp::tap_unplug($iface);
> + PVE::Network::tap_unplug($iface);
> +}
> +
> +sub veth_delete {
> + my ($veth) = @_;
> +
> + PVE::Network::SDN::Dhcp::tap_unplug($veth);
> + PVE::Network::veth_delete($veth);
> +}
> +
> sub add_bridge_fdb {
> my ($iface, $macaddr, $bridge) = @_;
>
> diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
> index db6fc472..783d0d47 100755
> --- a/src/test/run_test_vnets_blackbox.pl
> +++ b/src/test/run_test_vnets_blackbox.pl
> @@ -1844,6 +1844,135 @@ sub test_ebpf_backend_edge_cases {
> }
>
> run_test(\&test_ebpf_backend);
> +
> +sub test_zone_tap_plug {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> + my $zoneid = "TESTZONE";
> + my $vnetid = "testvnet";
> + my $mac = "da:65:8f:18:9b:6f";
> +
> + # the zone plug hands the MAC on to the dhcp backends, and a plug onto a
> + # plain bridge tells them where the interface went
> + create_zone({
> + type => "simple",
> + dhcp => "ebpf",
> + ipam => "pve",
> + zone => $zoneid,
> + });
> + create_vnet({
> + type => "vnet",
> + zone => $zoneid,
> + vnet => $vnetid,
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/24",
> + gateway => "10.0.0.1",
> + 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
> + });
> + create_ip({
> + zone => $zoneid,
> + vnet => $vnetid,
> + mac => $mac,
> + ip => "10.0.0.100",
> + });
> + my $bridged = [];
> + my $unplugged = [];
> + $mocked_sdn_zones_super_plugin->mock(
> + tap_plug => sub {
> + push @$bridged, $_[5];
> + push $test_state->{order}->@*, 'bridge';
> + },
> + );
> + my $mocked_pve_network = Test::MockModule->new('PVE::Network');
> + $mocked_pve_network->mock(
> + tap_plug => sub {
> + push @$bridged, $_[1];
> + push $test_state->{order}->@*, 'bridge';
> + },
> + tap_unplug => sub { push @$unplugged, $_[0]; },
> + veth_delete => sub { push @$unplugged, $_[0]; },
> + );
> + $mocked_pve_inotify->mock(read_file => sub { return { ifaces => {} }; });
> + take_ebpf_calls();
> + take_order();
> +
> + PVE::Network::SDN::Zones::tap_plug('tap999i0', $vnetid, undef, 0, undef, undef,
> + { mac => $mac });
> + eq_or_diff($bridged, [$vnetid], "$test_name: the zone plugin plugs the interface");
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{ method => 'attach', args => ['tap999i0', $vnetid, $ebpf_record->("10.0.0.100")] }],
> + "$test_name: and the dhcp backend attaches it with the MAC's record",
> + );
> + is_deeply(
> + take_order(),
> + ['lock', 'refresh', 'write', 'bridge'],
> + "$test_name: the responder attaches before the interface joins the bridge",
> + );
> +
> + @$bridged = ();
> + PVE::Network::SDN::Zones::tap_plug('tap999i0', 'vmbr0', undef, 0, undef, undef,
> + { mac => $mac });
> + eq_or_diff($bridged, ['vmbr0'], "$test_name: a plain bridge is plugged the classic way");
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{ method => 'detach', args => ['tap999i0', { vnet => undef }] }],
> + "$test_name: and the dhcp backends hear where the interface went",
> + );
> +
> + # a vnet of another backend is a place the ebpf backend hears of as well
> + create_zone({
> + type => "simple",
> + dhcp => "dnsmasq",
> + ipam => "pve",
> + zone => "DNSZONE",
> + });
> + create_vnet({
> + type => "vnet",
> + zone => "DNSZONE",
> + vnet => "dnsvnet",
> + });
> + @$bridged = ();
> + PVE::Network::SDN::Zones::tap_plug(
> + 'tap999i0',
> + 'dnsvnet',
> + undef,
> + 0,
> + undef,
> + undef,
> + { mac => $mac },
> + );
> + eq_or_diff(
> + $bridged,
> + ['dnsvnet'],
> + "$test_name: a vnet of another backend is plugged by its zone",
> + );
> + eq_or_diff(
> + take_ebpf_calls(),
> + [{ method => 'detach', args => ['tap999i0', { vnet => 'dnsvnet' }] }],
> + "$test_name: and the ebpf backend hears which vnet the interface went to",
> + );
> +
> + # the zone unplug and the veth delete report the interface gone before
> + # the classic ones run
> + PVE::Network::SDN::Zones::tap_unplug('tap999i0');
> + PVE::Network::SDN::Zones::veth_delete('veth999i0');
> + eq_or_diff($unplugged, ['tap999i0', 'veth999i0'], "$test_name: the classic unplugs run");
> + eq_or_diff(
> + take_ebpf_calls(),
> + [
> + { method => 'detach', args => ['tap999i0', undef] },
> + { method => 'detach', args => ['veth999i0', undef] },
> + ],
> + "$test_name: and the dhcp backends hear the interfaces are gone",
> + );
> + $mocked_sdn_zones_super_plugin->unmock('tap_plug');
> + $mocked_pve_inotify->unmock('read_file');
> +}
> +
> +run_test(\&test_zone_tap_plug);
> run_test(\&test_ebpf_backend_edge_cases);
>
> sub test_ebpf_mac_case {
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-manager v3 15/17] ui: sdn: add zone DHCP backend selector and subnet DHCP options
2026-10-20 8:00 ` [PATCH pve-manager v3 15/17] ui: sdn: add zone DHCP backend selector and subnet DHCP options Hannes Laimer
@ 2026-10-20 13:19 ` Stefan Hanreich
0 siblings, 0 replies; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:19 UTC (permalink / raw)
To: pve-devel
Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com>
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
On 10/20/26 10:02 AM, Hannes Laimer wrote:
> Zones of every type can now get DHCP from the eBPF backend, while
> dnsmasq stays limited to simple zones. The checkbox that enabled
> dnsmasq on simple zones becomes a backend selector in the advanced
> options of every zone dialog.
>
> The subnet dialog gains fields for the new DHCP lease time and for the
> DHCP DNS server, which could be set only through the API until now.
> The eBPF backend hands out a DNS server only when the subnet sets one.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> www/manager6/sdn/SubnetEdit.js | 23 +++++++++++++++++++++++
> www/manager6/sdn/zones/Base.js | 17 +++++++++++++++++
> www/manager6/sdn/zones/SimpleEdit.js | 11 -----------
> 3 files changed, 40 insertions(+), 11 deletions(-)
>
> diff --git a/www/manager6/sdn/SubnetEdit.js b/www/manager6/sdn/SubnetEdit.js
> index a3608428..a1d36eeb 100644
> --- a/www/manager6/sdn/SubnetEdit.js
> +++ b/www/manager6/sdn/SubnetEdit.js
> @@ -56,6 +56,29 @@ Ext.define('PVE.sdn.SubnetInputPanel', {
> deleteEmpty: '{!isCreate}',
> },
> },
> + {
> + xtype: 'proxmoxtextfield',
> + name: 'dhcp-dns-server',
> + vtype: 'IP64Address',
> + fieldLabel: gettext('DHCP DNS Server'),
> + allowBlank: true,
> + skipEmptyText: true,
> + cbind: {
> + deleteEmpty: '{!isCreate}',
> + },
> + },
> + {
> + xtype: 'proxmoxintegerfield',
> + name: 'dhcp-lease-time',
> + fieldLabel: gettext('DHCP Lease Time') + ' (s)',
> + emptyText: Proxmox.Utils.defaultText,
> + minValue: 60,
> + maxValue: 4294967295,
> + allowBlank: true,
> + cbind: {
> + deleteEmpty: '{!isCreate}',
> + },
> + },
> ],
> });
>
> diff --git a/www/manager6/sdn/zones/Base.js b/www/manager6/sdn/zones/Base.js
> index 66f93de0..7696136d 100644
> --- a/www/manager6/sdn/zones/Base.js
> +++ b/www/manager6/sdn/zones/Base.js
> @@ -82,6 +82,23 @@ Ext.define('PVE.panel.SDNZoneBase', {
> },
> );
>
> + let dhcpBackends = [
> + ['__default__', Proxmox.Utils.NoneText],
> + ['ebpf', 'eBPF'],
> + ];
> + if (me.type === 'simple') {
> + dhcpBackends.splice(1, 0, ['dnsmasq', 'dnsmasq']);
> + }
> +
> + me.advancedItems.push({
> + xtype: 'proxmoxKVComboBox',
> + name: 'dhcp',
> + fieldLabel: gettext('DHCP Backend'),
> + comboItems: dhcpBackends,
> + value: '__default__',
> + deleteEmpty: !me.isCreate,
> + });
> +
> me.callParent();
> },
> });
> diff --git a/www/manager6/sdn/zones/SimpleEdit.js b/www/manager6/sdn/zones/SimpleEdit.js
> index ba10bb36..aeb076ae 100644
> --- a/www/manager6/sdn/zones/SimpleEdit.js
> +++ b/www/manager6/sdn/zones/SimpleEdit.js
> @@ -19,17 +19,6 @@ Ext.define('PVE.sdn.zones.SimpleInputPanel', {
> var me = this;
>
> me.items = [];
> - me.advancedItems = [
> - {
> - xtype: 'proxmoxcheckbox',
> - name: 'dhcp',
> - inputValue: 'dnsmasq',
> - uncheckedValue: null,
> - checked: false,
> - fieldLabel: gettext('Automatic DHCP'),
> - deleteEmpty: !me.isCreate,
> - },
> - ];
>
> me.callParent();
> },
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-manager v3 16/17] sdn: bring up the DHCP backends at boot before the guests start
2026-10-20 8:00 ` [PATCH pve-manager v3 16/17] sdn: bring up the DHCP backends at boot before the guests start Hannes Laimer
@ 2026-10-20 13:21 ` Stefan Hanreich
2026-10-21 7:17 ` Hannes Laimer
0 siblings, 1 reply; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:21 UTC (permalink / raw)
To: pve-devel
On 10/20/26 10:03 AM, Hannes Laimer wrote:
> The eBPF DHCP responder keeps its programs and maps pinned in bpffs,
> so they are gone after a reboot. The SDN commit helper now brings the
> DHCP backends back up on every boot, not only when it has changes to
> commit.
>
> The helper runs before the node and HA start the guests at boot.
> Otherwise the first guest interface plug loads the responder with a
> full pass, which holds the responder's lock exclusively. A guest
> starting meanwhile waits for that lock. One that waits too long comes
> up without the responder on its interface.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> bin/pve-sdn-commit | 3 +++
> services/pve-sdn-commit.service | 1 +
> 2 files changed, 4 insertions(+)
>
> diff --git a/bin/pve-sdn-commit b/bin/pve-sdn-commit
> index aa7e9b29..ed2d6935 100644
> --- a/bin/pve-sdn-commit
> +++ b/bin/pve-sdn-commit
> @@ -96,6 +96,9 @@ sub sdn_changed {
> return fabrics_changed();
> }
>
> +# the dhcp backends bring back what a reboot took, pending changes or not
> +PVE::Network::SDN::Dhcp::boot();
Add some form of log message here. I've tested this and it worked but I'd
like at least some form of confirmation that this ran successfully?
> +
> if (!sdn_changed()) {
> print "No changes to SDN configuration detected, skipping reload\n";
> exit 0;
> diff --git a/services/pve-sdn-commit.service b/services/pve-sdn-commit.service
> index ff723725..825926be 100644
> --- a/services/pve-sdn-commit.service
> +++ b/services/pve-sdn-commit.service
> @@ -3,6 +3,7 @@ Description=Commit Proxmox VE SDN changes
> DefaultDependencies=no
> Wants=pve-cluster.service network.target
> After=frr.service network.target corosync.service
> +Before=pve-guests.service pve-ha-lrm.service
>
> [Service]
> ExecStart=/usr/share/pve-manager/helpers/pve-sdn-commit
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-container v3 14/17] net: report NIC plugs and removals to the SDN DHCP backends
2026-10-20 8:00 ` [PATCH pve-container v3 14/17] net: report NIC plugs and removals to the SDN DHCP backends Hannes Laimer
@ 2026-10-20 13:36 ` Stefan Hanreich
2026-10-21 7:29 ` Hannes Laimer
0 siblings, 1 reply; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:36 UTC (permalink / raw)
To: pve-devel
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
On 10/20/26 10:02 AM, Hannes Laimer wrote:
> An SDN DHCP backend answers a guest NIC with the address IPAM holds
> for its MAC. A NIC joining its bridge now hands its MAC to the SDN
> plug, so the backend of its vnet can answer its first request.
> Deleting a veth also reports it as gone to the backends, since their
> state for an interface can outlive it.
>
> On a running container, a bridge change now pushes the DHCP mapping
> for the new vnet, as adding a NIC or changing its MAC already does.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> src/PVE/LXC.pm | 13 +++++++++++--
> src/PVE/LXC/Config.pm | 3 ++-
> src/lxc-pve-poststop-hook | 6 +++---
> 3 files changed, 16 insertions(+), 6 deletions(-)
>
> diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
> index 1f11490..c3cf9e7 100644
> --- a/src/PVE/LXC.pm
> +++ b/src/PVE/LXC.pm
> @@ -1111,7 +1111,13 @@ sub net_tap_plug : prototype($$) {
> my $create_firewall_bridges = $net->{firewall} && PVE::Firewall::Helpers::needs_fwbr($bridge);
>
> PVE::Network::SDN::Zones::tap_plug(
> - $iface, $bridge, $tag, $create_firewall_bridges, $trunks, $rate,
> + $iface,
> + $bridge,
> + $tag,
> + $create_firewall_bridges,
> + $trunks,
> + $rate,
> + { mac => $hwaddr },
nit: I understand that this function already has quite a few arguments, but introducing a new
$opts and then setting the MAC there seems somewhat awkward as well? A matter of taste though, I
suppose..
> );
> PVE::Network::SDN::Zones::add_bridge_fdb($iface, $hwaddr, $bridge);
>
> @@ -1137,7 +1143,7 @@ sub update_net {
> || safe_string_ne($oldnet->{name}, $newnet->{name})
> ) {
>
> - PVE::Network::veth_delete($veth);
> + PVE::Network::SDN::Zones::veth_delete($veth);
>
> if (safe_string_ne($oldnet->{hwaddr}, $newnet->{hwaddr})) {
> eval {
> @@ -1192,6 +1198,9 @@ sub update_net {
> PVE::Network::SDN::Vnets::add_next_free_cidr(
> $newnet->{bridge}, $conf->{hostname}, $newnet->{hwaddr}, $vmid, undef, 1,
> );
> + PVE::Network::SDN::Vnets::add_dhcp_mapping(
> + $newnet->{bridge}, $newnet->{hwaddr}, $vmid, $conf->{hostname},
> + );
> }
> eval { PVE::LXC::net_tap_plug($veth, $newnet) };
> if (my $err = $@) {
> diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
> index 9823d00..b522413 100644
> --- a/src/PVE/LXC/Config.pm
> +++ b/src/PVE/LXC/Config.pm
> @@ -12,6 +12,7 @@ use PVE::GuestHelpers;
> use PVE::INotify;
> use PVE::JSONSchema qw(get_standard_option);
> use PVE::Network::SDN::Vnets;
> +use PVE::Network::SDN::Zones;
> use PVE::RESTEnvironment;
> use PVE::Tools;
>
> @@ -1696,7 +1697,7 @@ sub vmconfig_hotplug_pending {
> my $net = parse_lxc_network($conf->{$opt});
> PVE::LXC::kill_dhclients($vmid, $net->{name}) if $net->{'host-managed'};
>
> - PVE::Network::veth_delete("veth${vmid}i$netid");
> + PVE::Network::SDN::Zones::veth_delete("veth${vmid}i$netid");
> print "delete ips from $opt\n";
> eval {
> PVE::Network::SDN::Vnets::del_ips_from_mac(
> diff --git a/src/lxc-pve-poststop-hook b/src/lxc-pve-poststop-hook
> index c8bbb79..c5d1059 100755
> --- a/src/lxc-pve-poststop-hook
> +++ b/src/lxc-pve-poststop-hook
> @@ -11,7 +11,7 @@ use PVE::GuestHelpers;
> use PVE::LXC::Config;
> use PVE::LXC::Tools;
> use PVE::LXC;
> -use PVE::Network;
> +use PVE::Network::SDN::Zones;
> use PVE::RESTEnvironment;
> use PVE::Storage;
> use PVE::Tools;
> @@ -49,8 +49,8 @@ PVE::LXC::Tools::lxc_hook(
> my $ind = $1;
> my $net = PVE::LXC::Config->parse_lxc_network($conf->{$k});
> next if $net->{type} ne 'veth';
> - # veth_delete tests with '-d /sys/class/net/$name' before running the command
> - PVE::Network::veth_delete("veth${vmid}i$ind");
> + # reports the unplug and deletes the veth only where it still exists
> + PVE::Network::SDN::Zones::veth_delete("veth${vmid}i$ind");
> }
>
> my $config_updated = 0;
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH qemu-server v3 13/17] network: sdn: keep the DHCP backends current on guest NICs
2026-10-20 8:00 ` [PATCH qemu-server v3 13/17] network: sdn: keep the DHCP backends current on guest NICs Hannes Laimer
@ 2026-10-20 13:38 ` Stefan Hanreich
0 siblings, 0 replies; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:38 UTC (permalink / raw)
To: pve-devel
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
On 10/20/26 10:02 AM, Hannes Laimer wrote:
> A DHCP backend that serves the guest's tap itself has to hear of
> every plug and removal of that tap. The plug now carries the NIC's
> MAC, so the backend can look up the NIC's record. Removing a tap now
> goes through the SDN layer as well, so the backend drops its entry
> for that tap.
>
> A bridge change allocated the NIC's address in the new vnet, but
> unlike a start or a hotplug it never pushed the mapping to the
> backend of that vnet's zone. It now does.
>
> During a live migration the cluster lists the guest on the source
> until its config moves. A mapping edited in that window is sent to
> the source, while the target keeps the record it plugged the NIC
> with. When the target resumes the guest after the migration, it
> first pushes the current mapping of every NIC.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> src/PVE/CLI/qm.pm | 4 ++--
> src/PVE/QemuServer.pm | 4 ++++
> src/PVE/QemuServer/Network.pm | 19 +++++++++++++++++--
> src/PVE/QemuServer/RunState.pm | 2 ++
> src/usr/pve-bridge | 8 +++++++-
> src/usr/pve-bridgedown | 4 ++--
> 6 files changed, 34 insertions(+), 7 deletions(-)
>
> diff --git a/src/PVE/CLI/qm.pm b/src/PVE/CLI/qm.pm
> index b903c1f1..6c53581a 100755
> --- a/src/PVE/CLI/qm.pm
> +++ b/src/PVE/CLI/qm.pm
> @@ -22,7 +22,7 @@ use PVE::GuestHelpers;
> use PVE::GuestImport::OVF;
> use PVE::INotify;
> use PVE::JSONSchema qw(get_standard_option);
> -use PVE::Network;
> +use PVE::Network::SDN::Zones;
> use PVE::RPCEnvironment;
> use PVE::SafeSyslog;
> use PVE::Tools qw(extract_param file_get_contents);
> @@ -1143,7 +1143,7 @@ __PACKAGE__->register_method({
> foreach my $opt (keys %$conf) {
> next if $opt !~ m/^net(\d+)$/;
> my $interface = $1;
> - PVE::Network::tap_unplug("tap${vmid}i${interface}");
> + PVE::Network::SDN::Zones::tap_unplug("tap${vmid}i${interface}");
> }
> }
>
> diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
> index 135c993f..39bdfc8b 100644
> --- a/src/PVE/QemuServer.pm
> +++ b/src/PVE/QemuServer.pm
> @@ -4975,6 +4975,9 @@ sub vmconfig_update_net {
> PVE::Network::SDN::Vnets::add_next_free_cidr(
> $newnet->{bridge}, $conf->{name}, $newnet->{macaddr}, $vmid, undef, 1,
> );
> + PVE::Network::SDN::Vnets::add_dhcp_mapping(
> + $newnet->{bridge}, $newnet->{macaddr}, $vmid, $conf->{name},
> + );
> }
>
> PVE::QemuServer::Network::tap_plug(
> @@ -4984,6 +4987,7 @@ sub vmconfig_update_net {
> $newnet->{firewall},
> $newnet->{trunks},
> $newnet->{rate},
> + { mac => $newnet->{macaddr} },
> );
>
> } elsif (safe_num_ne($oldnet->{rate}, $newnet->{rate})) {
> diff --git a/src/PVE/QemuServer/Network.pm b/src/PVE/QemuServer/Network.pm
> index b4893c01..e95fa62d 100644
> --- a/src/PVE/QemuServer/Network.pm
> +++ b/src/PVE/QemuServer/Network.pm
> @@ -8,6 +8,7 @@ use PVE::Firewall::Helpers;
> use PVE::JSONSchema qw(get_standard_option parse_property_string);
> use PVE::Network::SDN::Vnets;
> use PVE::Network::SDN::Zones;
> +use PVE::Network::SDN::Dhcp;
> use PVE::RESTEnvironment qw(log_warn);
> use PVE::Tools qw($IPV6RE file_read_firstline);
>
> @@ -272,6 +273,20 @@ sub add_random_macs {
> }
> }
>
> +# an edit made while the VM migrated went to the source, which the vmlist
> +# still named. The target pushes every NIC's current mapping once it runs here
> +# for good
> +sub add_nets_dhcp_mapping {
> + my ($conf, $vmid) = @_;
> +
> + for my $opt (keys %$conf) {
> + next if $opt !~ m/^net\d+$/;
> + my $net = parse_net($conf->{$opt}, 1) or next;
> + next if !$net->{bridge} || !$net->{macaddr};
> + PVE::Network::SDN::Dhcp::update_mapping($net->{bridge}, $net->{macaddr}, $vmid);
> + }
> +}
> +
> sub add_nets_bridge_fdb {
> my ($conf, $vmid) = @_;
>
> @@ -349,10 +364,10 @@ sub delete_ifaces_ipams_ips {
> }
>
> sub tap_plug {
> - my ($iface, $bridge, $tag, $firewall, $trunks, $rate) = @_;
> + my ($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts) = @_;
same here w.r.t. $opts
>
> $firewall = $firewall && PVE::Firewall::Helpers::needs_fwbr($bridge);
> - PVE::Network::SDN::Zones::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate);
> + PVE::Network::SDN::Zones::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts);
> }
>
> sub get_nets_host_mtu {
> diff --git a/src/PVE/QemuServer/RunState.pm b/src/PVE/QemuServer/RunState.pm
> index 889ddd28..0fdd8807 100644
> --- a/src/PVE/QemuServer/RunState.pm
> +++ b/src/PVE/QemuServer/RunState.pm
> @@ -181,6 +181,8 @@ sub vm_resume($vmid, $skiplock = undef, $nocheck = undef) {
>
> PVE::QemuServer::Network::add_nets_bridge_fdb($conf, $vmid)
> if $resume_cmd eq 'cont';
> + PVE::QemuServer::Network::add_nets_dhcp_mapping($conf, $vmid)
> + if $nocheck && $resume_cmd eq 'cont';
>
> mon_cmd($vmid, $resume_cmd);
> },
> diff --git a/src/usr/pve-bridge b/src/usr/pve-bridge
> index c7260c6b..20724271 100755
> --- a/src/usr/pve-bridge
> +++ b/src/usr/pve-bridge
> @@ -43,7 +43,13 @@ die "unable to parse network config '$netid'\n" if !$net;
> PVE::Network::SDN::Vnets::add_dhcp_mapping($net->{bridge}, $net->{macaddr}, $vmid, $conf->{name});
> PVE::Network::SDN::Zones::tap_create($iface, $net->{bridge});
> PVE::QemuServer::Network::tap_plug(
> - $iface, $net->{bridge}, $net->{tag}, $net->{firewall}, $net->{trunks}, $net->{rate},
> + $iface,
> + $net->{bridge},
> + $net->{tag},
> + $net->{firewall},
> + $net->{trunks},
> + $net->{rate},
> + { mac => $net->{macaddr} },
> );
>
> exit 0;
> diff --git a/src/usr/pve-bridgedown b/src/usr/pve-bridgedown
> index e867b640..79a3722b 100755
> --- a/src/usr/pve-bridgedown
> +++ b/src/usr/pve-bridgedown
> @@ -2,7 +2,7 @@
>
> use strict;
> use warnings;
> -use PVE::Network;
> +use PVE::Network::SDN::Zones;
>
> my $iface = shift;
>
> @@ -11,6 +11,6 @@ die "no interface specified\n" if !$iface;
> die "got strange interface name '$iface'\n"
> if $iface !~ m/^tap(\d+)i(\d+)$/;
>
> -PVE::Network::tap_unplug($iface);
> +PVE::Network::SDN::Zones::tap_unplug($iface);
>
> exit 0;
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-network v3 05/17] sdn: dhcp: update reservations when ip mappings change
2026-10-20 7:59 ` [PATCH pve-network v3 05/17] sdn: dhcp: update reservations when ip mappings change Hannes Laimer
@ 2026-10-20 13:42 ` Stefan Hanreich
2026-10-21 7:21 ` Hannes Laimer
0 siblings, 1 reply; 43+ messages in thread
From: Stefan Hanreich @ 2026-10-20 13:42 UTC (permalink / raw)
To: pve-devel
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
On 10/20/26 10:03 AM, Hannes Laimer wrote:
> Creating, updating or deleting an IP mapping through the API changed
> the IPAM and the MAC cache, but never the zone's dnsmasq reservation
> file. dnsmasq kept serving the old reservation, or none for a new
> mapping, until the guest's next start. Deleting a guest NIC's
> addresses left its reservation in the file as well.
>
> A guest start, an API edit and the deletion of a NIC's addresses now
> all rewrite the MAC's reservation from the cache, on the node the
> change is made on. The other nodes catch up when the guest starts
> there. The zone and its backend come from the applied config, and a
> zone confined to other nodes is skipped. A failed update only warns,
> since the IPAM records are already written.
>
> The rewrite reads the cache only once it holds the file's lock. A
> read from before could undo a rewrite that ran in between. The lock
> is on a separate file, since a lock on the reservation file would not
> survive the rename that replaces it. The MAC's line is written in full
> from the cache, so a guest gaining an IPv6 address next to its IPv4
> one gets it reserved too. MACs and addresses are compared in lower
> case, since the cache keeps each as its writer spelled it. A cache
> entry without any address no longer keeps a stale line.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> src/PVE/API2/Network/SDN/Ips.pm | 7 +
> src/PVE/Network/SDN/Dhcp.pm | 48 +++---
> src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 110 ++++++++----
> src/PVE/Network/SDN/Dhcp/Plugin.pm | 7 +-
> src/PVE/Network/SDN/Vnets.pm | 5 +-
> src/test/run_test_vnets_blackbox.pl | 250 +++++++++++++++++++++++++++-
> 6 files changed, 352 insertions(+), 75 deletions(-)
>
> diff --git a/src/PVE/API2/Network/SDN/Ips.pm b/src/PVE/API2/Network/SDN/Ips.pm
> index 5ff05e75..d7b682dd 100644
> --- a/src/PVE/API2/Network/SDN/Ips.pm
> +++ b/src/PVE/API2/Network/SDN/Ips.pm
> @@ -46,6 +46,8 @@ __PACKAGE__->register_method({
> eval { PVE::Network::SDN::Vnets::del_ip($vnet, $ip, '', $mac); };
> die "$@\n" if $@;
>
> + PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
> +
> return undef;
> },
> });
> @@ -82,6 +84,8 @@ __PACKAGE__->register_method({
>
> PVE::Network::SDN::Vnets::add_ip($vnet, $ip, '', $mac, undef);
>
> + PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
> +
> return undef;
> },
> });
> @@ -132,6 +136,9 @@ __PACKAGE__->register_method({
> }
>
> die "$error\n" if $error;
> +
> + PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
> +
> return undef;
> },
> });
> diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
> index 65e40d40..ab5d6b5b 100644
> --- a/src/PVE/Network/SDN/Dhcp.pm
> +++ b/src/PVE/Network/SDN/Dhcp.pm
> @@ -6,7 +6,6 @@ use warnings;
> use PVE::Cluster;
>
> use PVE::Network::SDN;
> -use PVE::Network::SDN::Ipams;
> use PVE::Network::SDN::Subnets;
> use PVE::Network::SDN::Dhcp::Plugin;
> use PVE::Network::SDN::Dhcp::Dnsmasq;
> @@ -22,37 +21,32 @@ sub plugin_types {
> return PVE::Network::SDN::Dhcp::Plugin->lookup_types();
> }
>
> -sub add_mapping {
> - my ($vnetid, $mac, $ip4, $ip6) = @_;
> +# re-apply a MAC's mapping on this node from the current records, best effort
> +# since the record write stays authoritative. A caller that knows the guest
> +# passes its vmid, so only that guest's config is read
> +sub update_mapping {
> + my ($vnetid, $mac, $vmid) = @_;
>
> - my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid);
> - return if !$vnet;
> + # a record without a MAC has no guest NIC behind it, so no backend serves it
> + return if !$mac;
>
> - my $zoneid = $vnet->{zone};
> - my $zone = PVE::Network::SDN::Zones::get_zone($zoneid);
> + eval {
> + # the backend to push to is the applied one, since a zone edited but
> + # not applied has nothing running yet
> + my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid, 1);
> + return if !$vnet;
>
> - return if !$zone->{ipam} || !$zone->{dhcp};
> + my $zoneid = $vnet->{zone};
> + my $zone = PVE::Network::SDN::Zones::get_zone($zoneid, 1);
> + return if !$zone || !$zone->{ipam} || !$zone->{dhcp};
>
> - my $dhcptype = $zone->{dhcp};
> + # a zone confined to other nodes has no backend running here
> + return if defined($zone->{nodes}) && !$zone->{nodes}->{ PVE::INotify::nodename() };
>
> - my $macdb = PVE::Network::SDN::Ipams::read_macdb();
> - my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($dhcptype);
> - $dhcp_plugin->add_ip_mapping($zoneid, $macdb, $mac, $ip4, $ip6);
> -}
> -
> -sub remove_mapping {
> - my ($vnetid, $mac) = @_;
> -
> - my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid);
> - return if !$vnet;
> -
> - my $zoneid = $vnet->{zone};
> - my $zone = PVE::Network::SDN::Zones::get_zone($zoneid);
> -
> - return if !$zone->{ipam} || !$zone->{dhcp};
> -
> - my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($zone->{dhcp});
> - $dhcp_plugin->del_ip_mapping($zoneid, $mac);
> + PVE::Network::SDN::Dhcp::Plugin->lookup($zone->{dhcp})
> + ->update_ip_mapping($zoneid, $mac, $vmid);
> + };
> + warn "could not update dhcp mapping for $mac: $@" if $@;
> }
>
> sub regenerate_config {
> diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
> index 5c9496c0..57ce1f73 100644
> --- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
> +++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
> @@ -12,6 +12,7 @@ use File::Copy;
> use Net::DBus;
>
> use PVE::RESTEnvironment qw(log_warn);
> +use PVE::Network::SDN::Ipams;
>
> my $DNSMASQ_CONFIG_ROOT = '/etc/dnsmasq.d';
> my $DNSMASQ_DEFAULT_ROOT = '/etc/default';
> @@ -51,29 +52,65 @@ sub update_lease {
> $manager->AddDhcpLease($ip4, $mac, \@hostname, undef, 0, 0, 0) if $ip4;
> }
>
> -sub add_ip_mapping {
> - my ($class, $dhcpid, $macdb, $mac, $ip4, $ip6) = @_;
> +sub update_ip_mapping {
> + my ($class, $dhcpid, $mac) = @_;
>
> my $ethers_file = ethers_file($dhcpid);
> my $ethers_tmp_file = "$ethers_file.tmp";
>
> - my $reload = undef;
> + my ($ip4, $ip6);
> + my $reservation = undef;
> + my $changed = undef;
> +
> + my $rewriteFn = sub {
> + # read under the ethers lock. A snapshot from before it would sweep a
> + # line another push wrote meanwhile, and an older push would put its
> + # addresses back
> +
> + my $macdb = PVE::Network::SDN::Ipams::read_macdb();
> + # the cache spells a MAC and an address as their writer did, while the
> + # file holds one spelling of both. An entry without an address says
> + # nothing about its MAC, so a line of it is stale
> + my %macs;
> + for my $key (sort keys $macdb->{macs}->%*) {
> + my $entry = $macdb->{macs}->{$key};
> + my %addresses =
> + map { $_ => lc($entry->{$_}) } grep { defined($entry->{$_}) } qw(ip4 ip6);
> + next if !%addresses;
> + $macs{ lc($key) } = { %{ $macs{ lc($key) } // {} }, %addresses };
> + }
we could simplify this here as well with integrating case-folding into the read_macdb fn.
The more I think about, the more I'd prefer a small object that provides an API similar
to HashMaps that handle case-folding when querying / inserting / ... I think that'd save
us lots of duplication.
> +
> + my $entry = $macs{ lc($mac) };
> + ($ip4, $ip6) = ($entry->{ip4}, $entry->{ip6}) if $entry;
> + if ($ip4 || $ip6) {
> + $reservation = $mac;
> + $reservation .= ",$ip4" if $ip4;
> + $reservation .= ",[$ip6]" if $ip6;
> + # an external IPAM spells addresses its own way, while the file holds one spelling
> + $reservation = lc($reservation);
> + }
>
> - my $appendFn = sub {
> + # the first reservation of a zone finds no file yet
> + file_set_contents($ethers_file, '') if !-e $ethers_file;
> open(my $in, '<', $ethers_file) or die "Could not open file '$ethers_file' $!\n";
> open(my $out, '>', $ethers_tmp_file)
> or die "Could not open file '$ethers_tmp_file' $!\n";
>
> - my $match = undef;
> + my $kept = undef;
>
> - my $line_no = 0;
> while (my $line = <$in>) {
> - $line_no++;
> chomp($line);
> + next if $line !~ m/\S/;
> my ($parsed_mac, $parsed_ip1, $parsed_ip2) = split(/,/, $line);
>
> - if (!defined($parsed_mac)) {
> - warn "failed to parse MAC from $dhcpid ethers file on line $line_no: '$line'\n";
> + # the MAC's own reservation is rewritten. An unchanged one stays in place
> + if (lc($parsed_mac) eq lc($mac)) {
> + if (defined($reservation) && !$kept && lc($line) eq $reservation) {
> + $kept = 1;
> + print $out "$line\n";
> + } else {
> + $changed = 1;
> + }
> next;
> }
>
> @@ -81,50 +118,43 @@ sub add_ip_mapping {
> if ($parsed_ip2) {
> $parsed_ip4 = $parsed_ip1;
> $parsed_ip6 = $parsed_ip2;
> - } elsif (Net::IP::ip_is_ipv4($parsed_ip1)) {
> + } elsif (defined($parsed_ip1) && Net::IP::ip_is_ipv4($parsed_ip1)) {
> $parsed_ip4 = $parsed_ip1;
> - } else {
> + } elsif (defined($parsed_ip1)) {
> $parsed_ip6 = $parsed_ip1;
> }
> $parsed_ip6 = $1 if $parsed_ip6 && $parsed_ip6 =~ m/\[(\S+)\]/;
> + # a line from before addresses were lower-cased may spell one in upper case
> + $parsed_ip6 = lc($parsed_ip6) if $parsed_ip6;
>
> # a line that no longer matches the cache is replaced by the cache's
> # current one. Dropping it left that guest unknown to dnsmasq until
> # its next start
> if (
> - !defined($macdb->{macs}->{$parsed_mac})
> + !defined($macs{ lc($parsed_mac) })
> || ($parsed_ip4
> - && $macdb->{macs}->{$parsed_mac}->{'ip4'}
> - && $macdb->{macs}->{$parsed_mac}->{'ip4'} ne $parsed_ip4)
> + && $macs{ lc($parsed_mac) }->{'ip4'}
> + && $macs{ lc($parsed_mac) }->{'ip4'} ne $parsed_ip4)
> || ($parsed_ip6
> - && $macdb->{macs}->{$parsed_mac}->{'ip6'}
> - && $macdb->{macs}->{$parsed_mac}->{'ip6'} ne $parsed_ip6)
> + && $macs{ lc($parsed_mac) }->{'ip6'}
> + && $macs{ lc($parsed_mac) }->{'ip6'} ne $parsed_ip6)
> ) {
> - $reload = 1;
> - # the MAC being added gets its line below, like a missing one
> - my $current = $macdb->{macs}->{$parsed_mac};
> - next if $parsed_mac eq $mac || !$current;
> - my $replacement = $parsed_mac;
> + $changed = 1;
> + my $current = $macs{ lc($parsed_mac) };
> + next if !$current;
> + my $replacement = lc($parsed_mac);
a dedicated class could simplify this here a lot
> $replacement .= ",$current->{ip4}" if $current->{ip4};
> $replacement .= ",[$current->{ip6}]" if $current->{ip6};
> - print $out "$replacement\n" if $replacement ne $parsed_mac;
> + print $out "$replacement\n";
> next;
> }
>
> - if ($parsed_mac eq $mac) {
> - $match = 1 if $ip4 && $parsed_ip4 && $ip4;
> - $match = 1 if $ip6 && $parsed_ip6 && $ip6;
> - }
> -
> print $out "$line\n";
> }
>
> - if (!$match) {
> - my $reservation = $mac;
> - $reservation .= ",$ip4" if $ip4;
> - $reservation .= ",[$ip6]" if $ip6;
> + if (defined($reservation) && !$kept) {
> print $out "$reservation\n";
> - $reload = 1;
> + $changed = 1;
> }
>
> close $in;
> @@ -133,16 +163,22 @@ sub add_ip_mapping {
> chmod 0644, $ethers_file;
> };
>
> - PVE::Tools::lock_file($ethers_file, 10, $appendFn);
> + # the rewrite replaces the reservation file, so a lock on the file itself
> + # would not outlive the rename
> + PVE::Tools::lock_file("$ethers_file.lock", 10, $rewriteFn);
>
> if ($@) {
> - warn "Unable to add $mac to the dnsmasq configuration: $@\n";
> + warn "Unable to update $mac in the dnsmasq configuration: $@\n";
> return;
> }
>
> - my $service_name = "dnsmasq\@$dhcpid";
> - systemctl_service('reload', $service_name) if $reload;
> - update_lease($dhcpid, $ip4, $mac);
> + systemctl_service('reload', "dnsmasq\@$dhcpid") if $changed;
> +
> + # the reservation is written, so a lease refresh that fails only warns
> + if ($ip4) {
> + eval { update_lease($dhcpid, $ip4, $mac) };
> + log_warn("could not update the dnsmasq lease of $mac: $@") if $@;
> + }
> }
>
> sub configure_subnet {
> diff --git a/src/PVE/Network/SDN/Dhcp/Plugin.pm b/src/PVE/Network/SDN/Dhcp/Plugin.pm
> index b5d32fa1..2fa8683c 100644
> --- a/src/PVE/Network/SDN/Dhcp/Plugin.pm
> +++ b/src/PVE/Network/SDN/Dhcp/Plugin.pm
> @@ -22,8 +22,11 @@ sub private {
> return $defaultData;
> }
>
> -sub add_ip_mapping {
> - my ($class, $dhcpid, $macdb, $mac, $ip4, $ip6) = @_;
> +# the MAC's records changed, so make the backend serve the current ones. With no
> +# record left its reservation goes. The vmid names the guest when the caller
> +# knows it, which an API edit does not
> +sub update_ip_mapping {
> + my ($class, $dhcpid, $mac, $vmid) = @_;
> die 'implement in sub class';
> }
>
> diff --git a/src/PVE/Network/SDN/Vnets.pm b/src/PVE/Network/SDN/Vnets.pm
> index c327a4be..84b08037 100644
> --- a/src/PVE/Network/SDN/Vnets.pm
> +++ b/src/PVE/Network/SDN/Vnets.pm
> @@ -211,6 +211,8 @@ sub del_ips_from_mac {
> PVE::Network::SDN::Vnets::del_ip($vnetid, $ip4, $hostname, $mac) if $ip4;
> PVE::Network::SDN::Vnets::del_ip($vnetid, $ip6, $hostname, $mac) if $ip6;
>
> + PVE::Network::SDN::Dhcp::update_mapping($vnetid, $mac) if $ip4 || $ip6;
> +
> return ($ip4, $ip6);
> }
>
> @@ -225,11 +227,12 @@ sub add_dhcp_mapping {
> return if !$zone->{ipam} || !$zone->{dhcp};
>
> my ($ip4, $ip6) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
> + # one push at the end covers the allocated and the existing records alike
> add_next_free_cidr($vnetid, $name, $mac, "$vmid", undef, 1, 4) if !$ip4;
> add_next_free_cidr($vnetid, $name, $mac, "$vmid", undef, 1, 6) if !$ip6;
>
> ($ip4, $ip6) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
> - PVE::Network::SDN::Dhcp::add_mapping($vnetid, $mac, $ip4, $ip6) if $ip4 || $ip6;
> + PVE::Network::SDN::Dhcp::update_mapping($vnetid, $mac, $vmid) if $ip4 || $ip6;
> }
>
> 1;
> diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
> index 36225a92..d324538e 100755
> --- a/src/test/run_test_vnets_blackbox.pl
> +++ b/src/test/run_test_vnets_blackbox.pl
> @@ -46,6 +46,7 @@ sub clear_test_state {
> vnets_config => {},
> macdb => {},
> ipamdb => {},
> + dnsmasq_calls => [],
> ipam_config => {
> 'ids' => {
> 'pve' => {
> @@ -220,7 +221,9 @@ $mocked_sdn_dhcp_dnsmasq->mock(
> assert_dnsmasq_installed => sub { return 1; },
> before_configure => sub { },
> ethers_file => sub { return "/tmp/ethers"; },
> - systemctl_service => sub { },
> + systemctl_service => sub {
> + push $test_state->{dnsmasq_calls}->@*, $_[0];
> + },
> update_lease => sub { },
> );
>
> @@ -239,6 +242,11 @@ $mocked_rpc_env_obj->mock(
> check_any => sub { return 1; },
> );
>
> +my $mocked_pve_inotify = Test::MockModule->new('PVE::INotify');
> +$mocked_pve_inotify->mock(
> + nodename => sub { return 'localnode'; },
> +);
> +
> my $mocked_pve_cluster_obj = Test::MockModule->new('PVE::Cluster');
> $mocked_pve_cluster_obj->mock(
> check_cfs_quorum => sub { return 1; },
> @@ -288,6 +296,11 @@ sub create_zone {
> return $zone;
> }
>
> +sub update_zone {
> + my ($zoneid, $params) = @_;
> + PVE::API2::Network::SDN::Zones->update({ zone => $zoneid, %$params });
> +}
> +
> sub get_vnet {
> my ($id) = @_;
> return eval { PVE::API2::Network::SDN::Vnets->read({ vnet => $id }); };
> @@ -330,6 +343,16 @@ sub create_ip {
> return PVE::API2::Network::SDN::Ips->ipcreate($param);
> }
>
> +sub update_ip {
> + my ($param) = @_;
> + return PVE::API2::Network::SDN::Ips->ipupdate($param);
> +}
> +
> +sub delete_ip {
> + my ($param) = @_;
> + return PVE::API2::Network::SDN::Ips->ipdelete($param);
> +}
> +
> sub run_test {
> my $test = shift;
> clear_test_state();
> @@ -977,18 +1000,229 @@ sub test_dnsmasq_stale_reservation_replaced {
> };
> PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "$stale,10.0.1.10\n$gone,10.0.1.12\n");
>
> - PVE::Network::SDN::Dhcp::Dnsmasq->add_ip_mapping(
> - 'TESTZONE', $test_state->{macdb}, $added, '10.0.1.13', undef,
> - );
> + PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping('TESTZONE', $added);
>
> - my @lines = sort grep { /\S/ } split(/\n/, PVE::Tools::file_get_contents($TMP_ETHERS_FILE));
> - eq_or_diff(
> - \@lines,
> - ["$stale,10.0.1.11", "$added,10.0.1.13"],
> + is(
> + PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
> + "$stale,10.0.1.11\n$added,10.0.1.13\n",
> "$test_name: the stale line is replaced, the unknown one dropped",
> );
> }
>
> run_test(\&test_dnsmasq_stale_reservation_replaced);
>
> +# -------------- dnsmasq mapping pushes
> +
> +sub test_dnsmasq_mapping_push {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> + my $zoneid = "TESTZONE";
> + my $vnetid = "testvnet";
> + my $mac = "da:65:8f:18:9b:6f";
> +
> + create_zone({
> + type => "simple",
> + dhcp => "dnsmasq",
> + ipam => "pve",
> + zone => $zoneid,
> + });
> + create_vnet({
> + type => "vnet",
> + zone => $zoneid,
> + vnet => $vnetid,
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/24",
> + gateway => "10.0.0.1",
> + 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
> + });
> +
> + # the first push of a zone finds no ethers file yet
> + unlink($TMP_ETHERS_FILE);
> +
> + my $ethers = sub { PVE::Tools::file_get_contents($TMP_ETHERS_FILE) };
> + my $reloads = sub {
> + my $count = scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*);
> + $test_state->{dnsmasq_calls} = [];
> + return $count;
> + };
> +
> + eval { nic_start($vnetid, $mac, "999", "testhostname"); };
> + if ($@) {
> + fail("$test_name: nic_start: $@");
> + return;
> + }
> + is($ethers->(), "$mac,10.0.0.100\n", "$test_name: guest start reserves the allocation");
> + is($reloads->(), 1, "$test_name: guest start reloads once");
> +
> + # the records exist already, so the restart pushes them without a change
> + nic_start($vnetid, $mac, "999", "testhostname");
> + is($ethers->(), "$mac,10.0.0.100\n", "$test_name: restart keeps the reservation");
> + is($reloads->(), 0, "$test_name: restart without a change does not reload");
> +
> + update_ip({
> + zone => $zoneid,
> + vnet => $vnetid,
> + mac => $mac,
> + ip => "10.0.0.150",
> + });
> + is($ethers->(), "$mac,10.0.0.150\n", "$test_name: mapping edit replaces the reservation");
> + is($reloads->(), 1, "$test_name: mapping edit reloads once");
> +
> + # a push writes what the cache holds, so a repeated one rewrites nothing
> + PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
> + is($ethers->(), "$mac,10.0.0.150\n", "$test_name: a repeated push keeps the reservation");
> + is($reloads->(), 0, "$test_name: a repeated push changes nothing");
> +
> + # the cache and the file may spell a MAC differently, but it is one MAC
> + PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, uc($mac));
> + is(
> + $ethers->(),
> + "$mac,10.0.0.150\n",
> + "$test_name: a push spelled differently keeps the line as it is",
> + );
> + is($reloads->(), 0, "$test_name: and reloads nothing");
> +
> + delete_ip({
> + zone => $zoneid,
> + vnet => $vnetid,
> + mac => $mac,
> + ip => "10.0.0.150",
> + });
> + is($ethers->(), "", "$test_name: mapping delete drops the reservation");
> + is($reloads->(), 1, "$test_name: mapping delete reloads once");
> +
> + # a record without a MAC has no NIC to serve, so nothing is pushed for it
> + my @warnings;
> + {
> + local $SIG{__WARN__} = sub { push @warnings, @_; };
> + create_ip({ zone => $zoneid, vnet => $vnetid, ip => "10.0.0.160" });
> + delete_ip({ zone => $zoneid, vnet => $vnetid, ip => "10.0.0.160" });
> + }
> + is($reloads->(), 0, "$test_name: a record without a MAC is not pushed");
> + eq_or_diff(\@warnings, [], "$test_name: and warns nothing");
> +
> + # a zone confined to other nodes runs no dnsmasq here, so an edit is not
> + # pushed on this node
> + update_zone($zoneid, { nodes => 'other' });
> + create_ip({
> + zone => $zoneid,
> + vnet => $vnetid,
> + mac => $mac,
> + ip => "10.0.0.150",
> + });
> + is($ethers->(), "", "$test_name: a zone confined to other nodes gets no push here");
> + is($reloads->(), 0, "$test_name: nothing to reload for it");
> +}
> +
> +run_test(\&test_dnsmasq_mapping_push);
> +
> +sub test_dnsmasq_dual_stack_and_sweep {
> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
> + my $zoneid = "TESTZONE";
> + my $vnetid = "testvnet";
> + my $mac = "da:65:8f:18:9b:6f";
> +
> + create_zone({
> + type => "simple",
> + dhcp => "dnsmasq",
> + ipam => "pve",
> + zone => $zoneid,
> + });
> + create_vnet({
> + type => "vnet",
> + zone => $zoneid,
> + vnet => $vnetid,
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "10.0.0.0/24",
> + gateway => "10.0.0.1",
> + 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
> + });
> + create_subnet({
> + type => "subnet",
> + vnet => $vnetid,
> + subnet => "8888::/64",
> + gateway => "8888::1",
> + 'dhcp-range' => ["start-address=8888::100,end-address=8888::200"],
> + });
> +
> + # a reservation of a MAC the cache does not know is swept by the next
> + # rewrite, and both addresses of the guest go on one line
> + PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "aa:bb:cc:dd:ee:ff,10.0.0.77\n");
> + $test_state->{dnsmasq_calls} = [];
> +
> + eval { nic_start($vnetid, $mac, "999", "testhostname"); };
> + if ($@) {
> + fail("$test_name: nic_start: $@");
> + return;
> + }
> + is(
> + PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
> + "$mac,10.0.0.100,[8888::100]\n",
> + "$test_name: dual-stack reservation on one line, the unknown MAC swept",
> + );
> + is(
> + scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
> + 1,
> + "$test_name: one reload for the rewrite",
> + );
> +
> + # an external IPAM may spell an address in upper case, but the file holds
> + # one spelling
> + $test_state->{macdb}->{macs}->{$mac}->{ip6} = '8888::AB';
> + $test_state->{dnsmasq_calls} = [];
> + PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
> + is(
> + PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
> + "$mac,10.0.0.100,[8888::ab]\n",
> + "$test_name: the reservation is written in one spelling",
> + );
> + is(
> + scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
> + 1,
> + "$test_name: one reload for the changed address",
> + );
> +
> + # an entry without an address, as a miss once left behind, says nothing
> + # about its MAC, so a line of it is swept
> + my $stale = 'da:65:8f:18:9b:70';
> + $test_state->{macdb}->{macs}->{$stale} = { ip4 => undef, ip6 => undef };
> + PVE::Tools::file_set_contents(
> + $TMP_ETHERS_FILE,
> + "$mac,10.0.0.100,[8888::ab]\n$stale,10.0.0.101\n",
> + );
> + PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
> + is(
> + PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
> + "$mac,10.0.0.100,[8888::ab]\n",
> + "$test_name: a line of a MAC cached without an address is swept",
> + );
> +
> + # a line with an upper-case IPv6 address from before they were lower-cased is kept
> + my $other = 'da:65:8f:18:9b:71';
> + $test_state->{macdb}->{macs}->{$other} = { ip4 => '10.0.0.102', ip6 => '8888::AB' };
> + PVE::Tools::file_set_contents(
> + $TMP_ETHERS_FILE,
> + "$mac,10.0.0.100,[8888::ab]\n$other,10.0.0.102,[8888::AB]\n",
> + );
> + $test_state->{dnsmasq_calls} = [];
> + PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
> + is(
> + PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
> + "$mac,10.0.0.100,[8888::ab]\n$other,10.0.0.102,[8888::AB]\n",
> + "$test_name: a line of another MAC spelled by the old code is kept",
> + );
> + is(
> + scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
> + 0,
> + "$test_name: without a reload",
> + );
> +}
> +
> +run_test(\&test_dnsmasq_dual_stack_and_sweep);
> +
> done_testing();
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-network v3 09/17] sdn: dhcp: add an eBPF backend answering on each guest interface
2026-10-20 13:10 ` Stefan Hanreich
@ 2026-10-21 6:16 ` Hannes Laimer
0 siblings, 0 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-21 6:16 UTC (permalink / raw)
To: Stefan Hanreich, pve-devel
On 2026-10-20 15:11, Stefan Hanreich wrote:
> On 10/20/26 10:04 AM, Hannes Laimer wrote:
>> The ebpf DHCP backend answers DHCPv4 from a program in the kernel on
>> every guest interface of its zones, with the address the SDN MAC cache
>> holds for the interface's MAC. No server runs per zone, and IPv6
>> subnets are not served. A subnet without a gateway is served under a
>> link-local server identifier no host owns, so its guests renew by
>> broadcast at their rebinding time.
>>
>> An apply runs a full pass on each node, and on a node that no zone
>> with the ebpf backend includes, the pass removes the responder
>> altogether. A record change rewrites the answers for its MAC on the
>> node the change is made on, and the other nodes catch up at their
>> next apply.
>>
>> The responder is pinned in bpffs and does not survive a reboot, so a
>> boot hook runs a full pass. Removing the package deletes those pins,
>> since nothing would detach the responder afterward.
>>
>> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
>> ---
[..]
>> +# the NIC lines come from pmxcfs in one go, so no guest config gets parsed for
>> +# this. With a vmid only that guest's lines are fetched
>> +sub guest_nics {
>> + my ($vmid) = @_;
>> +
>> + # a hook script starts with the cluster cache empty
>> + PVE::Cluster::cfs_update();
>> + my $vmlist = PVE::Cluster::get_vmlist();
>> + my $nets = PVE::Cluster::get_guest_config_properties([map { "net$_" } 0 .. 31], $vmid);
>> + my $nics = [];
>> + for my $vmid (sort keys %$nets) {
>> + my $guest = $vmlist->{ids}->{$vmid} // next;
>> + my $prefix = ($guest->{type} // '') eq 'lxc' ? 'veth' : 'tap';
>> + for my $key (sort keys %{ $nets->{$vmid} }) {
>> + my ($index) = $key =~ m/^net(\d+)$/ or next;
>> + my $net = $nets->{$vmid}->{$key};
>> + my ($bridge) = $net =~ m/(?:^|,)bridge=([^,]+)/ or next;
>> + # a VM names the MAC as the value of its model, a container as hwaddr
>> + my ($mac) = $net =~ m/(?:^|,)[a-z0-9_-]+=([0-9a-f]{2}(?::[0-9a-f]{2}){5})(?:,|$)/i;
>
> Is there a reason for not using the pre-existing parse_net /
> parse_lxc_network functions in qemu-server / pve-container here?
> They'd provide a standalone way of parsing the property string only,
> without re-implementing the logic here.
>
> Circular dependencies potentially?
yes, both depend on pve-network :/
the lxc helper also fills the mac if missing, and for
`parse_property_string` we'd need the format
>
>> + push @$nics,
>> + {
>> + vmid => $vmid,
>> + node => $guest->{node},
>> + iface => "$prefix${vmid}i$index",
>> + bridge => $bridge,
>> + mac => $mac,
>> + };
>> + }
>> + }
>> +
>> + return $nics;
>> +}
>> +
[..]
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-network v3 06/17] sdn: ipam: key the MAC cache by lower-case MAC, stop caching misses
2026-10-20 13:10 ` Stefan Hanreich
@ 2026-10-21 6:17 ` Hannes Laimer
0 siblings, 0 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-21 6:17 UTC (permalink / raw)
To: Stefan Hanreich, pve-devel
On 2026-10-20 15:10, Stefan Hanreich wrote:
> On 10/20/26 10:01 AM, Hannes Laimer wrote:
>> Generated guest MACs are upper case, but the API accepts either
>> spelling. The cache was keyed by the MAC as its writer spelled it. A
>> deletion in the other spelling missed the entry, so the cache kept
>> answering with the deleted address. Entries cached before this
>> change keep their spelling. A lookup and a deletion still find them,
>> and the next write for their MAC moves them to the lower-case key.
>>
>> A lookup that missed the cache stored the IPAM's empty answer, and
>> any entry counted as a hit. So a record created later directly in the
>> IPAM stayed hidden, and a guest start allocated another address. An
>> empty answer is no longer stored, and an entry without an address
>> counts as a miss.
>>
>> The lookup also wrote the whole cache back without the cluster lock
>> the other writers take. A change made between its read and its write
>> was lost. The lookup now takes the lock and reads the cache again
>> before it stores its answer.
>>
>> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
>> ---
>> src/PVE/Network/SDN/Ipams.pm | 61 +++++++++++---
>> src/test/run_test_vnets_blackbox.pl | 118 ++++++++++++++++++++++++++++
>> 2 files changed, 168 insertions(+), 11 deletions(-)
>>
>> diff --git a/src/PVE/Network/SDN/Ipams.pm b/src/PVE/Network/SDN/Ipams.pm
>> index 179bdf76..d8051d8d 100644
>> --- a/src/PVE/Network/SDN/Ipams.pm
>> +++ b/src/PVE/Network/SDN/Ipams.pm
>> @@ -47,14 +47,27 @@ sub write_macdb {
>> cfs_write_file($macdb_filename, $data);
>> }
>>
>> +# an entry under another spelling moves to the lower-case key
>> +my sub fold_spellings {
>> + my ($db, $mac) = @_;
>> +
>> + for my $key (grep { $_ ne $mac && lc($_) eq $mac } reverse sort keys $db->{macs}->%*) {
>> + my $old = delete $db->{macs}->{$key};
>> + $db->{macs}->{$mac}->{$_} //= $old->{$_} for grep { defined($old->{$_}) } qw(ip4 ip6);
>> + }
>> +}
>> +
>> +# the cache is keyed by the MAC in lower case, whatever spelling its writers use
>> sub add_cache_mac_ip {
>> my ($mac, $ip) = @_;
>> + $mac = lc($mac);
>>
>> cfs_lock_file(
>> $macdb_filename,
>> undef,
>> sub {
>> my $db = read_macdb();
>> + fold_spellings($db, $mac);
>> if (Net::IP::ip_is_ipv4($ip)) {
>> $db->{macs}->{$mac}->{ip4} = $ip;
>> } else {
>> @@ -74,13 +87,17 @@ sub del_cache_mac_ip {
>> undef,
>> sub {
>> my $db = read_macdb();
>> - if (Net::IP::ip_is_ipv4($ip)) {
>> - delete $db->{macs}->{$mac}->{ip4};
>> - } else {
>> - delete $db->{macs}->{$mac}->{ip6};
>> + # entries from before the cache was keyed in lower case carry their writer's spelling
>> + for my $key (grep { lc($_) eq lc($mac) } keys $db->{macs}->%*) {
>> + if (Net::IP::ip_is_ipv4($ip)) {
>> + delete $db->{macs}->{$key}->{ip4};
>> + } else {
>> + delete $db->{macs}->{$key}->{ip6};
>> + }
>> + delete $db->{macs}->{$key}
>> + if !defined($db->{macs}->{$key}->{ip4})
>> + && !defined($db->{macs}->{$key}->{ip6});
>> }
>> - delete $db->{macs}->{$mac}
>> - if !defined($db->{macs}->{$mac}->{ip4}) && !defined($db->{macs}->{$mac}->{ip6});
>> write_macdb($db);
>> },
>> );
>> @@ -136,16 +153,38 @@ sub get_ips_from_mac {
>> my ($mac, $zoneid, $zone) = @_;
>>
>> my $macdb = read_macdb();
>> - return ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6}) if $macdb->{macs}->{$mac};
>> + # entries from before the cache was keyed in lower case carry their writer's
>> + # spelling, and the lower-case one wins where two hold the same family
>> + my %cached;
>> + for my $key (grep { lc($_) eq lc($mac) } sort keys $macdb->{macs}->%*) {
>> + my $entry = $macdb->{macs}->{$key};
>> + $cached{$_} = $entry->{$_} for grep { defined($entry->{$_}) } qw(ip4 ip6);
>> + }
>> +
>> + # an entry holding no address says nothing about the MAC, so ask the IPAM
>> + return ($cached{ip4}, $cached{ip6}) if defined($cached{ip4}) || defined($cached{ip6});
>>
>> my $plugin_config = get_plugin_config($zone);
>> my $plugin = PVE::Network::SDN::Ipams::Plugin->lookup($plugin_config->{type});
>> - ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6}) =
>> - $plugin->get_ips_from_mac($plugin_config, $mac, $zoneid);
>> + my ($ip4, $ip6) = $plugin->get_ips_from_mac($plugin_config, $mac, $zoneid);
>> +
>> + # an empty answer is not cached, since the record may simply not exist yet
>> + return if !defined($ip4) && !defined($ip6);
>>
>> - write_macdb($macdb);
>> + cfs_lock_file(
>> + $macdb_filename,
>> + undef,
>> + sub {
>> + my $db = read_macdb();
>> + fold_spellings($db, lc($mac));
>> + $db->{macs}->{ lc($mac) }->{ip4} = $ip4 if defined($ip4);
>> + $db->{macs}->{ lc($mac) }->{ip6} = $ip6 if defined($ip6);
>> + write_macdb($db);
>> + },
>> + );
>> + warn "$@" if $@;
>>
>> - return ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6});
>> + return ($ip4, $ip6);
>> }> 1;
>
> would it make sense to move all of this case handling into read_macdb itself? Currently
> we're doing it at every call site of read_macdb in Ipams.pm
>
yes, ill include something in v4!
>> diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
>> index d324538e..476e7f8d 100755
>> --- a/src/test/run_test_vnets_blackbox.pl
>> +++ b/src/test/run_test_vnets_blackbox.pl
>> @@ -1118,6 +1118,124 @@ sub test_dnsmasq_mapping_push {
>>
>> run_test(\&test_dnsmasq_mapping_push);
>>
>> +sub test_ipam_cache_misses {
>> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
>> + my $zoneid = "TESTZONE";
>> + my $vnetid = "testvnet";
>> + my $mac = "da:65:8f:18:9b:6f";
>> +
>> + create_zone({
>> + type => "simple",
>> + dhcp => "dnsmasq",
>> + ipam => "pve",
>> + zone => $zoneid,
>> + });
>> + create_vnet({
>> + type => "vnet",
>> + zone => $zoneid,
>> + vnet => $vnetid,
>> + });
>> + create_subnet({
>> + type => "subnet",
>> + vnet => $vnetid,
>> + subnet => "10.0.0.0/24",
>> + gateway => "10.0.0.1",
>> + 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
>> + });
>> +
>> + my @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
>> + is(scalar(grep { defined } @ips), 0, "$test_name: an unknown MAC has no answer");
>> + ok(!exists $test_state->{macdb}->{macs}->{$mac}, "$test_name: the miss is not cached");
>> +
>> + # an entry holding no address, left behind by an earlier miss, does not
>> + # hide a record created since
>> + create_ip({
>> + zone => $zoneid,
>> + vnet => $vnetid,
>> + mac => $mac,
>> + ip => "10.0.0.50",
>> + });
>> + $test_state->{macdb}->{macs}->{$mac} = { ip4 => undef, ip6 => undef };
>> + @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
>> + is($ips[0], "10.0.0.50", "$test_name: an address-less entry counts as a miss");
>> + is(
>> + $test_state->{macdb}->{macs}->{$mac}->{ip4},
>> + "10.0.0.50",
>> + "$test_name: the answer is cached",
>> + );
>> +}
>> +
>> +run_test(\&test_ipam_cache_misses);
>> +
>> +sub test_ipam_cache_case {
>> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
>> + my $zoneid = "TESTZONE";
>> + my $vnetid = "testvnet";
>> + my $mac = "da:65:8f:18:9b:6f";
>> +
>> + # an entry written under the config's spelling before the cache was keyed in
>> + # lower case is read and released like any other
>> + create_zone({
>> + type => "simple",
>> + dhcp => "dnsmasq",
>> + ipam => "pve",
>> + zone => $zoneid,
>> + });
>> + create_vnet({
>> + type => "vnet",
>> + zone => $zoneid,
>> + vnet => $vnetid,
>> + });
>> + create_subnet({
>> + type => "subnet",
>> + vnet => $vnetid,
>> + subnet => "10.0.0.0/24",
>> + gateway => "10.0.0.1",
>> + 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
>> + });
>> + $test_state->{macdb}->{macs}->{ uc($mac) } = { ip4 => '10.0.0.150' };
>> + my ($ip4) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
>> + is($ip4, '10.0.0.150', "$test_name: an old entry is found however it is spelled");
>> + PVE::Network::SDN::Ipams::del_cache_mac_ip($mac, '10.0.0.150');
>> + ok(
>> + !(grep { lc($_) eq $mac } keys $test_state->{macdb}->{macs}->%*),
>> + "$test_name: and released",
>> + );
>> +
>> + # a write spelled either way moves such an entry to the lower-case key
>> + # and keeps what it held
>> + $test_state->{macdb}->{macs}->{ uc($mac) } = { ip4 => '10.0.0.150' };
>> + PVE::Network::SDN::Ipams::add_cache_mac_ip(uc($mac), 'fd00::150');
>> + is_deeply(
>> + [grep { lc($_) eq $mac } keys $test_state->{macdb}->{macs}->%*],
>> + [$mac],
>> + "$test_name: a write leaves one entry for the MAC",
>> + );
>> + is_deeply(
>> + $test_state->{macdb}->{macs}->{$mac},
>> + { ip4 => '10.0.0.150', ip6 => 'fd00::150' },
>> + "$test_name: holding both addresses",
>> + );
>> +
>> + # the lookup's write moves an address-less entry under another spelling as well
>> + create_ip({
>> + zone => $zoneid,
>> + vnet => $vnetid,
>> + mac => $mac,
>> + ip => "10.0.0.160",
>> + });
>> + $test_state->{macdb}->{macs} = { uc($mac) => { ip4 => undef, ip6 => undef } };
>> + my @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
>> + is($ips[0], '10.0.0.160', "$test_name: a lookup past an address-less entry asks the IPAM");
>> + is_deeply(
>> + [sort keys $test_state->{macdb}->{macs}->%*],
>> + [$mac],
>> + "$test_name: and its write leaves one entry for the MAC",
>> + );
>> +}
>> +
>> +run_test(\&test_ipam_cache_case);
>> +
>> sub test_dnsmasq_dual_stack_and_sweep {
>> my $test_name = (split(/::/, (caller(0))[3]))[-1];
>> my $zoneid = "TESTZONE";
>
>
>
>
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-network v3 11/17] sdn: dhcp: push IP mapping edits to the guest's node
2026-10-20 13:09 ` Stefan Hanreich
@ 2026-10-21 6:44 ` Hannes Laimer
0 siblings, 0 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-21 6:44 UTC (permalink / raw)
To: Stefan Hanreich, pve-devel
On 2026-10-20 15:10, Stefan Hanreich wrote:
> On 10/20/26 10:02 AM, Hannes Laimer wrote:
>> IP mappings live in the cluster-wide IPAM and can be edited through
>> the API on any node. The DHCP backends hold their copy on each node,
>> dnsmasq in a reservation file and the ebpf responder in a kernel map.
>> An edit only updated the backend of the node it was made on, so a
>> guest on another node kept getting the old mapping.
>>
>> The new endpoint POST /nodes/{node}/sdn/dhcp-mapping updates the
>> mapping of one MAC on that node from the current records. An edit
>> made on a node other than the guest's calls it on the guest's node.
>> Only that node answers the guest's DHCP requests, so no other node is
>> called. The others catch up when the guest starts on one of them.
>>
>> The call runs in the background, so the request returns without
>> waiting on the other node. A call that fails logs its error to the
>> journal of the node the edit was made on. One that gets no answer is
>> stopped after a minute. The record stays as written either way, and
>> the guest's node picks it up at the latest when the guest next starts
>> there.
>>
>> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
>> ---
>> src/PVE/API2/Network/SDN/Ips.pm | 3 +
>> src/PVE/API2/Network/SDN/Nodes/Status.pm | 37 ++++++++++
>> src/PVE/Network/SDN/Dhcp.pm | 74 +++++++++++++++++++
>> src/test/run_test_vnets_blackbox.pl | 90 ++++++++++++++++++++++++
>> 4 files changed, 204 insertions(+)
>>
>> diff --git a/src/PVE/API2/Network/SDN/Ips.pm b/src/PVE/API2/Network/SDN/Ips.pm
>> index d7b682dd..5b45de7c 100644
>> --- a/src/PVE/API2/Network/SDN/Ips.pm
>> +++ b/src/PVE/API2/Network/SDN/Ips.pm
>> @@ -47,6 +47,7 @@ __PACKAGE__->register_method({
>> die "$@\n" if $@;
>>
>> PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
>> + PVE::Network::SDN::Dhcp::notify_guest_node($vnet, $mac);
>>
>> return undef;
>> },
>> @@ -85,6 +86,7 @@ __PACKAGE__->register_method({
>> PVE::Network::SDN::Vnets::add_ip($vnet, $ip, '', $mac, undef);
>>
>> PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
>> + PVE::Network::SDN::Dhcp::notify_guest_node($vnet, $mac);
>>
>> return undef;
>> },
>> @@ -138,6 +140,7 @@ __PACKAGE__->register_method({
>> die "$error\n" if $error;
>>
>> PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
>> + PVE::Network::SDN::Dhcp::notify_guest_node($vnet, $mac);
>>
>> return undef;
>> },
>> diff --git a/src/PVE/API2/Network/SDN/Nodes/Status.pm b/src/PVE/API2/Network/SDN/Nodes/Status.pm
>> index 7977e0c8..fe750a85 100644
>> --- a/src/PVE/API2/Network/SDN/Nodes/Status.pm
>> +++ b/src/PVE/API2/Network/SDN/Nodes/Status.pm
>> @@ -9,6 +9,9 @@ use PVE::API2::Network::SDN::Nodes::Vnets;
>>
>> use PVE::JSONSchema qw(get_standard_option);
>>
>> +use PVE::Network::SDN::Vnets;
>> +use PVE::Network::SDN::Dhcp;
>> +
>> use PVE::RESTHandler;
>> use base qw(PVE::RESTHandler);
>>
>> @@ -27,6 +30,40 @@ __PACKAGE__->register_method({
>> path => 'vnets',
>> });
>>
>> +__PACKAGE__->register_method({
>> + name => 'dhcp_mapping',
>> + path => 'dhcp-mapping',
>> + method => 'POST',
>> + description =>
>> + 'Re-apply the DHCP mapping of a MAC address on this node from the current records.',
>> + permissions => {
>> + check => ['perm', '/sdn/zones/{zone}/{vnet}', ['SDN.Allocate']],
>> + },
>> + protected => 1,
>> + proxyto => 'node',
>> + parameters => {
>> + additionalProperties => 0,
>> + properties => {
>> + node => get_standard_option('pve-node'),
>> + zone => get_standard_option('pve-sdn-zone-id'),
>> + vnet => get_standard_option('pve-sdn-vnet-id'),
>> + mac => get_standard_option('mac-addr'),
>> + },
>> + },
>> + returns => { type => 'null' },
>> + code => sub {
>> + my ($param) = @_;
>> +
>> + my $vnet = PVE::Network::SDN::Vnets::get_vnet($param->{vnet}, 1);
>> + die "vnet '$param->{vnet}' does not exist in zone '$param->{zone}'\n"
>> + if !$vnet || $vnet->{zone} ne $param->{zone};
>> +
>> + PVE::Network::SDN::Dhcp::update_mapping($param->{vnet}, $param->{mac});
>> +
>> + return undef;
>> + },
>> +});
>> +
>> __PACKAGE__->register_method({
>> name => 'sdnindex',
>> path => '',
>> diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
>> index cf805a03..1f43745f 100644
>> --- a/src/PVE/Network/SDN/Dhcp.pm
>> +++ b/src/PVE/Network/SDN/Dhcp.pm
>> @@ -3,6 +3,8 @@ package PVE::Network::SDN::Dhcp;
>> use strict;
>> use warnings;
>>
>> +use POSIX qw();
>> +
>> use PVE::Cluster;
>>
>> use PVE::Network::SDN;
>> @@ -86,6 +88,78 @@ sub guest_nics {
>> return $nics;
>> }
>>
>> +sub guest_node_by_mac {
>> + my ($vnetid, $mac) = @_;
>> +
>> + for my $nic (guest_nics()->@*) {
>> + return $nic->{node}
>> + if lc($nic->{mac} // '') eq lc($mac) && $nic->{bridge} eq $vnetid;
>> + }
>> +
>> + return undef;
>> +}
>
> seems like we're always (or at least most of the time) querying by MAC
> so would assembling a hash keyed by MAC addresses in the first place be sensible?
>
yes, keyed by lc mac with a list per mac since one can appear on several
guests
will do in v4
>
>> +sub notify_guest_node {
>> + my ($vnetid, $mac) = @_;
>> +
>> + return if !$mac;
>> +
>> + # best effort like the push itself, since the record is written by now
>> + my ($zoneid, $node) = eval {
>> + my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid, 1);
>> + return if !$vnet;
>> +
>> + my $zone = PVE::Network::SDN::Zones::get_zone($vnet->{zone}, 1);
>> + return if !$zone || !$zone->{ipam} || !$zone->{dhcp};
>> +
>> + return ($vnet->{zone}, guest_node_by_mac($vnetid, $mac));
>> + };
>> + if ($@) {
>> + warn "could not find the node to push the dhcp mapping of $mac to: $@";
>> + return;
>> + }
>> + return if !$node || $node eq PVE::INotify::nodename();
>> +
>> + # double fork, the grandchild is reparented to init so nothing on
>> + # the request path ever waits on it
>> + my $pid = fork();
>> + if (!defined($pid)) {
>> + warn "could not fork for the dhcp mapping push to $node: $!\n";
>> + return;
>> + }
>> + if ($pid) {
>> + waitpid($pid, 0);
>> + return;
>> + }
>> +
>> + POSIX::setsid();
>> + my $child = fork();
>> + POSIX::_exit(1) if !defined($child);
>> + POSIX::_exit(0) if $child;
>> +
>> + # nothing of the request stays with the push, and since the daemon's
>> + # stderr is closed a failed push reports through the journal
>> + open(STDIN, '<', '/dev/null') or POSIX::_exit(1);
>> + open(STDOUT, '>', '/dev/null') or POSIX::_exit(1);
>> + open(STDERR, '|-', 'logger', '-t', 'pve-sdn', '-p', 'daemon.warning')
>> + or POSIX::_exit(1);
>> + # a node that takes the connection but never answers must not keep the
>> + # push around
>> + exec(
>> + 'timeout',
>> + '60',
>> + 'pvesh',
>> + 'create',
>> + "/nodes/$node/sdn/dhcp-mapping",
>> + '--zone',
>> + $zoneid,
>> + '--vnet',
>> + $vnetid,
>> + '--mac',
>> + $mac,
>> + ) or POSIX::_exit(1);
>> +}
>> +
>
> There've been efforts of moving away from pvesh - see my other patch
> series introducing the api client instead of calling pvesh here:
>
[1]
> https://lore.proxmox.com/all/20260709140625.275618-1-s.hanreich@proxmox.com/
>
> Might be the better option here as well?
i did consider that, but at the time it seemed like "a lot" for
something where a different precedent exists already, but yes, `pvesh`
did seem *meh* here. also wasn't aware of [1]
>
> Also, there is proxyto_callback which we could use to proxy to the correct
> node in the first place?
yes, this should work, and be the better approach here. the only thing
it introduces is a cluster endpoint whose edit fails when another node
isn't reachable, which we don't have yet.. but tbf, it's niche, and
arguably also correct here
will include in v4!
>
>> sub tap_plug {
>> my ($bridge, $iface, $mac) = @_;
>>
>> diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
>> index 783d0d47..9a4dbb9c 100755
>> --- a/src/test/run_test_vnets_blackbox.pl
>> +++ b/src/test/run_test_vnets_blackbox.pl
>> @@ -49,6 +49,7 @@ sub clear_test_state {
>> dnsmasq_calls => [],
>> ebpf_calls => [],
>> order => [],
>> + notify_calls => [],
>> warnings => [],
>> ipam_config => {
>> 'ids' => {
>> @@ -238,6 +239,11 @@ $mocked_sdn_dhcp_dnsmasq->mock(
>> update_lease => sub { },
>> );
>>
>> +my $mocked_sdn_dhcp = Test::MockModule->new('PVE::Network::SDN::Dhcp');
>> +$mocked_sdn_dhcp->mock(
>> + notify_guest_node => sub { push $test_state->{notify_calls}->@*, [@_]; },
>> +);
>> +
>> my $mocked_pve_rs_dhcp = Test::MockModule->new('PVE::RS::SDN::Dhcp');
>> $mocked_pve_rs_dhcp->mock(
>> detach => sub {
>> @@ -1280,6 +1286,90 @@ sub test_dhcp_backend_needed_per_node {
>>
>> run_test(\&test_dhcp_backend_needed_per_node);
>>
>> +sub test_mapping_push_target {
>> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
>> + my $zoneid = "TESTZONE";
>> + my $vnetid = "testvnet";
>> + my $mac = "da:65:8f:18:9b:6f";
>> +
>> + create_zone({
>> + type => "simple",
>> + dhcp => "dnsmasq",
>> + ipam => "pve",
>> + zone => $zoneid,
>> + });
>> + create_vnet({
>> + type => "vnet",
>> + zone => $zoneid,
>> + vnet => $vnetid,
>> + });
>> + create_subnet({
>> + type => "subnet",
>> + vnet => $vnetid,
>> + subnet => "10.0.0.0/24",
>> + gateway => "10.0.0.1",
>> + });
>> +
>> + # the guest behind a MAC is found by its NIC on the vnet, and the config
>> + # writes the MAC in upper case
>> + $test_state->{vmlist} = {
>> + ids => {
>> + 100 => { type => 'qemu', node => 'other' },
>> + 101 => { type => 'lxc', node => 'third' },
>> + },
>> + };
>> + $test_state->{guest_nets} = {
>> + 1 => { net0 => "virtio=$mac,bridge=$vnetid" },
>> + 100 => { net0 => "virtio=" . uc($mac) . ",bridge=$vnetid" },
>> + 101 => { net0 => "name=eth0,bridge=vmbr0,hwaddr=00:11:22:33:44:55" },
>> + };
>> + is(
>> + PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, $mac),
>> + 'other',
>> + "$test_name: the MAC resolves to the node of its guest, a guest the list lacks is skipped",
>> + );
>> + is(
>> + PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, '00:11:22:33:44:55'),
>> + undef,
>> + "$test_name: a MAC on another bridge does not resolve",
>> + );
>> + is(
>> + PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, '00:11:22:33:44:66'),
>> + undef,
>> + "$test_name: an unknown MAC does not resolve",
>> + );
>> +
>> + # every mapping edit through the API pokes the guest's node once
>> + $test_state->{notify_calls} = [];
>> + create_ip({
>> + zone => $zoneid,
>> + vnet => $vnetid,
>> + mac => $mac,
>> + ip => "10.0.0.50",
>> + });
>> + update_ip({
>> + zone => $zoneid,
>> + vnet => $vnetid,
>> + mac => $mac,
>> + ip => "10.0.0.51",
>> + });
>> + delete_ip({
>> + zone => $zoneid,
>> + vnet => $vnetid,
>> + mac => $mac,
>> + ip => "10.0.0.51",
>> + });
>> + eq_or_diff(
>> + $test_state->{notify_calls},
>> + [[$vnetid, $mac], [$vnetid, $mac], [$vnetid, $mac]],
>> + "$test_name: create, edit and delete each poke the guest's node",
>> + );
>> + delete $test_state->{vmlist};
>> + delete $test_state->{guest_nets};
>> +}
>> +
>> +run_test(\&test_mapping_push_target);
>> +
>> sub test_ipam_cache_misses {
>> my $test_name = (split(/::/, (caller(0))[3]))[-1];
>> my $zoneid = "TESTZONE";
>
>
>
>
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH proxmox-ebpf v3 02/17] dhcp: add the subsystem that loads the responder and keeps its records
2026-10-20 13:09 ` Stefan Hanreich
@ 2026-10-21 7:09 ` Hannes Laimer
0 siblings, 0 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-21 7:09 UTC (permalink / raw)
To: Stefan Hanreich, pve-devel
On 2026-10-20 15:09, Stefan Hanreich wrote:
> On 10/20/26 10:03 AM, Hannes Laimer wrote:
>> The subsystem loads the program, pins a map with one entry per guest
>> interface and attaches the program to the interfaces it serves.
>> Callers run a full pass over every guest interface, plug or unplug one
>> interface, or change the records of a guest's interfaces.
>>
>> A load can leave the map empty, and only a full pass is handed every
>> guest interface. So the full pass is the only step that loads. A plug
>> or a record change that finds nothing loaded reports that a full pass
>> has to run first.
>>
>> A hotplug moves a tap before the guest config names the new bridge,
>> and no lock closes that gap. So every entry names the vnet its writer
>> saw the interface on. A pass or the record change of a mapping edit
>> leaves an entry naming another vnet alone. A plug writes outright, so
>> plugs share the lock. The edit's record change reads the entry before
>> it writes, so it takes the lock exclusively.
>>
>> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
>> ---
>> src/dhcp/mod.rs | 1031 +++++++++++++++++++++++++++++++++++++++++++++
>> src/dhcp/types.rs | 64 +++
>> src/lib.rs | 3 +
>> src/subsystem.rs | 2 +-
>> 4 files changed, 1099 insertions(+), 1 deletion(-)
>> create mode 100644 src/dhcp/mod.rs
>> create mode 100644 src/dhcp/types.rs
>>
>> diff --git a/src/dhcp/mod.rs b/src/dhcp/mod.rs
>> new file mode 100644
>> index 0000000..d0c4148
>> --- /dev/null
>> +++ b/src/dhcp/mod.rs
>> @@ -0,0 +1,1031 @@
>> +//! The dhcp subsystem, a per-tap DHCPv4 responder answering from a pinned map of one entry per
>> +//! interface.
>> +//!
>> +//! The map holds nothing of its own. The SDN side hands in every guest interface with the answer
>> +//! it gets, if any. A full pass makes the kernel match that for the interfaces it names. A record
>> +//! change or a plug writes its own interfaces only. A linked interface the pass does not name
>> +//! keeps its link and its entry. A plug runs under the shared lock and writes its entry whatever
>> +//! is there. A pass runs under the exclusive lock, and so does the record change an edit pushes,
>> +//! since it reads the entry's vnet before it writes. A guest's own start holds the guest lock, so
>> +//! nothing else plugs its interfaces meanwhile and it writes its records under the shared lock
>> +//! without a read.
>> +//!
>> +//! A hotplug plugs the tap before the guest config names the new bridge, and no lock orders that.
>> +//! So every entry names the vnet its writer saw the interface on. A pass or an edit's record change
>> +//! leaves an entry alone that names another vnet than the configs do, and the link follows such an
>> +//! entry. A plug replaces it. Plugs of one interface are serialized by the guest lock, so the plug
>> +//! is the latest word on where its interface sits. An unplug drops the interface's entry. An
>> +//! interface that went away without a link keeps its entry until the next pass. One plugged onto a
>> +//! bridge this subsystem does not serve trades its entry for a marker naming that place.
>> +mod types;
>> +
>> +use std::collections::{HashMap, HashSet};
>> +use std::io::ErrorKind;
>> +use std::net::Ipv4Addr;
>> +
>> +use anyhow::{Context, bail};
>> +use aya::maps::MapError;
>> +
>> +use self::types::*;
>> +use crate::subsystem::{ApplyLock, Exclusive, NotLoaded, Shared, TcPrograms};
>> +use crate::tc::{Direction, LinkPin};
>> +use crate::tc_programs;
>> +
>> +/// One guest interface as the configs know it. `serve` says whether a pass links it. A plug
>> +/// links its interface regardless. `record` is the answer it gets there, and without one the
>> +/// interface gets no answer. `vnet` is the vnet it sits on, none for a plain bridge.
>> +pub struct Iface {
>> + pub name: String,
>> + pub vnet: Option<String>,
>> + pub serve: bool,
>> + pub record: Option<Record>,
>> +}
>> +
>> +/// One interface's answer, the address and every option the reply carries.
>> +pub struct Record {
>> + pub ip: Ipv4Addr,
>> + pub prefixlen: u8,
>> + pub server_id: Ipv4Addr,
>> + pub lease: u32,
>> + pub router: Option<Ipv4Addr>,
>> + pub dns: Option<Ipv4Addr>,
>> + pub mtu: Option<u16>,
>> + pub mac: [u8; 6],
>
> proxmox-network-types has a MAC address type that would fit here and in other structs
>
not in its current shape, we need to access the bytes, with the current
type can't. but generally yes, we'd just also need to expose them on the
type. id rather have this as small follow-up that includes the change on
the type
>> +}
>> +
>> +/// What became of an unplugged interface.
>> +pub enum Unplugged {
>> + /// Gone, or going with the unplug.
>> + Gone,
>> + /// Plugged onto a bridge this subsystem does not serve, the vnet it sits on now, none for a
>> + /// plain bridge.
>> + MovedTo(Option<String>),
>> +}
>> +
>> +/// What a single-entry step did.
>> +#[derive(Debug, PartialEq, Eq)]
>> +pub enum Outcome {
>> + /// Nothing is left to do. The entry is in place, or its interface is not here.
>> + Done,
>> + /// Nothing is loaded to write into. Only a full pass loads, so the caller runs one and retries.
>> + NeedsFullPass,
>> +}
>> +
>> +const RECORDS_MAP: &str = "dhcp_iface_records";
>> +const MAPS: [&str; 1] = [RECORDS_MAP];
>> +
>> +// bump with any change to the key or value layout of a map in dhcp.bpf.c
>> +const SCHEMA_VERSION: u32 = 1;
>> +
>> +// requests only arrive from the guest side and replies leave through a redirect, so only ingress
>> +// carries a program
>> +const PROGRAM_NAMES: [(Direction, &str); 1] = [(Direction::Ingress, "tc_dhcp_ingress")];
>> +
>> +type RecordsMap = aya::maps::HashMap<aya::maps::MapData, u32, DhcpRecord>;
>> +
>> +static PROGRAMS: TcPrograms = tc_programs! {
>> + name: "dhcp",
>> + programs: &PROGRAM_NAMES,
>> + maps: &MAPS,
>> + schema_version: SCHEMA_VERSION,
>> +};
>> +
>> +pub struct DhcpSubsystem {
>> + programs: &'static TcPrograms,
>> +}
>> +
>> +impl Default for DhcpSubsystem {
>> + fn default() -> Self {
>> + Self::new()
>> + }
>> +}
>> +
>> +impl DhcpSubsystem {
>> + pub fn new() -> Self {
>> + Self {
>> + programs: &PROGRAMS,
>> + }
>> + }
>> +
>> + /// The lock a plug runs under, held until [`attach`](SharedLock::attach) returns. Plugs run
>> + /// next to each other and wait only behind an exclusive holder.
>> + pub fn lock_shared(&self) -> anyhow::Result<SharedLock> {
>> + Ok(SharedLock {
>> + lock: self.programs.lock_shared()?,
>> + })
>> + }
>> +
>> + /// The lock a full pass or a record change runs under, held until the change is written.
>> + pub fn lock_exclusive(&self) -> anyhow::Result<ExclusiveLock> {
>> + Ok(ExclusiveLock {
>> + lock: self.programs.lock_exclusive()?,
>> + })
>> + }
>> +
>> + /// A tap unplug. The interface may be gone by then, so its link is found by the name the pin
>> + /// carries. [`Unplugged`] decides what becomes of the entry.
>> + pub fn detach(&self, iface: &str, unplugged: Unplugged) -> anyhow::Result<()> {
>> + // resolved before the lock wait, an interface recreated under the name meanwhile is not
>> + // this unplug's and keeps its link
>> + let now = ifindex_of(iface)?;
>> + // every guest unplug on the host lands here. Most never had a link and must not wait
>> + // behind a pass. One without a link whose interface is gone leaves its entry to the
>> + // next pass
>> + if !self.programs.link_pinned(iface) && now.is_none() {
>> + return Ok(());
>> + }
>> + let lock = self.programs.lock_shared()?;
>> +
>> + if !lock.is_current() {
>> + log::debug!("dhcp detach: nothing current, {iface} is left to the next full pass");
>> + return Ok(());
>> + }
>> + // the map is opened and the marker built before the links go, so a failure there
>> + // leaves them attached
>> + let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?;
>> + let marker = match &unplugged {
>> + Unplugged::Gone => None,
>> + Unplugged::MovedTo(vnet) => Some(DhcpRecord::removed(place(vnet.as_deref())?)),
>> + };
>> + // an interface without a link has an entry as well, a marker where it is not served
>> + let mut ifindexes = lock.detach_named(iface, |pin| {
>
> the name confused me a bit here, maybe detach_named_if or something would be more fitting?
>
hmm, sure, but this is from the lib.. the fn doc mentions it, but i do
see you point. id address this in the lib series, maybe as a follow-up?
>> + Some(pin.ifindex) != now && pin_alive(pin).unwrap_or(false)
>> + })?;
>> + if let Some(now) = now.filter(|now| !ifindexes.contains(now)) {
>> + ifindexes.push(now);
>> + }
>> + // the marker is the moved interface's. Older incarnations under the name are gone
>> + unplug_entries(&mut map, &ifindexes, now.zip(marker))
>> + }
>> +}
>> +
>> +/// The shared lock of the responder, what a plug holds, and the record change of a guest's own
>> +/// start.
>> +pub struct SharedLock {
>> + lock: ApplyLock<'static, Shared>,
>> +}
>> +
>> +impl SharedLock {
>> + /// A tap plug, the entry goes in and the program on. The plug loads nothing, so with nothing
>> + /// pinned the caller runs a full pass first.
>> + pub fn attach(&self, iface: &Iface) -> anyhow::Result<Outcome> {
>> + if !self.lock.is_current() {
>> + return Ok(Outcome::NeedsFullPass);
>> + }
>> + let Some(ifindex) = ifindex_of(iface.name.as_str())? else {
>> + log::info!("dhcp attach: {} is gone, nothing to do", iface.name);
>> + return Ok(Outcome::Done);
>> + };
>> + // the entry goes in before the program runs, a marker when there is no record or one
>> + // the program cannot carry. A guest start must not fail on a bad record
>> + let mut map: RecordsMap = self.lock.hash_map(RECORDS_MAP)?;
>> + let entry = match iface.entry() {
>> + Ok(entry) => entry,
>> + Err(e) => {
>> + log::warn!(
>> + "dhcp: {} attaches with nothing to answer: {e:#}",
>> + iface.name
>> + );
>> + DhcpRecord::removed(iface.place()?)
>> + }
>> + };
>> + write_entry(&mut map, ifindex, &entry, Elsewhere::Override)?;
>> + match self.lock.attach_iface(ifindex) {
>> + Ok(()) => Ok(Outcome::Done),
>> + Err(e) if e.downcast_ref::<NotLoaded>().is_some() => Ok(Outcome::NeedsFullPass),
>> + Err(e) => Err(e).with_context(|| format!("dhcp responder for {}", iface.name)),
>> + }
>> + }
>> +
>> + /// A record change for a caller that holds the guest lock. Nothing else plugs the guest's
>> + /// interfaces under that lock, so each entry is written outright, one atomic write, without
>> + /// reading what it holds.
>> + pub fn update(&self, ifaces: &[Iface]) -> anyhow::Result<Outcome> {
>> + update_entries(&self.lock, ifaces, Elsewhere::Override)
>> + }
>> +}
>> +
>> +/// The exclusive lock of the responder, what a full pass and a record change hold.
>> +pub struct ExclusiveLock {
>> + lock: ApplyLock<'static, Exclusive>,
>> +}
>> +
>> +impl ExclusiveLock {
>> + /// The full pass. Afterward every interface the configs name that exists here has its entry,
>> + /// a record where it is served with one and a marker otherwise, and a link is on every
>> + /// served interface and on every interface whose entry names another vnet. Entries of
>> + /// interfaces that are gone are dropped, as are entries of live interfaces the configs do
>> + /// not name and that have no link. Bad input leaves its interface as it is, with a warning.
>> + pub fn apply(&self, ifaces: &[Iface]) -> anyhow::Result<()> {
>> + self.lock.ensure_loaded()?;
>> + let present = present_ifindexes(ifaces)?;
>> + // a link the configs do not name stays while its interface lives under the pin's
>> + // name. A reused index is not that interface
>> + let mut unknown_alive = HashSet::new();
>> + for pin in self.lock.attached()? {
>> + if !present.contains_key(&pin.ifindex) && pin_alive(&pin)? {
>> + unknown_alive.insert(pin.ifindex);
>> + }
>> + }
>> + let mut map: RecordsMap = self.lock.hash_map(RECORDS_MAP)?;
>> + let (elsewhere, failed) = sync_records(&mut map, &present, &unknown_alive)?;
>> + // a failed write or removal leaves its entry as it is. The links are still put right
>> + // and every failure is reported once the rest is done
>> + let links = self
>> + .lock
>> + .reconcile(&wanted_links(&present, &elsewhere, &unknown_alive));
>> + match (failed, links) {
>> + (0, links) => links,
>> + (failed, Ok(())) => bail!("{failed} entries could not be changed"),
>> + (failed, Err(e)) => {
>> + Err(e).with_context(|| format!("{failed} entries could not be changed"))
>> + }
>> + }
>> + }
>> +
>> + /// A record change. Interfaces not found here are skipped. The rest are written whether they
>> + /// have a link yet or not. A record the reply cannot carry leaves its interface as it is.
>> + /// With nothing loaded the caller runs a full pass first, like a plug.
>> + pub fn update(&self, ifaces: &[Iface]) -> anyhow::Result<Outcome> {
>> + update_entries(&self.lock, ifaces, Elsewhere::Skip)
>> + }
>> +
>> + /// Detach everywhere and drop the pinned state, for the last zone leaving the backend.
>> + pub fn clear(&self) -> anyhow::Result<()> {
>> + self.lock.clear()
>> + }
>> +}
>> +
>> +fn update_entries<M>(
>> + lock: &ApplyLock<'static, M>,
>> + ifaces: &[Iface],
>> + elsewhere: Elsewhere,
>> +) -> anyhow::Result<Outcome> {
>> + if !lock.is_current() {
>> + return Ok(Outcome::NeedsFullPass);
>> + }
>> + let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?;
>> + let mut failed = 0usize;
>> + for iface in ifaces {
>> + let ifindex = match ifindex_of(iface.name.as_str()) {
>> + Ok(Some(ifindex)) => ifindex,
>> + Ok(None) => continue,
>> + Err(e) => {
>> + log::warn!("dhcp: {e:#}");
>> + failed += 1;
>> + continue;
>> + }
>> + };
>> + let entry = match iface.entry() {
>> + Ok(entry) => entry,
>> + Err(e) => {
>> + log::warn!("dhcp: {} is left as it is: {e:#}", iface.name);
>> + continue;
>> + }
>> + };
>> + if let Err(e) = write_entry(&mut map, ifindex, &entry, elsewhere) {
>> + log::warn!("dhcp: writing the entry of {}: {e:#}", iface.name);
>> + failed += 1;
>> + }
>> + }
>> + if failed > 0 {
>> + bail!("{failed} interfaces could not be updated");
>> + }
>> + Ok(Outcome::Done)
>> +}
>> +
>> +const NO_VNET: [u8; 8] = [0; 8];
>> +
>> +// the map update flag that refuses a key the map does not hold, from the kernel's uapi bpf.h
>> +const BPF_EXIST: u64 = 2;
>> +
>> +fn unplug_entries(
>> + map: &mut impl RecordStore,
>> + ifindexes: &[u32],
>> + moved: Option<(u32, DhcpRecord)>,
>> +) -> anyhow::Result<()> {
>> + for &ifindex in ifindexes {
>> + match moved {
>> + // only an existing entry becomes a marker, so an interface without one gets none
>> + Some((current, marker)) if current == ifindex => map.mark(ifindex, marker)?,
>> + _ => map.remove(ifindex)?,
>> + }
>> + }
>> + Ok(())
>> +}
>> +
>> +// the vnet id padded with zeros, all zero for none
>> +fn place(vnet: Option<&str>) -> anyhow::Result<[u8; 8]> {
>> + let Some(vnet) = vnet else {
>> + return Ok(NO_VNET);
>> + };
>> + let bytes = vnet.as_bytes();
>> + if bytes.is_empty() || bytes.len() > NO_VNET.len() {
>> + bail!("vnet id {vnet:?} does not fit a record");
>> + }
>> + let mut place = NO_VNET;
>> + place[..bytes.len()].copy_from_slice(bytes);
>> + Ok(place)
>> +}
>> +
>> +// the record map as the writers see it, so their decisions run against a plain map in tests
>> +trait RecordStore {
>> + fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>>;
>> + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()>;
>> + // replace an entry that exists and do nothing for one that does not, in one step, so it is
>> + // safe under the shared lock
>> + fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()>;
>> + // a missing entry is removed already
>> + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()>;
>> + fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>>;
>> +}
>> +
>> +impl RecordStore for RecordsMap {
>> + fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
>> + match aya::maps::HashMap::get(self, &ifindex, 0) {
>> + Ok(live) => Ok(Some(live)),
>> + Err(MapError::KeyNotFound) => Ok(None),
>> + Err(e) => Err(e.into()),
>> + }
>> + }
>> +
>> + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
>> + aya::maps::HashMap::insert(self, ifindex, rec, 0)?;
>> + Ok(())
>> + }
>> +
>> + fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
>> + match aya::maps::HashMap::insert(self, ifindex, rec, BPF_EXIST) {
>> + Ok(()) => Ok(()),
>> + // the kernel answers a missing key with ENOENT, and aya passes that through as is
>> + Err(MapError::SyscallError(e)) if e.io_error.kind() == ErrorKind::NotFound => Ok(()),
>> + Err(e) => Err(e.into()),
>> + }
>> + }
>> +
>> + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
>> + match aya::maps::HashMap::remove(self, &ifindex) {
>> + Ok(()) => Ok(()),
>> + // the kernel answers a missing key with ENOENT, and aya passes that through as is
>> + Err(MapError::SyscallError(e)) if e.io_error.kind() == ErrorKind::NotFound => Ok(()),
>> + Err(e) => Err(e.into()),
>> + }
>> + }
>> +
>> + // a record the listing cannot read is one the pass cannot compare against, so the pass
>> + // fails rather than writes past it
>> + fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
>> + self.iter()
>> + .collect::<Result<Vec<_>, _>>()
>> + .context("listing the records")
>> + }
>> +}
>> +
>> +// what a write does with an entry naming another vnet than its own
>> +#[derive(Clone, Copy)]
>> +enum Elsewhere {
>> + // a record change, its configs are stale for the interface
>> + Skip,
>> + // a plug, or a record change under the guest lock, both see where the interface sits now
>> + Override,
>> +}
>> +
>> +fn write_entry(
>> + map: &mut impl RecordStore,
>> + ifindex: u32,
>> + rec: &DhcpRecord,
>> + elsewhere: Elsewhere,
>> +) -> anyhow::Result<()> {
>> + if matches!(elsewhere, Elsewhere::Skip)
>> + && let Some(live) = map.get(ifindex)?
>> + && live.vnet != rec.vnet
>> + {
>> + log::debug!("dhcp: ifindex {ifindex} sits elsewhere than the configs say, leaving it");
>> + return Ok(());
>> + }
>> + map.insert(ifindex, *rec)
>> +}
>> +
>> +// entries of live links the pass does not know stay, since their plug wrote them. The other
>> +// unknown ones go with their interface. Returns the interfaces whose entry names another vnet
>> +// than the configs, and how many writes failed
>> +fn sync_records(
>> + map: &mut impl RecordStore,
>> + present: &HashMap<u32, &Iface>,
>> + unknown_alive: &HashSet<u32>,
>> +) -> anyhow::Result<(HashSet<u32>, usize)> {
>> + let live: HashMap<u32, DhcpRecord> = map.entries()?.into_iter().collect();
>> + let dead: Vec<u32> = live
>> + .keys()
>> + .filter(|ifindex| !present.contains_key(ifindex) && !unknown_alive.contains(ifindex))
>> + .copied()
>> + .collect();
>> + let mut failed = 0usize;
>> + for &ifindex in &dead {
>> + if let Err(e) = map.remove(ifindex) {
>> + log::warn!("dhcp: removing the record of ifindex {ifindex}: {e:#}");
>
> error message is missing an error or some other prefix?
>
yes, ill reword for v4
>> + failed += 1;
>> + }
>> + }
>> + let mut written = 0usize;
>> + let mut elsewhere = HashSet::new();
>> +
>> + for (ifindex, iface) in present {
>> + // bad input leaves its interface as it is and the pass goes on
>> + let desired = match iface.entry() {
>> + Ok(desired) => desired,
>> + Err(e) => {
>> + log::warn!("dhcp: {} is left as it is: {e:#}", iface.name);
>> + continue;
>> + }
>> + };
>> + match live.get(ifindex) {
>> + Some(entry) if entry.vnet != desired.vnet => {
>> + elsewhere.insert(*ifindex);
>> + }
>> +
>> + Some(entry) if *entry == desired => {}
>> + _ => match map.insert(*ifindex, desired) {
>> + Ok(()) => written += 1,
>> + Err(e) => {
>> + log::warn!("dhcp: writing the entry of {}: {e:#}", iface.name);
>> + failed += 1;
>> + }
>> + },
>> + }
>> + }
>> +
>> + if written + dead.len() + elsewhere.len() > 0 {
>> + log::info!(
>> + "dhcp: {written} entries written, {} removed, {} left where their plug put them",
>> + dead.len(),
>> + elsewhere.len()
>> + );
>> + } else {
>> + log::debug!("dhcp: {} records, no changes", live.len());
>> + }
>> + Ok((elsewhere, failed))
>> +}
>> +
>> +// the interface's index here, none when it does not exist. Any other failure says nothing about
>> +// the interface, and a step taking it for gone would detach what it could not resolve
>> +fn ifindex_of(name: &str) -> anyhow::Result<Option<u32>> {
>> + match nix::net::if_::if_nametoindex(name) {
>> + Ok(ifindex) => Ok(Some(ifindex)),
>> + Err(nix::errno::Errno::ENODEV) => Ok(None),
>> + Err(e) => Err(e).with_context(|| format!("resolve interface {name}")),
>> + }
>> +}
>> +
>> +// the named interfaces that exist here by index, the names come from configs cluster-wide and
>> +// most belong to other nodes
>> +fn present_ifindexes(ifaces: &[Iface]) -> anyhow::Result<HashMap<u32, &Iface>> {
>> + let mut present = HashMap::new();
>> + for iface in ifaces {
>> + if let Some(ifindex) = ifindex_of(iface.name.as_str())? {
>> + present.insert(ifindex, iface);
>> + }
>> + }
>> + Ok(present)
>> +}
>
> Could we use if_nameindex here instead of querying if_nametoindex for every interface?
>
yes, will do for v4
>> +// the link follows the configs where they agree with the entry. An entry naming another vnet
>> +// keeps its link, and a served interface without a mapping gets one. So a record change filling
>> +// a marker attaches nothing itself. A live link the configs do not know stays
>> +fn wanted_links(
>> + present: &HashMap<u32, &Iface>,
>> + elsewhere: &HashSet<u32>,
>> + unknown_alive: &HashSet<u32>,
>> +) -> HashSet<u32> {
>> + let mut wanted: HashSet<u32> = present
>> + .iter()
>> + .filter(|(ifindex, iface)| elsewhere.contains(ifindex) || iface.serve)
>> + .map(|(ifindex, _)| *ifindex)
>> + .collect();
>> +
>> + wanted.extend(unknown_alive);
>> + wanted
>> +}
>> +
>> +// whether the link's interface still exists under the name and index its pin carries
>> +fn pin_alive(pin: &LinkPin) -> anyhow::Result<bool> {
>> + Ok(ifindex_of(pin.name.as_str())? == Some(pin.ifindex))
>> +}
>> +
>> +impl Iface {
>> + // a marker where nothing is served
>> + fn entry(&self) -> anyhow::Result<DhcpRecord> {
>> + let vnet = self.place()?;
>> + match &self.record {
>> + Some(record) if self.serve => record
>> + .entry(vnet)
>> + .with_context(|| format!("record for {}", self.name)),
>> + _ => Ok(DhcpRecord::removed(vnet)),
>> + }
>> + }
>> +
>> + fn place(&self) -> anyhow::Result<[u8; 8]> {
>> + place(self.vnet.as_deref()).with_context(|| format!("interface {}", self.name))
>> + }
>> +}
>> +
>> +impl Record {
>> + // refused here is what the reply cannot carry. Zero stands for absent in the entry, so an
>> + // address or an MTU of zero cannot be told from none
>> + fn entry(&self, vnet: [u8; 8]) -> anyhow::Result<DhcpRecord> {
>> + if self.prefixlen == 0 || self.prefixlen > 32 {
>> + bail!("prefixlen {} out of range", self.prefixlen);
>> + }
>> + if self.ip.is_unspecified() {
>> + bail!("no address");
>> + }
>> + if self.server_id.is_unspecified() {
>> + bail!("no server address");
>> + }
>> + if self.lease == 0 {
>> + bail!("lease of zero seconds");
>> + }
>> + if self.mtu.is_some_and(|mtu| mtu < 68) {
>> + bail!("mtu below the IPv4 minimum of 68");
>> + }
>> + if self.mac == [0; 6] {
>> + bail!("no MAC");
>> + }
>> + let netmask = u32::MAX << (32 - self.prefixlen);
>> + Ok(DhcpRecord {
>> + ip: u32::from(self.ip).to_be(),
>> + netmask: netmask.to_be(),
>> + router: u32::from(self.router.unwrap_or(Ipv4Addr::UNSPECIFIED)).to_be(),
>> + dns: u32::from(self.dns.unwrap_or(Ipv4Addr::UNSPECIFIED)).to_be(),
>> + server_id: u32::from(self.server_id).to_be(),
>> + lease: self.lease.to_be(),
>> + mtu: self.mtu.unwrap_or(0).to_be(),
>> + vnet,
>> + mac: self.mac,
>> + })
>> + }
>> +}
>> +
>> +#[cfg(test)]
>> +mod test {
>> + use std::collections::BTreeMap;
>> +
>> + use super::*;
>> +
>> + fn record() -> Record {
>> + Record {
>> + ip: Ipv4Addr::new(10, 0, 0, 100),
>> + prefixlen: 24,
>> + server_id: Ipv4Addr::new(10, 0, 0, 1),
>> + lease: 300,
>> + router: Some(Ipv4Addr::new(10, 0, 0, 1)),
>> + dns: Some(Ipv4Addr::new(1, 1, 1, 1)),
>> + mtu: Some(1500),
>> + mac: MAC,
>> + }
>> + }
>> +
>> + const MAC: [u8; 6] = [0xda, 0x65, 0x8f, 0x18, 0x9b, 0x6f];
>> +
>> + const VNET: [u8; 8] = *b"vnet1\0\0\0";
>> +
>> + #[test]
>> + fn converts_full_record() {
>> + let rec = record().entry(VNET).unwrap();
>> + assert_eq!(
>> + u32::from_be(rec.ip),
>> + u32::from(Ipv4Addr::new(10, 0, 0, 100))
>> + );
>> + assert_eq!(u32::from_be(rec.netmask), 0xffffff00);
>> + assert_eq!(
>> + u32::from_be(rec.router),
>> + u32::from(Ipv4Addr::new(10, 0, 0, 1))
>> + );
>> + assert_eq!(u32::from_be(rec.dns), u32::from(Ipv4Addr::new(1, 1, 1, 1)));
>> + assert_eq!(
>> + u32::from_be(rec.server_id),
>> + u32::from(Ipv4Addr::new(10, 0, 0, 1))
>> + );
>> + assert_eq!(u32::from_be(rec.lease), 300);
>> + assert_eq!(u16::from_be(rec.mtu), 1500);
>> + assert_eq!(rec.vnet, VNET);
>> + assert_eq!(rec.mac, MAC);
>> +
>> + let rec = Record {
>> + router: None,
>> + dns: None,
>> + mtu: None,
>> + ..record()
>> + }
>> + .entry(VNET)
>> + .unwrap();
>> + assert_eq!(rec.router, 0);
>> + assert_eq!(rec.dns, 0);
>> + assert_eq!(rec.mtu, 0);
>> + }
>> +
>> + #[test]
>> + fn entries_name_the_vnet_of_the_interface() {
>> + let iface = Iface {
>> + name: "tap100i0".into(),
>> + vnet: Some("testvnet".into()),
>> + serve: true,
>> + record: Some(record()),
>> + };
>> + assert_eq!(&iface.entry().unwrap().vnet, b"testvnet");
>> + let iface = Iface {
>> + name: "tap100i0".into(),
>> + vnet: None,
>> + serve: false,
>> + record: None,
>> + };
>> + assert_eq!(iface.entry().unwrap().vnet, NO_VNET);
>> + let iface = Iface {
>> + name: "tap100i0".into(),
>> + vnet: Some("ninechars".into()),
>> + serve: true,
>> + record: Some(record()),
>> + };
>> + assert!(iface.entry().is_err());
>> + let iface = Iface {
>> + name: "tap100i0".into(),
>> + vnet: Some(String::new()),
>> + serve: true,
>> + record: Some(record()),
>> + };
>> + assert!(iface.entry().is_err());
>> + }
>> +
>> + #[test]
>> + fn unserved_interfaces_get_a_marker() {
>> + let iface = Iface {
>> + name: "tap100i0".into(),
>> + vnet: Some("vnet1".into()),
>> + serve: false,
>> + record: Some(record()),
>> + };
>> + assert_eq!(iface.entry().unwrap().ip, 0);
>> + assert_eq!(iface.entry().unwrap().vnet, VNET);
>> + let iface = Iface {
>> + name: "tap100i0".into(),
>> + vnet: Some("vnet1".into()),
>> + serve: true,
>> + record: None,
>> + };
>> + assert_eq!(iface.entry().unwrap().ip, 0);
>> + let iface = Iface {
>> + name: "tap100i0".into(),
>> + vnet: Some("vnet1".into()),
>> + serve: true,
>> + record: Some(record()),
>> + };
>> + assert_ne!(iface.entry().unwrap().ip, 0);
>> + }
>> +
>> + #[test]
>> + fn refuses_what_the_reply_cannot_carry() {
>> + for rec in [
>> + Record {
>> + prefixlen: 33,
>> + ..record()
>> + },
>> + Record {
>> + prefixlen: 0,
>> + ..record()
>> + },
>> + Record {
>> + server_id: Ipv4Addr::UNSPECIFIED,
>> + ..record()
>> + },
>> + Record {
>> + lease: 0,
>> + ..record()
>> + },
>> + Record {
>> + ip: Ipv4Addr::UNSPECIFIED,
>> + ..record()
>> + },
>> + Record {
>> + mtu: Some(67),
>> + ..record()
>> + },
>> + Record {
>> + mac: [0; 6],
>> + ..record()
>> + },
>> + ] {
>> + assert!(rec.entry(VNET).is_err());
>> + }
>> + let rec = Record {
>> + mtu: Some(68),
>> + ..record()
>> + };
>> + assert_eq!(u16::from_be(rec.entry(VNET).unwrap().mtu), 68);
>> +
>> + let rec = Record {
>> + prefixlen: 32,
>> + ..record()
>> + };
>> + assert_eq!(u32::from_be(rec.entry(VNET).unwrap().netmask), u32::MAX);
>> + let rec = Record {
>> + prefixlen: 1,
>> + ..rec
>> + };
>> + assert_eq!(u32::from_be(rec.entry(VNET).unwrap().netmask), 1 << 31);
>> + }
>> +
>> + impl RecordStore for BTreeMap<u32, DhcpRecord> {
>> + fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
>> + Ok(BTreeMap::get(self, &ifindex).copied())
>> + }
>> +
>> + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
>> + BTreeMap::insert(self, ifindex, rec);
>> + Ok(())
>> + }
>> +
>> + fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
>> + if let Some(entry) = BTreeMap::get_mut(self, &ifindex) {
>> + *entry = rec;
>> + }
>> + Ok(())
>> + }
>> +
>> + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
>> + BTreeMap::remove(self, &ifindex);
>> + Ok(())
>> + }
>> +
>> + fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
>> + Ok(self.iter().map(|(k, v)| (*k, *v)).collect())
>> + }
>> + }
>> +
>> + const OTHER: [u8; 8] = *b"vnet2\0\0\0";
>> +
>> + fn entry(vnet: [u8; 8]) -> DhcpRecord {
>> + record().entry(vnet).unwrap()
>> + }
>> +
>> + fn other_answer(vnet: [u8; 8]) -> DhcpRecord {
>> + Record {
>> + ip: Ipv4Addr::new(10, 0, 0, 200),
>> + ..record()
>> + }
>> + .entry(vnet)
>> + .unwrap()
>> + }
>> +
>> + #[test]
>> + fn a_record_change_writes_on_its_own_vnet() {
>> + let mut map = BTreeMap::new();
>> + write_entry(&mut map, 1, &entry(VNET), Elsewhere::Skip).unwrap();
>> + assert_eq!(map[&1], entry(VNET));
>> + write_entry(&mut map, 1, &other_answer(VNET), Elsewhere::Skip).unwrap();
>> + assert_eq!(map[&1], other_answer(VNET));
>> + // an entry naming another vnet was placed by a plug and stays
>> + map.insert(1, entry(OTHER));
>> + write_entry(&mut map, 1, &other_answer(VNET), Elsewhere::Skip).unwrap();
>> + assert_eq!(map[&1], entry(OTHER));
>> + }
>> +
>> + #[test]
>> + fn a_plug_writes_whatever_is_there() {
>> + let mut map = BTreeMap::new();
>> + write_entry(&mut map, 1, &entry(VNET), Elsewhere::Override).unwrap();
>> + assert_eq!(map[&1], entry(VNET));
>> + map.insert(1, entry(OTHER));
>> + write_entry(&mut map, 1, &other_answer(VNET), Elsewhere::Override).unwrap();
>> + assert_eq!(map[&1], other_answer(VNET));
>> + }
>> +
>> + #[test]
>> + fn an_unplug_removes_or_marks_the_entries() {
>> + let mut map = BTreeMap::new();
>> + map.insert(1, entry(VNET));
>> + map.insert(2, entry(VNET));
>> + map.insert(3, entry(VNET));
>> + unplug_entries(&mut map, &[1, 2], None).unwrap();
>> + assert_eq!(map.keys().copied().collect::<Vec<_>>(), [3]);
>> + // a missing entry is removed already
>> + unplug_entries(&mut map, &[1], None).unwrap();
>> + // a move marks the interface that moved, and an older incarnation under its name is gone
>> + map.insert(4, entry(VNET));
>> + let marker = DhcpRecord::removed(NO_VNET);
>> + unplug_entries(&mut map, &[3, 4], Some((4, marker))).unwrap();
>> + assert_eq!(map.keys().copied().collect::<Vec<_>>(), [4]);
>> + assert_eq!(map[&4].ip, 0);
>> + assert_eq!(map[&4].vnet, NO_VNET);
>> + // an interface without an entry gets no marker
>> + unplug_entries(&mut map, &[5], Some((5, marker))).unwrap();
>> + assert!(!map.contains_key(&5));
>> + }
>> +
>> + fn iface(name: &str, vnet: Option<&str>, serve: bool, record: Option<Record>) -> Iface {
>> + Iface {
>> + name: name.into(),
>> + vnet: vnet.map(String::from),
>> + serve,
>> + record,
>> + }
>> + }
>> +
>> + #[test]
>> + fn the_pass_syncs_the_records_it_knows() {
>> + let mut map = BTreeMap::new();
>> + map.insert(1, entry(VNET)); // same answer as the config
>> + map.insert(2, other_answer(VNET)); // a changed answer
>> + map.insert(4, entry(VNET)); // not served anymore
>> + map.insert(6, entry(VNET)); // its interface is gone
>> + map.insert(7, entry(VNET)); // a live link the configs do not name
>> + map.insert(8, entry(VNET)); // its mapping is gone
>> + let ifaces = [
>> + iface("a", Some("vnet1"), true, Some(record())),
>> + iface("b", Some("vnet1"), true, Some(record())),
>> + iface("d", Some("vnet1"), false, None),
>> + iface("e", Some("vnet1"), true, Some(record())), // no entry yet
>> + iface("f", Some("vnet1"), true, None), // no entry, served, no mapping
>> + iface("g", Some("vnet1"), true, None),
>> + ];
>> + let present: HashMap<u32, &Iface> =
>> + [1, 2, 4, 5, 9, 8].into_iter().zip(ifaces.iter()).collect();
>> + let unknown_alive = HashSet::from([7]);
>> + let (elsewhere, failed) = sync_records(&mut map, &present, &unknown_alive).unwrap();
>> +
>> + assert!(elsewhere.is_empty());
>> + assert_eq!(failed, 0);
>> + assert_eq!(map[&1], entry(VNET));
>> + assert_eq!(map[&2], entry(VNET), "a changed answer is rewritten");
>> + assert_eq!(
>> + map[&4].ip, 0,
>> + "an interface not served anymore keeps a marker"
>> + );
>> + assert_eq!(map[&5], entry(VNET), "a new interface gets its entry");
>> + assert!(!map.contains_key(&6), "a gone interface loses its entry");
>> + assert_eq!(
>> + map[&9].ip, 0,
>> + "a served interface without a mapping gets a marker"
>> + );
>> + assert_eq!(map[&8].ip, 0, "a mapping that is gone leaves a marker");
>> + assert!(
>> + map.contains_key(&7),
>> + "a live link the configs do not name keeps its entry"
>> + );
>> + }
>> +
>> + #[test]
>> + fn the_links_follow_the_configs_or_the_entry() {
>> + let ifaces = [
>> + iface("a", Some("vnet1"), true, Some(record())), // served, entry agrees
>> + iface("b", Some("vnet1"), false, None), // not served
>> + iface("c", Some("vnet1"), false, None), // config stale, entry has a record
>> + iface("d", Some("vnet1"), true, Some(record())), // config stale, entry is a marker
>> + ];
>> + let present: HashMap<u32, &Iface> = (1..=4).zip(ifaces.iter()).collect();
>> + let elsewhere = HashSet::from([3, 4]);
>> + let unknown_alive = HashSet::from([9]);
>> + assert_eq!(
>> + wanted_links(&present, &elsewhere, &unknown_alive),
>> + HashSet::from([1, 3, 4, 9])
>> + );
>> + }
>> +
>> + // a store that refuses one interface's writes
>> + struct Flaky(BTreeMap<u32, DhcpRecord>, u32);
>> +
>> + impl RecordStore for Flaky {
>> + fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
>> + RecordStore::get(&self.0, ifindex)
>> + }
>> +
>> + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
>> + if ifindex == self.1 {
>> + bail!("map full");
>> + }
>> + RecordStore::insert(&mut self.0, ifindex, rec)
>> + }
>> +
>> + fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
>> + RecordStore::mark(&mut self.0, ifindex, rec)
>> + }
>> +
>> + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
>> + if ifindex == self.1 {
>> + bail!("map busy");
>> + }
>> + RecordStore::remove(&mut self.0, ifindex)
>> + }
>> +
>> + fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
>> + RecordStore::entries(&self.0)
>> + }
>> + }
>> +
>> + #[test]
>> + fn bad_input_leaves_its_interface_alone() {
>> + let mut map = BTreeMap::new();
>> + map.insert(1, entry(VNET));
>> + let bad = Record {
>> + lease: 0,
>> + ..record()
>> + };
>> + let ifaces = [
>> + iface("a", Some("vnet1"), true, Some(bad)),
>> + iface("b", Some("vnet1"), true, Some(record())),
>> + ];
>> + let present: HashMap<u32, &Iface> = (1..=2).zip(ifaces.iter()).collect();
>> + let (elsewhere, failed) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
>> + assert!(elsewhere.is_empty());
>> + assert_eq!(failed, 0);
>> + assert_eq!(map[&1], entry(VNET), "the old entry stays");
>> + assert_eq!(map[&2], entry(VNET), "the rest is written");
>> + }
>> +
>> + #[test]
>> + fn a_failed_write_does_not_stop_the_pass() {
>> + let mut map = Flaky(BTreeMap::new(), 2);
>> + let ifaces = [
>> + iface("a", Some("vnet1"), true, Some(record())),
>> + iface("b", Some("vnet1"), true, Some(record())),
>> + iface("c", Some("vnet1"), true, Some(record())),
>> + ];
>> + let present: HashMap<u32, &Iface> = (1..=3).zip(ifaces.iter()).collect();
>> + let (elsewhere, failed) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
>> + assert!(elsewhere.is_empty());
>> + assert_eq!(failed, 1);
>> + assert!(map.0.contains_key(&1));
>> + assert!(!map.0.contains_key(&2));
>> + assert!(map.0.contains_key(&3));
>> + }
>> +
>> + // a store whose listing fails
>> + struct Unlistable(BTreeMap<u32, DhcpRecord>);
>> +
>> + impl RecordStore for Unlistable {
>> + fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
>> + RecordStore::get(&self.0, ifindex)
>> + }
>> +
>> + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
>> + RecordStore::insert(&mut self.0, ifindex, rec)
>> + }
>> +
>> + fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
>> + RecordStore::mark(&mut self.0, ifindex, rec)
>> + }
>> +
>> + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
>> + RecordStore::remove(&mut self.0, ifindex)
>> + }
>> +
>> + fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
>> + bail!("map gone")
>> + }
>> + }
>> +
>> + #[test]
>> + fn a_failed_listing_fails_the_pass_before_it_writes() {
>> + let mut map = Unlistable(BTreeMap::new());
>> + map.0.insert(1, entry(OTHER));
>> + let ifaces = [iface("a", Some("vnet1"), true, Some(record()))];
>> + let present: HashMap<u32, &Iface> = [(1, &ifaces[0])].into_iter().collect();
>> + assert!(sync_records(&mut map, &present, &HashSet::new()).is_err());
>> + assert_eq!(map.0[&1].vnet, OTHER);
>> + }
>> +
>> + #[test]
>> + fn a_failed_removal_counts_like_a_failed_write() {
>> + let mut map = Flaky(BTreeMap::new(), 2);
>> + map.0.insert(2, entry(VNET));
>> + let ifaces = [iface("a", Some("vnet1"), true, Some(record()))];
>> + let present: HashMap<u32, &Iface> = [(1, &ifaces[0])].into_iter().collect();
>> + let (_, failed) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
>> + assert_eq!(failed, 1);
>> + assert!(map.0.contains_key(&1));
>> + assert!(map.0.contains_key(&2));
>> + }
>> +
>> + #[test]
>> + fn the_pass_leaves_entries_of_another_vnet_to_their_link() {
>> + let mut map = BTreeMap::new();
>> + map.insert(1, entry(OTHER)); // a plug the configs trail, with a record
>> + map.insert(2, DhcpRecord::removed(NO_VNET)); // moved to a plain bridge
>> + map.insert(3, DhcpRecord::removed(NO_VNET)); // the configs caught up with that
>> + let ifaces = [
>> + iface("a", Some("vnet1"), true, Some(record())),
>> + iface("b", Some("vnet1"), true, Some(record())),
>> + iface("c", None, false, None),
>> + ];
>> + let present: HashMap<u32, &Iface> = (1..=3).zip(ifaces.iter()).collect();
>> + let (elsewhere, _) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
>> +
>> + assert_eq!(elsewhere, HashSet::from([1, 2]));
>> +
>> + assert_eq!(map[&1].vnet, OTHER);
>> + assert_eq!(map[&2].vnet, NO_VNET);
>> + assert_eq!(
>> + map[&3].ip, 0,
>> + "a marker the configs agree with stays as it is"
>> + );
>> + }
>> +}
>> diff --git a/src/dhcp/types.rs b/src/dhcp/types.rs
>> new file mode 100644
>> index 0000000..71a1b94
>> --- /dev/null
>> +++ b/src/dhcp/types.rs
>> @@ -0,0 +1,64 @@
>> +//! Keep in sync with 'bpf/types.h'.
>> +
>> +/// Addresses, lease and mtu are stored in network byte order, so a reply copies them into the
>> +/// packet as-is.
>> +#[repr(C)]
>> +#[derive(Copy, Clone, Debug, PartialEq, Eq)]
>> +pub struct DhcpRecord {
>> + /// 0 = removed, the entry is a marker naming its vnet
>> + pub ip: u32,
>> + pub netmask: u32,
>> + /// 0 = not served
>> + pub router: u32,
>> + /// 0 = not served
>> + pub dns: u32,
>> + pub server_id: u32,
>> + pub lease: u32,
>> + /// 0 = not served
>> + pub mtu: u16,
>> + /// The vnet the writer saw the interface on, the id padded with zeros, all zero for none.
>> + /// Never read by the program.
>> + pub vnet: [u8; 8],
>> + /// The NIC's MAC, a request carrying another is passed on. Zero in a marker.
>> + pub mac: [u8; 6],
>> +}
>> +
>> +impl DhcpRecord {
>> + /// The marker a removal leaves behind, on the vnet the writer saw the interface on.
>> + pub fn removed(vnet: [u8; 8]) -> Self {
>> + Self {
>> + ip: 0,
>> + netmask: 0,
>> + router: 0,
>> + dns: 0,
>> + server_id: 0,
>> + lease: 0,
>> + mtu: 0,
>> + vnet,
>> + mac: [0; 6],
>> + }
>> + }
>> +}
>> +
>> +unsafe impl aya::Pod for DhcpRecord {}
>> +
>> +#[cfg(test)]
>> +mod layout {
>> + use core::mem::{offset_of, size_of};
>> +
>> + use super::*;
>> +
>> + #[test]
>> + fn matches_bpf_abi() {
>> + assert_eq!(size_of::<DhcpRecord>(), 40);
>> + assert_eq!(offset_of!(DhcpRecord, ip), 0);
>> + assert_eq!(offset_of!(DhcpRecord, netmask), 4);
>> + assert_eq!(offset_of!(DhcpRecord, router), 8);
>> + assert_eq!(offset_of!(DhcpRecord, dns), 12);
>> + assert_eq!(offset_of!(DhcpRecord, server_id), 16);
>> + assert_eq!(offset_of!(DhcpRecord, lease), 20);
>> + assert_eq!(offset_of!(DhcpRecord, mtu), 24);
>> + assert_eq!(offset_of!(DhcpRecord, vnet), 26);
>> + assert_eq!(offset_of!(DhcpRecord, mac), 34);
>> + }
>> +}
>> diff --git a/src/lib.rs b/src/lib.rs
>> index 572d861..a8c94ba 100644
>> --- a/src/lib.rs
>> +++ b/src/lib.rs
>> @@ -1,5 +1,8 @@
>> //! eBPF subsystems, one per concern, each behind a cargo feature of its name. Nothing here runs
>> //! on its own, consumers pull in only the subsystem they drive.
>>
>> +#[cfg(feature = "dhcp")]
>> +pub mod dhcp;
>> +
>> pub mod subsystem;
>> pub mod tc;
>> diff --git a/src/subsystem.rs b/src/subsystem.rs
>> index 3bf1634..6014dbe 100644
>> --- a/src/subsystem.rs
>> +++ b/src/subsystem.rs
>> @@ -42,7 +42,7 @@ const RUN_ROOT: &str = "/run/proxmox-ebpf";
>>
>> /// The subsystems in the order their programs run on an interface, the first runs first. A name
>> /// missing from the list fails the build.
>> -pub const RUN_ORDER: &[&str] = &[];
>> +pub const RUN_ORDER: &[&str] = &["dhcp"];
>>
>> const fn rank_of(name: &str) -> usize {
>> let mut i = 0;
>
>
>
>
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-manager v3 16/17] sdn: bring up the DHCP backends at boot before the guests start
2026-10-20 13:21 ` Stefan Hanreich
@ 2026-10-21 7:17 ` Hannes Laimer
0 siblings, 0 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-21 7:17 UTC (permalink / raw)
To: Stefan Hanreich, pve-devel
On 2026-10-20 15:22, Stefan Hanreich wrote:
> On 10/20/26 10:03 AM, Hannes Laimer wrote:
>> The eBPF DHCP responder keeps its programs and maps pinned in bpffs,
>> so they are gone after a reboot. The SDN commit helper now brings the
>> DHCP backends back up on every boot, not only when it has changes to
>> commit.
>>
>> The helper runs before the node and HA start the guests at boot.
>> Otherwise the first guest interface plug loads the responder with a
>> full pass, which holds the responder's lock exclusively. A guest
>> starting meanwhile waits for that lock. One that waits too long comes
>> up without the responder on its interface.
>>
>> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
>> ---
>> bin/pve-sdn-commit | 3 +++
>> services/pve-sdn-commit.service | 1 +
>> 2 files changed, 4 insertions(+)
>>
>> diff --git a/bin/pve-sdn-commit b/bin/pve-sdn-commit
>> index aa7e9b29..ed2d6935 100644
>> --- a/bin/pve-sdn-commit
>> +++ b/bin/pve-sdn-commit
>> @@ -96,6 +96,9 @@ sub sdn_changed {
>> return fabrics_changed();
>> }
>>
>> +# the dhcp backends bring back what a reboot took, pending changes or not
>> +PVE::Network::SDN::Dhcp::boot();
>
> Add some form of log message here. I've tested this and it worked but I'd
> like at least some form of confirmation that this ran successfully?
>
reasonable :) will add in v4
>> +
>> if (!sdn_changed()) {
>> print "No changes to SDN configuration detected, skipping reload\n";
>> exit 0;
>> diff --git a/services/pve-sdn-commit.service b/services/pve-sdn-commit.service
>> index ff723725..825926be 100644
>> --- a/services/pve-sdn-commit.service
>> +++ b/services/pve-sdn-commit.service
>> @@ -3,6 +3,7 @@ Description=Commit Proxmox VE SDN changes
>> DefaultDependencies=no
>> Wants=pve-cluster.service network.target
>> After=frr.service network.target corosync.service
>> +Before=pve-guests.service pve-ha-lrm.service
>>
>> [Service]
>> ExecStart=/usr/share/pve-manager/helpers/pve-sdn-commit
>
>
>
>
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-network v3 05/17] sdn: dhcp: update reservations when ip mappings change
2026-10-20 13:42 ` Stefan Hanreich
@ 2026-10-21 7:21 ` Hannes Laimer
0 siblings, 0 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-21 7:21 UTC (permalink / raw)
To: Stefan Hanreich, pve-devel
On 2026-10-20 15:42, Stefan Hanreich wrote:
> Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
>
> On 10/20/26 10:03 AM, Hannes Laimer wrote:
>> Creating, updating or deleting an IP mapping through the API changed
>> the IPAM and the MAC cache, but never the zone's dnsmasq reservation
>> file. dnsmasq kept serving the old reservation, or none for a new
>> mapping, until the guest's next start. Deleting a guest NIC's
>> addresses left its reservation in the file as well.
>>
>> A guest start, an API edit and the deletion of a NIC's addresses now
>> all rewrite the MAC's reservation from the cache, on the node the
>> change is made on. The other nodes catch up when the guest starts
>> there. The zone and its backend come from the applied config, and a
>> zone confined to other nodes is skipped. A failed update only warns,
>> since the IPAM records are already written.
>>
>> The rewrite reads the cache only once it holds the file's lock. A
>> read from before could undo a rewrite that ran in between. The lock
>> is on a separate file, since a lock on the reservation file would not
>> survive the rename that replaces it. The MAC's line is written in full
>> from the cache, so a guest gaining an IPv6 address next to its IPv4
>> one gets it reserved too. MACs and addresses are compared in lower
>> case, since the cache keeps each as its writer spelled it. A cache
>> entry without any address no longer keeps a stale line.
>>
>> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
>> ---
>> src/PVE/API2/Network/SDN/Ips.pm | 7 +
>> src/PVE/Network/SDN/Dhcp.pm | 48 +++---
>> src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 110 ++++++++----
>> src/PVE/Network/SDN/Dhcp/Plugin.pm | 7 +-
>> src/PVE/Network/SDN/Vnets.pm | 5 +-
>> src/test/run_test_vnets_blackbox.pl | 250 +++++++++++++++++++++++++++-
>> 6 files changed, 352 insertions(+), 75 deletions(-)
>>
>> diff --git a/src/PVE/API2/Network/SDN/Ips.pm b/src/PVE/API2/Network/SDN/Ips.pm
>> index 5ff05e75..d7b682dd 100644
>> --- a/src/PVE/API2/Network/SDN/Ips.pm
>> +++ b/src/PVE/API2/Network/SDN/Ips.pm
>> @@ -46,6 +46,8 @@ __PACKAGE__->register_method({
>> eval { PVE::Network::SDN::Vnets::del_ip($vnet, $ip, '', $mac); };
>> die "$@\n" if $@;
>>
>> + PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
>> +
>> return undef;
>> },
>> });
>> @@ -82,6 +84,8 @@ __PACKAGE__->register_method({
>>
>> PVE::Network::SDN::Vnets::add_ip($vnet, $ip, '', $mac, undef);
>>
>> + PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
>> +
>> return undef;
>> },
>> });
>> @@ -132,6 +136,9 @@ __PACKAGE__->register_method({
>> }
>>
>> die "$error\n" if $error;
>> +
>> + PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
>> +
>> return undef;
>> },
>> });
>> diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
>> index 65e40d40..ab5d6b5b 100644
>> --- a/src/PVE/Network/SDN/Dhcp.pm
>> +++ b/src/PVE/Network/SDN/Dhcp.pm
>> @@ -6,7 +6,6 @@ use warnings;
>> use PVE::Cluster;
>>
>> use PVE::Network::SDN;
>> -use PVE::Network::SDN::Ipams;
>> use PVE::Network::SDN::Subnets;
>> use PVE::Network::SDN::Dhcp::Plugin;
>> use PVE::Network::SDN::Dhcp::Dnsmasq;
>> @@ -22,37 +21,32 @@ sub plugin_types {
>> return PVE::Network::SDN::Dhcp::Plugin->lookup_types();
>> }
>>
>> -sub add_mapping {
>> - my ($vnetid, $mac, $ip4, $ip6) = @_;
>> +# re-apply a MAC's mapping on this node from the current records, best effort
>> +# since the record write stays authoritative. A caller that knows the guest
>> +# passes its vmid, so only that guest's config is read
>> +sub update_mapping {
>> + my ($vnetid, $mac, $vmid) = @_;
>>
>> - my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid);
>> - return if !$vnet;
>> + # a record without a MAC has no guest NIC behind it, so no backend serves it
>> + return if !$mac;
>>
>> - my $zoneid = $vnet->{zone};
>> - my $zone = PVE::Network::SDN::Zones::get_zone($zoneid);
>> + eval {
>> + # the backend to push to is the applied one, since a zone edited but
>> + # not applied has nothing running yet
>> + my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid, 1);
>> + return if !$vnet;
>>
>> - return if !$zone->{ipam} || !$zone->{dhcp};
>> + my $zoneid = $vnet->{zone};
>> + my $zone = PVE::Network::SDN::Zones::get_zone($zoneid, 1);
>> + return if !$zone || !$zone->{ipam} || !$zone->{dhcp};
>>
>> - my $dhcptype = $zone->{dhcp};
>> + # a zone confined to other nodes has no backend running here
>> + return if defined($zone->{nodes}) && !$zone->{nodes}->{ PVE::INotify::nodename() };
>>
>> - my $macdb = PVE::Network::SDN::Ipams::read_macdb();
>> - my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($dhcptype);
>> - $dhcp_plugin->add_ip_mapping($zoneid, $macdb, $mac, $ip4, $ip6);
>> -}
>> -
>> -sub remove_mapping {
>> - my ($vnetid, $mac) = @_;
>> -
>> - my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid);
>> - return if !$vnet;
>> -
>> - my $zoneid = $vnet->{zone};
>> - my $zone = PVE::Network::SDN::Zones::get_zone($zoneid);
>> -
>> - return if !$zone->{ipam} || !$zone->{dhcp};
>> -
>> - my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($zone->{dhcp});
>> - $dhcp_plugin->del_ip_mapping($zoneid, $mac);
>> + PVE::Network::SDN::Dhcp::Plugin->lookup($zone->{dhcp})
>> + ->update_ip_mapping($zoneid, $mac, $vmid);
>> + };
>> + warn "could not update dhcp mapping for $mac: $@" if $@;
>> }
>>
>> sub regenerate_config {
>> diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
>> index 5c9496c0..57ce1f73 100644
>> --- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
>> +++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
>> @@ -12,6 +12,7 @@ use File::Copy;
>> use Net::DBus;
>>
>> use PVE::RESTEnvironment qw(log_warn);
>> +use PVE::Network::SDN::Ipams;
>>
>> my $DNSMASQ_CONFIG_ROOT = '/etc/dnsmasq.d';
>> my $DNSMASQ_DEFAULT_ROOT = '/etc/default';
>> @@ -51,29 +52,65 @@ sub update_lease {
>> $manager->AddDhcpLease($ip4, $mac, \@hostname, undef, 0, 0, 0) if $ip4;
>> }
>>
>> -sub add_ip_mapping {
>> - my ($class, $dhcpid, $macdb, $mac, $ip4, $ip6) = @_;
>> +sub update_ip_mapping {
>> + my ($class, $dhcpid, $mac) = @_;
>>
>> my $ethers_file = ethers_file($dhcpid);
>> my $ethers_tmp_file = "$ethers_file.tmp";
>>
>> - my $reload = undef;
>> + my ($ip4, $ip6);
>> + my $reservation = undef;
>> + my $changed = undef;
>> +
>> + my $rewriteFn = sub {
>> + # read under the ethers lock. A snapshot from before it would sweep a
>> + # line another push wrote meanwhile, and an older push would put its
>> + # addresses back
>> +
>> + my $macdb = PVE::Network::SDN::Ipams::read_macdb();
>> + # the cache spells a MAC and an address as their writer did, while the
>> + # file holds one spelling of both. An entry without an address says
>> + # nothing about its MAC, so a line of it is stale
>> + my %macs;
>> + for my $key (sort keys $macdb->{macs}->%*) {
>> + my $entry = $macdb->{macs}->{$key};
>> + my %addresses =
>> + map { $_ => lc($entry->{$_}) } grep { defined($entry->{$_}) } qw(ip4 ip6);
>> + next if !%addresses;
>> + $macs{ lc($key) } = { %{ $macs{ lc($key) } // {} }, %addresses };
>> + }
>
> we could simplify this here as well with integrating case-folding into the read_macdb fn.
> The more I think about, the more I'd prefer a small object that provides an API similar
> to HashMaps that handle case-folding when querying / inserting / ... I think that'd save
> us lots of duplication.
>
like on 06/17 yes, not sure about the hash though, having the existing
functions do it should do the same i think
>> +
>> + my $entry = $macs{ lc($mac) };
>> + ($ip4, $ip6) = ($entry->{ip4}, $entry->{ip6}) if $entry;
>> + if ($ip4 || $ip6) {
>> + $reservation = $mac;
>> + $reservation .= ",$ip4" if $ip4;
>> + $reservation .= ",[$ip6]" if $ip6;
>> + # an external IPAM spells addresses its own way, while the file holds one spelling
>> + $reservation = lc($reservation);
>> + }
>>
>> - my $appendFn = sub {
>> + # the first reservation of a zone finds no file yet
>> + file_set_contents($ethers_file, '') if !-e $ethers_file;
>> open(my $in, '<', $ethers_file) or die "Could not open file '$ethers_file' $!\n";
>> open(my $out, '>', $ethers_tmp_file)
>> or die "Could not open file '$ethers_tmp_file' $!\n";
>>
>> - my $match = undef;
>> + my $kept = undef;
>>
>> - my $line_no = 0;
>> while (my $line = <$in>) {
>> - $line_no++;
>> chomp($line);
>> + next if $line !~ m/\S/;
>> my ($parsed_mac, $parsed_ip1, $parsed_ip2) = split(/,/, $line);
>>
>> - if (!defined($parsed_mac)) {
>> - warn "failed to parse MAC from $dhcpid ethers file on line $line_no: '$line'\n";
>> + # the MAC's own reservation is rewritten. An unchanged one stays in place
>> + if (lc($parsed_mac) eq lc($mac)) {
>> + if (defined($reservation) && !$kept && lc($line) eq $reservation) {
>> + $kept = 1;
>> + print $out "$line\n";
>> + } else {
>> + $changed = 1;
>> + }
>> next;
>> }
>>
>> @@ -81,50 +118,43 @@ sub add_ip_mapping {
>> if ($parsed_ip2) {
>> $parsed_ip4 = $parsed_ip1;
>> $parsed_ip6 = $parsed_ip2;
>> - } elsif (Net::IP::ip_is_ipv4($parsed_ip1)) {
>> + } elsif (defined($parsed_ip1) && Net::IP::ip_is_ipv4($parsed_ip1)) {
>> $parsed_ip4 = $parsed_ip1;
>> - } else {
>> + } elsif (defined($parsed_ip1)) {
>> $parsed_ip6 = $parsed_ip1;
>> }
>> $parsed_ip6 = $1 if $parsed_ip6 && $parsed_ip6 =~ m/\[(\S+)\]/;
>> + # a line from before addresses were lower-cased may spell one in upper case
>> + $parsed_ip6 = lc($parsed_ip6) if $parsed_ip6;
>>
>> # a line that no longer matches the cache is replaced by the cache's
>> # current one. Dropping it left that guest unknown to dnsmasq until
>> # its next start
>> if (
>> - !defined($macdb->{macs}->{$parsed_mac})
>> + !defined($macs{ lc($parsed_mac) })
>> || ($parsed_ip4
>> - && $macdb->{macs}->{$parsed_mac}->{'ip4'}
>> - && $macdb->{macs}->{$parsed_mac}->{'ip4'} ne $parsed_ip4)
>> + && $macs{ lc($parsed_mac) }->{'ip4'}
>> + && $macs{ lc($parsed_mac) }->{'ip4'} ne $parsed_ip4)
>> || ($parsed_ip6
>> - && $macdb->{macs}->{$parsed_mac}->{'ip6'}
>> - && $macdb->{macs}->{$parsed_mac}->{'ip6'} ne $parsed_ip6)
>> + && $macs{ lc($parsed_mac) }->{'ip6'}
>> + && $macs{ lc($parsed_mac) }->{'ip6'} ne $parsed_ip6)
>> ) {
>> - $reload = 1;
>> - # the MAC being added gets its line below, like a missing one
>> - my $current = $macdb->{macs}->{$parsed_mac};
>> - next if $parsed_mac eq $mac || !$current;
>> - my $replacement = $parsed_mac;
>> + $changed = 1;
>> + my $current = $macs{ lc($parsed_mac) };
>> + next if !$current;
>> + my $replacement = lc($parsed_mac);
>
> a dedicated class could simplify this here a lot
>
>> $replacement .= ",$current->{ip4}" if $current->{ip4};
>> $replacement .= ",[$current->{ip6}]" if $current->{ip6};
>> - print $out "$replacement\n" if $replacement ne $parsed_mac;
>> + print $out "$replacement\n";
>> next;
>> }
>>
>> - if ($parsed_mac eq $mac) {
>> - $match = 1 if $ip4 && $parsed_ip4 && $ip4;
>> - $match = 1 if $ip6 && $parsed_ip6 && $ip6;
>> - }
>> -
>> print $out "$line\n";
>> }
>>
>> - if (!$match) {
>> - my $reservation = $mac;
>> - $reservation .= ",$ip4" if $ip4;
>> - $reservation .= ",[$ip6]" if $ip6;
>> + if (defined($reservation) && !$kept) {
>> print $out "$reservation\n";
>> - $reload = 1;
>> + $changed = 1;
>> }
>>
>> close $in;
>> @@ -133,16 +163,22 @@ sub add_ip_mapping {
>> chmod 0644, $ethers_file;
>> };
>>
>> - PVE::Tools::lock_file($ethers_file, 10, $appendFn);
>> + # the rewrite replaces the reservation file, so a lock on the file itself
>> + # would not outlive the rename
>> + PVE::Tools::lock_file("$ethers_file.lock", 10, $rewriteFn);
>>
>> if ($@) {
>> - warn "Unable to add $mac to the dnsmasq configuration: $@\n";
>> + warn "Unable to update $mac in the dnsmasq configuration: $@\n";
>> return;
>> }
>>
>> - my $service_name = "dnsmasq\@$dhcpid";
>> - systemctl_service('reload', $service_name) if $reload;
>> - update_lease($dhcpid, $ip4, $mac);
>> + systemctl_service('reload', "dnsmasq\@$dhcpid") if $changed;
>> +
>> + # the reservation is written, so a lease refresh that fails only warns
>> + if ($ip4) {
>> + eval { update_lease($dhcpid, $ip4, $mac) };
>> + log_warn("could not update the dnsmasq lease of $mac: $@") if $@;
>> + }
>> }
>>
>> sub configure_subnet {
>> diff --git a/src/PVE/Network/SDN/Dhcp/Plugin.pm b/src/PVE/Network/SDN/Dhcp/Plugin.pm
>> index b5d32fa1..2fa8683c 100644
>> --- a/src/PVE/Network/SDN/Dhcp/Plugin.pm
>> +++ b/src/PVE/Network/SDN/Dhcp/Plugin.pm
>> @@ -22,8 +22,11 @@ sub private {
>> return $defaultData;
>> }
>>
>> -sub add_ip_mapping {
>> - my ($class, $dhcpid, $macdb, $mac, $ip4, $ip6) = @_;
>> +# the MAC's records changed, so make the backend serve the current ones. With no
>> +# record left its reservation goes. The vmid names the guest when the caller
>> +# knows it, which an API edit does not
>> +sub update_ip_mapping {
>> + my ($class, $dhcpid, $mac, $vmid) = @_;
>> die 'implement in sub class';
>> }
>>
>> diff --git a/src/PVE/Network/SDN/Vnets.pm b/src/PVE/Network/SDN/Vnets.pm
>> index c327a4be..84b08037 100644
>> --- a/src/PVE/Network/SDN/Vnets.pm
>> +++ b/src/PVE/Network/SDN/Vnets.pm
>> @@ -211,6 +211,8 @@ sub del_ips_from_mac {
>> PVE::Network::SDN::Vnets::del_ip($vnetid, $ip4, $hostname, $mac) if $ip4;
>> PVE::Network::SDN::Vnets::del_ip($vnetid, $ip6, $hostname, $mac) if $ip6;
>>
>> + PVE::Network::SDN::Dhcp::update_mapping($vnetid, $mac) if $ip4 || $ip6;
>> +
>> return ($ip4, $ip6);
>> }
>>
>> @@ -225,11 +227,12 @@ sub add_dhcp_mapping {
>> return if !$zone->{ipam} || !$zone->{dhcp};
>>
>> my ($ip4, $ip6) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
>> + # one push at the end covers the allocated and the existing records alike
>> add_next_free_cidr($vnetid, $name, $mac, "$vmid", undef, 1, 4) if !$ip4;
>> add_next_free_cidr($vnetid, $name, $mac, "$vmid", undef, 1, 6) if !$ip6;
>>
>> ($ip4, $ip6) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
>> - PVE::Network::SDN::Dhcp::add_mapping($vnetid, $mac, $ip4, $ip6) if $ip4 || $ip6;
>> + PVE::Network::SDN::Dhcp::update_mapping($vnetid, $mac, $vmid) if $ip4 || $ip6;
>> }
>>
>> 1;
>> diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
>> index 36225a92..d324538e 100755
>> --- a/src/test/run_test_vnets_blackbox.pl
>> +++ b/src/test/run_test_vnets_blackbox.pl
>> @@ -46,6 +46,7 @@ sub clear_test_state {
>> vnets_config => {},
>> macdb => {},
>> ipamdb => {},
>> + dnsmasq_calls => [],
>> ipam_config => {
>> 'ids' => {
>> 'pve' => {
>> @@ -220,7 +221,9 @@ $mocked_sdn_dhcp_dnsmasq->mock(
>> assert_dnsmasq_installed => sub { return 1; },
>> before_configure => sub { },
>> ethers_file => sub { return "/tmp/ethers"; },
>> - systemctl_service => sub { },
>> + systemctl_service => sub {
>> + push $test_state->{dnsmasq_calls}->@*, $_[0];
>> + },
>> update_lease => sub { },
>> );
>>
>> @@ -239,6 +242,11 @@ $mocked_rpc_env_obj->mock(
>> check_any => sub { return 1; },
>> );
>>
>> +my $mocked_pve_inotify = Test::MockModule->new('PVE::INotify');
>> +$mocked_pve_inotify->mock(
>> + nodename => sub { return 'localnode'; },
>> +);
>> +
>> my $mocked_pve_cluster_obj = Test::MockModule->new('PVE::Cluster');
>> $mocked_pve_cluster_obj->mock(
>> check_cfs_quorum => sub { return 1; },
>> @@ -288,6 +296,11 @@ sub create_zone {
>> return $zone;
>> }
>>
>> +sub update_zone {
>> + my ($zoneid, $params) = @_;
>> + PVE::API2::Network::SDN::Zones->update({ zone => $zoneid, %$params });
>> +}
>> +
>> sub get_vnet {
>> my ($id) = @_;
>> return eval { PVE::API2::Network::SDN::Vnets->read({ vnet => $id }); };
>> @@ -330,6 +343,16 @@ sub create_ip {
>> return PVE::API2::Network::SDN::Ips->ipcreate($param);
>> }
>>
>> +sub update_ip {
>> + my ($param) = @_;
>> + return PVE::API2::Network::SDN::Ips->ipupdate($param);
>> +}
>> +
>> +sub delete_ip {
>> + my ($param) = @_;
>> + return PVE::API2::Network::SDN::Ips->ipdelete($param);
>> +}
>> +
>> sub run_test {
>> my $test = shift;
>> clear_test_state();
>> @@ -977,18 +1000,229 @@ sub test_dnsmasq_stale_reservation_replaced {
>> };
>> PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "$stale,10.0.1.10\n$gone,10.0.1.12\n");
>>
>> - PVE::Network::SDN::Dhcp::Dnsmasq->add_ip_mapping(
>> - 'TESTZONE', $test_state->{macdb}, $added, '10.0.1.13', undef,
>> - );
>> + PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping('TESTZONE', $added);
>>
>> - my @lines = sort grep { /\S/ } split(/\n/, PVE::Tools::file_get_contents($TMP_ETHERS_FILE));
>> - eq_or_diff(
>> - \@lines,
>> - ["$stale,10.0.1.11", "$added,10.0.1.13"],
>> + is(
>> + PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
>> + "$stale,10.0.1.11\n$added,10.0.1.13\n",
>> "$test_name: the stale line is replaced, the unknown one dropped",
>> );
>> }
>>
>> run_test(\&test_dnsmasq_stale_reservation_replaced);
>>
>> +# -------------- dnsmasq mapping pushes
>> +
>> +sub test_dnsmasq_mapping_push {
>> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
>> + my $zoneid = "TESTZONE";
>> + my $vnetid = "testvnet";
>> + my $mac = "da:65:8f:18:9b:6f";
>> +
>> + create_zone({
>> + type => "simple",
>> + dhcp => "dnsmasq",
>> + ipam => "pve",
>> + zone => $zoneid,
>> + });
>> + create_vnet({
>> + type => "vnet",
>> + zone => $zoneid,
>> + vnet => $vnetid,
>> + });
>> + create_subnet({
>> + type => "subnet",
>> + vnet => $vnetid,
>> + subnet => "10.0.0.0/24",
>> + gateway => "10.0.0.1",
>> + 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
>> + });
>> +
>> + # the first push of a zone finds no ethers file yet
>> + unlink($TMP_ETHERS_FILE);
>> +
>> + my $ethers = sub { PVE::Tools::file_get_contents($TMP_ETHERS_FILE) };
>> + my $reloads = sub {
>> + my $count = scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*);
>> + $test_state->{dnsmasq_calls} = [];
>> + return $count;
>> + };
>> +
>> + eval { nic_start($vnetid, $mac, "999", "testhostname"); };
>> + if ($@) {
>> + fail("$test_name: nic_start: $@");
>> + return;
>> + }
>> + is($ethers->(), "$mac,10.0.0.100\n", "$test_name: guest start reserves the allocation");
>> + is($reloads->(), 1, "$test_name: guest start reloads once");
>> +
>> + # the records exist already, so the restart pushes them without a change
>> + nic_start($vnetid, $mac, "999", "testhostname");
>> + is($ethers->(), "$mac,10.0.0.100\n", "$test_name: restart keeps the reservation");
>> + is($reloads->(), 0, "$test_name: restart without a change does not reload");
>> +
>> + update_ip({
>> + zone => $zoneid,
>> + vnet => $vnetid,
>> + mac => $mac,
>> + ip => "10.0.0.150",
>> + });
>> + is($ethers->(), "$mac,10.0.0.150\n", "$test_name: mapping edit replaces the reservation");
>> + is($reloads->(), 1, "$test_name: mapping edit reloads once");
>> +
>> + # a push writes what the cache holds, so a repeated one rewrites nothing
>> + PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
>> + is($ethers->(), "$mac,10.0.0.150\n", "$test_name: a repeated push keeps the reservation");
>> + is($reloads->(), 0, "$test_name: a repeated push changes nothing");
>> +
>> + # the cache and the file may spell a MAC differently, but it is one MAC
>> + PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, uc($mac));
>> + is(
>> + $ethers->(),
>> + "$mac,10.0.0.150\n",
>> + "$test_name: a push spelled differently keeps the line as it is",
>> + );
>> + is($reloads->(), 0, "$test_name: and reloads nothing");
>> +
>> + delete_ip({
>> + zone => $zoneid,
>> + vnet => $vnetid,
>> + mac => $mac,
>> + ip => "10.0.0.150",
>> + });
>> + is($ethers->(), "", "$test_name: mapping delete drops the reservation");
>> + is($reloads->(), 1, "$test_name: mapping delete reloads once");
>> +
>> + # a record without a MAC has no NIC to serve, so nothing is pushed for it
>> + my @warnings;
>> + {
>> + local $SIG{__WARN__} = sub { push @warnings, @_; };
>> + create_ip({ zone => $zoneid, vnet => $vnetid, ip => "10.0.0.160" });
>> + delete_ip({ zone => $zoneid, vnet => $vnetid, ip => "10.0.0.160" });
>> + }
>> + is($reloads->(), 0, "$test_name: a record without a MAC is not pushed");
>> + eq_or_diff(\@warnings, [], "$test_name: and warns nothing");
>> +
>> + # a zone confined to other nodes runs no dnsmasq here, so an edit is not
>> + # pushed on this node
>> + update_zone($zoneid, { nodes => 'other' });
>> + create_ip({
>> + zone => $zoneid,
>> + vnet => $vnetid,
>> + mac => $mac,
>> + ip => "10.0.0.150",
>> + });
>> + is($ethers->(), "", "$test_name: a zone confined to other nodes gets no push here");
>> + is($reloads->(), 0, "$test_name: nothing to reload for it");
>> +}
>> +
>> +run_test(\&test_dnsmasq_mapping_push);
>> +
>> +sub test_dnsmasq_dual_stack_and_sweep {
>> + my $test_name = (split(/::/, (caller(0))[3]))[-1];
>> + my $zoneid = "TESTZONE";
>> + my $vnetid = "testvnet";
>> + my $mac = "da:65:8f:18:9b:6f";
>> +
>> + create_zone({
>> + type => "simple",
>> + dhcp => "dnsmasq",
>> + ipam => "pve",
>> + zone => $zoneid,
>> + });
>> + create_vnet({
>> + type => "vnet",
>> + zone => $zoneid,
>> + vnet => $vnetid,
>> + });
>> + create_subnet({
>> + type => "subnet",
>> + vnet => $vnetid,
>> + subnet => "10.0.0.0/24",
>> + gateway => "10.0.0.1",
>> + 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
>> + });
>> + create_subnet({
>> + type => "subnet",
>> + vnet => $vnetid,
>> + subnet => "8888::/64",
>> + gateway => "8888::1",
>> + 'dhcp-range' => ["start-address=8888::100,end-address=8888::200"],
>> + });
>> +
>> + # a reservation of a MAC the cache does not know is swept by the next
>> + # rewrite, and both addresses of the guest go on one line
>> + PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "aa:bb:cc:dd:ee:ff,10.0.0.77\n");
>> + $test_state->{dnsmasq_calls} = [];
>> +
>> + eval { nic_start($vnetid, $mac, "999", "testhostname"); };
>> + if ($@) {
>> + fail("$test_name: nic_start: $@");
>> + return;
>> + }
>> + is(
>> + PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
>> + "$mac,10.0.0.100,[8888::100]\n",
>> + "$test_name: dual-stack reservation on one line, the unknown MAC swept",
>> + );
>> + is(
>> + scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
>> + 1,
>> + "$test_name: one reload for the rewrite",
>> + );
>> +
>> + # an external IPAM may spell an address in upper case, but the file holds
>> + # one spelling
>> + $test_state->{macdb}->{macs}->{$mac}->{ip6} = '8888::AB';
>> + $test_state->{dnsmasq_calls} = [];
>> + PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
>> + is(
>> + PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
>> + "$mac,10.0.0.100,[8888::ab]\n",
>> + "$test_name: the reservation is written in one spelling",
>> + );
>> + is(
>> + scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
>> + 1,
>> + "$test_name: one reload for the changed address",
>> + );
>> +
>> + # an entry without an address, as a miss once left behind, says nothing
>> + # about its MAC, so a line of it is swept
>> + my $stale = 'da:65:8f:18:9b:70';
>> + $test_state->{macdb}->{macs}->{$stale} = { ip4 => undef, ip6 => undef };
>> + PVE::Tools::file_set_contents(
>> + $TMP_ETHERS_FILE,
>> + "$mac,10.0.0.100,[8888::ab]\n$stale,10.0.0.101\n",
>> + );
>> + PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
>> + is(
>> + PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
>> + "$mac,10.0.0.100,[8888::ab]\n",
>> + "$test_name: a line of a MAC cached without an address is swept",
>> + );
>> +
>> + # a line with an upper-case IPv6 address from before they were lower-cased is kept
>> + my $other = 'da:65:8f:18:9b:71';
>> + $test_state->{macdb}->{macs}->{$other} = { ip4 => '10.0.0.102', ip6 => '8888::AB' };
>> + PVE::Tools::file_set_contents(
>> + $TMP_ETHERS_FILE,
>> + "$mac,10.0.0.100,[8888::ab]\n$other,10.0.0.102,[8888::AB]\n",
>> + );
>> + $test_state->{dnsmasq_calls} = [];
>> + PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
>> + is(
>> + PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
>> + "$mac,10.0.0.100,[8888::ab]\n$other,10.0.0.102,[8888::AB]\n",
>> + "$test_name: a line of another MAC spelled by the old code is kept",
>> + );
>> + is(
>> + scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
>> + 0,
>> + "$test_name: without a reload",
>> + );
>> +}
>> +
>> +run_test(\&test_dnsmasq_dual_stack_and_sweep);
>> +
>> done_testing();
>
>
>
>
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH pve-container v3 14/17] net: report NIC plugs and removals to the SDN DHCP backends
2026-10-20 13:36 ` Stefan Hanreich
@ 2026-10-21 7:29 ` Hannes Laimer
0 siblings, 0 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-21 7:29 UTC (permalink / raw)
To: Stefan Hanreich, pve-devel
On 2026-10-20 15:37, Stefan Hanreich wrote:
> Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
>
> On 10/20/26 10:02 AM, Hannes Laimer wrote:
>> An SDN DHCP backend answers a guest NIC with the address IPAM holds
>> for its MAC. A NIC joining its bridge now hands its MAC to the SDN
>> plug, so the backend of its vnet can answer its first request.
>> Deleting a veth also reports it as gone to the backends, since their
>> state for an interface can outlive it.
>>
>> On a running container, a bridge change now pushes the DHCP mapping
>> for the new vnet, as adding a NIC or changing its MAC already does.
>>
>> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
>> ---
>> src/PVE/LXC.pm | 13 +++++++++++--
>> src/PVE/LXC/Config.pm | 3 ++-
>> src/lxc-pve-poststop-hook | 6 +++---
>> 3 files changed, 16 insertions(+), 6 deletions(-)
>>
>> diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
>> index 1f11490..c3cf9e7 100644
>> --- a/src/PVE/LXC.pm
>> +++ b/src/PVE/LXC.pm
>> @@ -1111,7 +1111,13 @@ sub net_tap_plug : prototype($$) {
>> my $create_firewall_bridges = $net->{firewall} && PVE::Firewall::Helpers::needs_fwbr($bridge);
>>
>> PVE::Network::SDN::Zones::tap_plug(
>> - $iface, $bridge, $tag, $create_firewall_bridges, $trunks, $rate,
>> + $iface,
>> + $bridge,
>> + $tag,
>> + $create_firewall_bridges,
>> + $trunks,
>> + $rate,
>> + { mac => $hwaddr },
>
> nit: I understand that this function already has quite a few arguments, but introducing a new
> $opts and then setting the MAC there seems somewhat awkward as well? A matter of taste though, I
> suppose..
>
yeah, `PVE::Network::tap_plug` underneath does the same for the 7th arg,
that's the only reason it is like this
>> );
>> PVE::Network::SDN::Zones::add_bridge_fdb($iface, $hwaddr, $bridge);
>>
>> @@ -1137,7 +1143,7 @@ sub update_net {
>> || safe_string_ne($oldnet->{name}, $newnet->{name})
>> ) {
>>
>> - PVE::Network::veth_delete($veth);
>> + PVE::Network::SDN::Zones::veth_delete($veth);
>>
>> if (safe_string_ne($oldnet->{hwaddr}, $newnet->{hwaddr})) {
>> eval {
>> @@ -1192,6 +1198,9 @@ sub update_net {
>> PVE::Network::SDN::Vnets::add_next_free_cidr(
>> $newnet->{bridge}, $conf->{hostname}, $newnet->{hwaddr}, $vmid, undef, 1,
>> );
>> + PVE::Network::SDN::Vnets::add_dhcp_mapping(
>> + $newnet->{bridge}, $newnet->{hwaddr}, $vmid, $conf->{hostname},
>> + );
>> }
>> eval { PVE::LXC::net_tap_plug($veth, $newnet) };
>> if (my $err = $@) {
>> diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
>> index 9823d00..b522413 100644
>> --- a/src/PVE/LXC/Config.pm
>> +++ b/src/PVE/LXC/Config.pm
>> @@ -12,6 +12,7 @@ use PVE::GuestHelpers;
>> use PVE::INotify;
>> use PVE::JSONSchema qw(get_standard_option);
>> use PVE::Network::SDN::Vnets;
>> +use PVE::Network::SDN::Zones;
>> use PVE::RESTEnvironment;
>> use PVE::Tools;
>>
>> @@ -1696,7 +1697,7 @@ sub vmconfig_hotplug_pending {
>> my $net = parse_lxc_network($conf->{$opt});
>> PVE::LXC::kill_dhclients($vmid, $net->{name}) if $net->{'host-managed'};
>>
>> - PVE::Network::veth_delete("veth${vmid}i$netid");
>> + PVE::Network::SDN::Zones::veth_delete("veth${vmid}i$netid");
>> print "delete ips from $opt\n";
>> eval {
>> PVE::Network::SDN::Vnets::del_ips_from_mac(
>> diff --git a/src/lxc-pve-poststop-hook b/src/lxc-pve-poststop-hook
>> index c8bbb79..c5d1059 100755
>> --- a/src/lxc-pve-poststop-hook
>> +++ b/src/lxc-pve-poststop-hook
>> @@ -11,7 +11,7 @@ use PVE::GuestHelpers;
>> use PVE::LXC::Config;
>> use PVE::LXC::Tools;
>> use PVE::LXC;
>> -use PVE::Network;
>> +use PVE::Network::SDN::Zones;
>> use PVE::RESTEnvironment;
>> use PVE::Storage;
>> use PVE::Tools;
>> @@ -49,8 +49,8 @@ PVE::LXC::Tools::lxc_hook(
>> my $ind = $1;
>> my $net = PVE::LXC::Config->parse_lxc_network($conf->{$k});
>> next if $net->{type} ne 'veth';
>> - # veth_delete tests with '-d /sys/class/net/$name' before running the command
>> - PVE::Network::veth_delete("veth${vmid}i$ind");
>> + # reports the unplug and deletes the veth only where it still exists
>> + PVE::Network::SDN::Zones::veth_delete("veth${vmid}i$ind");
>> }
>>
>> my $config_updated = 0;
>
>
>
>
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH proxmox-perl-rs v3 03/17] pve-rs: sdn: add bindings for the eBPF DHCP responder
2026-10-20 13:10 ` Stefan Hanreich
@ 2026-10-21 7:32 ` Hannes Laimer
0 siblings, 0 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-21 7:32 UTC (permalink / raw)
To: Stefan Hanreich, pve-devel
On 2026-10-20 15:10, Stefan Hanreich wrote:
> On 10/20/26 10:00 AM, Hannes Laimer wrote:
>> The responder answers with the records the caller hands in. The
>> caller has to build them under the responder's lock, or a full pass
>> could write older records over those a plug or record change built
>> later. So the lock is an object the caller takes before it builds the
>> records, and every call that hands in records is a method of that
>> lock.
>>
>> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
>> ---
>> pve-rs/Cargo.toml | 2 +
>> pve-rs/Makefile | 3 +
>> pve-rs/debian/control | 2 +
>> pve-rs/src/bindings/sdn/dhcp.rs | 229 ++++++++++++++++++++++++++++++++
>> pve-rs/src/bindings/sdn/mod.rs | 1 +
>> 5 files changed, 237 insertions(+)
>> create mode 100644 pve-rs/src/bindings/sdn/dhcp.rs
>>
>> diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
>> index 5ae9082..0adf00b 100644
>> --- a/pve-rs/Cargo.toml
>> +++ b/pve-rs/Cargo.toml
>> @@ -34,6 +34,7 @@ proxmox-apt = { version = "1.0.1", features = ["cache"] }
>> proxmox-apt-api-types = "3"
>> proxmox-base64 = "1"
>> proxmox-config-digest = "1"
>> +proxmox-ebpf = { version = "0.1", features = ["dhcp"] }
>> proxmox-frr = { version = "0.5.1" }
>> proxmox-http = { version = "1.0.2", features = ["client-sync", "client-trait"] }
>> proxmox-http-error = "1"
>> @@ -70,6 +71,7 @@ proxmox-wireguard = { version = "0.1.2" }
>> # proxmox-config-digest = { path = "../../proxmox/proxmox-config-digest" }
>> # proxmox-daemon = { path = "../../proxmox/proxmox-daemon" }
>> # proxmox-dns-api = { path = "../../proxmox/proxmox-dns-api" }
>> +# proxmox-ebpf = { path = "../../proxmox-ebpf" }
>> # proxmox-http-error = { path = "../../proxmox/proxmox-http-error" }
>> # proxmox-http = { path = "../../proxmox/proxmox-http" }
>> # proxmox-human-byte = { path = "../../proxmox/proxmox-human-byte" }
>> diff --git a/pve-rs/Makefile b/pve-rs/Makefile
>> index bb1cd2d..ebe8bd3 100644
>> --- a/pve-rs/Makefile
>> +++ b/pve-rs/Makefile
>> @@ -32,6 +32,9 @@ PERLMOD_PACKAGES := \
>> PVE::RS::OpenId \
>> PVE::RS::ResourceScheduling::Static \
>> PVE::RS::ResourceScheduling::Dynamic \
>> + PVE::RS::SDN::Dhcp \
>> + PVE::RS::SDN::Dhcp::ExclusiveLock \
>> + PVE::RS::SDN::Dhcp::SharedLock \
>> PVE::RS::SDN::Fabrics \
>> PVE::RS::SDN::PrefixLists \
>> PVE::RS::SDN::RouteMaps \
>> diff --git a/pve-rs/debian/control b/pve-rs/debian/control
>> index 70367dc..521d966 100644
>> --- a/pve-rs/debian/control
>> +++ b/pve-rs/debian/control
>> @@ -20,6 +20,8 @@ Build-Depends: debhelper-compat (= 13),
>> librust-proxmox-apt-api-types-3+default-dev (>= 3.0.0-~~),
>> librust-proxmox-base64-1+default-dev,
>> librust-proxmox-config-digest-1+default-dev,
>> + librust-proxmox-ebpf-0.1+default-dev,
>> + librust-proxmox-ebpf-0.1+dhcp-dev,
>> librust-proxmox-frr-0.5+default-dev (>= 0.5.1-~~),
>> librust-proxmox-http-1+client-sync-dev (>= 1.0.2-~~),
>> librust-proxmox-http-1+client-trait-dev (>= 1.0.2-~~),
>> diff --git a/pve-rs/src/bindings/sdn/dhcp.rs b/pve-rs/src/bindings/sdn/dhcp.rs
>> new file mode 100644
>> index 0000000..a6cdc95
>> --- /dev/null
>> +++ b/pve-rs/src/bindings/sdn/dhcp.rs
>> @@ -0,0 +1,229 @@
>> +#[perlmod::package(name = "PVE::RS::SDN::Dhcp", lib = "pve_rs")]
>> +pub mod pve_rs_sdn_dhcp {
>> + //! The `PVE::RS::SDN::Dhcp` package.
>> + //!
>> + //! Bindings for the per-tap eBPF DHCP responder. Interfaces and the records they answer with
>> + //! are handed in here and served directly from the kernel. The calls that change the responder
>> + //! are methods of a lock object, so nothing writes to it without holding the lock in the mode
>> + //! the change needs.
>> +
>> + use std::net::Ipv4Addr;
>> +
>> + use anyhow::Error;
>> + use serde::{Deserialize, Deserializer};
>> +
>> + use proxmox_ebpf::dhcp::{DhcpSubsystem, Iface, Record, Unplugged};
>> +
>> + /// One interface's DHCP answer, the address and every option the reply carries.
>> + #[derive(Deserialize)]
>> + pub struct DhcpRecord {
>> + ip: Ipv4Addr,
>> + prefixlen: u8,
>> + server_id: Ipv4Addr,
>> + lease: u32,
>> + #[serde(default)]
>> + router: Option<Ipv4Addr>,
>> + #[serde(default)]
>> + dns: Option<Ipv4Addr>,
>> + #[serde(default)]
>> + mtu: Option<u16>,
>> + #[serde(deserialize_with = "mac_from_str")]
>> + mac: [u8; 6],
>
> proxmox-network-types has a MAC address type that would fit here if we add an as_slice() method
grrr, patch ordering in thunderbird..
but yes, like in 02/17 i'd rather not include another repo in this
series, id do a small follow up with the mac type change included
>
>> + }
>> +
>> + // a MAC as six hex pairs
>> + fn mac_from_str<'de, D: Deserializer<'de>>(d: D) -> Result<[u8; 6], D::Error> {
>> + let text = String::deserialize(d)?;
>> + let mut mac = [0u8; 6];
>> + let mut parts = text.split(':');
>> + for byte in &mut mac {
>> + let part = parts
>> + .next()
>> + .ok_or_else(|| serde::de::Error::custom(format!("MAC {text} is too short")))?;
>> + if part.len() != 2 || !part.bytes().all(|b| b.is_ascii_hexdigit()) {
>> + return Err(serde::de::Error::custom(format!("MAC {text} is malformed")));
>> + }
>> + *byte = u8::from_str_radix(part, 16)
>> + .map_err(|_| serde::de::Error::custom(format!("MAC {text} is malformed")))?;
>> + }
>> + if parts.next().is_some() {
>> + return Err(serde::de::Error::custom(format!("MAC {text} is too long")));
>> + }
>> + Ok(mac)
>> + }
>
> would become obsolete if using the MacAddr type
>
>> + /// One guest interface, the vnet it sits on if any, whether the responder runs on it and what
>> + /// it answers there.
>> + #[derive(Deserialize)]
>> + pub struct DhcpIface {
>> + name: String,
>> + #[serde(default)]
>> + vnet: Option<String>,
>> + serve: bool,
>> + #[serde(default)]
>> + record: Option<DhcpRecord>,
>> + }
>> +
>> + /// Where an unplugged interface went, the vnet it sits on now, none for a plain bridge.
>> + #[derive(Deserialize)]
>> + pub struct DhcpTarget {
>> + #[serde(default)]
>> + vnet: Option<String>,
>> + }
>> +
>> + impl From<DhcpRecord> for Record {
>> + fn from(r: DhcpRecord) -> Self {
>> + Record {
>> + ip: r.ip,
>> + prefixlen: r.prefixlen,
>> + server_id: r.server_id,
>> + lease: r.lease,
>> + router: r.router,
>> + dns: r.dns,
>> + mtu: r.mtu,
>> + mac: r.mac,
>> + }
>> + }
>> + }
>> +
>> + impl From<DhcpIface> for Iface {
>> + fn from(i: DhcpIface) -> Self {
>> + Iface {
>> + name: i.name,
>> + vnet: i.vnet,
>> + serve: i.serve,
>> + record: i.record.map(Record::from),
>> + }
>> + }
>> + }
>> +
>> + pub(super) fn to_ifaces(ifaces: Vec<DhcpIface>) -> Vec<Iface> {
>> + ifaces.into_iter().map(Iface::from).collect()
>> + }
>> +
>> + /// A tap unplug. Without a target the interface is gone or going and takes its record with
>> + /// it. With one it was plugged onto a bridge the responder does not serve and keeps a marker
>> + /// naming that place.
>> + #[export]
>> + pub fn detach(iface: &str, target: Option<DhcpTarget>) -> Result<(), Error> {
>> + let unplugged = match target {
>> + None => Unplugged::Gone,
>> + Some(target) => Unplugged::MovedTo(target.vnet),
>> + };
>> + DhcpSubsystem::new().detach(iface, unplugged)
>> + }
>> +}
>> +
>> +#[perlmod::package(name = "PVE::RS::SDN::Dhcp::SharedLock", lib = "pve_rs")]
>> +pub mod pve_rs_sdn_dhcp_shared_lock {
>> + //! The `PVE::RS::SDN::Dhcp::SharedLock` package.
>> + //!
>> + //! The responder's lock taken shared, what a tap plug and a guest's own record change hold.
>> + //! Released when the object goes away.
>> +
>> + use anyhow::Error;
>> + use perlmod::Value;
>> +
>> + use proxmox_ebpf::dhcp::{DhcpSubsystem, Iface, Outcome, Record, SharedLock};
>> +
>> + use super::pve_rs_sdn_dhcp::{DhcpIface, DhcpRecord, to_ifaces};
>> +
>> + /// The lock as Perl holds it.
>> + pub struct PerlSharedLock {
>> + lock: SharedLock,
>> + }
>> +
>> + perlmod::declare_magic!(Box<PerlSharedLock> : &PerlSharedLock as "PVE::RS::SDN::Dhcp::SharedLock");
>> +
>> + /// Class method: take the lock. Plugs hold it next to each other and wait only behind a full
>> + /// pass or a record change.
>> + #[export]
>> + pub fn new(#[raw] class: Value) -> Result<Value, Error> {
>> + let lock = DhcpSubsystem::new().lock_shared()?;
>> + Ok(perlmod::instantiate_magic!(&class, MAGIC => Box::new(PerlSharedLock { lock })))
>> + }
>> +
>> + /// Method: a tap plug onto a vnet with the record it answers. Without a record the interface
>> + /// gets no answer. Returns false when nothing is loaded yet. An interface gone already counts
>> + /// as done.
>> + #[export]
>> + pub fn attach(
>> + #[try_from_ref] this: &PerlSharedLock,
>> + iface: String,
>> + vnet: String,
>> + record: Option<DhcpRecord>,
>> + ) -> Result<bool, Error> {
>> + let iface = Iface {
>> + name: iface,
>> + vnet: Some(vnet),
>> + serve: true,
>> + record: record.map(Record::from),
>> + };
>> + Ok(this.lock.attach(&iface)? == Outcome::Done)
>> + }
>> +
>> + /// Method: a record change for the listed interfaces by a caller holding the guest lock.
>> + /// Returns false when nothing is loaded to write into, so a full pass is due.
>> + #[export]
>> + pub fn update(
>> + #[try_from_ref] this: &PerlSharedLock,
>> + ifaces: Vec<DhcpIface>,
>> + ) -> Result<bool, Error> {
>> + Ok(this.lock.update(&to_ifaces(ifaces))? == Outcome::Done)
>> + }
>> +}
>> +
>> +#[perlmod::package(name = "PVE::RS::SDN::Dhcp::ExclusiveLock", lib = "pve_rs")]
>> +pub mod pve_rs_sdn_dhcp_exclusive_lock {
>> + //! The `PVE::RS::SDN::Dhcp::ExclusiveLock` package.
>> + //!
>> + //! The responder's lock taken exclusively, what a full pass and a record change hold. Released
>> + //! when the object goes away.
>> +
>> + use anyhow::Error;
>> + use perlmod::Value;
>> +
>> + use proxmox_ebpf::dhcp::{DhcpSubsystem, ExclusiveLock, Outcome};
>> +
>> + use super::pve_rs_sdn_dhcp::{DhcpIface, to_ifaces};
>> +
>> + /// The lock as Perl holds it.
>> + pub struct PerlExclusiveLock {
>> + lock: ExclusiveLock,
>> + }
>> +
>> + perlmod::declare_magic!(Box<PerlExclusiveLock> : &PerlExclusiveLock as "PVE::RS::SDN::Dhcp::ExclusiveLock");
>> +
>> + /// Class method: take the lock, waiting out every plug in flight.
>> + #[export]
>> + pub fn new(#[raw] class: Value) -> Result<Value, Error> {
>> + let lock = DhcpSubsystem::new().lock_exclusive()?;
>> + Ok(perlmod::instantiate_magic!(&class, MAGIC => Box::new(PerlExclusiveLock { lock })))
>> + }
>> +
>> + /// Method: the full pass over every guest interface of the cluster. Those found here get
>> + /// their entry and a link where served.
>> + #[export]
>> + pub fn apply(
>> + #[try_from_ref] this: &PerlExclusiveLock,
>> + ifaces: Vec<DhcpIface>,
>> + ) -> Result<(), Error> {
>> + this.lock.apply(&to_ifaces(ifaces))
>> + }
>> +
>> + /// Method: a record change for the listed interfaces. Those not found here are skipped.
>> + /// Returns false when nothing is loaded to write into, so a full pass is due.
>> + #[export]
>> + pub fn update(
>> + #[try_from_ref] this: &PerlExclusiveLock,
>> + ifaces: Vec<DhcpIface>,
>> + ) -> Result<bool, Error> {
>> + Ok(this.lock.update(&to_ifaces(ifaces))? == Outcome::Done)
>> + }
>> +
>> + /// Method: detach the responder everywhere and drop its pinned state.
>> + #[export]
>> + pub fn clear(#[try_from_ref] this: &PerlExclusiveLock) -> Result<(), Error> {
>> + this.lock.clear()
>> + }
>> +}
>> diff --git a/pve-rs/src/bindings/sdn/mod.rs b/pve-rs/src/bindings/sdn/mod.rs
>> index dcae046..4b99b8f 100644
>> --- a/pve-rs/src/bindings/sdn/mod.rs
>> +++ b/pve-rs/src/bindings/sdn/mod.rs
>> @@ -1,3 +1,4 @@
>> +pub(crate) mod dhcp;
>> pub(crate) mod fabrics;
>> pub(crate) mod prefix_lists;
>> pub(crate) mod route_maps;
>
>
>
>
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF
2026-10-20 13:11 ` [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Stefan Hanreich
@ 2026-10-21 7:35 ` Hannes Laimer
0 siblings, 0 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-21 7:35 UTC (permalink / raw)
To: Stefan Hanreich, pve-devel
On 2026-10-20 15:11, Stefan Hanreich wrote:
> Did some testing:
>
> * PVE IPAM / All zones / eBPF backend
> * PVE IPAM / Simple zone / dnsmasq backend
> * Netbox IPAM / All zones / eBPF backend
> * Netbox IPAM / Simple zone / dnsmasq backend
> * switching from dnsmasq -> eBPF and vice-versa
> -> pre-existing: dnsmasq doesn't clean its config folder
> * changing bridge on network devices
> * changing subnets on the vnet and then reacquiring a lease
>
> Everything worked really well, also the pre-existing dnsmasq zone. This
> revision seems a lot nicer already without the generation counter, which
> seemed a bit clunky. This is a more elegant solution imo.
>
> One thing I found:
>
> The new DHCP backend type needs to be added to proxmox-ve-rs as well,
> otherwise the config fails to parse (firewall). We should add fallback
> variants there...
could have caught that :) thanks!
>
> for code-related stuff see the respective patches.
Thanks for taking a look and the feedback! I'll prepare a v4
>
> On 10/20/26 10:00 AM, Hannes Laimer wrote:
>> This series adds a second DHCP backend, ebpf, next to dnsmasq. A zone
>> picks its backend. dnsmasq stays limited to simple zones, while ebpf
>> answers on the guest's own interface and works on every zone type. A
>> guest on a zone with the ebpf backend gets the address its NIC has in
>> the IPAM. There is no daemon per zone and no lease file. The answers
>> are held in the kernel and do not depend on the IPAM being reachable.
>>
>> The responder is a BPF program on the ingress of each guest interface
>> of a zone with the ebpf backend. It answers a request from the record
>> of that interface, and the exchange never reaches the bridge. A
>> request whose client MAC is not the interface's own passes on
>> unanswered. So a guest cannot read another guest's record, and a
>> nested client does not get the outer guest's address.
>>
>> An edit through /cluster/sdn/vnets/{vnet}/ips reaches the guest's node
>> by a detached call to a new node endpoint, POST
>> /nodes/{node}/sdn/dhcp-mapping. The guest sees the change at its next
>> renewal, so the lease time bounds how long an old answer survives.
>> Subnets get a dhcp-lease-time property for both backends. Without it
>> dnsmasq hands out infinite leases and the responder ten minutes. A
>> failed push is logged in the journal, and the responder on the guest's
>> node catches up at the node's next apply or the guest's next start. An
>> apply also attaches the responder to running guests, so a zone
>> switched to ebpf serves them without a restart.
>>
>> Changes reach the kernel in the order they read their input. So no
>> change overwrites a newer one with older input. A hotplug moves the
>> interface before the guest config names the new bridge. An apply or an
>> edit that read the older config in between leaves the moved interface
>> as its plug left it. A change that fails only warns, and what was
>> served stays served. Nothing of the responder survives a reboot, so it
>> is loaded again at boot before the guests start.
>>
>> Only DHCPv4 is answered. A request the guest tagged itself passes
>> unanswered as well, so a trunk port keeps working as before. The
>> request is answered on the guest interface, so the guest's firewall
>> rules, its DHCP firewall option and its rate limit do not apply to it.
>> The reply leaves through the interface's egress, where the rate limit
>> shapes it. A request the responder does not answer passes on to the
>> bridge and through the firewall as before. Mappings changed directly
>> in an external IPAM are not picked up, as with dnsmasq.
>>
>> In patch order, the series adds
>> - proxmox-ebpf, the dhcp subsystem, a tc program answering on the
>> guest interface and the per-interface map it answers from
>> - pve-rs, the bindings
>> - pve-network, the ebpf backend, the hooks that keep a backend current
>> when a guest interface or a mapping changes, dhcp on every zone
>> type, and fixes to the dnsmasq reservations and the MAC cache
>> - qemu-server and pve-container, the reports of a guest NIC coming,
>> going or moving, and for VMs the push at a migration's resume
>> - pve-manager, the backend and subnet options in the UI, and the
>> backends brought up at boot ahead of the guests
>> - pve-docs, documentation
>>
>> - proxmox-ebpf applies on top of the v4 of its own series [1]
>> - pve-rs needs proxmox-ebpf with the dhcp feature
>> - pve-network needs pve-rs, at build and run time
>> - qemu-server, pve-container and pve-manager need pve-network
>> An older pve-network fails its apply on the unknown backend, so every
>> node needs the new one before a zone gets ebpf.
>>
>> Changes since v2:
>> * drop the generation counter. Every change takes the responder's
>> lock before it reads its input instead. An edit takes it
>> exclusively
>> * the resume at the end of a live migration pushes the guest's
>> mappings again on the target, so an edit made during the migration
>> reaches it
>> * new first pve-network commit, a dnsmasq fix that replaces a stale
>> reservation of another MAC instead of dropping it
>> * no MTU option for a zone without an MTU, where v2 handed out 1500
>> * a mapping edit without a MAC no longer reaches the backends
>> * rewrite commit messages, cover letter and parts of the docs
>> * rebase onto current master
>>
>> (pre-build packages on sani)
>>
>> [1] https://lore.proxmox.com/pve-devel/20261019142413.2255644-1-h.laimer@proxmox.com/
>>
>>
>> proxmox-ebpf:
>>
>> Hannes Laimer (2):
>> dhcp: add the per-tap DHCPv4 responder program
>> dhcp: add the subsystem that loads the responder and keeps its records
>>
>> Cargo.toml | 5 +
>> debian/control | 6 +-
>> src/bpf-shim/bpf/bpf_helpers.h | 3 +
>> src/bpf-shim/vmlinux.h | 1 +
>> src/dhcp/bpf/dhcp.bpf.c | 401 +++++++++++++
>> src/dhcp/bpf/types.h | 24 +
>> src/dhcp/mod.rs | 1031 ++++++++++++++++++++++++++++++++
>> src/dhcp/types.rs | 64 ++
>> src/lib.rs | 3 +
>> src/subsystem.rs | 2 +-
>> tests/common/mod.rs | 63 +-
>> tests/dhcp.rs | 825 +++++++++++++++++++++++++
>> 12 files changed, 2422 insertions(+), 6 deletions(-)
>> create mode 100644 src/dhcp/bpf/dhcp.bpf.c
>> create mode 100644 src/dhcp/bpf/types.h
>> create mode 100644 src/dhcp/mod.rs
>> create mode 100644 src/dhcp/types.rs
>> create mode 100644 tests/dhcp.rs
>>
>>
>> proxmox-perl-rs:
>>
>> Hannes Laimer (1):
>> pve-rs: sdn: add bindings for the eBPF DHCP responder
>>
>> pve-rs/Cargo.toml | 2 +
>> pve-rs/Makefile | 3 +
>> pve-rs/debian/control | 2 +
>> pve-rs/src/bindings/sdn/dhcp.rs | 229 ++++++++++++++++++++++++++++++++
>> pve-rs/src/bindings/sdn/mod.rs | 1 +
>> 5 files changed, 237 insertions(+)
>> create mode 100644 pve-rs/src/bindings/sdn/dhcp.rs
>>
>>
>> pve-network:
>>
>> Hannes Laimer (9):
>> sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it
>> sdn: dhcp: update reservations when ip mappings change
>> sdn: ipam: key the MAC cache by lower-case MAC, stop caching misses
>> sdn: dhcp: make the lease time configurable per subnet
>> sdn: dhcp: require a backend only if a zone on this node uses it
>> sdn: dhcp: add an eBPF backend answering on each guest interface
>> sdn: zones: report interface plugs and unplugs to the DHCP backends
>> sdn: dhcp: push IP mapping edits to the guest's node
>> sdn: zones: allow the ebpf DHCP backend on every zone type
>>
>> debian/libpve-network-perl.prerm | 13 +
>> src/PVE/API2/Network/SDN/Ips.pm | 10 +
>> src/PVE/API2/Network/SDN/Nodes/Status.pm | 37 +
>> src/PVE/API2/Network/SDN/Zones.pm | 11 +-
>> src/PVE/Network/SDN/Dhcp.pm | 200 ++-
>> src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 114 +-
>> src/PVE/Network/SDN/Dhcp/Ebpf.pm | 308 +++++
>> src/PVE/Network/SDN/Dhcp/Makefile | 2 +-
>> src/PVE/Network/SDN/Dhcp/Plugin.pm | 26 +-
>> src/PVE/Network/SDN/Ipams.pm | 61 +-
>> src/PVE/Network/SDN/SubnetPlugin.pm | 11 +
>> src/PVE/Network/SDN/Vnets.pm | 5 +-
>> src/PVE/Network/SDN/Zones.pm | 29 +-
>> src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 32 +
>> src/PVE/Network/SDN/Zones/FaucetPlugin.pm | 1 +
>> src/PVE/Network/SDN/Zones/QinQPlugin.pm | 7 +
>> src/PVE/Network/SDN/Zones/SimplePlugin.pm | 2 +-
>> src/PVE/Network/SDN/Zones/VlanPlugin.pm | 7 +
>> src/PVE/Network/SDN/Zones/VxlanPlugin.pm | 25 +
>> src/test/run_test_vnets_blackbox.pl | 1442 ++++++++++++++++++++-
>> 20 files changed, 2263 insertions(+), 80 deletions(-)
>> create mode 100755 debian/libpve-network-perl.prerm
>> create mode 100644 src/PVE/Network/SDN/Dhcp/Ebpf.pm
>>
>>
>> qemu-server:
>>
>> Hannes Laimer (1):
>> network: sdn: keep the DHCP backends current on guest NICs
>>
>> src/PVE/CLI/qm.pm | 4 ++--
>> src/PVE/QemuServer.pm | 4 ++++
>> src/PVE/QemuServer/Network.pm | 19 +++++++++++++++++--
>> src/PVE/QemuServer/RunState.pm | 2 ++
>> src/usr/pve-bridge | 8 +++++++-
>> src/usr/pve-bridgedown | 4 ++--
>> 6 files changed, 34 insertions(+), 7 deletions(-)
>>
>>
>> pve-container:
>>
>> Hannes Laimer (1):
>> net: report NIC plugs and removals to the SDN DHCP backends
>>
>> src/PVE/LXC.pm | 13 +++++++++++--
>> src/PVE/LXC/Config.pm | 3 ++-
>> src/lxc-pve-poststop-hook | 6 +++---
>> 3 files changed, 16 insertions(+), 6 deletions(-)
>>
>>
>> pve-manager:
>>
>> Hannes Laimer (2):
>> ui: sdn: add zone DHCP backend selector and subnet DHCP options
>> sdn: bring up the DHCP backends at boot before the guests start
>>
>> bin/pve-sdn-commit | 3 +++
>> services/pve-sdn-commit.service | 1 +
>> www/manager6/sdn/SubnetEdit.js | 23 +++++++++++++++++++++++
>> www/manager6/sdn/zones/Base.js | 17 +++++++++++++++++
>> www/manager6/sdn/zones/SimpleEdit.js | 11 -----------
>> 5 files changed, 44 insertions(+), 11 deletions(-)
>>
>>
>> pve-docs:
>>
>> Hannes Laimer (1):
>> sdn: dhcp: add eBPF backend section
>>
>> pvesdn.adoc | 101 ++++++++++++++++++++++++++++++++++++++++++----------
>> 1 file changed, 83 insertions(+), 18 deletions(-)
>>
>>
>> Summary over all repositories:
>> 52 files changed, 5099 insertions(+), 128 deletions(-)
>>
>
>
>
>
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* superseded: [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (17 preceding siblings ...)
2026-10-20 13:11 ` [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Stefan Hanreich
@ 2026-10-21 8:46 ` Hannes Laimer
18 siblings, 0 replies; 43+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:46 UTC (permalink / raw)
To: pve-devel
superseded-by:
https://lore.proxmox.com/pve-devel/20261021084445.462195-1-h.laimer@proxmox.com/
On 2026-10-20 10:00, Hannes Laimer wrote:
> This series adds a second DHCP backend, ebpf, next to dnsmasq. A zone
> picks its backend. dnsmasq stays limited to simple zones, while ebpf
> answers on the guest's own interface and works on every zone type. A
> guest on a zone with the ebpf backend gets the address its NIC has in
> the IPAM. There is no daemon per zone and no lease file. The answers
> are held in the kernel and do not depend on the IPAM being reachable.
>
> The responder is a BPF program on the ingress of each guest interface
> of a zone with the ebpf backend. It answers a request from the record
> of that interface, and the exchange never reaches the bridge. A
> request whose client MAC is not the interface's own passes on
> unanswered. So a guest cannot read another guest's record, and a
> nested client does not get the outer guest's address.
>
> An edit through /cluster/sdn/vnets/{vnet}/ips reaches the guest's node
> by a detached call to a new node endpoint, POST
> /nodes/{node}/sdn/dhcp-mapping. The guest sees the change at its next
> renewal, so the lease time bounds how long an old answer survives.
> Subnets get a dhcp-lease-time property for both backends. Without it
> dnsmasq hands out infinite leases and the responder ten minutes. A
> failed push is logged in the journal, and the responder on the guest's
> node catches up at the node's next apply or the guest's next start. An
> apply also attaches the responder to running guests, so a zone
> switched to ebpf serves them without a restart.
>
> Changes reach the kernel in the order they read their input. So no
> change overwrites a newer one with older input. A hotplug moves the
> interface before the guest config names the new bridge. An apply or an
> edit that read the older config in between leaves the moved interface
> as its plug left it. A change that fails only warns, and what was
> served stays served. Nothing of the responder survives a reboot, so it
> is loaded again at boot before the guests start.
>
> Only DHCPv4 is answered. A request the guest tagged itself passes
> unanswered as well, so a trunk port keeps working as before. The
> request is answered on the guest interface, so the guest's firewall
> rules, its DHCP firewall option and its rate limit do not apply to it.
> The reply leaves through the interface's egress, where the rate limit
> shapes it. A request the responder does not answer passes on to the
> bridge and through the firewall as before. Mappings changed directly
> in an external IPAM are not picked up, as with dnsmasq.
>
> In patch order, the series adds
> - proxmox-ebpf, the dhcp subsystem, a tc program answering on the
> guest interface and the per-interface map it answers from
> - pve-rs, the bindings
> - pve-network, the ebpf backend, the hooks that keep a backend current
> when a guest interface or a mapping changes, dhcp on every zone
> type, and fixes to the dnsmasq reservations and the MAC cache
> - qemu-server and pve-container, the reports of a guest NIC coming,
> going or moving, and for VMs the push at a migration's resume
> - pve-manager, the backend and subnet options in the UI, and the
> backends brought up at boot ahead of the guests
> - pve-docs, documentation
>
> - proxmox-ebpf applies on top of the v4 of its own series [1]
> - pve-rs needs proxmox-ebpf with the dhcp feature
> - pve-network needs pve-rs, at build and run time
> - qemu-server, pve-container and pve-manager need pve-network
> An older pve-network fails its apply on the unknown backend, so every
> node needs the new one before a zone gets ebpf.
>
> Changes since v2:
> * drop the generation counter. Every change takes the responder's
> lock before it reads its input instead. An edit takes it
> exclusively
> * the resume at the end of a live migration pushes the guest's
> mappings again on the target, so an edit made during the migration
> reaches it
> * new first pve-network commit, a dnsmasq fix that replaces a stale
> reservation of another MAC instead of dropping it
> * no MTU option for a zone without an MTU, where v2 handed out 1500
> * a mapping edit without a MAC no longer reaches the backends
> * rewrite commit messages, cover letter and parts of the docs
> * rebase onto current master
>
> (pre-build packages on sani)
>
> [1] https://lore.proxmox.com/pve-devel/20261019142413.2255644-1-h.laimer@proxmox.com/
>
>
> proxmox-ebpf:
>
> Hannes Laimer (2):
> dhcp: add the per-tap DHCPv4 responder program
> dhcp: add the subsystem that loads the responder and keeps its records
>
> Cargo.toml | 5 +
> debian/control | 6 +-
> src/bpf-shim/bpf/bpf_helpers.h | 3 +
> src/bpf-shim/vmlinux.h | 1 +
> src/dhcp/bpf/dhcp.bpf.c | 401 +++++++++++++
> src/dhcp/bpf/types.h | 24 +
> src/dhcp/mod.rs | 1031 ++++++++++++++++++++++++++++++++
> src/dhcp/types.rs | 64 ++
> src/lib.rs | 3 +
> src/subsystem.rs | 2 +-
> tests/common/mod.rs | 63 +-
> tests/dhcp.rs | 825 +++++++++++++++++++++++++
> 12 files changed, 2422 insertions(+), 6 deletions(-)
> create mode 100644 src/dhcp/bpf/dhcp.bpf.c
> create mode 100644 src/dhcp/bpf/types.h
> create mode 100644 src/dhcp/mod.rs
> create mode 100644 src/dhcp/types.rs
> create mode 100644 tests/dhcp.rs
>
>
> proxmox-perl-rs:
>
> Hannes Laimer (1):
> pve-rs: sdn: add bindings for the eBPF DHCP responder
>
> pve-rs/Cargo.toml | 2 +
> pve-rs/Makefile | 3 +
> pve-rs/debian/control | 2 +
> pve-rs/src/bindings/sdn/dhcp.rs | 229 ++++++++++++++++++++++++++++++++
> pve-rs/src/bindings/sdn/mod.rs | 1 +
> 5 files changed, 237 insertions(+)
> create mode 100644 pve-rs/src/bindings/sdn/dhcp.rs
>
>
> pve-network:
>
> Hannes Laimer (9):
> sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it
> sdn: dhcp: update reservations when ip mappings change
> sdn: ipam: key the MAC cache by lower-case MAC, stop caching misses
> sdn: dhcp: make the lease time configurable per subnet
> sdn: dhcp: require a backend only if a zone on this node uses it
> sdn: dhcp: add an eBPF backend answering on each guest interface
> sdn: zones: report interface plugs and unplugs to the DHCP backends
> sdn: dhcp: push IP mapping edits to the guest's node
> sdn: zones: allow the ebpf DHCP backend on every zone type
>
> debian/libpve-network-perl.prerm | 13 +
> src/PVE/API2/Network/SDN/Ips.pm | 10 +
> src/PVE/API2/Network/SDN/Nodes/Status.pm | 37 +
> src/PVE/API2/Network/SDN/Zones.pm | 11 +-
> src/PVE/Network/SDN/Dhcp.pm | 200 ++-
> src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 114 +-
> src/PVE/Network/SDN/Dhcp/Ebpf.pm | 308 +++++
> src/PVE/Network/SDN/Dhcp/Makefile | 2 +-
> src/PVE/Network/SDN/Dhcp/Plugin.pm | 26 +-
> src/PVE/Network/SDN/Ipams.pm | 61 +-
> src/PVE/Network/SDN/SubnetPlugin.pm | 11 +
> src/PVE/Network/SDN/Vnets.pm | 5 +-
> src/PVE/Network/SDN/Zones.pm | 29 +-
> src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 32 +
> src/PVE/Network/SDN/Zones/FaucetPlugin.pm | 1 +
> src/PVE/Network/SDN/Zones/QinQPlugin.pm | 7 +
> src/PVE/Network/SDN/Zones/SimplePlugin.pm | 2 +-
> src/PVE/Network/SDN/Zones/VlanPlugin.pm | 7 +
> src/PVE/Network/SDN/Zones/VxlanPlugin.pm | 25 +
> src/test/run_test_vnets_blackbox.pl | 1442 ++++++++++++++++++++-
> 20 files changed, 2263 insertions(+), 80 deletions(-)
> create mode 100755 debian/libpve-network-perl.prerm
> create mode 100644 src/PVE/Network/SDN/Dhcp/Ebpf.pm
>
>
> qemu-server:
>
> Hannes Laimer (1):
> network: sdn: keep the DHCP backends current on guest NICs
>
> src/PVE/CLI/qm.pm | 4 ++--
> src/PVE/QemuServer.pm | 4 ++++
> src/PVE/QemuServer/Network.pm | 19 +++++++++++++++++--
> src/PVE/QemuServer/RunState.pm | 2 ++
> src/usr/pve-bridge | 8 +++++++-
> src/usr/pve-bridgedown | 4 ++--
> 6 files changed, 34 insertions(+), 7 deletions(-)
>
>
> pve-container:
>
> Hannes Laimer (1):
> net: report NIC plugs and removals to the SDN DHCP backends
>
> src/PVE/LXC.pm | 13 +++++++++++--
> src/PVE/LXC/Config.pm | 3 ++-
> src/lxc-pve-poststop-hook | 6 +++---
> 3 files changed, 16 insertions(+), 6 deletions(-)
>
>
> pve-manager:
>
> Hannes Laimer (2):
> ui: sdn: add zone DHCP backend selector and subnet DHCP options
> sdn: bring up the DHCP backends at boot before the guests start
>
> bin/pve-sdn-commit | 3 +++
> services/pve-sdn-commit.service | 1 +
> www/manager6/sdn/SubnetEdit.js | 23 +++++++++++++++++++++++
> www/manager6/sdn/zones/Base.js | 17 +++++++++++++++++
> www/manager6/sdn/zones/SimpleEdit.js | 11 -----------
> 5 files changed, 44 insertions(+), 11 deletions(-)
>
>
> pve-docs:
>
> Hannes Laimer (1):
> sdn: dhcp: add eBPF backend section
>
> pvesdn.adoc | 101 ++++++++++++++++++++++++++++++++++++++++++----------
> 1 file changed, 83 insertions(+), 18 deletions(-)
>
>
> Summary over all repositories:
> 52 files changed, 5099 insertions(+), 128 deletions(-)
>
^ permalink raw reply [flat|nested] 43+ messages in thread
end of thread, other threads:[~2026-10-21 8:48 UTC | newest]
Thread overview: 43+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-20 7:59 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Hannes Laimer
2026-10-20 7:59 ` [PATCH proxmox-ebpf v3 01/17] dhcp: add the per-tap DHCPv4 responder program Hannes Laimer
2026-10-20 7:59 ` [PATCH proxmox-ebpf v3 02/17] dhcp: add the subsystem that loads the responder and keeps its records Hannes Laimer
2026-10-20 13:09 ` Stefan Hanreich
2026-10-21 7:09 ` Hannes Laimer
2026-10-20 7:59 ` [PATCH proxmox-perl-rs v3 03/17] pve-rs: sdn: add bindings for the eBPF DHCP responder Hannes Laimer
2026-10-20 13:10 ` Stefan Hanreich
2026-10-21 7:32 ` Hannes Laimer
2026-10-20 7:59 ` [PATCH pve-network v3 04/17] sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it Hannes Laimer
2026-10-20 13:15 ` Stefan Hanreich
2026-10-20 7:59 ` [PATCH pve-network v3 05/17] sdn: dhcp: update reservations when ip mappings change Hannes Laimer
2026-10-20 13:42 ` Stefan Hanreich
2026-10-21 7:21 ` Hannes Laimer
2026-10-20 7:59 ` [PATCH pve-network v3 06/17] sdn: ipam: key the MAC cache by lower-case MAC, stop caching misses Hannes Laimer
2026-10-20 13:10 ` Stefan Hanreich
2026-10-21 6:17 ` Hannes Laimer
2026-10-20 7:59 ` [PATCH pve-network v3 07/17] sdn: dhcp: make the lease time configurable per subnet Hannes Laimer
2026-10-20 13:17 ` Stefan Hanreich
2026-10-20 7:59 ` [PATCH pve-network v3 08/17] sdn: dhcp: require a backend only if a zone on this node uses it Hannes Laimer
2026-10-20 13:17 ` Stefan Hanreich
2026-10-20 7:59 ` [PATCH pve-network v3 09/17] sdn: dhcp: add an eBPF backend answering on each guest interface Hannes Laimer
2026-10-20 13:10 ` Stefan Hanreich
2026-10-21 6:16 ` Hannes Laimer
2026-10-20 7:59 ` [PATCH pve-network v3 10/17] sdn: zones: report interface plugs and unplugs to the DHCP backends Hannes Laimer
2026-10-20 13:17 ` Stefan Hanreich
2026-10-20 7:59 ` [PATCH pve-network v3 11/17] sdn: dhcp: push IP mapping edits to the guest's node Hannes Laimer
2026-10-20 13:09 ` Stefan Hanreich
2026-10-21 6:44 ` Hannes Laimer
2026-10-20 8:00 ` [PATCH pve-network v3 12/17] sdn: zones: allow the ebpf DHCP backend on every zone type Hannes Laimer
2026-10-20 8:00 ` [PATCH qemu-server v3 13/17] network: sdn: keep the DHCP backends current on guest NICs Hannes Laimer
2026-10-20 13:38 ` Stefan Hanreich
2026-10-20 8:00 ` [PATCH pve-container v3 14/17] net: report NIC plugs and removals to the SDN DHCP backends Hannes Laimer
2026-10-20 13:36 ` Stefan Hanreich
2026-10-21 7:29 ` Hannes Laimer
2026-10-20 8:00 ` [PATCH pve-manager v3 15/17] ui: sdn: add zone DHCP backend selector and subnet DHCP options Hannes Laimer
2026-10-20 13:19 ` Stefan Hanreich
2026-10-20 8:00 ` [PATCH pve-manager v3 16/17] sdn: bring up the DHCP backends at boot before the guests start Hannes Laimer
2026-10-20 13:21 ` Stefan Hanreich
2026-10-21 7:17 ` Hannes Laimer
2026-10-20 8:00 ` [PATCH pve-docs v3 17/17] sdn: dhcp: add eBPF backend section Hannes Laimer
2026-10-20 13:11 ` [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v3 00/17] sdn: implement DHCP for all zones using eBPF Stefan Hanreich
2026-10-21 7:35 ` Hannes Laimer
2026-10-21 8:46 ` superseded: " Hannes Laimer
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox