* [PATCH proxmox-ebpf v2 01/16] dhcp: add per-tap responder BPF program
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH proxmox-ebpf v2 02/16] dhcp: add responder subsystem Hannes Laimer
` (14 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
Attached on tap ingress, the responder rewrites a request on an
interface with a record into the reply in place and redirects it back
out the tap. The exchange never reaches the bridge. Everything else
passes untouched, so attaching is harmless on interfaces that end up
unmanaged. A REQUEST for a stale address is NAKed, so edited records
converge within one lease.
The record is the interface's and names its NIC's MAC. A request
carrying another MAC passes on unanswered. That keeps a guest from
reading another guest's record by asking with its MAC, and a nested
client from being handed the outer guest's address. Equal subnets in two
zones do not mix either, each interface is answered from its own record.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Cargo.toml | 5 +
debian/control | 6 +-
src/bpf-shim/bpf/bpf_helpers.h | 3 +
src/bpf-shim/vmlinux.h | 1 +
src/dhcp/bpf/dhcp.bpf.c | 405 ++++++++++++++++
src/dhcp/bpf/types.h | 25 +
tests/common/mod.rs | 63 ++-
tests/dhcp.rs | 827 +++++++++++++++++++++++++++++++++
8 files changed, 1330 insertions(+), 5 deletions(-)
create mode 100644 src/dhcp/bpf/dhcp.bpf.c
create mode 100644 src/dhcp/bpf/types.h
create mode 100644 tests/dhcp.rs
diff --git a/Cargo.toml b/Cargo.toml
index 1e8703d..172042e 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -11,6 +11,7 @@ homepage = "https://proxmox.com"
exclude = [".cargo", ".gitignore", "Makefile", "build", "debian"]
[features]
+dhcp = []
# When enabled, BPF programs compile with -DBPF_DEBUG so their DBG() macros
# expand to bpf_printk. Off by default
bpf-debug = []
@@ -20,3 +21,7 @@ aya = "0.13"
anyhow = "1"
log = "0.4"
nix = { version = "0.29", features = ["fs", "net"] }
+
+[[test]]
+name = "dhcp"
+required-features = ["dhcp"]
diff --git a/debian/control b/debian/control
index d3a9cc8..2dc841a 100644
--- a/debian/control
+++ b/debian/control
@@ -37,14 +37,18 @@ Depends:
Provides:
librust-proxmox-ebpf+bpf-debug-dev (= ${binary:Version}),
librust-proxmox-ebpf+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf+dhcp-dev (= ${binary:Version}),
librust-proxmox-ebpf-0-dev (= ${binary:Version}),
librust-proxmox-ebpf-0+bpf-debug-dev (= ${binary:Version}),
librust-proxmox-ebpf-0+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0+dhcp-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1+bpf-debug-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1+dhcp-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1.0-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1.0+bpf-debug-dev (= ${binary:Version}),
- librust-proxmox-ebpf-0.1.0+default-dev (= ${binary:Version})
+ librust-proxmox-ebpf-0.1.0+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1.0+dhcp-dev (= ${binary:Version})
Description: Proxmox VE eBPF - Rust source code
Source code for Debianized Rust crate "proxmox-ebpf"
diff --git a/src/bpf-shim/bpf/bpf_helpers.h b/src/bpf-shim/bpf/bpf_helpers.h
index de3bd84..3118de2 100644
--- a/src/bpf-shim/bpf/bpf_helpers.h
+++ b/src/bpf-shim/bpf/bpf_helpers.h
@@ -29,6 +29,9 @@ extern long bpf_skb_load_bytes(const void *skb, __u32 offset, void *to, __u32 le
extern long bpf_skb_store_bytes(void *skb, __u32 offset, const void *from, __u32 len, __u64 flags);
extern long bpf_skb_pull_data(void *skb, __u32 len);
extern long bpf_skb_change_tail(void *skb, __u32 new_len, __u64 flags);
+extern long bpf_l4_csum_replace(void *skb, __u32 offset, __u64 from, __u64 to, __u64 flags);
extern long bpf_redirect(__u32 ifindex, __u64 flags);
+#define BPF_F_PSEUDO_HDR 0x10
+
#endif
diff --git a/src/bpf-shim/vmlinux.h b/src/bpf-shim/vmlinux.h
index 52954f2..d3ab11d 100644
--- a/src/bpf-shim/vmlinux.h
+++ b/src/bpf-shim/vmlinux.h
@@ -23,6 +23,7 @@ struct __sk_buff {
__u32 len;
__u32 ifindex;
__u32 mark;
+ __u32 vlan_present;
};
struct ethhdr {
diff --git a/src/dhcp/bpf/dhcp.bpf.c b/src/dhcp/bpf/dhcp.bpf.c
new file mode 100644
index 0000000..810688a
--- /dev/null
+++ b/src/dhcp/bpf/dhcp.bpf.c
@@ -0,0 +1,405 @@
+// Per-tap DHCPv4 responder. Attached on tap ingress, it answers DISCOVER and
+// REQUEST on interfaces with a record by rewriting the request into the reply
+// in place and redirecting it back out the same interface. The redirect
+// consumes the request, so nothing upstream on the bridge sees it. Everything
+// else, including requests on interfaces without a record, is handed on
+// untouched to whatever runs after this program on the interface.
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_endian.h>
+#include "types.h"
+#include "bpf_debug.h"
+
+char LICENSE[] SEC("license") = "GPL";
+
+// TC_ACT_UNSPEC hands the packet on, to the next tcx program and then to the
+// qdisc filters, any other verdict ends the chain right there
+#define TC_ACT_UNSPEC (-1)
+#define TC_ACT_SHOT 2
+
+#define ETH_HLEN 14
+#define ETH_P_IP 0x0800
+#define IP_PROTO_UDP 17
+
+#define BOOTP_LEN 236
+// replies are padded up to the minimal BOOTP message size (RFC 1542), some old
+// client stacks drop anything shorter
+#define BOOTP_MIN_LEN 300
+#define DHCP_MAGIC 0x63825363
+#define BOOTP_FLAG_BCAST 0x8000
+
+#define BOOTREQUEST 1
+#define BOOTREPLY 2
+
+#define DHCP_DISCOVER 1
+#define DHCP_OFFER 2
+#define DHCP_REQUEST 3
+#define DHCP_ACK 5
+#define DHCP_NAK 6
+
+#define OPT_PAD 0
+#define OPT_NETMASK 1
+#define OPT_ROUTER 3
+#define OPT_DNS 6
+#define OPT_MTU 26
+#define OPT_REQUESTED_IP 50
+#define OPT_LEASE 51
+#define OPT_MSGTYPE 53
+#define OPT_SERVER_ID 54
+#define OPT_WPAD 252
+#define OPT_END 255
+
+// the answer of every attached interface, keyed by ifindex
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __type(key, __u32);
+ __type(value, struct dhcp_record);
+ __uint(max_entries, 16384);
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __uint(pinning, LIBBPF_PIN_BY_NAME);
+} dhcp_iface_records SEC(".maps");
+
+struct bootp {
+ __u8 op;
+ __u8 htype;
+ __u8 hlen;
+ __u8 hops;
+ __u32 xid;
+ __u16 secs;
+ __u16 flags;
+ __u32 ciaddr;
+ __u32 yiaddr;
+ __u32 siaddr;
+ __u32 giaddr;
+ __u8 chaddr[16];
+ __u8 sname[64];
+ __u8 file[128];
+};
+
+struct reply_pkt {
+ struct iphdr ip;
+ struct udphdr udp;
+ struct bootp bp;
+ __u8 cookie[4];
+ __u8 opts[BOOTP_MIN_LEN - BOOTP_LEN - 4];
+} __attribute__((packed));
+
+// the responder has no address on the tap, replies carry a fixed locally
+// administered source MAC that nothing ever learns
+static const __u8 SRC_MAC[6] = { 0x02, 0x00, 0x00, 0x00, 0x00, 0x01 };
+static const __u8 BCAST_MAC[6] = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff };
+
+static __always_inline __u16 csum_fold(__u32 sum)
+{
+ sum = (sum & 0xffff) + (sum >> 16);
+ sum = (sum & 0xffff) + (sum >> 16);
+ return sum;
+}
+
+static __always_inline __u16 ip_csum(const __u8 *hdr)
+{
+ __u32 sum = 0;
+#pragma unroll
+ for (int i = 0; i < 10; i++) {
+ __u16 word;
+ __builtin_memcpy(&word, hdr + 2 * i, 2);
+ sum += word;
+ }
+ return ~csum_fold(sum);
+}
+
+// the pseudo-header part of the UDP checksum, the value a checksum the kernel
+// completes on the way out starts from
+static __always_inline __u16 udp_pseudo_csum(__u32 saddr, __u32 daddr, __u16 len)
+{
+ __u32 sum = (saddr & 0xffff) + (saddr >> 16) + (daddr & 0xffff) + (daddr >> 16);
+ sum += bpf_htons(IP_PROTO_UDP) + len;
+ return csum_fold(sum);
+}
+
+#define UDP_CHECK_OFF (ETH_HLEN + sizeof(struct iphdr) + 6)
+
+SEC("classifier")
+int tc_dhcp_ingress(struct __sk_buff *skb)
+{
+ __u32 min_len = ETH_HLEN + sizeof(struct iphdr) + sizeof(struct udphdr) + BOOTP_LEN + 4;
+
+ // the kernel strips a VLAN tag before this hook, a request the guest
+ // tagged itself belongs to a network of the guest's own and passes
+ if (skb->vlan_present)
+ return TC_ACT_UNSPEC;
+
+ // everything shorter is not a DHCP request
+ if (skb->len < min_len)
+ return TC_ACT_UNSPEC;
+
+ // the headers are read in place, a large packet from the guest is mostly
+ // outside the linear part and pulling it in would cost every one of them
+ struct {
+ struct ethhdr eth;
+ struct iphdr ip;
+ struct udphdr udp;
+ } __attribute__((packed)) hdr;
+ if (bpf_skb_load_bytes(skb, 0, &hdr, sizeof(hdr)))
+ return TC_ACT_UNSPEC;
+ if (hdr.eth.h_proto != bpf_htons(ETH_P_IP))
+ return TC_ACT_UNSPEC;
+ if (hdr.ip.version != 4 || hdr.ip.ihl != 5 || hdr.ip.protocol != IP_PROTO_UDP)
+ return TC_ACT_UNSPEC;
+ // a fragment is never a whole request, and past the first one the bytes at
+ // the UDP header offset are arbitrary payload
+ if (hdr.ip.frag_off & bpf_htons(0x3fff))
+ return TC_ACT_UNSPEC;
+ if (hdr.udp.dest != bpf_htons(67))
+ return TC_ACT_UNSPEC;
+
+ void *data = (void *)(long)skb->data;
+ void *data_end = (void *)(long)skb->data_end;
+
+ if (data + min_len > data_end) {
+ // a request may sit in a non-linear skb
+ if (bpf_skb_pull_data(skb, min_len))
+ return TC_ACT_UNSPEC;
+ data = (void *)(long)skb->data;
+ data_end = (void *)(long)skb->data_end;
+ if (data + min_len > data_end)
+ return TC_ACT_UNSPEC;
+ }
+
+ struct iphdr *ip = data + ETH_HLEN;
+
+ struct bootp *req = data + ETH_HLEN + sizeof(struct iphdr) + sizeof(struct udphdr);
+ if (req->op != BOOTREQUEST || req->htype != 1 || req->hlen != 6)
+ return TC_ACT_UNSPEC;
+
+ // a reply to a relayed request must go unicast to the relay instead of the
+ // client, leave those to a server that speaks relay
+ if (req->giaddr) {
+ DBG("dhcp: request relayed via %pI4 passed", &req->giaddr);
+ return TC_ACT_UNSPEC;
+ }
+
+ __u32 *magic = (void *)req + BOOTP_LEN;
+ if (*magic != bpf_htonl(DHCP_MAGIC))
+ return TC_ACT_UNSPEC;
+
+ // the answer is the interface's and only its NIC's MAC gets it, so a
+ // guest cannot read another one's record by asking with its MAC. The
+ // client identifier option is not honored either, the reply goes back
+ // to the hardware address the request came with. That is manual
+ // allocation in RFC 2131 terms
+ __u8 chaddr[6];
+ __builtin_memcpy(chaddr, req->chaddr, 6);
+ __u32 ifindex = skb->ifindex;
+ struct dhcp_record *rec = bpf_map_lookup_elem(&dhcp_iface_records, &ifindex);
+ // a removed record stays as a marker for the ordering and serves nothing
+ if (rec && !rec->ip)
+ rec = (void *)0;
+ if (!rec) {
+ DBG("dhcp: no record for ifindex %d", ifindex);
+ return TC_ACT_UNSPEC;
+ }
+ // a request carrying another MAC than the NIC's comes from a nested client
+ // or is spoofed, it is passed on unanswered like every frame the responder
+ // leaves alone
+ for (int i = 0; i < 6; i++) {
+ if (chaddr[i] != rec->mac[i]) {
+ DBG("dhcp: chaddr is not the NIC's on ifindex %d", ifindex);
+ return TC_ACT_UNSPEC;
+ }
+ }
+
+ __u8 msgtype = 0;
+ __u32 requested_ip = 0;
+ __u32 server_id = 0;
+ __u32 off = min_len;
+ // the options end with the UDP datagram. Whatever the IP datagram or the
+ // frame carries beyond it is padding or a trailer, not the guest's request
+ __u32 end = ETH_HLEN + sizeof(struct iphdr) + bpf_ntohs(hdr.udp.len);
+ if (end > ETH_HLEN + bpf_ntohs(ip->tot_len))
+ end = ETH_HLEN + bpf_ntohs(ip->tot_len);
+ if (end > skb->len)
+ end = skb->len;
+
+ for (int i = 0; i < 128; i++) {
+ __u8 code, len;
+ if (off >= end || bpf_skb_load_bytes(skb, off, &code, 1))
+ break;
+ if (code == OPT_END)
+ break;
+ if (code == OPT_PAD) {
+ off += 1;
+ continue;
+ }
+ if (off + 2 > end || bpf_skb_load_bytes(skb, off + 1, &len, 1))
+ break;
+ if (off + 2 + len > end)
+ break;
+ if (code == OPT_MSGTYPE && len == 1)
+ bpf_skb_load_bytes(skb, off + 2, &msgtype, 1);
+ else if (code == OPT_REQUESTED_IP && len == 4)
+ bpf_skb_load_bytes(skb, off + 2, &requested_ip, 4);
+ else if (code == OPT_SERVER_ID && len == 4)
+ bpf_skb_load_bytes(skb, off + 2, &server_id, 4);
+ off += 2 + len;
+ }
+
+ __u8 reply;
+ if (msgtype == DHCP_DISCOVER) {
+ reply = DHCP_OFFER;
+ } else if (msgtype == DHCP_REQUEST) {
+ // a REQUEST naming another server declines our offer and is
+ // answered, if at all, by that server
+ if (server_id && server_id != rec->server_id) {
+ DBG("dhcp: request selected server %pI4, passed", &server_id);
+ return TC_ACT_UNSPEC;
+ }
+ // a NAK on a stale address makes the client re-DISCOVER, which
+ // is how an edited record converges within one lease
+ __u32 want = requested_ip ? requested_ip : req->ciaddr;
+ reply = want == rec->ip ? DHCP_ACK : DHCP_NAK;
+ if (reply == DHCP_NAK)
+ DBG("dhcp: request for %pI4 against record %pI4, nak", &want, &rec->ip);
+ } else {
+ DBG("dhcp: message type %d passed", msgtype);
+ return TC_ACT_UNSPEC;
+ }
+
+ __u32 xid = req->xid;
+ __u16 flags = req->flags;
+ __u32 ciaddr = req->ciaddr;
+
+ struct reply_pkt buf = {};
+
+ buf.bp.op = BOOTREPLY;
+ buf.bp.htype = 1;
+ buf.bp.hlen = 6;
+ buf.bp.xid = xid;
+ buf.bp.flags = flags;
+ __builtin_memcpy(buf.bp.chaddr, chaddr, 6);
+
+ buf.cookie[0] = 0x63;
+ buf.cookie[1] = 0x82;
+ buf.cookie[2] = 0x53;
+ buf.cookie[3] = 0x63;
+
+ __u32 o = 0;
+ buf.opts[o++] = OPT_MSGTYPE;
+ buf.opts[o++] = 1;
+ buf.opts[o++] = reply;
+ buf.opts[o++] = OPT_SERVER_ID;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->server_id, 4);
+ o += 4;
+
+ if (reply != DHCP_NAK) {
+ buf.bp.yiaddr = rec->ip;
+
+ buf.opts[o++] = OPT_LEASE;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->lease, 4);
+ o += 4;
+ buf.opts[o++] = OPT_NETMASK;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->netmask, 4);
+ o += 4;
+ if (rec->router) {
+ buf.opts[o++] = OPT_ROUTER;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->router, 4);
+ o += 4;
+ }
+ if (rec->dns) {
+ buf.opts[o++] = OPT_DNS;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->dns, 4);
+ o += 4;
+ }
+ if (rec->mtu) {
+ buf.opts[o++] = OPT_MTU;
+ buf.opts[o++] = 2;
+ __builtin_memcpy(&buf.opts[o], &rec->mtu, 2);
+ o += 2;
+ }
+ // an empty WPAD answer, without it some windows versions
+ // probe for a proxy config indefinitely
+ buf.opts[o++] = OPT_WPAD;
+ buf.opts[o++] = 1;
+ buf.opts[o++] = '\n';
+ }
+ buf.opts[o++] = OPT_END;
+
+ // the zeroed tail of opts pads short replies out to the minimum, on the
+ // wire those zeros read as pad options after the end option
+ __u32 bootp_out = BOOTP_LEN + 4 + o;
+ if (bootp_out < BOOTP_MIN_LEN)
+ bootp_out = BOOTP_MIN_LEN;
+
+ __u32 payload = sizeof(struct udphdr) + bootp_out;
+
+ buf.ip.version = 4;
+ buf.ip.ihl = 5;
+ buf.ip.ttl = 64;
+ buf.ip.protocol = IP_PROTO_UDP;
+ buf.ip.tot_len = bpf_htons(sizeof(struct iphdr) + payload);
+ buf.ip.saddr = rec->server_id;
+ // a renewing client has an address and gets a unicast answer,
+ // everything else cannot be addressed yet
+ buf.ip.daddr = (reply != DHCP_NAK && ciaddr) ? ciaddr : 0xffffffff;
+ buf.ip.check = ip_csum((const __u8 *)&buf.ip);
+
+ buf.udp.source = bpf_htons(67);
+ buf.udp.dest = bpf_htons(68);
+ buf.udp.len = bpf_htons(payload);
+ buf.udp.check = 0;
+
+ __u32 wlen = sizeof(struct iphdr) + payload;
+ if (wlen > sizeof(buf))
+ return TC_ACT_UNSPEC;
+
+ if (bpf_skb_change_tail(skb, ETH_HLEN + wlen, 0))
+ return TC_ACT_UNSPEC;
+ // the request is gone from here on, a failed write drops the frame
+ // rather than forwarding what is left of it
+
+ // NAKs are always broadcast, and a client without an address that set the
+ // BROADCAST flag asked for a broadcast reply
+ __u8 eth_hdr[ETH_HLEN];
+ if (reply == DHCP_NAK || (!ciaddr && (flags & bpf_htons(BOOTP_FLAG_BCAST)))) {
+ __builtin_memcpy(eth_hdr, BCAST_MAC, 6);
+ } else {
+ __builtin_memcpy(eth_hdr, chaddr, 6);
+ }
+ __builtin_memcpy(eth_hdr + 6, SRC_MAC, 6);
+ eth_hdr[12] = 0x08;
+ eth_hdr[13] = 0x00;
+
+ if (bpf_skb_store_bytes(skb, 0, eth_hdr, ETH_HLEN, 0))
+ return TC_ACT_SHOT;
+ if (bpf_skb_store_bytes(skb, ETH_HLEN, &buf, wlen, 0))
+ return TC_ACT_SHOT;
+
+ // The reply reuses the request's skb, whose checksum state it cannot
+ // see. A request sent with checksum offload is completed by the kernel
+ // on the way out, it expects the pseudo-header sum in the field. Any
+ // other request needs the field valid as it is, and zero is valid. The
+ // helper below folds a checksum field once more when the kernel still
+ // has to complete it, so adding one to the zeroed field ends up as 1 in
+ // that state and as its complement otherwise. The low flag bits give
+ // the width of the field.
+ if (bpf_l4_csum_replace(skb, UDP_CHECK_OFF, 0, 1, BPF_F_PSEUDO_HDR | 2))
+ return TC_ACT_SHOT;
+ __u16 probe = 0;
+ if (bpf_skb_load_bytes(skb, UDP_CHECK_OFF, &probe, 2))
+ return TC_ACT_SHOT;
+ __u16 check = probe == 1 ? udp_pseudo_csum(buf.ip.saddr, buf.ip.daddr, buf.udp.len) : 0;
+ if (bpf_skb_store_bytes(skb, UDP_CHECK_OFF, &check, 2, 0))
+ return TC_ACT_SHOT;
+
+#ifdef BPF_DEBUG
+ __u32 yiaddr = buf.bp.yiaddr;
+ DBG("dhcp: reply type %d with %pI4 to ifindex %d", reply, &yiaddr, skb->ifindex);
+#endif
+ return bpf_redirect(skb->ifindex, 0);
+}
diff --git a/src/dhcp/bpf/types.h b/src/dhcp/bpf/types.h
new file mode 100644
index 0000000..d6ca261
--- /dev/null
+++ b/src/dhcp/bpf/types.h
@@ -0,0 +1,25 @@
+#ifndef PROXMOX_EBPF_DHCP_TYPES_H
+#define PROXMOX_EBPF_DHCP_TYPES_H
+
+// Keep in sync with ../types.rs
+// __u8/__u16/__u32 come from vmlinux.h
+
+// addresses, lease and mtu are stored in network byte order, so a
+// reply copies them into the packet as-is
+struct dhcp_record {
+ __u32 ip; // 0 = removed, the entry only keeps its generation
+ __u32 netmask;
+ __u32 router; // 0 = not served
+ __u32 dns; // 0 = not served
+ __u32 server_id;
+ __u32 lease;
+ __u16 mtu; // 0 = not served
+ __u8 vnet[8]; // the vnet the writer saw the interface on, zero for none, never read here
+ __u8 mac[6]; // the NIC's, a request carrying another is passed on
+ __u64 generation; // host order, the program never reads it
+};
+
+_Static_assert(sizeof(struct dhcp_record) == 48, "dhcp_record layout");
+_Static_assert(__builtin_offsetof(struct dhcp_record, generation) == 40, "dhcp_record layout");
+
+#endif
diff --git a/tests/common/mod.rs b/tests/common/mod.rs
index 7865dd8..016d8e1 100644
--- a/tests/common/mod.rs
+++ b/tests/common/mod.rs
@@ -28,13 +28,16 @@ pub struct SkBuff {
pub len: u32,
pub ifindex: u32,
pub mark: u32,
+ pub vlan_present: u32,
}
/// The skb handed to a program plus the buffer behind it. The helpers get the skb pointer and
-/// cast back to this, so the skb must stay the first field.
+/// cast back to this, so the skb must stay the first field. `partial` models a checksum the
+/// kernel still has to complete, the state a request sent with checksum offload carries.
#[repr(C)]
pub struct TestSkb {
skb: SkBuff,
+ pub partial: bool,
buf: Box<[u8; PKT_CAP]>,
}
@@ -50,7 +53,9 @@ impl TestSkb {
len: packet.len() as u32,
ifindex,
mark: 0,
+ vlan_present: 0,
},
+ partial: false,
buf,
};
t.sync();
@@ -63,6 +68,11 @@ impl TestSkb {
self.skb.data_end = base + self.skb.len as usize;
}
+ /// The frame arrived tagged, the kernel keeps the tag beside the data.
+ pub fn tag_vlan(&mut self) {
+ self.skb.vlan_present = 1;
+ }
+
pub fn packet(&self) -> &[u8] {
&self.buf[..self.skb.len as usize]
}
@@ -186,9 +196,10 @@ pub extern "C" fn bpf_skb_pull_data(skb: *mut c_void, len: u32) -> c_long {
#[unsafe(no_mangle)]
pub extern "C" fn bpf_skb_change_tail(skb: *mut c_void, new_len: u32, _flags: u64) -> c_long {
let t = unsafe { testskb(skb) };
- // the kernel refuses to cut into the headers it knows, this frame has its link header
- // only, and to grow past what fits the buffer
- let min_len = 14;
+ // the kernel refuses to cut into the headers it knows. tun sets the transport header on
+ // every frame it hands in, so an IPv4 UDP frame keeps 34 bytes. One with checksum offload
+ // keeps its checksum field as well. Growing past what fits the buffer is refused too
+ let min_len = if t.partial { 42 } else { 34 };
if new_len < min_len || new_len as usize > PKT_CAP {
return -EINVAL;
}
@@ -202,6 +213,50 @@ pub extern "C" fn bpf_skb_change_tail(skb: *mut c_void, new_len: u32, _flags: u6
0
}
+// the kernel's ones' complement primitives, csum_fold complements its result
+fn csum_add(a: u32, b: u32) -> u32 {
+ let (r, carry) = a.overflowing_add(b);
+ r + carry as u32
+}
+fn csum_fold(sum: u32) -> u16 {
+ let mut s = (sum & 0xffff) + (sum >> 16);
+ s = (s & 0xffff) + (s >> 16);
+ !(s as u16)
+}
+
+/// The kernel's field update, inet_proto_csum_replace4 for a 16-bit field. A partial checksum
+/// gets the difference folded in, any other is complemented around the update. So the same call
+/// ends in different values.
+#[unsafe(no_mangle)]
+pub extern "C" fn bpf_l4_csum_replace(
+ skb: *mut c_void,
+ offset: u32,
+ from: u64,
+ to: u64,
+ flags: u64,
+) -> c_long {
+ let t = unsafe { testskb(skb) };
+ let offset = offset as usize;
+ assert_eq!(flags & 0xf, 2, "the responder updates the 16-bit field");
+ assert!(
+ flags & 0x10 != 0,
+ "the responder passes the pseudo-header flag"
+ );
+ if offset + 2 > t.skb.len as usize {
+ return -EFAULT;
+ }
+ let field = u16::from_ne_bytes([t.buf[offset], t.buf[offset + 1]]) as u32;
+ // a 16-bit field takes 16-bit values, the kernel narrows both before the update
+ let (from, to) = (from as u16 as u32, to as u16 as u32);
+ let new = if t.partial {
+ !csum_fold(csum_add(csum_add(field, !from), to))
+ } else {
+ csum_fold(csum_add(csum_add(!field, !from), to))
+ };
+ t.buf[offset..offset + 2].copy_from_slice(&new.to_ne_bytes());
+ 0
+}
+
#[unsafe(no_mangle)]
pub extern "C" fn bpf_redirect(ifindex: u32, flags: u64) -> c_long {
REDIRECTED.with(|r| *r.borrow_mut() = Some((ifindex, flags)));
diff --git a/tests/dhcp.rs b/tests/dhcp.rs
new file mode 100644
index 0000000..1b98313
--- /dev/null
+++ b/tests/dhcp.rs
@@ -0,0 +1,827 @@
+//! The dhcp responder program run natively, crafted requests in and the rewritten reply out.
+
+mod common;
+
+use std::ffi::{c_int, c_void};
+
+use common::*;
+
+unsafe extern "C" {
+ fn tc_dhcp_ingress(skb: *mut SkBuff) -> c_int;
+ static dhcp_iface_records: u8;
+}
+
+const MAC: [u8; 6] = [0xda, 0x65, 0x8f, 0x18, 0x9b, 0x6f];
+const SRC_MAC: [u8; 6] = [0x02, 0x00, 0x00, 0x00, 0x00, 0x01];
+const BCAST: [u8; 6] = [0xff; 6];
+const IFINDEX: u32 = 42;
+
+const IP: [u8; 4] = [10, 0, 0, 100];
+const MASK: [u8; 4] = [255, 255, 255, 0];
+const GW: [u8; 4] = [10, 0, 0, 1];
+const DNS: [u8; 4] = [1, 1, 1, 1];
+const LEASE: u32 = 300;
+const MTU: u16 = 1500;
+
+const OPTS_OFF: usize = 14 + 20 + 8 + 236 + 4;
+
+const DISCOVER: &[u8] = &[53, 1, 1];
+const REQUEST: &[u8] = &[53, 1, 3];
+
+fn records_map() -> *const c_void {
+ (unsafe { &dhcp_iface_records } as *const u8).cast()
+}
+
+// the record map of a node, still empty
+fn setup_maps() {
+ register_map(records_map(), 4);
+}
+
+// the record map is keyed by the interface the request arrives on
+fn key() -> [u8; 4] {
+ IFINDEX.to_ne_bytes()
+}
+
+fn record(
+ ip: [u8; 4],
+ mask: [u8; 4],
+ router: [u8; 4],
+ dns: [u8; 4],
+ server: [u8; 4],
+ lease: u32,
+ mtu: u16,
+) -> [u8; 48] {
+ let mut r = [0u8; 48];
+ r[0..4].copy_from_slice(&ip);
+ r[4..8].copy_from_slice(&mask);
+ r[8..12].copy_from_slice(&router);
+ r[12..16].copy_from_slice(&dns);
+ r[16..20].copy_from_slice(&server);
+ r[20..24].copy_from_slice(&lease.to_be_bytes());
+ r[24..26].copy_from_slice(&mtu.to_be_bytes());
+ r[26..34].copy_from_slice(b"testvnet");
+ r[34..40].copy_from_slice(&MAC);
+ // the generation is the subsystem's, the program never reads it
+ r[40..48].copy_from_slice(&u64::MAX.to_ne_bytes());
+ r
+}
+
+fn setup_full_record() {
+ setup_maps();
+ map_insert(
+ records_map(),
+ &key(),
+ &record(IP, MASK, GW, DNS, GW, LEASE, MTU),
+ );
+}
+
+struct Request {
+ mac: [u8; 6],
+ xid: [u8; 4],
+ flags: [u8; 2],
+ ciaddr: [u8; 4],
+ giaddr: [u8; 4],
+ options: Vec<u8>,
+ dport: u16,
+ ethertype: [u8; 2],
+ ipproto: u8,
+ frag_off: [u8; 2],
+ ip_options: Vec<u8>,
+ op: u8,
+ htype: u8,
+ hlen: u8,
+ end: bool,
+ trailer: Vec<u8>,
+ // claimed on top of the real IP total length
+ tot_len_excess: u16,
+}
+
+impl Default for Request {
+ fn default() -> Self {
+ Request {
+ mac: MAC,
+ xid: [0xde, 0xad, 0xbe, 0xef],
+ flags: [0, 0],
+ ciaddr: [0; 4],
+ giaddr: [0; 4],
+ options: DISCOVER.to_vec(),
+ dport: 67,
+ ethertype: [0x08, 0x00],
+ ipproto: 17,
+ frag_off: [0, 0],
+ ip_options: Vec::new(),
+ op: 1,
+ htype: 1,
+ hlen: 6,
+ end: true,
+ trailer: Vec::new(),
+ tot_len_excess: 0,
+ }
+ }
+}
+
+impl Request {
+ fn build(&self) -> Vec<u8> {
+ assert_eq!(self.ip_options.len() % 4, 0);
+ let ihl = 5 + self.ip_options.len() / 4;
+ let payload = 8 + 236 + 4 + self.options.len() + usize::from(self.end);
+ let mut p = Vec::new();
+ // ethernet
+ p.extend_from_slice(&BCAST);
+ p.extend_from_slice(&self.mac);
+ p.extend_from_slice(&self.ethertype);
+ // ipv4
+ p.extend_from_slice(&[0x40 | ihl as u8, 0]);
+ p.extend_from_slice(&((ihl * 4 + payload) as u16 + self.tot_len_excess).to_be_bytes());
+ p.extend_from_slice(&[0, 0]); // id
+ p.extend_from_slice(&self.frag_off);
+ p.extend_from_slice(&[64, self.ipproto, 0, 0]);
+ p.extend_from_slice(&[0, 0, 0, 0]); // saddr
+ p.extend_from_slice(&[255, 255, 255, 255]); // daddr
+ p.extend_from_slice(&self.ip_options);
+ // udp
+ p.extend_from_slice(&68u16.to_be_bytes());
+ p.extend_from_slice(&self.dport.to_be_bytes());
+ p.extend_from_slice(&(payload as u16).to_be_bytes());
+ p.extend_from_slice(&[0, 0]);
+ // bootp
+ p.extend_from_slice(&[self.op, self.htype, self.hlen, 0]);
+ p.extend_from_slice(&self.xid);
+ p.extend_from_slice(&[0, 0]); // secs
+ p.extend_from_slice(&self.flags);
+ p.extend_from_slice(&self.ciaddr);
+ p.extend_from_slice(&[0; 8]); // yiaddr, siaddr
+ p.extend_from_slice(&self.giaddr);
+ p.extend_from_slice(&self.mac);
+ p.extend_from_slice(&[0; 10]); // chaddr padding
+ p.extend_from_slice(&[0; 192]); // sname, file
+ // cookie and options
+ p.extend_from_slice(&[0x63, 0x82, 0x53, 0x63]);
+ p.extend_from_slice(&self.options);
+ if self.end {
+ p.push(255);
+ }
+ // bytes past the datagram, padding or a trailer the frame carries
+ p.extend_from_slice(&self.trailer);
+ p
+ }
+
+ fn run(&self) -> (c_int, TestSkb) {
+ self.run_on(IFINDEX)
+ }
+
+ fn run_on(&self, ifindex: u32) -> (c_int, TestSkb) {
+ let mut skb = TestSkb::new(&self.build(), ifindex);
+ let verdict = skb.run(tc_dhcp_ingress);
+ (verdict, skb)
+ }
+}
+
+fn parse_opts(pkt: &[u8]) -> Vec<(u8, Vec<u8>)> {
+ let mut opts = Vec::new();
+ let mut off = OPTS_OFF;
+ loop {
+ let code = pkt[off];
+ if code == 255 {
+ break;
+ }
+ if code == 0 {
+ off += 1;
+ continue;
+ }
+ let len = pkt[off + 1] as usize;
+ opts.push((code, pkt[off + 2..off + 2 + len].to_vec()));
+ off += 2 + len;
+ }
+ opts
+}
+
+fn ip_csum_ok(pkt: &[u8]) -> bool {
+ let mut sum = 0u32;
+ for w in pkt[14..34].chunks(2) {
+ sum += u32::from(u16::from_be_bytes([w[0], w[1]]));
+ }
+ while sum > 0xffff {
+ sum = (sum & 0xffff) + (sum >> 16);
+ }
+ sum == 0xffff
+}
+
+/// The offset just past the end option of a reply.
+fn opts_end(pkt: &[u8]) -> usize {
+ let mut off = OPTS_OFF;
+ loop {
+ match pkt[off] {
+ 255 => return off + 1,
+ 0 => off += 1,
+ _ => off += 2 + pkt[off + 1] as usize,
+ }
+ }
+}
+
+/// What every reply has to look like whatever the request was. The addressing and the answer
+/// itself are the caller's to check.
+fn assert_reply_invariants(pkt: &[u8]) {
+ assert_eq!(&pkt[6..12], &SRC_MAC, "eth src");
+ assert_eq!(&pkt[12..14], &[0x08, 0x00], "ethertype");
+
+ assert_eq!(pkt[14], 0x45, "ihl/version");
+ assert_eq!(pkt[22], 64, "ttl");
+ assert_eq!(pkt[23], 17, "protocol");
+ assert_eq!(&pkt[26..30], &GW, "ip src is the server id");
+ assert!(ip_csum_ok(pkt), "ip checksum");
+ let tot_len = u16::from_be_bytes([pkt[16], pkt[17]]) as usize;
+ assert_eq!(tot_len, pkt.len() - 14, "ip total length");
+
+ assert_eq!(&pkt[34..36], &67u16.to_be_bytes(), "udp sport");
+ assert_eq!(&pkt[36..38], &68u16.to_be_bytes(), "udp dport");
+ let udp_len = u16::from_be_bytes([pkt[38], pkt[39]]) as usize;
+ assert_eq!(udp_len, pkt.len() - 34, "udp length");
+ assert_eq!(&pkt[40..42], &[0, 0], "udp checksum zero, none to complete");
+ assert!(pkt.len() - 42 >= 300, "bootp minimum message size");
+
+ assert_eq!(&pkt[42..45], &[2, 1, 6], "bootp reply op, htype, hlen");
+ assert_eq!(&pkt[46..50], &[0xde, 0xad, 0xbe, 0xef], "xid preserved");
+ assert_eq!(&pkt[70..76], &MAC, "chaddr");
+ assert_eq!(&pkt[278..282], &[0x63, 0x82, 0x53, 0x63], "cookie");
+ assert!(
+ matches!(parse_opts(pkt).first(), Some((53, t)) if t.len() == 1 && [2, 5, 6].contains(&t[0])),
+ "message type leads the options"
+ );
+ let end = opts_end(pkt);
+ assert!(
+ pkt[end..].iter().all(|&b| b == 0),
+ "padding after the end option is zero"
+ );
+}
+
+fn assert_reply_headers(pkt: &[u8], eth_dst: [u8; 6], ip_dst: [u8; 4], yiaddr: [u8; 4]) {
+ assert_reply_invariants(pkt);
+ assert_eq!(&pkt[0..6], ð_dst, "eth dst");
+ assert_eq!(&pkt[30..34], &ip_dst, "ip dst");
+ assert_eq!(&pkt[58..62], &yiaddr, "yiaddr");
+}
+
+#[test]
+fn discover_gets_full_offer() {
+ setup_full_record();
+ let (verdict, skb) = Request::default().run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_eq!(redirected(), Some(IFINDEX));
+ assert_eq!(
+ redirect_flags(),
+ Some(0),
+ "the reply leaves through the tap"
+ );
+
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, MAC, [255, 255, 255, 255], IP);
+ assert_eq!(
+ parse_opts(pkt),
+ vec![
+ (53, vec![2]),
+ (54, GW.to_vec()),
+ (51, LEASE.to_be_bytes().to_vec()),
+ (1, MASK.to_vec()),
+ (3, GW.to_vec()),
+ (6, DNS.to_vec()),
+ (26, MTU.to_be_bytes().to_vec()),
+ (252, vec![b'\n']),
+ ],
+ );
+}
+
+#[test]
+fn offer_skips_unset_options() {
+ setup_maps();
+ map_insert(
+ records_map(),
+ &key(),
+ &record(IP, MASK, [0; 4], [0; 4], GW, LEASE, 0),
+ );
+ let (verdict, skb) = Request::default().run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let codes: Vec<u8> = parse_opts(skb.packet()).iter().map(|(c, _)| *c).collect();
+ assert_eq!(codes, vec![53, 54, 51, 1, 252]);
+}
+
+#[test]
+fn matching_request_gets_ack() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[50, 4]);
+ options.extend_from_slice(&IP);
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, MAC, [255, 255, 255, 255], IP);
+ assert_eq!(parse_opts(pkt)[0], (53, vec![5]));
+}
+
+// the folded pseudo-header sum a checksum completed on the way out starts from
+fn udp_pseudo(pkt: &[u8]) -> [u8; 2] {
+ let words = [
+ u16::from_be_bytes([pkt[26], pkt[27]]),
+ u16::from_be_bytes([pkt[28], pkt[29]]),
+ u16::from_be_bytes([pkt[30], pkt[31]]),
+ u16::from_be_bytes([pkt[32], pkt[33]]),
+ 17,
+ u16::from_be_bytes([pkt[38], pkt[39]]),
+ ];
+ let mut s: u32 = words.iter().map(|&w| w as u32).sum();
+ while s >> 16 != 0 {
+ s = (s & 0xffff) + (s >> 16);
+ }
+ (s as u16).to_be_bytes()
+}
+
+#[test]
+fn offloaded_request_gets_the_pseudo_header_sum() {
+ setup_full_record();
+ let mut skb = TestSkb::new(&Request::default().build(), IFINDEX);
+ skb.partial = true;
+ assert_eq!(skb.run(tc_dhcp_ingress), TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_eq!(
+ &pkt[40..42],
+ &udp_pseudo(pkt),
+ "udp checksum holds the pseudo-header sum"
+ );
+ assert_ne!(&pkt[40..42], &[0, 0]);
+}
+
+#[test]
+fn removed_record_is_not_answered() {
+ setup_maps();
+ map_insert(records_map(), &key(), &[0u8; 48]);
+ let (verdict, _) = Request::default().run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+}
+
+#[test]
+fn tagged_request_passes() {
+ setup_full_record();
+ let mut skb = TestSkb::new(&Request::default().build(), IFINDEX);
+ skb.tag_vlan();
+ assert_eq!(skb.run(tc_dhcp_ingress), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn trailer_past_the_datagram_is_not_read_as_options() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[50, 4]);
+ options.extend_from_slice(&IP);
+ // no END option, and the frame goes on with bytes shaped like a server
+ // identifier naming another server
+ let (verdict, skb) = Request {
+ options,
+ end: false,
+ trailer: vec![54, 4, 10, 9, 9, 9],
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_eq!(parse_opts(skb.packet())[0], (53, vec![5]));
+}
+
+#[test]
+fn bytes_past_the_udp_datagram_are_not_read_as_options() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[50, 4]);
+ options.extend_from_slice(&IP);
+ // the IP length claims the trailer and the UDP length does not. The
+ // trailer is shaped like a server identifier naming another server
+ let (verdict, skb) = Request {
+ options,
+ end: false,
+ trailer: vec![54, 4, 10, 9, 9, 9],
+ tot_len_excess: 6,
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_eq!(parse_opts(skb.packet())[0], (53, vec![5]));
+}
+
+#[test]
+fn claimed_length_past_the_frame_is_clamped() {
+ // the message type is followed by a second one cut off by the frame, a walk trusting the
+ // claimed length would read past the frame and take the zeroed result as the type
+ setup_full_record();
+ let mut options = DISCOVER.to_vec();
+ options.extend_from_slice(&[53, 1]);
+ let req = Request {
+ options,
+ end: false,
+ tot_len_excess: 1,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_reply_invariants(skb.packet());
+}
+
+#[test]
+fn request_without_options_passes_untouched() {
+ setup_full_record();
+ let req = Request {
+ options: Vec::new(),
+ end: false,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(skb.packet(), &req.build()[..]);
+}
+
+#[test]
+fn stale_request_gets_nak() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[50, 4, 10, 0, 0, 77]);
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, BCAST, [255, 255, 255, 255], [0; 4]);
+ assert_eq!(parse_opts(pkt), vec![(53, vec![6]), (54, GW.to_vec())]);
+}
+
+#[test]
+fn renewal_is_answered_unicast() {
+ setup_full_record();
+ let (verdict, skb) = Request {
+ options: REQUEST.to_vec(),
+ ciaddr: IP,
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, MAC, IP, IP);
+ assert_eq!(parse_opts(pkt)[0], (53, vec![5]));
+}
+
+#[test]
+fn request_selecting_another_server_passes_untouched() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[54, 4, 192, 168, 7, 7]);
+ options.extend_from_slice(&[50, 4, 10, 0, 0, 77]);
+ let req = Request {
+ options,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+}
+
+#[test]
+fn relayed_request_passes_untouched() {
+ setup_full_record();
+ let req = Request {
+ giaddr: [10, 0, 0, 254],
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+}
+
+#[test]
+fn fragments_pass_untouched() {
+ setup_full_record();
+ for frag_off in [[0x20, 0x00], [0x00, 0x01]] {
+ let req = Request {
+ frag_off,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(skb.packet(), &req.build()[..]);
+ }
+
+ let (verdict, _) = Request {
+ frag_off: [0x40, 0x00],
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(verdict, TC_ACT_REDIRECT, "DF alone is not a fragment");
+}
+
+#[test]
+fn broadcast_flag_gets_l2_broadcast_reply() {
+ setup_full_record();
+ let (verdict, skb) = Request {
+ flags: [0x80, 0],
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, BCAST, [255, 255, 255, 255], IP);
+ assert_eq!(&pkt[52..54], &[0x80, 0], "flags echoed");
+
+ // ciaddr wins over the flag, a renewing client is addressable
+ let (verdict, skb) = Request {
+ flags: [0x80, 0],
+ ciaddr: IP,
+ options: REQUEST.to_vec(),
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_reply_headers(skb.packet(), MAC, IP, IP);
+}
+
+#[test]
+fn interface_without_a_record_passes_untouched() {
+ setup_maps();
+ let req = Request::default();
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+
+ // the record is one interface's, a request arriving on any other gets nothing from it
+ setup_full_record();
+ let (verdict, skb) = req.run_on(IFINDEX + 1);
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+}
+
+#[test]
+fn another_mac_on_the_interface_passes_untouched() {
+ // the record names the NIC's MAC. A nested client or a spoofed request carries another and
+ // is left to the bridge, it must not be handed the interface's address
+ setup_full_record();
+ let other = [0x02, 0x11, 0x22, 0x33, 0x44, 0x55];
+ let req = Request {
+ mac: other,
+ ..Request::default()
+ };
+ let before = req.build();
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(skb.packet(), &before[..], "untouched");
+}
+
+#[test]
+fn non_dhcp_traffic_passes_untouched() {
+ setup_full_record();
+ let variants = [
+ Request {
+ dport: 53,
+ ..Request::default()
+ },
+ Request {
+ ethertype: [0x86, 0xdd],
+ ..Request::default()
+ },
+ Request {
+ ipproto: 6,
+ ..Request::default()
+ },
+ ];
+ for req in variants {
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(skb.packet(), &req.build()[..]);
+ }
+
+ let frame = Request::default().build();
+ let mut skb = TestSkb::new(&frame[..100], IFINDEX);
+ assert_eq!(skb.run(tc_dhcp_ingress), TC_ACT_UNSPEC);
+
+ let mut corrupted = frame.clone();
+ corrupted[278] = 0;
+ let mut skb = TestSkb::new(&corrupted, IFINDEX);
+ assert_eq!(skb.run(tc_dhcp_ingress), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn options_walk_handles_padding_and_early_end() {
+ setup_full_record();
+
+ let mut options = vec![0, 0, 0];
+ options.extend_from_slice(DISCOVER);
+ let (verdict, _) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+
+ let mut options = vec![255];
+ options.extend_from_slice(DISCOVER);
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(
+ verdict, TC_ACT_UNSPEC,
+ "message type after end is not parsed"
+ );
+ assert_eq!(redirected(), None);
+ drop(skb);
+}
+
+#[test]
+fn stale_renewal_gets_broadcast_nak() {
+ setup_full_record();
+ let (verdict, skb) = Request {
+ options: REQUEST.to_vec(),
+ ciaddr: [10, 0, 0, 77],
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, BCAST, [255, 255, 255, 255], [0; 4]);
+ assert_eq!(parse_opts(pkt), vec![(53, vec![6]), (54, GW.to_vec())]);
+}
+
+#[test]
+fn request_without_address_gets_nak() {
+ setup_full_record();
+ let (verdict, skb) = Request {
+ options: REQUEST.to_vec(),
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, BCAST, [255, 255, 255, 255], [0; 4]);
+ assert_eq!(parse_opts(pkt)[0], (53, vec![6]));
+}
+
+#[test]
+fn realistic_request_is_answered_and_shrunk() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ // a client identifier naming something else is ignored, the chaddr decides
+ options.extend_from_slice(&[61, 7, 1, 0xde, 0xad, 0xbe, 0xef, 0x00, 0x01]);
+ options.extend_from_slice(&[50, 4]);
+ options.extend_from_slice(&IP);
+ options.extend_from_slice(&[54, 4]);
+ options.extend_from_slice(&GW);
+ options.extend_from_slice(&[12, 14]);
+ options.extend_from_slice(b"dhcp-client-01");
+ options.extend_from_slice(&[60, 32]);
+ options.extend_from_slice(b"PXEClient:Arch:00007:UNDI:003016");
+ options.extend_from_slice(&[55, 20]);
+ options.extend_from_slice(&[
+ 1, 2, 3, 6, 12, 15, 17, 26, 28, 40, 41, 42, 51, 54, 58, 59, 66, 67, 119, 121,
+ ]);
+ options.extend_from_slice(&[57, 2, 0x05, 0xdc]);
+ options.extend_from_slice(&[93, 2, 0, 7]);
+ options.extend_from_slice(&[94, 3, 1, 3, 16]);
+ options.extend_from_slice(&[97, 17, 0]);
+ options.extend_from_slice(&[0x11; 16]);
+ let req = Request {
+ options,
+ ..Request::default()
+ };
+ let request_len = req.build().len();
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert!(pkt.len() < request_len, "reply is shorter than the request");
+ assert_eq!(
+ pkt.len(),
+ 14 + 20 + 8 + 300,
+ "trimmed to the padded minimum"
+ );
+ assert_reply_headers(pkt, MAC, [255, 255, 255, 255], IP);
+ assert_eq!(parse_opts(pkt)[0], (53, vec![5]));
+}
+
+#[test]
+fn other_message_types_pass_untouched() {
+ setup_full_record();
+ for msgtype in [4, 7, 8] {
+ let req = Request {
+ options: vec![53, 1, msgtype],
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC, "message type {msgtype}");
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+ }
+}
+
+#[test]
+fn foreign_bootp_passes_untouched() {
+ setup_full_record();
+ let variants = [
+ // a reply coming in from the guest side, a guest running its own server
+ Request {
+ op: 2,
+ ..Request::default()
+ },
+ Request {
+ htype: 6,
+ ..Request::default()
+ },
+ Request {
+ hlen: 16,
+ ..Request::default()
+ },
+ Request {
+ ip_options: vec![1, 1, 1, 1],
+ ..Request::default()
+ },
+ ];
+ for req in variants {
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+ }
+}
+
+#[test]
+fn truncated_option_ends_the_walk() {
+ setup_full_record();
+
+ // a length running past the packet before the message type leaves nothing to answer
+ let mut options = vec![12, 200];
+ options.extend_from_slice(DISCOVER);
+ let req = Request {
+ options,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+
+ // after the message type it only cuts the walk short, what was read still counts
+ let mut options = DISCOVER.to_vec();
+ options.extend_from_slice(&[12, 200]);
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_reply_headers(skb.packet(), MAC, [255, 255, 255, 255], IP);
+}
+
+#[test]
+fn arbitrary_option_bytes_never_break_the_reply() {
+ setup_full_record();
+ let mut lcg = 0x2545f4914f6cdd1du64;
+ let mut rand = move || {
+ lcg = lcg
+ .wrapping_mul(6364136223846793005)
+ .wrapping_add(1442695040888963407);
+ (lcg >> 33) as u8
+ };
+
+ for _ in 0..500 {
+ let len = (rand() % 60) as usize;
+ let options: Vec<u8> = (0..len).map(|_| rand()).collect();
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+ assert!(verdict == TC_ACT_UNSPEC || verdict == TC_ACT_REDIRECT);
+ if verdict == TC_ACT_REDIRECT {
+ assert_reply_invariants(skb.packet());
+ }
+ }
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH proxmox-ebpf v2 02/16] dhcp: add responder subsystem
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
2026-09-09 10:41 ` [PATCH proxmox-ebpf v2 01/16] dhcp: add per-tap responder BPF program Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH proxmox-perl-rs v2 03/16] pve-rs: sdn: add dhcp responder bindings Hannes Laimer
` (13 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
The record map is fed entirely from outside, the subsystem keeps no
state source of its own. A full pass makes the programs current and
gives every interface named to it and found here an entry and the link
it is due. The entry is a record where the interface is served with one
and a marker otherwise. Bad input leaves its interface as it is. So the
empty state a rebuild leaves behind is refilled by the pass for every
interface the configs name. A single record or tap plug is written on
its own under the shared lock and touches nothing else. It reports when
nothing is loaded to write into, so the caller runs a full pass first.
An unplug drops its own link and leaves a later incarnation of the name
alone. The link is found by the name the pin carries, since the
interface may be gone by then.
Every record write carries a generation. A record change draws it after
its input was written, a plug before its cache read. A record keeps the
generation of the change that wrote it. So a full pass leaves alone what
changed since it read its input, and an older change never overwrites a
newer one. A record change writes the entry of every interface that
exists here, whether it has a link yet or not. The pass writes a marker
where nothing is served, so a plug still reading its input finds an
entry to lose against.
The configs trail the kernel around a hotplug. A tap is plugged before
the guest config names its new bridge, and a generation cannot order a
pass against that. So every entry also names the vnet its writer saw the
interface on. A pass or a record change leaves an entry alone that names
another vnet than the configs do, and its link follows the entry. A plug
replaces such an entry, it is the latest word on where its interface
sits. On its own vnet a plug compares by generation like every other
change. A plug finding no entry writes regardless, a pass that stamped
meanwhile never saw its interface. An unplug drops the entry, the
interface is gone or goes away with it. An interface plugged onto a
bridge this subsystem does not serve trades its entry for a marker
naming that place. So a pass still reading the old vnet leaves it alone,
and one reading the new place keeps it as it is. A linked interface the
pass does not know at all is left alone as well. One that is gone is
dropped with its entry, which a crashed guest leaves behind.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/dhcp/mod.rs | 1113 +++++++++++++++++++++++++++++++++++++++++++++
src/dhcp/types.rs | 80 ++++
src/lib.rs | 3 +
3 files changed, 1196 insertions(+)
create mode 100644 src/dhcp/mod.rs
create mode 100644 src/dhcp/types.rs
diff --git a/src/dhcp/mod.rs b/src/dhcp/mod.rs
new file mode 100644
index 0000000..a7e483b
--- /dev/null
+++ b/src/dhcp/mod.rs
@@ -0,0 +1,1113 @@
+//! The dhcp subsystem, a per-tap DHCPv4 responder answering from a pinned map of one entry per
+//! interface.
+//!
+//! The map holds nothing of its own. The SDN side hands in every guest interface with the answer
+//! it gets, if any. A full pass makes the kernel match that for the interfaces it names. A linked
+//! interface it does not name keeps its link and its entry. The single steps touch their own
+//! interface only, so they run under the shared lock and serialize on their map writes alone.
+//!
+//! Two things order the writers. The first is a generation. A record change draws it after its input
+//! is written, a pass and a plug draw before they read theirs. Older input never overwrites newer,
+//! see [`ApplyLock::with_generation`](crate::subsystem::ApplyLock::with_generation) and
+//! [`ApplyLock::change`](crate::subsystem::ApplyLock::change). The second is the vnet every entry
+//! names. A hotplug plugs the tap before the guest config names the new bridge, and no generation can
+//! order that. So a pass or a record change leaves an entry alone that names another vnet than the
+//! configs do, and the link follows such an entry. A plug replaces it. Plugs of one interface are
+//! serialized by the guest lock, so the plug is the latest word on where its interface sits. A plug
+//! finding no entry writes regardless of the stamp, a pass that stamped since never saw its
+//! interface. An unplugged interface takes its entry with it, one gone without a link leaves it to
+//! the next pass. One plugged onto a bridge this subsystem does not serve trades its entry for a
+//! marker naming that place.
+mod types;
+
+use std::collections::{HashMap, HashSet};
+use std::io::ErrorKind;
+use std::net::Ipv4Addr;
+
+use anyhow::{Context, bail};
+use aya::include_bytes_aligned;
+use aya::maps::MapError;
+
+use self::types::*;
+use crate::subsystem::{NotLoaded, TcPrograms};
+use crate::tc::{Direction, LinkPin};
+
+/// One guest interface as the configs know it. `serve` says whether a pass links it, a plug
+/// links its interface regardless. `record` is what it answers there, none answers nothing.
+/// `vnet` is what its bridge is, none for a plain bridge.
+pub struct Iface {
+ pub name: String,
+ pub vnet: Option<String>,
+ pub serve: bool,
+ pub record: Option<Record>,
+}
+
+/// One interface's answer, the address and every option the reply carries.
+pub struct Record {
+ pub ip: Ipv4Addr,
+ pub prefixlen: u8,
+ pub server_id: Ipv4Addr,
+ pub lease: u32,
+ pub router: Option<Ipv4Addr>,
+ pub dns: Option<Ipv4Addr>,
+ pub mtu: Option<u16>,
+ pub mac: [u8; 6],
+}
+
+/// What became of an unplugged interface.
+pub enum Unplugged {
+ /// Gone, or going with the unplug.
+ Gone,
+ /// Plugged onto a bridge this subsystem does not serve, the vnet that bridge is, if any.
+ MovedTo(Option<String>),
+}
+
+/// What a single-entry step did.
+#[derive(Debug, PartialEq, Eq)]
+pub enum Outcome {
+ /// Nothing is left to do. The entry is in place, or the change was older than what is there,
+ /// or its interface is not here.
+ Done,
+ /// Nothing is loaded to write into, only a full pass loads, the caller runs one and retries.
+ NeedsFullPass,
+}
+
+const NAME: &str = "dhcp";
+const RECORDS_MAP: &str = "dhcp_iface_records";
+const MAPS: [&str; 1] = [RECORDS_MAP];
+
+const DHCP_OBJ: &[u8] = include_bytes_aligned!(concat!(env!("OUT_DIR"), "/dhcp.bpf.o"));
+const DHCP_FINGERPRINT: u64 = TcPrograms::obj_fingerprint(DHCP_OBJ);
+
+// BUMP THIS when a semantically-incompatible change to a map definition in dhcp.bpf.c is made
+const SCHEMA_VERSION: u32 = 1;
+
+// requests only arrive from the guest side and replies leave through a redirect, so only ingress
+// carries a program
+const DIRECTIONS: [Direction; 1] = [Direction::Ingress];
+
+fn program_name(dir: Direction) -> &'static str {
+ match dir {
+ Direction::Ingress => "tc_dhcp_ingress",
+ Direction::Egress => unreachable!("dhcp is ingress-only"),
+ }
+}
+
+type RecordsMap = aya::maps::HashMap<aya::maps::MapData, u32, DhcpRecord>;
+
+const PROGRAMS: TcPrograms = TcPrograms {
+ name: NAME,
+ obj: DHCP_OBJ,
+ fingerprint: DHCP_FINGERPRINT,
+ prog_name: program_name,
+ directions: &DIRECTIONS,
+ maps: &MAPS,
+ schema_version: SCHEMA_VERSION,
+};
+
+pub struct DhcpSubsystem {
+ programs: TcPrograms,
+}
+
+impl Default for DhcpSubsystem {
+ fn default() -> Self {
+ Self::new()
+ }
+}
+
+impl DhcpSubsystem {
+ pub fn new() -> Self {
+ Self { programs: PROGRAMS }
+ }
+
+ pub fn next_generation(&self) -> anyhow::Result<u64> {
+ self.programs.next_generation()
+ }
+
+ /// The full pass. Every interface named and found here gets its entry, a record where served
+ /// with one and a marker otherwise, and a link where served or where the entry names another
+ /// vnet. Entries of gone interfaces are dropped, and of live ones it does not name that have
+ /// no link. Bad input leaves its interface as it is, with a warning. The rest is left alone
+ /// as the module doc says. Returns whether it applied, one older than a pass already in place
+ /// is dropped.
+ pub fn apply(&self, generation: u64, ifaces: &[Iface]) -> anyhow::Result<bool> {
+ let lock = self.programs.lock_exclusive()?;
+ // loading belongs to the ordered step, a pass dropped as stale must not rebuild the
+ // programs with an empty map it then never fills
+ let applied = lock.with_generation(generation, || {
+ lock.ensure_loaded()?;
+ let present = present_ifindexes(ifaces)?;
+ // a link the configs do not name stays while its interface lives under the pin's
+ // name, a reused index is not that interface
+ let mut unknown_alive = HashSet::new();
+ for pin in lock.attached()? {
+ if !present.contains_key(&pin.ifindex) && pin_alive(&pin)? {
+ unknown_alive.insert(pin.ifindex);
+ }
+ }
+ let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?;
+ let (elsewhere, failed) = sync_records(&mut map, &present, &unknown_alive, generation)?;
+ // a failed write or removal leaves its entry as it is, the links are still put right
+ // and every failure reported once the rest is done
+ let links = lock.reconcile(&wanted_links(&present, &elsewhere, &unknown_alive));
+ match (failed, links) {
+ (0, links) => links,
+ (failed, Ok(())) => bail!("{failed} entries could not be changed"),
+ (failed, Err(e)) => {
+ Err(e).with_context(|| format!("{failed} entries could not be changed"))
+ }
+ }
+ })?;
+ Ok(applied.is_some())
+ }
+
+ /// A record change. Interfaces not found here are skipped. The rest are written whether they
+ /// have a link yet or not, so a plug still reading its input loses to it. A record the reply
+ /// cannot carry leaves its interface as it is. With nothing loaded the caller runs a full pass
+ /// first, like a plug.
+ pub fn update(&self, generation: u64, ifaces: &[Iface]) -> anyhow::Result<Outcome> {
+ let lock = self.programs.lock_shared()?;
+ if !lock.is_current() {
+ return Ok(Outcome::NeedsFullPass);
+ }
+ let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?;
+ // resolved and built before the generation lock, it covers the map writes alone
+ let mut failed = 0usize;
+ let mut entries = Vec::new();
+ for iface in ifaces {
+ let ifindex = match ifindex_of(iface.name.as_str()) {
+ Ok(Some(ifindex)) => ifindex,
+ Ok(None) => continue,
+ Err(e) => {
+ log::warn!("dhcp: {e:#}");
+ failed += 1;
+ continue;
+ }
+ };
+ match iface.entry(generation) {
+ Ok(entry) => entries.push((ifindex, iface, entry)),
+ Err(e) => log::warn!("dhcp: {} is left as it is: {e:#}", iface.name),
+ }
+ }
+ lock.change(|applied| {
+ for (ifindex, iface, entry) in &entries {
+ if let Err(e) = write_entry(&mut map, applied, *ifindex, entry, Elsewhere::Skip) {
+ log::warn!("dhcp: writing the entry of {}: {e:#}", iface.name);
+ failed += 1;
+ }
+ }
+ Ok(())
+ })?;
+ if failed > 0 {
+ bail!("{failed} interfaces could not be updated");
+ }
+
+ Ok(Outcome::Done)
+ }
+
+ /// A tap plug, the entry goes in and the program on. Nothing is loaded here, with nothing
+ /// pinned the caller runs a full pass first.
+ pub fn attach(&self, generation: u64, iface: &Iface) -> anyhow::Result<Outcome> {
+ let Some(ifindex) = ifindex_of(iface.name.as_str())? else {
+ log::info!("dhcp attach: {} is gone, nothing to do", iface.name);
+ return Ok(Outcome::Done);
+ };
+ let lock = self.programs.lock_shared()?;
+ if !lock.is_current() {
+ return Ok(Outcome::NeedsFullPass);
+ }
+ // resolved again after the wait, a tap gone or recreated meanwhile is not this plug's
+ if ifindex_of(iface.name.as_str())? != Some(ifindex) {
+ log::info!("dhcp attach: {} went away, nothing to do", iface.name);
+ return Ok(Outcome::Done);
+ }
+ // the entry goes in before the program runs, a marker when there is no record or one
+ // the program cannot carry. A guest start must not fail on it
+ let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?;
+ let entry = match iface.entry(generation) {
+ Ok(entry) => entry,
+ Err(e) => {
+ log::warn!(
+ "dhcp: {} attaches with nothing to answer: {e:#}",
+ iface.name
+ );
+ DhcpRecord::removed(iface.place()?, generation)
+ }
+ };
+ lock.change(|applied| {
+ write_entry(&mut map, applied, ifindex, &entry, Elsewhere::Override)
+ })?;
+ match lock.attach_iface(ifindex) {
+ Ok(()) => Ok(Outcome::Done),
+ Err(e) if e.downcast_ref::<NotLoaded>().is_some() => Ok(Outcome::NeedsFullPass),
+ Err(e) => Err(e).with_context(|| format!("dhcp responder for {}", iface.name)),
+ }
+ }
+
+ /// A tap unplug. The interface may be gone by then, so its link is found by the name the pin
+ /// carries. [`Unplugged`] decides what becomes of the entry.
+ pub fn detach(&self, iface: &str, unplugged: Unplugged) -> anyhow::Result<()> {
+ // resolved before the lock wait, an interface recreated under the name meanwhile is not
+ // this unplug's and keeps its link
+ let now = ifindex_of(iface)?;
+ // every guest unplug on the host lands here. Most never had a link and must not wait
+ // behind a pass, one that is gone as well leaves its entry to the next pass
+ if !self.programs.link_pinned(iface) && now.is_none() {
+ return Ok(());
+ }
+ let lock = self.programs.lock_shared()?;
+
+ if !lock.is_current() {
+ log::debug!("dhcp detach: nothing current, {iface} is left to the next full pass");
+ return Ok(());
+ }
+ // the map is opened and the marker built before the links go, a failure there leaves
+ // them attached
+ let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?;
+ let marker = match &unplugged {
+ Unplugged::Gone => None,
+ Unplugged::MovedTo(vnet) => Some(DhcpRecord::removed(place(vnet.as_deref())?, 0)),
+ };
+ // an interface without a link has an entry as well, a marker where it is not served
+ let mut ifindexes = lock.detach_named(iface, |pin| {
+ Some(pin.ifindex) != now && pin_alive(pin).unwrap_or(false)
+ })?;
+ if let Some(now) = now.filter(|now| !ifindexes.contains(now)) {
+ ifindexes.push(now);
+ }
+ // the marker is the moved interface's, older incarnations under the name are gone
+ lock.change(|_| unplug_entries(&mut map, &ifindexes, now.zip(marker)))
+ }
+
+ /// Detach everywhere and drop the pinned state, for the last zone leaving the backend.
+ /// Ordered like a full pass, returns whether it applied.
+ pub fn clear(&self, generation: u64) -> anyhow::Result<bool> {
+ let lock = self.programs.lock_exclusive()?;
+ Ok(lock.with_generation(generation, || lock.clear())?.is_some())
+ }
+}
+
+const NO_VNET: [u8; 8] = [0; 8];
+
+fn unplug_entries(
+ map: &mut impl RecordStore,
+ ifindexes: &[u32],
+ moved: Option<(u32, DhcpRecord)>,
+) -> anyhow::Result<()> {
+ for &ifindex in ifindexes {
+ match moved {
+ // the marker replaces an entry, an interface without one was served by nothing a
+ // pass could see and gets none
+ Some((current, marker)) if current == ifindex => {
+ if map.get(ifindex)?.is_some() {
+ map.insert(ifindex, marker)?;
+ }
+ }
+ _ => map.remove(ifindex)?,
+ }
+ }
+ Ok(())
+}
+
+// the vnet id padded with zeros, all zero for none
+fn place(vnet: Option<&str>) -> anyhow::Result<[u8; 8]> {
+ let Some(vnet) = vnet else {
+ return Ok(NO_VNET);
+ };
+ let bytes = vnet.as_bytes();
+ if bytes.is_empty() || bytes.len() > NO_VNET.len() {
+ bail!("vnet id {vnet:?} does not fit a record");
+ }
+ let mut place = NO_VNET;
+ place[..bytes.len()].copy_from_slice(bytes);
+ Ok(place)
+}
+
+// the record map as the writers see it, so their decisions run against a plain map in tests
+trait RecordStore {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>>;
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()>;
+ // a missing entry is removed already
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()>;
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>>;
+}
+
+impl RecordStore for RecordsMap {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
+ match aya::maps::HashMap::get(self, &ifindex, 0) {
+ Ok(live) => Ok(Some(live)),
+ Err(MapError::KeyNotFound) => Ok(None),
+ Err(e) => Err(e.into()),
+ }
+ }
+
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ aya::maps::HashMap::insert(self, ifindex, rec, 0)?;
+ Ok(())
+ }
+
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
+ match aya::maps::HashMap::remove(self, &ifindex) {
+ Ok(()) => Ok(()),
+ // the kernel answers a missing key with ENOENT, aya passes that through as is
+ Err(MapError::SyscallError(e)) if e.io_error.kind() == ErrorKind::NotFound => Ok(()),
+ Err(e) => Err(e.into()),
+ }
+ }
+
+ // a record the listing cannot read is one the pass cannot compare against, so the pass
+ // fails rather than writes past it
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
+ self.iter()
+ .collect::<Result<Vec<_>, _>>()
+ .context("listing the records")
+ }
+}
+
+// what a write does with an entry naming another vnet than its own
+enum Elsewhere {
+ // a record change, its configs are stale for the interface
+ Skip,
+ // a plug, the latest word on where the interface sits
+ Override,
+}
+
+// a plug finding nothing writes regardless of the stamp, a pass that stamped since never saw its
+// interface
+fn write_entry(
+ map: &mut impl RecordStore,
+ applied: Option<u64>,
+ ifindex: u32,
+ rec: &DhcpRecord,
+ elsewhere: Elsewhere,
+) -> anyhow::Result<()> {
+ let live = map.get(ifindex)?;
+ match (&elsewhere, &live) {
+ (Elsewhere::Skip, Some(live)) if live.vnet != rec.vnet => {
+ log::debug!("dhcp: ifindex {ifindex} sits elsewhere than the configs say, leaving it");
+ return Ok(());
+ }
+ (Elsewhere::Override, None) => return map.insert(ifindex, *rec),
+ (Elsewhere::Override, Some(live)) if live.vnet != rec.vnet => {
+ return map.insert(ifindex, *rec);
+ }
+ _ => {}
+ }
+ if applied.is_some_and(|applied| applied >= rec.generation)
+ || live.is_some_and(|live| live.generation >= rec.generation)
+ {
+ log::debug!(
+ "dhcp: change of generation {} for ifindex {ifindex} is older than what is in place, \
+ dropping it",
+ rec.generation
+ );
+ return Ok(());
+ }
+ map.insert(ifindex, *rec)
+}
+
+// entries of live links the pass does not know stay, their plug wrote them. The other unknown
+// ones go with their interface. Returns the interfaces whose entry names another vnet than the
+// configs, and how many writes failed
+fn sync_records(
+ map: &mut impl RecordStore,
+ present: &HashMap<u32, &Iface>,
+ unknown_alive: &HashSet<u32>,
+ generation: u64,
+) -> anyhow::Result<(HashSet<u32>, usize)> {
+ let live: HashMap<u32, DhcpRecord> = map.entries()?.into_iter().collect();
+ let dead: Vec<u32> = live
+ .keys()
+ .filter(|ifindex| !present.contains_key(ifindex) && !unknown_alive.contains(ifindex))
+ .copied()
+ .collect();
+ let mut failed = 0usize;
+ for &ifindex in &dead {
+ if let Err(e) = map.remove(ifindex) {
+ log::warn!("dhcp: removing the record of ifindex {ifindex}: {e:#}");
+ failed += 1;
+ }
+ }
+ let (mut written, mut newer) = (0usize, 0usize);
+
+ let mut elsewhere = HashSet::new();
+
+ for (ifindex, iface) in present {
+ // bad input leaves its interface as it is and the pass goes on. The marker for an
+ // unserved interface is what a plug still reading its input loses against
+ let desired = match iface.entry(generation) {
+ Ok(desired) => desired,
+ Err(e) => {
+ log::warn!("dhcp: {} is left as it is: {e:#}", iface.name);
+ continue;
+ }
+ };
+ match live.get(ifindex) {
+ Some(entry) if entry.vnet != desired.vnet => {
+ elsewhere.insert(*ifindex);
+ }
+
+ Some(entry) if entry.generation > generation => newer += 1,
+ Some(entry) if entry.same_answer(&desired) => {}
+ _ => match map.insert(*ifindex, desired) {
+ Ok(()) => written += 1,
+ Err(e) => {
+ log::warn!("dhcp: writing the entry of {}: {e:#}", iface.name);
+ failed += 1;
+ }
+ },
+ }
+ }
+
+ if written + dead.len() + newer + elsewhere.len() > 0 {
+ log::info!(
+ "dhcp: {written} entries written, {} removed, {newer} left to newer changes, {} \
+ left where their plug put them",
+ dead.len(),
+ elsewhere.len()
+ );
+ } else {
+ log::debug!("dhcp: {} records, no changes", live.len());
+ }
+ Ok((elsewhere, failed))
+}
+
+// the interface's index here, none when it does not exist. Any other failure says nothing about
+// the interface, a step taking it for gone would detach what it could not resolve
+fn ifindex_of(name: &str) -> anyhow::Result<Option<u32>> {
+ match nix::net::if_::if_nametoindex(name) {
+ Ok(ifindex) => Ok(Some(ifindex)),
+ Err(nix::errno::Errno::ENODEV) => Ok(None),
+ Err(e) => Err(e).with_context(|| format!("resolve interface {name}")),
+ }
+}
+
+// the named interfaces that exist here by index, the names come from configs cluster-wide and
+// most belong to other nodes
+fn present_ifindexes(ifaces: &[Iface]) -> anyhow::Result<HashMap<u32, &Iface>> {
+ let mut present = HashMap::new();
+ for iface in ifaces {
+ if let Some(ifindex) = ifindex_of(iface.name.as_str())? {
+ present.insert(ifindex, iface);
+ }
+ }
+ Ok(present)
+}
+
+// the link follows the configs where they agree with the entry. An entry naming another vnet
+// keeps its link, and a served interface without a mapping gets one. So a record change filling
+// a marker attaches nothing itself. A live link the configs do not know stays
+
+fn wanted_links(
+ present: &HashMap<u32, &Iface>,
+ elsewhere: &HashSet<u32>,
+ unknown_alive: &HashSet<u32>,
+) -> HashSet<u32> {
+ let mut wanted: HashSet<u32> = present
+ .iter()
+ .filter(|(ifindex, iface)| elsewhere.contains(ifindex) || iface.serve)
+ .map(|(ifindex, _)| *ifindex)
+ .collect();
+
+ wanted.extend(unknown_alive);
+ wanted
+}
+
+// whether the link's interface still exists under the name and index its pin carries
+fn pin_alive(pin: &LinkPin) -> anyhow::Result<bool> {
+ Ok(ifindex_of(pin.name.as_str())? == Some(pin.ifindex))
+}
+
+impl Iface {
+ // a marker where nothing is served, so an older change arriving late still loses
+ fn entry(&self, generation: u64) -> anyhow::Result<DhcpRecord> {
+ let vnet = self.place()?;
+ match &self.record {
+ Some(record) if self.serve => record
+ .entry(vnet, generation)
+ .with_context(|| format!("record for {}", self.name)),
+ _ => Ok(DhcpRecord::removed(vnet, generation)),
+ }
+ }
+
+ fn place(&self) -> anyhow::Result<[u8; 8]> {
+ place(self.vnet.as_deref()).with_context(|| format!("interface {}", self.name))
+ }
+}
+
+impl Record {
+ // refused here is what the reply cannot carry. Zero stands for absent in the entry, so an
+ // address or an MTU of zero cannot be told from none
+ fn entry(&self, vnet: [u8; 8], generation: u64) -> anyhow::Result<DhcpRecord> {
+ if self.prefixlen == 0 || self.prefixlen > 32 {
+ bail!("prefixlen {} out of range", self.prefixlen);
+ }
+ if self.ip.is_unspecified() {
+ bail!("no address");
+ }
+ if self.server_id.is_unspecified() {
+ bail!("no server address");
+ }
+ if self.lease == 0 {
+ bail!("lease of zero seconds");
+ }
+ if self.mtu.is_some_and(|mtu| mtu < 68) {
+ bail!("mtu below the IPv4 minimum of 68");
+ }
+ if self.mac == [0; 6] {
+ bail!("no MAC");
+ }
+ let netmask = u32::MAX << (32 - self.prefixlen);
+ Ok(DhcpRecord {
+ ip: u32::from(self.ip).to_be(),
+ netmask: netmask.to_be(),
+ router: u32::from(self.router.unwrap_or(Ipv4Addr::UNSPECIFIED)).to_be(),
+ dns: u32::from(self.dns.unwrap_or(Ipv4Addr::UNSPECIFIED)).to_be(),
+ server_id: u32::from(self.server_id).to_be(),
+ lease: self.lease.to_be(),
+ mtu: self.mtu.unwrap_or(0).to_be(),
+ vnet,
+ mac: self.mac,
+ generation,
+ })
+ }
+}
+
+#[cfg(test)]
+mod test {
+ use std::collections::BTreeMap;
+
+ use super::*;
+
+ fn record() -> Record {
+ Record {
+ ip: Ipv4Addr::new(10, 0, 0, 100),
+ prefixlen: 24,
+ server_id: Ipv4Addr::new(10, 0, 0, 1),
+ lease: 300,
+ router: Some(Ipv4Addr::new(10, 0, 0, 1)),
+ dns: Some(Ipv4Addr::new(1, 1, 1, 1)),
+ mtu: Some(1500),
+ mac: MAC,
+ }
+ }
+
+ const MAC: [u8; 6] = [0xda, 0x65, 0x8f, 0x18, 0x9b, 0x6f];
+
+ const VNET: [u8; 8] = *b"vnet1\0\0\0";
+
+ #[test]
+ fn converts_full_record() {
+ let rec = record().entry(VNET, 7).unwrap();
+ assert_eq!(
+ u32::from_be(rec.ip),
+ u32::from(Ipv4Addr::new(10, 0, 0, 100))
+ );
+ assert_eq!(u32::from_be(rec.netmask), 0xffffff00);
+ assert_eq!(
+ u32::from_be(rec.router),
+ u32::from(Ipv4Addr::new(10, 0, 0, 1))
+ );
+ assert_eq!(u32::from_be(rec.dns), u32::from(Ipv4Addr::new(1, 1, 1, 1)));
+ assert_eq!(
+ u32::from_be(rec.server_id),
+ u32::from(Ipv4Addr::new(10, 0, 0, 1))
+ );
+ assert_eq!(u32::from_be(rec.lease), 300);
+ assert_eq!(u16::from_be(rec.mtu), 1500);
+ assert_eq!(rec.vnet, VNET);
+ assert_eq!(rec.mac, MAC);
+ assert_eq!(rec.generation, 7);
+
+ let rec = Record {
+ router: None,
+ dns: None,
+ mtu: None,
+ ..record()
+ }
+ .entry(VNET, 7)
+ .unwrap();
+ assert_eq!(rec.router, 0);
+ assert_eq!(rec.dns, 0);
+ assert_eq!(rec.mtu, 0);
+ }
+
+ #[test]
+ fn answers_compare_without_the_generation() {
+ let a = record().entry(VNET, 7).unwrap();
+ let b = record().entry(VNET, 9).unwrap();
+ assert!(a.same_answer(&b));
+ assert!(!a.same_answer(&DhcpRecord::removed(VNET, 9)));
+ assert_eq!(DhcpRecord::removed(VNET, 9).generation, 9);
+ assert_eq!(DhcpRecord::removed(VNET, 9).vnet, VNET);
+ }
+
+ #[test]
+ fn entries_name_the_vnet_of_the_interface() {
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("testvnet".into()),
+ serve: true,
+ record: Some(record()),
+ };
+ assert_eq!(&iface.entry(7).unwrap().vnet, b"testvnet");
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: None,
+ serve: false,
+ record: None,
+ };
+ assert_eq!(iface.entry(7).unwrap().vnet, NO_VNET);
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("ninechars".into()),
+ serve: true,
+ record: Some(record()),
+ };
+ assert!(iface.entry(7).is_err());
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some(String::new()),
+ serve: true,
+ record: Some(record()),
+ };
+ assert!(iface.entry(7).is_err());
+ }
+
+ #[test]
+ fn unserved_interfaces_get_a_marker() {
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("vnet1".into()),
+ serve: false,
+ record: Some(record()),
+ };
+ assert_eq!(iface.entry(7).unwrap().ip, 0);
+ assert_eq!(iface.entry(7).unwrap().vnet, VNET);
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("vnet1".into()),
+ serve: true,
+ record: None,
+ };
+ assert_eq!(iface.entry(7).unwrap().ip, 0);
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("vnet1".into()),
+ serve: true,
+ record: Some(record()),
+ };
+ assert_ne!(iface.entry(7).unwrap().ip, 0);
+ }
+
+ #[test]
+ fn refuses_what_the_reply_cannot_carry() {
+ for rec in [
+ Record {
+ prefixlen: 33,
+ ..record()
+ },
+ Record {
+ prefixlen: 0,
+ ..record()
+ },
+ Record {
+ server_id: Ipv4Addr::UNSPECIFIED,
+ ..record()
+ },
+ Record {
+ lease: 0,
+ ..record()
+ },
+ Record {
+ ip: Ipv4Addr::UNSPECIFIED,
+ ..record()
+ },
+ Record {
+ mtu: Some(67),
+ ..record()
+ },
+ Record {
+ mac: [0; 6],
+ ..record()
+ },
+ ] {
+ assert!(rec.entry(VNET, 7).is_err());
+ }
+ let rec = Record {
+ mtu: Some(68),
+ ..record()
+ };
+ assert_eq!(u16::from_be(rec.entry(VNET, 7).unwrap().mtu), 68);
+
+ let rec = Record {
+ prefixlen: 32,
+ ..record()
+ };
+ assert_eq!(u32::from_be(rec.entry(VNET, 7).unwrap().netmask), u32::MAX);
+ let rec = Record {
+ prefixlen: 1,
+ ..rec
+ };
+ assert_eq!(u32::from_be(rec.entry(VNET, 7).unwrap().netmask), 1 << 31);
+ }
+
+ impl RecordStore for BTreeMap<u32, DhcpRecord> {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
+ Ok(BTreeMap::get(self, &ifindex).copied())
+ }
+
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ BTreeMap::insert(self, ifindex, rec);
+ Ok(())
+ }
+
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
+ BTreeMap::remove(self, &ifindex);
+ Ok(())
+ }
+
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
+ Ok(self.iter().map(|(k, v)| (*k, *v)).collect())
+ }
+ }
+
+ const OTHER: [u8; 8] = *b"vnet2\0\0\0";
+
+ fn entry(vnet: [u8; 8], generation: u64) -> DhcpRecord {
+ record().entry(vnet, generation).unwrap()
+ }
+
+ fn other_answer(vnet: [u8; 8], generation: u64) -> DhcpRecord {
+ Record {
+ ip: Ipv4Addr::new(10, 0, 0, 200),
+ ..record()
+ }
+ .entry(vnet, generation)
+ .unwrap()
+ }
+
+ #[test]
+ fn a_record_change_is_ordered_by_generation() {
+ let mut map = BTreeMap::new();
+ write_entry(&mut map, None, 1, &entry(VNET, 5), Elsewhere::Skip).unwrap();
+ assert_eq!(map[&1].generation, 5);
+ // older than the live entry
+ write_entry(&mut map, None, 1, &other_answer(VNET, 4), Elsewhere::Skip).unwrap();
+ assert!(map[&1].same_answer(&entry(VNET, 0)));
+ // older than the last full apply
+ write_entry(
+ &mut map,
+ Some(6),
+ 1,
+ &other_answer(VNET, 6),
+ Elsewhere::Skip,
+ )
+ .unwrap();
+ assert!(map[&1].same_answer(&entry(VNET, 0)));
+ // newer than both
+ write_entry(
+ &mut map,
+ Some(6),
+ 1,
+ &other_answer(VNET, 7),
+ Elsewhere::Skip,
+ )
+ .unwrap();
+ assert!(map[&1].same_answer(&other_answer(VNET, 0)));
+ assert_eq!(map[&1].generation, 7);
+ }
+
+ #[test]
+ fn a_record_change_leaves_an_entry_of_another_vnet() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(OTHER, 1));
+ write_entry(&mut map, None, 1, &other_answer(VNET, 9), Elsewhere::Skip).unwrap();
+ assert_eq!(map[&1].vnet, OTHER);
+ assert_eq!(map[&1].generation, 1);
+ }
+
+ #[test]
+ fn a_plug_writes_where_nothing_is() {
+ let mut map = BTreeMap::new();
+ // a full apply stamped since the plug drew, it never saw the interface
+ write_entry(&mut map, Some(51), 1, &entry(VNET, 50), Elsewhere::Override).unwrap();
+ assert_eq!(map[&1].generation, 50);
+ // a record change finding nothing is still ordered against the stamp
+ let mut map = BTreeMap::new();
+ write_entry(&mut map, Some(51), 1, &entry(VNET, 50), Elsewhere::Skip).unwrap();
+ assert!(map.is_empty());
+ }
+
+ #[test]
+ fn a_plug_replaces_an_entry_of_another_vnet_and_compares_on_its_own() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(OTHER, 9));
+ // whatever the generations say, the tap sits on the plug's vnet now
+ write_entry(&mut map, Some(9), 1, &entry(VNET, 2), Elsewhere::Override).unwrap();
+ assert_eq!(map[&1].vnet, VNET);
+ assert_eq!(map[&1].generation, 2);
+ // on its own vnet an older plug loses like any other change
+ write_entry(
+ &mut map,
+ None,
+ 1,
+ &other_answer(VNET, 1),
+ Elsewhere::Override,
+ )
+ .unwrap();
+ assert!(map[&1].same_answer(&entry(VNET, 0)));
+ write_entry(
+ &mut map,
+ None,
+ 1,
+ &other_answer(VNET, 3),
+ Elsewhere::Override,
+ )
+ .unwrap();
+ assert!(map[&1].same_answer(&other_answer(VNET, 0)));
+ // and one older than the last full apply loses against an existing entry too
+ write_entry(&mut map, Some(9), 1, &entry(VNET, 5), Elsewhere::Override).unwrap();
+ assert!(map[&1].same_answer(&other_answer(VNET, 0)));
+ assert_eq!(map[&1].generation, 3);
+ }
+
+ #[test]
+ fn an_unplug_removes_or_marks_the_entries() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(VNET, 3));
+ map.insert(2, entry(VNET, 3));
+ map.insert(3, entry(VNET, 3));
+ unplug_entries(&mut map, &[1, 2], None).unwrap();
+ assert_eq!(map.keys().copied().collect::<Vec<_>>(), [3]);
+ // a missing entry is removed already
+ unplug_entries(&mut map, &[1], None).unwrap();
+ // a move marks the interface that moved, an older incarnation under its name is gone
+ map.insert(4, entry(VNET, 3));
+ let marker = DhcpRecord::removed(NO_VNET, 0);
+ unplug_entries(&mut map, &[3, 4], Some((4, marker))).unwrap();
+ assert_eq!(map.keys().copied().collect::<Vec<_>>(), [4]);
+ assert_eq!(map[&4].ip, 0);
+ assert_eq!(map[&4].vnet, NO_VNET);
+ assert_eq!(map[&4].generation, 0);
+ // an interface without an entry gets no marker
+ unplug_entries(&mut map, &[5], Some((5, marker))).unwrap();
+ assert!(!map.contains_key(&5));
+ }
+
+ fn iface(name: &str, vnet: Option<&str>, serve: bool, record: Option<Record>) -> Iface {
+ Iface {
+ name: name.into(),
+ vnet: vnet.map(String::from),
+ serve,
+ record,
+ }
+ }
+
+ #[test]
+ fn the_pass_syncs_the_records_it_knows() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(VNET, 3)); // same answer as the config, older
+ map.insert(2, other_answer(VNET, 3)); // a changed answer
+ map.insert(3, other_answer(VNET, 12)); // a change newer than the pass
+ map.insert(4, entry(VNET, 3)); // not served anymore
+ map.insert(6, entry(VNET, 3)); // its interface is gone
+ map.insert(7, entry(VNET, 3)); // a live link the configs do not name
+ map.insert(8, entry(VNET, 3)); // its mapping is gone
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(record())),
+ iface("b", Some("vnet1"), true, Some(record())),
+ iface("c", Some("vnet1"), true, Some(record())),
+ iface("d", Some("vnet1"), false, None),
+ iface("e", Some("vnet1"), true, Some(record())), // no entry yet
+ iface("f", Some("vnet1"), true, None), // no entry, served, no mapping
+ iface("g", Some("vnet1"), true, None),
+ ];
+ let present: HashMap<u32, &Iface> = [1, 2, 3, 4, 5, 9, 8]
+ .into_iter()
+ .zip(ifaces.iter())
+ .collect();
+ let unknown_alive = HashSet::from([7]);
+ let (elsewhere, _) = sync_records(&mut map, &present, &unknown_alive, 10).unwrap();
+
+ assert!(elsewhere.is_empty());
+ assert_eq!(
+ map[&1].generation, 3,
+ "an unchanged answer keeps its generation"
+ );
+ assert!(map[&2].same_answer(&entry(VNET, 0)));
+ assert_eq!(map[&2].generation, 10);
+ assert!(
+ map[&3].same_answer(&other_answer(VNET, 0)),
+ "a newer change stays"
+ );
+ assert_eq!(
+ map[&4].ip, 0,
+ "an interface not served anymore keeps a marker"
+ );
+ assert_eq!(map[&4].generation, 10);
+ assert_eq!(map[&5].generation, 10);
+
+ assert!(!map.contains_key(&6));
+ assert_eq!(
+ map[&9].ip, 0,
+ "a served interface without a mapping gets a marker"
+ );
+ assert_eq!(map[&9].generation, 10);
+ assert_eq!(map[&8].ip, 0, "a mapping that is gone leaves a marker");
+ assert_eq!(map[&8].generation, 10);
+ // a plug that read the cache before the pass now loses to the marker
+ write_entry(&mut map, Some(10), 8, &entry(VNET, 5), Elsewhere::Override).unwrap();
+ assert_eq!(map[&8].ip, 0);
+ assert!(map.contains_key(&7));
+ }
+
+ #[test]
+ fn the_links_follow_the_configs_or_the_entry() {
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(record())), // served, entry agrees
+ iface("b", Some("vnet1"), false, None), // not served
+ iface("c", Some("vnet1"), false, None), // config stale, entry has a record
+ iface("d", Some("vnet1"), true, Some(record())), // config stale, entry is a marker
+ ];
+ let present: HashMap<u32, &Iface> = (1..=4).zip(ifaces.iter()).collect();
+ let elsewhere = HashSet::from([3, 4]);
+ let unknown_alive = HashSet::from([9]);
+ assert_eq!(
+ wanted_links(&present, &elsewhere, &unknown_alive),
+ HashSet::from([1, 3, 4, 9])
+ );
+ }
+
+ // a store that refuses one interface's writes
+ struct Flaky(BTreeMap<u32, DhcpRecord>, u32);
+
+ impl RecordStore for Flaky {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
+ RecordStore::get(&self.0, ifindex)
+ }
+
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ if ifindex == self.1 {
+ bail!("map full");
+ }
+ RecordStore::insert(&mut self.0, ifindex, rec)
+ }
+
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
+ if ifindex == self.1 {
+ bail!("map busy");
+ }
+ RecordStore::remove(&mut self.0, ifindex)
+ }
+
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
+ RecordStore::entries(&self.0)
+ }
+ }
+
+ #[test]
+ fn bad_input_leaves_its_interface_alone() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(VNET, 3));
+ let bad = Record {
+ lease: 0,
+ ..record()
+ };
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(bad)),
+ iface("b", Some("vnet1"), true, Some(record())),
+ ];
+ let present: HashMap<u32, &Iface> = (1..=2).zip(ifaces.iter()).collect();
+ let (elsewhere, failed) = sync_records(&mut map, &present, &HashSet::new(), 10).unwrap();
+ assert!(elsewhere.is_empty());
+ assert_eq!(failed, 0);
+ assert_eq!(map[&1].generation, 3, "the old entry stays");
+ assert_eq!(map[&2].generation, 10, "the rest is written");
+ }
+
+ #[test]
+ fn a_failed_write_does_not_stop_the_pass() {
+ let mut map = Flaky(BTreeMap::new(), 2);
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(record())),
+ iface("b", Some("vnet1"), true, Some(record())),
+ iface("c", Some("vnet1"), true, Some(record())),
+ ];
+ let present: HashMap<u32, &Iface> = (1..=3).zip(ifaces.iter()).collect();
+ let (elsewhere, failed) = sync_records(&mut map, &present, &HashSet::new(), 10).unwrap();
+ assert!(elsewhere.is_empty());
+ assert_eq!(failed, 1);
+ assert!(map.0.contains_key(&1));
+ assert!(!map.0.contains_key(&2));
+ assert!(map.0.contains_key(&3));
+ }
+
+ // a store whose listing fails
+ struct Unlistable(BTreeMap<u32, DhcpRecord>);
+
+ impl RecordStore for Unlistable {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
+ RecordStore::get(&self.0, ifindex)
+ }
+
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ RecordStore::insert(&mut self.0, ifindex, rec)
+ }
+
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
+ RecordStore::remove(&mut self.0, ifindex)
+ }
+
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
+ bail!("map gone")
+ }
+ }
+
+ #[test]
+ fn a_failed_listing_fails_the_pass_before_it_writes() {
+ let mut map = Unlistable(BTreeMap::new());
+ map.0.insert(1, entry(OTHER, 1));
+ let ifaces = [iface("a", Some("vnet1"), true, Some(record()))];
+ let present: HashMap<u32, &Iface> = [(1, &ifaces[0])].into_iter().collect();
+ assert!(sync_records(&mut map, &present, &HashSet::new(), 10).is_err());
+ assert_eq!(map.0[&1].vnet, OTHER);
+ }
+
+ #[test]
+ fn a_failed_removal_counts_like_a_failed_write() {
+ let mut map = Flaky(BTreeMap::new(), 2);
+ map.0.insert(2, entry(VNET, 3));
+ let ifaces = [iface("a", Some("vnet1"), true, Some(record()))];
+ let present: HashMap<u32, &Iface> = [(1, &ifaces[0])].into_iter().collect();
+ let (_, failed) = sync_records(&mut map, &present, &HashSet::new(), 10).unwrap();
+ assert_eq!(failed, 1);
+ assert!(map.0.contains_key(&1));
+ assert!(map.0.contains_key(&2));
+ }
+
+ #[test]
+ fn the_pass_leaves_entries_of_another_vnet_to_their_link() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(OTHER, 1)); // a plug the configs trail, with a record
+ map.insert(2, DhcpRecord::removed(NO_VNET, 0)); // moved to a plain bridge
+ map.insert(3, DhcpRecord::removed(NO_VNET, 0)); // the configs caught up with that
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(record())),
+ iface("b", Some("vnet1"), true, Some(record())),
+ iface("c", None, false, None),
+ ];
+ let present: HashMap<u32, &Iface> = (1..=3).zip(ifaces.iter()).collect();
+ let (elsewhere, _) = sync_records(&mut map, &present, &HashSet::new(), 10).unwrap();
+
+ assert_eq!(elsewhere, HashSet::from([1, 2]));
+
+ assert_eq!(map[&1].vnet, OTHER);
+ assert_eq!(map[&2].vnet, NO_VNET);
+ assert_eq!(
+ map[&3].ip, 0,
+ "a marker the configs agree with stays as it is"
+ );
+ assert_eq!(map[&3].generation, 0);
+ }
+}
diff --git a/src/dhcp/types.rs b/src/dhcp/types.rs
new file mode 100644
index 0000000..e421f96
--- /dev/null
+++ b/src/dhcp/types.rs
@@ -0,0 +1,80 @@
+//! Keep in sync with 'bpf/types.h'.
+
+/// Addresses, lease and mtu are stored in network byte order, so a reply copies them into the
+/// packet as-is.
+#[repr(C)]
+#[derive(Copy, Clone)]
+pub struct DhcpRecord {
+ /// 0 = removed, the entry only keeps its generation
+ pub ip: u32,
+ pub netmask: u32,
+ /// 0 = not served
+ pub router: u32,
+ /// 0 = not served
+ pub dns: u32,
+ pub server_id: u32,
+ pub lease: u32,
+ /// 0 = not served
+ pub mtu: u16,
+ /// The vnet the writer saw the interface on, the id padded with zeros, all zero for none.
+ /// Never read by the program.
+ pub vnet: [u8; 8],
+ /// The NIC's MAC, a request carrying another is passed on. Zero in a marker.
+ pub mac: [u8; 6],
+ /// The generation of the change that wrote the entry, host order, never read by the program.
+ pub generation: u64,
+}
+
+impl DhcpRecord {
+ /// Whether two entries answer alike. The generation and the vnet are not part of the answer.
+ pub fn same_answer(&self, other: &Self) -> bool {
+ self.ip == other.ip
+ && self.netmask == other.netmask
+ && self.router == other.router
+ && self.dns == other.dns
+ && self.server_id == other.server_id
+ && self.lease == other.lease
+ && self.mtu == other.mtu
+ && self.mac == other.mac
+ }
+
+ /// The marker a removal leaves behind, on the vnet the writer saw the interface on.
+ pub fn removed(vnet: [u8; 8], generation: u64) -> Self {
+ Self {
+ ip: 0,
+ netmask: 0,
+ router: 0,
+ dns: 0,
+ server_id: 0,
+ lease: 0,
+ mtu: 0,
+ vnet,
+ mac: [0; 6],
+ generation,
+ }
+ }
+}
+
+unsafe impl aya::Pod for DhcpRecord {}
+
+#[cfg(test)]
+mod layout {
+ use core::mem::{offset_of, size_of};
+
+ use super::*;
+
+ #[test]
+ fn matches_bpf_abi() {
+ assert_eq!(size_of::<DhcpRecord>(), 48);
+ assert_eq!(offset_of!(DhcpRecord, ip), 0);
+ assert_eq!(offset_of!(DhcpRecord, netmask), 4);
+ assert_eq!(offset_of!(DhcpRecord, router), 8);
+ assert_eq!(offset_of!(DhcpRecord, dns), 12);
+ assert_eq!(offset_of!(DhcpRecord, server_id), 16);
+ assert_eq!(offset_of!(DhcpRecord, lease), 20);
+ assert_eq!(offset_of!(DhcpRecord, mtu), 24);
+ assert_eq!(offset_of!(DhcpRecord, vnet), 26);
+ assert_eq!(offset_of!(DhcpRecord, mac), 34);
+ assert_eq!(offset_of!(DhcpRecord, generation), 40);
+ }
+}
diff --git a/src/lib.rs b/src/lib.rs
index 572d861..a8c94ba 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -1,5 +1,8 @@
//! eBPF subsystems, one per concern, each behind a cargo feature of its name. Nothing here runs
//! on its own, consumers pull in only the subsystem they drive.
+#[cfg(feature = "dhcp")]
+pub mod dhcp;
+
pub mod subsystem;
pub mod tc;
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH proxmox-perl-rs v2 03/16] pve-rs: sdn: add dhcp responder bindings
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
2026-09-09 10:41 ` [PATCH proxmox-ebpf v2 01/16] dhcp: add per-tap responder BPF program Hannes Laimer
2026-09-09 10:41 ` [PATCH proxmox-ebpf v2 02/16] dhcp: add responder subsystem Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH pve-network v2 04/16] sdn: push mapping changes from the ipam API to the dhcp backend Hannes Laimer
` (12 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
The SDN side hands the responder subsystem of proxmox-ebpf its desired
state, a single interface on every change and the complete set on a full
pass. Every record write carries a generation drawn after its input was
written, a plug's before it read the cache. So the subsystem can drop a
change that arrives out of order. Every interface names the vnet it sits
on, so a pass whose configs trail a hotplug leaves the plug's entry
alone. An unplug says whether the interface is gone or was plugged onto
a bridge the responder does not serve. The latter trades its entry for a
marker naming that place.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
pve-rs/Cargo.toml | 2 +
pve-rs/Makefile | 1 +
pve-rs/debian/control | 2 +
pve-rs/src/bindings/sdn/dhcp.rs | 162 ++++++++++++++++++++++++++++++++
pve-rs/src/bindings/sdn/mod.rs | 1 +
5 files changed, 168 insertions(+)
create mode 100644 pve-rs/src/bindings/sdn/dhcp.rs
diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
index 5ae9082..0adf00b 100644
--- a/pve-rs/Cargo.toml
+++ b/pve-rs/Cargo.toml
@@ -34,6 +34,7 @@ proxmox-apt = { version = "1.0.1", features = ["cache"] }
proxmox-apt-api-types = "3"
proxmox-base64 = "1"
proxmox-config-digest = "1"
+proxmox-ebpf = { version = "0.1", features = ["dhcp"] }
proxmox-frr = { version = "0.5.1" }
proxmox-http = { version = "1.0.2", features = ["client-sync", "client-trait"] }
proxmox-http-error = "1"
@@ -70,6 +71,7 @@ proxmox-wireguard = { version = "0.1.2" }
# proxmox-config-digest = { path = "../../proxmox/proxmox-config-digest" }
# proxmox-daemon = { path = "../../proxmox/proxmox-daemon" }
# proxmox-dns-api = { path = "../../proxmox/proxmox-dns-api" }
+# proxmox-ebpf = { path = "../../proxmox-ebpf" }
# proxmox-http-error = { path = "../../proxmox/proxmox-http-error" }
# proxmox-http = { path = "../../proxmox/proxmox-http" }
# proxmox-human-byte = { path = "../../proxmox/proxmox-human-byte" }
diff --git a/pve-rs/Makefile b/pve-rs/Makefile
index bb1cd2d..7609a1e 100644
--- a/pve-rs/Makefile
+++ b/pve-rs/Makefile
@@ -32,6 +32,7 @@ PERLMOD_PACKAGES := \
PVE::RS::OpenId \
PVE::RS::ResourceScheduling::Static \
PVE::RS::ResourceScheduling::Dynamic \
+ PVE::RS::SDN::Dhcp \
PVE::RS::SDN::Fabrics \
PVE::RS::SDN::PrefixLists \
PVE::RS::SDN::RouteMaps \
diff --git a/pve-rs/debian/control b/pve-rs/debian/control
index 70367dc..521d966 100644
--- a/pve-rs/debian/control
+++ b/pve-rs/debian/control
@@ -20,6 +20,8 @@ Build-Depends: debhelper-compat (= 13),
librust-proxmox-apt-api-types-3+default-dev (>= 3.0.0-~~),
librust-proxmox-base64-1+default-dev,
librust-proxmox-config-digest-1+default-dev,
+ librust-proxmox-ebpf-0.1+default-dev,
+ librust-proxmox-ebpf-0.1+dhcp-dev,
librust-proxmox-frr-0.5+default-dev (>= 0.5.1-~~),
librust-proxmox-http-1+client-sync-dev (>= 1.0.2-~~),
librust-proxmox-http-1+client-trait-dev (>= 1.0.2-~~),
diff --git a/pve-rs/src/bindings/sdn/dhcp.rs b/pve-rs/src/bindings/sdn/dhcp.rs
new file mode 100644
index 0000000..2f193d4
--- /dev/null
+++ b/pve-rs/src/bindings/sdn/dhcp.rs
@@ -0,0 +1,162 @@
+#[perlmod::package(name = "PVE::RS::SDN::Dhcp", lib = "pve_rs")]
+pub mod pve_rs_sdn_dhcp {
+ //! The `PVE::RS::SDN::Dhcp` package.
+ //!
+ //! Bindings for the per-tap eBPF DHCP responder. Interfaces and the records they answer with
+ //! are handed in here and served directly from the kernel.
+
+ use std::net::Ipv4Addr;
+
+ use anyhow::Error;
+ use serde::{Deserialize, Deserializer};
+
+ use proxmox_ebpf::dhcp::{DhcpSubsystem, Iface, Outcome, Record, Unplugged};
+
+ /// One interface's DHCP answer, the address and every option the reply carries.
+ #[derive(Deserialize)]
+ pub struct DhcpRecord {
+ ip: Ipv4Addr,
+ prefixlen: u8,
+ server_id: Ipv4Addr,
+ lease: u32,
+ #[serde(default)]
+ router: Option<Ipv4Addr>,
+ #[serde(default)]
+ dns: Option<Ipv4Addr>,
+ #[serde(default)]
+ mtu: Option<u16>,
+ #[serde(deserialize_with = "mac_from_str")]
+ mac: [u8; 6],
+ }
+
+ // a MAC as six hex pairs
+ fn mac_from_str<'de, D: Deserializer<'de>>(d: D) -> Result<[u8; 6], D::Error> {
+ let text = String::deserialize(d)?;
+ let mut mac = [0u8; 6];
+ let mut parts = text.split(':');
+ for byte in &mut mac {
+ let part = parts
+ .next()
+ .ok_or_else(|| serde::de::Error::custom(format!("MAC {text} is too short")))?;
+ if part.len() != 2 || !part.bytes().all(|b| b.is_ascii_hexdigit()) {
+ return Err(serde::de::Error::custom(format!("MAC {text} is malformed")));
+ }
+ *byte = u8::from_str_radix(part, 16)
+ .map_err(|_| serde::de::Error::custom(format!("MAC {text} is malformed")))?;
+ }
+ if parts.next().is_some() {
+ return Err(serde::de::Error::custom(format!("MAC {text} is too long")));
+ }
+ Ok(mac)
+ }
+
+ /// One guest interface, the vnet its bridge is if any, whether the responder runs on it and
+ /// what it answers there.
+ #[derive(Deserialize)]
+ pub struct DhcpIface {
+ name: String,
+ #[serde(default)]
+ vnet: Option<String>,
+ serve: bool,
+ #[serde(default)]
+ record: Option<DhcpRecord>,
+ }
+
+ /// Where an unplugged interface went, a bridge the responder does not serve, the vnet that
+ /// bridge is if any.
+ #[derive(Deserialize)]
+ pub struct DhcpTarget {
+ #[serde(default)]
+ vnet: Option<String>,
+ }
+
+ impl From<DhcpRecord> for Record {
+ fn from(r: DhcpRecord) -> Self {
+ Record {
+ ip: r.ip,
+ prefixlen: r.prefixlen,
+ server_id: r.server_id,
+ lease: r.lease,
+ router: r.router,
+ dns: r.dns,
+ mtu: r.mtu,
+ mac: r.mac,
+ }
+ }
+ }
+
+ impl From<DhcpIface> for Iface {
+ fn from(i: DhcpIface) -> Self {
+ Iface {
+ name: i.name,
+ vnet: i.vnet,
+ serve: i.serve,
+ record: i.record.map(Record::from),
+ }
+ }
+ }
+
+ fn to_ifaces(ifaces: Vec<DhcpIface>) -> Vec<Iface> {
+ ifaces.into_iter().map(Iface::from).collect()
+ }
+
+ /// Draw the next generation. A record change draws after its input is written, a full pass
+ /// and a plug before they read theirs. So changes are ordered by the input they saw.
+ #[export]
+ pub fn next_generation() -> Result<u64, Error> {
+ DhcpSubsystem::new().next_generation()
+ }
+
+ /// The full pass over every guest interface of the cluster, those found here get their entry
+ /// and a link where served. Returns whether it applied, a pass older than one already in place
+ /// is dropped.
+ #[export]
+ pub fn apply(generation: u64, ifaces: Vec<DhcpIface>) -> Result<bool, Error> {
+ DhcpSubsystem::new().apply(generation, &to_ifaces(ifaces))
+ }
+
+ /// A record change for the listed interfaces, those not found here are skipped. Returns
+ /// false when nothing is loaded to write into, a full pass is due then.
+ #[export]
+ pub fn update(generation: u64, ifaces: Vec<DhcpIface>) -> Result<bool, Error> {
+ Ok(DhcpSubsystem::new().update(generation, &to_ifaces(ifaces))? == Outcome::Done)
+ }
+
+ /// A tap plug onto a vnet with the record it answers, none answers nothing. The generation was
+ /// drawn before the cache was read. Returns false when nothing is loaded yet, an interface gone
+ /// already counts as done.
+ #[export]
+ pub fn attach(
+ generation: u64,
+ iface: String,
+ vnet: String,
+ record: Option<DhcpRecord>,
+ ) -> Result<bool, Error> {
+ let iface = Iface {
+ name: iface,
+ vnet: Some(vnet),
+ serve: true,
+ record: record.map(Record::from),
+ };
+ Ok(DhcpSubsystem::new().attach(generation, &iface)? == Outcome::Done)
+ }
+
+ /// A tap unplug. Without a target the interface is gone or going and takes its record with
+ /// it. With one it was plugged onto a bridge the responder does not serve and keeps a marker
+ /// naming that place.
+ #[export]
+ pub fn detach(iface: &str, target: Option<DhcpTarget>) -> Result<(), Error> {
+ let unplugged = match target {
+ None => Unplugged::Gone,
+ Some(target) => Unplugged::MovedTo(target.vnet),
+ };
+ DhcpSubsystem::new().detach(iface, unplugged)
+ }
+
+ /// Detach the responder everywhere and drop its pinned state. Ordered by the generation
+ /// like a full pass, returns whether it applied.
+ #[export]
+ pub fn clear(generation: u64) -> Result<bool, Error> {
+ DhcpSubsystem::new().clear(generation)
+ }
+}
diff --git a/pve-rs/src/bindings/sdn/mod.rs b/pve-rs/src/bindings/sdn/mod.rs
index dcae046..4b99b8f 100644
--- a/pve-rs/src/bindings/sdn/mod.rs
+++ b/pve-rs/src/bindings/sdn/mod.rs
@@ -1,3 +1,4 @@
+pub(crate) mod dhcp;
pub(crate) mod fabrics;
pub(crate) mod prefix_lists;
pub(crate) mod route_maps;
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH pve-network v2 04/16] sdn: push mapping changes from the ipam API to the dhcp backend
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (2 preceding siblings ...)
2026-09-09 10:41 ` [PATCH proxmox-perl-rs v2 03/16] pve-rs: sdn: add dhcp responder bindings Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH pve-network v2 05/16] sdn: ipam: do not cache negative per-MAC answers, lock the write Hannes Laimer
` (11 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
Mapping edits through the API only wrote the IPAM record and the cache,
the dhcp backend was never told. So dnsmasq kept serving stale
reservations until a later guest start in the zone happened to sweep
them. The same held for a guest whose NIC moved to another bridge. The
old records were released and new ones allocated without the backend
hearing of either.
Push the MAC's current answer after every record change on the node the
change is made on, the other nodes still catch up at a guest start
there. A guest start pushes once after allocating what it lacks, so a
guest whose records exist already is pushed as well. The push is best
effort, the record write stays authoritative. It goes to the applied
zone's backend, a zone edited but not applied has nothing running yet.
dnsmasq rewrites the MAC's reservation in one pass and reloads only when
the file changed. So an unchanged record costs no reload, and a changed
one leaves no moment in which the MAC is unknown. The cache is read
under the ethers lock, a snapshot from before it would sweep a line
another push wrote meanwhile. So the dispatcher hands over only the MAC.
A reservation line is matched to its MAC regardless of case, the cache
and the file may spell it differently. Removing a mapping was dispatched
to the plugins but implemented nowhere, it is the empty case of that
rewrite. A zone confined to other nodes has no backend here and is
skipped. A failed lease refresh only warns, the reservation is written
by then.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/Network/SDN/Ips.pm | 7 +
src/PVE/Network/SDN/Dhcp.pm | 44 +++---
src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 101 ++++++++----
src/PVE/Network/SDN/Dhcp/Plugin.pm | 6 +-
src/PVE/Network/SDN/Vnets.pm | 5 +-
src/test/run_test_vnets_blackbox.pl | 229 +++++++++++++++++++++++++++-
6 files changed, 329 insertions(+), 63 deletions(-)
diff --git a/src/PVE/API2/Network/SDN/Ips.pm b/src/PVE/API2/Network/SDN/Ips.pm
index 5ff05e7..d7b682d 100644
--- a/src/PVE/API2/Network/SDN/Ips.pm
+++ b/src/PVE/API2/Network/SDN/Ips.pm
@@ -46,6 +46,8 @@ __PACKAGE__->register_method({
eval { PVE::Network::SDN::Vnets::del_ip($vnet, $ip, '', $mac); };
die "$@\n" if $@;
+ PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+
return undef;
},
});
@@ -82,6 +84,8 @@ __PACKAGE__->register_method({
PVE::Network::SDN::Vnets::add_ip($vnet, $ip, '', $mac, undef);
+ PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+
return undef;
},
});
@@ -132,6 +136,9 @@ __PACKAGE__->register_method({
}
die "$error\n" if $error;
+
+ PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+
return undef;
},
});
diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
index 65e40d4..28a9f2e 100644
--- a/src/PVE/Network/SDN/Dhcp.pm
+++ b/src/PVE/Network/SDN/Dhcp.pm
@@ -6,7 +6,6 @@ use warnings;
use PVE::Cluster;
use PVE::Network::SDN;
-use PVE::Network::SDN::Ipams;
use PVE::Network::SDN::Subnets;
use PVE::Network::SDN::Dhcp::Plugin;
use PVE::Network::SDN::Dhcp::Dnsmasq;
@@ -22,37 +21,28 @@ sub plugin_types {
return PVE::Network::SDN::Dhcp::Plugin->lookup_types();
}
-sub add_mapping {
- my ($vnetid, $mac, $ip4, $ip6) = @_;
-
- my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid);
- return if !$vnet;
-
- my $zoneid = $vnet->{zone};
- my $zone = PVE::Network::SDN::Zones::get_zone($zoneid);
-
- return if !$zone->{ipam} || !$zone->{dhcp};
-
- my $dhcptype = $zone->{dhcp};
-
- my $macdb = PVE::Network::SDN::Ipams::read_macdb();
- my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($dhcptype);
- $dhcp_plugin->add_ip_mapping($zoneid, $macdb, $mac, $ip4, $ip6);
-}
-
-sub remove_mapping {
+# re-apply a MAC's mapping on this node from the current records, best
+# effort, the record write stays authoritative
+sub update_mapping {
my ($vnetid, $mac) = @_;
- my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid);
- return if !$vnet;
+ eval {
+ # the backend to push to is the applied one, a zone edited but not
+ # applied has nothing running yet
+ my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid, 1);
+ return if !$vnet;
- my $zoneid = $vnet->{zone};
- my $zone = PVE::Network::SDN::Zones::get_zone($zoneid);
+ my $zoneid = $vnet->{zone};
+ my $zone = PVE::Network::SDN::Zones::get_zone($zoneid, 1);
+ return if !$zone || !$zone->{ipam} || !$zone->{dhcp};
- return if !$zone->{ipam} || !$zone->{dhcp};
+ # a zone confined to other nodes has no backend running here
+ return if defined($zone->{nodes}) && !$zone->{nodes}->{ PVE::INotify::nodename() };
- my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($zone->{dhcp});
- $dhcp_plugin->del_ip_mapping($zoneid, $mac);
+ PVE::Network::SDN::Dhcp::Plugin->lookup($zone->{dhcp})
+ ->update_ip_mapping($zoneid, $mac);
+ };
+ warn "could not update dhcp mapping for $mac: $@" if $@;
}
sub regenerate_config {
diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
index 477b700..f3c7739 100644
--- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
+++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
@@ -12,6 +12,7 @@ use File::Copy;
use Net::DBus;
use PVE::RESTEnvironment qw(log_warn);
+use PVE::Network::SDN::Ipams;
my $DNSMASQ_CONFIG_ROOT = '/etc/dnsmasq.d';
my $DNSMASQ_DEFAULT_ROOT = '/etc/default';
@@ -51,29 +52,65 @@ sub update_lease {
$manager->AddDhcpLease($ip4, $mac, \@hostname, undef, 0, 0, 0) if $ip4;
}
-sub add_ip_mapping {
- my ($class, $dhcpid, $macdb, $mac, $ip4, $ip6) = @_;
+sub update_ip_mapping {
+ my ($class, $dhcpid, $mac) = @_;
my $ethers_file = ethers_file($dhcpid);
my $ethers_tmp_file = "$ethers_file.tmp";
- my $reload = undef;
+ my ($ip4, $ip6);
+ my $reservation = undef;
+ my $changed = undef;
+
+ my $rewriteFn = sub {
+ # read under the ethers lock. A snapshot from before it would sweep a
+ # line another push wrote meanwhile, and an older push would put its
+ # addresses back
+
+ my $macdb = PVE::Network::SDN::Ipams::read_macdb();
+ # the cache spells a MAC and an address as their writer did, the file
+ # holds one spelling of both. An entry without an address says nothing
+ # about its MAC, a line of it is stale
+ my %macs;
+ for my $key (sort keys $macdb->{macs}->%*) {
+ my $entry = $macdb->{macs}->{$key};
+ my %addresses =
+ map { $_ => lc($entry->{$_}) } grep { defined($entry->{$_}) } qw(ip4 ip6);
+ next if !%addresses;
+ $macs{ lc($key) } = { %{ $macs{ lc($key) } // {} }, %addresses };
+ }
+
+ my $entry = $macs{ lc($mac) };
+ ($ip4, $ip6) = ($entry->{ip4}, $entry->{ip6}) if $entry;
+ if ($ip4 || $ip6) {
+ $reservation = $mac;
+ $reservation .= ",$ip4" if $ip4;
+ $reservation .= ",[$ip6]" if $ip6;
+ # an external IPAM spells addresses its own way, the file holds one spelling
+ $reservation = lc($reservation);
+ }
- my $appendFn = sub {
+ # the first reservation of a zone finds no file yet
+ file_set_contents($ethers_file, '') if !-e $ethers_file;
open(my $in, '<', $ethers_file) or die "Could not open file '$ethers_file' $!\n";
open(my $out, '>', $ethers_tmp_file)
or die "Could not open file '$ethers_tmp_file' $!\n";
- my $match = undef;
+ my $kept = undef;
- my $line_no = 0;
while (my $line = <$in>) {
- $line_no++;
chomp($line);
+ next if $line !~ m/\S/;
my ($parsed_mac, $parsed_ip1, $parsed_ip2) = split(/,/, $line);
- if (!defined($parsed_mac)) {
- warn "failed to parse MAC from $dhcpid ethers file on line $line_no: '$line'\n";
+ # the MAC's own reservation is rewritten, an unchanged one stays in place
+ if (lc($parsed_mac) eq lc($mac)) {
+ if (defined($reservation) && !$kept && lc($line) eq $reservation) {
+ $kept = 1;
+ print $out "$line\n";
+ } else {
+ $changed = 1;
+ }
next;
}
@@ -81,41 +118,35 @@ sub add_ip_mapping {
if ($parsed_ip2) {
$parsed_ip4 = $parsed_ip1;
$parsed_ip6 = $parsed_ip2;
- } elsif (Net::IP::ip_is_ipv4($parsed_ip1)) {
+ } elsif (defined($parsed_ip1) && Net::IP::ip_is_ipv4($parsed_ip1)) {
$parsed_ip4 = $parsed_ip1;
- } else {
+ } elsif (defined($parsed_ip1)) {
$parsed_ip6 = $parsed_ip1;
}
$parsed_ip6 = $1 if $parsed_ip6 && $parsed_ip6 =~ m/\[(\S+)\]/;
+ # a line of the old code may spell an address in upper case
+ $parsed_ip6 = lc($parsed_ip6) if $parsed_ip6;
#delete changed
if (
- !defined($macdb->{macs}->{$parsed_mac})
+ !defined($macs{ lc($parsed_mac) })
|| ($parsed_ip4
- && $macdb->{macs}->{$parsed_mac}->{'ip4'}
- && $macdb->{macs}->{$parsed_mac}->{'ip4'} ne $parsed_ip4)
+ && $macs{ lc($parsed_mac) }->{'ip4'}
+ && $macs{ lc($parsed_mac) }->{'ip4'} ne $parsed_ip4)
|| ($parsed_ip6
- && $macdb->{macs}->{$parsed_mac}->{'ip6'}
- && $macdb->{macs}->{$parsed_mac}->{'ip6'} ne $parsed_ip6)
+ && $macs{ lc($parsed_mac) }->{'ip6'}
+ && $macs{ lc($parsed_mac) }->{'ip6'} ne $parsed_ip6)
) {
- $reload = 1;
+ $changed = 1;
next;
}
- if ($parsed_mac eq $mac) {
- $match = 1 if $ip4 && $parsed_ip4 && $ip4;
- $match = 1 if $ip6 && $parsed_ip6 && $ip6;
- }
-
print $out "$line\n";
}
- if (!$match) {
- my $reservation = $mac;
- $reservation .= ",$ip4" if $ip4;
- $reservation .= ",[$ip6]" if $ip6;
+ if (defined($reservation) && !$kept) {
print $out "$reservation\n";
- $reload = 1;
+ $changed = 1;
}
close $in;
@@ -124,16 +155,22 @@ sub add_ip_mapping {
chmod 0644, $ethers_file;
};
- PVE::Tools::lock_file($ethers_file, 10, $appendFn);
+ # the rewrite replaces the reservation file, a lock on the file itself
+ # would not outlive the rename
+ PVE::Tools::lock_file("$ethers_file.lock", 10, $rewriteFn);
if ($@) {
- warn "Unable to add $mac to the dnsmasq configuration: $@\n";
+ warn "Unable to update $mac in the dnsmasq configuration: $@\n";
return;
}
- my $service_name = "dnsmasq\@$dhcpid";
- systemctl_service('reload', $service_name) if $reload;
- update_lease($dhcpid, $ip4, $mac);
+ systemctl_service('reload', "dnsmasq\@$dhcpid") if $changed;
+
+ # the reservation is written, a lease refresh that fails only warns
+ if ($ip4) {
+ eval { update_lease($dhcpid, $ip4, $mac) };
+ log_warn("could not update the dnsmasq lease of $mac: $@") if $@;
+ }
}
sub configure_subnet {
diff --git a/src/PVE/Network/SDN/Dhcp/Plugin.pm b/src/PVE/Network/SDN/Dhcp/Plugin.pm
index b5d32fa..3b95a68 100644
--- a/src/PVE/Network/SDN/Dhcp/Plugin.pm
+++ b/src/PVE/Network/SDN/Dhcp/Plugin.pm
@@ -22,8 +22,10 @@ sub private {
return $defaultData;
}
-sub add_ip_mapping {
- my ($class, $dhcpid, $macdb, $mac, $ip4, $ip6) = @_;
+# the MAC's records changed, make the backend serve the current ones, none
+# meaning its reservation goes
+sub update_ip_mapping {
+ my ($class, $dhcpid, $mac) = @_;
die 'implement in sub class';
}
diff --git a/src/PVE/Network/SDN/Vnets.pm b/src/PVE/Network/SDN/Vnets.pm
index c327a4b..8453a27 100644
--- a/src/PVE/Network/SDN/Vnets.pm
+++ b/src/PVE/Network/SDN/Vnets.pm
@@ -211,6 +211,8 @@ sub del_ips_from_mac {
PVE::Network::SDN::Vnets::del_ip($vnetid, $ip4, $hostname, $mac) if $ip4;
PVE::Network::SDN::Vnets::del_ip($vnetid, $ip6, $hostname, $mac) if $ip6;
+ PVE::Network::SDN::Dhcp::update_mapping($vnetid, $mac) if $ip4 || $ip6;
+
return ($ip4, $ip6);
}
@@ -225,11 +227,12 @@ sub add_dhcp_mapping {
return if !$zone->{ipam} || !$zone->{dhcp};
my ($ip4, $ip6) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ # one push at the end covers the allocated and the existing records alike
add_next_free_cidr($vnetid, $name, $mac, "$vmid", undef, 1, 4) if !$ip4;
add_next_free_cidr($vnetid, $name, $mac, "$vmid", undef, 1, 6) if !$ip6;
($ip4, $ip6) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
- PVE::Network::SDN::Dhcp::add_mapping($vnetid, $mac, $ip4, $ip6) if $ip4 || $ip6;
+ PVE::Network::SDN::Dhcp::update_mapping($vnetid, $mac) if $ip4 || $ip6;
}
1;
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 9f4c424..8273715 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -46,6 +46,7 @@ sub clear_test_state {
vnets_config => {},
macdb => {},
ipamdb => {},
+ dnsmasq_calls => [],
ipam_config => {
'ids' => {
'pve' => {
@@ -220,7 +221,9 @@ $mocked_sdn_dhcp_dnsmasq->mock(
assert_dnsmasq_installed => sub { return 1; },
before_configure => sub { },
ethers_file => sub { return "/tmp/ethers"; },
- systemctl_service => sub { },
+ systemctl_service => sub {
+ push $test_state->{dnsmasq_calls}->@*, $_[0];
+ },
update_lease => sub { },
);
@@ -239,6 +242,11 @@ $mocked_rpc_env_obj->mock(
check_any => sub { return 1; },
);
+my $mocked_pve_inotify = Test::MockModule->new('PVE::INotify');
+$mocked_pve_inotify->mock(
+ nodename => sub { return 'localnode'; },
+);
+
my $mocked_pve_cluster_obj = Test::MockModule->new('PVE::Cluster');
$mocked_pve_cluster_obj->mock(
check_cfs_quorum => sub { return 1; },
@@ -288,6 +296,11 @@ sub create_zone {
return $zone;
}
+sub update_zone {
+ my ($zoneid, $params) = @_;
+ PVE::API2::Network::SDN::Zones->update({ zone => $zoneid, %$params });
+}
+
sub get_vnet {
my ($id) = @_;
return eval { PVE::API2::Network::SDN::Vnets->read({ vnet => $id }); };
@@ -330,6 +343,16 @@ sub create_ip {
return PVE::API2::Network::SDN::Ips->ipcreate($param);
}
+sub update_ip {
+ my ($param) = @_;
+ return PVE::API2::Network::SDN::Ips->ipupdate($param);
+}
+
+sub delete_ip {
+ my ($param) = @_;
+ return PVE::API2::Network::SDN::Ips->ipdelete($param);
+}
+
sub run_test {
my $test = shift;
clear_test_state();
@@ -963,4 +986,208 @@ run_test(
2,
);
+# -------------- dnsmasq mapping pushes
+
+sub test_dnsmasq_mapping_push {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+
+ # the first push of a zone finds no ethers file yet
+ unlink($TMP_ETHERS_FILE);
+
+ my $ethers = sub { PVE::Tools::file_get_contents($TMP_ETHERS_FILE) };
+ my $reloads = sub {
+ my $count = scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*);
+ $test_state->{dnsmasq_calls} = [];
+ return $count;
+ };
+
+ eval { nic_start($vnetid, $mac, "999", "testhostname"); };
+ if ($@) {
+ fail("$test_name: nic_start: $@");
+ return;
+ }
+ is($ethers->(), "$mac,10.0.0.100\n", "$test_name: guest start reserves the allocation");
+ is($reloads->(), 1, "$test_name: guest start reloads once");
+
+ # the records exist already, the restart pushes them without a change
+ nic_start($vnetid, $mac, "999", "testhostname");
+ is($ethers->(), "$mac,10.0.0.100\n", "$test_name: restart keeps the reservation");
+ is($reloads->(), 0, "$test_name: restart without a change does not reload");
+
+ update_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ is($ethers->(), "$mac,10.0.0.150\n", "$test_name: mapping edit replaces the reservation");
+ is($reloads->(), 1, "$test_name: mapping edit reloads once");
+
+ # a push writes what the cache holds, so a repeated one rewrites nothing
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
+ is($ethers->(), "$mac,10.0.0.150\n", "$test_name: a repeated push keeps the reservation");
+ is($reloads->(), 0, "$test_name: a repeated push changes nothing");
+
+ # the cache and the file may spell a MAC differently, it is one MAC
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, uc($mac));
+ is(
+ $ethers->(),
+ "$mac,10.0.0.150\n",
+ "$test_name: a push spelled differently keeps the line as it is",
+ );
+ is($reloads->(), 0, "$test_name: and reloads nothing");
+
+ delete_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ is($ethers->(), "", "$test_name: mapping delete drops the reservation");
+ is($reloads->(), 1, "$test_name: mapping delete reloads once");
+
+ # a zone confined to other nodes runs no dnsmasq here, an edit is not
+ # pushed on this node
+ update_zone($zoneid, { nodes => 'other' });
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ is($ethers->(), "", "$test_name: a zone confined to other nodes gets no push here");
+ is($reloads->(), 0, "$test_name: nothing to reload for it");
+}
+
+run_test(\&test_dnsmasq_mapping_push);
+
+sub test_dnsmasq_dual_stack_and_sweep {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "8888::/64",
+ gateway => "8888::1",
+ 'dhcp-range' => ["start-address=8888::100,end-address=8888::200"],
+ });
+
+ # a reservation of a MAC the cache does not know is swept by the next
+ # rewrite, both addresses of the guest go on one line
+ PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "aa:bb:cc:dd:ee:ff,10.0.0.77\n");
+ $test_state->{dnsmasq_calls} = [];
+
+ eval { nic_start($vnetid, $mac, "999", "testhostname"); };
+ if ($@) {
+ fail("$test_name: nic_start: $@");
+ return;
+ }
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$mac,10.0.0.100,[8888::100]\n",
+ "$test_name: dual-stack reservation on one line, the unknown MAC swept",
+ );
+ is(
+ scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
+ 1,
+ "$test_name: one reload for the rewrite",
+ );
+
+ # an external IPAM may spell an address in upper case, the file holds
+ # one spelling
+ $test_state->{macdb}->{macs}->{$mac}->{ip6} = '8888::AB';
+ $test_state->{dnsmasq_calls} = [];
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$mac,10.0.0.100,[8888::ab]\n",
+ "$test_name: the reservation is written in one spelling",
+ );
+ is(
+ scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
+ 1,
+ "$test_name: one reload for the changed address",
+ );
+
+ # an entry without an address, as the old code wrote on a miss, says
+ # nothing about its MAC, so a line of it is swept
+ my $stale = 'da:65:8f:18:9b:70';
+ $test_state->{macdb}->{macs}->{$stale} = { ip4 => undef, ip6 => undef };
+ PVE::Tools::file_set_contents(
+ $TMP_ETHERS_FILE,
+ "$mac,10.0.0.100,[8888::ab]\n$stale,10.0.0.101\n",
+ );
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$mac,10.0.0.100,[8888::ab]\n",
+ "$test_name: a line of a MAC cached without an address is swept",
+ );
+
+ # a line the old code wrote with an upper-case IPv6 address is kept
+ my $other = 'da:65:8f:18:9b:71';
+ $test_state->{macdb}->{macs}->{$other} = { ip4 => '10.0.0.102', ip6 => '8888::AB' };
+ PVE::Tools::file_set_contents(
+ $TMP_ETHERS_FILE,
+ "$mac,10.0.0.100,[8888::ab]\n$other,10.0.0.102,[8888::AB]\n",
+ );
+ $test_state->{dnsmasq_calls} = [];
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$mac,10.0.0.100,[8888::ab]\n$other,10.0.0.102,[8888::AB]\n",
+ "$test_name: a line of another MAC spelled by the old code is kept",
+ );
+ is(
+ scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
+ 0,
+ "$test_name: without a reload",
+ );
+}
+
+run_test(\&test_dnsmasq_dual_stack_and_sweep);
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH pve-network v2 05/16] sdn: ipam: do not cache negative per-MAC answers, lock the write
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (3 preceding siblings ...)
2026-09-09 10:41 ` [PATCH pve-network v2 04/16] sdn: push mapping changes from the ipam API to the dhcp backend Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH pve-network v2 06/16] sdn: subnets: add dhcp-lease-time property Hannes Laimer
` (10 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
The lookup cache wrote an entry even when the plugin returned nothing,
and that entry short-circuits every later lookup. So a record created
after the first miss was never seen again. The read-modify-write also
ran without the cluster lock the other cache writers take, allowing
concurrent lookups to drop each other's entries.
Only cache actual answers and take the lock for the write, keeping the
common cache-hit path lock-free. An entry without an address, as
installs running the old code have them, counts as a miss. The cache is
keyed by the MAC in lower case, its writers spell it either way. A read
and a delete cover an entry of the old code under another spelling as
well, and a write moves it to the lower-case key. So a MAC keeps one
entry, and a mapping released after the upgrade releases its cache entry
too.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/Network/SDN/Ipams.pm | 61 +++++++++++---
src/test/run_test_vnets_blackbox.pl | 118 ++++++++++++++++++++++++++++
2 files changed, 168 insertions(+), 11 deletions(-)
diff --git a/src/PVE/Network/SDN/Ipams.pm b/src/PVE/Network/SDN/Ipams.pm
index 179bdf7..1612a11 100644
--- a/src/PVE/Network/SDN/Ipams.pm
+++ b/src/PVE/Network/SDN/Ipams.pm
@@ -47,14 +47,27 @@ sub write_macdb {
cfs_write_file($macdb_filename, $data);
}
+# an entry of the old code under another spelling moves to the lower-case key
+my sub fold_spellings {
+ my ($db, $mac) = @_;
+
+ for my $key (grep { $_ ne $mac && lc($_) eq $mac } reverse sort keys $db->{macs}->%*) {
+ my $old = delete $db->{macs}->{$key};
+ $db->{macs}->{$mac}->{$_} //= $old->{$_} for grep { defined($old->{$_}) } qw(ip4 ip6);
+ }
+}
+
+# the cache is keyed by the MAC in lower case, its writers spell it either way
sub add_cache_mac_ip {
my ($mac, $ip) = @_;
+ $mac = lc($mac);
cfs_lock_file(
$macdb_filename,
undef,
sub {
my $db = read_macdb();
+ fold_spellings($db, $mac);
if (Net::IP::ip_is_ipv4($ip)) {
$db->{macs}->{$mac}->{ip4} = $ip;
} else {
@@ -74,13 +87,17 @@ sub del_cache_mac_ip {
undef,
sub {
my $db = read_macdb();
- if (Net::IP::ip_is_ipv4($ip)) {
- delete $db->{macs}->{$mac}->{ip4};
- } else {
- delete $db->{macs}->{$mac}->{ip6};
+ # entries of the old code carry the spelling their writer used
+ for my $key (grep { lc($_) eq lc($mac) } keys $db->{macs}->%*) {
+ if (Net::IP::ip_is_ipv4($ip)) {
+ delete $db->{macs}->{$key}->{ip4};
+ } else {
+ delete $db->{macs}->{$key}->{ip6};
+ }
+ delete $db->{macs}->{$key}
+ if !defined($db->{macs}->{$key}->{ip4})
+ && !defined($db->{macs}->{$key}->{ip6});
}
- delete $db->{macs}->{$mac}
- if !defined($db->{macs}->{$mac}->{ip4}) && !defined($db->{macs}->{$mac}->{ip6});
write_macdb($db);
},
);
@@ -136,16 +153,38 @@ sub get_ips_from_mac {
my ($mac, $zoneid, $zone) = @_;
my $macdb = read_macdb();
- return ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6}) if $macdb->{macs}->{$mac};
+ # entries of the old code carry the spelling their writer used, the
+ # lower-case one wins where two hold the same family
+ my %cached;
+ for my $key (grep { lc($_) eq lc($mac) } sort keys $macdb->{macs}->%*) {
+ my $entry = $macdb->{macs}->{$key};
+ $cached{$_} = $entry->{$_} for grep { defined($entry->{$_}) } qw(ip4 ip6);
+ }
+
+ # an entry holding no address says nothing about the MAC, ask the IPAM
+ return ($cached{ip4}, $cached{ip6}) if defined($cached{ip4}) || defined($cached{ip6});
my $plugin_config = get_plugin_config($zone);
my $plugin = PVE::Network::SDN::Ipams::Plugin->lookup($plugin_config->{type});
- ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6}) =
- $plugin->get_ips_from_mac($plugin_config, $mac, $zoneid);
+ my ($ip4, $ip6) = $plugin->get_ips_from_mac($plugin_config, $mac, $zoneid);
+
+ # an empty answer is not cached, the record may simply not exist yet
+ return if !defined($ip4) && !defined($ip6);
- write_macdb($macdb);
+ cfs_lock_file(
+ $macdb_filename,
+ undef,
+ sub {
+ my $db = read_macdb();
+ fold_spellings($db, lc($mac));
+ $db->{macs}->{ lc($mac) }->{ip4} = $ip4 if defined($ip4);
+ $db->{macs}->{ lc($mac) }->{ip6} = $ip6 if defined($ip6);
+ write_macdb($db);
+ },
+ );
+ warn "$@" if $@;
- return ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6});
+ return ($ip4, $ip6);
}
1;
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 8273715..c1878dc 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -1083,6 +1083,124 @@ sub test_dnsmasq_mapping_push {
run_test(\&test_dnsmasq_mapping_push);
+sub test_ipam_cache_misses {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+
+ my @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ is(scalar(grep { defined } @ips), 0, "$test_name: an unknown MAC has no answer");
+ ok(!exists $test_state->{macdb}->{macs}->{$mac}, "$test_name: the miss is not cached");
+
+ # an entry holding no address, left behind by an earlier miss, does not
+ # hide a record created since
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.50",
+ });
+ $test_state->{macdb}->{macs}->{$mac} = { ip4 => undef, ip6 => undef };
+ @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ is($ips[0], "10.0.0.50", "$test_name: an address-less entry counts as a miss");
+ is(
+ $test_state->{macdb}->{macs}->{$mac}->{ip4},
+ "10.0.0.50",
+ "$test_name: the answer is cached",
+ );
+}
+
+run_test(\&test_ipam_cache_misses);
+
+sub test_ipam_cache_case {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # an entry the old code wrote under the config's spelling is read and
+ # released like one of the new code
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ $test_state->{macdb}->{macs}->{ uc($mac) } = { ip4 => '10.0.0.150' };
+ my ($ip4) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ is($ip4, '10.0.0.150', "$test_name: an old entry is found however it is spelled");
+ PVE::Network::SDN::Ipams::del_cache_mac_ip($mac, '10.0.0.150');
+ ok(
+ !(grep { lc($_) eq $mac } keys $test_state->{macdb}->{macs}->%*),
+ "$test_name: and released",
+ );
+
+ # a write spelled either way moves such an entry to the lower-case key
+ # and keeps what it held
+ $test_state->{macdb}->{macs}->{ uc($mac) } = { ip4 => '10.0.0.150' };
+ PVE::Network::SDN::Ipams::add_cache_mac_ip(uc($mac), 'fd00::150');
+ is_deeply(
+ [grep { lc($_) eq $mac } keys $test_state->{macdb}->{macs}->%*],
+ [$mac],
+ "$test_name: a write leaves one entry for the MAC",
+ );
+ is_deeply(
+ $test_state->{macdb}->{macs}->{$mac},
+ { ip4 => '10.0.0.150', ip6 => 'fd00::150' },
+ "$test_name: holding both addresses",
+ );
+
+ # the lookup's write moves an address-less entry of the old code as well
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.160",
+ });
+ $test_state->{macdb}->{macs} = { uc($mac) => { ip4 => undef, ip6 => undef } };
+ my @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ is($ips[0], '10.0.0.160', "$test_name: a lookup past an address-less entry asks the IPAM");
+ is_deeply(
+ [sort keys $test_state->{macdb}->{macs}->%*],
+ [$mac],
+ "$test_name: and its write leaves one entry for the MAC",
+ );
+}
+
+run_test(\&test_ipam_cache_case);
+
sub test_dnsmasq_dual_stack_and_sweep {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH pve-network v2 06/16] sdn: subnets: add dhcp-lease-time property
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (4 preceding siblings ...)
2026-09-09 10:41 ` [PATCH pve-network v2 05/16] sdn: ipam: do not cache negative per-MAC answers, lock the write Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH pve-network v2 07/16] sdn: dhcp: only assert a backend's availability for zones using it Hannes Laimer
` (9 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
Lease time is the convergence knob for mapping edits, a client picks
up a changed record at the latest one lease after the edit. Make it
configurable per subnet for dhcp backends that honor it.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 3 +-
src/PVE/Network/SDN/SubnetPlugin.pm | 11 +++++++
src/test/run_test_vnets_blackbox.pl | 51 +++++++++++++++++++++++++++++
3 files changed, 64 insertions(+), 1 deletion(-)
diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
index f3c7739..de7420a 100644
--- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
+++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
@@ -192,7 +192,8 @@ sub configure_subnet {
$mask = join('.', unpack("C4", pack("N", $mask)));
}
- push @{$config}, "dhcp-range=set:$tag,$network,static,$mask,infinite";
+ my $lease = $subnet_config->{'dhcp-lease-time'} // 'infinite';
+ push @{$config}, "dhcp-range=set:$tag,$network,static,$mask,$lease";
my $option_string;
if (ip_is_ipv6($subnet_config->{network})) {
diff --git a/src/PVE/Network/SDN/SubnetPlugin.pm b/src/PVE/Network/SDN/SubnetPlugin.pm
index e2a0e50..29edf7b 100644
--- a/src/PVE/Network/SDN/SubnetPlugin.pm
+++ b/src/PVE/Network/SDN/SubnetPlugin.pm
@@ -177,6 +177,16 @@ sub properties {
description => 'IP address for the DNS server',
optional => 1,
},
+ 'dhcp-lease-time' => {
+ type => 'integer',
+ minimum => 60,
+ maximum => 4294967295,
+ description =>
+ 'Lease time in seconds for DHCP answers. Without it dnsmasq hands out'
+ . ' infinite leases, and it raises anything below two minutes to two'
+ . ' minutes.',
+ optional => 1,
+ },
};
}
@@ -189,6 +199,7 @@ sub options {
dnszoneprefix => { optional => 1 },
'dhcp-range' => { optional => 1 },
'dhcp-dns-server' => { optional => 1 },
+ 'dhcp-lease-time' => { optional => 1 },
};
}
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index c1878dc..3e78ba9 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -334,6 +334,11 @@ sub create_subnet {
PVE::API2::Network::SDN::Subnets->create($params);
}
+sub update_subnet {
+ my ($params) = @_;
+ PVE::API2::Network::SDN::Subnets->update($params);
+}
+
sub get_ipam_entries {
return PVE::API2::Network::SDN::Ipams->ipamindex({ ipam => "pve" });
}
@@ -1083,6 +1088,52 @@ sub test_dnsmasq_mapping_push {
run_test(\&test_dnsmasq_mapping_push);
+sub test_dnsmasq_lease_time {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+
+ # the subnet's lease time reaches the range, infinite stays the default
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ my $range = sub {
+ my $subnets = PVE::Network::SDN::Vnets::get_subnets($vnetid, 1);
+ my $config = [];
+ for my $id (sort keys %$subnets) {
+ PVE::Network::SDN::Dhcp::Dnsmasq->configure_subnet(
+ $config, $zoneid, $vnetid, $subnets->{$id},
+ );
+ }
+ my ($line) = grep { m/^dhcp-range=/ } @$config;
+ return $line;
+ };
+ like($range->(), qr/,infinite$/, "$test_name: no lease time means infinite leases");
+ update_subnet({
+ vnet => $vnetid,
+ subnet => "$zoneid-10.0.0.0-24",
+ 'dhcp-lease-time' => 300,
+ });
+ like($range->(), qr/,300$/, "$test_name: the lease time reaches the range");
+}
+
+run_test(\&test_dnsmasq_lease_time);
+
sub test_ipam_cache_misses {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH pve-network v2 07/16] sdn: dhcp: only assert a backend's availability for zones using it
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (5 preceding siblings ...)
2026-09-09 10:41 ` [PATCH pve-network v2 06/16] sdn: subnets: add dhcp-lease-time property Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH pve-network v2 08/16] sdn: dhcp: add ebpf plugin Hannes Laimer
` (8 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
The regenerate told every registered backend whether any zone at all
uses dhcp. So any zone with dhcp configured made every backend assert
its own availability, and a failing assert aborts the whole regenerate.
With one backend that was the same thing. With a second one it breaks
network reloads on nodes that only run the other backend. Gate each
backend on the zones using it on this node, and leave the zones of other
nodes unconfigured here as well.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/Network/SDN/Dhcp.pm | 13 +++++++--
src/test/run_test_vnets_blackbox.pl | 43 +++++++++++++++++++++++++++++
2 files changed, 53 insertions(+), 3 deletions(-)
diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
index 28a9f2e..1f3619d 100644
--- a/src/PVE/Network/SDN/Dhcp.pm
+++ b/src/PVE/Network/SDN/Dhcp.pm
@@ -59,17 +59,24 @@ sub regenerate_config {
my $plugins = PVE::Network::SDN::Dhcp::Plugin->lookup_types();
- my $any_zone_needs_dhcp = grep { $_->{dhcp} } values $zone_cfg->{ids}->%*;
+ my %plugin_needed = ();
+ for my $zone (values $zone_cfg->{ids}->%*) {
+ next if !$zone->{dhcp};
+ # a zone confined to other nodes runs no backend here
+ next if defined($zone->{nodes}) && !$zone->{nodes}->{$nodename};
+ $plugin_needed{ $zone->{dhcp} } = 1;
+ }
foreach my $plugin_name (@$plugins) {
my $plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($plugin_name);
- eval { $plugin->before_regenerate(!$any_zone_needs_dhcp) };
+ eval { $plugin->before_regenerate(!$plugin_needed{$plugin_name}) };
die "Could not run before_regenerate for DHCP plugin $plugin_name $@\n" if $@;
}
foreach my $zoneid (sort keys %{ $zone_cfg->{ids} }) {
my $zone = $zone_cfg->{ids}->{$zoneid};
next if !$zone->{dhcp};
+ next if defined($zone->{nodes}) && !$zone->{nodes}->{$nodename};
my $dhcp_plugin_name = $zone->{dhcp};
my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($dhcp_plugin_name);
@@ -110,7 +117,7 @@ sub regenerate_config {
warn "Could not configure vnet $vnetid: $@\n" if $@;
}
- eval { $dhcp_plugin->after_configure($zoneid, !$any_zone_needs_dhcp) };
+ eval { $dhcp_plugin->after_configure($zoneid, !$plugin_needed{$dhcp_plugin_name}) };
warn "Could not run after_configure for DHCP server $zoneid $@\n" if $@;
}
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 3e78ba9..2429adf 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -104,6 +104,13 @@ my $mocked_pve_sdn;
$mocked_pve_sdn = Test::MockModule->new('PVE::Network::SDN');
$mocked_pve_sdn->mock(
cfs_lock_file => $mocked_cfs_lock_file,
+ running_config => sub {
+ return {
+ zones => $test_state->{zones_config},
+ vnets => $test_state->{vnets_config},
+ subnets => $test_state->{subnets_config},
+ };
+ },
);
my $mocked_pve_tools = Test::MockModule->new('PVE::Tools');
@@ -1134,6 +1141,42 @@ sub test_dnsmasq_lease_time {
run_test(\&test_dnsmasq_lease_time);
+sub test_dhcp_backend_needed_per_node {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+
+ # a backend is told whether a zone on this node uses it, a zone confined
+ # to other nodes runs nothing here
+ my $asked = [];
+ my $configured = [];
+ $mocked_sdn_dhcp_dnsmasq->mock(
+ before_regenerate => sub { push @$asked, $_[1] ? 'optional' : 'needed'; },
+ after_configure => sub { push @$configured, $_[1]; },
+ );
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => "DNSZONE",
+ nodes => 'other',
+ });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ $asked,
+ ['optional'],
+ "$test_name: a dnsmasq zone confined elsewhere needs no dnsmasq here",
+ );
+ eq_or_diff($configured, [], "$test_name: and is not configured here");
+
+ @$asked = ();
+ update_zone("DNSZONE", { delete => 'nodes' });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff($asked, ['needed'], "$test_name: a dnsmasq zone on this node needs it");
+ eq_or_diff($configured, ['DNSZONE'], "$test_name: and is configured");
+ $mocked_sdn_dhcp_dnsmasq->unmock($_) for qw(before_regenerate after_configure);
+}
+
+run_test(\&test_dhcp_backend_needed_per_node);
+
sub test_ipam_cache_misses {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH pve-network v2 08/16] sdn: dhcp: add ebpf plugin
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (6 preceding siblings ...)
2026-09-09 10:41 ` [PATCH pve-network v2 07/16] sdn: dhcp: only assert a backend's availability for zones using it Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH pve-network v2 09/16] sdn: zones: attach the dhcp responder on tap plug, detach on unplug Hannes Laimer
` (7 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
A dhcp backend that programs the proxmox-ebpf per-tap DHCP responder
instead of driving a dnsmasq instance, selectable per zone with
dhcp=ebpf. Answers come from the same per-MAC records dnsmasq serves
reservations from, handed in-process to the responder through the pve-rs
bindings. The record names the NIC's MAC and the responder answers no
other, as dnsmasq ignores an unknown MAC.
A record change writes that one MAC's record. The full pass on a
regenerate compiles the whole desired state and lets the responder
converge on it. That state is the records of the ebpf zones and the
guest NICs on their vnets, each interface with the vnet it sits on and
the answer its MAC gets there. It is read from the running config, the
MAC cache and the guest configs. Every change carries a generation drawn
before its input is read. So the responder drops one that arrives after
a newer one of the same record, and nothing has to serialize the
collection. A tap plug draws its generation before it reads the cache.
The number orders that read and says nothing about the guest config,
which a hotplug writes after the plug. The vnet in every record covers
that, the responder leaves a record naming another vnet than the config
alone with its link. A tap plugged onto a bridge the backend does not
serve trades its entry for a marker naming that place instead. So a pass
still reading the old vnet leaves it alone, and one reading the new
place keeps it. An unplug drops the entry, the interface goes away with
it. A responder with nothing loaded, as after boot, answers a single
change with a request for the full pass, which then runs first.
The cache is keyed by the MAC as its writer spelled it and the guest
configs spell it their own way, so the plugin matches them regardless of
case. An ebpf zone confined to other nodes runs no responder here. The
cluster state is refreshed before the cache is read. An IPv4 subnet
whose DNS server or a zone whose MTU the responder cannot hand out is
reported on every apply, guests on it or not. The MTU handed out is the
zone's, as dnsmasq does. The pinned state does not survive a reboot, so
the backends get a boot hook the SDN commit runs before the guests
start. A plug that still finds nothing loaded runs the pass itself, its
retry drawing afresh so its input is read after the pass.
Guests get answers without a DHCP daemon per zone and, once records are
pushed, independent of IPAM reachability. The responder identifies
itself with the gateway address. A subnet without one is served under a
link-local identifier and without a default route, so its guests renew
by broadcast. IPv4 only.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
debian/libpve-network-perl.prerm | 13 +
src/PVE/API2/Network/SDN/Zones.pm | 3 +-
src/PVE/Network/SDN/Dhcp.pm | 82 ++++
src/PVE/Network/SDN/Dhcp/Ebpf.pm | 355 +++++++++++++++++
src/PVE/Network/SDN/Dhcp/Makefile | 2 +-
src/PVE/Network/SDN/Dhcp/Plugin.pm | 19 +
src/PVE/Network/SDN/SubnetPlugin.pm | 4 +-
src/test/run_test_vnets_blackbox.pl | 596 ++++++++++++++++++++++++++++
8 files changed, 1070 insertions(+), 4 deletions(-)
create mode 100755 debian/libpve-network-perl.prerm
create mode 100644 src/PVE/Network/SDN/Dhcp/Ebpf.pm
diff --git a/debian/libpve-network-perl.prerm b/debian/libpve-network-perl.prerm
new file mode 100755
index 0000000..2a89236
--- /dev/null
+++ b/debian/libpve-network-perl.prerm
@@ -0,0 +1,13 @@
+#!/bin/sh
+
+set -e
+
+case "$1" in
+ remove)
+ # the eBPF DHCP responder stays attached through its pins in bpffs, with
+ # the package gone nothing would ever update or detach it
+ rm -rf /sys/fs/bpf/proxmox-ebpf/dhcp
+ ;;
+esac
+
+exit 0
diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm
index b897cbd..0e90726 100644
--- a/src/PVE/API2/Network/SDN/Zones.pm
+++ b/src/PVE/API2/Network/SDN/Zones.pm
@@ -16,6 +16,7 @@ use PVE::Network::SDN::Dns;
use PVE::Network::SDN::Subnets;
use PVE::Network::SDN::Vnets;
use PVE::Network::SDN;
+use PVE::Network::SDN::Dhcp;
use PVE::Network::SDN::Zones::EvpnPlugin;
use PVE::Network::SDN::Zones::FaucetPlugin;
@@ -90,7 +91,7 @@ my $ZONE_PROPERTIES = {
},
dhcp => {
type => 'string',
- enum => ['dnsmasq'],
+ enum => PVE::Network::SDN::Dhcp->plugin_types(),
optional => 1,
description => 'Name of DHCP server backend for this zone.',
},
diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
index 1f3619d..3b2d798 100644
--- a/src/PVE/Network/SDN/Dhcp.pm
+++ b/src/PVE/Network/SDN/Dhcp.pm
@@ -9,6 +9,7 @@ use PVE::Network::SDN;
use PVE::Network::SDN::Subnets;
use PVE::Network::SDN::Dhcp::Plugin;
use PVE::Network::SDN::Dhcp::Dnsmasq;
+use PVE::Network::SDN::Dhcp::Ebpf;
use PVE::INotify;
@@ -17,6 +18,9 @@ PVE::Network::SDN::Dhcp::Plugin->init();
PVE::Network::SDN::Dhcp::Dnsmasq->register();
PVE::Network::SDN::Dhcp::Dnsmasq->init();
+PVE::Network::SDN::Dhcp::Ebpf->register();
+PVE::Network::SDN::Dhcp::Ebpf->init();
+
sub plugin_types {
return PVE::Network::SDN::Dhcp::Plugin->lookup_types();
}
@@ -45,6 +49,84 @@ sub update_mapping {
warn "could not update dhcp mapping for $mac: $@" if $@;
}
+# every guest NIC in the cluster as its config names it, with the interface
+# the guest has for it on its node. The config lines come from pmxcfs in one
+# go, no guest config gets parsed for this. A guest the vmlist does not know
+# is skipped.
+sub guest_nics {
+ # a hook script starts with the cluster cache empty
+ PVE::Cluster::cfs_update();
+ my $vmlist = PVE::Cluster::get_vmlist();
+ my $nets = PVE::Cluster::get_guest_config_properties([map { "net$_" } 0 .. 31]);
+ my $nics = [];
+ for my $vmid (sort keys %$nets) {
+ my $guest = $vmlist->{ids}->{$vmid} // next;
+ my $prefix = ($guest->{type} // '') eq 'lxc' ? 'veth' : 'tap';
+ for my $key (sort keys %{ $nets->{$vmid} }) {
+ my ($index) = $key =~ m/^net(\d+)$/ or next;
+ my $net = $nets->{$vmid}->{$key};
+ my ($bridge) = $net =~ m/(?:^|,)bridge=([^,]+)/ or next;
+ # a VM names the MAC as the value of its model, a container as hwaddr
+ my ($mac) = $net =~ m/(?:^|,)[a-z0-9_-]+=([0-9a-f]{2}(?::[0-9a-f]{2}){5})(?:,|$)/i;
+ push @$nics,
+ {
+ vmid => $vmid,
+ node => $guest->{node},
+ iface => "$prefix${vmid}i$index",
+ bridge => $bridge,
+ mac => $mac,
+ };
+ }
+ }
+
+ return $nics;
+}
+
+# the interface may come from a vnet of another backend, or of none, so
+# every other backend drops what it holds for it first
+sub tap_plug {
+ my ($bridge, $iface, $mac) = @_;
+
+ my $vnet = PVE::Network::SDN::Vnets::get_vnet($bridge, 1);
+ my $zone = $vnet ? PVE::Network::SDN::Zones::get_zone($vnet->{zone}, 1) : undef;
+ my $backend = $zone ? $zone->{dhcp} : undef;
+
+ # every other backend hears where the interface sits now, a plain bridge
+ # is no vnet
+ my $target = { vnet => $vnet ? $bridge : undef };
+ for my $type (PVE::Network::SDN::Dhcp::Plugin->lookup_types()->@*) {
+ next if defined($backend) && $type eq $backend;
+ eval { PVE::Network::SDN::Dhcp::Plugin->lookup($type)->tap_unplug($iface, $target) };
+ warn "could not unplug $iface from the $type dhcp backend: $@" if $@;
+ }
+ return if !$backend;
+
+ eval {
+ PVE::Network::SDN::Dhcp::Plugin->lookup($backend)
+ ->tap_plug($vnet->{zone}, $bridge, $iface, $mac);
+ };
+ warn "could not plug $iface into the dhcp backend of zone $vnet->{zone}: $@" if $@;
+}
+
+# every backend brings back what a reboot took, run by the SDN commit at boot
+sub boot {
+ for my $type (PVE::Network::SDN::Dhcp::Plugin->lookup_types()->@*) {
+ eval { PVE::Network::SDN::Dhcp::Plugin->lookup($type)->boot() };
+ warn "could not bring up the $type dhcp backend: $@" if $@;
+ }
+}
+
+# the interface is usually gone already and with it any way to its zone,
+# every backend gets to drop what it holds for it
+sub tap_unplug {
+ my ($iface) = @_;
+
+ for my $type (PVE::Network::SDN::Dhcp::Plugin->lookup_types()->@*) {
+ eval { PVE::Network::SDN::Dhcp::Plugin->lookup($type)->tap_unplug($iface) };
+ warn "could not unplug $iface from the $type dhcp backend: $@" if $@;
+ }
+}
+
sub regenerate_config {
my ($reload) = @_;
diff --git a/src/PVE/Network/SDN/Dhcp/Ebpf.pm b/src/PVE/Network/SDN/Dhcp/Ebpf.pm
new file mode 100644
index 0000000..fa20da3
--- /dev/null
+++ b/src/PVE/Network/SDN/Dhcp/Ebpf.pm
@@ -0,0 +1,355 @@
+package PVE::Network::SDN::Dhcp::Ebpf;
+
+use strict;
+use warnings;
+
+use base qw(PVE::Network::SDN::Dhcp::Plugin);
+
+use Net::IP;
+use Net::Subnet qw(subnet_matcher);
+
+use PVE::Cluster;
+use PVE::INotify;
+use PVE::Network::SDN::Ipams;
+use PVE::RESTEnvironment qw(log_warn);
+
+use PVE::RS::SDN::Dhcp;
+
+my $DEFAULT_LEASE_TIME = 600;
+
+# a subnet without a gateway is served under a link-local server identifier,
+# which nothing on the subnet can carry. A guest cannot renew by unicast then
+# and broadcasts at its rebinding time instead
+my $SERVER_ID_NO_GATEWAY = '169.254.0.1';
+
+sub type {
+ return 'ebpf';
+}
+
+# the MTU option is 16 bits wide with 68 the smallest MTU IPv4 allows, a zone
+# MTU outside that is reported on an apply and not handed out.
+# TODO: bound the zone mtu in its schema, then this check can go
+my sub zone_mtu {
+ my ($zoneid, $zone, $report) = @_;
+
+ my $mtu = PVE::Network::SDN::Zones::get_mtu($zone);
+ return $mtu if $mtu >= 68 && $mtu <= 65535;
+ log_warn("zone $zoneid has MTU $mtu, not handing it out over DHCP") if $report;
+ return undef;
+}
+
+my sub dhcp_record {
+ my ($ip4, $subnet, $mtu, $mac) = @_;
+
+ my $gateway = $subnet->{gateway};
+
+ # a DNS server has to be reachable over the address family served
+ my $dns = $subnet->{'dhcp-dns-server'};
+ $dns = undef if defined($dns) && !Net::IP::ip_is_ipv4($dns);
+
+ # the config hands its numbers over as strings, the bindings take integers only
+ return {
+ ip => $ip4,
+ prefixlen => int($subnet->{mask}),
+ server_id => $gateway // $SERVER_ID_NO_GATEWAY,
+ lease => int($subnet->{'dhcp-lease-time'} // $DEFAULT_LEASE_TIME),
+ router => $gateway,
+ dns => $dns,
+ # the responder answers a request carrying this MAC and no other
+ mac => lc($mac),
+ mtu => defined($mtu) ? int($mtu) : undef,
+ };
+}
+
+# The desired state is compiled from the configs without a lock, so two
+# changes can reach the responder in the wrong order. Each carries a
+# generation drawn after its input was written and the responder drops a
+# change older than what it holds. A tap plug draws before it reads the
+# cache. Its number orders that read and says nothing about the guest
+# config, which is not written yet. The vnet in its record covers that.
+
+my sub generation {
+ return PVE::RS::SDN::Dhcp::next_generation();
+}
+
+# the IPv4 subnets of a vnet. What the responder leaves out of an answer
+# is reported only when asked. The full pass on an apply is where an admin
+# sees that once, a guest start must not repeat it
+
+my sub served_subnets {
+ my ($vnetid, $report) = @_;
+
+ my $subnets = PVE::Network::SDN::Vnets::get_subnets($vnetid, 1) // {};
+ my $served = [];
+ for my $subnetid (sort keys %$subnets) {
+ my $subnet = $subnets->{$subnetid};
+ next if !Net::IP::ip_is_ipv4($subnet->{network});
+ log_warn("subnet $subnetid has an IPv6 DNS server, not handing it out over IPv4")
+ if $report
+ && defined($subnet->{'dhcp-dns-server'})
+ && !Net::IP::ip_is_ipv4($subnet->{'dhcp-dns-server'});
+ push @$served, $subnet;
+ }
+
+ return $served;
+}
+
+# the cache is keyed by the MAC as its writer spelled it, the guest configs
+# spell it their own way
+my sub cached_macs {
+ my ($macdb) = @_;
+
+ my $macs = {};
+ for my $mac (sort keys $macdb->{macs}->%*) {
+ my $entry = $macdb->{macs}->{$mac};
+ $macs->{ lc($mac) }->{$_} = $entry->{$_} for grep { defined($entry->{$_}) } qw(ip4 ip6);
+ }
+ return $macs;
+}
+
+my $matchers = {};
+
+# the answer of a MAC on one vnet, none when nothing is to be served. The
+# vnet's subnet holding the cached address decides
+my sub vnet_record {
+ my ($subnets, $mtu, $macs, $mac) = @_;
+
+ my $ip4 = $macs->{ lc($mac) }->{ip4};
+ return undef if !$ip4;
+ # the most specific of overlapping subnets is the one to answer from
+ for my $subnet (sort { $b->{mask} <=> $a->{mask} } @$subnets) {
+ my $matcher = $matchers->{ $subnet->{cidr} } //= subnet_matcher($subnet->{cidr});
+ next if !$matcher->($ip4);
+ # the vnet's own gateway address is cached too and never a lease
+ return undef if defined($subnet->{gateway}) && $ip4 eq $subnet->{gateway};
+ return dhcp_record($ip4, $subnet, $mtu, $mac);
+ }
+
+ return undef;
+}
+
+# The complete desired state of this node's responder, built from the
+# running config, the MAC cache and the guest configs. It names every
+# guest NIC in the cluster with the vnet it sits on, whether the responder
+# runs on it and what it answers there. The responder keeps those present
+# on this node. The configs trail the kernel by a moment around a hotplug.
+# An interface they do not name yet, or still put on another vnet than its
+# plug did, is left alone by the responder while its link lives.
+
+my sub full_state {
+ my ($report) = @_;
+
+ # the configs and the cache are read after the cluster state is refreshed,
+ # a copy from before the generation was drawn would hand the pass older
+ # input
+ PVE::Cluster::cfs_update();
+ my $cfg = PVE::Network::SDN::running_config();
+ my $zones = $cfg->{zones}->{ids} // {};
+ my $vnets = $cfg->{vnets}->{ids} // {};
+ # an ebpf zone confined to other nodes runs no responder here
+ my $nodename = PVE::INotify::nodename();
+ my %ebpf = map { $_ => 1 } grep {
+ ($zones->{$_}->{dhcp} // '') eq 'ebpf'
+ && (!defined($zones->{$_}->{nodes}) || $zones->{$_}->{nodes}->{$nodename})
+ } keys %$zones;
+ return (0, []) if !%ebpf;
+
+ my $macs = cached_macs(PVE::Network::SDN::Ipams::read_macdb());
+ my (%mtu, %subnets);
+ # a subnet or a zone MTU the responder cannot serve is reported per apply,
+ # guests or not
+ if ($report) {
+ for my $vnetid (sort keys %$vnets) {
+ my $zoneid = $vnets->{$vnetid}->{zone};
+ next if !$zoneid || !$ebpf{$zoneid};
+ $subnets{$vnetid} //= served_subnets($vnetid, 1);
+ $mtu{$zoneid} = zone_mtu($zoneid, $zones->{$zoneid}, 1) if !exists $mtu{$zoneid};
+ }
+ }
+ my $ifaces = [];
+ for my $nic (PVE::Network::SDN::Dhcp::guest_nics()->@*) {
+ my $vnetid = $nic->{bridge};
+ my $zoneid = $vnets->{$vnetid} ? $vnets->{$vnetid}->{zone} : undef;
+ my $serve = $zoneid && $ebpf{$zoneid} ? 1 : 0;
+ my $record;
+ if ($serve) {
+ $mtu{$zoneid} = zone_mtu($zoneid, $zones->{$zoneid}, $report) if !exists $mtu{$zoneid};
+ $subnets{$vnetid} //= served_subnets($vnetid, $report);
+ $record = vnet_record($subnets{$vnetid}, $mtu{$zoneid}, $macs, $nic->{mac})
+ if $nic->{mac};
+ }
+ push @$ifaces,
+ {
+ name => $nic->{iface},
+ vnet => $vnets->{$vnetid} ? $vnetid : undef,
+ serve => $serve,
+ record => $record,
+ };
+ }
+
+ return (scalar(keys %ebpf), $ifaces);
+}
+
+# The full pass converges the responder from any starting point. It diffs
+# programs, links and records against its pinned state and leaves alone
+# what changed since it read its input. Once no zone uses the
+# backend anymore the state is torn down instead. A pass overtaken by a
+# newer one has nothing left to do, that one carried everything it would
+# have.
+my sub full_pass {
+ my ($report) = @_;
+
+ my $generation = eval { generation() };
+ if ($@) {
+ log_warn("could not draw a DHCP responder generation: $@");
+ return 0;
+ }
+
+ my ($zones, $ifaces) = eval { full_state($report) };
+ if ($@) {
+ log_warn("could not collect the DHCP responder state: $@");
+ return 0;
+ }
+
+ my $applied = eval {
+ $zones
+ ? PVE::RS::SDN::Dhcp::apply($generation, $ifaces)
+ : PVE::RS::SDN::Dhcp::clear($generation);
+ };
+ if ($@) {
+ log_warn("could not apply the DHCP responder state: $@");
+ return 0;
+ }
+
+ return $applied;
+}
+
+# the interfaces a MAC has on ebpf vnets with the answer the MAC gets
+# there, wherever the configs put them. The responder skips what is not
+# here
+
+my sub mac_ifaces {
+ my ($mac, $macs) = @_;
+
+ my $cfg = PVE::Network::SDN::running_config();
+ my $zones = $cfg->{zones}->{ids} // {};
+ my $vnets = $cfg->{vnets}->{ids} // {};
+ my $ifaces = [];
+ for my $nic (PVE::Network::SDN::Dhcp::guest_nics()->@*) {
+ next if lc($nic->{mac} // '') ne lc($mac);
+ my $vnet = $vnets->{ $nic->{bridge} } // next;
+ my $zone = $zones->{ $vnet->{zone} } // next;
+ next if ($zone->{dhcp} // '') ne 'ebpf';
+ my $record = vnet_record(
+ served_subnets($nic->{bridge}, 0),
+ zone_mtu($vnet->{zone}, $zone, 0),
+ $macs,
+ $mac,
+ );
+ push @$ifaces,
+ { name => $nic->{iface}, vnet => $nic->{bridge}, serve => 1, record => $record };
+ }
+
+ return $ifaces;
+}
+
+# One MAC's records changed, the answer of each of its interfaces here is
+# rewritten or dropped. The generation is drawn first and the cache read
+# after it. The full pass runs only when the responder has nothing loaded to
+# write into.
+sub update_ip_mapping {
+ my ($class, $dhcpid, $mac) = @_;
+
+ my $generation = eval { generation() };
+ if ($@) {
+ log_warn("could not draw a DHCP responder generation: $@");
+ return;
+ }
+ PVE::Cluster::cfs_update();
+ my $ifaces = mac_ifaces($mac, cached_macs(PVE::Network::SDN::Ipams::read_macdb()));
+ return if !@$ifaces;
+
+ my $done = eval { PVE::RS::SDN::Dhcp::update($generation, $ifaces) };
+ if ($@) {
+ log_warn("could not update the DHCP responder records of $mac: $@");
+ return;
+ }
+ full_pass(0) if !$done;
+}
+
+# the dispatcher's per-zone walk is not needed, the full pass collects
+# the state itself
+sub before_regenerate { }
+sub before_configure { }
+sub configure_subnet { }
+sub configure_range { }
+sub configure_vnet { }
+sub after_configure { }
+
+sub after_regenerate {
+ my ($class) = @_;
+
+ full_pass(1);
+}
+
+# the pinned state is gone after a reboot. The full pass loads the programs
+# before the first guest plugs, so no guest start has to
+sub boot {
+ my ($class) = @_;
+
+ full_pass(1);
+}
+
+# drops the link and the record of an unplugged guest interface, or turns
+# the record into a marker naming where it went. A crash that skipped this
+# leaves them to the next full pass
+
+sub tap_unplug {
+ my ($class, $iface, $target) = @_;
+
+ eval { PVE::RS::SDN::Dhcp::detach($iface, $target) };
+ log_warn("could not detach the DHCP responder from $iface: $@") if $@;
+}
+
+# attaches the responder program to the plugged guest interface with the
+# answer its MAC gets on the vnet. Best effort, a guest start must not fail
+# on it. Nothing loaded yet means the first plug after boot, the full pass
+# loads and the attach is retried.
+sub tap_plug {
+ my ($class, $dhcpid, $vnetid, $iface, $mac) = @_;
+
+ # the generation is drawn before the cache is read, so the record is at
+ # least as new as every change drawn before it
+ my $attach = sub {
+ my $generation = generation();
+ my $record;
+ if ($mac) {
+ PVE::Cluster::cfs_update();
+ my $zone = PVE::Network::SDN::Zones::get_zone($dhcpid, 1);
+ $record = vnet_record(
+ served_subnets($vnetid, 0),
+ zone_mtu($dhcpid, $zone, 0),
+ cached_macs(PVE::Network::SDN::Ipams::read_macdb()),
+ $mac,
+ );
+ }
+ return PVE::RS::SDN::Dhcp::attach($generation, $iface, $vnetid, $record);
+ };
+
+ my $done = eval { $attach->() };
+ if ($@) {
+ log_warn("could not attach DHCP responder to $iface: $@");
+ return;
+ }
+ return if $done;
+
+ # nothing is loaded, a rebuild or a boot the SDN commit did not load on.
+ # The pass loads. The retry draws afresh, its input is read after the pass
+
+ full_pass(0);
+ $done = eval { $attach->() };
+ log_warn("could not attach DHCP responder to $iface: " . ($@ || "nothing is loaded"))
+ if $@ || !$done;
+}
+
+1;
diff --git a/src/PVE/Network/SDN/Dhcp/Makefile b/src/PVE/Network/SDN/Dhcp/Makefile
index 6546513..ce86aae 100644
--- a/src/PVE/Network/SDN/Dhcp/Makefile
+++ b/src/PVE/Network/SDN/Dhcp/Makefile
@@ -1,4 +1,4 @@
-SOURCES=Plugin.pm Dnsmasq.pm
+SOURCES=Plugin.pm Dnsmasq.pm Ebpf.pm
PERL5DIR=${DESTDIR}/usr/share/perl5
diff --git a/src/PVE/Network/SDN/Dhcp/Plugin.pm b/src/PVE/Network/SDN/Dhcp/Plugin.pm
index 3b95a68..1653c29 100644
--- a/src/PVE/Network/SDN/Dhcp/Plugin.pm
+++ b/src/PVE/Network/SDN/Dhcp/Plugin.pm
@@ -59,9 +59,28 @@ sub before_regenerate {
die 'implement in sub class';
}
+# the node booted, a backend whose state does not survive a reboot brings
+# it back here, before the guests start
+sub boot {
+ my ($class) = @_;
+}
+
sub after_regenerate {
my ($class) = @_;
die 'implement in sub class';
}
+# a guest interface was plugged into a vnet of a zone using this backend,
+# nothing to do for backends serving the bridge rather than the interface
+sub tap_plug {
+ my ($class, $dhcpid, $vnetid, $iface, $mac) = @_;
+}
+
+# a guest interface left its vnet or went away, its zone is not known
+# anymore by then. A target names the bridge it sits on now, the vnet that
+# is if any. None means the interface is gone or goes away
+sub tap_unplug {
+ my ($class, $iface, $target) = @_;
+}
+
1;
diff --git a/src/PVE/Network/SDN/SubnetPlugin.pm b/src/PVE/Network/SDN/SubnetPlugin.pm
index 29edf7b..38579f2 100644
--- a/src/PVE/Network/SDN/SubnetPlugin.pm
+++ b/src/PVE/Network/SDN/SubnetPlugin.pm
@@ -183,8 +183,8 @@ sub properties {
maximum => 4294967295,
description =>
'Lease time in seconds for DHCP answers. Without it dnsmasq hands out'
- . ' infinite leases, and it raises anything below two minutes to two'
- . ' minutes.',
+ . ' infinite leases and the ebpf responder ten minutes. dnsmasq raises'
+ . ' anything below two minutes to two minutes.',
optional => 1,
},
};
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 2429adf..2af4880 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -47,6 +47,9 @@ sub clear_test_state {
macdb => {},
ipamdb => {},
dnsmasq_calls => [],
+ ebpf_calls => [],
+ order => [],
+ warnings => [],
ipam_config => {
'ids' => {
'pve' => {
@@ -54,6 +57,8 @@ sub clear_test_state {
},
},
},
+ ebpf_answer => undef,
+ generation => 0,
};
PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "\n");
}
@@ -234,6 +239,33 @@ $mocked_sdn_dhcp_dnsmasq->mock(
update_lease => sub { },
);
+my $mocked_pve_rs_dhcp = Test::MockModule->new('PVE::RS::SDN::Dhcp');
+$mocked_pve_rs_dhcp->mock(
+ next_generation => sub {
+ push $test_state->{order}->@*, 'draw';
+ return ++$test_state->{generation};
+ },
+ map {
+ my $method = $_;
+ $method => sub {
+ # the generation is a counter, not part of the expected state, it
+ # leads every ordered call
+ my @args = @_;
+ if ($method ne 'detach') {
+ shift @args;
+ push $test_state->{order}->@*, 'write';
+ }
+ push $test_state->{ebpf_calls}->@*, { method => $method, args => [@args] };
+ return $test_state->{ebpf_answer}->{$method} // 1;
+ };
+ } qw(apply attach clear update detach),
+);
+
+my $mocked_sdn_dhcp_ebpf = Test::MockModule->new('PVE::Network::SDN::Dhcp::Ebpf');
+$mocked_sdn_dhcp_ebpf->mock(
+ log_warn => sub { push $test_state->{warnings}->@*, $_[0]; },
+);
+
my $mocked_api_zones = Test::MockModule->new('PVE::API2::Network::SDN::Zones');
$mocked_api_zones->mock(
create_etc_interfaces_sdn_dir => sub { },
@@ -258,6 +290,9 @@ my $mocked_pve_cluster_obj = Test::MockModule->new('PVE::Cluster');
$mocked_pve_cluster_obj->mock(
check_cfs_quorum => sub { return 1; },
cfs_lock_domain => $mocked_cfs_lock_domain,
+ cfs_update => sub { push $test_state->{order}->@*, 'refresh'; },
+ get_vmlist => sub { return $test_state->{vmlist} // { ids => {} }; },
+ get_guest_config_properties => sub { return $test_state->{guest_nets} // {}; },
);
# ------- TEST FUNCTIONS --------------
@@ -365,6 +400,24 @@ sub delete_ip {
return PVE::API2::Network::SDN::Ips->ipdelete($param);
}
+sub take_ebpf_calls {
+ my $calls = $test_state->{ebpf_calls};
+ $test_state->{ebpf_calls} = [];
+ return $calls;
+}
+
+# the draws, cluster refreshes and responder writes in the order they ran,
+# a repeated step collapsed into one
+sub take_order {
+ my $order = $test_state->{order};
+ $test_state->{order} = [];
+ my @shape;
+ for my $step (@$order) {
+ push @shape, $step if !@shape || $shape[-1] ne $step;
+ }
+ return \@shape;
+}
+
sub run_test {
my $test = shift;
clear_test_state();
@@ -1402,4 +1455,547 @@ sub test_dnsmasq_dual_stack_and_sweep {
run_test(\&test_dnsmasq_dual_stack_and_sweep);
+# -------------- ebpf dhcp backend
+my $ebpf_record = sub {
+ my ($ip, %over) = @_;
+ return {
+ ip => $ip,
+ prefixlen => 24,
+ server_id => '10.0.0.1',
+ lease => 600,
+ router => '10.0.0.1',
+ dns => undef,
+ mtu => 1500,
+ mac => 'da:65:8f:18:9b:6f',
+ %over,
+ };
+};
+
+sub test_ebpf_backend {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+
+ # the guest behind the MAC runs here, its interface gets the record
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ take_ebpf_calls();
+ eval { nic_start($vnetid, $mac, "999", "testhostname"); };
+ if ($@) {
+ fail("$test_name: nic_start: $@");
+ return;
+ }
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.100"),
+ }]],
+ }],
+ "$test_name: a guest start writes the record of its interface",
+ );
+
+ # a mapping edit rewrites it, a delete leaves the interface without one
+ update_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.150"),
+ }]],
+ }],
+ "$test_name: a mapping edit rewrites the record",
+ );
+ delete_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{ name => 'tap999i0', vnet => $vnetid, serve => 1, record => undef }]],
+ }],
+ "$test_name: a mapping delete drops the record",
+ );
+
+ # a guest running elsewhere has no interface here to write to
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'other' } } };
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.150"),
+ }]],
+ }],
+ "$test_name: a guest elsewhere is handed over too, the responder skips what is not here",
+ );
+
+ # the full pass names every guest NIC in the cluster, whether the
+ # responder runs on it and what it answers. The responder keeps those
+ # present here. A guest the vmlist does not know is skipped, a NIC
+ # without a MAC is served nothing
+ $test_state->{vmlist} = {
+ ids => {
+ 100 => { type => 'qemu', node => 'other' },
+ 101 => { type => 'lxc', node => 'other' },
+ 103 => { type => 'qemu', node => 'other' },
+ },
+ };
+ $test_state->{guest_nets} = {
+ 100 => {
+ net0 => "virtio=$mac,bridge=$vnetid,firewall=1",
+ net1 => "virtio=00:11:22:33:44:55,bridge=vmbr0",
+ },
+ 101 => { net2 => "name=eth0,bridge=$vnetid,hwaddr=00:11:22:33:44:66" },
+ 102 => { net0 => "virtio=00:11:22:33:44:77,bridge=$vnetid" },
+ 103 => { net0 => "virtio,bridge=$vnetid" },
+ };
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'apply',
+ args => [[
+ {
+ name => 'tap100i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.150"),
+ },
+ { name => 'tap100i1', vnet => undef, serve => 0, record => undef },
+ { name => 'veth101i2', vnet => $vnetid, serve => 1, record => undef },
+ { name => 'tap103i0', vnet => $vnetid, serve => 1, record => undef },
+ ]],
+ }],
+ "$test_name: the full pass names every guest NIC with its answer",
+ );
+
+ # a responder with nothing loaded cannot take a single record, the
+ # full pass loads it and fills everything
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ $test_state->{ebpf_answer} = { update => 0 };
+ nic_start($vnetid, $mac, "999", "testhostname");
+ $test_state->{ebpf_answer} = undef;
+ eq_or_diff(
+ [map { $_->{method} } take_ebpf_calls()->@*],
+ ['update', 'apply'],
+ "$test_name: an unloaded responder gets the full pass",
+ );
+
+ # a plug attaches the interface with the answer its MAC gets on the
+ # vnet, with nothing loaded the full pass runs first and the attach is
+ # retried
+ PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0', $mac);
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'attach', args => ['tap999i0', $vnetid, $ebpf_record->("10.0.0.150")] }],
+ "$test_name: a plug attaches its interface with the record",
+ );
+ $test_state->{ebpf_answer} = { attach => 0 };
+ PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0', $mac);
+ $test_state->{ebpf_answer} = undef;
+ eq_or_diff(
+ [map { $_->{method} } take_ebpf_calls()->@*],
+ ['attach', 'apply', 'attach'],
+ "$test_name: a plug with nothing loaded runs the full pass first",
+ );
+ PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0');
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'attach', args => ['tap999i0', $vnetid, undef] }],
+ "$test_name: a plug without a MAC attaches with nothing to answer",
+ );
+}
+
+sub test_ebpf_backend_edge_cases {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a regenerate without any ebpf zone tears the responder state down
+ create_zone({
+ type => "simple",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ take_ebpf_calls();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'clear', args => [] }],
+ "$test_name: regenerate without an ebpf zone clears the responder",
+ );
+
+ # an ebpf zone confined to other nodes runs no responder here either
+ update_zone($zoneid, { dhcp => "ebpf", nodes => 'other' });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'clear', args => [] }],
+ "$test_name: an ebpf zone confined to other nodes clears the responder here",
+ );
+ update_zone($zoneid, { delete => 'nodes' });
+
+ # the SDN commit brings the responder back at boot, one full pass
+ PVE::Network::SDN::Dhcp::boot();
+ eq_or_diff(
+ [map { $_->{method} } take_ebpf_calls()->@*],
+ ['apply'],
+ "$test_name: the boot runs the full pass",
+ );
+
+ # the zone switches to ebpf, its subnets lack what a record needs
+ update_zone($zoneid, { dhcp => "ebpf" });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "fd00::/64",
+ gateway => "fd00::1",
+ 'dhcp-range' => ["start-address=fd00::100,end-address=fd00::200"],
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+
+ take_ebpf_calls();
+ eval { nic_start($vnetid, $mac, "999", "testhostname"); };
+ if ($@) {
+ fail("$test_name: nic_start: $@");
+ return;
+ }
+
+ # a gateway-less v4 subnet is served under a link-local identifier and
+ # without a router, the v6 subnet yields nothing
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->(
+ "10.0.0.100",
+ server_id => '169.254.0.1',
+ router => undef,
+ ),
+ }]],
+ }],
+ "$test_name: a subnet without a gateway is served without a router",
+ );
+ $test_state->{warnings} = [];
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ take_ebpf_calls();
+ is(scalar($test_state->{warnings}->@*), 0, "$test_name: and the full pass has nothing to say");
+
+ # a v6 resolver on a v4 subnet is dropped from the record instead of
+ # failing the whole set
+ update_subnet({
+ vnet => $vnetid,
+ subnet => "$zoneid-10.0.0.0-24",
+ gateway => "10.0.0.1",
+ 'dhcp-dns-server' => "fd00::53",
+ });
+ $test_state->{warnings} = [];
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ my $calls = take_ebpf_calls();
+ is(scalar(@$calls), 1, "$test_name: regenerate applies once");
+ my $iface = $calls->[0]->{args}->[0]->[0];
+ is($iface->{name}, 'tap999i0', "$test_name: the guest interface is listed");
+ is($iface->{record}->{ip}, '10.0.0.100', "$test_name: the v4 subnet now yields the record");
+ is($iface->{record}->{dns}, undef, "$test_name: the IPv6 resolver is not handed out");
+ is(
+ scalar(grep { m/IPv6 DNS server/ } $test_state->{warnings}->@*),
+ 1,
+ "$test_name: and the full pass says so",
+ );
+}
+
+run_test(\&test_ebpf_backend);
+run_test(\&test_ebpf_backend_edge_cases);
+
+sub test_ebpf_mac_case {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a mapping typed in lower case for a NIC whose config spells the MAC in
+ # upper case is the same mapping
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=" . uc($mac) . ",bridge=$vnetid" } };
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.100",
+ });
+ take_ebpf_calls();
+ take_order();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'apply',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.100"),
+ }]],
+ }],
+ "$test_name: the full pass finds the mapping however the config spells the MAC",
+ );
+ # the generation is drawn before the state is refreshed and read, so
+ # nothing read is older than the generation it is written under
+ is_deeply(
+ take_order(),
+ ['draw', 'refresh', 'write'],
+ "$test_name: the pass draws, refreshes, then writes",
+ );
+ PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0', uc($mac));
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'attach', args => ['tap999i0', $vnetid, $ebpf_record->("10.0.0.100")] }],
+ "$test_name: and so does a plug",
+ );
+
+ # an entry of the old code split across spellings is read as one
+ $test_state->{macdb} = {
+ macs => { uc($mac) => { ip4 => '10.0.0.100' }, $mac => { ip6 => '8888::100' } },
+ };
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{ip},
+ '10.0.0.100',
+ "$test_name: a cache entry split across spellings is read as one",
+ );
+}
+
+run_test(\&test_ebpf_mac_case);
+
+sub test_ebpf_equal_subnets {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+
+ # two ebpf zones carry the same subnet. Each interface is answered from
+ # the subnet of the vnet it sits on, whatever address the cache holds
+ # for its MAC
+ for my $net (['ZONEA', 'vneta', '10.0.0.1'], ['ZONEB', 'vnetb', '10.0.0.2']) {
+ my ($zoneid, $vnetid, $gateway) = @$net;
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => $gateway,
+ });
+ }
+ $test_state->{vmlist} = {
+ ids => {
+ 100 => { type => 'qemu', node => 'localnode' },
+ 101 => { type => 'qemu', node => 'localnode' },
+ },
+ };
+ $test_state->{guest_nets} = {
+ 100 => { net0 => "virtio=da:65:8f:18:9b:6f,bridge=vneta" },
+ 101 => { net0 => "virtio=da:65:8f:18:9b:70,bridge=vnetb" },
+ };
+ $test_state->{macdb} = {
+ macs => {
+ 'da:65:8f:18:9b:6f' => { ip4 => '10.0.0.150' },
+ 'da:65:8f:18:9b:70' => { ip4 => '10.0.0.150' },
+ },
+ };
+
+ take_ebpf_calls();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ my $ifaces = take_ebpf_calls()->[0]->{args}->[0];
+ eq_or_diff(
+ [map { [$_->{name}, $_->{record}->{router}] } @$ifaces],
+ [['tap100i0', '10.0.0.1'], ['tap101i0', '10.0.0.2']],
+ "$test_name: each interface is answered from its own vnet",
+ );
+}
+
+run_test(\&test_ebpf_equal_subnets);
+
+sub test_ebpf_zone_mtu {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a zone MTU the DHCP option cannot carry is reported on the apply and
+ # left out of the answer
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ mtu => 10,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ $test_state->{macdb} = { macs => { $mac => { ip4 => '10.0.0.100' } } };
+
+ take_ebpf_calls();
+ $test_state->{warnings} = [];
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ scalar(grep { m/MTU 10/ } $test_state->{warnings}->@*),
+ 1,
+ "$test_name: the zone MTU is reported once",
+ );
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{mtu},
+ undef,
+ "$test_name: and not handed out",
+ );
+}
+
+run_test(\&test_ebpf_zone_mtu);
+
+sub test_ebpf_overlapping_subnets {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a vnet with overlapping subnets answers from the most specific one
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/16",
+ gateway => "10.0.255.1",
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ $test_state->{macdb} = { macs => { $mac => { ip4 => '10.0.0.100' } } };
+
+ take_ebpf_calls();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ my $record = take_ebpf_calls()->[0]->{args}->[0]->[0]->{record};
+ is_deeply(
+ [$record->@{qw(prefixlen router)}],
+ [24, '10.0.0.1'],
+ "$test_name: the most specific subnet answers",
+ );
+}
+
+run_test(\&test_ebpf_overlapping_subnets);
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH pve-network v2 09/16] sdn: zones: attach the dhcp responder on tap plug, detach on unplug
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (7 preceding siblings ...)
2026-09-09 10:41 ` [PATCH pve-network v2 08/16] sdn: dhcp: add ebpf plugin Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH pve-network v2 10/16] sdn: dhcp: apply mapping edits on the node serving the guest Hannes Laimer
` (6 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
Program attach is per guest interface and the plug is the one moment
every such interface passes through on this node, covering guest
start, hotplug and incoming migration. The responder attaches before
the interface joins the bridge, so a guest already running cannot slip
its first request past it. The plug hands over the MAC of the guest
NIC, which the responder answers the interface for. Attach failures
only warn, a guest start must not depend on the responder. An unplug
reaches the backends the same way, with the interface alone, since its
zone is gone with it. A tap plugged onto a plain bridge is reported to
the backends as well, so a responder still attached to it lets go.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/Network/SDN/Zones.pm | 31 ++++++-
src/test/run_test_vnets_blackbox.pl | 129 ++++++++++++++++++++++++++++
2 files changed, 156 insertions(+), 4 deletions(-)
diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm
index f668303..cda8176 100644
--- a/src/PVE/Network/SDN/Zones.pm
+++ b/src/PVE/Network/SDN/Zones.pm
@@ -323,16 +323,18 @@ sub veth_create {
}
sub tap_plug {
- my ($iface, $bridge, $tag, $firewall, $trunks, $rate) = @_;
+ my ($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts) = @_;
my $vnet = PVE::Network::SDN::Vnets::get_vnet($bridge, 1);
if (!$vnet) { # fallback for classic bridge
my $interfaces_config = PVE::INotify::read_file('interfaces');
- my $opts = {};
- $opts->{learning} = 0
+ my $bridge_opts = {};
+ $bridge_opts->{learning} = 0
if $interfaces_config->{ifaces}->{$bridge}
&& $interfaces_config->{ifaces}->{$bridge}->{'bridge-disable-mac-learning'};
- PVE::Network::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts);
+ # the dhcp backends hear of an interface moving onto a plain bridge as well
+ PVE::Network::SDN::Dhcp::tap_plug($bridge, $iface, $opts->{mac});
+ PVE::Network::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate, $bridge_opts);
return;
}
@@ -343,9 +345,30 @@ sub tap_plug {
if $plugin_config->{nodes} && !defined($plugin_config->{nodes}->{$nodename});
my $plugin = PVE::Network::SDN::Zones::Plugin->lookup($plugin_config->{type});
+
+ # the responder attaches before the interface joins the bridge, so a running guest cannot
+ # slip its first request past it. Loaded through Vnets, a use here would close a
+ # load-order cycle
+ PVE::Network::SDN::Dhcp::tap_plug($bridge, $iface, $opts->{mac});
$plugin->tap_plug($plugin_config, $vnet, $tag, $iface, $bridge, $firewall, $trunks, $rate);
}
+# the interface is gone or leaving, the dhcp backends drop what they hold
+# for it before the bridge side is cleaned up
+sub tap_unplug {
+ my ($iface) = @_;
+
+ PVE::Network::SDN::Dhcp::tap_unplug($iface);
+ PVE::Network::tap_unplug($iface);
+}
+
+sub veth_delete {
+ my ($veth) = @_;
+
+ PVE::Network::SDN::Dhcp::tap_unplug($veth);
+ PVE::Network::veth_delete($veth);
+}
+
sub add_bridge_fdb {
my ($iface, $macaddr, $bridge) = @_;
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 2af4880..064a96c 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -1770,6 +1770,135 @@ sub test_ebpf_backend_edge_cases {
}
run_test(\&test_ebpf_backend);
+
+sub test_zone_tap_plug {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # the zone plug hands the MAC on to the dhcp backends, a plug onto a
+ # plain bridge tells them where the interface went
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.100",
+ });
+ my $bridged = [];
+ my $unplugged = [];
+ $mocked_sdn_zones_super_plugin->mock(
+ tap_plug => sub {
+ push @$bridged, $_[5];
+ push $test_state->{order}->@*, 'bridge';
+ },
+ );
+ my $mocked_pve_network = Test::MockModule->new('PVE::Network');
+ $mocked_pve_network->mock(
+ tap_plug => sub {
+ push @$bridged, $_[1];
+ push $test_state->{order}->@*, 'bridge';
+ },
+ tap_unplug => sub { push @$unplugged, $_[0]; },
+ veth_delete => sub { push @$unplugged, $_[0]; },
+ );
+ $mocked_pve_inotify->mock(read_file => sub { return { ifaces => {} }; });
+ take_ebpf_calls();
+ take_order();
+
+ PVE::Network::SDN::Zones::tap_plug('tap999i0', $vnetid, undef, 0, undef, undef,
+ { mac => $mac });
+ eq_or_diff($bridged, [$vnetid], "$test_name: the zone plugin plugs the interface");
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'attach', args => ['tap999i0', $vnetid, $ebpf_record->("10.0.0.100")] }],
+ "$test_name: and the dhcp backend attaches it with the MAC's record",
+ );
+ is_deeply(
+ take_order(),
+ ['draw', 'refresh', 'write', 'bridge'],
+ "$test_name: the responder attaches before the interface joins the bridge",
+ );
+
+ @$bridged = ();
+ PVE::Network::SDN::Zones::tap_plug('tap999i0', 'vmbr0', undef, 0, undef, undef,
+ { mac => $mac });
+ eq_or_diff($bridged, ['vmbr0'], "$test_name: a plain bridge is plugged the classic way");
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'detach', args => ['tap999i0', { vnet => undef }] }],
+ "$test_name: and the dhcp backends hear where the interface went",
+ );
+
+ # a vnet of another backend is a place the ebpf backend hears of as well
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => "DNSZONE",
+ });
+ create_vnet({
+ type => "vnet",
+ zone => "DNSZONE",
+ vnet => "dnsvnet",
+ });
+ @$bridged = ();
+ PVE::Network::SDN::Zones::tap_plug(
+ 'tap999i0',
+ 'dnsvnet',
+ undef,
+ 0,
+ undef,
+ undef,
+ { mac => $mac },
+ );
+ eq_or_diff(
+ $bridged,
+ ['dnsvnet'],
+ "$test_name: a vnet of another backend is plugged by its zone",
+ );
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'detach', args => ['tap999i0', { vnet => 'dnsvnet' }] }],
+ "$test_name: and the ebpf backend hears which vnet the interface went to",
+ );
+
+ # the zone unplug and the veth delete report the interface gone before
+ # the classic ones run
+ PVE::Network::SDN::Zones::tap_unplug('tap999i0');
+ PVE::Network::SDN::Zones::veth_delete('veth999i0');
+ eq_or_diff($unplugged, ['tap999i0', 'veth999i0'], "$test_name: the classic unplugs run");
+ eq_or_diff(
+ take_ebpf_calls(),
+ [
+ { method => 'detach', args => ['tap999i0', undef] },
+ { method => 'detach', args => ['veth999i0', undef] },
+ ],
+ "$test_name: and the dhcp backends hear the interfaces are gone",
+ );
+ $mocked_sdn_zones_super_plugin->unmock('tap_plug');
+ $mocked_pve_inotify->unmock('read_file');
+}
+
+run_test(\&test_zone_tap_plug);
run_test(\&test_ebpf_backend_edge_cases);
sub test_ebpf_mac_case {
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH pve-network v2 10/16] sdn: dhcp: apply mapping edits on the node serving the guest
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (8 preceding siblings ...)
2026-09-09 10:41 ` [PATCH pve-network v2 09/16] sdn: zones: attach the dhcp responder on tap plug, detach on unplug Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH pve-network v2 11/16] sdn: zones: offer dhcp on all zone types, keep dnsmasq simple-only Hannes Laimer
` (5 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
Mapping edits change cluster-wide records, but the ebpf backend answers
from a per-node map. So an edit made through another node's API left the
map of the node running the guest stale until the next apply or guest
start there. Resolve the MAC to its guest and re-apply the mapping on
that node through a node-scoped call, the same record update the editing
node runs locally. No other node needs the record, each picks it up at
that guest's next start there. So the cost of an edit stays one call
however large the cluster is. dnsmasq is covered by the same reasoning,
a simple zone's instance is node-local and only the guest's node serves
it.
The push runs detached from the edit request, so an unreachable node
cannot hold the edit up. It just catches up on its next apply or the
guest's next start.
The poke is bounded by a timeout, a node that takes the connection but
never answers must not keep it around.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/Network/SDN/Ips.pm | 3 +
src/PVE/API2/Network/SDN/Nodes/Status.pm | 37 ++++++++++
src/PVE/Network/SDN/Dhcp.pm | 77 ++++++++++++++++++++
src/test/run_test_vnets_blackbox.pl | 90 ++++++++++++++++++++++++
4 files changed, 207 insertions(+)
diff --git a/src/PVE/API2/Network/SDN/Ips.pm b/src/PVE/API2/Network/SDN/Ips.pm
index d7b682d..5b45de7 100644
--- a/src/PVE/API2/Network/SDN/Ips.pm
+++ b/src/PVE/API2/Network/SDN/Ips.pm
@@ -47,6 +47,7 @@ __PACKAGE__->register_method({
die "$@\n" if $@;
PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+ PVE::Network::SDN::Dhcp::notify_guest_node($vnet, $mac);
return undef;
},
@@ -85,6 +86,7 @@ __PACKAGE__->register_method({
PVE::Network::SDN::Vnets::add_ip($vnet, $ip, '', $mac, undef);
PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+ PVE::Network::SDN::Dhcp::notify_guest_node($vnet, $mac);
return undef;
},
@@ -138,6 +140,7 @@ __PACKAGE__->register_method({
die "$error\n" if $error;
PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+ PVE::Network::SDN::Dhcp::notify_guest_node($vnet, $mac);
return undef;
},
diff --git a/src/PVE/API2/Network/SDN/Nodes/Status.pm b/src/PVE/API2/Network/SDN/Nodes/Status.pm
index 7977e0c..fe750a8 100644
--- a/src/PVE/API2/Network/SDN/Nodes/Status.pm
+++ b/src/PVE/API2/Network/SDN/Nodes/Status.pm
@@ -9,6 +9,9 @@ use PVE::API2::Network::SDN::Nodes::Vnets;
use PVE::JSONSchema qw(get_standard_option);
+use PVE::Network::SDN::Vnets;
+use PVE::Network::SDN::Dhcp;
+
use PVE::RESTHandler;
use base qw(PVE::RESTHandler);
@@ -27,6 +30,40 @@ __PACKAGE__->register_method({
path => 'vnets',
});
+__PACKAGE__->register_method({
+ name => 'dhcp_mapping',
+ path => 'dhcp-mapping',
+ method => 'POST',
+ description =>
+ 'Re-apply the DHCP mapping of a MAC address on this node from the current records.',
+ permissions => {
+ check => ['perm', '/sdn/zones/{zone}/{vnet}', ['SDN.Allocate']],
+ },
+ protected => 1,
+ proxyto => 'node',
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ zone => get_standard_option('pve-sdn-zone-id'),
+ vnet => get_standard_option('pve-sdn-vnet-id'),
+ mac => get_standard_option('mac-addr'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $vnet = PVE::Network::SDN::Vnets::get_vnet($param->{vnet}, 1);
+ die "vnet '$param->{vnet}' does not exist in zone '$param->{zone}'\n"
+ if !$vnet || $vnet->{zone} ne $param->{zone};
+
+ PVE::Network::SDN::Dhcp::update_mapping($param->{vnet}, $param->{mac});
+
+ return undef;
+ },
+});
+
__PACKAGE__->register_method({
name => 'sdnindex',
path => '',
diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
index 3b2d798..e57468d 100644
--- a/src/PVE/Network/SDN/Dhcp.pm
+++ b/src/PVE/Network/SDN/Dhcp.pm
@@ -3,6 +3,8 @@ package PVE::Network::SDN::Dhcp;
use strict;
use warnings;
+use POSIX qw();
+
use PVE::Cluster;
use PVE::Network::SDN;
@@ -82,6 +84,81 @@ sub guest_nics {
return $nics;
}
+sub guest_node_by_mac {
+ my ($vnetid, $mac) = @_;
+
+ for my $nic (guest_nics()->@*) {
+ return $nic->{node}
+ if lc($nic->{mac} // '') eq lc($mac) && $nic->{bridge} eq $vnetid;
+ }
+
+ return undef;
+}
+
+# The records are cluster-wide, but a node answers from its own map.
+# So an edit has to reach the node running the guest behind the MAC.
+# Every other node picks the record up at that guest's next start
+# there. The push runs detached from the request, an unreachable node
+# must not hold the edit up.
+sub notify_guest_node {
+ my ($vnetid, $mac) = @_;
+
+ # best effort like the push itself, the record is written by now
+ my ($zoneid, $node) = eval {
+ my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid, 1);
+ return if !$vnet;
+
+ my $zone = PVE::Network::SDN::Zones::get_zone($vnet->{zone}, 1);
+ return if !$zone || !$zone->{ipam} || !$zone->{dhcp};
+
+ return ($vnet->{zone}, guest_node_by_mac($vnetid, $mac));
+ };
+ if ($@) {
+ warn "could not find the node to push the dhcp mapping of $mac to: $@";
+ return;
+ }
+ return if !$node || $node eq PVE::INotify::nodename();
+
+ # double fork, the grandchild is reparented to init so nothing on
+ # the request path ever waits on it
+ my $pid = fork();
+ if (!defined($pid)) {
+ warn "could not fork for the dhcp mapping push to $node: $!\n";
+ return;
+ }
+ if ($pid) {
+ waitpid($pid, 0);
+ return;
+ }
+
+ POSIX::setsid();
+ my $child = fork();
+ POSIX::_exit(1) if !defined($child);
+ POSIX::_exit(0) if $child;
+
+ # nothing of the request stays with the push, and since the daemon's
+ # stderr is closed a failed push reports through the journal
+ open(STDIN, '<', '/dev/null') or POSIX::_exit(1);
+ open(STDOUT, '>', '/dev/null') or POSIX::_exit(1);
+ open(STDERR, '|-', 'logger', '-t', 'pve-sdn', '-p', 'daemon.warning')
+ or POSIX::_exit(1);
+ # a node that takes the connection but never answers must not keep the
+ # push around
+ exec(
+ 'timeout',
+ '60',
+ 'pvesh',
+ 'create',
+ "/nodes/$node/sdn/dhcp-mapping",
+ '--zone',
+ $zoneid,
+ '--vnet',
+ $vnetid,
+ '--mac',
+ $mac,
+ ) or POSIX::_exit(1);
+}
+
# the interface may come from a vnet of another backend, or of none, so
# every other backend drops what it holds for it first
sub tap_plug {
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 064a96c..2c11dad 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -49,6 +49,7 @@ sub clear_test_state {
dnsmasq_calls => [],
ebpf_calls => [],
order => [],
+ notify_calls => [],
warnings => [],
ipam_config => {
'ids' => {
@@ -239,6 +240,11 @@ $mocked_sdn_dhcp_dnsmasq->mock(
update_lease => sub { },
);
+my $mocked_sdn_dhcp = Test::MockModule->new('PVE::Network::SDN::Dhcp');
+$mocked_sdn_dhcp->mock(
+ notify_guest_node => sub { push $test_state->{notify_calls}->@*, [@_]; },
+);
+
my $mocked_pve_rs_dhcp = Test::MockModule->new('PVE::RS::SDN::Dhcp');
$mocked_pve_rs_dhcp->mock(
next_generation => sub {
@@ -1230,6 +1236,90 @@ sub test_dhcp_backend_needed_per_node {
run_test(\&test_dhcp_backend_needed_per_node);
+sub test_mapping_push_target {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ });
+
+ # the guest behind a MAC is found by its NIC on the vnet, the config
+ # writes the MAC in upper case
+ $test_state->{vmlist} = {
+ ids => {
+ 100 => { type => 'qemu', node => 'other' },
+ 101 => { type => 'lxc', node => 'third' },
+ },
+ };
+ $test_state->{guest_nets} = {
+ 1 => { net0 => "virtio=$mac,bridge=$vnetid" },
+ 100 => { net0 => "virtio=" . uc($mac) . ",bridge=$vnetid" },
+ 101 => { net0 => "name=eth0,bridge=vmbr0,hwaddr=00:11:22:33:44:55" },
+ };
+ is(
+ PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, $mac),
+ 'other',
+ "$test_name: the MAC resolves to the node of its guest, a guest the list lacks is skipped",
+ );
+ is(
+ PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, '00:11:22:33:44:55'),
+ undef,
+ "$test_name: a MAC on another bridge does not resolve",
+ );
+ is(
+ PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, '00:11:22:33:44:66'),
+ undef,
+ "$test_name: an unknown MAC does not resolve",
+ );
+
+ # every mapping edit through the API pokes the guest's node once
+ $test_state->{notify_calls} = [];
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.50",
+ });
+ update_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.51",
+ });
+ delete_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.51",
+ });
+ eq_or_diff(
+ $test_state->{notify_calls},
+ [[$vnetid, $mac], [$vnetid, $mac], [$vnetid, $mac]],
+ "$test_name: create, edit and delete each poke the guest's node",
+ );
+ delete $test_state->{vmlist};
+ delete $test_state->{guest_nets};
+}
+
+run_test(\&test_mapping_push_target);
+
sub test_ipam_cache_misses {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH pve-network v2 11/16] sdn: zones: offer dhcp on all zone types, keep dnsmasq simple-only
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (9 preceding siblings ...)
2026-09-09 10:41 ` [PATCH pve-network v2 10/16] sdn: dhcp: apply mapping edits on the node serving the guest Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH qemu-server v2 12/16] network: report NIC plug and unplug to SDN with the MAC Hannes Laimer
` (4 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
The dnsmasq backend is confined to simple zones because it is a
node-local process on the vnet bridge. On a zone spanning nodes every
node's instance would answer the shared broadcast domain. The ebpf
backend answers on the guest's own tap and has no such restriction, so
the dhcp property is offered on every zone type, with dnsmasq rejected
on the others.
The backends ask the zone for the guest-facing MTU to serve, so the zone
types gaining dhcp report theirs where they have one. Vxlan derived
zones take the underlay's MTU minus the encapsulation overhead where
their peers name a local interface, the derivation their bridge
generation means to apply.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/Network/SDN/Zones.pm | 8 +-
src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 32 +++++++
src/PVE/Network/SDN/Zones/FaucetPlugin.pm | 1 +
src/PVE/Network/SDN/Zones/QinQPlugin.pm | 7 ++
src/PVE/Network/SDN/Zones/SimplePlugin.pm | 2 +-
src/PVE/Network/SDN/Zones/VlanPlugin.pm | 7 ++
src/PVE/Network/SDN/Zones/VxlanPlugin.pm | 25 +++++
src/test/run_test_vnets_blackbox.pl | 109 +++++++++++++++++++++-
8 files changed, 187 insertions(+), 4 deletions(-)
diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm
index 0e90726..2b2aba0 100644
--- a/src/PVE/API2/Network/SDN/Zones.pm
+++ b/src/PVE/API2/Network/SDN/Zones.pm
@@ -93,7 +93,7 @@ my $ZONE_PROPERTIES = {
type => 'string',
enum => PVE::Network::SDN::Dhcp->plugin_types(),
optional => 1,
- description => 'Name of DHCP server backend for this zone.',
+ description => 'DHCP backend serving the zone, dnsmasq works on simple zones only.',
},
'rt-import' => {
type => 'string',
@@ -448,6 +448,9 @@ __PACKAGE__->register_method({
raise_param_exc({ ipam => "$ipam not existing" })
if $ipam && !$ipam_cfg->{ids}->{$ipam};
+ raise_param_exc({ dhcp => "the dnsmasq backend only supports simple zones" })
+ if ($opts->{dhcp} // '') eq 'dnsmasq' && $plugin->type() ne 'simple';
+
$zone_cfg->{ids}->{$id} = $opts;
$plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
@@ -544,6 +547,9 @@ __PACKAGE__->register_method({
raise_param_exc({ ipam => "$ipam not existing" })
if $ipam && !$ipam_cfg->{ids}->{$ipam};
+ raise_param_exc({ dhcp => "the dnsmasq backend only supports simple zones" })
+ if ($scfg->{dhcp} // '') eq 'dnsmasq' && $plugin->type() ne 'simple';
+
$plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
PVE::Network::SDN::Zones::write_config($zone_cfg);
diff --git a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
index 0e79707..60fe788 100644
--- a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
@@ -125,10 +125,42 @@ sub options {
reversedns => { optional => 1 },
dnszone => { optional => 1 },
ipam => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
# Plugin implementation
+# the underlay interface is the one the controller's peers name, found the
+# way the bridge generation finds it
+sub get_mtu {
+ my ($class, $plugin_config) = @_;
+
+ return $plugin_config->{mtu} if $plugin_config->{mtu};
+
+ my $iface;
+ my $controller_cfg = PVE::Network::SDN::running_config()->{controllers};
+ my $controller = $controller_cfg->{ids}->{ $plugin_config->{controller} // '' };
+ if ($controller && $controller->{peers}) {
+ my @peers = PVE::Tools::split_list($controller->{peers});
+ my $local_node = PVE::INotify::nodename();
+ my $bgprouter = PVE::Network::SDN::Controllers::EvpnPlugin::find_bgp_controller(
+ $local_node, $controller_cfg,
+ );
+ my $isisrouter = PVE::Network::SDN::Controllers::EvpnPlugin::find_isis_controller(
+ $local_node, $controller_cfg,
+ );
+ my $loopback = $bgprouter->{loopback} // $isisrouter->{loopback};
+ (undef, $iface) = eval {
+ PVE::Network::SDN::Zones::Plugin::find_local_ip_interface_peers(\@peers, $loopback);
+ };
+ }
+ return PVE::Network::SDN::Zones::VxlanPlugin::vxlan_mtu(
+ $plugin_config,
+ $iface,
+ PVE::INotify::read_file('interfaces'),
+ );
+}
+
sub generate_sdn_config {
my (
$class,
diff --git a/src/PVE/Network/SDN/Zones/FaucetPlugin.pm b/src/PVE/Network/SDN/Zones/FaucetPlugin.pm
index 5f069ae..4e3f5ef 100644
--- a/src/PVE/Network/SDN/Zones/FaucetPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/FaucetPlugin.pm
@@ -30,6 +30,7 @@ sub options {
reversedns => { optional => 1 },
dnszone => { optional => 1 },
ipam => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
diff --git a/src/PVE/Network/SDN/Zones/QinQPlugin.pm b/src/PVE/Network/SDN/Zones/QinQPlugin.pm
index a75940c..e70912a 100644
--- a/src/PVE/Network/SDN/Zones/QinQPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/QinQPlugin.pm
@@ -36,6 +36,12 @@ sub properties {
};
}
+sub get_mtu {
+ my ($class, $plugin_config) = @_;
+
+ return $plugin_config->{mtu};
+}
+
sub options {
return {
nodes => { optional => 1 },
@@ -48,6 +54,7 @@ sub options {
reversedns => { optional => 1 },
dnszone => { optional => 1 },
ipam => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
diff --git a/src/PVE/Network/SDN/Zones/SimplePlugin.pm b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
index 347eee9..e8656cc 100644
--- a/src/PVE/Network/SDN/Zones/SimplePlugin.pm
+++ b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
@@ -31,7 +31,7 @@ sub properties {
description => "dns domain zone ex: mydomain.com",
},
dhcp => {
- description => 'Type of the DHCP backend for this zone',
+ description => 'DHCP backend serving the zone, dnsmasq works on simple zones only.',
type => 'string',
enum => PVE::Network::SDN::Dhcp->plugin_types(),
},
diff --git a/src/PVE/Network/SDN/Zones/VlanPlugin.pm b/src/PVE/Network/SDN/Zones/VlanPlugin.pm
index 9102b34..4f24b2a 100644
--- a/src/PVE/Network/SDN/Zones/VlanPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/VlanPlugin.pm
@@ -36,6 +36,12 @@ sub properties {
};
}
+sub get_mtu {
+ my ($class, $plugin_config) = @_;
+
+ return $plugin_config->{mtu};
+}
+
sub options {
return {
@@ -47,6 +53,7 @@ sub options {
reversedns => { optional => 1 },
dnszone => { optional => 1 },
ipam => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
diff --git a/src/PVE/Network/SDN/Zones/VxlanPlugin.pm b/src/PVE/Network/SDN/Zones/VxlanPlugin.pm
index a408261..0dab5d9 100644
--- a/src/PVE/Network/SDN/Zones/VxlanPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/VxlanPlugin.pm
@@ -47,6 +47,30 @@ sub properties {
};
}
+# the vnet bridge takes the underlay interface's MTU minus the vxlan
+# encapsulation where the peers name a local interface, the default
+# otherwise. An explicit zone MTU wins
+sub vxlan_mtu {
+ my ($plugin_config, $iface, $interfaces_config) = @_;
+
+ return $plugin_config->{mtu} if $plugin_config->{mtu};
+ return $interfaces_config->{ifaces}->{$iface}->{mtu} - 50
+ if $iface && $interfaces_config->{ifaces}->{$iface}->{mtu};
+ return 1450;
+}
+
+sub get_mtu {
+ my ($class, $plugin_config) = @_;
+
+ my $iface;
+ if ($plugin_config->{peers}) {
+ my @peers = PVE::Tools::split_list($plugin_config->{peers});
+ (undef, $iface) =
+ eval { PVE::Network::SDN::Zones::Plugin::find_local_ip_interface_peers(\@peers) };
+ }
+ return vxlan_mtu($plugin_config, $iface, PVE::INotify::read_file('interfaces'));
+}
+
sub options {
return {
nodes => { optional => 1 },
@@ -58,6 +82,7 @@ sub options {
dnszone => { optional => 1 },
ipam => { optional => 1 },
fabric => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 2c11dad..557600a 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -1438,6 +1438,55 @@ sub test_ipam_cache_case {
run_test(\&test_ipam_cache_case);
+sub test_zone_dhcp_backends {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+
+ # dnsmasq answers on the vnet bridge, so it only fits zones confined to
+ # one node. The ebpf backend answers on the guest's tap and fits any
+ my $zone = {
+ type => "vlan",
+ bridge => "vmbr0",
+ ipam => "pve",
+ zone => $zoneid,
+ };
+ eval { PVE::API2::Network::SDN::Zones->create({ %$zone, dhcp => "dnsmasq" }); };
+ like(
+ $@,
+ qr/dnsmasq backend only supports simple zones/,
+ "$test_name: dnsmasq is refused on a vlan zone",
+ );
+
+ create_zone({ %$zone, dhcp => "ebpf" });
+ is(get_zone($zoneid)->{dhcp}, "ebpf", "$test_name: ebpf is accepted on a vlan zone");
+
+ eval { update_zone($zoneid, { dhcp => "dnsmasq" }); };
+ like(
+ $@,
+ qr/dnsmasq backend only supports simple zones/,
+ "$test_name: switching a vlan zone to dnsmasq is refused",
+ );
+
+ # every other zone type offers the backend as well
+ $test_state->{controller_config} = {
+ ids => { ctrl => { type => 'evpn', asn => 65000, peers => '10.0.0.1' } },
+ };
+ my $params = {
+ qinq => { bridge => 'vmbr0', tag => 100 },
+ vxlan => { peers => '10.0.0.1' },
+ evpn => { controller => 'ctrl', 'vrf-vxlan' => 1000 },
+ faucet => { 'dp-id' => 1, controller => 'ctrl' },
+ };
+ for my $type (sort keys %$params) {
+ my $id = uc($type) . "Z";
+ my $zone = { type => $type, ipam => "pve", zone => $id, %{ $params->{$type} } };
+ create_zone({ %$zone, dhcp => "ebpf" });
+ is(get_zone($id)->{dhcp}, "ebpf", "$test_name: ebpf is accepted on a $type zone");
+ }
+}
+
+run_test(\&test_zone_dhcp_backends);
+
sub test_dnsmasq_dual_stack_and_sweep {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
@@ -1744,6 +1793,8 @@ sub test_ebpf_backend {
);
}
+run_test(\&test_ebpf_backend);
+
sub test_ebpf_backend_edge_cases {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
@@ -1859,7 +1910,7 @@ sub test_ebpf_backend_edge_cases {
);
}
-run_test(\&test_ebpf_backend);
+run_test(\&test_ebpf_backend_edge_cases);
sub test_zone_tap_plug {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
@@ -1989,7 +2040,6 @@ sub test_zone_tap_plug {
}
run_test(\&test_zone_tap_plug);
-run_test(\&test_ebpf_backend_edge_cases);
sub test_ebpf_mac_case {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
@@ -2171,6 +2221,61 @@ sub test_ebpf_zone_mtu {
run_test(\&test_ebpf_zone_mtu);
+sub test_ebpf_vxlan_mtu {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "VXZONE";
+ my $vnetid = "vxvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a vxlan zone without an MTU of its own answers with the one its bridge
+ # takes from the underlay, the encapsulation overhead taken off
+ my $mocked_zone_plugin = Test::MockModule->new('PVE::Network::SDN::Zones::Plugin');
+ $mocked_zone_plugin->mock(
+ find_local_ip_interface_peers => sub { return ('10.10.10.1', 'eno1'); });
+ $mocked_pve_inotify->mock(
+ read_file => sub { return { ifaces => { eno1 => { mtu => 1400 } } }; });
+ create_zone({
+ type => "vxlan",
+ peers => "10.10.10.2",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ tag => 100,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ $test_state->{macdb} = { macs => { $mac => { ip4 => '10.0.0.100' } } };
+
+ take_ebpf_calls();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{mtu},
+ 1350,
+ "$test_name: the underlay's MTU minus the encapsulation is handed out",
+ );
+ update_zone($zoneid, { mtu => 1500 });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{mtu},
+ 1500,
+ "$test_name: an explicit zone MTU wins",
+ );
+ $mocked_pve_inotify->unmock('read_file');
+}
+
+run_test(\&test_ebpf_vxlan_mtu);
+
sub test_ebpf_overlapping_subnets {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH qemu-server v2 12/16] network: report NIC plug and unplug to SDN with the MAC
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (10 preceding siblings ...)
2026-09-09 10:41 ` [PATCH pve-network v2 11/16] sdn: zones: offer dhcp on all zone types, keep dnsmasq simple-only Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH pve-container v2 13/16] net: report veth plug and unplug to SDN with the hwaddr Hannes Laimer
` (3 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
The SDN layer answers DHCP on a plugged interface from the mapping of
its guest NIC, and drops what it holds for the interface once it goes
away. So the down script and the cleanup after a crash report through
it. A bridge change keeps pve-common's unplug, allocates on the new vnet
and pushes through the mapping call, whose push writes the dnsmasq
reservation. The plug that follows tells the SDN layer where the
interface went and carries the responder's record. This needs a
libpve-network-perl with the SDN unplug. qemu-server lists the package
as a versioned Recommends today while it cannot start a VM without it,
so the floor is a Depends on that version.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/CLI/qm.pm | 4 ++--
src/PVE/QemuServer.pm | 4 ++++
src/PVE/QemuServer/Network.pm | 4 ++--
src/usr/pve-bridge | 8 +++++++-
src/usr/pve-bridgedown | 4 ++--
5 files changed, 17 insertions(+), 7 deletions(-)
diff --git a/src/PVE/CLI/qm.pm b/src/PVE/CLI/qm.pm
index b903c1f1..6c53581a 100755
--- a/src/PVE/CLI/qm.pm
+++ b/src/PVE/CLI/qm.pm
@@ -22,7 +22,7 @@ use PVE::GuestHelpers;
use PVE::GuestImport::OVF;
use PVE::INotify;
use PVE::JSONSchema qw(get_standard_option);
-use PVE::Network;
+use PVE::Network::SDN::Zones;
use PVE::RPCEnvironment;
use PVE::SafeSyslog;
use PVE::Tools qw(extract_param file_get_contents);
@@ -1143,7 +1143,7 @@ __PACKAGE__->register_method({
foreach my $opt (keys %$conf) {
next if $opt !~ m/^net(\d+)$/;
my $interface = $1;
- PVE::Network::tap_unplug("tap${vmid}i${interface}");
+ PVE::Network::SDN::Zones::tap_unplug("tap${vmid}i${interface}");
}
}
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 149f17be..360069f1 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -5158,6 +5158,9 @@ sub vmconfig_update_net {
PVE::Network::SDN::Vnets::add_next_free_cidr(
$newnet->{bridge}, $conf->{name}, $newnet->{macaddr}, $vmid, undef, 1,
);
+ PVE::Network::SDN::Vnets::add_dhcp_mapping(
+ $newnet->{bridge}, $newnet->{macaddr}, $vmid, $conf->{name},
+ );
}
PVE::QemuServer::Network::tap_plug(
@@ -5167,6 +5170,7 @@ sub vmconfig_update_net {
$newnet->{firewall},
$newnet->{trunks},
$newnet->{rate},
+ { mac => $newnet->{macaddr} },
);
} elsif (safe_num_ne($oldnet->{rate}, $newnet->{rate})) {
diff --git a/src/PVE/QemuServer/Network.pm b/src/PVE/QemuServer/Network.pm
index b4893c01..cc76cfb3 100644
--- a/src/PVE/QemuServer/Network.pm
+++ b/src/PVE/QemuServer/Network.pm
@@ -349,10 +349,10 @@ sub delete_ifaces_ipams_ips {
}
sub tap_plug {
- my ($iface, $bridge, $tag, $firewall, $trunks, $rate) = @_;
+ my ($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts) = @_;
$firewall = $firewall && PVE::Firewall::Helpers::needs_fwbr($bridge);
- PVE::Network::SDN::Zones::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate);
+ PVE::Network::SDN::Zones::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts);
}
sub get_nets_host_mtu {
diff --git a/src/usr/pve-bridge b/src/usr/pve-bridge
index c7260c6b..20724271 100755
--- a/src/usr/pve-bridge
+++ b/src/usr/pve-bridge
@@ -43,7 +43,13 @@ die "unable to parse network config '$netid'\n" if !$net;
PVE::Network::SDN::Vnets::add_dhcp_mapping($net->{bridge}, $net->{macaddr}, $vmid, $conf->{name});
PVE::Network::SDN::Zones::tap_create($iface, $net->{bridge});
PVE::QemuServer::Network::tap_plug(
- $iface, $net->{bridge}, $net->{tag}, $net->{firewall}, $net->{trunks}, $net->{rate},
+ $iface,
+ $net->{bridge},
+ $net->{tag},
+ $net->{firewall},
+ $net->{trunks},
+ $net->{rate},
+ { mac => $net->{macaddr} },
);
exit 0;
diff --git a/src/usr/pve-bridgedown b/src/usr/pve-bridgedown
index e867b640..79a3722b 100755
--- a/src/usr/pve-bridgedown
+++ b/src/usr/pve-bridgedown
@@ -2,7 +2,7 @@
use strict;
use warnings;
-use PVE::Network;
+use PVE::Network::SDN::Zones;
my $iface = shift;
@@ -11,6 +11,6 @@ die "no interface specified\n" if !$iface;
die "got strange interface name '$iface'\n"
if $iface !~ m/^tap(\d+)i(\d+)$/;
-PVE::Network::tap_unplug($iface);
+PVE::Network::SDN::Zones::tap_unplug($iface);
exit 0;
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH pve-container v2 13/16] net: report veth plug and unplug to SDN with the hwaddr
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (11 preceding siblings ...)
2026-09-09 10:41 ` [PATCH qemu-server v2 12/16] network: report NIC plug and unplug to SDN with the MAC Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH pve-manager v2 14/16] ui: sdn: dhcp backend selector on all zones, expose dhcp options Hannes Laimer
` (2 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
The SDN layer answers DHCP on a plugged interface from the mapping of
its guest NIC, and drops what it holds for the interface once it goes
away. So the veth delete reports through it. A bridge change keeps
pve-common's unplug, allocates on the new vnet and pushes through the
mapping call, whose push writes the dnsmasq reservation. The plug that
follows tells the SDN layer where the veth went and carries the
responder's record. A disconnected NIC tells it at its next plug. This
needs a libpve-network-perl with the SDN unplug when the package is
installed at all. pve-container keeps working without it, so the floor
is a Breaks on the older versions.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/LXC.pm | 24 ++++++++++++++++++++++--
src/PVE/LXC/Config.pm | 2 +-
src/lxc-pve-poststop-hook | 5 ++---
3 files changed, 25 insertions(+), 6 deletions(-)
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index dee073d..62b76b4 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -1117,7 +1117,13 @@ sub net_tap_plug : prototype($$) {
if ($have_sdn) {
PVE::Network::SDN::Zones::tap_plug(
- $iface, $bridge, $tag, $create_firewall_bridges, $trunks, $rate,
+ $iface,
+ $bridge,
+ $tag,
+ $create_firewall_bridges,
+ $trunks,
+ $rate,
+ { mac => $hwaddr },
);
PVE::Network::SDN::Zones::add_bridge_fdb($iface, $hwaddr, $bridge);
} else {
@@ -1135,6 +1141,17 @@ sub net_tap_plug : prototype($$) {
PVE::Tools::run_command(['/sbin/ip', 'link', 'set', 'dev', $iface, 'up']);
}
+# the SDN layer drops what its DHCP backends hold for the veth, which may be gone already
+sub net_veth_delete : prototype($) {
+ my ($veth) = @_;
+
+ if ($have_sdn) {
+ PVE::Network::SDN::Zones::veth_delete($veth);
+ } else {
+ PVE::Network::veth_delete($veth);
+ }
+}
+
sub update_net {
my ($vmid, $conf, $opt, $newnet, $netid, $rootdir) = @_;
@@ -1154,7 +1171,7 @@ sub update_net {
|| safe_string_ne($oldnet->{name}, $newnet->{name})
) {
- PVE::Network::veth_delete($veth);
+ PVE::LXC::net_veth_delete($veth);
if ($have_sdn && safe_string_ne($oldnet->{hwaddr}, $newnet->{hwaddr})) {
eval {
@@ -1209,6 +1226,9 @@ sub update_net {
PVE::Network::SDN::Vnets::add_next_free_cidr(
$newnet->{bridge}, $conf->{hostname}, $newnet->{hwaddr}, $vmid, undef, 1,
);
+ PVE::Network::SDN::Vnets::add_dhcp_mapping(
+ $newnet->{bridge}, $newnet->{hwaddr}, $vmid, $conf->{hostname},
+ );
}
eval { PVE::LXC::net_tap_plug($veth, $newnet) };
if (my $err = $@) {
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 3eb1905..15ea6ea 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -1702,7 +1702,7 @@ sub vmconfig_hotplug_pending {
my $net = parse_lxc_network($conf->{$opt});
PVE::LXC::kill_dhclients($vmid, $net->{name}) if $net->{'host-managed'};
- PVE::Network::veth_delete("veth${vmid}i$netid");
+ PVE::LXC::net_veth_delete("veth${vmid}i$netid");
if ($have_sdn) {
print "delete ips from $opt\n";
eval {
diff --git a/src/lxc-pve-poststop-hook b/src/lxc-pve-poststop-hook
index c8bbb79..aba6f4f 100755
--- a/src/lxc-pve-poststop-hook
+++ b/src/lxc-pve-poststop-hook
@@ -11,7 +11,6 @@ use PVE::GuestHelpers;
use PVE::LXC::Config;
use PVE::LXC::Tools;
use PVE::LXC;
-use PVE::Network;
use PVE::RESTEnvironment;
use PVE::Storage;
use PVE::Tools;
@@ -49,8 +48,8 @@ PVE::LXC::Tools::lxc_hook(
my $ind = $1;
my $net = PVE::LXC::Config->parse_lxc_network($conf->{$k});
next if $net->{type} ne 'veth';
- # veth_delete tests with '-d /sys/class/net/$name' before running the command
- PVE::Network::veth_delete("veth${vmid}i$ind");
+ # reports the unplug and deletes the veth only where it still exists
+ PVE::LXC::net_veth_delete("veth${vmid}i$ind");
}
my $config_updated = 0;
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH pve-manager v2 14/16] ui: sdn: dhcp backend selector on all zones, expose dhcp options
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (12 preceding siblings ...)
2026-09-09 10:41 ` [PATCH pve-container v2 13/16] net: report veth plug and unplug to SDN with the hwaddr Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH pve-manager v2 15/16] sdn: bring the dhcp backends up at boot before the guests start Hannes Laimer
2026-09-09 10:41 ` [PATCH pve-docs v2 16/16] sdn: dhcp: document the ebpf backend Hannes Laimer
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
The zone dhcp property is now common to all zone types with two
backends, so the simple-zone automatic-DHCP checkbox hardcoding dnsmasq
becomes a selector on the common zone panel. It offers dnsmasq only
where the backend is supported. The subnet gains fields for the
dns-server option and for the new lease-time knob. The dns-server option
was settable through the API only so far and is the only way guests get
a DNS server with the ebpf backend. The lease time bounds how long an
edited mapping takes to reach a leased guest.
The selector is sent or deleted on every zone edit and the lease time on
every subnet edit, so this needs the libpve-network-perl that knows the
backend on every zone type and the lease time.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
www/manager6/sdn/SubnetEdit.js | 23 +++++++++++++++++++++++
www/manager6/sdn/zones/Base.js | 17 +++++++++++++++++
www/manager6/sdn/zones/SimpleEdit.js | 11 -----------
3 files changed, 40 insertions(+), 11 deletions(-)
diff --git a/www/manager6/sdn/SubnetEdit.js b/www/manager6/sdn/SubnetEdit.js
index a3608428..a1d36eeb 100644
--- a/www/manager6/sdn/SubnetEdit.js
+++ b/www/manager6/sdn/SubnetEdit.js
@@ -56,6 +56,29 @@ Ext.define('PVE.sdn.SubnetInputPanel', {
deleteEmpty: '{!isCreate}',
},
},
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'dhcp-dns-server',
+ vtype: 'IP64Address',
+ fieldLabel: gettext('DHCP DNS Server'),
+ allowBlank: true,
+ skipEmptyText: true,
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
+ {
+ xtype: 'proxmoxintegerfield',
+ name: 'dhcp-lease-time',
+ fieldLabel: gettext('DHCP Lease Time') + ' (s)',
+ emptyText: Proxmox.Utils.defaultText,
+ minValue: 60,
+ maxValue: 4294967295,
+ allowBlank: true,
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
],
});
diff --git a/www/manager6/sdn/zones/Base.js b/www/manager6/sdn/zones/Base.js
index 66f93de0..7696136d 100644
--- a/www/manager6/sdn/zones/Base.js
+++ b/www/manager6/sdn/zones/Base.js
@@ -82,6 +82,23 @@ Ext.define('PVE.panel.SDNZoneBase', {
},
);
+ let dhcpBackends = [
+ ['__default__', Proxmox.Utils.NoneText],
+ ['ebpf', 'eBPF'],
+ ];
+ if (me.type === 'simple') {
+ dhcpBackends.splice(1, 0, ['dnsmasq', 'dnsmasq']);
+ }
+
+ me.advancedItems.push({
+ xtype: 'proxmoxKVComboBox',
+ name: 'dhcp',
+ fieldLabel: gettext('DHCP Backend'),
+ comboItems: dhcpBackends,
+ value: '__default__',
+ deleteEmpty: !me.isCreate,
+ });
+
me.callParent();
},
});
diff --git a/www/manager6/sdn/zones/SimpleEdit.js b/www/manager6/sdn/zones/SimpleEdit.js
index ba10bb36..aeb076ae 100644
--- a/www/manager6/sdn/zones/SimpleEdit.js
+++ b/www/manager6/sdn/zones/SimpleEdit.js
@@ -19,17 +19,6 @@ Ext.define('PVE.sdn.zones.SimpleInputPanel', {
var me = this;
me.items = [];
- me.advancedItems = [
- {
- xtype: 'proxmoxcheckbox',
- name: 'dhcp',
- inputValue: 'dnsmasq',
- uncheckedValue: null,
- checked: false,
- fieldLabel: gettext('Automatic DHCP'),
- deleteEmpty: !me.isCreate,
- },
- ];
me.callParent();
},
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH pve-manager v2 15/16] sdn: bring the dhcp backends up at boot before the guests start
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (13 preceding siblings ...)
2026-09-09 10:41 ` [PATCH pve-manager v2 14/16] ui: sdn: dhcp backend selector on all zones, expose dhcp options Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
2026-09-09 10:41 ` [PATCH pve-docs v2 16/16] sdn: dhcp: document the ebpf backend Hannes Laimer
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
The eBPF DHCP responder keeps its state in bpffs and under /run, both
gone after a reboot. Without pending SDN changes nothing loaded it again
until the first guest plugged an interface. Every guest starting
alongside then queued on that first plug's full pass, and one waiting
longer than the lock bound started without the responder. The SDN commit
runs on every boot, so it brings the backends up first, whether SDN
changes are pending or not. Its unit is ordered after the cluster
filesystem it reads, and the guests and the HA local resource manager
are ordered behind it. A node without quorum at boot holds them until it
has one, the guests waited for that anyway. The commit's own work holds
them as well, so the resource manager now waits with the guests.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
bin/pve-sdn-commit | 3 +++
services/pve-sdn-commit.service | 3 ++-
2 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/bin/pve-sdn-commit b/bin/pve-sdn-commit
index aa7e9b29..ed2d6935 100644
--- a/bin/pve-sdn-commit
+++ b/bin/pve-sdn-commit
@@ -96,6 +96,9 @@ sub sdn_changed {
return fabrics_changed();
}
+# the dhcp backends bring back what a reboot took, pending changes or not
+PVE::Network::SDN::Dhcp::boot();
+
if (!sdn_changed()) {
print "No changes to SDN configuration detected, skipping reload\n";
exit 0;
diff --git a/services/pve-sdn-commit.service b/services/pve-sdn-commit.service
index ff723725..c96a4b55 100644
--- a/services/pve-sdn-commit.service
+++ b/services/pve-sdn-commit.service
@@ -2,7 +2,8 @@
Description=Commit Proxmox VE SDN changes
DefaultDependencies=no
Wants=pve-cluster.service network.target
-After=frr.service network.target corosync.service
+After=frr.service network.target corosync.service pve-cluster.service
+Before=pve-guests.service pve-ha-lrm.service
[Service]
ExecStart=/usr/share/pve-manager/helpers/pve-sdn-commit
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH pve-docs v2 16/16] sdn: dhcp: document the ebpf backend
2026-09-09 10:41 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs}/qemu-server v2 00/16] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (14 preceding siblings ...)
2026-09-09 10:41 ` [PATCH pve-manager v2 15/16] sdn: bring the dhcp backends up at boot before the guests start Hannes Laimer
@ 2026-09-09 10:41 ` Hannes Laimer
15 siblings, 0 replies; 17+ messages in thread
From: Hannes Laimer @ 2026-09-09 10:41 UTC (permalink / raw)
To: pve-devel
The DHCP setting of a zone became a choice of backend, the eBPF
responder is available on every zone type while dnsmasq stays limited to
simple zones. Describe how the responder answers, what it needs and what
it does not do. Describe the two subnet options both backends read as
well, the DNS server handed out and the lease time.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
pvesdn.adoc | 113 +++++++++++++++++++++++++++++++++++++++++++---------
1 file changed, 95 insertions(+), 18 deletions(-)
diff --git a/pvesdn.adoc b/pvesdn.adoc
index 3fd3533..8394adb 100644
--- a/pvesdn.adoc
+++ b/pvesdn.adoc
@@ -98,10 +98,12 @@ source /etc/network/interfaces.d/*
DHCP IPAM
~~~~~~~~~
-The DHCP integration into the built-in 'PVE' IP Address Management stack
-currently uses `dnsmasq` for giving out DHCP leases. This is currently opt-in.
+The DHCP integration into the built-in 'PVE' IP Address Management stack gives
+out leases either through `dnsmasq` or through an eBPF responder built into
+{pve}, chosen per zone. This is currently opt-in.
-To use that feature you need to install the `dnsmasq` package on every node:
+The eBPF backend needs no additional package. To use the `dnsmasq` backend you
+need to install the `dnsmasq` package on every node:
----
apt update
@@ -153,9 +155,8 @@ in your SDN setup.
* xref:pvesdn_config_controllers[Controllers]: For controlling layer 3 routing
in complex setups
-* xref:pvesdn_config_dhcp[DHCP]: Define a DHCP server for a zone that
- automatically allocates IPs for guests in the IPAM and leases them to the
- guests via DHCP.
+* xref:pvesdn_config_dhcp[DHCP]: Let a zone automatically allocate IPs for
+ guests in the IPAM and lease them to the guests via DHCP.
* xref:pvesdn_config_ipam[IPAM]: Enables external for IP address management for
guests
@@ -229,6 +230,10 @@ DNSZone:: DNS domain name. Used to register hostnames, such as
`<hostname>.<domain>`. The DNS zone must already exist on the DNS server.
Optional.
+DHCP Backend:: The DHCP backend for the zone, see xref:pvesdn_config_dhcp[DHCP].
+ `dnsmasq` is limited to Simple Zones, `ebpf` is available for every zone
+ type. Optional.
+
[[pvesdn_zone_plugin_simple]]
Simple Zones
@@ -442,6 +447,14 @@ SNAT:: Enable Source NAT which allows VMs from inside a
DNS Zone Prefix:: Add a prefix to the domain registration, like
<hostname>.prefix.<domain> Optional.
+DHCP DNS Server:: The DNS server a lease points the guest at. The `ebpf`
+ backend hands out an IPv4 DNS server only, one of the other address family
+ is left out of its answers. Optional.
+
+DHCP Lease Time:: The lease time in seconds. Without it `dnsmasq` hands out
+ infinite leases and the `ebpf` backend leases of ten minutes. `dnsmasq`
+ raises anything below two minutes to two minutes. Optional.
+
[[pvesdn_config_controllers]]
Controllers
@@ -1342,9 +1355,10 @@ DHCP
------
The DHCP plugin in {pve} SDN can be used to automatically deploy a DHCP server
-for a Zone. It provides DHCP for all Subnets in a Zone that have a DHCP range
-configured. Currently the only available backend plugin for DHCP is the dnsmasq
-plugin.
+for a Zone. Two backend plugins are available, `dnsmasq` and `ebpf`, chosen per
+zone. Both answer from the mappings in the IPAM, `ebpf` for IPv4 Subnets only.
+An automatic allocation takes an address from the Subnet's DHCP ranges, or from
+the whole Subnet when it has none.
The DHCP plugin works by allocating an IP in the IPAM plugin configured in the
Zone when adding a new network interface to a VM/CT. You can find more
@@ -1362,10 +1376,11 @@ available when using the xref:pvesdn_ipam_plugin_pveipam[PVE IPAM plugin].
Configuration
~~~~~~~~~~~~~
-You can enable automatic DHCP for a zone in the Web UI via the Zones panel and
-enabling DHCP in the advanced options of a zone.
+You can enable automatic DHCP for a zone in the Web UI via the Zones panel by
+choosing a DHCP backend in the advanced options of the zone.
-NOTE: Currently only Simple Zones have support for automatic DHCP
+NOTE: The `dnsmasq` backend is only available for Simple Zones, the `ebpf`
+backend for every zone type.
After automatic DHCP has been enabled for a Zone, DHCP Ranges need to be
configured for the subnets in a Zone. In order to that, go to the Vnets panel and
@@ -1379,22 +1394,24 @@ pvesh set /cluster/sdn/vnets/<vnet>/subnets/<subnet>
-dhcp-range start-address=10.0.2.100,end-address=10.0.2.200
----
-You also need to have a gateway configured for the subnet - otherwise
-automatic DHCP will not work.
+A Subnet without a gateway is served by both backends, its guests then get no
+default route. An IPv6 Subnet needs a gateway with the `dnsmasq` backend.
The DHCP plugin will then allocate IPs in the IPAM only in the configured
ranges.
-Do not forget to follow the installation steps for the
-xref:pvesdn_install_dhcp_ipam[dnsmasq DHCP plugin] as well.
+The subnet options *DHCP DNS Server* and *DHCP Lease Time*, see
+xref:pvesdn_config_subnet[Subnets], shape the leases handed out.
+
+When using the `dnsmasq` backend, do not forget to follow the installation
+steps for the xref:pvesdn_install_dhcp_ipam[dnsmasq DHCP plugin] as well.
Plugins
~~~~~~~
Dnsmasq Plugin
^^^^^^^^^^^^^^
-Currently this is the only DHCP plugin and therefore the plugin that gets used
-when you enable DHCP for a zone.
+The plugin used by zones with `dnsmasq` as their DHCP backend.
.Installation
For installation see the xref:pvesdn_install_dhcp_ipam[DHCP IPAM] section.
@@ -1437,6 +1454,66 @@ For more information please consult the documentation of
xref:pvesdn_ipam_plugin_pveipam[the PVE IPAM plugin]. Changing DHCP leases is
currently not supported for the other IPAM plugins.
+[[pvesdn_dhcp_ebpf]]
+eBPF Plugin
+^^^^^^^^^^^
+The plugin used by zones with `ebpf` as their DHCP backend. It needs no
+additional package and is available for every zone type.
+
+Instead of running a DHCP server on the bridge, a small program is attached to
+the network interface of every guest on the zone's VNets. It answers the DHCP
+requests of that guest right there, from the cluster's cache of the MAC address
+and IP mappings, which every mapping change made through {pve} keeps current.
+The exchange never reaches the bridge. Each guest interface is answered from the
+mapping of its own network device only, and only a request carrying that
+device's MAC address is answered. Requests on an interface without a mapping
+pass through untouched, so such a guest can still reach another DHCP server on
+the network.
+
+A mapping changed through {pve} reaches a running guest at its next lease
+renewal, without a restart. A deleted one is no longer answered, the
+guest keeps its address until the lease runs out.
+
+The program is attached when a guest interface is plugged and detached when it
+is unplugged. Every SDN apply attaches what is missing, so a zone switched to
+`ebpf` serves its running guests that hold a mapping from the next apply on.
+
+.Requirements
+The responder needs kernel 6.6 or newer. Every kernel shipped with this {pve}
+release qualifies.
+
+.Limitations
+* IPv4 only. IPv6 subnets are ignored by this backend.
+* Only untagged frames are answered. A guest tagging its own traffic with a
+ VLAN, for example on a trunk port, is not served.
+* The answer is given before the firewall of the guest and the rate limit of its
+ interface see the request, so neither the guest's firewall rules nor its DHCP
+ firewall option nor the rate limit apply to the request. The reply is put on
+ the interface's egress, so its rate limit shapes the reply and the firewall
+ never sees it. A request on an interface without a mapping passes the firewall
+ and the rate limit as usual.
+* The answer carries the address, netmask, server identifier, lease time and
+ MTU, the router where the Subnet has a gateway and the DNS server where it has
+ an IPv4 one, plus a blank proxy autodiscovery option that keeps Windows guests
+ from searching for one. Options `dnsmasq` could add through its custom
+ configuration, such as a domain name or PXE boot parameters, are not
+ available. On an interface with a mapping, a DISCOVER and every REQUEST of the
+ guest itself that is not addressed to another server are answered or refused
+ here and never reach another DHCP server on the network. Every other message
+ type, an INFORM for example, passes on and gets no answer from this backend.
+* The responder keeps no lease file, the mapping is the lease. What a guest
+ holds is what the IPAM holds for its MAC address.
+* Only a request carrying the guest interface's MAC address is answered. A guest
+ bridging further clients, such as nested containers, or one that changed its
+ MAC address, gets no answer from this backend, as with an unknown MAC address
+ under `dnsmasq`.
+* The responder identifies itself with the subnet gateway, or with a
+ link-local address on a subnet without one, which is then served without a
+ default route. Where a guest cannot reach that address, it cannot renew its
+ lease by unicast and falls back to broadcasting at its rebinding time.
+* Mappings changed directly in an external IPAM are not picked up, as with
+ `dnsmasq`.
+
[[pvesdn_firewall_integration]]
Firewall Integration
--------------------
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread