* [PATCH proxmox-ebpf v4 01/20] dhcp: add the per-tap DHCPv4 responder program
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH proxmox-ebpf v4 02/20] dhcp: add the subsystem that loads the responder and keeps its records Hannes Laimer
` (18 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
The program answers a guest's DISCOVER and REQUEST on the guest's own
tap from the record stored for that interface, so the exchange never
reaches the bridge. It keeps no lease state and needs no address of
the host in the guest's subnet. Only the NIC's own MAC gets the
record's address. A REQUEST for another address gets a NAK, which
ends a stale lease right away. Anything else passes on unchanged,
such as a REQUEST that selects another server or one from a network
the guest runs itself.
A tc program cannot create a packet, so the reply overwrites the
request and is sent out as that packet. The kernel then treats it as
it would have treated the request, and completes the UDP checksum on
the way out only if the guest sent with checksum offload. The field
has to hold the pseudo-header sum in that case and a valid checksum
otherwise, and the program cannot read which. It finds out by adding
one to a zeroed field through the kernel's checksum helper, whose
result differs between the two.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Cargo.toml | 5 +
debian/control | 6 +-
src/bpf-shim/bpf/bpf_helpers.h | 3 +
src/bpf-shim/vmlinux.h | 1 +
src/dhcp/bpf/dhcp.bpf.c | 401 ++++++++++++++++
src/dhcp/bpf/types.h | 24 +
tests/common/mod.rs | 63 ++-
tests/dhcp.rs | 825 +++++++++++++++++++++++++++++++++
8 files changed, 1323 insertions(+), 5 deletions(-)
create mode 100644 src/dhcp/bpf/dhcp.bpf.c
create mode 100644 src/dhcp/bpf/types.h
create mode 100644 tests/dhcp.rs
diff --git a/Cargo.toml b/Cargo.toml
index 1e8703d..172042e 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -11,6 +11,7 @@ homepage = "https://proxmox.com"
exclude = [".cargo", ".gitignore", "Makefile", "build", "debian"]
[features]
+dhcp = []
# When enabled, BPF programs compile with -DBPF_DEBUG so their DBG() macros
# expand to bpf_printk. Off by default
bpf-debug = []
@@ -20,3 +21,7 @@ aya = "0.13"
anyhow = "1"
log = "0.4"
nix = { version = "0.29", features = ["fs", "net"] }
+
+[[test]]
+name = "dhcp"
+required-features = ["dhcp"]
diff --git a/debian/control b/debian/control
index d3a9cc8..2dc841a 100644
--- a/debian/control
+++ b/debian/control
@@ -37,14 +37,18 @@ Depends:
Provides:
librust-proxmox-ebpf+bpf-debug-dev (= ${binary:Version}),
librust-proxmox-ebpf+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf+dhcp-dev (= ${binary:Version}),
librust-proxmox-ebpf-0-dev (= ${binary:Version}),
librust-proxmox-ebpf-0+bpf-debug-dev (= ${binary:Version}),
librust-proxmox-ebpf-0+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0+dhcp-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1+bpf-debug-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1+dhcp-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1.0-dev (= ${binary:Version}),
librust-proxmox-ebpf-0.1.0+bpf-debug-dev (= ${binary:Version}),
- librust-proxmox-ebpf-0.1.0+default-dev (= ${binary:Version})
+ librust-proxmox-ebpf-0.1.0+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1.0+dhcp-dev (= ${binary:Version})
Description: Proxmox VE eBPF - Rust source code
Source code for Debianized Rust crate "proxmox-ebpf"
diff --git a/src/bpf-shim/bpf/bpf_helpers.h b/src/bpf-shim/bpf/bpf_helpers.h
index de3bd84..3118de2 100644
--- a/src/bpf-shim/bpf/bpf_helpers.h
+++ b/src/bpf-shim/bpf/bpf_helpers.h
@@ -29,6 +29,9 @@ extern long bpf_skb_load_bytes(const void *skb, __u32 offset, void *to, __u32 le
extern long bpf_skb_store_bytes(void *skb, __u32 offset, const void *from, __u32 len, __u64 flags);
extern long bpf_skb_pull_data(void *skb, __u32 len);
extern long bpf_skb_change_tail(void *skb, __u32 new_len, __u64 flags);
+extern long bpf_l4_csum_replace(void *skb, __u32 offset, __u64 from, __u64 to, __u64 flags);
extern long bpf_redirect(__u32 ifindex, __u64 flags);
+#define BPF_F_PSEUDO_HDR 0x10
+
#endif
diff --git a/src/bpf-shim/vmlinux.h b/src/bpf-shim/vmlinux.h
index 99a7aaf..677d2ee 100644
--- a/src/bpf-shim/vmlinux.h
+++ b/src/bpf-shim/vmlinux.h
@@ -23,6 +23,7 @@ struct __sk_buff {
__u32 len;
__u32 ifindex;
__u32 mark;
+ __u32 vlan_present;
};
struct ethhdr {
diff --git a/src/dhcp/bpf/dhcp.bpf.c b/src/dhcp/bpf/dhcp.bpf.c
new file mode 100644
index 0000000..ab21ffb
--- /dev/null
+++ b/src/dhcp/bpf/dhcp.bpf.c
@@ -0,0 +1,401 @@
+// Per-tap DHCPv4 responder. Attached on tap ingress, it answers DISCOVER and
+// REQUEST on interfaces with a record by rewriting the request into the reply
+// in place and redirecting it back out the same interface. The redirect
+// consumes the request, so nothing upstream on the bridge sees it. Everything
+// else, including requests on interfaces without a record, is handed on
+// untouched to whatever runs after this program on the interface.
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_endian.h>
+#include "types.h"
+#include "bpf_debug.h"
+
+char LICENSE[] SEC("license") = "GPL";
+
+// what is left alone gets TC_ACT_UNSPEC, since TC_ACT_OK would end the
+// chain before the qdisc filters, where the rate limiter sits
+#define TC_ACT_UNSPEC (-1)
+#define TC_ACT_SHOT 2
+
+#define ETH_HLEN 14
+#define ETH_P_IP 0x0800
+#define IP_PROTO_UDP 17
+
+#define BOOTP_LEN 236
+// replies are padded up to the minimal BOOTP message size (RFC 1542), since
+// some old client stacks drop anything shorter
+#define BOOTP_MIN_LEN 300
+#define DHCP_MAGIC 0x63825363
+#define BOOTP_FLAG_BCAST 0x8000
+
+#define BOOTREQUEST 1
+#define BOOTREPLY 2
+
+#define DHCP_DISCOVER 1
+#define DHCP_OFFER 2
+#define DHCP_REQUEST 3
+#define DHCP_ACK 5
+#define DHCP_NAK 6
+
+#define OPT_PAD 0
+#define OPT_NETMASK 1
+#define OPT_ROUTER 3
+#define OPT_DNS 6
+#define OPT_MTU 26
+#define OPT_REQUESTED_IP 50
+#define OPT_LEASE 51
+#define OPT_MSGTYPE 53
+#define OPT_SERVER_ID 54
+#define OPT_WPAD 252
+#define OPT_END 255
+
+// the answer of every attached interface, keyed by ifindex
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __type(key, __u32);
+ __type(value, struct dhcp_record);
+ __uint(max_entries, 16384);
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __uint(pinning, LIBBPF_PIN_BY_NAME);
+} dhcp_iface_records SEC(".maps");
+
+struct bootp {
+ __u8 op;
+ __u8 htype;
+ __u8 hlen;
+ __u8 hops;
+ __u32 xid;
+ __u16 secs;
+ __u16 flags;
+ __u32 ciaddr;
+ __u32 yiaddr;
+ __u32 siaddr;
+ __u32 giaddr;
+ __u8 chaddr[16];
+ __u8 sname[64];
+ __u8 file[128];
+};
+
+struct reply_pkt {
+ struct iphdr ip;
+ struct udphdr udp;
+ struct bootp bp;
+ __u8 cookie[4];
+ __u8 opts[BOOTP_MIN_LEN - BOOTP_LEN - 4];
+} __attribute__((packed));
+
+// the responder has no address on the tap, so replies carry a fixed locally
+// administered source MAC that nothing ever learns
+static const __u8 SRC_MAC[6] = { 0x02, 0x00, 0x00, 0x00, 0x00, 0x01 };
+static const __u8 BCAST_MAC[6] = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff };
+
+static __always_inline __u16 csum_fold(__u32 sum)
+{
+ sum = (sum & 0xffff) + (sum >> 16);
+ sum = (sum & 0xffff) + (sum >> 16);
+ return sum;
+}
+
+static __always_inline __u16 ip_csum(const __u8 *hdr)
+{
+ __u32 sum = 0;
+#pragma unroll
+ for (int i = 0; i < 10; i++) {
+ __u16 word;
+ __builtin_memcpy(&word, hdr + 2 * i, 2);
+ sum += word;
+ }
+ return ~csum_fold(sum);
+}
+
+// the pseudo-header sum, the starting value the kernel expects in a
+// checksum it completes on the way out
+static __always_inline __u16 udp_pseudo_csum(__u32 saddr, __u32 daddr, __u16 len)
+{
+ __u32 sum = (saddr & 0xffff) + (saddr >> 16) + (daddr & 0xffff) + (daddr >> 16);
+ sum += bpf_htons(IP_PROTO_UDP) + len;
+ return csum_fold(sum);
+}
+
+#define UDP_CHECK_OFF (ETH_HLEN + sizeof(struct iphdr) + 6)
+
+SEC("classifier")
+int tc_dhcp_ingress(struct __sk_buff *skb)
+{
+ __u32 min_len = ETH_HLEN + sizeof(struct iphdr) + sizeof(struct udphdr) + BOOTP_LEN + 4;
+
+ // the kernel strips a VLAN tag before this hook. A request the guest
+ // tagged itself belongs to a network of the guest's own and passes
+ if (skb->vlan_present)
+ return TC_ACT_UNSPEC;
+
+ // everything shorter is not a DHCP request
+ if (skb->len < min_len)
+ return TC_ACT_UNSPEC;
+
+ // the headers are read in place. A large packet from the guest is mostly
+ // outside the linear part, and pulling it in would cost every one of them
+ struct {
+ struct ethhdr eth;
+ struct iphdr ip;
+ struct udphdr udp;
+ } __attribute__((packed)) hdr;
+ if (bpf_skb_load_bytes(skb, 0, &hdr, sizeof(hdr)))
+ return TC_ACT_UNSPEC;
+ if (hdr.eth.h_proto != bpf_htons(ETH_P_IP))
+ return TC_ACT_UNSPEC;
+ if (hdr.ip.version != 4 || hdr.ip.ihl != 5 || hdr.ip.protocol != IP_PROTO_UDP)
+ return TC_ACT_UNSPEC;
+ // a fragment is never a whole request, and past the first one the bytes at
+ // the UDP header offset are arbitrary payload
+ if (hdr.ip.frag_off & bpf_htons(0x3fff))
+ return TC_ACT_UNSPEC;
+ if (hdr.udp.dest != bpf_htons(67))
+ return TC_ACT_UNSPEC;
+
+ void *data = (void *)(long)skb->data;
+ void *data_end = (void *)(long)skb->data_end;
+
+ if (data + min_len > data_end) {
+ // a request may sit in a non-linear skb
+ if (bpf_skb_pull_data(skb, min_len))
+ return TC_ACT_UNSPEC;
+ data = (void *)(long)skb->data;
+ data_end = (void *)(long)skb->data_end;
+ if (data + min_len > data_end)
+ return TC_ACT_UNSPEC;
+ }
+
+ struct iphdr *ip = data + ETH_HLEN;
+
+ struct bootp *req = data + ETH_HLEN + sizeof(struct iphdr) + sizeof(struct udphdr);
+ if (req->op != BOOTREQUEST || req->htype != 1 || req->hlen != 6)
+ return TC_ACT_UNSPEC;
+
+ // a reply to a relayed request must go unicast to the relay instead of the
+ // client. Those are left to a server that speaks relay
+ if (req->giaddr) {
+ DBG("dhcp: request relayed via %pI4 passed", &req->giaddr);
+ return TC_ACT_UNSPEC;
+ }
+
+ __u32 *magic = (void *)req + BOOTP_LEN;
+ if (*magic != bpf_htonl(DHCP_MAGIC))
+ return TC_ACT_UNSPEC;
+
+ // only the NIC's own MAC gets the interface's answer. The client identifier
+ // option is not honored either, so the reply goes back to the hardware
+ // address the request came with. That is manual allocation in RFC 2131
+ // terms
+ __u8 chaddr[6];
+ __builtin_memcpy(chaddr, req->chaddr, 6);
+ __u32 ifindex = skb->ifindex;
+ struct dhcp_record *rec = bpf_map_lookup_elem(&dhcp_iface_records, &ifindex);
+ // a marker only names its vnet and serves nothing
+ if (rec && !rec->ip)
+ rec = (void *)0;
+ if (!rec) {
+ DBG("dhcp: no record for ifindex %d", ifindex);
+ return TC_ACT_UNSPEC;
+ }
+ for (int i = 0; i < 6; i++) {
+ if (chaddr[i] != rec->mac[i]) {
+ DBG("dhcp: chaddr is not the NIC's on ifindex %d", ifindex);
+ return TC_ACT_UNSPEC;
+ }
+ }
+
+ __u8 msgtype = 0;
+ __u32 requested_ip = 0;
+ __u32 server_id = 0;
+ __u32 off = min_len;
+ // the options end with the UDP datagram. Whatever the IP datagram or the
+ // frame carries beyond it is padding or a trailer, not the guest's request
+ __u32 end = ETH_HLEN + sizeof(struct iphdr) + bpf_ntohs(hdr.udp.len);
+ if (end > ETH_HLEN + bpf_ntohs(ip->tot_len))
+ end = ETH_HLEN + bpf_ntohs(ip->tot_len);
+ if (end > skb->len)
+ end = skb->len;
+
+ for (int i = 0; i < 128; i++) {
+ __u8 code, len;
+ if (off >= end || bpf_skb_load_bytes(skb, off, &code, 1))
+ break;
+ if (code == OPT_END)
+ break;
+ if (code == OPT_PAD) {
+ off += 1;
+ continue;
+ }
+ if (off + 2 > end || bpf_skb_load_bytes(skb, off + 1, &len, 1))
+ break;
+ if (off + 2 + len > end)
+ break;
+ if (code == OPT_MSGTYPE && len == 1)
+ bpf_skb_load_bytes(skb, off + 2, &msgtype, 1);
+ else if (code == OPT_REQUESTED_IP && len == 4)
+ bpf_skb_load_bytes(skb, off + 2, &requested_ip, 4);
+ else if (code == OPT_SERVER_ID && len == 4)
+ bpf_skb_load_bytes(skb, off + 2, &server_id, 4);
+ off += 2 + len;
+ }
+
+ __u8 reply;
+ if (msgtype == DHCP_DISCOVER) {
+ reply = DHCP_OFFER;
+ } else if (msgtype == DHCP_REQUEST) {
+ // a REQUEST naming another server declines our offer and is
+ // answered, if at all, by that server
+ if (server_id && server_id != rec->server_id) {
+ DBG("dhcp: request selected server %pI4, passed", &server_id);
+ return TC_ACT_UNSPEC;
+ }
+ // a NAK on a stale address makes the client re-DISCOVER instead of
+ // keeping the old address until its lease ends
+ __u32 want = requested_ip ? requested_ip : req->ciaddr;
+ reply = want == rec->ip ? DHCP_ACK : DHCP_NAK;
+ if (reply == DHCP_NAK)
+ DBG("dhcp: request for %pI4 against record %pI4, nak", &want, &rec->ip);
+ } else {
+ DBG("dhcp: message type %d passed", msgtype);
+ return TC_ACT_UNSPEC;
+ }
+
+ __u32 xid = req->xid;
+ __u16 flags = req->flags;
+ __u32 ciaddr = req->ciaddr;
+
+ struct reply_pkt buf = {};
+
+ buf.bp.op = BOOTREPLY;
+ buf.bp.htype = 1;
+ buf.bp.hlen = 6;
+ buf.bp.xid = xid;
+ buf.bp.flags = flags;
+ __builtin_memcpy(buf.bp.chaddr, chaddr, 6);
+
+ buf.cookie[0] = 0x63;
+ buf.cookie[1] = 0x82;
+ buf.cookie[2] = 0x53;
+ buf.cookie[3] = 0x63;
+
+ __u32 o = 0;
+ buf.opts[o++] = OPT_MSGTYPE;
+ buf.opts[o++] = 1;
+ buf.opts[o++] = reply;
+ buf.opts[o++] = OPT_SERVER_ID;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->server_id, 4);
+ o += 4;
+
+ if (reply != DHCP_NAK) {
+ buf.bp.yiaddr = rec->ip;
+
+ buf.opts[o++] = OPT_LEASE;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->lease, 4);
+ o += 4;
+ buf.opts[o++] = OPT_NETMASK;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->netmask, 4);
+ o += 4;
+ if (rec->router) {
+ buf.opts[o++] = OPT_ROUTER;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->router, 4);
+ o += 4;
+ }
+ if (rec->dns) {
+ buf.opts[o++] = OPT_DNS;
+ buf.opts[o++] = 4;
+ __builtin_memcpy(&buf.opts[o], &rec->dns, 4);
+ o += 4;
+ }
+ if (rec->mtu) {
+ buf.opts[o++] = OPT_MTU;
+ buf.opts[o++] = 2;
+ __builtin_memcpy(&buf.opts[o], &rec->mtu, 2);
+ o += 2;
+ }
+ // an empty WPAD answer, without it some Windows versions
+ // probe for a proxy config indefinitely
+ buf.opts[o++] = OPT_WPAD;
+ buf.opts[o++] = 1;
+ buf.opts[o++] = '\n';
+ }
+ buf.opts[o++] = OPT_END;
+
+ // the zeroed tail of opts pads short replies out to the minimum, and on
+ // the wire those zeros read as pad options after the end option
+ __u32 bootp_out = BOOTP_LEN + 4 + o;
+ if (bootp_out < BOOTP_MIN_LEN)
+ bootp_out = BOOTP_MIN_LEN;
+
+ __u32 payload = sizeof(struct udphdr) + bootp_out;
+
+ buf.ip.version = 4;
+ buf.ip.ihl = 5;
+ buf.ip.ttl = 64;
+ buf.ip.protocol = IP_PROTO_UDP;
+ buf.ip.tot_len = bpf_htons(sizeof(struct iphdr) + payload);
+ buf.ip.saddr = rec->server_id;
+ // a renewing client has an address and gets a unicast answer.
+ // Everything else cannot be addressed yet
+ buf.ip.daddr = (reply != DHCP_NAK && ciaddr) ? ciaddr : 0xffffffff;
+ buf.ip.check = ip_csum((const __u8 *)&buf.ip);
+
+ buf.udp.source = bpf_htons(67);
+ buf.udp.dest = bpf_htons(68);
+ buf.udp.len = bpf_htons(payload);
+ buf.udp.check = 0;
+
+ __u32 wlen = sizeof(struct iphdr) + payload;
+ if (wlen > sizeof(buf))
+ return TC_ACT_UNSPEC;
+
+ if (bpf_skb_change_tail(skb, ETH_HLEN + wlen, 0))
+ return TC_ACT_UNSPEC;
+ // the request is gone from here on, so a failed write drops the frame
+ // rather than forwarding what is left of it
+
+ // NAKs are always broadcast, and a client without an address that set the
+ // BROADCAST flag asked for a broadcast reply
+ __u8 eth_hdr[ETH_HLEN];
+ if (reply == DHCP_NAK || (!ciaddr && (flags & bpf_htons(BOOTP_FLAG_BCAST)))) {
+ __builtin_memcpy(eth_hdr, BCAST_MAC, 6);
+ } else {
+ __builtin_memcpy(eth_hdr, chaddr, 6);
+ }
+ __builtin_memcpy(eth_hdr + 6, SRC_MAC, 6);
+ eth_hdr[12] = 0x08;
+ eth_hdr[13] = 0x00;
+
+ if (bpf_skb_store_bytes(skb, 0, eth_hdr, ETH_HLEN, 0))
+ return TC_ACT_SHOT;
+ if (bpf_skb_store_bytes(skb, ETH_HLEN, &buf, wlen, 0))
+ return TC_ACT_SHOT;
+
+ // The reply reuses the request's skb, whose checksum state the program
+ // cannot see. A request sent with checksum offload is completed by the kernel
+ // on the way out, and it expects the pseudo-header sum in the field. Any
+ // other request needs the field valid as it is, and zero is valid. The
+ // helper below folds a checksum field once more when the kernel still
+ // has to complete it, so adding one to the zeroed field ends up as 1 in
+ // that state and as its complement otherwise. The low flag bits give
+ // the width of the field.
+ if (bpf_l4_csum_replace(skb, UDP_CHECK_OFF, 0, 1, BPF_F_PSEUDO_HDR | 2))
+ return TC_ACT_SHOT;
+ __u16 probe = 0;
+ if (bpf_skb_load_bytes(skb, UDP_CHECK_OFF, &probe, 2))
+ return TC_ACT_SHOT;
+ __u16 check = probe == 1 ? udp_pseudo_csum(buf.ip.saddr, buf.ip.daddr, buf.udp.len) : 0;
+ if (bpf_skb_store_bytes(skb, UDP_CHECK_OFF, &check, 2, 0))
+ return TC_ACT_SHOT;
+
+#ifdef BPF_DEBUG
+ __u32 yiaddr = buf.bp.yiaddr;
+ DBG("dhcp: reply type %d with %pI4 to ifindex %d", reply, &yiaddr, skb->ifindex);
+#endif
+ return bpf_redirect(skb->ifindex, 0);
+}
diff --git a/src/dhcp/bpf/types.h b/src/dhcp/bpf/types.h
new file mode 100644
index 0000000..7da6969
--- /dev/null
+++ b/src/dhcp/bpf/types.h
@@ -0,0 +1,24 @@
+#ifndef PROXMOX_EBPF_DHCP_TYPES_H
+#define PROXMOX_EBPF_DHCP_TYPES_H
+
+// Keep in sync with ../types.rs
+// __u8/__u16/__u32 come from vmlinux.h
+
+// addresses, lease and mtu are stored in network byte order, so a
+// reply copies them into the packet as-is
+struct dhcp_record {
+ __u32 ip; // 0 = removed, the entry is a marker naming its vnet
+ __u32 netmask;
+ __u32 router; // 0 = not served
+ __u32 dns; // 0 = not served
+ __u32 server_id;
+ __u32 lease;
+ __u16 mtu; // 0 = not served
+ __u8 vnet[8]; // the vnet the writer saw the interface on, zero for none, never read here
+ __u8 mac[6]; // the NIC's, a request carrying another is passed on
+};
+
+_Static_assert(sizeof(struct dhcp_record) == 40, "dhcp_record layout");
+_Static_assert(__builtin_offsetof(struct dhcp_record, mac) == 34, "dhcp_record layout");
+
+#endif
diff --git a/tests/common/mod.rs b/tests/common/mod.rs
index 67fac33..c8fd285 100644
--- a/tests/common/mod.rs
+++ b/tests/common/mod.rs
@@ -27,13 +27,16 @@ pub struct SkBuff {
pub len: u32,
pub ifindex: u32,
pub mark: u32,
+ pub vlan_present: u32,
}
/// The skb handed to a program plus the buffer behind it. The helpers get the skb pointer and
-/// cast back to this, so the skb must stay the first field.
+/// cast back to this, so the skb must stay the first field. `partial` models a checksum the
+/// kernel still has to complete, the state a request sent with checksum offload carries.
#[repr(C)]
pub struct TestSkb {
skb: SkBuff,
+ pub partial: bool,
buf: Box<[u8; PKT_CAP]>,
}
@@ -49,7 +52,9 @@ impl TestSkb {
len: packet.len() as u32,
ifindex,
mark: 0,
+ vlan_present: 0,
},
+ partial: false,
buf,
};
t.sync();
@@ -62,6 +67,11 @@ impl TestSkb {
self.skb.data_end = base + self.skb.len as usize;
}
+ /// The frame arrived tagged, and the kernel keeps the tag beside the data.
+ pub fn tag_vlan(&mut self) {
+ self.skb.vlan_present = 1;
+ }
+
pub fn packet(&self) -> &[u8] {
&self.buf[..self.skb.len as usize]
}
@@ -184,9 +194,10 @@ pub extern "C" fn bpf_skb_pull_data(skb: *mut c_void, len: u32) -> c_long {
#[unsafe(no_mangle)]
pub extern "C" fn bpf_skb_change_tail(skb: *mut c_void, new_len: u32, _flags: u64) -> c_long {
let t = unsafe { testskb(skb) };
- // the kernel refuses to cut into the headers it knows, which is the link header here.
- // Growing stops at the harness buffer
- let min_len = 14;
+ // the kernel refuses to cut into the headers it knows. tun sets the transport header on
+ // every frame it hands in, so an IPv4 UDP frame keeps 34 bytes. One with checksum offload
+ // keeps its checksum field as well. Growing stops at the harness buffer
+ let min_len = if t.partial { 42 } else { 34 };
if new_len < min_len || new_len as usize > PKT_CAP {
return -EINVAL;
}
@@ -200,6 +211,50 @@ pub extern "C" fn bpf_skb_change_tail(skb: *mut c_void, new_len: u32, _flags: u6
0
}
+// the kernel's ones' complement primitives, csum_fold complements its result
+fn csum_add(a: u32, b: u32) -> u32 {
+ let (r, carry) = a.overflowing_add(b);
+ r + carry as u32
+}
+fn csum_fold(sum: u32) -> u16 {
+ let mut s = (sum & 0xffff) + (sum >> 16);
+ s = (s & 0xffff) + (s >> 16);
+ !(s as u16)
+}
+
+/// The kernel's field update, inet_proto_csum_replace4 for a 16-bit field. A partial checksum
+/// gets the difference folded in, and any other is complemented around the update. So the same
+/// call ends in different values.
+#[unsafe(no_mangle)]
+pub extern "C" fn bpf_l4_csum_replace(
+ skb: *mut c_void,
+ offset: u32,
+ from: u64,
+ to: u64,
+ flags: u64,
+) -> c_long {
+ let t = unsafe { testskb(skb) };
+ let offset = offset as usize;
+ assert_eq!(flags & 0xf, 2, "the responder updates the 16-bit field");
+ assert!(
+ flags & 0x10 != 0,
+ "the responder passes the pseudo-header flag"
+ );
+ if offset + 2 > t.skb.len as usize {
+ return -EFAULT;
+ }
+ let field = u16::from_ne_bytes([t.buf[offset], t.buf[offset + 1]]) as u32;
+ // a 16-bit field takes 16-bit values, so the kernel narrows both before the update
+ let (from, to) = (from as u16 as u32, to as u16 as u32);
+ let new = if t.partial {
+ !csum_fold(csum_add(csum_add(field, !from), to))
+ } else {
+ csum_fold(csum_add(csum_add(!field, !from), to))
+ };
+ t.buf[offset..offset + 2].copy_from_slice(&new.to_ne_bytes());
+ 0
+}
+
#[unsafe(no_mangle)]
pub extern "C" fn bpf_redirect(ifindex: u32, flags: u64) -> c_long {
REDIRECTED.with(|r| *r.borrow_mut() = Some((ifindex, flags)));
diff --git a/tests/dhcp.rs b/tests/dhcp.rs
new file mode 100644
index 0000000..8a00392
--- /dev/null
+++ b/tests/dhcp.rs
@@ -0,0 +1,825 @@
+//! The dhcp responder program run natively, crafted requests in and the rewritten reply out.
+
+mod common;
+
+use std::ffi::{c_int, c_void};
+
+use common::*;
+
+unsafe extern "C" {
+ fn tc_dhcp_ingress(skb: *mut SkBuff) -> c_int;
+ static dhcp_iface_records: u8;
+}
+
+const MAC: [u8; 6] = [0xda, 0x65, 0x8f, 0x18, 0x9b, 0x6f];
+const SRC_MAC: [u8; 6] = [0x02, 0x00, 0x00, 0x00, 0x00, 0x01];
+const BCAST: [u8; 6] = [0xff; 6];
+const IFINDEX: u32 = 42;
+
+const IP: [u8; 4] = [10, 0, 0, 100];
+const MASK: [u8; 4] = [255, 255, 255, 0];
+const GW: [u8; 4] = [10, 0, 0, 1];
+const DNS: [u8; 4] = [1, 1, 1, 1];
+const LEASE: u32 = 300;
+const MTU: u16 = 1500;
+
+const OPTS_OFF: usize = 14 + 20 + 8 + 236 + 4;
+
+const DISCOVER: &[u8] = &[53, 1, 1];
+const REQUEST: &[u8] = &[53, 1, 3];
+
+fn records_map() -> *const c_void {
+ (unsafe { &dhcp_iface_records } as *const u8).cast()
+}
+
+// the record map of a node, still empty
+fn setup_maps() {
+ register_map(records_map(), 4);
+}
+
+// the record map is keyed by the interface the request arrives on
+fn key() -> [u8; 4] {
+ IFINDEX.to_ne_bytes()
+}
+
+fn record(
+ ip: [u8; 4],
+ mask: [u8; 4],
+ router: [u8; 4],
+ dns: [u8; 4],
+ server: [u8; 4],
+ lease: u32,
+ mtu: u16,
+) -> [u8; 40] {
+ let mut r = [0u8; 40];
+ r[0..4].copy_from_slice(&ip);
+ r[4..8].copy_from_slice(&mask);
+ r[8..12].copy_from_slice(&router);
+ r[12..16].copy_from_slice(&dns);
+ r[16..20].copy_from_slice(&server);
+ r[20..24].copy_from_slice(&lease.to_be_bytes());
+ r[24..26].copy_from_slice(&mtu.to_be_bytes());
+ r[26..34].copy_from_slice(b"testvnet");
+ r[34..40].copy_from_slice(&MAC);
+ r
+}
+
+fn setup_full_record() {
+ setup_maps();
+ map_insert(
+ records_map(),
+ &key(),
+ &record(IP, MASK, GW, DNS, GW, LEASE, MTU),
+ );
+}
+
+struct Request {
+ mac: [u8; 6],
+ xid: [u8; 4],
+ flags: [u8; 2],
+ ciaddr: [u8; 4],
+ giaddr: [u8; 4],
+ options: Vec<u8>,
+ dport: u16,
+ ethertype: [u8; 2],
+ ipproto: u8,
+ frag_off: [u8; 2],
+ ip_options: Vec<u8>,
+ op: u8,
+ htype: u8,
+ hlen: u8,
+ end: bool,
+ trailer: Vec<u8>,
+ // claimed on top of the real IP total length
+ tot_len_excess: u16,
+}
+
+impl Default for Request {
+ fn default() -> Self {
+ Request {
+ mac: MAC,
+ xid: [0xde, 0xad, 0xbe, 0xef],
+ flags: [0, 0],
+ ciaddr: [0; 4],
+ giaddr: [0; 4],
+ options: DISCOVER.to_vec(),
+ dport: 67,
+ ethertype: [0x08, 0x00],
+ ipproto: 17,
+ frag_off: [0, 0],
+ ip_options: Vec::new(),
+ op: 1,
+ htype: 1,
+ hlen: 6,
+ end: true,
+ trailer: Vec::new(),
+ tot_len_excess: 0,
+ }
+ }
+}
+
+impl Request {
+ fn build(&self) -> Vec<u8> {
+ assert_eq!(self.ip_options.len() % 4, 0);
+ let ihl = 5 + self.ip_options.len() / 4;
+ let payload = 8 + 236 + 4 + self.options.len() + usize::from(self.end);
+ let mut p = Vec::new();
+ // ethernet
+ p.extend_from_slice(&BCAST);
+ p.extend_from_slice(&self.mac);
+ p.extend_from_slice(&self.ethertype);
+ // ipv4
+ p.extend_from_slice(&[0x40 | ihl as u8, 0]);
+ p.extend_from_slice(&((ihl * 4 + payload) as u16 + self.tot_len_excess).to_be_bytes());
+ p.extend_from_slice(&[0, 0]); // id
+ p.extend_from_slice(&self.frag_off);
+ p.extend_from_slice(&[64, self.ipproto, 0, 0]);
+ p.extend_from_slice(&[0, 0, 0, 0]); // saddr
+ p.extend_from_slice(&[255, 255, 255, 255]); // daddr
+ p.extend_from_slice(&self.ip_options);
+ // udp
+ p.extend_from_slice(&68u16.to_be_bytes());
+ p.extend_from_slice(&self.dport.to_be_bytes());
+ p.extend_from_slice(&(payload as u16).to_be_bytes());
+ p.extend_from_slice(&[0, 0]);
+ // bootp
+ p.extend_from_slice(&[self.op, self.htype, self.hlen, 0]);
+ p.extend_from_slice(&self.xid);
+ p.extend_from_slice(&[0, 0]); // secs
+ p.extend_from_slice(&self.flags);
+ p.extend_from_slice(&self.ciaddr);
+ p.extend_from_slice(&[0; 8]); // yiaddr, siaddr
+ p.extend_from_slice(&self.giaddr);
+ p.extend_from_slice(&self.mac);
+ p.extend_from_slice(&[0; 10]); // chaddr padding
+ p.extend_from_slice(&[0; 192]); // sname, file
+ // cookie and options
+ p.extend_from_slice(&[0x63, 0x82, 0x53, 0x63]);
+ p.extend_from_slice(&self.options);
+ if self.end {
+ p.push(255);
+ }
+ // bytes past the datagram, padding or a trailer the frame carries
+ p.extend_from_slice(&self.trailer);
+ p
+ }
+
+ fn run(&self) -> (c_int, TestSkb) {
+ self.run_on(IFINDEX)
+ }
+
+ fn run_on(&self, ifindex: u32) -> (c_int, TestSkb) {
+ let mut skb = TestSkb::new(&self.build(), ifindex);
+ let verdict = skb.run(tc_dhcp_ingress);
+ (verdict, skb)
+ }
+}
+
+fn parse_opts(pkt: &[u8]) -> Vec<(u8, Vec<u8>)> {
+ let mut opts = Vec::new();
+ let mut off = OPTS_OFF;
+ loop {
+ let code = pkt[off];
+ if code == 255 {
+ break;
+ }
+ if code == 0 {
+ off += 1;
+ continue;
+ }
+ let len = pkt[off + 1] as usize;
+ opts.push((code, pkt[off + 2..off + 2 + len].to_vec()));
+ off += 2 + len;
+ }
+ opts
+}
+
+fn ip_csum_ok(pkt: &[u8]) -> bool {
+ let mut sum = 0u32;
+ for w in pkt[14..34].chunks(2) {
+ sum += u32::from(u16::from_be_bytes([w[0], w[1]]));
+ }
+ while sum > 0xffff {
+ sum = (sum & 0xffff) + (sum >> 16);
+ }
+ sum == 0xffff
+}
+
+/// The offset just past the end option of a reply.
+fn opts_end(pkt: &[u8]) -> usize {
+ let mut off = OPTS_OFF;
+ loop {
+ match pkt[off] {
+ 255 => return off + 1,
+ 0 => off += 1,
+ _ => off += 2 + pkt[off + 1] as usize,
+ }
+ }
+}
+
+/// What every reply has to look like whatever the request was. The addressing and the answer
+/// itself are the caller's to check.
+fn assert_reply_invariants(pkt: &[u8]) {
+ assert_eq!(&pkt[6..12], &SRC_MAC, "eth src");
+ assert_eq!(&pkt[12..14], &[0x08, 0x00], "ethertype");
+
+ assert_eq!(pkt[14], 0x45, "ihl/version");
+ assert_eq!(pkt[22], 64, "ttl");
+ assert_eq!(pkt[23], 17, "protocol");
+ assert_eq!(&pkt[26..30], &GW, "ip src is the server id");
+ assert!(ip_csum_ok(pkt), "ip checksum");
+ let tot_len = u16::from_be_bytes([pkt[16], pkt[17]]) as usize;
+ assert_eq!(tot_len, pkt.len() - 14, "ip total length");
+
+ assert_eq!(&pkt[34..36], &67u16.to_be_bytes(), "udp sport");
+ assert_eq!(&pkt[36..38], &68u16.to_be_bytes(), "udp dport");
+ let udp_len = u16::from_be_bytes([pkt[38], pkt[39]]) as usize;
+ assert_eq!(udp_len, pkt.len() - 34, "udp length");
+ assert_eq!(&pkt[40..42], &[0, 0], "udp checksum zero, none to complete");
+ assert!(pkt.len() - 42 >= 300, "bootp minimum message size");
+
+ assert_eq!(&pkt[42..45], &[2, 1, 6], "bootp reply op, htype, hlen");
+ assert_eq!(&pkt[46..50], &[0xde, 0xad, 0xbe, 0xef], "xid preserved");
+ assert_eq!(&pkt[70..76], &MAC, "chaddr");
+ assert_eq!(&pkt[278..282], &[0x63, 0x82, 0x53, 0x63], "cookie");
+ assert!(
+ matches!(parse_opts(pkt).first(), Some((53, t)) if t.len() == 1 && [2, 5, 6].contains(&t[0])),
+ "message type leads the options"
+ );
+ let end = opts_end(pkt);
+ assert!(
+ pkt[end..].iter().all(|&b| b == 0),
+ "padding after the end option is zero"
+ );
+}
+
+fn assert_reply_headers(pkt: &[u8], eth_dst: [u8; 6], ip_dst: [u8; 4], yiaddr: [u8; 4]) {
+ assert_reply_invariants(pkt);
+ assert_eq!(&pkt[0..6], ð_dst, "eth dst");
+ assert_eq!(&pkt[30..34], &ip_dst, "ip dst");
+ assert_eq!(&pkt[58..62], &yiaddr, "yiaddr");
+}
+
+#[test]
+fn discover_gets_full_offer() {
+ setup_full_record();
+ let (verdict, skb) = Request::default().run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_eq!(redirected(), Some(IFINDEX));
+ assert_eq!(
+ redirect_flags(),
+ Some(0),
+ "the reply leaves through the tap"
+ );
+
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, MAC, [255, 255, 255, 255], IP);
+ assert_eq!(
+ parse_opts(pkt),
+ vec![
+ (53, vec![2]),
+ (54, GW.to_vec()),
+ (51, LEASE.to_be_bytes().to_vec()),
+ (1, MASK.to_vec()),
+ (3, GW.to_vec()),
+ (6, DNS.to_vec()),
+ (26, MTU.to_be_bytes().to_vec()),
+ (252, vec![b'\n']),
+ ],
+ );
+}
+
+#[test]
+fn offer_skips_unset_options() {
+ setup_maps();
+ map_insert(
+ records_map(),
+ &key(),
+ &record(IP, MASK, [0; 4], [0; 4], GW, LEASE, 0),
+ );
+ let (verdict, skb) = Request::default().run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let codes: Vec<u8> = parse_opts(skb.packet()).iter().map(|(c, _)| *c).collect();
+ assert_eq!(codes, vec![53, 54, 51, 1, 252]);
+}
+
+#[test]
+fn matching_request_gets_ack() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[50, 4]);
+ options.extend_from_slice(&IP);
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, MAC, [255, 255, 255, 255], IP);
+ assert_eq!(parse_opts(pkt)[0], (53, vec![5]));
+}
+
+// the pseudo-header sum, the starting value the kernel expects in a checksum it completes on
+// the way out
+fn udp_pseudo(pkt: &[u8]) -> [u8; 2] {
+ let words = [
+ u16::from_be_bytes([pkt[26], pkt[27]]),
+ u16::from_be_bytes([pkt[28], pkt[29]]),
+ u16::from_be_bytes([pkt[30], pkt[31]]),
+ u16::from_be_bytes([pkt[32], pkt[33]]),
+ 17,
+ u16::from_be_bytes([pkt[38], pkt[39]]),
+ ];
+ let mut s: u32 = words.iter().map(|&w| w as u32).sum();
+ while s >> 16 != 0 {
+ s = (s & 0xffff) + (s >> 16);
+ }
+ (s as u16).to_be_bytes()
+}
+
+#[test]
+fn offloaded_request_gets_the_pseudo_header_sum() {
+ setup_full_record();
+ let mut skb = TestSkb::new(&Request::default().build(), IFINDEX);
+ skb.partial = true;
+ assert_eq!(skb.run(tc_dhcp_ingress), TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_eq!(
+ &pkt[40..42],
+ &udp_pseudo(pkt),
+ "udp checksum holds the pseudo-header sum"
+ );
+ assert_ne!(&pkt[40..42], &[0, 0]);
+}
+
+#[test]
+fn removed_record_is_not_answered() {
+ setup_maps();
+ map_insert(records_map(), &key(), &[0u8; 40]);
+ let (verdict, _) = Request::default().run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+}
+
+#[test]
+fn tagged_request_passes() {
+ setup_full_record();
+ let mut skb = TestSkb::new(&Request::default().build(), IFINDEX);
+ skb.tag_vlan();
+ assert_eq!(skb.run(tc_dhcp_ingress), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn trailer_past_the_datagram_is_not_read_as_options() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[50, 4]);
+ options.extend_from_slice(&IP);
+ // no END option, and the frame goes on with bytes shaped like a server
+ // identifier naming another server
+ let (verdict, skb) = Request {
+ options,
+ end: false,
+ trailer: vec![54, 4, 10, 9, 9, 9],
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_eq!(parse_opts(skb.packet())[0], (53, vec![5]));
+}
+
+#[test]
+fn bytes_past_the_udp_datagram_are_not_read_as_options() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[50, 4]);
+ options.extend_from_slice(&IP);
+ // the IP length claims the trailer and the UDP length does not. The
+ // trailer is shaped like a server identifier naming another server
+ let (verdict, skb) = Request {
+ options,
+ end: false,
+ trailer: vec![54, 4, 10, 9, 9, 9],
+ tot_len_excess: 6,
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_eq!(parse_opts(skb.packet())[0], (53, vec![5]));
+}
+
+#[test]
+fn claimed_length_past_the_frame_is_clamped() {
+ // the message type is followed by a second one cut off by the frame. A walk trusting the
+ // claimed length would read past the frame and take the zeroed result as the type
+ setup_full_record();
+ let mut options = DISCOVER.to_vec();
+ options.extend_from_slice(&[53, 1]);
+ let req = Request {
+ options,
+ end: false,
+ tot_len_excess: 1,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_reply_invariants(skb.packet());
+}
+
+#[test]
+fn request_without_options_passes_untouched() {
+ setup_full_record();
+ let req = Request {
+ options: Vec::new(),
+ end: false,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(skb.packet(), &req.build()[..]);
+}
+
+#[test]
+fn stale_request_gets_nak() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[50, 4, 10, 0, 0, 77]);
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, BCAST, [255, 255, 255, 255], [0; 4]);
+ assert_eq!(parse_opts(pkt), vec![(53, vec![6]), (54, GW.to_vec())]);
+}
+
+#[test]
+fn renewal_is_answered_unicast() {
+ setup_full_record();
+ let (verdict, skb) = Request {
+ options: REQUEST.to_vec(),
+ ciaddr: IP,
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, MAC, IP, IP);
+ assert_eq!(parse_opts(pkt)[0], (53, vec![5]));
+}
+
+#[test]
+fn request_selecting_another_server_passes_untouched() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ options.extend_from_slice(&[54, 4, 192, 168, 7, 7]);
+ options.extend_from_slice(&[50, 4, 10, 0, 0, 77]);
+ let req = Request {
+ options,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+}
+
+#[test]
+fn relayed_request_passes_untouched() {
+ setup_full_record();
+ let req = Request {
+ giaddr: [10, 0, 0, 254],
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+}
+
+#[test]
+fn fragments_pass_untouched() {
+ setup_full_record();
+ for frag_off in [[0x20, 0x00], [0x00, 0x01]] {
+ let req = Request {
+ frag_off,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(skb.packet(), &req.build()[..]);
+ }
+
+ let (verdict, _) = Request {
+ frag_off: [0x40, 0x00],
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(verdict, TC_ACT_REDIRECT, "DF alone is not a fragment");
+}
+
+#[test]
+fn broadcast_flag_gets_l2_broadcast_reply() {
+ setup_full_record();
+ let (verdict, skb) = Request {
+ flags: [0x80, 0],
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, BCAST, [255, 255, 255, 255], IP);
+ assert_eq!(&pkt[52..54], &[0x80, 0], "flags echoed");
+
+ // ciaddr wins over the flag, since a renewing client is addressable
+ let (verdict, skb) = Request {
+ flags: [0x80, 0],
+ ciaddr: IP,
+ options: REQUEST.to_vec(),
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_reply_headers(skb.packet(), MAC, IP, IP);
+}
+
+#[test]
+fn interface_without_a_record_passes_untouched() {
+ setup_maps();
+ let req = Request::default();
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+
+ // the record is one interface's, so a request arriving on any other gets nothing from it
+ setup_full_record();
+ let (verdict, skb) = req.run_on(IFINDEX + 1);
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+}
+
+#[test]
+fn another_mac_on_the_interface_passes_untouched() {
+ // a request carrying another MAC than the NIC's is left to the bridge
+ setup_full_record();
+ let other = [0x02, 0x11, 0x22, 0x33, 0x44, 0x55];
+ let req = Request {
+ mac: other,
+ ..Request::default()
+ };
+ let before = req.build();
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(skb.packet(), &before[..], "untouched");
+}
+
+#[test]
+fn non_dhcp_traffic_passes_untouched() {
+ setup_full_record();
+ let variants = [
+ Request {
+ dport: 53,
+ ..Request::default()
+ },
+ Request {
+ ethertype: [0x86, 0xdd],
+ ..Request::default()
+ },
+ Request {
+ ipproto: 6,
+ ..Request::default()
+ },
+ ];
+ for req in variants {
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(skb.packet(), &req.build()[..]);
+ }
+
+ let frame = Request::default().build();
+ let mut skb = TestSkb::new(&frame[..100], IFINDEX);
+ assert_eq!(skb.run(tc_dhcp_ingress), TC_ACT_UNSPEC);
+
+ let mut corrupted = frame.clone();
+ corrupted[278] = 0;
+ let mut skb = TestSkb::new(&corrupted, IFINDEX);
+ assert_eq!(skb.run(tc_dhcp_ingress), TC_ACT_UNSPEC);
+}
+
+#[test]
+fn options_walk_handles_padding_and_early_end() {
+ setup_full_record();
+
+ let mut options = vec![0, 0, 0];
+ options.extend_from_slice(DISCOVER);
+ let (verdict, _) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+
+ let mut options = vec![255];
+ options.extend_from_slice(DISCOVER);
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(
+ verdict, TC_ACT_UNSPEC,
+ "message type after end is not parsed"
+ );
+ assert_eq!(redirected(), None);
+ drop(skb);
+}
+
+#[test]
+fn stale_renewal_gets_broadcast_nak() {
+ setup_full_record();
+ let (verdict, skb) = Request {
+ options: REQUEST.to_vec(),
+ ciaddr: [10, 0, 0, 77],
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, BCAST, [255, 255, 255, 255], [0; 4]);
+ assert_eq!(parse_opts(pkt), vec![(53, vec![6]), (54, GW.to_vec())]);
+}
+
+#[test]
+fn request_without_address_gets_nak() {
+ setup_full_record();
+ let (verdict, skb) = Request {
+ options: REQUEST.to_vec(),
+ ..Request::default()
+ }
+ .run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert_reply_headers(pkt, BCAST, [255, 255, 255, 255], [0; 4]);
+ assert_eq!(parse_opts(pkt)[0], (53, vec![6]));
+}
+
+#[test]
+fn realistic_request_is_answered_and_shrunk() {
+ setup_full_record();
+ let mut options = REQUEST.to_vec();
+ // a client identifier naming something else is ignored and the chaddr decides
+ options.extend_from_slice(&[61, 7, 1, 0xde, 0xad, 0xbe, 0xef, 0x00, 0x01]);
+ options.extend_from_slice(&[50, 4]);
+ options.extend_from_slice(&IP);
+ options.extend_from_slice(&[54, 4]);
+ options.extend_from_slice(&GW);
+ options.extend_from_slice(&[12, 14]);
+ options.extend_from_slice(b"dhcp-client-01");
+ options.extend_from_slice(&[60, 32]);
+ options.extend_from_slice(b"PXEClient:Arch:00007:UNDI:003016");
+ options.extend_from_slice(&[55, 20]);
+ options.extend_from_slice(&[
+ 1, 2, 3, 6, 12, 15, 17, 26, 28, 40, 41, 42, 51, 54, 58, 59, 66, 67, 119, 121,
+ ]);
+ options.extend_from_slice(&[57, 2, 0x05, 0xdc]);
+ options.extend_from_slice(&[93, 2, 0, 7]);
+ options.extend_from_slice(&[94, 3, 1, 3, 16]);
+ options.extend_from_slice(&[97, 17, 0]);
+ options.extend_from_slice(&[0x11; 16]);
+ let req = Request {
+ options,
+ ..Request::default()
+ };
+ let request_len = req.build().len();
+ let (verdict, skb) = req.run();
+
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ let pkt = skb.packet();
+ assert!(pkt.len() < request_len, "reply is shorter than the request");
+ assert_eq!(
+ pkt.len(),
+ 14 + 20 + 8 + 300,
+ "trimmed to the padded minimum"
+ );
+ assert_reply_headers(pkt, MAC, [255, 255, 255, 255], IP);
+ assert_eq!(parse_opts(pkt)[0], (53, vec![5]));
+}
+
+#[test]
+fn other_message_types_pass_untouched() {
+ setup_full_record();
+ for msgtype in [4, 7, 8] {
+ let req = Request {
+ options: vec![53, 1, msgtype],
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC, "message type {msgtype}");
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+ }
+}
+
+#[test]
+fn foreign_bootp_passes_untouched() {
+ setup_full_record();
+ let variants = [
+ // a reply coming in from the guest side, a guest running its own server
+ Request {
+ op: 2,
+ ..Request::default()
+ },
+ Request {
+ htype: 6,
+ ..Request::default()
+ },
+ Request {
+ hlen: 16,
+ ..Request::default()
+ },
+ Request {
+ ip_options: vec![1, 1, 1, 1],
+ ..Request::default()
+ },
+ ];
+ for req in variants {
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+ }
+}
+
+#[test]
+fn truncated_option_ends_the_walk() {
+ setup_full_record();
+
+ // a length running past the packet before the message type leaves nothing to answer
+ let mut options = vec![12, 200];
+ options.extend_from_slice(DISCOVER);
+ let req = Request {
+ options,
+ ..Request::default()
+ };
+ let (verdict, skb) = req.run();
+ assert_eq!(verdict, TC_ACT_UNSPEC);
+ assert_eq!(redirected(), None);
+ assert_eq!(skb.packet(), &req.build()[..]);
+
+ // after the message type it only cuts the walk short, and what was read still counts
+ let mut options = DISCOVER.to_vec();
+ options.extend_from_slice(&[12, 200]);
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+ assert_eq!(verdict, TC_ACT_REDIRECT);
+ assert_reply_headers(skb.packet(), MAC, [255, 255, 255, 255], IP);
+}
+
+#[test]
+fn arbitrary_option_bytes_never_break_the_reply() {
+ setup_full_record();
+ let mut lcg = 0x2545f4914f6cdd1du64;
+ let mut rand = move || {
+ lcg = lcg
+ .wrapping_mul(6364136223846793005)
+ .wrapping_add(1442695040888963407);
+ (lcg >> 33) as u8
+ };
+
+ for _ in 0..500 {
+ let len = (rand() % 60) as usize;
+ let options: Vec<u8> = (0..len).map(|_| rand()).collect();
+ let (verdict, skb) = Request {
+ options,
+ ..Request::default()
+ }
+ .run();
+ assert!(verdict == TC_ACT_UNSPEC || verdict == TC_ACT_REDIRECT);
+ if verdict == TC_ACT_REDIRECT {
+ assert_reply_invariants(skb.packet());
+ }
+ }
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH proxmox-ebpf v4 02/20] dhcp: add the subsystem that loads the responder and keeps its records
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
2026-10-21 8:44 ` [PATCH proxmox-ebpf v4 01/20] dhcp: add the per-tap DHCPv4 responder program Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH proxmox-perl-rs v4 03/20] pve-rs: sdn: add bindings for the eBPF DHCP responder Hannes Laimer
` (17 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
The subsystem loads the program, pins a map with one entry per guest
interface and attaches the program to the interfaces it serves.
Callers run a full pass over every guest interface, plug or unplug one
interface, or change the records of a guest's interfaces.
A load can leave the map empty, and only a full pass is handed every
guest interface. So the full pass is the only step that loads. A plug
or a record change that finds nothing loaded reports that a full pass
has to run first.
A hotplug moves a tap before the guest config names the new bridge,
and no lock closes that gap. So every entry names the vnet its writer
saw the interface on. A pass or the record change of a mapping edit
leaves an entry naming another vnet alone. A plug writes outright, so
plugs share the lock. The edit's record change reads the entry before
it writes, so it takes the lock exclusively.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Notes:
v4:
* use if_nameindex instead of if_nametoindex for the list of NICs
* say "could not" in the warnings about a failed record write or removal
src/dhcp/mod.rs | 1037 +++++++++++++++++++++++++++++++++++++++++++++
src/dhcp/types.rs | 64 +++
src/lib.rs | 3 +
src/subsystem.rs | 2 +-
4 files changed, 1105 insertions(+), 1 deletion(-)
create mode 100644 src/dhcp/mod.rs
create mode 100644 src/dhcp/types.rs
diff --git a/src/dhcp/mod.rs b/src/dhcp/mod.rs
new file mode 100644
index 0000000..868be4e
--- /dev/null
+++ b/src/dhcp/mod.rs
@@ -0,0 +1,1037 @@
+//! The dhcp subsystem, a per-tap DHCPv4 responder answering from a pinned map of one entry per
+//! interface.
+//!
+//! The map holds nothing of its own. The SDN side hands in every guest interface with the answer
+//! it gets, if any. A full pass makes the kernel match that for the interfaces it names. A record
+//! change or a plug writes its own interfaces only. A linked interface the pass does not name
+//! keeps its link and its entry. A plug runs under the shared lock and writes its entry whatever
+//! is there. A pass runs under the exclusive lock, and so does the record change an edit pushes,
+//! since it reads the entry's vnet before it writes. A guest's own start holds the guest lock, so
+//! nothing else plugs its interfaces meanwhile and it writes its records under the shared lock
+//! without a read.
+//!
+//! A hotplug plugs the tap before the guest config names the new bridge, and no lock orders that.
+//! So every entry names the vnet its writer saw the interface on. A pass or an edit's record change
+//! leaves an entry alone that names another vnet than the configs do, and the link follows such an
+//! entry. A plug replaces it. Plugs of one interface are serialized by the guest lock, so the plug
+//! is the latest word on where its interface sits. An unplug drops the interface's entry. An
+//! interface that went away without a link keeps its entry until the next pass. One plugged onto a
+//! bridge this subsystem does not serve trades its entry for a marker naming that place.
+mod types;
+
+use std::collections::{HashMap, HashSet};
+use std::io::ErrorKind;
+use std::net::Ipv4Addr;
+
+use anyhow::{Context, bail};
+use aya::maps::MapError;
+
+use self::types::*;
+use crate::subsystem::{ApplyLock, Exclusive, NotLoaded, Shared, TcPrograms};
+use crate::tc::{Direction, LinkPin};
+use crate::tc_programs;
+
+/// One guest interface as the configs know it. `serve` says whether a pass links it. A plug
+/// links its interface regardless. `record` is the answer it gets there, and without one the
+/// interface gets no answer. `vnet` is the vnet it sits on, none for a plain bridge.
+pub struct Iface {
+ pub name: String,
+ pub vnet: Option<String>,
+ pub serve: bool,
+ pub record: Option<Record>,
+}
+
+/// One interface's answer, the address and every option the reply carries.
+pub struct Record {
+ pub ip: Ipv4Addr,
+ pub prefixlen: u8,
+ pub server_id: Ipv4Addr,
+ pub lease: u32,
+ pub router: Option<Ipv4Addr>,
+ pub dns: Option<Ipv4Addr>,
+ pub mtu: Option<u16>,
+ pub mac: [u8; 6],
+}
+
+/// What became of an unplugged interface.
+pub enum Unplugged {
+ /// Gone, or going with the unplug.
+ Gone,
+ /// Plugged onto a bridge this subsystem does not serve, the vnet it sits on now, none for a
+ /// plain bridge.
+ MovedTo(Option<String>),
+}
+
+/// What a single-entry step did.
+#[derive(Debug, PartialEq, Eq)]
+pub enum Outcome {
+ /// Nothing is left to do. The entry is in place, or its interface is not here.
+ Done,
+ /// Nothing is loaded to write into. Only a full pass loads, so the caller runs one and retries.
+ NeedsFullPass,
+}
+
+const RECORDS_MAP: &str = "dhcp_iface_records";
+const MAPS: [&str; 1] = [RECORDS_MAP];
+
+// bump with any change to the key or value layout of a map in dhcp.bpf.c
+const SCHEMA_VERSION: u32 = 1;
+
+// requests only arrive from the guest side and replies leave through a redirect, so only ingress
+// carries a program
+const PROGRAM_NAMES: [(Direction, &str); 1] = [(Direction::Ingress, "tc_dhcp_ingress")];
+
+type RecordsMap = aya::maps::HashMap<aya::maps::MapData, u32, DhcpRecord>;
+
+static PROGRAMS: TcPrograms = tc_programs! {
+ name: "dhcp",
+ programs: &PROGRAM_NAMES,
+ maps: &MAPS,
+ schema_version: SCHEMA_VERSION,
+};
+
+pub struct DhcpSubsystem {
+ programs: &'static TcPrograms,
+}
+
+impl Default for DhcpSubsystem {
+ fn default() -> Self {
+ Self::new()
+ }
+}
+
+impl DhcpSubsystem {
+ pub fn new() -> Self {
+ Self {
+ programs: &PROGRAMS,
+ }
+ }
+
+ /// The lock a plug runs under, held until [`attach`](SharedLock::attach) returns. Plugs run
+ /// next to each other and wait only behind an exclusive holder.
+ pub fn lock_shared(&self) -> anyhow::Result<SharedLock> {
+ Ok(SharedLock {
+ lock: self.programs.lock_shared()?,
+ })
+ }
+
+ /// The lock a full pass or a record change runs under, held until the change is written.
+ pub fn lock_exclusive(&self) -> anyhow::Result<ExclusiveLock> {
+ Ok(ExclusiveLock {
+ lock: self.programs.lock_exclusive()?,
+ })
+ }
+
+ /// A tap unplug. The interface may be gone by then, so its link is found by the name the pin
+ /// carries. [`Unplugged`] decides what becomes of the entry.
+ pub fn detach(&self, iface: &str, unplugged: Unplugged) -> anyhow::Result<()> {
+ // resolved before the lock wait, an interface recreated under the name meanwhile is not
+ // this unplug's and keeps its link
+ let now = ifindex_of(iface)?;
+ // every guest unplug on the host lands here. Most never had a link and must not wait
+ // behind a pass. One without a link whose interface is gone leaves its entry to the
+ // next pass
+ if !self.programs.link_pinned(iface) && now.is_none() {
+ return Ok(());
+ }
+ let lock = self.programs.lock_shared()?;
+
+ if !lock.is_current() {
+ log::debug!("dhcp detach: nothing current, {iface} is left to the next full pass");
+ return Ok(());
+ }
+ // the map is opened and the marker built before the links go, so a failure there
+ // leaves them attached
+ let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?;
+ let marker = match &unplugged {
+ Unplugged::Gone => None,
+ Unplugged::MovedTo(vnet) => Some(DhcpRecord::removed(place(vnet.as_deref())?)),
+ };
+ // an interface without a link has an entry as well, a marker where it is not served
+ let mut ifindexes = lock.detach_named(iface, |pin| {
+ Some(pin.ifindex) != now && pin_alive(pin).unwrap_or(false)
+ })?;
+ if let Some(now) = now.filter(|now| !ifindexes.contains(now)) {
+ ifindexes.push(now);
+ }
+ // the marker is the moved interface's. Older incarnations under the name are gone
+ unplug_entries(&mut map, &ifindexes, now.zip(marker))
+ }
+}
+
+/// The shared lock of the responder, what a plug holds, and the record change of a guest's own
+/// start.
+pub struct SharedLock {
+ lock: ApplyLock<'static, Shared>,
+}
+
+impl SharedLock {
+ /// A tap plug, the entry goes in and the program on. The plug loads nothing, so with nothing
+ /// pinned the caller runs a full pass first.
+ pub fn attach(&self, iface: &Iface) -> anyhow::Result<Outcome> {
+ if !self.lock.is_current() {
+ return Ok(Outcome::NeedsFullPass);
+ }
+ let Some(ifindex) = ifindex_of(iface.name.as_str())? else {
+ log::info!("dhcp attach: {} is gone, nothing to do", iface.name);
+ return Ok(Outcome::Done);
+ };
+ // the entry goes in before the program runs, a marker when there is no record or one
+ // the program cannot carry. A guest start must not fail on a bad record
+ let mut map: RecordsMap = self.lock.hash_map(RECORDS_MAP)?;
+ let entry = match iface.entry() {
+ Ok(entry) => entry,
+ Err(e) => {
+ log::warn!(
+ "dhcp: {} attaches with nothing to answer: {e:#}",
+ iface.name
+ );
+ DhcpRecord::removed(iface.place()?)
+ }
+ };
+ write_entry(&mut map, ifindex, &entry, Elsewhere::Override)?;
+ match self.lock.attach_iface(ifindex) {
+ Ok(()) => Ok(Outcome::Done),
+ Err(e) if e.downcast_ref::<NotLoaded>().is_some() => Ok(Outcome::NeedsFullPass),
+ Err(e) => Err(e).with_context(|| format!("dhcp responder for {}", iface.name)),
+ }
+ }
+
+ /// A record change for a caller that holds the guest lock. Nothing else plugs the guest's
+ /// interfaces under that lock, so each entry is written outright, one atomic write, without
+ /// reading what it holds.
+ pub fn update(&self, ifaces: &[Iface]) -> anyhow::Result<Outcome> {
+ update_entries(&self.lock, ifaces, Elsewhere::Override)
+ }
+}
+
+/// The exclusive lock of the responder, what a full pass and a record change hold.
+pub struct ExclusiveLock {
+ lock: ApplyLock<'static, Exclusive>,
+}
+
+impl ExclusiveLock {
+ /// The full pass. Afterward every interface the configs name that exists here has its entry,
+ /// a record where it is served with one and a marker otherwise, and a link is on every
+ /// served interface and on every interface whose entry names another vnet. Entries of
+ /// interfaces that are gone are dropped, as are entries of live interfaces the configs do
+ /// not name and that have no link. Bad input leaves its interface as it is, with a warning.
+ pub fn apply(&self, ifaces: &[Iface]) -> anyhow::Result<()> {
+ self.lock.ensure_loaded()?;
+ let present = present_ifindexes(ifaces)?;
+ // a link the configs do not name stays while its interface lives under the pin's
+ // name. A reused index is not that interface
+ let mut unknown_alive = HashSet::new();
+ for pin in self.lock.attached()? {
+ if !present.contains_key(&pin.ifindex) && pin_alive(&pin)? {
+ unknown_alive.insert(pin.ifindex);
+ }
+ }
+ let mut map: RecordsMap = self.lock.hash_map(RECORDS_MAP)?;
+ let (elsewhere, failed) = sync_records(&mut map, &present, &unknown_alive)?;
+ // a failed write or removal leaves its entry as it is. The links are still put right
+ // and every failure is reported once the rest is done
+ let links = self
+ .lock
+ .reconcile(&wanted_links(&present, &elsewhere, &unknown_alive));
+ match (failed, links) {
+ (0, links) => links,
+ (failed, Ok(())) => bail!("{failed} entries could not be changed"),
+ (failed, Err(e)) => {
+ Err(e).with_context(|| format!("{failed} entries could not be changed"))
+ }
+ }
+ }
+
+ /// A record change. Interfaces not found here are skipped. The rest are written whether they
+ /// have a link yet or not. A record the reply cannot carry leaves its interface as it is.
+ /// With nothing loaded the caller runs a full pass first, like a plug.
+ pub fn update(&self, ifaces: &[Iface]) -> anyhow::Result<Outcome> {
+ update_entries(&self.lock, ifaces, Elsewhere::Skip)
+ }
+
+ /// Detach everywhere and drop the pinned state, for the last zone leaving the backend.
+ pub fn clear(&self) -> anyhow::Result<()> {
+ self.lock.clear()
+ }
+}
+
+fn update_entries<M>(
+ lock: &ApplyLock<'static, M>,
+ ifaces: &[Iface],
+ elsewhere: Elsewhere,
+) -> anyhow::Result<Outcome> {
+ if !lock.is_current() {
+ return Ok(Outcome::NeedsFullPass);
+ }
+ let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?;
+ let mut failed = 0usize;
+ for iface in ifaces {
+ let ifindex = match ifindex_of(iface.name.as_str()) {
+ Ok(Some(ifindex)) => ifindex,
+ Ok(None) => continue,
+ Err(e) => {
+ log::warn!("dhcp: {e:#}");
+ failed += 1;
+ continue;
+ }
+ };
+ let entry = match iface.entry() {
+ Ok(entry) => entry,
+ Err(e) => {
+ log::warn!("dhcp: {} is left as it is: {e:#}", iface.name);
+ continue;
+ }
+ };
+ if let Err(e) = write_entry(&mut map, ifindex, &entry, elsewhere) {
+ log::warn!("dhcp: could not write the entry of {}: {e:#}", iface.name);
+ failed += 1;
+ }
+ }
+ if failed > 0 {
+ bail!("{failed} interfaces could not be updated");
+ }
+ Ok(Outcome::Done)
+}
+
+const NO_VNET: [u8; 8] = [0; 8];
+
+// the map update flag that refuses a key the map does not hold, from the kernel's uapi bpf.h
+const BPF_EXIST: u64 = 2;
+
+fn unplug_entries(
+ map: &mut impl RecordStore,
+ ifindexes: &[u32],
+ moved: Option<(u32, DhcpRecord)>,
+) -> anyhow::Result<()> {
+ for &ifindex in ifindexes {
+ match moved {
+ // only an existing entry becomes a marker, so an interface without one gets none
+ Some((current, marker)) if current == ifindex => map.mark(ifindex, marker)?,
+ _ => map.remove(ifindex)?,
+ }
+ }
+ Ok(())
+}
+
+// the vnet id padded with zeros, all zero for none
+fn place(vnet: Option<&str>) -> anyhow::Result<[u8; 8]> {
+ let Some(vnet) = vnet else {
+ return Ok(NO_VNET);
+ };
+ let bytes = vnet.as_bytes();
+ if bytes.is_empty() || bytes.len() > NO_VNET.len() {
+ bail!("vnet id {vnet:?} does not fit a record");
+ }
+ let mut place = NO_VNET;
+ place[..bytes.len()].copy_from_slice(bytes);
+ Ok(place)
+}
+
+// the record map as the writers see it, so their decisions run against a plain map in tests
+trait RecordStore {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>>;
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()>;
+ // replace an entry that exists and do nothing for one that does not, in one step, so it is
+ // safe under the shared lock
+ fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()>;
+ // a missing entry is removed already
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()>;
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>>;
+}
+
+impl RecordStore for RecordsMap {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
+ match aya::maps::HashMap::get(self, &ifindex, 0) {
+ Ok(live) => Ok(Some(live)),
+ Err(MapError::KeyNotFound) => Ok(None),
+ Err(e) => Err(e.into()),
+ }
+ }
+
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ aya::maps::HashMap::insert(self, ifindex, rec, 0)?;
+ Ok(())
+ }
+
+ fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ match aya::maps::HashMap::insert(self, ifindex, rec, BPF_EXIST) {
+ Ok(()) => Ok(()),
+ // the kernel answers a missing key with ENOENT, and aya passes that through as is
+ Err(MapError::SyscallError(e)) if e.io_error.kind() == ErrorKind::NotFound => Ok(()),
+ Err(e) => Err(e.into()),
+ }
+ }
+
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
+ match aya::maps::HashMap::remove(self, &ifindex) {
+ Ok(()) => Ok(()),
+ // the kernel answers a missing key with ENOENT, and aya passes that through as is
+ Err(MapError::SyscallError(e)) if e.io_error.kind() == ErrorKind::NotFound => Ok(()),
+ Err(e) => Err(e.into()),
+ }
+ }
+
+ // a record the listing cannot read is one the pass cannot compare against, so the pass
+ // fails rather than writes past it
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
+ self.iter()
+ .collect::<Result<Vec<_>, _>>()
+ .context("listing the records")
+ }
+}
+
+// what a write does with an entry naming another vnet than its own
+#[derive(Clone, Copy)]
+enum Elsewhere {
+ // a record change, its configs are stale for the interface
+ Skip,
+ // a plug, or a record change under the guest lock, both see where the interface sits now
+ Override,
+}
+
+fn write_entry(
+ map: &mut impl RecordStore,
+ ifindex: u32,
+ rec: &DhcpRecord,
+ elsewhere: Elsewhere,
+) -> anyhow::Result<()> {
+ if matches!(elsewhere, Elsewhere::Skip)
+ && let Some(live) = map.get(ifindex)?
+ && live.vnet != rec.vnet
+ {
+ log::debug!("dhcp: ifindex {ifindex} sits elsewhere than the configs say, leaving it");
+ return Ok(());
+ }
+ map.insert(ifindex, *rec)
+}
+
+// entries of live links the pass does not know stay, since their plug wrote them. The other
+// unknown ones go with their interface. Returns the interfaces whose entry names another vnet
+// than the configs, and how many writes failed
+fn sync_records(
+ map: &mut impl RecordStore,
+ present: &HashMap<u32, &Iface>,
+ unknown_alive: &HashSet<u32>,
+) -> anyhow::Result<(HashSet<u32>, usize)> {
+ let live: HashMap<u32, DhcpRecord> = map.entries()?.into_iter().collect();
+ let dead: Vec<u32> = live
+ .keys()
+ .filter(|ifindex| !present.contains_key(ifindex) && !unknown_alive.contains(ifindex))
+ .copied()
+ .collect();
+ let mut failed = 0usize;
+ for &ifindex in &dead {
+ if let Err(e) = map.remove(ifindex) {
+ log::warn!("dhcp: could not remove the record of ifindex {ifindex}: {e:#}");
+ failed += 1;
+ }
+ }
+ let mut written = 0usize;
+ let mut elsewhere = HashSet::new();
+
+ for (ifindex, iface) in present {
+ // bad input leaves its interface as it is and the pass goes on
+ let desired = match iface.entry() {
+ Ok(desired) => desired,
+ Err(e) => {
+ log::warn!("dhcp: {} is left as it is: {e:#}", iface.name);
+ continue;
+ }
+ };
+ match live.get(ifindex) {
+ Some(entry) if entry.vnet != desired.vnet => {
+ elsewhere.insert(*ifindex);
+ }
+
+ Some(entry) if *entry == desired => {}
+ _ => match map.insert(*ifindex, desired) {
+ Ok(()) => written += 1,
+ Err(e) => {
+ log::warn!("dhcp: could not write the entry of {}: {e:#}", iface.name);
+ failed += 1;
+ }
+ },
+ }
+ }
+
+ if written + dead.len() + elsewhere.len() > 0 {
+ log::info!(
+ "dhcp: {written} entries written, {} removed, {} left where their plug put them",
+ dead.len(),
+ elsewhere.len()
+ );
+ } else {
+ log::debug!("dhcp: {} records, no changes", live.len());
+ }
+ Ok((elsewhere, failed))
+}
+
+// the interface's index here, none when it does not exist. Any other failure says nothing about
+// the interface, and a step taking it for gone would detach what it could not resolve
+fn ifindex_of(name: &str) -> anyhow::Result<Option<u32>> {
+ match nix::net::if_::if_nametoindex(name) {
+ Ok(ifindex) => Ok(Some(ifindex)),
+ Err(nix::errno::Errno::ENODEV) => Ok(None),
+ Err(e) => Err(e).with_context(|| format!("resolve interface {name}")),
+ }
+}
+
+// the named interfaces that exist here by index, the names come from configs cluster-wide and
+// most belong to other nodes
+fn present_ifindexes(ifaces: &[Iface]) -> anyhow::Result<HashMap<u32, &Iface>> {
+ let local = nix::net::if_::if_nameindex().context("list interfaces")?;
+ let by_name: HashMap<&str, u32> = local
+ .iter()
+ .filter_map(|i| i.name().to_str().ok().map(|name| (name, i.index())))
+ .collect();
+ Ok(ifaces
+ .iter()
+ .filter_map(|iface| {
+ by_name
+ .get(iface.name.as_str())
+ .map(|&ifindex| (ifindex, iface))
+ })
+ .collect())
+}
+
+// the link follows the configs where they agree with the entry. An entry naming another vnet
+// keeps its link, and a served interface without a mapping gets one. So a record change filling
+// a marker attaches nothing itself. A live link the configs do not know stays
+fn wanted_links(
+ present: &HashMap<u32, &Iface>,
+ elsewhere: &HashSet<u32>,
+ unknown_alive: &HashSet<u32>,
+) -> HashSet<u32> {
+ let mut wanted: HashSet<u32> = present
+ .iter()
+ .filter(|(ifindex, iface)| elsewhere.contains(ifindex) || iface.serve)
+ .map(|(ifindex, _)| *ifindex)
+ .collect();
+
+ wanted.extend(unknown_alive);
+ wanted
+}
+
+// whether the link's interface still exists under the name and index its pin carries
+fn pin_alive(pin: &LinkPin) -> anyhow::Result<bool> {
+ Ok(ifindex_of(pin.name.as_str())? == Some(pin.ifindex))
+}
+
+impl Iface {
+ // a marker where nothing is served
+ fn entry(&self) -> anyhow::Result<DhcpRecord> {
+ let vnet = self.place()?;
+ match &self.record {
+ Some(record) if self.serve => record
+ .entry(vnet)
+ .with_context(|| format!("record for {}", self.name)),
+ _ => Ok(DhcpRecord::removed(vnet)),
+ }
+ }
+
+ fn place(&self) -> anyhow::Result<[u8; 8]> {
+ place(self.vnet.as_deref()).with_context(|| format!("interface {}", self.name))
+ }
+}
+
+impl Record {
+ // refused here is what the reply cannot carry. Zero stands for absent in the entry, so an
+ // address or an MTU of zero cannot be told from none
+ fn entry(&self, vnet: [u8; 8]) -> anyhow::Result<DhcpRecord> {
+ if self.prefixlen == 0 || self.prefixlen > 32 {
+ bail!("prefixlen {} out of range", self.prefixlen);
+ }
+ if self.ip.is_unspecified() {
+ bail!("no address");
+ }
+ if self.server_id.is_unspecified() {
+ bail!("no server address");
+ }
+ if self.lease == 0 {
+ bail!("lease of zero seconds");
+ }
+ if self.mtu.is_some_and(|mtu| mtu < 68) {
+ bail!("mtu below the IPv4 minimum of 68");
+ }
+ if self.mac == [0; 6] {
+ bail!("no MAC");
+ }
+ let netmask = u32::MAX << (32 - self.prefixlen);
+ Ok(DhcpRecord {
+ ip: u32::from(self.ip).to_be(),
+ netmask: netmask.to_be(),
+ router: u32::from(self.router.unwrap_or(Ipv4Addr::UNSPECIFIED)).to_be(),
+ dns: u32::from(self.dns.unwrap_or(Ipv4Addr::UNSPECIFIED)).to_be(),
+ server_id: u32::from(self.server_id).to_be(),
+ lease: self.lease.to_be(),
+ mtu: self.mtu.unwrap_or(0).to_be(),
+ vnet,
+ mac: self.mac,
+ })
+ }
+}
+
+#[cfg(test)]
+mod test {
+ use std::collections::BTreeMap;
+
+ use super::*;
+
+ fn record() -> Record {
+ Record {
+ ip: Ipv4Addr::new(10, 0, 0, 100),
+ prefixlen: 24,
+ server_id: Ipv4Addr::new(10, 0, 0, 1),
+ lease: 300,
+ router: Some(Ipv4Addr::new(10, 0, 0, 1)),
+ dns: Some(Ipv4Addr::new(1, 1, 1, 1)),
+ mtu: Some(1500),
+ mac: MAC,
+ }
+ }
+
+ const MAC: [u8; 6] = [0xda, 0x65, 0x8f, 0x18, 0x9b, 0x6f];
+
+ const VNET: [u8; 8] = *b"vnet1\0\0\0";
+
+ #[test]
+ fn converts_full_record() {
+ let rec = record().entry(VNET).unwrap();
+ assert_eq!(
+ u32::from_be(rec.ip),
+ u32::from(Ipv4Addr::new(10, 0, 0, 100))
+ );
+ assert_eq!(u32::from_be(rec.netmask), 0xffffff00);
+ assert_eq!(
+ u32::from_be(rec.router),
+ u32::from(Ipv4Addr::new(10, 0, 0, 1))
+ );
+ assert_eq!(u32::from_be(rec.dns), u32::from(Ipv4Addr::new(1, 1, 1, 1)));
+ assert_eq!(
+ u32::from_be(rec.server_id),
+ u32::from(Ipv4Addr::new(10, 0, 0, 1))
+ );
+ assert_eq!(u32::from_be(rec.lease), 300);
+ assert_eq!(u16::from_be(rec.mtu), 1500);
+ assert_eq!(rec.vnet, VNET);
+ assert_eq!(rec.mac, MAC);
+
+ let rec = Record {
+ router: None,
+ dns: None,
+ mtu: None,
+ ..record()
+ }
+ .entry(VNET)
+ .unwrap();
+ assert_eq!(rec.router, 0);
+ assert_eq!(rec.dns, 0);
+ assert_eq!(rec.mtu, 0);
+ }
+
+ #[test]
+ fn entries_name_the_vnet_of_the_interface() {
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("testvnet".into()),
+ serve: true,
+ record: Some(record()),
+ };
+ assert_eq!(&iface.entry().unwrap().vnet, b"testvnet");
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: None,
+ serve: false,
+ record: None,
+ };
+ assert_eq!(iface.entry().unwrap().vnet, NO_VNET);
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("ninechars".into()),
+ serve: true,
+ record: Some(record()),
+ };
+ assert!(iface.entry().is_err());
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some(String::new()),
+ serve: true,
+ record: Some(record()),
+ };
+ assert!(iface.entry().is_err());
+ }
+
+ #[test]
+ fn unserved_interfaces_get_a_marker() {
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("vnet1".into()),
+ serve: false,
+ record: Some(record()),
+ };
+ assert_eq!(iface.entry().unwrap().ip, 0);
+ assert_eq!(iface.entry().unwrap().vnet, VNET);
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("vnet1".into()),
+ serve: true,
+ record: None,
+ };
+ assert_eq!(iface.entry().unwrap().ip, 0);
+ let iface = Iface {
+ name: "tap100i0".into(),
+ vnet: Some("vnet1".into()),
+ serve: true,
+ record: Some(record()),
+ };
+ assert_ne!(iface.entry().unwrap().ip, 0);
+ }
+
+ #[test]
+ fn refuses_what_the_reply_cannot_carry() {
+ for rec in [
+ Record {
+ prefixlen: 33,
+ ..record()
+ },
+ Record {
+ prefixlen: 0,
+ ..record()
+ },
+ Record {
+ server_id: Ipv4Addr::UNSPECIFIED,
+ ..record()
+ },
+ Record {
+ lease: 0,
+ ..record()
+ },
+ Record {
+ ip: Ipv4Addr::UNSPECIFIED,
+ ..record()
+ },
+ Record {
+ mtu: Some(67),
+ ..record()
+ },
+ Record {
+ mac: [0; 6],
+ ..record()
+ },
+ ] {
+ assert!(rec.entry(VNET).is_err());
+ }
+ let rec = Record {
+ mtu: Some(68),
+ ..record()
+ };
+ assert_eq!(u16::from_be(rec.entry(VNET).unwrap().mtu), 68);
+
+ let rec = Record {
+ prefixlen: 32,
+ ..record()
+ };
+ assert_eq!(u32::from_be(rec.entry(VNET).unwrap().netmask), u32::MAX);
+ let rec = Record {
+ prefixlen: 1,
+ ..rec
+ };
+ assert_eq!(u32::from_be(rec.entry(VNET).unwrap().netmask), 1 << 31);
+ }
+
+ impl RecordStore for BTreeMap<u32, DhcpRecord> {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
+ Ok(BTreeMap::get(self, &ifindex).copied())
+ }
+
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ BTreeMap::insert(self, ifindex, rec);
+ Ok(())
+ }
+
+ fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ if let Some(entry) = BTreeMap::get_mut(self, &ifindex) {
+ *entry = rec;
+ }
+ Ok(())
+ }
+
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
+ BTreeMap::remove(self, &ifindex);
+ Ok(())
+ }
+
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
+ Ok(self.iter().map(|(k, v)| (*k, *v)).collect())
+ }
+ }
+
+ const OTHER: [u8; 8] = *b"vnet2\0\0\0";
+
+ fn entry(vnet: [u8; 8]) -> DhcpRecord {
+ record().entry(vnet).unwrap()
+ }
+
+ fn other_answer(vnet: [u8; 8]) -> DhcpRecord {
+ Record {
+ ip: Ipv4Addr::new(10, 0, 0, 200),
+ ..record()
+ }
+ .entry(vnet)
+ .unwrap()
+ }
+
+ #[test]
+ fn a_record_change_writes_on_its_own_vnet() {
+ let mut map = BTreeMap::new();
+ write_entry(&mut map, 1, &entry(VNET), Elsewhere::Skip).unwrap();
+ assert_eq!(map[&1], entry(VNET));
+ write_entry(&mut map, 1, &other_answer(VNET), Elsewhere::Skip).unwrap();
+ assert_eq!(map[&1], other_answer(VNET));
+ // an entry naming another vnet was placed by a plug and stays
+ map.insert(1, entry(OTHER));
+ write_entry(&mut map, 1, &other_answer(VNET), Elsewhere::Skip).unwrap();
+ assert_eq!(map[&1], entry(OTHER));
+ }
+
+ #[test]
+ fn a_plug_writes_whatever_is_there() {
+ let mut map = BTreeMap::new();
+ write_entry(&mut map, 1, &entry(VNET), Elsewhere::Override).unwrap();
+ assert_eq!(map[&1], entry(VNET));
+ map.insert(1, entry(OTHER));
+ write_entry(&mut map, 1, &other_answer(VNET), Elsewhere::Override).unwrap();
+ assert_eq!(map[&1], other_answer(VNET));
+ }
+
+ #[test]
+ fn an_unplug_removes_or_marks_the_entries() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(VNET));
+ map.insert(2, entry(VNET));
+ map.insert(3, entry(VNET));
+ unplug_entries(&mut map, &[1, 2], None).unwrap();
+ assert_eq!(map.keys().copied().collect::<Vec<_>>(), [3]);
+ // a missing entry is removed already
+ unplug_entries(&mut map, &[1], None).unwrap();
+ // a move marks the interface that moved, and an older incarnation under its name is gone
+ map.insert(4, entry(VNET));
+ let marker = DhcpRecord::removed(NO_VNET);
+ unplug_entries(&mut map, &[3, 4], Some((4, marker))).unwrap();
+ assert_eq!(map.keys().copied().collect::<Vec<_>>(), [4]);
+ assert_eq!(map[&4].ip, 0);
+ assert_eq!(map[&4].vnet, NO_VNET);
+ // an interface without an entry gets no marker
+ unplug_entries(&mut map, &[5], Some((5, marker))).unwrap();
+ assert!(!map.contains_key(&5));
+ }
+
+ fn iface(name: &str, vnet: Option<&str>, serve: bool, record: Option<Record>) -> Iface {
+ Iface {
+ name: name.into(),
+ vnet: vnet.map(String::from),
+ serve,
+ record,
+ }
+ }
+
+ #[test]
+ fn the_pass_syncs_the_records_it_knows() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(VNET)); // same answer as the config
+ map.insert(2, other_answer(VNET)); // a changed answer
+ map.insert(4, entry(VNET)); // not served anymore
+ map.insert(6, entry(VNET)); // its interface is gone
+ map.insert(7, entry(VNET)); // a live link the configs do not name
+ map.insert(8, entry(VNET)); // its mapping is gone
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(record())),
+ iface("b", Some("vnet1"), true, Some(record())),
+ iface("d", Some("vnet1"), false, None),
+ iface("e", Some("vnet1"), true, Some(record())), // no entry yet
+ iface("f", Some("vnet1"), true, None), // no entry, served, no mapping
+ iface("g", Some("vnet1"), true, None),
+ ];
+ let present: HashMap<u32, &Iface> =
+ [1, 2, 4, 5, 9, 8].into_iter().zip(ifaces.iter()).collect();
+ let unknown_alive = HashSet::from([7]);
+ let (elsewhere, failed) = sync_records(&mut map, &present, &unknown_alive).unwrap();
+
+ assert!(elsewhere.is_empty());
+ assert_eq!(failed, 0);
+ assert_eq!(map[&1], entry(VNET));
+ assert_eq!(map[&2], entry(VNET), "a changed answer is rewritten");
+ assert_eq!(
+ map[&4].ip, 0,
+ "an interface not served anymore keeps a marker"
+ );
+ assert_eq!(map[&5], entry(VNET), "a new interface gets its entry");
+ assert!(!map.contains_key(&6), "a gone interface loses its entry");
+ assert_eq!(
+ map[&9].ip, 0,
+ "a served interface without a mapping gets a marker"
+ );
+ assert_eq!(map[&8].ip, 0, "a mapping that is gone leaves a marker");
+ assert!(
+ map.contains_key(&7),
+ "a live link the configs do not name keeps its entry"
+ );
+ }
+
+ #[test]
+ fn the_links_follow_the_configs_or_the_entry() {
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(record())), // served, entry agrees
+ iface("b", Some("vnet1"), false, None), // not served
+ iface("c", Some("vnet1"), false, None), // config stale, entry has a record
+ iface("d", Some("vnet1"), true, Some(record())), // config stale, entry is a marker
+ ];
+ let present: HashMap<u32, &Iface> = (1..=4).zip(ifaces.iter()).collect();
+ let elsewhere = HashSet::from([3, 4]);
+ let unknown_alive = HashSet::from([9]);
+ assert_eq!(
+ wanted_links(&present, &elsewhere, &unknown_alive),
+ HashSet::from([1, 3, 4, 9])
+ );
+ }
+
+ // a store that refuses one interface's writes
+ struct Flaky(BTreeMap<u32, DhcpRecord>, u32);
+
+ impl RecordStore for Flaky {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
+ RecordStore::get(&self.0, ifindex)
+ }
+
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ if ifindex == self.1 {
+ bail!("map full");
+ }
+ RecordStore::insert(&mut self.0, ifindex, rec)
+ }
+
+ fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ RecordStore::mark(&mut self.0, ifindex, rec)
+ }
+
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
+ if ifindex == self.1 {
+ bail!("map busy");
+ }
+ RecordStore::remove(&mut self.0, ifindex)
+ }
+
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
+ RecordStore::entries(&self.0)
+ }
+ }
+
+ #[test]
+ fn bad_input_leaves_its_interface_alone() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(VNET));
+ let bad = Record {
+ lease: 0,
+ ..record()
+ };
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(bad)),
+ iface("b", Some("vnet1"), true, Some(record())),
+ ];
+ let present: HashMap<u32, &Iface> = (1..=2).zip(ifaces.iter()).collect();
+ let (elsewhere, failed) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
+ assert!(elsewhere.is_empty());
+ assert_eq!(failed, 0);
+ assert_eq!(map[&1], entry(VNET), "the old entry stays");
+ assert_eq!(map[&2], entry(VNET), "the rest is written");
+ }
+
+ #[test]
+ fn a_failed_write_does_not_stop_the_pass() {
+ let mut map = Flaky(BTreeMap::new(), 2);
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(record())),
+ iface("b", Some("vnet1"), true, Some(record())),
+ iface("c", Some("vnet1"), true, Some(record())),
+ ];
+ let present: HashMap<u32, &Iface> = (1..=3).zip(ifaces.iter()).collect();
+ let (elsewhere, failed) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
+ assert!(elsewhere.is_empty());
+ assert_eq!(failed, 1);
+ assert!(map.0.contains_key(&1));
+ assert!(!map.0.contains_key(&2));
+ assert!(map.0.contains_key(&3));
+ }
+
+ // a store whose listing fails
+ struct Unlistable(BTreeMap<u32, DhcpRecord>);
+
+ impl RecordStore for Unlistable {
+ fn get(&self, ifindex: u32) -> anyhow::Result<Option<DhcpRecord>> {
+ RecordStore::get(&self.0, ifindex)
+ }
+
+ fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ RecordStore::insert(&mut self.0, ifindex, rec)
+ }
+
+ fn mark(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> {
+ RecordStore::mark(&mut self.0, ifindex, rec)
+ }
+
+ fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> {
+ RecordStore::remove(&mut self.0, ifindex)
+ }
+
+ fn entries(&self) -> anyhow::Result<Vec<(u32, DhcpRecord)>> {
+ bail!("map gone")
+ }
+ }
+
+ #[test]
+ fn a_failed_listing_fails_the_pass_before_it_writes() {
+ let mut map = Unlistable(BTreeMap::new());
+ map.0.insert(1, entry(OTHER));
+ let ifaces = [iface("a", Some("vnet1"), true, Some(record()))];
+ let present: HashMap<u32, &Iface> = [(1, &ifaces[0])].into_iter().collect();
+ assert!(sync_records(&mut map, &present, &HashSet::new()).is_err());
+ assert_eq!(map.0[&1].vnet, OTHER);
+ }
+
+ #[test]
+ fn a_failed_removal_counts_like_a_failed_write() {
+ let mut map = Flaky(BTreeMap::new(), 2);
+ map.0.insert(2, entry(VNET));
+ let ifaces = [iface("a", Some("vnet1"), true, Some(record()))];
+ let present: HashMap<u32, &Iface> = [(1, &ifaces[0])].into_iter().collect();
+ let (_, failed) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
+ assert_eq!(failed, 1);
+ assert!(map.0.contains_key(&1));
+ assert!(map.0.contains_key(&2));
+ }
+
+ #[test]
+ fn the_pass_leaves_entries_of_another_vnet_to_their_link() {
+ let mut map = BTreeMap::new();
+ map.insert(1, entry(OTHER)); // a plug the configs trail, with a record
+ map.insert(2, DhcpRecord::removed(NO_VNET)); // moved to a plain bridge
+ map.insert(3, DhcpRecord::removed(NO_VNET)); // the configs caught up with that
+ let ifaces = [
+ iface("a", Some("vnet1"), true, Some(record())),
+ iface("b", Some("vnet1"), true, Some(record())),
+ iface("c", None, false, None),
+ ];
+ let present: HashMap<u32, &Iface> = (1..=3).zip(ifaces.iter()).collect();
+ let (elsewhere, _) = sync_records(&mut map, &present, &HashSet::new()).unwrap();
+
+ assert_eq!(elsewhere, HashSet::from([1, 2]));
+
+ assert_eq!(map[&1].vnet, OTHER);
+ assert_eq!(map[&2].vnet, NO_VNET);
+ assert_eq!(
+ map[&3].ip, 0,
+ "a marker the configs agree with stays as it is"
+ );
+ }
+}
diff --git a/src/dhcp/types.rs b/src/dhcp/types.rs
new file mode 100644
index 0000000..71a1b94
--- /dev/null
+++ b/src/dhcp/types.rs
@@ -0,0 +1,64 @@
+//! Keep in sync with 'bpf/types.h'.
+
+/// Addresses, lease and mtu are stored in network byte order, so a reply copies them into the
+/// packet as-is.
+#[repr(C)]
+#[derive(Copy, Clone, Debug, PartialEq, Eq)]
+pub struct DhcpRecord {
+ /// 0 = removed, the entry is a marker naming its vnet
+ pub ip: u32,
+ pub netmask: u32,
+ /// 0 = not served
+ pub router: u32,
+ /// 0 = not served
+ pub dns: u32,
+ pub server_id: u32,
+ pub lease: u32,
+ /// 0 = not served
+ pub mtu: u16,
+ /// The vnet the writer saw the interface on, the id padded with zeros, all zero for none.
+ /// Never read by the program.
+ pub vnet: [u8; 8],
+ /// The NIC's MAC, a request carrying another is passed on. Zero in a marker.
+ pub mac: [u8; 6],
+}
+
+impl DhcpRecord {
+ /// The marker a removal leaves behind, on the vnet the writer saw the interface on.
+ pub fn removed(vnet: [u8; 8]) -> Self {
+ Self {
+ ip: 0,
+ netmask: 0,
+ router: 0,
+ dns: 0,
+ server_id: 0,
+ lease: 0,
+ mtu: 0,
+ vnet,
+ mac: [0; 6],
+ }
+ }
+}
+
+unsafe impl aya::Pod for DhcpRecord {}
+
+#[cfg(test)]
+mod layout {
+ use core::mem::{offset_of, size_of};
+
+ use super::*;
+
+ #[test]
+ fn matches_bpf_abi() {
+ assert_eq!(size_of::<DhcpRecord>(), 40);
+ assert_eq!(offset_of!(DhcpRecord, ip), 0);
+ assert_eq!(offset_of!(DhcpRecord, netmask), 4);
+ assert_eq!(offset_of!(DhcpRecord, router), 8);
+ assert_eq!(offset_of!(DhcpRecord, dns), 12);
+ assert_eq!(offset_of!(DhcpRecord, server_id), 16);
+ assert_eq!(offset_of!(DhcpRecord, lease), 20);
+ assert_eq!(offset_of!(DhcpRecord, mtu), 24);
+ assert_eq!(offset_of!(DhcpRecord, vnet), 26);
+ assert_eq!(offset_of!(DhcpRecord, mac), 34);
+ }
+}
diff --git a/src/lib.rs b/src/lib.rs
index 572d861..a8c94ba 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -1,5 +1,8 @@
//! eBPF subsystems, one per concern, each behind a cargo feature of its name. Nothing here runs
//! on its own, consumers pull in only the subsystem they drive.
+#[cfg(feature = "dhcp")]
+pub mod dhcp;
+
pub mod subsystem;
pub mod tc;
diff --git a/src/subsystem.rs b/src/subsystem.rs
index 3bf1634..6014dbe 100644
--- a/src/subsystem.rs
+++ b/src/subsystem.rs
@@ -42,7 +42,7 @@ const RUN_ROOT: &str = "/run/proxmox-ebpf";
/// The subsystems in the order their programs run on an interface, the first runs first. A name
/// missing from the list fails the build.
-pub const RUN_ORDER: &[&str] = &[];
+pub const RUN_ORDER: &[&str] = &["dhcp"];
const fn rank_of(name: &str) -> usize {
let mut i = 0;
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH proxmox-perl-rs v4 03/20] pve-rs: sdn: add bindings for the eBPF DHCP responder
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
2026-10-21 8:44 ` [PATCH proxmox-ebpf v4 01/20] dhcp: add the per-tap DHCPv4 responder program Hannes Laimer
2026-10-21 8:44 ` [PATCH proxmox-ebpf v4 02/20] dhcp: add the subsystem that loads the responder and keeps its records Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-network v4 04/20] sdn: ipam: stop caching IPAM misses Hannes Laimer
` (16 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
The responder answers with the records the caller hands in. The
caller has to build them under the responder's lock, or a full pass
could write older records over those a plug or record change built
later. So the lock is an object the caller takes before it builds the
records, and every call that hands in records is a method of that
lock.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
pve-rs/Cargo.toml | 2 +
pve-rs/Makefile | 3 +
pve-rs/debian/control | 2 +
pve-rs/src/bindings/sdn/dhcp.rs | 229 ++++++++++++++++++++++++++++++++
pve-rs/src/bindings/sdn/mod.rs | 1 +
5 files changed, 237 insertions(+)
create mode 100644 pve-rs/src/bindings/sdn/dhcp.rs
diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
index 5ae9082..0adf00b 100644
--- a/pve-rs/Cargo.toml
+++ b/pve-rs/Cargo.toml
@@ -34,6 +34,7 @@ proxmox-apt = { version = "1.0.1", features = ["cache"] }
proxmox-apt-api-types = "3"
proxmox-base64 = "1"
proxmox-config-digest = "1"
+proxmox-ebpf = { version = "0.1", features = ["dhcp"] }
proxmox-frr = { version = "0.5.1" }
proxmox-http = { version = "1.0.2", features = ["client-sync", "client-trait"] }
proxmox-http-error = "1"
@@ -70,6 +71,7 @@ proxmox-wireguard = { version = "0.1.2" }
# proxmox-config-digest = { path = "../../proxmox/proxmox-config-digest" }
# proxmox-daemon = { path = "../../proxmox/proxmox-daemon" }
# proxmox-dns-api = { path = "../../proxmox/proxmox-dns-api" }
+# proxmox-ebpf = { path = "../../proxmox-ebpf" }
# proxmox-http-error = { path = "../../proxmox/proxmox-http-error" }
# proxmox-http = { path = "../../proxmox/proxmox-http" }
# proxmox-human-byte = { path = "../../proxmox/proxmox-human-byte" }
diff --git a/pve-rs/Makefile b/pve-rs/Makefile
index bb1cd2d..ebe8bd3 100644
--- a/pve-rs/Makefile
+++ b/pve-rs/Makefile
@@ -32,6 +32,9 @@ PERLMOD_PACKAGES := \
PVE::RS::OpenId \
PVE::RS::ResourceScheduling::Static \
PVE::RS::ResourceScheduling::Dynamic \
+ PVE::RS::SDN::Dhcp \
+ PVE::RS::SDN::Dhcp::ExclusiveLock \
+ PVE::RS::SDN::Dhcp::SharedLock \
PVE::RS::SDN::Fabrics \
PVE::RS::SDN::PrefixLists \
PVE::RS::SDN::RouteMaps \
diff --git a/pve-rs/debian/control b/pve-rs/debian/control
index 70367dc..521d966 100644
--- a/pve-rs/debian/control
+++ b/pve-rs/debian/control
@@ -20,6 +20,8 @@ Build-Depends: debhelper-compat (= 13),
librust-proxmox-apt-api-types-3+default-dev (>= 3.0.0-~~),
librust-proxmox-base64-1+default-dev,
librust-proxmox-config-digest-1+default-dev,
+ librust-proxmox-ebpf-0.1+default-dev,
+ librust-proxmox-ebpf-0.1+dhcp-dev,
librust-proxmox-frr-0.5+default-dev (>= 0.5.1-~~),
librust-proxmox-http-1+client-sync-dev (>= 1.0.2-~~),
librust-proxmox-http-1+client-trait-dev (>= 1.0.2-~~),
diff --git a/pve-rs/src/bindings/sdn/dhcp.rs b/pve-rs/src/bindings/sdn/dhcp.rs
new file mode 100644
index 0000000..a6cdc95
--- /dev/null
+++ b/pve-rs/src/bindings/sdn/dhcp.rs
@@ -0,0 +1,229 @@
+#[perlmod::package(name = "PVE::RS::SDN::Dhcp", lib = "pve_rs")]
+pub mod pve_rs_sdn_dhcp {
+ //! The `PVE::RS::SDN::Dhcp` package.
+ //!
+ //! Bindings for the per-tap eBPF DHCP responder. Interfaces and the records they answer with
+ //! are handed in here and served directly from the kernel. The calls that change the responder
+ //! are methods of a lock object, so nothing writes to it without holding the lock in the mode
+ //! the change needs.
+
+ use std::net::Ipv4Addr;
+
+ use anyhow::Error;
+ use serde::{Deserialize, Deserializer};
+
+ use proxmox_ebpf::dhcp::{DhcpSubsystem, Iface, Record, Unplugged};
+
+ /// One interface's DHCP answer, the address and every option the reply carries.
+ #[derive(Deserialize)]
+ pub struct DhcpRecord {
+ ip: Ipv4Addr,
+ prefixlen: u8,
+ server_id: Ipv4Addr,
+ lease: u32,
+ #[serde(default)]
+ router: Option<Ipv4Addr>,
+ #[serde(default)]
+ dns: Option<Ipv4Addr>,
+ #[serde(default)]
+ mtu: Option<u16>,
+ #[serde(deserialize_with = "mac_from_str")]
+ mac: [u8; 6],
+ }
+
+ // a MAC as six hex pairs
+ fn mac_from_str<'de, D: Deserializer<'de>>(d: D) -> Result<[u8; 6], D::Error> {
+ let text = String::deserialize(d)?;
+ let mut mac = [0u8; 6];
+ let mut parts = text.split(':');
+ for byte in &mut mac {
+ let part = parts
+ .next()
+ .ok_or_else(|| serde::de::Error::custom(format!("MAC {text} is too short")))?;
+ if part.len() != 2 || !part.bytes().all(|b| b.is_ascii_hexdigit()) {
+ return Err(serde::de::Error::custom(format!("MAC {text} is malformed")));
+ }
+ *byte = u8::from_str_radix(part, 16)
+ .map_err(|_| serde::de::Error::custom(format!("MAC {text} is malformed")))?;
+ }
+ if parts.next().is_some() {
+ return Err(serde::de::Error::custom(format!("MAC {text} is too long")));
+ }
+ Ok(mac)
+ }
+
+ /// One guest interface, the vnet it sits on if any, whether the responder runs on it and what
+ /// it answers there.
+ #[derive(Deserialize)]
+ pub struct DhcpIface {
+ name: String,
+ #[serde(default)]
+ vnet: Option<String>,
+ serve: bool,
+ #[serde(default)]
+ record: Option<DhcpRecord>,
+ }
+
+ /// Where an unplugged interface went, the vnet it sits on now, none for a plain bridge.
+ #[derive(Deserialize)]
+ pub struct DhcpTarget {
+ #[serde(default)]
+ vnet: Option<String>,
+ }
+
+ impl From<DhcpRecord> for Record {
+ fn from(r: DhcpRecord) -> Self {
+ Record {
+ ip: r.ip,
+ prefixlen: r.prefixlen,
+ server_id: r.server_id,
+ lease: r.lease,
+ router: r.router,
+ dns: r.dns,
+ mtu: r.mtu,
+ mac: r.mac,
+ }
+ }
+ }
+
+ impl From<DhcpIface> for Iface {
+ fn from(i: DhcpIface) -> Self {
+ Iface {
+ name: i.name,
+ vnet: i.vnet,
+ serve: i.serve,
+ record: i.record.map(Record::from),
+ }
+ }
+ }
+
+ pub(super) fn to_ifaces(ifaces: Vec<DhcpIface>) -> Vec<Iface> {
+ ifaces.into_iter().map(Iface::from).collect()
+ }
+
+ /// A tap unplug. Without a target the interface is gone or going and takes its record with
+ /// it. With one it was plugged onto a bridge the responder does not serve and keeps a marker
+ /// naming that place.
+ #[export]
+ pub fn detach(iface: &str, target: Option<DhcpTarget>) -> Result<(), Error> {
+ let unplugged = match target {
+ None => Unplugged::Gone,
+ Some(target) => Unplugged::MovedTo(target.vnet),
+ };
+ DhcpSubsystem::new().detach(iface, unplugged)
+ }
+}
+
+#[perlmod::package(name = "PVE::RS::SDN::Dhcp::SharedLock", lib = "pve_rs")]
+pub mod pve_rs_sdn_dhcp_shared_lock {
+ //! The `PVE::RS::SDN::Dhcp::SharedLock` package.
+ //!
+ //! The responder's lock taken shared, what a tap plug and a guest's own record change hold.
+ //! Released when the object goes away.
+
+ use anyhow::Error;
+ use perlmod::Value;
+
+ use proxmox_ebpf::dhcp::{DhcpSubsystem, Iface, Outcome, Record, SharedLock};
+
+ use super::pve_rs_sdn_dhcp::{DhcpIface, DhcpRecord, to_ifaces};
+
+ /// The lock as Perl holds it.
+ pub struct PerlSharedLock {
+ lock: SharedLock,
+ }
+
+ perlmod::declare_magic!(Box<PerlSharedLock> : &PerlSharedLock as "PVE::RS::SDN::Dhcp::SharedLock");
+
+ /// Class method: take the lock. Plugs hold it next to each other and wait only behind a full
+ /// pass or a record change.
+ #[export]
+ pub fn new(#[raw] class: Value) -> Result<Value, Error> {
+ let lock = DhcpSubsystem::new().lock_shared()?;
+ Ok(perlmod::instantiate_magic!(&class, MAGIC => Box::new(PerlSharedLock { lock })))
+ }
+
+ /// Method: a tap plug onto a vnet with the record it answers. Without a record the interface
+ /// gets no answer. Returns false when nothing is loaded yet. An interface gone already counts
+ /// as done.
+ #[export]
+ pub fn attach(
+ #[try_from_ref] this: &PerlSharedLock,
+ iface: String,
+ vnet: String,
+ record: Option<DhcpRecord>,
+ ) -> Result<bool, Error> {
+ let iface = Iface {
+ name: iface,
+ vnet: Some(vnet),
+ serve: true,
+ record: record.map(Record::from),
+ };
+ Ok(this.lock.attach(&iface)? == Outcome::Done)
+ }
+
+ /// Method: a record change for the listed interfaces by a caller holding the guest lock.
+ /// Returns false when nothing is loaded to write into, so a full pass is due.
+ #[export]
+ pub fn update(
+ #[try_from_ref] this: &PerlSharedLock,
+ ifaces: Vec<DhcpIface>,
+ ) -> Result<bool, Error> {
+ Ok(this.lock.update(&to_ifaces(ifaces))? == Outcome::Done)
+ }
+}
+
+#[perlmod::package(name = "PVE::RS::SDN::Dhcp::ExclusiveLock", lib = "pve_rs")]
+pub mod pve_rs_sdn_dhcp_exclusive_lock {
+ //! The `PVE::RS::SDN::Dhcp::ExclusiveLock` package.
+ //!
+ //! The responder's lock taken exclusively, what a full pass and a record change hold. Released
+ //! when the object goes away.
+
+ use anyhow::Error;
+ use perlmod::Value;
+
+ use proxmox_ebpf::dhcp::{DhcpSubsystem, ExclusiveLock, Outcome};
+
+ use super::pve_rs_sdn_dhcp::{DhcpIface, to_ifaces};
+
+ /// The lock as Perl holds it.
+ pub struct PerlExclusiveLock {
+ lock: ExclusiveLock,
+ }
+
+ perlmod::declare_magic!(Box<PerlExclusiveLock> : &PerlExclusiveLock as "PVE::RS::SDN::Dhcp::ExclusiveLock");
+
+ /// Class method: take the lock, waiting out every plug in flight.
+ #[export]
+ pub fn new(#[raw] class: Value) -> Result<Value, Error> {
+ let lock = DhcpSubsystem::new().lock_exclusive()?;
+ Ok(perlmod::instantiate_magic!(&class, MAGIC => Box::new(PerlExclusiveLock { lock })))
+ }
+
+ /// Method: the full pass over every guest interface of the cluster. Those found here get
+ /// their entry and a link where served.
+ #[export]
+ pub fn apply(
+ #[try_from_ref] this: &PerlExclusiveLock,
+ ifaces: Vec<DhcpIface>,
+ ) -> Result<(), Error> {
+ this.lock.apply(&to_ifaces(ifaces))
+ }
+
+ /// Method: a record change for the listed interfaces. Those not found here are skipped.
+ /// Returns false when nothing is loaded to write into, so a full pass is due.
+ #[export]
+ pub fn update(
+ #[try_from_ref] this: &PerlExclusiveLock,
+ ifaces: Vec<DhcpIface>,
+ ) -> Result<bool, Error> {
+ Ok(this.lock.update(&to_ifaces(ifaces))? == Outcome::Done)
+ }
+
+ /// Method: detach the responder everywhere and drop its pinned state.
+ #[export]
+ pub fn clear(#[try_from_ref] this: &PerlExclusiveLock) -> Result<(), Error> {
+ this.lock.clear()
+ }
+}
diff --git a/pve-rs/src/bindings/sdn/mod.rs b/pve-rs/src/bindings/sdn/mod.rs
index dcae046..4b99b8f 100644
--- a/pve-rs/src/bindings/sdn/mod.rs
+++ b/pve-rs/src/bindings/sdn/mod.rs
@@ -1,3 +1,4 @@
+pub(crate) mod dhcp;
pub(crate) mod fabrics;
pub(crate) mod prefix_lists;
pub(crate) mod route_maps;
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-network v4 04/20] sdn: ipam: stop caching IPAM misses
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (2 preceding siblings ...)
2026-10-21 8:44 ` [PATCH proxmox-perl-rs v4 03/20] pve-rs: sdn: add bindings for the eBPF DHCP responder Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-network v4 05/20] sdn: ipam: write the lookup's cache entry under the cluster lock Hannes Laimer
` (15 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
A lookup that missed the cache stored the IPAM's empty answer, and
any entry counted as a hit. So a record created later directly in the
IPAM stayed hidden, and a guest start allocated another address. An
empty answer is no longer stored, and an entry without an address, as
earlier misses left them, is dropped when the cache is read.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Notes:
v4:
* new, split out of the MAC cache patch
src/PVE/Network/SDN/Ipams.pm | 19 ++++++++---
src/test/run_test_vnets_blackbox.pl | 52 ++++++++++++++++++++++++++++-
2 files changed, 66 insertions(+), 5 deletions(-)
diff --git a/src/PVE/Network/SDN/Ipams.pm b/src/PVE/Network/SDN/Ipams.pm
index 179bdf76..79591ea0 100644
--- a/src/PVE/Network/SDN/Ipams.pm
+++ b/src/PVE/Network/SDN/Ipams.pm
@@ -23,10 +23,17 @@ PVE::Network::SDN::Ipams::Plugin->init();
my $macdb_filename = "sdn/mac-cache.json";
cfs_register_file($macdb_filename, \&json_reader, \&json_writer);
+# an entry without an address was left behind by a lookup that missed, and
+# says nothing about its MAC
sub json_reader {
my ($filename, $data) = @_;
- return defined($data) && length($data) > 0 ? decode_json($data) : {};
+ my $db = defined($data) && length($data) > 0 ? decode_json($data) : {};
+ for my $mac (keys %{ $db->{macs} // {} }) {
+ my $entry = $db->{macs}->{$mac};
+ delete $db->{macs}->{$mac} if !defined($entry->{ip4}) && !defined($entry->{ip6});
+ }
+ return $db;
}
sub json_writer {
@@ -140,12 +147,16 @@ sub get_ips_from_mac {
my $plugin_config = get_plugin_config($zone);
my $plugin = PVE::Network::SDN::Ipams::Plugin->lookup($plugin_config->{type});
- ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6}) =
- $plugin->get_ips_from_mac($plugin_config, $mac, $zoneid);
+ my ($ip4, $ip6) = $plugin->get_ips_from_mac($plugin_config, $mac, $zoneid);
+ # an empty answer is not cached, since the record may simply not exist yet
+ return if !defined($ip4) && !defined($ip6);
+
+ $macdb->{macs}->{$mac}->{ip4} = $ip4 if defined($ip4);
+ $macdb->{macs}->{$mac}->{ip6} = $ip6 if defined($ip6);
write_macdb($macdb);
- return ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6});
+ return ($ip4, $ip6);
}
1;
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 9f4c424f..d8e35231 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -8,6 +8,7 @@ use File::Slurp;
use List::Util qw(first all);
use NetAddr::IP qw(:lower);
+use JSON;
use Test::More;
use Test::MockModule;
use Test::Differences;
@@ -193,7 +194,7 @@ $mocked_sdn_ipams->mock(
$test_state->{ipam_config} = $cfg;
},
read_macdb => sub {
- return $test_state->{macdb};
+ return PVE::Network::SDN::Ipams::json_reader(undef, encode_json($test_state->{macdb}));
},
write_macdb => sub {
my ($cfg) = @_;
@@ -421,6 +422,55 @@ sub test_without_subnet {
}
run_test(\&test_without_subnet);
+sub test_ipam_cache_misses {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+
+ my @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ is(scalar(grep { defined } @ips), 0, "$test_name: an unknown MAC has no answer");
+ ok(!exists $test_state->{macdb}->{macs}->{$mac}, "$test_name: the miss is not cached");
+
+ # an entry holding no address, left behind by an earlier miss, does not
+ # hide a record created since
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.50",
+ });
+ $test_state->{macdb}->{macs}->{$mac} = { ip4 => undef, ip6 => undef };
+ @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ is($ips[0], "10.0.0.50", "$test_name: an address-less entry counts as a miss");
+ is(
+ $test_state->{macdb}->{macs}->{$mac}->{ip4},
+ "10.0.0.50",
+ "$test_name: the answer is cached",
+ );
+}
+
+run_test(\&test_ipam_cache_misses);
+
sub test_nic_join {
my ($test_name, $subnets) = @_;
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-network v4 05/20] sdn: ipam: write the lookup's cache entry under the cluster lock
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (3 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-network v4 04/20] sdn: ipam: stop caching IPAM misses Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-network v4 06/20] sdn: ipam: key the MAC cache by lower-case MAC Hannes Laimer
` (14 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
The lookup that stores the IPAM's answer wrote the whole cache back
without the cluster lock the other writers take. A change made
between its read and its write was lost. The lookup now takes the
lock and reads the cache again before it stores its answer.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Notes:
v4:
* new, split out of the MAC cache patch
src/PVE/Network/SDN/Ipams.pm | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/src/PVE/Network/SDN/Ipams.pm b/src/PVE/Network/SDN/Ipams.pm
index 79591ea0..3d849d79 100644
--- a/src/PVE/Network/SDN/Ipams.pm
+++ b/src/PVE/Network/SDN/Ipams.pm
@@ -142,8 +142,8 @@ sub complete_sdn_vnet {
sub get_ips_from_mac {
my ($mac, $zoneid, $zone) = @_;
- my $macdb = read_macdb();
- return ($macdb->{macs}->{$mac}->{ip4}, $macdb->{macs}->{$mac}->{ip6}) if $macdb->{macs}->{$mac};
+ my $cached = read_macdb()->{macs}->{$mac};
+ return ($cached->{ip4}, $cached->{ip6}) if $cached;
my $plugin_config = get_plugin_config($zone);
my $plugin = PVE::Network::SDN::Ipams::Plugin->lookup($plugin_config->{type});
@@ -152,9 +152,17 @@ sub get_ips_from_mac {
# an empty answer is not cached, since the record may simply not exist yet
return if !defined($ip4) && !defined($ip6);
- $macdb->{macs}->{$mac}->{ip4} = $ip4 if defined($ip4);
- $macdb->{macs}->{$mac}->{ip6} = $ip6 if defined($ip6);
- write_macdb($macdb);
+ cfs_lock_file(
+ $macdb_filename,
+ undef,
+ sub {
+ my $db = read_macdb();
+ $db->{macs}->{$mac}->{ip4} = $ip4 if defined($ip4);
+ $db->{macs}->{$mac}->{ip6} = $ip6 if defined($ip6);
+ write_macdb($db);
+ },
+ );
+ warn "$@" if $@;
return ($ip4, $ip6);
}
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-network v4 06/20] sdn: ipam: key the MAC cache by lower-case MAC
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (4 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-network v4 05/20] sdn: ipam: write the lookup's cache entry under the cluster lock Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-network v4 07/20] sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it Hannes Laimer
` (13 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
Generated guest MACs are upper case, but the API accepts either
spelling. The cache was keyed by the MAC as its writer spelled it. A
deletion in the other spelling missed the entry, so the cache kept
answering with the deleted address. The cache is now read with every
MAC in lower case and written under the lower-case MAC. An entry
cached before this change under another spelling is read under the
lower-case key and leaves the file with the next write. Two spellings
that disagree on an address are both dropped on the read, since one
of them is stale and the cache cannot tell which. The next lookup
then asks the IPAM. The dnsmasq reservation file is written and read
with its MACs in lower case as well, since its lines from before keep
their writer's spelling.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Notes:
v4:
* fold the spelling handling into the cache read
* split the miss and lock fixes into their own commits
src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 3 ++
src/PVE/Network/SDN/Ipams.pm | 39 ++++++++++----
src/test/run_test_vnets_blackbox.pl | 82 +++++++++++++++++++++++++++++
3 files changed, 113 insertions(+), 11 deletions(-)
diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
index 477b700f..d3b78fe9 100644
--- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
+++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
@@ -53,6 +53,7 @@ sub update_lease {
sub add_ip_mapping {
my ($class, $dhcpid, $macdb, $mac, $ip4, $ip6) = @_;
+ $mac = lc($mac);
my $ethers_file = ethers_file($dhcpid);
my $ethers_tmp_file = "$ethers_file.tmp";
@@ -76,6 +77,8 @@ sub add_ip_mapping {
warn "failed to parse MAC from $dhcpid ethers file on line $line_no: '$line'\n";
next;
}
+ # the cache is keyed in lower case, a line from before keeps its writer's spelling
+ $parsed_mac = lc($parsed_mac);
my ($parsed_ip4, $parsed_ip6);
if ($parsed_ip2) {
diff --git a/src/PVE/Network/SDN/Ipams.pm b/src/PVE/Network/SDN/Ipams.pm
index 3d849d79..0e795a93 100644
--- a/src/PVE/Network/SDN/Ipams.pm
+++ b/src/PVE/Network/SDN/Ipams.pm
@@ -23,17 +23,31 @@ PVE::Network::SDN::Ipams::Plugin->init();
my $macdb_filename = "sdn/mac-cache.json";
cfs_register_file($macdb_filename, \&json_reader, \&json_writer);
-# an entry without an address was left behind by a lookup that missed, and
-# says nothing about its MAC
+# the file may hold a MAC under more than one spelling, and an entry without
+# an address says nothing about its MAC. Two spellings that disagree on an
+# address are both dropped, since one of them is stale and nothing here says
+# which. The next lookup then asks the IPAM
+my sub fold_macdb {
+ my ($db) = @_;
+
+ my $macs = {};
+ my %clash;
+ for my $key (sort keys %{ $db->{macs} // {} }) {
+ my $entry = $db->{macs}->{$key};
+ for my $family (grep { defined($entry->{$_}) } qw(ip4 ip6)) {
+ my $seen = $macs->{ lc($key) }->{$family};
+ $clash{ lc($key) } = 1 if defined($seen) && lc($seen) ne lc($entry->{$family});
+ $macs->{ lc($key) }->{$family} //= $entry->{$family};
+ }
+ }
+ delete $macs->{$_} for keys %clash;
+ return { %$db, macs => $macs };
+}
+
sub json_reader {
my ($filename, $data) = @_;
- my $db = defined($data) && length($data) > 0 ? decode_json($data) : {};
- for my $mac (keys %{ $db->{macs} // {} }) {
- my $entry = $db->{macs}->{$mac};
- delete $db->{macs}->{$mac} if !defined($entry->{ip4}) && !defined($entry->{ip6});
- }
- return $db;
+ return fold_macdb(defined($data) && length($data) > 0 ? decode_json($data) : {});
}
sub json_writer {
@@ -54,8 +68,10 @@ sub write_macdb {
cfs_write_file($macdb_filename, $data);
}
+# the cache is keyed by the MAC in lower case, whatever spelling its writers use
sub add_cache_mac_ip {
my ($mac, $ip) = @_;
+ $mac = lc($mac);
cfs_lock_file(
$macdb_filename,
@@ -75,6 +91,7 @@ sub add_cache_mac_ip {
sub del_cache_mac_ip {
my ($mac, $ip) = @_;
+ $mac = lc($mac);
cfs_lock_file(
$macdb_filename,
@@ -142,7 +159,7 @@ sub complete_sdn_vnet {
sub get_ips_from_mac {
my ($mac, $zoneid, $zone) = @_;
- my $cached = read_macdb()->{macs}->{$mac};
+ my $cached = read_macdb()->{macs}->{ lc($mac) };
return ($cached->{ip4}, $cached->{ip6}) if $cached;
my $plugin_config = get_plugin_config($zone);
@@ -157,8 +174,8 @@ sub get_ips_from_mac {
undef,
sub {
my $db = read_macdb();
- $db->{macs}->{$mac}->{ip4} = $ip4 if defined($ip4);
- $db->{macs}->{$mac}->{ip6} = $ip6 if defined($ip6);
+ $db->{macs}->{ lc($mac) }->{ip4} = $ip4 if defined($ip4);
+ $db->{macs}->{ lc($mac) }->{ip6} = $ip6 if defined($ip6);
write_macdb($db);
},
);
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index d8e35231..6b89778f 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -471,6 +471,88 @@ sub test_ipam_cache_misses {
run_test(\&test_ipam_cache_misses);
+sub test_ipam_cache_case {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # an entry written under the config's spelling before the cache was keyed in
+ # lower case is read and released like any other
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ $test_state->{macdb}->{macs}->{ uc($mac) } = { ip4 => '10.0.0.150' };
+ my ($ip4) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ is($ip4, '10.0.0.150', "$test_name: an old entry is found however it is spelled");
+ PVE::Network::SDN::Ipams::del_cache_mac_ip($mac, '10.0.0.150');
+ ok(
+ !(grep { lc($_) eq $mac } keys $test_state->{macdb}->{macs}->%*),
+ "$test_name: and released",
+ );
+
+ # a write spelled either way moves such an entry to the lower-case key
+ # and keeps what it held
+ $test_state->{macdb}->{macs}->{ uc($mac) } = { ip4 => '10.0.0.150' };
+ PVE::Network::SDN::Ipams::add_cache_mac_ip(uc($mac), 'fd00::150');
+ is_deeply(
+ [grep { lc($_) eq $mac } keys $test_state->{macdb}->{macs}->%*],
+ [$mac],
+ "$test_name: a write leaves one entry for the MAC",
+ );
+ is_deeply(
+ $test_state->{macdb}->{macs}->{$mac},
+ { ip4 => '10.0.0.150', ip6 => 'fd00::150' },
+ "$test_name: holding both addresses",
+ );
+
+ # the lookup's write moves an address-less entry under another spelling as well
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.160",
+ });
+ $test_state->{macdb}->{macs} = { uc($mac) => { ip4 => undef, ip6 => undef } };
+ my @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ is($ips[0], '10.0.0.160', "$test_name: a lookup past an address-less entry asks the IPAM");
+ is_deeply(
+ [sort keys $test_state->{macdb}->{macs}->%*],
+ [$mac],
+ "$test_name: and its write leaves one entry for the MAC",
+ );
+
+ # two spellings that disagree are not trusted, the lookup asks the IPAM
+ $test_state->{macdb}->{macs} = {
+ uc($mac) => { ip4 => '10.0.0.150' },
+ $mac => { ip4 => '10.0.0.151' },
+ };
+ @ips = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ is($ips[0], '10.0.0.160', "$test_name: a clash of spellings counts as a miss");
+ is_deeply(
+ $test_state->{macdb}->{macs},
+ { $mac => { ip4 => '10.0.0.160' } },
+ "$test_name: and its write leaves the IPAM's answer alone",
+ );
+}
+
+run_test(\&test_ipam_cache_case);
+
sub test_nic_join {
my ($test_name, $subnets) = @_;
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-network v4 07/20] sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (5 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-network v4 06/20] sdn: ipam: key the MAC cache by lower-case MAC Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-network v4 08/20] sdn: dhcp: update reservations when ip mappings change Hannes Laimer
` (12 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
A reservation rewrite dropped every line that no longer matched the
MAC cache and wrote nothing in its place. The guest behind such a
line was unknown to dnsmasq until its next start, so its requests
were ignored. The line is now replaced by the cache's current one.
Only a MAC the cache holds no address for is dropped.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com>
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 11 ++++++++++-
src/test/run_test_vnets_blackbox.pl | 28 ++++++++++++++++++++++++++++
2 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
index d3b78fe9..a1e4c70f 100644
--- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
+++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
@@ -91,7 +91,9 @@ sub add_ip_mapping {
}
$parsed_ip6 = $1 if $parsed_ip6 && $parsed_ip6 =~ m/\[(\S+)\]/;
- #delete changed
+ # a line that no longer matches the cache is replaced by the cache's
+ # current one. Dropping it left that guest unknown to dnsmasq until
+ # its next start
if (
!defined($macdb->{macs}->{$parsed_mac})
|| ($parsed_ip4
@@ -102,6 +104,13 @@ sub add_ip_mapping {
&& $macdb->{macs}->{$parsed_mac}->{'ip6'} ne $parsed_ip6)
) {
$reload = 1;
+ # the MAC being added gets its line below, like a missing one
+ my $current = $macdb->{macs}->{$parsed_mac};
+ next if $parsed_mac eq $mac || !$current;
+ my $replacement = $parsed_mac;
+ $replacement .= ",$current->{ip4}" if $current->{ip4};
+ $replacement .= ",[$current->{ip6}]" if $current->{ip6};
+ print $out "$replacement\n" if $replacement ne $parsed_mac;
next;
}
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 6b89778f..adad5e9a 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -1095,4 +1095,32 @@ run_test(
2,
);
+sub test_dnsmasq_stale_reservation_replaced {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+
+ my $stale = "da:65:8f:18:9b:01";
+ my $gone = "da:65:8f:18:9b:02";
+ my $added = "da:65:8f:18:9b:03";
+ $test_state->{macdb} = {
+ macs => {
+ $stale => { ip4 => '10.0.1.11' },
+ $added => { ip4 => '10.0.1.13' },
+ },
+ };
+ PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "$stale,10.0.1.10\n$gone,10.0.1.12\n");
+
+ PVE::Network::SDN::Dhcp::Dnsmasq->add_ip_mapping(
+ 'TESTZONE', $test_state->{macdb}, $added, '10.0.1.13', undef,
+ );
+
+ my @lines = sort grep { /\S/ } split(/\n/, PVE::Tools::file_get_contents($TMP_ETHERS_FILE));
+ eq_or_diff(
+ \@lines,
+ ["$stale,10.0.1.11", "$added,10.0.1.13"],
+ "$test_name: the stale line is replaced, the unknown one dropped",
+ );
+}
+
+run_test(\&test_dnsmasq_stale_reservation_replaced);
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-network v4 08/20] sdn: dhcp: update reservations when ip mappings change
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (6 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-network v4 07/20] sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-network v4 09/20] sdn: dhcp: make the lease time configurable per subnet Hannes Laimer
` (11 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
Creating, updating or deleting an IP mapping through the API changed
the IPAM and the MAC cache, but never the zone's dnsmasq reservation
file. dnsmasq kept serving the old reservation, or none for a new
mapping, until the guest's next start. Deleting a guest NIC's
addresses left its reservation in the file as well.
A guest start, an API edit and the deletion of a NIC's addresses now
all rewrite the MAC's reservation from the cache, on the node the
change is made on. The other nodes catch up when the guest starts
there. The zone and its backend come from the applied config, and a
zone confined to other nodes is skipped. A failed update only warns,
since the IPAM records are already written.
The rewrite reads the cache only once it holds the file's lock. A
read from before could undo a rewrite that ran in between. The lock
is on a separate file, since a lock on the reservation file would not
survive the rename that replaces it. The MAC's line is written in full
from the cache, so a guest gaining an IPv6 address next to its IPv4
one gets it reserved too. The file's lines are written and compared
in lower case, since lines from before carry their writer's spelling.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Notes:
v4:
* use the cache as read, with its lower-cased keys
src/PVE/API2/Network/SDN/Ips.pm | 7 +
src/PVE/Network/SDN/Dhcp.pm | 48 +++---
src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 101 ++++++-----
src/PVE/Network/SDN/Dhcp/Plugin.pm | 7 +-
src/PVE/Network/SDN/Vnets.pm | 5 +-
src/test/run_test_vnets_blackbox.pl | 250 +++++++++++++++++++++++++++-
6 files changed, 340 insertions(+), 78 deletions(-)
diff --git a/src/PVE/API2/Network/SDN/Ips.pm b/src/PVE/API2/Network/SDN/Ips.pm
index 5ff05e75..d7b682dd 100644
--- a/src/PVE/API2/Network/SDN/Ips.pm
+++ b/src/PVE/API2/Network/SDN/Ips.pm
@@ -46,6 +46,8 @@ __PACKAGE__->register_method({
eval { PVE::Network::SDN::Vnets::del_ip($vnet, $ip, '', $mac); };
die "$@\n" if $@;
+ PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+
return undef;
},
});
@@ -82,6 +84,8 @@ __PACKAGE__->register_method({
PVE::Network::SDN::Vnets::add_ip($vnet, $ip, '', $mac, undef);
+ PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+
return undef;
},
});
@@ -132,6 +136,9 @@ __PACKAGE__->register_method({
}
die "$error\n" if $error;
+
+ PVE::Network::SDN::Dhcp::update_mapping($vnet, $mac);
+
return undef;
},
});
diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
index 65e40d40..ab5d6b5b 100644
--- a/src/PVE/Network/SDN/Dhcp.pm
+++ b/src/PVE/Network/SDN/Dhcp.pm
@@ -6,7 +6,6 @@ use warnings;
use PVE::Cluster;
use PVE::Network::SDN;
-use PVE::Network::SDN::Ipams;
use PVE::Network::SDN::Subnets;
use PVE::Network::SDN::Dhcp::Plugin;
use PVE::Network::SDN::Dhcp::Dnsmasq;
@@ -22,37 +21,32 @@ sub plugin_types {
return PVE::Network::SDN::Dhcp::Plugin->lookup_types();
}
-sub add_mapping {
- my ($vnetid, $mac, $ip4, $ip6) = @_;
+# re-apply a MAC's mapping on this node from the current records, best effort
+# since the record write stays authoritative. A caller that knows the guest
+# passes its vmid, so only that guest's config is read
+sub update_mapping {
+ my ($vnetid, $mac, $vmid) = @_;
- my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid);
- return if !$vnet;
+ # a record without a MAC has no guest NIC behind it, so no backend serves it
+ return if !$mac;
- my $zoneid = $vnet->{zone};
- my $zone = PVE::Network::SDN::Zones::get_zone($zoneid);
+ eval {
+ # the backend to push to is the applied one, since a zone edited but
+ # not applied has nothing running yet
+ my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid, 1);
+ return if !$vnet;
- return if !$zone->{ipam} || !$zone->{dhcp};
+ my $zoneid = $vnet->{zone};
+ my $zone = PVE::Network::SDN::Zones::get_zone($zoneid, 1);
+ return if !$zone || !$zone->{ipam} || !$zone->{dhcp};
- my $dhcptype = $zone->{dhcp};
+ # a zone confined to other nodes has no backend running here
+ return if defined($zone->{nodes}) && !$zone->{nodes}->{ PVE::INotify::nodename() };
- my $macdb = PVE::Network::SDN::Ipams::read_macdb();
- my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($dhcptype);
- $dhcp_plugin->add_ip_mapping($zoneid, $macdb, $mac, $ip4, $ip6);
-}
-
-sub remove_mapping {
- my ($vnetid, $mac) = @_;
-
- my $vnet = PVE::Network::SDN::Vnets::get_vnet($vnetid);
- return if !$vnet;
-
- my $zoneid = $vnet->{zone};
- my $zone = PVE::Network::SDN::Zones::get_zone($zoneid);
-
- return if !$zone->{ipam} || !$zone->{dhcp};
-
- my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($zone->{dhcp});
- $dhcp_plugin->del_ip_mapping($zoneid, $mac);
+ PVE::Network::SDN::Dhcp::Plugin->lookup($zone->{dhcp})
+ ->update_ip_mapping($zoneid, $mac, $vmid);
+ };
+ warn "could not update dhcp mapping for $mac: $@" if $@;
}
sub regenerate_config {
diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
index a1e4c70f..2d18706f 100644
--- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
+++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
@@ -12,6 +12,7 @@ use File::Copy;
use Net::DBus;
use PVE::RESTEnvironment qw(log_warn);
+use PVE::Network::SDN::Ipams;
my $DNSMASQ_CONFIG_ROOT = '/etc/dnsmasq.d';
my $DNSMASQ_DEFAULT_ROOT = '/etc/default';
@@ -51,83 +52,97 @@ sub update_lease {
$manager->AddDhcpLease($ip4, $mac, \@hostname, undef, 0, 0, 0) if $ip4;
}
-sub add_ip_mapping {
- my ($class, $dhcpid, $macdb, $mac, $ip4, $ip6) = @_;
+sub update_ip_mapping {
+ my ($class, $dhcpid, $mac) = @_;
$mac = lc($mac);
my $ethers_file = ethers_file($dhcpid);
my $ethers_tmp_file = "$ethers_file.tmp";
- my $reload = undef;
+ my ($ip4, $ip6);
+ my $reservation = undef;
+ my $changed = undef;
- my $appendFn = sub {
+ my $rewriteFn = sub {
+ # read under the ethers lock. A snapshot from before it would sweep a
+ # line another push wrote meanwhile, and an older push would put its
+ # addresses back
+
+ my $macs = PVE::Network::SDN::Ipams::read_macdb()->{macs};
+
+ my $entry = $macs->{$mac};
+ ($ip4, $ip6) = ($entry->{ip4}, $entry->{ip6}) if $entry;
+ if ($ip4 || $ip6) {
+ $reservation = $mac;
+ $reservation .= ",$ip4" if $ip4;
+ $reservation .= ",[$ip6]" if $ip6;
+ # an external IPAM spells addresses its own way, while the file holds one spelling
+ $reservation = lc($reservation);
+ }
+
+ # the first reservation of a zone finds no file yet
+ file_set_contents($ethers_file, '') if !-e $ethers_file;
open(my $in, '<', $ethers_file) or die "Could not open file '$ethers_file' $!\n";
open(my $out, '>', $ethers_tmp_file)
or die "Could not open file '$ethers_tmp_file' $!\n";
- my $match = undef;
+ my $kept = undef;
- my $line_no = 0;
while (my $line = <$in>) {
- $line_no++;
chomp($line);
+ next if $line !~ m/\S/;
my ($parsed_mac, $parsed_ip1, $parsed_ip2) = split(/,/, $line);
+ # a line from before the file was written in lower case keeps its writer's spelling
+ $parsed_mac = lc($parsed_mac);
- if (!defined($parsed_mac)) {
- warn "failed to parse MAC from $dhcpid ethers file on line $line_no: '$line'\n";
+ # the MAC's own reservation is rewritten. An unchanged one stays in place
+ if ($parsed_mac eq $mac) {
+ if (defined($reservation) && !$kept && lc($line) eq $reservation) {
+ $kept = 1;
+ print $out "$line\n";
+ } else {
+ $changed = 1;
+ }
next;
}
- # the cache is keyed in lower case, a line from before keeps its writer's spelling
- $parsed_mac = lc($parsed_mac);
my ($parsed_ip4, $parsed_ip6);
if ($parsed_ip2) {
$parsed_ip4 = $parsed_ip1;
$parsed_ip6 = $parsed_ip2;
- } elsif (Net::IP::ip_is_ipv4($parsed_ip1)) {
+ } elsif (defined($parsed_ip1) && Net::IP::ip_is_ipv4($parsed_ip1)) {
$parsed_ip4 = $parsed_ip1;
- } else {
+ } elsif (defined($parsed_ip1)) {
$parsed_ip6 = $parsed_ip1;
}
$parsed_ip6 = $1 if $parsed_ip6 && $parsed_ip6 =~ m/\[(\S+)\]/;
+ # a line from before addresses were lower-cased may spell one in upper case
+ $parsed_ip6 = lc($parsed_ip6) if $parsed_ip6;
# a line that no longer matches the cache is replaced by the cache's
# current one. Dropping it left that guest unknown to dnsmasq until
# its next start
+ my $current = $macs->{$parsed_mac};
if (
- !defined($macdb->{macs}->{$parsed_mac})
- || ($parsed_ip4
- && $macdb->{macs}->{$parsed_mac}->{'ip4'}
- && $macdb->{macs}->{$parsed_mac}->{'ip4'} ne $parsed_ip4)
- || ($parsed_ip6
- && $macdb->{macs}->{$parsed_mac}->{'ip6'}
- && $macdb->{macs}->{$parsed_mac}->{'ip6'} ne $parsed_ip6)
+ !$current
+ || ($parsed_ip4 && $current->{ip4} && $current->{ip4} ne $parsed_ip4)
+ || ($parsed_ip6 && $current->{ip6} && lc($current->{ip6}) ne $parsed_ip6)
) {
- $reload = 1;
- # the MAC being added gets its line below, like a missing one
- my $current = $macdb->{macs}->{$parsed_mac};
- next if $parsed_mac eq $mac || !$current;
+ $changed = 1;
+ next if !$current;
my $replacement = $parsed_mac;
$replacement .= ",$current->{ip4}" if $current->{ip4};
$replacement .= ",[$current->{ip6}]" if $current->{ip6};
- print $out "$replacement\n" if $replacement ne $parsed_mac;
+ print $out lc($replacement) . "\n";
next;
}
- if ($parsed_mac eq $mac) {
- $match = 1 if $ip4 && $parsed_ip4 && $ip4;
- $match = 1 if $ip6 && $parsed_ip6 && $ip6;
- }
-
print $out "$line\n";
}
- if (!$match) {
- my $reservation = $mac;
- $reservation .= ",$ip4" if $ip4;
- $reservation .= ",[$ip6]" if $ip6;
+ if (defined($reservation) && !$kept) {
print $out "$reservation\n";
- $reload = 1;
+ $changed = 1;
}
close $in;
@@ -136,16 +151,22 @@ sub add_ip_mapping {
chmod 0644, $ethers_file;
};
- PVE::Tools::lock_file($ethers_file, 10, $appendFn);
+ # the rewrite replaces the reservation file, so a lock on the file itself
+ # would not outlive the rename
+ PVE::Tools::lock_file("$ethers_file.lock", 10, $rewriteFn);
if ($@) {
- warn "Unable to add $mac to the dnsmasq configuration: $@\n";
+ warn "Unable to update $mac in the dnsmasq configuration: $@\n";
return;
}
- my $service_name = "dnsmasq\@$dhcpid";
- systemctl_service('reload', $service_name) if $reload;
- update_lease($dhcpid, $ip4, $mac);
+ systemctl_service('reload', "dnsmasq\@$dhcpid") if $changed;
+
+ # the reservation is written, so a lease refresh that fails only warns
+ if ($ip4) {
+ eval { update_lease($dhcpid, $ip4, $mac) };
+ log_warn("could not update the dnsmasq lease of $mac: $@") if $@;
+ }
}
sub configure_subnet {
diff --git a/src/PVE/Network/SDN/Dhcp/Plugin.pm b/src/PVE/Network/SDN/Dhcp/Plugin.pm
index b5d32fa1..2fa8683c 100644
--- a/src/PVE/Network/SDN/Dhcp/Plugin.pm
+++ b/src/PVE/Network/SDN/Dhcp/Plugin.pm
@@ -22,8 +22,11 @@ sub private {
return $defaultData;
}
-sub add_ip_mapping {
- my ($class, $dhcpid, $macdb, $mac, $ip4, $ip6) = @_;
+# the MAC's records changed, so make the backend serve the current ones. With no
+# record left its reservation goes. The vmid names the guest when the caller
+# knows it, which an API edit does not
+sub update_ip_mapping {
+ my ($class, $dhcpid, $mac, $vmid) = @_;
die 'implement in sub class';
}
diff --git a/src/PVE/Network/SDN/Vnets.pm b/src/PVE/Network/SDN/Vnets.pm
index c327a4be..84b08037 100644
--- a/src/PVE/Network/SDN/Vnets.pm
+++ b/src/PVE/Network/SDN/Vnets.pm
@@ -211,6 +211,8 @@ sub del_ips_from_mac {
PVE::Network::SDN::Vnets::del_ip($vnetid, $ip4, $hostname, $mac) if $ip4;
PVE::Network::SDN::Vnets::del_ip($vnetid, $ip6, $hostname, $mac) if $ip6;
+ PVE::Network::SDN::Dhcp::update_mapping($vnetid, $mac) if $ip4 || $ip6;
+
return ($ip4, $ip6);
}
@@ -225,11 +227,12 @@ sub add_dhcp_mapping {
return if !$zone->{ipam} || !$zone->{dhcp};
my ($ip4, $ip6) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
+ # one push at the end covers the allocated and the existing records alike
add_next_free_cidr($vnetid, $name, $mac, "$vmid", undef, 1, 4) if !$ip4;
add_next_free_cidr($vnetid, $name, $mac, "$vmid", undef, 1, 6) if !$ip6;
($ip4, $ip6) = PVE::Network::SDN::Vnets::get_ips_from_mac($vnetid, $mac);
- PVE::Network::SDN::Dhcp::add_mapping($vnetid, $mac, $ip4, $ip6) if $ip4 || $ip6;
+ PVE::Network::SDN::Dhcp::update_mapping($vnetid, $mac, $vmid) if $ip4 || $ip6;
}
1;
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index adad5e9a..33a5b44e 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -47,6 +47,7 @@ sub clear_test_state {
vnets_config => {},
macdb => {},
ipamdb => {},
+ dnsmasq_calls => [],
ipam_config => {
'ids' => {
'pve' => {
@@ -221,7 +222,9 @@ $mocked_sdn_dhcp_dnsmasq->mock(
assert_dnsmasq_installed => sub { return 1; },
before_configure => sub { },
ethers_file => sub { return "/tmp/ethers"; },
- systemctl_service => sub { },
+ systemctl_service => sub {
+ push $test_state->{dnsmasq_calls}->@*, $_[0];
+ },
update_lease => sub { },
);
@@ -240,6 +243,11 @@ $mocked_rpc_env_obj->mock(
check_any => sub { return 1; },
);
+my $mocked_pve_inotify = Test::MockModule->new('PVE::INotify');
+$mocked_pve_inotify->mock(
+ nodename => sub { return 'localnode'; },
+);
+
my $mocked_pve_cluster_obj = Test::MockModule->new('PVE::Cluster');
$mocked_pve_cluster_obj->mock(
check_cfs_quorum => sub { return 1; },
@@ -289,6 +297,11 @@ sub create_zone {
return $zone;
}
+sub update_zone {
+ my ($zoneid, $params) = @_;
+ PVE::API2::Network::SDN::Zones->update({ zone => $zoneid, %$params });
+}
+
sub get_vnet {
my ($id) = @_;
return eval { PVE::API2::Network::SDN::Vnets->read({ vnet => $id }); };
@@ -331,6 +344,16 @@ sub create_ip {
return PVE::API2::Network::SDN::Ips->ipcreate($param);
}
+sub update_ip {
+ my ($param) = @_;
+ return PVE::API2::Network::SDN::Ips->ipupdate($param);
+}
+
+sub delete_ip {
+ my ($param) = @_;
+ return PVE::API2::Network::SDN::Ips->ipdelete($param);
+}
+
sub run_test {
my $test = shift;
clear_test_state();
@@ -1109,18 +1132,229 @@ sub test_dnsmasq_stale_reservation_replaced {
};
PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "$stale,10.0.1.10\n$gone,10.0.1.12\n");
- PVE::Network::SDN::Dhcp::Dnsmasq->add_ip_mapping(
- 'TESTZONE', $test_state->{macdb}, $added, '10.0.1.13', undef,
- );
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping('TESTZONE', $added);
- my @lines = sort grep { /\S/ } split(/\n/, PVE::Tools::file_get_contents($TMP_ETHERS_FILE));
- eq_or_diff(
- \@lines,
- ["$stale,10.0.1.11", "$added,10.0.1.13"],
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$stale,10.0.1.11\n$added,10.0.1.13\n",
"$test_name: the stale line is replaced, the unknown one dropped",
);
}
run_test(\&test_dnsmasq_stale_reservation_replaced);
+# -------------- dnsmasq mapping pushes
+
+sub test_dnsmasq_mapping_push {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+
+ # the first push of a zone finds no ethers file yet
+ unlink($TMP_ETHERS_FILE);
+
+ my $ethers = sub { PVE::Tools::file_get_contents($TMP_ETHERS_FILE) };
+ my $reloads = sub {
+ my $count = scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*);
+ $test_state->{dnsmasq_calls} = [];
+ return $count;
+ };
+
+ eval { nic_start($vnetid, $mac, "999", "testhostname"); };
+ if ($@) {
+ fail("$test_name: nic_start: $@");
+ return;
+ }
+ is($ethers->(), "$mac,10.0.0.100\n", "$test_name: guest start reserves the allocation");
+ is($reloads->(), 1, "$test_name: guest start reloads once");
+
+ # the records exist already, so the restart pushes them without a change
+ nic_start($vnetid, $mac, "999", "testhostname");
+ is($ethers->(), "$mac,10.0.0.100\n", "$test_name: restart keeps the reservation");
+ is($reloads->(), 0, "$test_name: restart without a change does not reload");
+
+ update_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ is($ethers->(), "$mac,10.0.0.150\n", "$test_name: mapping edit replaces the reservation");
+ is($reloads->(), 1, "$test_name: mapping edit reloads once");
+
+ # a push writes what the cache holds, so a repeated one rewrites nothing
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
+ is($ethers->(), "$mac,10.0.0.150\n", "$test_name: a repeated push keeps the reservation");
+ is($reloads->(), 0, "$test_name: a repeated push changes nothing");
+
+ # the cache and the file may spell a MAC differently, but it is one MAC
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, uc($mac));
+ is(
+ $ethers->(),
+ "$mac,10.0.0.150\n",
+ "$test_name: a push spelled differently keeps the line as it is",
+ );
+ is($reloads->(), 0, "$test_name: and reloads nothing");
+
+ delete_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ is($ethers->(), "", "$test_name: mapping delete drops the reservation");
+ is($reloads->(), 1, "$test_name: mapping delete reloads once");
+
+ # a record without a MAC has no NIC to serve, so nothing is pushed for it
+ my @warnings;
+ {
+ local $SIG{__WARN__} = sub { push @warnings, @_; };
+ create_ip({ zone => $zoneid, vnet => $vnetid, ip => "10.0.0.160" });
+ delete_ip({ zone => $zoneid, vnet => $vnetid, ip => "10.0.0.160" });
+ }
+ is($reloads->(), 0, "$test_name: a record without a MAC is not pushed");
+ eq_or_diff(\@warnings, [], "$test_name: and warns nothing");
+
+ # a zone confined to other nodes runs no dnsmasq here, so an edit is not
+ # pushed on this node
+ update_zone($zoneid, { nodes => 'other' });
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ is($ethers->(), "", "$test_name: a zone confined to other nodes gets no push here");
+ is($reloads->(), 0, "$test_name: nothing to reload for it");
+}
+
+run_test(\&test_dnsmasq_mapping_push);
+
+sub test_dnsmasq_dual_stack_and_sweep {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "8888::/64",
+ gateway => "8888::1",
+ 'dhcp-range' => ["start-address=8888::100,end-address=8888::200"],
+ });
+
+ # a reservation of a MAC the cache does not know is swept by the next
+ # rewrite, and both addresses of the guest go on one line
+ PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "aa:bb:cc:dd:ee:ff,10.0.0.77\n");
+ $test_state->{dnsmasq_calls} = [];
+
+ eval { nic_start($vnetid, $mac, "999", "testhostname"); };
+ if ($@) {
+ fail("$test_name: nic_start: $@");
+ return;
+ }
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$mac,10.0.0.100,[8888::100]\n",
+ "$test_name: dual-stack reservation on one line, the unknown MAC swept",
+ );
+ is(
+ scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
+ 1,
+ "$test_name: one reload for the rewrite",
+ );
+
+ # an external IPAM may spell an address in upper case, but the file holds
+ # one spelling
+ $test_state->{macdb}->{macs}->{$mac}->{ip6} = '8888::AB';
+ $test_state->{dnsmasq_calls} = [];
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$mac,10.0.0.100,[8888::ab]\n",
+ "$test_name: the reservation is written in one spelling",
+ );
+ is(
+ scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
+ 1,
+ "$test_name: one reload for the changed address",
+ );
+
+ # an entry without an address, as a miss once left behind, says nothing
+ # about its MAC, so a line of it is swept
+ my $stale = 'da:65:8f:18:9b:70';
+ $test_state->{macdb}->{macs}->{$stale} = { ip4 => undef, ip6 => undef };
+ PVE::Tools::file_set_contents(
+ $TMP_ETHERS_FILE,
+ "$mac,10.0.0.100,[8888::ab]\n$stale,10.0.0.101\n",
+ );
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$mac,10.0.0.100,[8888::ab]\n",
+ "$test_name: a line of a MAC cached without an address is swept",
+ );
+
+ # a line with an upper-case IPv6 address from before they were lower-cased is kept
+ my $other = 'da:65:8f:18:9b:71';
+ $test_state->{macdb}->{macs}->{$other} = { ip4 => '10.0.0.102', ip6 => '8888::AB' };
+ PVE::Tools::file_set_contents(
+ $TMP_ETHERS_FILE,
+ "$mac,10.0.0.100,[8888::ab]\n$other,10.0.0.102,[8888::AB]\n",
+ );
+ $test_state->{dnsmasq_calls} = [];
+ PVE::Network::SDN::Dhcp::Dnsmasq->update_ip_mapping($zoneid, $mac);
+ is(
+ PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
+ "$mac,10.0.0.100,[8888::ab]\n$other,10.0.0.102,[8888::AB]\n",
+ "$test_name: a line of another MAC spelled by the old code is kept",
+ );
+ is(
+ scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
+ 0,
+ "$test_name: without a reload",
+ );
+}
+
+run_test(\&test_dnsmasq_dual_stack_and_sweep);
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-network v4 09/20] sdn: dhcp: make the lease time configurable per subnet
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (7 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-network v4 08/20] sdn: dhcp: update reservations when ip mappings change Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-network v4 10/20] sdn: dhcp: require a backend only if a zone on this node uses it Hannes Laimer
` (10 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
dnsmasq hands out infinite leases, so a guest asks for its address
again only at its next reboot or network restart. A finite lease makes
it renew within the lease time. An address changed in the IPAM goes to
dnsmasq on the node the change is made on, so a guest there picks up
the new address at its next renewal. Guests on other nodes get it only
at their next start there.
A subnet without a lease time keeps infinite leases.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com>
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 3 +-
src/PVE/Network/SDN/SubnetPlugin.pm | 11 +++++++
src/test/run_test_vnets_blackbox.pl | 51 +++++++++++++++++++++++++++++
3 files changed, 64 insertions(+), 1 deletion(-)
diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
index 2d18706f..79315e0f 100644
--- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
+++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
@@ -188,7 +188,8 @@ sub configure_subnet {
$mask = join('.', unpack("C4", pack("N", $mask)));
}
- push @{$config}, "dhcp-range=set:$tag,$network,static,$mask,infinite";
+ my $lease = $subnet_config->{'dhcp-lease-time'} // 'infinite';
+ push @{$config}, "dhcp-range=set:$tag,$network,static,$mask,$lease";
my $option_string;
if (ip_is_ipv6($subnet_config->{network})) {
diff --git a/src/PVE/Network/SDN/SubnetPlugin.pm b/src/PVE/Network/SDN/SubnetPlugin.pm
index e2a0e50d..29edf7b6 100644
--- a/src/PVE/Network/SDN/SubnetPlugin.pm
+++ b/src/PVE/Network/SDN/SubnetPlugin.pm
@@ -177,6 +177,16 @@ sub properties {
description => 'IP address for the DNS server',
optional => 1,
},
+ 'dhcp-lease-time' => {
+ type => 'integer',
+ minimum => 60,
+ maximum => 4294967295,
+ description =>
+ 'Lease time in seconds for DHCP answers. Without it dnsmasq hands out'
+ . ' infinite leases, and it raises anything below two minutes to two'
+ . ' minutes.',
+ optional => 1,
+ },
};
}
@@ -189,6 +199,7 @@ sub options {
dnszoneprefix => { optional => 1 },
'dhcp-range' => { optional => 1 },
'dhcp-dns-server' => { optional => 1 },
+ 'dhcp-lease-time' => { optional => 1 },
};
}
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 33a5b44e..7bc18de1 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -335,6 +335,11 @@ sub create_subnet {
PVE::API2::Network::SDN::Subnets->create($params);
}
+sub update_subnet {
+ my ($params) = @_;
+ PVE::API2::Network::SDN::Subnets->update($params);
+}
+
sub get_ipam_entries {
return PVE::API2::Network::SDN::Ipams->ipamindex({ ipam => "pve" });
}
@@ -1250,6 +1255,52 @@ sub test_dnsmasq_mapping_push {
run_test(\&test_dnsmasq_mapping_push);
+sub test_dnsmasq_lease_time {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+
+ # the subnet's lease time reaches the range, and infinite stays the default
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ my $range = sub {
+ my $subnets = PVE::Network::SDN::Vnets::get_subnets($vnetid, 1);
+ my $config = [];
+ for my $id (sort keys %$subnets) {
+ PVE::Network::SDN::Dhcp::Dnsmasq->configure_subnet(
+ $config, $zoneid, $vnetid, $subnets->{$id},
+ );
+ }
+ my ($line) = grep { m/^dhcp-range=/ } @$config;
+ return $line;
+ };
+ like($range->(), qr/,infinite$/, "$test_name: no lease time means infinite leases");
+ update_subnet({
+ vnet => $vnetid,
+ subnet => "$zoneid-10.0.0.0-24",
+ 'dhcp-lease-time' => 300,
+ });
+ like($range->(), qr/,300$/, "$test_name: the lease time reaches the range");
+}
+
+run_test(\&test_dnsmasq_lease_time);
+
sub test_dnsmasq_dual_stack_and_sweep {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-network v4 10/20] sdn: dhcp: require a backend only if a zone on this node uses it
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (8 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-network v4 09/20] sdn: dhcp: make the lease time configurable per subnet Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-network v4 11/20] sdn: dhcp: add an eBPF backend answering on each guest interface Hannes Laimer
` (9 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
A network reload failed on a node without the dnsmasq package as soon
as any zone had DHCP, even a zone confined to other nodes. On a node
with dnsmasq, the reload wrote such a zone's configuration and started
an instance for it, although none of the zone's vnets exist there.
A reload now skips the zones confined to other nodes, so a backend is
required and started only for the zones on the reloading node.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com>
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
src/PVE/Network/SDN/Dhcp.pm | 13 +++++++--
src/test/run_test_vnets_blackbox.pl | 43 +++++++++++++++++++++++++++++
2 files changed, 53 insertions(+), 3 deletions(-)
diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
index ab5d6b5b..2a023534 100644
--- a/src/PVE/Network/SDN/Dhcp.pm
+++ b/src/PVE/Network/SDN/Dhcp.pm
@@ -63,17 +63,24 @@ sub regenerate_config {
my $plugins = PVE::Network::SDN::Dhcp::Plugin->lookup_types();
- my $any_zone_needs_dhcp = grep { $_->{dhcp} } values $zone_cfg->{ids}->%*;
+ my %plugin_needed = ();
+ for my $zone (values $zone_cfg->{ids}->%*) {
+ next if !$zone->{dhcp};
+ # a zone confined to other nodes runs no backend here
+ next if defined($zone->{nodes}) && !$zone->{nodes}->{$nodename};
+ $plugin_needed{ $zone->{dhcp} } = 1;
+ }
foreach my $plugin_name (@$plugins) {
my $plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($plugin_name);
- eval { $plugin->before_regenerate(!$any_zone_needs_dhcp) };
+ eval { $plugin->before_regenerate(!$plugin_needed{$plugin_name}) };
die "Could not run before_regenerate for DHCP plugin $plugin_name $@\n" if $@;
}
foreach my $zoneid (sort keys %{ $zone_cfg->{ids} }) {
my $zone = $zone_cfg->{ids}->{$zoneid};
next if !$zone->{dhcp};
+ next if defined($zone->{nodes}) && !$zone->{nodes}->{$nodename};
my $dhcp_plugin_name = $zone->{dhcp};
my $dhcp_plugin = PVE::Network::SDN::Dhcp::Plugin->lookup($dhcp_plugin_name);
@@ -114,7 +121,7 @@ sub regenerate_config {
warn "Could not configure vnet $vnetid: $@\n" if $@;
}
- eval { $dhcp_plugin->after_configure($zoneid, !$any_zone_needs_dhcp) };
+ eval { $dhcp_plugin->after_configure($zoneid, !$plugin_needed{$dhcp_plugin_name}) };
warn "Could not run after_configure for DHCP server $zoneid $@\n" if $@;
}
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 7bc18de1..8ad1e255 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -105,6 +105,13 @@ my $mocked_pve_sdn;
$mocked_pve_sdn = Test::MockModule->new('PVE::Network::SDN');
$mocked_pve_sdn->mock(
cfs_lock_file => $mocked_cfs_lock_file,
+ running_config => sub {
+ return {
+ zones => $test_state->{zones_config},
+ vnets => $test_state->{vnets_config},
+ subnets => $test_state->{subnets_config},
+ };
+ },
);
my $mocked_pve_tools = Test::MockModule->new('PVE::Tools');
@@ -1301,6 +1308,42 @@ sub test_dnsmasq_lease_time {
run_test(\&test_dnsmasq_lease_time);
+sub test_dhcp_backend_needed_per_node {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+
+ # a backend is told whether a zone on this node uses it. A zone confined
+ # to other nodes runs nothing here
+ my $asked = [];
+ my $configured = [];
+ $mocked_sdn_dhcp_dnsmasq->mock(
+ before_regenerate => sub { push @$asked, $_[1] ? 'optional' : 'needed'; },
+ after_configure => sub { push @$configured, $_[1]; },
+ );
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => "DNSZONE",
+ nodes => 'other',
+ });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ $asked,
+ ['optional'],
+ "$test_name: a dnsmasq zone confined elsewhere needs no dnsmasq here",
+ );
+ eq_or_diff($configured, [], "$test_name: and is not configured here");
+
+ @$asked = ();
+ update_zone("DNSZONE", { delete => 'nodes' });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff($asked, ['needed'], "$test_name: a dnsmasq zone on this node needs it");
+ eq_or_diff($configured, ['DNSZONE'], "$test_name: and is configured");
+ $mocked_sdn_dhcp_dnsmasq->unmock($_) for qw(before_regenerate after_configure);
+}
+
+run_test(\&test_dhcp_backend_needed_per_node);
+
sub test_dnsmasq_dual_stack_and_sweep {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-network v4 11/20] sdn: dhcp: add an eBPF backend answering on each guest interface
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (9 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-network v4 10/20] sdn: dhcp: require a backend only if a zone on this node uses it Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-network v4 12/20] sdn: zones: report interface plugs and unplugs to the DHCP backends Hannes Laimer
` (8 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
The ebpf DHCP backend answers DHCPv4 from a program in the kernel on
every guest interface of its zones, with the address the SDN MAC cache
holds for the interface's MAC. No server runs per zone, and IPv6
subnets are not served. A subnet without a gateway is served under a
link-local server identifier no host owns, so its guests renew by
broadcast at their rebinding time.
An apply runs a full pass on each node, and on a node that no zone
with the ebpf backend includes, the pass removes the responder
altogether. A record change rewrites the answers for its MAC on the
node the change is made on, and the other nodes catch up at their
next apply.
The responder is pinned in bpffs and does not survive a reboot, so a
boot hook runs a full pass. Removing the package deletes those pins,
since nothing would detach the responder afterward.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Notes:
v4:
* key the guest NIC list by lower-cased MAC, a NIC line without a MAC
is left out
debian/libpve-network-perl.prerm | 13 +
src/PVE/API2/Network/SDN/Zones.pm | 3 +-
src/PVE/Network/SDN/Dhcp.pm | 82 ++++
src/PVE/Network/SDN/Dhcp/Ebpf.pm | 295 +++++++++++++
src/PVE/Network/SDN/Dhcp/Makefile | 2 +-
src/PVE/Network/SDN/Dhcp/Plugin.pm | 19 +
src/PVE/Network/SDN/SubnetPlugin.pm | 4 +-
src/test/run_test_vnets_blackbox.pl | 642 ++++++++++++++++++++++++++++
8 files changed, 1056 insertions(+), 4 deletions(-)
create mode 100755 debian/libpve-network-perl.prerm
create mode 100644 src/PVE/Network/SDN/Dhcp/Ebpf.pm
diff --git a/debian/libpve-network-perl.prerm b/debian/libpve-network-perl.prerm
new file mode 100755
index 00000000..d28c74fe
--- /dev/null
+++ b/debian/libpve-network-perl.prerm
@@ -0,0 +1,13 @@
+#!/bin/sh
+
+set -e
+
+case "$1" in
+ remove)
+ # the eBPF DHCP responder stays attached through its pins in bpffs, and with
+ # the package gone nothing would ever update or detach it
+ rm -rf /sys/fs/bpf/proxmox-ebpf/dhcp
+ ;;
+esac
+
+exit 0
diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm
index b897cbdf..0e907260 100644
--- a/src/PVE/API2/Network/SDN/Zones.pm
+++ b/src/PVE/API2/Network/SDN/Zones.pm
@@ -16,6 +16,7 @@ use PVE::Network::SDN::Dns;
use PVE::Network::SDN::Subnets;
use PVE::Network::SDN::Vnets;
use PVE::Network::SDN;
+use PVE::Network::SDN::Dhcp;
use PVE::Network::SDN::Zones::EvpnPlugin;
use PVE::Network::SDN::Zones::FaucetPlugin;
@@ -90,7 +91,7 @@ my $ZONE_PROPERTIES = {
},
dhcp => {
type => 'string',
- enum => ['dnsmasq'],
+ enum => PVE::Network::SDN::Dhcp->plugin_types(),
optional => 1,
description => 'Name of DHCP server backend for this zone.',
},
diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
index 2a023534..5a76cfda 100644
--- a/src/PVE/Network/SDN/Dhcp.pm
+++ b/src/PVE/Network/SDN/Dhcp.pm
@@ -9,6 +9,7 @@ use PVE::Network::SDN;
use PVE::Network::SDN::Subnets;
use PVE::Network::SDN::Dhcp::Plugin;
use PVE::Network::SDN::Dhcp::Dnsmasq;
+use PVE::Network::SDN::Dhcp::Ebpf;
use PVE::INotify;
@@ -17,6 +18,9 @@ PVE::Network::SDN::Dhcp::Plugin->init();
PVE::Network::SDN::Dhcp::Dnsmasq->register();
PVE::Network::SDN::Dhcp::Dnsmasq->init();
+PVE::Network::SDN::Dhcp::Ebpf->register();
+PVE::Network::SDN::Dhcp::Ebpf->init();
+
sub plugin_types {
return PVE::Network::SDN::Dhcp::Plugin->lookup_types();
}
@@ -49,6 +53,84 @@ sub update_mapping {
warn "could not update dhcp mapping for $mac: $@" if $@;
}
+# the NIC lines come from pmxcfs in one go, so no guest config gets parsed for
+# this. With a vmid only that guest's lines are fetched. The map is keyed by
+# the lower-cased MAC and holds a list per key, since a MAC is unique within a
+# guest but not across the cluster
+sub guest_nics {
+ my ($vmid) = @_;
+
+ # a hook script starts with the cluster cache empty
+ PVE::Cluster::cfs_update();
+ my $vmlist = PVE::Cluster::get_vmlist();
+ my $nets = PVE::Cluster::get_guest_config_properties([map { "net$_" } 0 .. 31], $vmid);
+ my $nics = {};
+ for my $vmid (sort keys %$nets) {
+ my $guest = $vmlist->{ids}->{$vmid} // next;
+ my $prefix = ($guest->{type} // '') eq 'lxc' ? 'veth' : 'tap';
+ for my $key (sort keys %{ $nets->{$vmid} }) {
+ my ($index) = $key =~ m/^net(\d+)$/ or next;
+ my $net = $nets->{$vmid}->{$key};
+ my ($bridge) = $net =~ m/(?:^|,)bridge=([^,]+)/ or next;
+ # a VM names the MAC as the value of its model, a container as hwaddr
+ my ($mac) = $net =~ m/(?:^|,)[a-z0-9_-]+=([0-9a-f]{2}(?::[0-9a-f]{2}){5})(?:,|$)/i
+ or next;
+ push $nics->{ lc($mac) }->@*,
+ {
+ vmid => $vmid,
+ node => $guest->{node},
+ iface => "$prefix${vmid}i$index",
+ bridge => $bridge,
+ mac => lc($mac),
+ };
+ }
+ }
+
+ return $nics;
+}
+
+sub tap_plug {
+ my ($bridge, $iface, $mac) = @_;
+
+ my $vnet = PVE::Network::SDN::Vnets::get_vnet($bridge, 1);
+ my $zone = $vnet ? PVE::Network::SDN::Zones::get_zone($vnet->{zone}, 1) : undef;
+ my $backend = $zone ? $zone->{dhcp} : undef;
+
+ # every other backend hears where the interface sits now. A plain bridge
+ # is no vnet
+ my $target = { vnet => $vnet ? $bridge : undef };
+ for my $type (PVE::Network::SDN::Dhcp::Plugin->lookup_types()->@*) {
+ next if defined($backend) && $type eq $backend;
+ eval { PVE::Network::SDN::Dhcp::Plugin->lookup($type)->tap_unplug($iface, $target) };
+ warn "could not unplug $iface from the $type dhcp backend: $@" if $@;
+ }
+ return if !$backend;
+
+ eval {
+ PVE::Network::SDN::Dhcp::Plugin->lookup($backend)
+ ->tap_plug($vnet->{zone}, $bridge, $iface, $mac);
+ };
+ warn "could not plug $iface into the dhcp backend of zone $vnet->{zone}: $@" if $@;
+}
+
+sub boot {
+ for my $type (PVE::Network::SDN::Dhcp::Plugin->lookup_types()->@*) {
+ eval { PVE::Network::SDN::Dhcp::Plugin->lookup($type)->boot() };
+ warn "could not bring up the $type dhcp backend: $@" if $@;
+ }
+}
+
+# the interface is usually gone already and with it any way to its zone, so
+# every backend is told
+sub tap_unplug {
+ my ($iface) = @_;
+
+ for my $type (PVE::Network::SDN::Dhcp::Plugin->lookup_types()->@*) {
+ eval { PVE::Network::SDN::Dhcp::Plugin->lookup($type)->tap_unplug($iface) };
+ warn "could not unplug $iface from the $type dhcp backend: $@" if $@;
+ }
+}
+
sub regenerate_config {
my ($reload) = @_;
diff --git a/src/PVE/Network/SDN/Dhcp/Ebpf.pm b/src/PVE/Network/SDN/Dhcp/Ebpf.pm
new file mode 100644
index 00000000..f5f5474d
--- /dev/null
+++ b/src/PVE/Network/SDN/Dhcp/Ebpf.pm
@@ -0,0 +1,295 @@
+package PVE::Network::SDN::Dhcp::Ebpf;
+
+use strict;
+use warnings;
+
+use base qw(PVE::Network::SDN::Dhcp::Plugin);
+
+use Net::IP;
+use Net::Subnet qw(subnet_matcher);
+
+use PVE::Cluster;
+use PVE::INotify;
+use PVE::Network::SDN::Ipams;
+use PVE::RESTEnvironment qw(log_warn);
+
+use PVE::RS::SDN::Dhcp;
+use PVE::RS::SDN::Dhcp::ExclusiveLock;
+use PVE::RS::SDN::Dhcp::SharedLock;
+
+my $DEFAULT_LEASE_TIME = 600;
+
+# a subnet without a gateway is served under a link-local server identifier,
+# which no host on the subnet owns. A guest cannot renew by unicast then and
+# broadcasts at its rebinding time instead
+my $SERVER_ID_NO_GATEWAY = '169.254.0.1';
+
+sub type {
+ return 'ebpf';
+}
+
+# the MTU option is 16 bits wide with 68 the smallest MTU IPv4 allows, so a
+# zone MTU outside that is reported on an apply and not handed out. A zone
+# without one hands out none and the guest keeps its own setting
+my sub zone_mtu {
+ my ($zoneid, $zone, $report) = @_;
+
+ my $mtu = PVE::Network::SDN::Zones::Plugin->lookup($zone->{type})->get_mtu($zone);
+ return undef if !defined($mtu);
+ return $mtu if $mtu >= 68 && $mtu <= 65535;
+ log_warn("zone $zoneid has MTU $mtu, not handing it out over DHCP") if $report;
+ return undef;
+}
+
+my sub dhcp_record {
+ my ($ip4, $subnet, $mtu, $mac) = @_;
+
+ my $gateway = $subnet->{gateway};
+
+ # a DNS server has to be reachable over the address family served
+ my $dns = $subnet->{'dhcp-dns-server'};
+ $dns = undef if defined($dns) && !Net::IP::ip_is_ipv4($dns);
+
+ # the config hands its numbers over as strings, while the bindings take integers only
+ return {
+ ip => $ip4,
+ prefixlen => int($subnet->{mask}),
+ server_id => $gateway // $SERVER_ID_NO_GATEWAY,
+ lease => int($subnet->{'dhcp-lease-time'} // $DEFAULT_LEASE_TIME),
+ router => $gateway,
+ dns => $dns,
+ mac => lc($mac),
+ mtu => defined($mtu) ? int($mtu) : undef,
+ };
+}
+
+# the IPv4 subnets of a vnet, warning about a DNS server the reply cannot
+# carry when asked to. Only the apply asks, so a guest start does not repeat
+# the warning
+my sub served_subnets {
+ my ($vnetid, $report) = @_;
+
+ my $subnets = PVE::Network::SDN::Vnets::get_subnets($vnetid, 1) // {};
+ my $served = [];
+ for my $subnetid (sort keys %$subnets) {
+ my $subnet = $subnets->{$subnetid};
+ next if !Net::IP::ip_is_ipv4($subnet->{network});
+ log_warn("subnet $subnetid has an IPv6 DNS server, not handing it out over IPv4")
+ if $report
+ && defined($subnet->{'dhcp-dns-server'})
+ && !Net::IP::ip_is_ipv4($subnet->{'dhcp-dns-server'});
+ push @$served, $subnet;
+ }
+
+ return $served;
+}
+
+my $matchers = {};
+
+# the vnet's subnet holding the cached address decides the answer
+my sub vnet_record {
+ my ($subnets, $mtu, $macs, $mac) = @_;
+
+ my $ip4 = $macs->{ lc($mac) }->{ip4};
+ return undef if !$ip4;
+ # the most specific of overlapping subnets is the one to answer from
+ for my $subnet (sort { $b->{mask} <=> $a->{mask} } @$subnets) {
+ my $matcher = $matchers->{ $subnet->{cidr} } //= subnet_matcher($subnet->{cidr});
+ next if !$matcher->($ip4);
+ # the vnet's own gateway address is cached too and never a lease
+ return undef if defined($subnet->{gateway}) && $ip4 eq $subnet->{gateway};
+ return dhcp_record($ip4, $subnet, $mtu, $mac);
+ }
+
+ return undef;
+}
+
+my sub full_state {
+ my ($report) = @_;
+
+ # the configs and the cache are read after the cluster state is refreshed,
+ # since a copy from before the lock was taken would hand the pass older input
+ PVE::Cluster::cfs_update();
+ my $cfg = PVE::Network::SDN::running_config();
+ my $zones = $cfg->{zones}->{ids} // {};
+ my $vnets = $cfg->{vnets}->{ids} // {};
+ # an ebpf zone confined to other nodes runs no responder here
+ my $nodename = PVE::INotify::nodename();
+ my %ebpf = map { $_ => 1 } grep {
+ ($zones->{$_}->{dhcp} // '') eq 'ebpf'
+ && (!defined($zones->{$_}->{nodes}) || $zones->{$_}->{nodes}->{$nodename})
+ } keys %$zones;
+ return (0, []) if !%ebpf;
+
+ my $macs = PVE::Network::SDN::Ipams::read_macdb()->{macs};
+ my (%mtu, %subnets);
+ # a subnet or a zone MTU the responder cannot serve is reported per apply,
+ # guests or not
+ if ($report) {
+ for my $vnetid (sort keys %$vnets) {
+ my $zoneid = $vnets->{$vnetid}->{zone};
+ next if !$zoneid || !$ebpf{$zoneid};
+ $subnets{$vnetid} //= served_subnets($vnetid, 1);
+ $mtu{$zoneid} = zone_mtu($zoneid, $zones->{$zoneid}, 1) if !exists $mtu{$zoneid};
+ }
+ }
+ my $ifaces = [];
+ my @nics = map { $_->@* } values PVE::Network::SDN::Dhcp::guest_nics()->%*;
+ for my $nic (sort { $a->{vmid} cmp $b->{vmid} || $a->{iface} cmp $b->{iface} } @nics) {
+ my $vnetid = $nic->{bridge};
+ my $zoneid = $vnets->{$vnetid} ? $vnets->{$vnetid}->{zone} : undef;
+ my $serve = $zoneid && $ebpf{$zoneid} ? 1 : 0;
+ my $record;
+ if ($serve) {
+ $mtu{$zoneid} = zone_mtu($zoneid, $zones->{$zoneid}, $report) if !exists $mtu{$zoneid};
+ $subnets{$vnetid} //= served_subnets($vnetid, $report);
+ $record = vnet_record($subnets{$vnetid}, $mtu{$zoneid}, $macs, $nic->{mac})
+ if $nic->{mac};
+ }
+ push @$ifaces,
+ {
+ name => $nic->{iface},
+ vnet => $vnets->{$vnetid} ? $vnetid : undef,
+ serve => $serve,
+ record => $record,
+ };
+ }
+
+ return (scalar(keys %ebpf), $ifaces);
+}
+
+my sub full_pass {
+ my ($report) = @_;
+
+ my $lock = eval { PVE::RS::SDN::Dhcp::ExclusiveLock->new() };
+ if (!$lock) {
+ log_warn("could not lock the DHCP responder: $@");
+ return;
+ }
+
+ my ($zones, $ifaces) = eval { full_state($report) };
+ if ($@) {
+ log_warn("could not collect the DHCP responder state: $@");
+ return;
+ }
+
+ eval { $zones ? $lock->apply($ifaces) : $lock->clear() };
+ log_warn("could not apply the DHCP responder state: $@") if $@;
+}
+
+my sub mac_ifaces {
+ my ($mac, $macs, $vmid) = @_;
+
+ my $cfg = PVE::Network::SDN::running_config();
+ my $zones = $cfg->{zones}->{ids} // {};
+ my $vnets = $cfg->{vnets}->{ids} // {};
+ my $ifaces = [];
+ for my $nic ((PVE::Network::SDN::Dhcp::guest_nics($vmid)->{ lc($mac) } // [])->@*) {
+ my $vnet = $vnets->{ $nic->{bridge} } // next;
+ my $zone = $zones->{ $vnet->{zone} } // next;
+ next if ($zone->{dhcp} // '') ne 'ebpf';
+ my $record = vnet_record(
+ served_subnets($nic->{bridge}, 0),
+ zone_mtu($vnet->{zone}, $zone, 0),
+ $macs,
+ $mac,
+ );
+ push @$ifaces,
+ { name => $nic->{iface}, vnet => $nic->{bridge}, serve => 1, record => $record };
+ }
+
+ return $ifaces;
+}
+
+# under the guest lock nothing else writes the guest's entries, so a caller
+# that names the guest takes the shared lock. An API edit names no guest and
+# takes the exclusive one
+sub update_ip_mapping {
+ my ($class, $dhcpid, $mac, $vmid) = @_;
+
+ my $update = sub {
+ my $lock =
+ defined($vmid)
+ ? PVE::RS::SDN::Dhcp::SharedLock->new()
+ : PVE::RS::SDN::Dhcp::ExclusiveLock->new();
+ PVE::Cluster::cfs_update();
+ my $ifaces =
+ mac_ifaces($mac, PVE::Network::SDN::Ipams::read_macdb()->{macs}, $vmid);
+ return 1 if !@$ifaces;
+ return $lock->update($ifaces);
+ };
+
+ my $done = eval { $update->() };
+ if ($@) {
+ log_warn("could not update the DHCP responder records of $mac: $@");
+ return;
+ }
+ return if $done;
+
+ # nothing loaded, the pass takes its own lock once the update's is gone
+ full_pass(0);
+}
+
+# the dispatcher's per-zone walk is not needed, since the full pass collects
+# the state itself
+sub before_regenerate { }
+sub before_configure { }
+sub configure_subnet { }
+sub configure_range { }
+sub configure_vnet { }
+sub after_configure { }
+
+sub after_regenerate {
+ my ($class) = @_;
+
+ full_pass(1);
+}
+
+sub boot {
+ my ($class) = @_;
+
+ full_pass(1);
+}
+
+sub tap_unplug {
+ my ($class, $iface, $target) = @_;
+
+ eval { PVE::RS::SDN::Dhcp::detach($iface, $target) };
+ log_warn("could not detach the DHCP responder from $iface: $@") if $@;
+}
+
+sub tap_plug {
+ my ($class, $dhcpid, $vnetid, $iface, $mac) = @_;
+
+ my $attach = sub {
+ my $lock = PVE::RS::SDN::Dhcp::SharedLock->new();
+ my $record;
+ if ($mac) {
+ PVE::Cluster::cfs_update();
+ my $zone = PVE::Network::SDN::Zones::get_zone($dhcpid, 1);
+ $record = vnet_record(
+ served_subnets($vnetid, 0),
+ zone_mtu($dhcpid, $zone, 0),
+ PVE::Network::SDN::Ipams::read_macdb()->{macs},
+ $mac,
+ );
+ }
+ return $lock->attach($iface, $vnetid, $record);
+ };
+
+ my $done = eval { $attach->() };
+ if ($@) {
+ log_warn("could not attach DHCP responder to $iface: $@");
+ return;
+ }
+ return if $done;
+
+ # nothing is loaded, after a rebuild or a boot where the SDN commit loaded
+ # nothing. The plug's lock is gone by here, so the pass can take its own
+ full_pass(0);
+ $done = eval { $attach->() };
+ log_warn("could not attach DHCP responder to $iface: " . ($@ || "nothing is loaded"))
+ if $@ || !$done;
+}
+
+1;
diff --git a/src/PVE/Network/SDN/Dhcp/Makefile b/src/PVE/Network/SDN/Dhcp/Makefile
index 65465135..ce86aaef 100644
--- a/src/PVE/Network/SDN/Dhcp/Makefile
+++ b/src/PVE/Network/SDN/Dhcp/Makefile
@@ -1,4 +1,4 @@
-SOURCES=Plugin.pm Dnsmasq.pm
+SOURCES=Plugin.pm Dnsmasq.pm Ebpf.pm
PERL5DIR=${DESTDIR}/usr/share/perl5
diff --git a/src/PVE/Network/SDN/Dhcp/Plugin.pm b/src/PVE/Network/SDN/Dhcp/Plugin.pm
index 2fa8683c..149873e6 100644
--- a/src/PVE/Network/SDN/Dhcp/Plugin.pm
+++ b/src/PVE/Network/SDN/Dhcp/Plugin.pm
@@ -60,9 +60,28 @@ sub before_regenerate {
die 'implement in sub class';
}
+# the node booted. A backend whose state does not survive a reboot brings it
+# back here, before the guests start
+sub boot {
+ my ($class) = @_;
+}
+
sub after_regenerate {
my ($class) = @_;
die 'implement in sub class';
}
+# a guest interface was plugged into a vnet of a zone using this backend.
+# Backends serving the bridge rather than the interface have nothing to do
+sub tap_plug {
+ my ($class, $dhcpid, $vnetid, $iface, $mac) = @_;
+}
+
+# a guest interface left its vnet or went away, and by then nothing says which
+# zone it was on. The target is the vnet it sits on now, none for a plain
+# bridge, and no target means it is gone
+sub tap_unplug {
+ my ($class, $iface, $target) = @_;
+}
+
1;
diff --git a/src/PVE/Network/SDN/SubnetPlugin.pm b/src/PVE/Network/SDN/SubnetPlugin.pm
index 29edf7b6..38579f21 100644
--- a/src/PVE/Network/SDN/SubnetPlugin.pm
+++ b/src/PVE/Network/SDN/SubnetPlugin.pm
@@ -183,8 +183,8 @@ sub properties {
maximum => 4294967295,
description =>
'Lease time in seconds for DHCP answers. Without it dnsmasq hands out'
- . ' infinite leases, and it raises anything below two minutes to two'
- . ' minutes.',
+ . ' infinite leases and the ebpf responder ten minutes. dnsmasq raises'
+ . ' anything below two minutes to two minutes.',
optional => 1,
},
};
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 8ad1e255..30a4ab82 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -48,6 +48,9 @@ sub clear_test_state {
macdb => {},
ipamdb => {},
dnsmasq_calls => [],
+ ebpf_calls => [],
+ order => [],
+ warnings => [],
ipam_config => {
'ids' => {
'pve' => {
@@ -55,6 +58,7 @@ sub clear_test_state {
},
},
},
+ ebpf_answer => undef,
};
PVE::Tools::file_set_contents($TMP_ETHERS_FILE, "\n");
}
@@ -235,6 +239,43 @@ $mocked_sdn_dhcp_dnsmasq->mock(
update_lease => sub { },
);
+my $mocked_pve_rs_dhcp = Test::MockModule->new('PVE::RS::SDN::Dhcp');
+$mocked_pve_rs_dhcp->mock(
+ detach => sub {
+ push $test_state->{ebpf_calls}->@*, { method => 'detach', args => [@_] };
+ return $test_state->{ebpf_answer}->{detach} // 1;
+ },
+);
+
+# taking a lock is the ordered step, and the responder calls are its methods
+my %mocked_pve_rs_dhcp_locks;
+for my $class (qw(SharedLock ExclusiveLock)) {
+ my $package = "PVE::RS::SDN::Dhcp::$class";
+ my $mock = Test::MockModule->new($package, no_auto => 1);
+ $mock->mock(
+ new => sub {
+ push $test_state->{order}->@*, 'lock';
+ push $test_state->{lock_modes}->@*, $class eq 'SharedLock' ? 'shared' : 'exclusive';
+ return bless {}, $package;
+ },
+ map {
+ my $method = $_;
+ $method => sub {
+ my ($self, @args) = @_;
+ push $test_state->{order}->@*, 'write';
+ push $test_state->{ebpf_calls}->@*, { method => $method, args => [@args] };
+ return $test_state->{ebpf_answer}->{$method} // 1;
+ };
+ } ($class eq 'SharedLock' ? qw(attach update) : qw(apply update clear)),
+ );
+ $mocked_pve_rs_dhcp_locks{$class} = $mock;
+}
+
+my $mocked_sdn_dhcp_ebpf = Test::MockModule->new('PVE::Network::SDN::Dhcp::Ebpf');
+$mocked_sdn_dhcp_ebpf->mock(
+ log_warn => sub { push $test_state->{warnings}->@*, $_[0]; },
+);
+
my $mocked_api_zones = Test::MockModule->new('PVE::API2::Network::SDN::Zones');
$mocked_api_zones->mock(
create_etc_interfaces_sdn_dir => sub { },
@@ -259,6 +300,15 @@ my $mocked_pve_cluster_obj = Test::MockModule->new('PVE::Cluster');
$mocked_pve_cluster_obj->mock(
check_cfs_quorum => sub { return 1; },
cfs_lock_domain => $mocked_cfs_lock_domain,
+ cfs_update => sub { push $test_state->{order}->@*, 'refresh'; },
+ get_vmlist => sub { return $test_state->{vmlist} // { ids => {} }; },
+ get_guest_config_properties => sub {
+ my ($properties, $vmid) = @_;
+ my $nets = $test_state->{guest_nets} // {};
+ push $test_state->{nic_listings}->@*, $vmid;
+ return $nets if !defined($vmid);
+ return exists $nets->{$vmid} ? { $vmid => $nets->{$vmid} } : {};
+ },
);
# ------- TEST FUNCTIONS --------------
@@ -366,6 +416,24 @@ sub delete_ip {
return PVE::API2::Network::SDN::Ips->ipdelete($param);
}
+sub take_ebpf_calls {
+ my $calls = $test_state->{ebpf_calls};
+ $test_state->{ebpf_calls} = [];
+ return $calls;
+}
+
+# the lock takes, cluster refreshes and responder writes in the order they
+# ran, a repeated step collapsed into one
+sub take_order {
+ my $order = $test_state->{order};
+ $test_state->{order} = [];
+ my @shape;
+ for my $step (@$order) {
+ push @shape, $step if !@shape || $shape[-1] ne $step;
+ }
+ return \@shape;
+}
+
sub run_test {
my $test = shift;
clear_test_state();
@@ -1451,4 +1519,578 @@ sub test_dnsmasq_dual_stack_and_sweep {
run_test(\&test_dnsmasq_dual_stack_and_sweep);
+# -------------- ebpf dhcp backend
+my $ebpf_record = sub {
+ my ($ip, %over) = @_;
+ return {
+ ip => $ip,
+ prefixlen => 24,
+ server_id => '10.0.0.1',
+ lease => 600,
+ router => '10.0.0.1',
+ dns => undef,
+ mtu => undef,
+ mac => 'da:65:8f:18:9b:6f',
+ %over,
+ };
+};
+
+sub test_ebpf_backend {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+
+ # the guest behind the MAC runs here, so its interface gets the record
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ take_ebpf_calls();
+ $test_state->{nic_listings} = [];
+ $test_state->{lock_modes} = [];
+ eval { nic_start($vnetid, $mac, "999", "testhostname"); };
+ if ($@) {
+ fail("$test_name: nic_start: $@");
+ return;
+ }
+ is(
+ $test_state->{nic_listings}->[0],
+ 999,
+ "$test_name: the start reads its own guest's config only",
+ );
+ eq_or_diff(
+ $test_state->{lock_modes},
+ ['shared'],
+ "$test_name: and writes its record under the shared lock",
+ );
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.100"),
+ }]],
+ }],
+ "$test_name: a guest start writes the record of its interface",
+ );
+
+ # a mapping edit rewrites it, and a delete leaves the interface without one
+ $test_state->{lock_modes} = [];
+ update_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ eq_or_diff(
+ $test_state->{lock_modes},
+ ['exclusive'],
+ "$test_name: an edit names no guest and writes under the exclusive lock",
+ );
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.150"),
+ }]],
+ }],
+ "$test_name: a mapping edit rewrites the record",
+ );
+ delete_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{ name => 'tap999i0', vnet => $vnetid, serve => 1, record => undef }]],
+ }],
+ "$test_name: a mapping delete drops the record",
+ );
+
+ # a guest running elsewhere has no interface here to write to
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'other' } } };
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.150",
+ });
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.150"),
+ }]],
+ }],
+ "$test_name: a guest elsewhere is handed over too, the responder skips what is not here",
+ );
+
+ # the full pass names every guest NIC in the cluster, whether the
+ # responder runs on it and what it answers. The responder keeps those
+ # present here. A guest the vmlist does not know is skipped, and a NIC
+ # without a MAC is served nothing
+ $test_state->{vmlist} = {
+ ids => {
+ 100 => { type => 'qemu', node => 'other' },
+ 101 => { type => 'lxc', node => 'other' },
+ 103 => { type => 'qemu', node => 'other' },
+ },
+ };
+ $test_state->{guest_nets} = {
+ 100 => {
+ net0 => "virtio=$mac,bridge=$vnetid,firewall=1",
+ net1 => "virtio=00:11:22:33:44:55,bridge=vmbr0",
+ },
+ 101 => { net2 => "name=eth0,bridge=$vnetid,hwaddr=00:11:22:33:44:66" },
+ 102 => { net0 => "virtio=00:11:22:33:44:77,bridge=$vnetid" },
+ # a line without a MAC has nothing to answer and is left out
+ 103 => { net0 => "virtio,bridge=$vnetid" },
+ };
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'apply',
+ args => [[
+ {
+ name => 'tap100i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.150"),
+ },
+ { name => 'tap100i1', vnet => undef, serve => 0, record => undef },
+ { name => 'veth101i2', vnet => $vnetid, serve => 1, record => undef },
+ ]],
+ }],
+ "$test_name: the full pass names every guest NIC with its answer",
+ );
+
+ # a responder with nothing loaded cannot take a single record, so the
+ # full pass loads it and fills everything
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ $test_state->{ebpf_answer} = { update => 0 };
+ $test_state->{lock_modes} = [];
+ nic_start($vnetid, $mac, "999", "testhostname");
+ $test_state->{ebpf_answer} = undef;
+ eq_or_diff(
+ [map { $_->{method} } take_ebpf_calls()->@*],
+ ['update', 'apply'],
+ "$test_name: an unloaded responder gets the full pass",
+ );
+ eq_or_diff(
+ $test_state->{lock_modes},
+ ['shared', 'exclusive'],
+ "$test_name: which takes its own lock after the record change let go",
+ );
+
+ # a plug attaches the interface with the answer its MAC gets on the
+ # vnet. With nothing loaded the full pass runs first and the attach is
+ # retried
+ PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0', $mac);
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'attach', args => ['tap999i0', $vnetid, $ebpf_record->("10.0.0.150")] }],
+ "$test_name: a plug attaches its interface with the record",
+ );
+ $test_state->{ebpf_answer} = { attach => 0 };
+ PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0', $mac);
+ $test_state->{ebpf_answer} = undef;
+ eq_or_diff(
+ [map { $_->{method} } take_ebpf_calls()->@*],
+ ['attach', 'apply', 'attach'],
+ "$test_name: a plug with nothing loaded runs the full pass first",
+ );
+ PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0');
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'attach', args => ['tap999i0', $vnetid, undef] }],
+ "$test_name: a plug without a MAC attaches with nothing to answer",
+ );
+}
+
+sub test_ebpf_backend_edge_cases {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a regenerate without any ebpf zone tears the responder state down
+ create_zone({
+ type => "simple",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ take_ebpf_calls();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'clear', args => [] }],
+ "$test_name: regenerate without an ebpf zone clears the responder",
+ );
+
+ # an ebpf zone confined to other nodes runs no responder here either
+ update_zone($zoneid, { dhcp => "ebpf", nodes => 'other' });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'clear', args => [] }],
+ "$test_name: an ebpf zone confined to other nodes clears the responder here",
+ );
+ update_zone($zoneid, { delete => 'nodes' });
+
+ # the SDN commit brings the responder back at boot, one full pass
+ PVE::Network::SDN::Dhcp::boot();
+ eq_or_diff(
+ [map { $_->{method} } take_ebpf_calls()->@*],
+ ['apply'],
+ "$test_name: the boot runs the full pass",
+ );
+
+ # the zone switches to ebpf, but its subnets lack what a record needs
+ update_zone($zoneid, { dhcp => "ebpf" });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "fd00::/64",
+ gateway => "fd00::1",
+ 'dhcp-range' => ["start-address=fd00::100,end-address=fd00::200"],
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+
+ take_ebpf_calls();
+ eval { nic_start($vnetid, $mac, "999", "testhostname"); };
+ if ($@) {
+ fail("$test_name: nic_start: $@");
+ return;
+ }
+
+ # a gateway-less v4 subnet is served under a link-local identifier and
+ # without a router. The v6 subnet yields nothing
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'update',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->(
+ "10.0.0.100",
+ server_id => '169.254.0.1',
+ router => undef,
+ ),
+ }]],
+ }],
+ "$test_name: a subnet without a gateway is served without a router",
+ );
+ $test_state->{warnings} = [];
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ take_ebpf_calls();
+ is(scalar($test_state->{warnings}->@*), 0, "$test_name: and the full pass has nothing to say");
+
+ # a v6 resolver on a v4 subnet is dropped from the record instead of
+ # failing the whole set
+ update_subnet({
+ vnet => $vnetid,
+ subnet => "$zoneid-10.0.0.0-24",
+ gateway => "10.0.0.1",
+ 'dhcp-dns-server' => "fd00::53",
+ });
+ $test_state->{warnings} = [];
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ my $calls = take_ebpf_calls();
+ is(scalar(@$calls), 1, "$test_name: regenerate applies once");
+ my $iface = $calls->[0]->{args}->[0]->[0];
+ is($iface->{name}, 'tap999i0', "$test_name: the guest interface is listed");
+ is($iface->{record}->{ip}, '10.0.0.100', "$test_name: the v4 subnet now yields the record");
+ is($iface->{record}->{dns}, undef, "$test_name: the IPv6 resolver is not handed out");
+ is(
+ scalar(grep { m/IPv6 DNS server/ } $test_state->{warnings}->@*),
+ 1,
+ "$test_name: and the full pass says so",
+ );
+}
+
+run_test(\&test_ebpf_backend);
+run_test(\&test_ebpf_backend_edge_cases);
+
+sub test_ebpf_mac_case {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a mapping typed in lower case for a NIC whose config spells the MAC in
+ # upper case is the same mapping
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=" . uc($mac) . ",bridge=$vnetid" } };
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.100",
+ });
+ take_ebpf_calls();
+ take_order();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{
+ method => 'apply',
+ args => [[{
+ name => 'tap999i0',
+ vnet => $vnetid,
+ serve => 1,
+ record => $ebpf_record->("10.0.0.100"),
+ }]],
+ }],
+ "$test_name: the full pass finds the mapping however the config spells the MAC",
+ );
+ # the lock is taken before the refresh and the reads
+ is_deeply(
+ take_order(),
+ ['lock', 'refresh', 'write'],
+ "$test_name: the pass locks, refreshes, then writes",
+ );
+ PVE::Network::SDN::Dhcp::Ebpf->tap_plug($zoneid, $vnetid, 'tap999i0', uc($mac));
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'attach', args => ['tap999i0', $vnetid, $ebpf_record->("10.0.0.100")] }],
+ "$test_name: and so does a plug",
+ );
+
+ # an entry split across spellings is read as one
+ $test_state->{macdb} = {
+ macs => { uc($mac) => { ip4 => '10.0.0.100' }, $mac => { ip6 => '8888::100' } },
+ };
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{ip},
+ '10.0.0.100',
+ "$test_name: a cache entry split across spellings is read as one",
+ );
+}
+
+run_test(\&test_ebpf_mac_case);
+
+sub test_ebpf_equal_subnets {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+
+ # two ebpf zones carry the same subnet. Each interface is answered from
+ # the subnet of the vnet it sits on, whatever address the cache holds
+ # for its MAC
+ for my $net (['ZONEA', 'vneta', '10.0.0.1'], ['ZONEB', 'vnetb', '10.0.0.2']) {
+ my ($zoneid, $vnetid, $gateway) = @$net;
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => $gateway,
+ });
+ }
+ $test_state->{vmlist} = {
+ ids => {
+ 100 => { type => 'qemu', node => 'localnode' },
+ 101 => { type => 'qemu', node => 'localnode' },
+ },
+ };
+ $test_state->{guest_nets} = {
+ 100 => { net0 => "virtio=da:65:8f:18:9b:6f,bridge=vneta" },
+ 101 => { net0 => "virtio=da:65:8f:18:9b:70,bridge=vnetb" },
+ };
+ $test_state->{macdb} = {
+ macs => {
+ 'da:65:8f:18:9b:6f' => { ip4 => '10.0.0.150' },
+ 'da:65:8f:18:9b:70' => { ip4 => '10.0.0.150' },
+ },
+ };
+
+ take_ebpf_calls();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ my $ifaces = take_ebpf_calls()->[0]->{args}->[0];
+ eq_or_diff(
+ [map { [$_->{name}, $_->{record}->{router}] } @$ifaces],
+ [['tap100i0', '10.0.0.1'], ['tap101i0', '10.0.0.2']],
+ "$test_name: each interface is answered from its own vnet",
+ );
+}
+
+run_test(\&test_ebpf_equal_subnets);
+
+sub test_ebpf_zone_mtu {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a zone MTU the DHCP option cannot carry is reported on the apply and
+ # left out of the answer
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ mtu => 10,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ $test_state->{macdb} = { macs => { $mac => { ip4 => '10.0.0.100' } } };
+
+ take_ebpf_calls();
+ $test_state->{warnings} = [];
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ scalar(grep { m/MTU 10/ } $test_state->{warnings}->@*),
+ 1,
+ "$test_name: the zone MTU is reported once",
+ );
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{mtu},
+ undef,
+ "$test_name: and not handed out",
+ );
+
+ update_zone($zoneid, { mtu => 9000 });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{mtu},
+ 9000,
+ "$test_name: a zone MTU the option carries is handed out",
+ );
+}
+
+run_test(\&test_ebpf_zone_mtu);
+
+sub test_ebpf_overlapping_subnets {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a vnet with overlapping subnets answers from the most specific one
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/16",
+ gateway => "10.0.255.1",
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ $test_state->{macdb} = { macs => { $mac => { ip4 => '10.0.0.100' } } };
+
+ take_ebpf_calls();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ my $record = take_ebpf_calls()->[0]->{args}->[0]->[0]->{record};
+ is_deeply(
+ [$record->@{qw(prefixlen router)}],
+ [24, '10.0.0.1'],
+ "$test_name: the most specific subnet answers",
+ );
+}
+
+run_test(\&test_ebpf_overlapping_subnets);
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-network v4 12/20] sdn: zones: report interface plugs and unplugs to the DHCP backends
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (10 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-network v4 11/20] sdn: dhcp: add an eBPF backend answering on each guest interface Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-network v4 13/20] sdn: dhcp: run IP mapping edits on the guest's node Hannes Laimer
` (7 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
The ebpf DHCP backend answers on each guest interface rather than on
the bridge, so the zone layer now reports every plug to the DHCP
backends. The plug takes the guest's MAC along, since the address to
hand out is recorded per MAC. A plug onto a plain bridge is reported
too, since a running guest's interface can move there from a vnet.
A failing backend only warns and the plug goes ahead.
The zone layer also gains an unplug and a veth removal, which report
the interface to the backends as gone.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com>
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
src/PVE/Network/SDN/Zones.pm | 29 ++++++-
src/test/run_test_vnets_blackbox.pl | 129 ++++++++++++++++++++++++++++
2 files changed, 154 insertions(+), 4 deletions(-)
diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm
index 4229feda..10fc0731 100644
--- a/src/PVE/Network/SDN/Zones.pm
+++ b/src/PVE/Network/SDN/Zones.pm
@@ -326,16 +326,18 @@ sub veth_create {
}
sub tap_plug {
- my ($iface, $bridge, $tag, $firewall, $trunks, $rate) = @_;
+ my ($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts) = @_;
my $vnet = PVE::Network::SDN::Vnets::get_vnet($bridge, 1);
if (!$vnet) { # fallback for classic bridge
my $interfaces_config = PVE::INotify::read_file('interfaces');
- my $opts = {};
- $opts->{learning} = 0
+ my $bridge_opts = {};
+ $bridge_opts->{learning} = 0
if $interfaces_config->{ifaces}->{$bridge}
&& $interfaces_config->{ifaces}->{$bridge}->{'bridge-disable-mac-learning'};
- PVE::Network::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts);
+ # the dhcp backends hear of an interface moving onto a plain bridge as well
+ PVE::Network::SDN::Dhcp::tap_plug($bridge, $iface, $opts->{mac});
+ PVE::Network::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate, $bridge_opts);
return;
}
@@ -346,9 +348,28 @@ sub tap_plug {
if $plugin_config->{nodes} && !defined($plugin_config->{nodes}->{$nodename});
my $plugin = PVE::Network::SDN::Zones::Plugin->lookup($plugin_config->{type});
+
+ # the dhcp backends act before the interface joins the bridge, so no request slips past
+ # them. Dhcp.pm comes in through Vnets.pm, since a use of it here would create a load-order
+ # cycle
+ PVE::Network::SDN::Dhcp::tap_plug($bridge, $iface, $opts->{mac});
$plugin->tap_plug($plugin_config, $vnet, $tag, $iface, $bridge, $firewall, $trunks, $rate);
}
+sub tap_unplug {
+ my ($iface) = @_;
+
+ PVE::Network::SDN::Dhcp::tap_unplug($iface);
+ PVE::Network::tap_unplug($iface);
+}
+
+sub veth_delete {
+ my ($veth) = @_;
+
+ PVE::Network::SDN::Dhcp::tap_unplug($veth);
+ PVE::Network::veth_delete($veth);
+}
+
sub add_bridge_fdb {
my ($iface, $macaddr, $bridge) = @_;
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 30a4ab82..516b13c6 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -1858,6 +1858,135 @@ sub test_ebpf_backend_edge_cases {
}
run_test(\&test_ebpf_backend);
+
+sub test_zone_tap_plug {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # the zone plug hands the MAC on to the dhcp backends, and a plug onto a
+ # plain bridge tells them where the interface went
+ create_zone({
+ type => "simple",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ 'dhcp-range' => ["start-address=10.0.0.100,end-address=10.0.0.200"],
+ });
+ create_ip({
+ zone => $zoneid,
+ vnet => $vnetid,
+ mac => $mac,
+ ip => "10.0.0.100",
+ });
+ my $bridged = [];
+ my $unplugged = [];
+ $mocked_sdn_zones_super_plugin->mock(
+ tap_plug => sub {
+ push @$bridged, $_[5];
+ push $test_state->{order}->@*, 'bridge';
+ },
+ );
+ my $mocked_pve_network = Test::MockModule->new('PVE::Network');
+ $mocked_pve_network->mock(
+ tap_plug => sub {
+ push @$bridged, $_[1];
+ push $test_state->{order}->@*, 'bridge';
+ },
+ tap_unplug => sub { push @$unplugged, $_[0]; },
+ veth_delete => sub { push @$unplugged, $_[0]; },
+ );
+ $mocked_pve_inotify->mock(read_file => sub { return { ifaces => {} }; });
+ take_ebpf_calls();
+ take_order();
+
+ PVE::Network::SDN::Zones::tap_plug('tap999i0', $vnetid, undef, 0, undef, undef,
+ { mac => $mac });
+ eq_or_diff($bridged, [$vnetid], "$test_name: the zone plugin plugs the interface");
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'attach', args => ['tap999i0', $vnetid, $ebpf_record->("10.0.0.100")] }],
+ "$test_name: and the dhcp backend attaches it with the MAC's record",
+ );
+ is_deeply(
+ take_order(),
+ ['lock', 'refresh', 'write', 'bridge'],
+ "$test_name: the responder attaches before the interface joins the bridge",
+ );
+
+ @$bridged = ();
+ PVE::Network::SDN::Zones::tap_plug('tap999i0', 'vmbr0', undef, 0, undef, undef,
+ { mac => $mac });
+ eq_or_diff($bridged, ['vmbr0'], "$test_name: a plain bridge is plugged the classic way");
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'detach', args => ['tap999i0', { vnet => undef }] }],
+ "$test_name: and the dhcp backends hear where the interface went",
+ );
+
+ # a vnet of another backend is a place the ebpf backend hears of as well
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => "DNSZONE",
+ });
+ create_vnet({
+ type => "vnet",
+ zone => "DNSZONE",
+ vnet => "dnsvnet",
+ });
+ @$bridged = ();
+ PVE::Network::SDN::Zones::tap_plug(
+ 'tap999i0',
+ 'dnsvnet',
+ undef,
+ 0,
+ undef,
+ undef,
+ { mac => $mac },
+ );
+ eq_or_diff(
+ $bridged,
+ ['dnsvnet'],
+ "$test_name: a vnet of another backend is plugged by its zone",
+ );
+ eq_or_diff(
+ take_ebpf_calls(),
+ [{ method => 'detach', args => ['tap999i0', { vnet => 'dnsvnet' }] }],
+ "$test_name: and the ebpf backend hears which vnet the interface went to",
+ );
+
+ # the zone unplug and the veth delete report the interface gone before
+ # the classic ones run
+ PVE::Network::SDN::Zones::tap_unplug('tap999i0');
+ PVE::Network::SDN::Zones::veth_delete('veth999i0');
+ eq_or_diff($unplugged, ['tap999i0', 'veth999i0'], "$test_name: the classic unplugs run");
+ eq_or_diff(
+ take_ebpf_calls(),
+ [
+ { method => 'detach', args => ['tap999i0', undef] },
+ { method => 'detach', args => ['veth999i0', undef] },
+ ],
+ "$test_name: and the dhcp backends hear the interfaces are gone",
+ );
+ $mocked_sdn_zones_super_plugin->unmock('tap_plug');
+ $mocked_pve_inotify->unmock('read_file');
+}
+
+run_test(\&test_zone_tap_plug);
run_test(\&test_ebpf_backend_edge_cases);
sub test_ebpf_mac_case {
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-network v4 13/20] sdn: dhcp: run IP mapping edits on the guest's node
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (11 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-network v4 12/20] sdn: zones: report interface plugs and unplugs to the DHCP backends Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-network v4 14/20] sdn: zones: allow the ebpf DHCP backend on every zone type Hannes Laimer
` (6 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
IP mappings live in the cluster-wide IPAM and can be edited through
the API on any node. The DHCP backends hold their copy on each node,
dnsmasq in a reservation file and the ebpf responder in a kernel map.
An edit only updated the backend of the node it was made on, so a
guest on another node kept getting the old mapping.
The API proxies an edit through /cluster/sdn/vnets/{vnet}/ips to the
node of the guest whose NIC carries the MAC on that vnet, and the
backend there is updated along with the record. Only that node answers
the guest's DHCP requests, so no other node needs the update. The
others catch up when the guest starts on one of them. An edit in a
zone without a DHCP backend, or for a MAC no guest has, runs where it
was made. When the guest's node cannot be reached the edit fails, as
any proxied request does.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Notes:
v4:
* use a proxyto callback to run the edit on the guest's node
instead of pushing it there through a node endpoint
* look the guest's node up in the NIC list by MAC
src/PVE/API2/Network/SDN/Ips.pm | 16 +++++
src/PVE/Network/SDN/Dhcp.pm | 10 +++
src/test/run_test_vnets_blackbox.pl | 96 +++++++++++++++++++++++++++++
3 files changed, 122 insertions(+)
diff --git a/src/PVE/API2/Network/SDN/Ips.pm b/src/PVE/API2/Network/SDN/Ips.pm
index d7b682dd..e92de511 100644
--- a/src/PVE/API2/Network/SDN/Ips.pm
+++ b/src/PVE/API2/Network/SDN/Ips.pm
@@ -6,6 +6,7 @@ use warnings;
use PVE::Tools qw(extract_param);
use PVE::Network::SDN::Vnets;
+use PVE::Network::SDN::Zones;
use PVE::Network::SDN::Dhcp;
use PVE::JSONSchema qw(get_standard_option);
@@ -13,12 +14,25 @@ use PVE::RESTHandler;
use base qw(PVE::RESTHandler);
+# only the guest's node answers its DHCP requests, so the edit runs there
+my $proxy_to_guest_node = sub {
+ my ($rpcenv, $proxyto, $param) = @_;
+
+ my $vnet = PVE::Network::SDN::Vnets::get_vnet($param->{vnet}, 1);
+ my $zone = $vnet ? PVE::Network::SDN::Zones::get_zone($vnet->{zone}, 1) : undef;
+ return 'localhost' if !$zone || !$zone->{dhcp};
+
+ my $node = PVE::Network::SDN::Dhcp::guest_node_by_mac($param->{vnet}, $param->{mac});
+ return $node // 'localhost';
+};
+
__PACKAGE__->register_method({
name => 'ipdelete',
path => '',
method => 'DELETE',
description => 'Delete IP Mappings in a VNet',
protected => 1,
+ proxyto_callback => $proxy_to_guest_node,
permissions => {
check => ['perm', '/sdn/zones/{zone}/{vnet}', ['SDN.Allocate']],
},
@@ -58,6 +72,7 @@ __PACKAGE__->register_method({
method => 'POST',
description => 'Create IP Mapping in a VNet',
protected => 1,
+ proxyto_callback => $proxy_to_guest_node,
permissions => {
check => ['perm', '/sdn/zones/{zone}/{vnet}', ['SDN.Allocate']],
},
@@ -95,6 +110,7 @@ __PACKAGE__->register_method({
method => 'PUT',
description => 'Update IP Mapping in a VNet',
protected => 1,
+ proxyto_callback => $proxy_to_guest_node,
permissions => {
check => ['perm', '/sdn/zones/{zone}/{vnet}', ['SDN.Allocate']],
},
diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm
index 5a76cfda..32faefe5 100644
--- a/src/PVE/Network/SDN/Dhcp.pm
+++ b/src/PVE/Network/SDN/Dhcp.pm
@@ -89,6 +89,16 @@ sub guest_nics {
return $nics;
}
+sub guest_node_by_mac {
+ my ($vnetid, $mac) = @_;
+
+ for my $nic ((guest_nics()->{ lc($mac) } // [])->@*) {
+ return $nic->{node} if $nic->{bridge} eq $vnetid;
+ }
+
+ return undef;
+}
+
sub tap_plug {
my ($bridge, $iface, $mac) = @_;
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 516b13c6..28d58de1 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -1412,6 +1412,102 @@ sub test_dhcp_backend_needed_per_node {
run_test(\&test_dhcp_backend_needed_per_node);
+sub test_mapping_edit_target {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+ my $vnetid = "testvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ create_zone({
+ type => "simple",
+ dhcp => "dnsmasq",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ });
+ create_zone({
+ type => "simple",
+ ipam => "pve",
+ zone => "NODHCP",
+ });
+ create_vnet({
+ type => "vnet",
+ zone => "NODHCP",
+ vnet => "nodhcp",
+ });
+
+ # the guest behind a MAC is found by its NIC on the vnet, and the config
+ # writes the MAC in upper case
+ $test_state->{vmlist} = {
+ ids => {
+ 100 => { type => 'qemu', node => 'other' },
+ 101 => { type => 'lxc', node => 'third' },
+ },
+ };
+ $test_state->{guest_nets} = {
+ 1 => { net0 => "virtio=$mac,bridge=$vnetid" },
+ 100 => {
+ net0 => "virtio=" . uc($mac) . ",bridge=$vnetid",
+ net1 => "virtio=$mac,bridge=nodhcp",
+ },
+ 101 => { net0 => "name=eth0,bridge=vmbr0,hwaddr=00:11:22:33:44:55" },
+ };
+ is(
+ PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, $mac),
+ 'other',
+ "$test_name: the MAC resolves to the node of its guest, a guest the list lacks is skipped",
+ );
+ is(
+ PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, '00:11:22:33:44:55'),
+ undef,
+ "$test_name: a MAC on another bridge does not resolve",
+ );
+ is(
+ PVE::Network::SDN::Dhcp::guest_node_by_mac($vnetid, '00:11:22:33:44:66'),
+ undef,
+ "$test_name: an unknown MAC does not resolve",
+ );
+
+ # every mapping edit through the API runs on the guest's node
+ for my $method (qw(ipcreate ipupdate ipdelete)) {
+ my $info = PVE::API2::Network::SDN::Ips->map_method_by_name($method);
+ my $proxyto = $info->{proxyto_callback};
+ is(
+ $proxyto->(undef, undef, { zone => $zoneid, vnet => $vnetid, mac => $mac }),
+ 'other',
+ "$test_name: $method runs on the guest's node",
+ );
+ is(
+ $proxyto->(
+ undef,
+ undef,
+ { zone => $zoneid, vnet => $vnetid, mac => '00:11:22:33:44:66' },
+ ),
+ 'localhost',
+ "$test_name: $method for a MAC without a guest runs here",
+ );
+ is(
+ $proxyto->(undef, undef, { zone => 'NODHCP', vnet => 'nodhcp', mac => $mac }),
+ 'localhost',
+ "$test_name: $method in a zone without a backend runs here",
+ );
+ }
+ delete $test_state->{vmlist};
+ delete $test_state->{guest_nets};
+}
+
+run_test(\&test_mapping_edit_target);
+
sub test_dnsmasq_dual_stack_and_sweep {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-network v4 14/20] sdn: zones: allow the ebpf DHCP backend on every zone type
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (12 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-network v4 13/20] sdn: dhcp: run IP mapping edits on the guest's node Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH qemu-server v4 15/20] network: sdn: keep the DHCP backends current on guest NICs Hannes Laimer
` (5 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
The ebpf backend answers each guest on the guest's own tap or veth
device, so the zone type makes no difference to it. dnsmasq stays
limited to simple zones, since every node runs its own instance on
the vnet bridge and only a simple zone's bridge is cut off from the
other nodes. Choosing it elsewhere fails on create and on update.
Every zone type now reports the MTU its guests should use. The ebpf
backend hands it out, and until now only simple zones could answer.
A VXLAN or EVPN zone without an MTU setting takes it from the
interface that reaches its peers.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/PVE/API2/Network/SDN/Zones.pm | 8 +-
src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 32 +++++++
src/PVE/Network/SDN/Zones/FaucetPlugin.pm | 1 +
src/PVE/Network/SDN/Zones/QinQPlugin.pm | 7 ++
src/PVE/Network/SDN/Zones/SimplePlugin.pm | 2 +-
src/PVE/Network/SDN/Zones/VlanPlugin.pm | 7 ++
src/PVE/Network/SDN/Zones/VxlanPlugin.pm | 25 +++++
src/test/run_test_vnets_blackbox.pl | 111 +++++++++++++++++++++-
8 files changed, 188 insertions(+), 5 deletions(-)
diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm
index 0e907260..181d2872 100644
--- a/src/PVE/API2/Network/SDN/Zones.pm
+++ b/src/PVE/API2/Network/SDN/Zones.pm
@@ -93,7 +93,7 @@ my $ZONE_PROPERTIES = {
type => 'string',
enum => PVE::Network::SDN::Dhcp->plugin_types(),
optional => 1,
- description => 'Name of DHCP server backend for this zone.',
+ description => 'DHCP backend serving the zone. dnsmasq works on simple zones only.',
},
'rt-import' => {
type => 'string',
@@ -448,6 +448,9 @@ __PACKAGE__->register_method({
raise_param_exc({ ipam => "$ipam not existing" })
if $ipam && !$ipam_cfg->{ids}->{$ipam};
+ raise_param_exc({ dhcp => "the dnsmasq backend only supports simple zones" })
+ if ($opts->{dhcp} // '') eq 'dnsmasq' && $plugin->type() ne 'simple';
+
$zone_cfg->{ids}->{$id} = $opts;
$plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
@@ -544,6 +547,9 @@ __PACKAGE__->register_method({
raise_param_exc({ ipam => "$ipam not existing" })
if $ipam && !$ipam_cfg->{ids}->{$ipam};
+ raise_param_exc({ dhcp => "the dnsmasq backend only supports simple zones" })
+ if ($scfg->{dhcp} // '') eq 'dnsmasq' && $plugin->type() ne 'simple';
+
$plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
PVE::Network::SDN::Zones::write_config($zone_cfg);
diff --git a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
index 0e79707c..60fe788f 100644
--- a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
@@ -125,10 +125,42 @@ sub options {
reversedns => { optional => 1 },
dnszone => { optional => 1 },
ipam => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
# Plugin implementation
+# the underlay interface is the one the controller's peers name, found the
+# way the bridge generation finds it
+sub get_mtu {
+ my ($class, $plugin_config) = @_;
+
+ return $plugin_config->{mtu} if $plugin_config->{mtu};
+
+ my $iface;
+ my $controller_cfg = PVE::Network::SDN::running_config()->{controllers};
+ my $controller = $controller_cfg->{ids}->{ $plugin_config->{controller} // '' };
+ if ($controller && $controller->{peers}) {
+ my @peers = PVE::Tools::split_list($controller->{peers});
+ my $local_node = PVE::INotify::nodename();
+ my $bgprouter = PVE::Network::SDN::Controllers::EvpnPlugin::find_bgp_controller(
+ $local_node, $controller_cfg,
+ );
+ my $isisrouter = PVE::Network::SDN::Controllers::EvpnPlugin::find_isis_controller(
+ $local_node, $controller_cfg,
+ );
+ my $loopback = $bgprouter->{loopback} // $isisrouter->{loopback};
+ (undef, $iface) = eval {
+ PVE::Network::SDN::Zones::Plugin::find_local_ip_interface_peers(\@peers, $loopback);
+ };
+ }
+ return PVE::Network::SDN::Zones::VxlanPlugin::vxlan_mtu(
+ $plugin_config,
+ $iface,
+ PVE::INotify::read_file('interfaces'),
+ );
+}
+
sub generate_sdn_config {
my (
$class,
diff --git a/src/PVE/Network/SDN/Zones/FaucetPlugin.pm b/src/PVE/Network/SDN/Zones/FaucetPlugin.pm
index 5f069aea..4e3f5eff 100644
--- a/src/PVE/Network/SDN/Zones/FaucetPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/FaucetPlugin.pm
@@ -30,6 +30,7 @@ sub options {
reversedns => { optional => 1 },
dnszone => { optional => 1 },
ipam => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
diff --git a/src/PVE/Network/SDN/Zones/QinQPlugin.pm b/src/PVE/Network/SDN/Zones/QinQPlugin.pm
index 642420e9..85d9ce98 100644
--- a/src/PVE/Network/SDN/Zones/QinQPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/QinQPlugin.pm
@@ -36,6 +36,12 @@ sub properties {
};
}
+sub get_mtu {
+ my ($class, $plugin_config) = @_;
+
+ return $plugin_config->{mtu};
+}
+
sub options {
return {
nodes => { optional => 1 },
@@ -48,6 +54,7 @@ sub options {
reversedns => { optional => 1 },
dnszone => { optional => 1 },
ipam => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
diff --git a/src/PVE/Network/SDN/Zones/SimplePlugin.pm b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
index 347eee9a..655f7e41 100644
--- a/src/PVE/Network/SDN/Zones/SimplePlugin.pm
+++ b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
@@ -31,7 +31,7 @@ sub properties {
description => "dns domain zone ex: mydomain.com",
},
dhcp => {
- description => 'Type of the DHCP backend for this zone',
+ description => 'DHCP backend serving the zone. dnsmasq works on simple zones only.',
type => 'string',
enum => PVE::Network::SDN::Dhcp->plugin_types(),
},
diff --git a/src/PVE/Network/SDN/Zones/VlanPlugin.pm b/src/PVE/Network/SDN/Zones/VlanPlugin.pm
index 271b3fab..a5d0fb0d 100644
--- a/src/PVE/Network/SDN/Zones/VlanPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/VlanPlugin.pm
@@ -36,6 +36,12 @@ sub properties {
};
}
+sub get_mtu {
+ my ($class, $plugin_config) = @_;
+
+ return $plugin_config->{mtu};
+}
+
sub options {
return {
@@ -47,6 +53,7 @@ sub options {
reversedns => { optional => 1 },
dnszone => { optional => 1 },
ipam => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
diff --git a/src/PVE/Network/SDN/Zones/VxlanPlugin.pm b/src/PVE/Network/SDN/Zones/VxlanPlugin.pm
index a4082615..0dab5d95 100644
--- a/src/PVE/Network/SDN/Zones/VxlanPlugin.pm
+++ b/src/PVE/Network/SDN/Zones/VxlanPlugin.pm
@@ -47,6 +47,30 @@ sub properties {
};
}
+# the vnet bridge takes the underlay interface's MTU minus the vxlan
+# encapsulation where the peers name a local interface, the default
+# otherwise. An explicit zone MTU wins
+sub vxlan_mtu {
+ my ($plugin_config, $iface, $interfaces_config) = @_;
+
+ return $plugin_config->{mtu} if $plugin_config->{mtu};
+ return $interfaces_config->{ifaces}->{$iface}->{mtu} - 50
+ if $iface && $interfaces_config->{ifaces}->{$iface}->{mtu};
+ return 1450;
+}
+
+sub get_mtu {
+ my ($class, $plugin_config) = @_;
+
+ my $iface;
+ if ($plugin_config->{peers}) {
+ my @peers = PVE::Tools::split_list($plugin_config->{peers});
+ (undef, $iface) =
+ eval { PVE::Network::SDN::Zones::Plugin::find_local_ip_interface_peers(\@peers) };
+ }
+ return vxlan_mtu($plugin_config, $iface, PVE::INotify::read_file('interfaces'));
+}
+
sub options {
return {
nodes => { optional => 1 },
@@ -58,6 +82,7 @@ sub options {
dnszone => { optional => 1 },
ipam => { optional => 1 },
fabric => { optional => 1 },
+ dhcp => { optional => 1 },
};
}
diff --git a/src/test/run_test_vnets_blackbox.pl b/src/test/run_test_vnets_blackbox.pl
index 28d58de1..5722c601 100755
--- a/src/test/run_test_vnets_blackbox.pl
+++ b/src/test/run_test_vnets_blackbox.pl
@@ -1508,6 +1508,55 @@ sub test_mapping_edit_target {
run_test(\&test_mapping_edit_target);
+sub test_zone_dhcp_backends {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "TESTZONE";
+
+ # dnsmasq answers on the vnet bridge, so it only fits zones confined to
+ # one node. The ebpf backend answers on the guest's tap and fits any
+ my $zone = {
+ type => "vlan",
+ bridge => "vmbr0",
+ ipam => "pve",
+ zone => $zoneid,
+ };
+ eval { PVE::API2::Network::SDN::Zones->create({ %$zone, dhcp => "dnsmasq" }); };
+ like(
+ $@,
+ qr/dnsmasq backend only supports simple zones/,
+ "$test_name: dnsmasq is refused on a vlan zone",
+ );
+
+ create_zone({ %$zone, dhcp => "ebpf" });
+ is(get_zone($zoneid)->{dhcp}, "ebpf", "$test_name: ebpf is accepted on a vlan zone");
+
+ eval { update_zone($zoneid, { dhcp => "dnsmasq" }); };
+ like(
+ $@,
+ qr/dnsmasq backend only supports simple zones/,
+ "$test_name: switching a vlan zone to dnsmasq is refused",
+ );
+
+ # every other zone type offers the backend as well
+ $test_state->{controller_config} = {
+ ids => { ctrl => { type => 'evpn', asn => 65000, peers => '10.0.0.1' } },
+ };
+ my $params = {
+ qinq => { bridge => 'vmbr0', tag => 100 },
+ vxlan => { peers => '10.0.0.1' },
+ evpn => { controller => 'ctrl', 'vrf-vxlan' => 1000 },
+ faucet => { 'dp-id' => 1, controller => 'ctrl' },
+ };
+ for my $type (sort keys %$params) {
+ my $id = uc($type) . "Z";
+ my $zone = { type => $type, ipam => "pve", zone => $id, %{ $params->{$type} } };
+ create_zone({ %$zone, dhcp => "ebpf" });
+ is(get_zone($id)->{dhcp}, "ebpf", "$test_name: ebpf is accepted on a $type zone");
+ }
+}
+
+run_test(\&test_zone_dhcp_backends);
+
sub test_dnsmasq_dual_stack_and_sweep {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
@@ -1604,7 +1653,7 @@ sub test_dnsmasq_dual_stack_and_sweep {
is(
PVE::Tools::file_get_contents($TMP_ETHERS_FILE),
"$mac,10.0.0.100,[8888::ab]\n$other,10.0.0.102,[8888::AB]\n",
- "$test_name: a line of another MAC spelled by the old code is kept",
+ "$test_name: a line of another MAC in its earlier spelling is kept",
);
is(
scalar(grep { $_ eq 'reload' } $test_state->{dnsmasq_calls}->@*),
@@ -1838,6 +1887,8 @@ sub test_ebpf_backend {
);
}
+run_test(\&test_ebpf_backend);
+
sub test_ebpf_backend_edge_cases {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
@@ -1953,7 +2004,7 @@ sub test_ebpf_backend_edge_cases {
);
}
-run_test(\&test_ebpf_backend);
+run_test(\&test_ebpf_backend_edge_cases);
sub test_zone_tap_plug {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
@@ -2083,7 +2134,6 @@ sub test_zone_tap_plug {
}
run_test(\&test_zone_tap_plug);
-run_test(\&test_ebpf_backend_edge_cases);
sub test_ebpf_mac_case {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
@@ -2272,6 +2322,61 @@ sub test_ebpf_zone_mtu {
run_test(\&test_ebpf_zone_mtu);
+sub test_ebpf_vxlan_mtu {
+ my $test_name = (split(/::/, (caller(0))[3]))[-1];
+ my $zoneid = "VXZONE";
+ my $vnetid = "vxvnet";
+ my $mac = "da:65:8f:18:9b:6f";
+
+ # a vxlan zone without an MTU of its own answers with the one its bridge
+ # takes from the underlay, the encapsulation overhead taken off
+ my $mocked_zone_plugin = Test::MockModule->new('PVE::Network::SDN::Zones::Plugin');
+ $mocked_zone_plugin->mock(
+ find_local_ip_interface_peers => sub { return ('10.10.10.1', 'eno1'); });
+ $mocked_pve_inotify->mock(
+ read_file => sub { return { ifaces => { eno1 => { mtu => 1400 } } }; });
+ create_zone({
+ type => "vxlan",
+ peers => "10.10.10.2",
+ dhcp => "ebpf",
+ ipam => "pve",
+ zone => $zoneid,
+ });
+ create_vnet({
+ type => "vnet",
+ zone => $zoneid,
+ vnet => $vnetid,
+ tag => 100,
+ });
+ create_subnet({
+ type => "subnet",
+ vnet => $vnetid,
+ subnet => "10.0.0.0/24",
+ gateway => "10.0.0.1",
+ });
+ $test_state->{vmlist} = { ids => { 999 => { type => 'qemu', node => 'localnode' } } };
+ $test_state->{guest_nets} = { 999 => { net0 => "virtio=$mac,bridge=$vnetid" } };
+ $test_state->{macdb} = { macs => { $mac => { ip4 => '10.0.0.100' } } };
+
+ take_ebpf_calls();
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{mtu},
+ 1350,
+ "$test_name: the underlay's MTU minus the encapsulation is handed out",
+ );
+ update_zone($zoneid, { mtu => 1500 });
+ PVE::Network::SDN::Dhcp::regenerate_config();
+ is(
+ take_ebpf_calls()->[0]->{args}->[0]->[0]->{record}->{mtu},
+ 1500,
+ "$test_name: an explicit zone MTU wins",
+ );
+ $mocked_pve_inotify->unmock('read_file');
+}
+
+run_test(\&test_ebpf_vxlan_mtu);
+
sub test_ebpf_overlapping_subnets {
my $test_name = (split(/::/, (caller(0))[3]))[-1];
my $zoneid = "TESTZONE";
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH qemu-server v4 15/20] network: sdn: keep the DHCP backends current on guest NICs
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (13 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-network v4 14/20] sdn: zones: allow the ebpf DHCP backend on every zone type Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-container v4 16/20] net: report NIC plugs and removals to the SDN DHCP backends Hannes Laimer
` (4 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
A DHCP backend that serves the guest's tap itself has to hear of
every plug and removal of that tap. The plug now carries the NIC's
MAC, so the backend can look up the NIC's record. Removing a tap now
goes through the SDN layer as well, so the backend drops its entry
for that tap.
A bridge change allocated the NIC's address in the new vnet, but
unlike a start or a hotplug it never pushed the mapping to the
backend of that vnet's zone. It now does.
During a live migration the cluster lists the guest on the source
until its config moves. A mapping edited in that window is sent to
the source, while the target keeps the record it plugged the NIC
with. When the target resumes the guest after the migration, it
first pushes the current mapping of every NIC.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
src/PVE/CLI/qm.pm | 4 ++--
src/PVE/QemuServer.pm | 4 ++++
src/PVE/QemuServer/Network.pm | 19 +++++++++++++++++--
src/PVE/QemuServer/RunState.pm | 2 ++
src/usr/pve-bridge | 8 +++++++-
src/usr/pve-bridgedown | 4 ++--
6 files changed, 34 insertions(+), 7 deletions(-)
diff --git a/src/PVE/CLI/qm.pm b/src/PVE/CLI/qm.pm
index b903c1f1..6c53581a 100755
--- a/src/PVE/CLI/qm.pm
+++ b/src/PVE/CLI/qm.pm
@@ -22,7 +22,7 @@ use PVE::GuestHelpers;
use PVE::GuestImport::OVF;
use PVE::INotify;
use PVE::JSONSchema qw(get_standard_option);
-use PVE::Network;
+use PVE::Network::SDN::Zones;
use PVE::RPCEnvironment;
use PVE::SafeSyslog;
use PVE::Tools qw(extract_param file_get_contents);
@@ -1143,7 +1143,7 @@ __PACKAGE__->register_method({
foreach my $opt (keys %$conf) {
next if $opt !~ m/^net(\d+)$/;
my $interface = $1;
- PVE::Network::tap_unplug("tap${vmid}i${interface}");
+ PVE::Network::SDN::Zones::tap_unplug("tap${vmid}i${interface}");
}
}
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 135c993f..39bdfc8b 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -4975,6 +4975,9 @@ sub vmconfig_update_net {
PVE::Network::SDN::Vnets::add_next_free_cidr(
$newnet->{bridge}, $conf->{name}, $newnet->{macaddr}, $vmid, undef, 1,
);
+ PVE::Network::SDN::Vnets::add_dhcp_mapping(
+ $newnet->{bridge}, $newnet->{macaddr}, $vmid, $conf->{name},
+ );
}
PVE::QemuServer::Network::tap_plug(
@@ -4984,6 +4987,7 @@ sub vmconfig_update_net {
$newnet->{firewall},
$newnet->{trunks},
$newnet->{rate},
+ { mac => $newnet->{macaddr} },
);
} elsif (safe_num_ne($oldnet->{rate}, $newnet->{rate})) {
diff --git a/src/PVE/QemuServer/Network.pm b/src/PVE/QemuServer/Network.pm
index b4893c01..e95fa62d 100644
--- a/src/PVE/QemuServer/Network.pm
+++ b/src/PVE/QemuServer/Network.pm
@@ -8,6 +8,7 @@ use PVE::Firewall::Helpers;
use PVE::JSONSchema qw(get_standard_option parse_property_string);
use PVE::Network::SDN::Vnets;
use PVE::Network::SDN::Zones;
+use PVE::Network::SDN::Dhcp;
use PVE::RESTEnvironment qw(log_warn);
use PVE::Tools qw($IPV6RE file_read_firstline);
@@ -272,6 +273,20 @@ sub add_random_macs {
}
}
+# an edit made while the VM migrated went to the source, which the vmlist
+# still named. The target pushes every NIC's current mapping once it runs here
+# for good
+sub add_nets_dhcp_mapping {
+ my ($conf, $vmid) = @_;
+
+ for my $opt (keys %$conf) {
+ next if $opt !~ m/^net\d+$/;
+ my $net = parse_net($conf->{$opt}, 1) or next;
+ next if !$net->{bridge} || !$net->{macaddr};
+ PVE::Network::SDN::Dhcp::update_mapping($net->{bridge}, $net->{macaddr}, $vmid);
+ }
+}
+
sub add_nets_bridge_fdb {
my ($conf, $vmid) = @_;
@@ -349,10 +364,10 @@ sub delete_ifaces_ipams_ips {
}
sub tap_plug {
- my ($iface, $bridge, $tag, $firewall, $trunks, $rate) = @_;
+ my ($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts) = @_;
$firewall = $firewall && PVE::Firewall::Helpers::needs_fwbr($bridge);
- PVE::Network::SDN::Zones::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate);
+ PVE::Network::SDN::Zones::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate, $opts);
}
sub get_nets_host_mtu {
diff --git a/src/PVE/QemuServer/RunState.pm b/src/PVE/QemuServer/RunState.pm
index 889ddd28..0fdd8807 100644
--- a/src/PVE/QemuServer/RunState.pm
+++ b/src/PVE/QemuServer/RunState.pm
@@ -181,6 +181,8 @@ sub vm_resume($vmid, $skiplock = undef, $nocheck = undef) {
PVE::QemuServer::Network::add_nets_bridge_fdb($conf, $vmid)
if $resume_cmd eq 'cont';
+ PVE::QemuServer::Network::add_nets_dhcp_mapping($conf, $vmid)
+ if $nocheck && $resume_cmd eq 'cont';
mon_cmd($vmid, $resume_cmd);
},
diff --git a/src/usr/pve-bridge b/src/usr/pve-bridge
index c7260c6b..20724271 100755
--- a/src/usr/pve-bridge
+++ b/src/usr/pve-bridge
@@ -43,7 +43,13 @@ die "unable to parse network config '$netid'\n" if !$net;
PVE::Network::SDN::Vnets::add_dhcp_mapping($net->{bridge}, $net->{macaddr}, $vmid, $conf->{name});
PVE::Network::SDN::Zones::tap_create($iface, $net->{bridge});
PVE::QemuServer::Network::tap_plug(
- $iface, $net->{bridge}, $net->{tag}, $net->{firewall}, $net->{trunks}, $net->{rate},
+ $iface,
+ $net->{bridge},
+ $net->{tag},
+ $net->{firewall},
+ $net->{trunks},
+ $net->{rate},
+ { mac => $net->{macaddr} },
);
exit 0;
diff --git a/src/usr/pve-bridgedown b/src/usr/pve-bridgedown
index e867b640..79a3722b 100755
--- a/src/usr/pve-bridgedown
+++ b/src/usr/pve-bridgedown
@@ -2,7 +2,7 @@
use strict;
use warnings;
-use PVE::Network;
+use PVE::Network::SDN::Zones;
my $iface = shift;
@@ -11,6 +11,6 @@ die "no interface specified\n" if !$iface;
die "got strange interface name '$iface'\n"
if $iface !~ m/^tap(\d+)i(\d+)$/;
-PVE::Network::tap_unplug($iface);
+PVE::Network::SDN::Zones::tap_unplug($iface);
exit 0;
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-container v4 16/20] net: report NIC plugs and removals to the SDN DHCP backends
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (14 preceding siblings ...)
2026-10-21 8:44 ` [PATCH qemu-server v4 15/20] network: sdn: keep the DHCP backends current on guest NICs Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-manager v4 17/20] ui: sdn: add zone DHCP backend selector and subnet DHCP options Hannes Laimer
` (3 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
An SDN DHCP backend answers a guest NIC with the address IPAM holds
for its MAC. A NIC joining its bridge now hands its MAC to the SDN
plug, so the backend of its vnet can answer its first request.
Deleting a veth also reports it as gone to the backends, since their
state for an interface can outlive it.
On a running container, a bridge change now pushes the DHCP mapping
for the new vnet, as adding a NIC or changing its MAC already does.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
src/PVE/LXC.pm | 13 +++++++++++--
src/PVE/LXC/Config.pm | 3 ++-
src/lxc-pve-poststop-hook | 6 +++---
3 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 1f11490..c3cf9e7 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -1111,7 +1111,13 @@ sub net_tap_plug : prototype($$) {
my $create_firewall_bridges = $net->{firewall} && PVE::Firewall::Helpers::needs_fwbr($bridge);
PVE::Network::SDN::Zones::tap_plug(
- $iface, $bridge, $tag, $create_firewall_bridges, $trunks, $rate,
+ $iface,
+ $bridge,
+ $tag,
+ $create_firewall_bridges,
+ $trunks,
+ $rate,
+ { mac => $hwaddr },
);
PVE::Network::SDN::Zones::add_bridge_fdb($iface, $hwaddr, $bridge);
@@ -1137,7 +1143,7 @@ sub update_net {
|| safe_string_ne($oldnet->{name}, $newnet->{name})
) {
- PVE::Network::veth_delete($veth);
+ PVE::Network::SDN::Zones::veth_delete($veth);
if (safe_string_ne($oldnet->{hwaddr}, $newnet->{hwaddr})) {
eval {
@@ -1192,6 +1198,9 @@ sub update_net {
PVE::Network::SDN::Vnets::add_next_free_cidr(
$newnet->{bridge}, $conf->{hostname}, $newnet->{hwaddr}, $vmid, undef, 1,
);
+ PVE::Network::SDN::Vnets::add_dhcp_mapping(
+ $newnet->{bridge}, $newnet->{hwaddr}, $vmid, $conf->{hostname},
+ );
}
eval { PVE::LXC::net_tap_plug($veth, $newnet) };
if (my $err = $@) {
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 9823d00..b522413 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -12,6 +12,7 @@ use PVE::GuestHelpers;
use PVE::INotify;
use PVE::JSONSchema qw(get_standard_option);
use PVE::Network::SDN::Vnets;
+use PVE::Network::SDN::Zones;
use PVE::RESTEnvironment;
use PVE::Tools;
@@ -1696,7 +1697,7 @@ sub vmconfig_hotplug_pending {
my $net = parse_lxc_network($conf->{$opt});
PVE::LXC::kill_dhclients($vmid, $net->{name}) if $net->{'host-managed'};
- PVE::Network::veth_delete("veth${vmid}i$netid");
+ PVE::Network::SDN::Zones::veth_delete("veth${vmid}i$netid");
print "delete ips from $opt\n";
eval {
PVE::Network::SDN::Vnets::del_ips_from_mac(
diff --git a/src/lxc-pve-poststop-hook b/src/lxc-pve-poststop-hook
index c8bbb79..c5d1059 100755
--- a/src/lxc-pve-poststop-hook
+++ b/src/lxc-pve-poststop-hook
@@ -11,7 +11,7 @@ use PVE::GuestHelpers;
use PVE::LXC::Config;
use PVE::LXC::Tools;
use PVE::LXC;
-use PVE::Network;
+use PVE::Network::SDN::Zones;
use PVE::RESTEnvironment;
use PVE::Storage;
use PVE::Tools;
@@ -49,8 +49,8 @@ PVE::LXC::Tools::lxc_hook(
my $ind = $1;
my $net = PVE::LXC::Config->parse_lxc_network($conf->{$k});
next if $net->{type} ne 'veth';
- # veth_delete tests with '-d /sys/class/net/$name' before running the command
- PVE::Network::veth_delete("veth${vmid}i$ind");
+ # reports the unplug and deletes the veth only where it still exists
+ PVE::Network::SDN::Zones::veth_delete("veth${vmid}i$ind");
}
my $config_updated = 0;
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-manager v4 17/20] ui: sdn: add zone DHCP backend selector and subnet DHCP options
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (15 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-container v4 16/20] net: report NIC plugs and removals to the SDN DHCP backends Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-manager v4 18/20] sdn: bring up the DHCP backends at boot before the guests start Hannes Laimer
` (2 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
Zones of every type can now get DHCP from the eBPF backend, while
dnsmasq stays limited to simple zones. The checkbox that enabled
dnsmasq on simple zones becomes a backend selector in the advanced
options of every zone dialog.
The subnet dialog gains fields for the new DHCP lease time and for the
DHCP DNS server, which could be set only through the API until now.
The eBPF backend hands out a DNS server only when the subnet sets one.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com>
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
www/manager6/sdn/SubnetEdit.js | 23 +++++++++++++++++++++++
www/manager6/sdn/zones/Base.js | 17 +++++++++++++++++
www/manager6/sdn/zones/SimpleEdit.js | 11 -----------
3 files changed, 40 insertions(+), 11 deletions(-)
diff --git a/www/manager6/sdn/SubnetEdit.js b/www/manager6/sdn/SubnetEdit.js
index a3608428..a1d36eeb 100644
--- a/www/manager6/sdn/SubnetEdit.js
+++ b/www/manager6/sdn/SubnetEdit.js
@@ -56,6 +56,29 @@ Ext.define('PVE.sdn.SubnetInputPanel', {
deleteEmpty: '{!isCreate}',
},
},
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'dhcp-dns-server',
+ vtype: 'IP64Address',
+ fieldLabel: gettext('DHCP DNS Server'),
+ allowBlank: true,
+ skipEmptyText: true,
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
+ {
+ xtype: 'proxmoxintegerfield',
+ name: 'dhcp-lease-time',
+ fieldLabel: gettext('DHCP Lease Time') + ' (s)',
+ emptyText: Proxmox.Utils.defaultText,
+ minValue: 60,
+ maxValue: 4294967295,
+ allowBlank: true,
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
],
});
diff --git a/www/manager6/sdn/zones/Base.js b/www/manager6/sdn/zones/Base.js
index 66f93de0..7696136d 100644
--- a/www/manager6/sdn/zones/Base.js
+++ b/www/manager6/sdn/zones/Base.js
@@ -82,6 +82,23 @@ Ext.define('PVE.panel.SDNZoneBase', {
},
);
+ let dhcpBackends = [
+ ['__default__', Proxmox.Utils.NoneText],
+ ['ebpf', 'eBPF'],
+ ];
+ if (me.type === 'simple') {
+ dhcpBackends.splice(1, 0, ['dnsmasq', 'dnsmasq']);
+ }
+
+ me.advancedItems.push({
+ xtype: 'proxmoxKVComboBox',
+ name: 'dhcp',
+ fieldLabel: gettext('DHCP Backend'),
+ comboItems: dhcpBackends,
+ value: '__default__',
+ deleteEmpty: !me.isCreate,
+ });
+
me.callParent();
},
});
diff --git a/www/manager6/sdn/zones/SimpleEdit.js b/www/manager6/sdn/zones/SimpleEdit.js
index ba10bb36..aeb076ae 100644
--- a/www/manager6/sdn/zones/SimpleEdit.js
+++ b/www/manager6/sdn/zones/SimpleEdit.js
@@ -19,17 +19,6 @@ Ext.define('PVE.sdn.zones.SimpleInputPanel', {
var me = this;
me.items = [];
- me.advancedItems = [
- {
- xtype: 'proxmoxcheckbox',
- name: 'dhcp',
- inputValue: 'dnsmasq',
- uncheckedValue: null,
- checked: false,
- fieldLabel: gettext('Automatic DHCP'),
- deleteEmpty: !me.isCreate,
- },
- ];
me.callParent();
},
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-manager v4 18/20] sdn: bring up the DHCP backends at boot before the guests start
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (16 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-manager v4 17/20] ui: sdn: add zone DHCP backend selector and subnet DHCP options Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH proxmox-ve-rs v4 19/20] ve-config: sdn: add the ebpf dhcp backend Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-docs v4 20/20] sdn: dhcp: add eBPF backend section Hannes Laimer
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
The eBPF DHCP responder keeps its programs and maps pinned in bpffs,
so they are gone after a reboot. The SDN commit helper now brings the
DHCP backends back up on every boot, not only when it has changes to
commit.
The helper runs before the node and HA start the guests at boot.
Otherwise the first guest interface plug loads the responder with a
full pass, which holds the responder's lock exclusively. A guest
starting meanwhile waits for that lock. One that waits too long comes
up without the responder on its interface.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Notes:
v4:
* log that the DHCP backends are being brought up
bin/pve-sdn-commit | 4 ++++
services/pve-sdn-commit.service | 1 +
2 files changed, 5 insertions(+)
diff --git a/bin/pve-sdn-commit b/bin/pve-sdn-commit
index aa7e9b29..13231e4f 100644
--- a/bin/pve-sdn-commit
+++ b/bin/pve-sdn-commit
@@ -96,6 +96,10 @@ sub sdn_changed {
return fabrics_changed();
}
+# the dhcp backends bring back what a reboot took, pending changes or not
+print "bringing up the DHCP backends\n";
+PVE::Network::SDN::Dhcp::boot();
+
if (!sdn_changed()) {
print "No changes to SDN configuration detected, skipping reload\n";
exit 0;
diff --git a/services/pve-sdn-commit.service b/services/pve-sdn-commit.service
index ff723725..825926be 100644
--- a/services/pve-sdn-commit.service
+++ b/services/pve-sdn-commit.service
@@ -3,6 +3,7 @@ Description=Commit Proxmox VE SDN changes
DefaultDependencies=no
Wants=pve-cluster.service network.target
After=frr.service network.target corosync.service
+Before=pve-guests.service pve-ha-lrm.service
[Service]
ExecStart=/usr/share/pve-manager/helpers/pve-sdn-commit
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH proxmox-ve-rs v4 19/20] ve-config: sdn: add the ebpf dhcp backend
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (17 preceding siblings ...)
2026-10-21 8:44 ` [PATCH pve-manager v4 18/20] sdn: bring up the DHCP backends at boot before the guests start Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
2026-10-21 8:44 ` [PATCH pve-docs v4 20/20] sdn: dhcp: add eBPF backend section Hannes Laimer
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Notes:
v4:
* new
proxmox-ve-config/src/sdn/config.rs | 3 +++
1 file changed, 3 insertions(+)
diff --git a/proxmox-ve-config/src/sdn/config.rs b/proxmox-ve-config/src/sdn/config.rs
index 2f30cf2..50c6947 100644
--- a/proxmox-ve-config/src/sdn/config.rs
+++ b/proxmox-ve-config/src/sdn/config.rs
@@ -104,6 +104,7 @@ impl Display for ZoneType {
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub enum DhcpType {
Dnsmasq,
+ Ebpf,
}
proxmox_serde::forward_deserialize_to_from_str!(DhcpType);
proxmox_serde::forward_serialize_to_display!(DhcpType);
@@ -114,6 +115,7 @@ impl FromStr for DhcpType {
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s {
"dnsmasq" => Ok(DhcpType::Dnsmasq),
+ "ebpf" => Ok(DhcpType::Ebpf),
_ => Err(SdnConfigError::InvalidDhcpType),
}
}
@@ -123,6 +125,7 @@ impl Display for DhcpType {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(match self {
DhcpType::Dnsmasq => "dnsmasq",
+ DhcpType::Ebpf => "ebpf",
})
}
}
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH pve-docs v4 20/20] sdn: dhcp: add eBPF backend section
2026-10-21 8:44 [PATCH container/docs/manager/network/proxmox{-ebpf,-perl-rs,-ve-rs}/qemu-server v4 00/20] sdn: implement DHCP for all zones using eBPF Hannes Laimer
` (18 preceding siblings ...)
2026-10-21 8:44 ` [PATCH proxmox-ve-rs v4 19/20] ve-config: sdn: add the ebpf dhcp backend Hannes Laimer
@ 2026-10-21 8:44 ` Hannes Laimer
19 siblings, 0 replies; 21+ messages in thread
From: Hannes Laimer @ 2026-10-21 8:44 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
pvesdn.adoc | 101 ++++++++++++++++++++++++++++++++++++++++++----------
1 file changed, 83 insertions(+), 18 deletions(-)
diff --git a/pvesdn.adoc b/pvesdn.adoc
index 3fd3533..ec5c500 100644
--- a/pvesdn.adoc
+++ b/pvesdn.adoc
@@ -98,10 +98,12 @@ source /etc/network/interfaces.d/*
DHCP IPAM
~~~~~~~~~
-The DHCP integration into the built-in 'PVE' IP Address Management stack
-currently uses `dnsmasq` for giving out DHCP leases. This is currently opt-in.
+The DHCP integration into the built-in 'PVE' IP Address Management stack gives
+out leases either through `dnsmasq` or through an eBPF backend built into
+{pve}. Each zone selects its backend, and DHCP is opt-in per zone.
-To use that feature you need to install the `dnsmasq` package on every node:
+The eBPF backend needs no additional package. To use the `dnsmasq` backend you
+need to install the `dnsmasq` package on every node:
----
apt update
@@ -153,9 +155,8 @@ in your SDN setup.
* xref:pvesdn_config_controllers[Controllers]: For controlling layer 3 routing
in complex setups
-* xref:pvesdn_config_dhcp[DHCP]: Define a DHCP server for a zone that
- automatically allocates IPs for guests in the IPAM and leases them to the
- guests via DHCP.
+* xref:pvesdn_config_dhcp[DHCP]: Let a zone automatically allocate IPs for
+ guests in the IPAM and lease them to the guests via DHCP.
* xref:pvesdn_config_ipam[IPAM]: Enables external for IP address management for
guests
@@ -229,6 +230,11 @@ DNSZone:: DNS domain name. Used to register hostnames, such as
`<hostname>.<domain>`. The DNS zone must already exist on the DNS server.
Optional.
+DHCP Backend:: Hand out leases to the zone's guests through this backend, see
+ xref:pvesdn_config_dhcp[DHCP]. The `dnsmasq` backend is only available for
+ Simple Zones. The `ebpf` backend works with every zone type. Optional,
+ defaults to none, which disables DHCP.
+
[[pvesdn_zone_plugin_simple]]
Simple Zones
@@ -442,6 +448,15 @@ SNAT:: Enable Source NAT which allows VMs from inside a
DNS Zone Prefix:: Add a prefix to the domain registration, like
<hostname>.prefix.<domain> Optional.
+DHCP DNS Server:: The DNS server handed out to guests with their lease.
+ Optional.
+
+DHCP Lease Time:: The lease time in seconds, at least 60. A guest picks up a
+ changed mapping when it renews its lease, so a shorter lease time applies
+ changes sooner. Without it `dnsmasq` hands out infinite leases and the `ebpf`
+ backend ten-minute leases. `dnsmasq` raises anything below two minutes to
+ two minutes. Optional.
+
[[pvesdn_config_controllers]]
Controllers
@@ -1343,8 +1358,9 @@ DHCP
The DHCP plugin in {pve} SDN can be used to automatically deploy a DHCP server
for a Zone. It provides DHCP for all Subnets in a Zone that have a DHCP range
-configured. Currently the only available backend plugin for DHCP is the dnsmasq
-plugin.
+configured. Each zone selects one of two backend plugins, `dnsmasq` or `ebpf`.
+Both answer from the MAC address and IP mappings in the IPAM. The `ebpf`
+backend serves IPv4 subnets only.
The DHCP plugin works by allocating an IP in the IPAM plugin configured in the
Zone when adding a new network interface to a VM/CT. You can find more
@@ -1362,10 +1378,12 @@ available when using the xref:pvesdn_ipam_plugin_pveipam[PVE IPAM plugin].
Configuration
~~~~~~~~~~~~~
-You can enable automatic DHCP for a zone in the Web UI via the Zones panel and
-enabling DHCP in the advanced options of a zone.
+You can enable automatic DHCP for a zone in the web interface via the Zones
+panel by selecting a backend in the *DHCP Backend* field of the zone's advanced
+options.
-NOTE: Currently only Simple Zones have support for automatic DHCP
+NOTE: The `dnsmasq` backend is only available for Simple Zones. The `ebpf`
+backend works with every zone type.
After automatic DHCP has been enabled for a Zone, DHCP Ranges need to be
configured for the subnets in a Zone. In order to that, go to the Vnets panel and
@@ -1379,22 +1397,24 @@ pvesh set /cluster/sdn/vnets/<vnet>/subnets/<subnet>
-dhcp-range start-address=10.0.2.100,end-address=10.0.2.200
----
-You also need to have a gateway configured for the subnet - otherwise
-automatic DHCP will not work.
-
The DHCP plugin will then allocate IPs in the IPAM only in the configured
ranges.
-Do not forget to follow the installation steps for the
-xref:pvesdn_install_dhcp_ipam[dnsmasq DHCP plugin] as well.
+The `dnsmasq` backend needs a gateway on the subnet. The `ebpf` backend also
+serves a subnet without one, but then hands out no default route.
+
+The subnet options *DHCP DNS Server* and *DHCP Lease Time* set the DNS server
+and the lease time handed out, see xref:pvesdn_config_subnet[Subnets].
+
+When using the `dnsmasq` backend, do not forget to follow the installation
+steps for the xref:pvesdn_install_dhcp_ipam[dnsmasq DHCP plugin] as well.
Plugins
~~~~~~~
Dnsmasq Plugin
^^^^^^^^^^^^^^
-Currently this is the only DHCP plugin and therefore the plugin that gets used
-when you enable DHCP for a zone.
+This plugin serves zones with `dnsmasq` as their DHCP backend.
.Installation
For installation see the xref:pvesdn_install_dhcp_ipam[DHCP IPAM] section.
@@ -1437,6 +1457,51 @@ For more information please consult the documentation of
xref:pvesdn_ipam_plugin_pveipam[the PVE IPAM plugin]. Changing DHCP leases is
currently not supported for the other IPAM plugins.
+[[pvesdn_dhcp_plugin_ebpf]]
+eBPF Plugin
+^^^^^^^^^^^
+This plugin serves zones with `ebpf` as their DHCP backend.
+
+.Installation
+The backend needs no additional package. It requires host kernel version 6.6
+or newer.
+
+Instead of a DHCP server on the bridge, the backend attaches a small eBPF
+program to the network interface of every guest on the zone's VNets. The
+program answers the guest's DHCP requests from the MAC address and IP mapping
+in the IPAM, so these requests never reach the bridge. It only answers
+requests that carry the MAC address of its own network interface. Requests on
+a network interface without a mapping pass through untouched, so such a guest
+can still reach another DHCP server on the network.
+
+The backend attaches the program when a guest starts or a network interface
+is hot-plugged. Applying the SDN configuration attaches it to running guests
+as well, so a zone switched to `ebpf` serves them without a restart.
+
+A mapping changed through {pve} reaches a running guest when it next renews
+its lease. After a mapping is deleted, the backend stops answering the guest,
+which keeps its address until the lease runs out.
+
+The answer carries the address, netmask, lease time, and server address. The
+zone's MTU, the subnet's gateway, and its IPv4 DNS server are included where
+the zone or subnet has one. Options `dnsmasq` can add through its custom
+configuration, such as a domain name or PXE boot parameters, are not
+available.
+
+.Notes
+The backend serves IPv4 only. Only untagged requests are answered, so a guest
+tagging its own traffic, for example on a trunk port, is not served. A guest
+with a mapping cannot use another DHCP server on the network, since its
+requests never leave its network interface. Clients behind a guest, such as
+containers bridged inside a VM, get no answer, and neither does a guest that
+changed its MAC address in the guest OS.
+
+NOTE: Requests this backend answers never reach the firewall of the guest's
+network interface, so the guest's firewall rules and the DHCP option of its
+firewall have no effect on them. The rate limit applies to the reply only.
+Requests the backend does not answer pass the firewall and the rate limit as
+usual.
+
[[pvesdn_firewall_integration]]
Firewall Integration
--------------------
--
2.47.3
^ permalink raw reply related [flat|nested] 21+ messages in thread