From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 51C041FF09C for ; Mon, 02 Nov 2026 15:56:41 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 0C2CC216F3; Mon, 02 Nov 2026 15:56:39 +0100 (CET) From: Hannes Laimer To: pve-devel@lists.proxmox.com Subject: [PATCH docs/network/qemu-server v2 00/11] sdn: improve MAC cache, DHCP reservations and scope both to the vnet Date: Mon, 2 Nov 2026 15:56:19 +0100 Message-ID: <20261102145630.1555953-1-h.laimer@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1793631393864 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.367 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: RPO4EIDYSEVLFKCQE35HGSHLIBNZAZE3 X-Message-ID-Hash: RPO4EIDYSEVLFKCQE35HGSHLIBNZAZE3 X-MailFrom: h.laimer@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: This series addresses some problems in the IPAM code, mostly around caching. It also solves the problem that the MAC cache and the lookup behind it held a single address per MAC. This is a problem since MACs only have to be unique within a vnet and can repeat across vnets, and it led to a guest getting the IP of a guest with the same MAC on another vnet. Patches 1-5 address general, mostly cache related problems. Patches 1-4 were already included in [1], 1-3 changed since then. Patch 1 leaves entries without an address in the file and treats them as a miss, patch 2 skips the fill when the file changed while the IPAM was asked, and patch 3 leaves the cache keys as written, since nodes not yet upgraded still use them. Patch 5 is new. Patch 6 adds updating of the dnsmasq reservations when a mapping changes, so dnsmasq picks the change up immediately and a guest gets its new address without a restart. Patches 7-8 scope the lookup and the cache to the vnet, so guests don't get addresses of another vnet. The cache file now contains a `vnets` map next to `macs`. `macs` stays and is still written, since nodes not yet upgraded read and write only it. Its entries are never replaced, a write only adds an address where the MAC has none of that family yet, and they serve as fallback for a vnet only with addresses inside that vnet's subnets. A miss in the cache asks the IPAM within the vnet's subnets, and that answer is put into `vnets`. A `vnets` address is trusted only while `macs` still holds the MAC's family and names no address inside the vnet's subnets that none of the MAC's entries holds, so a release or an edit on a node not yet upgraded is noticed and the IPAM is asked again. So an old entry answers only for the vnet it was written for. The one exception is the same or an overlapping range on two vnets, where an old entry answers for the other vnet too, as it did before, until that vnet has an entry of its own or the address is released. Patch 9 adds separation by zone for the two external IPAM providers we support, since two zones can use the same range. - NetBox: each zone gets its own `pve-` VRF, and the prefixes of its subnets go in there - phpIPAM: each zone gets a sub-section `pve-` under the configured one, and its subnets go in there(only if the configured section is not a sub-section already, phpIPAM only allows 1 deep nesting) Patch 10 is qemu-server's side. The vmlist names the source of a live migration until the config moves at its end, so an edit in that window updates the source. Every resume now pushes the current mapping of each NIC, and the migration's last step resumes the target after the move, so it catches up there. The patch needs the pve-network of this series. Taken from [1] as well. Patch 11 just adds docs explaining this for the two external IPAMs. This is not a breaking change, existing assignments are returned exactly like they are already. Cache entries from before keep their addresses, since nodes not yet upgraded still use them, and answer for a vnet with the addresses in its subnets, so a running guest keeps its reservation. Records in an external IPAM stay where they are, in NetBox's global table or in phpIPAM's configured section. Like the cache, an IPAM lookup first checks the zone's VRF or section for the subnet and falls back to the global table or the configured section. Unlike with the cache, migration can be done manually, by moving a prefix together with its addresses and IP ranges into the zone's VRF in NetBox, or a subnet into the zone's sub-section in phpIPAM, since the lookup checks those first. Migration is not necessary though. Creating a subnet creates the VRF or sub-section if it doesn't exist yet, nothing else does. A subnet whose range is still in the global table or the configured section is refused, since a copy would hide the records in it. The error says to move that prefix or subnet into the zone's VRF or sub-section, or to remove it. We don't move anything and we don't have to. If users are bothered by guests getting IPs not assigned to them, they can move the global entries into the `pve-` VRF or sub-section, where they'll be picked up correctly. note: I'll rebase [1] onto this, it'll lose some patches that moved here. With [1] things addressed here become more apparent/visible eventhough they are pre-existing, that is why [1] will be based on this. [1] https://lore.proxmox.com/pve-devel/20261021084445.462195-1-h.laimer@proxmox.com/ v2: - the cache and the lookup are kept per vnet, the old map stays for nodes not yet upgraded - entries from before are left as written, a fill is skipped when the file changed meanwhile - a release leaves a record that belongs to another MAC alone - each zone gets its own VRF or section in the external IPAMs, and phpIPAM gets the lookup by MAC - an edit for a guest whose node is offline runs here, and the reservation rewrite reads the cache once - the target of a live migration catches up when the guest resumes pve-network: Hannes Laimer (9): sdn: ipam: stop caching IPAM misses sdn: ipam: write the lookup's cache entry under the cluster lock sdn: ipam: read the MAC cache case-insensitively sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it sdn: ipam: release an address the built-in IPAM no longer holds sdn: dhcp: update reservations when ip mappings change sdn: ipam: look a MAC up within the vnet's subnets sdn: ipam: keep the MAC cache per vnet sdn: ipam: give each zone a VRF in NetBox and a section in phpIPAM src/PVE/API2/Network/SDN/Ips.pm | 38 +- src/PVE/Network/SDN/Dhcp.pm | 56 +- src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 148 +++-- src/PVE/Network/SDN/Dhcp/Plugin.pm | 6 +- src/PVE/Network/SDN/Ipams.pm | 223 ++++++- src/PVE/Network/SDN/Ipams/NetboxPlugin.pm | 209 +++++-- src/PVE/Network/SDN/Ipams/PVEPlugin.pm | 27 +- src/PVE/Network/SDN/Ipams/PhpIpamPlugin.pm | 220 +++++-- src/PVE/Network/SDN/Ipams/Plugin.pm | 7 +- src/PVE/Network/SDN/Subnets.pm | 14 +- src/PVE/Network/SDN/Vnets.pm | 14 +- src/test/ipams/netbox/expected.add_ip | 2 +- .../ipams/netbox/expected.add_ip_notgateway | 2 +- src/test/ipams/netbox/expected.add_subnet | 2 +- src/test/ipams/netbox/expected.update_ip | 2 +- src/test/run_test_ipams.pl | 366 +++++++++++ src/test/run_test_vnets_blackbox.pl | 569 +++++++++++++++++- 17 files changed, 1692 insertions(+), 213 deletions(-) qemu-server: Hannes Laimer (1): network: sdn: push the DHCP mappings of a guest when it resumes src/PVE/QemuServer/Network.pm | 14 ++++++++++++++ src/PVE/QemuServer/RunState.pm | 2 ++ 2 files changed, 16 insertions(+) pve-docs: Hannes Laimer (1): sdn: ipam: document the per-zone VRF and section in NetBox and phpIPAM pvesdn.adoc | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) Summary over all repositories: 20 files changed, 1733 insertions(+), 213 deletions(-) -- Generated by murpp 0.12.0