From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 906E01FF0C0 for ; Wed, 04 Nov 2026 10:57:53 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 0537A2141D; Wed, 04 Nov 2026 10:57:40 +0100 (CET) Message-ID: <89ffa1b1-4fe6-4c45-a348-73c95009a232@proxmox.com> Date: Wed, 4 Nov 2026 10:56:21 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: superseded: [PATCH docs/manager/network/qemu-server v4 00/13] sdn: improve MAC cache, DHCP reservations and scope both to the vnet To: pve-devel@lists.proxmox.com References: <20261103185559.420277-1-h.laimer@proxmox.com> From: Hannes Laimer Content-Language: en-US In-Reply-To: <20261103185559.420277-1-h.laimer@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1793786181320 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.313 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: MI2PI3CYQ532GJMXS44VAFTZOHHLPGIH X-Message-ID-Hash: MI2PI3CYQ532GJMXS44VAFTZOHHLPGIH X-MailFrom: h.laimer@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: superseded-by: https://lore.proxmox.com/pve-devel/20261104095520.398544-1-h.laimer@proxmox.com/ On 2026-11-03 19:56, Hannes Laimer wrote: > This series addresses some problems in the IPAM code, mostly around > caching. It also solves the problem that the MAC cache and the lookup > behind it held a single address per MAC. This is a problem since MACs > only have to be unique within a vnet and can repeat across vnets, and > it led to a guest getting the IP of a guest with the same MAC on > another vnet. > > Patches 1-5 address general, mostly cache related problems. Patch 1 > leaves entries without an address in the file and treats them as a > miss, patch 2 skips the fill when the file changed while the IPAM was > asked, and patch 3 leaves the cache keys as written, since nodes not > yet upgraded still use them. Patch 4 replaces a stale reservation > line instead of dropping it, and patch 5 lets the built-in IPAM > release an address it no longer holds, so its cache entry is cleared. > > Patch 6 adds updating of the dnsmasq reservations when a mapping > changes, so dnsmasq picks the change up immediately and a guest gets > its new address without a restart. The edit runs on the guest's node > under its migration lock, so pve-network now depends on > pve-guest-common. > > Patches 7-8 scope the lookup and the cache to the vnet, so guests > don't get addresses of another vnet. The cache file now contains a > `vnets` map next to `macs`. `macs` stays, since nodes not yet > upgraded read and write only it, and keeps holding one address per > MAC as before. Every write goes into both maps. A lookup answers > from `vnets` as long as `macs` still names the same address. > Otherwise it asks the IPAM within the vnet's subnets, since a write > for the MAC on another vnet or a node not yet upgraded changed or > released the address in between. An entry only `macs` has is one > from before, it answers for a vnet only with an address in the > vnet's subnets. A MAC with an address of the same family on two > vnets can match `macs` on one of them at most, so a start on the > other asks the IPAM. So an old entry answers only for the vnet it > was written for. The one exception is the same or an overlapping range > on two vnets, where an old entry answers for the other vnet too, as > it did before, until one of the two has an entry of its own or the > address is released. > > Patch 9 adds separation by zone for the two external IPAM providers > we support, since two zones can use the same range. It is behind a > `zone-namespaces` option of the IPAM entry, off by default, to be > turned on once every node runs this version. > - NetBox: each zone gets its own `pve-` VRF, and the prefixes > of its subnets go in there > - phpIPAM: each zone gets a sub-section `pve-` under the > configured one, and its subnets go in there(only if the configured > section is not a sub-section already, phpIPAM only allows 1 deep > nesting) > The names start with a `zone-namespace-prefix` of the IPAM entry, > `pve` by default, so clusters sharing an IPAM stay apart. Patch 9 > also gives phpIPAM the lookup by MAC, which failed before. > > Patch 10 is qemu-server's side. An edit and a migration of the guest > exclude each other since patch 6. An edit through a node not yet > upgraded still goes through during one, and it writes the IPAM and > the cache but no reservation. A resume now pushes the current mapping > of each NIC from the cache, and a migration resumes the target after > the config moved, so the target serves what the cache vouches for. > The patch needs the pve-network of this series. > > Patches 11-12 are pve-manager's side, the DHCP tree keeping the > subnets of two zones with one range apart, it merged them under the > first zone before, and the two options in the NetBox and phpIPAM > dialogs. Patch 12 needs the pve-network of this series. > > Patch 13 just adds docs explaining this for the two external IPAMs. > > This is not a breaking change, existing assignments stay as they are > and keep answering for their vnet. Cache entries from before keep > their addresses, since nodes not yet upgraded still use them, and > answer for a vnet with the addresses in its subnets, so a running > guest keeps its reservation. Records in an external IPAM stay where > they are, in NetBox's global table or in phpIPAM's configured section. > Like the cache, an IPAM lookup first checks the zone's VRF or section > for the subnet and falls back to the global table or the configured > section. Unlike with the cache, migration can be done manually, by > moving a prefix together with its addresses and IP ranges into the > zone's VRF in NetBox, or a subnet into the zone's sub-section in > phpIPAM, since the lookup checks those first. Migration is not > necessary though. With the option on, creating a subnet creates the > VRF or sub-section if it doesn't exist yet, nothing else does. A > subnet whose range is still in the global table or the configured > section is then refused, since a copy would hide the records in it. > The error says to move that prefix or subnet into the VRF or > sub-section of its zone, or to remove it. We don't move anything and > we don't have to. With the option off nothing changes in the external > IPAM, so a rolling upgrade is safe, and a node not yet upgraded never > meets a subnet in a VRF or sub-section. > > v4: > - add IPAM config flag to use zone scoped layout, tried avoiding this, > but that added much more problems than it solved > - a mapping edit and a migration of the guest exclude each other > - the resume of a guest pushes its reservations from the cache only > - the same address on two vnets of a MAC is checked with the IPAM at > every start > - a prefix option of the IPAM entry names the zone namespaces, pve > by default, so clusters sharing an IPAM stay apart > - pve-manager: the two options in the dialogs, and the DHCP tree > keeps the subnets of two zones with one range apart > > v3: > - a start asks the IPAM when the MAC sits on two vnets and the cache > can't vouch for it > - ask the IPAM for the changed MAC before walking the reservations, > so its stale line is replaced > - phpIPAM: fail subnet creation on api error > - an API mapping edit fails when the reservation update does, the > records are written by then > > v2: > - the cache and the lookup are kept per vnet, the old map stays > for nodes not yet upgraded > - entries from before are left as written, a fill is skipped when > the file changed meanwhile > - a release leaves a record that belongs to another MAC alone > - each zone gets its own VRF or section in the external IPAMs, and > phpIPAM gets the lookup by MAC > - an edit for a guest whose node is offline runs here, and the > reservation rewrite reads the cache once > - the target of a live migration catches up when the guest resumes > > > pve-network: > > Hannes Laimer (9): > sdn: ipam: stop caching IPAM misses > sdn: ipam: cache a MAC lookup's answer under the cluster lock > sdn: ipam: read the MAC cache case-insensitively > sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it > sdn: ipam: release an address the built-in IPAM no longer holds > sdn: dhcp: update reservations when IP mappings change > sdn: ipam: look a MAC up within the vnet's subnets > sdn: ipam: keep the MAC cache per vnet > sdn: ipam: give each zone a VRF in NetBox and a section in phpIPAM > > debian/control | 2 + > src/PVE/API2/Network/SDN/Ips.pm | 108 ++- > src/PVE/Network/SDN/Dhcp.pm | 56 +- > src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 163 +++-- > src/PVE/Network/SDN/Dhcp/Plugin.pm | 8 +- > src/PVE/Network/SDN/Ipams.pm | 188 ++++- > src/PVE/Network/SDN/Ipams/NetboxPlugin.pm | 239 +++++-- > src/PVE/Network/SDN/Ipams/PVEPlugin.pm | 27 +- > src/PVE/Network/SDN/Ipams/PhpIpamPlugin.pm | 245 ++++++- > src/PVE/Network/SDN/Ipams/Plugin.pm | 24 +- > src/PVE/Network/SDN/Subnets.pm | 14 +- > src/PVE/Network/SDN/Vnets.pm | 21 +- > src/test/ipams/netbox/expected.add_ip | 2 +- > .../ipams/netbox/expected.add_ip_notgateway | 2 +- > src/test/ipams/netbox/expected.update_ip | 2 +- > src/test/run_test_ipams.pl | 582 +++++++++++++++ > src/test/run_test_vnets_blackbox.pl | 664 +++++++++++++++++- > 17 files changed, 2116 insertions(+), 231 deletions(-) > > > qemu-server: > > Hannes Laimer (1): > network: sdn: push the DHCP mappings of a guest when it resumes > > src/PVE/QemuServer/Network.pm | 17 +++++++++++++++++ > src/PVE/QemuServer/RunState.pm | 2 ++ > 2 files changed, 19 insertions(+) > > > pve-manager: > > Hannes Laimer (2): > ui: sdn: dhcp tree: do not merge the same subnet in different zones > ui: sdn: ipam: add the zone namespace options > > www/manager6/sdn/ipams/Base.js | 25 +++++++++++++++++++++++++ > www/manager6/sdn/ipams/NetboxEdit.js | 2 ++ > www/manager6/sdn/ipams/PhpIpamEdit.js | 2 ++ > www/manager6/tree/DhcpTree.js | 8 +++++--- > 4 files changed, 34 insertions(+), 3 deletions(-) > > > pve-docs: > > Hannes Laimer (1): > sdn: ipam: document zone namespaces in NetBox and phpIPAM > > pvesdn.adoc | 33 +++++++++++++++++++++++++++++++++ > 1 file changed, 33 insertions(+) > > > Summary over all repositories: > 24 files changed, 2202 insertions(+), 234 deletions(-) >