[PATCH docs/network/qemu-server v2 00/11] sdn: improve MAC cache, DHCP reservations and scope both to the vnet

[Hannes Laimer <h.laimer@proxmox.com>]

This series addresses some problems in the IPAM code, mostly around
caching. It also solves the problem that the MAC cache and the lookup
behind it held a single address per MAC. This is a problem since MACs
only have to be unique within a vnet and can repeat across vnets, and
it led to a guest getting the IP of a guest with the same MAC on
another vnet.

Patches 1-5 address general, mostly cache related problems. Patches
1-4 were already included in [1], 1-3 changed since then. Patch 1
leaves entries without an address in the file and treats them as a
miss, patch 2 skips the fill when the file changed while the IPAM was
asked, and patch 3 leaves the cache keys as written, since nodes not
yet upgraded still use them. Patch 5 is new.

Patch 6 adds updating of the dnsmasq reservations when a mapping
changes, so dnsmasq picks the change up immediately and a guest gets
its new address without a restart.

Patches 7-8 scope the lookup and the cache to the vnet, so guests
don't get addresses of another vnet. The cache file now contains a
`vnets` map next to `macs`. `macs` stays, since nodes not yet
upgraded read and write only it, and keeps holding one address per
MAC as before. Every write goes into both maps. A lookup answers
from `vnets` as long as `macs` still names the same address.
Otherwise it asks the IPAM within the vnet's subnets, since a node
not yet upgraded changed or released the address in between. An
entry only `macs` has is one from before, it answers for a vnet
only with an address in the vnet's subnets. A MAC on two vnets can
match `macs` on one of them only, so the other one asks the IPAM at
every start. So an old entry answers only for the vnet it was
written for. The one exception is the same or an overlapping range
on two vnets, where an old entry answers for the other vnet too, as
it did before, until that vnet has an entry of its own or the
address is released.

Patch 9 adds separation by zone for the two external IPAM providers
we support, since two zones can use the same range.
 - NetBox: each zone gets its own `pve-<zone>` VRF, and the prefixes
   of its subnets go in there
 - phpIPAM: each zone gets a sub-section `pve-<zone>` under the
   configured one, and its subnets go in there(only if the configured
   section is not a sub-section already, phpIPAM only allows 1 deep
   nesting). Section names are global in phpIPAM, so zone ids must
   be unique across the clusters sharing one

Patch 10 is qemu-server's side. The vmlist names the source of a live
migration until the config moves at its end, so an edit in that
window updates the source. Every resume now pushes the current
mapping of each NIC, and the migration's last step resumes the target
after the move, so it catches up there. The patch needs the
pve-network of this series. Taken from [1] as well.

Patch 11 just adds docs explaining this for the two external IPAMs.

This is not a breaking change, existing assignments are returned
exactly like they are already. Cache entries from before keep their
addresses, since nodes not yet upgraded still use them, and answer for
a vnet with the addresses in its subnets, so a running guest keeps its
reservation. Records in an external IPAM stay where they are, in
NetBox's global table or in phpIPAM's configured section. Like the
cache, an IPAM lookup first checks the zone's VRF or section for the
subnet and falls back to the global table or the configured section.
Unlike with the cache, migration can be done manually, by moving a
prefix together with its addresses and IP ranges into the zone's VRF
in NetBox, or a subnet into the zone's sub-section in phpIPAM, since
the lookup checks those first. Migration is not necessary though.
Creating a subnet creates the VRF or sub-section if it doesn't exist
yet, nothing else does. A subnet whose range is still in the global
table or the configured section is refused, since a copy would hide
the records in it. The error says to move that prefix or subnet into
the zone's VRF or sub-section, or to remove it. We don't move
anything and we don't have to.
If users are bothered by guests getting IPs not assigned to them, they
can move the global entries into the `pve-<zone>` VRF or sub-section,
where they'll be picked up correctly. While a rolling upgrade is
under way, a node not yet upgraded puts an explicit mapping into
NetBox's global table, also for a subnet created on an upgraded
node, such addresses get moved like the ones from before.

note: I'll rebase [1] onto this, it'll lose some patches that moved
here. With [1] things addressed here become more apparent/visible
eventhough they are pre-existing, that is why [1] will be based on this.

[1] https://lore.proxmox.com/pve-devel/20261021084445.462195-1-h.laimer@proxmox.com/

v3:
 - a start asks the IPAM when the MAC sits on two vnets and the cache
   can't vouch for it
 - ask the IPAM for the changed MAC before walking the reservations,
   so its stale line is replaced
 - phpIPAM: fail subnet creation on api error
 - an API mapping edit fails when the reservation update does, the
   records are written by then

v2:
 - the cache and the lookup are kept per vnet, the old map stays
   for nodes not yet upgraded
 - entries from before are left as written, a fill is skipped when
   the file changed meanwhile
 - a release leaves a record that belongs to another MAC alone
 - each zone gets its own VRF or section in the external IPAMs, and
   phpIPAM gets the lookup by MAC
 - an edit for a guest whose node is offline runs here, and the
   reservation rewrite reads the cache once
 - the target of a live migration catches up when the guest resumes


pve-network:

Hannes Laimer (9):
  sdn: ipam: stop caching IPAM misses
  sdn: ipam: write the lookup's cache entry under the cluster lock
  sdn: ipam: read the MAC cache case-insensitively
  sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it
  sdn: ipam: release an address the built-in IPAM no longer holds
  sdn: dhcp: update reservations when ip mappings change
  sdn: ipam: look a MAC up within the vnet's subnets
  sdn: ipam: keep the MAC cache per vnet
  sdn: ipam: give each zone a VRF in NetBox and a section in phpIPAM

 src/PVE/API2/Network/SDN/Ips.pm               |  38 +-
 src/PVE/Network/SDN/Dhcp.pm                   |  56 +-
 src/PVE/Network/SDN/Dhcp/Dnsmasq.pm           | 153 +++--
 src/PVE/Network/SDN/Dhcp/Plugin.pm            |   7 +-
 src/PVE/Network/SDN/Ipams.pm                  | 180 ++++-
 src/PVE/Network/SDN/Ipams/NetboxPlugin.pm     | 223 +++++--
 src/PVE/Network/SDN/Ipams/PVEPlugin.pm        |  27 +-
 src/PVE/Network/SDN/Ipams/PhpIpamPlugin.pm    | 237 ++++++-
 src/PVE/Network/SDN/Ipams/Plugin.pm           |   7 +-
 src/PVE/Network/SDN/Subnets.pm                |  14 +-
 src/PVE/Network/SDN/Vnets.pm                  |  21 +-
 src/test/ipams/netbox/expected.add_ip         |   2 +-
 .../ipams/netbox/expected.add_ip_notgateway   |   2 +-
 src/test/ipams/netbox/expected.add_subnet     |   2 +-
 src/test/ipams/netbox/expected.update_ip      |   2 +-
 src/test/run_test_ipams.pl                    | 468 +++++++++++++
 src/test/run_test_vnets_blackbox.pl           | 618 +++++++++++++++++-
 17 files changed, 1836 insertions(+), 221 deletions(-)


qemu-server:

Hannes Laimer (1):
  network: sdn: push the DHCP mappings of a guest when it resumes

 src/PVE/QemuServer/Network.pm  | 19 +++++++++++++++++++
 src/PVE/QemuServer/RunState.pm |  2 ++
 2 files changed, 21 insertions(+)


pve-docs:

Hannes Laimer (1):
  sdn: ipam: document the per-zone VRF and section in NetBox and phpIPAM

 pvesdn.adoc | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)


Summary over all repositories:
  20 files changed, 1888 insertions(+), 221 deletions(-)

-- 
Generated by murpp 0.12.0