[PATCH docs/network/qemu-server v2 00/11] sdn: improve MAC cache, DHCP reservations and scope both to the vnet

[Hannes Laimer <h.laimer@proxmox.com>]

This series addresses some problems in the IPAM code, mostly around
caching. It also solves the problem that the MAC cache and the lookup
behind it held a single address per MAC. This is a problem since MACs
only have to be unique within a vnet and can repeat across vnets, and
it led to a guest getting the IP of a guest with the same MAC on
another vnet.

Patches 1-5 address general, mostly cache related problems. Patches
1-4 were already included in [1], 1-3 changed since then. Patch 1
leaves entries without an address in the file and treats them as a
miss, patch 2 skips the fill when the file changed while the IPAM was
asked, and patch 3 leaves the cache keys as written, since nodes not
yet upgraded still use them. Patch 5 is new.

Patch 6 adds updating of the dnsmasq reservations when a mapping
changes, so dnsmasq picks the change up immediately and a guest gets
its new address without a restart.

Patches 7-8 scope the lookup and the cache to the vnet, so guests
don't get addresses of another vnet. The cache file now contains a
`vnets` map next to `macs`. `macs` stays and is still written, since
nodes not yet upgraded read and write only it. Its entries are never
replaced, a write only adds an address where the MAC has none of that
family yet, and they serve as fallback for a vnet only with addresses
inside that vnet's subnets. A miss in the cache asks the IPAM within
the vnet's subnets, and that answer is put into `vnets`. A `vnets`
address is trusted only while `macs` still holds the MAC's family and
names no address inside the vnet's subnets that none of the MAC's
entries holds, so a release or an edit on a node not yet upgraded is
noticed and the IPAM is asked again. So an old entry answers only for
the vnet it was written for. The one exception is the same or an
overlapping range on two vnets, where an old entry answers for the
other vnet too, as it did before, until that vnet has an entry of its
own or the address is released.

Patch 9 adds separation by zone for the two external IPAM providers
we support, since two zones can use the same range.
 - NetBox: each zone gets its own `pve-<zone>` VRF, and the prefixes
   of its subnets go in there
 - phpIPAM: each zone gets a sub-section `pve-<zone>` under the
   configured one, and its subnets go in there(only if the configured
   section is not a sub-section already, phpIPAM only allows 1 deep
   nesting)

Patch 10 is qemu-server's side. The vmlist names the source of a live
migration until the config moves at its end, so an edit in that
window updates the source. Every resume now pushes the current
mapping of each NIC, and the migration's last step resumes the target
after the move, so it catches up there. The patch needs the
pve-network of this series. Taken from [1] as well.

Patch 11 just adds docs explaining this for the two external IPAMs.

This is not a breaking change, existing assignments are returned
exactly like they are already. Cache entries from before keep their
addresses, since nodes not yet upgraded still use them, and answer for
a vnet with the addresses in its subnets, so a running guest keeps its
reservation. Records in an external IPAM stay where they are, in
NetBox's global table or in phpIPAM's configured section. Like the
cache, an IPAM lookup first checks the zone's VRF or section for the
subnet and falls back to the global table or the configured section.
Unlike with the cache, migration can be done manually, by moving a
prefix together with its addresses and IP ranges into the zone's VRF
in NetBox, or a subnet into the zone's sub-section in phpIPAM, since
the lookup checks those first. Migration is not necessary though.
Creating a subnet creates the VRF or sub-section if it doesn't exist
yet, nothing else does. A subnet whose range is still in the global
table or the configured section is refused, since a copy would hide
the records in it. The error says to move that prefix or subnet into
the zone's VRF or sub-section, or to remove it. We don't move
anything and we don't have to.
If users are bothered by guests getting IPs not assigned to them, they
can move the global entries into the `pve-<zone>` VRF or sub-section,
where they'll be picked up correctly.

note: I'll rebase [1] onto this, it'll lose some patches that moved
here. With [1] things addressed here become more apparent/visible
eventhough they are pre-existing, that is why [1] will be based on this.

[1] https://lore.proxmox.com/pve-devel/20261021084445.462195-1-h.laimer@proxmox.com/


v2:
 - the cache and the lookup are kept per vnet, the old map stays
   for nodes not yet upgraded
 - entries from before are left as written, a fill is skipped when
   the file changed meanwhile
 - a release leaves a record that belongs to another MAC alone
 - each zone gets its own VRF or section in the external IPAMs, and
   phpIPAM gets the lookup by MAC
 - an edit for a guest whose node is offline runs here, and the
   reservation rewrite reads the cache once
 - the target of a live migration catches up when the guest resumes


pve-network:

Hannes Laimer (9):
  sdn: ipam: stop caching IPAM misses
  sdn: ipam: write the lookup's cache entry under the cluster lock
  sdn: ipam: read the MAC cache case-insensitively
  sdn: dhcp: replace a stale dnsmasq reservation instead of dropping it
  sdn: ipam: release an address the built-in IPAM no longer holds
  sdn: dhcp: update reservations when ip mappings change
  sdn: ipam: look a MAC up within the vnet's subnets
  sdn: ipam: keep the MAC cache per vnet
  sdn: ipam: give each zone a VRF in NetBox and a section in phpIPAM

 src/PVE/API2/Network/SDN/Ips.pm               |  38 +-
 src/PVE/Network/SDN/Dhcp.pm                   |  56 +-
 src/PVE/Network/SDN/Dhcp/Dnsmasq.pm           | 148 +++--
 src/PVE/Network/SDN/Dhcp/Plugin.pm            |   6 +-
 src/PVE/Network/SDN/Ipams.pm                  | 223 ++++++-
 src/PVE/Network/SDN/Ipams/NetboxPlugin.pm     | 209 +++++--
 src/PVE/Network/SDN/Ipams/PVEPlugin.pm        |  27 +-
 src/PVE/Network/SDN/Ipams/PhpIpamPlugin.pm    | 220 +++++--
 src/PVE/Network/SDN/Ipams/Plugin.pm           |   7 +-
 src/PVE/Network/SDN/Subnets.pm                |  14 +-
 src/PVE/Network/SDN/Vnets.pm                  |  14 +-
 src/test/ipams/netbox/expected.add_ip         |   2 +-
 .../ipams/netbox/expected.add_ip_notgateway   |   2 +-
 src/test/ipams/netbox/expected.add_subnet     |   2 +-
 src/test/ipams/netbox/expected.update_ip      |   2 +-
 src/test/run_test_ipams.pl                    | 366 +++++++++++
 src/test/run_test_vnets_blackbox.pl           | 569 +++++++++++++++++-
 17 files changed, 1692 insertions(+), 213 deletions(-)


qemu-server:

Hannes Laimer (1):
  network: sdn: push the DHCP mappings of a guest when it resumes

 src/PVE/QemuServer/Network.pm  | 14 ++++++++++++++
 src/PVE/QemuServer/RunState.pm |  2 ++
 2 files changed, 16 insertions(+)


pve-docs:

Hannes Laimer (1):
  sdn: ipam: document the per-zone VRF and section in NetBox and phpIPAM

 pvesdn.adoc | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)


Summary over all repositories:
  20 files changed, 1733 insertions(+), 213 deletions(-)

-- 
Generated by murpp 0.12.0