Re: [pve-devel] [PATCH container 1/1] Signed-off-by: Maurice Klein <klein@aetherus.de>

[Stefan Hanreich <s.hanreich@proxmox.com>]

On 2/1/26 3:31 PM, Maurice Klein wrote:
> Basicly the vnet and subnet part I see as in issue.
> Since in this kind of setup there is no defined subsets required the
> current configuration doesn't fully make sense.
> I guess you could still have a subnet configuration and configure all
> the host addresses inside that subnet, but it's not really necessary .
> Every VM route would be a /32 route and also the configured address on
> that bridge (gateway field) would be a /32.

We would still need a local IP on the PVE host that acts as a gateway
and preferably an IP for the VM inside the subnet so you can route the
traffic for the /32 IPs there. So we'd need to configure e.g.
192.0.2.0/24 as subnet, then have the host as gateway (e.g. 192.0.2.1)
and each VM gets an IP inside that subnet (which could automatically be
handled via IPAM / DHCP). Looking at other implementations (e.g.
kube-router) there's even a whole subnet pool and each node gets one
subnet from that pool - but that's easier done with containers than VMs,
so I think the approach with one shared subnet seems easier
(particularly for VM mobility).


> When the tap interface of a vm gets plugged a route needs to be created.
> Routes per VM get created with the comand ip route add 192.168.1.5/32
> dev routedbridge.
> The /32 gateway address needs to be configured on the bridge as well.

This could be done in in the respective tap_plug / veth_create functions
inside pve-network [1]. You can override them on a per-zone basis so
that would fit right in. We'd have to implement analogous functions for
teardown though so we can remove the routes when updating / deleting the
tap / veth.

Someone has actually implemented a quite similar thing via utilizing
hooks and a dedicated config files for each VM - see [2]. They're using
IPv6-LL addresses though (which I would personally also prefer), but I'm
unsure how it would work with windows guests for instance and it might
be weird / unintuitive for some users (see my previous mail).


> There needs to be some way to configure the guests IPs as well, but in
> ipam there is currently no way to set a ip for a vm, it's only ip mac
> bindings.

That's imo the real question left, where to store the additional IPs.
Zone config is awkward, PVE IPAM might be workable with introducing
additional fields (and, for a PoC we could just deny using any other
IPAM plugin than that and implement it later).

Network device is probably the best bet, since we can then utilize the
hotplug code in case an IP gets reassigned, which would be more
complicated with the other approaches. The only reason why I'm reluctant
is because we're introducing a property there that is specific to one
particular SDN zone and unused by everything else.


> A potential security flaw is also devices on that bridge can steal a
> configured ip by just replying to arp.
> That could be mitigated by disabling bridge learning and creating static
> atp entires as well for those configured IPs.

That setting should be exposed in the zone configuration and probably be
on by default. There's also always the option of using IP / MAC filters
in the firewall although the static fdb / neighbor table approach is
preferable imo.


[0] https://docs.cilium.io/en/stable/network/lb-ipam/#requesting-ips
[1]
https://git.proxmox.com/?p=pve-network.git;a=blob;f=src/PVE/Network/SDN/Zones.pm;h=4da94580e07d6b3dcb794f19ce9335412fa7bc41;hb=HEAD#l298
[2] https://siewert.io/posts/2022/announce-proxmox-vm-ips-via-bgp-1/