Re: [pve-devel] [RFC cluster/manager/network 0/6] Add support for DHCP servers to SDN

[Stefan Hanreich <s.hanreich@proxmox.com>]

Sorry for my late reply, I was a bit busy the last two days and I also 
wanted some time to think about your suggestions.

On 9/11/23 05:53, DERUMIER, Alexandre wrote:
> Hi,
> 
> I think we should think how we want to attribute ips to the vms before
> continue the implementation. >
> I think they are 2 models:
> 
> 1)
> 
> - we want that dhcp server attribute itself ips && leases from the
> subnets/ranges configured.
> 
> That mean that leases need to be shared across nodes.  (from the same
> cluster maybe with /etc/pve tricks,   but in real world, it should also
> works across multiple clusters, as it's not uncommon to shared subnets
> in differents cluster, public network,...)
> 
> So we don't have that 2 differents vms starting on the same time on 2
> differents cluster, receive the same ips. (so dhcp servers need to use
> some kind of central lock,...)
>

This is also something I have thought about, but I assume dnsmasq is not 
really built in mind with multiple instances accessing the same leases file.

This problem would be solved by using distributed DHCP servers like kea. 
kea on the other hand has the issue that it we need to set up a SQL 
database or other external storage. Alternatively we need to write a new 
backend for kea that integrates with our pmxcfs.

This is partly why I think Thomas mentioned implementing our own DHCP 
server, where we have the flexibility of handling things as we see fit.

Then we can just recommend the dnsmasq plugin for simple setups (e.g. 
single node setups), while more advanced setups should opt for other 
DHCP backends.

> 
> 2)
> 
> The other way (my preferred way), could be to use ipam. (where we
> already have local ipam, or external ipams like netbox/phpipam for
> sharing between multiple cluster).
> 
> 
> The ip is reserved in ipam  (automatic find next free ip at vm creation
> for example, or manually in the gui, or maybe at vm start if we want
> ephemeral ip), then registered dns,
> and generated dhcp server config with mac-ip reserversation. (for dhcp
> server config generation, it could be a daemon pooling the ipam
> database change for example)
> 
> Like this, no need to handle lease sharing, so it can work with any
> dhcp server.
> 

Implementing this via IPAM plugins seems like a good idea, but if we 
want to use distributed DHCP servers like kea (or our own 
implementation) then this might not be needed in those cases. It also 
adds quite a bit of complexity.

With dnsmasq there is even the possibility of running scripts (via 
--dhcp-script, see the docs [1]) when a lease is added / changed / 
deleted. But as far as I can tell this can not be used to override the 
IP that dnsmasq provides via DHCP, so it is probably not really useful 
for our use-case.

------

Another method that I had in mind was providing a DHCP forwarding plugin 
that proxies the DHCP requests to another DHCP server (that can then 
even be outside the cluster). This way there is only one DHCP server 
that handles keeping track of the leases and you do not have the issue 
of having to handle sharing a lease database / using IPAM. So, for 
instance, you have a DHCP server running on one node and the other nodes 
just proxy their requests to the one DHCP server.

I was also thinking we could implement setting the IP for a specific VM 
on interfaces where we have a DHCP server, since we can then just 
provide fixed IPs for specific MAC-addresses. This could be quite 
convenient.



[1] https://thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html