Re: [pve-devel] [RFC cluster/manager/network 0/6] Add support for DHCP servers to SDN

["DERUMIER, Alexandre" <alexandre.derumier@groupe-cyllene.com>]

Le mercredi 13 septembre 2023 à 10:54 +0200, Stefan Hanreich a écrit :
> Sorry for my late reply, I was a bit busy the last two days and I
> also 
> wanted some time to think about your suggestions.
> 
> On 9/11/23 05:53, DERUMIER, Alexandre wrote:
> > Hi,
> > 
> > I think we should think how we want to attribute ips to the vms
> > before
> > continue the implementation. >
> > I think they are 2 models:
> > 
> > 1)
> > 
> > - we want that dhcp server attribute itself ips && leases from the
> > subnets/ranges configured.
> > 
> > That mean that leases need to be shared across nodes.  (from the
> > same
> > cluster maybe with /etc/pve tricks,   but in real world, it should
> > also
> > works across multiple clusters, as it's not uncommon to shared
> > subnets
> > in differents cluster, public network,...)
> > 
> > So we don't have that 2 differents vms starting on the same time on
> > 2
> > differents cluster, receive the same ips. (so dhcp servers need to
> > use
> > some kind of central lock,...)
> > 
> 
> This is also something I have thought about, but I assume dnsmasq is
> not 
> really built in mind with multiple instances accessing the same
> leases file.
> 
> This problem would be solved by using distributed DHCP servers like
> kea. 
> kea on the other hand has the issue that it we need to set up a SQL 
> database or other external storage. Alternatively we need to write a
> new 
> backend for kea that integrates with our pmxcfs.

using pmxcfs could be great for 1 cluster, but if you are multiple
clusters sharing same subnet it'll not work.


Maybe, for cross-cluster, only ip reservations should be used.
and for (dynamic|ephemeral) ip using a subnet specific to the cluster?


> 
> This is partly why I think Thomas mentioned implementing our own DHCP
> server, where we have the flexibility of handling things as we see
> fit.
> 
> Then we can just recommend the dnsmasq plugin for simple setups (e.g.
> single node setups), while more advanced setups should opt for other 
> DHCP backends.
> 
> > 
> > 2)
> > 
> > The other way (my preferred way), could be to use ipam. (where we
> > already have local ipam, or external ipams like netbox/phpipam for
> > sharing between multiple cluster).
> > 
> > 
> > The ip is reserved in ipam  (automatic find next free ip at vm
> > creation
> > for example, or manually in the gui, or maybe at vm start if we
> > want
> > ephemeral ip), then registered dns,
> > and generated dhcp server config with mac-ip reserversation. (for
> > dhcp
> > server config generation, it could be a daemon pooling the ipam
> > database change for example)
> > 
> > Like this, no need to handle lease sharing, so it can work with any
> > dhcp server.
> > 
> 
> Implementing this via IPAM plugins seems like a good idea, but if we 
> want to use distributed DHCP servers like kea (or our own 
> implementation) then this might not be needed in those cases. It also
> adds quite a bit of complexity.
> 
> With dnsmasq there is even the possibility of running scripts (via 
> --dhcp-script, see the docs [1]) when a lease is added / changed / 
> deleted. But as far as I can tell this can not be used to override
> the 
> IP that dnsmasq provides via DHCP, so it is probably not really
> useful 
> for our use-case.
> 
> ------

(I have sent another mail with more detail of what I was thinking to
implement)
> 

> Another method that I had in mind was providing a DHCP forwarding
> plugin 
> that proxies the DHCP requests to another DHCP server (that can then 
> even be outside the cluster). This way there is only one DHCP server 
> that handles keeping track of the leases and you do not have the
> issue 
> of having to handle sharing a lease database / using IPAM. So, for 
> instance, you have a DHCP server running on one node and the other
> nodes 
> just proxy their requests to the one DHCP server.
> 
> I was also thinking we could implement setting the IP for a specific
> VM 
> on interfaces where we have a DHCP server, since we can then just 
> provide fixed IPs for specific MAC-addresses. This could be quite 
> convenient.
> 
> 
I'm always a little bit afraid to use a central dhcp (or a couple in
HA) for my production. Because if a problem occur on dhcp when vms are
starting after a major outage for example.

Personnally, I'm still using static ips in my vms for this.

(And also, I'm using multiple datacenters, with public ips range, so 1
central dhcp is really not possible )





> 
> [1]
> https://antiphishing.cetsi.fr/proxy/v3?i=d09ZU0Z5WTAyTG85WWdYbIX9F1yND7gsvpr6o9NYFYg&r=UTEzTUpQcktwRVdhdEg1TKCFOzhw8CGaAiMfyFTpTR_LTspF9zP2JS-LN0ctA-XBzHeMG-sD1OqL3ihNxDMXJg&f=TmtFVlNVNmxSYnFaWFhxYgWRqBlL4orB2JX8m5oXcL1BuLSwOjOIAbslpc0EWkZJ&u=https%3A//thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html&k=DWI7
>