[pve-devel] [PATCH pve-network] dnsmasq: configure static range for each subnet

[pve-devel] [PATCH pve-network] dnsmasq: configure static range for each subnet

[Alexandre Derumier]

we don't want dynamic lease, simply define each subnet as a static range.

dhcp-range defined on a subnet is only used by ipam plugin.

This will also allow to use dhcp subnet without need to define a range.
Can be usefull for external ipam like phpipam, where you can't define ranges.

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
---
 src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
index 46172c5..2db7f4f 100644
--- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
+++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
@@ -112,11 +112,18 @@ sub configure_subnet {
 sub configure_range {
     my ($class, $dhcpid, $subnet_config, $range_config) = @_;
 
-    my $range_file = "$DNSMASQ_CONFIG_ROOT/$dhcpid/10-$subnet_config->{id}.ranges.conf",
+    my $subnet_file = "$DNSMASQ_CONFIG_ROOT/$dhcpid/10-$subnet_config->{id}.conf";
     my $tag = $subnet_config->{id};
 
-    open(my $fh, '>>', $range_file) or die "Could not open file '$range_file' $!\n";
-    print $fh "dhcp-range=set:$tag,$range_config->{'start-address'},$range_config->{'end-address'}\n";
+    my ($zone, $network, $mask) = split(/-/, $tag);
+
+    if (Net::IP::ip_is_ipv4($network)) {
+	$mask = (2 ** $mask - 1) << (32 - $mask);
+	$mask = join( '.', unpack( "C4", pack( "N", $mask ) ) );
+    }
+
+    open(my $fh, '>>', $subnet_file) or die "Could not open file '$subnet_file' $!\n";
+    print $fh "dhcp-range=set:$tag,$network,static,$mask,infinite\n";
     close $fh;
 }
 
-- 
2.39.2

Re: [pve-devel] [PATCH pve-network] dnsmasq: configure static range for each subnet

[DERUMIER, Alexandre]

mmmm,it's not so easy.

dnsmasq still use a lease file (and need it) for static reversation...


and it keep in memory the reservation too,
that mean than we can't reattribute an existing ip, even if we have
delete it previouly from reservation/ipam

the only way is to delete the lease file, and restart dnsmasq.
(reload is not enough :/ )


It'll try to look at dnsmasq code && maybe do test with kea to compare.



-------- Message initial --------
De: Alexandre Derumier <aderumier@odiso.com>
Répondre à: Proxmox VE development discussion <pve-
devel@lists.proxmox.com>
À: pve-devel@lists.proxmox.com
Objet: [pve-devel] [PATCH pve-network] dnsmasq: configure static range
for each subnet
Date: 15/11/2023 16:13:50

we don't want dynamic lease, simply define each subnet as a static
range.

dhcp-range defined on a subnet is only used by ipam plugin.

This will also allow to use dhcp subnet without need to define a range.
Can be usefull for external ipam like phpipam, where you can't define
ranges.

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
---
 src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
index 46172c5..2db7f4f 100644
--- a/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
+++ b/src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
@@ -112,11 +112,18 @@ sub configure_subnet {
 sub configure_range {
     my ($class, $dhcpid, $subnet_config, $range_config) = @_;
 
-    my $range_file = "$DNSMASQ_CONFIG_ROOT/$dhcpid/10-$subnet_config-
>{id}.ranges.conf",
+    my $subnet_file = "$DNSMASQ_CONFIG_ROOT/$dhcpid/10-$subnet_config-
>{id}.conf";
     my $tag = $subnet_config->{id};
 
-    open(my $fh, '>>', $range_file) or die "Could not open file
'$range_file' $!\n";
-    print $fh "dhcp-range=set:$tag,$range_config->{'start-
address'},$range_config->{'end-address'}\n";
+    my ($zone, $network, $mask) = split(/-/, $tag);
+
+    if (Net::IP::ip_is_ipv4($network)) {
+	$mask = (2 ** $mask - 1) << (32 - $mask);
+	$mask = join( '.', unpack( "C4", pack( "N", $mask ) ) );
+    }
+
+    open(my $fh, '>>', $subnet_file) or die "Could not open file
'$subnet_file' $!\n";
+    print $fh "dhcp-range=set:$tag,$network,static,$mask,infinite\n";
     close $fh;
 }
 

Re: [pve-devel] [PATCH pve-network] dnsmasq: configure static range for each subnet

[DERUMIER, Alexandre]

>>
>>I have a similar solution for the dhcp-range.
>>I played around with adding and removing IPs from IPAM and It looks
>>like
>>dnsmasq is offering the correct IP, as stored in IPAM and ethers
>>file.
>>
>>I tried with a regular reboot to get a new IP.
>>
>>What I saw when testing with the previous dhcp-range config.
>>VM with an IP in IPAM, manually remove IP from ethers file,
>>the VM will still get an IP in the range, but not the one originally
>>offered.
>>If dnsmasq never saw the MAC, it will not offer an IP.
>>
>>If you find a specific scenario that does not work, please post a
>>step
>>by step description so I can try to reproduce and to get a better
>>understanding of the dnsmasq config.

I can reproduce easily 100%:


- create a nic with mac:xx:xx:xx:xY  ip: 192.168.0.10
- start vm. (the ether file is filed + reload)
- the guest do a dhcp request, the dnsmasq respond  a store the lease
in /var/lib/misc/zone.lease

- delete the nic


- add a new nic in same vm or another vm,  free found ip is
192.168.0.10  (because it was removed)


- start the vm (the ether file is upgrade with the new ip mac + reload)

- the guest do a dhcp request: the dnsmasq can't respond (with my last
patch) or give a dynamic ip in the range (with current implementation)
because it's still see his lease file the old mac:ip assocation



so, the solution is to remove lease file and restart dnsmasq




others dhcp daemons:


KEA
----
With kea, it possible to update/del/add lease directly through unix
socket. (but not static reservation, it's a commercial plugin).
That mean that if an unknown client is doing a request, it can return a
lease in the pool range (and we don't known it, and could allocate it)


echo '{ "command": "lease4-get-all" }'  | socat /run/kea/kea4-ctrl-
socket -,ignoreeof


echo '{ "command": "lease4-del", "arguments": {"ip-address":
"192.168.2.20"} }'  | socat /run/kea/kea4-ctrl-socket -,ignoreeof


echo '{ "command": "lease4-add", "arguments": {"ip-address":
"192.168.2.20", "hw-address": "1a:1b:1c:1d:1e:1f"} }'  | socat
/run/kea/kea4-ctrl-socket -,ignoreeof

echo '{ "command": "lease4-update", "arguments": {"ip-address":
"192.168.2.20", "hw-address": "1a:1b:1c:1d:1e:1f"} }'  | socat
/run/kea/kea4-ctrl-socket -,ignoreeof



FREERADIUS
----------

freeradius seem interesting, as it's possible to do custom
plugins,including perl


https://serverfault.com/questions/1098827/freeradius-with-dhcp-server-calls-to-perl-module-returns-error

So, maybe it could be possible to read directly the macs.db database in
/etc/pve dynamically.

I need to read the doc to see how it's works





What we need is just a stupid daemon replying to dhcp request with dhcp
offers using static mac:ip  (with correct dhcp protocol implementation


Maybe some pure perl daemon exist ?
or python like https://github.com/flan/staticdhcpd ?

Re: [pve-devel] [PATCH pve-network] dnsmasq: configure static range for each subnet

[Stefan Lendl]

"DERUMIER, Alexandre" <alexandre.derumier@groupe-cyllene.com> writes:

>
> I can reproduce easily 100%:
>
>
> - create a nic with mac:xx:xx:xx:xY  ip: 192.168.0.10
> - start vm. (the ether file is filed + reload)
> - the guest do a dhcp request, the dnsmasq respond  a store the lease
> in /var/lib/misc/zone.lease
>
> - delete the nic
>
>
> - add a new nic in same vm or another vm,  free found ip is
> 192.168.0.10  (because it was removed)
>
>
> - start the vm (the ether file is upgrade with the new ip mac + reload)
>
> - the guest do a dhcp request: the dnsmasq can't respond (with my last
> patch) or give a dynamic ip in the range (with current implementation)
> because it's still see his lease file the old mac:ip assocation
>

I experimented with several approaches with dnsmasq leases.
I cannot reproduce your example because it works in my examples.
My procedure:

dnsmasq config:
dhcp-range=set:DHCPNAT-10.1.0.0-16,10.1.0.0,static,255.255.0.0,infinite

VM 108 net1: MAC: bc:24:11:ad:0e:2e

qm set 108 --delete net1

dnsmasq lease file still contains the lease for MAC bc:24:11:ad:0e:2e

qm set 108 --net1 model=virtio,bridge=dhcpnat

- ethers file gets updated to new mac: BC:24:11:51:10:AD
- soon after dnsmasq lease gets updated to the new lease as well!
- correct IP assigned in the VM

I also tried:
- ip link set down > ip link set upi
- reboot
- force Stop the VM

So far, *this all works!*


I also tried with a short dhcp lease in dnsmasq. With this
configuration, the new IP will even propagate to the VM and set
correctly after IPAM update.

In my tests I used 30s but something like 5 or 10min should be fine as well.

dhcp-range=set:DHCPNAT-10.1.0.0-16,10.1.0.0,static,255.255.0.0,30

The VM is polling every ~60s as seen on the wire with tcpdump:

tcpdump -i dhcpnat -n port 67 or port 68

After I manually update the ethers file and `systemctl *reload*
dnsmasq`, it will respond with the new IP.

dnsmasq is running *locally only* so any DHCP queries are limited to the
local bridge.

The biggest problem and for me the reason I think it's not a feasible
solution, is that dnsmasq becomes a single point of failure.
If dnsmasq is offline, all of the VMs will have *NO IP*.

Re: [pve-devel] [PATCH pve-network] dnsmasq: configure static range for each subnet

[DERUMIER, Alexandre]

>>I experimented with several approaches with dnsmasq leases.
>>I cannot reproduce your example because it works in my examples.
>>My procedure:

>>dnsmasq config:
dhcp-range=set:DHCPNAT-10.1.0.0-16,10.1.0.0,static,255.255.0.0,infinite

>>VM 108 net1: MAC: bc:24:11:ad:0e:2e
>>
>>qm set 108 --delete net1
>>
>>dnsmasq lease file still contains the lease for MAC bc:24:11:ad:0e:2e
>>
>>qm set 108 --net1 model=virtio,bridge=dhcpnat
>>
>>- ethers file gets updated to new mac: BC:24:11:51:10:AD
>>- soon after dnsmasq lease gets updated to the new lease as well!
>>- correct IP assigned in the VM

>>I also tried:
>>- ip link set down > ip link set upi
>>- reboot
>>- force Stop the VM
>>
>>So far, *this all works!*

mmm, and you reassign the same ip address to the new mac ?





>>I also tried with a short dhcp lease in dnsmasq. With this
>>configuration, the new IP will even propagate to the VM and set
>>correctly after IPAM update.
>>
>>In my tests I used 30s but something like 5 or 10min should be fine
>>as well.
>>
>>dhcp-range=set:DHCPNAT-10.1.0.0-16,10.1.0.0,static,255.255.0.0,30
>>
>>The VM is polling every ~60s as seen on the wire with tcpdump:
>>
>>tcpdump -i dhcpnat -n port 67 or port 68
>>
>>After I manually update the ethers file and `systemctl *reload*
>>dnsmasq`, it will respond with the new IP.
>>
>>dnsmasq is running *locally only* so any DHCP queries are limited to
>>the
>>local bridge.
>>
>>The biggest problem and for me the reason I think it's not a feasible
>>solution, is that dnsmasq becomes a single point of failure.
>>If dnsmasq is offline, all of the VMs will have *NO IP*.


Yes, personnality, I'm more for infinite leases to not have spof

Re: [pve-devel] [PATCH pve-network] dnsmasq: configure static range for each subnet

[DERUMIER, Alexandre]

the debug log without my patch:

vm with 12:45:db:3a:04:97  got  192.168.2.10

Nov 15 21:33:31 formationkvm3 dnsmasq-dhcp[796025]: DHCPDISCOVER(vnetpve) 192.168.2.10 12:45:db:3a:04:97
Nov 15 21:33:31 formationkvm3 dnsmasq-dhcp[796025]: DHCPOFFER(vnetpve) 192.168.2.10 12:45:db:3a:04:97
Nov 15 21:33:31 formationkvm3 dnsmasq-dhcp[796025]: DHCPREQUEST(vnetpve) 192.168.2.10 12:45:db:3a:04:97
Nov 15 21:33:31 formationkvm3 dnsmasq-dhcp[796025]: DHCPACK(vnetpve) 192.168.2.10 12:45:db:3a:04:97 testovn1
Nov 15 21:34:47 formationkvm3 systemd[1]: Reloading dnsmasq@simpve.service - dnsmasq (simpve) - A lightweight DHCP and caching DNS server...

remove nic, assign another mac 12:45:db:3a:04:97 with 192.168.2.10

vm start, generate ether file + reload

Nov 15 21:34:47 formationkvm3 dnsmasq[796025]: cleared cache
Nov 15 21:34:47 formationkvm3 dnsmasq-dhcp[796025]: read /etc/dnsmasq.d/simpve/ethers
Nov 15 21:34:47 formationkvm3 systemd[1]: Reloaded dnsmasq@simpve.service - dnsmasq (simpve) - A lightweight DHCP and caching DNS server.

vm do the dhcp request and the server refuse because 192.168.2.10  is still leased to 12:45:db:3a:04:97

Nov 15 21:35:05 formationkvm3 dnsmasq-dhcp[796025]: not using configured address 192.168.2.10 because it is leased to 12:45:db:3a:04:97
Nov 15 21:35:05 formationkvm3 dnsmasq-dhcp[796025]: DHCPDISCOVER(vnetpve) 192.168.2.10 12:45:10:22:fb:fd no address available
Nov 15 21:35:09 formationkvm3 dnsmasq-dhcp[796025]: not using configured address 192.168.2.10 because it is leased to 12:45:db:3a:04:97
Nov 15 21:35:09 formationkvm3 dnsmasq-dhcp[796025]: DHCPDISCOVER(vnetpve) 192.168.2.10 12:45:10:22:fb:fd no address available
Nov 15 21:35:15 formationkvm3 dnsmasq-dhcp[796025]: not using configured address 192.168.2.10 because it is leased to 12:45:db:3a:04:97
Nov 15 21:35:15 formationkvm3 dnsmasq-dhcp[796025]: DHCPDISCOVER(vnetpve) 192.168.2.10 12:45:10:22:fb:fd no address available
Nov 15 21:35:26 formationkvm3 dnsmasq-dhcp[796025]: not using configured address 192.168.2.10 because it is leased to 12:45:db:3a:04:97
Nov 15 21:35:26 formationkvm3 dnsmasq-dhcp[796025]: DHCPDISCOVER(vnetpve) 192.168.2.10 12:45:10:22:fb:fd no address available


-------- Message initial --------
De: Stefan Lendl <s.lendl@proxmox.com>
À: "DERUMIER, Alexandre" <alexandre.derumier@groupe-cyllene.com>
Cc: pve-devel@lists.proxmox.com <pve-devel@lists.proxmox.com>
Objet: Re: [pve-devel] [PATCH pve-network] dnsmasq: configure static range for each subnet
Date: 16/11/2023 13:53:45

"DERUMIER, Alexandre" <alexandre.derumier@groupe-cyllene.com> writes:


I can reproduce easily 100%:


- create a nic with mac:xx:xx:xx:xY  ip: 192.168.0.10
- start vm. (the ether file is filed + reload)
- the guest do a dhcp request, the dnsmasq respond  a store the lease
in /var/lib/misc/zone.lease

- delete the nic


- add a new nic in same vm or another vm,  free found ip is
192.168.0.10  (because it was removed)


- start the vm (the ether file is upgrade with the new ip mac + reload)

- the guest do a dhcp request: the dnsmasq can't respond (with my last
patch) or give a dynamic ip in the range (with current implementation)
because it's still see his lease file the old mac:ip assocation


I experimented with several approaches with dnsmasq leases.
I cannot reproduce your example because it works in my examples.
My procedure:

dnsmasq config:
dhcp-range=set:DHCPNAT-10.1.0.0-16,10.1.0.0,static,255.255.0.0,infinite

VM 108 net1: MAC: bc:24:11:ad:0e:2e

qm set 108 --delete net1

dnsmasq lease file still contains the lease for MAC bc:24:11:ad:0e:2e

qm set 108 --net1 model=virtio,bridge=dhcpnat

- ethers file gets updated to new mac: BC:24:11:51:10:AD
- soon after dnsmasq lease gets updated to the new lease as well!
- correct IP assigned in the VM

I also tried:
- ip link set down > ip link set upi
- reboot
- force Stop the VM

So far, *this all works!*


I also tried with a short dhcp lease in dnsmasq. With this
configuration, the new IP will even propagate to the VM and set
correctly after IPAM update.

In my tests I used 30s but something like 5 or 10min should be fine as well.

dhcp-range=set:DHCPNAT-10.1.0.0-16,10.1.0.0,static,255.255.0.0,30

The VM is polling every ~60s as seen on the wire with tcpdump:

tcpdump -i dhcpnat -n port 67 or port 68

After I manually update the ethers file and `systemctl *reload*
dnsmasq`, it will respond with the new IP.

dnsmasq is running *locally only* so any DHCP queries are limited to the
local bridge.

The biggest problem and for me the reason I think it's not a feasible
solution, is that dnsmasq becomes a single point of failure.
If dnsmasq is offline, all of the VMs will have *NO IP*.

Re: [pve-devel] [PATCH pve-network] dnsmasq: configure static range for each subnet

[Stefan Lendl]

"DERUMIER, Alexandre" <alexandre.derumier@groupe-cyllene.com> writes:

>>>So far, *this all works!*
>
> mmm, and you reassign the same ip address to the new mac ?
>

If that is the first free IP in the range, the vNic gets the same IP.
Otherwise it will get a new IP.

Re: [pve-devel] [PATCH pve-network] dnsmasq: configure static range for each subnet

[DERUMIER, Alexandre]

> > > So far, *this all works!*
> 
> mmm, and you reassign the same ip address to the new mac ?
> 

>>If that is the first free IP in the range, the vNic gets the same IP.
>>Otherwise it will get a new IP.


Can you share your dnsmasq config file ? Maybe do you have extra option
make it working for you ?


from my test, once a lease it done, they are no way to manually release
it without restart dnsmasq.

I have also tried the "leasefile-ro" option, it's not writing in the
lease file, but still keep leases in memory.