public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] [PATCH v4 cluster/network/manager/qemu-server/container/docs 00/33] Add support for DHCP servers to SDN
@ 2023-11-17 11:39 Stefan Hanreich
  2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-cluster 01/33] add priv/macs.db Stefan Hanreich
                   ` (35 more replies)
  0 siblings, 36 replies; 66+ messages in thread
From: Stefan Hanreich @ 2023-11-17 11:39 UTC (permalink / raw)
  To: pve-devel

This patch series adds support for automatically deploying dnsmasq as a DHCP
server to a simple SDN Zone. This series is now in a state where I think it is
okay to integrate into the respective repositories **except** for pve-container

You can (and should) check out the documentation, but here is a quick setup
guide for your convenienve:

You need to install dnsmasq (and disable it afterwards):

  apt install dnsmasq && systemctl disable --now dnsmasq


You can use the following example configuration for deploying a DHCP server in
a SDN subnet, you should also be able to recreate this configuration in the
web UI:


/etc/pve/sdn/zones.cfg:

  simple: DHCPNAT
          ipam pve
          dhcp dnsmasq


/etc/pve/sdn/vnets.cfg:

  vnet: dhcpnat
          zone DHCPNAT


/etc/pve/sdn/subnets.cfg:

  subnet: DHCPNAT-10.1.0.0-16
          vnet dhcpnat
          dhcp-dns-server 10.1.0.1
          dhcp-range start-address=10.1.0.100,end-address=10.1.0.200
          gateway 10.1.0.1
          snat 1

Don't forget to apply the new configuration!

For testing it can be helpful to monitor the following files (e.g. with watch)
to find out what is happening
  * /etc/dnsmasq.d/<zone_id>/ethers (on each node)
  * /etc/pve/priv/ipam.db
  * /etc/pve/priv/macs.db

Known Issues (working on fixes currently):
  * removing an entry with a duplicate MAC address removes all entries with
    MAC address
  * dnsmasq and IPv6 (and DHCP in general) do not really play well together,
    so using subnets with IPv6 configured is wonky
  * removing custom mappings fails due to a missing VMID
  * Changing IP addresses and MAC addresses can be buggy due to the way dnsmasq
    handles leases
  * DHCP DNS server setting is not available in the UI

Changes from v3 -> v4:
  * Improved API (permissions, validation, error handling)
  * UI polishing
  * added support for containers
  * countless small bug fixes

Changes from v2 -> v3:
  * Removed dhcp.cfg, DHCP server now get configured at the zone
  * added UI
  * added / updated API
  * DHCP acquires IPs at vNIC creation instead of VM start
  * DHCP releases IPs at vNIC removal instead of VM stop
  * improved dnsmasq configuration generation
  * added priv/macs.db for caching mac/IP mappings
  * refactored IPAM plugins
  * updated tests

Changes from v1 -> v2:
  * added hooks for handling DHCP when starting / stopping / .. VMs and CTs
  * Get an IP from IPAM and register that IP in the DHCP server
    (pve only for now)
  * remove lease-time, since it is now infinite and managed by the VM lifecycle
  * add hooks for setting & deleting DHCP mappings to DHCP plugins
  * modified interface of the abstract class to reflect new requirements
  * added helpers in existing SDN classes
  * simplified DHCP configuration settings



pve-cluster:

Alexandre Derumier (1):
  add priv/macs.db

 src/PVE/Cluster.pm  | 1 +
 src/pmxcfs/status.c | 1 +
 2 files changed, 2 insertions(+)


pve-network:

Alexandre Derumier (3):
  sdn: fix tests
  sdn: fix subnets && netbox ipam tests
  add add_dhcp_mapping

Stefan Hanreich (12):
  sdn: preparations for DHCP plugin
  subnet: add dhcp options
  sdn: zone: add dhcp option
  ipam: plugins: preparations for DHCP
  subnet: vnet: refactor IPAM related methods
  dhcp: add abstract class for DHCP plugins
  sdn: dhcp: add dnsmasq plugin
  sdn: dhcp: add helper for creating DHCP leases
  api: add endpoints for managing PVE IPAM
  api: subnet: add dhcp ranges
  api: zone: add dhcp option
  dhcp: regenerate config for DHCP plugins on applying configuration

 debian/control                                |   1 +
 src/PVE/API2/Network/SDN.pm                   |   6 +
 src/PVE/API2/Network/SDN/Ipam.pm              | 221 +++++++++++++++++
 src/PVE/API2/Network/SDN/Makefile             |   2 +-
 src/PVE/API2/Network/SDN/Subnets.pm           |   1 +
 src/PVE/API2/Network/SDN/Zones.pm             |   1 +
 src/PVE/Network/SDN.pm                        |   9 +-
 src/PVE/Network/SDN/Dhcp.pm                   | 118 +++++++++
 src/PVE/Network/SDN/Dhcp/Dnsmasq.pm           | 226 ++++++++++++++++++
 src/PVE/Network/SDN/Dhcp/Makefile             |   8 +
 src/PVE/Network/SDN/Dhcp/Plugin.pm            |  65 +++++
 src/PVE/Network/SDN/Ipams.pm                  |  80 ++++++-
 src/PVE/Network/SDN/Ipams/NetboxPlugin.pm     |  86 ++++++-
 src/PVE/Network/SDN/Ipams/PVEPlugin.pm        |  85 ++++++-
 src/PVE/Network/SDN/Ipams/PhpIpamPlugin.pm    |  29 +++
 src/PVE/Network/SDN/Ipams/Plugin.pm           |  19 +-
 src/PVE/Network/SDN/Makefile                  |   3 +-
 src/PVE/Network/SDN/SubnetPlugin.pm           |  32 ++-
 src/PVE/Network/SDN/Subnets.pm                |  98 +++++---
 src/PVE/Network/SDN/Vnets.pm                  | 147 ++++++++----
 src/PVE/Network/SDN/Zones.pm                  |  34 ++-
 src/PVE/Network/SDN/Zones/SimplePlugin.pm     |   7 +-
 src/test/ipams/netbox/expected.add_ip         |   2 +-
 .../ipams/netbox/expected.add_ip_notgateway   |   2 +-
 .../ipams/netbox/expected.add_next_freeip     |   2 +-
 src/test/ipams/netbox/expected.update_ip      |   2 +-
 src/test/run_test_subnets.pl                  |  16 +-
 src/test/run_test_vnets.pl                    |   4 +-
 28 files changed, 1181 insertions(+), 125 deletions(-)
 create mode 100644 src/PVE/API2/Network/SDN/Ipam.pm
 create mode 100644 src/PVE/Network/SDN/Dhcp.pm
 create mode 100644 src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
 create mode 100644 src/PVE/Network/SDN/Dhcp/Makefile
 create mode 100644 src/PVE/Network/SDN/Dhcp/Plugin.pm


pve-manager:

Stefan Hanreich (4):
  sdn: regenerate DHCP config on reload
  sdn: add DHCP option to Zone dialogue
  sdn: subnet: add panel for editing dhcp ranges
  sdn: ipam: add ipam panel

 PVE/API2/Network.pm                  |   1 +
 www/css/ext6-pve.css                 |  22 ++-
 www/manager6/Makefile                |   2 +
 www/manager6/dc/Config.js            |  12 +-
 www/manager6/sdn/IpamEdit.js         |  78 ++++++++
 www/manager6/sdn/SubnetEdit.js       | 160 +++++++++++++++-
 www/manager6/sdn/zones/Base.js       |   6 +-
 www/manager6/sdn/zones/SimpleEdit.js |  10 +
 www/manager6/tree/DhcpTree.js        | 267 +++++++++++++++++++++++++++
 9 files changed, 547 insertions(+), 11 deletions(-)
 create mode 100644 www/manager6/sdn/IpamEdit.js
 create mode 100644 www/manager6/tree/DhcpTree.js


qemu-server:

Alexandre Derumier (6):
  vmnic add|remove : add|del ip in ipam
  vm_start : vm-network-scripts: add_dhcp_reservation
  api2: create|restore|clone: add_free_ip
  vm_destroy: delete ip from ipam
  nic hotplug: add_dhcp_mapping
  nic online bridge/vlan change: link disconnect/reconnect

 PVE/API2/Qemu.pm              |  6 +++
 PVE/QemuServer.pm             | 86 +++++++++++++++++++++++++++++++++++
 vm-network-scripts/pve-bridge |  2 +
 3 files changed, 94 insertions(+)


pve-container:

Alexandre Derumier (6):
  nic hotplug : add|del ips in ipam
  vm_destroy: remove ips from ipam for all interfaces
  vm_create|restore: create ips in ipam
  vm_clone : create ips in ipams
  vm_apply_pending: add|del ips from ipam for offline changes
  lxc-pve-prestart-hook : add_dhcp_mapping

 src/PVE/API2/LXC.pm       | 10 ++++++++-
 src/PVE/LXC.pm            | 46 +++++++++++++++++++++++++++++++++++++++
 src/PVE/LXC/Config.pm     | 27 +++++++++++++++++++++++
 src/lxc-pve-prestart-hook | 15 +++++++++++++
 4 files changed, 97 insertions(+), 1 deletion(-)


pve-docs:

Stefan Hanreich (1):
  sdn: dhcp: Add documentation for DHCP

 pvesdn.adoc | 122 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 122 insertions(+)


Summary over all repositories:
  47 files changed, 2043 insertions(+), 137 deletions(-)

-- 
murpp v0.4.0




^ permalink raw reply	[flat|nested] 66+ messages in thread

end of thread, other threads:[~2023-11-22  8:06 UTC | newest]

Thread overview: 66+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-11-17 11:39 [pve-devel] [PATCH v4 cluster/network/manager/qemu-server/container/docs 00/33] Add support for DHCP servers to SDN Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-cluster 01/33] add priv/macs.db Stefan Hanreich
2023-11-17 13:54   ` [pve-devel] applied: " Thomas Lamprecht
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 02/33] sdn: preparations for DHCP plugin Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 03/33] subnet: add dhcp options Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 04/33] sdn: zone: add dhcp option Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 05/33] ipam: plugins: preparations for DHCP Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 06/33] subnet: vnet: refactor IPAM related methods Stefan Hanreich
2023-11-17 14:13   ` Stefan Lendl
2023-11-17 15:12     ` Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 07/33] dhcp: add abstract class for DHCP plugins Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 08/33] sdn: dhcp: add dnsmasq plugin Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 09/33] sdn: dhcp: add helper for creating DHCP leases Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 10/33] api: add endpoints for managing PVE IPAM Stefan Hanreich
2023-11-18 16:27   ` Thomas Lamprecht
2023-11-20 10:55     ` Stefan Hanreich
2023-11-20 12:28       ` DERUMIER, Alexandre
2023-11-20 12:34         ` Stefan Hanreich
2023-11-20 12:50           ` Stefan Hanreich
2023-11-20 16:25           ` DERUMIER, Alexandre
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 11/33] api: subnet: add dhcp ranges Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 12/33] api: zone: add dhcp option Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 13/33] dhcp: regenerate config for DHCP plugins on applying configuration Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 14/33] sdn: fix tests Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 15/33] sdn: fix subnets && netbox ipam tests Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-network 16/33] add add_dhcp_mapping Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-manager 17/33] sdn: regenerate DHCP config on reload Stefan Hanreich
2023-11-21 21:15   ` [pve-devel] applied: " Thomas Lamprecht
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-manager 18/33] sdn: add DHCP option to Zone dialogue Stefan Hanreich
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-manager 19/33] sdn: subnet: add panel for editing dhcp ranges Stefan Hanreich
2023-11-20 13:20   ` Dominik Csapak
2023-11-17 11:39 ` [pve-devel] [PATCH v4 pve-manager 20/33] sdn: ipam: add ipam panel Stefan Hanreich
2023-11-17 15:04   ` DERUMIER, Alexandre
2023-11-17 15:15     ` Stefan Hanreich
2023-11-18 14:25       ` DERUMIER, Alexandre
2023-11-20 13:44   ` Dominik Csapak
2023-11-17 11:39 ` [pve-devel] [PATCH v4 qemu-server 21/33] vmnic add|remove : add|del ip in ipam Stefan Hanreich
2023-11-21 13:53   ` [pve-devel] applied-series: " Wolfgang Bumiller
2023-11-17 11:40 ` [pve-devel] [PATCH v4 qemu-server 22/33] vm_start : vm-network-scripts: add_dhcp_reservation Stefan Hanreich
2023-11-17 11:40 ` [pve-devel] [PATCH v4 qemu-server 23/33] api2: create|restore|clone: add_free_ip Stefan Hanreich
2023-11-17 11:40 ` [pve-devel] [PATCH v4 qemu-server 24/33] vm_destroy: delete ip from ipam Stefan Hanreich
2023-11-17 11:40 ` [pve-devel] [PATCH v4 qemu-server 25/33] nic hotplug: add_dhcp_mapping Stefan Hanreich
2023-11-17 11:40 ` [pve-devel] [PATCH v4 qemu-server 26/33] nic online bridge/vlan change: link disconnect/reconnect Stefan Hanreich
2023-11-17 11:40 ` [pve-devel] [PATCH v4 pve-container 27/33] nic hotplug : add|del ips in ipam Stefan Hanreich
2023-11-21 13:47   ` [pve-devel] applied-series: " Wolfgang Bumiller
2023-11-17 11:40 ` [pve-devel] [PATCH v4 pve-container 28/33] vm_destroy: remove ips from ipam for all interfaces Stefan Hanreich
2023-11-17 11:40 ` [pve-devel] [PATCH v4 pve-container 29/33] vm_create|restore: create ips in ipam Stefan Hanreich
2023-11-17 11:40 ` [pve-devel] [PATCH v4 pve-container 30/33] vm_clone : create ips in ipams Stefan Hanreich
2023-11-17 11:40 ` [pve-devel] [PATCH v4 pve-container 31/33] vm_apply_pending: add|del ips from ipam for offline changes Stefan Hanreich
2023-11-17 11:40 ` [pve-devel] [PATCH v4 pve-container 32/33] lxc-pve-prestart-hook : add_dhcp_mapping Stefan Hanreich
2023-11-17 11:40 ` [pve-devel] [PATCH v4 pve-docs 33/33] sdn: dhcp: Add documentation for DHCP Stefan Hanreich
2023-11-21 19:03   ` [pve-devel] applied: " Thomas Lamprecht
2023-11-17 15:47 ` [pve-devel] [PATCH v4 cluster/network/manager/qemu-server/container/docs 00/33] Add support for DHCP servers to SDN DERUMIER, Alexandre
2023-11-17 16:05   ` Stefan Hanreich
2023-11-17 16:07     ` Stefan Hanreich
2023-11-17 16:09     ` DERUMIER, Alexandre
2023-11-17 20:44       ` DERUMIER, Alexandre
2023-11-21 11:23   ` Stefan Lendl
2023-11-21 13:02     ` DERUMIER, Alexandre
2023-11-21 13:25     ` DERUMIER, Alexandre
2023-11-21 13:28     ` DERUMIER, Alexandre
2023-11-21 16:34       ` Stefan Lendl
2023-11-21 18:15         ` DERUMIER, Alexandre
2023-11-22  8:06         ` DERUMIER, Alexandre
2023-11-18 14:38 ` DERUMIER, Alexandre
2023-11-20 16:42 ` Thomas Lamprecht

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal