[pve-devel] [PATCH v4 cluster/network/manager/qemu-server/container/docs 00/33] Add support for DHCP servers to SDN

[Stefan Hanreich <s.hanreich@proxmox.com>]

This patch series adds support for automatically deploying dnsmasq as a DHCP
server to a simple SDN Zone. This series is now in a state where I think it is
okay to integrate into the respective repositories **except** for pve-container

You can (and should) check out the documentation, but here is a quick setup
guide for your convenienve:

You need to install dnsmasq (and disable it afterwards):

  apt install dnsmasq && systemctl disable --now dnsmasq


You can use the following example configuration for deploying a DHCP server in
a SDN subnet, you should also be able to recreate this configuration in the
web UI:


/etc/pve/sdn/zones.cfg:

  simple: DHCPNAT
          ipam pve
          dhcp dnsmasq


/etc/pve/sdn/vnets.cfg:

  vnet: dhcpnat
          zone DHCPNAT


/etc/pve/sdn/subnets.cfg:

  subnet: DHCPNAT-10.1.0.0-16
          vnet dhcpnat
          dhcp-dns-server 10.1.0.1
          dhcp-range start-address=10.1.0.100,end-address=10.1.0.200
          gateway 10.1.0.1
          snat 1

Don't forget to apply the new configuration!

For testing it can be helpful to monitor the following files (e.g. with watch)
to find out what is happening
  * /etc/dnsmasq.d/<zone_id>/ethers (on each node)
  * /etc/pve/priv/ipam.db
  * /etc/pve/priv/macs.db

Known Issues (working on fixes currently):
  * removing an entry with a duplicate MAC address removes all entries with
    MAC address
  * dnsmasq and IPv6 (and DHCP in general) do not really play well together,
    so using subnets with IPv6 configured is wonky
  * removing custom mappings fails due to a missing VMID
  * Changing IP addresses and MAC addresses can be buggy due to the way dnsmasq
    handles leases
  * DHCP DNS server setting is not available in the UI

Changes from v3 -> v4:
  * Improved API (permissions, validation, error handling)
  * UI polishing
  * added support for containers
  * countless small bug fixes

Changes from v2 -> v3:
  * Removed dhcp.cfg, DHCP server now get configured at the zone
  * added UI
  * added / updated API
  * DHCP acquires IPs at vNIC creation instead of VM start
  * DHCP releases IPs at vNIC removal instead of VM stop
  * improved dnsmasq configuration generation
  * added priv/macs.db for caching mac/IP mappings
  * refactored IPAM plugins
  * updated tests

Changes from v1 -> v2:
  * added hooks for handling DHCP when starting / stopping / .. VMs and CTs
  * Get an IP from IPAM and register that IP in the DHCP server
    (pve only for now)
  * remove lease-time, since it is now infinite and managed by the VM lifecycle
  * add hooks for setting & deleting DHCP mappings to DHCP plugins
  * modified interface of the abstract class to reflect new requirements
  * added helpers in existing SDN classes
  * simplified DHCP configuration settings



pve-cluster:

Alexandre Derumier (1):
  add priv/macs.db

 src/PVE/Cluster.pm  | 1 +
 src/pmxcfs/status.c | 1 +
 2 files changed, 2 insertions(+)


pve-network:

Alexandre Derumier (3):
  sdn: fix tests
  sdn: fix subnets && netbox ipam tests
  add add_dhcp_mapping

Stefan Hanreich (12):
  sdn: preparations for DHCP plugin
  subnet: add dhcp options
  sdn: zone: add dhcp option
  ipam: plugins: preparations for DHCP
  subnet: vnet: refactor IPAM related methods
  dhcp: add abstract class for DHCP plugins
  sdn: dhcp: add dnsmasq plugin
  sdn: dhcp: add helper for creating DHCP leases
  api: add endpoints for managing PVE IPAM
  api: subnet: add dhcp ranges
  api: zone: add dhcp option
  dhcp: regenerate config for DHCP plugins on applying configuration

 debian/control                                |   1 +
 src/PVE/API2/Network/SDN.pm                   |   6 +
 src/PVE/API2/Network/SDN/Ipam.pm              | 221 +++++++++++++++++
 src/PVE/API2/Network/SDN/Makefile             |   2 +-
 src/PVE/API2/Network/SDN/Subnets.pm           |   1 +
 src/PVE/API2/Network/SDN/Zones.pm             |   1 +
 src/PVE/Network/SDN.pm                        |   9 +-
 src/PVE/Network/SDN/Dhcp.pm                   | 118 +++++++++
 src/PVE/Network/SDN/Dhcp/Dnsmasq.pm           | 226 ++++++++++++++++++
 src/PVE/Network/SDN/Dhcp/Makefile             |   8 +
 src/PVE/Network/SDN/Dhcp/Plugin.pm            |  65 +++++
 src/PVE/Network/SDN/Ipams.pm                  |  80 ++++++-
 src/PVE/Network/SDN/Ipams/NetboxPlugin.pm     |  86 ++++++-
 src/PVE/Network/SDN/Ipams/PVEPlugin.pm        |  85 ++++++-
 src/PVE/Network/SDN/Ipams/PhpIpamPlugin.pm    |  29 +++
 src/PVE/Network/SDN/Ipams/Plugin.pm           |  19 +-
 src/PVE/Network/SDN/Makefile                  |   3 +-
 src/PVE/Network/SDN/SubnetPlugin.pm           |  32 ++-
 src/PVE/Network/SDN/Subnets.pm                |  98 +++++---
 src/PVE/Network/SDN/Vnets.pm                  | 147 ++++++++----
 src/PVE/Network/SDN/Zones.pm                  |  34 ++-
 src/PVE/Network/SDN/Zones/SimplePlugin.pm     |   7 +-
 src/test/ipams/netbox/expected.add_ip         |   2 +-
 .../ipams/netbox/expected.add_ip_notgateway   |   2 +-
 .../ipams/netbox/expected.add_next_freeip     |   2 +-
 src/test/ipams/netbox/expected.update_ip      |   2 +-
 src/test/run_test_subnets.pl                  |  16 +-
 src/test/run_test_vnets.pl                    |   4 +-
 28 files changed, 1181 insertions(+), 125 deletions(-)
 create mode 100644 src/PVE/API2/Network/SDN/Ipam.pm
 create mode 100644 src/PVE/Network/SDN/Dhcp.pm
 create mode 100644 src/PVE/Network/SDN/Dhcp/Dnsmasq.pm
 create mode 100644 src/PVE/Network/SDN/Dhcp/Makefile
 create mode 100644 src/PVE/Network/SDN/Dhcp/Plugin.pm


pve-manager:

Stefan Hanreich (4):
  sdn: regenerate DHCP config on reload
  sdn: add DHCP option to Zone dialogue
  sdn: subnet: add panel for editing dhcp ranges
  sdn: ipam: add ipam panel

 PVE/API2/Network.pm                  |   1 +
 www/css/ext6-pve.css                 |  22 ++-
 www/manager6/Makefile                |   2 +
 www/manager6/dc/Config.js            |  12 +-
 www/manager6/sdn/IpamEdit.js         |  78 ++++++++
 www/manager6/sdn/SubnetEdit.js       | 160 +++++++++++++++-
 www/manager6/sdn/zones/Base.js       |   6 +-
 www/manager6/sdn/zones/SimpleEdit.js |  10 +
 www/manager6/tree/DhcpTree.js        | 267 +++++++++++++++++++++++++++
 9 files changed, 547 insertions(+), 11 deletions(-)
 create mode 100644 www/manager6/sdn/IpamEdit.js
 create mode 100644 www/manager6/tree/DhcpTree.js


qemu-server:

Alexandre Derumier (6):
  vmnic add|remove : add|del ip in ipam
  vm_start : vm-network-scripts: add_dhcp_reservation
  api2: create|restore|clone: add_free_ip
  vm_destroy: delete ip from ipam
  nic hotplug: add_dhcp_mapping
  nic online bridge/vlan change: link disconnect/reconnect

 PVE/API2/Qemu.pm              |  6 +++
 PVE/QemuServer.pm             | 86 +++++++++++++++++++++++++++++++++++
 vm-network-scripts/pve-bridge |  2 +
 3 files changed, 94 insertions(+)


pve-container:

Alexandre Derumier (6):
  nic hotplug : add|del ips in ipam
  vm_destroy: remove ips from ipam for all interfaces
  vm_create|restore: create ips in ipam
  vm_clone : create ips in ipams
  vm_apply_pending: add|del ips from ipam for offline changes
  lxc-pve-prestart-hook : add_dhcp_mapping

 src/PVE/API2/LXC.pm       | 10 ++++++++-
 src/PVE/LXC.pm            | 46 +++++++++++++++++++++++++++++++++++++++
 src/PVE/LXC/Config.pm     | 27 +++++++++++++++++++++++
 src/lxc-pve-prestart-hook | 15 +++++++++++++
 4 files changed, 97 insertions(+), 1 deletion(-)


pve-docs:

Stefan Hanreich (1):
  sdn: dhcp: Add documentation for DHCP

 pvesdn.adoc | 122 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 122 insertions(+)


Summary over all repositories:
  47 files changed, 2043 insertions(+), 137 deletions(-)

-- 
murpp v0.4.0