From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id C31D69A22D for ; Fri, 17 Nov 2023 12:40:20 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 47F1F30B70 for ; Fri, 17 Nov 2023 12:40:20 +0100 (CET) Received: from lana.proxmox.com (unknown [94.136.29.99]) by firstgate.proxmox.com (Proxmox) with ESMTP for ; Fri, 17 Nov 2023 12:40:18 +0100 (CET) Received: by lana.proxmox.com (Postfix, from userid 10043) id 0317C2C20B9; Fri, 17 Nov 2023 12:40:17 +0100 (CET) From: Stefan Hanreich To: pve-devel@lists.proxmox.com Date: Fri, 17 Nov 2023 12:39:38 +0100 Message-Id: <20231117114011.834002-1-s.hanreich@proxmox.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.509 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record T_SCC_BODY_TEXT_LINE -0.01 - URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [plugin.pm, qemuserver.pm, dhcp.pm, qemu.pm, cluster.pm, netboxplugin.pm, vnets.pm, zones.pm, phpipamplugin.pm, ipam.pm, sdn.pm, ipams.pm, network.pm, lxc.pm, pveplugin.pm, dnsmasq.pm, simpleplugin.pm, subnetplugin.pm, subnets.pm, config.pm] Subject: [pve-devel] [PATCH v4 cluster/network/manager/qemu-server/container/docs 00/33] Add support for DHCP servers to SDN X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Nov 2023 11:40:20 -0000 This patch series adds support for automatically deploying dnsmasq as a DHCP server to a simple SDN Zone. This series is now in a state where I think it is okay to integrate into the respective repositories **except** for pve-container You can (and should) check out the documentation, but here is a quick setup guide for your convenienve: You need to install dnsmasq (and disable it afterwards): apt install dnsmasq && systemctl disable --now dnsmasq You can use the following example configuration for deploying a DHCP server in a SDN subnet, you should also be able to recreate this configuration in the web UI: /etc/pve/sdn/zones.cfg: simple: DHCPNAT ipam pve dhcp dnsmasq /etc/pve/sdn/vnets.cfg: vnet: dhcpnat zone DHCPNAT /etc/pve/sdn/subnets.cfg: subnet: DHCPNAT-10.1.0.0-16 vnet dhcpnat dhcp-dns-server 10.1.0.1 dhcp-range start-address=10.1.0.100,end-address=10.1.0.200 gateway 10.1.0.1 snat 1 Don't forget to apply the new configuration! For testing it can be helpful to monitor the following files (e.g. with watch) to find out what is happening * /etc/dnsmasq.d//ethers (on each node) * /etc/pve/priv/ipam.db * /etc/pve/priv/macs.db Known Issues (working on fixes currently): * removing an entry with a duplicate MAC address removes all entries with MAC address * dnsmasq and IPv6 (and DHCP in general) do not really play well together, so using subnets with IPv6 configured is wonky * removing custom mappings fails due to a missing VMID * Changing IP addresses and MAC addresses can be buggy due to the way dnsmasq handles leases * DHCP DNS server setting is not available in the UI Changes from v3 -> v4: * Improved API (permissions, validation, error handling) * UI polishing * added support for containers * countless small bug fixes Changes from v2 -> v3: * Removed dhcp.cfg, DHCP server now get configured at the zone * added UI * added / updated API * DHCP acquires IPs at vNIC creation instead of VM start * DHCP releases IPs at vNIC removal instead of VM stop * improved dnsmasq configuration generation * added priv/macs.db for caching mac/IP mappings * refactored IPAM plugins * updated tests Changes from v1 -> v2: * added hooks for handling DHCP when starting / stopping / .. VMs and CTs * Get an IP from IPAM and register that IP in the DHCP server (pve only for now) * remove lease-time, since it is now infinite and managed by the VM lifecycle * add hooks for setting & deleting DHCP mappings to DHCP plugins * modified interface of the abstract class to reflect new requirements * added helpers in existing SDN classes * simplified DHCP configuration settings pve-cluster: Alexandre Derumier (1): add priv/macs.db src/PVE/Cluster.pm | 1 + src/pmxcfs/status.c | 1 + 2 files changed, 2 insertions(+) pve-network: Alexandre Derumier (3): sdn: fix tests sdn: fix subnets && netbox ipam tests add add_dhcp_mapping Stefan Hanreich (12): sdn: preparations for DHCP plugin subnet: add dhcp options sdn: zone: add dhcp option ipam: plugins: preparations for DHCP subnet: vnet: refactor IPAM related methods dhcp: add abstract class for DHCP plugins sdn: dhcp: add dnsmasq plugin sdn: dhcp: add helper for creating DHCP leases api: add endpoints for managing PVE IPAM api: subnet: add dhcp ranges api: zone: add dhcp option dhcp: regenerate config for DHCP plugins on applying configuration debian/control | 1 + src/PVE/API2/Network/SDN.pm | 6 + src/PVE/API2/Network/SDN/Ipam.pm | 221 +++++++++++++++++ src/PVE/API2/Network/SDN/Makefile | 2 +- src/PVE/API2/Network/SDN/Subnets.pm | 1 + src/PVE/API2/Network/SDN/Zones.pm | 1 + src/PVE/Network/SDN.pm | 9 +- src/PVE/Network/SDN/Dhcp.pm | 118 +++++++++ src/PVE/Network/SDN/Dhcp/Dnsmasq.pm | 226 ++++++++++++++++++ src/PVE/Network/SDN/Dhcp/Makefile | 8 + src/PVE/Network/SDN/Dhcp/Plugin.pm | 65 +++++ src/PVE/Network/SDN/Ipams.pm | 80 ++++++- src/PVE/Network/SDN/Ipams/NetboxPlugin.pm | 86 ++++++- src/PVE/Network/SDN/Ipams/PVEPlugin.pm | 85 ++++++- src/PVE/Network/SDN/Ipams/PhpIpamPlugin.pm | 29 +++ src/PVE/Network/SDN/Ipams/Plugin.pm | 19 +- src/PVE/Network/SDN/Makefile | 3 +- src/PVE/Network/SDN/SubnetPlugin.pm | 32 ++- src/PVE/Network/SDN/Subnets.pm | 98 +++++--- src/PVE/Network/SDN/Vnets.pm | 147 ++++++++---- src/PVE/Network/SDN/Zones.pm | 34 ++- src/PVE/Network/SDN/Zones/SimplePlugin.pm | 7 +- src/test/ipams/netbox/expected.add_ip | 2 +- .../ipams/netbox/expected.add_ip_notgateway | 2 +- .../ipams/netbox/expected.add_next_freeip | 2 +- src/test/ipams/netbox/expected.update_ip | 2 +- src/test/run_test_subnets.pl | 16 +- src/test/run_test_vnets.pl | 4 +- 28 files changed, 1181 insertions(+), 125 deletions(-) create mode 100644 src/PVE/API2/Network/SDN/Ipam.pm create mode 100644 src/PVE/Network/SDN/Dhcp.pm create mode 100644 src/PVE/Network/SDN/Dhcp/Dnsmasq.pm create mode 100644 src/PVE/Network/SDN/Dhcp/Makefile create mode 100644 src/PVE/Network/SDN/Dhcp/Plugin.pm pve-manager: Stefan Hanreich (4): sdn: regenerate DHCP config on reload sdn: add DHCP option to Zone dialogue sdn: subnet: add panel for editing dhcp ranges sdn: ipam: add ipam panel PVE/API2/Network.pm | 1 + www/css/ext6-pve.css | 22 ++- www/manager6/Makefile | 2 + www/manager6/dc/Config.js | 12 +- www/manager6/sdn/IpamEdit.js | 78 ++++++++ www/manager6/sdn/SubnetEdit.js | 160 +++++++++++++++- www/manager6/sdn/zones/Base.js | 6 +- www/manager6/sdn/zones/SimpleEdit.js | 10 + www/manager6/tree/DhcpTree.js | 267 +++++++++++++++++++++++++++ 9 files changed, 547 insertions(+), 11 deletions(-) create mode 100644 www/manager6/sdn/IpamEdit.js create mode 100644 www/manager6/tree/DhcpTree.js qemu-server: Alexandre Derumier (6): vmnic add|remove : add|del ip in ipam vm_start : vm-network-scripts: add_dhcp_reservation api2: create|restore|clone: add_free_ip vm_destroy: delete ip from ipam nic hotplug: add_dhcp_mapping nic online bridge/vlan change: link disconnect/reconnect PVE/API2/Qemu.pm | 6 +++ PVE/QemuServer.pm | 86 +++++++++++++++++++++++++++++++++++ vm-network-scripts/pve-bridge | 2 + 3 files changed, 94 insertions(+) pve-container: Alexandre Derumier (6): nic hotplug : add|del ips in ipam vm_destroy: remove ips from ipam for all interfaces vm_create|restore: create ips in ipam vm_clone : create ips in ipams vm_apply_pending: add|del ips from ipam for offline changes lxc-pve-prestart-hook : add_dhcp_mapping src/PVE/API2/LXC.pm | 10 ++++++++- src/PVE/LXC.pm | 46 +++++++++++++++++++++++++++++++++++++++ src/PVE/LXC/Config.pm | 27 +++++++++++++++++++++++ src/lxc-pve-prestart-hook | 15 +++++++++++++ 4 files changed, 97 insertions(+), 1 deletion(-) pve-docs: Stefan Hanreich (1): sdn: dhcp: Add documentation for DHCP pvesdn.adoc | 122 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) Summary over all repositories: 47 files changed, 2043 insertions(+), 137 deletions(-) -- murpp v0.4.0