From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 555DD1FF13C for ; Thu, 02 Apr 2026 10:16:13 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id ABF7BE60F; Thu, 2 Apr 2026 10:16:42 +0200 (CEST) Message-ID: Date: Thu, 2 Apr 2026 10:16:35 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: superseded: [RFC manager/network/proxmox{,-ve-rs,-perl-rs} 00/27] Add WireGuard as protocol to SDN fabrics To: pve-devel@lists.proxmox.com References: <20260219145649.441418-1-s.hanreich@proxmox.com> Content-Language: en-US From: Stefan Hanreich In-Reply-To: <20260219145649.441418-1-s.hanreich@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.804 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_SHORT 0.001 Use of a URL Shortener for very short URL RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 1 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 1 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 1 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 6HSW3WJXB5NZOFSFNGCRTOPJLWY2ZXSD X-Message-ID-Hash: 6HSW3WJXB5NZOFSFNGCRTOPJLWY2ZXSD X-MailFrom: s.hanreich@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: https://lore.proxmox.com/pve-devel/20260402081148.76276-1-s.hanreich@proxmox.com/T/#t On 2/19/26 3:56 PM, Stefan Hanreich wrote: > This patch series builds upon the patch series from Christoph, that introduces > the proxmox-wireguard crate [1]. Sending this as an RFC to gather feedback, > particularly on the configuration format as well as the key generation logic. > > > ## Introduction > > This patch series introduces WireGuard as fabric protocol. Potential use-cases > include: > > * Connecting to remote PBS / PDM instances > * Simple encryption layer for intra-DC VXLAN tunnels > * Secure migration network > * Connecting with remote PVE clusters > > It utilizes the wg(8) tool for generating the interface configuration [2] and > the section config format leans heavily into the keys defined there. > > > ## Configuration format > > The configuration format is quite similar to OSPF and Openfabric with the main > difference being that WireGuard nodes have been split into two subtypes > (external and internal), in order to support nodes that are not part of the > cluster. > > ### Nodes > > WireGuard nodes have been split into two different types. Those are not distinct > section config types, due to how the internal representation of the FabricConfig > has been structured (which maps exactly one Fabric type to one Node type). So > instead there is one Node type that is an enum. The 'role' field is used for > distinguishing between different WireGuard node types. > > #### Internal > > This represents a node that is part of the Proxmox VE cluster. > > An example configuration looks like this: > > wireguard_node: vpn_elementalist > endpoint 192.0.2.1 > allowed_ips 203.0.113.128/25 > interfaces name=wg0,listen_port=50000,public_key=O+Kzrochm6klMILjSKVw83xb3YyXXLpmZj9n/ICM5xE=,ip=198.51.100.1/24 > role internal > > The endpoint value will be used by other nodes inside the Proxmox VE cluster for > connecting to the defined node. IPs are defined on a per-interface basis, not a > per-node basis. The interface key represents the [Interface] section in the > WireGuard configuration. All values (except for public key) are overridable in > the peer definition. > > #### External > > External nodes represent any peer that is not a Proxmox VE node. They provide a > mechanism for defining a reusable peer definition (see below for more details). > > This allows for easily re-using and updating the information of an external > peer, without having to re-type all information for every Proxmox VE node that > wants to utilize the definition. > > An example configuration looks like this: > > wireguard_node: vpn_berserker > endpoint berserker:51337 > allowed_ips 203.0.113.0/25 > public_key GDPUAnPOY5xGIjYXmcGyXZXbocjBr21dGQ5vwnjmdzA= > role external > > Those keys map 1:1 to the peer entries in the respective WireGuard configuration > format and are used for generating the peer definition wherever they are > referenced. > > ### Peers > > Interfaces on Proxmox nodes can have one or more peers. A peer is a reference to > either the interface of an internal node, or an external node. Due to > limitations in dealing with nested data in the section config, peers are an > array field in the node, instead of being configured on the interface directly. > > An example configuration for a Proxmox VE node with an interface that has an > internal and external node as peer looks as follows: > > wireguard_node: vpn_occultist > endpoint 192.0.2.2 > interfaces name=wg0,listen_port=50000,public_key=y0kOpXfo9ff4KoUwO3H1cRuwObbKwsK8mAkwXxNvKUc=,ip=198.51.100.2/24 > peers type=internal,node=elementalist,node_iface=wg0,iface=wg0 > peers type=external,node=berserker,iface=wg0 > role internal > > This would generate the following wg0.conf file: > > [Interface] > PrivateKey = > ListenPort = 50000 > > [Peer] > PublicKey = O+Kzrochm6klMILjSKVw83xb3YyXXLpmZj9n/ICM5xE= > AllowedIPs = 198.51.100.1/32 > Endpoint = 192.0.2.1:50000 > AllowedIPs = 203.0.113.128/25 > > [Peer] > PublicKey = GDPUAnPOY5xGIjYXmcGyXZXbocjBr21dGQ5vwnjmdzA= > Endpoint = berserker:51337 > AllowedIPs = 203.0.113.0/25 > > > Peer definitions allow overriding properties from the node definition (e.g. > endpoint). This is currently not implemented in the frontend. This is also the > main reason for choosing to store peers as an array in a different key. > Referencing peer defintions by id would have been possible in the interface > property string, but if the possibility of overriding certain attributes should > be available, then a separate key with property strings is required. > > > ## Key handling > > Keys are automatically generated in the backend on demand, whenever a interface > is created / updated or deleted. This is accomplished by proxying API calls to > the respective node and generating / deleting the WireGuard keys there. After a > key has been generated, the respective public key gets stored in the section > config. The main reason for this is that it allows returning the public key from > the API without having to read one configuration file. This is particularly > relevant when implementing support for establishing cross-cluster communication > in PDM, which requires the information for creating the respective peer > definitions on each cluster. > > All WireGuard keys and configuration files are stored locally on the node in the > newly established '/etc/wireguard/proxmox' folder, and managed by the node > itself. Private keys are never transmitted across the wire and are also not > shared via pmxcfs. A WireGuard interface configuration usually requires 3 files: > public/private key and a wireguard configuration file. > > > ## Open questions / issues > > ### Peers > > The main issue I see with the configuration format is that peers reference > arbitrary node sections / interface definitions in the fabric config. This poses > some problems, particularly when updating the referenced entities. For instance, > users could delete a referenced interface, invalidating the configuration. This > is quite similar to the problems we currently encounter with firewall ipsets and > aliases. > > In order to avoid re-creating the same issues there are a few restrictions in > the UI that should prevent the most common mistakes: > > * Renaming nodes and interfaces is not allowed. > * The configuration is validated after every modification and invalid > configurations are outright rejected. This is particularly important for > delete operations. > > In the future we could lift some restrictions by implementing smarter CRUD > operations. For instance, when deleting an interface all peer entries, that > reference that interface, could be deleted as well. Even for accidental > deletions this isn't too bad imo, since we have a mechanism of restoring the > current running configuration, which users can always use. > > For updates to the interfaces of a node this is harder, since it is impossible > to say whether an interface has been renamed or an interface has been deleted > and another one created. I don't really see a good heuristic (even when tracking > this in the UI) that works particularly well for all potential cases. > > ### Section Types > > The split of one section type ('wireguard_node') into two different subtypes is > breaking a bit with section config principles. Another solution would be to > introduce two section config types (e.g. wireguard_node_{external,internal}), > although that would require quite some refactoring effort. > > > ## Todo: > > Since this is an RFC, there are still some unpolished / rough edges that will be > improved until a v1: > > * currently there is no documentation > * add more integration tests > * potentially expose the overridable properties in the UI > * implement deletion of keys when deleting the whole fabric (bit tricky with how > keygen is currently handled). > > ## Future work > > * implement status reporting > * implement ECMP for routing the same subnet via multiple wg interfaces > * provide QoL features for easier config (e.g. auto-"fullmeshify" PVE cluster) > * Implement some backend-only features in the UI (e.g. per-peer overrides, > pre-shared keys) > * Integration into PDM / PBS > > [1] https://lore.proxmox.com/all/20260213143601.1424613-1-c.heiss@proxmox.com/ > [2] https://man7.org/linux/man-pages/man8/wg.8.html > > proxmox: > > Stefan Hanreich (2): > wireguard: skip serializing preshared_key if unset > wireguard: implement ApiType for endpoints and hostnames > > proxmox-network-types/src/endpoint.rs | 30 +++++++++++++++++++++++++-- > proxmox-wireguard/src/lib.rs | 1 + > 2 files changed, 29 insertions(+), 2 deletions(-) > > > proxmox-ve-rs: > > Christoph Heiss (2): > sdn-types: add wireguard-specific PersistentKeepalive api type > ve-config: fabric: refactor fabric config entry impl using macro > > Stefan Hanreich (7): > debian: update control file > clippy: fix 'hiding a lifetime that's elided elsewhere is confusing' > ve-config: fabrics: split interface name regex into two parts > ve-config: fabrics: add protocol-specific properties for wireguard > ve-config: sdn: fabrics: add wireguard to the fabric config > ve-config: fabrics: wireguard add validation for wireguard config > ve-config: fabrics: implement wireguard config generation > > proxmox-sdn-types/debian/control | 3 +- > proxmox-sdn-types/src/lib.rs | 1 + > proxmox-sdn-types/src/wireguard.rs | 43 ++ > proxmox-ve-config/Cargo.toml | 3 + > proxmox-ve-config/debian/control | 6 + > proxmox-ve-config/src/firewall/types/ipset.rs | 2 +- > proxmox-ve-config/src/sdn/fabric/frr.rs | 1 + > proxmox-ve-config/src/sdn/fabric/mod.rs | 399 ++++++++++--- > .../src/sdn/fabric/section_config/fabric.rs | 25 + > .../sdn/fabric/section_config/interface.rs | 5 +- > .../src/sdn/fabric/section_config/mod.rs | 58 ++ > .../src/sdn/fabric/section_config/node.rs | 44 +- > .../sdn/fabric/section_config/protocol/mod.rs | 1 + > .../section_config/protocol/wireguard.rs | 546 ++++++++++++++++++ > proxmox-ve-config/src/sdn/mod.rs | 1 + > proxmox-ve-config/src/sdn/wireguard.rs | 311 ++++++++++ > 16 files changed, 1375 insertions(+), 74 deletions(-) > create mode 100644 proxmox-sdn-types/src/wireguard.rs > create mode 100644 proxmox-ve-config/src/sdn/fabric/section_config/protocol/wireguard.rs > create mode 100644 proxmox-ve-config/src/sdn/wireguard.rs > > > proxmox-perl-rs: > > Christoph Heiss (1): > pve-rs: fabrics: wireguard: generate ifupdown2 configuration > > Stefan Hanreich (1): > pve-rs: fabrics: add helpers for parsing interface property strings > > pve-rs/Cargo.toml | 1 + > pve-rs/src/bindings/sdn/fabrics.rs | 215 +++++++++++++++++++++++++---- > pve-rs/src/sdn/status.rs | 16 +++ > 3 files changed, 203 insertions(+), 29 deletions(-) > > > pve-network: > > Christoph Heiss (1): > sdn: add wireguard helper module > > Stefan Hanreich (2): > fabrics: wireguard: add schema definitions for wireguard > fabrics: wireguard: implement wireguard key auto-generation > > src/PVE/API2/Network/SDN.pm | 2 +- > .../API2/Network/SDN/Fabrics/FabricNode.pm | 129 ++++++++- > src/PVE/Network/SDN.pm | 9 +- > src/PVE/Network/SDN/Fabrics.pm | 257 +++++++++++++++++- > src/PVE/Network/SDN/Makefile | 15 +- > src/PVE/Network/SDN/WireGuard.pm | 163 +++++++++++ > 6 files changed, 560 insertions(+), 15 deletions(-) > create mode 100644 src/PVE/Network/SDN/WireGuard.pm > > > pve-manager: > > Christoph Heiss (2): > ui: fabrics: edit: make ipv4/6 support generic over fabric panels > ui: fabrics: interface: make ipv4/6 support generic over edit panels > > Stefan Hanreich (9): > network: sdn: generate wireguard configuration on apply > ui: fix parsing of property-strings when values contain = > ui: fabrics: i18n: make node loading string translatable > ui: fabrics: split node selector creation and config > ui: fabrics: node: make ipv4/6 support generic over edit panels > ui: fabrics: wireguard: add interface edit panel > ui: fabrics: wireguard: add node edit panel > ui: fabrics: wireguard: add fabric edit panel > ui: fabrics: hook up wireguard components > > PVE/API2/Network.pm | 1 + > www/manager6/Makefile | 3 + > www/manager6/Parser.js | 7 +- > www/manager6/sdn/FabricsView.js | 12 + > www/manager6/sdn/fabrics/FabricEdit.js | 68 ++- > www/manager6/sdn/fabrics/InterfacePanel.js | 18 + > www/manager6/sdn/fabrics/NodeEdit.js | 64 ++- > .../sdn/fabrics/openfabric/FabricEdit.js | 32 -- > .../sdn/fabrics/openfabric/InterfacePanel.js | 13 - > .../sdn/fabrics/openfabric/NodeEdit.js | 14 - > www/manager6/sdn/fabrics/ospf/FabricEdit.js | 2 + > .../sdn/fabrics/ospf/InterfacePanel.js | 2 + > www/manager6/sdn/fabrics/ospf/NodeEdit.js | 2 + > .../sdn/fabrics/wireguard/FabricEdit.js | 29 ++ > .../sdn/fabrics/wireguard/InterfacePanel.js | 427 ++++++++++++++++++ > .../sdn/fabrics/wireguard/NodeEdit.js | 230 ++++++++++ > 16 files changed, 835 insertions(+), 89 deletions(-) > create mode 100644 www/manager6/sdn/fabrics/wireguard/FabricEdit.js > create mode 100644 www/manager6/sdn/fabrics/wireguard/InterfacePanel.js > create mode 100644 www/manager6/sdn/fabrics/wireguard/NodeEdit.js > > > Summary over all repositories: > 43 files changed, 3002 insertions(+), 209 deletions(-) >