From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 3DB241FF146 for ; Tue, 12 May 2026 19:38:47 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 56A3D1E208; Tue, 12 May 2026 19:38:46 +0200 (CEST) Message-ID: <5db8f6f7-fc06-4476-9712-49a14e0aaf19@proxmox.com> Date: Tue, 12 May 2026 19:38:34 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: superseded: [PATCH cluster/manager/network/proxmox{,-ve-rs,-perl-rs} v4 00/31] Add WireGuard as protocol to SDN fabrics To: pve-devel@lists.proxmox.com References: <20260507124008.417223-1-s.hanreich@proxmox.com> Content-Language: en-US From: Stefan Hanreich In-Reply-To: <20260507124008.417223-1-s.hanreich@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.612 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_SHORT 0.001 Use of a URL Shortener for very short URL SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com,status.rs,sdn.pm,lib.rs,mod.rs,fabric.rs,frr.rs,interface.rs,wireguard.pm,cluster.pm,fabrics.rs,fabricnode.pm,fabrics.pm,node.rs,man7.org,network.pm,endpoint.rs,wireguard.rs] Message-ID-Hash: IRYJHZY74JYUX4NEEFS766F4TGGBZ6NN X-Message-ID-Hash: IRYJHZY74JYUX4NEEFS766F4TGGBZ6NN X-MailFrom: s.hanreich@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: https://lore.proxmox.com/pve-devel/20260512173145.596958-1-s.hanreich@proxmox.com/T/#t On 5/7/26 2:38 PM, Stefan Hanreich wrote: > This patch series is based on top of the route-maps series [1]. While it does > not rely on any of the features / code included there, there are some merge > conflicts when applying either series first, so I'm sending it based on top of > the route-maps series so they can be applied conveniently after another. > > ## Introduction > > This patch series introduces WireGuard as fabric protocol. Potential use-cases > include: > > * Connecting to remote PBS / PDM instances > * Simple encryption layer for intra-DC VXLAN tunnels > * Secure migration network > * Connecting with remote PVE clusters > > It utilizes the wg(8) tool for generating the interface configuration [2] and > the section config format leans heavily into the keys defined there. > > > ## Configuration format > > The configuration format is quite similar to OSPF and Openfabric with the main > difference being that WireGuard nodes have been split into two subtypes > (external and internal), in order to support nodes that are not part of the > cluster. > > ### Nodes > > WireGuard nodes have been split into two different types. Those are not distinct > section config types, due to how the internal representation of the FabricConfig > has been structured (which maps exactly one Fabric type to one Node type). So > instead there is one Node type that is an enum. The 'role' field is used for > distinguishing between different WireGuard node types. > > #### Internal > > This represents a node that is part of the Proxmox VE cluster. > > An example configuration looks like this: > > wireguard_node: vpn_elementalist > endpoint 192.0.2.1 > allowed_ips 203.0.113.128/25 > interfaces name=wg0,listen_port=50000,public_key=O+Kzrochm6klMILjSKVw83xb3YyXXLpmZj9n/ICM5xE=,ip=198.51.100.1/24 > role internal > > The endpoint value will be used by other nodes inside the Proxmox VE cluster for > connecting to the defined node. IPs are defined on a per-interface basis, not a > per-node basis. The interface key represents the [Interface] section in the > WireGuard configuration. All values (except for public key) are overridable in > the peer definition. > > #### External > > External nodes represent any peer that is not a Proxmox VE node. They provide a > mechanism for defining a reusable peer definition (see below for more details). > > This allows for easily re-using and updating the information of an external > peer, without having to re-type all information for every Proxmox VE node that > wants to utilize the definition. > > An example configuration looks like this: > > wireguard_node: vpn_berserker > endpoint berserker:51337 > allowed_ips 203.0.113.0/25 > public_key GDPUAnPOY5xGIjYXmcGyXZXbocjBr21dGQ5vwnjmdzA= > role external > > Those keys map 1:1 to the peer entries in the respective WireGuard configuration > format and are used for generating the peer definition wherever they are > referenced. > > ### Peers > > Interfaces on Proxmox nodes can have one or more peers. A peer is a reference to > either the interface of an internal node, or an external node. Due to > limitations in dealing with nested data in the section config, peers are an > array field in the node, instead of being configured on the interface directly. > > An example configuration for a Proxmox VE node with an interface that has an > internal and external node as peer looks as follows: > > wireguard_node: vpn_occultist > endpoint 192.0.2.2 > interfaces name=wg0,listen_port=50000,public_key=y0kOpXfo9ff4KoUwO3H1cRuwObbKwsK8mAkwXxNvKUc=,ip=198.51.100.2/24 > peers type=internal,node=elementalist,node_iface=wg0,iface=wg0 > peers type=external,node=berserker,iface=wg0 > role internal > > This would generate the following wg0.conf file: > > [Interface] > PrivateKey = > ListenPort = 50000 > > [Peer] > PublicKey = O+Kzrochm6klMILjSKVw83xb3YyXXLpmZj9n/ICM5xE= > AllowedIPs = 198.51.100.1/32 > Endpoint = 192.0.2.1:50000 > AllowedIPs = 203.0.113.128/25 > > [Peer] > PublicKey = GDPUAnPOY5xGIjYXmcGyXZXbocjBr21dGQ5vwnjmdzA= > Endpoint = berserker:51337 > AllowedIPs = 203.0.113.0/25 > > > Peer definitions allow overriding properties from the node definition (e.g. > endpoint). This is currently not implemented in the frontend. This is also the > main reason for choosing to store peers as an array in a different key. > Referencing peer defintions by id would have been possible in the interface > property string, but if the possibility of overriding certain attributes should > be available, then a separate key with property strings is required. > > > ## Key handling > > Keys are automatically generated in the backend on demand, whenever an interface > is created. Keys are deleted upon applying the SDN configuration. After a > key has been generated, the respective public key gets stored in the section > config. > > The WireGuard configuration files are stored locally on the node in the newly > established '/etc/wireguard/proxmox' folder, and managed by the node itself. > > > ## Open questions / issues > > ### Peers > > The main issue I see with the configuration format is that peers reference > arbitrary node sections / interface definitions in the fabric config. This poses > some problems, particularly when updating the referenced entities. For instance, > users could delete a referenced interface, invalidating the configuration. This > is quite similar to the problems we currently encounter with firewall ipsets and > aliases. > > In order to avoid re-creating the same issues there are a few restrictions in > the UI that should prevent the most common mistakes: > > * Renaming nodes and interfaces is not allowed. > * The configuration is validated after every modification and invalid > configurations are outright rejected. This is particularly important for > delete operations. > > In the future we could lift some restrictions by implementing smarter CRUD > operations. For instance, when deleting an interface all peer entries, that > reference that interface, could be deleted as well. Even for accidental > deletions this isn't too bad imo, since we have a mechanism of restoring the > current running configuration, which users can always use. > > For updates to the interfaces of a node this is harder, since it is impossible > to say whether an interface has been renamed or an interface has been deleted > and another one created. I don't really see a good heuristic (even when tracking > this in the UI) that works particularly well for all potential cases. > > ### Section Types > > The split of one section type ('wireguard_node') into two different subtypes is > breaking a bit with section config principles. Another solution would be to > introduce two section config types (e.g. wireguard_node_{external,internal}), > although that would require quite some refactoring effort. > > > ## Future work > > * implement status reporting > * provide QoL features for easier config (e.g. auto-"fullmeshify" PVE cluster) > * Implement some backend-only features in the UI (e.g. per-peer overrides, > pre-shared keys) > * Integration into PDM / PBS > > > ## Dependencies > > * proxmox-ve-config depends on proxmox-sdn-types > * proxmox-ve-config depends on proxmox-network-types > * proxmox-ve-config depends on proxmox-wireguard > * proxmox-perl-rs depends on proxmox-ve-config > * pve-network depends on proxmox-perl-rs > * pve-network depends on pve-cluster > * pve-manager depends on pve-network > > > Changes from v3 (Thanks @Thomas): > * rebased on top of current master > * use x25519 instead of ed25519 for public key derivation (which is the correct > algorithm) > * moved keys to pmxcfs into a section config file under /etc/pve/priv > * delete keys on applying the SDN config, not when calling DELETE API call > * fix error message when referenced interface does not exist > * fix validating the existence of interfaces > * fix editing an external node > > Changes from v2 (Thanks @Gabriel): > * rebased branches on top of current master + route-maps series > * added backend-only option to skip auto-generating routes > * added possibility to include wireguard interfaces when selecting interfaces > for nodes in other fabric types > * show auto-generated public key in Web UI > * improved validation error messages > * added better descriptions in the UI for the endpoint / allowed ips options > * added newline to generated ifupdown2 config stanza > * added early failure in case wireguard-tools isn't installed > > Changes from RFC: > * rebased on top of current master branches > > [1] https://lore.proxmox.com/pve-devel/20260504160350.395470-2-s.hanreich@proxmox.com/T/#t > [2] https://man7.org/linux/man-pages/man8/wg.8.html > > > pve-cluster: > > Stefan Hanreich (1): > cfs: add 'priv/wg-keys.cfg' to observed files > > src/PVE/Cluster.pm | 1 + > src/pmxcfs/status.c | 1 + > 2 files changed, 2 insertions(+) > > > proxmox: > > Stefan Hanreich (4): > wireguard: utilize x25519 for public key generation > wireguard: skip serializing preshared_key if unset > wireguard: implement ApiType for private key > network-types: implement ApiType for endpoints and hostnames > > proxmox-network-types/src/endpoint.rs | 30 ++++++++++- > proxmox-wireguard/Cargo.toml | 1 + > proxmox-wireguard/src/lib.rs | 72 +++++++++++++-------------- > 3 files changed, 64 insertions(+), 39 deletions(-) > > > proxmox-ve-rs: > > Christoph Heiss (2): > sdn-types: add wireguard-specific PersistentKeepalive api type > ve-config: fabric: refactor fabric config entry impl using macro > > Stefan Hanreich (6): > ve-config: fabrics: split interface name regex into two parts > ve-config: fabrics: add protocol-specific properties for wireguard > ve-config: wireguard: add private keys section config > ve-config: sdn: fabrics: add wireguard to the fabric config > ve-config: fabrics: wireguard add validation for wireguard config > ve-config: fabrics: implement wireguard config generation > > proxmox-sdn-types/src/lib.rs | 1 + > proxmox-sdn-types/src/wireguard.rs | 43 + > proxmox-ve-config/Cargo.toml | 3 + > proxmox-ve-config/debian/control | 6 + > proxmox-ve-config/src/sdn/fabric/frr.rs | 1 + > proxmox-ve-config/src/sdn/fabric/mod.rs | 405 +++++++-- > .../src/sdn/fabric/section_config/fabric.rs | 25 + > .../sdn/fabric/section_config/interface.rs | 5 +- > .../src/sdn/fabric/section_config/mod.rs | 58 ++ > .../src/sdn/fabric/section_config/node.rs | 32 +- > .../sdn/fabric/section_config/protocol/mod.rs | 1 + > .../section_config/protocol/wireguard.rs | 788 ++++++++++++++++++ > proxmox-ve-config/src/sdn/mod.rs | 1 + > proxmox-ve-config/src/sdn/wireguard.rs | 309 +++++++ > 14 files changed, 1607 insertions(+), 71 deletions(-) > create mode 100644 proxmox-sdn-types/src/wireguard.rs > create mode 100644 proxmox-ve-config/src/sdn/fabric/section_config/protocol/wireguard.rs > create mode 100644 proxmox-ve-config/src/sdn/wireguard.rs > > > proxmox-perl-rs: > > Christoph Heiss (1): > pve-rs: fabrics: wireguard: generate ifupdown2 configuration > > Stefan Hanreich (2): > pve-rs: fabrics: add helpers for parsing interface property strings > pve-rs: sdn: wireguard: add private keys module > > pve-rs/Cargo.toml | 1 + > pve-rs/Makefile | 1 + > pve-rs/src/bindings/sdn/fabrics.rs | 217 +++++++++++++++++++++++---- > pve-rs/src/bindings/sdn/mod.rs | 1 + > pve-rs/src/bindings/sdn/wireguard.rs | 103 +++++++++++++ > pve-rs/src/sdn/status.rs | 16 ++ > 6 files changed, 308 insertions(+), 31 deletions(-) > create mode 100644 pve-rs/src/bindings/sdn/wireguard.rs > > > pve-network: > > Christoph Heiss (1): > sdn: add wireguard helper module > > Stefan Hanreich (2): > fabrics: wireguard: add schema definitions for wireguard > fabrics: wireguard: implement wireguard key auto-generation > > src/PVE/API2/Network/SDN.pm | 4 +- > .../API2/Network/SDN/Fabrics/FabricNode.pm | 105 ++++++++++- > src/PVE/Network/SDN.pm | 2 + > src/PVE/Network/SDN/Fabrics.pm | 173 +++++++++++++++++- > src/PVE/Network/SDN/Makefile | 3 +- > src/PVE/Network/SDN/WireGuard.pm | 162 ++++++++++++++++ > 6 files changed, 435 insertions(+), 14 deletions(-) > create mode 100644 src/PVE/Network/SDN/WireGuard.pm > > > pve-manager: > > Christoph Heiss (2): > ui: fabrics: edit: make ipv4/6 support generic over fabric panels > ui: fabrics: interface: make ipv4/6 support generic over edit panels > > Stefan Hanreich (10): > network: sdn: generate wireguard configuration on apply > ui: fix parsing of property-strings when values contain = > ui: fabrics: i18n: make node loading string translatable > ui: fabrics: split node selector creation and config > ui: fabrics: node: make ipv4/6 support generic over edit panels > ui: fabrics: wireguard: add interface edit panel > ui: fabrics: wireguard: add node edit panel > ui: fabrics: wireguard: add fabric edit panel > ui: fabrics: hook up wireguard components > fabrics: node edit: add option to include wireguard interfaces > > PVE/API2/Network.pm | 1 + > www/manager6/Makefile | 3 + > www/manager6/Parser.js | 7 +- > www/manager6/sdn/FabricsView.js | 12 + > www/manager6/sdn/fabrics/FabricEdit.js | 68 ++- > www/manager6/sdn/fabrics/InterfacePanel.js | 18 + > www/manager6/sdn/fabrics/NodeEdit.js | 108 ++++- > .../sdn/fabrics/openfabric/FabricEdit.js | 32 -- > .../sdn/fabrics/openfabric/InterfacePanel.js | 13 - > .../sdn/fabrics/openfabric/NodeEdit.js | 14 - > www/manager6/sdn/fabrics/ospf/FabricEdit.js | 2 + > .../sdn/fabrics/ospf/InterfacePanel.js | 2 + > www/manager6/sdn/fabrics/ospf/NodeEdit.js | 3 + > .../sdn/fabrics/wireguard/FabricEdit.js | 29 ++ > .../sdn/fabrics/wireguard/InterfacePanel.js | 435 ++++++++++++++++++ > .../sdn/fabrics/wireguard/NodeEdit.js | 229 +++++++++ > 16 files changed, 882 insertions(+), 94 deletions(-) > create mode 100644 www/manager6/sdn/fabrics/wireguard/FabricEdit.js > create mode 100644 www/manager6/sdn/fabrics/wireguard/InterfacePanel.js > create mode 100644 www/manager6/sdn/fabrics/wireguard/NodeEdit.js > > > Summary over all repositories: > 47 files changed, 3298 insertions(+), 249 deletions(-) >