[RFC manager/network/proxmox{,-ve-rs,-perl-rs} 00/27] Add WireGuard as protocol to SDN fabrics

[Stefan Hanreich <s.hanreich@proxmox.com>]

This patch series builds upon the patch series from Christoph, that introduces
the proxmox-wireguard crate [1]. Sending this as an RFC to gather feedback,
particularly on the configuration format as well as the key generation logic.


## Introduction

This patch series introduces WireGuard as fabric protocol. Potential use-cases
include:

* Connecting to remote PBS / PDM instances
* Simple encryption layer for intra-DC VXLAN tunnels
* Secure migration network
* Connecting with remote PVE clusters

It utilizes the wg(8) tool for generating the interface configuration [2] and
the section config format leans heavily into the keys defined there.


## Configuration format

The configuration format is quite similar to OSPF and Openfabric with the main
difference being that WireGuard nodes have been split into two subtypes
(external and internal), in order to support nodes that are not part of the
cluster.

### Nodes

WireGuard nodes have been split into two different types. Those are not distinct
section config types, due to how the internal representation of the FabricConfig
has been structured (which maps exactly one Fabric type to one Node type). So
instead there is one Node type that is an enum. The 'role' field is used for
distinguishing between different WireGuard node types.

#### Internal

This represents a node that is part of the Proxmox VE cluster.

An example configuration looks like this:

    wireguard_node: vpn_elementalist
            endpoint 192.0.2.1
            allowed_ips 203.0.113.128/25
            interfaces name=wg0,listen_port=50000,public_key=O+Kzrochm6klMILjSKVw83xb3YyXXLpmZj9n/ICM5xE=,ip=198.51.100.1/24
            role internal

The endpoint value will be used by other nodes inside the Proxmox VE cluster for
connecting to the defined node. IPs are defined on a per-interface basis, not a
per-node basis. The interface key represents the [Interface] section in the
WireGuard configuration. All values (except for public key) are overridable in
the peer definition.

#### External

External nodes represent any peer that is not a Proxmox VE node. They provide a
mechanism for defining a reusable peer definition (see below for more details).

This allows for easily re-using and updating the information of an external
peer, without having to re-type all information for every Proxmox VE node that
wants to utilize the definition.

An example configuration looks like this:

    wireguard_node: vpn_berserker
            endpoint berserker:51337
            allowed_ips 203.0.113.0/25
            public_key GDPUAnPOY5xGIjYXmcGyXZXbocjBr21dGQ5vwnjmdzA=
            role external

Those keys map 1:1 to the peer entries in the respective WireGuard configuration
format and are used for generating the peer definition wherever they are
referenced.

### Peers

Interfaces on Proxmox nodes can have one or more peers. A peer is a reference to
either the interface of an internal node, or an external node. Due to
limitations in dealing with nested data in the section config, peers are an
array field in the node, instead of being configured on the interface directly.

An example configuration for a Proxmox VE node with an interface that has an
internal and external node as peer looks as follows:

    wireguard_node: vpn_occultist
            endpoint 192.0.2.2
            interfaces name=wg0,listen_port=50000,public_key=y0kOpXfo9ff4KoUwO3H1cRuwObbKwsK8mAkwXxNvKUc=,ip=198.51.100.2/24
            peers type=internal,node=elementalist,node_iface=wg0,iface=wg0
            peers type=external,node=berserker,iface=wg0
            role internal

This would generate the following wg0.conf file:

    [Interface]
    PrivateKey = <some_private_key>
    ListenPort = 50000

    [Peer]
    PublicKey = O+Kzrochm6klMILjSKVw83xb3YyXXLpmZj9n/ICM5xE=
    AllowedIPs = 198.51.100.1/32
    Endpoint = 192.0.2.1:50000
    AllowedIPs = 203.0.113.128/25

    [Peer]
    PublicKey = GDPUAnPOY5xGIjYXmcGyXZXbocjBr21dGQ5vwnjmdzA=
    Endpoint = berserker:51337
    AllowedIPs = 203.0.113.0/25


Peer definitions allow overriding properties from the node definition (e.g.
endpoint). This is currently not implemented in the frontend. This is also the
main reason for choosing to store peers as an array in a different key.
Referencing peer defintions by id would have been possible in the interface
property string, but if the possibility of overriding certain attributes should
be available, then a separate key with property strings is required.


## Key handling

Keys are automatically generated in the backend on demand, whenever a interface
is created / updated or deleted. This is accomplished by proxying API calls to
the respective node and generating / deleting the WireGuard keys there. After a
key has been generated, the respective public key gets stored in the section
config. The main reason for this is that it allows returning the public key from
the API without having to read one configuration file. This is particularly
relevant when implementing support for establishing cross-cluster communication
in PDM, which requires the information for creating the respective peer
definitions on each cluster.

All WireGuard keys and configuration files are stored locally on the node in the
newly established '/etc/wireguard/proxmox' folder, and managed by the node
itself. Private keys are never transmitted across the wire and are also not
shared via pmxcfs. A WireGuard interface configuration usually requires 3 files:
public/private key and a wireguard configuration file.


## Open questions / issues

### Peers

The main issue I see with the configuration format is that peers reference
arbitrary node sections / interface definitions in the fabric config. This poses
some problems, particularly when updating the referenced entities. For instance,
users could delete a referenced interface, invalidating the configuration. This
is quite similar to the problems we currently encounter with firewall ipsets and
aliases.

In order to avoid re-creating the same issues there are a few restrictions in
the UI that should prevent the most common mistakes:

* Renaming nodes and interfaces is not allowed.
* The configuration is validated after every modification and invalid
  configurations are outright rejected. This is particularly important for
  delete operations.

In the future we could lift some restrictions by implementing smarter CRUD
operations. For instance, when deleting an interface all peer entries, that
reference that interface, could be deleted as well. Even for accidental
deletions this isn't too bad imo, since we have a mechanism of restoring the
current running configuration, which users can always use.

For updates to the interfaces of a node this is harder, since it is impossible
to say whether an interface has been renamed or an interface has been deleted
and another one created. I don't really see a good heuristic (even when tracking
this in the UI) that works particularly well for all potential cases.

### Section Types

The split of one section type ('wireguard_node') into two different subtypes is
breaking a bit with section config principles. Another solution would be to
introduce two section config types (e.g. wireguard_node_{external,internal}),
although that would require quite some refactoring effort.


## Todo:

Since this is an RFC, there are still some unpolished / rough edges that will be
improved until a v1:

* currently there is no documentation
* add more integration tests
* potentially expose the overridable properties in the UI
* implement deletion of keys when deleting the whole fabric (bit tricky with how
  keygen is currently handled).

## Future work

* implement status reporting
* implement ECMP for routing the same subnet via multiple wg interfaces
* provide QoL features for easier config (e.g. auto-"fullmeshify" PVE cluster)
* Implement some backend-only features in the UI (e.g. per-peer overrides,
  pre-shared keys)
* Integration into PDM / PBS

[1] https://lore.proxmox.com/all/20260213143601.1424613-1-c.heiss@proxmox.com/
[2] https://man7.org/linux/man-pages/man8/wg.8.html

proxmox:

Stefan Hanreich (2):
  wireguard: skip serializing preshared_key if unset
  wireguard: implement ApiType for endpoints and hostnames

 proxmox-network-types/src/endpoint.rs | 30 +++++++++++++++++++++++++--
 proxmox-wireguard/src/lib.rs          |  1 +
 2 files changed, 29 insertions(+), 2 deletions(-)


proxmox-ve-rs:

Christoph Heiss (2):
  sdn-types: add wireguard-specific PersistentKeepalive api type
  ve-config: fabric: refactor fabric config entry impl using macro

Stefan Hanreich (7):
  debian: update control file
  clippy: fix 'hiding a lifetime that's elided elsewhere is confusing'
  ve-config: fabrics: split interface name regex into two parts
  ve-config: fabrics: add protocol-specific properties for wireguard
  ve-config: sdn: fabrics: add wireguard to the fabric config
  ve-config: fabrics: wireguard add validation for wireguard config
  ve-config: fabrics: implement wireguard config generation

 proxmox-sdn-types/debian/control              |   3 +-
 proxmox-sdn-types/src/lib.rs                  |   1 +
 proxmox-sdn-types/src/wireguard.rs            |  43 ++
 proxmox-ve-config/Cargo.toml                  |   3 +
 proxmox-ve-config/debian/control              |   6 +
 proxmox-ve-config/src/firewall/types/ipset.rs |   2 +-
 proxmox-ve-config/src/sdn/fabric/frr.rs       |   1 +
 proxmox-ve-config/src/sdn/fabric/mod.rs       | 399 ++++++++++---
 .../src/sdn/fabric/section_config/fabric.rs   |  25 +
 .../sdn/fabric/section_config/interface.rs    |   5 +-
 .../src/sdn/fabric/section_config/mod.rs      |  58 ++
 .../src/sdn/fabric/section_config/node.rs     |  44 +-
 .../sdn/fabric/section_config/protocol/mod.rs |   1 +
 .../section_config/protocol/wireguard.rs      | 546 ++++++++++++++++++
 proxmox-ve-config/src/sdn/mod.rs              |   1 +
 proxmox-ve-config/src/sdn/wireguard.rs        | 311 ++++++++++
 16 files changed, 1375 insertions(+), 74 deletions(-)
 create mode 100644 proxmox-sdn-types/src/wireguard.rs
 create mode 100644 proxmox-ve-config/src/sdn/fabric/section_config/protocol/wireguard.rs
 create mode 100644 proxmox-ve-config/src/sdn/wireguard.rs


proxmox-perl-rs:

Christoph Heiss (1):
  pve-rs: fabrics: wireguard: generate ifupdown2 configuration

Stefan Hanreich (1):
  pve-rs: fabrics: add helpers for parsing interface property strings

 pve-rs/Cargo.toml                  |   1 +
 pve-rs/src/bindings/sdn/fabrics.rs | 215 +++++++++++++++++++++++++----
 pve-rs/src/sdn/status.rs           |  16 +++
 3 files changed, 203 insertions(+), 29 deletions(-)


pve-network:

Christoph Heiss (1):
  sdn: add wireguard helper module

Stefan Hanreich (2):
  fabrics: wireguard: add schema definitions for wireguard
  fabrics: wireguard: implement wireguard key auto-generation

 src/PVE/API2/Network/SDN.pm                   |   2 +-
 .../API2/Network/SDN/Fabrics/FabricNode.pm    | 129 ++++++++-
 src/PVE/Network/SDN.pm                        |   9 +-
 src/PVE/Network/SDN/Fabrics.pm                | 257 +++++++++++++++++-
 src/PVE/Network/SDN/Makefile                  |  15 +-
 src/PVE/Network/SDN/WireGuard.pm              | 163 +++++++++++
 6 files changed, 560 insertions(+), 15 deletions(-)
 create mode 100644 src/PVE/Network/SDN/WireGuard.pm


pve-manager:

Christoph Heiss (2):
  ui: fabrics: edit: make ipv4/6 support generic over fabric panels
  ui: fabrics: interface: make ipv4/6 support generic over edit panels

Stefan Hanreich (9):
  network: sdn: generate wireguard configuration on apply
  ui: fix parsing of property-strings when values contain =
  ui: fabrics: i18n: make node loading string translatable
  ui: fabrics: split node selector creation and config
  ui: fabrics: node: make ipv4/6 support generic over edit panels
  ui: fabrics: wireguard: add interface edit panel
  ui: fabrics: wireguard: add node edit panel
  ui: fabrics: wireguard: add fabric edit panel
  ui: fabrics: hook up wireguard components

 PVE/API2/Network.pm                           |   1 +
 www/manager6/Makefile                         |   3 +
 www/manager6/Parser.js                        |   7 +-
 www/manager6/sdn/FabricsView.js               |  12 +
 www/manager6/sdn/fabrics/FabricEdit.js        |  68 ++-
 www/manager6/sdn/fabrics/InterfacePanel.js    |  18 +
 www/manager6/sdn/fabrics/NodeEdit.js          |  64 ++-
 .../sdn/fabrics/openfabric/FabricEdit.js      |  32 --
 .../sdn/fabrics/openfabric/InterfacePanel.js  |  13 -
 .../sdn/fabrics/openfabric/NodeEdit.js        |  14 -
 www/manager6/sdn/fabrics/ospf/FabricEdit.js   |   2 +
 .../sdn/fabrics/ospf/InterfacePanel.js        |   2 +
 www/manager6/sdn/fabrics/ospf/NodeEdit.js     |   2 +
 .../sdn/fabrics/wireguard/FabricEdit.js       |  29 ++
 .../sdn/fabrics/wireguard/InterfacePanel.js   | 427 ++++++++++++++++++
 .../sdn/fabrics/wireguard/NodeEdit.js         | 230 ++++++++++
 16 files changed, 835 insertions(+), 89 deletions(-)
 create mode 100644 www/manager6/sdn/fabrics/wireguard/FabricEdit.js
 create mode 100644 www/manager6/sdn/fabrics/wireguard/InterfacePanel.js
 create mode 100644 www/manager6/sdn/fabrics/wireguard/NodeEdit.js


Summary over all repositories:
  43 files changed, 3002 insertions(+), 209 deletions(-)

-- 
Generated by git-murpp 0.8.0