[pve-devel] applied-series: [PATCH access-control/cluster/docs/gui-tests/manager/network/proxmox{, -firewall, -ve-rs, -perl-rs, -widget-toolkit} v5 00/76] Add SDN Fabrics

[Thomas Lamprecht <t.lamprecht@proxmox.com>]

Am 16.07.25 um 15:07 schrieb Gabriel Goller:
> Overview
> ========
> 
> This series allows the user to easily use dynamic routing protocols such as
> OpenFabric and OSPF in their clusters. It also integrates existing features,
> such as Ceph with the new SDN fabrics feature to enable users simple
> configuration of e.g. full-mesh Ceph clusters via the Web UI.
> 
> This patch series adds the initial support for two routing protocols:
> * OpenFabric
> * OSPF
> 
> In the future we plan on moving the existing IS-IS and BGP controllers into the
> fabric structure. Christoph Heiss is also currently working on adding a new
> Wireguard fabric, which can be combined with any other fabric types. This
> feature allows layering different fabrics on top of each other, so adding
> encryption to an existing fabric is as simple as just putting a Wireguard fabric
> on top, or using a Wireguard fabric as the basis.
> 
> Packages are available on sani: packages/sdn-fabrics-v4
> 
> 
> Implementation
> ==============
> 
> Every fabric consists of zero or more nodes, which themselves consist of zero or
> more interfaces. Fabrics and nodes are modeled as different section config types
> (which means two section types for each protocol), interfaces are an array
> contained in a node section.
> 
> For now, nodes in the fabric configuration always represent PVE nodes, but in
> the future nodes could also represent external members of the fabric (e.g. in a
> potential Wireguard fabric). An example use case for this would be securely
> connecting PBS or PDM instances to the PVE cluster via Wireguard.
> 
> Most of the functionality is implemented in Rust and exposed to the existing SDN
> module via perlmod. This includes configuration reading / writing, FRR config
> generation from the section config and API CRUD methods. Some functionality,
> like digest matching and permission checking is still handled on the perl side,
> due to the lack of facilities in Rust for that.
> 
> 
> Configuration Format
> --------------------
> 
> The whole configuration is now contained in just one configuration file
> `/etc/pve/sdn/fabrics.cfg`. This makes handling the fabrics configuration easier
> in many different areas: locking, digest calculation, validation.
> 
> For every protocol there are two different section types (fabric and node). As
> an example the two section types for OSPF are 'ospf_fabric' and 'ospf_node'.
> 
> The ID of a fabric is a simple name, at most 8 alphanumeric characters since we
> use it for generating network interfaces names with a prefix. This is analogous
> to existing SDN entities, e.g. VNet. A node can only be uniquely identified by
> its id (which is equivalent to the hostname of the node), as well as the
> fabric_id. This is because a node can be part of multiple fabrics.
> 
> An example how the configuration looks like for a full-mesh 3-node Openfabric
> fabric called 'example':
> 
>     openfabric_fabric: example
>         csnp_interval 3
>         hello_interval 3
>         ip6_prefix 2001:db8::/64
>         ip_prefix 192.0.2.0/24
> 
>     openfabric_node: example_deadeye
>         interfaces name=eth1,ip=198.51.100.0/31
>         interfaces name=eth2
>         ip 192.0.2.1
>         ip6 2001:db8::1
> 
>     openfabric_node: example_pathfinder
>         interfaces name=eth1,ip=198.51.100.2/31
>         interfaces name=eth2
>         ip 192.0.2.2
>         ip6 2001:db8::2
> 
>     openfabric_node: example_raider
>         interfaces name=eth1,ip=198.51.100.4/31
>         interfaces name=eth2
>         ip 192.0.2.3
>         ip6 2001:db8::3
> 
> We parse the configuration file flat in rust, and then afterwards split them
> into Fabric / Node structs and store them hierarchically (fabric -> node) in a
> dedicated FabricConfig struct. This struct provides the CRUD methods for
> manipulating the FabricConfig safely as well as serializing the FabricConfig
> back into its section config format.
> 
> To prevent having to duplicate common properties for every protocol, we
> introduced a generic (Fabric|Node)Section<T> struct that contains all common
> properties. Protocol-specific properties can be defined by the generic type
> parameter T. This saves us from duplicating a lot of code (which was an initial
> problem with the intermediate configuration) and conversions can be simplified
> by providing a generic implementation that every protocol uses.
> 
> This design also means that adding new protocols to the configuration is quite
> straightforward: It is only required to add structs with the protocol-specific
> properties in Rust and add them to the enums defining the Section Config. The
> commit adding OSPF support shows how simple it is to add a new protocol.
> 
> 
> Validation
> ----------
> 
> The hierarchical nature of the configuration and the relationship between nodes
> inside the fabrics requires validation of sections relative to other sections.
> For this matter we introduced a new Validatable trait as well as a struct that
> wraps valid configuration in a Valid<T> struct. For more information on that see
> the respective commit.
> 
> 
> API & Permissions
> -----------------
> 
> The whole API is contained in the /cluster/sdn/fabrics subfolder and contains
> submodules for fabric / node.
> 
> A quick overview of the methods provided by the API:
> 
> GET /all - list fabrics & nodes
> 
> GET /fabric - list all fabrics
> POST /fabric - create a fabric
> GET /fabric/{fabric_id} - get a single fabric
> PUT /fabric/{fabric_id} - update a fabric
> DELETE /fabric/{fabric_id} - delete a fabric
> 
> GET /node - list all nodes (regardless of fabric)
> 
> GET /node/{fabric_id} - list all nodes belonging to fabric {fabric_id}
> POST /node/{fabric_id} - create a node in fabric {fabric_id}
> GET /node/{fabric_id}/{node_id} - get a single node
> PUT /node/{fabric_id}/{node_id} - update a single node
> DELETE /node/{fabric_id}/{node_id} - delete a single node
> 
> 
> FRR Configuration
> -----------------
> 
> For the FRR-specific functionality we introduced a new proxmox-frr crate that
> models the different entities in the FRR configuration format (routers,
> interfaces, route-maps, ...) and provides serializers for those structs. For
> more information see the respective FRR commits. When applying the SDN
> configuration, perl calls into perlmod to utilize the proxmox-frr crate for
> generating the FRR configuration of the fabrics.
> 
> We also introduce a proxmox-sdn-types crate, where we extracted generic
> fabric types (e.g., openfabric::HelloInterval), so we can reuse them across
> multiple crates (proxmox-frr, proxmox-ve-config, ..).
> 
> 
> UI
> --
> 
> The UI allows users to easily create different types of fabrics. One can add
> Nodes to the fabrics by selecting them from a dropdown which shows all the nodes
> in the cluster. Additionally the user can then select the interfaces of the node
> which should be added to the fabric. There are also protocol-specific options
> such as "passive", "hello-interval" etc. available to select on the interface.
> There are also options spanning whole fabrics: the "hello-interval" option on
> openfabric for example, can be set on the fabric and will be applied to every
> interface.
> 
> We are also working on the integration of status reporting into the sidebar,
> which also includes an integration into pvestatd. The plan is to show the status
> for each node, which routes are learned, neighbor status and possibly topology
> from the POV of each node. Since this patch series is already quite huge and the
> sidebar integration is still a work in progress it is not included here.
> 
> Integration with existing features
> ----------------------------------
> 
> We also provide a UI for the Ceph, Migration Network, VXLAN zone and EVPN
> controller integrations. Users can configure fabrics for those components simply
> by selecting them from a dropdown, providing a streamlined experience and nice
> integration with existing features.
> 
> 
> Refactoring
> ===========
> 
> This patch series required some rework of existing functionality, mostly how SDN
> generates the FRR configuration and writes /etc/network/interfaces. Prior the
> FRR configuration was generated exclusively from the controllers, but fabrics
> need to write it as well. Same goes for the interfaces file, which got written
> by the Zone plugin, but Fabrics need to write this file as well.
> 
> For this we moved the FRR and ifupdown config generation one level up to the SDN
> module, which now calls into the respective child modules to generate the FRR /
> ifupdown configuration.
> 
> 
> Dependencies
> ============
> This series relies on the FRR 10.2.2 backport series, since it fixes potential
> issues with EVPN + Openfabric/OSPF:
> 
> https://lore.proxmox.com/all/20250418112114.2747673-1-s.hanreich@proxmox.com/
> 
> 
> proxmox-frr depends on proxmox-network-types
> proxmox-frr depends on proxmox-sdn-types
> 
> proxmox-ve-config depends on proxmox-frr
> proxmox-ve-config depends on proxmox-network-types
> proxmox-ve-config depends on proxmox-sdn-types
> proxmox-ve-config depends on proxmox-serde
> proxmox-ve-config depends on proxmox-api-macro
> 
> proxmox-firewall depends on proxmox-ve-config
> 
> proxmox-perl-rs depends on proxmox-ve-config
> proxmox-perl-rs depends on proxmox-frr
> proxmox-perl-rs depends on proxmox-network-types
> 
> pve-network depends on proxmox-perl-rs
> pve-network depends on pve-cluster
> pve-network depends on pve-access-control
> 
> pve-docs depends on pve-gui-tests
> 
> pve-manager depends on proxmox-widget-toolkit
> pve-manager depends on pve-docs
> pve-manager depends on pve-network
> pve-manager depends on pve-access-control
> 
> 
> pve-network commits 4-7 do not build independently, because it's one refactor
> but split across multiple commits so it's easier to follow the steps during the
> refactor. We could consider squashing those commits on applying, so each commit
> still builds indepedently.
> 
> Shoutout to Stefan for his great work on this patch series!
> 


Talked with Hannes quickly before he left and in his opinion he saw no big blocker,
so applied series, *huge* thanks to all involved, nice work and great details in
the patch submission!

While I'm certain things can still be polished anything, that's (more) true for
basically everything else, and doing t will need actual user feedback anyway.

The screenshots are still missing from the docs, I commented the thumbnail
references out for now. It would be great if you could push a commit adding them
to your staff repo.


Tiny UX nits: For the UI it might be nice to add some emptyTexts in the input
fields of the add/edit dialogue. Making those windows a bit wider wouldn't hurt
either IMO.

Some simple how-to's for the wiki for some real world use cases might be nice
too, but maybe it's best to wait on some user feedback and get in other in progress
stuff here.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel