[pve-devel] [PATCH cluster/docs/manager/network/proxmox{, -ve-rs, -firewall, -perl-rs} 00/52] Add SDN Fabrics

[Gabriel Goller <g.goller@proxmox.com>]

This series allows the user to add fabrics such as OpenFabric and OSPF over
their clusters.

Overview
========

This series allows the user to create routed networks ('fabrics') across their
clusters, which can be used as the underlay network for a EVPN cluster, or for
creating Ceph full mesh clusters easily.

This patch series adds the initial support for two routing protocols:
* OpenFabric
* OSPF

In the future we plan on moving the existing IS-IS and BGP controllers into the
fabric structure. There are also plans for adding a new Wireguard fabric to
this.


Implementation
==============

Every fabric consists of one or more nodes, which themselves consists of one or
more interfaces. Fabrics and nodes are modeled as different section config
types, interfaces are an array contained in a node section. We have a separate
configuration file for each fabric type. This is because the basic structure
(fabric, node, interface) is the same, but the specific options vary wildly.
This makes serialization / validation from the Rust side a lot easier.

For now, nodes in the fabric configuration are always PVE nodes, but in the
future nodes could also represent external members of the fabric (e.g. in a
potential wireguard fabric).

Settings can be configured on a fabric-level, so they apply to all interfaces,
or overridden on a interface-level (hidden in the UI by default).

Most of the functionality is implemented by rust and exposed to the existing SDN
module via perlmod. This includes configuration reading / writing, FRR config
generation from the section config and API CRUD methods.

The API provides one common GET method, to get the configuration of all
different fabric types (for the tree overview), but otherwise there are separate
CRUD endpoints for every fabric type, to mimic the split of configuration files.
Another upside of this is, that the generated rust structs for the API endpoints
(for PDM) will be much nicer.

For the FRR-specific functionality we introduced a new proxmox-frr crate that
models the different entities in the FRR configuration format (routers,
interfaces, route-maps, ...) and provides serializers for those structs. For
more information see the respective FRR commits. When applying the SDN
configuration, perl calls into perlmod to utilize the proxmox-frr crate for
generating the FRR configuration of the fabrics.

We also introduce a proxmox-network-types crate, where we extracted generic
network types (Cidr for instance), so we can reuse them across multiple crates
(proxmox-frr, proxmox-firewall, proxmox-ve-config, ..).

The hierarchical nature of the configuration and the relationship between nodes
inside the fabrics requires validation of sections relative to other sections.
For this matter we introduced an intermediate configuration in the initial RFC,
but that turned out to be unwieldy (lots of additional code & conversions).
Because of this we introduced a Validation trait, that handles validation of
section config data.

The UI allows users to easily create different types of fabrics. One can add
Nodes to the fabrics by selecting them from a dropdown which shows all the nodes
in the cluster. Additionally the user can then select the interfaces of the node
which should be added to the fabric. There are also protocol-specific options
such as "passive", "hello-interval" etc. available to select on the interface.
There are also options spanning whole fabrics: the "hello-interval" option on
openfabric for example, can be set on the fabric and will be applied to every
interface.


Refactoring
===========
This patch series required some rework of existing functionality, mostly how SDN
generates the FRR configuration and writes /etc/network/interfaces. Prior the
FRR configuration was generated exclusively from the controllers, but fabrics
need to write it as well. Same goes for the interfaces file, which got written
by the Zone plugin, but Fabrics need to write this file as well.

For this we moved the FRR and ifupdown config generation one level up to the SDN
module, which now calls into the respective child modules to generate the FRR /
ifupdown configuration.


Dependencies
============
pve-manager depends on pve-docs
pve-manager depends on pve-network
pve-network depends on proxmox-perl-rs
pve-network depends on pve-cluster
proxmox-perl-rs depends on proxmox-ve-config
proxmox-perl-rs depends on proxmox-frr
proxmox-perl-rs depends on proxmox-network-types
proxmox-ve-config depends on proxmox-frr
proxmox-ve-config depends on proxmox-network-types
proxmox-frr depends on proxmox-network-types
proxmox-ve-config depends on proxmox-serde
proxmox-firewall depends on proxmox-network-types

Changelog v1:
=============
proxmox-ve-rs
-------------
 * remove intermediate-config, convert section-config directly to frr-types.
 * add validation layer to validate the section-config
 * simplify openfabric `net` to `router-id`
 * add loopback prefixes to ensure that all router-ids are in a specific subnet
 * generate router-map and access-lists to rewrite the source address of all
   the routes received through openfabric and ospf
 * add integration tests
 * add option for ospf unnumbered
 * only allow ipv4 on ospf

pve-network
-------------
 * rework frr config generation
 * rework etc/network/interfaces config generation
 * revert "return loopback interface"

proxmox-perl-rs
-------------
 * generate /etc/network/interfaces config to set ip-addresses
 * auto-generate dummy interface for every fabric

pve-manager
-------------
 * simplify a lot
 * remove interface entries in tree
 * hide specific openfabric/ospf options (hello-interval, passive etc.)

frr (external)
--------------
 * fix --dummy_as_loopback bug (already on staging)

RFC
===
Changelog v2:
=============
proxmox-ve-rs
-------------
 * serialize internal representation directly to the frr format 
 * add integration tests to proxmox-frr
 * change internal representation to use BTreeMap instead of HashMap (so that
   the test output is ordered)
 * move some stuff from proxmox-frr and proxmox-ve-config to proxmox-network-types

pve-network
-----------
 * generate frr config and append to running config directly (without going
   through perl frr merging)
 * check permissions on each fabric when listing

pve-manager
-----------
 * autogenerate net and router-id when selecting the first interface

pve-cluster
-----------
 * update the config files in status.c (pve-cluster) (thanks @Thomas)

frr (external)
--------------
 * got this one merged: https://github.com/FRRouting/frr/pull/18242, so we
   *could* automatically add dummy interfaces


Big thanks to Stefan Hanreich for his help and support throughout this series!

proxmox:

Gabriel Goller (1):
  serde: add string_as_bool module for boolean string parsing

 proxmox-serde/src/lib.rs | 84 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 84 insertions(+)


proxmox-ve-rs:

Gabriel Goller (17):
  add proxmox-network-types crate
  network-types: add common hostname and openfabric types
  network-types: add openfabric NET type
  network-types: move Ipv4Cidr and Ipv6Cidr types
  frr: create proxmox-frr crate
  frr: add common frr types
  frr: add openfabric types
  frr: add ospf types
  frr: add route-map types
  frr: add generic types over openfabric and ospf
  frr: add serializer for all FRR types
  ve-config: add openfabric section-config
  ve-config: add ospf section-config
  ve-config: add FRR conversion helpers for openfabric and ospf
  ve-config: add validation for section-config
  ve-config: add section-config to frr types conversion
  ve-config: add integrations tests

 .gitignore                                    |   1 +
 Cargo.toml                                    |   7 +
 proxmox-frr/Cargo.toml                        |  22 +
 proxmox-frr/debian/changelog                  |   5 +
 proxmox-frr/debian/control                    |  47 ++
 proxmox-frr/debian/copyright                  |  18 +
 proxmox-frr/debian/debcargo.toml              |   7 +
 proxmox-frr/src/lib.rs                        | 218 +++++++++
 proxmox-frr/src/openfabric.rs                 |  93 ++++
 proxmox-frr/src/ospf.rs                       | 135 +++++
 proxmox-frr/src/route_map.rs                  | 128 +++++
 proxmox-frr/src/serializer.rs                 | 192 ++++++++
 proxmox-network-types/Cargo.toml              |  15 +
 proxmox-network-types/debian/changelog        |   5 +
 proxmox-network-types/debian/control          |  41 ++
 proxmox-network-types/debian/copyright        |  18 +
 proxmox-network-types/debian/debcargo.toml    |   7 +
 .../src}/address.rs                           |  14 +-
 proxmox-network-types/src/hostname.rs         |  34 ++
 proxmox-network-types/src/lib.rs              |   4 +
 proxmox-network-types/src/net.rs              | 382 +++++++++++++++
 proxmox-network-types/src/openfabric.rs       |  89 ++++
 proxmox-ve-config/Cargo.toml                  |  17 +-
 proxmox-ve-config/debian/control              |  37 +-
 proxmox-ve-config/src/firewall/cluster.rs     |   3 +-
 proxmox-ve-config/src/firewall/ct_helper.rs   |   8 +-
 proxmox-ve-config/src/firewall/host.rs        |   3 +-
 proxmox-ve-config/src/firewall/types/alias.rs |   3 +-
 proxmox-ve-config/src/firewall/types/ipset.rs |  26 +-
 proxmox-ve-config/src/firewall/types/mod.rs   |   2 -
 proxmox-ve-config/src/firewall/types/rule.rs  |   3 +-
 .../src/firewall/types/rule_match.rs          |   5 +-
 proxmox-ve-config/src/guest/vm.rs             |   4 +-
 proxmox-ve-config/src/host/utils.rs           |   2 +-
 proxmox-ve-config/src/sdn/config.rs           |   8 +-
 proxmox-ve-config/src/sdn/fabric/mod.rs       | 460 ++++++++++++++++++
 .../src/sdn/fabric/openfabric/frr.rs          |  24 +
 .../src/sdn/fabric/openfabric/mod.rs          | 291 +++++++++++
 .../src/sdn/fabric/openfabric/validation.rs   |  56 +++
 proxmox-ve-config/src/sdn/fabric/ospf/frr.rs  |  29 ++
 proxmox-ve-config/src/sdn/fabric/ospf/mod.rs  | 245 ++++++++++
 .../src/sdn/fabric/ospf/validation.rs         |  53 ++
 proxmox-ve-config/src/sdn/ipam.rs             |   4 +-
 proxmox-ve-config/src/sdn/mod.rs              |   3 +-
 proxmox-ve-config/tests/fabric/helper.rs      |  44 ++
 proxmox-ve-config/tests/fabric/main.rs        |  80 +++
 .../resources/cfg/openfabric_default.cfg      |  17 +
 .../cfg/openfabric_verification_fail.cfg      |  11 +
 .../fabric/resources/cfg/ospf_default.cfg     |  10 +
 .../resources/cfg/ospf_verification_fail.cfg  |  11 +
 .../resources/frr/openfabric_default.pve.frr  |  32 ++
 .../resources/frr/openfabric_default.pve1.frr |  28 ++
 .../fabric/resources/frr/ospf_default.pve.frr |  26 +
 .../resources/frr/ospf_default.pve1.frr       |  21 +
 proxmox-ve-config/tests/sdn/main.rs           |   3 +-
 55 files changed, 3011 insertions(+), 40 deletions(-)
 create mode 100644 proxmox-frr/Cargo.toml
 create mode 100644 proxmox-frr/debian/changelog
 create mode 100644 proxmox-frr/debian/control
 create mode 100644 proxmox-frr/debian/copyright
 create mode 100644 proxmox-frr/debian/debcargo.toml
 create mode 100644 proxmox-frr/src/lib.rs
 create mode 100644 proxmox-frr/src/openfabric.rs
 create mode 100644 proxmox-frr/src/ospf.rs
 create mode 100644 proxmox-frr/src/route_map.rs
 create mode 100644 proxmox-frr/src/serializer.rs
 create mode 100644 proxmox-network-types/Cargo.toml
 create mode 100644 proxmox-network-types/debian/changelog
 create mode 100644 proxmox-network-types/debian/control
 create mode 100644 proxmox-network-types/debian/copyright
 create mode 100644 proxmox-network-types/debian/debcargo.toml
 rename {proxmox-ve-config/src/firewall/types => proxmox-network-types/src}/address.rs (99%)
 create mode 100644 proxmox-network-types/src/hostname.rs
 create mode 100644 proxmox-network-types/src/lib.rs
 create mode 100644 proxmox-network-types/src/net.rs
 create mode 100644 proxmox-network-types/src/openfabric.rs
 create mode 100644 proxmox-ve-config/src/sdn/fabric/mod.rs
 create mode 100644 proxmox-ve-config/src/sdn/fabric/openfabric/frr.rs
 create mode 100644 proxmox-ve-config/src/sdn/fabric/openfabric/mod.rs
 create mode 100644 proxmox-ve-config/src/sdn/fabric/openfabric/validation.rs
 create mode 100644 proxmox-ve-config/src/sdn/fabric/ospf/frr.rs
 create mode 100644 proxmox-ve-config/src/sdn/fabric/ospf/mod.rs
 create mode 100644 proxmox-ve-config/src/sdn/fabric/ospf/validation.rs
 create mode 100644 proxmox-ve-config/tests/fabric/helper.rs
 create mode 100644 proxmox-ve-config/tests/fabric/main.rs
 create mode 100644 proxmox-ve-config/tests/fabric/resources/cfg/openfabric_default.cfg
 create mode 100644 proxmox-ve-config/tests/fabric/resources/cfg/openfabric_verification_fail.cfg
 create mode 100644 proxmox-ve-config/tests/fabric/resources/cfg/ospf_default.cfg
 create mode 100644 proxmox-ve-config/tests/fabric/resources/cfg/ospf_verification_fail.cfg
 create mode 100644 proxmox-ve-config/tests/fabric/resources/frr/openfabric_default.pve.frr
 create mode 100644 proxmox-ve-config/tests/fabric/resources/frr/openfabric_default.pve1.frr
 create mode 100644 proxmox-ve-config/tests/fabric/resources/frr/ospf_default.pve.frr
 create mode 100644 proxmox-ve-config/tests/fabric/resources/frr/ospf_default.pve1.frr


proxmox-firewall:

Stefan Hanreich (1):
  firewall: nftables: migrate to proxmox-network-types

 Cargo.toml                         | 1 +
 proxmox-firewall/Cargo.toml        | 1 +
 proxmox-firewall/src/firewall.rs   | 2 +-
 proxmox-firewall/src/object.rs     | 4 +++-
 proxmox-firewall/src/rule.rs       | 3 ++-
 proxmox-nftables/Cargo.toml        | 3 ++-
 proxmox-nftables/src/expression.rs | 5 +----
 proxmox-nftables/src/types.rs      | 2 +-
 8 files changed, 12 insertions(+), 9 deletions(-)


proxmox-perl-rs:

Gabriel Goller (7):
  perl-rs: sdn: initial fabric infrastructure
  perl-rs: sdn: add CRUD helpers for OpenFabric fabric management
  perl-rs: sdn: OpenFabric perlmod methods
  perl-rs: sdn: implement OSPF interface file configuration generation
  perl-rs: sdn: add CRUD helpers for OSPF fabric management
  perl-rs: sdn: OSPF perlmod methods
  perl-rs: sdn: implement OSPF interface file configuration generation

 pve-rs/Cargo.toml            |   6 +-
 pve-rs/Makefile              |   3 +
 pve-rs/src/lib.rs            |   1 +
 pve-rs/src/sdn/fabrics.rs    |  50 ++++
 pve-rs/src/sdn/mod.rs        |   3 +
 pve-rs/src/sdn/openfabric.rs | 462 +++++++++++++++++++++++++++++++++++
 pve-rs/src/sdn/ospf.rs       | 427 ++++++++++++++++++++++++++++++++
 7 files changed, 951 insertions(+), 1 deletion(-)
 create mode 100644 pve-rs/src/sdn/fabrics.rs
 create mode 100644 pve-rs/src/sdn/mod.rs
 create mode 100644 pve-rs/src/sdn/openfabric.rs
 create mode 100644 pve-rs/src/sdn/ospf.rs


pve-cluster:

Gabriel Goller (1):
  cluster: add sdn fabrics config files

 src/PVE/Cluster.pm  | 2 ++
 src/pmxcfs/status.c | 2 ++
 2 files changed, 4 insertions(+)


pve-network:

Gabriel Goller (1):
  debian: add dependency to proxmox-perl-rs

Stefan Hanreich (16):
  sdn: fix value returned by pending_config
  fabrics: add fabrics module
  refactor: controller: move frr methods into helper
  controllers: implement new api for frr config generation
  sdn: add frr config generation helper
  test: isis: add test for standalone configuration
  sdn: frr: add daemon status to frr helper
  sdn: running: apply fabrics config
  fabrics: generate ifupdown configuration
  api: add fabrics subfolder
  api: fabrics: add common helpers
  fabric: openfabric: add api endpoints
  fabric: ospf: add api endpoints
  test: fabrics: add test cases for ospf and openfabric + evpn
  frr: bump frr config version to 10.2.1
  frr: fix reloading frr configuration

 debian/control                                |   2 +
 src/PVE/API2/Network/SDN.pm                   |   7 +
 src/PVE/API2/Network/SDN/Fabrics.pm           | 294 ++++++++++++
 src/PVE/API2/Network/SDN/Fabrics/Common.pm    |  80 ++++
 src/PVE/API2/Network/SDN/Fabrics/Makefile     |   9 +
 .../API2/Network/SDN/Fabrics/OpenFabric.pm    | 348 +++++++++++++++
 src/PVE/API2/Network/SDN/Fabrics/Ospf.pm      | 345 ++++++++++++++
 src/PVE/API2/Network/SDN/Makefile             |   3 +-
 src/PVE/Network/SDN.pm                        | 138 +++++-
 src/PVE/Network/SDN/Controllers.pm            |  67 +--
 src/PVE/Network/SDN/Controllers/BgpPlugin.pm  |  21 +-
 src/PVE/Network/SDN/Controllers/EvpnPlugin.pm | 295 +-----------
 src/PVE/Network/SDN/Controllers/IsisPlugin.pm |  21 +-
 src/PVE/Network/SDN/Controllers/Plugin.pm     |  31 +-
 src/PVE/Network/SDN/Fabrics.pm                | 130 ++++++
 src/PVE/Network/SDN/Frr.pm                    | 420 ++++++++++++++++++
 src/PVE/Network/SDN/Makefile                  |   2 +-
 src/PVE/Network/SDN/Zones.pm                  |  10 -
 src/test/run_test_zones.pl                    |  11 +-
 .../expected_controller_config                |   2 +-
 .../expected_controller_config                |   2 +-
 .../evpn/ebgp/expected_controller_config      |   2 +-
 .../ebgp_loopback/expected_controller_config  |   2 +-
 .../evpn/exitnode/expected_controller_config  |   2 +-
 .../expected_controller_config                |   2 +-
 .../expected_controller_config                |   2 +-
 .../exitnode_snat/expected_controller_config  |   2 +-
 .../expected_controller_config                |   2 +-
 .../evpn/ipv4/expected_controller_config      |   2 +-
 .../evpn/ipv4ipv6/expected_controller_config  |   2 +-
 .../expected_controller_config                |   2 +-
 .../evpn/ipv6/expected_controller_config      |   2 +-
 .../ipv6underlay/expected_controller_config   |   2 +-
 .../evpn/isis/expected_controller_config      |   2 +-
 .../isis_loopback/expected_controller_config  |   2 +-
 .../expected_controller_config                |  22 +
 .../isis_standalone/expected_sdn_interfaces   |   1 +
 .../zones/evpn/isis_standalone/interfaces     |  12 +
 .../zones/evpn/isis_standalone/sdn_config     |  21 +
 .../expected_controller_config                |   2 +-
 .../multiplezones/expected_controller_config  |   2 +-
 .../expected_controller_config                |  72 +++
 .../openfabric_fabric/expected_sdn_interfaces |  56 +++
 .../zones/evpn/openfabric_fabric/interfaces   |   6 +
 .../zones/evpn/openfabric_fabric/sdn_config   |  79 ++++
 .../ospf_fabric/expected_controller_config    |  66 +++
 .../evpn/ospf_fabric/expected_sdn_interfaces  |  53 +++
 src/test/zones/evpn/ospf_fabric/interfaces    |   6 +
 src/test/zones/evpn/ospf_fabric/sdn_config    |  75 ++++
 .../evpn/rt_import/expected_controller_config |   2 +-
 .../evpn/vxlanport/expected_controller_config |   2 +-
 51 files changed, 2270 insertions(+), 473 deletions(-)
 create mode 100644 src/PVE/API2/Network/SDN/Fabrics.pm
 create mode 100644 src/PVE/API2/Network/SDN/Fabrics/Common.pm
 create mode 100644 src/PVE/API2/Network/SDN/Fabrics/Makefile
 create mode 100644 src/PVE/API2/Network/SDN/Fabrics/OpenFabric.pm
 create mode 100644 src/PVE/API2/Network/SDN/Fabrics/Ospf.pm
 create mode 100644 src/PVE/Network/SDN/Fabrics.pm
 create mode 100644 src/PVE/Network/SDN/Frr.pm
 create mode 100644 src/test/zones/evpn/isis_standalone/expected_controller_config
 create mode 100644 src/test/zones/evpn/isis_standalone/expected_sdn_interfaces
 create mode 100644 src/test/zones/evpn/isis_standalone/interfaces
 create mode 100644 src/test/zones/evpn/isis_standalone/sdn_config
 create mode 100644 src/test/zones/evpn/openfabric_fabric/expected_controller_config
 create mode 100644 src/test/zones/evpn/openfabric_fabric/expected_sdn_interfaces
 create mode 100644 src/test/zones/evpn/openfabric_fabric/interfaces
 create mode 100644 src/test/zones/evpn/openfabric_fabric/sdn_config
 create mode 100644 src/test/zones/evpn/ospf_fabric/expected_controller_config
 create mode 100644 src/test/zones/evpn/ospf_fabric/expected_sdn_interfaces
 create mode 100644 src/test/zones/evpn/ospf_fabric/interfaces
 create mode 100644 src/test/zones/evpn/ospf_fabric/sdn_config


pve-manager:

Gabriel Goller (7):
  api: use new generalized frr and etc network config helper functions
  fabrics: add common interface panel
  fabrics: add additional interface fields for openfabric and ospf
  fabrics: add FabricEdit components
  fabrics: add NodeEdit components
  fabrics: Add main FabricView
  utils: avoid line-break in pending changes message

 PVE/API2/Network.pm                           |   6 +-
 www/manager6/Makefile                         |   8 +
 www/manager6/Utils.js                         |   2 +-
 www/manager6/dc/Config.js                     |   8 +
 www/manager6/sdn/FabricsView.js               | 430 ++++++++++++++++++
 www/manager6/sdn/fabrics/Common.js            | 285 ++++++++++++
 .../sdn/fabrics/openfabric/FabricEdit.js      |  71 +++
 .../sdn/fabrics/openfabric/InterfaceEdit.js   |  64 +++
 .../sdn/fabrics/openfabric/NodeEdit.js        | 205 +++++++++
 www/manager6/sdn/fabrics/ospf/FabricEdit.js   |  64 +++
 .../sdn/fabrics/ospf/InterfaceEdit.js         |  27 ++
 www/manager6/sdn/fabrics/ospf/NodeEdit.js     | 207 +++++++++
 12 files changed, 1374 insertions(+), 3 deletions(-)
 create mode 100644 www/manager6/sdn/FabricsView.js
 create mode 100644 www/manager6/sdn/fabrics/Common.js
 create mode 100644 www/manager6/sdn/fabrics/openfabric/FabricEdit.js
 create mode 100644 www/manager6/sdn/fabrics/openfabric/InterfaceEdit.js
 create mode 100644 www/manager6/sdn/fabrics/openfabric/NodeEdit.js
 create mode 100644 www/manager6/sdn/fabrics/ospf/FabricEdit.js
 create mode 100644 www/manager6/sdn/fabrics/ospf/InterfaceEdit.js
 create mode 100644 www/manager6/sdn/fabrics/ospf/NodeEdit.js


pve-docs:

Gabriel Goller (1):
  fabrics: add initial documentation for sdn fabrics

 pvesdn.adoc | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 155 insertions(+)


Summary over all repositories:
  137 files changed, 7861 insertions(+), 526 deletions(-)

-- 
Generated by git-murpp 0.8.0


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel