From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <pve-devel-bounces@lists.proxmox.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 1E5801FF164 for <inbox@lore.proxmox.com>; Fri, 28 Mar 2025 18:15:51 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 213D28ACF; Fri, 28 Mar 2025 18:14:08 +0100 (CET) From: Gabriel Goller <g.goller@proxmox.com> To: pve-devel@lists.proxmox.com Date: Fri, 28 Mar 2025 18:12:48 +0100 Message-Id: <20250328171340.885413-1-g.goller@proxmox.com> X-Mailer: git-send-email 2.39.5 MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.025 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pve-devel] [PATCH cluster/docs/manager/network/proxmox{, -ve-rs, -firewall, -perl-rs} 00/52] Add SDN Fabrics X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/> List-Post: <mailto:pve-devel@lists.proxmox.com> List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe> Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com> Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com> This series allows the user to add fabrics such as OpenFabric and OSPF over their clusters. Overview ======== This series allows the user to create routed networks ('fabrics') across their clusters, which can be used as the underlay network for a EVPN cluster, or for creating Ceph full mesh clusters easily. This patch series adds the initial support for two routing protocols: * OpenFabric * OSPF In the future we plan on moving the existing IS-IS and BGP controllers into the fabric structure. There are also plans for adding a new Wireguard fabric to this. Implementation ============== Every fabric consists of one or more nodes, which themselves consists of one or more interfaces. Fabrics and nodes are modeled as different section config types, interfaces are an array contained in a node section. We have a separate configuration file for each fabric type. This is because the basic structure (fabric, node, interface) is the same, but the specific options vary wildly. This makes serialization / validation from the Rust side a lot easier. For now, nodes in the fabric configuration are always PVE nodes, but in the future nodes could also represent external members of the fabric (e.g. in a potential wireguard fabric). Settings can be configured on a fabric-level, so they apply to all interfaces, or overridden on a interface-level (hidden in the UI by default). Most of the functionality is implemented by rust and exposed to the existing SDN module via perlmod. This includes configuration reading / writing, FRR config generation from the section config and API CRUD methods. The API provides one common GET method, to get the configuration of all different fabric types (for the tree overview), but otherwise there are separate CRUD endpoints for every fabric type, to mimic the split of configuration files. Another upside of this is, that the generated rust structs for the API endpoints (for PDM) will be much nicer. For the FRR-specific functionality we introduced a new proxmox-frr crate that models the different entities in the FRR configuration format (routers, interfaces, route-maps, ...) and provides serializers for those structs. For more information see the respective FRR commits. When applying the SDN configuration, perl calls into perlmod to utilize the proxmox-frr crate for generating the FRR configuration of the fabrics. We also introduce a proxmox-network-types crate, where we extracted generic network types (Cidr for instance), so we can reuse them across multiple crates (proxmox-frr, proxmox-firewall, proxmox-ve-config, ..). The hierarchical nature of the configuration and the relationship between nodes inside the fabrics requires validation of sections relative to other sections. For this matter we introduced an intermediate configuration in the initial RFC, but that turned out to be unwieldy (lots of additional code & conversions). Because of this we introduced a Validation trait, that handles validation of section config data. The UI allows users to easily create different types of fabrics. One can add Nodes to the fabrics by selecting them from a dropdown which shows all the nodes in the cluster. Additionally the user can then select the interfaces of the node which should be added to the fabric. There are also protocol-specific options such as "passive", "hello-interval" etc. available to select on the interface. There are also options spanning whole fabrics: the "hello-interval" option on openfabric for example, can be set on the fabric and will be applied to every interface. Refactoring =========== This patch series required some rework of existing functionality, mostly how SDN generates the FRR configuration and writes /etc/network/interfaces. Prior the FRR configuration was generated exclusively from the controllers, but fabrics need to write it as well. Same goes for the interfaces file, which got written by the Zone plugin, but Fabrics need to write this file as well. For this we moved the FRR and ifupdown config generation one level up to the SDN module, which now calls into the respective child modules to generate the FRR / ifupdown configuration. Dependencies ============ pve-manager depends on pve-docs pve-manager depends on pve-network pve-network depends on proxmox-perl-rs pve-network depends on pve-cluster proxmox-perl-rs depends on proxmox-ve-config proxmox-perl-rs depends on proxmox-frr proxmox-perl-rs depends on proxmox-network-types proxmox-ve-config depends on proxmox-frr proxmox-ve-config depends on proxmox-network-types proxmox-frr depends on proxmox-network-types proxmox-ve-config depends on proxmox-serde proxmox-firewall depends on proxmox-network-types Changelog v1: ============= proxmox-ve-rs ------------- * remove intermediate-config, convert section-config directly to frr-types. * add validation layer to validate the section-config * simplify openfabric `net` to `router-id` * add loopback prefixes to ensure that all router-ids are in a specific subnet * generate router-map and access-lists to rewrite the source address of all the routes received through openfabric and ospf * add integration tests * add option for ospf unnumbered * only allow ipv4 on ospf pve-network ------------- * rework frr config generation * rework etc/network/interfaces config generation * revert "return loopback interface" proxmox-perl-rs ------------- * generate /etc/network/interfaces config to set ip-addresses * auto-generate dummy interface for every fabric pve-manager ------------- * simplify a lot * remove interface entries in tree * hide specific openfabric/ospf options (hello-interval, passive etc.) frr (external) -------------- * fix --dummy_as_loopback bug (already on staging) RFC === Changelog v2: ============= proxmox-ve-rs ------------- * serialize internal representation directly to the frr format * add integration tests to proxmox-frr * change internal representation to use BTreeMap instead of HashMap (so that the test output is ordered) * move some stuff from proxmox-frr and proxmox-ve-config to proxmox-network-types pve-network ----------- * generate frr config and append to running config directly (without going through perl frr merging) * check permissions on each fabric when listing pve-manager ----------- * autogenerate net and router-id when selecting the first interface pve-cluster ----------- * update the config files in status.c (pve-cluster) (thanks @Thomas) frr (external) -------------- * got this one merged: https://github.com/FRRouting/frr/pull/18242, so we *could* automatically add dummy interfaces Big thanks to Stefan Hanreich for his help and support throughout this series! proxmox: Gabriel Goller (1): serde: add string_as_bool module for boolean string parsing proxmox-serde/src/lib.rs | 84 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) proxmox-ve-rs: Gabriel Goller (17): add proxmox-network-types crate network-types: add common hostname and openfabric types network-types: add openfabric NET type network-types: move Ipv4Cidr and Ipv6Cidr types frr: create proxmox-frr crate frr: add common frr types frr: add openfabric types frr: add ospf types frr: add route-map types frr: add generic types over openfabric and ospf frr: add serializer for all FRR types ve-config: add openfabric section-config ve-config: add ospf section-config ve-config: add FRR conversion helpers for openfabric and ospf ve-config: add validation for section-config ve-config: add section-config to frr types conversion ve-config: add integrations tests .gitignore | 1 + Cargo.toml | 7 + proxmox-frr/Cargo.toml | 22 + proxmox-frr/debian/changelog | 5 + proxmox-frr/debian/control | 47 ++ proxmox-frr/debian/copyright | 18 + proxmox-frr/debian/debcargo.toml | 7 + proxmox-frr/src/lib.rs | 218 +++++++++ proxmox-frr/src/openfabric.rs | 93 ++++ proxmox-frr/src/ospf.rs | 135 +++++ proxmox-frr/src/route_map.rs | 128 +++++ proxmox-frr/src/serializer.rs | 192 ++++++++ proxmox-network-types/Cargo.toml | 15 + proxmox-network-types/debian/changelog | 5 + proxmox-network-types/debian/control | 41 ++ proxmox-network-types/debian/copyright | 18 + proxmox-network-types/debian/debcargo.toml | 7 + .../src}/address.rs | 14 +- proxmox-network-types/src/hostname.rs | 34 ++ proxmox-network-types/src/lib.rs | 4 + proxmox-network-types/src/net.rs | 382 +++++++++++++++ proxmox-network-types/src/openfabric.rs | 89 ++++ proxmox-ve-config/Cargo.toml | 17 +- proxmox-ve-config/debian/control | 37 +- proxmox-ve-config/src/firewall/cluster.rs | 3 +- proxmox-ve-config/src/firewall/ct_helper.rs | 8 +- proxmox-ve-config/src/firewall/host.rs | 3 +- proxmox-ve-config/src/firewall/types/alias.rs | 3 +- proxmox-ve-config/src/firewall/types/ipset.rs | 26 +- proxmox-ve-config/src/firewall/types/mod.rs | 2 - proxmox-ve-config/src/firewall/types/rule.rs | 3 +- .../src/firewall/types/rule_match.rs | 5 +- proxmox-ve-config/src/guest/vm.rs | 4 +- proxmox-ve-config/src/host/utils.rs | 2 +- proxmox-ve-config/src/sdn/config.rs | 8 +- proxmox-ve-config/src/sdn/fabric/mod.rs | 460 ++++++++++++++++++ .../src/sdn/fabric/openfabric/frr.rs | 24 + .../src/sdn/fabric/openfabric/mod.rs | 291 +++++++++++ .../src/sdn/fabric/openfabric/validation.rs | 56 +++ proxmox-ve-config/src/sdn/fabric/ospf/frr.rs | 29 ++ proxmox-ve-config/src/sdn/fabric/ospf/mod.rs | 245 ++++++++++ .../src/sdn/fabric/ospf/validation.rs | 53 ++ proxmox-ve-config/src/sdn/ipam.rs | 4 +- proxmox-ve-config/src/sdn/mod.rs | 3 +- proxmox-ve-config/tests/fabric/helper.rs | 44 ++ proxmox-ve-config/tests/fabric/main.rs | 80 +++ .../resources/cfg/openfabric_default.cfg | 17 + .../cfg/openfabric_verification_fail.cfg | 11 + .../fabric/resources/cfg/ospf_default.cfg | 10 + .../resources/cfg/ospf_verification_fail.cfg | 11 + .../resources/frr/openfabric_default.pve.frr | 32 ++ .../resources/frr/openfabric_default.pve1.frr | 28 ++ .../fabric/resources/frr/ospf_default.pve.frr | 26 + .../resources/frr/ospf_default.pve1.frr | 21 + proxmox-ve-config/tests/sdn/main.rs | 3 +- 55 files changed, 3011 insertions(+), 40 deletions(-) create mode 100644 proxmox-frr/Cargo.toml create mode 100644 proxmox-frr/debian/changelog create mode 100644 proxmox-frr/debian/control create mode 100644 proxmox-frr/debian/copyright create mode 100644 proxmox-frr/debian/debcargo.toml create mode 100644 proxmox-frr/src/lib.rs create mode 100644 proxmox-frr/src/openfabric.rs create mode 100644 proxmox-frr/src/ospf.rs create mode 100644 proxmox-frr/src/route_map.rs create mode 100644 proxmox-frr/src/serializer.rs create mode 100644 proxmox-network-types/Cargo.toml create mode 100644 proxmox-network-types/debian/changelog create mode 100644 proxmox-network-types/debian/control create mode 100644 proxmox-network-types/debian/copyright create mode 100644 proxmox-network-types/debian/debcargo.toml rename {proxmox-ve-config/src/firewall/types => proxmox-network-types/src}/address.rs (99%) create mode 100644 proxmox-network-types/src/hostname.rs create mode 100644 proxmox-network-types/src/lib.rs create mode 100644 proxmox-network-types/src/net.rs create mode 100644 proxmox-network-types/src/openfabric.rs create mode 100644 proxmox-ve-config/src/sdn/fabric/mod.rs create mode 100644 proxmox-ve-config/src/sdn/fabric/openfabric/frr.rs create mode 100644 proxmox-ve-config/src/sdn/fabric/openfabric/mod.rs create mode 100644 proxmox-ve-config/src/sdn/fabric/openfabric/validation.rs create mode 100644 proxmox-ve-config/src/sdn/fabric/ospf/frr.rs create mode 100644 proxmox-ve-config/src/sdn/fabric/ospf/mod.rs create mode 100644 proxmox-ve-config/src/sdn/fabric/ospf/validation.rs create mode 100644 proxmox-ve-config/tests/fabric/helper.rs create mode 100644 proxmox-ve-config/tests/fabric/main.rs create mode 100644 proxmox-ve-config/tests/fabric/resources/cfg/openfabric_default.cfg create mode 100644 proxmox-ve-config/tests/fabric/resources/cfg/openfabric_verification_fail.cfg create mode 100644 proxmox-ve-config/tests/fabric/resources/cfg/ospf_default.cfg create mode 100644 proxmox-ve-config/tests/fabric/resources/cfg/ospf_verification_fail.cfg create mode 100644 proxmox-ve-config/tests/fabric/resources/frr/openfabric_default.pve.frr create mode 100644 proxmox-ve-config/tests/fabric/resources/frr/openfabric_default.pve1.frr create mode 100644 proxmox-ve-config/tests/fabric/resources/frr/ospf_default.pve.frr create mode 100644 proxmox-ve-config/tests/fabric/resources/frr/ospf_default.pve1.frr proxmox-firewall: Stefan Hanreich (1): firewall: nftables: migrate to proxmox-network-types Cargo.toml | 1 + proxmox-firewall/Cargo.toml | 1 + proxmox-firewall/src/firewall.rs | 2 +- proxmox-firewall/src/object.rs | 4 +++- proxmox-firewall/src/rule.rs | 3 ++- proxmox-nftables/Cargo.toml | 3 ++- proxmox-nftables/src/expression.rs | 5 +---- proxmox-nftables/src/types.rs | 2 +- 8 files changed, 12 insertions(+), 9 deletions(-) proxmox-perl-rs: Gabriel Goller (7): perl-rs: sdn: initial fabric infrastructure perl-rs: sdn: add CRUD helpers for OpenFabric fabric management perl-rs: sdn: OpenFabric perlmod methods perl-rs: sdn: implement OSPF interface file configuration generation perl-rs: sdn: add CRUD helpers for OSPF fabric management perl-rs: sdn: OSPF perlmod methods perl-rs: sdn: implement OSPF interface file configuration generation pve-rs/Cargo.toml | 6 +- pve-rs/Makefile | 3 + pve-rs/src/lib.rs | 1 + pve-rs/src/sdn/fabrics.rs | 50 ++++ pve-rs/src/sdn/mod.rs | 3 + pve-rs/src/sdn/openfabric.rs | 462 +++++++++++++++++++++++++++++++++++ pve-rs/src/sdn/ospf.rs | 427 ++++++++++++++++++++++++++++++++ 7 files changed, 951 insertions(+), 1 deletion(-) create mode 100644 pve-rs/src/sdn/fabrics.rs create mode 100644 pve-rs/src/sdn/mod.rs create mode 100644 pve-rs/src/sdn/openfabric.rs create mode 100644 pve-rs/src/sdn/ospf.rs pve-cluster: Gabriel Goller (1): cluster: add sdn fabrics config files src/PVE/Cluster.pm | 2 ++ src/pmxcfs/status.c | 2 ++ 2 files changed, 4 insertions(+) pve-network: Gabriel Goller (1): debian: add dependency to proxmox-perl-rs Stefan Hanreich (16): sdn: fix value returned by pending_config fabrics: add fabrics module refactor: controller: move frr methods into helper controllers: implement new api for frr config generation sdn: add frr config generation helper test: isis: add test for standalone configuration sdn: frr: add daemon status to frr helper sdn: running: apply fabrics config fabrics: generate ifupdown configuration api: add fabrics subfolder api: fabrics: add common helpers fabric: openfabric: add api endpoints fabric: ospf: add api endpoints test: fabrics: add test cases for ospf and openfabric + evpn frr: bump frr config version to 10.2.1 frr: fix reloading frr configuration debian/control | 2 + src/PVE/API2/Network/SDN.pm | 7 + src/PVE/API2/Network/SDN/Fabrics.pm | 294 ++++++++++++ src/PVE/API2/Network/SDN/Fabrics/Common.pm | 80 ++++ src/PVE/API2/Network/SDN/Fabrics/Makefile | 9 + .../API2/Network/SDN/Fabrics/OpenFabric.pm | 348 +++++++++++++++ src/PVE/API2/Network/SDN/Fabrics/Ospf.pm | 345 ++++++++++++++ src/PVE/API2/Network/SDN/Makefile | 3 +- src/PVE/Network/SDN.pm | 138 +++++- src/PVE/Network/SDN/Controllers.pm | 67 +-- src/PVE/Network/SDN/Controllers/BgpPlugin.pm | 21 +- src/PVE/Network/SDN/Controllers/EvpnPlugin.pm | 295 +----------- src/PVE/Network/SDN/Controllers/IsisPlugin.pm | 21 +- src/PVE/Network/SDN/Controllers/Plugin.pm | 31 +- src/PVE/Network/SDN/Fabrics.pm | 130 ++++++ src/PVE/Network/SDN/Frr.pm | 420 ++++++++++++++++++ src/PVE/Network/SDN/Makefile | 2 +- src/PVE/Network/SDN/Zones.pm | 10 - src/test/run_test_zones.pl | 11 +- .../expected_controller_config | 2 +- .../expected_controller_config | 2 +- .../evpn/ebgp/expected_controller_config | 2 +- .../ebgp_loopback/expected_controller_config | 2 +- .../evpn/exitnode/expected_controller_config | 2 +- .../expected_controller_config | 2 +- .../expected_controller_config | 2 +- .../exitnode_snat/expected_controller_config | 2 +- .../expected_controller_config | 2 +- .../evpn/ipv4/expected_controller_config | 2 +- .../evpn/ipv4ipv6/expected_controller_config | 2 +- .../expected_controller_config | 2 +- .../evpn/ipv6/expected_controller_config | 2 +- .../ipv6underlay/expected_controller_config | 2 +- .../evpn/isis/expected_controller_config | 2 +- .../isis_loopback/expected_controller_config | 2 +- .../expected_controller_config | 22 + .../isis_standalone/expected_sdn_interfaces | 1 + .../zones/evpn/isis_standalone/interfaces | 12 + .../zones/evpn/isis_standalone/sdn_config | 21 + .../expected_controller_config | 2 +- .../multiplezones/expected_controller_config | 2 +- .../expected_controller_config | 72 +++ .../openfabric_fabric/expected_sdn_interfaces | 56 +++ .../zones/evpn/openfabric_fabric/interfaces | 6 + .../zones/evpn/openfabric_fabric/sdn_config | 79 ++++ .../ospf_fabric/expected_controller_config | 66 +++ .../evpn/ospf_fabric/expected_sdn_interfaces | 53 +++ src/test/zones/evpn/ospf_fabric/interfaces | 6 + src/test/zones/evpn/ospf_fabric/sdn_config | 75 ++++ .../evpn/rt_import/expected_controller_config | 2 +- .../evpn/vxlanport/expected_controller_config | 2 +- 51 files changed, 2270 insertions(+), 473 deletions(-) create mode 100644 src/PVE/API2/Network/SDN/Fabrics.pm create mode 100644 src/PVE/API2/Network/SDN/Fabrics/Common.pm create mode 100644 src/PVE/API2/Network/SDN/Fabrics/Makefile create mode 100644 src/PVE/API2/Network/SDN/Fabrics/OpenFabric.pm create mode 100644 src/PVE/API2/Network/SDN/Fabrics/Ospf.pm create mode 100644 src/PVE/Network/SDN/Fabrics.pm create mode 100644 src/PVE/Network/SDN/Frr.pm create mode 100644 src/test/zones/evpn/isis_standalone/expected_controller_config create mode 100644 src/test/zones/evpn/isis_standalone/expected_sdn_interfaces create mode 100644 src/test/zones/evpn/isis_standalone/interfaces create mode 100644 src/test/zones/evpn/isis_standalone/sdn_config create mode 100644 src/test/zones/evpn/openfabric_fabric/expected_controller_config create mode 100644 src/test/zones/evpn/openfabric_fabric/expected_sdn_interfaces create mode 100644 src/test/zones/evpn/openfabric_fabric/interfaces create mode 100644 src/test/zones/evpn/openfabric_fabric/sdn_config create mode 100644 src/test/zones/evpn/ospf_fabric/expected_controller_config create mode 100644 src/test/zones/evpn/ospf_fabric/expected_sdn_interfaces create mode 100644 src/test/zones/evpn/ospf_fabric/interfaces create mode 100644 src/test/zones/evpn/ospf_fabric/sdn_config pve-manager: Gabriel Goller (7): api: use new generalized frr and etc network config helper functions fabrics: add common interface panel fabrics: add additional interface fields for openfabric and ospf fabrics: add FabricEdit components fabrics: add NodeEdit components fabrics: Add main FabricView utils: avoid line-break in pending changes message PVE/API2/Network.pm | 6 +- www/manager6/Makefile | 8 + www/manager6/Utils.js | 2 +- www/manager6/dc/Config.js | 8 + www/manager6/sdn/FabricsView.js | 430 ++++++++++++++++++ www/manager6/sdn/fabrics/Common.js | 285 ++++++++++++ .../sdn/fabrics/openfabric/FabricEdit.js | 71 +++ .../sdn/fabrics/openfabric/InterfaceEdit.js | 64 +++ .../sdn/fabrics/openfabric/NodeEdit.js | 205 +++++++++ www/manager6/sdn/fabrics/ospf/FabricEdit.js | 64 +++ .../sdn/fabrics/ospf/InterfaceEdit.js | 27 ++ www/manager6/sdn/fabrics/ospf/NodeEdit.js | 207 +++++++++ 12 files changed, 1374 insertions(+), 3 deletions(-) create mode 100644 www/manager6/sdn/FabricsView.js create mode 100644 www/manager6/sdn/fabrics/Common.js create mode 100644 www/manager6/sdn/fabrics/openfabric/FabricEdit.js create mode 100644 www/manager6/sdn/fabrics/openfabric/InterfaceEdit.js create mode 100644 www/manager6/sdn/fabrics/openfabric/NodeEdit.js create mode 100644 www/manager6/sdn/fabrics/ospf/FabricEdit.js create mode 100644 www/manager6/sdn/fabrics/ospf/InterfaceEdit.js create mode 100644 www/manager6/sdn/fabrics/ospf/NodeEdit.js pve-docs: Gabriel Goller (1): fabrics: add initial documentation for sdn fabrics pvesdn.adoc | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 155 insertions(+) Summary over all repositories: 137 files changed, 7861 insertions(+), 526 deletions(-) -- Generated by git-murpp 0.8.0 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel