From: Stefan Hanreich <s.hanreich@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: Re: [PATCH pve-network v3 16/32] sdn: allow fabrics to use simple zone VRFs
Date: Wed, 18 Nov 2026 17:27:57 +0100 [thread overview]
Message-ID: <26c8b2b0-e107-49d7-b761-bfd5b5cc8e38@proxmox.com> (raw)
In-Reply-To: <20261116161154.343743-17-g.goller@proxmox.com>
generally we should check in the API for where we should use the new _checked helpers
and which validate_property_references methods should get extended? Here are some,
and there are probably other in this series as well - please re-check this!
On 11/16/26 5:15 PM, Gabriel Goller wrote:
> Let BGP and OSPF fabrics use a simple zone with dedicated-vrf enabled
> to keep their routes separate from the default VRF.
>
> The zone must be enabled on every fabric node. Prevent zone changes
> or deletion that would leave a fabric without its VRF. A fabric's zone
> cannot be changed after creation, since that resets all its sessions
> anyway and other objects rely on the fabric staying in its VRF.
>
> Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
> ---
> src/PVE/API2/Network/SDN/Fabrics/Fabric.pm | 57 ++++++++++++++++---
> .../API2/Network/SDN/Fabrics/FabricNode.pm | 18 ++++++
> src/PVE/API2/Network/SDN/Zones.pm | 28 +++++++++
> src/PVE/Network/SDN/Controllers/BgpPlugin.pm | 2 +-
> src/PVE/Network/SDN/Fabrics.pm | 26 +++++++++
> 5 files changed, 122 insertions(+), 9 deletions(-)
>
> diff --git a/src/PVE/API2/Network/SDN/Fabrics/Fabric.pm b/src/PVE/API2/Network/SDN/Fabrics/Fabric.pm
> index bbc33e047154..20b7b1f98e4b 100644
> --- a/src/PVE/API2/Network/SDN/Fabrics/Fabric.pm
> +++ b/src/PVE/API2/Network/SDN/Fabrics/Fabric.pm
> @@ -6,13 +6,51 @@ use warnings;
> use PVE::Network::SDN;
> use PVE::Network::SDN::Controllers;
> use PVE::Network::SDN::Fabrics;
> +use PVE::Network::SDN::Zones;
>
> +use PVE::Exception qw(raise_param_exc);
> use PVE::JSONSchema qw(get_standard_option);
> use PVE::Tools qw(extract_param);
>
> use PVE::RESTHandler;
> use base qw(PVE::RESTHandler);
>
> +sub validate_fabric_zone {
> + my ($fabric, $config) = @_;
> +
> + my $protocol = $fabric->{protocol} // '';
> + if ($protocol eq 'bgp' && !$fabric->{zone}) {
> + my $controller_cfg = PVE::Network::SDN::Controllers::config();
> + for my $controller_id (keys %{ $controller_cfg->{ids} // {} }) {
> + my $controller = $controller_cfg->{ids}->{$controller_id};
> + if ($controller->{type} eq 'bgp') {
> + die "cannot configure a BGP fabric while BGP controller '$controller_id' exists:"
> + . " both target the default-VRF BGP router\n";
> + }
> + }
> + }
> +
> + my $zone_id = $fabric->{zone};
> + return if !$zone_id;
> +
> + raise_param_exc({ zone => 'VRF zones are only supported for BGP and OSPF fabrics' })
> + if $protocol !~ /^(?:bgp|ospf)$/;
> +
> + my $zone = PVE::Network::SDN::Zones::get_zone($zone_id);
should this use get_zone_checked instead?
> +
> + raise_param_exc({ zone => "zone '$zone_id' does not exist" }) if !$zone;
> + raise_param_exc({ zone => "zone '$zone_id' is not a simple zone" })
> + if $zone->{type} ne 'simple';
> + raise_param_exc({ zone => "zone '$zone_id' is configured to use the default VRF" })
> + if $zone->{'default-vrf'} // 1;
Isn't that dedicated-vrf?
> +
> + if ($config) {
> + PVE::Network::SDN::Fabrics::assert_fabric_nodes_in_zone(
> + $config, $fabric->{id}, $zone_id, $zone,
> + );
> + }
> +}
> +
> __PACKAGE__->register_method({
> name => 'index',
> path => '',
> @@ -129,6 +167,8 @@ __PACKAGE__->register_method({
> protected => 1,
> permissions => {
> check => ['perm', '/sdn/fabrics', ['SDN.Allocate']],
> + description => "Assigning a VRF zone additionally requires 'SDN.Allocate' on"
> + . " '/sdn/zones/<zone>'.",
> },
> parameters => {
> properties => PVE::Network::SDN::Fabrics::fabric_properties(0),
> @@ -148,14 +188,10 @@ __PACKAGE__->register_method({
> my $digest = extract_param($param, 'digest');
> PVE::Tools::assert_if_modified($config->digest(), $digest) if $digest;
>
> - if (($param->{protocol} // '') eq 'bgp') {
> - my $controller_cfg = PVE::Network::SDN::Controllers::config();
> - for my $id (keys %{ $controller_cfg->{ids} // {} }) {
> - die "cannot add a BGP fabric while BGP controller '$id' exists:"
> - . " both target the default-VRF BGP router\n"
> - if $controller_cfg->{ids}->{$id}->{type} eq 'bgp';
> - }
> - }
> + PVE::Network::SDN::Zones::api_get_zone_checked($param->{zone}, ['SDN.Allocate'])
> + if $param->{zone};
> +
> + validate_fabric_zone($param);
>
> my $privs = ['SDN.Use', 'SDN.Allocate'];
> PVE::Network::SDN::Fabrics::api_validate_property_references($param, $privs);
> @@ -200,7 +236,12 @@ __PACKAGE__->register_method({
> my $privs = ['SDN.Use', 'SDN.Allocate'];
> PVE::Network::SDN::Fabrics::api_validate_property_references($param, $privs);
does this fn also cover the zone permissions now?
>
> + my $old_zone = $config->get_fabric($id)->{zone} // '';
> + raise_param_exc({ zone => "cannot change the VRF zone of an existing fabric" })
> + if defined($param->{zone}) && $param->{zone} ne $old_zone;
this doesn't deny deleting the zone param.
> +
> $config->update_fabric($id, $param);
> + validate_fabric_zone($config->get_fabric($id), $config);
> PVE::Network::SDN::Fabrics::write_config($config);
> },
> "updating fabric failed",
> diff --git a/src/PVE/API2/Network/SDN/Fabrics/FabricNode.pm b/src/PVE/API2/Network/SDN/Fabrics/FabricNode.pm
> index 7a148b550f7d..eb3b429075e5 100644
> --- a/src/PVE/API2/Network/SDN/Fabrics/FabricNode.pm
> +++ b/src/PVE/API2/Network/SDN/Fabrics/FabricNode.pm
> @@ -9,6 +9,7 @@ use PVE::Tools qw(extract_param run_command);
> use PVE::Network::SDN;
> use PVE::Network::SDN::Fabrics;
> use PVE::Network::SDN::WireGuard;
> +use PVE::Network::SDN::Zones;
> use PVE::RS::SDN::Fabrics;
>
> use PVE::RESTHandler;
> @@ -138,6 +139,21 @@ my sub is_internal_wireguard_node {
> return $node->{protocol} eq 'wireguard' && $node->{role} eq 'internal';
> }
>
> +my sub validate_node_fabric_zone {
> + my ($config, $fabric_id, $node_id) = @_;
> +
> + my $fabric = $config->get_fabric($fabric_id);
> + my $zone_id = $fabric->{zone};
> + return if !$zone_id;
> +
> + my $zone = PVE::Network::SDN::Zones::get_zone($zone_id);
should this use get_zone_checked instead?
> +
> + die "zone '$zone_id' does not exist\n" if !$zone;
> + PVE::Network::SDN::Fabrics::assert_fabric_node_in_zone(
> + $fabric_id, $node_id, $zone_id, $zone,
> + );
> +}
> +
> __PACKAGE__->register_method({
> name => 'add_node',
> path => '',
> @@ -169,6 +185,8 @@ __PACKAGE__->register_method({
> my $digest = extract_param($param, 'digest');
> PVE::Tools::assert_if_modified($config->digest(), $digest) if $digest;
>
> + validate_node_fabric_zone($config, $param->{fabric_id}, $param->{node_id});
> +
> if (is_internal_wireguard_node($param) && $param->{interfaces}) {
> my $private_keys = PVE::Network::SDN::WireGuard::private_keys();
>
> diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm
> index 38d950fda076..e4f4b174466e 100644
> --- a/src/PVE/API2/Network/SDN/Zones.pm
> +++ b/src/PVE/API2/Network/SDN/Zones.pm
> @@ -11,6 +11,7 @@ use PVE::SafeSyslog;
> use PVE::Tools qw(extract_param);
>
> use PVE::Network::SDN::Dns;
> +use PVE::Network::SDN::Fabrics;
> use PVE::Network::SDN::Subnets;
> use PVE::Network::SDN::Vnets;
> use PVE::Network::SDN;
> @@ -363,6 +364,13 @@ sub create_etc_interfaces_sdn_dir {
> mkdir("/etc/pve/sdn");
> }
>
> +sub fabrics_using_zone {
> + my ($zone_id, $fabric_config) = @_;
> +
> + my $fabrics = $fabric_config->list_fabrics();
> + return grep { ($fabrics->{$_}->{zone} // '') eq $zone_id } sort keys $fabrics->%*;
> +}
> +
> __PACKAGE__->register_method({
> name => 'create',
> protected => 1,
> @@ -475,6 +483,22 @@ __PACKAGE__->register_method({
>
> $scfg->{$_} = $opts->{$_} for keys $opts->%*;
>
> + my $fabric_config = PVE::Network::SDN::Fabrics::config();
> + my @fabric_ids = fabrics_using_zone($id, $fabric_config);
> +
> + if (@fabric_ids && $scfg->{type} eq 'simple' && !$scfg->{'dedicated-vrf'}) {
> + raise_param_exc({
> + 'dedicated-vrf' =>
> + "zone is still used as VRF by fabric '$fabric_ids[0]'",
> + });
> + }
> +
> + for my $fabric_id (@fabric_ids) {
> + PVE::Network::SDN::Fabrics::assert_fabric_nodes_in_zone(
> + $fabric_config, $fabric_id, $id, $scfg,
> + );
> + }
> +
> my $new_ipam = $scfg->{ipam};
> if (!$new_ipam != !$old_ipam || (($new_ipam // '') ne ($old_ipam // ''))) {
> # don't allow ipam change if subnet are defined for now, need to implement resync ipam content
> @@ -547,6 +571,10 @@ __PACKAGE__->register_method({
> my $plugin = PVE::Network::SDN::Zones::Plugin->lookup($scfg->{type});
> my $vnet_cfg = PVE::Network::SDN::Vnets::config();
>
> + my $fabric_config = PVE::Network::SDN::Fabrics::config();
> + my @fabric_ids = fabrics_using_zone($id, $fabric_config);
> + die "zone is still used as VRF by fabric '$fabric_ids[0]'\n" if @fabric_ids;
> +
> $plugin->on_delete_hook($id, $vnet_cfg);
>
> delete $cfg->{ids}->{$id};
> diff --git a/src/PVE/Network/SDN/Controllers/BgpPlugin.pm b/src/PVE/Network/SDN/Controllers/BgpPlugin.pm
> index 537b46229e58..328246ce7d77 100644
> --- a/src/PVE/Network/SDN/Controllers/BgpPlugin.pm
> +++ b/src/PVE/Network/SDN/Controllers/BgpPlugin.pm
> @@ -209,7 +209,7 @@ sub on_update_hook {
> for my $id (keys %$fabrics) {
> die "cannot configure a BGP controller while BGP fabric '$id' exists:"
> . " both target the default-VRF BGP router\n"
> - if $fabrics->{$id}->{protocol} eq 'bgp';
> + if $fabrics->{$id}->{protocol} eq 'bgp' && !$fabrics->{$id}->{zone};
> }
>
> my $controller = $controller_cfg->{ids}->{$controllerid};
> diff --git a/src/PVE/Network/SDN/Fabrics.pm b/src/PVE/Network/SDN/Fabrics.pm
> index 41119b2ec782..e7cec1fe9582 100644
> --- a/src/PVE/Network/SDN/Fabrics.pm
> +++ b/src/PVE/Network/SDN/Fabrics.pm
> @@ -229,6 +229,23 @@ sub write_config {
> cfs_write_file("sdn/fabrics.cfg", $config->to_raw(), 1);
> }
>
> +sub assert_fabric_node_in_zone {
> + my ($fabric_id, $node_id, $zone_id, $zone) = @_;
> +
> + return if !defined($zone->{nodes}) || $zone->{nodes}->{$node_id};
> +
> + die "fabric '$fabric_id' contains node '$node_id', which is not enabled for zone '$zone_id'\n";
> +}
> +
> +sub assert_fabric_nodes_in_zone {
> + my ($config, $fabric_id, $zone_id, $zone) = @_;
> +
> + my $nodes = $config->list_nodes_fabric($fabric_id);
> + for my $node_id (sort keys $nodes->%*) {
> + assert_fabric_node_in_zone($fabric_id, $node_id, $zone_id, $zone);
> + }
> +}
> +
> sub get_frr_daemon_status {
> my ($fabric_config) = @_;
>
> @@ -512,6 +529,15 @@ sub fabric_properties {
> protocol => get_standard_option('pve-sdn-fabric-protocol'),
> digest => get_standard_option('pve-config-digest'),
> 'lock-token' => get_standard_option('pve-sdn-lock-token'),
> + zone => get_standard_option(
> + 'pve-sdn-zone-id',
> + {
> + description => 'Simple zone whose VRF contains this fabric.',
> + 'type-property' => 'protocol',
> + 'instance-types' => ['bgp', 'ospf'],
> + optional => 1,
> + },
> + ),
> ip_prefix => {
> type => 'string',
> format => 'CIDR',
next prev parent reply other threads:[~2026-11-18 16:28 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-16 16:11 [PATCH common/manager/network/proxmox{-ve-rs,-perl-rs,-widget-toolkit} v3 00/32] SDN VRF support Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-common v3 01/32] interfaces: preserve VRF membership across address family changes Gabriel Goller
2026-11-18 14:13 ` Stefan Hanreich
2026-11-18 15:41 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 02/32] frr: add VRF-aware OSPF rendering Gabriel Goller
2026-11-18 14:41 ` Stefan Hanreich
2026-11-18 16:21 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 03/32] sdn: add zone references to OSPF and BGP fabrics Gabriel Goller
2026-11-18 14:10 ` Stefan Hanreich
2026-11-18 16:55 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 04/32] sdn: generate fabric routing configuration in zone VRFs Gabriel Goller
2026-11-18 14:53 ` Stefan Hanreich
2026-11-18 17:26 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 05/32] sdn: fix VRF route-map scoping and zone ID validation Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 06/32] tests: fabrics: add test for fabrics in VRFs Gabriel Goller
2026-11-18 15:01 ` Stefan Hanreich
2026-11-18 17:33 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 07/32] sdn: allow static routes in simple zone VRFs Gabriel Goller
2026-11-18 15:29 ` Stefan Hanreich
2026-11-18 18:10 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 08/32] sdn: support static routes in EVPN " Gabriel Goller
2026-11-18 15:32 ` Stefan Hanreich
2026-11-19 13:12 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-perl-rs v3 09/32] pve-rs: fabrics: assign network interfaces to configured VRFs Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-perl-rs v3 10/32] pve-rs: fabrics: make per-fabric status queries VRF-aware Gabriel Goller
2026-11-18 15:42 ` Stefan Hanreich
2026-11-19 13:30 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-perl-rs v3 11/32] pve-rs: fabrics: include VRF routes in aggregate status Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-perl-rs v3 12/32] pve-rs: fabrics: expose host interface membership Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-perl-rs v3 13/32] sdn: expose static routes for simple zone uplinks Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 14/32] sdn: evpn: add unreachable IPv6 default route to zone VRFs Gabriel Goller
2026-11-18 16:08 ` Stefan Hanreich
2026-11-18 16:39 ` Stefan Hanreich
2026-11-19 13:38 ` Gabriel Goller
2026-11-19 15:51 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 15/32] sdn: add optional VRFs for simple zones Gabriel Goller
2026-11-18 16:18 ` Stefan Hanreich
2026-11-19 13:44 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 16/32] sdn: allow fabrics to use simple zone VRFs Gabriel Goller
2026-11-18 16:27 ` Stefan Hanreich [this message]
2026-11-19 14:14 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 17/32] sdn: require underlay fabrics to use the default VRF Gabriel Goller
2026-11-18 16:30 ` Stefan Hanreich
2026-11-19 14:33 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 18/32] sdn: always generate EVPN zone VRFs Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 19/32] api: sdn: allow EVPN zones as fabric VRFs Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 20/32] api: sdn: disallow BGP fabrics in EVPN zone VRFs Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 21/32] sdn: simple: support static routes in zone routing tables Gabriel Goller
2026-11-18 16:49 ` Stefan Hanreich
2026-11-19 14:53 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 22/32] sdn: evpn: allow static routes Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 23/32] sdn: zones: SNAT via dedicated uplinks in zone VRFs Gabriel Goller
2026-11-18 18:07 ` Stefan Hanreich
2026-11-16 16:11 ` [PATCH pve-network v3 24/32] sdn: dhcp: only configure DHCP for zones on the local node Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 25/32] sdn: dhcp: run dnsmasq in the zone VRF Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-widget-toolkit v3 26/32] ui: network: allow selecting a VRF for host interfaces Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-manager v3 27/32] ui: sdn: add VRF zone selection for fabrics Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-manager v3 28/32] ui: sdn: expose EVPN zones as fabric VRFs Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-manager v3 29/32] network: allow setting the VRF of host interfaces Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-manager v3 30/32] ui: sdn: allow static routes for Simple zone VRFs Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-manager v3 31/32] ui: sdn: allow static routes in EVPN zones Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-manager v3 32/32] network: generate SDN DHCP config after ifreload Gabriel Goller
2026-11-18 18:27 ` [PATCH common/manager/network/proxmox{-ve-rs,-perl-rs,-widget-toolkit} v3 00/32] SDN VRF support Stefan Hanreich
2026-11-19 15:33 ` Gabriel Goller
2026-11-19 20:10 ` Gabriel Goller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=26c8b2b0-e107-49d7-b761-bfd5b5cc8e38@proxmox.com \
--to=s.hanreich@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.