From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 2D4361FF18C for ; Tue, 14 Apr 2026 18:35:16 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 158FD1FBFD; Tue, 14 Apr 2026 18:34:09 +0200 (CEST) From: Stefan Hanreich To: pve-devel@lists.proxmox.com Subject: [PATCH pve-network 08/16] evpn zone: add secondary-controllers and rt filtering Date: Tue, 14 Apr 2026 18:33:05 +0200 Message-ID: <20260414163315.419384-9-s.hanreich@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260414163315.419384-1-s.hanreich@proxmox.com> References: <20260414163315.419384-1-s.hanreich@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1776184326640 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.691 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: J7Q4DJUHARXLHB6PT2SXQNF6QZKYLMOJ X-Message-ID-Hash: J7Q4DJUHARXLHB6PT2SXQNF6QZKYLMOJ X-MailFrom: s.hanreich@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: With the ability to define multiple EVPN controllers, enable the possibility to assign multiple EVPN controllers to a single zone. This can be used to announce the L2VPN EVPN routes of a single zone to multiple peer groups. By having to specify the controllers responsible for announcing the routes explicitly, one can selectively announce the routes for a given zone to specific peer groups. Because the EVPN controller is used to auto-derive VTEP IPs (and the MTU of the VNets), based on the peer definition in the EVPN controller, it is not sufficient to extend the existing field and convert it to a list. Instead, the old controller field now defines the 'primary' controller for the zone, which is utilized to generate the VTEP IPs and MTU. All other secondary controllers only announce the L2VPN EVPN routes - without having any effect on VTEP IP auto-derivation and MTU. Since the generated FRR configuration uses `advertise-all-vni`, which advertise all routes from all zones by default (instead of importing VRFs / route-targets explicitly), it is required to filter outgoing EVPN routes in order to preserve backwards-compatibility. Otherwise an EVPN controller configured on a node would announce the routes from all zones, irregardless of the assigned zones. To prevent this, outgoing routes for EVPN controllers need to be filtered, based on the Route Targets of the zones and vnets. This is implemented via route maps and extcommunity lists. An EVPN controller will only announce routes with a route target that matches the route target of a zone / vnet. To prevent accidental breakage and preserve full backwards-compatibility, the base case (one EVPN controller in the cluster) is special cased. Since it was only possible to configure a single EVPN controller, and EVPN zones need a controller assigned, it is not required to filter outgoing routes when there is only one EVPN controller configured. This, in turn, means that only if there are multiple EVPN controllers configured, RT filtering for outgoing routes takes place. This makes the new behavior opt-in and the generated FRR configuration fully backwards-compatible. This is evidenced by the fact that all existing test cases still work without any changes. Signed-off-by: Stefan Hanreich --- src/PVE/API2/Network/SDN/Zones.pm | 11 +++ src/PVE/Network/SDN.pm | 1 + src/PVE/Network/SDN/Controllers.pm | 71 +++++++++++++++++++ src/PVE/Network/SDN/Controllers/EvpnPlugin.pm | 53 +++++++++++++- src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 39 ++++++++-- 5 files changed, 167 insertions(+), 8 deletions(-) diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm index 8d829a9..cfdc195 100644 --- a/src/PVE/API2/Network/SDN/Zones.pm +++ b/src/PVE/API2/Network/SDN/Zones.pm @@ -202,6 +202,17 @@ my $ZONE_PROPERTIES = { description => "Disable auto mac learning. VLAN zone only.", optional => 1, }, + 'secondary-controllers' => { + type => 'array', + description => 'Additional controllers.', + items => { + type => 'string', + minLength => 2, + maxLength => 64, + pattern => '[a-zA-Z][a-zA-Z0-9_-]*[a-zA-Z0-9]', + }, + optional => 1, + }, }; __PACKAGE__->register_method({ diff --git a/src/PVE/Network/SDN.pm b/src/PVE/Network/SDN.pm index 2b3dead..58a21f2 100644 --- a/src/PVE/Network/SDN.pm +++ b/src/PVE/Network/SDN.pm @@ -508,6 +508,7 @@ sub encode_value { || $key eq 'entries' || $key eq 'match' || $key eq 'set' + || $key eq 'secondary-controllers' ) { if (ref($value) eq 'HASH') { return join(',', sort keys(%$value)); diff --git a/src/PVE/Network/SDN/Controllers.pm b/src/PVE/Network/SDN/Controllers.pm index 3c18552..6167fa2 100644 --- a/src/PVE/Network/SDN/Controllers.pm +++ b/src/PVE/Network/SDN/Controllers.pm @@ -103,23 +103,41 @@ sub generate_frr_config { } } + my $allowed_communities = {}; + foreach my $id (sort keys %{ $controller_cfg->{ids} }) { my $plugin_config = $controller_cfg->{ids}->{$id}; my $plugin = PVE::Network::SDN::Controllers::Plugin->lookup($plugin_config->{type}); $plugin->generate_frr_config($plugin_config, $controller_cfg, $id, $uplinks, $frr_config); + + if ($plugin_config->{type} eq 'evpn') { + $allowed_communities->{$id} = []; + } } foreach my $id (sort keys %{ $zone_cfg->{ids} }) { my $plugin_config = $zone_cfg->{ids}->{$id}; + my $controllerid = $plugin_config->{controller}; next if !$controllerid; + my $controller = $controller_cfg->{ids}->{$controllerid}; + my $route_target = "$controller->{asn}:$plugin_config->{'vrf-vxlan'}"; + if ($controller) { my $controller_plugin = PVE::Network::SDN::Controllers::Plugin->lookup($controller->{type}); $controller_plugin->generate_zone_frr_config( $plugin_config, $controller, $controller_cfg, $id, $uplinks, $frr_config, ); + + push $allowed_communities->{$controllerid}->@*, $route_target; + } + + if ($plugin_config->{'secondary-controllers'}) { + for my $id ($plugin_config->{'secondary-controllers'}->@*) { + push $allowed_communities->{$id}->@*, $route_target; + } } } @@ -133,12 +151,65 @@ sub generate_frr_config { next if !$controllerid; my $controller = $controller_cfg->{ids}->{$controllerid}; + my $route_target = "$controller->{asn}:$plugin_config->{'tag'}"; + if ($controller) { my $controller_plugin = PVE::Network::SDN::Controllers::Plugin->lookup($controller->{type}); $controller_plugin->generate_vnet_frr_config( $plugin_config, $controller, $zone, $zoneid, $id, $frr_config, ); + + push $allowed_communities->{$controllerid}->@*, $route_target; + } + + if ($zone->{'secondary-controllers'}) { + for my $id ($zone->{'secondary-controllers'}->@*) { + push $allowed_communities->{$id}->@*, $route_target; + } + } + } + + if (!PVE::Network::SDN::Controllers::EvpnPlugin::skip_route_target_filtering($controller_cfg)) { + $frr_config->{frr}->{bgp}->{ext_community_lists} = {}; + + for my $controller_id (sort keys $allowed_communities->%*) { + my $route_targets = $allowed_communities->{$controller_id}; + my $community_list_name = "pve_controller_$controller_id"; + + if (scalar($route_targets->@*)) { + my @entries = map { { + action => 'permit', + match_entry => { + type => 'rt', + value => $_, + }, + } } $route_targets->@*; + + $frr_config->{frr}->{bgp}->{ext_community_lists}->{$community_list_name} = { + type => 'standard', + entries => \@entries, + }; + } else { + # Since it's impossible to create empty community lists create a + # community list with one deny entry instead. This works, + # because the default verdict is to deny any extcommunity that + # doesn't match an entry in the community list. So this + # extcommunity-list effectively blocks *every* route. + + $frr_config->{frr}->{bgp}->{ext_community_lists}->{$community_list_name} = { + type => 'standard', + entries => [ + { + action => 'deny', + match_entry => { + type => 'rt', + value => "0:0", + }, + }, + ], + }; + } } } } diff --git a/src/PVE/Network/SDN/Controllers/EvpnPlugin.pm b/src/PVE/Network/SDN/Controllers/EvpnPlugin.pm index db844ff..e9bee33 100644 --- a/src/PVE/Network/SDN/Controllers/EvpnPlugin.pm +++ b/src/PVE/Network/SDN/Controllers/EvpnPlugin.pm @@ -68,6 +68,32 @@ sub options { }; } +=head3 skip_route_target_filtering(\%controller_config) + +When multiple EVPN controllers are configured for multiple zones, outgoing +routes need to get filtered, since otherwise EVPN controllers would simply +announce all routes for all zones located on the node. To preserve +backwards-compatibility and unnecessary filtering when only one EVPN controller +is defined, this function can be used to check whether route target filtering +should be performed or not. + +=cut + +sub skip_route_target_filtering { + my ($controller_config) = @_; + + my $evpn_controller = undef; + + for my $controller (values $controller_config->{ids}->%*) { + next if $controller->{type} ne 'evpn'; + + return 0 if $evpn_controller; + $evpn_controller = $controller; + } + + return 1; +} + # Plugin implementation sub generate_frr_config { my ($class, $plugin_config, $controller_cfg, $id, $uplinks, $config) = @_; @@ -228,9 +254,26 @@ sub generate_frr_config { } if (!$config->{frr}->{routemaps}->{$route_map_out}) { - my $entry = { seq => 1, action => "permit" }; - $entry->{call} = $plugin_config->{'route-map-out'} if $plugin_config->{'route-map-out'}; + my $entry = { + seq => 1, + action => "permit", + }; + + # only filter outgoing routes if there are multiple EVPN controllers, to + # preserve backwards-compatibility + if (!skip_route_target_filtering($controller_cfg)) { + $entry->{matches} = [ + { + key => 'extcommunity', + value => { + name => "pve_controller_$id", + mode => 'any', + }, + }, + ]; + } + $entry->{call} = $plugin_config->{'route-map-out'} if $plugin_config->{'route-map-out'}; push($config->{frr}->{routemaps}->{$route_map_out}->@*, $entry); } @@ -538,8 +581,14 @@ sub on_delete_hook { # verify that zone is associated to this controller foreach my $id (keys %{ $zone_cfg->{ids} }) { my $zone = $zone_cfg->{ids}->{$id}; + die "controller $controllerid is used by $id" if (defined($zone->{controller}) && $zone->{controller} eq $controllerid); + + for my $secondary_controller ($zone->{'secondary-controllers'}->@*) { + die "controller $controllerid is used by $id" + if $secondary_controller eq $controllerid; + } } } diff --git a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm index 8e7ddfd..5df7e5f 100644 --- a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm +++ b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm @@ -53,6 +53,17 @@ sub properties { type => 'string', description => 'Controller for this zone.', }, + 'secondary-controllers' => { + type => 'array', + description => 'Additional controllers.', + items => { + type => 'string', + minLength => 2, + maxLength => 64, + pattern => '[a-zA-Z][a-zA-Z0-9_-]*[a-zA-Z0-9]', + }, + optional => 1, + }, 'mac' => { type => 'string', description => "Anycast logical router mac address.", @@ -97,6 +108,7 @@ sub options { nodes => { optional => 1 }, 'vrf-vxlan' => { optional => 0 }, controller => { optional => 0 }, + 'secondary-controllers' => { optional => 1 }, exitnodes => { optional => 1 }, 'exitnodes-local-routing' => { optional => 1 }, 'exitnodes-primary' => { optional => 1 }, @@ -348,13 +360,28 @@ sub generate_sdn_config { sub on_update_hook { my ($class, $zoneid, $zone_cfg, $controller_cfg) = @_; + my $controllers = {}; + # verify that controller exist - my $controller = $zone_cfg->{ids}->{$zoneid}->{controller}; - if (!defined($controller_cfg->{ids}->{$controller})) { - die "controller $controller don't exist"; - } else { - die "$controller is not a evpn controller type" - if $controller_cfg->{ids}->{$controller}->{type} ne 'evpn'; + my $zone = $zone_cfg->{ids}->{$zoneid}; + my $controller = $zone->{controller}; + + $controllers->{$controller} = undef; + + for my $secondary_controller ($zone->{'secondary-controllers'}->@*) { + die "can not configure the same controller twice" + if exists($controllers->{$secondary_controller}); + + $controllers->{$secondary_controller} = undef; + } + + for my $controller_id (keys $controllers->%*) { + if (!defined($controller_cfg->{ids}->{$controller_id})) { + die "controller $controller don't exist"; + } else { + die "$controller_id is not a evpn controller type" + if $controller_cfg->{ids}->{$controller_id}->{type} ne 'evpn'; + } } #vrf-vxlan need to be defined -- 2.47.3