From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 61A8B1FF09C for ; Mon, 05 Oct 2026 15:01:51 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 553AC214C3; Mon, 05 Oct 2026 15:01:49 +0200 (CEST) Message-ID: Date: Mon, 5 Oct 2026 15:01:44 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH pve-network] sdn: zones: vxlan: restore the flood entries after an FRR reload To: Gabriel Goller References: <20260929092845.189004-1-h.laimer@proxmox.com> From: Hannes Laimer Content-Language: en-US In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1791205304847 X-SPAM-LEVEL: Spam detection results: 0 AWL -2.024 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_DBL_SPAM 5 Contains a spam URL listed in the Spamhaus DBL blocklist [sdn.pm] Message-ID-Hash: KIVQHBP2BW42PUEYRLM34AY3FPT3QZWH X-Message-ID-Hash: KIVQHBP2BW42PUEYRLM34AY3FPT3QZWH X-MailFrom: h.laimer@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: pve-devel@lists.proxmox.com X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 2026-10-05 14:31, Gabriel Goller wrote: > On 29.09.2026 11:28, Hannes Laimer wrote: >> With advertise-all-vni, which an EVPN controller sets even without an >> EVPN zone, FRR treats every vxlan device on a node as an EVPN VNI. >> That includes the devices of a VXLAN zone, for which the other nodes >> running FRR become remote VTEPs. >> >> When the controller is removed, FRR drops its VNIs and deletes the >> flood entries of their remote VTEPs. The kernel identifies such an >> entry by its address and not by who added it, so the zone's static >> flood entries to those nodes are deleted. The zone can no longer >> flood, so nothing resolves across nodes, and no reload repairs it, >> since ifupdown2 re-applies the peer list only when the configured list >> changed. >> >> Re-append the entries after every FRR apply. The kernel treats an >> existing remote as a no-op, so the entries are simply added where >> missing. >> >> Signed-off-by: Hannes Laimer >> --- >> src/PVE/Network/SDN.pm | 7 ++++++- >> src/PVE/Network/SDN/Zones.pm | 18 ++++++++++++++++++ >> 2 files changed, 24 insertions(+), 1 deletion(-) >> >> diff --git a/src/PVE/Network/SDN.pm b/src/PVE/Network/SDN.pm >> index 33a3cf3..42af00d 100644 >> --- a/src/PVE/Network/SDN.pm >> +++ b/src/PVE/Network/SDN.pm >> @@ -492,7 +492,12 @@ sub generate_frr_config { >> my $raw_config = PVE::Network::SDN::generate_frr_raw_config($running_config, $fabric_config); >> PVE::Network::SDN::Frr::write_raw_config($raw_config); >> >> - PVE::Network::SDN::Frr::apply($needs_restart) if $apply; >> + return if !$apply; >> + >> + PVE::Network::SDN::Frr::apply($needs_restart); >> + >> + # zebra removes a VXLAN zone's static flood entries with the VTEPs it withdraws >> + PVE::Network::SDN::Zones::restore_vxlan_flood_entries(); > > Hmm so this is tricky, on reload bgpd tells zebra to remove these changes > and zebra enqueues this to the dplane, so this is asynchronous in many > ways. So we can't guarantee that the entries have been removed when running > restore_vxlan_flood_entries(). > yes, this is unfortunately not something we can guarantee. restart happens already at the end of FRR::apply, we could force a `reload` of there is a removed evpn controller and an existing vxlan zone. I'm not sure there is a difference for this here tough.. > Not sure what we could do here. > > Three options that came to my mind where: > 1) sleep(5) not sure what kind of timeframes are realistic here, since we can't guarantee anything anyway, something like 1s is probably enough. but in my (limited) testing i also couldn't hit this without any sleep > 2) `bridge monitor fdb` before running the frr reload/restart and then check if the entries have been removed (kind of overkill) we still wouldn't know how long to wait.. > 3) force a frr restart (not 100% sure if this works) > ideally we could just tell frr 'hands off these VNIs..', but we can't, and given `advertise-all-vni` that might also be weird/wrong to begin with >> } >> >> sub generate_dhcp_config { >> diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm >> index f668303..2c3c69c 100644 >> --- a/src/PVE/Network/SDN/Zones.pm >> +++ b/src/PVE/Network/SDN/Zones.pm >> @@ -178,6 +178,24 @@ sub generate_etc_network_config { >> return $raw_network_config; >> } >> >> +sub restore_vxlan_flood_entries { >> + my $raw_config = eval { PVE::Tools::file_get_contents($local_network_sdn_file) }; >> + return if !defined($raw_config); > > Maybe an error here would be nice instead of a silent return. Although this > probably can't happen anyway... > yeah, not really an error imho.. but a warning should be fine, because the caller assumed there is a config file thanks for taking a look! :) >> + >> + my $iface; >> + for my $line (split(/\n/, $raw_config)) { >> + if ($line =~ m/^iface (\S+)$/) { >> + $iface = $1; >> + } elsif ($line =~ m/^\s+vxlan_remoteip (\S+)$/) { >> + my $address = $1; >> + my $cmd = >> + ['bridge', 'fdb', 'append', '00:00:00:00:00:00', 'dev', $iface, 'dst', $address]; >> + eval { run_command($cmd) }; >> + warn "$iface: restoring the flood entry to $address failed: $@" if $@; >> + } >> + } >> +} >> + >> sub read_etc_network_config_version { >> my $versionstr = PVE::Tools::file_read_firstline($local_network_sdn_file); >> >> -- >> 2.47.3