Re: [PATCH pve-network] sdn: zones: vxlan: restore the flood entries after an FRR reload

[Hannes Laimer <h.laimer@proxmox.com>]

On 2026-10-05 14:31, Gabriel Goller wrote:
> On 29.09.2026 11:28, Hannes Laimer wrote:
>> With advertise-all-vni, which an EVPN controller sets even without an
>> EVPN zone, FRR treats every vxlan device on a node as an EVPN VNI.
>> That includes the devices of a VXLAN zone, for which the other nodes
>> running FRR become remote VTEPs.
>>
>> When the controller is removed, FRR drops its VNIs and deletes the
>> flood entries of their remote VTEPs. The kernel identifies such an
>> entry by its address and not by who added it, so the zone's static
>> flood entries to those nodes are deleted. The zone can no longer
>> flood, so nothing resolves across nodes, and no reload repairs it,
>> since ifupdown2 re-applies the peer list only when the configured list
>> changed.
>>
>> Re-append the entries after every FRR apply. The kernel treats an
>> existing remote as a no-op, so the entries are simply added where
>> missing.
>>
>> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
>> ---
>>  src/PVE/Network/SDN.pm       |  7 ++++++-
>>  src/PVE/Network/SDN/Zones.pm | 18 ++++++++++++++++++
>>  2 files changed, 24 insertions(+), 1 deletion(-)
>>
>> diff --git a/src/PVE/Network/SDN.pm b/src/PVE/Network/SDN.pm
>> index 33a3cf3..42af00d 100644
>> --- a/src/PVE/Network/SDN.pm
>> +++ b/src/PVE/Network/SDN.pm
>> @@ -492,7 +492,12 @@ sub generate_frr_config {
>>      my $raw_config = PVE::Network::SDN::generate_frr_raw_config($running_config, $fabric_config);
>>      PVE::Network::SDN::Frr::write_raw_config($raw_config);
>>  
>> -    PVE::Network::SDN::Frr::apply($needs_restart) if $apply;
>> +    return if !$apply;
>> +
>> +    PVE::Network::SDN::Frr::apply($needs_restart);
>> +
>> +    # zebra removes a VXLAN zone's static flood entries with the VTEPs it withdraws
>> +    PVE::Network::SDN::Zones::restore_vxlan_flood_entries();
> 
> Hmm so this is tricky, on reload bgpd tells zebra to remove these changes
> and zebra enqueues this to the dplane, so this is asynchronous in many
> ways. So we can't guarantee that the entries have been removed when running
> restore_vxlan_flood_entries().
> 

yes, this is unfortunately not something we can guarantee. restart
happens already at the end of FRR::apply, we could force a `reload` of
there is a removed evpn controller and an existing vxlan zone. I'm not
sure there is a difference for this here tough..


> Not sure what we could do here.
> 
> Three options that came to my mind where:
> 1) sleep(5)

not sure what kind of timeframes are realistic here, since we can't
guarantee anything anyway, something like 1s is probably enough. but in
my (limited) testing i also couldn't hit this without any sleep

> 2) `bridge monitor fdb` before running the frr reload/restart and then check if the entries have been removed (kind of overkill)

we still wouldn't know how long to wait..

> 3) force a frr restart (not 100% sure if this works)
> 

ideally we could just tell frr 'hands off these VNIs..', but we can't,
and given `advertise-all-vni` that might also be weird/wrong to begin
with


>>  }
>>  
>>  sub generate_dhcp_config {
>> diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm
>> index f668303..2c3c69c 100644
>> --- a/src/PVE/Network/SDN/Zones.pm
>> +++ b/src/PVE/Network/SDN/Zones.pm
>> @@ -178,6 +178,24 @@ sub generate_etc_network_config {
>>      return $raw_network_config;
>>  }
>>  
>> +sub restore_vxlan_flood_entries {
>> +    my $raw_config = eval { PVE::Tools::file_get_contents($local_network_sdn_file) };
>> +    return if !defined($raw_config);
> 
> Maybe an error here would be nice instead of a silent return. Although this
> probably can't happen anyway...
> 

yeah, not really an error imho.. but a warning should be fine,
because the caller assumed there is a config file


thanks for taking a look! :)

>> +
>> +    my $iface;
>> +    for my $line (split(/\n/, $raw_config)) {
>> +        if ($line =~ m/^iface (\S+)$/) {
>> +            $iface = $1;
>> +        } elsif ($line =~ m/^\s+vxlan_remoteip (\S+)$/) {
>> +            my $address = $1;
>> +            my $cmd =
>> +                ['bridge', 'fdb', 'append', '00:00:00:00:00:00', 'dev', $iface, 'dst', $address];
>> +            eval { run_command($cmd) };
>> +            warn "$iface: restoring the flood entry to $address failed: $@" if $@;
>> +        }
>> +    }
>> +}
>> +
>>  sub read_etc_network_config_version {
>>      my $versionstr = PVE::Tools::file_read_firstline($local_network_sdn_file);
>>  
>> -- 
>> 2.47.3