superseded: [PATCH common/network 0/3] Performance improvements for applying VLAN / QinQ zones

[Stefan Hanreich <s.hanreich@proxmox.com>]

https://lore.proxmox.com/pve-devel/20261019112640.181849-2-s.hanreich@proxmox.com/T/#t

On 7/9/26 3:28 PM, Stefan Hanreich wrote:
> ## Problem
> 
> When creating a VLAN / QinQ zone on a non VLAN-aware bridge, the SDN config
> generation enters a code path that calls ip_link_details from PVE::IPRoute2
> repeatedly. This causes a severe slowdown when applying configurations with a
> large amount of VNets. What exacerbated this problem was that, aside from
> calling iproute2 twice, every vnet recalculated the bridge ports for its bridge
> by iterating over all existing interfaces on the host which leads to a quadratic
> increase in runtime once the SDN configuration had been applied once.
> 
> ## Solution
> 
> This patch series introduces a new helper to pve-common that pre-computes the
> bridge port mappings used in the SDN config generation. The SDN stack then calls
> that function once at the beginning of the 'Apply' step, then re-uses the result
> for each vnet. This avoids calling iproute2 repeatedly and the nested loop
> inside the generate_sdn_config function in the VLAN / QinQ plugins.
> 
> An initial approach only cached the ip_link_details output and reused that, but
> this approach did not avoid the nested loops, so I opted for pre-calculating the
> required information instead and passing only that to the zone plugins. The
> first patch is a remnant of the initial approach, which was still included
> because it is useful for the other callsites.
> 
> ## Benchmarks
> 
> This slashes the runtime for generating the SDN configuration considerably on a
> test setup with 1000 VLAN VNets present on the host / config:
> 
> Without these patches applied:
> 
>   root@pve-vlan-test:~# time perl -e 'use PVE::Network::SDN; PVE::Network::SDN::generate_raw_etc_network_config(PVE::Network::SDN::compile_running_cfg(1))'
>   real    5m59.853s
>   user    5m49.009s
>   sys     0m34.887s
> 
> With these patches applied:
> 
>   root@pve-vlan-test:~# time perl -e 'use PVE::Network::SDN; PVE::Network::SDN::generate_raw_etc_network_config(PVE::Network::SDN::compile_running_cfg(1))'
>   real    0m0.868s
>   user    0m0.818s
>   sys     0m0.109s
> 
> ## Future Work
> 
> This patch series does not address any bottlenecks in ifupdown2, which is now
> the limiting factor when applying large configurations. Applying a SDN
> configuration via ifreload with 1000 VLANs takes 22 seconds. I'll be exploring
> potential improvements there with Christoph, but currently we do not have high
> hopes of improving the runtime there considerably.
> 
> Currently the SDN configuration is applied sequentially on each node, I'll be
> looking into creating a patch series that applies the network configuration in
> parallel.
> 
> 
> pve-common:
> 
> Stefan Hanreich (2):
>   iproute2: restructure get_physical_bridge_ports
>   iproute2: add helper for getting bridge port mappings
> 
>  src/PVE/IPRoute2.pm | 30 +++++++++++++++++++++++++++---
>  1 file changed, 27 insertions(+), 3 deletions(-)
> 
> 
> pve-network:
> 
> Stefan Hanreich (1):
>   fix #7738: zones: use pre-computed bridge port list
> 
>  src/PVE/Network/SDN/Zones.pm            |  3 +++
>  src/PVE/Network/SDN/Zones/Plugin.pm     |  7 +------
>  src/PVE/Network/SDN/Zones/QinQPlugin.pm |  5 +++--
>  src/PVE/Network/SDN/Zones/VlanPlugin.pm |  5 +++--
>  src/test/run_test_zones.pl              | 13 ++++++++++---
>  5 files changed, 20 insertions(+), 13 deletions(-)
> 
> 
> Summary over all repositories:
>   6 files changed, 47 insertions(+), 16 deletions(-)
>