Subject: Re: [PATCH access-control/cluster/network/proxmox{-ve-rs,-perl-rs} v2 00/34] Add support for route maps / prefix lists to SDN

Subject: Re: [PATCH access-control/cluster/network/proxmox{-ve-rs,-perl-rs} v2 00/34] Add support for route maps / prefix lists to SDN

[Hugh Blandford]

Hi Stefan and all,

I am new, so please forgive me if I am bringing up something that has
already been discarded.

I saw your thread referenced above supporting route maps / prefix lists in
SDN.

I believe my changes address a gap not covered by this particular series —
specifically, the ability to establish BGP peers inside a VRF rather than
in the default routing table, which ended up with asymmetric routing issues.

What I have done is not complete at this stage, but I believe it is useful
in solving an issue and wanted to see if there is any interest in
completing this as a patch.

I recently started experimenting with an EVPN setup and encountered an
asymmetric routing problem with pfSense as my gateway router using a
dedicated transit vlan to my SDN workloads.  pfSense doesn't support ECMP
so I went with having a primary exit node, but traffic to my
existing management PC would shortcut the pfSense box and go via the
existing interfaces.


I patched three files:
/usr/share/perl5/PVE/Network/SDN/Zones/BgpPlugin.pm
/usr/share/perl5/PVE/Network/SDN/Zones/EvpnPlugin.pm
/usr/share/pve-manager/js/pvemanagerlib.js

1. I added a vrf_zone property to the peer configuration in BgpPlugin.pm
2. Changed EvpnPlugin.pm to use the vrf_zone property to define the BGP
session inside the vrf
3. Added an extra field to the web interface for the BGP controller option
to select a valid vrf from the dropdown box
4. Added additional fields to the web interface for the BGP controller to
specify a prefix list to pull in information from frr.conf.local.  (Not
needed anymore thanks to this patch series)
5. I added the vmbrX interface for the transit vlan into my vrf by manually
adding vrf vrf_<vrfname) to the /etc/network/interfaces file and
reloading.  I have not explored what would be required to handle this in
the web interface at this stage.


My frr.conf.local:
ip prefix-list EVPN_SUBNETS seq 10 deny 192.168.250.0/23 ge 24 le 24
ip prefix-list EVPN_SUBNETS seq 20 deny 192.168.252.0/22 ge 24 le 24
ip prefix-list EVPN_SUBNETS seq 30 permit 192.168.192.0/18 ge 24 le 24
!
route-map TO_PFSENSE permit 10
 match ip address prefix-list EVPN_SUBNETS
exit
!
route-map TO_PFSENSE deny 20
exit
!

Result:
This allows you to have a BGP peering session running inside a VRF, with
prefix lists controlling the route distribution without requiring any route
leaking.
Traffic is routed into and out of the VRF via the specified transit vlan,
no shortcuts via other interfaces.
This has resulted in the following output which is what I intended at the
start:
Normal proxmox node routing table with no vrf entries in it:

root@prox1:/# ip route
default via 192.168.132.1 dev vmbr0 proto kernel onlink
10.10.20.0/24 dev vmbr20 proto kernel scope link src 10.10.20.7
10.10.21.0/24 dev vmbr21 proto kernel scope link src 10.10.21.7
192.168.132.0/24 dev vmbr0 proto kernel scope link src 192.168.132.7

Below is the vrf routing table for the SDN VRF with its respective entries:
192.168.222.0/24 an SDN subnet with VMs in it
192.168.254.0/24 a transit vlan between the SDN exit nodes and a pfSense VM
gateway to the internet, which is publishing a default route to the SDN
exit nodes and learning whatever routes are available in the range
192.168.192.0-192.168.249.255.


root@prox1:/# ip route show vrf vrf_public
default nhid 38 via 192.168.254.1 dev vmbr14 proto bgp metric 20
192.168.222.0/24 dev dmz proto kernel scope link src 192.168.222.1
192.168.254.0/24 dev vmbr14 proto kernel scope link src 192.168.254.7

With the BGP peer configuration automatically built for you on the exit
node in frr:

router bgp 65001 vrf vrf_public
 bgp router-id 10.10.21.7
 no bgp hard-administrative-reset
 no bgp graceful-restart notification
 neighbor BGP peer-group
 neighbor BGP remote-as external
 neighbor BGP bfd
 neighbor 192.168.254.1 peer-group BGP
 !
 address-family ipv4 unicast
  redistribute connected
  neighbor BGP soft-reconfiguration inbound
  neighbor BGP route-map TO_PFSENSE out
 exit-address-family
 !
 address-family ipv6 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  default-originate ipv4
  default-originate ipv6
 exit-address-family
exit

Regards,

Hugh Blandford

Re: Subject: Re: [PATCH access-control/cluster/network/proxmox{-ve-rs,-perl-rs} v2 00/34] Add support for route maps / prefix lists to SDN

[Gabriel Goller]

Hi!

On 12.04.2026 15:10, Hugh Blandford wrote:
> Hi Stefan and all,
> 
> I am new, so please forgive me if I am bringing up something that has
> already been discarded.
> 
> I saw your thread referenced above supporting route maps / prefix lists in
> SDN.
> 
> I believe my changes address a gap not covered by this particular series —
> specifically, the ability to establish BGP peers inside a VRF rather than
> in the default routing table, which ended up with asymmetric routing issues.
> 
> What I have done is not complete at this stage, but I believe it is useful
> in solving an issue and wanted to see if there is any interest in
> completing this as a patch.

We're currently revamping the EVPN-Controller, introducing a BGP-Fabric and
VRF-Handling in the UI. This will hopefully make it possible to select a
specific VRF in the BGP-Fabric and then peer inside of the VRF.

> I recently started experimenting with an EVPN setup and encountered an
> asymmetric routing problem with pfSense as my gateway router using a
> dedicated transit vlan to my SDN workloads.  pfSense doesn't support ECMP
> so I went with having a primary exit node, but traffic to my
> existing management PC would shortcut the pfSense box and go via the
> existing interfaces.
> 
> 
> I patched three files:
> /usr/share/perl5/PVE/Network/SDN/Zones/BgpPlugin.pm
> /usr/share/perl5/PVE/Network/SDN/Zones/EvpnPlugin.pm
> /usr/share/pve-manager/js/pvemanagerlib.js
> 
> 1. I added a vrf_zone property to the peer configuration in BgpPlugin.pm
> 2. Changed EvpnPlugin.pm to use the vrf_zone property to define the BGP
> session inside the vrf
> 3. Added an extra field to the web interface for the BGP controller option
> to select a valid vrf from the dropdown box
> 4. Added additional fields to the web interface for the BGP controller to
> specify a prefix list to pull in information from frr.conf.local.  (Not
> needed anymore thanks to this patch series)
> 5. I added the vmbrX interface for the transit vlan into my vrf by manually
> adding vrf vrf_<vrfname) to the /etc/network/interfaces file and
> reloading.  I have not explored what would be required to handle this in
> the web interface at this stage.
> 
> 
> My frr.conf.local:
> ip prefix-list EVPN_SUBNETS seq 10 deny 192.168.250.0/23 ge 24 le 24
> ip prefix-list EVPN_SUBNETS seq 20 deny 192.168.252.0/22 ge 24 le 24
> ip prefix-list EVPN_SUBNETS seq 30 permit 192.168.192.0/18 ge 24 le 24
> !
> route-map TO_PFSENSE permit 10
>  match ip address prefix-list EVPN_SUBNETS
> exit
> !
> route-map TO_PFSENSE deny 20
> exit
> !
> 
> Result:
> This allows you to have a BGP peering session running inside a VRF, with
> prefix lists controlling the route distribution without requiring any route
> leaking.
> Traffic is routed into and out of the VRF via the specified transit vlan,
> no shortcuts via other interfaces.
> This has resulted in the following output which is what I intended at the
> start:
> Normal proxmox node routing table with no vrf entries in it:
> 
> root@prox1:/# ip route
> default via 192.168.132.1 dev vmbr0 proto kernel onlink
> 10.10.20.0/24 dev vmbr20 proto kernel scope link src 10.10.20.7
> 10.10.21.0/24 dev vmbr21 proto kernel scope link src 10.10.21.7
> 192.168.132.0/24 dev vmbr0 proto kernel scope link src 192.168.132.7
> 
> Below is the vrf routing table for the SDN VRF with its respective entries:
> 192.168.222.0/24 an SDN subnet with VMs in it
> 192.168.254.0/24 a transit vlan between the SDN exit nodes and a pfSense VM
> gateway to the internet, which is publishing a default route to the SDN
> exit nodes and learning whatever routes are available in the range
> 192.168.192.0-192.168.249.255.
> 
> 
> root@prox1:/# ip route show vrf vrf_public
> default nhid 38 via 192.168.254.1 dev vmbr14 proto bgp metric 20
> 192.168.222.0/24 dev dmz proto kernel scope link src 192.168.222.1
> 192.168.254.0/24 dev vmbr14 proto kernel scope link src 192.168.254.7
> 
> With the BGP peer configuration automatically built for you on the exit
> node in frr:
> 
> router bgp 65001 vrf vrf_public
>  bgp router-id 10.10.21.7
>  no bgp hard-administrative-reset
>  no bgp graceful-restart notification
>  neighbor BGP peer-group
>  neighbor BGP remote-as external
>  neighbor BGP bfd
>  neighbor 192.168.254.1 peer-group BGP
>  !
>  address-family ipv4 unicast
>   redistribute connected
>   neighbor BGP soft-reconfiguration inbound
>   neighbor BGP route-map TO_PFSENSE out
>  exit-address-family
>  !
>  address-family ipv6 unicast
>   redistribute connected
>  exit-address-family
>  !
>  address-family l2vpn evpn
>   default-originate ipv4
>   default-originate ipv6
>  exit-address-family
> exit

Yep, this is the sort of config we're going for. Sadly when using EVPN+BGP, the
EVPN peers have to be in the default VRF, because FRR doesn't allow EVPN peering
in a custom VRF. So the config of EVPN Controller + BGP Fabric is probably going
to look some thing like:

vrf vrf_evpn
 vni 1000
exit-vrf
!
router bgp 65000
 bgp router-id 10.0.0.1
 no bgp hard-administrative-reset
 no bgp default ipv4-unicast
 coalesce-time 1000
 no bgp graceful-restart notification
 !
 neighbor VTEP peer-group
 neighbor VTEP remote-as internal
 neighbor ens19 interface peer-group VTEP
 !
 address-family l2vpn evpn
  neighbor VTEP activate
  advertise-all-vni
 exit-address-family
 !
exit
!
router bgp 65011 vrf vrf_export
 bgp router-id 10.0.0.1
 no bgp hard-administrative-reset
 no bgp default ipv4-unicast
 coalesce-time 1000
 no bgp graceful-restart notification
 neighbor BGP peer-group
 neighbor BGP remote-as external
 neighbor ens21 interface peer-group BGP
 !
 address-family ipv4 unicast
  neighbor BGP activate
  import vpn
  rt vpn import 65000:500
  route-map vpn import IMPORT_RM
 exit-address-family
 !
exit
!
router bgp 65000 vrf vrf_evpn
 bgp router-id 10.0.0.1
 no bgp hard-administrative-reset
 no bgp graceful-restart notification
 !
 address-family ipv4 unicast
  export vpn
  rd vpn export 65000:1
  rt vpn export 65000:500
  redistribute connected
 exit-address-family
 !
 address-family ipv6 unicast
  redistribute connected
 exit-address-family
 !
exit
!
access-list LOCAL seq 1 permit 10.0.0.1
access-list LOCAL seq 2 permit 0.0.0.0
!
route-map IMPORT_RM permit 20
 match ip next-hop LOCAL
route-map IMPORT_RM deny 30

> Regards,
> 
> Hugh Blandford