From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 5523C1FF137 for ; Tue, 14 Apr 2026 10:10:00 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id D93B51108E; Tue, 14 Apr 2026 10:10:46 +0200 (CEST) Date: Tue, 14 Apr 2026 10:10:11 +0200 From: Gabriel Goller To: Hugh Blandford Subject: Re: Subject: Re: [PATCH access-control/cluster/network/proxmox{-ve-rs,-perl-rs} v2 00/34] Add support for route maps / prefix lists to SDN Message-ID: Mail-Followup-To: Hugh Blandford , pve-devel@lists.proxmox.com References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: NeoMutt/20241002-35-39f9a6 X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1776154136666 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.027 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [evpnplugin.pm,bgpplugin.pm] Message-ID-Hash: KTWY5HD44IY6MXD4ALA2AWGIJVFWTPZY X-Message-ID-Hash: KTWY5HD44IY6MXD4ALA2AWGIJVFWTPZY X-MailFrom: g.goller@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: pve-devel@lists.proxmox.com X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Hi! On 12.04.2026 15:10, Hugh Blandford wrote: > Hi Stefan and all, > > I am new, so please forgive me if I am bringing up something that has > already been discarded. > > I saw your thread referenced above supporting route maps / prefix lists in > SDN. > > I believe my changes address a gap not covered by this particular series — > specifically, the ability to establish BGP peers inside a VRF rather than > in the default routing table, which ended up with asymmetric routing issues. > > What I have done is not complete at this stage, but I believe it is useful > in solving an issue and wanted to see if there is any interest in > completing this as a patch. We're currently revamping the EVPN-Controller, introducing a BGP-Fabric and VRF-Handling in the UI. This will hopefully make it possible to select a specific VRF in the BGP-Fabric and then peer inside of the VRF. > I recently started experimenting with an EVPN setup and encountered an > asymmetric routing problem with pfSense as my gateway router using a > dedicated transit vlan to my SDN workloads. pfSense doesn't support ECMP > so I went with having a primary exit node, but traffic to my > existing management PC would shortcut the pfSense box and go via the > existing interfaces. > > > I patched three files: > /usr/share/perl5/PVE/Network/SDN/Zones/BgpPlugin.pm > /usr/share/perl5/PVE/Network/SDN/Zones/EvpnPlugin.pm > /usr/share/pve-manager/js/pvemanagerlib.js > > 1. I added a vrf_zone property to the peer configuration in BgpPlugin.pm > 2. Changed EvpnPlugin.pm to use the vrf_zone property to define the BGP > session inside the vrf > 3. Added an extra field to the web interface for the BGP controller option > to select a valid vrf from the dropdown box > 4. Added additional fields to the web interface for the BGP controller to > specify a prefix list to pull in information from frr.conf.local. (Not > needed anymore thanks to this patch series) > 5. I added the vmbrX interface for the transit vlan into my vrf by manually > adding vrf vrf_ reloading. I have not explored what would be required to handle this in > the web interface at this stage. > > > My frr.conf.local: > ip prefix-list EVPN_SUBNETS seq 10 deny 192.168.250.0/23 ge 24 le 24 > ip prefix-list EVPN_SUBNETS seq 20 deny 192.168.252.0/22 ge 24 le 24 > ip prefix-list EVPN_SUBNETS seq 30 permit 192.168.192.0/18 ge 24 le 24 > ! > route-map TO_PFSENSE permit 10 > match ip address prefix-list EVPN_SUBNETS > exit > ! > route-map TO_PFSENSE deny 20 > exit > ! > > Result: > This allows you to have a BGP peering session running inside a VRF, with > prefix lists controlling the route distribution without requiring any route > leaking. > Traffic is routed into and out of the VRF via the specified transit vlan, > no shortcuts via other interfaces. > This has resulted in the following output which is what I intended at the > start: > Normal proxmox node routing table with no vrf entries in it: > > root@prox1:/# ip route > default via 192.168.132.1 dev vmbr0 proto kernel onlink > 10.10.20.0/24 dev vmbr20 proto kernel scope link src 10.10.20.7 > 10.10.21.0/24 dev vmbr21 proto kernel scope link src 10.10.21.7 > 192.168.132.0/24 dev vmbr0 proto kernel scope link src 192.168.132.7 > > Below is the vrf routing table for the SDN VRF with its respective entries: > 192.168.222.0/24 an SDN subnet with VMs in it > 192.168.254.0/24 a transit vlan between the SDN exit nodes and a pfSense VM > gateway to the internet, which is publishing a default route to the SDN > exit nodes and learning whatever routes are available in the range > 192.168.192.0-192.168.249.255. > > > root@prox1:/# ip route show vrf vrf_public > default nhid 38 via 192.168.254.1 dev vmbr14 proto bgp metric 20 > 192.168.222.0/24 dev dmz proto kernel scope link src 192.168.222.1 > 192.168.254.0/24 dev vmbr14 proto kernel scope link src 192.168.254.7 > > With the BGP peer configuration automatically built for you on the exit > node in frr: > > router bgp 65001 vrf vrf_public > bgp router-id 10.10.21.7 > no bgp hard-administrative-reset > no bgp graceful-restart notification > neighbor BGP peer-group > neighbor BGP remote-as external > neighbor BGP bfd > neighbor 192.168.254.1 peer-group BGP > ! > address-family ipv4 unicast > redistribute connected > neighbor BGP soft-reconfiguration inbound > neighbor BGP route-map TO_PFSENSE out > exit-address-family > ! > address-family ipv6 unicast > redistribute connected > exit-address-family > ! > address-family l2vpn evpn > default-originate ipv4 > default-originate ipv6 > exit-address-family > exit Yep, this is the sort of config we're going for. Sadly when using EVPN+BGP, the EVPN peers have to be in the default VRF, because FRR doesn't allow EVPN peering in a custom VRF. So the config of EVPN Controller + BGP Fabric is probably going to look some thing like: vrf vrf_evpn vni 1000 exit-vrf ! router bgp 65000 bgp router-id 10.0.0.1 no bgp hard-administrative-reset no bgp default ipv4-unicast coalesce-time 1000 no bgp graceful-restart notification ! neighbor VTEP peer-group neighbor VTEP remote-as internal neighbor ens19 interface peer-group VTEP ! address-family l2vpn evpn neighbor VTEP activate advertise-all-vni exit-address-family ! exit ! router bgp 65011 vrf vrf_export bgp router-id 10.0.0.1 no bgp hard-administrative-reset no bgp default ipv4-unicast coalesce-time 1000 no bgp graceful-restart notification neighbor BGP peer-group neighbor BGP remote-as external neighbor ens21 interface peer-group BGP ! address-family ipv4 unicast neighbor BGP activate import vpn rt vpn import 65000:500 route-map vpn import IMPORT_RM exit-address-family ! exit ! router bgp 65000 vrf vrf_evpn bgp router-id 10.0.0.1 no bgp hard-administrative-reset no bgp graceful-restart notification ! address-family ipv4 unicast export vpn rd vpn export 65000:1 rt vpn export 65000:500 redistribute connected exit-address-family ! address-family ipv6 unicast redistribute connected exit-address-family ! exit ! access-list LOCAL seq 1 permit 10.0.0.1 access-list LOCAL seq 2 permit 0.0.0.0 ! route-map IMPORT_RM permit 20 match ip next-hop LOCAL route-map IMPORT_RM deny 30 > Regards, > > Hugh Blandford