From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id AE4B11FF0AB for ; Wed, 09 Sep 2026 15:01:23 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 55376215BB; Wed, 09 Sep 2026 15:01:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=glanzmann.de; s=infra26101010; t=1788958872; bh=2L5o8+kP8OopzKJq1UWdWSZpQt7qRqLL8t5mSQPe9+M=; h=Date:From:To:Cc:Subject:In-Reply-To:From; b=RhccYi8k2eAFw9p1wHyMUiLgWKGSP6Vfmy7VxlV4GcrDLiwxgrRDgZbrUpxJeowJR wh/NfFSdfgzBAkFb8WvYNLNmV2XScLJhPbOt6VO8mTlKiSFocpYbMcdNfTj0HF3FI2 ZEbuvBBc/0EV+tO+K+Wm262A/wi/mjByxbM6Q/Eyx708TSWB3pAJtrcvnOSMZtWluo JmejyxW9mcBXGaCfmpfFGkbVqg37ySfXNpH28mnRNQ4xPwLy9tnmHrmHk9Wgjfkykj wmebRMnvbeMNJr0ZKigxxcfx4ngz/ZTA32+SoLP+JP1bqSkPzqQE9I7uF5dbTeVHpe A6nDIvxhwIMwUzlsSRzMY0HqCAa2nWlm7R55TBzH2H2qOTTB1yQdrDEh1GQpmitoyG SLMMEJOLkv1m3n2Gb1w+Gm1CZl9GQpN6uFBC/Zpzjfmaap3T0IRVoz9tyhUg+UOpTu 0WEHVl74P4UXVOfNM6k+sjLRncdy2QwGcEPYwa1Y4WEVSzq8/ZkTHDeQ20hgLfjlLZ RzUV2nTiv1dw+D6lLBqiQiUGicThkOGuEmJ2KCCar9ur8XvCezECH4dg/d1Q8+vEFf a6cZqMq0IAAzy2o1Ji91VL/PvqxE3kXwuOdosAGEqyLXujcDhHEpGuXkaqJlbCjs1f WwTDXzFE0iMFhQlbmmVtbeNk= Date: Wed, 9 Sep 2026 15:01:12 +0200 From: Thomas Glanzmann To: Gabriel Goller Subject: Re: [PATCH] sdn: evpn: allow vlan-aware vnets Message-ID: MIME-Version: 1.0 In-Reply-To: X-SPAM-LEVEL: Spam detection results: 0 DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 5QJNDC2GZTXMNTBCZUQ7OH5VFFLGYDTX X-Message-ID-Hash: 5QJNDC2GZTXMNTBCZUQ7OH5VFFLGYDTX X-MailFrom: thomas@glanzmann.de X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 3.3.10 CC: pve-devel@lists.proxmox.com X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Hello Gabriel, > Yes, this should be fixed with https://github.com/FRRouting/frr/pull/22555. > Note that `show evpn vni x` shows the zebra vtep, and zebra can't find a > vtep as there is no vxlan-local-tunnelip set on the interface. bgpd will > fallback to the router-ip, so `show bgp l2vpn evpn route` should show routes > with the correct derived vtep ip. You are right and I was wrong. I re-measured it today and my "no type-3 is originated, no VTEP is learned, nothing forwards" was a misattribution - please disregard that part of my last mail. Details below, since I think the exact failure mode is worth having in the archive. Setup unchanged from my last mail: PVE 9.2.11 with stock libpve-network-perl 1.6.7, frr 10.6.1-1+pve3, ifupdown2 3.3.0-1+pmx12 (VTEP 10.101.0.18), against MikroTik CRS304-4XG on RouterOS 7.25beta3 (172.31.0.4), Cisco N9K-C9372TX on NX-OS 9.3(16) (172.31.0.7) and BIRD 3.3.2 (172.31.0.118). The fix is indeed in the package I tested with: frr (10.6.1-1+pve3) trixie; urgency=medium * fix VTEP derivation for VXLAN interfaces without a local IP address a5ad7bd "frr: backport VXLAN local VTEP fallback fix" debian/patches/upstream/0028-bgpd-zebra-fix-VXLAN-interface-local-IP-fallback.patch debian/patches/upstream/0029-tests-add-EVPN-L2VNI-unspecified-VTEP-topotest.patch 1. THE FALLBACK WORKS - my workaround file is gone for good I removed /etc/network/interfaces.d/zz-vxlan-local-tunnelip, so there is now no vxlan-local-tunnelip anywhere on the node, and the kernel device really has no local address: # ip -d link show vxlan_e1 21: vxlan_e1: mtu 9004 master e1 vxlan id 1 srcport 0 0 dstport 4789 ttl auto ageing 300 # vtysh -c 'show evpn vni 1' VNI: 1 / Type: L2 / Vlan: 1 / Bridge: e1 Local VTEP IP: (null) <- exactly as you describe Remote VTEPs for this VNI: 172.31.0.7 flood: HER 172.31.0.4 flood: HER 172.31.0.118 flood: HER # vtysh -c 'show bgp l2vpn evpn neighbors 172.31.0.7 advertised-routes' *> [3]:[0]:[32]:[10.101.0.18] <- derived from bgp router-id Total number of prefixes 1 and on the wire the outer header carries the same address, with the customer tag inside the VNI where it belongs (this one is the N9K leg, i.e. the QinVNI peer that is fussiest about all of this): 10.101.0.18.42551 > 172.31.0.7.4789: VXLAN, flags [I] (0x08), vni 1 bc:24:11:ec:33:f2 > 58:11:22:b6:19:fa, ethertype 802.1Q (0x8100), vlan 5, p 0, ethertype IPv4, 10.0.0.1 > 10.0.0.4: ICMP echo request All 12 directed pairs over {10.0.0.1 PVE VM, .3 MikroTik, .4 N9K, .5 BIRD} at 0% loss, `ping -M do -s 1338` passes and `-s 1340` fails end to end, so the 1366-byte tenant MTU is still exact to the byte. The N9K learns the guest MAC where it should, on the data-plane port and in the customer VLAN, not as a control-plane MAC in the VNI bridge domain: # show mac address-table * 5 bc24.11ec.33f2 dynamic 0 F F Eth1/3 and it survives a full `pvesh set /cluster/sdn` (regenerated interfaces + frr, still 0% loss). So the proposed VLAN-aware-VXLAN-zone + EVPN-controller + route-map setup does work out of the box on stock packages, with no patched Perl and no local tunnel IP, against all three foreign implementations. The "packaging gap" in my last mail does not exist; there is nothing to fix in VxlanPlugin.pm. One packaging-adjacent trap while re-testing, in case someone else chases it: after deleting a vxlan-local-tunnelip stanza, the FIRST `ifreload -a` still programs the old address, because the module-global vxlan._vxlan_local_tunnelip is also populated from the saved old ifaceobjs in get_dependent_ifacenames(). A second `ifreload -a` clears it. I mistook that for "the workaround is load bearing" more than once. 2. WHAT I ACTUALLY SAW - `Vlan: 0`, and it is not about the tunnel IP What I reported as "nothing forwards" came with `VNI: 1 / Vlan: 0 / Local VTEP IP: (null)`. The `Vlan: 0` is real and reproducible, but it is independent of vxlan-local-tunnelip. Control: systemctl restart frr, WITHOUT vxlan-local-tunnelip -> Vlan: 0 systemctl restart frr, WITH vxlan-local-tunnelip -> Vlan: 0 i.e. after an frr restart zebra does not re-resolve the bridge VLAN for the L2VNI. A subsequent `ifreload -a` does not heal it either; bouncing the vxlan interface (`ifdown vxlan_e1 && ifup vxlan_e1`) does, and then it is `Vlan: 1` again. In the `Vlan: 0` state the remote VTEPs stay learned and forwarding stays at 0% loss in my topology, so it is not fatal here - but zebra does advertise the local MAC as an Ethernet-Tag-0 type-2 in that state, which is precisely the route the route-map on the controller has to filter, so on a QinVNI peer such as the N9K it would matter (with the route-map in place it is filtered on the way out, which is why it stayed invisible). Tell me if you want that as a separate report, I can produce a clean reproducer. 3. THE ifupdown2 WARNING - patch attached > The ifupdown2 warning is only printed on the syntax-check, which gets > executed when using e.g. `ifreload -s`. We don't do that anywhere AFAICS. > Maybe we should patch this warning out as it's technically false. Confirmed on both counts. `_module_syntax_check()` in ifupdownmain.py is only reached under the `syntaxcheck` flag, and PVE only ever calls `ifreload -a` (API2/Network.pm) and `ifquery -a -c -o json` (SDN.pm, SDN/Zones.pm), neither of which takes that path. One thing to add though: it is not only a cosmetic warning. vxlan.py's syntax_check() *returns False* on the missing attribute, so `ifreload -s -a` exits non-zero on a perfectly good configuration - which is what an admin validating a config before applying it would see. Patch attached as git format-patch against the ifupdown2 packaging repo (new debian/patches/pve/0016-*.patch plus series entry). It drops the whole block rather than just the log line, for the return-value reason above; the clagd anycast check below it is unaffected and copes with local == None. Tested by applying it to /usr/share/ifupdown2/addons/vxlan.py on the node: before: warning: vxlan_e1: missing vxlan-local-tunnelip ; rc=1 after: (gone) 4. A SECOND FALSE POSITIVE IN THE SAME SYNTAX CHECK With the patch applied, `ifreload -s -a` still exits 1 on this configuration, because of bridge.py::_error_vxlan_in_vlan_aware_br(): warning: vxlan_e1: `bridge-access` attribute is mandatory when vxlan device (vxlan_e1) is part of vlan aware bridge (e1) That is a log_error(), so it also sets ifaceStatus.ERROR on the interface. It is equally false for the setup we are discussing: a VLAN bundle service maps a whole VLAN range into one VNI, so the vxlan port must be a trunk in the vlan-aware bridge and must NOT have bridge-access. It is exactly the config that the VXLAN zone plugin generates for a vlanaware vnet, so any EVPN-driven VLAN-aware zone hits it. I left it out of the attached patch since you only asked about the tunnel-ip one - say the word and I will send a second patch (it needs `and not ` rather than a plain removal, since the check is legitimate for the VNI-per-VLAN case). Cheers, Thomas