From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <pve-user-bounces@lists.proxmox.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id CE2C81FF15E for <inbox@lore.proxmox.com>; Tue, 11 Mar 2025 09:15:24 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id E109F8799; Tue, 11 Mar 2025 09:15:12 +0100 (CET) From: "Schunke, Alexander" <schunke@eth.mpg.de> To: Proxmox VE user list <pve-user@lists.proxmox.com> Thread-Topic: [PVE-User] Inter VRF traffic Thread-Index: AQHbkfydxJoGHWjvXkqimDXhD7X38LNtjJIw Date: Tue, 11 Mar 2025 07:50:40 +0000 Message-ID: <c5febd155c884b11b2c644a597d3ea1d@eth.mpg.de> References: <CAEaLa5E2VUvhecTwpiR5FGWU1xp3BKB-r31JsGQP+hVHvQwt5A@mail.gmail.com> In-Reply-To: <CAEaLa5E2VUvhecTwpiR5FGWU1xp3BKB-r31JsGQP+hVHvQwt5A@mail.gmail.com> Accept-Language: en-US, de-DE Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.250.9.205] MIME-Version: 1.0 X-Virus-Scanned: (clean) by clamav X-SPAM-LEVEL: Spam detection results: 0 BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_LOW -0.7 Sender listed at https://www.dnswl.org/, low trust RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [pastebin.com, proxmox.com] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [PVE-User] Inter VRF traffic X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list <pve-user.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-user>, <mailto:pve-user-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-user/> List-Post: <mailto:pve-user@lists.proxmox.com> List-Help: <mailto:pve-user-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user>, <mailto:pve-user-request@lists.proxmox.com?subject=subscribe> Reply-To: Proxmox VE user list <pve-user@lists.proxmox.com> Content-Type: multipart/mixed; boundary="===============8740897285513246720==" Errors-To: pve-user-bounces@lists.proxmox.com Sender: "pve-user" <pve-user-bounces@lists.proxmox.com> --===============8740897285513246720== Content-Language: en-US Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=SHA1; boundary="----=_NextPart_000_01A4_01DB9262.A9CE4700" ------=_NextPart_000_01A4_01DB9262.A9CE4700 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Hi Cyrus, from a first glance I would say this makes sense since from what I understand the VMs are running on the same server and are thus connected to the same bridge. Since the bridge knows both MAC addresses it forwards the traffic between the VMs at layer 2. In other words, the packages never get to layer 3 and thus never get routed upstream. What you could try is to enable "Isolate Ports" in your vnet config.[1] This should set the `isolated` flag on each interface connected to the bridge, forcing the kernel to route the packages. At least in theory. I am yet to migrate my setup to SDN to make use of that future myself. All the best, Alex. [1]: https://pve.proxmox.com/wiki/Software-Defined_Network#pvesdn_config_vnet -----Original Message----- From: pve-user <pve-user-bounces@lists.proxmox.com> On Behalf Of Cyrus Sent: 10 March 2025 21:39 To: pve-user@lists.proxmox.com Subject: [PVE-User] Inter VRF traffic Hello!, I'm trying to make traffic work between VRFs passing through a an external firewall (opnsense+frr) but traffic seems to be resolved locally by the node, even though source/destination are on different VRFs (and ultimately doesn't work): root@pve-01:~/bin# ip route get 192.168.111.10 192.168.111.10 via 192.168.203.145 dev vrfbr_L01VPN01 src 192.168.203.212 uid 0 cache root@pve-01:~/bin# ip route get 192.168.111.10 192.168.111.10 dev ol111001 src 192.168.111.1 uid 0 cache root@pve-01:~/bin# ip addr show dev ol111001 191: ol111001: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vrf_L01VPN01 state UP group default qlen 1000 link/ether bc:24:11:e6:34:58 brd ff:ff:ff:ff:ff:ff inet 192.168.111.1/25 scope global ol111001 valid_lft forever preferred_lft forever inet6 fe80::be24:11ff:fee6:3458/64 scope link valid_lft forever preferred_lft forever root@pve-01:~/bin# ip addr show dev ol107001 63: ol107001: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vrf_SDCVPN01 state UP group default qlen 1000 link/ether bc:24:11:a9:f9:46 brd ff:ff:ff:ff:ff:ff inet 192.168.107.1/27 scope global ol107001 valid_lft forever preferred_lft forever inet6 fe80::be24:11ff:fea9:f946/64 scope link valid_lft forever preferred_lft forever "ip r" output: https://pastebin.com/Q9sF8uMv "frr.conf.local" content: https://pastebin.com/KAqNqKB1 rendered "frr.conf": https://pastebin.com/gUpYnuc0 /etc/pve/sdn/*: https://pastebin.com/U7yjNe5N "/etc/network/interfaces" for pve-01: https://pastebin.com/smEfYUJw North/South traffic works anytime destination is an external network. North/South traffic fails if destination is the host or a network in another VRF and traffic should be forwarded via an external firewall. Any hints?, is something missing? Regards, _______________________________________________ pve-user mailing list pve-user@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user ------=_NextPart_000_01A4_01DB9262.A9CE4700-- --===============8740897285513246720== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-user mailing list pve-user@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user --===============8740897285513246720==--