From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id C852C1FF17E for ; Thu, 2 Oct 2025 19:05:12 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id A02FB1571C; Thu, 2 Oct 2025 19:05:19 +0200 (CEST) Date: Thu, 2 Oct 2025 19:04:44 +0200 From: Gabriel Goller To: Stefan Hanreich Message-ID: Mail-Followup-To: Stefan Hanreich , Proxmox VE development discussion References: <20250916124108.209042-1-g.goller@proxmox.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20241002-35-39f9a6 X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1759424660903 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.004 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] [PATCH perl-rs v2] fix: sdn: fabrics: always add node-ip to all fabric interfaces X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Cc: Proxmox VE development discussion Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" On 01.10.2025 17:00, Stefan Hanreich wrote: >Have some additional comments I wanted to post on the ML so they don't >get lost, but they're more food for though for further improvements of >the whole SDN / Fabrics stack wrt VRFs. > >Tested the patch series and looked at the code - LGTM! > >Reviewed-by: Stefan Hanreich >Tested-by: Stefan Hanreich Thanks for the review! >On 9/16/25 2:41 PM, Gabriel Goller wrote: >> [snip] >> In this scenario we ping from Node1 to Node3, which means that Node2 >> needs to forward our packet. When the packet arrives at Node2, we again >> check the routing table, where we see the same entry as before, so >> 10.0.1.3 is available at (e.g.) ens20 onlink with the src address >> 10.0.1.2. This is fine, but we still need to do an ARP request to lookup >> the mac address of the neighbor which is attached at ens20. So we take >> the packet we get from Node1, which has a source address of 10.0.1.1, and >> call arp_solicit on it to make an ARP request and find the neighbors mac >> address. arp_solicit will take the source address of the packet >> (10.0.1.1) and lookup to search it locally. This check fails because >> 10.0.1.1 is not available locally (there is a direct route to it, but >> it's not configured on any local interface (RTN_LOCAL)). arp_solicit >> will thus [2] call inet_select_addr, which goes through all the ip >> addresses on the current interface (there are none, because this >> interface is unnumbered) and then iterate through all the other >> interfaces on the host and select the first one with 'scope link'. This >> ip will then be selected as the source address for the outgoing ARP >> package. Now if we're lucky this is the dummy interface on our node and >> we select the correct source address (10.0.1.2) -- but we could also be >> unlucky and it selects a completely random address from another >> interface e.g. 172.16.0.26. If we're unlucky arp_solicit will send out >> the following ARP packet: >> >> Request who-has 10.0.1.3 tell 172.16.0.26 >> >> We will get a correct response but the response will end up on another >> interface (because 172.16.0.26 is not on the same interface as >> 10.0.1.2). This means we will send out these ARP requests repeatedly and >> never get an answer, so the ping from Node1 to Node3 will get >> "Destination host unreachable errors". > >Interesting, that the src IP address directive from the route is not >considered at all: > >172.16.123.2 nhid 162 via 172.16.123.2 dev ens22 [..] src 172.16.123.1 > >Didn't dig further into it, there's a good chance there's a good reason >for that I just didn't see immediately. Hmm yeah, I'll look into this. Currently the whole neighbor system is quite generic and limited, I'll see if there is any way we can add "hints" to the arp_solicit function when it's called from ip_forward. >What I also found interesting while jumping down this rabbit hole is the >following comment / code section in the inet_select_addr function [1]: > >/* For VRFs, the VRF device takes the place of the loopback device, > * with addresses on it being preferred. Note in such cases the > * loopback device will be among the devices that fail the master_idx > * equality check in the loop below. > */ > >So, in that case (iiuc) one could side-step that problem by >compartmentalizing the fabric inside its own VRF (further reinforcing my >belief in implementing VRF support sooner than later to avoid issues >like this when running everything in one routing table, particularly >multiple fabrics). > >fabricd has no VRF support atm though (could potentially run it via ip >vrf exec, but that seems hacky) - OSPF and BGP do. I agree, in theory this would be very nice, but hacking vrf support into lots of stuff that doesn't have vrf support might be tricky. Another example is wireguard, where vrf support is also kinda tricky. I'll look into OpenFabric VRF support though, this shouldn't be too tricky to implement as ISIS already supports it. >[1] >https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/net/ipv4/devinet.c?h=v6.17#n1359 > >> For more information check out the simplified version of the arp_solicit >> function below: >> [snip] _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel