From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 403C675272 for ; Wed, 13 Oct 2021 01:03:58 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 2FA14A42C for ; Wed, 13 Oct 2021 01:03:28 +0200 (CEST) Received: from host.marinov.us (host.marinov.us [212.56.5.50]) by firstgate.proxmox.com (Proxmox) with ESMTP id B2F8CA417 for ; Wed, 13 Oct 2021 01:03:26 +0200 (CEST) Received: from localhost (unknown [127.0.0.1]) by host.marinov.us (Postfix) with ESMTP id 5CA5A2027EC for ; Wed, 13 Oct 2021 02:03:26 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=marinov.us; h= x-mailer:message-id:in-reply-to:to:references:date:date:subject :subject:mime-version:content-type:content-type:from:from; s= dkim; t=1634079806; x=1634943807; bh=K64WoPCgwypYdEzf2E0dnzG9yga KljrxD9+ooq5eo8Y=; b=bzwuCxr1EX8LFSlc0Blv4cEqHhbywDvmV/0OmUXIimF 61YxM+26QBgJEyjWPzP0ujEO+aNa2wWU//IOEuADG8jujrvy78LeRb1Ihjc51Bv4 TnIb0+A4LFNxazwXL52njdj5xG2bIsY5mopUM7TBLO5usUXBQr92Z/LhXcYW4ZBw = Received: from host.marinov.us ([127.0.0.1]) by localhost (host.marinov.us [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Op_J_6jkpW_H for ; Wed, 13 Oct 2021 02:03:26 +0300 (EEST) Received: from smtpclient.apple (unknown [62.73.121.18]) by host.marinov.us (Postfix) with ESMTPSA id 38BF7201A94 for ; Wed, 13 Oct 2021 02:03:26 +0300 (EEST) From: Stoyan Marinov Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.22\)) Date: Wed, 13 Oct 2021 02:03:25 +0300 References: <2b417bee43cb4484bcba66afc6076113@velartis.at> <093EC041-0E5D-41F2-99C9-CF8A5E767313@marinov.us> <4F0DFA30-F1ED-4322-857A-4F4C24B463FE@marinov.us> <1FAB115F-FD40-41E1-AC81-A781DA29B378@marinov.us> To: Proxmox VE development discussion In-Reply-To: <1FAB115F-FD40-41E1-AC81-A781DA29B378@marinov.us> Message-Id: <284AFD0B-54B1-49EE-9211-2BD3F90C44F4@marinov.us> X-Mailer: Apple Mail (2.3654.100.0.2.22) X-SPAM-LEVEL: Spam detection results: 0 AWL 0.266 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain HTML_MESSAGE 0.001 HTML included in message KAM_INFOUSMEBIZ 0.75 Prevalent use of .info|.us|.me|.me.uk|.biz|xyz|id|rocks|life domains in spam/malware SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [marinov.us, proxmox.com] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [pve-devel] BUG in vlan aware bridge X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Oct 2021 23:03:58 -0000 Alright, I said tomorrow, but I did a bit more fiddling with tcpdump and = that's quite odd: 01:59:31.462866 d2:24:de:13:3d:6e > c6:29:5b:54:e3:b9, ethertype IPv4 = (0x0800), length 1514: 10.3.4.111 > 192.168.0.220: ICMP echo reply, id = 112, seq 29, length 1480 01:59:31.462866 d2:24:de:13:3d:6e > c6:29:5b:54:e3:b9, ethertype 802.1Q = (0x8100), length 566: vlan 10, p 0, ethertype IPv4, 10.3.4.111 > = 192.168.0.220: ip-proto-1 01:59:32.486719 d2:24:de:13:3d:6e > c6:29:5b:54:e3:b9, ethertype IPv4 = (0x0800), length 1514: 10.3.4.111 > 192.168.0.220: ICMP echo reply, id = 112, seq 30, length 1480 01:59:32.486719 d2:24:de:13:3d:6e > c6:29:5b:54:e3:b9, ethertype 802.1Q = (0x8100), length 566: vlan 10, p 0, ethertype IPv4, 10.3.4.111 > = 192.168.0.220: ip-proto-1 It seems like the first fragment arrives properly and the 2nd one is a = vlan tagged ethernet frame. That's weird. On top of all I noticed that if I run a tcpdump on proxmox host, = listening on vmbr0 - it all works! > On 13 Oct 2021, at 1:45 AM, Stoyan Marinov wrote: >=20 > OK, I have just verified it has nothing to do with bonds. I get the = same behavior with vlan aware bridge, bridge-nf-call-iptables=3D1 with = regular eth0 being part of the bridge. Packets arrive fragmented on tap, = reassembled by netfilter and then re-injected in bridge assembled (full = size). >=20 > I did have limited success by setting = net.bridge.bridge-nf-filter-vlan-tagged to 1. Now packets seem to get = fragmented on the way out and back in, but there are still issues: >=20 > 1. I'm testing with ping -s 2000 (1500 mtu everywhere) to an external = box. I do see reply packets arrive on the vm nic, but ping doesn't see = them. Haven't analyzed much further. > 2. While watching with tcpdump (inside the vm) i notice "ip reassembly = time exceeded" messages being generated from the vm. >=20 > I'll try to investigate a bit further tomorrow. >=20 >> On 12 Oct 2021, at 11:26 PM, Stoyan Marinov = wrote: >>=20 >> That's an interesting observation. Now that I think about it, it = could be caused by bonding and not the underlying device. When I tested = this (about an year ago) I was using bonding on the mlx adapters and not = using bonding on intel ones. >>=20 >>> On 12 Oct 2021, at 3:36 PM, VELARTIS Philipp D=C3=BCrhammer = wrote: >>>=20 >>> HI, >>>=20 >>> we use HP Server with Intel Cards or the standard hp nic ( ithink = also intel) >>>=20 >>> Also I see the I did a mistake: >>>=20 >>> Setup working: >>> tapX (UNtagged) <- -> vmbr0 <- - > bond0 >>>=20 >>> is correct. (before I had also tagged)=20 >>>=20 >>> it should be : >>>=20 >>> Setup not working: >>> tapX (tagged) <- -> vmbr0 <- - > bond0 >>>=20 >>> Setup working: >>> tapX (untagged) <- -> vmbr0 <- - > bond0 >>>=20 >>> Setup also working: >>> tapX < - - > vmbr0v350 < -- > bond0.350 < -- > bond0 >>>=20 >>> -----Urspr=C3=BCngliche Nachricht----- >>> Von: pve-devel Im Auftrag von = Stoyan Marinov >>> Gesendet: Dienstag, 12. Oktober 2021 13:16 >>> An: Proxmox VE development discussion >>> Betreff: Re: [pve-devel] BUG in vlan aware bridge >>>=20 >>> I'm having the very same issue with Mellanox ethernet adapters. I = don't see this behavior with Intel nics. What network cards do you have? >>>=20 >>>> On 12 Oct 2021, at 1:48 PM, VELARTIS Philipp D=C3=BCrhammer = wrote: >>>>=20 >>>> HI, >>>>=20 >>>> i am playing around since days because we have strange packet = losses. >>>> Finally I can report following (Linux 5.11.22-4-pve, Proxmox 7, all = devices MTU 1500): >>>>=20 >>>> Packet with sizes > 1500 without VLAN working well but at the = moment they are Tagged they are dropped by the bond device. >>>> Netfilter (set to 1) always reassembles the packets when they = arrive a bridge. But they don't get fragmented again I they are VLAN = tagged. So the bond device drops them. If the bridge is NOT Vlan aware = they also get fragmented and it works well. >>>>=20 >>>> Setup not working: >>>>=20 >>>> tapX (tagged) <- -> vmbr0 <- - > bond0 >>>>=20 >>>> Setup working: >>>>=20 >>>> tapX (tagged) <- -> vmbr0 <- - > bond0 >>>>=20 >>>> Setup also working: >>>>=20 >>>> tapX < - - > vmbr0v350 < -- > bond0.350 < -- > bond0 >>>>=20 >>>> Have you got any idea where to search? I don't understand who is in = charge of fragmenting packages again if they get reassembled by = netfilter. (and why it is not working with vlan aware bridges) >>>>=20 >>>>=20 >>>> _______________________________________________ >>>> pve-devel mailing list >>>> pve-devel@lists.proxmox.com >>>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel >>>>=20 >>>=20 >>>=20 >>> _______________________________________________ >>> pve-devel mailing list >>> pve-devel@lists.proxmox.com >>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel >>> _______________________________________________ >>> pve-devel mailing list >>> pve-devel@lists.proxmox.com >>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel >>=20 >>=20 >> _______________________________________________ >> pve-devel mailing list >> pve-devel@lists.proxmox.com >> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel >=20 >=20 > _______________________________________________ > pve-devel mailing list > pve-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel