From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 7C00D721AF for ; Sun, 22 May 2022 06:13:10 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 6A3276B1 for ; Sun, 22 May 2022 06:12:40 +0200 (CEST) Received: from mail-oi1-x22f.google.com (mail-oi1-x22f.google.com [IPv6:2607:f8b0:4864:20::22f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 30CA469C for ; Sun, 22 May 2022 06:12:38 +0200 (CEST) Received: by mail-oi1-x22f.google.com with SMTP id q10so14285785oia.9 for ; Sat, 21 May 2022 21:12:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wolfspyre-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:from:mime-version:subject:message-id :references:in-reply-to:to:date; bh=PA6HMBHEigMaswb6QgE0rm4CkfWOWu162EIOci234Jo=; b=3Kgv8SrDIA7tayHxXgOHQ1Zt3AjSEmjGziLTVENR6v+3Y9Hm4zMkxqokRupXFVDlGa eSczl9l3d8vWulNjqKeffyNaIZaxVHLxtPTGMVBuBxfn9aayKPqxjp/PtX7BqSNc3FO4 UGPj+YN61JlBzmAdiS3/Ad9YwzHCI/VrjgRmvsl69EwasJjwoymfO8Yd5iUBvPHtiCfk ZDRdbLDSqkgnwld9ebJGBwgYbzpSOBtG07FgizCYqaobd4Q2MGkXrJLDqs+VsDDe5MqP yGjgJe3SrI9U2by+mcnb4AGXQqTMtxsMaaSXT7c2ntOEKjTY7QOzqOUhXSENyjQ/4Nck onVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:message-id:references:in-reply-to:to:date; bh=PA6HMBHEigMaswb6QgE0rm4CkfWOWu162EIOci234Jo=; b=qvm8/GaDuYcQPBXWIs+ZhnY5FPo1CoduWfPLhsLUtHlmfAfDBZbgzr9DxvCCzhFo6M FBdXwxpknmz4h5wvcmxqg1WpqlWjvyKPDA6NdNNIyx1jr//q9Cx94dX8wklY0KjJDuad 0fypzkl6waUfEtA3AWuiVqXRjGDyd9sWuuR/qs9OfXtuRU+iAsc2SYlMatCKKtiBwNBm HZRcnUNg0+3MSX5ZMAglY8M6moI/tTg6I3iF5S5Y2yf0JUfiS2PcEldHscS+9unsmv/M ZkH8oNgghP2NFUHluTf/lkgLJ4Vap4AWIGvBfuiRDW9Dpk4OsFB6TA0EfaNZPfOiw8bg npUQ== X-Gm-Message-State: AOAM53060LeNCEZUaf6V3TkTlAkVdy5+rhtPkaTfUUXzp5irDninGYdJ lBaQJm7sqZftyudSFijoCufgXA== X-Google-Smtp-Source: ABdhPJx8MND4Q3KvcIHv7VKFXnWPOV8n80meWJnTJPrGlUsv0NpbFzFHZI9/JvjISQaYcw9Kr7I4gw== X-Received: by 2002:a05:6808:1416:b0:326:a373:84d4 with SMTP id w22-20020a056808141600b00326a37384d4mr9175916oiv.132.1653192750745; Sat, 21 May 2022 21:12:30 -0700 (PDT) Received: from smtpclient.apple (who.wolfspaw.com. [108.221.46.19]) by smtp.gmail.com with ESMTPSA id z21-20020a05687042d500b000e686d13872sm2563606oah.12.2022.05.21.21.12.28 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 21 May 2022 21:12:29 -0700 (PDT) Content-Type: multipart/mixed; boundary=Apple-Mail-CBEFEEF5-7F1E-41F3-9065-D3671D3DE3FB Content-Transfer-Encoding: 7bit From: Wolf Noble Mime-Version: 1.0 (1.0) Message-Id: References: In-Reply-To: To: Proxmox VE user list Date: Sat, 21 May 2022 23:12:28 -0500 X-Mailer: iPad Mail (19F77) X-SPAM-LEVEL: Spam detection results: 0 AWL 0.003 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [PVE-User] Experimenting with bond on a non-LACP switch... X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 May 2022 04:13:10 -0000 --Apple-Mail-CBEFEEF5-7F1E-41F3-9065-D3671D3DE3FB Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Good Catch Marco! I'd not seen that when I read through that page, but I just re-read it.... M= y read is that it can introduce ODD edge-case complications. my synthesis of this information is outlined here. I encourage anyone to co= rrect my misunderstandings.=20 network abstraction gets complicated QUICKLY. Network gear vendors implemen= t their support for the different bonding modes in subtly different ways. Firewalls have their own quirks. abstractions on top of abstractions on top of abstractions on top of abstrac= tions on top of .... okay you get the point. we want to avoid asymmetric pathing where possible, because stuff gets quirk= y and edge-casey quickly. the fewer explicitly supported virtual topologies,= the fewer scenarios the engineering teams need to scrutinize the COMPLEX ed= ge case behaviors of, resulting in a better experience for EVERYONE.... here= s what I mean: LACP:=20 This is a pretty well known and consistently implemented aggregation mechani= sm. the behaviors of network interfaces and switching hardware that are invo= lved are pretty consistent.=20 This GENERALLY works fine. the only time I've seen it get a little wonky is L= ACP across switch chassis behavior can be odd.. =20 --Apple-Mail-CBEFEEF5-7F1E-41F3-9065-D3671D3DE3FB Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable When node 2 sends traffic destined for node 1 via lacp_b.1 ... it may traver= se the trunked interconnect, it may not. (it depends) active-backup: =20 I talk, and listen on link0... if link 0 goes down, I switchover... I don't r= eally pay attention to linkl1 otherwise. active-passive: I talk on link0. I listen on link0 and link1. The downside that I've seen here: Arp caching can get wonky, and packets that SHOULD be directed to node0 link= 0=20 get directed to node0 link1... or sometimes packets directed to nod= e0.link0 have a destination mac address of the hwaddr of link1 and so get de= livered to link1 ... =20 There MAY be some oddities that manifest with this configuration.=20 depending on (node scope configuration) sysctl settings, node0 could just ig= nore those packets, resulting in weird behavior with the various balance algorithms nodes will see a different hardware addr= esses for each other, again, this isn't *USUALLY* a problem, but there are s= till some dragons that lurk within the trunking/bonding code... hardware che= cksumming can get whacky... especially when VLANs get mixed in...=20 My gut tells me that the main reason for this advise is that using LACP or a= ctive/backup provides sufficient durability while introducing as little edge= -case wonky as possible, which generally speaking is a GoodThing=E2=84=A2=EF= =B8=8F when it comes to intra-cluster-comms. I could be wrong, so don't take this as gospel... if anyone has a better exp= lanation, or can point out my flawed logic, by all means, chime in! :)=20 Hope my understanding HELPS... if it doesn't, throw it away and ignore it ;)= =F0=9F=90=BAW This message created and transmitted using 100% recycled electrons. > On May 21, 2022, at 03:11, Marco Gaiarin wrote: >=20 > =EF=BB=BF > I'm doing some experimentation on a switch that seems does not support LAC= P, > even thus claim that; is a Netgear GS724Tv2: >=20 > https://www.downloads.netgear.com/files/GDC/GS724Tv2/enus_ds_gs724t.pdf= >=20 > data sheet say: >=20 > Port Trunking - Manual as per IEEE802.3ad Link Aggregation >=20 > and 'IEEE802.3ad Link Aggregation' is LACP, right? >=20 >=20 > Anyway, i'm experimenting a bit with other bonding mode, having (un)expect= ed > results and troubles, but in: >=20 > https://pve.proxmox.com/wiki/Network_Configuration#_linux_bond >=20 > i've stumble upon that sentence: >=20 > If you intend to run your cluster network on the bonding interfaces, th= en you have to use active-passive mode on the bonding interfaces, other mode= s are unsupported. >=20 > What exactly mean?! Thanks. >=20 > --=20 > Molti italiani sognavano di vedere Berlusconi in un cellulare, > prima o poi... (Stardust=C2=AE, da i.n.n-a) >=20 >=20 >=20 > _______________________________________________ > pve-user mailing list > pve-user@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user >=20 --Apple-Mail-CBEFEEF5-7F1E-41F3-9065-D3671D3DE3FB--