Re: [PVE-User] Experimenting with bond on a non-LACP switch...

[Wolf Noble <wolf@wolfspyre.com>]

[-- Attachment #1: Type: text/plain, Size: 1249 bytes --]


Good Catch Marco!

I'd not seen that when I read through that page, but I just re-read it.... My read is that it can introduce ODD edge-case complications.

my synthesis of this information is  outlined here. I encourage anyone to correct my misunderstandings. 

network abstraction gets complicated QUICKLY.  Network gear vendors implement their support for the different bonding modes in subtly different ways.
Firewalls have their own quirks.

abstractions on top of abstractions on top of abstractions on top of abstractions on top of .... okay you get the point.


we want to avoid asymmetric pathing where possible, because stuff gets quirky and edge-casey quickly. the fewer explicitly supported virtual topologies, the fewer scenarios the engineering teams need to scrutinize the COMPLEX edge case behaviors of, resulting in a better experience for EVERYONE.... heres what I mean:

LACP: 
This is a pretty well known and consistently implemented aggregation mechanism. the behaviors of network interfaces and switching hardware that are involved are pretty consistent. 

This GENERALLY works fine. the only time I've seen it get a little wonky is LACP across switch chassis behavior can be odd..
                       

[-- Attachment #2: Type: text/plain, Size: 2938 bytes --]




When node 2 sends traffic destined for node 1 via lacp_b.1 ... it may traverse the trunked interconnect, it may not. (it depends)



active-backup:  

I talk, and listen on link0... if link 0 goes down, I switchover... I don't really pay attention to linkl1 otherwise.

active-passive:
  I talk on link0. I listen on link0 and link1.

The downside that I've seen here:


Arp caching can get wonky, and packets that SHOULD be directed to node0 link0 
        get directed to node0 link1... or  sometimes packets directed to node0.link0 have a destination mac address of the hwaddr of link1 and so get delivered to link1 ...  

There MAY be some oddities that manifest with this configuration. 

depending on (node scope configuration) sysctl settings, node0 could just ignore those packets, resulting in weird behavior

with the various balance algorithms nodes will see a different hardware addresses for each other, again, this isn't *USUALLY* a problem, but there are still some dragons that lurk within the trunking/bonding code... hardware checksumming can get whacky...  especially when VLANs get mixed in... 

My gut tells me that the main reason for this advise is that using LACP or active/backup provides sufficient durability while introducing as little edge-case wonky as possible, which generally speaking is a GoodThing™️ when it comes to intra-cluster-comms.


I could be wrong, so don't take this as gospel... if anyone has a better explanation, or can point out my flawed logic, by all means, chime in! :) 


Hope my understanding HELPS... if it doesn't, throw it away and ignore it ;)

🐺W



This message created and transmitted using 100% recycled electrons.

> On May 21, 2022, at 03:11, Marco Gaiarin <gaio@lilliput.linux.it> wrote:
> 
> 
> I'm doing some experimentation on a switch that seems does not support LACP,
> even thus claim that; is a Netgear GS724Tv2:
> 
>    https://www.downloads.netgear.com/files/GDC/GS724Tv2/enus_ds_gs724t.pdf
> 
> data sheet say:
> 
>    Port Trunking - Manual as per IEEE802.3ad Link Aggregation
> 
> and 'IEEE802.3ad Link Aggregation' is LACP, right?
> 
> 
> Anyway, i'm experimenting a bit with other bonding mode, having (un)expected
> results and troubles, but in:
> 
>    https://pve.proxmox.com/wiki/Network_Configuration#_linux_bond
> 
> i've stumble upon that sentence:
> 
>    If you intend to run your cluster network on the bonding interfaces, then you have to use active-passive mode on the bonding interfaces, other modes are unsupported.
> 
> What exactly mean?! Thanks.
> 
> -- 
>  Molti italiani sognavano di vedere Berlusconi in un cellulare,
>  prima o poi...            (Stardust®, da i.n.n-a)
> 
> 
> 
> _______________________________________________
> pve-user mailing list
> pve-user@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>