[PVE-User] Kernel panics when using OpenVSwitch bridges (Proxmox VE 7.3)

[PVE-User] Kernel panics when using OpenVSwitch bridges (Proxmox VE 7.3)

[Benjamin Hofer]

Hello community,

we're using OpenVSwitch bridges on a productive Proxxmox 7.3 cluster with 4
nodes (different hardware). Some weeks ago, a sudden reboot happened on one
of the cluster nodes. Further analysis showed that we had  kernel panics /
CPU stalls which seem to be related to OpenVSwitch. After more analysis, we
found out that we're able to reliably reproduce the OVS related kernel
panic jst by restarting running LXC containers with network interfaces. The
behaviour could be reproduced on all our nodes. As these nodes are quite
different in their hardware specifications, we assume that it's caused by
some software-related (OVS) bug.

The kernel panics do NOT occur when switching to linux bridged on a node.

See kernel log extract attached.

Did someone have a similar behaviour?
What experiences do you have with linux bridges compared to OVS bridges
regarding network performance?

We could do without OVS features but must rely on good enough performance.

System:
pveversion: pve-manager/7.3-6/723bb6ec (running kernel: 5.15.102-1-pve)
OVS 2.15.0

Thank you in advance.

All the best
Benjamin

Re: [PVE-User] Kernel panics when using OpenVSwitch bridges (Proxmox VE 7.3)

[DERUMIER, Alexandre]

bridge performance is same (without the bugs).


The only was to have better performance with ovs, is to implemented
dpdk + vhost-user nic, and it's not implemented in proxmox currently.

Le mercredi 31 mai 2023 à 14:28 +0200, Benjamin Hofer a écrit :
> Hello community,
> 
> we're using OpenVSwitch bridges on a productive Proxxmox 7.3 cluster
> with 4
> nodes (different hardware). Some weeks ago, a sudden reboot happened
> on one
> of the cluster nodes. Further analysis showed that we had  kernel
> panics /
> CPU stalls which seem to be related to OpenVSwitch. After more
> analysis, we
> found out that we're able to reliably reproduce the OVS related
> kernel
> panic jst by restarting running LXC containers with network
> interfaces. The
> behaviour could be reproduced on all our nodes. As these nodes are
> quite
> different in their hardware specifications, we assume that it's
> caused by
> some software-related (OVS) bug.
> 
> The kernel panics do NOT occur when switching to linux bridged on a
> node.
> 
> See kernel log extract attached.
> 
> Did someone have a similar behaviour?
> What experiences do you have with linux bridges compared to OVS
> bridges
> regarding network performance?
> 
> We could do without OVS features but must rely on good enough
> performance.
> 
> System:
> pveversion: pve-manager/7.3-6/723bb6ec (running kernel: 5.15.102-1-
> pve)
> OVS 2.15.0
> 
> Thank you in advance.
> 
> All the best
> Benjamin
> _______________________________________________
> pve-user mailing list
> pve-user@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>