[PATCH pve-docs v3 1/1] network/sdn: update and clean up VLAN information

[Thomas Ellmenreich <t.ellmenreich@proxmox.com>]

Adjusted some of the details related to VLAN setups, both with the traditional
methods as well as the SDN setup.

- Removed an outdated example and added more information to the remaining
  examples. First, by adding the VLAN interface on the bridge, and then
  showcasing that VLAN tags can be set in the guest network configurations.

- Added SDN as a possible way to configure VLANs for guests, as well as adding
  an example for SDN.

- Added a "Common Issues" section to collect common networking issues.

- Added a "Troubleshooting" section, to collect information on how to fix VLAN
  related issues.

Suggested-by: Stefan Hanreich <s.hanreich@proxmox.com>
Signed-off-by: Thomas Ellmenreich <t.ellmenreich@proxmox.com>
---
Some changes for the VLAN configuration documentation, would apprechiate any
help in regards to the wording and details of said documentation.

Changelog:

- PATCH v2 -> v3
    * Removed the broken examples in the "Common Issues" section so that
      they are not copied as working configs.

    * Rewrote all of the "Common Issues" to have very precise wording.

    * A few smaller adjustments and corrections

- PATCH v1 -> v2
    * Added a "Common Issues" section, that recounts different problems that
      might appear.

    * Added a "Troubleshooting" section to show different ways to troubleshoot
      problems with the networking.

    * Added a SDN VLAN example.

    * Some rewording, and adjustment of different sections.

 pve-network.adoc | 197 +++++++++++++++++++++++++++++++++++++----------
 pvesdn.adoc      |   2 +-
 2 files changed, 157 insertions(+), 42 deletions(-)

diff --git a/pve-network.adoc b/pve-network.adoc
index 2b3e664..4d8e6b9 100644
--- a/pve-network.adoc
+++ b/pve-network.adoc
@@ -637,6 +637,54 @@ completely done inside the guest and can not be influenced from the
 outside. The benefit is that you can use more than one VLAN on a
 single virtual NIC.
 
+* *SDN configured VLAN:*
+It is also possible to xref:pvesdn_config_vnet[configure VLANs in SDN] by
+creating a VLAN zone on the bridge and then creating VNets for every VLAN tag.
+In this case, it is also recommended to use a VLAN-aware bridge.
+
+[[vlan_example_vlan_through_sdn]]
+.Example: SDN VLAN Zones with VLAN-aware bridges
+
+This is the preferred solution if users have a small number of VLANs inside
+their cluster. On every node, users should configure a VLAN-aware bridge with
+the respective carrier for their VLANs:
+
+----
+auto vmbr0
+iface vmbr0 inet manual
+        bridge-ports nic0
+        bridge-stp off
+        bridge-fd 0
+        bridge-vlan-aware yes
+        bridge-vids 2-4094
+----
+
+Then create an SDN zone on top of that bridge:
+
+----
+vlan: vlanz
+        bridge vmbr0
+        ipam pve
+----
+
+Then, for each VLAN they want to use, create a VNet:
+
+----
+vnet: vnet1234
+        zone vlanz
+        tag 1234
+----
+
+After applying the SDN configuration, users can then select vnet1234 in the
+network device configuration of their guests. This method does not allow to
+configure an IP for the host directly on a VNet - but it can be done manually
+by adding an entry for the VNet to /etc/network/interfaces:
+
+----
+auto vnet1234
+iface vnet1234 inet static
+        address 192.0.2.1/24
+----
 
 VLAN on the Host
 ^^^^^^^^^^^^^^^^
@@ -644,63 +692,49 @@ VLAN on the Host
 To allow host communication with an isolated network. It is possible
 to apply VLAN tags to any network device (NIC, Bond, Bridge). In
 general, you should configure the VLAN on the interface with the least
-abstraction layers between itself and the physical NIC.
+abstraction layers between itself and the physical NIC. But it would still
+be best to configure the VLAN on a bridge instead of a bond or the NIC
+directly to avoid some of the issues mentioned xref:vlan_common_issues[here].
 
 For example, in a default configuration where you want to place
 the host management address on a separate VLAN.
 
-
-.Example: Use VLAN 5 for the {pve} management IP with traditional Linux bridge
+[[vlan_example_management_ip_and_vlan_aware_bridge]]
+.Example: Use VLAN 5 for the {pve} management IP with VLAN-aware Linux bridge
 ----
 auto lo
 iface lo inet loopback
 
 iface eno1 inet manual
 
-iface eno1.5 inet manual
-
-auto vmbr0v5
-iface vmbr0v5 inet static
-        address  10.10.10.2/24
-        gateway  10.10.10.1
-        bridge-ports eno1.5
-        bridge-stp off
-        bridge-fd 0
-
 auto vmbr0
 iface vmbr0 inet manual
         bridge-ports eno1
         bridge-stp off
         bridge-fd 0
-
-----
-
-.Example: Use VLAN 5 for the {pve} management IP with VLAN aware Linux bridge
-----
-auto lo
-iface lo inet loopback
-
-iface eno1 inet manual
-
+        bridge-vlan-aware yes
+        bridge-vids 2-4094
 
 auto vmbr0.5
 iface vmbr0.5 inet static
         address  10.10.10.2/24
         gateway  10.10.10.1
+----
 
-auto vmbr0
-iface vmbr0 inet manual
-        bridge-ports eno1
-        bridge-stp off
-        bridge-fd 0
-        bridge-vlan-aware yes
-        bridge-vids 2-4094
+Guests can then reference the VLAN tag directly in their network
+configuration. Compared to an SDN-configured solution, this has the
+disadvantage that specific VLANs cannot be named, but the advantage that
+each VLAN does not have to be configured individually.
+
+----
+net0: bridge=vmbr0,tag=5,...
 ----
 
 The next example is the same setup but a bond is used to
 make this network fail-safe.
 
-.Example: Use VLAN 5 with bond0 for the {pve} management IP with traditional Linux bridge
+[[vlan_example_bond_with_management_ip_and_vlan_aware_bridge]]
+.Example: Use VLAN 5 with bond0 for the {pve} management IP with VLAN-aware Linux bridge
 ----
 auto lo
 iface lo inet loopback
@@ -716,24 +750,105 @@ iface bond0 inet manual
       bond-mode 802.3ad
       bond-xmit-hash-policy layer2+3
 
-iface bond0.5 inet manual
-
-auto vmbr0v5
-iface vmbr0v5 inet static
-        address  10.10.10.2/24
-        gateway  10.10.10.1
-        bridge-ports bond0.5
-        bridge-stp off
-        bridge-fd 0
-
 auto vmbr0
 iface vmbr0 inet manual
         bridge-ports bond0
+        bridge-vlan-aware yes
+        bridge-vids 2-4094
         bridge-stp off
         bridge-fd 0
 
+auto vmbr0.5
+iface vmbr0.5 inet static
+        address 10.10.10.2/24
+        gateway 10.10.10.1
+
+----
+
+[[vlan_common_issues]]
+Common Issues
+^^^^^^^^^^^^^
+
+.Issue: Host not reachable on VLAN interface after guest start
+
+A management IP can be configured in a VLAN interface whose underlying
+interface is a bridge port, for example on nic0.5 or bond0.5. If the bridge
+the underlying interface is on is not VLAN-aware, the IP stops working as
+soon as a guest on that bridge and VLAN is started. Stopping the guest will
+not rectify this problem, the host has to be rebooted. To fix this, the VLAN
+interface must be removed, and the management IP configured on the bridge
+VLAN interface, as well as making the bridge VLAN-aware. xref:vlan_example_management_ip_and_vlan_aware_bridge[This example].
+shows how to do this.
+
+.Issue: ifupdown2 fails after configuring gateway on NIC
+
+When configuring a default gateway on the VLAN interface, where the unterlying
+interface is part of a VLAN-aware bridge, ifupdown2 will not be able to
+configure the default gateway and fail with a similar error:
+
+----
+>  2026-12-19 14:05:57,411: MainThread: ifupdown: scheduler.py:114:run_iface_op(): error: nic0.1337: cmd '/bin/ip route replace default via 192
+.0.2.1 proto kernel dev nic0.1337 onlink' failed: returned 2 (Error: Nexthop device is not up.)
+----
+
+To avoid this issue, configure the address and default gateway on the bridge
+directly, as is shown in xref:vlan_example_management_ip_and_vlan_aware_bridge[this example].
+
+.Issue: Limited number of tags when VLAN offloading to NIC
+
+Certain NICs only support VLAN filter offloading for a reduced number of tags.
+In these cases it's best to either reduce the tag range on the VLAN-aware
+bridge or disable offloading altogether. To determine if offloading is enabled
+`ethtool` can be used, (but do replace `nic0` with the name of your interface).
+
+----
+ethtool -k nic0 | grep rx-vlan-filter
+----
+
+If the output contains "[fixed]", the value cannot be changed. In the case that
+it can, this should be done in a post-up, so that It may survive a reboot.
+
+----
+iface nic0 inet manual
+        post-up ethtool -K nic0 rx-vlan-filter off
+----
+
+Troubleshooting
+^^^^^^^^^^^^^^^
+
+There are some helpful commands that can be utilized when debugging such issues:
+
+----
+$ bridge -compressvlans vlan show
 ----
 
+This prints a list of bridges and their ports, as well as the VLANs that are
+currently configured for them:
+
+----
+port              vlan-id
+nic0              1 PVID Egress Untagged
+                  2-4094
+vmbr0             1 PVID Egress Untagged
+veth100i0         1234 PVID Egress Untagged
+----
+
+----
+$ bridge fdb show
+----
+
+This prints a list of the current forwarding database entries of all bridges.
+The bridge decides how to forward a given packet based on this table. An entry
+of the fdb generally looks like this:
+
+----
+aa:aa:aa:d8:36:7c dev nic0 vlan 49 master vmbr0 permanent
+----
+
+This means that packets arriving on vmbr0, that have VLAN Tag 49 set and
+destination MAC address aa:aa:aa:d8:36:7c will not be forwarded, but be
+terminated locally.
+
 Disabling IPv6 on the Node
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/pvesdn.adoc b/pvesdn.adoc
index 3fd3533..4c87ede 100644
--- a/pvesdn.adoc
+++ b/pvesdn.adoc
@@ -251,7 +251,7 @@ the network segments.  This allows connectivity of VMs between different nodes.
 VLAN zone configuration options:
 
 Bridge:: The local bridge or OVS switch, already configured on *each* node that
-  allows node-to-node connection.
+  allows node-to-node connection. This bridge should be VLAN-aware to work properly.
 
 
 [[pvesdn_zone_plugin_qinq]]
-- 
2.47.3