From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id E5A0C1FF0AA for ; Tue, 20 Oct 2026 08:45:29 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id EA39421607; Tue, 20 Oct 2026 08:45:26 +0200 (CEST) From: Thomas Ellmenreich To: pve-devel@lists.proxmox.com Subject: [PATCH pve-docs v2 1/1] network/sdn: update and clean up VLAN information Date: Tue, 20 Oct 2026 08:45:08 +0200 Message-ID: <20261020064508.32479-1-t.ellmenreich@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1792478719180 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.429 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: XOK2IJCBGUT56BJEAXG3536GT5BDXKA4 X-Message-ID-Hash: XOK2IJCBGUT56BJEAXG3536GT5BDXKA4 X-MailFrom: t.ellmenreich@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Thomas Ellmenreich X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Adjusted some of the details related to VLAN setups, both with the traditional methods as well as the SDN setup. - Removed an outdated example and added more information to the remaining examples. First, by adding the VLAN interface on the bridge, and then showcasing that VLAN tags can be set in the guest network configurations. - Added SDN as a possible way to configure VLANs for guests, as well as adding an example for SDN. - Added a "Common Issues" section to collect common networking issues. - Added a "Troubleshooting" section, to collect information on how to fix VLAN related issues. Suggested-by: Stefan Hanreich Signed-off-by: Thomas Ellmenreich --- Some changes for the VLAN configuration documentation, would apprechiate any help in regards to the wording and details of said documentation. Changelog: - PATCH v1 -> v2 * Added a "Common Issues" section, that recounts different problems that might appear. * Added a "Troubleshooting" section to show different ways to troubleshoot problems with the networking. * Added a SDN VLAN example. * Some rewording, and adjustment of different sections. pve-network.adoc | 161 +++++++++++++++++++++++++++++++++++------------ pvesdn.adoc | 2 +- 2 files changed, 121 insertions(+), 42 deletions(-) diff --git a/pve-network.adoc b/pve-network.adoc index 2b3e664..b1a598c 100644 --- a/pve-network.adoc +++ b/pve-network.adoc @@ -637,6 +637,53 @@ completely done inside the guest and can not be influenced from the outside. The benefit is that you can use more than one VLAN on a single virtual NIC. +* *SDN configured VLAN:* +It is also possible to xref:pvesdn_config_vnet[configure VLANs in SDN] by +creating a VLAN zone on the bridge and then creating VNets for every VLAN tag. +In this case, it is also recommended to use a VLAN-aware bridge. + +.Example: SDN VLAN Zones with VLAN aware bridges + +This is the preferred solution if users have a small amount of VLANs inside +their cluster. On every node, users should configure a VLAN-aware bridge with +the respective carrier for their VLANs: + +---- +auto vmbr0 +iface vmbr0 inet manual + bridge-ports nic0 + bridge-stp off + bridge-fd 0 + bridge-vlan-aware yes + bridge-vids 2-4094 +---- + +Then create an SDN zone on top of that bridge: + +---- +vlan: vlanz + bridge vmbr0 + ipam pve +---- + +Then, for each VLAN they want to use, create a VNet: + +---- +vnet: vnet1234 + zone vlanz + tag 1234 +---- + +After applying the SDN configuration, users can then select vnet1234 in the +network device configuration of their guests. This method does not allow to +configure an IP for the host directly on a VNet - but it can be done manually +by adding an entry for the VNet to /etc/network/interfaces: + +---- +auto vnet1234 +iface vnet1234 inet static + address 192.0.2.1/24 +---- VLAN on the Host ^^^^^^^^^^^^^^^^ @@ -644,63 +691,47 @@ VLAN on the Host To allow host communication with an isolated network. It is possible to apply VLAN tags to any network device (NIC, Bond, Bridge). In general, you should configure the VLAN on the interface with the least -abstraction layers between itself and the physical NIC. +abstraction layers between itself and the physical NIC. But it would still +be best to configure the VLAN on a bridge instead of a bond or the NIC +directly to avoid some of the issues mentioned xref:vlan_common_issues[here]. For example, in a default configuration where you want to place the host management address on a separate VLAN. - -.Example: Use VLAN 5 for the {pve} management IP with traditional Linux bridge +.Example: Use VLAN 5 for the {pve} management IP with VLAN aware Linux bridge ---- auto lo iface lo inet loopback iface eno1 inet manual -iface eno1.5 inet manual - -auto vmbr0v5 -iface vmbr0v5 inet static - address 10.10.10.2/24 - gateway 10.10.10.1 - bridge-ports eno1.5 - bridge-stp off - bridge-fd 0 - auto vmbr0 iface vmbr0 inet manual bridge-ports eno1 bridge-stp off bridge-fd 0 - ----- - -.Example: Use VLAN 5 for the {pve} management IP with VLAN aware Linux bridge ----- -auto lo -iface lo inet loopback - -iface eno1 inet manual - + bridge-vlan-aware yes + bridge-vids 2-4094 auto vmbr0.5 iface vmbr0.5 inet static address 10.10.10.2/24 gateway 10.10.10.1 +---- + +Guests can then reference the VLAN tag directly in their network +configuration. Compared to an SDN-configured solution, this has the +disadvantage that specific VLANs cannot be named, but the advantage that +each VLAN does not have to be configured individually. -auto vmbr0 -iface vmbr0 inet manual - bridge-ports eno1 - bridge-stp off - bridge-fd 0 - bridge-vlan-aware yes - bridge-vids 2-4094 +---- +net0: name=eth0,bridge=vmbr0,tag=5,... ---- The next example is the same setup but a bond is used to make this network fail-safe. -.Example: Use VLAN 5 with bond0 for the {pve} management IP with traditional Linux bridge +.Example: Use VLAN 5 with bond0 for the {pve} management IP with VLAN aware Linux bridge ---- auto lo iface lo inet loopback @@ -716,24 +747,72 @@ iface bond0 inet manual bond-mode 802.3ad bond-xmit-hash-policy layer2+3 -iface bond0.5 inet manual - -auto vmbr0v5 -iface vmbr0v5 inet static - address 10.10.10.2/24 - gateway 10.10.10.1 - bridge-ports bond0.5 - bridge-stp off - bridge-fd 0 - auto vmbr0 iface vmbr0 inet manual bridge-ports bond0 + bridge-vlan-aware yes + bridge-vids 2-4094 bridge-stp off bridge-fd 0 +auto vmbr0.5 +iface vmbr0.5 inet static + address 10.10.10.2/24 + gateway 10.10.10.1 + ---- +Troubleshooting +^^^^^^^^^^^^^^^ + +There are some helpful commands that can be utilised when debugging such issues: + +---- +$ bridge -compressvlans vlan show +---- + +This prints a list of bridges and their ports, as well as the VLANs that are +currently configured for them: + +---- +port vlan-id +nic0 1 PVID Egress Untagged + 2-4094 +vmbr0 1 PVID Egress Untagged +veth100i0 1234 PVID Egress Untagged +---- + +---- +$ bridge fdb show +---- + +This prints a list of the current forwarding database entries of all bridges. +The bridge decides how to forward a given packet based on this table. An entry +of the fdb generally looks like this: + +---- +aa:aa:aa:d8:36:7c dev nic0 vlan 49 master vmbr0 permanent +---- + +This means that packets arriving on vmbr0, that have VLAN Tag 49 set and +destination MAC address aa:aa:aa:d8:36:7c will get forwarded to the port nic0. + +[[vlan_common_issues]] +Common Issues +^^^^^^^^^^^^^ + +.Issue: Configuring VLAN on interface that is member of bridge + +It's not recommended to configure a VLAN on an interface that is a member of a +bridge. Doing so will "blackhole" all traffic of that VLAN on the bridge. + +.Issue: Limited number of tags when VLAN offloading to NIC + +Certain NIC's, only support VLAN offloading only for a reduced number of tags. +In these cases it is best to either reduce the VLAN tag range on the bridge or +disable offloading altogether. The exact problems that can arise depend on +the exact hardware used. + Disabling IPv6 on the Node ~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/pvesdn.adoc b/pvesdn.adoc index 3fd3533..8e40ab4 100644 --- a/pvesdn.adoc +++ b/pvesdn.adoc @@ -251,7 +251,7 @@ the network segments. This allows connectivity of VMs between different nodes. VLAN zone configuration options: Bridge:: The local bridge or OVS switch, already configured on *each* node that - allows node-to-node connection. + allows node-to-node connection. This bridge should be VLAN aware to work properly. [[pvesdn_zone_plugin_qinq]] -- 2.47.3