From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id B669E9A875 for ; Fri, 17 Nov 2023 14:56:09 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 962DF341C2 for ; Fri, 17 Nov 2023 14:55:39 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Fri, 17 Nov 2023 14:55:37 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id C42AE43E34 for ; Fri, 17 Nov 2023 14:55:37 +0100 (CET) From: Stefan Lendl To: pve-devel@lists.proxmox.com Date: Fri, 17 Nov 2023 14:55:27 +0100 Message-ID: <20231117135531.3198353-3-s.lendl@proxmox.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231117135531.3198353-1-s.lendl@proxmox.com> References: <20231117135531.3198353-1-s.lendl@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.062 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: [pve-devel] [PATCH v2 pve-docs 2/6] sdn: Zones X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Nov 2023 13:56:09 -0000 Signed-off-by: Stefan Lendl --- pvesdn.adoc | 185 ++++++++++++++++++++++++++-------------------------- 1 file changed, 93 insertions(+), 92 deletions(-) diff --git a/pvesdn.adoc b/pvesdn.adoc index 562e081..8a71c03 100644 --- a/pvesdn.adoc +++ b/pvesdn.adoc @@ -86,189 +86,190 @@ in your SDN setup. virtual guests' hostname and IP addresses -[[pvesdn_config_main_sdn]] +[[pvesdn_config_main_sdn]] SDN -~~~ +~~~~~~~~~~~~~ This is the main status panel. Here you can see the deployment status of zones on different nodes. -The 'Apply' button is used to push and reload local configuration on all cluster +Pressing the 'Apply' button reloads the local configuration on all cluster nodes. -[[pvesdn_local_deployment_monitoring]] -Local Deployment Monitoring -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -After applying the configuration through the main SDN panel, -the local network configuration is generated locally on each node in -the file `/etc/network/interfaces.d/sdn`, and reloaded with ifupdown2. - -You can monitor the status of local zones and VNets through the main tree. - - [[pvesdn_config_zone]] Zones ------ +------------- -A zone defines a virtually separated network. Zones can be restricted to +A zone defines a virtually separated network. Zones are restricted to specific nodes and assigned permissions, in order to restrict users to a certain zone and its contained VNets. Different technologies can be used for separation: +* Simple: Isolated Bridge. A simple layer 3 routing bridge (NAT) + * VLAN: Virtual LANs are the classic method of subdividing a LAN * QinQ: Stacked VLAN (formally known as `IEEE 802.1ad`) -* VXLAN: Layer2 VXLAN +* VXLAN: Layer 2 VXLAN network via a UDP tunnel -* Simple: Isolated Bridge. A simple layer 3 routing bridge (NAT) +* EVPN (BGP EVPN): VXLAN with BGP to establish Layer 3 routing -* EVPN (BGP EVPN): VXLAN using layer 3 border gateway protocol (BGP) routing +[[pvesdn_config_common_options]] Common options -~~~~~~~~~~~~~~ +~~~~~~~~~~~~~ The following options are available for all zone types: -nodes:: The nodes which the zone and associated VNets should be deployed on +Nodes:: The nodes which the zone and associated VNets should be deployed on. -ipam:: Optional. Use an IP Address Management (IPAM) tool to manage IPs in the - zone. +IPAM:: Use an IP Address Management (IPAM) tool to manage IPs in the + zone. Optional, defaults to `pve`. -dns:: Optional. DNS API server. +DNS:: DNS API server. Optional. -reversedns:: Optional. Reverse DNS API server. +ReverseDNS:: Reverse DNS API server. Optional. -dnszone:: Optional. DNS domain name. Used to register hostnames, such as - `.`. The DNS zone must already exist on the DNS server. +DNSZone:: DNS domain name. Used to register hostnames, such as + `.`. The DNS zone must already exist on the DNS server. Optional. [[pvesdn_zone_plugin_simple]] Simple Zones -~~~~~~~~~~~~ +~~~~~~~~~~~~~ + +This is the simplest plugin. It will create an isolated VNet bridge. This +bridge is not linked to a physical interface, and VM traffic is only local on +each the node. +It can be used in NAT or routed setups. -This is the simplest plugin. It will create an isolated VNet bridge. -This bridge is not linked to a physical interface, and VM traffic is only -local between the node(s). -It can also be used in NAT or routed setups. [[pvesdn_zone_plugin_vlan]] VLAN Zones -~~~~~~~~~~ +~~~~~~~~~~~~~ -This plugin reuses an existing local Linux or OVS bridge, and manages the VLANs -on it. The benefit of using the SDN module is that you can create different -zones with specific VNet VLAN tags, and restrict virtual machines to separated -zones. +The VLAN plugin uses an existing local Linux or OVS bridge to connect to the +node's physical interface. It uses VLAN tagging defined in the VNet to isolate +the network segments. This allows connectivity of VMs between different nodes. -Specific `VLAN` configuration options: +VLAN zone configuration options: + +Bridge:: The local bridge or OVS switch, already configured on *each* node that + allows node-to-node connection. -bridge:: Reuse this local bridge or OVS switch, already configured on *each* - local node. [[pvesdn_zone_plugin_qinq]] QinQ Zones -~~~~~~~~~~ +~~~~~~~~~~~~~ -QinQ also known as VLAN stacking, wherein the first VLAN tag is defined for the -zone (the 'service-vlan'), and the second VLAN tag is defined for the -VNets. +QinQ also known as VLAN stacking, that uses multiple layers of VLAN tags for +isolation. The QinQ zone defines the outer VLAN tag (the 'Service VLAN') +whereas the inner VLAN tag is defined by the VNet. NOTE: Your physical network switches must support stacked VLANs for this -configuration! +configuration. -Below are the configuration options specific to QinQ: +QinQ zone configuration options: -bridge:: A local, VLAN-aware bridge that is already configured on each local +Bridge:: A local, VLAN-aware bridge that is already configured on each local node -service vlan:: The main VLAN tag of this zone +Service VLAN:: The main VLAN tag of this zone -service vlan protocol:: Allows you to choose between an 802.1q (default) or +Service VLAN Protocol:: Allows you to choose between an 802.1q (default) or 802.1ad service VLAN type. -mtu:: Due to the double stacking of tags, you need 4 more bytes for QinQ VLANs. +MTU:: Due to the double stacking of tags, you need 4 more bytes for QinQ VLANs. For example, you must reduce the MTU to `1496` if you physical interface MTU is `1500`. + [[pvesdn_zone_plugin_vxlan]] VXLAN Zones -~~~~~~~~~~~ +~~~~~~~~~~~~~ -The VXLAN plugin establishes a tunnel (overlay) on top of an existing -network (underlay). This encapsulates layer 2 Ethernet frames within layer -4 UDP datagrams, using `4789` as the default destination port. You can, for -example, create a private IPv4 VXLAN network on top of public internet network -nodes. +The VXLAN plugin establishes a tunnel (overlay) on top of an existing network +(underlay). This encapsulates layer 2 Ethernet frames within layer 4 UDP +datagrams using the default destination port `4789`. + +You have to configure the underlay network yourself to enable UDP connectivity +between all peers. -This is a layer 2 tunnel only, so no routing between different VNets is -possible. +You can, for example, create a VXLAN overlay network on top of public internet, +appearing to the VMs as if they share the same local Layer 2 network. -Each VNet will have a specific VXLAN ID in the range 1 - 16777215. +WARNING: VXLAN on its own does does not provide any encryption. When joining + multiple sites via VXLAN, make sure to establish a secure connection between + the site, for example by using a site-to-site VPN. -Specific EVPN configuration options: +VXLAN zone configuration options: -peers address list:: A list of IP addresses from each node through which you - want to communicate. Can also be external nodes. +Peers Address List:: A list of IP addresses of each node in the VXLAN zone. This + can be external nodes reachable at this IP address. + All nodes in the cluster need to be mentioned here. -mtu:: Because VXLAN encapsulation uses 50 bytes, the MTU needs to be 50 bytes +MTU:: Because VXLAN encapsulation uses 50 bytes, the MTU needs to be 50 bytes lower than the outgoing physical interface. + [[pvesdn_zone_plugin_evpn]] EVPN Zones -~~~~~~~~~~ +~~~~~~~~~~~~~ -This is the most complex of all the supported plugins. +The EVPN zone creates a routable Layer 3 network, capable of spanning across +multiple clusters. This is achieved by establishing a VPN and utilizing BGP as +the routing protocol. -BGP-EVPN allows you to create a routable layer 3 network. The VNet of EVPN can -have an anycast IP address and/or MAC address. The bridge IP is the same on each -node, meaning a virtual guest can use this address as gateway. +The VNet of EVPN can have an anycast IP address and/or MAC address. The bridge +IP is the same on each node, meaning a virtual guest can use this address as +gateway. Routing can work across VNets from different zones through a VRF (Virtual Routing and Forwarding) interface. -The configuration options specific to EVPN are as follows: +EVPN zone configuration options: -VRF VXLAN tag:: This is a VXLAN-ID used for routing interconnect between VNets. +VRF VXLAN ID:: A VXLAN-ID used for dedicated routing interconnect between VNets. It must be different than the VXLAN-ID of the VNets. -controller:: An EVPN-controller must to be defined first (see controller plugins +Controller:: The EVPN-controller to use for this zone. (See controller plugins section). -VNet MAC address:: A unique, anycast MAC address for all VNets in this zone. - Will be auto-generated if not defined. +VNet MAC Address:: Anycast MAC address that gets assigned to all VNets in this + zone. Will be auto-generated if not defined. -Exit Nodes:: Optional. This is used if you want to define some {pve} nodes as - exit gateways from the EVPN network, through the real network. The configured - nodes will announce a default route in the EVPN network. +Exit Nodes:: Nodes that shall be configured as exit gateways from the EVPN + network, through the real network. The configured nodes will announce a + default route in the EVPN network. Optional. -Primary Exit Node:: Optional. If you use multiple exit nodes, this forces - traffic to a primary exit node, instead of load-balancing on all nodes. This - is required if you want to use SNAT or if your upstream router doesn't support +Primary Exit Node:: If you use multiple exit nodes, force traffic through this + primary exit node, instead of load-balancing on all nodes. Optional but + necessary if you want to use SNAT or if your upstream router doesn't support ECMP. -Exit Nodes local routing:: Optional. This is a special option if you need to - reach a VM/CT service from an exit node. (By default, the exit nodes only - allow forwarding traffic between real network and EVPN network). +Exit Nodes Local Routing:: This is a special option if you need to reach a VM/CT + service from an exit node. (By default, the exit nodes only allow forwarding + traffic between real network and EVPN network). Optional. -Advertise Subnets:: Optional. If you have silent VMs/CTs (for example, if you - have multiple IPs and the anycast gateway doesn't see traffic from theses IPs, - the IP addresses won't be able to be reach inside the EVPN network). This - option will announce the full subnet in the EVPN network in this case. +Advertise Subnets:: Announce the full subnet in the EVPN network. + If you have silent VMs/CTs (for example, if you have multiple IPs and the + anycast gateway doesn't see traffic from theses IPs, the IP addresses won't be + able to be reached inside the EVPN network). Optional. -Disable Arp-Nd Suppression:: Optional. Don't suppress ARP or ND packets. - This is required if you use floating IPs in your guest VMs - (IP are MAC addresses are being moved between systems). +Disable ARP ND Suppression:: Don't suppress ARP or ND (Neighbor Discovery) + packets. This is required if you use floating IPs in your VMs (IP and MAC + addresses are being moved between systems). Optional. -Route-target import:: Optional. Allows you to import a list of external EVPN - route targets. Used for cross-DC or different EVPN network interconnects. +Route-target Import:: Allows you to import a list of external EVPN route + targets. Used for cross-DC or different EVPN network interconnects. Optional. MTU:: Because VXLAN encapsulation uses 50 bytes, the MTU needs to be 50 bytes - less than the maximal MTU of the outgoing physical interface. + less than the maximal MTU of the outgoing physical interface. Optional, + defaults to 1450. [[pvesdn_config_vnet]] -- 2.42.0