From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 58E176E3CB for ; Mon, 23 Aug 2021 15:23:03 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 4EABD2434E for ; Mon, 23 Aug 2021 15:22:33 +0200 (CEST) Received: from kvmformation3.odiso.net (globalOdiso.M6Lille.odiso.net [89.248.211.242]) by firstgate.proxmox.com (Proxmox) with ESMTP id 2B4D124316 for ; Mon, 23 Aug 2021 15:22:27 +0200 (CEST) Received: by kvmformation3.odiso.net (Postfix, from userid 0) id 07F1011453D; Mon, 23 Aug 2021 15:22:20 +0200 (CEST) From: Alexandre Derumier To: pve-devel@lists.proxmox.com Date: Mon, 23 Aug 2021 15:22:16 +0200 Message-Id: <20210823132218.3558949-4-aderumier@odiso.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210823132218.3558949-1-aderumier@odiso.com> References: <20210823132218.3558949-1-aderumier@odiso.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.888 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% HEADER_FROM_DIFFERENT_DOMAINS 0.25 From and EnvelopeFrom 2nd level mail domains are different KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods KHOP_HELO_FCRDNS 0.399 Relay HELO differs from its IP's reverse DNS NO_DNS_FOR_FROM 0.001 Envelope sender has no MX or A DNS records SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record Subject: [pve-devel] [PATCH pve-network 3/5] evpn: add exitnodes-local-routing X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2021 13:23:03 -0000 This option allow an exit-node to reach itself a guest in evpn network. Forum user have requested it, the exitnode need to reach virtual dns server in evpn. This use a veth-pair instead a simple leak. It's not enable by default is slowing down a little bit the routing. Signed-off-by: Alexandre Derumier --- PVE/Network/SDN/Controllers.pm | 2 +- PVE/Network/SDN/Controllers/EvpnPlugin.pm | 48 ++++++++++++---- PVE/Network/SDN/Controllers/FaucetPlugin.pm | 2 +- PVE/Network/SDN/Zones/EvpnPlugin.pm | 30 +++++++++- .../expected_controller_config | 37 ++++++++++++ .../expected_sdn_interfaces | 56 +++++++++++++++++++ .../evpn/exitnode_local_routing/interfaces | 7 +++ .../evpn/exitnode_local_routing/sdn_config | 27 +++++++++ 8 files changed, 195 insertions(+), 14 deletions(-) create mode 100644 test/zones/evpn/exitnode_local_routing/expected_controller_config create mode 100644 test/zones/evpn/exitnode_local_routing/expected_sdn_interfaces create mode 100644 test/zones/evpn/exitnode_local_routing/interfaces create mode 100644 test/zones/evpn/exitnode_local_routing/sdn_config diff --git a/PVE/Network/SDN/Controllers.pm b/PVE/Network/SDN/Controllers.pm index e1cf9e0..a23048e 100644 --- a/PVE/Network/SDN/Controllers.pm +++ b/PVE/Network/SDN/Controllers.pm @@ -125,7 +125,7 @@ sub generate_controller_config { my $controller = $controller_cfg->{ids}->{$controllerid}; if ($controller) { my $controller_plugin = PVE::Network::SDN::Controllers::Plugin->lookup($controller->{type}); - $controller_plugin->generate_controller_vnet_config($plugin_config, $controller, $zoneid, $id, $config); + $controller_plugin->generate_controller_vnet_config($plugin_config, $controller, $zone, $zoneid, $id, $config); } } diff --git a/PVE/Network/SDN/Controllers/EvpnPlugin.pm b/PVE/Network/SDN/Controllers/EvpnPlugin.pm index d629140..3e450cf 100644 --- a/PVE/Network/SDN/Controllers/EvpnPlugin.pm +++ b/PVE/Network/SDN/Controllers/EvpnPlugin.pm @@ -114,6 +114,7 @@ sub generate_controller_zone_config { my $vrfvxlan = $plugin_config->{'vrf-vxlan'}; my $exitnodes = $plugin_config->{'exitnodes'}; my $advertisesubnets = $plugin_config->{'advertise-subnets'}; + my $exitnodes_local_routing = $plugin_config->{'exitnodes-local-routing'}; my $asn = $controller->{asn}; my $ebgp = undef; @@ -149,17 +150,19 @@ sub generate_controller_zone_config { if ($is_gateway) { - @controller_config = (); - #import /32 routes of evpn network from vrf1 to default vrf (for packet return) - push @controller_config, "import vrf $vrf"; - push(@{$config->{frr}->{router}->{"bgp $asn"}->{"address-family"}->{"ipv4 unicast"}}, @controller_config); - push(@{$config->{frr}->{router}->{"bgp $asn"}->{"address-family"}->{"ipv6 unicast"}}, @controller_config); - - @controller_config = (); - #redistribute connected to be able to route to local vms on the gateway - push @controller_config, "redistribute connected"; - push(@{$config->{frr}->{router}->{"bgp $asn vrf $vrf"}->{"address-family"}->{"ipv4 unicast"}}, @controller_config); - push(@{$config->{frr}->{router}->{"bgp $asn vrf $vrf"}->{"address-family"}->{"ipv6 unicast"}}, @controller_config); + if (!$exitnodes_local_routing) { + @controller_config = (); + #import /32 routes of evpn network from vrf1 to default vrf (for packet return) + push @controller_config, "import vrf $vrf"; + push(@{$config->{frr}->{router}->{"bgp $asn"}->{"address-family"}->{"ipv4 unicast"}}, @controller_config); + push(@{$config->{frr}->{router}->{"bgp $asn"}->{"address-family"}->{"ipv6 unicast"}}, @controller_config); + + @controller_config = (); + #redistribute connected to be able to route to local vms on the gateway + push @controller_config, "redistribute connected"; + push(@{$config->{frr}->{router}->{"bgp $asn vrf $vrf"}->{"address-family"}->{"ipv4 unicast"}}, @controller_config); + push(@{$config->{frr}->{router}->{"bgp $asn vrf $vrf"}->{"address-family"}->{"ipv6 unicast"}}, @controller_config); + } @controller_config = (); #add default originate to announce 0.0.0.0/0 type5 route in evpn @@ -184,6 +187,29 @@ sub generate_controller_zone_config { return $config; } +sub generate_controller_vnet_config { + my ($class, $plugin_config, $controller, $zone, $zoneid, $vnetid, $config) = @_; + + my $exitnodes = $zone->{'exitnodes'}; + my $exitnodes_local_routing = $zone->{'exitnodes-local-routing'}; + + return if !$exitnodes_local_routing; + + my $local_node = PVE::INotify::nodename(); + my $is_gateway = $exitnodes->{$local_node}; + + return if !$is_gateway; + + my $subnets = PVE::Network::SDN::Vnets::get_subnets($vnetid, 1); + my @controller_config = (); + foreach my $subnetid (sort keys %{$subnets}) { + my $subnet = $subnets->{$subnetid}; + my $cidr = $subnet->{cidr}; + push @controller_config, "ip route $cidr 10.255.255.2 xvrf_$zoneid"; + } + push(@{$config->{frr}->{''}}, @controller_config); +} + sub on_delete_hook { my ($class, $controllerid, $zone_cfg) = @_; diff --git a/PVE/Network/SDN/Controllers/FaucetPlugin.pm b/PVE/Network/SDN/Controllers/FaucetPlugin.pm index 5742187..4f3bb5c 100644 --- a/PVE/Network/SDN/Controllers/FaucetPlugin.pm +++ b/PVE/Network/SDN/Controllers/FaucetPlugin.pm @@ -43,7 +43,7 @@ sub generate_controller_zone_config { sub generate_controller_vnet_config { - my ($class, $plugin_config, $controller, $zoneid, $vnetid, $config) = @_; + my ($class, $plugin_config, $controller, $zone, $zoneid, $vnetid, $config) = @_; my $mac = $plugin_config->{mac}; my $ipv4 = $plugin_config->{ipv4}; diff --git a/PVE/Network/SDN/Zones/EvpnPlugin.pm b/PVE/Network/SDN/Zones/EvpnPlugin.pm index c463851..648f7c7 100644 --- a/PVE/Network/SDN/Zones/EvpnPlugin.pm +++ b/PVE/Network/SDN/Zones/EvpnPlugin.pm @@ -35,6 +35,11 @@ sub properties { optional => 1, format => 'mac-addr' }, 'exitnodes' => get_standard_option('pve-node-list'), + 'exitnodes-local-routing' => { + type => 'boolean', + description => "Allow exitnodes to connect to evpn guests", + optional => 1 + }, 'advertise-subnets' => { type => 'boolean', description => "Advertise evpn subnets if you have silent hosts", @@ -49,6 +54,7 @@ sub options { 'vrf-vxlan' => { optional => 0 }, controller => { optional => 0 }, exitnodes => { optional => 1 }, + 'exitnodes-local-routing' => { optional => 1 }, 'advertise-subnets' => { optional => 1 }, mtu => { optional => 1 }, mac => { optional => 1 }, @@ -80,6 +86,8 @@ sub generate_sdn_config { my $loopback = $bgprouter->{loopback} if $bgprouter->{loopback}; my ($ifaceip, $iface) = PVE::Network::SDN::Zones::Plugin::find_local_ip_interface_peers(\@peers, $loopback); my $is_evpn_gateway = $plugin_config->{'exitnodes'}->{$local_node}; + my $exitnodes_local_routing = $plugin_config->{'exitnodes-local-routing'}; + my $mtu = 1450; $mtu = $interfaces_config->{$iface}->{mtu} - 50 if $interfaces_config->{$iface}->{mtu}; @@ -192,8 +200,28 @@ sub generate_sdn_config { push @iface_config, "vrf $vrf_iface"; push(@{$config->{$brvrf}}, @iface_config) if !$config->{$brvrf}; } - } + if ( $is_evpn_gateway && $exitnodes_local_routing ) { + #add a veth pair for local cross-vrf routing + my $iface_xvrf = "xvrf_$zoneid"; + my $iface_xvrfp = "xvrfp_$zoneid"; + + @iface_config = (); + push @iface_config, "link-type veth"; + push @iface_config, "address 10.255.255.1/30"; + push @iface_config, "veth-peer-name $iface_xvrfp"; + push @iface_config, "mtu ".($mtu+50) if $mtu; + push(@{$config->{$iface_xvrf}}, @iface_config) if !$config->{$iface_xvrf}; + + @iface_config = (); + push @iface_config, "link-type veth"; + push @iface_config, "address 10.255.255.2/30"; + push @iface_config, "veth-peer-name $iface_xvrf"; + push @iface_config, "vrf $vrf_iface"; + push @iface_config, "mtu ".($mtu+50) if $mtu; + push(@{$config->{$iface_xvrfp}}, @iface_config) if !$config->{$iface_xvrfp}; + } + } return $config; } diff --git a/test/zones/evpn/exitnode_local_routing/expected_controller_config b/test/zones/evpn/exitnode_local_routing/expected_controller_config new file mode 100644 index 0000000..abd065b --- /dev/null +++ b/test/zones/evpn/exitnode_local_routing/expected_controller_config @@ -0,0 +1,37 @@ +log syslog informational +ip forwarding +ipv6 forwarding +frr defaults datacenter +service integrated-vtysh-config +hostname localhost +! +ip route 10.0.0.0/24 10.255.255.2 xvrf_myzone +! +vrf vrf_myzone + vni 1000 +exit-vrf +! +router bgp 65000 + bgp router-id 192.168.0.1 + no bgp default ipv4-unicast + coalesce-time 1000 + neighbor VTEP peer-group + neighbor VTEP remote-as 65000 + neighbor VTEP bfd + neighbor 192.168.0.2 peer-group VTEP + neighbor 192.168.0.3 peer-group VTEP + ! + address-family l2vpn evpn + neighbor VTEP activate + advertise-all-vni + exit-address-family +! +router bgp 65000 vrf vrf_myzone + ! + address-family l2vpn evpn + default-originate ipv4 + default-originate ipv6 + exit-address-family +! +line vty +! \ No newline at end of file diff --git a/test/zones/evpn/exitnode_local_routing/expected_sdn_interfaces b/test/zones/evpn/exitnode_local_routing/expected_sdn_interfaces new file mode 100644 index 0000000..301f5b3 --- /dev/null +++ b/test/zones/evpn/exitnode_local_routing/expected_sdn_interfaces @@ -0,0 +1,56 @@ +#version:1 + +auto myvnet +iface myvnet + address 10.0.0.1/24 + bridge_ports vxlan_myvnet + bridge_stp off + bridge_fd 0 + mtu 1450 + ip-forward on + arp-accept on + vrf vrf_myzone + +auto vrf_myzone +iface vrf_myzone + vrf-table auto + post-up ip route del vrf vrf_myzone unreachable default metric 4278198272 + +auto vrfbr_myzone +iface vrfbr_myzone + bridge-ports vrfvx_myzone + bridge_stp off + bridge_fd 0 + mtu 1450 + vrf vrf_myzone + +auto vrfvx_myzone +iface vrfvx_myzone + vxlan-id 1000 + vxlan-local-tunnelip 192.168.0.1 + bridge-learning off + bridge-arp-nd-suppress on + mtu 1450 + +auto vxlan_myvnet +iface vxlan_myvnet + vxlan-id 100 + vxlan-local-tunnelip 192.168.0.1 + bridge-learning off + bridge-arp-nd-suppress on + mtu 1450 + +auto xvrf_myzone +iface xvrf_myzone + link-type veth + address 10.255.255.1/30 + veth-peer-name xvrfp_myzone + mtu 1500 + +auto xvrfp_myzone +iface xvrfp_myzone + link-type veth + address 10.255.255.2/30 + veth-peer-name xvrf_myzone + vrf vrf_myzone + mtu 1500 diff --git a/test/zones/evpn/exitnode_local_routing/interfaces b/test/zones/evpn/exitnode_local_routing/interfaces new file mode 100644 index 0000000..66bb826 --- /dev/null +++ b/test/zones/evpn/exitnode_local_routing/interfaces @@ -0,0 +1,7 @@ +auto vmbr0 +iface vmbr0 inet static + address 192.168.0.1/24 + gateway 192.168.0.254 + bridge-ports eth0 + bridge-stp off + bridge-fd 0 diff --git a/test/zones/evpn/exitnode_local_routing/sdn_config b/test/zones/evpn/exitnode_local_routing/sdn_config new file mode 100644 index 0000000..f5f7ca1 --- /dev/null +++ b/test/zones/evpn/exitnode_local_routing/sdn_config @@ -0,0 +1,27 @@ +{ + version => 1, + vnets => { + ids => { + myvnet => { tag => "100", type => "vnet", zone => "myzone" }, + }, + }, + + zones => { + ids => { myzone => { ipam => "pve", type => "evpn", controller => "evpnctl", 'vrf-vxlan' => 1000, exitnodes => { 'localhost' => 1 }, 'exitnodes-local-routing' => 1 }, + }, + }, + controllers => { + ids => { evpnctl => { type => "evpn", 'peers' => '192.168.0.1,192.168.0.2,192.168.0.3', asn => "65000" } }, + }, + + subnets => { + ids => { 'myzone-10.0.0.0-24' => { + 'type' => 'subnet', + 'vnet' => 'myvnet', + 'gateway' => '10.0.0.1', + }, + } + } +} + + -- 2.30.2