From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 256611FF0BC for ; Mon, 16 Nov 2026 17:16:59 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id AC1282164B; Mon, 16 Nov 2026 17:13:07 +0100 (CET) From: Gabriel Goller To: pve-devel@lists.proxmox.com Subject: [PATCH pve-network v3 23/32] sdn: zones: SNAT via dedicated uplinks in zone VRFs Date: Mon, 16 Nov 2026 17:11:43 +0100 Message-ID: <20261116161154.343743-24-g.goller@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261116161154.343743-1-g.goller@proxmox.com> References: <20261116161154.343743-1-g.goller@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1794845521430 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.244 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 4KEV3RA3RFY3R2XUIQWS3DEFGTVZFGJ5 X-Message-ID-Hash: 4KEV3RA3RFY3R2XUIQWS3DEFGTVZFGJ5 X-MailFrom: g.goller@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: SNAT for subnets in a zone VRF resolved the outgoing interface and source address in the default routing table, so the generated rule never matched traffic leaving through the VRF. Allow moving a dedicated uplink into the VRF instead, which keeps the zone isolated from the default routing table. The uplink is derived from the zone's default route, or else from the gateway of a host interface in the VRF, since the config is generated before ifreload creates the VRF and moves the interface into it. Only without either, e.g. for a DHCP uplink, the VRF routing table is queried. MASQUERADE is used if the uplink has no static address. Simple zones with a VRF never fall back to the default routing table, EVPN exit nodes without a dedicated uplink keep the current behavior. As subnets can overlap between zones, connections from the vnets of such zones are put into a conntrack zone per VRF in the original direction. Replies are found via their NATed reply tuple in the default zone, so the uplink needs no rule. The conntrack zone ID is hashed from the VRF name to stay stable across reloads. Signed-off-by: Gabriel Goller --- src/PVE/Network/SDN/Zones.pm | 3 +- src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 51 +++--- src/PVE/Network/SDN/Zones/Plugin.pm | 157 +++++++++++++++++- src/PVE/Network/SDN/Zones/SimplePlugin.pm | 46 +++-- src/test/run_test_zones.pl | 4 +- .../expected_controller_config | 79 +++++++++ .../expected_sdn_interfaces | 73 ++++++++ .../evpn/exitnode_snat_vrf_uplink/interfaces | 16 ++ .../evpn/exitnode_snat_vrf_uplink/sdn_config | 42 +++++ .../vrf_snat_uplink/expected_sdn_interfaces | 92 ++++++++++ .../zones/simple/vrf_snat_uplink/interfaces | 26 +++ .../zones/simple/vrf_snat_uplink/sdn_config | 84 ++++++++++ 12 files changed, 632 insertions(+), 41 deletions(-) create mode 100644 src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_controller_config create mode 100644 src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_sdn_interfaces create mode 100644 src/test/zones/evpn/exitnode_snat_vrf_uplink/interfaces create mode 100644 src/test/zones/evpn/exitnode_snat_vrf_uplink/sdn_config create mode 100644 src/test/zones/simple/vrf_snat_uplink/expected_sdn_interfaces create mode 100644 src/test/zones/simple/vrf_snat_uplink/interfaces create mode 100644 src/test/zones/simple/vrf_snat_uplink/sdn_config diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm index 1d3ad22c1cd5..0f64f39f6ac6 100644 --- a/src/PVE/Network/SDN/Zones.pm +++ b/src/PVE/Network/SDN/Zones.pm @@ -133,6 +133,7 @@ sub generate_etc_network_config { #generate configuration my $config = {}; my $nodename = PVE::INotify::nodename(); + my $vrf_ct_zones = PVE::Network::SDN::Zones::Plugin::get_vrf_ct_zones($zone_ids); for my $zone_id (sort keys %$zone_ids) { my $plugin_config = $zone_ids->{$zone_id}; @@ -168,7 +169,7 @@ sub generate_etc_network_config { my $plugin = PVE::Network::SDN::Zones::Plugin->lookup($plugin_config->{type}); eval { $plugin->generate_sdn_config( - $plugin_config, + { %$plugin_config, 'vrf-ct-zone' => $vrf_ct_zones->{$zone} }, $zone, $id, $vnet, diff --git a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm index 1933ce86f997..b4b4ab450384 100644 --- a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm +++ b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm @@ -268,6 +268,7 @@ sub generate_sdn_config { my $ipv6 = undef; my $enable_forward_v4 = undef; my $enable_forward_v6 = undef; + my $ct_zones = {}; my $subnets = PVE::Network::SDN::Vnets::get_subnets($vnetid, 1); foreach my $subnetid (sort keys %{$subnets}) { my $subnet = $subnets->{$subnetid}; @@ -280,50 +281,62 @@ sub generate_sdn_config { $address->{$gateway} = 1; } - my $iptables = undef; my $checkrouteip = undef; my $ipversion = Net::IP::ip_is_ipv6($gateway) ? 6 : 4; if ($ipversion == 6) { $ipv6 = 1; - $iptables = "ip6tables"; $checkrouteip = '2001:4860:4860::8888'; $enable_forward_v6 = 1 if $gateway; } else { $ipv4 = 1; - $iptables = "iptables"; $checkrouteip = '8.8.8.8'; $enable_forward_v4 = 1 if $gateway; } - if ($subnet->{snat}) { + if ($subnet->{snat} && $is_evpn_gateway) { #find outgoing interface - my ($outip, $outiface); + my ($outip, $outiface, $ct_zone); eval { - ($outip, $outiface) = - PVE::Network::SDN::Zones::Plugin::get_local_route_ip($checkrouteip); + ($outip, $outiface) = PVE::Network::SDN::Zones::Plugin::find_dedicated_uplink( + $zoneid, $plugin_config, $ipversion, $interfaces_config, + ); + if ($outiface) { + $ct_zone = $plugin_config->{'vrf-ct-zone'}; + } else { + # the VRF of exit nodes falls through to the default routing table + ($outip, $outiface) = + PVE::Network::SDN::Zones::Plugin::get_local_route_ip($checkrouteip); + $outiface = undef if !$outip; + } }; if ($@) { my $msg = "interface for SNAT could not be resolved: $@"; log_warn($msg); next; } - if ($outip && $outiface && $is_evpn_gateway) { - #use snat, faster than masquerade - push @iface_config, - "post-up $iptables -t nat -A POSTROUTING -s '$cidr' -o $outiface -j SNAT --to-source $outip"; - push @iface_config, - "post-down $iptables -t nat -D POSTROUTING -s '$cidr' -o $outiface -j SNAT --to-source $outip"; - #add conntrack zone once on outgoing interface - push @iface_config, - "post-up $iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1"; - push @iface_config, - "post-down $iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1"; - } + next if !$outiface; + + $ct_zones->{$ipversion} = $ct_zone if defined($ct_zone); + push @iface_config, + PVE::Network::SDN::Zones::Plugin::generate_snat_config( + $ipversion, $cidr, $outip, $outiface, + ); } } + # also for traffic routed from other nodes via the L3VNI + for my $ipversion (sort keys %$ct_zones) { + push @iface_config, + PVE::Network::SDN::Zones::Plugin::generate_vrf_ct_zone_config( + $ipversion, + $ct_zones->{$ipversion}, + $vnetid, + "vrfbr_$zoneid", + ); + } + push @iface_config, "hwaddress $mac" if $mac; push @iface_config, "bridge_ports $vxlan_iface"; push @iface_config, "bridge_stp off"; diff --git a/src/PVE/Network/SDN/Zones/Plugin.pm b/src/PVE/Network/SDN/Zones/Plugin.pm index a3863bc9818c..5c4dafd01c9f 100644 --- a/src/PVE/Network/SDN/Zones/Plugin.pm +++ b/src/PVE/Network/SDN/Zones/Plugin.pm @@ -4,6 +4,7 @@ use strict; use warnings; use PVE::Tools qw(run_command); +use Digest::SHA; use Net::IP qw(ip_get_version); use PVE::JSONSchema; use PVE::Cluster; @@ -378,13 +379,15 @@ sub get_iface_addresses { } sub get_local_route_ip { - my ($targetip) = @_; + my ($targetip, $vrf) = @_; my $ip = undef; my $interface = undef; run_command( - ['/sbin/ip', 'route', 'get', $targetip], + ['/sbin/ip', 'route', 'get', $targetip, defined($vrf) ? ('vrf', $vrf) : ()], + # a missing VRF or route is expected on the first apply, and handled by the caller + defined($vrf) ? (errfunc => sub { }) : (), outfunc => sub { if ($_[0] =~ m/src ($PVE::Tools::IPRE)/) { $ip = $1; @@ -398,6 +401,156 @@ sub get_local_route_ip { return ($ip, $interface); } +# Assign each zone with a VRF a conntrack zone, so that identical connections +# from overlapping subnets in different VRFs do not clash. +# +# The ID is derived from the VRF name, so it only changes if a zone with a +# colliding hash is added. Zones 0 (default) and 1 (fwbr+ bridges) are reserved. +sub get_vrf_ct_zones { + my ($zones) = @_; + + my $ct_zones = {}; + my $used = {}; + + for my $zone_id (sort keys %$zones) { + my $zone = $zones->{$zone_id}; + next if $zone->{type} ne 'evpn' && !($zone->{type} eq 'simple' && $zone->{'dedicated-vrf'}); + + my $ct_zone = 2 + unpack('n', Digest::SHA::sha256("vrf_$zone_id")) % 65534; + $ct_zone = $ct_zone == 65535 ? 2 : $ct_zone + 1 while $used->{$ct_zone}; + + $used->{$ct_zone} = 1; + $ct_zones->{$zone_id} = $ct_zone; + } + + return $ct_zones; +} + +# Return the zone's default route of the given IP version with the lowest +# administrative distance. +sub find_zone_default_route { + my ($plugin_config, $ipversion) = @_; + + my @routes; + for my $route_str (@{ $plugin_config->{route} // [] }) { + my $route = PVE::JSONSchema::parse_property_string('pve-sdn-static-route', $route_str); + my ($ip, $mask) = split(/\//, $route->{prefix}, 2); + next if !defined($mask) || $mask != 0 || (ip_get_version($ip) // 0) != $ipversion; + push @routes, $route; + } + + my ($route) = sort { ($a->{distance} // 1) <=> ($b->{distance} // 1) } @routes; + return $route; +} + +# Resolve the dedicated SNAT uplink of a zone, i.e. the host interface in the +# zone VRF that the default route of the given IP version points to. +# +# The uplink is resolved from the configuration if possible, since neither the +# VRF nor its routes exist on the first apply. A default route of the zone takes +# precedence over an interface in the VRF with a gateway. Without either, e.g. +# if the uplink gets its gateway via DHCP, the VRF routing table is queried. +# +# Returns ($ip, $iface), with $ip undef if the uplink has no static address, or +# nothing if the zone has no dedicated uplink. Dies if the default route has its +# next hop in the zone VRF, but not via an interface in it. +sub find_dedicated_uplink { + my ($zone_id, $plugin_config, $ipversion, $interfaces_config) = @_; + + my $vrf = "vrf_$zone_id"; + my $ifaces = $interfaces_config->{ifaces} // {}; + my ($address_key, $method_key, $cidr_key, $gateway_key) = + $ipversion == 6 + ? ('address6', 'method6', 'cidr6', 'gateway6') + : ('address', 'method', 'cidr', 'gateway'); + + my $is_member = sub { + my ($iface) = @_; + return defined($iface) && ($ifaces->{$iface}->{vrf} // '') eq $vrf; + }; + + my $iface; + if (my $route = find_zone_default_route($plugin_config, $ipversion)) { + return if ($route->{'nexthop-vrf'} // $vrf) ne $vrf; + + $iface = $route->{iface}; + if (!defined($iface) && defined(my $via = $route->{via})) { + ($iface) = grep { + $is_member->($_) + && $ifaces->{$_}->{$cidr_key} + && PVE::Network::is_ip_in_cidr($via, $ifaces->{$_}->{$cidr_key}, $ipversion) + } sort keys %$ifaces; + } + + die "default route of zone '$zone_id' is not via an interface in $vrf\n" + if !$is_member->($iface); + } elsif ( + my @gw_ifaces = + grep { $is_member->($_) && $ifaces->{$_}->{$gateway_key} } sort keys %$ifaces + ) { + # ifupdown2 installs the gateway into the VRF routing table + $iface = $gw_ifaces[0]; + } else { + my $targetip = $ipversion == 6 ? '2001:4860:4860::8888' : '8.8.8.8'; + # fails if the VRF or its default route do not exist yet + (undef, $iface) = eval { get_local_route_ip($targetip, $vrf) }; + return if !$is_member->($iface); + } + + my $ip; + if (($ifaces->{$iface}->{$method_key} // '') eq 'static') { + $ip = $ifaces->{$iface}->{$address_key}; + $ip = $ip->[0] if ref($ip) eq 'ARRAY'; + } + + return ($ip, $iface); +} + +my sub generate_iptables_rules { + my ($ipversion, @rules) = @_; + + my $iptables = $ipversion == 6 ? 'ip6tables' : 'iptables'; + + my @iface_config; + for my $rule (@rules) { + my ($table, $action, $spec) = @$rule; + push @iface_config, "post-up $iptables -t $table $action $spec"; + push @iface_config, "post-down $iptables -t $table -D $spec"; + } + + return @iface_config; +} + +# Generate the ifupdown2 commands to SNAT $cidr out of $outiface, using +# MASQUERADE if the source address is not known. +sub generate_snat_config { + my ($ipversion, $cidr, $outip, $outiface) = @_; + + my $target = defined($outip) ? "SNAT --to-source $outip" : 'MASQUERADE'; + + return generate_iptables_rules( + $ipversion, + #use snat, faster than masquerade + ['nat', '-A', "POSTROUTING -s '$cidr' -o $outiface -j $target"], + #add conntrack zone once on outgoing interface + ['raw', '-I', 'PREROUTING -i fwbr+ -j CT --zone 1'], + ); +} + +# Put connections received on @ifaces into conntrack zone $ct_zone in the +# original direction, so identical connections from overlapping subnets in +# other VRFs do not clash before being NATed. Replies are looked up by their +# reply tuple in the default zone, which NAT keeps unique, so neither the +# uplink nor local replies need a rule. +sub generate_vrf_ct_zone_config { + my ($ipversion, $ct_zone, @ifaces) = @_; + + return generate_iptables_rules( + $ipversion, + map { ['raw', '-I', "PREROUTING -i $_ -j CT --zone-orig $ct_zone"] } @ifaces, + ); +} + sub find_local_ip_interface_peers { my ($peers, $iface) = @_; diff --git a/src/PVE/Network/SDN/Zones/SimplePlugin.pm b/src/PVE/Network/SDN/Zones/SimplePlugin.pm index 1d44f0b078f5..c8718629ac28 100644 --- a/src/PVE/Network/SDN/Zones/SimplePlugin.pm +++ b/src/PVE/Network/SDN/Zones/SimplePlugin.pm @@ -106,6 +106,7 @@ sub generate_sdn_config { my $ipv6 = undef; my $enable_forward_v4 = undef; my $enable_forward_v6 = undef; + my $ct_zones = {}; foreach my $subnetid (sort keys %{$subnets}) { my $subnet = $subnets->{$subnetid}; @@ -118,18 +119,15 @@ sub generate_sdn_config { $address->{$gateway} = 1; } - my $iptables = undef; my $checkrouteip = undef; my $ipversion = Net::IP::ip_is_ipv6($gateway) ? 6 : 4; if ($ipversion == 6) { $ipv6 = 1; - $iptables = "ip6tables"; $checkrouteip = '2001:4860:4860::8888'; $enable_forward_v6 = 1 if $gateway; } else { $ipv4 = 1; - $iptables = "iptables"; $checkrouteip = '8.8.8.8'; $enable_forward_v4 = 1 if $gateway; } @@ -141,31 +139,43 @@ sub generate_sdn_config { } if ($subnet->{snat}) { #find outgoing interface - my ($outip, $outiface); + my ($outip, $outiface, $ct_zone); eval { - ($outip, $outiface) = - PVE::Network::SDN::Zones::Plugin::get_local_route_ip($checkrouteip); + if ($plugin_config->{'dedicated-vrf'}) { + ($outip, $outiface) = + PVE::Network::SDN::Zones::Plugin::find_dedicated_uplink( + $zone_id, $plugin_config, $ipversion, $interfaces_config, + ); + die "no uplink interface in vrf_$zone_id\n" if !$outiface; + $ct_zone = $plugin_config->{'vrf-ct-zone'}; + } else { + ($outip, $outiface) = + PVE::Network::SDN::Zones::Plugin::get_local_route_ip($checkrouteip); + $outiface = undef if !$outip; + } }; if ($@) { my $msg = "interface for SNAT could not be resolved: $@"; log_warn($msg); next; } - if ($outip && $outiface) { - #use snat, faster than masquerade - push @iface_config, - "post-up $iptables -t nat -A POSTROUTING -s '$cidr' -o $outiface -j SNAT --to-source $outip"; - push @iface_config, - "post-down $iptables -t nat -D POSTROUTING -s '$cidr' -o $outiface -j SNAT --to-source $outip"; - #add conntrack zone once on outgoing interface - push @iface_config, - "post-up $iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1"; - push @iface_config, - "post-down $iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1"; - } + next if !$outiface; + + $ct_zones->{$ipversion} = $ct_zone if defined($ct_zone); + push @iface_config, + PVE::Network::SDN::Zones::Plugin::generate_snat_config( + $ipversion, $cidr, $outip, $outiface, + ); } } + for my $ipversion (sort keys %$ct_zones) { + push @iface_config, + PVE::Network::SDN::Zones::Plugin::generate_vrf_ct_zone_config( + $ipversion, $ct_zones->{$ipversion}, $vnetid, + ); + } + push @iface_config, "hwaddress $mac" if $mac; push @iface_config, "bridge_ports none"; push @iface_config, "bridge_stp off"; diff --git a/src/test/run_test_zones.pl b/src/test/run_test_zones.pl index f8ef7375cdc0..afb0ed5d8a00 100755 --- a/src/test/run_test_zones.pl +++ b/src/test/run_test_zones.pl @@ -93,7 +93,9 @@ foreach my $test (@tests) { $pve_sdn_zones_plugin = Test::MockModule->new('PVE::Network::SDN::Zones::Plugin'); $pve_sdn_zones_plugin->mock( get_local_route_ip => sub { - my ($targetip) = @_; + my ($targetip, $vrf) = @_; + # VRFs do not exist yet on the first apply + die "Cannot find device \"$vrf\"\n" if defined($vrf); my $outiface = "vmbr0"; my $outip = $interfaces_config->{ifaces}->{$outiface}->{address}; if (Net::IP::ip_is_ipv6($targetip)) { diff --git a/src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_controller_config b/src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_controller_config new file mode 100644 index 000000000000..0a19fc711bf6 --- /dev/null +++ b/src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_controller_config @@ -0,0 +1,79 @@ +frr version 10.6.1 +frr defaults datacenter +hostname localhost +log syslog informational +service integrated-vtysh-config +! +vrf vrf_myzone + vni 1000 +exit-vrf +! +router bgp 65000 + bgp router-id 192.168.0.1 + no bgp hard-administrative-reset + no bgp default ipv4-unicast + coalesce-time 1000 + no bgp graceful-restart notification + neighbor VTEP peer-group + neighbor VTEP remote-as 65000 + neighbor VTEP bfd + neighbor 192.168.0.2 peer-group VTEP + neighbor 192.168.0.3 peer-group VTEP + ! + address-family ipv4 unicast + import vrf vrf_myzone + exit-address-family + ! + address-family ipv6 unicast + import vrf vrf_myzone + exit-address-family + ! + address-family l2vpn evpn + neighbor VTEP activate + neighbor VTEP route-map MAP_VTEP_IN in + neighbor VTEP route-map MAP_VTEP_OUT out + advertise-all-vni + exit-address-family +exit +! +router bgp 65000 vrf vrf_myzone + bgp router-id 192.168.0.1 + no bgp hard-administrative-reset + no bgp graceful-restart notification + ! + address-family ipv4 unicast + redistribute connected + exit-address-family + ! + address-family ipv6 unicast + redistribute connected + exit-address-family + ! + address-family l2vpn evpn + default-originate ipv4 + default-originate ipv6 + exit-address-family +exit +! +ip prefix-list only_default seq 1 permit 0.0.0.0/0 +! +ipv6 prefix-list only_default_v6 seq 1 permit ::/0 +! +route-map MAP_VTEP_IN deny 1 + match ip address prefix-list only_default +exit +! +route-map MAP_VTEP_IN deny 2 + match ipv6 address prefix-list only_default_v6 +exit +! +route-map MAP_VTEP_IN permit 3 +exit +! +route-map MAP_VTEP_OUT permit 1 +exit +! +ip route 0.0.0.0/0 203.0.113.1 vrf vrf_myzone +! +line vty +! diff --git a/src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_sdn_interfaces b/src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_sdn_interfaces new file mode 100644 index 000000000000..1848a62e657e --- /dev/null +++ b/src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_sdn_interfaces @@ -0,0 +1,73 @@ +#version:1 + +auto myvnet +iface myvnet + address 10.0.0.1/24 + post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth1 -j SNAT --to-source 203.0.113.10 + post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth1 -j SNAT --to-source 203.0.113.10 + post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1 + post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1 + post-up iptables -t raw -I PREROUTING -i myvnet -j CT --zone-orig 7767 + post-down iptables -t raw -D PREROUTING -i myvnet -j CT --zone-orig 7767 + post-up iptables -t raw -I PREROUTING -i vrfbr_myzone -j CT --zone-orig 7767 + post-down iptables -t raw -D PREROUTING -i vrfbr_myzone -j CT --zone-orig 7767 + bridge_ports vxlan_myvnet + bridge_stp off + bridge_fd 0 + mtu 1450 + ip-forward on + arp-accept on + vrf vrf_myzone + +auto myvnet2 +iface myvnet2 + address 2a08:2142:302:3::1/64 + post-up ip6tables -t nat -A POSTROUTING -s '2a08:2142:302:3::/64' -o vmbr0 -j SNAT --to-source 2001:db8::2 + post-down ip6tables -t nat -D POSTROUTING -s '2a08:2142:302:3::/64' -o vmbr0 -j SNAT --to-source 2001:db8::2 + post-up ip6tables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1 + post-down ip6tables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1 + bridge_ports vxlan_myvnet2 + bridge_stp off + bridge_fd 0 + mtu 1450 + ip6-forward on + arp-accept on + vrf vrf_myzone + +auto vrf_myzone +iface vrf_myzone + vrf-table auto + post-up ip route del vrf vrf_myzone unreachable default metric 4278198272 + post-up ip -6 route del vrf vrf_myzone unreachable default metric 4278198272 + +auto vrfbr_myzone +iface vrfbr_myzone + bridge-ports vrfvx_myzone + bridge_stp off + bridge_fd 0 + mtu 1450 + vrf vrf_myzone + +auto vrfvx_myzone +iface vrfvx_myzone + vxlan-id 1000 + vxlan-local-tunnelip 192.168.0.1 + bridge-learning off + bridge-arp-nd-suppress on + mtu 1450 + +auto vxlan_myvnet +iface vxlan_myvnet + vxlan-id 100 + vxlan-local-tunnelip 192.168.0.1 + bridge-learning off + bridge-arp-nd-suppress on + mtu 1450 + +auto vxlan_myvnet2 +iface vxlan_myvnet2 + vxlan-id 200 + vxlan-local-tunnelip 192.168.0.1 + bridge-learning off + bridge-arp-nd-suppress on + mtu 1450 diff --git a/src/test/zones/evpn/exitnode_snat_vrf_uplink/interfaces b/src/test/zones/evpn/exitnode_snat_vrf_uplink/interfaces new file mode 100644 index 000000000000..f51116efdde5 --- /dev/null +++ b/src/test/zones/evpn/exitnode_snat_vrf_uplink/interfaces @@ -0,0 +1,16 @@ +auto vmbr0 +iface vmbr0 inet static + address 192.168.0.1/24 + gateway 192.168.0.254 + bridge-ports eth0 + bridge-stp off + bridge-fd 0 + +iface vmbr0 inet6 static + address 2001:db8::2/64 + gateway 2001:db8::1 + +auto eth1 +iface eth1 inet static + address 203.0.113.10/24 + vrf vrf_myzone diff --git a/src/test/zones/evpn/exitnode_snat_vrf_uplink/sdn_config b/src/test/zones/evpn/exitnode_snat_vrf_uplink/sdn_config new file mode 100644 index 000000000000..ac0db7778083 --- /dev/null +++ b/src/test/zones/evpn/exitnode_snat_vrf_uplink/sdn_config @@ -0,0 +1,42 @@ +{ + version => 1, + vnets => { + ids => { + myvnet => { tag => "100", type => "vnet", zone => "myzone" }, + myvnet2 => { tag => "200", type => "vnet", zone => "myzone" }, + }, + }, + + zones => { + ids => { + myzone => { + ipam => "pve", + type => "evpn", + controller => "evpnctl", + 'vrf-vxlan' => 1000, + exitnodes => { 'localhost' => 1 }, + route => [ 'prefix=0.0.0.0/0,via=203.0.113.1' ], + }, + }, + }, + controllers => { + ids => { evpnctl => { type => "evpn", 'peers' => '192.168.0.1,192.168.0.2,192.168.0.3', asn => "65000" } }, + }, + + subnets => { + ids => { + 'myzone-10.0.0.0-24' => { + 'type' => 'subnet', + 'vnet' => 'myvnet', + 'gateway' => '10.0.0.1', + 'snat' => 1 + }, + 'myzone-2a08:2142:302:3::-64' => { + 'type' => 'subnet', + 'vnet' => 'myvnet2', + 'gateway' => '2a08:2142:302:3::1', + 'snat' => 1 + }, + } + } +} diff --git a/src/test/zones/simple/vrf_snat_uplink/expected_sdn_interfaces b/src/test/zones/simple/vrf_snat_uplink/expected_sdn_interfaces new file mode 100644 index 000000000000..f639eb132a64 --- /dev/null +++ b/src/test/zones/simple/vrf_snat_uplink/expected_sdn_interfaces @@ -0,0 +1,92 @@ +#version:1 + +auto vneta +iface vneta + address 10.0.0.1/24 + post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth1 -j SNAT --to-source 203.0.113.10 + post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth1 -j SNAT --to-source 203.0.113.10 + post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1 + post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1 + address 10.1.0.1/24 + post-up iptables -t nat -A POSTROUTING -s '10.1.0.0/24' -o eth1 -j SNAT --to-source 203.0.113.10 + post-down iptables -t nat -D POSTROUTING -s '10.1.0.0/24' -o eth1 -j SNAT --to-source 203.0.113.10 + post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1 + post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1 + address 2001:db8:a::1/64 + post-up ip6tables -t nat -A POSTROUTING -s '2001:db8:a::/64' -o eth1 -j MASQUERADE + post-down ip6tables -t nat -D POSTROUTING -s '2001:db8:a::/64' -o eth1 -j MASQUERADE + post-up ip6tables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1 + post-down ip6tables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1 + post-up iptables -t raw -I PREROUTING -i vneta -j CT --zone-orig 64008 + post-down iptables -t raw -D PREROUTING -i vneta -j CT --zone-orig 64008 + post-up ip6tables -t raw -I PREROUTING -i vneta -j CT --zone-orig 64008 + post-down ip6tables -t raw -D PREROUTING -i vneta -j CT --zone-orig 64008 + bridge_ports none + bridge_stp off + bridge_fd 0 + ip-forward on + ip6-forward on + vrf vrf_zonea + +auto vnetb +iface vnetb + address 10.0.0.1/24 + post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth2 -j SNAT --to-source 198.51.100.10 + post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth2 -j SNAT --to-source 198.51.100.10 + post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1 + post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1 + post-up iptables -t raw -I PREROUTING -i vnetb -j CT --zone-orig 48540 + post-down iptables -t raw -D PREROUTING -i vnetb -j CT --zone-orig 48540 + bridge_ports none + bridge_stp off + bridge_fd 0 + ip-forward on + vrf vrf_zoneb + +auto vnetc +iface vnetc + address 10.0.0.1/24 + bridge_ports none + bridge_stp off + bridge_fd 0 + ip-forward on + vrf vrf_zonec + +auto vnetd +iface vnetd + address 10.0.0.1/24 + post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth3 -j SNAT --to-source 10.100.0.4 + post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth3 -j SNAT --to-source 10.100.0.4 + post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1 + post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1 + post-up iptables -t raw -I PREROUTING -i vnetd -j CT --zone-orig 27935 + post-down iptables -t raw -D PREROUTING -i vnetd -j CT --zone-orig 27935 + bridge_ports none + bridge_stp off + bridge_fd 0 + ip-forward on + vrf vrf_zoned + +auto vrf_zonea +iface vrf_zonea + vrf-table auto + post-up ip route add vrf vrf_zonea unreachable default metric 4278198272 + post-up ip -6 route add vrf vrf_zonea unreachable default metric 4278198272 + +auto vrf_zoneb +iface vrf_zoneb + vrf-table auto + post-up ip route add vrf vrf_zoneb unreachable default metric 4278198272 + post-up ip -6 route add vrf vrf_zoneb unreachable default metric 4278198272 + +auto vrf_zonec +iface vrf_zonec + vrf-table auto + post-up ip route add vrf vrf_zonec unreachable default metric 4278198272 + post-up ip -6 route add vrf vrf_zonec unreachable default metric 4278198272 + +auto vrf_zoned +iface vrf_zoned + vrf-table auto + post-up ip route add vrf vrf_zoned unreachable default metric 4278198272 + post-up ip -6 route add vrf vrf_zoned unreachable default metric 4278198272 diff --git a/src/test/zones/simple/vrf_snat_uplink/interfaces b/src/test/zones/simple/vrf_snat_uplink/interfaces new file mode 100644 index 000000000000..de646d3f6b43 --- /dev/null +++ b/src/test/zones/simple/vrf_snat_uplink/interfaces @@ -0,0 +1,26 @@ +auto vmbr0 +iface vmbr0 inet static + address 192.168.0.1/24 + gateway 192.168.0.254 + bridge-ports eth0 + bridge-stp off + bridge-fd 0 + +auto eth1 +iface eth1 inet static + address 203.0.113.10/24 + vrf vrf_zonea + +iface eth1 inet6 auto + vrf vrf_zonea + +auto eth2 +iface eth2 inet static + address 198.51.100.10/24 + vrf vrf_zoneb + +auto eth3 +iface eth3 inet static + address 10.100.0.4/24 + gateway 10.100.0.1 + vrf vrf_zoned diff --git a/src/test/zones/simple/vrf_snat_uplink/sdn_config b/src/test/zones/simple/vrf_snat_uplink/sdn_config new file mode 100644 index 000000000000..7cd558dcc671 --- /dev/null +++ b/src/test/zones/simple/vrf_snat_uplink/sdn_config @@ -0,0 +1,84 @@ +{ + version => 1, + vnets => { + ids => { + vneta => { type => "vnet", zone => "zonea" }, + vnetb => { type => "vnet", zone => "zoneb" }, + vnetc => { type => "vnet", zone => "zonec" }, + vnetd => { type => "vnet", zone => "zoned" }, + }, + }, + zones => { + ids => { + zonea => { + ipam => "pve", + type => "simple", + "dedicated-vrf" => 1, + route => [ + 'prefix=0.0.0.0/0,via=203.0.113.1', + 'prefix=::/0,via=fe80::1,iface=eth1', + ], + }, + zoneb => { + ipam => "pve", + type => "simple", + "dedicated-vrf" => 1, + route => [ + 'prefix=0.0.0.0/0,via=198.51.100.2,distance=10', + 'prefix=0.0.0.0/0,via=198.51.100.1', + ], + }, + zonec => { + ipam => "pve", + type => "simple", + "dedicated-vrf" => 1, + route => [ 'prefix=0.0.0.0/0,via=192.168.0.254,nexthop-vrf=default' ], + }, + zoned => { + ipam => "pve", + type => "simple", + "dedicated-vrf" => 1, + }, + }, + }, + subnets => { + ids => { + 'zonea-10.0.0.0-24' => { + type => 'subnet', + vnet => 'vneta', + gateway => '10.0.0.1', + snat => 1, + }, + 'zonea-10.1.0.0-24' => { + type => 'subnet', + vnet => 'vneta', + gateway => '10.1.0.1', + snat => 1, + }, + 'zonea-2001:db8:a::-64' => { + type => 'subnet', + vnet => 'vneta', + gateway => '2001:db8:a::1', + snat => 1, + }, + 'zoneb-10.0.0.0-24' => { + type => 'subnet', + vnet => 'vnetb', + gateway => '10.0.0.1', + snat => 1, + }, + 'zonec-10.0.0.0-24' => { + type => 'subnet', + vnet => 'vnetc', + gateway => '10.0.0.1', + snat => 1, + }, + 'zoned-10.0.0.0-24' => { + type => 'subnet', + vnet => 'vnetd', + gateway => '10.0.0.1', + snat => 1, + }, + }, + }, +} -- 2.47.3