From: Stefan Hanreich <s.hanreich@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: Re: [PATCH pve-network v3 23/32] sdn: zones: SNAT via dedicated uplinks in zone VRFs
Date: Wed, 18 Nov 2026 19:07:51 +0100 [thread overview]
Message-ID: <df54d1df-e47f-464e-8382-79526cd5acb0@proxmox.com> (raw)
In-Reply-To: <20261116161154.343743-24-g.goller@proxmox.com>
SNAT has some problems, as it doesn't work with the nftables firewall at all,
but we have some CT problems regardless methinks. I think we have to set the
CT zone for *every* packet in *every* direction as soon as the packet enters
the networking stack. That's because we have overlapping subnets now in
simple zones even if we don't implement SNAT at all as otherwise
this could be problematic for hosts that have the firewall enabled and
use VRFs since CT entries can clash!
We have to set the zone probably at all of those points:
* in PREROUTING on the vnet / fwbr with appropriate priority (-300)
for traffic exiting guests
* in OUTPUT for non-guest interfaces
for traffic being sent by interfaces in the VRF (dnsmasq!)
* in PREROUTING for the non-guest interfaces with appropriate priority
for traffic entering the zone through physical interfaces
We can only do this for simple zones for now. I think we could manage with
SNAT in simple zones, but only when the uplink is dedicated to the VRF
(with the changes above!) - shared uplink I'd drop.
There are also some other issues, that we've also already touched in my
initial reply, I have looked at some of those things more in-depth and
there's a lot of stuff I need to dive in deeper here - I've stoppped for
now to finish the review...
Also, zone CT IDs aren't stable when creating a new zone that is
alphabetically before another zone that uses VRFs.
On 11/16/26 5:17 PM, Gabriel Goller wrote:
> SNAT for subnets in a zone VRF resolved the outgoing interface and source
> address in the default routing table, so the generated rule never matched
> traffic leaving through the VRF. Allow moving a dedicated uplink into the
> VRF instead, which keeps the zone isolated from the default routing table.
>
> The uplink is derived from the zone's default route, or else from the
> gateway of a host interface in the VRF, since the config is generated before
> ifreload creates the VRF and moves the interface into it. Only without
> either, e.g. for a DHCP uplink, the VRF routing table is queried. MASQUERADE is used if the
> uplink has no static address. Simple zones with a VRF never fall back to
> the default routing table, EVPN exit nodes without a dedicated uplink keep
> the current behavior.
>
> As subnets can overlap between zones, connections from the vnets of such
> zones are put into a conntrack zone per VRF in the original direction.
> Replies are found via their NATed reply tuple in the default zone, so the
> uplink needs no rule. The conntrack zone ID is hashed from the VRF name to
> stay stable across reloads.
>
> Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
> ---
> src/PVE/Network/SDN/Zones.pm | 3 +-
> src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 51 +++---
> src/PVE/Network/SDN/Zones/Plugin.pm | 157 +++++++++++++++++-
> src/PVE/Network/SDN/Zones/SimplePlugin.pm | 46 +++--
> src/test/run_test_zones.pl | 4 +-
> .../expected_controller_config | 79 +++++++++
> .../expected_sdn_interfaces | 73 ++++++++
> .../evpn/exitnode_snat_vrf_uplink/interfaces | 16 ++
> .../evpn/exitnode_snat_vrf_uplink/sdn_config | 42 +++++
> .../vrf_snat_uplink/expected_sdn_interfaces | 92 ++++++++++
> .../zones/simple/vrf_snat_uplink/interfaces | 26 +++
> .../zones/simple/vrf_snat_uplink/sdn_config | 84 ++++++++++
> 12 files changed, 632 insertions(+), 41 deletions(-)
> create mode 100644 src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_controller_config
> create mode 100644 src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_sdn_interfaces
> create mode 100644 src/test/zones/evpn/exitnode_snat_vrf_uplink/interfaces
> create mode 100644 src/test/zones/evpn/exitnode_snat_vrf_uplink/sdn_config
> create mode 100644 src/test/zones/simple/vrf_snat_uplink/expected_sdn_interfaces
> create mode 100644 src/test/zones/simple/vrf_snat_uplink/interfaces
> create mode 100644 src/test/zones/simple/vrf_snat_uplink/sdn_config
>
> diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm
> index 1d3ad22c1cd5..0f64f39f6ac6 100644
> --- a/src/PVE/Network/SDN/Zones.pm
> +++ b/src/PVE/Network/SDN/Zones.pm
> @@ -133,6 +133,7 @@ sub generate_etc_network_config {
> #generate configuration
> my $config = {};
> my $nodename = PVE::INotify::nodename();
> + my $vrf_ct_zones = PVE::Network::SDN::Zones::Plugin::get_vrf_ct_zones($zone_ids);
>
> for my $zone_id (sort keys %$zone_ids) {
> my $plugin_config = $zone_ids->{$zone_id};
> @@ -168,7 +169,7 @@ sub generate_etc_network_config {
> my $plugin = PVE::Network::SDN::Zones::Plugin->lookup($plugin_config->{type});
> eval {
> $plugin->generate_sdn_config(
> - $plugin_config,
> + { %$plugin_config, 'vrf-ct-zone' => $vrf_ct_zones->{$zone} },
> $zone,
> $id,
> $vnet,
> diff --git a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
> index 1933ce86f997..b4b4ab450384 100644
> --- a/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
> +++ b/src/PVE/Network/SDN/Zones/EvpnPlugin.pm
> @@ -268,6 +268,7 @@ sub generate_sdn_config {
> my $ipv6 = undef;
> my $enable_forward_v4 = undef;
> my $enable_forward_v6 = undef;
> + my $ct_zones = {};
> my $subnets = PVE::Network::SDN::Vnets::get_subnets($vnetid, 1);
> foreach my $subnetid (sort keys %{$subnets}) {
> my $subnet = $subnets->{$subnetid};
> @@ -280,50 +281,62 @@ sub generate_sdn_config {
> $address->{$gateway} = 1;
> }
>
> - my $iptables = undef;
> my $checkrouteip = undef;
> my $ipversion = Net::IP::ip_is_ipv6($gateway) ? 6 : 4;
>
> if ($ipversion == 6) {
> $ipv6 = 1;
> - $iptables = "ip6tables";
> $checkrouteip = '2001:4860:4860::8888';
> $enable_forward_v6 = 1 if $gateway;
> } else {
> $ipv4 = 1;
> - $iptables = "iptables";
> $checkrouteip = '8.8.8.8';
> $enable_forward_v4 = 1 if $gateway;
> }
>
> - if ($subnet->{snat}) {
> + if ($subnet->{snat} && $is_evpn_gateway) {
>
> #find outgoing interface
> - my ($outip, $outiface);
> + my ($outip, $outiface, $ct_zone);
> eval {
> - ($outip, $outiface) =
> - PVE::Network::SDN::Zones::Plugin::get_local_route_ip($checkrouteip);
> + ($outip, $outiface) = PVE::Network::SDN::Zones::Plugin::find_dedicated_uplink(
> + $zoneid, $plugin_config, $ipversion, $interfaces_config,
> + );
> + if ($outiface) {
> + $ct_zone = $plugin_config->{'vrf-ct-zone'};
> + } else {
> + # the VRF of exit nodes falls through to the default routing table
> + ($outip, $outiface) =
> + PVE::Network::SDN::Zones::Plugin::get_local_route_ip($checkrouteip);
> + $outiface = undef if !$outip;
> + }
> };
> if ($@) {
> my $msg = "interface for SNAT could not be resolved: $@";
> log_warn($msg);
> next;
> }
> - if ($outip && $outiface && $is_evpn_gateway) {
> - #use snat, faster than masquerade
> - push @iface_config,
> - "post-up $iptables -t nat -A POSTROUTING -s '$cidr' -o $outiface -j SNAT --to-source $outip";
> - push @iface_config,
> - "post-down $iptables -t nat -D POSTROUTING -s '$cidr' -o $outiface -j SNAT --to-source $outip";
> - #add conntrack zone once on outgoing interface
> - push @iface_config,
> - "post-up $iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1";
> - push @iface_config,
> - "post-down $iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1";
> - }
> + next if !$outiface;
> +
> + $ct_zones->{$ipversion} = $ct_zone if defined($ct_zone);
> + push @iface_config,
> + PVE::Network::SDN::Zones::Plugin::generate_snat_config(
> + $ipversion, $cidr, $outip, $outiface,
> + );
> }
> }
>
> + # also for traffic routed from other nodes via the L3VNI
> + for my $ipversion (sort keys %$ct_zones) {
> + push @iface_config,
> + PVE::Network::SDN::Zones::Plugin::generate_vrf_ct_zone_config(
> + $ipversion,
> + $ct_zones->{$ipversion},
> + $vnetid,
> + "vrfbr_$zoneid",
> + );
> + }
> +
> push @iface_config, "hwaddress $mac" if $mac;
> push @iface_config, "bridge_ports $vxlan_iface";
> push @iface_config, "bridge_stp off";
> diff --git a/src/PVE/Network/SDN/Zones/Plugin.pm b/src/PVE/Network/SDN/Zones/Plugin.pm
> index a3863bc9818c..5c4dafd01c9f 100644
> --- a/src/PVE/Network/SDN/Zones/Plugin.pm
> +++ b/src/PVE/Network/SDN/Zones/Plugin.pm
> @@ -4,6 +4,7 @@ use strict;
> use warnings;
>
> use PVE::Tools qw(run_command);
> +use Digest::SHA;
> use Net::IP qw(ip_get_version);
> use PVE::JSONSchema;
> use PVE::Cluster;
> @@ -378,13 +379,15 @@ sub get_iface_addresses {
> }
>
> sub get_local_route_ip {
> - my ($targetip) = @_;
> + my ($targetip, $vrf) = @_;
>
> my $ip = undef;
> my $interface = undef;
>
> run_command(
> - ['/sbin/ip', 'route', 'get', $targetip],
> + ['/sbin/ip', 'route', 'get', $targetip, defined($vrf) ? ('vrf', $vrf) : ()],
> + # a missing VRF or route is expected on the first apply, and handled by the caller
> + defined($vrf) ? (errfunc => sub { }) : (),
> outfunc => sub {
> if ($_[0] =~ m/src ($PVE::Tools::IPRE)/) {
> $ip = $1;
> @@ -398,6 +401,156 @@ sub get_local_route_ip {
> return ($ip, $interface);
> }
>
> +# Assign each zone with a VRF a conntrack zone, so that identical connections
> +# from overlapping subnets in different VRFs do not clash.
> +#
> +# The ID is derived from the VRF name, so it only changes if a zone with a
> +# colliding hash is added. Zones 0 (default) and 1 (fwbr+ bridges) are reserved.
> +sub get_vrf_ct_zones {
> + my ($zones) = @_;
> +
> + my $ct_zones = {};
> + my $used = {};
> +
> + for my $zone_id (sort keys %$zones) {
> + my $zone = $zones->{$zone_id};
> + next if $zone->{type} ne 'evpn' && !($zone->{type} eq 'simple' && $zone->{'dedicated-vrf'});
> +
> + my $ct_zone = 2 + unpack('n', Digest::SHA::sha256("vrf_$zone_id")) % 65534;
> + $ct_zone = $ct_zone == 65535 ? 2 : $ct_zone + 1 while $used->{$ct_zone};
> +
> + $used->{$ct_zone} = 1;
> + $ct_zones->{$zone_id} = $ct_zone;
> + }
> +
> + return $ct_zones;
> +}
> +
> +# Return the zone's default route of the given IP version with the lowest
> +# administrative distance.
> +sub find_zone_default_route {
> + my ($plugin_config, $ipversion) = @_;
> +
> + my @routes;
> + for my $route_str (@{ $plugin_config->{route} // [] }) {
> + my $route = PVE::JSONSchema::parse_property_string('pve-sdn-static-route', $route_str);
> + my ($ip, $mask) = split(/\//, $route->{prefix}, 2);
> + next if !defined($mask) || $mask != 0 || (ip_get_version($ip) // 0) != $ipversion;
> + push @routes, $route;
> + }
> +
> + my ($route) = sort { ($a->{distance} // 1) <=> ($b->{distance} // 1) } @routes;
> + return $route;
> +}
> +
> +# Resolve the dedicated SNAT uplink of a zone, i.e. the host interface in the
> +# zone VRF that the default route of the given IP version points to.
> +#
> +# The uplink is resolved from the configuration if possible, since neither the
> +# VRF nor its routes exist on the first apply. A default route of the zone takes
> +# precedence over an interface in the VRF with a gateway. Without either, e.g.
> +# if the uplink gets its gateway via DHCP, the VRF routing table is queried.
> +#
> +# Returns ($ip, $iface), with $ip undef if the uplink has no static address, or
> +# nothing if the zone has no dedicated uplink. Dies if the default route has its
> +# next hop in the zone VRF, but not via an interface in it.
> +sub find_dedicated_uplink {
> + my ($zone_id, $plugin_config, $ipversion, $interfaces_config) = @_;
> +
> + my $vrf = "vrf_$zone_id";
> + my $ifaces = $interfaces_config->{ifaces} // {};
> + my ($address_key, $method_key, $cidr_key, $gateway_key) =
> + $ipversion == 6
> + ? ('address6', 'method6', 'cidr6', 'gateway6')
> + : ('address', 'method', 'cidr', 'gateway');
> +
> + my $is_member = sub {
> + my ($iface) = @_;
> + return defined($iface) && ($ifaces->{$iface}->{vrf} // '') eq $vrf;
> + };
> +
> + my $iface;
> + if (my $route = find_zone_default_route($plugin_config, $ipversion)) {
> + return if ($route->{'nexthop-vrf'} // $vrf) ne $vrf;
> +
> + $iface = $route->{iface};
> + if (!defined($iface) && defined(my $via = $route->{via})) {
> + ($iface) = grep {
> + $is_member->($_)
> + && $ifaces->{$_}->{$cidr_key}
> + && PVE::Network::is_ip_in_cidr($via, $ifaces->{$_}->{$cidr_key}, $ipversion)
> + } sort keys %$ifaces;
> + }
> +
> + die "default route of zone '$zone_id' is not via an interface in $vrf\n"
> + if !$is_member->($iface);
> + } elsif (
> + my @gw_ifaces =
> + grep { $is_member->($_) && $ifaces->{$_}->{$gateway_key} } sort keys %$ifaces
> + ) {
> + # ifupdown2 installs the gateway into the VRF routing table
> + $iface = $gw_ifaces[0];
> + } else {
> + my $targetip = $ipversion == 6 ? '2001:4860:4860::8888' : '8.8.8.8';
> + # fails if the VRF or its default route do not exist yet
> + (undef, $iface) = eval { get_local_route_ip($targetip, $vrf) };
> + return if !$is_member->($iface);
> + }
> +
> + my $ip;
> + if (($ifaces->{$iface}->{$method_key} // '') eq 'static') {
> + $ip = $ifaces->{$iface}->{$address_key};
> + $ip = $ip->[0] if ref($ip) eq 'ARRAY';
> + }
> +
> + return ($ip, $iface);
> +}
> +
> +my sub generate_iptables_rules {
> + my ($ipversion, @rules) = @_;
> +
> + my $iptables = $ipversion == 6 ? 'ip6tables' : 'iptables';
> +
> + my @iface_config;
> + for my $rule (@rules) {
> + my ($table, $action, $spec) = @$rule;
> + push @iface_config, "post-up $iptables -t $table $action $spec";
> + push @iface_config, "post-down $iptables -t $table -D $spec";
> + }
> +
> + return @iface_config;
> +}
> +
> +# Generate the ifupdown2 commands to SNAT $cidr out of $outiface, using
> +# MASQUERADE if the source address is not known.
> +sub generate_snat_config {
> + my ($ipversion, $cidr, $outip, $outiface) = @_;
> +
> + my $target = defined($outip) ? "SNAT --to-source $outip" : 'MASQUERADE';
> +
> + return generate_iptables_rules(
> + $ipversion,
> + #use snat, faster than masquerade
> + ['nat', '-A', "POSTROUTING -s '$cidr' -o $outiface -j $target"],
> + #add conntrack zone once on outgoing interface
> + ['raw', '-I', 'PREROUTING -i fwbr+ -j CT --zone 1'],
> + );
> +}
> +
> +# Put connections received on @ifaces into conntrack zone $ct_zone in the
> +# original direction, so identical connections from overlapping subnets in
> +# other VRFs do not clash before being NATed. Replies are looked up by their
> +# reply tuple in the default zone, which NAT keeps unique, so neither the
> +# uplink nor local replies need a rule.
> +sub generate_vrf_ct_zone_config {
> + my ($ipversion, $ct_zone, @ifaces) = @_;
> +
> + return generate_iptables_rules(
> + $ipversion,
> + map { ['raw', '-I', "PREROUTING -i $_ -j CT --zone-orig $ct_zone"] } @ifaces,
> + );
> +}
> +
> sub find_local_ip_interface_peers {
> my ($peers, $iface) = @_;
>
> diff --git a/src/PVE/Network/SDN/Zones/SimplePlugin.pm b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
> index 1d44f0b078f5..c8718629ac28 100644
> --- a/src/PVE/Network/SDN/Zones/SimplePlugin.pm
> +++ b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
> @@ -106,6 +106,7 @@ sub generate_sdn_config {
> my $ipv6 = undef;
> my $enable_forward_v4 = undef;
> my $enable_forward_v6 = undef;
> + my $ct_zones = {};
>
> foreach my $subnetid (sort keys %{$subnets}) {
> my $subnet = $subnets->{$subnetid};
> @@ -118,18 +119,15 @@ sub generate_sdn_config {
> $address->{$gateway} = 1;
> }
>
> - my $iptables = undef;
> my $checkrouteip = undef;
> my $ipversion = Net::IP::ip_is_ipv6($gateway) ? 6 : 4;
>
> if ($ipversion == 6) {
> $ipv6 = 1;
> - $iptables = "ip6tables";
> $checkrouteip = '2001:4860:4860::8888';
> $enable_forward_v6 = 1 if $gateway;
> } else {
> $ipv4 = 1;
> - $iptables = "iptables";
> $checkrouteip = '8.8.8.8';
> $enable_forward_v4 = 1 if $gateway;
> }
> @@ -141,31 +139,43 @@ sub generate_sdn_config {
> }
> if ($subnet->{snat}) {
> #find outgoing interface
> - my ($outip, $outiface);
> + my ($outip, $outiface, $ct_zone);
> eval {
> - ($outip, $outiface) =
> - PVE::Network::SDN::Zones::Plugin::get_local_route_ip($checkrouteip);
> + if ($plugin_config->{'dedicated-vrf'}) {
> + ($outip, $outiface) =
> + PVE::Network::SDN::Zones::Plugin::find_dedicated_uplink(
> + $zone_id, $plugin_config, $ipversion, $interfaces_config,
> + );
> + die "no uplink interface in vrf_$zone_id\n" if !$outiface;
> + $ct_zone = $plugin_config->{'vrf-ct-zone'};
> + } else {
> + ($outip, $outiface) =
> + PVE::Network::SDN::Zones::Plugin::get_local_route_ip($checkrouteip);
> + $outiface = undef if !$outip;
> + }
> };
> if ($@) {
> my $msg = "interface for SNAT could not be resolved: $@";
> log_warn($msg);
> next;
> }
> - if ($outip && $outiface) {
> - #use snat, faster than masquerade
> - push @iface_config,
> - "post-up $iptables -t nat -A POSTROUTING -s '$cidr' -o $outiface -j SNAT --to-source $outip";
> - push @iface_config,
> - "post-down $iptables -t nat -D POSTROUTING -s '$cidr' -o $outiface -j SNAT --to-source $outip";
> - #add conntrack zone once on outgoing interface
> - push @iface_config,
> - "post-up $iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1";
> - push @iface_config,
> - "post-down $iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1";
> - }
> + next if !$outiface;
> +
> + $ct_zones->{$ipversion} = $ct_zone if defined($ct_zone);
> + push @iface_config,
> + PVE::Network::SDN::Zones::Plugin::generate_snat_config(
> + $ipversion, $cidr, $outip, $outiface,
> + );
> }
> }
>
> + for my $ipversion (sort keys %$ct_zones) {
> + push @iface_config,
> + PVE::Network::SDN::Zones::Plugin::generate_vrf_ct_zone_config(
> + $ipversion, $ct_zones->{$ipversion}, $vnetid,
> + );
> + }
> +
> push @iface_config, "hwaddress $mac" if $mac;
> push @iface_config, "bridge_ports none";
> push @iface_config, "bridge_stp off";
> diff --git a/src/test/run_test_zones.pl b/src/test/run_test_zones.pl
> index f8ef7375cdc0..afb0ed5d8a00 100755
> --- a/src/test/run_test_zones.pl
> +++ b/src/test/run_test_zones.pl
> @@ -93,7 +93,9 @@ foreach my $test (@tests) {
> $pve_sdn_zones_plugin = Test::MockModule->new('PVE::Network::SDN::Zones::Plugin');
> $pve_sdn_zones_plugin->mock(
> get_local_route_ip => sub {
> - my ($targetip) = @_;
> + my ($targetip, $vrf) = @_;
> + # VRFs do not exist yet on the first apply
> + die "Cannot find device \"$vrf\"\n" if defined($vrf);
> my $outiface = "vmbr0";
> my $outip = $interfaces_config->{ifaces}->{$outiface}->{address};
> if (Net::IP::ip_is_ipv6($targetip)) {
> diff --git a/src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_controller_config b/src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_controller_config
> new file mode 100644
> index 000000000000..0a19fc711bf6
> --- /dev/null
> +++ b/src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_controller_config
> @@ -0,0 +1,79 @@
> +frr version 10.6.1
> +frr defaults datacenter
> +hostname localhost
> +log syslog informational
> +service integrated-vtysh-config
> +!
> +vrf vrf_myzone
> + vni 1000
> +exit-vrf
> +!
> +router bgp 65000
> + bgp router-id 192.168.0.1
> + no bgp hard-administrative-reset
> + no bgp default ipv4-unicast
> + coalesce-time 1000
> + no bgp graceful-restart notification
> + neighbor VTEP peer-group
> + neighbor VTEP remote-as 65000
> + neighbor VTEP bfd
> + neighbor 192.168.0.2 peer-group VTEP
> + neighbor 192.168.0.3 peer-group VTEP
> + !
> + address-family ipv4 unicast
> + import vrf vrf_myzone
> + exit-address-family
> + !
> + address-family ipv6 unicast
> + import vrf vrf_myzone
> + exit-address-family
> + !
> + address-family l2vpn evpn
> + neighbor VTEP activate
> + neighbor VTEP route-map MAP_VTEP_IN in
> + neighbor VTEP route-map MAP_VTEP_OUT out
> + advertise-all-vni
> + exit-address-family
> +exit
> +!
> +router bgp 65000 vrf vrf_myzone
> + bgp router-id 192.168.0.1
> + no bgp hard-administrative-reset
> + no bgp graceful-restart notification
> + !
> + address-family ipv4 unicast
> + redistribute connected
> + exit-address-family
> + !
> + address-family ipv6 unicast
> + redistribute connected
> + exit-address-family
> + !
> + address-family l2vpn evpn
> + default-originate ipv4
> + default-originate ipv6
> + exit-address-family
> +exit
> +!
> +ip prefix-list only_default seq 1 permit 0.0.0.0/0
> +!
> +ipv6 prefix-list only_default_v6 seq 1 permit ::/0
> +!
> +route-map MAP_VTEP_IN deny 1
> + match ip address prefix-list only_default
> +exit
> +!
> +route-map MAP_VTEP_IN deny 2
> + match ipv6 address prefix-list only_default_v6
> +exit
> +!
> +route-map MAP_VTEP_IN permit 3
> +exit
> +!
> +route-map MAP_VTEP_OUT permit 1
> +exit
> +!
> +ip route 0.0.0.0/0 203.0.113.1 vrf vrf_myzone
> +!
> +line vty
> +!
> diff --git a/src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_sdn_interfaces b/src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_sdn_interfaces
> new file mode 100644
> index 000000000000..1848a62e657e
> --- /dev/null
> +++ b/src/test/zones/evpn/exitnode_snat_vrf_uplink/expected_sdn_interfaces
> @@ -0,0 +1,73 @@
> +#version:1
> +
> +auto myvnet
> +iface myvnet
> + address 10.0.0.1/24
> + post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth1 -j SNAT --to-source 203.0.113.10
> + post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth1 -j SNAT --to-source 203.0.113.10
> + post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
> + post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
> + post-up iptables -t raw -I PREROUTING -i myvnet -j CT --zone-orig 7767
> + post-down iptables -t raw -D PREROUTING -i myvnet -j CT --zone-orig 7767
> + post-up iptables -t raw -I PREROUTING -i vrfbr_myzone -j CT --zone-orig 7767
> + post-down iptables -t raw -D PREROUTING -i vrfbr_myzone -j CT --zone-orig 7767
> + bridge_ports vxlan_myvnet
> + bridge_stp off
> + bridge_fd 0
> + mtu 1450
> + ip-forward on
> + arp-accept on
> + vrf vrf_myzone
> +
> +auto myvnet2
> +iface myvnet2
> + address 2a08:2142:302:3::1/64
> + post-up ip6tables -t nat -A POSTROUTING -s '2a08:2142:302:3::/64' -o vmbr0 -j SNAT --to-source 2001:db8::2
> + post-down ip6tables -t nat -D POSTROUTING -s '2a08:2142:302:3::/64' -o vmbr0 -j SNAT --to-source 2001:db8::2
> + post-up ip6tables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
> + post-down ip6tables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
> + bridge_ports vxlan_myvnet2
> + bridge_stp off
> + bridge_fd 0
> + mtu 1450
> + ip6-forward on
> + arp-accept on
> + vrf vrf_myzone
> +
> +auto vrf_myzone
> +iface vrf_myzone
> + vrf-table auto
> + post-up ip route del vrf vrf_myzone unreachable default metric 4278198272
> + post-up ip -6 route del vrf vrf_myzone unreachable default metric 4278198272
> +
> +auto vrfbr_myzone
> +iface vrfbr_myzone
> + bridge-ports vrfvx_myzone
> + bridge_stp off
> + bridge_fd 0
> + mtu 1450
> + vrf vrf_myzone
> +
> +auto vrfvx_myzone
> +iface vrfvx_myzone
> + vxlan-id 1000
> + vxlan-local-tunnelip 192.168.0.1
> + bridge-learning off
> + bridge-arp-nd-suppress on
> + mtu 1450
> +
> +auto vxlan_myvnet
> +iface vxlan_myvnet
> + vxlan-id 100
> + vxlan-local-tunnelip 192.168.0.1
> + bridge-learning off
> + bridge-arp-nd-suppress on
> + mtu 1450
> +
> +auto vxlan_myvnet2
> +iface vxlan_myvnet2
> + vxlan-id 200
> + vxlan-local-tunnelip 192.168.0.1
> + bridge-learning off
> + bridge-arp-nd-suppress on
> + mtu 1450
> diff --git a/src/test/zones/evpn/exitnode_snat_vrf_uplink/interfaces b/src/test/zones/evpn/exitnode_snat_vrf_uplink/interfaces
> new file mode 100644
> index 000000000000..f51116efdde5
> --- /dev/null
> +++ b/src/test/zones/evpn/exitnode_snat_vrf_uplink/interfaces
> @@ -0,0 +1,16 @@
> +auto vmbr0
> +iface vmbr0 inet static
> + address 192.168.0.1/24
> + gateway 192.168.0.254
> + bridge-ports eth0
> + bridge-stp off
> + bridge-fd 0
> +
> +iface vmbr0 inet6 static
> + address 2001:db8::2/64
> + gateway 2001:db8::1
> +
> +auto eth1
> +iface eth1 inet static
> + address 203.0.113.10/24
> + vrf vrf_myzone
> diff --git a/src/test/zones/evpn/exitnode_snat_vrf_uplink/sdn_config b/src/test/zones/evpn/exitnode_snat_vrf_uplink/sdn_config
> new file mode 100644
> index 000000000000..ac0db7778083
> --- /dev/null
> +++ b/src/test/zones/evpn/exitnode_snat_vrf_uplink/sdn_config
> @@ -0,0 +1,42 @@
> +{
> + version => 1,
> + vnets => {
> + ids => {
> + myvnet => { tag => "100", type => "vnet", zone => "myzone" },
> + myvnet2 => { tag => "200", type => "vnet", zone => "myzone" },
> + },
> + },
> +
> + zones => {
> + ids => {
> + myzone => {
> + ipam => "pve",
> + type => "evpn",
> + controller => "evpnctl",
> + 'vrf-vxlan' => 1000,
> + exitnodes => { 'localhost' => 1 },
> + route => [ 'prefix=0.0.0.0/0,via=203.0.113.1' ],
> + },
> + },
> + },
> + controllers => {
> + ids => { evpnctl => { type => "evpn", 'peers' => '192.168.0.1,192.168.0.2,192.168.0.3', asn => "65000" } },
> + },
> +
> + subnets => {
> + ids => {
> + 'myzone-10.0.0.0-24' => {
> + 'type' => 'subnet',
> + 'vnet' => 'myvnet',
> + 'gateway' => '10.0.0.1',
> + 'snat' => 1
> + },
> + 'myzone-2a08:2142:302:3::-64' => {
> + 'type' => 'subnet',
> + 'vnet' => 'myvnet2',
> + 'gateway' => '2a08:2142:302:3::1',
> + 'snat' => 1
> + },
> + }
> + }
> +}
> diff --git a/src/test/zones/simple/vrf_snat_uplink/expected_sdn_interfaces b/src/test/zones/simple/vrf_snat_uplink/expected_sdn_interfaces
> new file mode 100644
> index 000000000000..f639eb132a64
> --- /dev/null
> +++ b/src/test/zones/simple/vrf_snat_uplink/expected_sdn_interfaces
> @@ -0,0 +1,92 @@
> +#version:1
> +
> +auto vneta
> +iface vneta
> + address 10.0.0.1/24
> + post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth1 -j SNAT --to-source 203.0.113.10
> + post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth1 -j SNAT --to-source 203.0.113.10
> + post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
> + post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
> + address 10.1.0.1/24
> + post-up iptables -t nat -A POSTROUTING -s '10.1.0.0/24' -o eth1 -j SNAT --to-source 203.0.113.10
> + post-down iptables -t nat -D POSTROUTING -s '10.1.0.0/24' -o eth1 -j SNAT --to-source 203.0.113.10
> + post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
> + post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
> + address 2001:db8:a::1/64
> + post-up ip6tables -t nat -A POSTROUTING -s '2001:db8:a::/64' -o eth1 -j MASQUERADE
> + post-down ip6tables -t nat -D POSTROUTING -s '2001:db8:a::/64' -o eth1 -j MASQUERADE
> + post-up ip6tables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
> + post-down ip6tables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
> + post-up iptables -t raw -I PREROUTING -i vneta -j CT --zone-orig 64008
> + post-down iptables -t raw -D PREROUTING -i vneta -j CT --zone-orig 64008
> + post-up ip6tables -t raw -I PREROUTING -i vneta -j CT --zone-orig 64008
> + post-down ip6tables -t raw -D PREROUTING -i vneta -j CT --zone-orig 64008
> + bridge_ports none
> + bridge_stp off
> + bridge_fd 0
> + ip-forward on
> + ip6-forward on
> + vrf vrf_zonea
> +
> +auto vnetb
> +iface vnetb
> + address 10.0.0.1/24
> + post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth2 -j SNAT --to-source 198.51.100.10
> + post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth2 -j SNAT --to-source 198.51.100.10
> + post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
> + post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
> + post-up iptables -t raw -I PREROUTING -i vnetb -j CT --zone-orig 48540
> + post-down iptables -t raw -D PREROUTING -i vnetb -j CT --zone-orig 48540
> + bridge_ports none
> + bridge_stp off
> + bridge_fd 0
> + ip-forward on
> + vrf vrf_zoneb
> +
> +auto vnetc
> +iface vnetc
> + address 10.0.0.1/24
> + bridge_ports none
> + bridge_stp off
> + bridge_fd 0
> + ip-forward on
> + vrf vrf_zonec
> +
> +auto vnetd
> +iface vnetd
> + address 10.0.0.1/24
> + post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth3 -j SNAT --to-source 10.100.0.4
> + post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth3 -j SNAT --to-source 10.100.0.4
> + post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
> + post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
> + post-up iptables -t raw -I PREROUTING -i vnetd -j CT --zone-orig 27935
> + post-down iptables -t raw -D PREROUTING -i vnetd -j CT --zone-orig 27935
> + bridge_ports none
> + bridge_stp off
> + bridge_fd 0
> + ip-forward on
> + vrf vrf_zoned
> +
> +auto vrf_zonea
> +iface vrf_zonea
> + vrf-table auto
> + post-up ip route add vrf vrf_zonea unreachable default metric 4278198272
> + post-up ip -6 route add vrf vrf_zonea unreachable default metric 4278198272
> +
> +auto vrf_zoneb
> +iface vrf_zoneb
> + vrf-table auto
> + post-up ip route add vrf vrf_zoneb unreachable default metric 4278198272
> + post-up ip -6 route add vrf vrf_zoneb unreachable default metric 4278198272
> +
> +auto vrf_zonec
> +iface vrf_zonec
> + vrf-table auto
> + post-up ip route add vrf vrf_zonec unreachable default metric 4278198272
> + post-up ip -6 route add vrf vrf_zonec unreachable default metric 4278198272
> +
> +auto vrf_zoned
> +iface vrf_zoned
> + vrf-table auto
> + post-up ip route add vrf vrf_zoned unreachable default metric 4278198272
> + post-up ip -6 route add vrf vrf_zoned unreachable default metric 4278198272
> diff --git a/src/test/zones/simple/vrf_snat_uplink/interfaces b/src/test/zones/simple/vrf_snat_uplink/interfaces
> new file mode 100644
> index 000000000000..de646d3f6b43
> --- /dev/null
> +++ b/src/test/zones/simple/vrf_snat_uplink/interfaces
> @@ -0,0 +1,26 @@
> +auto vmbr0
> +iface vmbr0 inet static
> + address 192.168.0.1/24
> + gateway 192.168.0.254
> + bridge-ports eth0
> + bridge-stp off
> + bridge-fd 0
> +
> +auto eth1
> +iface eth1 inet static
> + address 203.0.113.10/24
> + vrf vrf_zonea
> +
> +iface eth1 inet6 auto
> + vrf vrf_zonea
> +
> +auto eth2
> +iface eth2 inet static
> + address 198.51.100.10/24
> + vrf vrf_zoneb
> +
> +auto eth3
> +iface eth3 inet static
> + address 10.100.0.4/24
> + gateway 10.100.0.1
> + vrf vrf_zoned
> diff --git a/src/test/zones/simple/vrf_snat_uplink/sdn_config b/src/test/zones/simple/vrf_snat_uplink/sdn_config
> new file mode 100644
> index 000000000000..7cd558dcc671
> --- /dev/null
> +++ b/src/test/zones/simple/vrf_snat_uplink/sdn_config
> @@ -0,0 +1,84 @@
> +{
> + version => 1,
> + vnets => {
> + ids => {
> + vneta => { type => "vnet", zone => "zonea" },
> + vnetb => { type => "vnet", zone => "zoneb" },
> + vnetc => { type => "vnet", zone => "zonec" },
> + vnetd => { type => "vnet", zone => "zoned" },
> + },
> + },
> + zones => {
> + ids => {
> + zonea => {
> + ipam => "pve",
> + type => "simple",
> + "dedicated-vrf" => 1,
> + route => [
> + 'prefix=0.0.0.0/0,via=203.0.113.1',
> + 'prefix=::/0,via=fe80::1,iface=eth1',
> + ],
> + },
> + zoneb => {
> + ipam => "pve",
> + type => "simple",
> + "dedicated-vrf" => 1,
> + route => [
> + 'prefix=0.0.0.0/0,via=198.51.100.2,distance=10',
> + 'prefix=0.0.0.0/0,via=198.51.100.1',
> + ],
> + },
> + zonec => {
> + ipam => "pve",
> + type => "simple",
> + "dedicated-vrf" => 1,
> + route => [ 'prefix=0.0.0.0/0,via=192.168.0.254,nexthop-vrf=default' ],
> + },
> + zoned => {
> + ipam => "pve",
> + type => "simple",
> + "dedicated-vrf" => 1,
> + },
> + },
> + },
> + subnets => {
> + ids => {
> + 'zonea-10.0.0.0-24' => {
> + type => 'subnet',
> + vnet => 'vneta',
> + gateway => '10.0.0.1',
> + snat => 1,
> + },
> + 'zonea-10.1.0.0-24' => {
> + type => 'subnet',
> + vnet => 'vneta',
> + gateway => '10.1.0.1',
> + snat => 1,
> + },
> + 'zonea-2001:db8:a::-64' => {
> + type => 'subnet',
> + vnet => 'vneta',
> + gateway => '2001:db8:a::1',
> + snat => 1,
> + },
> + 'zoneb-10.0.0.0-24' => {
> + type => 'subnet',
> + vnet => 'vnetb',
> + gateway => '10.0.0.1',
> + snat => 1,
> + },
> + 'zonec-10.0.0.0-24' => {
> + type => 'subnet',
> + vnet => 'vnetc',
> + gateway => '10.0.0.1',
> + snat => 1,
> + },
> + 'zoned-10.0.0.0-24' => {
> + type => 'subnet',
> + vnet => 'vnetd',
> + gateway => '10.0.0.1',
> + snat => 1,
> + },
> + },
> + },
> +}
next prev parent reply other threads:[~2026-11-18 18:08 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-16 16:11 [PATCH common/manager/network/proxmox{-ve-rs,-perl-rs,-widget-toolkit} v3 00/32] SDN VRF support Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-common v3 01/32] interfaces: preserve VRF membership across address family changes Gabriel Goller
2026-11-18 14:13 ` Stefan Hanreich
2026-11-18 15:41 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 02/32] frr: add VRF-aware OSPF rendering Gabriel Goller
2026-11-18 14:41 ` Stefan Hanreich
2026-11-18 16:21 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 03/32] sdn: add zone references to OSPF and BGP fabrics Gabriel Goller
2026-11-18 14:10 ` Stefan Hanreich
2026-11-18 16:55 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 04/32] sdn: generate fabric routing configuration in zone VRFs Gabriel Goller
2026-11-18 14:53 ` Stefan Hanreich
2026-11-18 17:26 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 05/32] sdn: fix VRF route-map scoping and zone ID validation Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 06/32] tests: fabrics: add test for fabrics in VRFs Gabriel Goller
2026-11-18 15:01 ` Stefan Hanreich
2026-11-18 17:33 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 07/32] sdn: allow static routes in simple zone VRFs Gabriel Goller
2026-11-18 15:29 ` Stefan Hanreich
2026-11-18 18:10 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-ve-rs v3 08/32] sdn: support static routes in EVPN " Gabriel Goller
2026-11-18 15:32 ` Stefan Hanreich
2026-11-19 13:12 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-perl-rs v3 09/32] pve-rs: fabrics: assign network interfaces to configured VRFs Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-perl-rs v3 10/32] pve-rs: fabrics: make per-fabric status queries VRF-aware Gabriel Goller
2026-11-18 15:42 ` Stefan Hanreich
2026-11-19 13:30 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-perl-rs v3 11/32] pve-rs: fabrics: include VRF routes in aggregate status Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-perl-rs v3 12/32] pve-rs: fabrics: expose host interface membership Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-perl-rs v3 13/32] sdn: expose static routes for simple zone uplinks Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 14/32] sdn: evpn: add unreachable IPv6 default route to zone VRFs Gabriel Goller
2026-11-18 16:08 ` Stefan Hanreich
2026-11-18 16:39 ` Stefan Hanreich
2026-11-19 13:38 ` Gabriel Goller
2026-11-19 15:51 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 15/32] sdn: add optional VRFs for simple zones Gabriel Goller
2026-11-18 16:18 ` Stefan Hanreich
2026-11-19 13:44 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 16/32] sdn: allow fabrics to use simple zone VRFs Gabriel Goller
2026-11-18 16:27 ` Stefan Hanreich
2026-11-19 14:14 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 17/32] sdn: require underlay fabrics to use the default VRF Gabriel Goller
2026-11-18 16:30 ` Stefan Hanreich
2026-11-19 14:33 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 18/32] sdn: always generate EVPN zone VRFs Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 19/32] api: sdn: allow EVPN zones as fabric VRFs Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 20/32] api: sdn: disallow BGP fabrics in EVPN zone VRFs Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 21/32] sdn: simple: support static routes in zone routing tables Gabriel Goller
2026-11-18 16:49 ` Stefan Hanreich
2026-11-19 14:53 ` Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 22/32] sdn: evpn: allow static routes Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 23/32] sdn: zones: SNAT via dedicated uplinks in zone VRFs Gabriel Goller
2026-11-18 18:07 ` Stefan Hanreich [this message]
2026-11-16 16:11 ` [PATCH pve-network v3 24/32] sdn: dhcp: only configure DHCP for zones on the local node Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-network v3 25/32] sdn: dhcp: run dnsmasq in the zone VRF Gabriel Goller
2026-11-16 16:11 ` [PATCH proxmox-widget-toolkit v3 26/32] ui: network: allow selecting a VRF for host interfaces Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-manager v3 27/32] ui: sdn: add VRF zone selection for fabrics Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-manager v3 28/32] ui: sdn: expose EVPN zones as fabric VRFs Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-manager v3 29/32] network: allow setting the VRF of host interfaces Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-manager v3 30/32] ui: sdn: allow static routes for Simple zone VRFs Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-manager v3 31/32] ui: sdn: allow static routes in EVPN zones Gabriel Goller
2026-11-16 16:11 ` [PATCH pve-manager v3 32/32] network: generate SDN DHCP config after ifreload Gabriel Goller
2026-11-18 18:27 ` [PATCH common/manager/network/proxmox{-ve-rs,-perl-rs,-widget-toolkit} v3 00/32] SDN VRF support Stefan Hanreich
2026-11-19 15:33 ` Gabriel Goller
2026-11-19 20:10 ` Gabriel Goller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=df54d1df-e47f-464e-8382-79526cd5acb0@proxmox.com \
--to=s.hanreich@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.