all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Stefan Hanreich <s.hanreich@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH pve-network 04/14] sdn: simple: SNAT via the uplinks of zone VRFs
Date: Thu, 19 Nov 2026 20:45:36 +0100	[thread overview]
Message-ID: <20261119194547.674512-5-s.hanreich@proxmox.com> (raw)
In-Reply-To: <20261119194547.674512-1-s.hanreich@proxmox.com>

SNAT for subnets in a zone with a dedicated VRF resolved the outgoing
interface and source address in the default routing table, so the
generated rule never matched traffic leaving through the VRF.

Add a MASQUERADE rule for every interface that the host network
configuration puts into the zone VRF. Routing picks the uplink per
connection, so this also works with several uplinks and ECMP, and
MASQUERADE needs no address when generating the configuration, which
covers uplinks configured via DHCP or SLAAC as well. Interfaces that
only become VRF members otherwise, such as fabric interfaces, are not
used for SNAT.

Without an uplink in the VRF, warn and skip SNAT for the subnet. Zones
without a dedicated VRF keep using SNAT via the default routing table.

Originally-by: Gabriel Goller <g.goller@proxmox.com>
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 src/PVE/Network/SDN/Zones/Plugin.pm           |  9 +++
 src/PVE/Network/SDN/Zones/SimplePlugin.pm     | 20 ++++-
 .../simple/vrf_snat/expected_sdn_interfaces   | 81 +++++++++++++++++++
 src/test/zones/simple/vrf_snat/interfaces     | 29 +++++++
 src/test/zones/simple/vrf_snat/sdn_config     | 53 ++++++++++++
 5 files changed, 191 insertions(+), 1 deletion(-)
 create mode 100644 src/test/zones/simple/vrf_snat/expected_sdn_interfaces
 create mode 100644 src/test/zones/simple/vrf_snat/interfaces
 create mode 100644 src/test/zones/simple/vrf_snat/sdn_config

diff --git a/src/PVE/Network/SDN/Zones/Plugin.pm b/src/PVE/Network/SDN/Zones/Plugin.pm
index 2c561ea..288edfc 100644
--- a/src/PVE/Network/SDN/Zones/Plugin.pm
+++ b/src/PVE/Network/SDN/Zones/Plugin.pm
@@ -398,6 +398,15 @@ sub get_local_route_ip {
     return ($ip, $interface);
 }
 
+# Returns the interfaces of the host network configuration that are members of $vrf, sorted by
+# name.
+sub get_vrf_member_interfaces {
+    my ($vrf, $interfaces_config) = @_;
+
+    my $ifaces = $interfaces_config->{ifaces} // {};
+    return sort grep { ($ifaces->{$_}->{vrf} // '') eq $vrf } keys %$ifaces;
+}
+
 sub find_local_ip_interface_peers {
     my ($peers, $iface) = @_;
 
diff --git a/src/PVE/Network/SDN/Zones/SimplePlugin.pm b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
index 1d44f0b..4221521 100644
--- a/src/PVE/Network/SDN/Zones/SimplePlugin.pm
+++ b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
@@ -139,7 +139,25 @@ sub generate_sdn_config {
             my $vrf = $plugin_config->{'dedicated-vrf'} ? " vrf vrf_$zone_id" : '';
             push @iface_config, "up ip route add $cidr dev $vnetid$vrf";
         }
-        if ($subnet->{snat}) {
+        if ($subnet->{snat} && $plugin_config->{'dedicated-vrf'}) {
+            my @uplinks = PVE::Network::SDN::Zones::Plugin::get_vrf_member_interfaces(
+                "vrf_$zone_id", $interfaces_config,
+            );
+            if (!@uplinks) {
+                log_warn("no uplink in vrf_$zone_id, not adding SNAT for subnet $cidr");
+                next;
+            }
+            # the routing table picks the uplink per connection (e.g. with ECMP), and its
+            # address may be dynamic, so let each uplink masquerade with its own address
+            for my $uplink (@uplinks) {
+                push @iface_config,
+                    "post-up $iptables -t nat -A POSTROUTING -s '$cidr' -o $uplink -j MASQUERADE";
+                push @iface_config,
+                    "post-down $iptables -t nat -D POSTROUTING -s '$cidr' -o $uplink -j MASQUERADE";
+            }
+            push @iface_config, "post-up $iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1";
+            push @iface_config, "post-down $iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1";
+        } elsif ($subnet->{snat}) {
             #find outgoing interface
             my ($outip, $outiface);
             eval {
diff --git a/src/test/zones/simple/vrf_snat/expected_sdn_interfaces b/src/test/zones/simple/vrf_snat/expected_sdn_interfaces
new file mode 100644
index 0000000..2a9804c
--- /dev/null
+++ b/src/test/zones/simple/vrf_snat/expected_sdn_interfaces
@@ -0,0 +1,81 @@
+#version:1
+
+auto vneta
+iface vneta
+	address 10.0.0.1/24
+	post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth1 -j MASQUERADE
+	post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth1 -j MASQUERADE
+	post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
+	post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
+	address 2001:db8:a::1/64
+	post-up ip6tables -t nat -A POSTROUTING -s '2001:db8:a::/64' -o eth1 -j MASQUERADE
+	post-down ip6tables -t nat -D POSTROUTING -s '2001:db8:a::/64' -o eth1 -j MASQUERADE
+	post-up ip6tables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
+	post-down ip6tables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
+	bridge_ports none
+	bridge_stp off
+	bridge_fd 0
+	ip-forward on
+	ip6-forward on
+	vrf vrf_zonea
+
+auto vnetb
+iface vnetb
+	address 10.0.0.1/24
+	post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth2 -j MASQUERADE
+	post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth2 -j MASQUERADE
+	post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
+	post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
+	bridge_ports none
+	bridge_stp off
+	bridge_fd 0
+	ip-forward on
+	vrf vrf_zoneb
+
+auto vnetc
+iface vnetc
+	address 10.0.0.1/24
+	bridge_ports none
+	bridge_stp off
+	bridge_fd 0
+	ip-forward on
+	vrf vrf_zonec
+
+auto vnetd
+iface vnetd
+	address 10.0.0.1/24
+	post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth3 -j MASQUERADE
+	post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth3 -j MASQUERADE
+	post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth4 -j MASQUERADE
+	post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth4 -j MASQUERADE
+	post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
+	post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
+	bridge_ports none
+	bridge_stp off
+	bridge_fd 0
+	ip-forward on
+	vrf vrf_zoned
+
+auto vrf_zonea
+iface vrf_zonea
+	vrf-table auto
+	post-up ip route add vrf vrf_zonea unreachable default metric 4278198272
+	post-up ip -6 route add vrf vrf_zonea unreachable default metric 4278198272 || true
+
+auto vrf_zoneb
+iface vrf_zoneb
+	vrf-table auto
+	post-up ip route add vrf vrf_zoneb unreachable default metric 4278198272
+	post-up ip -6 route add vrf vrf_zoneb unreachable default metric 4278198272 || true
+
+auto vrf_zonec
+iface vrf_zonec
+	vrf-table auto
+	post-up ip route add vrf vrf_zonec unreachable default metric 4278198272
+	post-up ip -6 route add vrf vrf_zonec unreachable default metric 4278198272 || true
+
+auto vrf_zoned
+iface vrf_zoned
+	vrf-table auto
+	post-up ip route add vrf vrf_zoned unreachable default metric 4278198272
+	post-up ip -6 route add vrf vrf_zoned unreachable default metric 4278198272 || true
diff --git a/src/test/zones/simple/vrf_snat/interfaces b/src/test/zones/simple/vrf_snat/interfaces
new file mode 100644
index 0000000..67ad22f
--- /dev/null
+++ b/src/test/zones/simple/vrf_snat/interfaces
@@ -0,0 +1,29 @@
+auto vmbr0
+iface vmbr0 inet static
+	address 192.168.0.1/24
+	gateway 192.168.0.254
+        bridge-ports eth0
+        bridge-stp off
+        bridge-fd 0
+
+auto eth1
+iface eth1 inet static
+	address 203.0.113.10/24
+	vrf vrf_zonea
+
+iface eth1 inet6 auto
+	vrf vrf_zonea
+
+auto eth2
+iface eth2 inet dhcp
+	vrf vrf_zoneb
+
+auto eth3
+iface eth3 inet static
+	address 198.51.100.10/29
+	vrf vrf_zoned
+
+auto eth4
+iface eth4 inet static
+	address 198.51.100.18/29
+	vrf vrf_zoned
diff --git a/src/test/zones/simple/vrf_snat/sdn_config b/src/test/zones/simple/vrf_snat/sdn_config
new file mode 100644
index 0000000..fa74493
--- /dev/null
+++ b/src/test/zones/simple/vrf_snat/sdn_config
@@ -0,0 +1,53 @@
+{
+  version => 1,
+  vnets   => {
+               ids => {
+                        vneta => { type => "vnet", zone => "zonea" },
+                        vnetb => { type => "vnet", zone => "zoneb" },
+                        vnetc => { type => "vnet", zone => "zonec" },
+                        vnetd => { type => "vnet", zone => "zoned" },
+                      },
+             },
+  zones   => {
+               ids => {
+                        zonea => { ipam => "pve", type => "simple", "dedicated-vrf" => 1 },
+                        zoneb => { ipam => "pve", type => "simple", "dedicated-vrf" => 1 },
+                        zonec => { ipam => "pve", type => "simple", "dedicated-vrf" => 1 },
+                        zoned => { ipam => "pve", type => "simple", "dedicated-vrf" => 1 },
+                      },
+             },
+  subnets => {
+               ids => {
+                        'zonea-10.0.0.0-24' => {
+                                                 type => 'subnet',
+                                                 vnet => 'vneta',
+                                                 gateway => '10.0.0.1',
+                                                 snat => 1,
+                                               },
+                        'zonea-2001:db8:a::-64' => {
+                                                     type => 'subnet',
+                                                     vnet => 'vneta',
+                                                     gateway => '2001:db8:a::1',
+                                                     snat => 1,
+                                                   },
+                        'zoneb-10.0.0.0-24' => {
+                                                 type => 'subnet',
+                                                 vnet => 'vnetb',
+                                                 gateway => '10.0.0.1',
+                                                 snat => 1,
+                                               },
+                        'zonec-10.0.0.0-24' => {
+                                                 type => 'subnet',
+                                                 vnet => 'vnetc',
+                                                 gateway => '10.0.0.1',
+                                                 snat => 1,
+                                               },
+                        'zoned-10.0.0.0-24' => {
+                                                 type => 'subnet',
+                                                 vnet => 'vnetd',
+                                                 gateway => '10.0.0.1',
+                                                 snat => 1,
+                                               },
+                      },
+             },
+}
-- 
2.47.3





  parent reply	other threads:[~2026-11-19 19:47 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox 01/14] network-api: expose the master device of a link Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-ve-rs 02/14] sdn: config: read the conntrack zone of zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-ve-rs 03/14] sdn: conntrack: map the interfaces of zone VRFs to their conntrack zone Stefan Hanreich
2026-11-19 19:45 ` Stefan Hanreich [this message]
2026-11-19 19:45 ` [PATCH pve-network 05/14] sdn: zones: assign a conntrack zone to zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 06/14] firewall: apply vnet rules to host traffic from vnets in a VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 07/14] nftables: expression: add a constructor for map lookups Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 08/14] nftables: statement: add a helper for setting the conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 09/14] nftables: expression: convert the conntrack zones of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 10/14] firewall: add a command applying " Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 11/14] firewall: put guest traffic of zone VRFs into their conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-manager 12/14] network: apply the conntrack zones of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-manager 13/14] network: warn if pve-firewall cannot isolate guests in " Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-docs 14/14] pvesdn: simple zones: document dedicated VRFs Stefan Hanreich

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119194547.674512-5-s.hanreich@proxmox.com \
    --to=s.hanreich@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal