all lists on lists.proxmox.com
 help / color / mirror / Atom feed
* [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF
@ 2026-11-19 19:45 Stefan Hanreich
  2026-11-19 19:45 ` [PATCH proxmox 01/14] network-api: expose the master device of a link Stefan Hanreich
                   ` (13 more replies)
  0 siblings, 14 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

The subnets of simple zones with a dedicated VRF may overlap, but all
VRFs share conntrack zone 0. A connection in one VRF could then match
the conntrack entry of an identical connection in another VRF, so the
stateful firewall accepts a new connection as established. Conntrack
state migration also overwrites the entries of other guests when
loading entries with identical tuples on the target node. NAT stores
its mappings in the same conntrack entries, so it is affected as well.

This series gives every simple zone with a dedicated VRF its own
conntrack zone and makes SNAT work through the uplinks of the zone VRF.

Conntrack zones:

* The conntrack zones 16384 to 32767 (0x4000 to 0x7fff) are considered
  reserved for Proxmox VE from now on. These are exactly the zones whose
  two most significant bits are 01, so the whole area is an aligned
  block that can be matched with a single bitmask
  (ct zone & 0xc000 == 0x4000). It should stay clear of the lower zone
  ids that custom rules typically use, including zone 1, which the SNAT
  rules of pve-network use for the firewall bridges of pve-firewall.
  This does not affect existing users until they actually utilize the
  new feature.

* Each simple zone with a dedicated VRF gets a conntrack zone from the
  first 1024 zones of this area (16384 to 17407). It is stored with the
  zone as ct-zone, assigned automatically by the zone API in
  pve-network, and stays the same as long as the zone has a dedicated
  VRF, so it is the same on all nodes. This ensures that conntrack
  state migration works inside the cluster. Remote migrations do not
  support conntrack state migration.

* The zone is set at the following hook points:

  Always:

  - inet PREROUTING (for traffic arriving at an interface in the VRF)
  - inet OUTPUT (for host traffic originated in the VRF)

  Only while proxmox-firewall is active:

  - bridge PREROUTING (for guest traffic)

  The chains run at priority raw - 10, before conntrack and before the
  iptables raw table, so the zone takes effect independent of the order
  in which rules are loaded. It doesn't affect the existing pve-network
  rules that set CT zone 1 for the firewall bridges of pve-firewall,
  because the firewall bridges are never members of a VRF.

* pve-manager runs the proxmox-firewall command after ifreload when
  applying the network configuration and in pve-sdn-commit at boot,
  before starting guests.

With pve-firewall, connections of guests with the firewall enabled on
a network device are tracked on their firewall bridge in a conntrack
zone shared by all VRFs, so they are not isolated. Applying the
network configuration warns about this.

SNAT:

Subnets with SNAT enabled are masquerading outgoing traffic on every
interface that /etc/network/interfaces puts into the zone VRF, i.e. the
uplinks of the zone, with one rule per uplink.

The routing table of the VRF still decides which uplink a connection
leaves through, e.g. via the gateway of an uplink, which ifupdown2
installs into the VRF routing table, or a default route of the zone.
So this works with several uplinks and ECMP, and with uplinks
configured via DHCP or SLAAC, as no address is needed when generating
the configuration.

Neither vnets nor fabric interfaces are uplinks. This means that
traffic between the vnets of a zone and routed traffic over fabrics
keeps the guest addresses.

Without an uplink in the VRF, SNAT is skipped with a warning. Zones
without a dedicated VRF and EVPN zones are unchanged. As NAT uses the
conntrack zones of the VRFs, SNAT needs no conntrack zones of its own.

The host-bridge-input chain in proxmox-firewall now additionally looks
up the slave device when inspecting traffic. This allows also matching
traffic originating from a VRF, as the bridges there are enslaved to
VRFs and the input interface is the VRF interface. For bridges inside
VRFs we need to match on the enslaved interface instead.

Dependencies:

This series depends on v3 of Gabriel's VRF series [1].

proxmox-firewall needs proxmox-network-api and proxmox-ve-config
pve-manager needs proxmox-firewall and pve-network

[1] https://lore.proxmox.com/pve-devel/20261119194116.575520-1-g.goller@proxmox.com/


proxmox:

Stefan Hanreich (1):
  network-api: expose the master device of a link

 proxmox-network-api/src/config/helper.rs | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)


proxmox-ve-rs:

Stefan Hanreich (2):
  sdn: config: read the conntrack zone of zones with a dedicated VRF
  sdn: conntrack: map the interfaces of zone VRFs to their conntrack
    zone

 proxmox-ve-config/src/sdn/config.rs    |  38 ++++-
 proxmox-ve-config/src/sdn/conntrack.rs | 225 +++++++++++++++++++++++++
 proxmox-ve-config/src/sdn/mod.rs       |   1 +
 proxmox-ve-config/tests/sdn/main.rs    |  47 ++++++
 4 files changed, 304 insertions(+), 7 deletions(-)
 create mode 100644 proxmox-ve-config/src/sdn/conntrack.rs


pve-network:

Stefan Hanreich (2):
  sdn: simple: SNAT via the uplinks of zone VRFs
  sdn: zones: assign a conntrack zone to zones with a dedicated VRF

 src/PVE/API2/Network/SDN/Zones.pm             |  13 +++
 src/PVE/Network/SDN/Zones.pm                  |  42 +++++++
 src/PVE/Network/SDN/Zones/Plugin.pm           |  13 +++
 src/PVE/Network/SDN/Zones/SimplePlugin.pm     |  27 ++++-
 src/test/Makefile                             |   5 +-
 src/test/run_test_ct_zones.pl                 | 103 ++++++++++++++++++
 .../simple/vrf_snat/expected_sdn_interfaces   |  81 ++++++++++++++
 src/test/zones/simple/vrf_snat/interfaces     |  29 +++++
 src/test/zones/simple/vrf_snat/sdn_config     |  53 +++++++++
 9 files changed, 364 insertions(+), 2 deletions(-)
 create mode 100755 src/test/run_test_ct_zones.pl
 create mode 100644 src/test/zones/simple/vrf_snat/expected_sdn_interfaces
 create mode 100644 src/test/zones/simple/vrf_snat/interfaces
 create mode 100644 src/test/zones/simple/vrf_snat/sdn_config


proxmox-firewall:

Stefan Hanreich (6):
  firewall: apply vnet rules to host traffic from vnets in a VRF
  nftables: expression: add a constructor for map lookups
  nftables: statement: add a helper for setting the conntrack zone
  nftables: expression: convert the conntrack zones of zone VRFs
  firewall: add a command applying the conntrack zones of zone VRFs
  firewall: put guest traffic of zone VRFs into their conntrack zone

 .../resources/proxmox-firewall.nft            |  16 ++
 proxmox-firewall/src/bin/proxmox-firewall.rs  |  30 ++-
 proxmox-firewall/src/config.rs                |  23 +-
 proxmox-firewall/src/firewall.rs              |  30 +++
 proxmox-firewall/src/lib.rs                   |   1 +
 proxmox-firewall/src/sdn.rs                   | 235 ++++++++++++++++++
 .../tests/input/.running-config.json          |   4 +-
 .../integration_tests__firewall.snap          |  24 ++
 proxmox-nftables/src/expression.rs            |  69 +++++
 proxmox-nftables/src/statement.rs             |   7 +
 10 files changed, 430 insertions(+), 9 deletions(-)
 create mode 100644 proxmox-firewall/src/sdn.rs


pve-manager:

Stefan Hanreich (2):
  network: apply the conntrack zones of zone VRFs
  network: warn if pve-firewall cannot isolate guests in zone VRFs

 PVE/API2/Network.pm             | 35 +++++++++++++++++++++++++++++++++
 bin/pve-sdn-commit              | 20 +++++++++++++++++++
 services/pve-sdn-commit.service |  2 ++
 3 files changed, 57 insertions(+)


pve-docs:

Stefan Hanreich (1):
  pvesdn: simple zones: document dedicated VRFs

 pvesdn.adoc | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 100 insertions(+)


Summary over all repositories:
  28 files changed, 1265 insertions(+), 20 deletions(-)

-- 
Generated by murpp 0.12.0




^ permalink raw reply	[flat|nested] 15+ messages in thread

* [PATCH proxmox 01/14] network-api: expose the master device of a link
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  2026-11-19 19:45 ` [PATCH proxmox-ve-rs 02/14] sdn: config: read the conntrack zone of zones with a dedicated VRF Stefan Hanreich
                   ` (12 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

The master of a link, as reported by iproute2, tells which bridge, bond
or VRF an interface is enslaved to. Initially this will be used by the
SDN stack to determine if an interface is the member of a VRF.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 proxmox-network-api/src/config/helper.rs | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/proxmox-network-api/src/config/helper.rs b/proxmox-network-api/src/config/helper.rs
index a9ae35fc..e33c8e3d 100644
--- a/proxmox-network-api/src/config/helper.rs
+++ b/proxmox-network-api/src/config/helper.rs
@@ -177,6 +177,7 @@ pub struct IpLink {
     link_type: Link,
     linkinfo: Option<LinkInfo>,
     operstate: String,
+    master: Option<String>,
 }
 
 impl IpLink {
@@ -240,6 +241,11 @@ impl IpLink {
     pub fn active(&self) -> bool {
         self.operstate == "UP"
     }
+
+    /// The name of the device this interface is enslaved to, e.g. a bridge, bond or VRF.
+    pub fn master(&self) -> Option<&str> {
+        self.master.as_deref()
+    }
 }
 
 /// A mapping of altnames to the interfaces' ifname.
@@ -380,7 +386,8 @@ mod tests {
   "gro_ipv4_max_size": 65536
 }"#;
 
-        serde_json::from_str::<IpLink>(interface).unwrap();
+        let link = serde_json::from_str::<IpLink>(interface).unwrap();
+        assert_eq!(link.master(), None);
     }
 
     #[test]
@@ -464,7 +471,8 @@ mod tests {
     ]
 }"#;
 
-        serde_json::from_str::<IpLink>(interface).unwrap();
+        let link = serde_json::from_str::<IpLink>(interface).unwrap();
+        assert_eq!(link.master(), Some("vmbr0"));
     }
 
     #[test]
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH proxmox-ve-rs 02/14] sdn: config: read the conntrack zone of zones with a dedicated VRF
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
  2026-11-19 19:45 ` [PATCH proxmox 01/14] network-api: expose the master device of a link Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  2026-11-19 19:45 ` [PATCH proxmox-ve-rs 03/14] sdn: conntrack: map the interfaces of zone VRFs to their conntrack zone Stefan Hanreich
                   ` (11 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

Proxmox VE reserves the conntrack zones 16384 to 32767 (0x4000 to
0x7fff). These are all zones whose two most significant bits are 01, so
the whole block can be matched with a single bitmask
(ct zone & 0xc000 == 0x4000). Zones with a dedicated VRF get their
conntrack zone from the first 1024 zones of this block, 16384 to 17407,
which leaves the rest of the block for future use.

Add the VrfCtZone type, which only accepts conntrack zones from that
range, and read the ct-zone property of zones from the running config.
A conntrack zone out of range is ignored with a warning, so that it
does not prevent reading the rest of the SDN configuration.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 proxmox-ve-config/src/sdn/config.rs    | 38 ++++++++++++---
 proxmox-ve-config/src/sdn/conntrack.rs | 65 ++++++++++++++++++++++++++
 proxmox-ve-config/src/sdn/mod.rs       |  1 +
 proxmox-ve-config/tests/sdn/main.rs    | 34 ++++++++++++++
 4 files changed, 131 insertions(+), 7 deletions(-)
 create mode 100644 proxmox-ve-config/src/sdn/conntrack.rs

diff --git a/proxmox-ve-config/src/sdn/config.rs b/proxmox-ve-config/src/sdn/config.rs
index 55df9dd2..db61094e 100644
--- a/proxmox-ve-config/src/sdn/config.rs
+++ b/proxmox-ve-config/src/sdn/config.rs
@@ -17,7 +17,7 @@ use crate::{
         Ipset,
         ipset::{IpsetEntry, IpsetName, IpsetScope},
     },
-    sdn::{SdnNameError, SubnetName, VnetName, ZoneName},
+    sdn::{SdnNameError, SubnetName, VnetName, ZoneName, conntrack::VrfCtZone},
 };
 
 #[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
@@ -142,6 +142,12 @@ pub struct ZoneRunningConfig {
     #[serde(rename = "type")]
     ty: ZoneType,
     dhcp: Option<DhcpType>,
+    #[serde(
+        default,
+        rename = "ct-zone",
+        deserialize_with = "proxmox_serde::perl::deserialize_u16"
+    )]
+    ct_zone: Option<u16>,
 }
 
 /// Struct for deserializing the zones of the SDN running config
@@ -391,6 +397,8 @@ pub struct ZoneConfig {
     name: ZoneName,
     ty: ZoneType,
     vnets: BTreeMap<VnetName, VnetConfig>,
+    #[serde(default)]
+    ct_zone: Option<VrfCtZone>,
 }
 
 impl ZoneConfig {
@@ -399,6 +407,7 @@ impl ZoneConfig {
             name,
             ty,
             vnets: BTreeMap::default(),
+            ct_zone: None,
         }
     }
 
@@ -445,6 +454,15 @@ impl ZoneConfig {
     pub fn ty(&self) -> &ZoneType {
         &self.ty
     }
+
+    /// The conntrack zone of a zone with a dedicated VRF.
+    pub fn ct_zone(&self) -> Option<VrfCtZone> {
+        self.ct_zone
+    }
+
+    pub fn set_ct_zone(&mut self, ct_zone: Option<VrfCtZone>) {
+        self.ct_zone = ct_zone;
+    }
 }
 
 /// Representation of a Proxmox VE SDN configuration
@@ -649,12 +667,18 @@ impl TryFrom<RunningConfig> for SdnConfig {
         let mut config = SdnConfig::default();
 
         if let Some(running_zones) = value.zones.take() {
-            config.add_zones(
-                running_zones
-                    .ids
-                    .into_iter()
-                    .map(|(name, running_config)| ZoneConfig::new(name, running_config.ty)),
-            )?;
+            config.add_zones(running_zones.ids.into_iter().map(|(name, running_config)| {
+                // an invalid conntrack zone must not prevent reading the rest of the configuration
+                let ct_zone = running_config.ct_zone.and_then(|ct_zone| {
+                    VrfCtZone::try_from(ct_zone)
+                        .inspect_err(|err| log::warn!("zone {name}: ignoring the {err}"))
+                        .ok()
+                });
+
+                let mut zone = ZoneConfig::new(name, running_config.ty);
+                zone.set_ct_zone(ct_zone);
+                zone
+            }))?;
         }
 
         if let Some(running_vnets) = value.vnets.take() {
diff --git a/proxmox-ve-config/src/sdn/conntrack.rs b/proxmox-ve-config/src/sdn/conntrack.rs
new file mode 100644
index 00000000..ff363646
--- /dev/null
+++ b/proxmox-ve-config/src/sdn/conntrack.rs
@@ -0,0 +1,65 @@
+//! Conntrack zones of SDN zones.
+//!
+//! Proxmox VE reserves the conntrack zones 16384 to 32767 (0x4000 to 0x7fff). These are all zones
+//! whose two most significant bits are 01, so the whole block can be matched with a single bitmask
+//! (`ct zone & 0xc000 == 0x4000`). Zones with a dedicated VRF get their conntrack zone from the
+//! first 1024 zones of this block.
+
+use serde::Deserialize;
+
+/// The conntrack zone of an SDN zone with a dedicated VRF.
+#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, Deserialize)]
+#[serde(try_from = "u16")]
+pub struct VrfCtZone(u16);
+
+impl VrfCtZone {
+    /// The first conntrack zone for zones with a dedicated VRF, the start of the reserved block.
+    pub const FIRST: u16 = 16384;
+
+    /// The number of conntrack zones for zones with a dedicated VRF.
+    pub const COUNT: u16 = 1024;
+
+    /// Returns `None` if `ct_zone` is not in the range for zones with a dedicated VRF.
+    pub fn new(ct_zone: u16) -> Option<Self> {
+        (Self::FIRST..Self::FIRST + Self::COUNT)
+            .contains(&ct_zone)
+            .then_some(Self(ct_zone))
+    }
+
+    /// The conntrack zone ID.
+    pub fn id(self) -> u16 {
+        self.0
+    }
+
+    /// The position of the conntrack zone within the range for zones with a dedicated VRF.
+    pub fn index(self) -> u16 {
+        self.0 - Self::FIRST
+    }
+}
+
+impl TryFrom<u16> for VrfCtZone {
+    type Error = String;
+
+    fn try_from(ct_zone: u16) -> Result<Self, Self::Error> {
+        Self::new(ct_zone).ok_or_else(|| {
+            format!(
+                "conntrack zone {ct_zone} is out of range ({} - {})",
+                Self::FIRST,
+                Self::FIRST + Self::COUNT - 1
+            )
+        })
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn vrf_ct_zone_range() {
+        assert_eq!(VrfCtZone::new(16384).map(VrfCtZone::index), Some(0));
+        assert_eq!(VrfCtZone::new(17407).map(VrfCtZone::index), Some(1023));
+        assert_eq!(VrfCtZone::new(16383), None);
+        assert_eq!(VrfCtZone::new(17408), None);
+    }
+}
diff --git a/proxmox-ve-config/src/sdn/mod.rs b/proxmox-ve-config/src/sdn/mod.rs
index 88987cd5..3b96ec18 100644
--- a/proxmox-ve-config/src/sdn/mod.rs
+++ b/proxmox-ve-config/src/sdn/mod.rs
@@ -1,4 +1,5 @@
 pub mod config;
+pub mod conntrack;
 pub mod fabric;
 pub mod ipam;
 pub mod prefix_list;
diff --git a/proxmox-ve-config/tests/sdn/main.rs b/proxmox-ve-config/tests/sdn/main.rs
index 7d29d69f..ad982bd1 100644
--- a/proxmox-ve-config/tests/sdn/main.rs
+++ b/proxmox-ve-config/tests/sdn/main.rs
@@ -11,6 +11,7 @@ use proxmox_ve_config::sdn::{
     config::{
         RunningConfig, SdnConfig, SdnConfigError, SubnetConfig, VnetConfig, ZoneConfig, ZoneType,
     },
+    conntrack::VrfCtZone,
     ipam::{Ipam, IpamDataVm, IpamEntry, IpamJson},
 };
 
@@ -137,6 +138,39 @@ fn parse_running_config_from_newer_pve_network() {
     assert_eq!(subnet_v6.gateway(), None);
 }
 
+#[test]
+fn parse_running_config_ct_zone() {
+    // perl may store the zone as string, a zone out of range is ignored
+    let running_config: RunningConfig = serde_json::from_value(serde_json::json!({
+        "zones": {
+            "ids": {
+                "zone0": { "type": "simple", "dedicated-vrf": 1, "ct-zone": 16384 },
+                "zone1": { "type": "simple", "dedicated-vrf": 1, "ct-zone": "17407" },
+                "zone2": { "type": "simple", "dedicated-vrf": 1, "ct-zone": 17408 },
+                "zone3": { "type": "simple", "dedicated-vrf": 1, "ct-zone": 5 },
+                "zone4": { "type": "simple" },
+            },
+        },
+    }))
+    .expect("valid running config");
+
+    let sdn_config = SdnConfig::try_from(running_config).expect("valid SDN config");
+
+    let ct_zone = |zone: &str| {
+        sdn_config
+            .zone(&ZoneName::from_str(zone).unwrap())
+            .expect("zone exists")
+            .ct_zone()
+            .map(VrfCtZone::id)
+    };
+
+    assert_eq!(ct_zone("zone0"), Some(16384));
+    assert_eq!(ct_zone("zone1"), Some(17407));
+    assert_eq!(ct_zone("zone2"), None);
+    assert_eq!(ct_zone("zone3"), None);
+    assert_eq!(ct_zone("zone4"), None);
+}
+
 #[test]
 fn sdn_config() {
     let mut sdn_config = SdnConfig::new();
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH proxmox-ve-rs 03/14] sdn: conntrack: map the interfaces of zone VRFs to their conntrack zone
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
  2026-11-19 19:45 ` [PATCH proxmox 01/14] network-api: expose the master device of a link Stefan Hanreich
  2026-11-19 19:45 ` [PATCH proxmox-ve-rs 02/14] sdn: config: read the conntrack zone of zones with a dedicated VRF Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  2026-11-19 19:45 ` [PATCH pve-network 04/14] sdn: simple: SNAT via the uplinks of zone VRFs Stefan Hanreich
                   ` (10 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

Add VrfCtZones, which maps the VRF device of each zone with a conntrack
zone to the interfaces in that VRF. Interfaces are added by the name of
their VRF device, so the mapping can be filled from the kernel, which
reports the master of every interface. The kernel state is what
forwards the traffic once the network configuration is applied, and it
covers every way an interface ends up in a VRF (vnets, fabrics,
/etc/network/interfaces) without collecting them from the
configuration.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 proxmox-ve-config/src/sdn/conntrack.rs | 160 +++++++++++++++++++++++++
 proxmox-ve-config/tests/sdn/main.rs    |  15 ++-
 2 files changed, 174 insertions(+), 1 deletion(-)

diff --git a/proxmox-ve-config/src/sdn/conntrack.rs b/proxmox-ve-config/src/sdn/conntrack.rs
index ff363646..82f93de0 100644
--- a/proxmox-ve-config/src/sdn/conntrack.rs
+++ b/proxmox-ve-config/src/sdn/conntrack.rs
@@ -5,8 +5,12 @@
 //! (`ct zone & 0xc000 == 0x4000`). Zones with a dedicated VRF get their conntrack zone from the
 //! first 1024 zones of this block.
 
+use std::collections::{BTreeMap, BTreeSet};
+
 use serde::Deserialize;
 
+use crate::sdn::config::{SdnConfig, ZoneConfig};
+
 /// The conntrack zone of an SDN zone with a dedicated VRF.
 #[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, Deserialize)]
 #[serde(try_from = "u16")]
@@ -51,6 +55,98 @@ impl TryFrom<u16> for VrfCtZone {
     }
 }
 
+/// A zone VRF with a conntrack zone and the interfaces enslaved to it.
+#[derive(Clone, Debug, PartialEq, Eq)]
+pub struct VrfCtZoneEntry {
+    ct_zone: VrfCtZone,
+    interfaces: BTreeSet<String>,
+}
+
+impl VrfCtZoneEntry {
+    /// The conntrack zone of the VRF.
+    pub fn ct_zone(&self) -> VrfCtZone {
+        self.ct_zone
+    }
+
+    /// The interfaces enslaved to the VRF, not including the VRF device itself.
+    pub fn interfaces(&self) -> impl Iterator<Item = &str> + '_ {
+        self.interfaces.iter().map(String::as_str)
+    }
+}
+
+/// The zone VRFs with a conntrack zone, keyed by the name of their VRF device.
+#[derive(Clone, Debug, Default, PartialEq, Eq)]
+pub struct VrfCtZones {
+    vrfs: BTreeMap<String, VrfCtZoneEntry>,
+}
+
+impl VrfCtZones {
+    /// The name of the VRF device of a zone with a dedicated VRF.
+    pub fn vrf_name(zone: &ZoneConfig) -> String {
+        format!("vrf_{}", zone.name())
+    }
+
+    /// Adds the VRF of a zone, if the zone has a conntrack zone.
+    pub fn add_zone(&mut self, zone: &ZoneConfig) {
+        if let Some(ct_zone) = zone.ct_zone() {
+            self.vrfs.insert(
+                Self::vrf_name(zone),
+                VrfCtZoneEntry {
+                    ct_zone,
+                    interfaces: BTreeSet::new(),
+                },
+            );
+        }
+    }
+
+    /// Adds an interface to the VRF device `vrf`.
+    ///
+    /// Returns `false` and ignores the interface if `vrf` is not the VRF of a zone with a
+    /// conntrack zone.
+    pub fn add_interface(&mut self, vrf: &str, interface: impl Into<String>) -> bool {
+        match self.vrfs.get_mut(vrf) {
+            Some(entry) => {
+                entry.interfaces.insert(interface.into());
+                true
+            }
+            None => false,
+        }
+    }
+
+    pub fn is_empty(&self) -> bool {
+        self.vrfs.is_empty()
+    }
+
+    /// Iterates over the VRF devices and their entries.
+    pub fn iter(&self) -> impl Iterator<Item = (&str, &VrfCtZoneEntry)> + '_ {
+        self.vrfs.iter().map(|(vrf, entry)| (vrf.as_str(), entry))
+    }
+
+    /// Iterates over all interfaces whose traffic belongs to a conntrack zone.
+    ///
+    /// These are the VRF devices themselves, as packets are looked up again when passing them, and
+    /// the interfaces enslaved to them.
+    pub fn interfaces(&self) -> impl Iterator<Item = (&str, VrfCtZone)> + '_ {
+        self.iter().flat_map(|(vrf, entry)| {
+            std::iter::once(vrf)
+                .chain(entry.interfaces())
+                .map(|interface| (interface, entry.ct_zone()))
+        })
+    }
+}
+
+impl From<&SdnConfig> for VrfCtZones {
+    fn from(config: &SdnConfig) -> Self {
+        let mut ct_zones = Self::default();
+
+        for zone in config.zones() {
+            ct_zones.add_zone(zone);
+        }
+
+        ct_zones
+    }
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;
@@ -62,4 +158,68 @@ mod tests {
         assert_eq!(VrfCtZone::new(16383), None);
         assert_eq!(VrfCtZone::new(17408), None);
     }
+
+    #[test]
+    fn vrf_ct_zones() {
+        use crate::sdn::ZoneName;
+        use crate::sdn::config::ZoneType;
+
+        let zone = |name: &str, ct_zone: Option<u16>| {
+            let mut zone =
+                ZoneConfig::new(ZoneName::new(name.to_string()).unwrap(), ZoneType::Simple);
+            zone.set_ct_zone(ct_zone.and_then(VrfCtZone::new));
+            zone
+        };
+
+        let mut ct_zones = VrfCtZones::default();
+        assert!(ct_zones.is_empty());
+
+        ct_zones.add_zone(&zone("red", Some(16384)));
+        ct_zones.add_zone(&zone("blue", Some(16389)));
+        ct_zones.add_zone(&zone("green", None));
+
+        assert!(ct_zones.add_interface("vrf_red", "vnetr1"));
+        assert!(ct_zones.add_interface("vrf_red", "bond0.101"));
+        assert!(ct_zones.add_interface("vrf_blue", "vnetb"));
+        // VRFs that are not zone VRFs with a conntrack zone
+        assert!(!ct_zones.add_interface("vrf_green", "vnetg"));
+        assert!(!ct_zones.add_interface("vrf_mgmt", "eth0"));
+        // the prefix is part of the name
+        assert!(!ct_zones.add_interface("red", "eth1"));
+
+        let entries: Vec<_> = ct_zones
+            .iter()
+            .map(|(vrf, entry)| {
+                (
+                    vrf,
+                    entry.ct_zone().id(),
+                    entry.interfaces().collect::<Vec<_>>(),
+                )
+            })
+            .collect();
+
+        assert_eq!(
+            entries,
+            [
+                ("vrf_blue", 16389, vec!["vnetb"]),
+                ("vrf_red", 16384, vec!["bond0.101", "vnetr1"]),
+            ]
+        );
+
+        let interfaces: Vec<_> = ct_zones
+            .interfaces()
+            .map(|(interface, ct_zone)| (interface, ct_zone.id()))
+            .collect();
+
+        assert_eq!(
+            interfaces,
+            [
+                ("vrf_blue", 16389),
+                ("vnetb", 16389),
+                ("vrf_red", 16384),
+                ("bond0.101", 16384),
+                ("vnetr1", 16384),
+            ]
+        );
+    }
 }
diff --git a/proxmox-ve-config/tests/sdn/main.rs b/proxmox-ve-config/tests/sdn/main.rs
index ad982bd1..76ef25f2 100644
--- a/proxmox-ve-config/tests/sdn/main.rs
+++ b/proxmox-ve-config/tests/sdn/main.rs
@@ -11,7 +11,7 @@ use proxmox_ve_config::sdn::{
     config::{
         RunningConfig, SdnConfig, SdnConfigError, SubnetConfig, VnetConfig, ZoneConfig, ZoneType,
     },
-    conntrack::VrfCtZone,
+    conntrack::{VrfCtZone, VrfCtZones},
     ipam::{Ipam, IpamDataVm, IpamEntry, IpamJson},
 };
 
@@ -169,6 +169,19 @@ fn parse_running_config_ct_zone() {
     assert_eq!(ct_zone("zone2"), None);
     assert_eq!(ct_zone("zone3"), None);
     assert_eq!(ct_zone("zone4"), None);
+
+    let vrfs: Vec<_> = VrfCtZones::from(&sdn_config)
+        .iter()
+        .map(|(vrf, entry)| (vrf.to_string(), entry.ct_zone().id()))
+        .collect();
+
+    assert_eq!(
+        vrfs,
+        [
+            ("vrf_zone0".to_string(), 16384),
+            ("vrf_zone1".to_string(), 17407)
+        ]
+    );
 }
 
 #[test]
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH pve-network 04/14] sdn: simple: SNAT via the uplinks of zone VRFs
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
                   ` (2 preceding siblings ...)
  2026-11-19 19:45 ` [PATCH proxmox-ve-rs 03/14] sdn: conntrack: map the interfaces of zone VRFs to their conntrack zone Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  2026-11-19 19:45 ` [PATCH pve-network 05/14] sdn: zones: assign a conntrack zone to zones with a dedicated VRF Stefan Hanreich
                   ` (9 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

SNAT for subnets in a zone with a dedicated VRF resolved the outgoing
interface and source address in the default routing table, so the
generated rule never matched traffic leaving through the VRF.

Add a MASQUERADE rule for every interface that the host network
configuration puts into the zone VRF. Routing picks the uplink per
connection, so this also works with several uplinks and ECMP, and
MASQUERADE needs no address when generating the configuration, which
covers uplinks configured via DHCP or SLAAC as well. Interfaces that
only become VRF members otherwise, such as fabric interfaces, are not
used for SNAT.

Without an uplink in the VRF, warn and skip SNAT for the subnet. Zones
without a dedicated VRF keep using SNAT via the default routing table.

Originally-by: Gabriel Goller <g.goller@proxmox.com>
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 src/PVE/Network/SDN/Zones/Plugin.pm           |  9 +++
 src/PVE/Network/SDN/Zones/SimplePlugin.pm     | 20 ++++-
 .../simple/vrf_snat/expected_sdn_interfaces   | 81 +++++++++++++++++++
 src/test/zones/simple/vrf_snat/interfaces     | 29 +++++++
 src/test/zones/simple/vrf_snat/sdn_config     | 53 ++++++++++++
 5 files changed, 191 insertions(+), 1 deletion(-)
 create mode 100644 src/test/zones/simple/vrf_snat/expected_sdn_interfaces
 create mode 100644 src/test/zones/simple/vrf_snat/interfaces
 create mode 100644 src/test/zones/simple/vrf_snat/sdn_config

diff --git a/src/PVE/Network/SDN/Zones/Plugin.pm b/src/PVE/Network/SDN/Zones/Plugin.pm
index 2c561ea..288edfc 100644
--- a/src/PVE/Network/SDN/Zones/Plugin.pm
+++ b/src/PVE/Network/SDN/Zones/Plugin.pm
@@ -398,6 +398,15 @@ sub get_local_route_ip {
     return ($ip, $interface);
 }
 
+# Returns the interfaces of the host network configuration that are members of $vrf, sorted by
+# name.
+sub get_vrf_member_interfaces {
+    my ($vrf, $interfaces_config) = @_;
+
+    my $ifaces = $interfaces_config->{ifaces} // {};
+    return sort grep { ($ifaces->{$_}->{vrf} // '') eq $vrf } keys %$ifaces;
+}
+
 sub find_local_ip_interface_peers {
     my ($peers, $iface) = @_;
 
diff --git a/src/PVE/Network/SDN/Zones/SimplePlugin.pm b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
index 1d44f0b..4221521 100644
--- a/src/PVE/Network/SDN/Zones/SimplePlugin.pm
+++ b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
@@ -139,7 +139,25 @@ sub generate_sdn_config {
             my $vrf = $plugin_config->{'dedicated-vrf'} ? " vrf vrf_$zone_id" : '';
             push @iface_config, "up ip route add $cidr dev $vnetid$vrf";
         }
-        if ($subnet->{snat}) {
+        if ($subnet->{snat} && $plugin_config->{'dedicated-vrf'}) {
+            my @uplinks = PVE::Network::SDN::Zones::Plugin::get_vrf_member_interfaces(
+                "vrf_$zone_id", $interfaces_config,
+            );
+            if (!@uplinks) {
+                log_warn("no uplink in vrf_$zone_id, not adding SNAT for subnet $cidr");
+                next;
+            }
+            # the routing table picks the uplink per connection (e.g. with ECMP), and its
+            # address may be dynamic, so let each uplink masquerade with its own address
+            for my $uplink (@uplinks) {
+                push @iface_config,
+                    "post-up $iptables -t nat -A POSTROUTING -s '$cidr' -o $uplink -j MASQUERADE";
+                push @iface_config,
+                    "post-down $iptables -t nat -D POSTROUTING -s '$cidr' -o $uplink -j MASQUERADE";
+            }
+            push @iface_config, "post-up $iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1";
+            push @iface_config, "post-down $iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1";
+        } elsif ($subnet->{snat}) {
             #find outgoing interface
             my ($outip, $outiface);
             eval {
diff --git a/src/test/zones/simple/vrf_snat/expected_sdn_interfaces b/src/test/zones/simple/vrf_snat/expected_sdn_interfaces
new file mode 100644
index 0000000..2a9804c
--- /dev/null
+++ b/src/test/zones/simple/vrf_snat/expected_sdn_interfaces
@@ -0,0 +1,81 @@
+#version:1
+
+auto vneta
+iface vneta
+	address 10.0.0.1/24
+	post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth1 -j MASQUERADE
+	post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth1 -j MASQUERADE
+	post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
+	post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
+	address 2001:db8:a::1/64
+	post-up ip6tables -t nat -A POSTROUTING -s '2001:db8:a::/64' -o eth1 -j MASQUERADE
+	post-down ip6tables -t nat -D POSTROUTING -s '2001:db8:a::/64' -o eth1 -j MASQUERADE
+	post-up ip6tables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
+	post-down ip6tables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
+	bridge_ports none
+	bridge_stp off
+	bridge_fd 0
+	ip-forward on
+	ip6-forward on
+	vrf vrf_zonea
+
+auto vnetb
+iface vnetb
+	address 10.0.0.1/24
+	post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth2 -j MASQUERADE
+	post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth2 -j MASQUERADE
+	post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
+	post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
+	bridge_ports none
+	bridge_stp off
+	bridge_fd 0
+	ip-forward on
+	vrf vrf_zoneb
+
+auto vnetc
+iface vnetc
+	address 10.0.0.1/24
+	bridge_ports none
+	bridge_stp off
+	bridge_fd 0
+	ip-forward on
+	vrf vrf_zonec
+
+auto vnetd
+iface vnetd
+	address 10.0.0.1/24
+	post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth3 -j MASQUERADE
+	post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth3 -j MASQUERADE
+	post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth4 -j MASQUERADE
+	post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth4 -j MASQUERADE
+	post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
+	post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
+	bridge_ports none
+	bridge_stp off
+	bridge_fd 0
+	ip-forward on
+	vrf vrf_zoned
+
+auto vrf_zonea
+iface vrf_zonea
+	vrf-table auto
+	post-up ip route add vrf vrf_zonea unreachable default metric 4278198272
+	post-up ip -6 route add vrf vrf_zonea unreachable default metric 4278198272 || true
+
+auto vrf_zoneb
+iface vrf_zoneb
+	vrf-table auto
+	post-up ip route add vrf vrf_zoneb unreachable default metric 4278198272
+	post-up ip -6 route add vrf vrf_zoneb unreachable default metric 4278198272 || true
+
+auto vrf_zonec
+iface vrf_zonec
+	vrf-table auto
+	post-up ip route add vrf vrf_zonec unreachable default metric 4278198272
+	post-up ip -6 route add vrf vrf_zonec unreachable default metric 4278198272 || true
+
+auto vrf_zoned
+iface vrf_zoned
+	vrf-table auto
+	post-up ip route add vrf vrf_zoned unreachable default metric 4278198272
+	post-up ip -6 route add vrf vrf_zoned unreachable default metric 4278198272 || true
diff --git a/src/test/zones/simple/vrf_snat/interfaces b/src/test/zones/simple/vrf_snat/interfaces
new file mode 100644
index 0000000..67ad22f
--- /dev/null
+++ b/src/test/zones/simple/vrf_snat/interfaces
@@ -0,0 +1,29 @@
+auto vmbr0
+iface vmbr0 inet static
+	address 192.168.0.1/24
+	gateway 192.168.0.254
+        bridge-ports eth0
+        bridge-stp off
+        bridge-fd 0
+
+auto eth1
+iface eth1 inet static
+	address 203.0.113.10/24
+	vrf vrf_zonea
+
+iface eth1 inet6 auto
+	vrf vrf_zonea
+
+auto eth2
+iface eth2 inet dhcp
+	vrf vrf_zoneb
+
+auto eth3
+iface eth3 inet static
+	address 198.51.100.10/29
+	vrf vrf_zoned
+
+auto eth4
+iface eth4 inet static
+	address 198.51.100.18/29
+	vrf vrf_zoned
diff --git a/src/test/zones/simple/vrf_snat/sdn_config b/src/test/zones/simple/vrf_snat/sdn_config
new file mode 100644
index 0000000..fa74493
--- /dev/null
+++ b/src/test/zones/simple/vrf_snat/sdn_config
@@ -0,0 +1,53 @@
+{
+  version => 1,
+  vnets   => {
+               ids => {
+                        vneta => { type => "vnet", zone => "zonea" },
+                        vnetb => { type => "vnet", zone => "zoneb" },
+                        vnetc => { type => "vnet", zone => "zonec" },
+                        vnetd => { type => "vnet", zone => "zoned" },
+                      },
+             },
+  zones   => {
+               ids => {
+                        zonea => { ipam => "pve", type => "simple", "dedicated-vrf" => 1 },
+                        zoneb => { ipam => "pve", type => "simple", "dedicated-vrf" => 1 },
+                        zonec => { ipam => "pve", type => "simple", "dedicated-vrf" => 1 },
+                        zoned => { ipam => "pve", type => "simple", "dedicated-vrf" => 1 },
+                      },
+             },
+  subnets => {
+               ids => {
+                        'zonea-10.0.0.0-24' => {
+                                                 type => 'subnet',
+                                                 vnet => 'vneta',
+                                                 gateway => '10.0.0.1',
+                                                 snat => 1,
+                                               },
+                        'zonea-2001:db8:a::-64' => {
+                                                     type => 'subnet',
+                                                     vnet => 'vneta',
+                                                     gateway => '2001:db8:a::1',
+                                                     snat => 1,
+                                                   },
+                        'zoneb-10.0.0.0-24' => {
+                                                 type => 'subnet',
+                                                 vnet => 'vnetb',
+                                                 gateway => '10.0.0.1',
+                                                 snat => 1,
+                                               },
+                        'zonec-10.0.0.0-24' => {
+                                                 type => 'subnet',
+                                                 vnet => 'vnetc',
+                                                 gateway => '10.0.0.1',
+                                                 snat => 1,
+                                               },
+                        'zoned-10.0.0.0-24' => {
+                                                 type => 'subnet',
+                                                 vnet => 'vnetd',
+                                                 gateway => '10.0.0.1',
+                                                 snat => 1,
+                                               },
+                      },
+             },
+}
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH pve-network 05/14] sdn: zones: assign a conntrack zone to zones with a dedicated VRF
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
                   ` (3 preceding siblings ...)
  2026-11-19 19:45 ` [PATCH pve-network 04/14] sdn: simple: SNAT via the uplinks of zone VRFs Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  2026-11-19 19:45 ` [PATCH proxmox-firewall 06/14] firewall: apply vnet rules to host traffic from vnets in a VRF Stefan Hanreich
                   ` (8 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

The subnets of zones with a dedicated VRF may overlap, so connections in
different VRFs must not share a conntrack entry, or the firewall would
treat a new connection in one VRF as part of an established one in
another VRF, and conntrack state migration would mix up the entries.

Assign each such zone a conntrack zone from the first 1024 zones (16384
to 17407) of the block 16384 to 32767 that Proxmox VE reserves, stored
with the zone as ct-zone, so that it is the same on all nodes and stays
the same as long as the zone has a dedicated VRF. It is removed when
the dedicated VRF is disabled, and cannot be set via the API.

A new conntrack zone follows the highest one in use, including the
running config, so the conntrack zone of a zone that is still active or
was just deleted is not reused right away, which could match stale
conntrack entries of that zone.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 src/PVE/API2/Network/SDN/Zones.pm         |  13 +++
 src/PVE/Network/SDN/Zones.pm              |  42 +++++++++
 src/PVE/Network/SDN/Zones/Plugin.pm       |   4 +
 src/PVE/Network/SDN/Zones/SimplePlugin.pm |   7 ++
 src/test/Makefile                         |   5 +-
 src/test/run_test_ct_zones.pl             | 103 ++++++++++++++++++++++
 6 files changed, 173 insertions(+), 1 deletion(-)
 create mode 100755 src/test/run_test_ct_zones.pl

diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm
index 5952f40..5ff7bb4 100644
--- a/src/PVE/API2/Network/SDN/Zones.pm
+++ b/src/PVE/API2/Network/SDN/Zones.pm
@@ -70,6 +70,11 @@ my $ZONE_PROPERTIES = {
         optional => 1,
         description => 'Create a dedicated VRF for a simple zone.',
     },
+    'ct-zone' => {
+        type => 'integer',
+        optional => 1,
+        description => 'Conntrack zone of the dedicated VRF, assigned automatically.',
+    },
     route => {
         type => 'array',
         optional => 1,
@@ -407,6 +412,9 @@ __PACKAGE__->register_method({
         my $id = extract_param($param, 'zone');
         my $lock_token = extract_param($param, 'lock-token');
 
+        raise_param_exc({ 'ct-zone' => 'is assigned automatically' })
+            if defined($param->{'ct-zone'});
+
         my $plugin = PVE::Network::SDN::Zones::Plugin->lookup($type);
         my $opts = $plugin->check_config($id, $param, 1, 1);
 
@@ -434,6 +442,7 @@ __PACKAGE__->register_method({
                     if $dnszone && !$dnsserver;
 
                 $zone_cfg->{ids}->{$id} = $opts;
+                PVE::Network::SDN::Zones::update_ct_zone($zone_cfg, $id);
                 $plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
 
                 PVE::Network::SDN::Zones::write_config($zone_cfg);
@@ -478,6 +487,9 @@ __PACKAGE__->register_method({
             $delete = [PVE::Tools::split_list($delete)];
         }
 
+        raise_param_exc({ 'ct-zone' => 'is assigned automatically' })
+            if defined($param->{'ct-zone'}) || grep { $_ eq 'ct-zone' } ($delete // [])->@*;
+
         PVE::Network::SDN::lock_sdn_config(
             sub {
                 my $zone_cfg = PVE::Network::SDN::Zones::config();
@@ -541,6 +553,7 @@ __PACKAGE__->register_method({
                 my $dnszone = $opts->{dnszone};
                 raise_param_exc({ dnszone => "missing dns server" }) if $dnszone && !$dnsserver;
 
+                PVE::Network::SDN::Zones::update_ct_zone($zone_cfg, $id);
                 $plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
 
                 PVE::Network::SDN::Zones::write_config($zone_cfg);
diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm
index 3b59f4f..c8ff20b 100644
--- a/src/PVE/Network/SDN/Zones.pm
+++ b/src/PVE/Network/SDN/Zones.pm
@@ -59,6 +59,48 @@ sub config {
     return cfs_read_file("sdn/zones.cfg");
 }
 
+# Assigns a conntrack zone to a simple zone with a dedicated VRF and removes it from other zones.
+#
+# A new conntrack zone follows the highest one in use, also in the running config, so that the
+# conntrack zone of a zone that is still active or was just deleted is not reused right away.
+sub update_ct_zone {
+    my ($zone_cfg, $id) = @_;
+
+    my $zone = $zone_cfg->{ids}->{$id};
+
+    if ($zone->{type} ne 'simple' || !$zone->{'dedicated-vrf'}) {
+        delete $zone->{'ct-zone'};
+        return;
+    }
+
+    return if defined($zone->{'ct-zone'});
+
+    my $first = $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_FIRST;
+    my $last = $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_LAST;
+
+    my $running_zones = PVE::Network::SDN::Zones::config(1)->{ids};
+    my $used = {};
+    for my $zones ($zone_cfg->{ids}, $running_zones) {
+        for my $ct_zone (map { $_->{'ct-zone'} } values $zones->%*) {
+            $used->{$ct_zone} = 1 if defined($ct_zone) && $ct_zone >= $first && $ct_zone <= $last;
+        }
+    }
+
+    my $count = $last - $first + 1;
+    die "no conntrack zone left for the VRF of zone '$id'\n" if scalar(keys $used->%*) >= $count;
+
+    my ($highest) = sort { $b <=> $a } keys $used->%*;
+    $highest //= $last;
+
+    for my $offset (1 .. $count) {
+        my $ct_zone = $first + ($highest - $first + $offset) % $count;
+        if (!$used->{$ct_zone}) {
+            $zone->{'ct-zone'} = $ct_zone;
+            return;
+        }
+    }
+}
+
 sub get_plugin_config {
     my ($vnet) = @_;
     my $zoneid = $vnet->{zone};
diff --git a/src/PVE/Network/SDN/Zones/Plugin.pm b/src/PVE/Network/SDN/Zones/Plugin.pm
index 288edfc..d92bfd1 100644
--- a/src/PVE/Network/SDN/Zones/Plugin.pm
+++ b/src/PVE/Network/SDN/Zones/Plugin.pm
@@ -31,6 +31,10 @@ PVE::JSONSchema::register_standard_option(
     },
 );
 
+# conntrack zones reserved for the VRFs of zones, see also VrfCtZone in proxmox-ve-config
+our $VRF_CT_ZONE_FIRST = 16384;
+our $VRF_CT_ZONE_LAST = 17407;
+
 my $static_route_fmt = {
     prefix => {
         type => 'string',
diff --git a/src/PVE/Network/SDN/Zones/SimplePlugin.pm b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
index 4221521..3e866ac 100644
--- a/src/PVE/Network/SDN/Zones/SimplePlugin.pm
+++ b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
@@ -40,6 +40,12 @@ sub properties {
             type => 'boolean',
             default => 0,
         },
+        'ct-zone' => {
+            description => 'Conntrack zone of the dedicated VRF, assigned automatically.',
+            type => 'integer',
+            minimum => $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_FIRST,
+            maximum => $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_LAST,
+        },
     };
 }
 
@@ -53,6 +59,7 @@ sub options {
         ipam => { optional => 1 },
         dhcp => { optional => 1 },
         'dedicated-vrf' => { optional => 1 },
+        'ct-zone' => { optional => 1 },
         route => { optional => 1 },
     };
 }
diff --git a/src/test/Makefile b/src/test/Makefile
index 0d5f3c1..242901b 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
 all: test
 
-test: test_zones test_static_routes test_fabric_host_vrf test_ipams test_dns test_subnets test_vnets_blackbox test_running_config test_api_index
+test: test_zones test_static_routes test_ct_zones test_fabric_host_vrf test_ipams test_dns test_subnets test_vnets_blackbox test_running_config test_api_index
 
 test_zones: run_test_zones.pl
 	./run_test_zones.pl
@@ -11,6 +11,9 @@ test_static_routes: run_test_static_routes.pl
 test_fabric_host_vrf: run_test_fabric_host_vrf.pl
 	./run_test_fabric_host_vrf.pl
 
+test_ct_zones: run_test_ct_zones.pl
+	./run_test_ct_zones.pl
+
 test_ipams: run_test_ipams.pl
 	./run_test_ipams.pl
 
diff --git a/src/test/run_test_ct_zones.pl b/src/test/run_test_ct_zones.pl
new file mode 100755
index 0000000..1b71522
--- /dev/null
+++ b/src/test/run_test_ct_zones.pl
@@ -0,0 +1,103 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use lib qw(..);
+
+use Test::More;
+use Test::MockModule;
+
+use PVE::API2::Network::SDN::Zones;
+use PVE::Network::SDN::Zones;
+
+my $running_zones = {};
+
+my $mocked_zones = Test::MockModule->new('PVE::Network::SDN::Zones');
+$mocked_zones->mock(
+    config => sub {
+        my ($running) = @_;
+        die "unexpected read of zones.cfg\n" if !$running;
+        return { ids => $running_zones };
+    },
+);
+
+sub vrf_zone {
+    my ($ct_zone) = @_;
+    my $zone = { type => 'simple', 'dedicated-vrf' => 1 };
+    $zone->{'ct-zone'} = $ct_zone if defined($ct_zone);
+    return $zone;
+}
+
+sub ct_zone_of {
+    my ($zones, $id) = @_;
+    my $zone_cfg = { ids => $zones };
+    PVE::Network::SDN::Zones::update_ct_zone($zone_cfg, $id);
+    return $zone_cfg->{ids}->{$id}->{'ct-zone'};
+}
+
+is(ct_zone_of({ new => vrf_zone() }, 'new'), 16384, 'first zone VRF gets the first conntrack zone');
+
+is(
+    ct_zone_of({ a => vrf_zone(16384), b => vrf_zone(16390), new => vrf_zone() }, 'new'),
+    16391,
+    'new conntrack zone follows the highest one in use',
+);
+
+is(ct_zone_of({ a => vrf_zone(16500) }, 'a'), 16500, 'existing conntrack zone is kept');
+
+is(
+    ct_zone_of({ a => { type => 'simple', 'ct-zone' => 16400 } }, 'a'),
+    undef,
+    'conntrack zone is removed without a dedicated VRF',
+);
+
+is(
+    ct_zone_of({ a => { type => 'evpn', 'dedicated-vrf' => 1, 'ct-zone' => 16400 } }, 'a'),
+    undef,
+    'only simple zones get a conntrack zone',
+);
+
+$running_zones = { old => vrf_zone(16395) };
+is(
+    ct_zone_of({ a => vrf_zone(16384), new => vrf_zone() }, 'new'),
+    16396,
+    'conntrack zone of a zone that is only in the running config is not reused',
+);
+$running_zones = {};
+
+is(
+    ct_zone_of({ a => vrf_zone(17407), b => vrf_zone(16384), new => vrf_zone() }, 'new'),
+    16385,
+    'allocation wraps around after the last conntrack zone',
+);
+
+is(
+    ct_zone_of({ a => { type => 'vlan', 'ct-zone' => 17000 }, new => vrf_zone() }, 'new'),
+    17001,
+    'conntrack zones of other zones are taken into account',
+);
+
+is(
+    ct_zone_of({ a => vrf_zone(5), new => vrf_zone() }, 'new'),
+    16384,
+    'conntrack zones out of range are ignored',
+);
+
+my $all = { map { ("z$_" => vrf_zone($_)) } 16384 .. 17407 };
+$all->{new} = vrf_zone();
+eval { ct_zone_of($all, 'new') };
+like($@, qr/no conntrack zone left/, 'allocation fails if all conntrack zones are in use');
+
+for my $method ('create', 'update') {
+    my $param = { zone => 'zone0', 'ct-zone' => 16384 };
+    $param->{type} = 'simple' if $method eq 'create';
+
+    eval { PVE::API2::Network::SDN::Zones->$method($param) };
+    like($@, qr/ct-zone.*assigned automatically/s, "$method rejects setting ct-zone");
+}
+
+eval { PVE::API2::Network::SDN::Zones->update({ zone => 'zone0', delete => 'ct-zone' }) };
+like($@, qr/ct-zone.*assigned automatically/s, 'update rejects deleting ct-zone');
+
+done_testing();
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH proxmox-firewall 06/14] firewall: apply vnet rules to host traffic from vnets in a VRF
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
                   ` (4 preceding siblings ...)
  2026-11-19 19:45 ` [PATCH pve-network 05/14] sdn: zones: assign a conntrack zone to zones with a dedicated VRF Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  2026-11-19 19:45 ` [PATCH proxmox-firewall 07/14] nftables: expression: add a constructor for map lookups Stefan Hanreich
                   ` (7 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

The vnet firewall checks traffic from guests to the host by looking up
the input interface in the bridge map. For a vnet in a VRF, such as the
vnets of EVPN zones, the input interface of locally delivered traffic
is the VRF device, so the lookup never matched and the vnet rules for
traffic towards the host were silently skipped.

Additionally look up the slave device, which holds the bridge the
packet arrived on for traffic received in a VRF and is unset otherwise.

This only adds verdicts in a chain that runs before the host input
chain, so traffic that was allowed by the host rules before can now
be dropped by the vnet rules. With a restrictive forward policy on such
a vnet, traffic to the gateway, e.g. ICMP, needs to be allowed
explicitly, as for vnets outside a VRF.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 proxmox-firewall/resources/proxmox-firewall.nft | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/proxmox-firewall/resources/proxmox-firewall.nft b/proxmox-firewall/resources/proxmox-firewall.nft
index 265ed80..2675727 100644
--- a/proxmox-firewall/resources/proxmox-firewall.nft
+++ b/proxmox-firewall/resources/proxmox-firewall.nft
@@ -240,6 +240,9 @@ table inet proxmox-firewall {
     chain host-bridge-input {
         type filter hook input priority filter - 1; policy accept;
         meta iifname vmap @bridge-map
+        # traffic received in a VRF has the VRF device as input interface, the bridge it
+        # arrived on is the slave device
+        meta sdifname vmap @bridge-map
     }
 
     chain host-bridge-output {
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH proxmox-firewall 07/14] nftables: expression: add a constructor for map lookups
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
                   ` (5 preceding siblings ...)
  2026-11-19 19:45 ` [PATCH proxmox-firewall 06/14] firewall: apply vnet rules to host traffic from vnets in a VRF Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  2026-11-19 19:45 ` [PATCH proxmox-firewall 08/14] nftables: statement: add a helper for setting the conntrack zone Stefan Hanreich
                   ` (6 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

Map lookups could only be deserialized so far, but not constructed, as
the fields of the struct are private.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 proxmox-nftables/src/expression.rs | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/proxmox-nftables/src/expression.rs b/proxmox-nftables/src/expression.rs
index ca7a4e7..010c0c6 100644
--- a/proxmox-nftables/src/expression.rs
+++ b/proxmox-nftables/src/expression.rs
@@ -293,6 +293,22 @@ pub struct Map {
     data: Expression,
 }
 
+impl Map {
+    pub fn new(key: impl Into<Expression>, data: impl Into<Expression>) -> Self {
+        Self {
+            key: key.into(),
+            data: data.into(),
+        }
+    }
+}
+
+impl From<Map> for Expression {
+    #[inline]
+    fn from(map: Map) -> Self {
+        Expression::Map(Box::new(map))
+    }
+}
+
 #[derive(Clone, Debug, Deserialize, Serialize)]
 pub struct Ct {
     key: String,
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH proxmox-firewall 08/14] nftables: statement: add a helper for setting the conntrack zone
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
                   ` (6 preceding siblings ...)
  2026-11-19 19:45 ` [PATCH proxmox-firewall 07/14] nftables: expression: add a constructor for map lookups Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  2026-11-19 19:45 ` [PATCH proxmox-firewall 09/14] nftables: expression: convert the conntrack zones of zone VRFs Stefan Hanreich
                   ` (5 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 proxmox-nftables/src/statement.rs | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/proxmox-nftables/src/statement.rs b/proxmox-nftables/src/statement.rs
index 4433975..e500be5 100644
--- a/proxmox-nftables/src/statement.rs
+++ b/proxmox-nftables/src/statement.rs
@@ -394,6 +394,13 @@ impl Mangle {
             value: value.into(),
         }
     }
+
+    pub fn ct_zone(value: impl Into<Expression>) -> Self {
+        Self {
+            key: Ct::new("zone", None).into(),
+            value: value.into(),
+        }
+    }
 }
 
 #[derive(Clone, Copy, Debug, Deserialize, Serialize)]
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH proxmox-firewall 09/14] nftables: expression: convert the conntrack zones of zone VRFs
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
                   ` (7 preceding siblings ...)
  2026-11-19 19:45 ` [PATCH proxmox-firewall 08/14] nftables: statement: add a helper for setting the conntrack zone Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  2026-11-19 19:45 ` [PATCH proxmox-firewall 10/14] firewall: add a command applying " Stefan Hanreich
                   ` (4 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

The interfaces of zone VRFs need to be mapped to the conntrack zone of
their VRF. Convert them into the data of an anonymous map, so that a
rule can set the conntrack zone with a map lookup of the interface name.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 proxmox-nftables/src/expression.rs | 53 ++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/proxmox-nftables/src/expression.rs b/proxmox-nftables/src/expression.rs
index 010c0c6..cfd6a97 100644
--- a/proxmox-nftables/src/expression.rs
+++ b/proxmox-nftables/src/expression.rs
@@ -14,6 +14,8 @@ use proxmox_ve_config::firewall::types::rule_match::{IcmpCode, IcmpType, Icmpv6C
 use proxmox_ve_config::guest::types::Vmid;
 #[cfg(feature = "config-ext")]
 use proxmox_ve_config::host::types::BridgeName;
+#[cfg(feature = "config-ext")]
+use proxmox_ve_config::sdn::conntrack::VrfCtZones;
 
 #[derive(Clone, Debug, Deserialize, Serialize)]
 #[serde(rename_all = "lowercase")]
@@ -276,6 +278,16 @@ impl From<Vmid> for Expression {
     }
 }
 
+/// The interfaces of the zone VRFs with their conntrack zone, for use as data of a map lookup.
+#[cfg(feature = "config-ext")]
+impl From<&VrfCtZones> for Expression {
+    fn from(value: &VrfCtZones) -> Self {
+        Expression::set(value.interfaces().map(|(interface, ct_zone)| {
+            Expression::List(vec![interface.into(), ct_zone.id().into()])
+        }))
+    }
+}
+
 #[derive(Clone, Debug, Deserialize, Serialize)]
 pub struct Meta {
     key: String,
@@ -432,3 +444,44 @@ pub struct Element {
     config: ElemConfig,
     val: Expression,
 }
+
+#[cfg(all(test, feature = "config-ext"))]
+mod tests {
+    use std::str::FromStr;
+
+    use proxmox_ve_config::sdn::ZoneName;
+    use proxmox_ve_config::sdn::config::{ZoneConfig, ZoneType};
+    use proxmox_ve_config::sdn::conntrack::VrfCtZone;
+    use serde_json::json;
+
+    use super::*;
+
+    #[test]
+    fn vrf_ct_zones_map_data() {
+        let mut ct_zones = VrfCtZones::default();
+
+        assert_eq!(
+            serde_json::to_value(Expression::from(&ct_zones)).unwrap(),
+            json!({ "set": [] })
+        );
+
+        for (name, ct_zone) in [("red", 16384), ("blue", 16389)] {
+            let mut zone = ZoneConfig::new(ZoneName::from_str(name).unwrap(), ZoneType::Simple);
+            zone.set_ct_zone(VrfCtZone::new(ct_zone));
+            ct_zones.add_zone(&zone);
+        }
+
+        ct_zones.add_interface("vrf_red", "vnetr1");
+        ct_zones.add_interface("vrf_blue", "vnetb");
+
+        assert_eq!(
+            serde_json::to_value(Expression::from(&ct_zones)).unwrap(),
+            json!({ "set": [
+                ["vrf_blue", 16389],
+                ["vnetb", 16389],
+                ["vrf_red", 16384],
+                ["vnetr1", 16384],
+            ]})
+        );
+    }
+}
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH proxmox-firewall 10/14] firewall: add a command applying the conntrack zones of zone VRFs
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
                   ` (8 preceding siblings ...)
  2026-11-19 19:45 ` [PATCH proxmox-firewall 09/14] nftables: expression: convert the conntrack zones of zone VRFs Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  2026-11-19 19:45 ` [PATCH proxmox-firewall 11/14] firewall: put guest traffic of zone VRFs into their conntrack zone Stefan Hanreich
                   ` (3 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

The new command sdn-ct-zones sets the conntrack zone of a VRF for the
VRF device and all interfaces enslaved to it, as reported by the
kernel. Interfaces end up in a zone VRF via vnets, fabrics and
/etc/network/interfaces, and the kernel state reflects all of them once
the network configuration got applied. The command is therefore meant
for internal use by pve-manager after applying the network
configuration, which the help text points out. With --print, the rules
are printed instead of applied.

NAT relies on conntrack independent of the firewall, so the rules live
in a table of their own, which neither the firewall service nor its
removal touches.

The zone is set in prerouting and, for locally generated traffic, in
output. Both chains run before conntrack and before the iptables raw
table, so the zone takes effect independent of the order in which the
rules were loaded. The VRF device needs an entry as well, as received
packets are looked up a second time when passing it, and locally
generated packets are tracked when passing output on it.

The table is replaced atomically and removed if there is no zone VRF
with a conntrack zone. If the SDN running configuration cannot be read
or parsed, the command fails and keeps the existing rules.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 proxmox-firewall/src/bin/proxmox-firewall.rs |  30 ++-
 proxmox-firewall/src/config.rs               |  15 +-
 proxmox-firewall/src/lib.rs                  |   1 +
 proxmox-firewall/src/sdn.rs                  | 235 +++++++++++++++++++
 4 files changed, 273 insertions(+), 8 deletions(-)
 create mode 100644 proxmox-firewall/src/sdn.rs

diff --git a/proxmox-firewall/src/bin/proxmox-firewall.rs b/proxmox-firewall/src/bin/proxmox-firewall.rs
index e8ed477..41ab2d3 100644
--- a/proxmox-firewall/src/bin/proxmox-firewall.rs
+++ b/proxmox-firewall/src/bin/proxmox-firewall.rs
@@ -5,10 +5,13 @@ use std::time::{Duration, Instant};
 use anyhow::{Context, Error, bail, format_err};
 use pico_args::Arguments;
 
+use proxmox_firewall::config::FirewallSdnConfig;
 use proxmox_firewall::config::{FirewallConfig, PveFirewallConfigLoader, PveNftConfigLoader};
 use proxmox_firewall::firewall::Firewall;
+use proxmox_firewall::sdn::{load_sdn_config, vrf_ct_zones, vrf_ct_zones_commands};
 use proxmox_log as log;
 use proxmox_log::{LevelFilter, Logger};
+use proxmox_network_api::get_network_interfaces;
 use proxmox_nftables::{NftClient, client::NftError};
 use proxmox_ve_config::firewall::host::Config as HostConfig;
 
@@ -22,6 +25,9 @@ COMMANDS:
   compile           Compile and print firewall rules as accepted by 'nft -j -f -'
   start             Execute proxmox-firewall service in foreground
   localnet          Print the contents of the management ipset
+  sdn-ct-zones      Apply the conntrack zones of SDN zone VRFs (internal, run by
+                    pve-manager when applying the network configuration)
+                    --print: Print the rules as accepted by 'nft -j -f -' instead
 "#;
 
 const RULE_BASE: &str = include_str!("../../resources/proxmox-firewall.nft");
@@ -72,6 +78,23 @@ fn handle_firewall() -> Result<(), Error> {
     Ok(())
 }
 
+fn handle_sdn_ct_zones(print: bool) -> Result<(), Error> {
+    let sdn = load_sdn_config(&PveFirewallConfigLoader::new())?;
+    let links = get_network_interfaces()?.into_values();
+
+    let ct_zones = vrf_ct_zones(sdn.as_ref().map(FirewallSdnConfig::config), links);
+    let commands = vrf_ct_zones_commands(&ct_zones);
+
+    if print {
+        println!("{}", serde_json::to_string_pretty(&commands)?);
+    } else {
+        NftClient::run_json_commands(&commands)
+            .context("could not apply the conntrack zones of SDN zone VRFs")?;
+    }
+
+    Ok(())
+}
+
 fn init_logger(command: Command) -> Result<(), Error> {
     let mut logger = Logger::from_env("PVE_LOG", LevelFilter::WARN);
 
@@ -125,6 +148,7 @@ pub enum Command {
     Skeleton,
     Start,
     Localnet,
+    SdnCtZones,
 }
 
 impl std::str::FromStr for Command {
@@ -137,6 +161,7 @@ impl std::str::FromStr for Command {
             "skeleton" => Command::Skeleton,
             "start" => Command::Start,
             "localnet" => Command::Localnet,
+            "sdn-ct-zones" => Command::SdnCtZones,
             cmd => {
                 bail!("{cmd} is not a valid command")
             }
@@ -144,7 +169,7 @@ impl std::str::FromStr for Command {
     }
 }
 
-fn run_command(command: Command) -> Result<(), Error> {
+fn run_command(command: Command, args: &mut Arguments) -> Result<(), Error> {
     init_logger(command)?;
 
     match command {
@@ -169,6 +194,7 @@ fn run_command(command: Command) -> Result<(), Error> {
                 println!("{ip}");
             }
         }
+        Command::SdnCtZones => handle_sdn_ct_zones(args.contains("--print"))?,
     };
 
     Ok(())
@@ -183,7 +209,7 @@ fn main() -> Result<(), Error> {
         .parse();
 
     if let Ok(command) = parsed_command {
-        run_command(command)
+        run_command(command, &mut args)
     } else {
         eprintln!("Invalid command specified!\n{}", HELP);
         std::process::exit(1);
diff --git a/proxmox-firewall/src/config.rs b/proxmox-firewall/src/config.rs
index acc4b56..7e055c0 100644
--- a/proxmox-firewall/src/config.rs
+++ b/proxmox-firewall/src/config.rs
@@ -115,7 +115,9 @@ fn bridge_name_from_fw_name(name: &str) -> Result<BridgeName, Error> {
 ///
 /// Reading the content before parsing it separates read errors, which are most likely transient,
 /// from errors in the content.
-fn read_config(data: Option<Box<dyn io::BufRead>>) -> Result<Option<Vec<u8>>, io::Error> {
+pub(crate) fn read_config(
+    data: Option<Box<dyn io::BufRead>>,
+) -> Result<Option<Vec<u8>>, io::Error> {
     data.map(|mut data| {
         let mut content = Vec::new();
         data.read_to_end(&mut content)?;
@@ -308,11 +310,15 @@ impl NftConfigLoader for PveNftConfigLoader {
 }
 
 pub struct FirewallSdnConfig {
-    _config: SdnConfig,
+    config: SdnConfig,
     ipsets: BTreeMap<String, Ipset>,
 }
 
 impl FirewallSdnConfig {
+    pub fn config(&self) -> &SdnConfig {
+        &self.config
+    }
+
     pub fn ipsets(&self) -> &BTreeMap<String, Ipset> {
         &self.ipsets
     }
@@ -416,10 +422,7 @@ impl FirewallConfig {
                     .map(|ipset| (ipset.name().name().to_string(), ipset))
                     .collect();
 
-                Some(FirewallSdnConfig {
-                    _config: config,
-                    ipsets,
-                })
+                Some(FirewallSdnConfig { config, ipsets })
             }
             _ => None,
         })
diff --git a/proxmox-firewall/src/lib.rs b/proxmox-firewall/src/lib.rs
index c4b037a..25db727 100644
--- a/proxmox-firewall/src/lib.rs
+++ b/proxmox-firewall/src/lib.rs
@@ -2,3 +2,4 @@ pub mod config;
 pub mod firewall;
 pub mod object;
 pub mod rule;
+pub mod sdn;
diff --git a/proxmox-firewall/src/sdn.rs b/proxmox-firewall/src/sdn.rs
new file mode 100644
index 0000000..ded5740
--- /dev/null
+++ b/proxmox-firewall/src/sdn.rs
@@ -0,0 +1,235 @@
+//! Conntrack zones of SDN zone VRFs.
+//!
+//! Sets the conntrack zone of each zone VRF for the VRF device and the interfaces in the VRF. NAT
+//! relies on conntrack as well, so the rules are independent of the firewall and live in a table of
+//! their own.
+
+use anyhow::{Context, Error};
+
+use proxmox_network_api::IpLink;
+use proxmox_nftables::command::{Add, Commands, Delete};
+use proxmox_nftables::expression::{Map, Meta};
+use proxmox_nftables::statement::Mangle;
+use proxmox_nftables::types::{
+    AddChain, AddRule, AddTable, BaseChainConfig, ChainPart, ChainPolicy, ChainType, Hook,
+    TableFamily, TablePart,
+};
+use proxmox_nftables::{Expression, Statement};
+use proxmox_ve_config::sdn::config::SdnConfig;
+use proxmox_ve_config::sdn::conntrack::VrfCtZones;
+
+use crate::config::{FirewallConfig, FirewallConfigLoader, FirewallSdnConfig, read_config};
+
+pub const VRF_CT_ZONES_TABLE: &str = "proxmox-sdn-ct-zones";
+
+// before the iptables raw table (-300) and conntrack (-200), so that the zone takes effect
+// regardless of the order in which the tables were loaded
+const VRF_CT_ZONES_PRIORITY: i64 = -310;
+
+/// Loads the SDN running configuration.
+///
+/// Fails if the configuration cannot be read or parsed, so that the existing rules are kept
+/// instead of being removed.
+pub fn load_sdn_config(
+    loader: &dyn FirewallConfigLoader,
+) -> Result<Option<FirewallSdnConfig>, Error> {
+    loader.check_mounted()?;
+
+    let data = read_config(loader.sdn_running_config()?)
+        .context("failed to read the SDN running configuration")?;
+
+    FirewallConfig::parse_sdn(data.as_deref())
+}
+
+/// Collects the zone VRFs with a conntrack zone and the interfaces that are enslaved to them.
+///
+/// The members are taken from the kernel, which reflects all sources that put an interface into a
+/// VRF once the network configuration got applied.
+pub fn vrf_ct_zones(
+    sdn: Option<&SdnConfig>,
+    links: impl IntoIterator<Item = IpLink>,
+) -> VrfCtZones {
+    let mut ct_zones = sdn.map(VrfCtZones::from).unwrap_or_default();
+
+    for link in links {
+        if let Some(master) = link.master() {
+            ct_zones.add_interface(master, link.name());
+        }
+    }
+
+    ct_zones
+}
+
+/// Generates the commands that atomically replace the table for the conntrack zones.
+///
+/// Without any zone VRF, the commands only remove the table.
+pub fn vrf_ct_zones_commands(ct_zones: &VrfCtZones) -> Commands {
+    let table = TablePart::new(TableFamily::Inet, VRF_CT_ZONES_TABLE);
+
+    let mut commands = Commands::new(vec![
+        Add::table(AddTable::new(TableFamily::Inet, VRF_CT_ZONES_TABLE)),
+        Delete::table(table.clone()),
+    ]);
+
+    if ct_zones.is_empty() {
+        return commands;
+    }
+
+    commands.push(Add::table(AddTable::new(
+        TableFamily::Inet,
+        VRF_CT_ZONES_TABLE,
+    )));
+
+    // received packets pass prerouting on the receiving interface and on the VRF device, locally
+    // generated ones pass output on the VRF device and on the outgoing interface
+    for (hook, key) in [(Hook::Prerouting, "iifname"), (Hook::Output, "oifname")] {
+        let chain = ChainPart::new(table.clone(), hook.to_string());
+
+        commands.push(Add::chain(AddChain::new_base_chain(
+            table.clone(),
+            chain.name(),
+            BaseChainConfig::new(
+                ChainType::Filter,
+                hook,
+                VRF_CT_ZONES_PRIORITY,
+                ChainPolicy::Accept,
+            ),
+        )));
+
+        commands.push(Add::rule(AddRule::from_statement(
+            chain,
+            Statement::from(Mangle::ct_zone(Map::new(
+                Meta::new(key),
+                Expression::from(ct_zones),
+            ))),
+        )));
+    }
+
+    commands
+}
+
+#[cfg(test)]
+mod tests {
+    use proxmox_ve_config::sdn::config::RunningConfig;
+    use serde_json::json;
+
+    use super::*;
+
+    fn link(name: &str, master: Option<&str>) -> IpLink {
+        serde_json::from_value(json!({
+            "ifindex": 1,
+            "ifname": name,
+            "link_type": "ether",
+            "operstate": "UP",
+            "master": master,
+        }))
+        .unwrap()
+    }
+
+    fn sdn_config() -> SdnConfig {
+        let running_config: RunningConfig = serde_json::from_value(json!({
+            "zones": {
+                "ids": {
+                    "red": { "type": "simple", "dedicated-vrf": 1, "ct-zone": 16384 },
+                    "blue": { "type": "simple", "dedicated-vrf": 1, "ct-zone": "16389" },
+                    "green": { "type": "simple" },
+                },
+            },
+        }))
+        .unwrap();
+
+        SdnConfig::try_from(running_config).unwrap()
+    }
+
+    #[test]
+    fn vrf_ct_zones_from_links() {
+        let links = [
+            link("vrf_red", None),
+            link("vnetr1", Some("vrf_red")),
+            link("bond0.101", Some("vrf_red")),
+            link("vnetb", Some("vrf_blue")),
+            link("ens18", Some("vmbr0")),
+            link("eth5", Some("vrf_other")),
+            link("vnetg", Some("vrf_green")),
+            link("lo", None),
+        ];
+
+        let interfaces: Vec<_> = vrf_ct_zones(Some(&sdn_config()), links)
+            .interfaces()
+            .map(|(interface, ct_zone)| (interface.to_string(), ct_zone.id()))
+            .collect();
+
+        assert_eq!(
+            interfaces,
+            [
+                ("vrf_blue".to_string(), 16389),
+                ("vnetb".to_string(), 16389),
+                ("vrf_red".to_string(), 16384),
+                ("bond0.101".to_string(), 16384),
+                ("vnetr1".to_string(), 16384),
+            ]
+        );
+
+        assert!(vrf_ct_zones(None, [link("vnetr1", Some("vrf_red"))]).is_empty());
+    }
+
+    #[test]
+    fn vrf_ct_zones_commands_without_zones() {
+        let commands = vrf_ct_zones_commands(&VrfCtZones::default());
+
+        assert_eq!(
+            serde_json::to_value(&commands).unwrap(),
+            json!({"nftables": [
+                {"add": {"table": {"family": "inet", "name": "proxmox-sdn-ct-zones"}}},
+                {"delete": {"table": {"family": "inet", "name": "proxmox-sdn-ct-zones"}}},
+            ]})
+        );
+    }
+
+    #[test]
+    fn vrf_ct_zones_commands_with_zones() {
+        let ct_zones = vrf_ct_zones(Some(&sdn_config()), [link("vnetb", Some("vrf_blue"))]);
+
+        let elements = json!({"set": [
+            ["vrf_blue", 16389],
+            ["vnetb", 16389],
+            ["vrf_red", 16384],
+        ]});
+
+        let chain = |hook: &str, key: &str| {
+            [
+                json!({"add": {"chain": {
+                    "family": "inet",
+                    "table": "proxmox-sdn-ct-zones",
+                    "name": hook,
+                    "type": "filter",
+                    "hook": hook,
+                    "prio": -310,
+                    "policy": "accept",
+                }}}),
+                json!({"add": {"rule": {
+                    "family": "inet",
+                    "table": "proxmox-sdn-ct-zones",
+                    "chain": hook,
+                    "expr": [{"mangle": {
+                        "key": {"ct": {"key": "zone"}},
+                        "value": {"map": {"key": {"meta": {"key": key}}, "data": elements}},
+                    }}],
+                }}}),
+            ]
+        };
+
+        let mut expected = vec![
+            json!({"add": {"table": {"family": "inet", "name": "proxmox-sdn-ct-zones"}}}),
+            json!({"delete": {"table": {"family": "inet", "name": "proxmox-sdn-ct-zones"}}}),
+            json!({"add": {"table": {"family": "inet", "name": "proxmox-sdn-ct-zones"}}}),
+        ];
+        expected.extend(chain("prerouting", "iifname"));
+        expected.extend(chain("output", "oifname"));
+
+        assert_eq!(
+            serde_json::to_value(vrf_ct_zones_commands(&ct_zones)).unwrap(),
+            json!({ "nftables": expected })
+        );
+    }
+}
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH proxmox-firewall 11/14] firewall: put guest traffic of zone VRFs into their conntrack zone
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
                   ` (9 preceding siblings ...)
  2026-11-19 19:45 ` [PATCH proxmox-firewall 10/14] firewall: add a command applying " Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  2026-11-19 19:45 ` [PATCH pve-manager 12/14] network: apply the conntrack zones of zone VRFs Stefan Hanreich
                   ` (2 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

Bridge conntrack tracks the connections of guests as soon as the guest
table uses conntrack, and routed packets keep that entry in the inet
hooks. The conntrack zones set by the sdn-ct-zones command therefore
come too late for guest traffic.

Set the conntrack zone of the VRF for the traffic of its vnets in a
chain of the guest table that runs before bridge conntrack. As the
chain is part of the guest table, it exists exactly when bridge
conntrack is in use. The conntrack zones are taken from the SDN running
config.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 .../resources/proxmox-firewall.nft            | 13 ++++++++
 proxmox-firewall/src/config.rs                |  8 +++++
 proxmox-firewall/src/firewall.rs              | 30 +++++++++++++++++++
 .../tests/input/.running-config.json          |  4 ++-
 .../integration_tests__firewall.snap          | 24 +++++++++++++++
 5 files changed, 78 insertions(+), 1 deletion(-)

diff --git a/proxmox-firewall/resources/proxmox-firewall.nft b/proxmox-firewall/resources/proxmox-firewall.nft
index 2675727..af69248 100644
--- a/proxmox-firewall/resources/proxmox-firewall.nft
+++ b/proxmox-firewall/resources/proxmox-firewall.nft
@@ -44,6 +44,7 @@ add chain bridge proxmox-firewall-guests pre-vm-in
 add chain bridge proxmox-firewall-guests vm-in {type filter hook postrouting priority 0; policy accept;}
 add chain bridge proxmox-firewall-guests before-bridge
 add chain bridge proxmox-firewall-guests forward {type filter hook forward priority 0; policy accept;}
+add chain bridge proxmox-firewall-guests sdn-ct-zone {type filter hook prerouting priority -310; policy accept;}
 
 flush chain inet proxmox-firewall do-reject
 flush chain inet proxmox-firewall accept-management
@@ -84,6 +85,7 @@ flush chain bridge proxmox-firewall-guests pre-vm-in
 flush chain bridge proxmox-firewall-guests vm-in
 flush chain bridge proxmox-firewall-guests before-bridge
 flush chain bridge proxmox-firewall-guests forward
+flush chain bridge proxmox-firewall-guests sdn-ct-zone
 
 table inet proxmox-firewall {
     chain do-reject {
@@ -296,6 +298,17 @@ table bridge proxmox-firewall-guests {
         typeof iifname : verdict
     }
 
+    map sdn-ct-zones {
+        typeof meta ibrname : ct zone
+    }
+
+    # Bridge conntrack tracks the connections of the guests, and routed packets keep the entry in
+    # the inet hooks, so the conntrack zone of their VRF has to be set before it (priority -200).
+    chain sdn-ct-zone {
+        type filter hook prerouting priority -310; policy accept;
+        ct zone set meta ibrname map @sdn-ct-zones
+    }
+
     chain allow-dhcp-in {
         udp sport . udp dport { 67 . 68, 547 . 546 } accept
     }
diff --git a/proxmox-firewall/src/config.rs b/proxmox-firewall/src/config.rs
index 7e055c0..cf82bc6 100644
--- a/proxmox-firewall/src/config.rs
+++ b/proxmox-firewall/src/config.rs
@@ -26,6 +26,7 @@ use proxmox_nftables::types::ListChain;
 use proxmox_ve_config::sdn::{
     VnetName,
     config::{RunningConfig, SdnConfig},
+    conntrack::VrfCtZone,
     ipam::{Ipam, IpamJson},
 };
 
@@ -319,6 +320,13 @@ impl FirewallSdnConfig {
         &self.config
     }
 
+    /// The vnets of zones with a dedicated VRF, together with the conntrack zone of the VRF.
+    pub fn vnet_ct_zones(&self) -> impl Iterator<Item = (&VnetName, VrfCtZone)> + '_ {
+        self.config
+            .vnets()
+            .filter_map(|(zone, vnet)| Some((vnet.name(), zone.ct_zone()?)))
+    }
+
     pub fn ipsets(&self) -> &BTreeMap<String, Ipset> {
         &self.ipsets
     }
diff --git a/proxmox-firewall/src/firewall.rs b/proxmox-firewall/src/firewall.rs
index d620a2e..d80a463 100644
--- a/proxmox-firewall/src/firewall.rs
+++ b/proxmox-firewall/src/firewall.rs
@@ -82,6 +82,10 @@ impl Firewall {
         SetName::new(Self::guest_table(), format!("vm-map-{dir}"))
     }
 
+    fn sdn_ct_zone_map() -> SetName {
+        SetName::new(Self::guest_table(), "sdn-ct-zones")
+    }
+
     fn cluster_chain(dir: Direction) -> ChainPart {
         ChainPart::new(Self::cluster_table(), format!("cluster-{dir}"))
     }
@@ -156,6 +160,7 @@ impl Firewall {
             Flush::map(Self::guest_vmap(Direction::Out)),
             Flush::map(Self::bridge_vmap(Self::guest_table())),
             Flush::map(Self::bridge_vmap(Self::host_table())),
+            Flush::map(Self::sdn_ct_zone_map()),
             Flush::chain(Self::host_conntrack_chain()),
             Flush::chain(Self::synflood_limit_chain()),
             Flush::chain(Self::log_invalid_tcp_chain()),
@@ -331,6 +336,7 @@ impl Firewall {
         if !(enabled_guests.is_empty() && enabled_bridges.is_empty()) {
             log::info!("creating guest configuration");
 
+            self.create_sdn_ct_zones(&mut commands);
             self.create_management_ipset(&mut commands, &guest_table)?;
 
             self.create_ipsets(
@@ -455,6 +461,30 @@ impl Firewall {
         Ok(())
     }
 
+    /// Puts the traffic of vnets in zones with a dedicated VRF into the conntrack zone of the VRF.
+    fn create_sdn_ct_zones(&self, commands: &mut Commands) {
+        let Some(sdn) = self.config.sdn() else {
+            return;
+        };
+
+        let elements: Vec<(Expression, MapValue)> = sdn
+            .vnet_ct_zones()
+            .map(|(vnet, ct_zone)| {
+                (
+                    Expression::from(vnet.as_ref()),
+                    MapValue::from(Expression::from(ct_zone.id())),
+                )
+            })
+            .collect();
+
+        if !elements.is_empty() {
+            commands.push(Add::element(AddElement::map_from_expressions(
+                Self::sdn_ct_zone_map(),
+                elements,
+            )));
+        }
+    }
+
     fn handle_host_options(&self, commands: &mut Commands) -> Result<(), Error> {
         log::info!("setting host options");
 
diff --git a/proxmox-firewall/tests/input/.running-config.json b/proxmox-firewall/tests/input/.running-config.json
index a4511f0..c674352 100644
--- a/proxmox-firewall/tests/input/.running-config.json
+++ b/proxmox-firewall/tests/input/.running-config.json
@@ -35,7 +35,9 @@
       "test": {
         "dhcp": "dnsmasq",
         "ipam": "pve",
-        "type": "simple"
+        "type": "simple",
+        "dedicated-vrf": 1,
+        "ct-zone": 16384
       }
     }
   },
diff --git a/proxmox-firewall/tests/snapshots/integration_tests__firewall.snap b/proxmox-firewall/tests/snapshots/integration_tests__firewall.snap
index 69fb10a..a434586 100644
--- a/proxmox-firewall/tests/snapshots/integration_tests__firewall.snap
+++ b/proxmox-firewall/tests/snapshots/integration_tests__firewall.snap
@@ -148,6 +148,15 @@ expression: "firewall.full_host_fw().expect(\"firewall can be generated\")"
         }
       }
     },
+    {
+      "flush": {
+        "map": {
+          "family": "bridge",
+          "table": "proxmox-firewall-guests",
+          "name": "sdn-ct-zones"
+        }
+      }
+    },
     {
       "flush": {
         "chain": {
@@ -4366,6 +4375,21 @@ expression: "firewall.full_host_fw().expect(\"firewall can be generated\")"
         }
       }
     },
+    {
+      "add": {
+        "element": {
+          "family": "bridge",
+          "table": "proxmox-firewall-guests",
+          "name": "sdn-ct-zones",
+          "elem": [
+            [
+              "public",
+              16384
+            ]
+          ]
+        }
+      }
+    },
     {
       "add": {
         "set": {
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH pve-manager 12/14] network: apply the conntrack zones of zone VRFs
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
                   ` (10 preceding siblings ...)
  2026-11-19 19:45 ` [PATCH proxmox-firewall 11/14] firewall: put guest traffic of zone VRFs into their conntrack zone Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  2026-11-19 19:45 ` [PATCH pve-manager 13/14] network: warn if pve-firewall cannot isolate guests in " Stefan Hanreich
  2026-11-19 19:45 ` [PATCH pve-docs 14/14] pvesdn: simple zones: document dedicated VRFs Stefan Hanreich
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

Apply the conntrack zones of zone VRFs with 'proxmox-firewall
sdn-ct-zones' whenever the network configuration is applied. The
command sets them for the interfaces that the kernel reports as
enslaved to a zone VRF, so it has to run after ifreload. Failing to
apply them only causes a warning, as the network configuration is
already applied at that point.

At boot, apply them in pve-sdn-commit after the cluster file system is
quorate, also if there are no pending changes, as the running config
may have changed while the node was offline. Start guests only after
that, so their connections are tracked in the right conntrack zone from
the start.

proxmox-firewall is only recommended, so warn if it is not installed
while there are zone VRFs with a conntrack zone.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 PVE/API2/Network.pm             | 18 ++++++++++++++++++
 bin/pve-sdn-commit              | 20 ++++++++++++++++++++
 services/pve-sdn-commit.service |  2 ++
 3 files changed, 40 insertions(+)

diff --git a/PVE/API2/Network.pm b/PVE/API2/Network.pm
index 21157877..064fe40a 100644
--- a/PVE/API2/Network.pm
+++ b/PVE/API2/Network.pm
@@ -26,6 +26,22 @@ eval {
 
 my $iflockfn = "/etc/network/.pve-interfaces.lock";
 
+my $proxmox_firewall = '/usr/libexec/proxmox/proxmox-firewall';
+
+# the conntrack zones are set for the interfaces that the kernel reports as enslaved to a zone VRF,
+# so this needs to run after ifreload
+my sub apply_vrf_ct_zones {
+    if (!-x $proxmox_firewall) {
+        my $zones = PVE::Network::SDN::running_config()->{zones}->{ids} // {};
+        warn "proxmox-firewall is not installed, connections are not isolated between zone VRFs\n"
+            if grep { defined($_->{'ct-zone'}) } values $zones->%*;
+        return;
+    }
+
+    eval { PVE::Tools::run_command([$proxmox_firewall, 'sdn-ct-zones']) };
+    warn "failed to apply the conntrack zones of zone VRFs - $@" if $@;
+}
+
 my $bond_mode_enum = [
     'balance-rr',
     'active-backup', # OVS and Linux
@@ -969,6 +985,8 @@ __PACKAGE__->register_method({
             };
             PVE::Tools::run_command(['ifreload', '-a'], errfunc => $err);
 
+            apply_vrf_ct_zones() if $have_sdn;
+
             # needs the zone VRFs created by ifreload, because dnsmasq is started inside
             # them with 'ip vrf exec'
             PVE::Network::SDN::generate_dhcp_config() if $have_sdn;
diff --git a/bin/pve-sdn-commit b/bin/pve-sdn-commit
index ab4175b8..57c2feba 100644
--- a/bin/pve-sdn-commit
+++ b/bin/pve-sdn-commit
@@ -22,6 +22,23 @@ for (my $i = 0; !PVE::Cluster::check_cfs_quorum(1); $i++) {
     usleep(100 * 1000);
 }
 
+my $proxmox_firewall = '/usr/libexec/proxmox/proxmox-firewall';
+
+# The conntrack zones are set for the interfaces that the kernel reports as enslaved to a zone VRF,
+# so this needs to run after ifreload. The running config may have changed while the node was
+# offline, so apply them even without pending changes.
+sub apply_vrf_ct_zones {
+    if (!-x $proxmox_firewall) {
+        my $zones = PVE::Network::SDN::running_config()->{zones}->{ids} // {};
+        warn "proxmox-firewall is not installed, connections are not isolated between zone VRFs\n"
+            if grep { defined($_->{'ct-zone'}) } values $zones->%*;
+        return;
+    }
+
+    eval { PVE::Tools::run_command([$proxmox_firewall, 'sdn-ct-zones']) };
+    warn "failed to apply the conntrack zones of zone VRFs - $@" if $@;
+}
+
 sub has_pending_changes {
     my ($pending_config) = @_;
 
@@ -98,6 +115,7 @@ sub sdn_changed {
 
 if (!sdn_changed()) {
     print "No changes to SDN configuration detected, skipping reload\n";
+    apply_vrf_ct_zones();
     exit 0;
 }
 
@@ -118,6 +136,8 @@ my $err = sub {
 
 PVE::Tools::run_command(['ifreload', '-a'], errfunc => $err);
 
+apply_vrf_ct_zones();
+
 # needs the zone VRFs created by ifreload, because dnsmasq is started inside
 # them with 'ip vrf exec'
 PVE::Network::SDN::generate_dhcp_config();
diff --git a/services/pve-sdn-commit.service b/services/pve-sdn-commit.service
index ff723725..2dfe7872 100644
--- a/services/pve-sdn-commit.service
+++ b/services/pve-sdn-commit.service
@@ -3,6 +3,8 @@ Description=Commit Proxmox VE SDN changes
 DefaultDependencies=no
 Wants=pve-cluster.service network.target
 After=frr.service network.target corosync.service
+# guests must only start once the zone VRFs have their conntrack zones
+Before=pve-guests.service
 
 [Service]
 ExecStart=/usr/share/pve-manager/helpers/pve-sdn-commit
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH pve-manager 13/14] network: warn if pve-firewall cannot isolate guests in zone VRFs
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
                   ` (11 preceding siblings ...)
  2026-11-19 19:45 ` [PATCH pve-manager 12/14] network: apply the conntrack zones of zone VRFs Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  2026-11-19 19:45 ` [PATCH pve-docs 14/14] pvesdn: simple zones: document dedicated VRFs Stefan Hanreich
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

pve-firewall tracks the connections of guests with the firewall enabled
on a network device on their firewall bridges, in a conntrack zone that
is shared by all VRFs, so the conntrack zones of zone VRFs do not keep
these connections apart.

Warn when applying the network configuration if zone VRFs exist and the
enabled firewall is pve-firewall instead of the nftables firewall, which
sets the conntrack zone before tracking the connections of guests.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 PVE/API2/Network.pm | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/PVE/API2/Network.pm b/PVE/API2/Network.pm
index 064fe40a..a10a47e1 100644
--- a/PVE/API2/Network.pm
+++ b/PVE/API2/Network.pm
@@ -12,6 +12,8 @@ use PVE::RESTHandler;
 use PVE::RPCEnvironment;
 use PVE::JSONSchema qw(get_standard_option);
 use PVE::AccessControl;
+use PVE::Firewall;
+use PVE::Firewall::Helpers;
 use PVE::Network;
 use IO::File;
 
@@ -42,6 +44,18 @@ my sub apply_vrf_ct_zones {
     warn "failed to apply the conntrack zones of zone VRFs - $@" if $@;
 }
 
+# pve-firewall tracks the connections of guests with the firewall enabled on a network device on
+# the firewall bridges, in a conntrack zone that is shared by all zone VRFs
+my sub warn_vrf_zones_with_legacy_firewall {
+    my $zones = PVE::Network::SDN::running_config()->{zones}->{ids} // {};
+    return if !grep { defined($_->{'ct-zone'}) } values $zones->%*;
+    return if PVE::Firewall::Helpers::is_nftables();
+    return if !PVE::Firewall::load_clusterfw_conf()->{options}->{enable};
+
+    warn "guests with the firewall enabled on a network device are not isolated between zone"
+        . " VRFs with pve-firewall, use the nftables firewall instead\n";
+}
+
 my $bond_mode_enum = [
     'balance-rr',
     'active-backup', # OVS and Linux
@@ -985,7 +999,10 @@ __PACKAGE__->register_method({
             };
             PVE::Tools::run_command(['ifreload', '-a'], errfunc => $err);
 
-            apply_vrf_ct_zones() if $have_sdn;
+            if ($have_sdn) {
+                apply_vrf_ct_zones();
+                eval { warn_vrf_zones_with_legacy_firewall() };
+            }
 
             # needs the zone VRFs created by ifreload, because dnsmasq is started inside
             # them with 'ip vrf exec'
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH pve-docs 14/14] pvesdn: simple zones: document dedicated VRFs
  2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
                   ` (12 preceding siblings ...)
  2026-11-19 19:45 ` [PATCH pve-manager 13/14] network: warn if pve-firewall cannot isolate guests in " Stefan Hanreich
@ 2026-11-19 19:45 ` Stefan Hanreich
  13 siblings, 0 replies; 15+ messages in thread
From: Stefan Hanreich @ 2026-11-19 19:45 UTC (permalink / raw)
  To: pve-devel

Describe how traffic of a simple zone with a dedicated VRF can leave the
zone, which interfaces belong to the VRF, and what this means for DHCP,
SNAT and the firewall.

Document the conntrack zones reserved for Proxmox VE, so that admins
do not use them in their own rules, that proxmox-firewall needs to be
installed to set them, and how to apply them again after changing the
interfaces of a VRF outside of the network API.

Also document the limitations that admins need to be aware of: no
isolation of guests with pve-firewall, firewall rules and SDN IPSets
matching addresses only, and the need to block access to infrastructure
addresses in the VRF themselves. For SNAT, every host interface in the
VRF acts as uplink, guests appear with the address of the node, so
access lists of an upstream router cannot tell them apart, and packets
with a forged source address from the subnet are masqueraded unless
reverse path filtering or the firewall drops them.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 pvesdn.adoc | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 100 insertions(+)

diff --git a/pvesdn.adoc b/pvesdn.adoc
index b701636..0c7ebb4 100644
--- a/pvesdn.adoc
+++ b/pvesdn.adoc
@@ -239,6 +239,106 @@ bridge is not linked to a physical interface, and VM traffic is only local on
 each the node.
 It can be used in NAT or routed setups.
 
+Simple zone configuration options:
+
+Dedicated VRF:: Put the VNets of the zone into a VRF of their own, see
+  xref:pvesdn_zone_plugin_simple_vrf[Dedicated VRF]. Optional, disabled by
+  default.
+
+[[pvesdn_zone_plugin_simple_vrf]]
+Dedicated VRF
+^^^^^^^^^^^^^
+
+With a dedicated VRF, the VNets of a simple zone are put into the VRF
+`vrf_<zone>`, which has a routing table of its own. Traffic of the zone is
+routed separately from the routing table of the host and from other zones, so
+the subnets of different zones with a dedicated VRF may overlap.
+
+Traffic can only leave the zone through interfaces in its VRF:
+
+* Host interfaces that the network configuration of the node puts into the VRF,
+  for example a VLAN interface to an upstream router.
+* Interfaces of xref:pvesdn_config_fabrics[fabrics] in the zone.
+
+Without any such interface, the zone is isolated. Services of the host, such as
+SSH or the web interface, cannot be reached from the zone, as long as the
+`net.ipv4.tcp_l3mdev_accept` and `net.ipv4.udp_l3mdev_accept` sysctls are not
+enabled.
+
+DHCP:: The DHCP server of the zone runs in its VRF. Guests use it as DNS server
+  by default, which needs a route to the upstream DNS servers through an
+  interface in the VRF. Otherwise, set a DNS server reachable from the zone with
+  the `dhcp-dns-server` option of the subnet.
+
+SNAT:: Subnets with SNAT enabled are masqueraded on every host interface in the
+  VRF. Only interfaces configured in `/etc/network/interfaces` itself are taken
+  into account, not the ones in files included from it. Interfaces of fabrics
+  are not used for SNAT. If routing balances connections across several such
+  interfaces, connections of a guest leave with different source addresses.
+  Connections are dropped if the interface goes down, even briefly.
++
+Every such interface acts as an uplink: traffic of the subnet routed through it
+carries the address of the node, also towards networks that should see the
+addresses of the guests. Only put interfaces into the VRF that should be used
+as uplinks.
++
+SNAT does not protect guests from incoming connections, use the firewall for
+that. Packets that arrive on an uplink with a source address from the subnet
+are masqueraded as well if they are routed out through an uplink. To drop them,
+enable strict reverse path filtering for IPv4 on the uplinks, by setting the
+`net.ipv4.conf.<interface>.rp_filter` sysctl to `1`, or drop such packets with
+the firewall. Strict reverse path filtering also drops replies that arrive on
+another uplink than the one routing would use to reach their source.
+
+Connection tracking:: Each zone with a dedicated VRF gets a conntrack zone, so
+  that connections of different VRFs never share a conntrack entry. It is
+  assigned automatically from the range 16384 to 17407 and shown as `ct-zone`
+  of the zone. {pve} reserves the conntrack zones 16384 to 32767, do not use
+  them in your own rules.
++
+The conntrack zone is set for the VRF device and all interfaces in the VRF by
+the `proxmox-firewall` package, which needs to be installed, even if the
+nftables-based firewall is not used. The rules are loaded into the nftables
+table `inet proxmox-sdn-ct-zones` when the network configuration is applied
+and at boot. If the interfaces in a VRF are changed in another way, for example
+by running `ifreload -a` manually, or if the nftables ruleset is flushed, for
+example by enabling `nftables.service` with its default configuration, apply
+the network configuration again.
++
+After a zone is deleted, a new zone may reuse its conntrack zone. Remaining
+connection tracking entries of the deleted zone expire after their timeout.
+
+Firewall:: Only the xref:pve_firewall_nft[nftables-based firewall] isolates the
+  connections of guests between zone VRFs. With `pve-firewall`, guests with the
+  firewall enabled on a network device share their connection tracking state
+  across all VRFs, and applying the configuration warns about this. Conntrack
+  helpers of `pve-firewall` do not apply in zone VRFs.
++
+Firewall rules and the IPSets generated for VNets match addresses only. With
+overlapping subnets, a rule applies to every zone with a matching subnet. For
+traffic of a zone VRF, the input interface seen by the host firewall is the VRF
+device `vrf_<zone>`.
+
+Infrastructure addresses in the VRF, such as the addresses of fabric interfaces
+and nodes or the address of an upstream router, are reachable from guests in the
+zone. To block them with the nftables-based firewall, add a datacenter IPSet,
+for example `vrf-infra`, with these addresses and drop traffic from the subnets
+of the zone to them:
+
+----
+# /etc/pve/firewall/cluster.fw
+
+[RULES]
+
+FORWARD DROP -source +sdn/vnet0-all -dest +dc/vrf-infra
+IN DROP -source +sdn/vnet0-all -dest +dc/vrf-infra
+----
+
+An upstream router should additionally protect its own addresses with access
+lists on its interface to the zone. This does not work for subnets with SNAT, as
+the router only sees the address of the node for their traffic, so only the
+firewall of the node can block it.
+
 
 [[pvesdn_zone_plugin_vlan]]
 VLAN Zones
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2026-11-19 19:48 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox 01/14] network-api: expose the master device of a link Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-ve-rs 02/14] sdn: config: read the conntrack zone of zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-ve-rs 03/14] sdn: conntrack: map the interfaces of zone VRFs to their conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-network 04/14] sdn: simple: SNAT via the uplinks of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-network 05/14] sdn: zones: assign a conntrack zone to zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 06/14] firewall: apply vnet rules to host traffic from vnets in a VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 07/14] nftables: expression: add a constructor for map lookups Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 08/14] nftables: statement: add a helper for setting the conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 09/14] nftables: expression: convert the conntrack zones of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 10/14] firewall: add a command applying " Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 11/14] firewall: put guest traffic of zone VRFs into their conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-manager 12/14] network: apply the conntrack zones of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-manager 13/14] network: warn if pve-firewall cannot isolate guests in " Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-docs 14/14] pvesdn: simple zones: document dedicated VRFs Stefan Hanreich

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal