From: Stefan Hanreich <s.hanreich@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH proxmox-firewall 10/14] firewall: add a command applying the conntrack zones of zone VRFs
Date: Thu, 19 Nov 2026 20:45:42 +0100 [thread overview]
Message-ID: <20261119194547.674512-11-s.hanreich@proxmox.com> (raw)
In-Reply-To: <20261119194547.674512-1-s.hanreich@proxmox.com>
The new command sdn-ct-zones sets the conntrack zone of a VRF for the
VRF device and all interfaces enslaved to it, as reported by the
kernel. Interfaces end up in a zone VRF via vnets, fabrics and
/etc/network/interfaces, and the kernel state reflects all of them once
the network configuration got applied. The command is therefore meant
for internal use by pve-manager after applying the network
configuration, which the help text points out. With --print, the rules
are printed instead of applied.
NAT relies on conntrack independent of the firewall, so the rules live
in a table of their own, which neither the firewall service nor its
removal touches.
The zone is set in prerouting and, for locally generated traffic, in
output. Both chains run before conntrack and before the iptables raw
table, so the zone takes effect independent of the order in which the
rules were loaded. The VRF device needs an entry as well, as received
packets are looked up a second time when passing it, and locally
generated packets are tracked when passing output on it.
The table is replaced atomically and removed if there is no zone VRF
with a conntrack zone. If the SDN running configuration cannot be read
or parsed, the command fails and keeps the existing rules.
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
proxmox-firewall/src/bin/proxmox-firewall.rs | 30 ++-
proxmox-firewall/src/config.rs | 15 +-
proxmox-firewall/src/lib.rs | 1 +
proxmox-firewall/src/sdn.rs | 235 +++++++++++++++++++
4 files changed, 273 insertions(+), 8 deletions(-)
create mode 100644 proxmox-firewall/src/sdn.rs
diff --git a/proxmox-firewall/src/bin/proxmox-firewall.rs b/proxmox-firewall/src/bin/proxmox-firewall.rs
index e8ed477..41ab2d3 100644
--- a/proxmox-firewall/src/bin/proxmox-firewall.rs
+++ b/proxmox-firewall/src/bin/proxmox-firewall.rs
@@ -5,10 +5,13 @@ use std::time::{Duration, Instant};
use anyhow::{Context, Error, bail, format_err};
use pico_args::Arguments;
+use proxmox_firewall::config::FirewallSdnConfig;
use proxmox_firewall::config::{FirewallConfig, PveFirewallConfigLoader, PveNftConfigLoader};
use proxmox_firewall::firewall::Firewall;
+use proxmox_firewall::sdn::{load_sdn_config, vrf_ct_zones, vrf_ct_zones_commands};
use proxmox_log as log;
use proxmox_log::{LevelFilter, Logger};
+use proxmox_network_api::get_network_interfaces;
use proxmox_nftables::{NftClient, client::NftError};
use proxmox_ve_config::firewall::host::Config as HostConfig;
@@ -22,6 +25,9 @@ COMMANDS:
compile Compile and print firewall rules as accepted by 'nft -j -f -'
start Execute proxmox-firewall service in foreground
localnet Print the contents of the management ipset
+ sdn-ct-zones Apply the conntrack zones of SDN zone VRFs (internal, run by
+ pve-manager when applying the network configuration)
+ --print: Print the rules as accepted by 'nft -j -f -' instead
"#;
const RULE_BASE: &str = include_str!("../../resources/proxmox-firewall.nft");
@@ -72,6 +78,23 @@ fn handle_firewall() -> Result<(), Error> {
Ok(())
}
+fn handle_sdn_ct_zones(print: bool) -> Result<(), Error> {
+ let sdn = load_sdn_config(&PveFirewallConfigLoader::new())?;
+ let links = get_network_interfaces()?.into_values();
+
+ let ct_zones = vrf_ct_zones(sdn.as_ref().map(FirewallSdnConfig::config), links);
+ let commands = vrf_ct_zones_commands(&ct_zones);
+
+ if print {
+ println!("{}", serde_json::to_string_pretty(&commands)?);
+ } else {
+ NftClient::run_json_commands(&commands)
+ .context("could not apply the conntrack zones of SDN zone VRFs")?;
+ }
+
+ Ok(())
+}
+
fn init_logger(command: Command) -> Result<(), Error> {
let mut logger = Logger::from_env("PVE_LOG", LevelFilter::WARN);
@@ -125,6 +148,7 @@ pub enum Command {
Skeleton,
Start,
Localnet,
+ SdnCtZones,
}
impl std::str::FromStr for Command {
@@ -137,6 +161,7 @@ impl std::str::FromStr for Command {
"skeleton" => Command::Skeleton,
"start" => Command::Start,
"localnet" => Command::Localnet,
+ "sdn-ct-zones" => Command::SdnCtZones,
cmd => {
bail!("{cmd} is not a valid command")
}
@@ -144,7 +169,7 @@ impl std::str::FromStr for Command {
}
}
-fn run_command(command: Command) -> Result<(), Error> {
+fn run_command(command: Command, args: &mut Arguments) -> Result<(), Error> {
init_logger(command)?;
match command {
@@ -169,6 +194,7 @@ fn run_command(command: Command) -> Result<(), Error> {
println!("{ip}");
}
}
+ Command::SdnCtZones => handle_sdn_ct_zones(args.contains("--print"))?,
};
Ok(())
@@ -183,7 +209,7 @@ fn main() -> Result<(), Error> {
.parse();
if let Ok(command) = parsed_command {
- run_command(command)
+ run_command(command, &mut args)
} else {
eprintln!("Invalid command specified!\n{}", HELP);
std::process::exit(1);
diff --git a/proxmox-firewall/src/config.rs b/proxmox-firewall/src/config.rs
index acc4b56..7e055c0 100644
--- a/proxmox-firewall/src/config.rs
+++ b/proxmox-firewall/src/config.rs
@@ -115,7 +115,9 @@ fn bridge_name_from_fw_name(name: &str) -> Result<BridgeName, Error> {
///
/// Reading the content before parsing it separates read errors, which are most likely transient,
/// from errors in the content.
-fn read_config(data: Option<Box<dyn io::BufRead>>) -> Result<Option<Vec<u8>>, io::Error> {
+pub(crate) fn read_config(
+ data: Option<Box<dyn io::BufRead>>,
+) -> Result<Option<Vec<u8>>, io::Error> {
data.map(|mut data| {
let mut content = Vec::new();
data.read_to_end(&mut content)?;
@@ -308,11 +310,15 @@ impl NftConfigLoader for PveNftConfigLoader {
}
pub struct FirewallSdnConfig {
- _config: SdnConfig,
+ config: SdnConfig,
ipsets: BTreeMap<String, Ipset>,
}
impl FirewallSdnConfig {
+ pub fn config(&self) -> &SdnConfig {
+ &self.config
+ }
+
pub fn ipsets(&self) -> &BTreeMap<String, Ipset> {
&self.ipsets
}
@@ -416,10 +422,7 @@ impl FirewallConfig {
.map(|ipset| (ipset.name().name().to_string(), ipset))
.collect();
- Some(FirewallSdnConfig {
- _config: config,
- ipsets,
- })
+ Some(FirewallSdnConfig { config, ipsets })
}
_ => None,
})
diff --git a/proxmox-firewall/src/lib.rs b/proxmox-firewall/src/lib.rs
index c4b037a..25db727 100644
--- a/proxmox-firewall/src/lib.rs
+++ b/proxmox-firewall/src/lib.rs
@@ -2,3 +2,4 @@ pub mod config;
pub mod firewall;
pub mod object;
pub mod rule;
+pub mod sdn;
diff --git a/proxmox-firewall/src/sdn.rs b/proxmox-firewall/src/sdn.rs
new file mode 100644
index 0000000..ded5740
--- /dev/null
+++ b/proxmox-firewall/src/sdn.rs
@@ -0,0 +1,235 @@
+//! Conntrack zones of SDN zone VRFs.
+//!
+//! Sets the conntrack zone of each zone VRF for the VRF device and the interfaces in the VRF. NAT
+//! relies on conntrack as well, so the rules are independent of the firewall and live in a table of
+//! their own.
+
+use anyhow::{Context, Error};
+
+use proxmox_network_api::IpLink;
+use proxmox_nftables::command::{Add, Commands, Delete};
+use proxmox_nftables::expression::{Map, Meta};
+use proxmox_nftables::statement::Mangle;
+use proxmox_nftables::types::{
+ AddChain, AddRule, AddTable, BaseChainConfig, ChainPart, ChainPolicy, ChainType, Hook,
+ TableFamily, TablePart,
+};
+use proxmox_nftables::{Expression, Statement};
+use proxmox_ve_config::sdn::config::SdnConfig;
+use proxmox_ve_config::sdn::conntrack::VrfCtZones;
+
+use crate::config::{FirewallConfig, FirewallConfigLoader, FirewallSdnConfig, read_config};
+
+pub const VRF_CT_ZONES_TABLE: &str = "proxmox-sdn-ct-zones";
+
+// before the iptables raw table (-300) and conntrack (-200), so that the zone takes effect
+// regardless of the order in which the tables were loaded
+const VRF_CT_ZONES_PRIORITY: i64 = -310;
+
+/// Loads the SDN running configuration.
+///
+/// Fails if the configuration cannot be read or parsed, so that the existing rules are kept
+/// instead of being removed.
+pub fn load_sdn_config(
+ loader: &dyn FirewallConfigLoader,
+) -> Result<Option<FirewallSdnConfig>, Error> {
+ loader.check_mounted()?;
+
+ let data = read_config(loader.sdn_running_config()?)
+ .context("failed to read the SDN running configuration")?;
+
+ FirewallConfig::parse_sdn(data.as_deref())
+}
+
+/// Collects the zone VRFs with a conntrack zone and the interfaces that are enslaved to them.
+///
+/// The members are taken from the kernel, which reflects all sources that put an interface into a
+/// VRF once the network configuration got applied.
+pub fn vrf_ct_zones(
+ sdn: Option<&SdnConfig>,
+ links: impl IntoIterator<Item = IpLink>,
+) -> VrfCtZones {
+ let mut ct_zones = sdn.map(VrfCtZones::from).unwrap_or_default();
+
+ for link in links {
+ if let Some(master) = link.master() {
+ ct_zones.add_interface(master, link.name());
+ }
+ }
+
+ ct_zones
+}
+
+/// Generates the commands that atomically replace the table for the conntrack zones.
+///
+/// Without any zone VRF, the commands only remove the table.
+pub fn vrf_ct_zones_commands(ct_zones: &VrfCtZones) -> Commands {
+ let table = TablePart::new(TableFamily::Inet, VRF_CT_ZONES_TABLE);
+
+ let mut commands = Commands::new(vec![
+ Add::table(AddTable::new(TableFamily::Inet, VRF_CT_ZONES_TABLE)),
+ Delete::table(table.clone()),
+ ]);
+
+ if ct_zones.is_empty() {
+ return commands;
+ }
+
+ commands.push(Add::table(AddTable::new(
+ TableFamily::Inet,
+ VRF_CT_ZONES_TABLE,
+ )));
+
+ // received packets pass prerouting on the receiving interface and on the VRF device, locally
+ // generated ones pass output on the VRF device and on the outgoing interface
+ for (hook, key) in [(Hook::Prerouting, "iifname"), (Hook::Output, "oifname")] {
+ let chain = ChainPart::new(table.clone(), hook.to_string());
+
+ commands.push(Add::chain(AddChain::new_base_chain(
+ table.clone(),
+ chain.name(),
+ BaseChainConfig::new(
+ ChainType::Filter,
+ hook,
+ VRF_CT_ZONES_PRIORITY,
+ ChainPolicy::Accept,
+ ),
+ )));
+
+ commands.push(Add::rule(AddRule::from_statement(
+ chain,
+ Statement::from(Mangle::ct_zone(Map::new(
+ Meta::new(key),
+ Expression::from(ct_zones),
+ ))),
+ )));
+ }
+
+ commands
+}
+
+#[cfg(test)]
+mod tests {
+ use proxmox_ve_config::sdn::config::RunningConfig;
+ use serde_json::json;
+
+ use super::*;
+
+ fn link(name: &str, master: Option<&str>) -> IpLink {
+ serde_json::from_value(json!({
+ "ifindex": 1,
+ "ifname": name,
+ "link_type": "ether",
+ "operstate": "UP",
+ "master": master,
+ }))
+ .unwrap()
+ }
+
+ fn sdn_config() -> SdnConfig {
+ let running_config: RunningConfig = serde_json::from_value(json!({
+ "zones": {
+ "ids": {
+ "red": { "type": "simple", "dedicated-vrf": 1, "ct-zone": 16384 },
+ "blue": { "type": "simple", "dedicated-vrf": 1, "ct-zone": "16389" },
+ "green": { "type": "simple" },
+ },
+ },
+ }))
+ .unwrap();
+
+ SdnConfig::try_from(running_config).unwrap()
+ }
+
+ #[test]
+ fn vrf_ct_zones_from_links() {
+ let links = [
+ link("vrf_red", None),
+ link("vnetr1", Some("vrf_red")),
+ link("bond0.101", Some("vrf_red")),
+ link("vnetb", Some("vrf_blue")),
+ link("ens18", Some("vmbr0")),
+ link("eth5", Some("vrf_other")),
+ link("vnetg", Some("vrf_green")),
+ link("lo", None),
+ ];
+
+ let interfaces: Vec<_> = vrf_ct_zones(Some(&sdn_config()), links)
+ .interfaces()
+ .map(|(interface, ct_zone)| (interface.to_string(), ct_zone.id()))
+ .collect();
+
+ assert_eq!(
+ interfaces,
+ [
+ ("vrf_blue".to_string(), 16389),
+ ("vnetb".to_string(), 16389),
+ ("vrf_red".to_string(), 16384),
+ ("bond0.101".to_string(), 16384),
+ ("vnetr1".to_string(), 16384),
+ ]
+ );
+
+ assert!(vrf_ct_zones(None, [link("vnetr1", Some("vrf_red"))]).is_empty());
+ }
+
+ #[test]
+ fn vrf_ct_zones_commands_without_zones() {
+ let commands = vrf_ct_zones_commands(&VrfCtZones::default());
+
+ assert_eq!(
+ serde_json::to_value(&commands).unwrap(),
+ json!({"nftables": [
+ {"add": {"table": {"family": "inet", "name": "proxmox-sdn-ct-zones"}}},
+ {"delete": {"table": {"family": "inet", "name": "proxmox-sdn-ct-zones"}}},
+ ]})
+ );
+ }
+
+ #[test]
+ fn vrf_ct_zones_commands_with_zones() {
+ let ct_zones = vrf_ct_zones(Some(&sdn_config()), [link("vnetb", Some("vrf_blue"))]);
+
+ let elements = json!({"set": [
+ ["vrf_blue", 16389],
+ ["vnetb", 16389],
+ ["vrf_red", 16384],
+ ]});
+
+ let chain = |hook: &str, key: &str| {
+ [
+ json!({"add": {"chain": {
+ "family": "inet",
+ "table": "proxmox-sdn-ct-zones",
+ "name": hook,
+ "type": "filter",
+ "hook": hook,
+ "prio": -310,
+ "policy": "accept",
+ }}}),
+ json!({"add": {"rule": {
+ "family": "inet",
+ "table": "proxmox-sdn-ct-zones",
+ "chain": hook,
+ "expr": [{"mangle": {
+ "key": {"ct": {"key": "zone"}},
+ "value": {"map": {"key": {"meta": {"key": key}}, "data": elements}},
+ }}],
+ }}}),
+ ]
+ };
+
+ let mut expected = vec![
+ json!({"add": {"table": {"family": "inet", "name": "proxmox-sdn-ct-zones"}}}),
+ json!({"delete": {"table": {"family": "inet", "name": "proxmox-sdn-ct-zones"}}}),
+ json!({"add": {"table": {"family": "inet", "name": "proxmox-sdn-ct-zones"}}}),
+ ];
+ expected.extend(chain("prerouting", "iifname"));
+ expected.extend(chain("output", "oifname"));
+
+ assert_eq!(
+ serde_json::to_value(vrf_ct_zones_commands(&ct_zones)).unwrap(),
+ json!({ "nftables": expected })
+ );
+ }
+}
--
2.47.3
next prev parent reply other threads:[~2026-11-19 19:48 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox 01/14] network-api: expose the master device of a link Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-ve-rs 02/14] sdn: config: read the conntrack zone of zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-ve-rs 03/14] sdn: conntrack: map the interfaces of zone VRFs to their conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-network 04/14] sdn: simple: SNAT via the uplinks of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-network 05/14] sdn: zones: assign a conntrack zone to zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 06/14] firewall: apply vnet rules to host traffic from vnets in a VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 07/14] nftables: expression: add a constructor for map lookups Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 08/14] nftables: statement: add a helper for setting the conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 09/14] nftables: expression: convert the conntrack zones of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` Stefan Hanreich [this message]
2026-11-19 19:45 ` [PATCH proxmox-firewall 11/14] firewall: put guest traffic of zone VRFs into their conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-manager 12/14] network: apply the conntrack zones of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-manager 13/14] network: warn if pve-firewall cannot isolate guests in " Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-docs 14/14] pvesdn: simple zones: document dedicated VRFs Stefan Hanreich
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119194547.674512-11-s.hanreich@proxmox.com \
--to=s.hanreich@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox