From: Stefan Hanreich <s.hanreich@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH pve-network 05/14] sdn: zones: assign a conntrack zone to zones with a dedicated VRF
Date: Thu, 19 Nov 2026 20:45:37 +0100 [thread overview]
Message-ID: <20261119194547.674512-6-s.hanreich@proxmox.com> (raw)
In-Reply-To: <20261119194547.674512-1-s.hanreich@proxmox.com>
The subnets of zones with a dedicated VRF may overlap, so connections in
different VRFs must not share a conntrack entry, or the firewall would
treat a new connection in one VRF as part of an established one in
another VRF, and conntrack state migration would mix up the entries.
Assign each such zone a conntrack zone from the first 1024 zones (16384
to 17407) of the block 16384 to 32767 that Proxmox VE reserves, stored
with the zone as ct-zone, so that it is the same on all nodes and stays
the same as long as the zone has a dedicated VRF. It is removed when
the dedicated VRF is disabled, and cannot be set via the API.
A new conntrack zone follows the highest one in use, including the
running config, so the conntrack zone of a zone that is still active or
was just deleted is not reused right away, which could match stale
conntrack entries of that zone.
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
src/PVE/API2/Network/SDN/Zones.pm | 13 +++
src/PVE/Network/SDN/Zones.pm | 42 +++++++++
src/PVE/Network/SDN/Zones/Plugin.pm | 4 +
src/PVE/Network/SDN/Zones/SimplePlugin.pm | 7 ++
src/test/Makefile | 5 +-
src/test/run_test_ct_zones.pl | 103 ++++++++++++++++++++++
6 files changed, 173 insertions(+), 1 deletion(-)
create mode 100755 src/test/run_test_ct_zones.pl
diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm
index 5952f40..5ff7bb4 100644
--- a/src/PVE/API2/Network/SDN/Zones.pm
+++ b/src/PVE/API2/Network/SDN/Zones.pm
@@ -70,6 +70,11 @@ my $ZONE_PROPERTIES = {
optional => 1,
description => 'Create a dedicated VRF for a simple zone.',
},
+ 'ct-zone' => {
+ type => 'integer',
+ optional => 1,
+ description => 'Conntrack zone of the dedicated VRF, assigned automatically.',
+ },
route => {
type => 'array',
optional => 1,
@@ -407,6 +412,9 @@ __PACKAGE__->register_method({
my $id = extract_param($param, 'zone');
my $lock_token = extract_param($param, 'lock-token');
+ raise_param_exc({ 'ct-zone' => 'is assigned automatically' })
+ if defined($param->{'ct-zone'});
+
my $plugin = PVE::Network::SDN::Zones::Plugin->lookup($type);
my $opts = $plugin->check_config($id, $param, 1, 1);
@@ -434,6 +442,7 @@ __PACKAGE__->register_method({
if $dnszone && !$dnsserver;
$zone_cfg->{ids}->{$id} = $opts;
+ PVE::Network::SDN::Zones::update_ct_zone($zone_cfg, $id);
$plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
PVE::Network::SDN::Zones::write_config($zone_cfg);
@@ -478,6 +487,9 @@ __PACKAGE__->register_method({
$delete = [PVE::Tools::split_list($delete)];
}
+ raise_param_exc({ 'ct-zone' => 'is assigned automatically' })
+ if defined($param->{'ct-zone'}) || grep { $_ eq 'ct-zone' } ($delete // [])->@*;
+
PVE::Network::SDN::lock_sdn_config(
sub {
my $zone_cfg = PVE::Network::SDN::Zones::config();
@@ -541,6 +553,7 @@ __PACKAGE__->register_method({
my $dnszone = $opts->{dnszone};
raise_param_exc({ dnszone => "missing dns server" }) if $dnszone && !$dnsserver;
+ PVE::Network::SDN::Zones::update_ct_zone($zone_cfg, $id);
$plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
PVE::Network::SDN::Zones::write_config($zone_cfg);
diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm
index 3b59f4f..c8ff20b 100644
--- a/src/PVE/Network/SDN/Zones.pm
+++ b/src/PVE/Network/SDN/Zones.pm
@@ -59,6 +59,48 @@ sub config {
return cfs_read_file("sdn/zones.cfg");
}
+# Assigns a conntrack zone to a simple zone with a dedicated VRF and removes it from other zones.
+#
+# A new conntrack zone follows the highest one in use, also in the running config, so that the
+# conntrack zone of a zone that is still active or was just deleted is not reused right away.
+sub update_ct_zone {
+ my ($zone_cfg, $id) = @_;
+
+ my $zone = $zone_cfg->{ids}->{$id};
+
+ if ($zone->{type} ne 'simple' || !$zone->{'dedicated-vrf'}) {
+ delete $zone->{'ct-zone'};
+ return;
+ }
+
+ return if defined($zone->{'ct-zone'});
+
+ my $first = $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_FIRST;
+ my $last = $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_LAST;
+
+ my $running_zones = PVE::Network::SDN::Zones::config(1)->{ids};
+ my $used = {};
+ for my $zones ($zone_cfg->{ids}, $running_zones) {
+ for my $ct_zone (map { $_->{'ct-zone'} } values $zones->%*) {
+ $used->{$ct_zone} = 1 if defined($ct_zone) && $ct_zone >= $first && $ct_zone <= $last;
+ }
+ }
+
+ my $count = $last - $first + 1;
+ die "no conntrack zone left for the VRF of zone '$id'\n" if scalar(keys $used->%*) >= $count;
+
+ my ($highest) = sort { $b <=> $a } keys $used->%*;
+ $highest //= $last;
+
+ for my $offset (1 .. $count) {
+ my $ct_zone = $first + ($highest - $first + $offset) % $count;
+ if (!$used->{$ct_zone}) {
+ $zone->{'ct-zone'} = $ct_zone;
+ return;
+ }
+ }
+}
+
sub get_plugin_config {
my ($vnet) = @_;
my $zoneid = $vnet->{zone};
diff --git a/src/PVE/Network/SDN/Zones/Plugin.pm b/src/PVE/Network/SDN/Zones/Plugin.pm
index 288edfc..d92bfd1 100644
--- a/src/PVE/Network/SDN/Zones/Plugin.pm
+++ b/src/PVE/Network/SDN/Zones/Plugin.pm
@@ -31,6 +31,10 @@ PVE::JSONSchema::register_standard_option(
},
);
+# conntrack zones reserved for the VRFs of zones, see also VrfCtZone in proxmox-ve-config
+our $VRF_CT_ZONE_FIRST = 16384;
+our $VRF_CT_ZONE_LAST = 17407;
+
my $static_route_fmt = {
prefix => {
type => 'string',
diff --git a/src/PVE/Network/SDN/Zones/SimplePlugin.pm b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
index 4221521..3e866ac 100644
--- a/src/PVE/Network/SDN/Zones/SimplePlugin.pm
+++ b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
@@ -40,6 +40,12 @@ sub properties {
type => 'boolean',
default => 0,
},
+ 'ct-zone' => {
+ description => 'Conntrack zone of the dedicated VRF, assigned automatically.',
+ type => 'integer',
+ minimum => $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_FIRST,
+ maximum => $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_LAST,
+ },
};
}
@@ -53,6 +59,7 @@ sub options {
ipam => { optional => 1 },
dhcp => { optional => 1 },
'dedicated-vrf' => { optional => 1 },
+ 'ct-zone' => { optional => 1 },
route => { optional => 1 },
};
}
diff --git a/src/test/Makefile b/src/test/Makefile
index 0d5f3c1..242901b 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_zones test_static_routes test_fabric_host_vrf test_ipams test_dns test_subnets test_vnets_blackbox test_running_config test_api_index
+test: test_zones test_static_routes test_ct_zones test_fabric_host_vrf test_ipams test_dns test_subnets test_vnets_blackbox test_running_config test_api_index
test_zones: run_test_zones.pl
./run_test_zones.pl
@@ -11,6 +11,9 @@ test_static_routes: run_test_static_routes.pl
test_fabric_host_vrf: run_test_fabric_host_vrf.pl
./run_test_fabric_host_vrf.pl
+test_ct_zones: run_test_ct_zones.pl
+ ./run_test_ct_zones.pl
+
test_ipams: run_test_ipams.pl
./run_test_ipams.pl
diff --git a/src/test/run_test_ct_zones.pl b/src/test/run_test_ct_zones.pl
new file mode 100755
index 0000000..1b71522
--- /dev/null
+++ b/src/test/run_test_ct_zones.pl
@@ -0,0 +1,103 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use lib qw(..);
+
+use Test::More;
+use Test::MockModule;
+
+use PVE::API2::Network::SDN::Zones;
+use PVE::Network::SDN::Zones;
+
+my $running_zones = {};
+
+my $mocked_zones = Test::MockModule->new('PVE::Network::SDN::Zones');
+$mocked_zones->mock(
+ config => sub {
+ my ($running) = @_;
+ die "unexpected read of zones.cfg\n" if !$running;
+ return { ids => $running_zones };
+ },
+);
+
+sub vrf_zone {
+ my ($ct_zone) = @_;
+ my $zone = { type => 'simple', 'dedicated-vrf' => 1 };
+ $zone->{'ct-zone'} = $ct_zone if defined($ct_zone);
+ return $zone;
+}
+
+sub ct_zone_of {
+ my ($zones, $id) = @_;
+ my $zone_cfg = { ids => $zones };
+ PVE::Network::SDN::Zones::update_ct_zone($zone_cfg, $id);
+ return $zone_cfg->{ids}->{$id}->{'ct-zone'};
+}
+
+is(ct_zone_of({ new => vrf_zone() }, 'new'), 16384, 'first zone VRF gets the first conntrack zone');
+
+is(
+ ct_zone_of({ a => vrf_zone(16384), b => vrf_zone(16390), new => vrf_zone() }, 'new'),
+ 16391,
+ 'new conntrack zone follows the highest one in use',
+);
+
+is(ct_zone_of({ a => vrf_zone(16500) }, 'a'), 16500, 'existing conntrack zone is kept');
+
+is(
+ ct_zone_of({ a => { type => 'simple', 'ct-zone' => 16400 } }, 'a'),
+ undef,
+ 'conntrack zone is removed without a dedicated VRF',
+);
+
+is(
+ ct_zone_of({ a => { type => 'evpn', 'dedicated-vrf' => 1, 'ct-zone' => 16400 } }, 'a'),
+ undef,
+ 'only simple zones get a conntrack zone',
+);
+
+$running_zones = { old => vrf_zone(16395) };
+is(
+ ct_zone_of({ a => vrf_zone(16384), new => vrf_zone() }, 'new'),
+ 16396,
+ 'conntrack zone of a zone that is only in the running config is not reused',
+);
+$running_zones = {};
+
+is(
+ ct_zone_of({ a => vrf_zone(17407), b => vrf_zone(16384), new => vrf_zone() }, 'new'),
+ 16385,
+ 'allocation wraps around after the last conntrack zone',
+);
+
+is(
+ ct_zone_of({ a => { type => 'vlan', 'ct-zone' => 17000 }, new => vrf_zone() }, 'new'),
+ 17001,
+ 'conntrack zones of other zones are taken into account',
+);
+
+is(
+ ct_zone_of({ a => vrf_zone(5), new => vrf_zone() }, 'new'),
+ 16384,
+ 'conntrack zones out of range are ignored',
+);
+
+my $all = { map { ("z$_" => vrf_zone($_)) } 16384 .. 17407 };
+$all->{new} = vrf_zone();
+eval { ct_zone_of($all, 'new') };
+like($@, qr/no conntrack zone left/, 'allocation fails if all conntrack zones are in use');
+
+for my $method ('create', 'update') {
+ my $param = { zone => 'zone0', 'ct-zone' => 16384 };
+ $param->{type} = 'simple' if $method eq 'create';
+
+ eval { PVE::API2::Network::SDN::Zones->$method($param) };
+ like($@, qr/ct-zone.*assigned automatically/s, "$method rejects setting ct-zone");
+}
+
+eval { PVE::API2::Network::SDN::Zones->update({ zone => 'zone0', delete => 'ct-zone' }) };
+like($@, qr/ct-zone.*assigned automatically/s, 'update rejects deleting ct-zone');
+
+done_testing();
--
2.47.3
next prev parent reply other threads:[~2026-11-19 19:47 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox 01/14] network-api: expose the master device of a link Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-ve-rs 02/14] sdn: config: read the conntrack zone of zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-ve-rs 03/14] sdn: conntrack: map the interfaces of zone VRFs to their conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-network 04/14] sdn: simple: SNAT via the uplinks of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` Stefan Hanreich [this message]
2026-11-19 19:45 ` [PATCH proxmox-firewall 06/14] firewall: apply vnet rules to host traffic from vnets in a VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 07/14] nftables: expression: add a constructor for map lookups Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 08/14] nftables: statement: add a helper for setting the conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 09/14] nftables: expression: convert the conntrack zones of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 10/14] firewall: add a command applying " Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 11/14] firewall: put guest traffic of zone VRFs into their conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-manager 12/14] network: apply the conntrack zones of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-manager 13/14] network: warn if pve-firewall cannot isolate guests in " Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-docs 14/14] pvesdn: simple zones: document dedicated VRFs Stefan Hanreich
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119194547.674512-6-s.hanreich@proxmox.com \
--to=s.hanreich@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox