all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Stefan Hanreich <s.hanreich@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH pve-network 05/14] sdn: zones: assign a conntrack zone to zones with a dedicated VRF
Date: Thu, 19 Nov 2026 20:45:37 +0100	[thread overview]
Message-ID: <20261119194547.674512-6-s.hanreich@proxmox.com> (raw)
In-Reply-To: <20261119194547.674512-1-s.hanreich@proxmox.com>

The subnets of zones with a dedicated VRF may overlap, so connections in
different VRFs must not share a conntrack entry, or the firewall would
treat a new connection in one VRF as part of an established one in
another VRF, and conntrack state migration would mix up the entries.

Assign each such zone a conntrack zone from the first 1024 zones (16384
to 17407) of the block 16384 to 32767 that Proxmox VE reserves, stored
with the zone as ct-zone, so that it is the same on all nodes and stays
the same as long as the zone has a dedicated VRF. It is removed when
the dedicated VRF is disabled, and cannot be set via the API.

A new conntrack zone follows the highest one in use, including the
running config, so the conntrack zone of a zone that is still active or
was just deleted is not reused right away, which could match stale
conntrack entries of that zone.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 src/PVE/API2/Network/SDN/Zones.pm         |  13 +++
 src/PVE/Network/SDN/Zones.pm              |  42 +++++++++
 src/PVE/Network/SDN/Zones/Plugin.pm       |   4 +
 src/PVE/Network/SDN/Zones/SimplePlugin.pm |   7 ++
 src/test/Makefile                         |   5 +-
 src/test/run_test_ct_zones.pl             | 103 ++++++++++++++++++++++
 6 files changed, 173 insertions(+), 1 deletion(-)
 create mode 100755 src/test/run_test_ct_zones.pl

diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm
index 5952f40..5ff7bb4 100644
--- a/src/PVE/API2/Network/SDN/Zones.pm
+++ b/src/PVE/API2/Network/SDN/Zones.pm
@@ -70,6 +70,11 @@ my $ZONE_PROPERTIES = {
         optional => 1,
         description => 'Create a dedicated VRF for a simple zone.',
     },
+    'ct-zone' => {
+        type => 'integer',
+        optional => 1,
+        description => 'Conntrack zone of the dedicated VRF, assigned automatically.',
+    },
     route => {
         type => 'array',
         optional => 1,
@@ -407,6 +412,9 @@ __PACKAGE__->register_method({
         my $id = extract_param($param, 'zone');
         my $lock_token = extract_param($param, 'lock-token');
 
+        raise_param_exc({ 'ct-zone' => 'is assigned automatically' })
+            if defined($param->{'ct-zone'});
+
         my $plugin = PVE::Network::SDN::Zones::Plugin->lookup($type);
         my $opts = $plugin->check_config($id, $param, 1, 1);
 
@@ -434,6 +442,7 @@ __PACKAGE__->register_method({
                     if $dnszone && !$dnsserver;
 
                 $zone_cfg->{ids}->{$id} = $opts;
+                PVE::Network::SDN::Zones::update_ct_zone($zone_cfg, $id);
                 $plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
 
                 PVE::Network::SDN::Zones::write_config($zone_cfg);
@@ -478,6 +487,9 @@ __PACKAGE__->register_method({
             $delete = [PVE::Tools::split_list($delete)];
         }
 
+        raise_param_exc({ 'ct-zone' => 'is assigned automatically' })
+            if defined($param->{'ct-zone'}) || grep { $_ eq 'ct-zone' } ($delete // [])->@*;
+
         PVE::Network::SDN::lock_sdn_config(
             sub {
                 my $zone_cfg = PVE::Network::SDN::Zones::config();
@@ -541,6 +553,7 @@ __PACKAGE__->register_method({
                 my $dnszone = $opts->{dnszone};
                 raise_param_exc({ dnszone => "missing dns server" }) if $dnszone && !$dnsserver;
 
+                PVE::Network::SDN::Zones::update_ct_zone($zone_cfg, $id);
                 $plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
 
                 PVE::Network::SDN::Zones::write_config($zone_cfg);
diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm
index 3b59f4f..c8ff20b 100644
--- a/src/PVE/Network/SDN/Zones.pm
+++ b/src/PVE/Network/SDN/Zones.pm
@@ -59,6 +59,48 @@ sub config {
     return cfs_read_file("sdn/zones.cfg");
 }
 
+# Assigns a conntrack zone to a simple zone with a dedicated VRF and removes it from other zones.
+#
+# A new conntrack zone follows the highest one in use, also in the running config, so that the
+# conntrack zone of a zone that is still active or was just deleted is not reused right away.
+sub update_ct_zone {
+    my ($zone_cfg, $id) = @_;
+
+    my $zone = $zone_cfg->{ids}->{$id};
+
+    if ($zone->{type} ne 'simple' || !$zone->{'dedicated-vrf'}) {
+        delete $zone->{'ct-zone'};
+        return;
+    }
+
+    return if defined($zone->{'ct-zone'});
+
+    my $first = $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_FIRST;
+    my $last = $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_LAST;
+
+    my $running_zones = PVE::Network::SDN::Zones::config(1)->{ids};
+    my $used = {};
+    for my $zones ($zone_cfg->{ids}, $running_zones) {
+        for my $ct_zone (map { $_->{'ct-zone'} } values $zones->%*) {
+            $used->{$ct_zone} = 1 if defined($ct_zone) && $ct_zone >= $first && $ct_zone <= $last;
+        }
+    }
+
+    my $count = $last - $first + 1;
+    die "no conntrack zone left for the VRF of zone '$id'\n" if scalar(keys $used->%*) >= $count;
+
+    my ($highest) = sort { $b <=> $a } keys $used->%*;
+    $highest //= $last;
+
+    for my $offset (1 .. $count) {
+        my $ct_zone = $first + ($highest - $first + $offset) % $count;
+        if (!$used->{$ct_zone}) {
+            $zone->{'ct-zone'} = $ct_zone;
+            return;
+        }
+    }
+}
+
 sub get_plugin_config {
     my ($vnet) = @_;
     my $zoneid = $vnet->{zone};
diff --git a/src/PVE/Network/SDN/Zones/Plugin.pm b/src/PVE/Network/SDN/Zones/Plugin.pm
index 288edfc..d92bfd1 100644
--- a/src/PVE/Network/SDN/Zones/Plugin.pm
+++ b/src/PVE/Network/SDN/Zones/Plugin.pm
@@ -31,6 +31,10 @@ PVE::JSONSchema::register_standard_option(
     },
 );
 
+# conntrack zones reserved for the VRFs of zones, see also VrfCtZone in proxmox-ve-config
+our $VRF_CT_ZONE_FIRST = 16384;
+our $VRF_CT_ZONE_LAST = 17407;
+
 my $static_route_fmt = {
     prefix => {
         type => 'string',
diff --git a/src/PVE/Network/SDN/Zones/SimplePlugin.pm b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
index 4221521..3e866ac 100644
--- a/src/PVE/Network/SDN/Zones/SimplePlugin.pm
+++ b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
@@ -40,6 +40,12 @@ sub properties {
             type => 'boolean',
             default => 0,
         },
+        'ct-zone' => {
+            description => 'Conntrack zone of the dedicated VRF, assigned automatically.',
+            type => 'integer',
+            minimum => $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_FIRST,
+            maximum => $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_LAST,
+        },
     };
 }
 
@@ -53,6 +59,7 @@ sub options {
         ipam => { optional => 1 },
         dhcp => { optional => 1 },
         'dedicated-vrf' => { optional => 1 },
+        'ct-zone' => { optional => 1 },
         route => { optional => 1 },
     };
 }
diff --git a/src/test/Makefile b/src/test/Makefile
index 0d5f3c1..242901b 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
 all: test
 
-test: test_zones test_static_routes test_fabric_host_vrf test_ipams test_dns test_subnets test_vnets_blackbox test_running_config test_api_index
+test: test_zones test_static_routes test_ct_zones test_fabric_host_vrf test_ipams test_dns test_subnets test_vnets_blackbox test_running_config test_api_index
 
 test_zones: run_test_zones.pl
 	./run_test_zones.pl
@@ -11,6 +11,9 @@ test_static_routes: run_test_static_routes.pl
 test_fabric_host_vrf: run_test_fabric_host_vrf.pl
 	./run_test_fabric_host_vrf.pl
 
+test_ct_zones: run_test_ct_zones.pl
+	./run_test_ct_zones.pl
+
 test_ipams: run_test_ipams.pl
 	./run_test_ipams.pl
 
diff --git a/src/test/run_test_ct_zones.pl b/src/test/run_test_ct_zones.pl
new file mode 100755
index 0000000..1b71522
--- /dev/null
+++ b/src/test/run_test_ct_zones.pl
@@ -0,0 +1,103 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use lib qw(..);
+
+use Test::More;
+use Test::MockModule;
+
+use PVE::API2::Network::SDN::Zones;
+use PVE::Network::SDN::Zones;
+
+my $running_zones = {};
+
+my $mocked_zones = Test::MockModule->new('PVE::Network::SDN::Zones');
+$mocked_zones->mock(
+    config => sub {
+        my ($running) = @_;
+        die "unexpected read of zones.cfg\n" if !$running;
+        return { ids => $running_zones };
+    },
+);
+
+sub vrf_zone {
+    my ($ct_zone) = @_;
+    my $zone = { type => 'simple', 'dedicated-vrf' => 1 };
+    $zone->{'ct-zone'} = $ct_zone if defined($ct_zone);
+    return $zone;
+}
+
+sub ct_zone_of {
+    my ($zones, $id) = @_;
+    my $zone_cfg = { ids => $zones };
+    PVE::Network::SDN::Zones::update_ct_zone($zone_cfg, $id);
+    return $zone_cfg->{ids}->{$id}->{'ct-zone'};
+}
+
+is(ct_zone_of({ new => vrf_zone() }, 'new'), 16384, 'first zone VRF gets the first conntrack zone');
+
+is(
+    ct_zone_of({ a => vrf_zone(16384), b => vrf_zone(16390), new => vrf_zone() }, 'new'),
+    16391,
+    'new conntrack zone follows the highest one in use',
+);
+
+is(ct_zone_of({ a => vrf_zone(16500) }, 'a'), 16500, 'existing conntrack zone is kept');
+
+is(
+    ct_zone_of({ a => { type => 'simple', 'ct-zone' => 16400 } }, 'a'),
+    undef,
+    'conntrack zone is removed without a dedicated VRF',
+);
+
+is(
+    ct_zone_of({ a => { type => 'evpn', 'dedicated-vrf' => 1, 'ct-zone' => 16400 } }, 'a'),
+    undef,
+    'only simple zones get a conntrack zone',
+);
+
+$running_zones = { old => vrf_zone(16395) };
+is(
+    ct_zone_of({ a => vrf_zone(16384), new => vrf_zone() }, 'new'),
+    16396,
+    'conntrack zone of a zone that is only in the running config is not reused',
+);
+$running_zones = {};
+
+is(
+    ct_zone_of({ a => vrf_zone(17407), b => vrf_zone(16384), new => vrf_zone() }, 'new'),
+    16385,
+    'allocation wraps around after the last conntrack zone',
+);
+
+is(
+    ct_zone_of({ a => { type => 'vlan', 'ct-zone' => 17000 }, new => vrf_zone() }, 'new'),
+    17001,
+    'conntrack zones of other zones are taken into account',
+);
+
+is(
+    ct_zone_of({ a => vrf_zone(5), new => vrf_zone() }, 'new'),
+    16384,
+    'conntrack zones out of range are ignored',
+);
+
+my $all = { map { ("z$_" => vrf_zone($_)) } 16384 .. 17407 };
+$all->{new} = vrf_zone();
+eval { ct_zone_of($all, 'new') };
+like($@, qr/no conntrack zone left/, 'allocation fails if all conntrack zones are in use');
+
+for my $method ('create', 'update') {
+    my $param = { zone => 'zone0', 'ct-zone' => 16384 };
+    $param->{type} = 'simple' if $method eq 'create';
+
+    eval { PVE::API2::Network::SDN::Zones->$method($param) };
+    like($@, qr/ct-zone.*assigned automatically/s, "$method rejects setting ct-zone");
+}
+
+eval { PVE::API2::Network::SDN::Zones->update({ zone => 'zone0', delete => 'ct-zone' }) };
+like($@, qr/ct-zone.*assigned automatically/s, 'update rejects deleting ct-zone');
+
+done_testing();
-- 
2.47.3





  parent reply	other threads:[~2026-11-19 19:47 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox 01/14] network-api: expose the master device of a link Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-ve-rs 02/14] sdn: config: read the conntrack zone of zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-ve-rs 03/14] sdn: conntrack: map the interfaces of zone VRFs to their conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-network 04/14] sdn: simple: SNAT via the uplinks of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` Stefan Hanreich [this message]
2026-11-19 19:45 ` [PATCH proxmox-firewall 06/14] firewall: apply vnet rules to host traffic from vnets in a VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 07/14] nftables: expression: add a constructor for map lookups Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 08/14] nftables: statement: add a helper for setting the conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 09/14] nftables: expression: convert the conntrack zones of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 10/14] firewall: add a command applying " Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 11/14] firewall: put guest traffic of zone VRFs into their conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-manager 12/14] network: apply the conntrack zones of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-manager 13/14] network: warn if pve-firewall cannot isolate guests in " Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-docs 14/14] pvesdn: simple zones: document dedicated VRFs Stefan Hanreich

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119194547.674512-6-s.hanreich@proxmox.com \
    --to=s.hanreich@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal