public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Stefan Hanreich <s.hanreich@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH pve-network 05/14] sdn: zones: assign a conntrack zone to zones with a dedicated VRF
Date: Thu, 19 Nov 2026 20:45:37 +0100	[thread overview]
Message-ID: <20261119194547.674512-6-s.hanreich@proxmox.com> (raw)
In-Reply-To: <20261119194547.674512-1-s.hanreich@proxmox.com>

The subnets of zones with a dedicated VRF may overlap, so connections in
different VRFs must not share a conntrack entry, or the firewall would
treat a new connection in one VRF as part of an established one in
another VRF, and conntrack state migration would mix up the entries.

Assign each such zone a conntrack zone from the first 1024 zones (16384
to 17407) of the block 16384 to 32767 that Proxmox VE reserves, stored
with the zone as ct-zone, so that it is the same on all nodes and stays
the same as long as the zone has a dedicated VRF. It is removed when
the dedicated VRF is disabled, and cannot be set via the API.

A new conntrack zone follows the highest one in use, including the
running config, so the conntrack zone of a zone that is still active or
was just deleted is not reused right away, which could match stale
conntrack entries of that zone.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 src/PVE/API2/Network/SDN/Zones.pm         |  13 +++
 src/PVE/Network/SDN/Zones.pm              |  42 +++++++++
 src/PVE/Network/SDN/Zones/Plugin.pm       |   4 +
 src/PVE/Network/SDN/Zones/SimplePlugin.pm |   7 ++
 src/test/Makefile                         |   5 +-
 src/test/run_test_ct_zones.pl             | 103 ++++++++++++++++++++++
 6 files changed, 173 insertions(+), 1 deletion(-)
 create mode 100755 src/test/run_test_ct_zones.pl

diff --git a/src/PVE/API2/Network/SDN/Zones.pm b/src/PVE/API2/Network/SDN/Zones.pm
index 5952f40..5ff7bb4 100644
--- a/src/PVE/API2/Network/SDN/Zones.pm
+++ b/src/PVE/API2/Network/SDN/Zones.pm
@@ -70,6 +70,11 @@ my $ZONE_PROPERTIES = {
         optional => 1,
         description => 'Create a dedicated VRF for a simple zone.',
     },
+    'ct-zone' => {
+        type => 'integer',
+        optional => 1,
+        description => 'Conntrack zone of the dedicated VRF, assigned automatically.',
+    },
     route => {
         type => 'array',
         optional => 1,
@@ -407,6 +412,9 @@ __PACKAGE__->register_method({
         my $id = extract_param($param, 'zone');
         my $lock_token = extract_param($param, 'lock-token');
 
+        raise_param_exc({ 'ct-zone' => 'is assigned automatically' })
+            if defined($param->{'ct-zone'});
+
         my $plugin = PVE::Network::SDN::Zones::Plugin->lookup($type);
         my $opts = $plugin->check_config($id, $param, 1, 1);
 
@@ -434,6 +442,7 @@ __PACKAGE__->register_method({
                     if $dnszone && !$dnsserver;
 
                 $zone_cfg->{ids}->{$id} = $opts;
+                PVE::Network::SDN::Zones::update_ct_zone($zone_cfg, $id);
                 $plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
 
                 PVE::Network::SDN::Zones::write_config($zone_cfg);
@@ -478,6 +487,9 @@ __PACKAGE__->register_method({
             $delete = [PVE::Tools::split_list($delete)];
         }
 
+        raise_param_exc({ 'ct-zone' => 'is assigned automatically' })
+            if defined($param->{'ct-zone'}) || grep { $_ eq 'ct-zone' } ($delete // [])->@*;
+
         PVE::Network::SDN::lock_sdn_config(
             sub {
                 my $zone_cfg = PVE::Network::SDN::Zones::config();
@@ -541,6 +553,7 @@ __PACKAGE__->register_method({
                 my $dnszone = $opts->{dnszone};
                 raise_param_exc({ dnszone => "missing dns server" }) if $dnszone && !$dnsserver;
 
+                PVE::Network::SDN::Zones::update_ct_zone($zone_cfg, $id);
                 $plugin->on_update_hook($id, $zone_cfg, $controller_cfg);
 
                 PVE::Network::SDN::Zones::write_config($zone_cfg);
diff --git a/src/PVE/Network/SDN/Zones.pm b/src/PVE/Network/SDN/Zones.pm
index 3b59f4f..c8ff20b 100644
--- a/src/PVE/Network/SDN/Zones.pm
+++ b/src/PVE/Network/SDN/Zones.pm
@@ -59,6 +59,48 @@ sub config {
     return cfs_read_file("sdn/zones.cfg");
 }
 
+# Assigns a conntrack zone to a simple zone with a dedicated VRF and removes it from other zones.
+#
+# A new conntrack zone follows the highest one in use, also in the running config, so that the
+# conntrack zone of a zone that is still active or was just deleted is not reused right away.
+sub update_ct_zone {
+    my ($zone_cfg, $id) = @_;
+
+    my $zone = $zone_cfg->{ids}->{$id};
+
+    if ($zone->{type} ne 'simple' || !$zone->{'dedicated-vrf'}) {
+        delete $zone->{'ct-zone'};
+        return;
+    }
+
+    return if defined($zone->{'ct-zone'});
+
+    my $first = $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_FIRST;
+    my $last = $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_LAST;
+
+    my $running_zones = PVE::Network::SDN::Zones::config(1)->{ids};
+    my $used = {};
+    for my $zones ($zone_cfg->{ids}, $running_zones) {
+        for my $ct_zone (map { $_->{'ct-zone'} } values $zones->%*) {
+            $used->{$ct_zone} = 1 if defined($ct_zone) && $ct_zone >= $first && $ct_zone <= $last;
+        }
+    }
+
+    my $count = $last - $first + 1;
+    die "no conntrack zone left for the VRF of zone '$id'\n" if scalar(keys $used->%*) >= $count;
+
+    my ($highest) = sort { $b <=> $a } keys $used->%*;
+    $highest //= $last;
+
+    for my $offset (1 .. $count) {
+        my $ct_zone = $first + ($highest - $first + $offset) % $count;
+        if (!$used->{$ct_zone}) {
+            $zone->{'ct-zone'} = $ct_zone;
+            return;
+        }
+    }
+}
+
 sub get_plugin_config {
     my ($vnet) = @_;
     my $zoneid = $vnet->{zone};
diff --git a/src/PVE/Network/SDN/Zones/Plugin.pm b/src/PVE/Network/SDN/Zones/Plugin.pm
index 288edfc..d92bfd1 100644
--- a/src/PVE/Network/SDN/Zones/Plugin.pm
+++ b/src/PVE/Network/SDN/Zones/Plugin.pm
@@ -31,6 +31,10 @@ PVE::JSONSchema::register_standard_option(
     },
 );
 
+# conntrack zones reserved for the VRFs of zones, see also VrfCtZone in proxmox-ve-config
+our $VRF_CT_ZONE_FIRST = 16384;
+our $VRF_CT_ZONE_LAST = 17407;
+
 my $static_route_fmt = {
     prefix => {
         type => 'string',
diff --git a/src/PVE/Network/SDN/Zones/SimplePlugin.pm b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
index 4221521..3e866ac 100644
--- a/src/PVE/Network/SDN/Zones/SimplePlugin.pm
+++ b/src/PVE/Network/SDN/Zones/SimplePlugin.pm
@@ -40,6 +40,12 @@ sub properties {
             type => 'boolean',
             default => 0,
         },
+        'ct-zone' => {
+            description => 'Conntrack zone of the dedicated VRF, assigned automatically.',
+            type => 'integer',
+            minimum => $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_FIRST,
+            maximum => $PVE::Network::SDN::Zones::Plugin::VRF_CT_ZONE_LAST,
+        },
     };
 }
 
@@ -53,6 +59,7 @@ sub options {
         ipam => { optional => 1 },
         dhcp => { optional => 1 },
         'dedicated-vrf' => { optional => 1 },
+        'ct-zone' => { optional => 1 },
         route => { optional => 1 },
     };
 }
diff --git a/src/test/Makefile b/src/test/Makefile
index 0d5f3c1..242901b 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
 all: test
 
-test: test_zones test_static_routes test_fabric_host_vrf test_ipams test_dns test_subnets test_vnets_blackbox test_running_config test_api_index
+test: test_zones test_static_routes test_ct_zones test_fabric_host_vrf test_ipams test_dns test_subnets test_vnets_blackbox test_running_config test_api_index
 
 test_zones: run_test_zones.pl
 	./run_test_zones.pl
@@ -11,6 +11,9 @@ test_static_routes: run_test_static_routes.pl
 test_fabric_host_vrf: run_test_fabric_host_vrf.pl
 	./run_test_fabric_host_vrf.pl
 
+test_ct_zones: run_test_ct_zones.pl
+	./run_test_ct_zones.pl
+
 test_ipams: run_test_ipams.pl
 	./run_test_ipams.pl
 
diff --git a/src/test/run_test_ct_zones.pl b/src/test/run_test_ct_zones.pl
new file mode 100755
index 0000000..1b71522
--- /dev/null
+++ b/src/test/run_test_ct_zones.pl
@@ -0,0 +1,103 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use lib qw(..);
+
+use Test::More;
+use Test::MockModule;
+
+use PVE::API2::Network::SDN::Zones;
+use PVE::Network::SDN::Zones;
+
+my $running_zones = {};
+
+my $mocked_zones = Test::MockModule->new('PVE::Network::SDN::Zones');
+$mocked_zones->mock(
+    config => sub {
+        my ($running) = @_;
+        die "unexpected read of zones.cfg\n" if !$running;
+        return { ids => $running_zones };
+    },
+);
+
+sub vrf_zone {
+    my ($ct_zone) = @_;
+    my $zone = { type => 'simple', 'dedicated-vrf' => 1 };
+    $zone->{'ct-zone'} = $ct_zone if defined($ct_zone);
+    return $zone;
+}
+
+sub ct_zone_of {
+    my ($zones, $id) = @_;
+    my $zone_cfg = { ids => $zones };
+    PVE::Network::SDN::Zones::update_ct_zone($zone_cfg, $id);
+    return $zone_cfg->{ids}->{$id}->{'ct-zone'};
+}
+
+is(ct_zone_of({ new => vrf_zone() }, 'new'), 16384, 'first zone VRF gets the first conntrack zone');
+
+is(
+    ct_zone_of({ a => vrf_zone(16384), b => vrf_zone(16390), new => vrf_zone() }, 'new'),
+    16391,
+    'new conntrack zone follows the highest one in use',
+);
+
+is(ct_zone_of({ a => vrf_zone(16500) }, 'a'), 16500, 'existing conntrack zone is kept');
+
+is(
+    ct_zone_of({ a => { type => 'simple', 'ct-zone' => 16400 } }, 'a'),
+    undef,
+    'conntrack zone is removed without a dedicated VRF',
+);
+
+is(
+    ct_zone_of({ a => { type => 'evpn', 'dedicated-vrf' => 1, 'ct-zone' => 16400 } }, 'a'),
+    undef,
+    'only simple zones get a conntrack zone',
+);
+
+$running_zones = { old => vrf_zone(16395) };
+is(
+    ct_zone_of({ a => vrf_zone(16384), new => vrf_zone() }, 'new'),
+    16396,
+    'conntrack zone of a zone that is only in the running config is not reused',
+);
+$running_zones = {};
+
+is(
+    ct_zone_of({ a => vrf_zone(17407), b => vrf_zone(16384), new => vrf_zone() }, 'new'),
+    16385,
+    'allocation wraps around after the last conntrack zone',
+);
+
+is(
+    ct_zone_of({ a => { type => 'vlan', 'ct-zone' => 17000 }, new => vrf_zone() }, 'new'),
+    17001,
+    'conntrack zones of other zones are taken into account',
+);
+
+is(
+    ct_zone_of({ a => vrf_zone(5), new => vrf_zone() }, 'new'),
+    16384,
+    'conntrack zones out of range are ignored',
+);
+
+my $all = { map { ("z$_" => vrf_zone($_)) } 16384 .. 17407 };
+$all->{new} = vrf_zone();
+eval { ct_zone_of($all, 'new') };
+like($@, qr/no conntrack zone left/, 'allocation fails if all conntrack zones are in use');
+
+for my $method ('create', 'update') {
+    my $param = { zone => 'zone0', 'ct-zone' => 16384 };
+    $param->{type} = 'simple' if $method eq 'create';
+
+    eval { PVE::API2::Network::SDN::Zones->$method($param) };
+    like($@, qr/ct-zone.*assigned automatically/s, "$method rejects setting ct-zone");
+}
+
+eval { PVE::API2::Network::SDN::Zones->update({ zone => 'zone0', delete => 'ct-zone' }) };
+like($@, qr/ct-zone.*assigned automatically/s, 'update rejects deleting ct-zone');
+
+done_testing();
-- 
2.47.3





  parent reply	other threads:[~2026-11-19 19:47 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 19:45 [PATCH docs/manager/network/proxmox{,-ve-rs,-firewall} 00/14] sdn: conntrack zones and SNAT for simple zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox 01/14] network-api: expose the master device of a link Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-ve-rs 02/14] sdn: config: read the conntrack zone of zones with a dedicated VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-ve-rs 03/14] sdn: conntrack: map the interfaces of zone VRFs to their conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-network 04/14] sdn: simple: SNAT via the uplinks of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` Stefan Hanreich [this message]
2026-11-19 19:45 ` [PATCH proxmox-firewall 06/14] firewall: apply vnet rules to host traffic from vnets in a VRF Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 07/14] nftables: expression: add a constructor for map lookups Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 08/14] nftables: statement: add a helper for setting the conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 09/14] nftables: expression: convert the conntrack zones of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 10/14] firewall: add a command applying " Stefan Hanreich
2026-11-19 19:45 ` [PATCH proxmox-firewall 11/14] firewall: put guest traffic of zone VRFs into their conntrack zone Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-manager 12/14] network: apply the conntrack zones of zone VRFs Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-manager 13/14] network: warn if pve-firewall cannot isolate guests in " Stefan Hanreich
2026-11-19 19:45 ` [PATCH pve-docs 14/14] pvesdn: simple zones: document dedicated VRFs Stefan Hanreich

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119194547.674512-6-s.hanreich@proxmox.com \
    --to=s.hanreich@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal