[pve-devel] [PATCH docs/firewall/manager v7 0/4] autogenerate ipsets for sdn objects

[pve-devel] [PATCH docs/firewall/manager v7 0/4] autogenerate ipsets for sdn objects

[Stefan Hanreich]

This patch series adds support for autogenerating ipsets for SDN objects. It
autogenerates ipsets for every VNet as follows:

* ipset containing all IP ranges of the VNet
* ipset containing all gateways of the VNet
* ipset containing all IP ranges of the subnet - except gateways
* ipset containing all dhcp ranges of the vnet

Additionally it generates an IPSet for every guest that has one or more IPAM
entries in the pve IPAM.

Those can then be used in the cluster / host / guest firewalls. Firewall rules
automatically update on changes of the SDN / IPAM configuration. This patch
series works for the old firewall as well as the new firewall.

The ipsets in nftables currently get generated as named ipsets in every table,
this means that the `nft list ruleset` output can get quite crowded for large
SDN configurations or large IPAM databases. Another option would be to only
include them as anonymous IPsets in the rules, which would make the nft output
far less crowded but this way would use more memory when making extensive use of
the sdn ipsets, since everytime it is used in a rule we create an entirely new
ipset.

Dependencies:
* proxmox-perl-rs and proxmox-firewall depend on proxmox-ve-rs
* pve-firewall depends on proxmox-perl-rs
* pve-manager depends on pve-firewall

Changes from v6 to v7:
* cleaned up RPCEnvironment import in Firewall
* fix rule verification accepting all SDN IPsets for security groups / cluster

Changes from v5 to v6:
* Always load the full SDN configuration for the firewall instead of checking
  for the scope of the current user
* Filter the output of the refs endpoints to only show IPSets that the user has
  permission for
* Adapt create/update rule endpoints to only use IPSets from SDN config that the
  user has permission for

Changes from v4 to v5:
* extracted the API changes setting protected into a separate commit and put
  them up front
* fixed perl style issues - thanks @Thomas

Changes from v3 to v4:
* omitted proxmox-ve-rs since it is merged
* always load SDN configuration now when loading cluster config
* adapt is_nftables to check the flag file instead of reading the config
* gracefully fail when RPCEnvironment is not available

Changes from v2:
* rename end in IpRange to last to avoid confusion - thanks @Wolfgang
* bump Rust to 1.82 - thanks @Wolfgang
* improvements to the code generating IPSets - thanks @Wolfgang
* implement AsRef<str> for SDN name types - thanks @Wolfgang
* improve docstrings (proper capitalization and punctuation) - thanks @Wolfgang
* included a patch that removes proxmox-ve-config from proxmox-firewall

Changes from RFC:
* added documentation
* added separate SDN scope for IPSets
* rustfmt fixes

pve-firewall:

Stefan Hanreich (2):
  add support for loading sdn firewall configuration
  ipsets: return sdn ipsets from api

 src/PVE/API2/Firewall/Cluster.pm | 12 +++++++-
 src/PVE/API2/Firewall/Helpers.pm | 50 ++++++++++++++++++++++++++++++++
 src/PVE/API2/Firewall/Makefile   |  1 +
 src/PVE/API2/Firewall/Rules.pm   | 25 ++++++++++++++++
 src/PVE/API2/Firewall/VM.pm      | 10 ++++++-
 src/PVE/Firewall.pm              | 42 ++++++++++++++++++++++-----
 src/PVE/Service/pve_firewall.pm  |  4 +--
 7 files changed, 132 insertions(+), 12 deletions(-)
 create mode 100644 src/PVE/API2/Firewall/Helpers.pm


pve-manager:

Stefan Hanreich (1):
  firewall: add sdn scope to IPRefSelector

 www/manager6/form/IPRefSelector.js | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)


pve-docs:

Stefan Hanreich (1):
  sdn: add documentation for firewall integration

 pvesdn.adoc | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 92 insertions(+)


Summary over all repositories:
  9 files changed, 231 insertions(+), 13 deletions(-)

-- 
Generated by git-murpp 0.6.0

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-firewall v7 1/4] add support for loading sdn firewall configuration

[Stefan Hanreich]

This also includes support for parsing rules referencing IPSets in the
new SDN scope and generating those IPSets in the firewall. We always
load the new configuration, since loading the configuration always
includes validating the loaded rules. Validation fails without
including the SDN ipsets, leading to syslog error messages.

In the API, we only use the IPSets the user is actually allowed to use
for validating the rules - preventing users from using autogenerated
IPSets they have no permission for.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 src/PVE/API2/Firewall/Helpers.pm | 50 ++++++++++++++++++++++++++++++++
 src/PVE/API2/Firewall/Makefile   |  1 +
 src/PVE/API2/Firewall/Rules.pm   | 25 ++++++++++++++++
 src/PVE/Firewall.pm              | 42 ++++++++++++++++++++++-----
 src/PVE/Service/pve_firewall.pm  |  4 +--
 5 files changed, 112 insertions(+), 10 deletions(-)
 create mode 100644 src/PVE/API2/Firewall/Helpers.pm

diff --git a/src/PVE/API2/Firewall/Helpers.pm b/src/PVE/API2/Firewall/Helpers.pm
new file mode 100644
index 0000000..a8c6fea
--- /dev/null
+++ b/src/PVE/API2/Firewall/Helpers.pm
@@ -0,0 +1,50 @@
+package PVE::API2::Firewall::Helpers;
+
+use strict;
+use warnings;
+
+use PVE::Cluster;
+use PVE::Network::SDN::Vnets;
+use PVE::RPCEnvironment;
+
+sub get_allowed_vnets {
+    my $rpcenv = eval { PVE::RPCEnvironment::get() };
+
+    if ($@) {
+	warn "could not initialize RPCEnvironment";
+	return {};
+    }
+
+    my $authuser = $rpcenv->get_user();
+
+    my $vnets = PVE::Network::SDN::Vnets::config(1);
+    my $privs = [ 'SDN.Audit', 'SDN.Allocate' ];
+
+    my $allowed_vnets = [];
+    foreach my $vnet (sort keys %{$vnets->{ids}}) {
+	my $zone = $vnets->{ids}->{$vnet}->{zone};
+	next if !$rpcenv->check_any($authuser, "/sdn/zones/$zone/$vnet", $privs, 1);
+	push @$allowed_vnets, $vnet;
+    }
+
+    return $allowed_vnets;
+}
+
+sub get_allowed_vms {
+    my $rpcenv = eval { PVE::RPCEnvironment::get() };
+
+    if ($@) {
+	warn "could not initialize RPCEnvironment";
+	return {};
+    }
+
+    my $authuser = $rpcenv->get_user();
+
+    my $guests = PVE::Cluster::get_vmlist();
+
+    return [
+	grep { $rpcenv->check($authuser, "/vms/$_", [ 'VM.Audit' ], 1) } sort keys $guests->{ids}->%*
+    ];
+}
+
+1;
diff --git a/src/PVE/API2/Firewall/Makefile b/src/PVE/API2/Firewall/Makefile
index e916755..6be8261 100644
--- a/src/PVE/API2/Firewall/Makefile
+++ b/src/PVE/API2/Firewall/Makefile
@@ -4,6 +4,7 @@ PERLDIR=$(DESTDIR)/$(PREFIX)/share/perl5
 
 LIB_SOURCES=			\
 	Aliases.pm		\
+	Helpers.pm		\
 	IPSet.pm		\
 	Rules.pm		\
 	Cluster.pm		\
diff --git a/src/PVE/API2/Firewall/Rules.pm b/src/PVE/API2/Firewall/Rules.pm
index 9fcfb20..bbcf732 100644
--- a/src/PVE/API2/Firewall/Rules.pm
+++ b/src/PVE/API2/Firewall/Rules.pm
@@ -7,6 +7,7 @@ use PVE::JSONSchema qw(get_standard_option);
 use PVE::Exception qw(raise raise_param_exc);
 
 use PVE::Firewall;
+use PVE::API2::Firewall::Helpers;
 
 use base qw(PVE::RESTHandler);
 
@@ -237,6 +238,18 @@ sub register_create_rule {
 
 		my $rule = {};
 
+		# reloading the scoped SDN config for verification, so users can
+		# only use IPSets they have permissions for
+		my $allowed_vms = PVE::API2::Firewall::Helpers::get_allowed_vms();
+		my $allowed_vnets = PVE::API2::Firewall::Helpers::get_allowed_vnets();
+		my $sdn_conf = PVE::Firewall::load_sdn_conf($allowed_vms, $allowed_vnets);
+
+		if ($cluster_conf) {
+		    $cluster_conf->{sdn} = $sdn_conf;
+		} else {
+		    $fw_conf->{sdn} = $sdn_conf;
+		}
+
 		PVE::Firewall::copy_rule_data($rule, $param);
 		PVE::Firewall::verify_rule($rule, $cluster_conf, $fw_conf, $class->rule_env());
 
@@ -320,6 +333,18 @@ sub register_update_rule {
 
 		    PVE::Firewall::delete_rule_properties($rule, $param->{'delete'}) if $param->{'delete'};
 
+		    # reloading the scoped SDN config for verification, so users can
+		    # only use IPSets they have permissions for
+		    my $allowed_vms = PVE::API2::Firewall::Helpers::get_allowed_vms();
+		    my $allowed_vnets = PVE::API2::Firewall::Helpers::get_allowed_vnets();
+		    my $sdn_conf = PVE::Firewall::load_sdn_conf($allowed_vms, $allowed_vnets);
+
+		    if ($cluster_conf) {
+			$cluster_conf->{sdn} = $sdn_conf;
+		    } else {
+			$fw_conf->{sdn} = $sdn_conf;
+		    }
+
 		    PVE::Firewall::verify_rule($rule, $cluster_conf, $fw_conf, $class->rule_env());
 		}
 
diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm
index a69d5dd..6437db0 100644
--- a/src/PVE/Firewall.pm
+++ b/src/PVE/Firewall.pm
@@ -25,6 +25,7 @@ use PVE::Tools qw($IPV4RE $IPV6RE);
 use PVE::Tools qw(run_command lock_file dir_glob_foreach);
 
 use PVE::Firewall::Helpers;
+use PVE::RS::Firewall::SDN;
 
 my $pvefw_conf_dir = "/etc/pve/firewall";
 my $clusterfw_conf_filename = "$pvefw_conf_dir/cluster.fw";
@@ -1689,10 +1690,15 @@ sub verify_rule {
 
 	if (my $value = $rule->{$name}) {
 	    if ($value =~ m/^\+/) {
-		if ($value =~ m@^\+(guest/|dc/)?(${ipset_name_pattern})$@) {
-		    &$add_error($name, "no such ipset '$2'")
-			if !($cluster_conf->{ipset}->{$2} || ($fw_conf && $fw_conf->{ipset}->{$2}));
-
+		if ($value =~ m@^\+(guest/|dc/|sdn/)?(${ipset_name_pattern})$@) {
+		    if (
+			!($cluster_conf->{ipset}->{$2})
+			&& !($fw_conf && $fw_conf->{ipset}->{$2})
+			&& !($cluster_conf->{sdn} && $cluster_conf->{sdn}->{ipset}->{$2})
+			&& !($fw_conf->{sdn} && $fw_conf->{sdn}->{ipset}->{$2})
+		    ) {
+			$add_error->($name, "no such ipset '$2'")
+		    }
 		} else {
 		    &$add_error($name, "invalid ipset name '$value'");
 		}
@@ -2108,13 +2114,18 @@ sub ipt_gen_src_or_dst_match {
 
     my $match;
     if ($adr =~ m/^\+/) {
-	if ($adr =~ m@^\+(guest/|dc/)?(${ipset_name_pattern})$@) {
+	if ($adr =~ m@^\+(guest/|dc/|sdn/)?(${ipset_name_pattern})$@) {
 	    my $scope = $1 // "";
 	    my $name = $2;
 	    my $ipset_chain;
-	    if ($scope ne 'dc/' && $fw_conf && $fw_conf->{ipset}->{$name}) {
+
+	    my $is_scope = sub { return !$scope || $scope eq "$_[0]/" };
+
+	    if ($is_scope->('guest') && $fw_conf && $fw_conf->{ipset}->{$name}) {
 		$ipset_chain = compute_ipset_chain_name($fw_conf->{vmid}, $name, $ipversion);
-	    } elsif ($scope ne 'guest/' && $cluster_conf && $cluster_conf->{ipset}->{$name}) {
+	    } elsif ($is_scope->('dc') && $cluster_conf && $cluster_conf->{ipset}->{$name}) {
+		$ipset_chain = compute_ipset_chain_name(0, $name, $ipversion);
+	    } elsif ($is_scope->('sdn') && $cluster_conf->{sdn} && $cluster_conf->{sdn}->{ipset}->{$name}) {
 		$ipset_chain = compute_ipset_chain_name(0, $name, $ipversion);
 	    } else {
 		die "no such ipset '$name'\n";
@@ -3655,6 +3666,7 @@ sub load_clusterfw_conf {
 	group_comments => {},
 	ipset => {} ,
 	ipset_comments => {},
+	sdn => load_sdn_conf(),
     };
 
     my $cluster_conf = generic_fw_config_parser($filename, $empty_conf, $empty_conf, 'cluster');
@@ -3663,6 +3675,19 @@ sub load_clusterfw_conf {
     return $cluster_conf;
 }
 
+sub load_sdn_conf {
+    my ($allowed_vms, $allowed_vnets) = @_;
+
+    my $empty_sdn_config = { ipset => {} , ipset_comments => {} };
+
+    my $sdn_config = eval {
+	PVE::RS::Firewall::SDN::config($allowed_vnets, $allowed_vms)
+    };
+    warn $@ if $@;
+
+    return $sdn_config // $empty_sdn_config;
+}
+
 sub save_clusterfw_conf {
     my ($cluster_conf) = @_;
 
@@ -3768,7 +3793,7 @@ sub compile {
 
 	$vmfw_configs = read_vm_firewall_configs($cluster_conf, $vmdata, $testdir);
     } else { # normal operation
-	$cluster_conf = load_clusterfw_conf(undef) if !$cluster_conf;
+	$cluster_conf = load_clusterfw_conf() if !$cluster_conf;
 
 	$hostfw_conf = load_hostfw_conf($cluster_conf, undef) if !$hostfw_conf;
 
@@ -4043,6 +4068,7 @@ sub compile_ipsets {
     }
 
     generate_ipset_chains($ipset_ruleset, undef, $cluster_conf, undef, $cluster_conf->{ipset});
+    generate_ipset_chains($ipset_ruleset, undef, $cluster_conf, undef, $cluster_conf->{sdn}->{ipset});
 
     return $ipset_ruleset;
 }
diff --git a/src/PVE/Service/pve_firewall.pm b/src/PVE/Service/pve_firewall.pm
index 65cb2b8..02b507a 100755
--- a/src/PVE/Service/pve_firewall.pm
+++ b/src/PVE/Service/pve_firewall.pm
@@ -158,7 +158,7 @@ __PACKAGE__->register_method ({
 
 	    PVE::Firewall::set_verbose(1); # show syntax errors
 
-	    my $cluster_conf = PVE::Firewall::load_clusterfw_conf(undef);
+	    my $cluster_conf = PVE::Firewall::load_clusterfw_conf();
 	    $res->{enable} = $cluster_conf->{options}->{enable} ? 1 : 0;
 
 	    if ($status eq 'running') {
@@ -202,7 +202,7 @@ __PACKAGE__->register_method ({
 
 	    PVE::Firewall::set_verbose(1);
 
-	    my $cluster_conf = PVE::Firewall::load_clusterfw_conf(undef);
+	    my $cluster_conf = PVE::Firewall::load_clusterfw_conf();
 	    my ($ruleset, $ipset_ruleset, $rulesetv6, $ebtables_ruleset) = PVE::Firewall::compile($cluster_conf, undef, undef);
 
 	    print "ipset cmdlist:\n";
-- 
2.39.5


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-firewall v7 2/4] ipsets: return sdn ipsets from api

[Stefan Hanreich]

In order for the new SDN ipsets to show up we need to adapt the
existing API endpoints so they read the SDN configuration. We reload
the SDN configuration explicitly, in order to return only the IPSets
the user is allowed to see.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 src/PVE/API2/Firewall/Cluster.pm | 12 +++++++++++-
 src/PVE/API2/Firewall/VM.pm      | 10 +++++++++-
 2 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/src/PVE/API2/Firewall/Cluster.pm b/src/PVE/API2/Firewall/Cluster.pm
index 48ad90d..4d51b34 100644
--- a/src/PVE/API2/Firewall/Cluster.pm
+++ b/src/PVE/API2/Firewall/Cluster.pm
@@ -9,6 +9,7 @@ use PVE::Firewall;
 use PVE::API2::Firewall::Aliases;
 use PVE::API2::Firewall::Rules;
 use PVE::API2::Firewall::Groups;
+use PVE::API2::Firewall::Helpers;
 use PVE::API2::Firewall::IPSet;
 
 #fixme: locking?
@@ -255,7 +256,16 @@ __PACKAGE__->register_method({
 
 	my $conf = PVE::Firewall::load_clusterfw_conf();
 
-	return PVE::Firewall::Helpers::collect_refs($conf, $param->{type}, "dc");
+	# we are explicitly loading the SDN config here with the scope of the current
+	# API user, so we only return the IPSets that the user can actually use
+	my $allowed_vms = PVE::API2::Firewall::Helpers::get_allowed_vms();
+	my $allowed_vnets = PVE::API2::Firewall::Helpers::get_allowed_vnets();
+	my $sdn_conf = PVE::Firewall::load_sdn_conf($allowed_vms, $allowed_vnets);
+
+	my $cluster_refs = PVE::Firewall::Helpers::collect_refs($conf, $param->{type}, "dc");
+	my $sdn_refs = PVE::Firewall::Helpers::collect_refs($sdn_conf, $param->{type}, "sdn");
+
+	return [@$sdn_refs, @$cluster_refs];
     }});
 
 1;
diff --git a/src/PVE/API2/Firewall/VM.pm b/src/PVE/API2/Firewall/VM.pm
index 4222103..2d25735 100644
--- a/src/PVE/API2/Firewall/VM.pm
+++ b/src/PVE/API2/Firewall/VM.pm
@@ -8,6 +8,7 @@ use PVE::JSONSchema qw(get_standard_option);
 use PVE::Cluster;
 use PVE::Firewall;
 use PVE::API2::Firewall::Rules;
+use PVE::API2::Firewall::Helpers;
 use PVE::API2::Firewall::Aliases;
 
 
@@ -281,10 +282,17 @@ sub register_handlers {
 	    my $cluster_conf = PVE::Firewall::load_clusterfw_conf();
 	    my $fw_conf = PVE::Firewall::load_vmfw_conf($cluster_conf, $rule_env, $param->{vmid});
 
+	    # we are explicitly loading the SDN config here with the scope of the current
+	    # API user, so we only return the IPSets that the user can actually use
+	    my $allowed_vms = PVE::API2::Firewall::Helpers::get_allowed_vms();
+	    my $allowed_vnets = PVE::API2::Firewall::Helpers::get_allowed_vnets();
+	    my $sdn_conf = PVE::Firewall::load_sdn_conf($allowed_vms, $allowed_vnets);
+
 	    my $dc_refs = PVE::Firewall::Helpers::collect_refs($cluster_conf, $param->{type}, 'dc');
+	    my $sdn_refs = PVE::Firewall::Helpers::collect_refs($sdn_conf, $param->{type}, "sdn");
 	    my $vm_refs = PVE::Firewall::Helpers::collect_refs($fw_conf, $param->{type}, 'guest');
 
-	    return [@$dc_refs, @$vm_refs];
+	    return [@$dc_refs, @$sdn_refs, @$vm_refs];
 	}});
 }
 
-- 
2.39.5


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-manager v7 3/4] firewall: add sdn scope to IPRefSelector

[Stefan Hanreich]

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Hannes Dürr <h.duerr@proxmox.com>
---
 www/manager6/form/IPRefSelector.js | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/www/manager6/form/IPRefSelector.js b/www/manager6/form/IPRefSelector.js
index d41cde5f5..16078e428 100644
--- a/www/manager6/form/IPRefSelector.js
+++ b/www/manager6/form/IPRefSelector.js
@@ -67,6 +67,12 @@ Ext.define('PVE.form.IPRefSelector', {
 	    });
 	}
 
+	let scopes = {
+	    'dc': gettext("Datacenter"),
+	    'guest': gettext("Guest"),
+	    'sdn': gettext("SDN"),
+	};
+
 	columns.push(
 	    {
 		header: gettext('Name'),
@@ -80,7 +86,7 @@ Ext.define('PVE.form.IPRefSelector', {
 		hideable: false,
 		width: 140,
 		renderer: function(value) {
-		    return value === 'dc' ? gettext("Datacenter") : gettext("Guest");
+		    return scopes[value] ?? "unknown scope";
 		},
 	    },
 	    {
-- 
2.39.5


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-docs v7 4/4] sdn: add documentation for firewall integration

[Stefan Hanreich]

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 pvesdn.adoc | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 92 insertions(+)

diff --git a/pvesdn.adoc b/pvesdn.adoc
index 39de80f..c187365 100644
--- a/pvesdn.adoc
+++ b/pvesdn.adoc
@@ -702,6 +702,98 @@ For more information please consult the documentation of
 xref:pvesdn_ipam_plugin_pveipam[the PVE IPAM plugin]. Changing DHCP leases is
 currently not supported for the other IPAM plugins.
 
+Firewall Integration
+--------------------
+
+SDN integrates with the Proxmox VE firewall by automatically generating IPSets
+which can then be referenced in the source / destination fields of firewall
+rules. This happens automatically for VNets and IPAM entries.
+
+VNets and Subnets
+~~~~~~~~~~~~~~~~~
+
+The firewall automatically generates the following IPSets in the SDN scope for
+every VNet:
+
+`vnet-all`::
+  Contains the CIDRs of all subnets in a VNet
+`vnet-gateway`::
+  Contains the IPs of the gateways of all subnets in a VNet
+`vnet-no-gateway`::
+  Contains the CIDRs of all subnets in a VNet, but excludes the gateways
+`vnet-dhcp`::
+  Contains all DHCP ranges configured in the subnets in a VNet
+
+When making changes to your configuration, the IPSets update automatically, so
+you do not have to update your firewall rules when changing the configuration of
+your Subnets.
+
+Simple Zone Example
+^^^^^^^^^^^^^^^^^^^
+
+Assuming the configuration below for a VNet and its contained subnets:
+
+----
+# /etc/pve/sdn/vnets.cfg
+
+vnet: vnet0
+        zone simple
+
+# /etc/pve/sdn/subnets.cfg
+
+subnet: simple-192.0.2.0-24
+        vnet vnet0
+        dhcp-range start-address=192.0.2.100,end-address=192.0.2.199
+        gateway 192.0.2.1
+
+subnet: simple-2001:db8::-64
+        vnet vnet0
+        dhcp-range start-address=2001:db8::1000,end-address=2001:db8::1999
+        gateway 2001:db8::1
+----
+
+In this example we configured an IPv4 subnet in the VNet `vnet0`, with
+'192.0.2.0/24' as its IP Range, '192.0.2.1' as the gateway and the DHCP range is
+'192.0.2.100' - '192.0.2.199'.
+
+Additionally we configured an IPv6 subnet with '2001:db8::/64' as the IP range,
+'2001:db8::1' as the gateway and a DHCP range of '2001:db8::1000' -
+'2001:db8::1999'.
+
+The respective auto-generated IPsets for vnet0 would then contain the following
+elements:
+
+`vnet0-all`::
+* '192.0.2.0/24'
+* '2001:db8::/64'
+`vnet0-gateway`::
+* '192.0.2.1'
+* '2001:db8::1'
+`vnet0-no-gateway`::
+* '192.0.2.0/24'
+* '2001:db8::/64'
+* '!192.0.2.1'
+* '!2001:db8::1'
+`vnet0-dhcp`::
+* '192.0.2.100 - 192.0.2.199'
+* '2001:db8::1000 - 2001:db8::1999'
+
+IPAM
+~~~~
+
+If you are using the built-in PVE IPAM, then the firewall automatically
+generates an IPset for every guest that has entries in the IPAM. The respective
+IPset for a guest with ID 100 would be `guest-ipam-100`. It contains all IP
+addresses from all IPAM entries. So if guest 100 is member of multiple VNets,
+then the IPset would contain the IPs from *all* VNets.
+
+When entries get added / updated / deleted, then the respective IPSets will be
+updated accordingly.
+
+WARNING: When removing all entries for a guest and there are firewall rules
+still referencing the auto-generated IPSet then the firewall will fail to update
+the ruleset, since it references a non-existing IPSet.
+
 [[pvesdn_setup_examples]]
 Examples
 --------
-- 
2.39.5


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel