[pve-devel] [PATCH container/docs/firewall/manager/qemu-server v4 0/5] proxmox firewall nftables

[pve-devel] [PATCH container/docs/firewall/manager/qemu-server v4 0/5] proxmox firewall nftables

[Stefan Hanreich]

This patch series contains the remaining patches that are necessary for
proxmox-firewall to work. It adds documentation as well as changes how
firewall-bridges are created when proxmox-firewall is activated. It also patches
pve-firewall to not generate rules when proxmox-firewall is active.

Dependencies:
* qemu-server, pve-container & pve-manager depend on a bump of pve-firewall

Changes from v3 -> v4:
* additionally check for the existence of proxmox-firewall bin
* extracted checks into helper functions
* update docs to reflect the changes in behavior

(omitted description & changes only relevant for the firewall itself)

qemu-server:

Stefan Hanreich (1):
  firewall: add handling for new nft firewall

 vm-network-scripts/pve-bridge | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)


pve-container:

Stefan Hanreich (1):
  firewall: add handling for new nft firewall

 src/PVE/LXC.pm | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)


pve-firewall:

Stefan Hanreich (1):
  add configuration option for new nftables firewall

 src/PVE/Firewall.pm | 41 ++++++++++++++++++++++++++++++++---------
 1 file changed, 32 insertions(+), 9 deletions(-)


pve-manager:

Stefan Hanreich (1):
  firewall: expose configuration option for new nftables firewall

 www/manager6/grid/FirewallOptions.js | 1 +
 1 file changed, 1 insertion(+)


pve-docs:

Stefan Hanreich (1):
  firewall: add documentation for proxmox-firewall

 pve-firewall.adoc | 181 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 181 insertions(+)


Summary over all repositories:
  5 files changed, 224 insertions(+), 13 deletions(-)

-- 
Generated by git-murpp 0.6.0

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v4 1/5] firewall: add handling for new nft firewall

[Stefan Hanreich]

When the nftables firewall is enabled, we do not need to create
firewall bridges.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 vm-network-scripts/pve-bridge | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/vm-network-scripts/pve-bridge b/vm-network-scripts/pve-bridge
index 85997a0..fe5a702 100755
--- a/vm-network-scripts/pve-bridge
+++ b/vm-network-scripts/pve-bridge
@@ -6,6 +6,7 @@ use warnings;
 use PVE::QemuServer;
 use PVE::Tools qw(run_command);
 use PVE::Network;
+use PVE::Firewall;
 
 my $have_sdn;
 eval {
@@ -44,13 +45,15 @@ die "unable to get network config '$netid'\n"
 my $net = PVE::QemuServer::parse_net($netconf);
 die "unable to parse network config '$netid'\n" if !$net;
 
+my $firewall = $net->{firewall} && !PVE::Firewall::is_nftables();
+
 if ($have_sdn) {
     PVE::Network::SDN::Vnets::add_dhcp_mapping($net->{bridge}, $net->{macaddr}, $vmid, $conf->{name});
     PVE::Network::SDN::Zones::tap_create($iface, $net->{bridge});
-    PVE::Network::SDN::Zones::tap_plug($iface, $net->{bridge}, $net->{tag}, $net->{firewall}, $net->{trunks}, $net->{rate});
+    PVE::Network::SDN::Zones::tap_plug($iface, $net->{bridge}, $net->{tag}, $firewall, $net->{trunks}, $net->{rate});
 } else {
     PVE::Network::tap_create($iface, $net->{bridge});
-    PVE::Network::tap_plug($iface, $net->{bridge}, $net->{tag}, $net->{firewall}, $net->{trunks}, $net->{rate});
+    PVE::Network::tap_plug($iface, $net->{bridge}, $net->{tag}, $firewall, $net->{trunks}, $net->{rate});
 }
 
 exit 0;
-- 
2.39.2


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-container v4 2/5] firewall: add handling for new nft firewall

[Stefan Hanreich]

When the nftables firewall is enabled, we do not need to create
firewall bridges.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 src/PVE/LXC.pm | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 400cf4f..44f5ccf 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -18,6 +18,7 @@ use PVE::AccessControl;
 use PVE::CGroup;
 use PVE::CpuSet;
 use PVE::Exception qw(raise_perm_exc);
+use PVE::Firewall;
 use PVE::GuestHelpers qw(check_vnet_access safe_string_ne safe_num_ne safe_boolean_ne);
 use PVE::INotify;
 use PVE::JSONSchema qw(get_standard_option);
@@ -946,8 +947,10 @@ sub net_tap_plug : prototype($$) {
 	return;
     }
 
-    my ($bridge, $tag, $firewall, $trunks, $rate, $hwaddr) =
-	$net->@{'bridge', 'tag', 'firewall', 'trunks', 'rate', 'hwaddr'};
+    my ($bridge, $tag, $trunks, $rate, $hwaddr) =
+	$net->@{'bridge', 'tag', 'trunks', 'rate', 'hwaddr'};
+
+    my $firewall = $net->{firewall} && !PVE::Firewall::is_nftables();
 
     if ($have_sdn) {
 	PVE::Network::SDN::Zones::tap_plug($iface, $bridge, $tag, $firewall, $trunks, $rate);
-- 
2.39.2


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-firewall v4 3/5] add configuration option for new nftables firewall

[Stefan Hanreich]

Introduces new nftables configuration option that en/disables the new
nftables firewall.

pve-firewall reads this option and only generates iptables rules when
nftables is set to `0` or if the proxmox-firewall package is not
installed at all. Conversely, proxmox-firewall only generates rules
when the option is set to `1`.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
This looks a bit awkward, but I wanted to avoid having to re-parse the
configuration when calling from pve-firewall but also avoid having to
load the config manually when calling from qemu-server / pve-container

 src/PVE/Firewall.pm | 41 ++++++++++++++++++++++++++++++++---------
 1 file changed, 32 insertions(+), 9 deletions(-)

diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm
index 81a8798..21eb5fc 100644
--- a/src/PVE/Firewall.pm
+++ b/src/PVE/Firewall.pm
@@ -1408,6 +1408,12 @@ our $host_option_properties = {
 	default => 0,
 	optional => 1
     },
+    nftables => {
+	description => "Enable nftables based firewall",
+	type => 'boolean',
+	default => 0,
+	optional => 1,
+    },
 };
 
 our $vm_option_properties = {
@@ -2929,7 +2935,7 @@ sub parse_hostfw_option {
 
     my $loglevels = "emerg|alert|crit|err|warning|notice|info|debug|nolog";
 
-    if ($line =~ m/^(enable|nosmurfs|tcpflags|ndp|log_nf_conntrack|nf_conntrack_allow_invalid|protection_synflood):\s*(0|1)\s*$/i) {
+    if ($line =~ m/^(enable|nosmurfs|tcpflags|ndp|log_nf_conntrack|nf_conntrack_allow_invalid|protection_synflood|nftables):\s*(0|1)\s*$/i) {
 	$opt = lc($1);
 	$value = int($2);
     } elsif ($line =~ m/^(log_level_in|log_level_out|tcp_flags_log_level|smurf_log_level):\s*(($loglevels)\s*)?$/i) {
@@ -4673,12 +4679,30 @@ sub remove_pvefw_chains_ebtables {
     ebtables_restore_cmdlist(get_ebtables_cmdlist({}));
 }
 
-sub init {
-    my $cluster_conf = load_clusterfw_conf();
-    my $cluster_options = $cluster_conf->{options};
-    my $enable = $cluster_options->{enable};
+sub is_nftables {
+    my ($cluster_conf, $host_conf) = @_;
+
+    if (!-x "/usr/libexec/proxmox/proxmox-firewall") {
+	return 0;
+    }
+
+    $cluster_conf = load_clusterfw_conf() if !defined($cluster_conf);
+    $host_conf = load_hostfw_conf($cluster_conf) if !defined($host_conf);
 
-    return if !$enable;
+    return $host_conf->{options}->{nftables};
+}
+
+sub is_enabled {
+    my ($cluster_conf, $host_conf) = @_;
+
+    $cluster_conf = load_clusterfw_conf() if !defined($cluster_conf);
+    $host_conf = load_hostfw_conf($cluster_conf) if !defined($host_conf);
+
+    return $cluster_conf->{options}->{enable} && !is_nftables($cluster_conf, $host_conf);
+}
+
+sub init {
+    return if !is_enabled();
 
     # load required modules here
 }
@@ -4687,14 +4711,13 @@ sub update {
     my $code = sub {
 
 	my $cluster_conf = load_clusterfw_conf();
-	my $cluster_options = $cluster_conf->{options};
+	my $hostfw_conf = load_hostfw_conf($cluster_conf);
 
-	if (!$cluster_options->{enable}) {
+	if (!is_enabled($cluster_conf, $hostfw_conf)) {
 	    PVE::Firewall::remove_pvefw_chains();
 	    return;
 	}
 
-	my $hostfw_conf = load_hostfw_conf($cluster_conf);
 
 	my ($ruleset, $ipset_ruleset, $rulesetv6, $ebtables_ruleset) = compile($cluster_conf, $hostfw_conf);
 
-- 
2.39.2


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-manager v4 4/5] firewall: expose configuration option for new nftables firewall

[Stefan Hanreich]

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 www/manager6/grid/FirewallOptions.js | 1 +
 1 file changed, 1 insertion(+)

diff --git a/www/manager6/grid/FirewallOptions.js b/www/manager6/grid/FirewallOptions.js
index 0ac9979c4..6aacb47be 100644
--- a/www/manager6/grid/FirewallOptions.js
+++ b/www/manager6/grid/FirewallOptions.js
@@ -83,6 +83,7 @@ Ext.define('PVE.FirewallOptions', {
 	    add_log_row('log_level_out');
 	    add_log_row('tcp_flags_log_level', 120);
 	    add_log_row('smurf_log_level');
+	    add_boolean_row('nftables', gettext('nftables (tech preview)'), 0);
 	} else if (me.fwtype === 'vm') {
 	    me.rows.enable = {
 		required: true,
-- 
2.39.2


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-docs v4 5/5] firewall: add documentation for proxmox-firewall

[Stefan Hanreich]

Add a section that explains how to use the new nftables-based
proxmox-firewall.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 pve-firewall.adoc | 181 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 181 insertions(+)

diff --git a/pve-firewall.adoc b/pve-firewall.adoc
index a5e40f9..9fb4e46 100644
--- a/pve-firewall.adoc
+++ b/pve-firewall.adoc
@@ -379,6 +379,7 @@ discovery protocol to work.
 ----
 
 
+[[pve_firewall_services_commands]]
 Services and Commands
 ---------------------
 
@@ -637,6 +638,186 @@ Ports used by {pve}
 * corosync cluster traffic: 5405-5412 UDP
 * live migration (VM memory and local-disk data): 60000-60050 (TCP)
 
+
+nftables
+--------
+
+As an alternative to `pve-firewall` we offer `proxmox-firewall`, which is an
+implementation of the Proxmox VE firewall based on the newer
+https://wiki.nftables.org/wiki-nftables/index.php/What_is_nftables%3F[nftables]
+rather than iptables.
+
+WARNING: `proxmox-firewall` is currently in tech preview. There might be bugs or
+incompatibilies with the original firewall. It is currently not suited for
+production use.
+
+This implementation uses the same configuration files and configuration format,
+so you can use your old configuration when switching. It provides the exact same
+functionality with a few exceptions:
+
+* REJECT is currently not possible for guest traffic (traffic will instead be
+  dropped).
+* Using the `NDP`, `Router Advertisement` or `DHCP` options will *always* create
+  firewall rules, irregardless of your default policy.
+* firewall rules for guests are evaluated even for connections that have
+  conntrack table entries.
+
+
+Installation and Usage
+~~~~~~~~~~~~~~~~~~~~~~
+
+Install the `proxmox-firewall` package:
+
+----
+apt install proxmox-firewall
+----
+
+Enable the nftables backend via the Web UI on your hosts (Host > Firewall >
+Options > nftables), or by enabling it in the configuration file for your hosts
+(`/etc/pve/nodes/<node_name>/host.fw`):
+
+----
+[OPTIONS]
+
+nftables: 1
+----
+
+NOTE: After enabling/disabling `proxmox-firewall`, all running VMs and
+containers need to be restarted for the old/new firewall to work properly.
+
+After setting the `nftables` configuration key, the new `proxmox-firewall`
+service will take over. You can check if the new service is working by
+checking the systemctl status of `proxmox-firewall`:
+
+----
+systemctl status proxmox-firewall
+----
+
+You can also examine the generated ruleset. You can find more information about
+this in the section xref:pve_firewall_nft_helpful_commands[Helpful Commands].
+You should also check whether `pve-firewall` is no longer generating iptables
+rules, you can find the respective commands in the
+xref:pve_firewall_services_commands[Services and Commands] section.
+
+Switching back to the old firewall can be done by simply setting the
+configuration value back to 0 / No.
+
+Usage
+~~~~~
+
+`proxmox-firewall` will create two tables that are managed by the
+`proxmox-firewall` service: `proxmox-firewall` and `proxmox-firewall-guests`. If
+you want to create custom rules that live outside the Proxmox VE firewall
+configuration you can create your own tables to manage your custom firewall
+rules. `proxmox-firewall` will only touch the tables it generates, so you can
+easily extend and modify the behavior of the `proxmox-firewall` by adding your
+own tables.
+
+Instead of using the `pve-firewall` command, the nftables-based firewall uses
+`proxmox-firewall`. It is a systemd service, so you can start and stop it via
+`systemctl`:
+
+----
+systemctl start proxmox-firewall
+systemctl stop proxmox-firewall
+----
+
+Stopping the firewall service will remove all generated rules.
+
+To query the status of the firewall, you can query the status of the systemctl
+service:
+
+----
+systemctl status proxmox-firewall
+----
+
+
+[[pve_firewall_nft_helpful_commands]]
+Helpful Commands
+~~~~~~~~~~~~~~~~
+You can check the generated ruleset via the following command:
+
+----
+nft list ruleset
+----
+
+If you want to debug `proxmox-firewall` you can simply run the daemon in
+foreground with the `RUST_LOG` environment variable set to `trace`. This should
+provide you with detailed debugging output:
+
+----
+RUST_LOG=trace /usr/libexec/proxmox/proxmox-firewall
+----
+
+You can also edit the systemctl service if you want to have detailed output for
+your firewall daemon:
+
+----
+systemctl edit proxmox-firewall
+----
+
+Then you need to add the override for the `RUST_LOG` environment variable:
+
+----
+[Service]
+Environment="RUST_LOG=trace"
+----
+
+This will generate a large amount of logs very quickly, so only use this for
+debugging purposes. Other, less verbose, log levels are `info` and `debug`.
+
+Running in foreground writes the log output to STDERR, so you can redirect it
+with the following command (e.g. for submitting logs to the community forum):
+
+----
+RUST_LOG=trace /usr/libexec/proxmox/proxmox-firewall 2> firewall_log_$(hostname).txt
+----
+
+It can be helpful to trace packet flow through the different chains in order to
+debug firewall rules. This can be achieved by setting `nftrace` to 1 for packets
+that you want to track. It is advisable that you do not set this flag for *all*
+packets, in the example below we only examine ICMP packets.
+
+----
+#!/usr/sbin/nft -f
+table bridge tracebridge
+delete table bridge tracebridge
+
+table bridge tracebridge {
+    chain trace {
+        meta l4proto icmp meta nftrace set 1
+    }
+
+    chain prerouting {
+        type filter hook prerouting priority -350; policy accept;
+        jump trace
+    }
+
+    chain postrouting {
+        type filter hook postrouting priority -350; policy accept;
+        jump trace
+    }
+}
+----
+
+Saving this file, making it executable, and then running it once will create the
+respective tracing chains. You can then inspect the tracing output via the
+Proxmox VE Web UI (Firewall > Log) or via `nft monitor trace`.
+
+The above example traces traffic on all bridges, which is usually where guest
+traffic flows through. If you want to examine host traffic, create those chains
+in the `inet` table instead of the `bridge` table.
+
+NOTE: Be aware that this can generate a *lot* of log spam and slow down the
+performance of your networking stack significantly.
+
+You can remove the tracing rules via running the following command:
+
+----
+nft delete table bridge tracebridge
+----
+
+
 ifdef::manvolnum[]
 
 Macro Definitions
-- 
2.39.2


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] applied: [PATCH pve-docs v4 5/5] firewall: add documentation for proxmox-firewall

[Thomas Lamprecht]

Am 19/04/2024 um 11:42 schrieb Stefan Hanreich:
> Add a section that explains how to use the new nftables-based
> proxmox-firewall.
> 
> Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
> ---
>  pve-firewall.adoc | 181 ++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 181 insertions(+)
> 
>

applied this one too now, thanks!


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] applied: [PATCH container/docs/firewall/manager/qemu-server v4 0/5] proxmox firewall nftables

[Thomas Lamprecht]

Am 19/04/2024 um 11:42 schrieb Stefan Hanreich:
> This patch series contains the remaining patches that are necessary for
> proxmox-firewall to work. It adds documentation as well as changes how
> firewall-bridges are created when proxmox-firewall is activated. It also patches
> pve-firewall to not generate rules when proxmox-firewall is active.
> 
> Dependencies:
> * qemu-server, pve-container & pve-manager depend on a bump of pve-firewall
> 
> Changes from v3 -> v4:
> * additionally check for the existence of proxmox-firewall bin
> * extracted checks into helper functions
> * update docs to reflect the changes in behavior
> 
> (omitted description & changes only relevant for the firewall itself)
> 
> qemu-server:
> 
> Stefan Hanreich (1):
>   firewall: add handling for new nft firewall
> 
>  vm-network-scripts/pve-bridge | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> 
> pve-container:
> 
> Stefan Hanreich (1):
>   firewall: add handling for new nft firewall
> 
>  src/PVE/LXC.pm | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> 
> pve-firewall:
> 
> Stefan Hanreich (1):
>   add configuration option for new nftables firewall
> 
>  src/PVE/Firewall.pm | 41 ++++++++++++++++++++++++++++++++---------
>  1 file changed, 32 insertions(+), 9 deletions(-)
> 
> 
> pve-manager:
> 
> Stefan Hanreich (1):
>   firewall: expose configuration option for new nftables firewall
> 
>  www/manager6/grid/FirewallOptions.js | 1 +
>  1 file changed, 1 insertion(+)
> 
> 
> pve-docs:
> 
> Stefan Hanreich (1):
>   firewall: add documentation for proxmox-firewall
> 
>  pve-firewall.adoc | 181 ++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 181 insertions(+)
> 
> 
> Summary over all repositories:
>   5 files changed, 224 insertions(+), 13 deletions(-)
> 


applied, thanks!


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel