From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 750F01FF16F for ; Fri, 15 Nov 2024 13:13:17 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id E567812D75; Fri, 15 Nov 2024 13:12:52 +0100 (CET) Message-ID: Date: Fri, 15 Nov 2024 13:12:48 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: pve-devel@lists.proxmox.com References: <20241112122602.88598-1-s.hanreich@proxmox.com> Content-Language: en-US From: Stefan Hanreich In-Reply-To: <20241112122602.88598-1-s.hanreich@proxmox.com> X-SPAM-LEVEL: Spam detection results: 0 AWL 0.616 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_CSS_A 0.1 Contains URL's A record listed in the Spamhaus CSS blocklist [195.154.200.217] Subject: Re: [pve-devel] [PATCH docs/firewall/manager/proxmox{-ve-rs, -firewall, -perl-rs} v3 00/24] autogenerate ipsets for sdn objects X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" v4 here: https://lore.proxmox.com/pve-devel/20241115120937.169342-1-s.hanreich@proxmox.com/T/ On 11/12/24 13:25, Stefan Hanreich wrote: > This patch series adds support for autogenerating ipsets for SDN objects. It > autogenerates ipsets for every VNet as follows: > > * ipset containing all IP ranges of the VNet > * ipset containing all gateways of the VNet > * ipset containing all IP ranges of the subnet - except gateways > * ipset containing all dhcp ranges of the vnet > > Additionally it generates an IPSet for every guest that has one or more IPAM > entries in the pve IPAM. > > Those can then be used in the cluster / host / guest firewalls. Firewall rules > automatically update on changes of the SDN / IPAM configuration. This patch > series works for the old firewall as well as the new firewall. > > The ipsets in nftables currently get generated as named ipsets in every table, > this means that the `nft list ruleset` output can get quite crowded for large > SDN configurations or large IPAM databases. Another option would be to only > include them as anonymous IPsets in the rules, which would make the nft output > far less crowded but this way would use more memory when making extensive use of > the sdn ipsets, since everytime it is used in a rule we create an entirely new > ipset. > > The base for proxmox-ve-rs (which is a filtered version of the proxmox-firewall > repository can be found here:) > > staff/s.hanreich/proxmox-ve-rs.git master > > Dependencies: > * proxmox-perl-rs and proxmox-firewall depend on proxmox-ve-rs > * pve-firewall depends on proxmox-perl-rs > * pve-manager depends on pve-firewall > > Changes from v2: > * rename end in IpRange to last to avoid confusion - thanks @Wolfgang > * bump Rust to 1.82 - thanks @Wolfgang > * improvements to the code generating IPSets - thanks @Wolfgang > * implement AsRef for SDN name types - thanks @Wolfgang > * improve docstrings (proper capitalization and punctuation) - thanks @Wolfgang > * included a patch that removes proxmox-ve-config from proxmox-firewall > > Changes from RFC: > * added documentation > * added separate SDN scope for IPSets > * rustfmt fixes > > proxmox-ve-rs: > > Stefan Hanreich (16): > debian: add files for packaging > firewall: add sdn scope for ipsets > firewall: add ip range types > firewall: address: use new iprange type for ip entries > ipset: add range variant to addresses > iprange: add methods for converting an ip range to cidrs > ipset: address: add helper methods > firewall: guest: derive traits according to rust api guidelines > common: add allowlist > sdn: add name types > sdn: add ipam module > sdn: ipam: add method for generating ipsets > sdn: add config module > sdn: config: add method for generating ipsets > tests: add sdn config tests > tests: add ipam tests > > .cargo/config.toml | 5 + > .gitignore | 8 + > Cargo.toml | 17 + > Makefile | 69 + > build.sh | 35 + > bump.sh | 44 + > proxmox-ve-config/Cargo.toml | 19 +- > proxmox-ve-config/debian/changelog | 5 + > proxmox-ve-config/debian/control | 46 + > proxmox-ve-config/debian/copyright | 19 + > proxmox-ve-config/debian/debcargo.toml | 4 + > proxmox-ve-config/src/common/mod.rs | 31 + > .../src/firewall/types/address.rs | 1171 ++++++++++++++++- > proxmox-ve-config/src/firewall/types/alias.rs | 4 +- > proxmox-ve-config/src/firewall/types/ipset.rs | 32 +- > proxmox-ve-config/src/firewall/types/rule.rs | 6 +- > proxmox-ve-config/src/guest/types.rs | 7 +- > proxmox-ve-config/src/guest/vm.rs | 11 +- > proxmox-ve-config/src/lib.rs | 2 + > proxmox-ve-config/src/sdn/config.rs | 640 +++++++++ > proxmox-ve-config/src/sdn/ipam.rs | 368 ++++++ > proxmox-ve-config/src/sdn/mod.rs | 251 ++++ > proxmox-ve-config/tests/sdn/main.rs | 189 +++ > proxmox-ve-config/tests/sdn/resources/ipam.db | 26 + > .../tests/sdn/resources/running-config.json | 54 + > 25 files changed, 2976 insertions(+), 87 deletions(-) > create mode 100644 .cargo/config.toml > create mode 100644 .gitignore > create mode 100644 Cargo.toml > create mode 100644 Makefile > create mode 100755 build.sh > create mode 100755 bump.sh > create mode 100644 proxmox-ve-config/debian/changelog > create mode 100644 proxmox-ve-config/debian/control > create mode 100644 proxmox-ve-config/debian/copyright > create mode 100644 proxmox-ve-config/debian/debcargo.toml > create mode 100644 proxmox-ve-config/src/common/mod.rs > create mode 100644 proxmox-ve-config/src/sdn/config.rs > create mode 100644 proxmox-ve-config/src/sdn/ipam.rs > create mode 100644 proxmox-ve-config/src/sdn/mod.rs > create mode 100644 proxmox-ve-config/tests/sdn/main.rs > create mode 100644 proxmox-ve-config/tests/sdn/resources/ipam.db > create mode 100644 proxmox-ve-config/tests/sdn/resources/running-config.json > > > proxmox-firewall: > > Stefan Hanreich (3): > add proxmox-ve-rs crate - move proxmox-ve-config there > config: tests: add support for loading sdn and ipam config > ipsets: autogenerate ipsets for vnets and ipam > > Cargo.toml | 4 +- > Makefile | 2 +- > proxmox-firewall/Cargo.toml | 2 +- > proxmox-firewall/src/config.rs | 69 + > proxmox-firewall/src/firewall.rs | 22 +- > proxmox-firewall/src/object.rs | 41 +- > .../tests/input/.running-config.json | 45 + > proxmox-firewall/tests/input/ipam.db | 32 + > proxmox-firewall/tests/integration_tests.rs | 10 + > .../integration_tests__firewall.snap | 1288 +++++++++++++++++ > proxmox-nftables/Cargo.toml | 2 +- > proxmox-nftables/src/expression.rs | 17 +- > proxmox-nftables/src/types.rs | 2 +- > proxmox-ve-config/Cargo.toml | 25 - > proxmox-ve-config/resources/ct_helper.json | 52 - > proxmox-ve-config/resources/macros.json | 923 ------------ > proxmox-ve-config/src/firewall/cluster.rs | 374 ----- > proxmox-ve-config/src/firewall/common.rs | 184 --- > proxmox-ve-config/src/firewall/ct_helper.rs | 115 -- > proxmox-ve-config/src/firewall/fw_macros.rs | 69 - > proxmox-ve-config/src/firewall/guest.rs | 237 --- > proxmox-ve-config/src/firewall/host.rs | 372 ----- > proxmox-ve-config/src/firewall/mod.rs | 10 - > proxmox-ve-config/src/firewall/parse.rs | 494 ------- > proxmox-ve-config/src/firewall/ports.rs | 80 - > .../src/firewall/types/address.rs | 615 -------- > proxmox-ve-config/src/firewall/types/alias.rs | 174 --- > proxmox-ve-config/src/firewall/types/group.rs | 36 - > proxmox-ve-config/src/firewall/types/ipset.rs | 349 ----- > proxmox-ve-config/src/firewall/types/log.rs | 222 --- > proxmox-ve-config/src/firewall/types/mod.rs | 14 - > proxmox-ve-config/src/firewall/types/port.rs | 181 --- > proxmox-ve-config/src/firewall/types/rule.rs | 412 ------ > .../src/firewall/types/rule_match.rs | 977 ------------- > proxmox-ve-config/src/guest/mod.rs | 115 -- > proxmox-ve-config/src/guest/types.rs | 38 - > proxmox-ve-config/src/guest/vm.rs | 510 ------- > proxmox-ve-config/src/host/mod.rs | 1 - > proxmox-ve-config/src/host/utils.rs | 70 - > proxmox-ve-config/src/lib.rs | 3 - > 40 files changed, 1517 insertions(+), 6671 deletions(-) > create mode 100644 proxmox-firewall/tests/input/.running-config.json > create mode 100644 proxmox-firewall/tests/input/ipam.db > delete mode 100644 proxmox-ve-config/Cargo.toml > delete mode 100644 proxmox-ve-config/resources/ct_helper.json > delete mode 100644 proxmox-ve-config/resources/macros.json > delete mode 100644 proxmox-ve-config/src/firewall/cluster.rs > delete mode 100644 proxmox-ve-config/src/firewall/common.rs > delete mode 100644 proxmox-ve-config/src/firewall/ct_helper.rs > delete mode 100644 proxmox-ve-config/src/firewall/fw_macros.rs > delete mode 100644 proxmox-ve-config/src/firewall/guest.rs > delete mode 100644 proxmox-ve-config/src/firewall/host.rs > delete mode 100644 proxmox-ve-config/src/firewall/mod.rs > delete mode 100644 proxmox-ve-config/src/firewall/parse.rs > delete mode 100644 proxmox-ve-config/src/firewall/ports.rs > delete mode 100644 proxmox-ve-config/src/firewall/types/address.rs > delete mode 100644 proxmox-ve-config/src/firewall/types/alias.rs > delete mode 100644 proxmox-ve-config/src/firewall/types/group.rs > delete mode 100644 proxmox-ve-config/src/firewall/types/ipset.rs > delete mode 100644 proxmox-ve-config/src/firewall/types/log.rs > delete mode 100644 proxmox-ve-config/src/firewall/types/mod.rs > delete mode 100644 proxmox-ve-config/src/firewall/types/port.rs > delete mode 100644 proxmox-ve-config/src/firewall/types/rule.rs > delete mode 100644 proxmox-ve-config/src/firewall/types/rule_match.rs > delete mode 100644 proxmox-ve-config/src/guest/mod.rs > delete mode 100644 proxmox-ve-config/src/guest/types.rs > delete mode 100644 proxmox-ve-config/src/guest/vm.rs > delete mode 100644 proxmox-ve-config/src/host/mod.rs > delete mode 100644 proxmox-ve-config/src/host/utils.rs > delete mode 100644 proxmox-ve-config/src/lib.rs > > > pve-firewall: > > Stefan Hanreich (2): > add support for loading sdn firewall configuration > api: load sdn ipsets > > src/PVE/API2/Firewall/Cluster.pm | 8 +++-- > src/PVE/API2/Firewall/Rules.pm | 12 ++++--- > src/PVE/API2/Firewall/VM.pm | 3 +- > src/PVE/Firewall.pm | 59 ++++++++++++++++++++++++++++---- > 4 files changed, 67 insertions(+), 15 deletions(-) > > > proxmox-perl-rs: > > Stefan Hanreich (1): > add PVE::RS::Firewall::SDN module > > pve-rs/Cargo.toml | 1 + > pve-rs/Makefile | 1 + > pve-rs/src/firewall/mod.rs | 1 + > pve-rs/src/firewall/sdn.rs | 130 +++++++++++++++++++++++++++++++++++++ > pve-rs/src/lib.rs | 1 + > 5 files changed, 134 insertions(+) > create mode 100644 pve-rs/src/firewall/mod.rs > create mode 100644 pve-rs/src/firewall/sdn.rs > > > pve-manager: > > Stefan Hanreich (1): > firewall: add sdn scope to IPRefSelector > > www/manager6/form/IPRefSelector.js | 8 +++++++- > 1 file changed, 7 insertions(+), 1 deletion(-) > > > pve-docs: > > Stefan Hanreich (1): > sdn: add documentation for firewall integration > > pvesdn.adoc | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 92 insertions(+) > > > Summary over all repositories: > 76 files changed, 4793 insertions(+), 6774 deletions(-) > _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel