public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] [PATCH docs/firewall/manager/proxmox{-ve-rs, -firewall, -perl-rs} v3 00/24] autogenerate ipsets for sdn objects
@ 2024-11-12 12:25 Stefan Hanreich
  2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 01/24] debian: add files for packaging Stefan Hanreich
                   ` (23 more replies)
  0 siblings, 24 replies; 26+ messages in thread
From: Stefan Hanreich @ 2024-11-12 12:25 UTC (permalink / raw)
  To: pve-devel

This patch series adds support for autogenerating ipsets for SDN objects. It
autogenerates ipsets for every VNet as follows:

* ipset containing all IP ranges of the VNet
* ipset containing all gateways of the VNet
* ipset containing all IP ranges of the subnet - except gateways
* ipset containing all dhcp ranges of the vnet

Additionally it generates an IPSet for every guest that has one or more IPAM
entries in the pve IPAM.

Those can then be used in the cluster / host / guest firewalls. Firewall rules
automatically update on changes of the SDN / IPAM configuration. This patch
series works for the old firewall as well as the new firewall.

The ipsets in nftables currently get generated as named ipsets in every table,
this means that the `nft list ruleset` output can get quite crowded for large
SDN configurations or large IPAM databases. Another option would be to only
include them as anonymous IPsets in the rules, which would make the nft output
far less crowded but this way would use more memory when making extensive use of
the sdn ipsets, since everytime it is used in a rule we create an entirely new
ipset.

The base for proxmox-ve-rs (which is a filtered version of the proxmox-firewall
repository can be found here:)

staff/s.hanreich/proxmox-ve-rs.git master

Dependencies:
* proxmox-perl-rs and proxmox-firewall depend on proxmox-ve-rs
* pve-firewall depends on proxmox-perl-rs
* pve-manager depends on pve-firewall

Changes from v2:
* rename end in IpRange to last to avoid confusion - thanks @Wolfgang
* bump Rust to 1.82 - thanks @Wolfgang
* improvements to the code generating IPSets - thanks @Wolfgang
* implement AsRef<str> for SDN name types - thanks @Wolfgang
* improve docstrings (proper capitalization and punctuation) - thanks @Wolfgang
* included a patch that removes proxmox-ve-config from proxmox-firewall

Changes from RFC:
* added documentation
* added separate SDN scope for IPSets
* rustfmt fixes

proxmox-ve-rs:

Stefan Hanreich (16):
  debian: add files for packaging
  firewall: add sdn scope for ipsets
  firewall: add ip range types
  firewall: address: use new iprange type for ip entries
  ipset: add range variant to addresses
  iprange: add methods for converting an ip range to cidrs
  ipset: address: add helper methods
  firewall: guest: derive traits according to rust api guidelines
  common: add allowlist
  sdn: add name types
  sdn: add ipam module
  sdn: ipam: add method for generating ipsets
  sdn: add config module
  sdn: config: add method for generating ipsets
  tests: add sdn config tests
  tests: add ipam tests

 .cargo/config.toml                            |    5 +
 .gitignore                                    |    8 +
 Cargo.toml                                    |   17 +
 Makefile                                      |   69 +
 build.sh                                      |   35 +
 bump.sh                                       |   44 +
 proxmox-ve-config/Cargo.toml                  |   19 +-
 proxmox-ve-config/debian/changelog            |    5 +
 proxmox-ve-config/debian/control              |   46 +
 proxmox-ve-config/debian/copyright            |   19 +
 proxmox-ve-config/debian/debcargo.toml        |    4 +
 proxmox-ve-config/src/common/mod.rs           |   31 +
 .../src/firewall/types/address.rs             | 1171 ++++++++++++++++-
 proxmox-ve-config/src/firewall/types/alias.rs |    4 +-
 proxmox-ve-config/src/firewall/types/ipset.rs |   32 +-
 proxmox-ve-config/src/firewall/types/rule.rs  |    6 +-
 proxmox-ve-config/src/guest/types.rs          |    7 +-
 proxmox-ve-config/src/guest/vm.rs             |   11 +-
 proxmox-ve-config/src/lib.rs                  |    2 +
 proxmox-ve-config/src/sdn/config.rs           |  640 +++++++++
 proxmox-ve-config/src/sdn/ipam.rs             |  368 ++++++
 proxmox-ve-config/src/sdn/mod.rs              |  251 ++++
 proxmox-ve-config/tests/sdn/main.rs           |  189 +++
 proxmox-ve-config/tests/sdn/resources/ipam.db |   26 +
 .../tests/sdn/resources/running-config.json   |   54 +
 25 files changed, 2976 insertions(+), 87 deletions(-)
 create mode 100644 .cargo/config.toml
 create mode 100644 .gitignore
 create mode 100644 Cargo.toml
 create mode 100644 Makefile
 create mode 100755 build.sh
 create mode 100755 bump.sh
 create mode 100644 proxmox-ve-config/debian/changelog
 create mode 100644 proxmox-ve-config/debian/control
 create mode 100644 proxmox-ve-config/debian/copyright
 create mode 100644 proxmox-ve-config/debian/debcargo.toml
 create mode 100644 proxmox-ve-config/src/common/mod.rs
 create mode 100644 proxmox-ve-config/src/sdn/config.rs
 create mode 100644 proxmox-ve-config/src/sdn/ipam.rs
 create mode 100644 proxmox-ve-config/src/sdn/mod.rs
 create mode 100644 proxmox-ve-config/tests/sdn/main.rs
 create mode 100644 proxmox-ve-config/tests/sdn/resources/ipam.db
 create mode 100644 proxmox-ve-config/tests/sdn/resources/running-config.json


proxmox-firewall:

Stefan Hanreich (3):
  add proxmox-ve-rs crate - move proxmox-ve-config there
  config: tests: add support for loading sdn and ipam config
  ipsets: autogenerate ipsets for vnets and ipam

 Cargo.toml                                    |    4 +-
 Makefile                                      |    2 +-
 proxmox-firewall/Cargo.toml                   |    2 +-
 proxmox-firewall/src/config.rs                |   69 +
 proxmox-firewall/src/firewall.rs              |   22 +-
 proxmox-firewall/src/object.rs                |   41 +-
 .../tests/input/.running-config.json          |   45 +
 proxmox-firewall/tests/input/ipam.db          |   32 +
 proxmox-firewall/tests/integration_tests.rs   |   10 +
 .../integration_tests__firewall.snap          | 1288 +++++++++++++++++
 proxmox-nftables/Cargo.toml                   |    2 +-
 proxmox-nftables/src/expression.rs            |   17 +-
 proxmox-nftables/src/types.rs                 |    2 +-
 proxmox-ve-config/Cargo.toml                  |   25 -
 proxmox-ve-config/resources/ct_helper.json    |   52 -
 proxmox-ve-config/resources/macros.json       |  923 ------------
 proxmox-ve-config/src/firewall/cluster.rs     |  374 -----
 proxmox-ve-config/src/firewall/common.rs      |  184 ---
 proxmox-ve-config/src/firewall/ct_helper.rs   |  115 --
 proxmox-ve-config/src/firewall/fw_macros.rs   |   69 -
 proxmox-ve-config/src/firewall/guest.rs       |  237 ---
 proxmox-ve-config/src/firewall/host.rs        |  372 -----
 proxmox-ve-config/src/firewall/mod.rs         |   10 -
 proxmox-ve-config/src/firewall/parse.rs       |  494 -------
 proxmox-ve-config/src/firewall/ports.rs       |   80 -
 .../src/firewall/types/address.rs             |  615 --------
 proxmox-ve-config/src/firewall/types/alias.rs |  174 ---
 proxmox-ve-config/src/firewall/types/group.rs |   36 -
 proxmox-ve-config/src/firewall/types/ipset.rs |  349 -----
 proxmox-ve-config/src/firewall/types/log.rs   |  222 ---
 proxmox-ve-config/src/firewall/types/mod.rs   |   14 -
 proxmox-ve-config/src/firewall/types/port.rs  |  181 ---
 proxmox-ve-config/src/firewall/types/rule.rs  |  412 ------
 .../src/firewall/types/rule_match.rs          |  977 -------------
 proxmox-ve-config/src/guest/mod.rs            |  115 --
 proxmox-ve-config/src/guest/types.rs          |   38 -
 proxmox-ve-config/src/guest/vm.rs             |  510 -------
 proxmox-ve-config/src/host/mod.rs             |    1 -
 proxmox-ve-config/src/host/utils.rs           |   70 -
 proxmox-ve-config/src/lib.rs                  |    3 -
 40 files changed, 1517 insertions(+), 6671 deletions(-)
 create mode 100644 proxmox-firewall/tests/input/.running-config.json
 create mode 100644 proxmox-firewall/tests/input/ipam.db
 delete mode 100644 proxmox-ve-config/Cargo.toml
 delete mode 100644 proxmox-ve-config/resources/ct_helper.json
 delete mode 100644 proxmox-ve-config/resources/macros.json
 delete mode 100644 proxmox-ve-config/src/firewall/cluster.rs
 delete mode 100644 proxmox-ve-config/src/firewall/common.rs
 delete mode 100644 proxmox-ve-config/src/firewall/ct_helper.rs
 delete mode 100644 proxmox-ve-config/src/firewall/fw_macros.rs
 delete mode 100644 proxmox-ve-config/src/firewall/guest.rs
 delete mode 100644 proxmox-ve-config/src/firewall/host.rs
 delete mode 100644 proxmox-ve-config/src/firewall/mod.rs
 delete mode 100644 proxmox-ve-config/src/firewall/parse.rs
 delete mode 100644 proxmox-ve-config/src/firewall/ports.rs
 delete mode 100644 proxmox-ve-config/src/firewall/types/address.rs
 delete mode 100644 proxmox-ve-config/src/firewall/types/alias.rs
 delete mode 100644 proxmox-ve-config/src/firewall/types/group.rs
 delete mode 100644 proxmox-ve-config/src/firewall/types/ipset.rs
 delete mode 100644 proxmox-ve-config/src/firewall/types/log.rs
 delete mode 100644 proxmox-ve-config/src/firewall/types/mod.rs
 delete mode 100644 proxmox-ve-config/src/firewall/types/port.rs
 delete mode 100644 proxmox-ve-config/src/firewall/types/rule.rs
 delete mode 100644 proxmox-ve-config/src/firewall/types/rule_match.rs
 delete mode 100644 proxmox-ve-config/src/guest/mod.rs
 delete mode 100644 proxmox-ve-config/src/guest/types.rs
 delete mode 100644 proxmox-ve-config/src/guest/vm.rs
 delete mode 100644 proxmox-ve-config/src/host/mod.rs
 delete mode 100644 proxmox-ve-config/src/host/utils.rs
 delete mode 100644 proxmox-ve-config/src/lib.rs


pve-firewall:

Stefan Hanreich (2):
  add support for loading sdn firewall configuration
  api: load sdn ipsets

 src/PVE/API2/Firewall/Cluster.pm |  8 +++--
 src/PVE/API2/Firewall/Rules.pm   | 12 ++++---
 src/PVE/API2/Firewall/VM.pm      |  3 +-
 src/PVE/Firewall.pm              | 59 ++++++++++++++++++++++++++++----
 4 files changed, 67 insertions(+), 15 deletions(-)


proxmox-perl-rs:

Stefan Hanreich (1):
  add PVE::RS::Firewall::SDN module

 pve-rs/Cargo.toml          |   1 +
 pve-rs/Makefile            |   1 +
 pve-rs/src/firewall/mod.rs |   1 +
 pve-rs/src/firewall/sdn.rs | 130 +++++++++++++++++++++++++++++++++++++
 pve-rs/src/lib.rs          |   1 +
 5 files changed, 134 insertions(+)
 create mode 100644 pve-rs/src/firewall/mod.rs
 create mode 100644 pve-rs/src/firewall/sdn.rs


pve-manager:

Stefan Hanreich (1):
  firewall: add sdn scope to IPRefSelector

 www/manager6/form/IPRefSelector.js | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)


pve-docs:

Stefan Hanreich (1):
  sdn: add documentation for firewall integration

 pvesdn.adoc | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 92 insertions(+)


Summary over all repositories:
  76 files changed, 4793 insertions(+), 6774 deletions(-)

-- 
Generated by git-murpp 0.6.0

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2024-11-12 19:16 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-11-12 12:25 [pve-devel] [PATCH docs/firewall/manager/proxmox{-ve-rs, -firewall, -perl-rs} v3 00/24] autogenerate ipsets for sdn objects Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 01/24] debian: add files for packaging Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 02/24] firewall: add sdn scope for ipsets Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 03/24] firewall: add ip range types Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 04/24] firewall: address: use new iprange type for ip entries Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 05/24] ipset: add range variant to addresses Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 06/24] iprange: add methods for converting an ip range to cidrs Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 07/24] ipset: address: add helper methods Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 08/24] firewall: guest: derive traits according to rust api guidelines Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 09/24] common: add allowlist Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 10/24] sdn: add name types Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 11/24] sdn: add ipam module Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 12/24] sdn: ipam: add method for generating ipsets Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 13/24] sdn: add config module Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 14/24] sdn: config: add method for generating ipsets Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 15/24] tests: add sdn config tests Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-ve-rs v3 16/24] tests: add ipam tests Stefan Hanreich
2024-11-12 19:16   ` [pve-devel] partially-applied-series: " Thomas Lamprecht
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-firewall v3 17/24] add proxmox-ve-rs crate - move proxmox-ve-config there Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-firewall v3 18/24] config: tests: add support for loading sdn and ipam config Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH proxmox-firewall v3 19/24] ipsets: autogenerate ipsets for vnets and ipam Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH pve-firewall v3 20/24] add support for loading sdn firewall configuration Stefan Hanreich
2024-11-12 12:25 ` [pve-devel] [PATCH pve-firewall v3 21/24] api: load sdn ipsets Stefan Hanreich
2024-11-12 12:26 ` [pve-devel] [PATCH proxmox-perl-rs v3 22/24] add PVE::RS::Firewall::SDN module Stefan Hanreich
2024-11-12 12:26 ` [pve-devel] [PATCH pve-manager v3 23/24] firewall: add sdn scope to IPRefSelector Stefan Hanreich
2024-11-12 12:26 ` [pve-devel] [PATCH pve-docs v3 24/24] sdn: add documentation for firewall integration Stefan Hanreich

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal