[PATCH access-control/cluster/docs/manager/network/proxmox{-ve-rs,-firewall,-perl-rs} v3 00/38] sdn: add microsegmentation support

[Hannes Laimer <h.laimer@proxmox.com>]

Guests on a vnet are separated only by IP addressing and the guest
firewall. Its address rules have to follow every move and renumbering.
No identity crosses nodes with a frame, and tags and names carry no
policy.

This series adds a group-based policy with a datacenter panel over
/etc/pve/sdn/microseg.cfg and /cluster/sdn/microseg. A rule has a
priority and a verdict and matches a source and a destination predicate
over groups, any, all or exact. It can be narrowed to a protocol and
ports, an ICMP type or a firewall macro. Assignments bind guest
interfaces to groups by guest, tag or name regex, and a subgroup member
is in every group above it. VXLAN and EVPN zones gain a vxlan-gbp option
that carries the identity across nodes. The host stamps it where a frame
enters the bridge, so a guest cannot forge its own.

Default is deny. Rules are stateless and directional, so a reply needs
its own allow, or the rule is marked both ways, which is exactly that
pair with the ports swapped. The highest priority wins, at a tie the
more specific all and exact predicates decide, and deny wins among
equals. Identities stay stable across config changes, and a retired one
is reused only after a quarantine. A full apply swaps a node's policy in
one transaction, and a guest interface is enforced before it is bridged.
A running config a node cannot read fails every guest start there, one
it cannot parse or build fails the start of guests with assigned
interfaces, as does an interface headed for an Open vSwitch port without
a firewall bridge. One already on such a port is named in the node's
network reload task and stays unenforced until the firewall option is
set. A guest matched after the last apply stays unmanaged until the next
one.

proxmox-ve-config carries the types, the identity engine and the desired
state, the policy in a form no enforcing mechanism depends on.
pve-cluster adds the observed file, pve-access-control the ACL path.
pve-network holds the config, the API, the zone option and the plug
trigger. proxmox-microseg, a library crate in the proxmox-firewall
workspace, renders the desired state into a bridge family nftables
table, and pve-rs binds it next to the config, so Perl applies it on an
SDN apply and on a NIC plug. pve-manager adds the panel and applies on
network reload. pve-docs adds the section. Connection tracking, rule
logging, the SRv6 carrier and an eBPF plane are left out on purpose.

libpve-network-perl loads only with the pve-cluster of this series and
needs its libpve-rs-perl, or every guest start on the node fails at the
plug. It gains dependencies on nftables and libpve-guest-common-perl.
The API package needs the libpve-access-control of this series, or no
ACL can be granted on /sdn/microseg and its endpoints only see what /sdn
passes down. pve-manager builds only with the pve-docs of this series,
whose anchors its help buttons name, and at runtime needs the new
libpve-network-perl, since its network reload applies the microseg
policy through it. pve-rs builds against the proxmox-microseg crate,
which pulls in proxmox-nftables. pve-rs and the crate both need a
proxmox-ve-config with the new module, and so does the proxmox-firewall
source package that carries the crate. None of these floors is bumped
here. The ifupdown2 vxlan-gbp attribute and four proxmox-nftables
patches (building without the config extension, mark-keyed verdict
maps, shared protocol rendering, crate packaging) are sent separately.

# changelog
v3:
 - data plane moved from eBPF programs to an nftables bridge-family
   table, proxmox-ebpf is gone and the data plane is a library crate in
   the proxmox-firewall workspace that pve-rs binds, without an agent
   binary or a boot-time unit since the first plug builds the table, the
   desired state it renders lives in proxmox-ve-config, and
   libpve-network-perl depends on nftables
 - rules can be narrowed to a protocol, ports, an ICMP type or a
   firewall macro, the protocol and port rendering is shared with the
   firewall
 - a rule can be marked both ways, which applies it with the predicates
   and the ports swapped as well, under one id, so the reply rule of a
   stateless pair no longer has to be written by hand
 - groups can form a hierarchy, membership in a subgroup implies the
   groups above it, and for all and exact predicates specificity breaks
   priority ties so a subgroup rule outranks its parent's
 - a plug fails when the running config cannot be read, or cannot be
   parsed or built and the NIC is assigned, or an assigned NIC would
   become an Open vSwitch port with no firewall bridge, an unreachable
   pmxcfs fails the plug instead of reading as an empty config, and a
   realized set without an identity fails the apply instead of passing
   as untagged
 - the microseg endpoints check their privileges on a new /sdn/microseg
   ACL path, added by a pve-access-control patch in the series
 - the overview endpoint reports identity id space usage, shown in the
   tree header
 - the pve-cluster patch also registers the file with the version array
   pmxcfs keeps for observed files
 - validation rejects mark 0 for a group, `untagged` in an all or exact
   destination, and `untagged` next to real groups in an all or exact
   source
 - an assignment naming an interface index is realized whether or not
   the guest has that interface yet, and tag matcher tags are
   normalized like guest tags
 - the running config stores the realized assignments per group set
   with per-guest interface lists, so the file stays small at scale
 - the network reload warns about an assigned interface that is already
   up on an Open vSwitch port without a firewall bridge, since only a
   plug can refuse one
 - two-column rule editor reusing the firewall's selectors, the tree
   shows subgroups
 - the SRv6 carrier is out of the series, it belongs with an SRv6 zone
 - the ifupdown2 `vxlan-gbp` attribute and the proxmox-nftables
   additions with the crate packaging are sent as standalone patches,
   the zone option stays in the series and the evpn vxlan-learning fix
   is folded into it
 - docs updated, cover letter rewritten
v2:
 - core model reworked: each interface now holds a *set* of groups
   compiled into a per-NIC wire identity, replacing the single-group
   parent/child tree
 - rules gained explicit `src`/`dst` predicates (any/all/exact) and a
   `prio`, replacing the implicit tree-distance ordering
 - `untagged` is now a nameable built-in group (mark 0)
 - assignments are additive: a NIC's set is the union of all that
   resolve to it
 - added tag-matcher and name-regex assignments
 - rule and dynamic-assignment ids are now generated
 - identity allocation is fresh-first with quarantined reclamation, so
   churn cannot exhaust the id space
 - split the SRv6 carrier support into droppable tail patches
 - updated docs
 - updated cover-letter


proxmox-ve-rs:

Hannes Laimer (8):
  ve-config: firewall: add a constructor for sctp matches
  ve-config: sdn: add microseg signature-identity engine
  ve-config: sdn: add microseg config types
  ve-config: sdn: microseg: add tag matcher
  ve-config: sdn: microseg: add name regex matcher
  ve-config: sdn: microseg: narrow rules by protocol and ports
  ve-config: sdn: microseg: add group hierarchy
  ve-config: sdn: microseg: add the desired state

 .../src/firewall/types/rule_match.rs          |    4 +
 .../src/sdn/microseg/identity.rs              |  625 +++
 proxmox-ve-config/src/sdn/microseg/mod.rs     | 3819 +++++++++++++++++
 proxmox-ve-config/src/sdn/microseg/state.rs   |  316 ++
 proxmox-ve-config/src/sdn/mod.rs              |    1 +
 5 files changed, 4765 insertions(+)
 create mode 100644 proxmox-ve-config/src/sdn/microseg/identity.rs
 create mode 100644 proxmox-ve-config/src/sdn/microseg/mod.rs
 create mode 100644 proxmox-ve-config/src/sdn/microseg/state.rs


proxmox-firewall:

Hannes Laimer (5):
  nftables: build without the config extension
  nftables: add support for mark-keyed verdict maps
  nftables: move protocol match rendering into the lib
  nftables: package the crate for debcargo
  microseg: add the SDN microsegmentation data plane

 .gitignore                                    |   4 +
 Cargo.toml                                    |   4 +
 Makefile                                      |  12 +-
 build.sh                                      |  46 ++
 proxmox-firewall/src/rule.rs                  | 166 +---
 proxmox-microseg/Cargo.toml                   |  22 +
 proxmox-microseg/debian/changelog             |   6 +
 proxmox-microseg/debian/control               |  46 ++
 proxmox-microseg/debian/copyright             |  18 +
 proxmox-microseg/debian/debcargo.toml         |   7 +
 proxmox-microseg/debian/source/format         |   1 +
 proxmox-microseg/src/dataplane.rs             |  12 +
 proxmox-microseg/src/lib.rs                   |  41 +
 proxmox-microseg/src/nft.rs                   | 458 +++++++++++
 proxmox-microseg/src/nic.rs                   |  54 ++
 proxmox-microseg/src/running_config.rs        |  28 +
 ..._tests__empty_state_removes_the_table.snap |  24 +
 ..._tests__full_apply_replaces_the_table.snap | 726 ++++++++++++++++++
 ...t__tests__nic_apply_adds_its_elements.snap |  46 ++
 proxmox-nftables/Cargo.toml                   |   7 +-
 proxmox-nftables/debian/changelog             |   6 +
 proxmox-nftables/debian/control               |  60 ++
 proxmox-nftables/debian/copyright             |  18 +
 proxmox-nftables/debian/debcargo.toml         |   7 +
 proxmox-nftables/debian/source/format         |   1 +
 proxmox-nftables/src/expression.rs            |   3 +-
 proxmox-nftables/src/lib.rs                   |   2 +
 proxmox-nftables/src/protocol.rs              |  70 ++
 proxmox-nftables/src/statement.rs             |  10 +
 proxmox-nftables/src/types.rs                 |   1 +
 30 files changed, 1748 insertions(+), 158 deletions(-)
 create mode 100755 build.sh
 create mode 100644 proxmox-microseg/Cargo.toml
 create mode 100644 proxmox-microseg/debian/changelog
 create mode 100644 proxmox-microseg/debian/control
 create mode 100644 proxmox-microseg/debian/copyright
 create mode 100644 proxmox-microseg/debian/debcargo.toml
 create mode 100644 proxmox-microseg/debian/source/format
 create mode 100644 proxmox-microseg/src/dataplane.rs
 create mode 100644 proxmox-microseg/src/lib.rs
 create mode 100644 proxmox-microseg/src/nft.rs
 create mode 100644 proxmox-microseg/src/nic.rs
 create mode 100644 proxmox-microseg/src/running_config.rs
 create mode 100644 proxmox-microseg/src/snapshots/proxmox_microseg__nft__tests__empty_state_removes_the_table.snap
 create mode 100644 proxmox-microseg/src/snapshots/proxmox_microseg__nft__tests__full_apply_replaces_the_table.snap
 create mode 100644 proxmox-microseg/src/snapshots/proxmox_microseg__nft__tests__nic_apply_adds_its_elements.snap
 create mode 100644 proxmox-nftables/debian/changelog
 create mode 100644 proxmox-nftables/debian/control
 create mode 100644 proxmox-nftables/debian/copyright
 create mode 100644 proxmox-nftables/debian/debcargo.toml
 create mode 100644 proxmox-nftables/debian/source/format
 create mode 100644 proxmox-nftables/src/protocol.rs


pve-cluster:

Hannes Laimer (1):
  cfs: add 'sdn/microseg.cfg' to observed files

 src/PVE/Cluster.pm  | 1 +
 src/pmxcfs/status.c | 1 +
 2 files changed, 2 insertions(+)


pve-access-control:

Hannes Laimer (1):
  permissions: add ACL path for SDN microsegmentation

 src/PVE/AccessControl.pm | 1 +
 1 file changed, 1 insertion(+)


proxmox-perl-rs:

Hannes Laimer (2):
  pve-rs: sdn: add microseg config binding
  pve-rs: sdn: microseg: bind the data plane

 pve-rs/Cargo.toml                   |   2 +
 pve-rs/Makefile                     |   1 +
 pve-rs/debian/control               |   1 +
 pve-rs/src/bindings/sdn/microseg.rs | 278 ++++++++++++++++++++++++++++
 pve-rs/src/bindings/sdn/mod.rs      |   1 +
 5 files changed, 283 insertions(+)
 create mode 100644 pve-rs/src/bindings/sdn/microseg.rs


pve-network:

Hannes Laimer (8):
  sdn: zones: add vxlan-gbp option to vxlan and evpn zones
  sdn: microseg: add config, API and guest inventory
  sdn: dry-run: show pending microseg changes
  sdn: zones: trigger microseg apply on tap_plug
  sdn: microseg: add tag matcher
  sdn: microseg: add name regex matcher
  sdn: microseg: expose protocol, port and macro rule options
  sdn: microseg: expose the group parent

 debian/control                                |   2 +
 src/PVE/API2/Network/SDN.pm                   |  38 +-
 src/PVE/API2/Network/SDN/Makefile             |   2 +
 src/PVE/API2/Network/SDN/Microseg.pm          | 277 +++++++
 .../API2/Network/SDN/Microseg/Assignment.pm   | 192 +++++
 src/PVE/API2/Network/SDN/Microseg/Group.pm    | 191 +++++
 src/PVE/API2/Network/SDN/Microseg/Makefile    |  10 +
 src/PVE/API2/Network/SDN/Microseg/Rule.pm     | 194 +++++
 src/PVE/API2/Network/SDN/Zones.pm             |   8 +
 src/PVE/Network/SDN.pm                        |  36 +
 src/PVE/Network/SDN/Makefile                  |   1 +
 src/PVE/Network/SDN/Microseg.pm               | 774 ++++++++++++++++++
 src/PVE/Network/SDN/Zones.pm                  |   5 +
 src/PVE/Network/SDN/Zones/EvpnPlugin.pm       |   5 +
 src/PVE/Network/SDN/Zones/VxlanPlugin.pm      |  10 +
 .../zones/evpn/gbp/expected_controller_config |  77 ++
 .../zones/evpn/gbp/expected_sdn_interfaces    |  45 +
 src/test/zones/evpn/gbp/interfaces            |   7 +
 src/test/zones/evpn/gbp/sdn_config            |  26 +
 .../zones/vxlan/gbp/expected_sdn_interfaces   |  16 +
 src/test/zones/vxlan/gbp/interfaces           |   7 +
 src/test/zones/vxlan/gbp/sdn_config           |  11 +
 22 files changed, 1932 insertions(+), 2 deletions(-)
 create mode 100644 src/PVE/API2/Network/SDN/Microseg.pm
 create mode 100644 src/PVE/API2/Network/SDN/Microseg/Assignment.pm
 create mode 100644 src/PVE/API2/Network/SDN/Microseg/Group.pm
 create mode 100644 src/PVE/API2/Network/SDN/Microseg/Makefile
 create mode 100644 src/PVE/API2/Network/SDN/Microseg/Rule.pm
 create mode 100644 src/PVE/Network/SDN/Microseg.pm
 create mode 100644 src/test/zones/evpn/gbp/expected_controller_config
 create mode 100644 src/test/zones/evpn/gbp/expected_sdn_interfaces
 create mode 100644 src/test/zones/evpn/gbp/interfaces
 create mode 100644 src/test/zones/evpn/gbp/sdn_config
 create mode 100644 src/test/zones/vxlan/gbp/expected_sdn_interfaces
 create mode 100644 src/test/zones/vxlan/gbp/interfaces
 create mode 100644 src/test/zones/vxlan/gbp/sdn_config


pve-manager:

Hannes Laimer (9):
  network: apply microseg state on reload
  ui: firewall: register the ICMP type stores by id
  ui: sdn: zones: add vxlan-gbp checkbox to vxlan and evpn
  ui: sdn: add microsegmentation panel
  ui: sdn: dry-run: show pending microseg diff
  ui: sdn: microseg: add tag matcher
  ui: sdn: microseg: add name regex matcher
  ui: sdn: microseg: add narrowing to the rule form
  ui: sdn: microseg: show and edit the group hierarchy

 PVE/API2/Network.pm                         |   4 +
 www/css/ext6-pve.css                        |   8 +
 www/manager6/Makefile                       |   8 +
 www/manager6/Utils.js                       |  34 ++
 www/manager6/data/PermPathStore.js          |   1 +
 www/manager6/dc/Config.js                   |   8 +
 www/manager6/form/MicrosegGroupSelector.js  |  95 ++++
 www/manager6/grid/FirewallRules.js          |   2 +
 www/manager6/sdn/MicrosegView.js            |  43 ++
 www/manager6/sdn/SdnDiffView.js             |  24 +
 www/manager6/sdn/microseg/AssignmentEdit.js | 238 ++++++++++
 www/manager6/sdn/microseg/Base.js           | 123 ++++++
 www/manager6/sdn/microseg/GroupEdit.js      |  47 ++
 www/manager6/sdn/microseg/PolicyView.js     | 401 +++++++++++++++++
 www/manager6/sdn/microseg/RuleEdit.js       | 285 ++++++++++++
 www/manager6/sdn/microseg/Tree.js           | 460 ++++++++++++++++++++
 www/manager6/sdn/zones/EvpnEdit.js          |   8 +
 www/manager6/sdn/zones/VxlanEdit.js         |  11 +
 18 files changed, 1800 insertions(+)
 create mode 100644 www/manager6/form/MicrosegGroupSelector.js
 create mode 100644 www/manager6/sdn/MicrosegView.js
 create mode 100644 www/manager6/sdn/microseg/AssignmentEdit.js
 create mode 100644 www/manager6/sdn/microseg/Base.js
 create mode 100644 www/manager6/sdn/microseg/GroupEdit.js
 create mode 100644 www/manager6/sdn/microseg/PolicyView.js
 create mode 100644 www/manager6/sdn/microseg/RuleEdit.js
 create mode 100644 www/manager6/sdn/microseg/Tree.js


pve-docs:

Hannes Laimer (4):
  sdn: add VXLAN-GBP flag to evpn/vxlan zone sections
  sdn: add microsegmentation section
  sdn: microseg: document narrowed rules
  sdn: microseg: document group hierarchy and the specificity tie-break

 pvesdn.adoc | 264 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 264 insertions(+)


Summary over all repositories:
  84 files changed, 10795 insertions(+), 160 deletions(-)

-- 
Generated by murpp 0.12.0