[PATCH manager/network/proxmox{-ve-rs,-perl-rs} v6 00/24] Add support for route maps / prefix lists to SDN

[Stefan Hanreich <s.hanreich@proxmox.com>]

Following the off-list discussion between Thomas, Dominik and me this patch
series contains the remaining pve-manager patches from the initial series and
implements changes to the API structure of route maps and prefix lists. For more
details on the exact API changes, see the respective pve-network commits that
give an overview of the new API structure, as well as provide reasoning for why
they were changed.

Handling of sequence numbers in prefix list entries has been changed as well.
They are now required in the section config, but not the API. If users do not
explicitly set a sequence number in a prefix list entry, then it will be
auto-generated in the same way as FRR auto-generates them (highest existing
sequence number + 5).

To make reviewing easier, the changes have been made on top of the existing
commits, so it is easy to see what changed from the initial UI patches.

(omitted the rest of the cover letter since it only talks about already merged
stuff)

## Changelog

Changes from v5 (Thanks @Thomas, @Dominik, @Robert, @Lukas, @Gabriel):
* overhaul API structure for both route maps / prefix lists
* adapt UI to the new API structure
* add new CRUD endpoints for prefix list entries
* expose sequence number field in the UI
* implement sequence number auto-generation
* change the prefix >=/<= fields to integerfields
* change the route map order field to an integerfield
* implement backend validation for prefix list entries
* add tests for prefix list entry validation
* fix le and ge not getting deleted when left empty on updating

Changes from v4 (Thanks @Hannes):
* Properly persist route maps / prefix lists to running config
* Consider route maps / prefix lists when rolling back
* Consider route maps / prefix lists when doing a check if FRR needs to be
  reloaded
* Fix change detection for exit action field in the UI
* Fix issue with adding a prefix list if one had already been added in the UI
* Fix exit policy rendering in route map overview
* Add default setting to exit policy dropdown
* fix error message when failing to delete prefix list
* fix check_reference using wrong property string format when deleting prefix
  lists

Changes from v3:
* added 3 commits in ve-rs that were missing due to layer-8 errors in rebasing

Changes from v2(Thanks @Wolfgang, @Gabriel, @Hannes):
* Add UI integration for prefix list / route map generation
* Add route filter based on prefix lists to openfabric / OSPF
* integrate routemap in / out parameters in BGP / EVPN controller UI
* generate route maps / prefix lists in FRR dry-run
* improve validation in the backend considerably
* add protected flag to API endpoints that require elevated privileges
* fix jinja templates for FRR config due to minijinja whitespace handling 
  changes
* refactored IntegerWithSign into ModifyNumber

Changes from v1 (Thanks @Gabriel, @Hannes, @Wolfgang):
* rebase on top of current master
* fix newly introduced vtysh tests
* include missing access-control patch
* fix an error in the permission API path of GET /route-maps/{route-map-id}
* fix permission check in list route maps / prefix lists endpoint
* implement From instead of Into for section config to frr conversions
* replace core::* imports with std::*
* improve comments in both pve-rs modules
* use get() instead of iter().find() in get methods of both pve-rs modules
* use entry API when creating new entities in both pve-rs modules
* removed duplicate PrefixList implementation block
* fixed pending parameter in GET endpoints
* add route maps / prefix lists to has_pending_changes method
* fixed change detection for newly introduced fields in prefix lists / route
  maps
* fixed reserved id 'loopbacks_ips' for prefix lists (instead of reserving
  loopback_ips)
* properly pass delete parameter to the route map update pve-rs method
* remove additional prefix list / route map rendering methods and just use dump
  instead in the ve-config FRR integration tests
* improved documentation of the FRR route map generation logic, so it better
  explains *how* the configuration gets merged.
* added another test-case for EVPN zones with a controller with custom route-map
  + exit nodes
* implement exit action and call features of route maps
* jump into user-supplied route maps instead of replacing them directly, to
  avoid breaking exit-node setups if users do not recreate the auto-generated
  route map
* improve indentation of FRR template
* update tests to reflect changes w.r.t. FRR config generation
* improve error message on trying to GET non-existing route map entry
* move the tests from the frr module in route maps / prefix lists to
  the integration tests in proxmox-ve-config
* make order u16 instead of u32, because in FRR it is an u16 as well
* add unit tests to some new types
* change route map merging logic to overwrite existing route maps, if an entry
  with the same route map name exists in the section config
* added separate patch for PrefixListName::new, since the vtysh patch from
  gabriel hasn't been applied yet, but this patch series requires the new
  function


proxmox-ve-rs:

Stefan Hanreich (2):
  sdn: prefix lists: refactor section config and api format
  prefix lists: implement validation for prefix lists

 proxmox-ve-config/src/sdn/prefix_list.rs     | 551 ++++++++++++++++++-
 proxmox-ve-config/tests/prefix_lists/main.rs |  30 +-
 2 files changed, 550 insertions(+), 31 deletions(-)


proxmox-perl-rs:

Stefan Hanreich (4):
  sdn: prefix lists: refactor existing API endpoint
  sdn: prefix lists: add crud methods for prefix list entries
  sdn: prefix lists: validate prefix lists
  sdn: route maps: add route map list method

 pve-rs/src/bindings/sdn/prefix_lists.rs | 157 ++++++++++++++++++------
 pve-rs/src/bindings/sdn/route_maps.rs   |  27 ++++
 2 files changed, 148 insertions(+), 36 deletions(-)


pve-network:

Stefan Hanreich (2):
  api: refactor route map api structure
  api: refactor prefix list api structure

 src/PVE/API2/Network/SDN/Makefile             |   1 +
 src/PVE/API2/Network/SDN/PrefixLists.pm       | 145 +++---------
 src/PVE/API2/Network/SDN/PrefixLists/Makefile |   9 +
 .../Network/SDN/PrefixLists/PrefixList.pm     | 139 ++++++++++++
 .../SDN/PrefixLists/PrefixListEntry.pm        | 208 ++++++++++++++++++
 src/PVE/API2/Network/SDN/RouteMaps.pm         |  93 ++------
 src/PVE/API2/Network/SDN/RouteMaps/Makefile   |   2 +-
 .../API2/Network/SDN/RouteMaps/RouteMap.pm    |   7 +-
 .../Network/SDN/RouteMaps/RouteMapEntries.pm  | 142 ++++++++++++
 .../Network/SDN/RouteMaps/RouteMapEntry.pm    |   9 +-
 src/PVE/Network/SDN/PrefixLists.pm            |  80 ++++---
 11 files changed, 596 insertions(+), 239 deletions(-)
 create mode 100644 src/PVE/API2/Network/SDN/PrefixLists/Makefile
 create mode 100644 src/PVE/API2/Network/SDN/PrefixLists/PrefixList.pm
 create mode 100644 src/PVE/API2/Network/SDN/PrefixLists/PrefixListEntry.pm
 create mode 100644 src/PVE/API2/Network/SDN/RouteMaps/RouteMapEntries.pm


pve-manager:

Dominik Csapak (2):
  ui: sdn: prefix list: add missing subjects
  ui: sdn: prefix list: adapt to changed api structure

Stefan Hanreich (14):
  ui: sdn: add route map selector
  ui: sdn: add prefix list selector
  ui: sdn: add panel for managing prefix lists
  ui: sdn: add panel for managing route map entries
  ui: sdn: bgp controller: allow configuring route maps
  ui: sdn: evpn controller: allow configuring route maps
  ui: sdn: openfabric: add route filter
  ui: sdn: ospf: add route filter setting
  sdn: do not fail rendering record data if pending property is missing
  ui: sdn: route maps: adapt to new route map api structure
  ui: sdn: prefix lists: route maps: use integerfields for numbers
  ui: sdn: prefix list panel: reload data on deleting prefix list entry
  ui: prefix list panel: delete empty le and get properties
  ui: prefix list entry panel: make prefix required

 www/manager6/Makefile                         |   4 +
 www/manager6/Utils.js                         |   5 +-
 www/manager6/dc/Config.js                     |  16 +
 www/manager6/sdn/PrefixListPanel.js           | 386 +++++++
 www/manager6/sdn/PrefixListSelector.js        |  30 +
 www/manager6/sdn/RouteMapPanel.js             | 977 ++++++++++++++++++
 www/manager6/sdn/RouteMapSelector.js          |  39 +
 www/manager6/sdn/controllers/BgpEdit.js       |  14 +
 www/manager6/sdn/controllers/EvpnEdit.js      |  17 +
 .../sdn/fabrics/openfabric/FabricEdit.js      |   8 +
 www/manager6/sdn/fabrics/ospf/FabricEdit.js   |   8 +
 11 files changed, 1502 insertions(+), 2 deletions(-)
 create mode 100644 www/manager6/sdn/PrefixListPanel.js
 create mode 100644 www/manager6/sdn/PrefixListSelector.js
 create mode 100644 www/manager6/sdn/RouteMapPanel.js
 create mode 100644 www/manager6/sdn/RouteMapSelector.js


Summary over all repositories:
  26 files changed, 2796 insertions(+), 308 deletions(-)

-- 
Generated by murpp 0.11.0