Re: [pdm-devel] [PATCH network/proxmox{, -backup, -api-types, -datacenter-manager} v4 00/30] Add initial SDN / EVPN integration

[Dominik Csapak <d.csapak@proxmox.com>]

Changes look good to me (i diffed the older version to this one instead
of looking at every patch again)

Tested again, and everything went as expected.

Consider this version:

Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>

On 9/4/25 10:19 AM, Stefan Hanreich wrote:
> ## Introduction
> 
> This patch series adds a new panel to the PDM that shows an overview of the
> current state of all EVPN zones across all remotes. It includes two different
> tree views:
> 
> * IP-VRFs: that shows the contents of all IP-VRFs (identified by their Route
>    Target = ASN:VNI) across all remotes.
> * Zones: that shows the contents of a specific zone on a specific remote.
> 
> For more information on the two tree views, consult the respective commits that
> introduce the components.
> 
> The panel also allows users to create new Zones / VNets on multiple remotes
> simultaneously by utilizing the new SDN locking functionality.
> 
> I have provided prebuilt packages on the share in the folder pdm-evpn.
> 
> This patch series requires the ParallelFetcher patch series from Lukas in order
> to work.
> 
> ## API
> 
> It introduces the following API endpoints on PDM:
> 
> /sdn
>      GET /controllers - list the controllers of all remotes
>      GET /zones - list the zones of all remotes
>      POST /zones - create a zone on multiple remotes
>      GET /vnets - list the vnets of all remotes
>      POST /vnets - create a vnet on multiple remotes
> 
> 
> ## Additional remarks
> 
> This patch series contains some preparatory patches that are not directly
> related to the implemented functionality:
> 
> * One fix for proxmox-schema so values that are larger than i32 can be used in
>    the integer schema definition (required for e.g. 64-bit ASNs)
> * Add JSONSchema to a lot of SDN API endpoints that were previously undocumented
> 
> I have sent them initially as separate patch series, but since they are a hard
> requirement for this patch series I have merged all of them into one patch
> series now. This way it is easier to keep track of the requirements.
> 
> 
> ## Open questions for reviewers
> 
> * The LockedSdnClient(s) are abstractions for locked SDN remotes. I'm still a
> bit unsure about the design / implementation but for future features I will be
> making more complex changes across multiple remotes so I figured an abstraction
> for this will come in handy in the future.
> 
> I'd love some inputs / opinions on the API design as well as the general concept
> of locking config -> making changes -> rolling back / applying.
> 
> I will work on a more sophisticated implementation utilizing tokio-specific
> functions in the following days, but I wanted to get the patch series out now
> and validate the API / general idea.
> 
> * We might wanna move the EvpnRouteTarget type out of the UI, even though it is
> currently only used there.
> 
> * Should we introduce a caching mechanism for the SDN API calls?
> 
> I have shortly talked about this with @Lukas, but we decided against
> implementing such a mechanism for now after some deliberation.
> 
> Showing outdated information is particularly problematic with configuration,
> especially because the create dialogues rely on that information.
> 
> After creating a new zone / vnet we would have to hit the remotes anyway, in
> order to be able to show the updated data immediately.
> 
> The downside is of course a long load time for the EVPN panel, as well as a long
> load if even one of the remotes is not available.
> 
> For an initial release I think it is fine to go forward without caching and see
> how it works out in practice based on reports from our users. Any input on this
> matter would be greatly appreciated!
> 
> 
> ## Future Work
> * show the output of the new status API calls created by Gabriel in the views.
> * add a functionality for grouping remotes together, instead of implicitly
>    grouping them based on ASN:VNI
> * introduce a caching mechanism for the SDN API calls (?)
> * integration tests with mocked SDN clients
> * add some QoL to the UI (e.g expand/collapse all)
> 
> 
> Huge thanks to @Lukas and @Dominik for helping me greatly on moving this patch
> series forward the last few days!
> 
> ## Changelog
> 
> Changes from v3 (Thanks @Shannon, @Wolfang):
> * created dedicated verification functions for SDN IDs
> * improved / fixed regex for SDN ids in the process
> * improved API documentation for SDN zones in PVE
> * use new verification functions in PDM types as well
> * SDN client prints the correct remote when releasing the lock
> 
> Changes from v2:
> * detect invalid response from rollback endpoint to gracefully handle unpatched
>    libpve-network-api-perl
> * use create_toolbar instead of implementing a whole component
> * pass is_loading to Refresh button
> * show spinner on initial load instead of empty trees
> * improved default sorting order for remotes tree
> * sort PveClients in LockedSdnClients to provide ordered output
> * use HashSet for all list endpoints for deduplication and efficient filtering
> 
> Changes from v1:
> * detect legacy PVE remotes without SDN locking API capability
> * remove already applied patch
> * parallelize list endpoints via Lukas' ParallelFetcher
> * reversed toolbar / grid order in EVPN panel
> * updated and improved commit messages
> * added missing translation macro invocations
> * replaced thread_local in components
> * store columns in component to avoid re-creating them on update
> * add better error message in add_zone/vnet dialogues if there is no
>    controller / zone
> * remove unused message from vrf/remote tree components
> * use update_root_tree for restoring tree state
> * moved EVPN above remotes in the main menu
> * added instructions on how to unlock SDN configuration in cases of errors
> 
> Changes from RFC v2:
> * rebased on top of current master
> * improved error handling for the yew components considerably
> * tinkered with column sizes in the remote view
> * preserve collapsed state on refresh
> * fix SDN ID schema definition
> * improved EVPN icon
> * moved task descriptions from yew-comp to pdm
> * improved default sorting order for the remote view
> 
> Changes from RFC v1:
> * overhauled the structure of the trees completely
>    * split the initial tree view into two distinct tree views
>    * changed the grouping of elements
>    * improved and unified the terms used across all UI elements
> * improved toolbar design
> * removed the controller data table, since the tree views should now include
>    that information
> * improved locked SDN client and added a collection type for locked SDN clients
> * improved error handling and logging considerably for the worker tasks
> 
> 
> ## Dependencies:
> pbs-api-types depends on proxmox-schema
> proxmox-api-types depends on proxmox-schema
> proxmox-backup depends on proxmox-schema
> proxmox-datacenter-manager depends on proxmox-schema
> 
> proxmox-api-types depends on pve-network
> proxmox-datacenter-manager depends on proxmox-api-types
> proxmox-datacenter-manager depends on pve-network
> 
> proxmox:
> 
> Stefan Hanreich (2):
>    schema: use i64 for minimum / maximum / default integer values
>    pbs-api-types: fix values for integer schemas
> 
>   pbs-api-types/src/datastore.rs  |  6 +++---
>   proxmox-schema/src/de/mod.rs    |  3 +--
>   proxmox-schema/src/de/verify.rs | 13 ++++++++-----
>   proxmox-schema/src/schema.rs    | 18 +++++++++---------
>   4 files changed, 21 insertions(+), 19 deletions(-)
> 
> 
> proxmox-backup:
> 
> Stefan Hanreich (1):
>    api: change integer schema parameters to i64
> 
>   pbs-tape/src/bin/pmt.rs           |  6 +++---
>   proxmox-backup-client/src/main.rs |  2 +-
>   pxar-bin/src/main.rs              |  6 +++---
>   src/api2/backup/upload_chunk.rs   | 15 ++++++---------
>   4 files changed, 13 insertions(+), 16 deletions(-)
> 
> 
> pve-network:
> 
> Stefan Hanreich (6):
>    sdn: api: return null for rollback / lock endpoints
>    controllers: fix maximum value for ASN
>    api: add state standard option
>    api: controllers: update schema of endpoints
>    api: vnets: update schema of endpoints
>    api: zones: update schema of endpoints
> 
>   src/PVE/API2/Network/SDN.pm                   |   4 +
>   src/PVE/API2/Network/SDN/Controllers.pm       | 116 +++++++++-
>   src/PVE/API2/Network/SDN/Vnets.pm             |  92 +++++++-
>   src/PVE/API2/Network/SDN/Zones.pm             | 204 ++++++++++++++++--
>   src/PVE/Network/SDN.pm                        |  10 +
>   src/PVE/Network/SDN/Controllers/BgpPlugin.pm  |   7 +-
>   src/PVE/Network/SDN/Controllers/EvpnPlugin.pm |   2 +-
>   src/PVE/Network/SDN/Controllers/IsisPlugin.pm |   6 +-
>   src/PVE/Network/SDN/VnetPlugin.pm             |  21 +-
>   src/PVE/Network/SDN/Zones/EvpnPlugin.pm       |  22 +-
>   src/PVE/Network/SDN/Zones/QinQPlugin.pm       |   6 +-
>   src/PVE/Network/SDN/Zones/VlanPlugin.pm       |   1 +
>   src/PVE/Network/SDN/Zones/VxlanPlugin.pm      |  16 +-
>   13 files changed, 459 insertions(+), 48 deletions(-)
> 
> 
> proxmox-api-types:
> 
> Stefan Hanreich (6):
>    sdn: add list/create zone endpoints
>    sdn: add list/create vnet endpoints
>    sdn: add list/create controller endpoints
>    sdn: add sdn configuration locking endpoints
>    tasks: add helper for querying successfully finished tasks
>    sdn: add helpers for pending values
> 
>   pve-api-types/generate.pl            | 38 +++++++++++++++++++
>   pve-api-types/src/lib.rs             |  1 +
>   pve-api-types/src/sdn.rs             | 33 ++++++++++++++++
>   pve-api-types/src/types/mod.rs       |  4 ++
>   pve-api-types/src/types/verifiers.rs | 56 ++++++++++++++++++++++++++++
>   5 files changed, 132 insertions(+)
>   create mode 100644 pve-api-types/src/sdn.rs
> 
> 
> proxmox-datacenter-manager:
> 
> Stefan Hanreich (15):
>    server: add locked sdn client helpers
>    ui: pve: sdn: add descriptions for sdn tasks
>    api: sdn: add list_zones endpoint
>    api: sdn: add create_zone endpoint
>    api: sdn: add list_vnets endpoint
>    api: sdn: add create_vnet endpoint
>    api: sdn: add list_controllers endpoint
>    ui: sdn: add EvpnRouteTarget type
>    ui: sdn: add vnet icon
>    ui: sdn: add view for showing evpn zones
>    ui: sdn: add view for showing ip vrfs
>    ui: sdn: add component for creating evpn vnets
>    ui: sdn: add component for creatin evpn zones
>    ui: sdn: add evpn overview panel
>    ui: sdn: add evpn panel to main menu
> 
>   lib/pdm-api-types/Cargo.toml      |   2 +
>   lib/pdm-api-types/src/lib.rs      |   2 +
>   lib/pdm-api-types/src/sdn.rs      | 171 ++++++++++
>   lib/pdm-client/src/lib.rs         |  61 ++++
>   server/src/api/mod.rs             |   2 +
>   server/src/api/sdn/controllers.rs | 114 +++++++
>   server/src/api/sdn/mod.rs         |  17 +
>   server/src/api/sdn/vnets.rs       | 180 +++++++++++
>   server/src/api/sdn/zones.rs       | 206 +++++++++++++
>   server/src/lib.rs                 |   1 +
>   server/src/sdn_client.rs          | 432 ++++++++++++++++++++++++++
>   ui/css/pdm.scss                   |  14 +-
>   ui/images/icon-sdn-vnet.svg       |   6 +
>   ui/src/lib.rs                     |   2 +
>   ui/src/main_menu.rs               |  10 +
>   ui/src/sdn/evpn/add_vnet.rs       | 313 +++++++++++++++++++
>   ui/src/sdn/evpn/add_zone.rs       | 328 ++++++++++++++++++++
>   ui/src/sdn/evpn/evpn_panel.rs     | 262 ++++++++++++++++
>   ui/src/sdn/evpn/mod.rs            |  41 +++
>   ui/src/sdn/evpn/remote_tree.rs    | 496 ++++++++++++++++++++++++++++++
>   ui/src/sdn/evpn/vrf_tree.rs       | 409 ++++++++++++++++++++++++
>   ui/src/sdn/mod.rs                 |   1 +
>   ui/src/tasks.rs                   |   4 +
>   23 files changed, 3073 insertions(+), 1 deletion(-)
>   create mode 100644 lib/pdm-api-types/src/sdn.rs
>   create mode 100644 server/src/api/sdn/controllers.rs
>   create mode 100644 server/src/api/sdn/mod.rs
>   create mode 100644 server/src/api/sdn/vnets.rs
>   create mode 100644 server/src/api/sdn/zones.rs
>   create mode 100644 server/src/sdn_client.rs
>   create mode 100644 ui/images/icon-sdn-vnet.svg
>   create mode 100644 ui/src/sdn/evpn/add_vnet.rs
>   create mode 100644 ui/src/sdn/evpn/add_zone.rs
>   create mode 100644 ui/src/sdn/evpn/evpn_panel.rs
>   create mode 100644 ui/src/sdn/evpn/mod.rs
>   create mode 100644 ui/src/sdn/evpn/remote_tree.rs
>   create mode 100644 ui/src/sdn/evpn/vrf_tree.rs
>   create mode 100644 ui/src/sdn/mod.rs
> 
> 
> Summary over all repositories:
>    49 files changed, 3698 insertions(+), 84 deletions(-)
> 



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel