From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 9906D1FF16F for ; Tue, 2 Sep 2025 16:10:32 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id A7C7B15D38; Tue, 2 Sep 2025 16:10:39 +0200 (CEST) Message-ID: <926ed3c7-a6f3-45b0-8d5c-d2caa47f7807@proxmox.com> Date: Tue, 2 Sep 2025 16:10:34 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: pdm-devel@lists.proxmox.com References: <20250829145313.329114-1-s.hanreich@proxmox.com> Content-Language: en-US From: Stefan Hanreich In-Reply-To: <20250829145313.329114-1-s.hanreich@proxmox.com> X-SPAM-LEVEL: Spam detection results: 0 AWL 0.561 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pdm-devel] superseded: [PATCH network/proxmox{, -backup, -api-types, -datacenter-manager} v2 00/30] Add initial SDN / EVPN integration X-BeenThere: pdm-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Datacenter Manager development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Datacenter Manager development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pdm-devel-bounces@lists.proxmox.com Sender: "pdm-devel" https://lore.proxmox.com/pdm-devel/20250902140956.228031-1-s.hanreich@proxmox.com/T/#t On 8/29/25 4:53 PM, Stefan Hanreich wrote: > ## Introduction > > This patch series adds a new panel to the PDM that shows an overview of the > current state of all EVPN zones across all remotes. It includes two different > tree views: > > * IP-VRFs: that shows the contents of all IP-VRFs (identified by their Route > Target = ASN:VNI) across all remotes. > * Zones: that shows the contents of a specific zone on a specific remote. > > For more information on the two tree views, consult the respective commits that > introduce the components. > > The panel also allows users to create new Zones / VNets on multiple remotes > simultaneously by utilizing the new SDN locking functionality. > > I have provided prebuilt packages on the share in the folder pdm-evpn. > > This patch series requires the ParallelFetcher patch series from Lukas in order > to work. > > ## API > > It introduces the following API endpoints on PDM: > > /sdn > GET /controllers - list the controllers of all remotes > GET /zones - list the zones of all remotes > POST /zones - create a zone on multiple remotes > GET /vnets - list the vnets of all remotes > POST /vnets - create a vnet on multiple remotes > > > ## Additional remarks > > This patch series contains some preparatory patches that are not directly > related to the implemented functionality: > > * One fix for proxmox-schema so values that are larger than i32 can be used in > the integer schema definition (required for e.g. 64-bit ASNs) > * Add JSONSchema to a lot of SDN API endpoints that were previously undocumented > > I have sent them initially as separate patch series, but since they are a hard > requirement for this patch series I have merged all of them into one patch > series now. This way it is easier to keep track of the requirements. > > > ## Open questions for reviewers > > * The LockedSdnClient(s) are abstractions for locked SDN remotes. I'm still a > bit unsure about the design / implementation but for future features I will be > making more complex changes across multiple remotes so I figured an abstraction > for this will come in handy in the future. > > I'd love some inputs / opinions on the API design as well as the general concept > of locking config -> making changes -> rolling back / applying. > > I will work on a more sophisticated implementation utilizing tokio-specific > functions in the following days, but I wanted to get the patch series out now > and validate the API / general idea. > > * We might wanna move the EvpnRouteTarget type out of the UI, even though it is > currently only used there. > > * Should we introduce a caching mechanism for the SDN API calls? > > I have shortly talked about this with @Lukas, but we decided against > implementing such a mechanism for now after some deliberation. > > Showing outdated information is particularly problematic with configuration, > especially because the create dialogues rely on that information. > > After creating a new zone / vnet we would have to hit the remotes anyway, in > order to be able to show the updated data immediately. > > The downside is of course a long load time for the EVPN panel, as well as a long > load if even one of the remotes is not available. > > For an initial release I think it is fine to go forward without caching and see > how it works out in practice based on reports from our users. Any input on this > matter would be greatly appreciated! > > > ## Future Work > * show the output of the new status API calls created by Gabriel in the views. > * add a functionality for grouping remotes together, instead of implicitly > grouping them based on ASN:VNI > * introduce a caching mechanism for the SDN API calls (?) > * integration tests with mocked SDN clients > * add some QoL to the UI (e.g expand/collapse all) > > > Huge thanks to @Lukas and @Dominik for helping me greatly on moving this patch > series forward the last few days! > > ## Changelog > > Changes from v1: > * detect legacy PVE remotes without SDN locking API capability > * remove already applied patch > * parallelize list endpoints via Lukas' ParallelFetcher > * reversed toolbar / grid order in EVPN panel > * updated and improved commit messages > * added missing translation macro invocations > * replaced thread_local in components > * store columns in component to avoid re-creating them on update > * add better error message in add_zone/vnet dialogues if there is no > controller / zone > * remove unused message from vrf/remote tree components > * use update_root_tree for restoring tree state > * moved EVPN above remotes in the main menu > * added instructions on how to unlock SDN configuration in cases of errors > > Changes from RFC v2: > * rebased on top of current master > * improved error handling for the yew components considerably > * tinkered with column sizes in the remote view > * preserve collapsed state on refresh > * fix SDN ID schema definition > * improved EVPN icon > * moved task descriptions from yew-comp to pdm > * improved default sorting order for the remote view > > Changes from RFC v1: > * overhauled the structure of the trees completely > * split the initial tree view into two distinct tree views > * changed the grouping of elements > * improved and unified the terms used across all UI elements > * improved toolbar design > * removed the controller data table, since the tree views should now include > that information > * improved locked SDN client and added a collection type for locked SDN clients > * improved error handling and logging considerably for the worker tasks > > > ## Dependencies: > pbs-api-types depends on proxmox-schema > proxmox-api-types depends on proxmox-schema > proxmox-backup depends on proxmox-schema > proxmox-datacenter-manager depends on proxmox-schema > > proxmox-api-types depends on pve-network > proxmox-datacenter-manager depends on proxmox-api-types > proxmox-datacenter-manager depends on pve-network > > proxmox: > > Stefan Hanreich (2): > schema: use i64 for minimum / maximum / default integer values > pbs-api-types: fix values for integer schemas > > pbs-api-types/src/datastore.rs | 6 +++--- > proxmox-schema/src/de/mod.rs | 3 +-- > proxmox-schema/src/de/verify.rs | 13 ++++++++----- > proxmox-schema/src/schema.rs | 18 +++++++++--------- > 4 files changed, 21 insertions(+), 19 deletions(-) > > > proxmox-backup: > > Stefan Hanreich (1): > api: change integer schema parameters to i64 > > pbs-tape/src/bin/pmt.rs | 6 +++--- > proxmox-backup-client/src/main.rs | 2 +- > pxar-bin/src/main.rs | 6 +++--- > src/api2/backup/upload_chunk.rs | 15 ++++++--------- > 4 files changed, 13 insertions(+), 16 deletions(-) > > > pve-network: > > Stefan Hanreich (6): > sdn: api: return null for rollback / lock endpoints > controllers: fix maximum value for ASN > api: add state standard option > api: controllers: update schema of endpoints > api: vnets: update schema of endpoints > api: zones: update schema of endpoints > > src/PVE/API2/Network/SDN.pm | 4 + > src/PVE/API2/Network/SDN/Controllers.pm | 116 +++++++++- > src/PVE/API2/Network/SDN/Vnets.pm | 92 +++++++- > src/PVE/API2/Network/SDN/Zones.pm | 203 ++++++++++++++++-- > src/PVE/Network/SDN.pm | 10 + > src/PVE/Network/SDN/Controllers/BgpPlugin.pm | 7 +- > src/PVE/Network/SDN/Controllers/EvpnPlugin.pm | 2 +- > src/PVE/Network/SDN/Controllers/IsisPlugin.pm | 6 +- > src/PVE/Network/SDN/VnetPlugin.pm | 21 +- > src/PVE/Network/SDN/Zones/EvpnPlugin.pm | 22 +- > src/PVE/Network/SDN/Zones/QinQPlugin.pm | 6 +- > src/PVE/Network/SDN/Zones/VlanPlugin.pm | 1 + > src/PVE/Network/SDN/Zones/VxlanPlugin.pm | 15 +- > 13 files changed, 457 insertions(+), 48 deletions(-) > > > proxmox-api-types: > > Stefan Hanreich (6): > sdn: add list/create zone endpoints > sdn: add list/create vnet endpoints > sdn: add list/create controller endpoints > sdn: add sdn configuration locking endpoints > tasks: add helper for querying successfully finished tasks > sdn: add helpers for pending values > > pve-api-types/generate.pl | 38 ++++++++++++++++++++++++++++++++++ > pve-api-types/src/lib.rs | 1 + > pve-api-types/src/sdn.rs | 33 +++++++++++++++++++++++++++++ > pve-api-types/src/types/mod.rs | 4 ++++ > 4 files changed, 76 insertions(+) > create mode 100644 pve-api-types/src/sdn.rs > > > proxmox-datacenter-manager: > > Stefan Hanreich (15): > server: add locked sdn client helpers > ui: pve: sdn: add descriptions for sdn tasks > api: sdn: add list_zones endpoint > api: sdn: add create_zone endpoint > api: sdn: add list_vnets endpoint > api: sdn: add create_vnet endpoint > api: sdn: add list_controllers endpoint > ui: sdn: add EvpnRouteTarget type > ui: sdn: add vnet icon > ui: sdn: add view for showing evpn zones > ui: sdn: add view for showing ip vrfs > ui: sdn: add component for creating evpn vnets > ui: sdn: add component for creatin evpn zones > ui: sdn: add evpn overview panel > ui: sdn: add evpn panel to main menu > > lib/pdm-api-types/Cargo.toml | 2 + > lib/pdm-api-types/src/lib.rs | 2 + > lib/pdm-api-types/src/sdn.rs | 168 +++++++++++ > lib/pdm-client/src/lib.rs | 61 ++++ > server/src/api/mod.rs | 2 + > server/src/api/sdn/controllers.rs | 112 +++++++ > server/src/api/sdn/mod.rs | 17 ++ > server/src/api/sdn/vnets.rs | 180 +++++++++++ > server/src/api/sdn/zones.rs | 204 +++++++++++++ > server/src/lib.rs | 1 + > server/src/sdn_client.rs | 427 ++++++++++++++++++++++++++ > ui/css/pdm.scss | 14 +- > ui/images/icon-sdn-vnet.svg | 6 + > ui/src/lib.rs | 2 + > ui/src/main_menu.rs | 10 + > ui/src/sdn/evpn/add_vnet.rs | 313 +++++++++++++++++++ > ui/src/sdn/evpn/add_zone.rs | 328 ++++++++++++++++++++ > ui/src/sdn/evpn/evpn_panel.rs | 275 +++++++++++++++++ > ui/src/sdn/evpn/mod.rs | 41 +++ > ui/src/sdn/evpn/remote_tree.rs | 480 ++++++++++++++++++++++++++++++ > ui/src/sdn/evpn/vrf_tree.rs | 409 +++++++++++++++++++++++++ > ui/src/sdn/mod.rs | 1 + > ui/src/tasks.rs | 4 + > 23 files changed, 3058 insertions(+), 1 deletion(-) > create mode 100644 lib/pdm-api-types/src/sdn.rs > create mode 100644 server/src/api/sdn/controllers.rs > create mode 100644 server/src/api/sdn/mod.rs > create mode 100644 server/src/api/sdn/vnets.rs > create mode 100644 server/src/api/sdn/zones.rs > create mode 100644 server/src/sdn_client.rs > create mode 100644 ui/images/icon-sdn-vnet.svg > create mode 100644 ui/src/sdn/evpn/add_vnet.rs > create mode 100644 ui/src/sdn/evpn/add_zone.rs > create mode 100644 ui/src/sdn/evpn/evpn_panel.rs > create mode 100644 ui/src/sdn/evpn/mod.rs > create mode 100644 ui/src/sdn/evpn/remote_tree.rs > create mode 100644 ui/src/sdn/evpn/vrf_tree.rs > create mode 100644 ui/src/sdn/mod.rs > > > Summary over all repositories: > 48 files changed, 3625 insertions(+), 84 deletions(-) > _______________________________________________ pdm-devel mailing list pdm-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel