From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pdm-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
by lore.proxmox.com (Postfix) with ESMTPS id 92D911FF164
for <inbox@lore.proxmox.com>; Fri, 28 Feb 2025 16:18:46 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 0DCA31208E;
Fri, 28 Feb 2025 16:18:39 +0100 (CET)
From: Stefan Hanreich <s.hanreich@proxmox.com>
To: pdm-devel@lists.proxmox.com
Date: Fri, 28 Feb 2025 16:17:37 +0100
Message-Id: <20250228151803.158984-1-s.hanreich@proxmox.com>
X-Mailer: git-send-email 2.39.5
MIME-Version: 1.0
X-SPAM-LEVEL: Spam detection results: 1
AWL -1.488 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery
methods
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_NONE 0.001 SPF: sender does not publish an SPF Record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
information. [generate.pl, controllers.rs, sdn.rs, types.rs, proxmox.com,
tasks.rs, utils.rs, lib.rs, vnets.rs, mod.rs, zones.rs, code.rs]
URIBL_DBL_SPAM 2.5 Contains a spam URL listed in the Spamhaus DBL blocklist
[tasks.rs]
Subject: [pdm-devel] [RFC proxmox{-api-types, -yew-comp,
-datacenter-manager} 00/26] Add initial SDN / EVPN integration
X-BeenThere: pdm-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Datacenter Manager development discussion
<pdm-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pdm-devel>,
<mailto:pdm-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pdm-devel/>
List-Post: <mailto:pdm-devel@lists.proxmox.com>
List-Help: <mailto:pdm-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel>,
<mailto:pdm-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox Datacenter Manager development discussion
<pdm-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pdm-devel-bounces@lists.proxmox.com
Sender: "pdm-devel" <pdm-devel-bounces@lists.proxmox.com>
## Introduction
This patch series introduces the first SDN features to the Proxmox Datacenter
Manager. I sent it as an RFC, since I wanted to get some early feedback and
nothing presented here is set in stone.
This patch series is mainly intended for laying the groundwork on how to handle
SDN configuration changes across multiple remotes. To demonstrate the concept it
implements two, relatively simple, features using the newly introduced lock
functionality:
* Creating a new VRF (= zone) across multiple remotes
* Creating a new VNet across multiple remotes
Additionally there is an aggregated overview of all EVPN zones / controllers of
all PVE remotes.
Once everything has been properly fleshed out, the groundwork laid here will be
used to build the full SDN/EVPN integration into PDM with more complex
configuration changes.
## Prerequisites
This patch series relies on the following preparatory patch series, that I've
sent separately:
https://lore.proxmox.com/pve-devel/20250228130549.100357-1-s.hanreich@proxmox.com/T/
https://lore.proxmox.com/pve-devel/20250228140136.124286-1-s.hanreich@proxmox.com/T/
https://lore.proxmox.com/pbs-devel/20250129104250.934927-1-s.hanreich@proxmox.com/T/
I've sent others, but as far as I can tell they have already been merged. Please
tell me if I missed something and you cannot build this patch series! Not all of
them have been bumped yet, so make sure to apply the patches on top of the
current master.
## How it works
I introduced a new locking mechanism to the SDN API, that locks the
configuration and enables subsequent modifications only when the lock secret is
provided. Locking the configuration only succeeds, if there are no pending
changes, but there's a knob for tuning that behavior (that we could expose in
the PDM UI as well). See my other patch series for more detailed information.
The two PDM API calls introduced in this patch series do the following sequence
of actions when trying to change the SDN configuration:
* lock the sdn configuration of all involved remotes
* if locking at least one remote fails, release the lock on all remotes and
return unsuccessfully without making any changes
* invoke the API endpoints to make the changes to the SDN configuration
* if a call fails, no further changes will be made to the remote and the
configuration will stay locked - no rollbacks
* no further changes will be made to the remote after one call fails
* apply the configuration on all remotes where the changes were successful
* any errors during applying the configuration on a remote will be logged
* the configuration will stay locked if applying the configuration fails
* reload the network configuration on all nodes where applying was succesful
* if reloading was unsuccessful, the configuration will be unlocked, but the
user will get an error message (This is also how applying the configuration
on a single cluster currently works).
* the task will wait for all remotes to finish reloading and only return
successfully if changing the configuration on all remotes was successful
We are doing no automatic rollback of configuration changes. If we want to
introduce automatic rollback from PDM, this is how we could do it with how the
new SDN lock functionality works: We lock the SDN configuration from PDM if
there are no pending changes. We then proceed to make our changes to the config.
If we run into any error, we can be sure that all pending changes were made by
PDM. This fact enables us to safely rollback the configuration changes we made
and unlock the SDN configuration. This could be opt-in as a global PDM setting
(or when starting the task).
In the case of network issues, we cannot really proceed with making
configuration changes or rolling back the configuration. The configuration
changes will stay on PVE side, and users can choose to unlock & rollback.
## Open issues
Currently, we need information about 3 types of entities in the UI (zones,
vnets, controllers). The PDM API calls make an API call for each of those types
to the remotes and aggregates them before returning them. This makes the calls
really expensive, since we need to make 3 API calls to every remote configured
in PDM.
In this series, the yew component makes this call to PDM once and stores all the
results, passing the entities to every child component. This way we do not need
to repeat those calls when opening an EditWindow, for instance. In the future I
want to add a proper caching mechanism on the server side. Since this patch
series was already quite extensive I've decided to cut it out of scope for now.
I will think a bit on the best way to implement caching for this and implement
it in a follow-up patch series or a v2.
The matching of zones to a VRF is based solely on their VRF VXLAN VNI, in the
future we should also consider the rt-import configuration key for matching VRFs
to each other.
## Dependencies
* proxmox-api-types depends on pve-network (from the separate patch series)
* proxmox-datacenter-manager depends on proxmox-api-types / proxmox-yew-comp
proxmox-api-types:
Stefan Hanreich (12):
sdn: add list/create zone endpoints
sdn: generate zones endpoints
sdn: add list/create vnet endpoints
sdn: generate vnet endpoints
sdn: add list/create controller endpoints
sdn: generate controller endpoints
sdn: add acquire/release lock endpoints
sdn: generate lock endpoints
sdn: add apply configuration endpoint
sdn: generate apply configuration endpoint
tasks: add helper for querying successfully finished tasks
sdn: add helpers for pending values
pve-api-types/generate.pl | 36 +
pve-api-types/src/generated/code.rs | 150 +-
pve-api-types/src/generated/types.rs | 5259 +++++++++++++++++---------
pve-api-types/src/lib.rs | 1 +
pve-api-types/src/sdn.rs | 33 +
pve-api-types/src/types/mod.rs | 4 +
6 files changed, 3609 insertions(+), 1874 deletions(-)
create mode 100644 pve-api-types/src/sdn.rs
proxmox-yew-comp:
Stefan Hanreich (1):
sdn: add descriptions for sdn tasks
src/utils.rs | 3 +++
1 file changed, 3 insertions(+)
proxmox-datacenter-manager:
Stefan Hanreich (13):
server: add locked sdn client and helper methods
api: sdn: add list_zones endpoint
api: sdn: add create_zone endpoint
api: sdn: add list_vnets endpoint
api: sdn: add create_vnet endpoint
api: sdn: add list_controllers endpoint
ui: add VrfTree component
ui: sdn: add RouterTable component
ui: sdn: add AddVnetWindow component
ui: sdn: add AddZoneWindow component
ui: sdn: add EvpnPanel
ui: sdn: add EvpnPanel to main menu
pve: sdn: add descriptions for sdn tasks
lib/pdm-api-types/Cargo.toml | 2 +
lib/pdm-api-types/src/lib.rs | 2 +
lib/pdm-api-types/src/sdn.rs | 168 +++++++++++++++++
lib/pdm-client/src/lib.rs | 64 +++++++
server/src/api/mod.rs | 2 +
server/src/api/sdn/controllers.rs | 78 ++++++++
server/src/api/sdn/mod.rs | 17 ++
server/src/api/sdn/vnets.rs | 151 ++++++++++++++++
server/src/api/sdn/zones.rs | 177 ++++++++++++++++++
server/src/lib.rs | 1 +
server/src/sdn_client.rs | 234 ++++++++++++++++++++++++
ui/src/lib.rs | 2 +
ui/src/main_menu.rs | 10 +
ui/src/sdn/evpn/add_vnet.rs | 216 ++++++++++++++++++++++
ui/src/sdn/evpn/add_zone.rs | 229 +++++++++++++++++++++++
ui/src/sdn/evpn/evpn_panel.rs | 249 +++++++++++++++++++++++++
ui/src/sdn/evpn/mod.rs | 14 ++
ui/src/sdn/evpn/router_table.rs | 125 +++++++++++++
ui/src/sdn/evpn/vrf_tree.rs | 291 ++++++++++++++++++++++++++++++
ui/src/sdn/mod.rs | 1 +
ui/src/tasks.rs | 2 +
21 files changed, 2035 insertions(+)
create mode 100644 lib/pdm-api-types/src/sdn.rs
create mode 100644 server/src/api/sdn/controllers.rs
create mode 100644 server/src/api/sdn/mod.rs
create mode 100644 server/src/api/sdn/vnets.rs
create mode 100644 server/src/api/sdn/zones.rs
create mode 100644 server/src/sdn_client.rs
create mode 100644 ui/src/sdn/evpn/add_vnet.rs
create mode 100644 ui/src/sdn/evpn/add_zone.rs
create mode 100644 ui/src/sdn/evpn/evpn_panel.rs
create mode 100644 ui/src/sdn/evpn/mod.rs
create mode 100644 ui/src/sdn/evpn/router_table.rs
create mode 100644 ui/src/sdn/evpn/vrf_tree.rs
create mode 100644 ui/src/sdn/mod.rs
Summary over all repositories:
28 files changed, 5647 insertions(+), 1874 deletions(-)
--
Generated by git-murpp 0.8.0
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel