From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 0E8011FF0ED for ; Fri, 31 Jul 2026 11:19:46 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id B815521480; Fri, 31 Jul 2026 11:19:45 +0200 (CEST) Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 Date: Fri, 31 Jul 2026 11:19:41 +0200 Subject: Superseded: Re: [RFC cluster/datacenter-manager/manager/proxmox 00/17] TLS Certificate Staging To: "Shannon Sterz" , Message-Id: X-Mailer: aerc 0.20.0 References: <20260611120327.257523-1-s.sterz@proxmox.com> In-Reply-To: <20260611120327.257523-1-s.sterz@proxmox.com> From: "Shannon Sterz" X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1785489571795 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.146 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_LOW -0.7 Sender listed at https://www.dnswl.org/, low trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: N5L44JQY53MJ3XDXG42A3DVUZIMXXRLO X-Message-ID-Hash: N5L44JQY53MJ3XDXG42A3DVUZIMXXRLO X-MailFrom: s.sterz@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Datacenter Manager development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Supersed-by: https://lore.proxmox.com/pdm-devel/20260731091655.93282-2-s.st= erz@proxmox.com/T/#t On Thu Jun 11, 2026 at 2:03 PM CEST, Shannon Sterz wrote: > the aim of this series is to allow clients to automatically adapt to regu= lar > certificate rotation. the top-level overview of the mechanism proposed he= re is > as follows: > > - hosts that rotate their certificate create a new certificate at the ear= liest > four weeks before their current certificate expires. this certificate i= s > considered as "staged" up until it becomes actively used. > - clients can query a host for a staged certificate at any moment, the ho= st > will provide information such as the fingerprint for the active and sta= ged > certificate(s). > - at the earliest two weeks before their current certificate expires, hos= ts may > start using the "staged" certificate. the two week window is needed to = give > clients enough time to query a potential staged certificate. > - clients, that use fingerprints to validate a TLS certificate, should di= scard > the previously used fingerprint and update to the new certificate's > fingerprint (the previously staged certificate) as soon as they detect = its > usage. connections trying to authenticate themselves with the old Certi= ficate > should be rejected at this point. > > this series implements the host part of this mechanism for pve 9 and pdm.= the > first three patches in the series are intended for pve and implement the > staging mechanism. they also make it a little easier to query the certifi= cate > of a node when we don't know the node name specifically. > > the next few patches improve how fingerprints are handled for the > proxmox-client and pdm specifically. they also add the certificate info > endpoint to the pve client. specifically the following improvements are > provided: > > * if a fingerprint is provided, never fall back to the system's trust sto= re. > providing proper pinning semantics (patches 4 and 6) > * if a fingerprint of a remote does not match, but pdm-client is in inter= active > mode, allow a user to accept the updated fingerprint then and there. th= is > better matches the behaviour in interactive mode of connecting to a > non-trusted node (patch 7). > * report mismatching fingerprints as untrusted when probing a remote and > improve how the ui handles such situations by adding more context (patc= hes > 9-11) > > the remaining commits mostly prepare and the implement the rotation mecha= nism > within pdm. pdm will query pve remote nodes once every twelve hours to se= e if a > new staged certificate becomes available. if a new fingerprint is encount= ered, > it will be stored in the remotes.cfg. once a staged fingerprint is encoun= tered, > it will replace the active fingerprint. > > How to Test > ----------- > > the easiest way is probably to force pve to rotate and stage certificates= by > setting a date with `date --set` that's far enough in the future to trigg= er the > action and then running `pveupdate`. to force pdm to query its remotes, i= t's > easiest to run `systemctl restart proxmox-datacenter-api.service`. the da= emon > will execute the task query its remotes once on start. > > How to Apply & Bump > ------------------- > > the first patch for pve-manager (02/17) depends on the changes for pve-cl= uster > (01/17). the second pve-manager patch can be applied independently. > > the patches for proxmox-datacenter-manager can all be applied independent= ly, > with the exception of the last one (17/17), which needs the patch for the > pve-api-types (05/17) to be applied and bumped. > > Future Work > ----------- > > 1. pbs remotes currently do not rotate their certificates. a series that = is as > of yet not applied would add such a mechanism to pbs too. for now pbs rem= otes > are ignored by the staged certificates mechanism for the most part. > > 2. backporting of the pve patches to the bookworm branch probably makes s= ense > to improve compatibility. i'll send such patches once this series is. > > 3. somewhat orthogonal to this series: the mechanism outlined in the note= s of > patch 16 would probably improve adding tasks to pdm. > > > pve-cluster: > > Shannon Sterz (1): > setup: allow caller to provide the certificate filename > > src/PVE/Cluster/Setup.pm | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > > pve-manager: > > Shannon Sterz (2): > bin/api: add a new staged certificate when renewing self-signed cert > api: certificates: if node parameter is 'localhost' return local certs > > PVE/API2/Certificates.pm | 10 +++++++-- > PVE/CertHelpers.pm | 6 ++++++ > bin/pveupdate | 44 ++++++++++++++++++++++++++++++++-------- > 3 files changed, 49 insertions(+), 11 deletions(-) > > > proxmox: > > Shannon Sterz (2): > client: ignore certificate trust store validation result on fp option > pve-api-types: expose certificates info endpoint > > proxmox-client/src/client.rs | 5 +---- > pve-api-types/Cargo.toml | 1 + > pve-api-types/generate.pl | 3 +++ > pve-api-types/src/generated/code.rs | 15 ++++++++++++++- > pve-api-types/src/types/mod.rs | 1 + > 5 files changed, 20 insertions(+), 5 deletions(-) > > > proxmox-datacenter-manager: > > Shannon Sterz (12): > client: don't short-circuit on valid certificate when tls fp exists > client: allow users to update a changed fingerprint interactively > cli/api-types: move Fingerprint to common api type crate > server: connection: report mismatching fingerprint as untrusted on > probe > ui: wizzard: add context if a provided fingerprint did not match > remote > ui: wizzard: nodes page: always update fingerprints on user > confirmation > pdm-api-types: implement ApiType for Fingerprint > pdm-api-types: add staged_fingerprints field to NodeUrl > server: remotes: lock remotes config when updating it > server: connection: rotate in staged fingerprints when encountering > them > server: api: tasks: move `spawn_aborted_on_shutdown()` to super module > server: bin: api: tasks: add task to discover new staged certificates > > cli/client/src/env/fingerprint_cache.rs | 91 ++-------- > cli/client/src/env/mod.rs | 10 +- > cli/client/src/main.rs | 6 +- > lib/pdm-api-types/Cargo.toml | 1 + > lib/pdm-api-types/src/fingerprint.rs | 84 +++++++++ > lib/pdm-api-types/src/lib.rs | 3 + > lib/pdm-api-types/src/remotes.rs | 15 +- > server/src/api/pbs/mod.rs | 2 + > server/src/api/pve/mod.rs | 3 + > server/src/api/remotes/mod.rs | 28 ++- > server/src/bin/proxmox-datacenter-api/main.rs | 1 + > .../tasks/ceph_detection.rs | 18 +- > .../bin/proxmox-datacenter-api/tasks/mod.rs | 17 ++ > .../tasks/remote_staged_fingerprints.rs | 149 ++++++++++++++++ > server/src/connection.rs | 166 +++++++++++++++--- > ui/src/remotes/config.rs | 1 + > ui/src/remotes/node_url_list.rs | 1 + > ui/src/remotes/wizard_page_connect.rs | 26 ++- > ui/src/remotes/wizard_page_info.rs | 1 + > ui/src/remotes/wizard_page_nodes.rs | 40 ++++- > 20 files changed, 525 insertions(+), 138 deletions(-) > create mode 100644 lib/pdm-api-types/src/fingerprint.rs > create mode 100644 server/src/bin/proxmox-datacenter-api/tasks/remote_st= aged_fingerprints.rs > > > Summary over all repositories: > 29 files changed, 596 insertions(+), 156 deletions(-)