Superseded: Re: [RFC cluster/datacenter-manager/manager/proxmox 00/17] TLS Certificate Staging

["Shannon Sterz" <s.sterz@proxmox.com>]

Supersed-by: https://lore.proxmox.com/pdm-devel/20260731091655.93282-2-s.sterz@proxmox.com/T/#t

On Thu Jun 11, 2026 at 2:03 PM CEST, Shannon Sterz wrote:
> the aim of this series is to allow clients to automatically adapt to regular
> certificate rotation. the top-level overview of the mechanism proposed here is
> as follows:
>
> - hosts that rotate their certificate create a new certificate at the earliest
>   four weeks before their current certificate expires. this certificate is
>   considered as "staged" up until it becomes actively used.
> - clients can query a host for a staged certificate at any moment, the host
>   will provide information such as the fingerprint for the active and staged
>   certificate(s).
> - at the earliest two weeks before their current certificate expires, hosts may
>   start using the "staged" certificate. the two week window is needed to give
>   clients enough time to query a potential staged certificate.
> - clients, that use fingerprints to validate a TLS certificate, should discard
>   the previously used fingerprint and update to the new certificate's
>   fingerprint (the previously staged certificate) as soon as they detect its
>   usage. connections trying to authenticate themselves with the old Certificate
>   should be rejected at this point.
>
> this series implements the host part of this mechanism for pve 9 and pdm. the
> first three patches in the series are intended for pve and implement the
> staging mechanism. they also make it a little easier to query the certificate
> of a node when we don't know the node name specifically.
>
> the next few patches improve how fingerprints are handled for the
> proxmox-client and pdm specifically. they also add the certificate info
> endpoint to the pve client. specifically the following improvements are
> provided:
>
> * if a fingerprint is provided, never fall back to the system's trust store.
>   providing proper pinning semantics (patches 4 and 6)
> * if a fingerprint of a remote does not match, but pdm-client is in interactive
>   mode, allow a user to accept the updated fingerprint then and there. this
>   better matches the behaviour in interactive mode of connecting to a
>   non-trusted node (patch 7).
> * report mismatching fingerprints as untrusted when probing a remote and
>   improve how the ui handles such situations by adding more context (patches
>   9-11)
>
> the remaining commits mostly prepare and the implement the rotation mechanism
> within pdm. pdm will query pve remote nodes once every twelve hours to see if a
> new staged certificate becomes available. if a new fingerprint is encountered,
> it will be stored in the remotes.cfg. once a staged fingerprint is encountered,
> it will replace the active fingerprint.
>
> How to Test
> -----------
>
> the easiest way is probably to force pve to rotate and stage certificates by
> setting a date with `date --set` that's far enough in the future to trigger the
> action and then running `pveupdate`. to force pdm to query its remotes, it's
> easiest to run `systemctl restart proxmox-datacenter-api.service`. the daemon
> will execute the task query its remotes once on start.
>
> How to Apply & Bump
> -------------------
>
> the first patch for pve-manager (02/17) depends on the changes for pve-cluster
> (01/17). the second pve-manager patch can be applied independently.
>
> the patches for proxmox-datacenter-manager can all be applied independently,
> with the exception of the last one (17/17), which needs the patch for the
> pve-api-types (05/17) to be applied and bumped.
>
> Future Work
> -----------
>
> 1. pbs remotes currently do not rotate their certificates. a series that is as
> of yet not applied would add such a mechanism to pbs too. for now pbs remotes
> are ignored by the staged certificates mechanism for the most part.
>
> 2. backporting of the pve patches to the bookworm branch probably makes sense
> to improve compatibility. i'll send such patches once this series is.
>
> 3. somewhat orthogonal to this series: the mechanism outlined in the notes of
> patch 16 would probably improve adding tasks to pdm.
>
>
> pve-cluster:
>
> Shannon Sterz (1):
>   setup: allow caller to provide the certificate filename
>
>  src/PVE/Cluster/Setup.pm | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
>
> pve-manager:
>
> Shannon Sterz (2):
>   bin/api: add a new staged certificate when renewing self-signed cert
>   api: certificates: if node parameter is 'localhost' return local certs
>
>  PVE/API2/Certificates.pm | 10 +++++++--
>  PVE/CertHelpers.pm       |  6 ++++++
>  bin/pveupdate            | 44 ++++++++++++++++++++++++++++++++--------
>  3 files changed, 49 insertions(+), 11 deletions(-)
>
>
> proxmox:
>
> Shannon Sterz (2):
>   client: ignore certificate trust store validation result on fp option
>   pve-api-types: expose certificates info endpoint
>
>  proxmox-client/src/client.rs        |  5 +----
>  pve-api-types/Cargo.toml            |  1 +
>  pve-api-types/generate.pl           |  3 +++
>  pve-api-types/src/generated/code.rs | 15 ++++++++++++++-
>  pve-api-types/src/types/mod.rs      |  1 +
>  5 files changed, 20 insertions(+), 5 deletions(-)
>
>
> proxmox-datacenter-manager:
>
> Shannon Sterz (12):
>   client: don't short-circuit on valid certificate when tls fp exists
>   client: allow users to update a changed fingerprint interactively
>   cli/api-types: move Fingerprint to common api type crate
>   server: connection: report mismatching fingerprint as untrusted on
>     probe
>   ui: wizzard: add context if a provided fingerprint did not match
>     remote
>   ui: wizzard: nodes page: always update fingerprints on user
>     confirmation
>   pdm-api-types: implement ApiType for Fingerprint
>   pdm-api-types: add staged_fingerprints field to NodeUrl
>   server: remotes: lock remotes config when updating it
>   server: connection: rotate in staged fingerprints when encountering
>     them
>   server: api: tasks: move `spawn_aborted_on_shutdown()` to super module
>   server: bin: api: tasks: add task to discover new staged certificates
>
>  cli/client/src/env/fingerprint_cache.rs       |  91 ++--------
>  cli/client/src/env/mod.rs                     |  10 +-
>  cli/client/src/main.rs                        |   6 +-
>  lib/pdm-api-types/Cargo.toml                  |   1 +
>  lib/pdm-api-types/src/fingerprint.rs          |  84 +++++++++
>  lib/pdm-api-types/src/lib.rs                  |   3 +
>  lib/pdm-api-types/src/remotes.rs              |  15 +-
>  server/src/api/pbs/mod.rs                     |   2 +
>  server/src/api/pve/mod.rs                     |   3 +
>  server/src/api/remotes/mod.rs                 |  28 ++-
>  server/src/bin/proxmox-datacenter-api/main.rs |   1 +
>  .../tasks/ceph_detection.rs                   |  18 +-
>  .../bin/proxmox-datacenter-api/tasks/mod.rs   |  17 ++
>  .../tasks/remote_staged_fingerprints.rs       | 149 ++++++++++++++++
>  server/src/connection.rs                      | 166 +++++++++++++++---
>  ui/src/remotes/config.rs                      |   1 +
>  ui/src/remotes/node_url_list.rs               |   1 +
>  ui/src/remotes/wizard_page_connect.rs         |  26 ++-
>  ui/src/remotes/wizard_page_info.rs            |   1 +
>  ui/src/remotes/wizard_page_nodes.rs           |  40 ++++-
>  20 files changed, 525 insertions(+), 138 deletions(-)
>  create mode 100644 lib/pdm-api-types/src/fingerprint.rs
>  create mode 100644 server/src/bin/proxmox-datacenter-api/tasks/remote_staged_fingerprints.rs
>
>
> Summary over all repositories:
>   29 files changed, 596 insertions(+), 156 deletions(-)