[PATCH cluster/datacenter-manager/manager/proxmox 00/15] TLS Certificate Staging

[Shannon Sterz <s.sterz@proxmox.com>]

the aim of this series is to allow clients to automatically adapt to regular
certificate rotation. the top-level overview of the mechanism proposed here is
as follows:

- hosts that rotate their certificate create a new certificate at the earliest
  four weeks before their current certificate expires. this certificate is
  considered as "staged" up until it becomes actively used.
- clients can query a host for a staged certificate at any moment, the host
  will provide information such as the fingerprint for the active and staged
  certificate(s).
- at the earliest two weeks before their current certificate expires, hosts may
  start using the "staged" certificate. the two week window is needed to give
  clients enough time to query a potential staged certificate.
- clients, that use fingerprints to validate a TLS certificate, should discard
  the previously used fingerprint and update to the new certificate's
  fingerprint (the previously staged certificate) as soon as they detect its
  usage. connections trying to authenticate themselves with the old Certificate
  should be rejected at this point.

this series implements the host part of this mechanism for pve 9 and pdm. the
first three patches in the series are intended for pve and implement the
staging mechanism. they also make it a little easier to query the certificate
of a node when we don't know the node name specifically.

the next few patches improve how fingerprints are handled for pdm. they also
add the certificate info endpoint to the pve client. specifically the following
improvements are provided:

* if a fingerprint of a remote does not match, but pdm-client is in interactive
  mode, allow a user to accept the updated fingerprint then and there. this
  better matches the behaviour in interactive mode of connecting to a
  non-trusted node (patch 5).
* report mismatching fingerprints as untrusted when probing a remote and
  improve how the ui handles such situations by adding more context (patches
  7-9)

the remaining patches mostly prepare and then implement the rotation mechanism
within pdm. pdm will query pve remote nodes once every twelve hours to see if a
new staged certificate becomes available. if a new fingerprint is encountered,
it will be stored in the remotes.cfg. once a staged fingerprint is encountered,
it will replace the active fingerprint.

How to Test
-----------

the easiest way is probably to force pve to rotate and stage certificates by
setting a date with `date --set` that's far enough in the future to trigger the
action and then running `pveupdate`. to force pdm to query its remotes, it's
easiest to run `systemctl restart proxmox-datacenter-api.service`. the daemon
will execute the task query its remotes once on start.

How to Apply & Bump
-------------------

the first patch for pve-manager (02/15) depends on the changes for pve-cluster
(01/15). the second pve-manager patch can be applied independently. note that
Elias' has send a patches [1] that would address the same issue as patch
(03/15), so that patch can be dropped in case elias's series makes it in first.

the patches for proxmox-datacenter-manager can all be applied independently,
with the exception of the last one (15/15), which needs the patch for the
pve-api-types (04/15) to be applied and bumped.

Future Work
-----------

1. pbs remotes currently do not rotate their certificates. a series that is as
of yet not applied would add such a mechanism to pbs too [1]. for now pbs
remotes are ignored by the staged certificates mechanism for the most part.

2. the `Fingerprint` type should be replaced by one from a shared proxmox-*
crate. Dominik's series for unifying tls callbacks adds such a type to
proxmox-http [2]. i'll adapt this series depending on how things are applied.

3. backporting of the pve patches to the bookworm branch probably makes sense
to improve compatibility. i'll send such patches once this series is. this may
have been more prescient  when this series was first submitted. since pve 8 is
eol by now (or tomorrow as of sending this), this may no longer apply.

4. somewhat orthogonal to this series: the mechanism outlined in the notes of
patch 16 would probably improve adding tasks to pdm.

Changelog
---------

* rfc: https://lore.proxmox.com/all/20260611120327.257523-1-s.sterz@proxmox.com/

    changes since the rfc:

    + dropped a patch adjust the tls verify callback in proxmox-client. it
      should instead be replaced by Dominik's implementation of a unified
      callback once that's applied
    + dropped a similar patch for proxmox-datacenter-client for the same
      reason.
    + fixed an issues where a certificate chain could be validated incorrectly
    + rebased on current master for all repos

[1]: https://lore.proxmox.com/all/20260714145027.53038-1-e.huhsovitz@proxmox.com/
[2]: https://lore.proxmox.com/all/20260730133158.418015-1-s.sterz@proxmox.com/
[3]: https://lore.proxmox.com/all/20260701103120.1593265-6-d.csapak@proxmox.com/


pve-cluster:

Shannon Sterz (1):
  setup: allow caller to provide the certificate filename

 src/PVE/Cluster/Setup.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)


pve-manager:

Shannon Sterz (2):
  bin/api: add a new staged certificate when renewing self-signed cert
  api: certificates: if node parameter is 'localhost' return local certs

 PVE/API2/Certificates.pm | 10 +++++++--
 PVE/CertHelpers.pm       |  6 ++++++
 bin/pveupdate            | 44 ++++++++++++++++++++++++++++++++--------
 3 files changed, 49 insertions(+), 11 deletions(-)


proxmox:

Shannon Sterz (1):
  pve-api-types: expose certificates info endpoint

 pve-api-types/Cargo.toml            |  1 +
 pve-api-types/generate.pl           |  3 +++
 pve-api-types/src/generated/code.rs | 15 ++++++++++++++-
 pve-api-types/src/types/mod.rs      |  1 +
 4 files changed, 19 insertions(+), 1 deletion(-)


proxmox-datacenter-manager:

Shannon Sterz (11):
  client: allow users to update a changed fingerprint interactively
  cli/api-types: move Fingerprint to common api type crate
  server: connection: report mismatching fingerprint as untrusted on
    probe
  ui: wizzard: add context if a provided fingerprint did not match
    remote
  ui: wizzard: nodes page: always update fingerprints on user
    confirmation
  pdm-api-types: implement ApiType for Fingerprint
  pdm-api-types: add staged_fingerprints field to NodeUrl
  server: remotes: lock remotes config when updating it
  server: connection: rotate in staged fingerprints when encountering
    them
  server: api: tasks: move `spawn_aborted_on_shutdown()` to super module
  server: bin: api: tasks: add task to discover new staged certificates

 cli/client/src/env/fingerprint_cache.rs       |  90 +--------
 cli/client/src/env/mod.rs                     |   2 +-
 lib/pdm-api-types/Cargo.toml                  |   1 +
 lib/pdm-api-types/src/fingerprint.rs          |  84 +++++++++
 lib/pdm-api-types/src/lib.rs                  |   3 +
 lib/pdm-api-types/src/remotes.rs              |  15 +-
 server/src/api/pbs/mod.rs                     |   2 +
 server/src/api/pve/mod.rs                     |   3 +
 server/src/api/remotes/mod.rs                 |  28 ++-
 server/src/bin/proxmox-datacenter-api/main.rs |   1 +
 .../tasks/ceph_detection.rs                   |  18 +-
 .../bin/proxmox-datacenter-api/tasks/mod.rs   |  17 ++
 .../tasks/remote_staged_fingerprints.rs       | 149 +++++++++++++++
 server/src/connection.rs                      | 173 +++++++++++++++---
 ui/src/remotes/config.rs                      |   1 +
 ui/src/remotes/node_url_list.rs               |   1 +
 ui/src/remotes/wizard_page_connect.rs         |  26 ++-
 ui/src/remotes/wizard_page_info.rs            |   1 +
 ui/src/remotes/wizard_page_nodes.rs           |  40 +++-
 19 files changed, 523 insertions(+), 132 deletions(-)
 create mode 100644 lib/pdm-api-types/src/fingerprint.rs
 create mode 100644 server/src/bin/proxmox-datacenter-api/tasks/remote_staged_fingerprints.rs


Summary over all repositories:
  27 files changed, 593 insertions(+), 146 deletions(-)

-- 
Generated by murpp 0.12.0