[PATCH datacenter-manager v2 0/8] subscription: add central key pool registry with reissue support

[Thomas Lamprecht <t.lamprecht@proxmox.com>]

Add a Subscription Registry to PDM: a central pool of PVE and PBS
subscription keys that an operator can assign to remote nodes from one
place, with an explicit Apply/Clear lifecycle for staged changes plus a
Reissue Key action for freeing a key bound to a node so it can be
reassigned elsewhere.

Motivation: managing subscriptions across many remotes today means
doing this for each node individually. PDM already has the remote
inventory; with a key pool plus per-remote query data we can show "which
nodes need a subscription" and "which keys are unused" together, and let
an admin batch-assign and tear down from one place.
In the near/mid-term we can also make polling keys from customers more
integrated, but that needs a bit adaption in our shop infa and does not
block the base work here in anyway. Actually, the implementation here
was split out from a more complete work, so most parts of it are already
prepared to adopt this relatively easily.

Design points worth flagging for review:

* Storage layout. subscriptions.cfg holds key entries via the typed
  section-config layer, with `product-type` as the section type so PVE
  and PBS sections live side-by-side.
  The subscriptions.shadow file is reserved for a future shop-bundle
  import flow (signed info blobs) and stays empty for manually-added
  keys. I can drop that part for now too, but figured it might be nice
  and potentially relevant for review to see the direction this probably
  goes now already.

* Endpoints take PRIV_SYS_AUDIT/MODIFY at the macro
  level for the pool itself, with per-remote PRIV_RESOURCE_* enforced
  inside the handlers when a specific remote is touched.
  A dedicated subscription privilege seemed not like a necessity and
  also not fit that well into our general priv approach in PDM.

* The pending lifecycle goes like: Pool entries with a (remote, node)
* binding whose
  live state does not match are "pending push"; entries with the new
  pending-reissue flag are "pending removal". Apply Pending walks both
  queues; Clear Pending drops the queue without touching any remote
  (binding-clear for push, flag-only for reissue so the operator can
  retry without re-importing the key).
  The per-remote subscription cache is invalidated after each successful
  apply step so the next panel load reflects the change rather than a
  5-minute-stale snapshot, which is highly confusing UI/UX wise. This
  might warrant a closer look though, might be currently done in a
  rather heavier handed fashion as potentially needed (had no time to
  recheck).

* Locking is best-effort, but here that should be fine in practice given
  that another entity can always alter the state on a remote node in
  parallel anyway.

The lib/pdm-api-types/tests/test_import.rs test should provide basic
coverage for section-config roundtrip for both subscription.cfg and the
shadow file (which is why I'd be fine with keeping it, but not _that_
hard feelings), schema acceptance and rejection (for now only accept
PVE/PBS; everything else rejected), ProductType classification, the
SubscriptionLevel display/from-str backward-compat (single-letter and
full-name forms both parse), and pick_best_pve_socket_key edge cases.

Open follow-ups deliberately out of scope here:
* Auto-import existing remote-side keys into the pool on first
  observation (the reissue path already adopts; an explicit import for
  legacy onboarding would be cleaner).
* Make reissue a full reissue, if it goes in like this it should be
  rather called "Clear Key", but that can be handled on applying too, if
  really nothing else comes up (which I doubt)
* A shop-bundle import path (the shadow file plumbing is already in),
  either manual copy+paste or through an api token.
* Some polishing code and ui/ux wise (e.g., a reload button), but wante
  to finally get this out now.
* ...

changes v1 -> v2:
 * cleanly adopt (naughty) in-patch edit that broke docs changes of
   subsequent patch.
 * add correct subjectprefix

Thomas Lamprecht (8):
  api: subscription cache: ensure max_age=0 forces a fresh fetch
  api types: subscription level: render full names
  subscription: add key pool data model and config layer
  subscription: add key pool and node status API endpoints
  ui: add subscription registry with key pool and node status
  cli: add subscription key pool management subcommands
  docs: add subscription registry chapter
  subscription: add Reissue Key action with pending-reissue queue

 cli/client/src/subscriptions.rs               |  226 +++-
 docs/index.rst                                |    1 +
 docs/subscription-registry.rst                |   64 +
 lib/pdm-api-types/Cargo.toml                  |    4 +
 lib/pdm-api-types/src/subscription.rs         |  422 +++++-
 lib/pdm-api-types/tests/test_import.rs        |  310 +++++
 lib/pdm-client/src/lib.rs                     |  157 ++-
 lib/pdm-config/src/lib.rs                     |    1 +
 lib/pdm-config/src/subscriptions.rs           |  102 ++
 server/src/api/mod.rs                         |    2 +
 server/src/api/resources.rs                   |   13 +-
 server/src/api/subscriptions/mod.rs           | 1199 +++++++++++++++++
 server/src/context.rs                         |    7 +
 ui/src/configuration/mod.rs                   |    2 +
 ui/src/configuration/subscription_keys.rs     |  458 +++++++
 ui/src/configuration/subscription_registry.rs |  791 +++++++++++
 ui/src/dashboard/subscriptions_list.rs        |   18 +-
 ui/src/main_menu.rs                           |   10 +
 18 files changed, 3751 insertions(+), 36 deletions(-)
 create mode 100644 docs/subscription-registry.rst
 create mode 100644 lib/pdm-api-types/tests/test_import.rs
 create mode 100644 lib/pdm-config/src/subscriptions.rs
 create mode 100644 server/src/api/subscriptions/mod.rs
 create mode 100644 ui/src/configuration/subscription_keys.rs
 create mode 100644 ui/src/configuration/subscription_registry.rs

-- 
2.47.3