[PATCH datacenter-manager v3 0/4] add generic, per-remote (and global) cache for remote API responses

[Lukas Wagner <l.wagner@proxmox.com>]

The main intention is to avoid a sprawl of different caching approaches by
establishing a simple, easy to use cache implementation that can be used to
persistently cache API responses from remotes (and derived aggregations).

The `namespaced_cache` module is pretty generic and can be moved to proxmox.git
(maybe in proxmox-shared-cache) once it has sufficiently stabilized.

Changes since the RFC:
  - change storage location to /run/proxmox-datacenter-manager/api-cache
  - change name of pdm_cache to api_cache
  - use freestanding functions in api_cache
  - add async interface
  - minor code style improvements
  - improve test coverage and documentation
  - add basic sanity checks for namespaces and keys (e.g. prohibiting ../)
  - add cleanup code for the old remote-updates cachefile

Thanks to Dominik C. and Wolfgang for their reviews of the RFC, highly appreciated!

Changes since v1:
  - The only changes happened in the last two patches:
     - Improved behavior in case the cache entry could not be read,
       mostly to ensure a self-healing behavior
     - Slightly restructured code to avoid some clone calls
     - When requesting the list of updates for a single remote, update the
       global summary cache asynchronously

Changes since v2 (thanks @ Thomas):
  - Use 750 directory permissions consistently for the API cache
  - Verify keys and namespaces using SAFE_ID_REGEX
  - Add set_if_newer{,_with_timestamp} method
    These only set the cache entry if the existing entry is actually
    older than the timestamp we provide (or that is implicitly provided, since
    set_if_newer just uses the current time).
    If the existing entry is newer, it is returned as Ok(Some(existing_entry)).
    This makes it quite convenient to ensure that a caller *always*
    gets the latest possible data from the cache, even if there is
    a race when updating the cache.
  - Fixed some typos
  - Fixed a race condition when updating the global update cache


proxmox-datacenter-manager:

Lukas Wagner (4):
  add persistent, generic, namespaced key-value cache implementation
  add api_cache as a specialized wrapper around the namespaced cache
  api: resources: subscriptions: switch over to api_cache
  remote-updates: switch over to new api_cache

 Cargo.toml                                    |   1 +
 debian/control                                |   1 +
 server/Cargo.toml                             |   4 +
 server/src/api/pve/mod.rs                     |   4 +-
 server/src/api/remotes/updates.rs             |   6 +-
 server/src/api/resources.rs                   |  88 +-
 server/src/api_cache.rs                       | 126 +++
 .../bin/proxmox-datacenter-privileged-api.rs  |   9 +-
 server/src/lib.rs                             |   2 +
 server/src/namespaced_cache.rs                | 923 ++++++++++++++++++
 server/src/remote_updates.rs                  | 123 ++-
 11 files changed, 1187 insertions(+), 100 deletions(-)
 create mode 100644 server/src/api_cache.rs
 create mode 100644 server/src/namespaced_cache.rs


Summary over all repositories:
  11 files changed, 1187 insertions(+), 100 deletions(-)

-- 
Generated by murpp 0.12.0