[PATCH proxmox{,-backup} 00/28] append-only sync jobs and snapshot retention timespan

[Christian Ebner <c.ebner@proxmox.com>]

Currently sync jobs cannot be configured to be fully append-only
since namespace creation requires datastore modify privileges to do
so. Further, permissions would also allow to restore or modify owned
content.

This patch series therefore extends the current permissions
and roles to allow for append only sync jobs, by only allowing
the minimally required permissions and roles.

In particular, for push the sync jobs local user on the source
requires RemoteSyncAppendOperator as well as DatastoreReader on the
source datastore, with DatastoreAppend and DatastoreAudit (latter for
listing privs of pre-existing contents without restore) permissions
for the user on the remote instance used for connection.

For pull, the user on the target must be able to append to the
datastore via DatastoreAppend and able to read from the remote source
by the respective RemoteSyncOperator permissions on the remote and
by either DatastoreBackup or DatastoreReader permissions to access the
contents.

Further, sync jobs are extended to allow setting a retention timespan
for which synced snapshots cannot be pruned, neither by the sync job,
nor by prune jobs. Only root@pam is allowed to change the retention
period. After the retention period, snapshots behave like regular
snapshots again and can be pruned.

To protect from sync jobs setting unintended retention timespans,
it is now also possible to configure a maximum allowed reteniton time
on the datastore.

Sending this as RFC for some initial feedback on the overall
implementation approach, plan to further have a look into object
locking and retention on s3 object stores [0] and changes required
for immutable storage [1].

[0] https://bugzilla.proxmox.com/show_bug.cgi?id=6780
[1] https://bugzilla.proxmox.com/show_bug.cgi?id=4293


proxmox:

Christian Ebner (5):
  pbs-api-types: add append only permission and role
  pbs-api-types: add remote datastore append privs and role
  pbs-api-types: extend snapshot list items by retention timestamp
  pbs-api-types: extend sync job config by retention-timespan parameter
  pbs-api-types: add maximum retention timespan property to datastore

 pbs-api-types/src/acl.rs       | 24 +++++++++++++
 pbs-api-types/src/datastore.rs | 16 +++++++--
 pbs-api-types/src/jobs.rs      | 64 +++++++++++++++++++++++++++++++++-
 3 files changed, 101 insertions(+), 3 deletions(-)


proxmox-backup:

Christian Ebner (23):
  api: config: extend sync job config by new retention-timespan
  api: admin: improve code style for status endpoint
  client: avoid error in status if user lacks permissions
  server: allow iterating contents for Datastore.Append permissions
  api: backup: fix possible information leak in multi-tenant datastores
  api: backup: allow backup for user/token with append permission
  api: allow namespace creation on append permissions
  api: sync: allow pull to target for user/token with append permission
  sync: pull: allow pulling for user/token with append permission
  sync: push: allow push and ns creation on Remote.DatastoreAppend
  datastore: conditionally treat missing manifest as error or bening
  api: backup: provide retain-until timestamp for extended prune
    protection
  tools: include retain-until timestamp in snapshot list items
  client: backup writer: allow to send retain-until timestamp on backup
  sync: push: allow to set retention timestamp for synced snapshots
  sync: pull: allow to set retention timestamp for synced snapshots
  sync: pull: protect retained snapshot from being overwritten
  api: config: allow to set or delete reteniton timespan for sync jobs
  ui: add retention timespan form and use it for sync job edit window
  datastore/config: parse and enforce maximum retention timespan
  ui: allow datastore wide max retention timespan configuration
  api: admin: allow to update snapshot retention for root user
  ui: show retention in datastore contents

 examples/upload-speed.rs               |  1 +
 pbs-client/src/backup_writer.rs        |  5 ++
 pbs-datastore/src/backup_info.rs       | 63 ++++++++++++++++---
 pbs-datastore/src/datastore.rs         | 10 ++-
 pbs-datastore/src/manifest.rs          | 27 ++++++++
 proxmox-backup-client/src/benchmark.rs |  1 +
 proxmox-backup-client/src/main.rs      | 19 +++---
 src/api2/admin/datastore.rs            | 85 ++++++++++++++++++++------
 src/api2/admin/namespace.rs            |  8 +--
 src/api2/backup/environment.rs         |  4 ++
 src/api2/backup/mod.rs                 | 35 +++++++++--
 src/api2/config/datastore.rs           |  9 +++
 src/api2/config/sync.rs                | 22 +++++--
 src/api2/pull.rs                       | 19 ++++--
 src/api2/push.rs                       |  8 ++-
 src/api2/tape/restore.rs               |  6 +-
 src/backup/hierarchy.rs                | 23 +++++--
 src/server/pull.rs                     | 69 +++++++++++++++++----
 src/server/push.rs                     | 30 +++++++--
 src/server/sync.rs                     |  1 +
 src/tools/mod.rs                       | 23 +++----
 www/Makefile                           |  2 +
 www/datastore/Content.js               | 67 ++++++++++++++++++++
 www/datastore/OptionView.js            |  8 +++
 www/form/RetentionTimespan.js          | 21 +++++++
 www/window/MaxRetentionTimespanEdit.js | 27 ++++++++
 www/window/SyncJobEdit.js              |  9 +++
 27 files changed, 510 insertions(+), 92 deletions(-)
 create mode 100644 www/form/RetentionTimespan.js
 create mode 100644 www/window/MaxRetentionTimespanEdit.js


Summary over all repositories:
  30 files changed, 611 insertions(+), 95 deletions(-)

-- 
Generated by murpp 0.11.0