From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id BF7B91FF0E7 for ; Thu, 13 Aug 2026 19:11:00 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id E794121AF6; Thu, 13 Aug 2026 19:10:36 +0200 (CEST) From: Christian Ebner To: pbs-devel@lists.proxmox.com Subject: [PATCH proxmox{,-backup} 00/28] append-only sync jobs and snapshot retention timespan Date: Thu, 13 Aug 2026 19:09:34 +0200 Message-ID: <20260813171002.809441-1-c.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1786641012565 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.211 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust RDNS_NONE 1.274 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: G37O2VKNEVRDHNULTJNG6S35L2SV7SMJ X-Message-ID-Hash: G37O2VKNEVRDHNULTJNG6S35L2SV7SMJ X-MailFrom: c.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Currently sync jobs cannot be configured to be fully append-only since namespace creation requires datastore modify privileges to do so. Further, permissions would also allow to restore or modify owned content. This patch series therefore extends the current permissions and roles to allow for append only sync jobs, by only allowing the minimally required permissions and roles. In particular, for push the sync jobs local user on the source requires RemoteSyncAppendOperator as well as DatastoreReader on the source datastore, with DatastoreAppend and DatastoreAudit (latter for listing privs of pre-existing contents without restore) permissions for the user on the remote instance used for connection. For pull, the user on the target must be able to append to the datastore via DatastoreAppend and able to read from the remote source by the respective RemoteSyncOperator permissions on the remote and by either DatastoreBackup or DatastoreReader permissions to access the contents. Further, sync jobs are extended to allow setting a retention timespan for which synced snapshots cannot be pruned, neither by the sync job, nor by prune jobs. Only root@pam is allowed to change the retention period. After the retention period, snapshots behave like regular snapshots again and can be pruned. To protect from sync jobs setting unintended retention timespans, it is now also possible to configure a maximum allowed reteniton time on the datastore. Sending this as RFC for some initial feedback on the overall implementation approach, plan to further have a look into object locking and retention on s3 object stores [0] and changes required for immutable storage [1]. [0] https://bugzilla.proxmox.com/show_bug.cgi?id=6780 [1] https://bugzilla.proxmox.com/show_bug.cgi?id=4293 proxmox: Christian Ebner (5): pbs-api-types: add append only permission and role pbs-api-types: add remote datastore append privs and role pbs-api-types: extend snapshot list items by retention timestamp pbs-api-types: extend sync job config by retention-timespan parameter pbs-api-types: add maximum retention timespan property to datastore pbs-api-types/src/acl.rs | 24 +++++++++++++ pbs-api-types/src/datastore.rs | 16 +++++++-- pbs-api-types/src/jobs.rs | 64 +++++++++++++++++++++++++++++++++- 3 files changed, 101 insertions(+), 3 deletions(-) proxmox-backup: Christian Ebner (23): api: config: extend sync job config by new retention-timespan api: admin: improve code style for status endpoint client: avoid error in status if user lacks permissions server: allow iterating contents for Datastore.Append permissions api: backup: fix possible information leak in multi-tenant datastores api: backup: allow backup for user/token with append permission api: allow namespace creation on append permissions api: sync: allow pull to target for user/token with append permission sync: pull: allow pulling for user/token with append permission sync: push: allow push and ns creation on Remote.DatastoreAppend datastore: conditionally treat missing manifest as error or bening api: backup: provide retain-until timestamp for extended prune protection tools: include retain-until timestamp in snapshot list items client: backup writer: allow to send retain-until timestamp on backup sync: push: allow to set retention timestamp for synced snapshots sync: pull: allow to set retention timestamp for synced snapshots sync: pull: protect retained snapshot from being overwritten api: config: allow to set or delete reteniton timespan for sync jobs ui: add retention timespan form and use it for sync job edit window datastore/config: parse and enforce maximum retention timespan ui: allow datastore wide max retention timespan configuration api: admin: allow to update snapshot retention for root user ui: show retention in datastore contents examples/upload-speed.rs | 1 + pbs-client/src/backup_writer.rs | 5 ++ pbs-datastore/src/backup_info.rs | 63 ++++++++++++++++--- pbs-datastore/src/datastore.rs | 10 ++- pbs-datastore/src/manifest.rs | 27 ++++++++ proxmox-backup-client/src/benchmark.rs | 1 + proxmox-backup-client/src/main.rs | 19 +++--- src/api2/admin/datastore.rs | 85 ++++++++++++++++++++------ src/api2/admin/namespace.rs | 8 +-- src/api2/backup/environment.rs | 4 ++ src/api2/backup/mod.rs | 35 +++++++++-- src/api2/config/datastore.rs | 9 +++ src/api2/config/sync.rs | 22 +++++-- src/api2/pull.rs | 19 ++++-- src/api2/push.rs | 8 ++- src/api2/tape/restore.rs | 6 +- src/backup/hierarchy.rs | 23 +++++-- src/server/pull.rs | 69 +++++++++++++++++---- src/server/push.rs | 30 +++++++-- src/server/sync.rs | 1 + src/tools/mod.rs | 23 +++---- www/Makefile | 2 + www/datastore/Content.js | 67 ++++++++++++++++++++ www/datastore/OptionView.js | 8 +++ www/form/RetentionTimespan.js | 21 +++++++ www/window/MaxRetentionTimespanEdit.js | 27 ++++++++ www/window/SyncJobEdit.js | 9 +++ 27 files changed, 510 insertions(+), 92 deletions(-) create mode 100644 www/form/RetentionTimespan.js create mode 100644 www/window/MaxRetentionTimespanEdit.js Summary over all repositories: 30 files changed, 611 insertions(+), 95 deletions(-) -- Generated by murpp 0.11.0