From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 6DBAD1FF140 for ; Fri, 10 Apr 2026 18:54:30 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 9BB8F22B83; Fri, 10 Apr 2026 18:55:13 +0200 (CEST) From: Christian Ebner To: pbs-devel@lists.proxmox.com Subject: [PATCH proxmox{,-backup} v2 00/27] fix #7251: implement server side encryption support for push sync jobs Date: Fri, 10 Apr 2026 18:54:27 +0200 Message-ID: <20260410165454.1578501-1-c.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1775840035034 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.068 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: OSUBANZYS35C5RPEWWAYIQKMSYEK7QTO X-Message-ID-Hash: OSUBANZYS35C5RPEWWAYIQKMSYEK7QTO X-MailFrom: c.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: This patch series implements support for encrypting backup snapshots when pushing from a source PBS instance to an untrusted remote target PBS instance. Further, it adds support to decrypt snapshots being encrypted on the remote source PBS when pulling the contents to the local target PBS instance. This allows to perform full server side encryption/decryption when syncing with a less trusted remote PBS. In order to encrypt/decrypt snapshots, a new encryption key entity is introduced, to be created as global instance on the PBS, placed and managed by it's own dedicated config. Keys with secret are stored in dedicated files so they only need to be loaded when accessing the key, not for listing of configuration. Sync encryption keys can be archived, rendering them no longer usable to encrypt new contents, but still allowing to decrypt. In order to remove a sync encryption key, it must be archived first and no longer associated to any sync job config, a constrained added as safety net to avoid accidental key removal. The same centralized key management is also used for tape encryption keys, so they are on-par ui wise, the configs remain however separated for the time being. The sync jobs in push direction are extended to receive an additional active encryption key parameter, which will be used to encrypt unencrypted snapshot when pushing to the remote target. A list of associated keys is kept, adding the previous encryption key of the push sync job if the key is rotated. For pull sync jobs, the active encryption key parameter is not considered, rather all associated keys will be loaded and used to decrypt snapshots with matching fingerprint as found in the source manifest. In order to encrypt/decrypt the contents, chunks, index files, blobs and manifest are additionally processed, rewritten when required. Changes since version 1 (thanks a lot to @all reviewers/testers!): - Implement encryption key archiving and key rotation logic, allowing to specify active encryption key for push syncs, and a list of previously used ones. For pull multiple decryption keys can now be configured. - Rework the UI to add support for key archiving, manage key association in sync jobs and to also manage tape encryption keys in the same centralized grid. - Check for key still being in-use by sync job before removing it - Fully encrypted snapshots are now pushed as-is if an encryption key is configured. - Fixed inefficient resync of pre-existing target snapshot on pull, detect file changes in manifest via fingerprinting. - Avoid overwriting pre-existing decrypted local snapshot by encrypted snapshot when no (or mismatching) decryption key is passed for pull job. - Rename EncryptionKey to CyrptKey, as the key is also used for decryption. - Remove key from config before removing keyfile - Add locking mechansism to avoid races in key config writing - Fix gathering of known chunks from previous snapshot in push for dynamic index files - Detect config changes by checking for digest mismatch - Guard key loading by PRIV_SYS_MODIFY - Use tracing::info! instead of log::info! - Fix clearing of encryption/decryption key via sync job config window - Fix creating new sync job without crypt key configured - Check key exists and can be accessed when set in sync job - Fix min key id length for key edit window - Fixed drag-and-drop for key file upload - Fix outdated comments, typos, ecc. Link to the bugtracker issue: https://bugzilla.proxmox.com/show_bug.cgi?id=7251 proxmox: Christian Ebner (2): pbs-api-types: define en-/decryption key type and schema pbs-api-types: sync job: add optional cryptographic keys to config pbs-api-types/src/jobs.rs | 21 ++++++++++++++-- pbs-api-types/src/key_derivation.rs | 38 ++++++++++++++++++++++++++--- pbs-api-types/src/lib.rs | 2 +- 3 files changed, 55 insertions(+), 6 deletions(-) proxmox-backup: Christian Ebner (25): datastore: blob: implement async reader for data blobs datastore: manifest: add helper for change detection fingerprint pbs-key-config: introduce store_with() for KeyConfig pbs-config: implement encryption key config handling pbs-config: acls: add 'encryption-keys' as valid 'system' subpath ui: expose 'encryption-keys' as acl subpath for 'system' sync: add helper to check encryption key acls and load key api: config: add endpoints for encryption key manipulation api: config: check sync owner has access to en-/decryption keys api: config: allow encryption key manipulation for sync job sync: push: rewrite manifest instead of pushing pre-existing one api: push sync: expose optional encryption key for push sync sync: push: optionally encrypt data blob on upload sync: push: optionally encrypt client log on upload if key is given sync: push: add helper for loading known chunks from previous snapshot fix #7251: api: push: encrypt snapshots using configured encryption key ui: define and expose encryption key management menu item and windows ui: expose assigning encryption key to sync jobs sync: pull: load encryption key if given in job config sync: expand source chunk reader trait by crypt config sync: pull: introduce and use decrypt index writer if crypt config sync: pull: extend encountered chunk by optional decrypted digest sync: pull: decrypt blob files on pull if encryption key is configured sync: pull: decrypt chunks and rewrite index file for matching key sync: pull: decrypt snapshots with matching encryption key fingerprint pbs-config/Cargo.toml | 2 + pbs-config/src/acl.rs | 4 +- pbs-config/src/encryption_keys.rs | 210 +++++++++++++ pbs-config/src/lib.rs | 1 + pbs-datastore/src/data_blob.rs | 18 +- pbs-datastore/src/manifest.rs | 20 ++ pbs-key-config/src/lib.rs | 36 ++- src/api2/config/encryption_keys.rs | 203 +++++++++++++ src/api2/config/mod.rs | 2 + src/api2/config/sync.rs | 78 ++++- src/api2/pull.rs | 15 +- src/api2/push.rs | 8 +- src/server/pull.rs | 455 ++++++++++++++++++++++++----- src/server/push.rs | 297 ++++++++++++++----- src/server/sync.rs | 58 +++- www/Makefile | 3 + www/NavigationTree.js | 6 + www/Utils.js | 1 + www/config/EncryptionKeysView.js | 324 ++++++++++++++++++++ www/form/EncryptionKeySelector.js | 96 ++++++ www/form/PermissionPathSelector.js | 1 + www/window/EncryptionKeysEdit.js | 383 ++++++++++++++++++++++++ www/window/SyncJobEdit.js | 30 ++ 23 files changed, 2092 insertions(+), 159 deletions(-) create mode 100644 pbs-config/src/encryption_keys.rs create mode 100644 src/api2/config/encryption_keys.rs create mode 100644 www/config/EncryptionKeysView.js create mode 100644 www/form/EncryptionKeySelector.js create mode 100644 www/window/EncryptionKeysEdit.js Summary over all repositories: 26 files changed, 2147 insertions(+), 165 deletions(-) -- Generated by murpp 0.11.0