From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 7B0241FF137 for ; Tue, 14 Apr 2026 14:59:39 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id C182018BD6; Tue, 14 Apr 2026 15:00:10 +0200 (CEST) From: Christian Ebner To: pbs-devel@lists.proxmox.com Subject: [PATCH proxmox{,-backup} v3 00/30] fix #7251: implement server side encryption support for push sync jobs Date: Tue, 14 Apr 2026 14:58:53 +0200 Message-ID: <20260414125923.892345-1-c.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1776171496791 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.070 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [acl.rs,lib.rs,proxmox.com,jobs.rs,mod.rs,push.rs,manifest.rs,sync.rs,pull.rs] Message-ID-Hash: ZTXIPA4YWGDGMF3QHY56OLHNHZO6DQC7 X-Message-ID-Hash: ZTXIPA4YWGDGMF3QHY56OLHNHZO6DQC7 X-MailFrom: c.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: This patch series implements support for encrypting backup snapshots when pushing from a source PBS instance to an untrusted remote target PBS instance. Further, it adds support to decrypt snapshots being encrypted on the remote source PBS when pulling the contents to the local target PBS instance. This allows to perform full server side encryption/decryption when syncing with a less trusted remote PBS. In order to encrypt/decrypt snapshots, a new encryption key entity is introduced, to be created as global instance on the PBS, placed and managed by it's own dedicated config. Keys with secret are stored in dedicated files so they only need to be loaded when accessing the key, not for listing of configuration. Sync encryption keys can be archived, rendering them no longer usable to encrypt new contents, but still allowing to decrypt. In order to remove a sync encryption key, it must be archived first and no longer associated to any sync job config, a constrained added as safety net to avoid accidental key removal. The same centralized key management is also used for tape encryption keys, so they are on-par ui wise, the configs remain however separated for the time being. The sync jobs in push direction are extended to receive an additional active encryption key parameter, which will be used to encrypt unencrypted snapshot when pushing to the remote target. A list of associated keys is kept, adding the previous encryption key of the push sync job if the key is rotated. For pull sync jobs, the active encryption key parameter is not considered, rather all associated keys will be loaded and used to decrypt snapshots with matching fingerprint as found in the source manifest. In order to encrypt/decrypt the contents, chunks, index files, blobs and manifest are additionally processed, rewritten when required. Changes since version 2 (thanks a lot to @Thomas for review): - Add dedicated lock file for per-key file locks when creating/deleting sync keys. - Add initial documentation for server side encryption/decription during sync jobs. - Adapt key archive endpoint to be able to toggle, kept as dedicated patch as unsure about impl details. - Early detect unusable keys provided on key creation as upload via api. - List all associated sync jobs when checking with encryption_key_in_use(). - Fix check for key access when setting active encryption key. It must fail for archived keys. - Add flag to check for not allowing to set archived key as active encryption key. - Drop associated keys also on active encryption key update, readd rotated one afterwards if required. - Refactor check for un-/partially-/fully-encrypted backup snapshots. - Include snapshot name in log message for skipped snapshots. - Add missing return on error when requesting key archivation for tape. - Handle errors for api calls to load tape and sync keys in ui by wrapping into try-catch-block. - Also drop verify state on pull, do not rely on inherent check to better protect against bugs and corruptions. - Awitch field label for associated keys based on sync direction. - Add comment field explaining active encryption key and associated keys and their relation on key rotation. - Also store key id together with key config when loading associated keys, so it can be logged later when key fingerprint matched. - Squash new manifest registration into patch 26, keeping logic together - Fix boguous check, must use change-detection-fingerprint, not key-fingerprint to detect changes on already existing manifest. - Convert unprotected manifest part to json value to drop key-fingerprint. - Log id of key used for decryption, not just fingerprint - Switch all remaining `log` macros for sync to use `tracing`. - Fix typos in commit message for async DataBlob reader patch. - Double column width for `hint` field. - Fix icons for type based menu buttons and type column - Drop dead code `crypt-key-fp`. - Fix error messages by s/seems/seem/ and wrap in gettext() - Document config lock requirements for delete_key(). - Drop outdated comment on key file lock drop, it's a dedicated file now. Changes since version 1 (thanks a lot to @all reviewers/testers!): - Implement encryption key archiving and key rotation logic, allowing to specify active encryption key for push syncs, and a list of previously used ones. For pull multiple decryption keys can now be configured. - Rework the UI to add support for key archiving, manage key association in sync jobs and to also manage tape encryption keys in the same centralized grid. - Check for key still being in-use by sync job before removing it - Fully encrypted snapshots are now pushed as-is if an encryption key is configured. - Fixed inefficient resync of pre-existing target snapshot on pull, detect file changes in manifest via fingerprinting. - Avoid overwriting pre-existing decrypted local snapshot by encrypted snapshot when no (or mismatching) decryption key is passed for pull job. - Rename EncryptionKey to CyrptKey, as the key is also used for decryption. - Remove key from config before removing keyfile - Add locking mechansism to avoid races in key config writing - Fix gathering of known chunks from previous snapshot in push for dynamic index files - Detect config changes by checking for digest mismatch - Guard key loading by PRIV_SYS_MODIFY - Use tracing::info! instead of log::info! - Fix clearing of encryption/decryption key via sync job config window - Fix creating new sync job without crypt key configured - Check key exists and can be accessed when set in sync job - Fix min key id length for key edit window - Fixed drag-and-drop for key file upload - Fix outdated comments, typos, ecc. Link to the bugtracker issue: https://bugzilla.proxmox.com/show_bug.cgi?id=7251 proxmox: Christian Ebner (2): pbs-api-types: define en-/decryption key type and schema pbs-api-types: sync job: add optional cryptographic keys to config pbs-api-types/src/jobs.rs | 21 ++++++++++++++-- pbs-api-types/src/key_derivation.rs | 38 ++++++++++++++++++++++++++--- pbs-api-types/src/lib.rs | 2 +- 3 files changed, 55 insertions(+), 6 deletions(-) proxmox-backup: Christian Ebner (28): sync: push: use tracing macros instead of log datastore: blob: implement async reader for data blobs datastore: manifest: add helper for change detection fingerprint pbs-key-config: introduce store_with() for KeyConfig pbs-config: implement encryption key config handling pbs-config: acls: add 'encryption-keys' as valid 'system' subpath ui: expose 'encryption-keys' as acl subpath for 'system' sync: add helper to check encryption key acls and load key api: config: add endpoints for encryption key manipulation api: config: check sync owner has access to en-/decryption keys api: config: allow encryption key manipulation for sync job sync: push: rewrite manifest instead of pushing pre-existing one api: push sync: expose optional encryption key for push sync sync: push: optionally encrypt data blob on upload sync: push: optionally encrypt client log on upload if key is given sync: push: add helper for loading known chunks from previous snapshot fix #7251: api: push: encrypt snapshots using configured encryption key ui: define and expose encryption key management menu item and windows ui: expose assigning encryption key to sync jobs sync: pull: load encryption key if given in job config sync: expand source chunk reader trait by crypt config sync: pull: introduce and use decrypt index writer if crypt config sync: pull: extend encountered chunk by optional decrypted digest sync: pull: decrypt blob files on pull if encryption key is configured sync: pull: decrypt chunks and rewrite index file for matching key sync: pull: decrypt snapshots with matching encryption key fingerprint api: encryption keys: allow to toggle the archived state for keys docs: add section describing server side encryption for sync jobs docs/managing-remotes.rst | 49 +++ pbs-config/Cargo.toml | 2 + pbs-config/src/acl.rs | 4 +- pbs-config/src/encryption_keys.rs | 217 ++++++++++++++ pbs-config/src/lib.rs | 1 + pbs-datastore/src/data_blob.rs | 18 +- pbs-datastore/src/manifest.rs | 20 ++ pbs-key-config/src/lib.rs | 36 ++- src/api2/config/encryption_keys.rs | 219 ++++++++++++++ src/api2/config/mod.rs | 2 + src/api2/config/sync.rs | 94 +++++- src/api2/pull.rs | 15 +- src/api2/push.rs | 8 +- src/server/pull.rs | 459 ++++++++++++++++++++++++----- src/server/push.rs | 311 ++++++++++++++----- src/server/sync.rs | 58 +++- www/Makefile | 3 + www/NavigationTree.js | 6 + www/Utils.js | 1 + www/config/EncryptionKeysView.js | 346 ++++++++++++++++++++++ www/form/EncryptionKeySelector.js | 96 ++++++ www/form/PermissionPathSelector.js | 1 + www/window/EncryptionKeysEdit.js | 382 ++++++++++++++++++++++++ www/window/SyncJobEdit.js | 62 ++++ 24 files changed, 2248 insertions(+), 162 deletions(-) create mode 100644 pbs-config/src/encryption_keys.rs create mode 100644 src/api2/config/encryption_keys.rs create mode 100644 www/config/EncryptionKeysView.js create mode 100644 www/form/EncryptionKeySelector.js create mode 100644 www/window/EncryptionKeysEdit.js Summary over all repositories: 27 files changed, 2303 insertions(+), 168 deletions(-) -- Generated by murpp 0.11.0