public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Christian Ebner <c.ebner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [PATCH proxmox-backup v2 15/27] sync: push: optionally encrypt data blob on upload
Date: Fri, 10 Apr 2026 18:54:42 +0200	[thread overview]
Message-ID: <20260410165454.1578501-16-c.ebner@proxmox.com> (raw)
In-Reply-To: <20260410165454.1578501-1-c.ebner@proxmox.com>

Encrypt the data blob with given encryption key during syncs in push
direction, if given.

Introduces a helper to read and decode the data blob from source into
raw data and re-encrypt, so the new blob is compressed and encrypted,
including the correct header when uploading. The same helper will be
reused for client log uploads in subsequent code changes.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 src/server/push.rs | 39 +++++++++++++++++++++++++++++++++------
 1 file changed, 33 insertions(+), 6 deletions(-)

diff --git a/src/server/push.rs b/src/server/push.rs
index 93434bedc..7ce47e32e 100644
--- a/src/server/push.rs
+++ b/src/server/push.rs
@@ -1,6 +1,7 @@
 //! Sync datastore by pushing contents to remote server
 
 use std::collections::HashSet;
+use std::path::Path;
 use std::sync::{Arc, Mutex};
 
 use anyhow::{bail, format_err, Context, Error};
@@ -26,7 +27,7 @@ use pbs_datastore::dynamic_index::DynamicIndexReader;
 use pbs_datastore::fixed_index::FixedIndexReader;
 use pbs_datastore::index::IndexFile;
 use pbs_datastore::read_chunk::AsyncReadChunk;
-use pbs_datastore::{BackupManifest, DataStore, StoreProgress};
+use pbs_datastore::{BackupManifest, DataBlob, DataStore, StoreProgress};
 use pbs_tools::crypt_config::CryptConfig;
 
 use super::sync::{
@@ -849,6 +850,8 @@ pub(crate) async fn push_snapshot(
         return Ok(stats);
     }
 
+    let mut encrypt_using_key = None;
+
     // Writer instance locks the snapshot on the remote side
     let backup_writer = BackupWriter::start(
         &params.target.client,
@@ -856,7 +859,7 @@ pub(crate) async fn push_snapshot(
             datastore: params.target.repo.store(),
             ns: &target_ns,
             backup: snapshot,
-            crypt_config: None,
+            crypt_config: encrypt_using_key.clone(),
             debug: false,
             benchmark: false,
             no_cache: false,
@@ -901,10 +904,20 @@ pub(crate) async fn push_snapshot(
             let archive_name = BackupArchiveName::from_path(&entry.filename)?;
             match archive_name.archive_type() {
                 ArchiveType::Blob => {
-                    let file = std::fs::File::open(&path)?;
-                    let backup_stats = backup_writer
-                        .upload_blob(file, archive_name.as_ref())
-                        .await?;
+                    let backup_stats = if encrypt_using_key.is_some() {
+                        reencode_encrypted_and_upload_blob(
+                            path,
+                            &archive_name,
+                            &backup_writer,
+                            &upload_options,
+                        )
+                        .await?
+                    } else {
+                        let file = std::fs::File::open(&path)?;
+                        backup_writer
+                            .upload_blob(file, archive_name.as_ref())
+                            .await?
+                    };
                     target_manifest.add_file(
                         &archive_name,
                         backup_stats.size,
@@ -1039,6 +1052,20 @@ pub(crate) async fn push_snapshot(
     Ok(stats)
 }
 
+async fn reencode_encrypted_and_upload_blob<P: AsRef<Path>>(
+    path: P,
+    archive_name: &BackupArchiveName,
+    backup_writer: &BackupWriter,
+    upload_options: &UploadOptions,
+) -> Result<BackupStats, Error> {
+    let mut file = tokio::fs::File::open(&path).await?;
+    let data_blob = DataBlob::load_from_async_reader(&mut file).await?;
+    let raw_data = data_blob.decode(None, None)?;
+    backup_writer
+        .upload_blob_from_data(raw_data, archive_name.as_ref(), upload_options.clone())
+        .await
+}
+
 // Read fixed or dynamic index and push to target by uploading via the backup writer instance
 //
 // For fixed indexes, the size must be provided as given by the index reader.
-- 
2.47.3





  parent reply	other threads:[~2026-04-10 16:55 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-04-10 16:54 [PATCH proxmox{,-backup} v2 00/27] fix #7251: implement server side encryption support for push sync jobs Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox v2 01/27] pbs-api-types: define en-/decryption key type and schema Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox v2 02/27] pbs-api-types: sync job: add optional cryptographic keys to config Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 03/27] datastore: blob: implement async reader for data blobs Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 04/27] datastore: manifest: add helper for change detection fingerprint Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 05/27] pbs-key-config: introduce store_with() for KeyConfig Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 06/27] pbs-config: implement encryption key config handling Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 07/27] pbs-config: acls: add 'encryption-keys' as valid 'system' subpath Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 08/27] ui: expose 'encryption-keys' as acl subpath for 'system' Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 09/27] sync: add helper to check encryption key acls and load key Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 10/27] api: config: add endpoints for encryption key manipulation Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 11/27] api: config: check sync owner has access to en-/decryption keys Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 12/27] api: config: allow encryption key manipulation for sync job Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 13/27] sync: push: rewrite manifest instead of pushing pre-existing one Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 14/27] api: push sync: expose optional encryption key for push sync Christian Ebner
2026-04-10 16:54 ` Christian Ebner [this message]
2026-04-10 16:54 ` [PATCH proxmox-backup v2 16/27] sync: push: optionally encrypt client log on upload if key is given Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 17/27] sync: push: add helper for loading known chunks from previous snapshot Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 18/27] fix #7251: api: push: encrypt snapshots using configured encryption key Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 19/27] ui: define and expose encryption key management menu item and windows Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 20/27] ui: expose assigning encryption key to sync jobs Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 21/27] sync: pull: load encryption key if given in job config Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 22/27] sync: expand source chunk reader trait by crypt config Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 23/27] sync: pull: introduce and use decrypt index writer if " Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 24/27] sync: pull: extend encountered chunk by optional decrypted digest Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 25/27] sync: pull: decrypt blob files on pull if encryption key is configured Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 26/27] sync: pull: decrypt chunks and rewrite index file for matching key Christian Ebner
2026-04-10 16:54 ` [PATCH proxmox-backup v2 27/27] sync: pull: decrypt snapshots with matching encryption key fingerprint Christian Ebner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260410165454.1578501-16-c.ebner@proxmox.com \
    --to=c.ebner@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal