From: Christian Ebner <c.ebner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [PATCH proxmox-backup v3 04/30] datastore: blob: implement async reader for data blobs
Date: Tue, 14 Apr 2026 14:58:57 +0200 [thread overview]
Message-ID: <20260414125923.892345-5-c.ebner@proxmox.com> (raw)
In-Reply-To: <20260414125923.892345-1-c.ebner@proxmox.com>
So it can be used to load the blob file when server side encryption
is required during push sync jobs, which runs in async context.
Factor out the DataBlob and CRC check, which is identical for sync
and async reader implementation.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
changes since version 2:
- fix typos in commit message for async DataBlob reader patch.
pbs-datastore/src/data_blob.rs | 18 +++++++++++++++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/pbs-datastore/src/data_blob.rs b/pbs-datastore/src/data_blob.rs
index 0c05c5a40..b434243c0 100644
--- a/pbs-datastore/src/data_blob.rs
+++ b/pbs-datastore/src/data_blob.rs
@@ -2,6 +2,7 @@ use std::io::Write;
use anyhow::{bail, Error};
use openssl::symm::{decrypt_aead, Mode};
+use tokio::io::{AsyncRead, AsyncReadExt};
use proxmox_io::{ReadExt, WriteExt};
@@ -238,15 +239,26 @@ impl DataBlob {
}
}
- /// Load blob from ``reader``, verify CRC
+ /// Load data blob via given sync ``reader`` and verify its CRC
pub fn load_from_reader(reader: &mut dyn std::io::Read) -> Result<Self, Error> {
let mut data = Vec::with_capacity(1024 * 1024);
reader.read_to_end(&mut data)?;
+ Self::from_raw_with_crc_check(data)
+ }
- let blob = Self::from_raw(data)?;
+ /// Load data blob via given async ``reader`` and verify its CRC
+ pub async fn load_from_async_reader(
+ reader: &mut (dyn AsyncRead + Unpin + Send),
+ ) -> Result<Self, Error> {
+ let mut data = Vec::with_capacity(1024 * 1024);
+ reader.read_to_end(&mut data).await?;
+ Self::from_raw_with_crc_check(data)
+ }
+ /// Generates a data blob from raw input data and checks for matching CRC in header
+ fn from_raw_with_crc_check(raw_data: Vec<u8>) -> Result<Self, Error> {
+ let blob = Self::from_raw(raw_data)?;
blob.verify_crc()?;
-
Ok(blob)
}
--
2.47.3
next prev parent reply other threads:[~2026-04-14 12:58 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-04-14 12:58 [PATCH proxmox{,-backup} v3 00/30] fix #7251: implement server side encryption support for push sync jobs Christian Ebner
2026-04-14 12:58 ` [PATCH proxmox v3 01/30] pbs-api-types: define en-/decryption key type and schema Christian Ebner
2026-04-14 12:58 ` [PATCH proxmox v3 02/30] pbs-api-types: sync job: add optional cryptographic keys to config Christian Ebner
2026-04-14 12:58 ` [PATCH proxmox-backup v3 03/30] sync: push: use tracing macros instead of log Christian Ebner
2026-04-14 12:58 ` Christian Ebner [this message]
2026-04-14 12:58 ` [PATCH proxmox-backup v3 05/30] datastore: manifest: add helper for change detection fingerprint Christian Ebner
2026-04-14 12:58 ` [PATCH proxmox-backup v3 06/30] pbs-key-config: introduce store_with() for KeyConfig Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 07/30] pbs-config: implement encryption key config handling Christian Ebner
2026-04-14 14:32 ` Michael Köppl
2026-04-15 6:48 ` Christian Ebner
2026-04-15 8:03 ` Daniel Kral
2026-04-15 8:21 ` Christian Ebner
2026-04-15 8:06 ` Thomas Lamprecht
2026-04-14 12:59 ` [PATCH proxmox-backup v3 08/30] pbs-config: acls: add 'encryption-keys' as valid 'system' subpath Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 09/30] ui: expose 'encryption-keys' as acl subpath for 'system' Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 10/30] sync: add helper to check encryption key acls and load key Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 11/30] api: config: add endpoints for encryption key manipulation Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 12/30] api: config: check sync owner has access to en-/decryption keys Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 13/30] api: config: allow encryption key manipulation for sync job Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 14/30] sync: push: rewrite manifest instead of pushing pre-existing one Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 15/30] api: push sync: expose optional encryption key for push sync Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 16/30] sync: push: optionally encrypt data blob on upload Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 17/30] sync: push: optionally encrypt client log on upload if key is given Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 18/30] sync: push: add helper for loading known chunks from previous snapshot Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 19/30] fix #7251: api: push: encrypt snapshots using configured encryption key Christian Ebner
2026-04-15 14:49 ` Michael Köppl
2026-04-15 15:25 ` Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 20/30] ui: define and expose encryption key management menu item and windows Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 21/30] ui: expose assigning encryption key to sync jobs Christian Ebner
2026-04-15 14:49 ` Michael Köppl
2026-04-15 15:20 ` Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 22/30] sync: pull: load encryption key if given in job config Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 23/30] sync: expand source chunk reader trait by crypt config Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 24/30] sync: pull: introduce and use decrypt index writer if " Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 25/30] sync: pull: extend encountered chunk by optional decrypted digest Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 26/30] sync: pull: decrypt blob files on pull if encryption key is configured Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 27/30] sync: pull: decrypt chunks and rewrite index file for matching key Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 28/30] sync: pull: decrypt snapshots with matching encryption key fingerprint Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 29/30] api: encryption keys: allow to toggle the archived state for keys Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 30/30] docs: add section describing server side encryption for sync jobs Christian Ebner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260414125923.892345-5-c.ebner@proxmox.com \
--to=c.ebner@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.