From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id A843A1FF136 for ; Mon, 20 Apr 2026 18:16:34 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 51F128CB1; Mon, 20 Apr 2026 18:16:25 +0200 (CEST) From: Christian Ebner To: pbs-devel@lists.proxmox.com Subject: [PATCH proxmox-backup v4 04/30] datastore: blob: implement async reader for data blobs Date: Mon, 20 Apr 2026 18:15:07 +0200 Message-ID: <20260420161533.1055484-5-c.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260420161533.1055484-1-c.ebner@proxmox.com> References: <20260420161533.1055484-1-c.ebner@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1776701663318 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.070 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: JTOTB2MARKKDBPC2QHGEPVAT76EJQ3FI X-Message-ID-Hash: JTOTB2MARKKDBPC2QHGEPVAT76EJQ3FI X-MailFrom: c.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: So it can be used to load the blob file when server side encryption is required during push sync jobs, which runs in async context. Factor out the DataBlob and CRC check, which is identical for sync and async reader implementation. Signed-off-by: Christian Ebner --- pbs-datastore/src/data_blob.rs | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/pbs-datastore/src/data_blob.rs b/pbs-datastore/src/data_blob.rs index 0c05c5a40..b434243c0 100644 --- a/pbs-datastore/src/data_blob.rs +++ b/pbs-datastore/src/data_blob.rs @@ -2,6 +2,7 @@ use std::io::Write; use anyhow::{bail, Error}; use openssl::symm::{decrypt_aead, Mode}; +use tokio::io::{AsyncRead, AsyncReadExt}; use proxmox_io::{ReadExt, WriteExt}; @@ -238,15 +239,26 @@ impl DataBlob { } } - /// Load blob from ``reader``, verify CRC + /// Load data blob via given sync ``reader`` and verify its CRC pub fn load_from_reader(reader: &mut dyn std::io::Read) -> Result { let mut data = Vec::with_capacity(1024 * 1024); reader.read_to_end(&mut data)?; + Self::from_raw_with_crc_check(data) + } - let blob = Self::from_raw(data)?; + /// Load data blob via given async ``reader`` and verify its CRC + pub async fn load_from_async_reader( + reader: &mut (dyn AsyncRead + Unpin + Send), + ) -> Result { + let mut data = Vec::with_capacity(1024 * 1024); + reader.read_to_end(&mut data).await?; + Self::from_raw_with_crc_check(data) + } + /// Generates a data blob from raw input data and checks for matching CRC in header + fn from_raw_with_crc_check(raw_data: Vec) -> Result { + let blob = Self::from_raw(raw_data)?; blob.verify_crc()?; - Ok(blob) } -- 2.47.3