From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id D4E6D1FF140 for ; Fri, 10 Apr 2026 18:54:57 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id C79E522DBC; Fri, 10 Apr 2026 18:55:42 +0200 (CEST) From: Christian Ebner To: pbs-devel@lists.proxmox.com Subject: [PATCH proxmox-backup v2 03/27] datastore: blob: implement async reader for data blobs Date: Fri, 10 Apr 2026 18:54:30 +0200 Message-ID: <20260410165454.1578501-4-c.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260410165454.1578501-1-c.ebner@proxmox.com> References: <20260410165454.1578501-1-c.ebner@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1775840035754 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.068 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: DAPPYCBS4B7RJB4L7SONRKXPFVWLRSAI X-Message-ID-Hash: DAPPYCBS4B7RJB4L7SONRKXPFVWLRSAI X-MailFrom: c.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: So it can be used to load the blob file when server side encryption is required during push sync jobs, which runs in async context. Factor out the DataBlob and CRC check, which is identical for sync and async reader implementation. Signed-off-by: Christian Ebner --- pbs-datastore/src/data_blob.rs | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/pbs-datastore/src/data_blob.rs b/pbs-datastore/src/data_blob.rs index 0c05c5a40..22b5dfaf9 100644 --- a/pbs-datastore/src/data_blob.rs +++ b/pbs-datastore/src/data_blob.rs @@ -2,6 +2,7 @@ use std::io::Write; use anyhow::{bail, Error}; use openssl::symm::{decrypt_aead, Mode}; +use tokio::io::{AsyncRead, AsyncReadExt}; use proxmox_io::{ReadExt, WriteExt}; @@ -238,15 +239,26 @@ impl DataBlob { } } - /// Load blob from ``reader``, verify CRC + /// Load data blob via given sync ``reader`` and verify it's CRC pub fn load_from_reader(reader: &mut dyn std::io::Read) -> Result { let mut data = Vec::with_capacity(1024 * 1024); reader.read_to_end(&mut data)?; + Self::from_raw_with_crc_check(data) + } - let blob = Self::from_raw(data)?; + /// Load data blob via given async ``reader`` and verify it's CRC + pub async fn load_from_async_reader( + reader: &mut (dyn AsyncRead + Unpin + Send), + ) -> Result { + let mut data = Vec::with_capacity(1024 * 1024); + reader.read_to_end(&mut data).await?; + Self::from_raw_with_crc_check(data) + } + /// Generates a data blob from raw input data and checks for matching CRC in header + fn from_raw_with_crc_check(raw_data: Vec) -> Result { + let blob = Self::from_raw(raw_data)?; blob.verify_crc()?; - Ok(blob) } -- 2.47.3