From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 0A8381FF185 for ; Mon, 21 Jul 2025 15:23:55 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id EB3C7FB8E; Mon, 21 Jul 2025 15:25:05 +0200 (CEST) Message-ID: Date: Mon, 21 Jul 2025 15:24:31 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: Proxmox Backup Server development discussion , Hannes Laimer References: <20250719125035.9926-1-c.ebner@proxmox.com> <20250719125035.9926-19-c.ebner@proxmox.com> Content-Language: en-US, de-DE From: Christian Ebner In-Reply-To: X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1753104265140 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.046 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] [PATCH proxmox-backup v9 15/46] datastore: local chunk reader: read chunks based on backend X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Cc: pbs-devel Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" On 7/21/25 3:13 PM, Hannes Laimer wrote: > On Sat Jul 19, 2025 at 2:50 PM CEST, Christian Ebner wrote: >> Get and store the datastore's backend on local chunk reader >> instantiantion and fetch chunks based on the variant from either the >> filesystem or the s3 object store. >> >> By storing the backend variant, the s3 client is instantiated only >> once and reused until the local chunk reader instance is dropped. >> >> Signed-off-by: Christian Ebner >> --- >> changes since version 8: >> - use Arc::clone() over .clone() >> >> pbs-datastore/Cargo.toml | 1 + >> pbs-datastore/src/local_chunk_reader.rs | 38 +++++++++++++++++++++---- >> 2 files changed, 33 insertions(+), 6 deletions(-) >> >> diff --git a/pbs-datastore/Cargo.toml b/pbs-datastore/Cargo.toml >> index 7e56dbd31..8ce930a94 100644 >> --- a/pbs-datastore/Cargo.toml >> +++ b/pbs-datastore/Cargo.toml >> @@ -13,6 +13,7 @@ crc32fast.workspace = true >> endian_trait.workspace = true >> futures.workspace = true >> hex = { workspace = true, features = [ "serde" ] } >> +http-body-util.workspace = true >> hyper.workspace = true >> libc.workspace = true >> log.workspace = true >> diff --git a/pbs-datastore/src/local_chunk_reader.rs b/pbs-datastore/src/local_chunk_reader.rs >> index 05a70c068..667c97206 100644 >> --- a/pbs-datastore/src/local_chunk_reader.rs >> +++ b/pbs-datastore/src/local_chunk_reader.rs >> @@ -3,17 +3,21 @@ use std::pin::Pin; >> use std::sync::Arc; >> >> use anyhow::{bail, Error}; >> +use http_body_util::BodyExt; >> >> use pbs_api_types::CryptMode; >> use pbs_tools::crypt_config::CryptConfig; >> +use proxmox_s3_client::S3Client; >> >> use crate::data_blob::DataBlob; >> +use crate::datastore::DatastoreBackend; >> use crate::read_chunk::{AsyncReadChunk, ReadChunk}; >> use crate::DataStore; >> >> #[derive(Clone)] >> pub struct LocalChunkReader { >> store: Arc, >> + backend: DatastoreBackend, >> crypt_config: Option>, >> crypt_mode: CryptMode, >> } >> @@ -24,8 +28,11 @@ impl LocalChunkReader { >> crypt_config: Option>, >> crypt_mode: CryptMode, >> ) -> Self { >> + // TODO: Error handling! >> + let backend = store.backend().unwrap(); > > Was this missed, or was this intentionally left in? > I feel like we don't want to panic here :P > (correct me if I'm wrong, but I think we would whenever anything goes > wrong when connecting to s3?) Ah, good catch, this was overlooked indeed, will fix this, thanks! _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel