From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 8CA661FF17E for ; Thu, 16 Oct 2025 15:18:27 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id DE5EE13C07; Thu, 16 Oct 2025 15:18:42 +0200 (CEST) From: Christian Ebner To: pbs-devel@lists.proxmox.com Date: Thu, 16 Oct 2025 15:18:19 +0200 Message-ID: <20251016131819.349049-7-c.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251016131819.349049-1-c.ebner@proxmox.com> References: <20251016131819.349049-1-c.ebner@proxmox.com> MIME-Version: 1.0 X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1760620715103 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.042 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pbs-devel] [PATCH proxmox-backup 6/6] verify: distinguish s3 object fetching and chunk loading error X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" Errors while loading chunks from the object store might be cause by transient issues, and must therefore handled so they do not incorrectly mark chunks as corrupt. On creating the chunk from the response data, which includes the chunk header and validity checks, errors must however lead to the chunk being flagged as bad. Adapt the code so these errors are correctly distinguished. Signed-off-by: Christian Ebner --- src/backup/verify.rs | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/src/backup/verify.rs b/src/backup/verify.rs index b1066f6f5..4cfd81350 100644 --- a/src/backup/verify.rs +++ b/src/backup/verify.rs @@ -225,19 +225,24 @@ impl VerifyWorker { let object_key = pbs_datastore::s3::object_key_from_digest(&info.digest)?; match proxmox_async::runtime::block_on(s3_client.get_object(object_key)) { Ok(Some(response)) => { - let chunk_result = proxmox_lang::try_block!({ - let bytes = - proxmox_async::runtime::block_on(response.content.collect())? - .to_bytes(); - DataBlob::from_raw(bytes.to_vec()) - }); - - match chunk_result { - Ok(chunk) => { - let size = info.size(); - *read_bytes += chunk.raw_size(); - decoder_pool.send((chunk, info.digest, size))?; - *decoded_bytes += size; + match proxmox_async::runtime::block_on(response.content.collect()) { + Ok(raw_chunk) => { + match DataBlob::from_raw(raw_chunk.to_bytes().to_vec()) { + Ok(chunk) => { + let size = info.size(); + *read_bytes += chunk.raw_size(); + decoder_pool.send((chunk, info.digest, size))?; + *decoded_bytes += size; + } + Err(err) => self.add_corrupt_chunk( + info.digest, + errors, + &format!( + "can't verify chunk with digest {} - {err}", + hex::encode(info.digest) + ), + ), + } } Err(err) => { errors.fetch_add(1, Ordering::SeqCst); -- 2.47.3 _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel