From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 493F61FF179 for ; Wed, 29 Oct 2025 12:06:14 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id C231479F5; Wed, 29 Oct 2025 12:06:45 +0100 (CET) From: Christian Ebner To: pbs-devel@lists.proxmox.com Date: Wed, 29 Oct 2025 12:06:25 +0100 Message-ID: <20251029110625.375975-3-c.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251029110625.375975-1-c.ebner@proxmox.com> References: <20251029110625.375975-1-c.ebner@proxmox.com> MIME-Version: 1.0 X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1761735989059 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.046 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pbs-devel] [PATCH proxmox-backup 2/2] verify: distinguish s3 object fetching and chunk loading error X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" Errors while loading chunks from the object store might be cause by transient issues, and must therefore handled so they do not incorrectly mark chunks as corrupt. On creating the chunk from the response data, which includes the chunk header and validity checks, errors must however lead to the chunk being flagged as bad. Adapt the code so these errors are correctly distinguished. This is a followup to commit 3c350f35 ("fix #6665: never rename chunks on s3 client fetch errors") which did not take that into account. Fixes: 3c350f35 ("fix #6665: never rename chunks on s3 client fetch errors") Signed-off-by: Christian Ebner --- Originally on list as https://lore.proxmox.com/pbs-devel/20251016131819.349049-7-c.ebner@proxmox.com/ Extended the commit message to include the commit ref since the original version of the patch. src/backup/verify.rs | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/src/backup/verify.rs b/src/backup/verify.rs index 7172e81e1..f01345b04 100644 --- a/src/backup/verify.rs +++ b/src/backup/verify.rs @@ -292,19 +292,24 @@ impl VerifyWorker { let object_key = pbs_datastore::s3::object_key_from_digest(&info.digest)?; match proxmox_async::runtime::block_on(s3_client.get_object(object_key)) { Ok(Some(response)) => { - let chunk_result = proxmox_lang::try_block!({ - let bytes = - proxmox_async::runtime::block_on(response.content.collect())? - .to_bytes(); - DataBlob::from_raw(bytes.to_vec()) - }); - - match chunk_result { - Ok(chunk) => { - let size = info.size(); - *read_bytes += chunk.raw_size(); - decoder_pool.send((chunk, info.digest, size))?; - *decoded_bytes += size; + match proxmox_async::runtime::block_on(response.content.collect()) { + Ok(raw_chunk) => { + match DataBlob::from_raw(raw_chunk.to_bytes().to_vec()) { + Ok(chunk) => { + let size = info.size(); + *read_bytes += chunk.raw_size(); + decoder_pool.send((chunk, info.digest, size))?; + *decoded_bytes += size; + } + Err(err) => self.add_corrupt_chunk( + info.digest, + errors, + &format!( + "can't verify chunk with digest {} - {err}", + hex::encode(info.digest) + ), + ), + } } Err(err) => { errors.fetch_add(1, Ordering::SeqCst); -- 2.47.3 _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel