From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 3F14D1FF17E for ; Thu, 16 Oct 2025 15:18:21 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id F036613AD3; Thu, 16 Oct 2025 15:18:40 +0200 (CEST) From: Christian Ebner To: pbs-devel@lists.proxmox.com Date: Thu, 16 Oct 2025 15:18:13 +0200 Message-ID: <20251016131819.349049-1-c.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1760620713706 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.042 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [datastore.rs, verify.rs] Subject: [pbs-devel] [PATCH proxmox-backup 0/6] s3 store verify: fix concurrency issues and add missing in-memory cache eviction X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" This patch series addresses issues with verification tasks on s3 backends. Most importantly, it is assured that corrupt chunks are correctly evicted also from the in-memory LRU cache, as otherwise new, non-corrupt chunk will not be re-inserted into the chunk store if the digest is still cached as recently used. Also, it refines the distinction between fetching error and chunk parsing error handling, as the latter must flag the chunk as bad, the former not. Further, avoid issues by concurrent chunk inserts and or garbage collection while renaming corrupt chunks. The chunk store mutex lock was never acquired, independent of the datastore backend, although being less problematic on filesystem backends due to the atomic nature of the chunk file rename. Finally, make sure that the mutex lock guarding the corrupt chunk list is not held when entering the rename helper, as that performs async api calls to the object store for datastores backed by S3, which could potentially lead to deadlocks. proxmox-backup: Christian Ebner (6): verify/datastore: make rename corrupt chunk a datastore helper method datastore: refactor rename_corrupted_chunk error handling verify: never hold mutex lock in async scope on corrupt chunk rename datastore: acquire chunk store mutex lock when renaming corrupt chunk datastore: verify: evict corrupt chunks from in-memory LRU cache verify: distinguish s3 object fetching and chunk loading error pbs-datastore/src/datastore.rs | 63 ++++++++++++++++++ src/backup/verify.rs | 117 ++++++++------------------------- 2 files changed, 92 insertions(+), 88 deletions(-) Summary over all repositories: 2 files changed, 92 insertions(+), 88 deletions(-) -- Generated by git-murpp 0.8.1 _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel