From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 456A51FF17A for ; Tue, 25 Nov 2025 13:58:27 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 917D516833; Tue, 25 Nov 2025 13:58:38 +0100 (CET) From: Christian Ebner To: pbs-devel@lists.proxmox.com Date: Tue, 25 Nov 2025 13:58:26 +0100 Message-ID: <20251125125826.433235-1-c.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1764075479674 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.048 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pbs-devel] [PATCH proxmox-backup] GC: chunk store: fix chunk using markers cleanup X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" Since commit 9510ef1a ("GC: assure chunk exists on s3 store when creating missing chunk marker") chunks which are referenced by an index file but do not have a local marker file are marked by a file with the `using` extension, so they are not cleaned up during phase 2 if the chunk is still present on the backend. If the chunk is however not encountered, phase 3 will see the marker and tries to clean it up, which currently however fails because it is first tried to be cleaned up from the LRU cache, the filename being converted to the chunk digest. Therefore, as stop gap make sure using markers are only cleaned up by unlinking the file, not by going over the datastore cache. Fixes: https://forum.proxmox.com/threads/176567/post-819437 Signed-off-by: Christian Ebner --- Note: Make sure to set gc-atime-cutoff to be low enough to not have to wait for the default. pbs-datastore/src/chunk_store.rs | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/pbs-datastore/src/chunk_store.rs b/pbs-datastore/src/chunk_store.rs index f53460664..f3fe8ec16 100644 --- a/pbs-datastore/src/chunk_store.rs +++ b/pbs-datastore/src/chunk_store.rs @@ -25,6 +25,8 @@ use crate::file_formats::{ }; use crate::{DataBlob, LocalDatastoreLruCache}; +const USING_MARKER_FILENAME_EXT: &str = "using"; + /// File system based chunk store pub struct ChunkStore { name: String, // used for error reporting @@ -440,7 +442,11 @@ impl ChunkStore { || { // non-bad S3 chunks need to be removed via cache if let Some(cache) = cache { - if !bad { + if !bad + && !filename + .to_bytes() + .ends_with(USING_MARKER_FILENAME_EXT.as_bytes()) + { let digest = <[u8; 32]>::from_hex(filename.to_bytes())?; // unless there is a concurrent upload pending, @@ -776,7 +782,7 @@ impl ChunkStore { /// Helper to generate marker file path for expected chunks fn chunk_expected_marker_path(&self, digest: &[u8; 32]) -> PathBuf { let (mut path, _digest_str) = self.chunk_path(digest); - path.set_extension("using"); + path.set_extension(USING_MARKER_FILENAME_EXT); path } -- 2.47.3 _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel