From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 3E5601FF189 for ; Sat, 22 Nov 2025 11:41:50 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id DFF885C36; Sat, 22 Nov 2025 11:41:56 +0100 (CET) From: Christian Ebner To: pbs-devel@lists.proxmox.com Date: Sat, 22 Nov 2025 11:41:18 +0100 Message-ID: <20251122104118.205994-1-c.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1763808049664 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.048 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pbs-devel] [PATCH proxmox-backup] GC: s3: fix local marker cleanup for unreferenced, s3 only chunks X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" If a chunk object is located on the s3 object store only, not being referenced by any index file and having no local marker file it is marked for cleanup by pretending an atime equal to the unix epoch. While this will mark the chunk for deletion from the backend and include it in the delete list for the next s3 delete objects call, it also will lead to the chunk marker and LRU cache entry being tried to clean up locally, which however fails since there is no marker to be cleaned up. In order to treat this edge case with the same cleanup logic, simply insert the marker file if not present, for it to get correctly cleaned up as expected afterwards. This should not happen under normal datastore operation anyways, most likely to appear after re-creation of the datastore from existing bucket contents containing such unreferenced chunks. Fixes: https://forum.proxmox.com/threads/176567/ Signed-off-by: Christian Ebner --- pbs-datastore/src/datastore.rs | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs index 65299cca9..a24392d9f 100644 --- a/pbs-datastore/src/datastore.rs +++ b/pbs-datastore/src/datastore.rs @@ -1711,11 +1711,12 @@ impl DataStore { let atime = match std::fs::metadata(&chunk_path) { Ok(stat) => stat.accessed()?, Err(err) if err.kind() == std::io::ErrorKind::NotFound => { + unsafe { + // chunk store lock held + // insert marke unconditionally, cleaned up again below if required + self.inner.chunk_store.replace_chunk_with_marker(&digest)?; + } if self.inner.chunk_store.clear_chunk_expected_mark(&digest)? { - unsafe { - // chunk store lock held - self.inner.chunk_store.replace_chunk_with_marker(&digest)?; - } SystemTime::now() } else { // File not found, delete by setting atime to unix epoch -- 2.47.3 _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel