public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Christian Ebner <c.ebner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-backup v2 09/12] chunk store: refactor method for chunk insertion
Date: Wed,  8 Oct 2025 17:21:22 +0200	[thread overview]
Message-ID: <20251008152125.849216-10-c.ebner@proxmox.com> (raw)
In-Reply-To: <20251008152125.849216-1-c.ebner@proxmox.com>

Extend the current implementation of the chunk insertion by a method
to provide a callback to be executed before writing the chunk file,
but after checking if the file is pre-existing.

This is in preparation for different methods for chunk insertion,
with differentiated handling of the non pre-existing chunk file case.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 pbs-datastore/src/chunk_store.rs | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/pbs-datastore/src/chunk_store.rs b/pbs-datastore/src/chunk_store.rs
index 6e50327cb..65d74d68e 100644
--- a/pbs-datastore/src/chunk_store.rs
+++ b/pbs-datastore/src/chunk_store.rs
@@ -558,6 +558,15 @@ impl ChunkStore {
     }
 
     pub fn insert_chunk(&self, chunk: &DataBlob, digest: &[u8; 32]) -> Result<(bool, u64), Error> {
+        self.insert_chunk_impl(chunk, digest, |_, _| Ok(()))
+    }
+
+    fn insert_chunk_impl(
+        &self,
+        chunk: &DataBlob,
+        digest: &[u8; 32],
+        before_create: impl FnOnce(&[u8; 32], bool) -> Result<(), Error>,
+    ) -> Result<(bool, u64), Error> {
         // unwrap: only `None` in unit tests
         assert!(self.locker.is_some());
 
@@ -572,7 +581,7 @@ impl ChunkStore {
 
         let name = &self.name;
 
-        if let Ok(metadata) = std::fs::metadata(&chunk_path) {
+        let pre_existing = if let Ok(metadata) = std::fs::metadata(&chunk_path) {
             if !metadata.is_file() {
                 bail!("got unexpected file type on store '{name}' for chunk {digest_str}");
             }
@@ -612,7 +621,12 @@ impl ChunkStore {
             } else {
                 log::debug!("Got another copy of chunk with digest '{digest_str}', existing chunk is bigger, replacing with uploaded one.");
             }
-        }
+            true
+        } else {
+            false
+        };
+
+        before_create(digest, pre_existing)?;
 
         let chunk_dir_path = chunk_path
             .parent()
-- 
2.47.3



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


  parent reply	other threads:[~2025-10-08 15:22 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-10-08 15:21 [pbs-devel] [PATCH proxmox-backup v2 00/12] s3 store: fix issues with chunk s3 backend upload and cache eviction Christian Ebner
2025-10-08 15:21 ` [pbs-devel] [PATCH proxmox-backup v2 01/12] datastore: gc: inline single callsite method Christian Ebner
2025-10-08 15:21 ` [pbs-devel] [PATCH proxmox-backup v2 02/12] gc: chunk store: rework atime check and gc status into common helper Christian Ebner
2025-10-08 15:21 ` [pbs-devel] [PATCH proxmox-backup v2 03/12] chunk store: add unsafe signature to cache remove method Christian Ebner
2025-10-08 15:21 ` [pbs-devel] [PATCH proxmox-backup v2 04/12] local store cache: replace evicted cache chunks instead of truncate Christian Ebner
2025-10-08 15:21 ` [pbs-devel] [PATCH proxmox-backup v2 05/12] local store cache: serve response fetched from s3 backend Christian Ebner
2025-10-08 15:21 ` [pbs-devel] [PATCH proxmox-backup v2 06/12] local store cache: refactor fetch and insert of chunks for " Christian Ebner
2025-10-08 15:21 ` [pbs-devel] [PATCH proxmox-backup v2 07/12] local store cache: rework access cache fetching and insert logic Christian Ebner
2025-10-08 15:21 ` [pbs-devel] [PATCH proxmox-backup v2 08/12] local store cache: drop obsolete cacher implementation Christian Ebner
2025-10-08 15:21 ` Christian Ebner [this message]
2025-10-08 15:21 ` [pbs-devel] [PATCH proxmox-backup v2 10/12] api: chunk upload: fix race between chunk backend upload and insert Christian Ebner
2025-10-08 15:21 ` [pbs-devel] [PATCH proxmox-backup v2 11/12] api: chunk upload: fix race with garbage collection for no-cache on s3 Christian Ebner
2025-10-08 15:21 ` [pbs-devel] [PATCH proxmox-backup v2 12/12] pull: guard chunk upload and only insert into cache after upload Christian Ebner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20251008152125.849216-10-c.ebner@proxmox.com \
    --to=c.ebner@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal