[pbs-devel] [PATCH v2 proxmox-backup 3/4] garbage collection: allow to keep track of already touched chunks

[Christian Ebner <c.ebner@proxmox.com>]

Implements the `TouchedChunks` struct and methods to keep track of
already touched chunks during garbage collection phase 1, to avoid
multiple computational and I/O intensive atime updates via a syscall.

By inserting a digest, the chunk will be considered as touched and
can be ignored for subsequent encounters. To limit memory usage, the
structure allows to reset the chunk status, flagging them as seen
previous to the reset. A subsequent insert will then flag it as seen
after the reset. Chunks not seen after a reset, will be cleared from
the structure by the next reset call, eliminating them from memory.

This allows to reset the tracking stat after each processes image
index file, to mimic the incremental backup behaviour of known chunks
and limit memory footprint.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
changes since version 1:
- no changes

 pbs-datastore/src/datastore.rs | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs
index 72bc9f77f..fdbb33a98 100644
--- a/pbs-datastore/src/datastore.rs
+++ b/pbs-datastore/src/datastore.rs
@@ -1585,3 +1585,32 @@ impl DataStore {
         Ok(())
     }
 }
+
+struct TouchedChunks {
+    list: HashMap<[u8; 32], bool>,
+}
+
+impl TouchedChunks {
+    fn new() -> Self {
+        Self {
+            list: HashMap::new(),
+        }
+    }
+
+    // Clear untouched chunks and reset the touched marker for others.
+    fn reset(&mut self) {
+        let mut new_list = HashMap::new();
+        for (digest, touched) in self.list.drain() {
+            if touched {
+                new_list.insert(digest, false);
+            }
+        }
+        self.list = new_list;
+    }
+
+    // Insert the digest in the list of touched chunks.
+    // Returns true if the chunk was already present, false otherwise.
+    fn insert(&mut self, digest: [u8; 32]) -> bool {
+        self.list.insert(digest, true).is_some()
+    }
+}
-- 
2.39.5



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel