[pbs-devel] [PATCH proxmox{, -backup} v3 0/2] add GC chunk LRU cache stats

[pbs-devel] [PATCH proxmox{, -backup} v3 0/2] add GC chunk LRU cache stats

[Christian Ebner]

Allows better fine-tuning of the garbage collection cache capacity by
providing the hit and miss count, as well as the hit ratio as output
to the garbage collection task log.

Changes since version 2 (thanks Fabian for feedback):
- Combine stats counters into struct and make them optional. This
  avoids parsing issues during upgrade.

Changes since version 1 (thanks Lukas for feedback):
- Also display the cache hit ratio
- Fix the cache not being disabled when the capacity is set to 0,
  discovered while investigating the hit ratio for different
  capacities.

proxmox:

Christian Ebner (1):
  pbs api types: extend garbage collection status by cache stats

 pbs-api-types/src/datastore.rs | 13 +++++++++++++
 1 file changed, 13 insertions(+)


proxmox-backup:

Christian Ebner (1):
  garbage collection: track chunk cache stats and show in task log

 pbs-datastore/src/datastore.rs | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)


Summary over all repositories:
  2 files changed, 33 insertions(+), 2 deletions(-)

-- 
Generated by git-murpp 0.8.1


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox master stable-bookworm v3 1/1] pbs api types: extend garbage collection status by cache stats

[Christian Ebner]

Add cache stats, containing the number of cache hits and cache misses
encountered during phase 1 of garbage collection in order to display
this information in the garbage collection task log summary.

Make stats optional for backwards compatibility when parsing garbage
collection status during upgrades.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
changes since version 2:
- Factor out cache counters into own api type
- Make cache stats optional to avoid parsing issues

 pbs-api-types/src/datastore.rs | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/pbs-api-types/src/datastore.rs b/pbs-api-types/src/datastore.rs
index 5bd953ac..4fa4c091 100644
--- a/pbs-api-types/src/datastore.rs
+++ b/pbs-api-types/src/datastore.rs
@@ -1426,6 +1426,16 @@ pub struct TypeCounts {
     pub snapshots: u64,
 }
 
+#[api()]
+#[derive(Clone, Debug, Default, Serialize, Deserialize, PartialEq)]
+/// Garbage collection cache statistics
+pub struct GarbageCollectionCacheStats {
+    /// Number of atime update cache hits
+    pub hits: usize,
+    /// Number of atime update cache misses
+    pub misses: usize,
+}
+
 #[api(
     properties: {
         "upid": {
@@ -1459,6 +1469,9 @@ pub struct GarbageCollectionStatus {
     pub removed_bad: usize,
     /// Number of chunks still marked as .bad after garbage collection.
     pub still_bad: usize,
+    /// Statistics of atime update cache
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub cache_stats: Option<GarbageCollectionCacheStats>,
 }
 
 #[api(
-- 
2.39.5



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox-backup v3 1/1] garbage collection: track chunk cache stats and show in task log

[Christian Ebner]

Count the chunk cache hits and misses and display the resulting
values and the hit ratio in the garbage collection task log summary.

This allows to investigate possible issues and tune cache capacity,
also by being able to compare to other values in the summary such
as the on disk chunk count.

Exemplary output
```
2025-05-16T22:31:53+02:00: Chunk cache: hits 15817, misses 873 (hit ratio 94.77%)
2025-05-16T22:31:53+02:00: Removed garbage: 0 B
2025-05-16T22:31:53+02:00: Removed chunks: 0
2025-05-16T22:31:53+02:00: Original data usage: 64.961 GiB
2025-05-16T22:31:53+02:00: On-Disk usage: 1.037 GiB (1.60%)
2025-05-16T22:31:53+02:00: On-Disk chunks: 874
2025-05-16T22:31:53+02:00: Deduplication factor: 62.66
2025-05-16T22:31:53+02:00: Average chunk size: 1.215 MiB
```

Sidenote: the discrepancy between cache miss counter and on-disk
chunk count in the output shown above can be attributed to the all
zero chunk, inserted during the atime update check at the start of
garbage collection, however not being referenced by any index file in
this examplary case.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
changes since version 2:
- Refactor and initialize to use GarbageCollectionCacheStats for cache
  stats

 pbs-datastore/src/datastore.rs | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs
index 924d8cf9c..d663465e2 100644
--- a/pbs-datastore/src/datastore.rs
+++ b/pbs-datastore/src/datastore.rs
@@ -23,8 +23,8 @@ use proxmox_worker_task::WorkerTaskContext;
 
 use pbs_api_types::{
     ArchiveType, Authid, BackupGroupDeleteStats, BackupNamespace, BackupType, ChunkOrder,
-    DataStoreConfig, DatastoreFSyncLevel, DatastoreTuning, GarbageCollectionStatus,
-    MaintenanceMode, MaintenanceType, Operation, UPID,
+    DataStoreConfig, DatastoreFSyncLevel, DatastoreTuning, GarbageCollectionCacheStats,
+    GarbageCollectionStatus, MaintenanceMode, MaintenanceType, Operation, UPID,
 };
 use pbs_config::BackupLockGuard;
 
@@ -1098,8 +1098,14 @@ impl DataStore {
             // Avoid multiple expensive atime updates by utimensat
             if let Some(chunk_lru_cache) = chunk_lru_cache {
                 if chunk_lru_cache.insert(*digest, ()) {
+                    if let Some(cache_stats) = status.cache_stats.as_mut() {
+                        cache_stats.hits += 1;
+                    }
                     continue;
                 }
+                if let Some(cache_stats) = status.cache_stats.as_mut() {
+                    cache_stats.misses += 1;
+                }
             }
 
             if !self.inner.chunk_store.cond_touch_chunk(digest, false)? {
@@ -1304,6 +1310,7 @@ impl DataStore {
 
             let mut gc_status = GarbageCollectionStatus {
                 upid: Some(upid.to_string()),
+                cache_stats: Some(GarbageCollectionCacheStats::default()),
                 ..Default::default()
             };
             let tuning: DatastoreTuning = serde_json::from_value(
@@ -1366,6 +1373,17 @@ impl DataStore {
                 worker,
             )?;
 
+            if let Some(cache_stats) = &gc_status.cache_stats {
+                let total_cache_counts = cache_stats.hits + cache_stats.misses;
+                if total_cache_counts > 0 {
+                    let cache_hit_ratio =
+                        (cache_stats.hits as f64 * 100.) / total_cache_counts as f64;
+                    info!(
+                        "Chunk cache: hits {}, misses {} (hit ratio {cache_hit_ratio:.2}%)",
+                        cache_stats.hits, cache_stats.misses,
+                    );
+                }
+            }
             info!(
                 "Removed garbage: {}",
                 HumanByte::from(gc_status.removed_bytes),
-- 
2.39.5



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel