[pbs-devel] [PATCH v2 proxmox] pbs api types: add garbage collection cache capacity tuning option

[Christian Ebner <c.ebner@proxmox.com>]

Allows to adjust the capacity for the LRU cache used to keep track of
recently touched chunks during phase 1 of garbage collection.

Values are provided as multiples of 1024 cache entries, the default
value of 1024 * 1024  was chosen as tradeoff between runtime
improvements and memory usage [0]. The maximum of 8192 * 1024 was
chosen based on the linear regression from [1], resulting in about
8 * 80 MiB = 640 MiB of memory requirement, while allowing to keep
chunks which can reference about 32 TiB of data in case of 4 MiB
fixed size chunks.

[0] https://git.proxmox.com/?p=proxmox-backup.git;a=commit;h=03143eee0a59cf319be0052e139f7e20e124d572
[1] https://lore.proxmox.com/pbs-devel/fa3800dd-e812-4c9a-9d3d-2d8673e05355@proxmox.com/

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
changes since version 1:
- use raw value for cache capacity, not multiples of 1024

 pbs-api-types/src/datastore.rs | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/pbs-api-types/src/datastore.rs b/pbs-api-types/src/datastore.rs
index ddd8d3c6..020d8aa7 100644
--- a/pbs-api-types/src/datastore.rs
+++ b/pbs-api-types/src/datastore.rs
@@ -223,12 +223,23 @@ pub enum DatastoreFSyncLevel {
     Filesystem,
 }
 
+pub const GC_CACHE_CAPACITY_SCHEMA: Schema =
+    IntegerSchema::new("Garbage collection chunk digest cache capacity")
+        .minimum(0)
+        .maximum(8192 * 1024)
+        .default(1024 * 1024)
+        .schema();
+
 #[api(
     properties: {
         "chunk-order": {
             type: ChunkOrder,
             optional: true,
         },
+        "gc-cache-capacity": {
+            schema: GC_CACHE_CAPACITY_SCHEMA,
+            optional: true,
+        },
     },
 )]
 #[derive(Serialize, Deserialize, Default)]
@@ -240,6 +251,8 @@ pub struct DatastoreTuning {
     pub chunk_order: Option<ChunkOrder>,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub sync_level: Option<DatastoreFSyncLevel>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub gc_cache_capacity: Option<usize>,
 }
 
 pub const DATASTORE_TUNING_STRING_SCHEMA: Schema = StringSchema::new("Datastore tuning options")
-- 
2.39.5



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel