Re: [pbs-devel] applied-series: [PATCH v5 proxmox-backup 0/5] fix #5331: GC: avoid multiple atime updates

From: Christian Ebner <c.ebner@proxmox.com>
To: Thomas Lamprecht <t.lamprecht@proxmox.com>,
	Proxmox Backup Server development discussion
	<pbs-devel@lists.proxmox.com>
Subject: Re: [pbs-devel] applied-series: [PATCH v5 proxmox-backup 0/5] fix #5331: GC: avoid multiple atime updates
Date: Thu, 3 Apr 2025 11:55:57 +0200	[thread overview]
Message-ID: <fa3800dd-e812-4c9a-9d3d-2d8673e05355@proxmox.com> (raw)
In-Reply-To: <552903da-7ed8-4b93-9277-e7d612121b82@proxmox.com>

On 4/2/25 19:45, Thomas Lamprecht wrote:
> I decided to move this forward as is, getting the other metrics measured
> like proposed in my reply to patch 5/5 might be still nice, if that really
> results in us choosing a different default or what not we can record those
> metrics in a new commit, else they are at least on the list and all is
> fine anyway. So:
> 
> applied, thanks!

Here the results of the suggested additional tests with focus
on the memory usage dependence on the LRU cache capacity
(average of 3 runs):

cache capacity  runtime [s]  peak RSS [MiB]  peak PSS [MiB]
     128 * 1024   45.3 ± 6.3      92.7 ± 9.3     88.5 ± 12.7
     256 * 1024   41.2 ± 5.3     117.6 ± 5.3    110.8 ±  4.2
     512 * 1024   31.2 ± 5.6     162.7 ± 8.1    165.0 ±  0.1
    1024 * 1024   27.8 ± 3.0     227.3 ± 0.1    232.5 ±  9.1
    2048 * 1024   26.1 ± 3.0     339.3 ± 6.0    332.5 ±  4.3

More of interest is however the difference between GC phase1 and
phase2, observed by a distinct drop in memory usage (not observed if
cache capacity is 0, so I would speculate this can mostly be
attributed to the cached items being dropped). 128k is a bit of
an outlier, not sure about that datapoint.

cache capacity  runtime [s]  delta RSS [MiB]  delta PSS [MiB]
     128 * 1024   45.3 ± 6.3       26.2 ± 9.5       24.0 ± 6.2
     256 * 1024   41.2 ± 5.3       17.5 ± 5.4       17.5 ± 5.5
     512 * 1024   31.2 ± 5.6       41.2 ± 0.1       41.2 ± 0.1
    1024 * 1024   27.8 ± 3.0       82.3 ± 0.1       82.5 ± 0.5
    2048 * 1024   26.1 ± 3.0      163.7 ± 0.6      163.6 ± 0.6

Given these values, halving the currently default capacity of 1024k
might be a reasonable tradeoff and I can send the patch to implement
the datastore tuning option to allow configuring larger capacities if
desired by system administrators.

Rough datastore statistics for these runs were:

Index files: 6837
Original data usage: 120.447 TiB
On-Disk usage: 2.753 TiB (2.29%)
On-Disk chunks: 1525362
Deduplication factor: 43.74
Average chunk size: 1.893 MiB

What do you think?

_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel