Re: [pbs-devel] [PATCH v8 pxar proxmox-backup 00/69] fix #3174: improve file-level backup

[Christian Ebner <c.ebner@proxmox.com>]

[-- Attachment #1: Type: text/plain, Size: 724 bytes --]

On 5/31/24 12:40, Dominik Csapak wrote:
> for reference, as discussed off-list
> 
> there seems to be an issue (with this version at least, not sure if it's 
> actually a regression)
> that does not correctly reuse entries after encountering a changed file
> 
> (changed 1 file + added one file (~20MiB) and only ~600 out of ~500000 
> (~10GiB) were reused)

Thanks for reporting this and swift help to find the underlying issue 
Dominik!
The fix will be included in an upcoming version 9 of the patches, I 
still want to wait a bit for eventual further feedback before spamming 
the mailing list with to many intermediate iterations.
For now, the attached patch can be applied as followup to the series to 
fix this issue.

[-- Attachment #2: 0001-client-pxar-conditional-flushing-of-cached-entries.patch --]
[-- Type: text/x-diff, Size: 1352 bytes --]

From 18e8b6868780869e9d46292b73a56687ca44483b Mon Sep 17 00:00:00 2001
From: Christian Ebner <c.ebner@proxmox.com>
Date: Fri, 31 May 2024 13:07:52 +0200
Subject: [PATCH proxmox-backup] client: pxar: conditional flushing of cached
 entries

Entries with unchanged metadata are placed on the lookahead cache in
order to postpone a decition whether to reuse or reencode the
entries. In case of a non-reusable entry, the cached entries must be
reencoded if their padding exceeds the threshold, they should however
be reused otherwise.
This patch adds the missing check to allow the reuse of cached
entries in such a case.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 pbs-client/src/pxar/create.rs | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/pbs-client/src/pxar/create.rs b/pbs-client/src/pxar/create.rs
index c30ba340f..da7807b49 100644
--- a/pbs-client/src/pxar/create.rs
+++ b/pbs-client/src/pxar/create.rs
@@ -791,6 +791,9 @@ impl Archiver {
                 self.cache
                     .insert(fd, c_file_name.into(), *stat, metadata.clone(), offset);
                 return Ok(());
+            } else {
+                self.flush_cached_reusing_if_below_threshold(encoder, false)
+                    .await?;
             }
         } else if self.cache.caching_enabled() {
             self.cache.insert(
-- 
2.39.2


[-- Attachment #3: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel