Re: [pbs-devel] [PATCH v2 0/5] Improve corrupt chunk handling

[Dietmar Maurer <dietmar@proxmox.com>]

applied 0, 1,2,3 and 5

patch 4/5 needs some improvements.

> On 09/07/2020 5:30 PM Stefan Reiter <s.reiter@proxmox.com> wrote:
> 
>  
> Verify will now rename chunks it detects as corrupted, so future backups will be
> forced to write them. The next GC will then clean these ".bad" files up, since
> it has to scan each chunk directory anyway.
> 
> In case the last backup uses some of these chunks, but is not the one that
> failed verification, the client may still omit these chunks, which could lead to
> a broken backup. Patch 4 detects these cases by checking all referenced chunks
> for existance (which certainly adds a bit of overhead, especially to otherwise
> minimal dirty-bitmap backups).
> 
> Additionally, the last patch makes sure all chunks have their atime updated,
> even if they won't be written (when they already exist), to eliminate a race
> with GC where the chunk might be missing after a successful backup.
> 
> Also, friendly ping on:
> https://lists.proxmox.com/pipermail/pbs-devel/2020-September/000479.html
> This series makes the most sense with that patch already applied, the feedback
> from Dominik is addressed here.
> 
> v2:
> * address Thomas' feedback (as well as Dietmar's comment)
> * add patch 5
> 
> 
> proxmox-backup: Stefan Reiter (5):
>   verify: fix log units
>   verify: rename corrupted chunks with .bad extension
>   gc: remove .bad files on garbage collect
>   backup: check all referenced chunks actually exist
>   backup: touch all chunks, even if they exist
> 
>  src/api2/backup/environment.rs | 21 +++++++++-
>  src/api2/types/mod.rs          |  3 ++
>  src/backup/chunk_store.rs      | 73 ++++++++++++++++++++++++++++------
>  src/backup/datastore.rs        |  5 ++-
>  src/backup/verify.rs           | 34 +++++++++++++++-
>  5 files changed, 120 insertions(+), 16 deletions(-)
> 
> -- 
> 2.20.1
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel