[pbs-devel] [PATCH v2 0/5] Improve corrupt chunk handling

[Stefan Reiter <s.reiter@proxmox.com>]

Verify will now rename chunks it detects as corrupted, so future backups will be
forced to write them. The next GC will then clean these ".bad" files up, since
it has to scan each chunk directory anyway.

In case the last backup uses some of these chunks, but is not the one that
failed verification, the client may still omit these chunks, which could lead to
a broken backup. Patch 4 detects these cases by checking all referenced chunks
for existance (which certainly adds a bit of overhead, especially to otherwise
minimal dirty-bitmap backups).

Additionally, the last patch makes sure all chunks have their atime updated,
even if they won't be written (when they already exist), to eliminate a race
with GC where the chunk might be missing after a successful backup.

Also, friendly ping on:
https://lists.proxmox.com/pipermail/pbs-devel/2020-September/000479.html
This series makes the most sense with that patch already applied, the feedback
from Dominik is addressed here.

v2:
* address Thomas' feedback (as well as Dietmar's comment)
* add patch 5


proxmox-backup: Stefan Reiter (5):
  verify: fix log units
  verify: rename corrupted chunks with .bad extension
  gc: remove .bad files on garbage collect
  backup: check all referenced chunks actually exist
  backup: touch all chunks, even if they exist

 src/api2/backup/environment.rs | 21 +++++++++-
 src/api2/types/mod.rs          |  3 ++
 src/backup/chunk_store.rs      | 73 ++++++++++++++++++++++++++++------
 src/backup/datastore.rs        |  5 ++-
 src/backup/verify.rs           | 34 +++++++++++++++-
 5 files changed, 120 insertions(+), 16 deletions(-)

-- 
2.20.1