[PATCH proxmox-backup v3 0/3] fix #6990: server: drop verify state on push & pull job

[Jakob Klocker <j.klocker@proxmox.com>]

When syncing a snapshot to another datastore, the source's verify_state
is currently carried over to the target. This reports the target
snapshot as verified even though its stored copy was never checked
there. Since verify jobs skip snapshots that already carry a
verify_state, the target's copy can't be verified again.

Chunks are checksummed in memory while being transferred, but a verify
serves a different purpose: confirming the write to the (external)
target actually succeeded.

This series stops the target from inheriting the source's
verify_state, so the target is verified independently. On pull, a 
verify_state the target obtained on its own is preserved; on push the
state is dropped unconditionally. It also moves the blocking calls in
the touched code paths off the runtime's worker threads onto the
blocking thread pool.

Tested (verify_state correctly stripped on target):
 * Pull, verified source, new content, clean sync
 * Pull, verified source, existing content, clean sync
 * Pull, corrupted target, resync-corrupt
 * Push, verified source (has no corrupt path)
 * Pull, independently-verified target, clean re-sync - state preserved

changes from v2 to v3 (thanks @Christian):
 * offload remaining removes to `spawn_blocking()`
 * add `target_path` parameter to `write_to_path` and perform the
   rename inside the helper
 * add a helper to return expected files as hashset
 * refactor `cleanup_unreferenced_files` to take the hashset instead
   of the manifest

changes from v1 to v2 (thanks @Christian):
 * move the manifest write helper onto `BackupManifest`
 * offload fsync, rename and cleanup to `spawn_blocking()`

Link: https://bugzilla.proxmox.com/show_bug.cgi?id=6990


proxmox-backup:

Jakob Klocker (3):
  server: pull: run blocking file operations on the blocking pool
  fix #6990: server: drop verify state on push job
  fix #6990: server: drop verify state on non-decrypt pull job

 pbs-datastore/src/backup_info.rs | 21 +++++-----
 pbs-datastore/src/manifest.rs    | 50 ++++++++++++++++++++++-
 src/server/pull.rs               | 69 +++++++++++++++++++-------------
 src/server/push.rs               |  7 ++++
 4 files changed, 106 insertions(+), 41 deletions(-)


Summary over all repositories:
  4 files changed, 106 insertions(+), 41 deletions(-)

-- 
Generated by murpp 0.12.0