From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id D17BB1FF0BC for ; Thu, 15 Oct 2026 14:17:12 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 26D2421541; Thu, 15 Oct 2026 14:17:04 +0200 (CEST) From: Jakob Klocker To: pbs-devel@lists.proxmox.com Subject: [PATCH proxmox{,-backup} v4 0/5] fix #6990: server: drop verify state on push & pull job Date: Thu, 15 Oct 2026 14:16:54 +0200 Message-ID: <20261015121659.239652-1-j.klocker@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 1 AWL -0.666 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods RDNS_NONE 1.274 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record Message-ID-Hash: 7G4UECX3VMT34UWMC46YN6VQRFZ5JLAB X-Message-ID-Hash: 7G4UECX3VMT34UWMC46YN6VQRFZ5JLAB X-MailFrom: jklocker@dev.proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: When syncing a snapshot to another datastore, the source's verify_state is currently carried over to the target. This reports the target snapshot as verified even though its stored copy was never checked there. Since verify jobs skip snapshots that already carry a verify_state, the target's copy can't be verified again. Chunks are checksummed in memory while being transferred, but a verify serves a different purpose: confirming the write to the (external) target actually succeeded. This series stops the target from inheriting the source's verify_state, so the target is verified independently. On pull, a verify_state the target obtained on its own is preserved; on push the state is dropped unconditionally. It also moves the blocking calls in the touched code paths off the runtime's worker threads onto the blocking thread pool. Tested (verify_state correctly stripped on target): * Pull, verified source, new content, clean sync * Pull, verified source, existing content, clean sync * Pull, corrupted target, resync-corrupt * Push, verified source (has no corrupt path) * Pull, independently-verified target, clean re-sync - state preserved changes from v3 to v4 (thanks @Christian): * new patch for pbs-api-types: implement `Hash` and `Borrow` for `BackupArchiveName` * use `HashSet` instead of `HashSet` for the set of files to keep * split the `cleanup_unreferenced_files` / `expected_files` refactor into its own preparatory patch * reword the `expected_files` doc comment, as the client log is optional and not necessarily present changes from v2 to v3 (thanks @Christian): * offload remaining removes to `spawn_blocking()` * add `target_path` parameter to `write_to_path` and perform the rename inside the helper * add a helper to return expected files as hashset * refactor `cleanup_unreferenced_files` to take the hashset instead of the manifest changes from v1 to v2 (thanks @Christian): * move the manifest write helper onto `BackupManifest` * offload fsync, rename and cleanup to `spawn_blocking()` Link: https://bugzilla.proxmox.com/show_bug.cgi?id=6990 proxmox: Jakob Klocker (1): pbs-api-types: implement Hash and Borrow for BackupArchiveName pbs-api-types/src/datastore.rs | 14 ++++++++++++++ 1 file changed, 14 insertions(+) proxmox-backup: Jakob Klocker (4): server: pull: run blocking file operations on the blocking pool pbs-datastore: pass expected files to cleanup_unreferenced_files fix #6990: server: drop verify state on non-decrypt pull job fix #6990: server: drop verify state on push job pbs-datastore/src/backup_info.rs | 24 +++++------ pbs-datastore/src/manifest.rs | 51 ++++++++++++++++++++++- src/server/pull.rs | 69 +++++++++++++++++++------------- src/server/push.rs | 7 ++++ 4 files changed, 110 insertions(+), 41 deletions(-) Summary over all repositories: 5 files changed, 124 insertions(+), 41 deletions(-) -- Generated by murpp 0.12.0