From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id DE9C51FF17E for ; Thu, 13 Nov 2025 10:30:39 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B436F1630C; Thu, 13 Nov 2025 10:31:34 +0100 (CET) From: Nicolas Frey To: pbs-devel@lists.proxmox.com Date: Thu, 13 Nov 2025 10:31:13 +0100 Message-ID: <20251113093118.195229-8-n.frey@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251113093118.195229-1-n.frey@proxmox.com> References: <20251113093118.195229-1-n.frey@proxmox.com> MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.098 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record Subject: [pbs-devel] [PATCH proxmox-backup v4 4/9] verify: move chunk loading into parallel handler X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" This way, the chunks will be loaded in parallel in addition to being checked in parallel. Depending on the underlying storage, this can speed up reading chunks from disk, especially when the underlying storage is IO depth dependent, and the CPU is faster than the storage. Originally-by: Dominik Csapak Signed-off-by: Nicolas Frey --- src/backup/verify.rs | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/src/backup/verify.rs b/src/backup/verify.rs index e1eda039..07d51dcf 100644 --- a/src/backup/verify.rs +++ b/src/backup/verify.rs @@ -177,6 +177,20 @@ impl VerifyWorker { .datastore .get_chunks_in_order(&*index, skip_chunk, check_abort)?; + let reader_pool = ParallelHandler::new("read chunks", 1, { + let decoder_pool = decoder_pool.channel(); + let verify_state = Arc::clone(&verify_state); + let backend = self.backend.clone(); + + move |info: ChunkReadInfo| { + Self::verify_chunk_by_backend( + &backend, + Arc::clone(&verify_state), + &decoder_pool, + &info, + ) + } + }); for (pos, _) in chunk_list { self.worker.check_abort()?; self.worker.fail_on_shutdown()?; @@ -188,15 +202,10 @@ impl VerifyWorker { continue; // already verified or marked corrupt } - Self::verify_chunk_by_backend( - &self.backend, - Arc::clone(&verify_state), - &decoder_pool.channel(), - &info, - )?; + reader_pool.send(info)?; } - decoder_pool.complete()?; + reader_pool.complete()?; let elapsed = verify_state.start_time.elapsed().as_secs_f64(); -- 2.47.3 _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel