From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 902791FF183 for ; Wed, 5 Nov 2025 16:51:22 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id D2B3B2299; Wed, 5 Nov 2025 16:52:02 +0100 (CET) From: Nicolas Frey To: pbs-devel@lists.proxmox.com Date: Wed, 5 Nov 2025 16:51:24 +0100 Message-ID: <20251105155129.517430-1-n.frey@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.108 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com, jobs.rs, environment.rs, datastore.rs, verify.rs] Subject: [pbs-devel] [PATCH proxmox{, -backup} 0/5] parallelize chunk reads in verification X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" This patch series aims to expand on Dominik's series [0] written for pbs 3, parallelizing chunk reads in `VerifyWorker` using a seperate thread pool from the verification. The number of threads was previously hard-coded, but is now configurable via the API and GUI with a new property called `worker-threads`, similarly to tape backups. The number of `worker-threads` only controls the thread pool for reading, though if it makes sense to reuse this for the verification pool, it could be adjusted to do so too. In my local tests I measured the following speed difference: verified a single snapshot with ~32 GiB (4x the RAM size) with 4 cores 1 thread: ~440MiB/s 2 threads: ~780MiB/s 4 threads: ~1140MiB/s [0] https://lore.proxmox.com/pbs-devel/20250707132706.2854973-1-d.csapak@proxmox.com/#t proxmox: Nicolas Frey (1): pbs-api-types: jobs: verify: add worker-threads to VerificationJobSetup pbs-api-types/src/jobs.rs | 10 ++++++++++ 1 file changed, 10 insertions(+) proxmox-backup: Nicolas Frey (4): api: verify: move chunk loading into parallel handler api: verify: use worker-threads to determine the number of threads to use api: verify: add worker-threads to update endpoint ui: verify: add option to set number of threads for job src/api2/admin/datastore.rs | 13 +++- src/api2/backup/environment.rs | 2 +- src/api2/config/verify.rs | 8 +++ src/backup/verify.rs | 123 +++++++++++++++++++++------------ src/server/verify_job.rs | 3 +- www/window/VerifyAll.js | 12 ++++ www/window/VerifyJobEdit.js | 13 ++++ 7 files changed, 125 insertions(+), 49 deletions(-) Summary over all repositories: 8 files changed, 135 insertions(+), 49 deletions(-) -- Generated by git-murpp 0.8.1 _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel