[pbs-devel] [PATCH proxmox{, -backup} 0/5] parallelize chunk reads in verification

[Nicolas Frey <n.frey@proxmox.com>]

This patch series aims to expand on Dominik's series [0] written for
pbs 3, parallelizing chunk reads in `VerifyWorker` using a seperate
thread pool from the verification.

The number of threads was previously hard-coded, but is now
configurable via the API and GUI with a new property called
`worker-threads`, similarly to tape backups.

The number of `worker-threads` only controls the thread pool for
reading, though if it makes sense to reuse this for the verification
pool, it could be adjusted to do so too.

In my local tests I measured the following speed difference:
verified a single snapshot with ~32 GiB (4x the RAM size) with 4 cores

1 thread:    ~440MiB/s
2 threads:   ~780MiB/s
4 threads:   ~1140MiB/s

[0] https://lore.proxmox.com/pbs-devel/20250707132706.2854973-1-d.csapak@proxmox.com/#t

proxmox:

Nicolas Frey (1):
  pbs-api-types: jobs: verify: add worker-threads to
    VerificationJobSetup

 pbs-api-types/src/jobs.rs | 10 ++++++++++
 1 file changed, 10 insertions(+)


proxmox-backup:

Nicolas Frey (4):
  api: verify: move chunk loading into parallel handler
  api: verify: use worker-threads to determine the number of threads to
    use
  api: verify: add worker-threads to update endpoint
  ui: verify: add option to set number of threads for job

 src/api2/admin/datastore.rs    |  13 +++-
 src/api2/backup/environment.rs |   2 +-
 src/api2/config/verify.rs      |   8 +++
 src/backup/verify.rs           | 123 +++++++++++++++++++++------------
 src/server/verify_job.rs       |   3 +-
 www/window/VerifyAll.js        |  12 ++++
 www/window/VerifyJobEdit.js    |  13 ++++
 7 files changed, 125 insertions(+), 49 deletions(-)


Summary over all repositories:
  8 files changed, 135 insertions(+), 49 deletions(-)

-- 
Generated by git-murpp 0.8.1

_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel