all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Christian Ebner <c.ebner@proxmox.com>
To: Proxmox Backup Server development discussion
	<pbs-devel@lists.proxmox.com>, Nicolas Frey <n.frey@proxmox.com>
Subject: Re: [pbs-devel] [PATCH proxmox{, -backup} 0/5] parallelize chunk reads in verification
Date: Thu, 6 Nov 2025 09:02:07 +0100	[thread overview]
Message-ID: <3578656e-d487-4e05-88fb-d63554469cf0@proxmox.com> (raw)
In-Reply-To: <20251105155129.517430-1-n.frey@proxmox.com>

On 11/5/25 4:51 PM, Nicolas Frey wrote:
> This patch series aims to expand on Dominik's series [0] written for
> pbs 3, parallelizing chunk reads in `VerifyWorker` using a seperate
> thread pool from the verification.
> 
> The number of threads was previously hard-coded, but is now
> configurable via the API and GUI with a new property called
> `worker-threads`, similarly to tape backups.
> 
> The number of `worker-threads` only controls the thread pool for
> reading, though if it makes sense to reuse this for the verification
> pool, it could be adjusted to do so too.
> 
> In my local tests I measured the following speed difference:
> verified a single snapshot with ~32 GiB (4x the RAM size) with 4 cores
> 
> 1 thread:    ~440MiB/s
> 2 threads:   ~780MiB/s
> 4 threads:   ~1140MiB/s
> 
> [0] https://lore.proxmox.com/pbs-devel/20250707132706.2854973-1-d.csapak@proxmox.com/#t

Thanks for the patches, nice work!

I will go trough and comment on individual patches, but one thing upfront:

Could you also include a concise description of what the additional 
parameter for the verification job config controlles to the docs? So we 
can point users directly to that.


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


  parent reply	other threads:[~2025-11-06  8:02 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-11-05 15:51 Nicolas Frey
2025-11-05 15:51 ` [pbs-devel] [PATCH proxmox 1/1] pbs-api-types: jobs: verify: add worker-threads to VerificationJobConfig Nicolas Frey
2025-11-06  8:14   ` Christian Ebner
2025-11-05 15:51 ` [pbs-devel] [PATCH proxmox-backup 1/4] api: verify: move chunk loading into parallel handler Nicolas Frey
2025-11-06  8:54   ` Christian Ebner
2025-11-06  9:04     ` Nicolas Frey
2025-11-06  9:26       ` Christian Ebner
2025-11-05 15:51 ` [pbs-devel] [PATCH proxmox-backup 2/4] api: verify: use worker-threads to determine the number of threads to use Nicolas Frey
2025-11-06  9:09   ` Christian Ebner
2025-11-06  9:23     ` Nicolas Frey
2025-11-06  9:32       ` Christian Ebner
2025-11-06 11:22         ` Nicolas Frey
2025-11-06 11:57           ` Christian Ebner
2025-11-05 15:51 ` [pbs-devel] [PATCH proxmox-backup 3/4] api: verify: add worker-threads to update endpoint Nicolas Frey
2025-11-06  9:13   ` Christian Ebner
2025-11-05 15:51 ` [pbs-devel] [PATCH proxmox-backup 4/4] ui: verify: add option to set number of threads for job Nicolas Frey
2025-11-06  9:22   ` Christian Ebner
2025-11-06  9:25     ` Nicolas Frey
2025-11-06  8:02 ` Christian Ebner [this message]
2025-11-06 16:15 ` [pbs-devel] superseded: [PATCH proxmox{, -backup} 0/5] parallelize chunk reads in verification Nicolas Frey

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3578656e-d487-4e05-88fb-d63554469cf0@proxmox.com \
    --to=c.ebner@proxmox.com \
    --cc=n.frey@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal