all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Dominik Csapak <d.csapak@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: Re: [pbs-devel] [PATCH proxmox-backup 2/2] tape: use datastores 'read-thread' for tape backup
Date: Tue, 30 Apr 2024 13:00:41 +0200	[thread overview]
Message-ID: <14667c59-bea8-43a3-998c-a13a7ae31927@proxmox.com> (raw)
In-Reply-To: <20240430093939.1318786-2-d.csapak@proxmox.com>

On 4/30/24 11:39, Dominik Csapak wrote:
> using a single thread for reading is not optimal in some cases, e.g.
> when the underlying storage can handle more reads in parallel than
> with a single thread.
> 
> This depends largely on the storage and cpu.
> 
> We use the ParallelHandler to handle the actual reads.
> Make the sync_channel buffer size depending on the number of threads
> so we have space for two chunks per thread.
> 
> Did some benchmarks on my (virtual) pbs with a real tape drive (lto8
> tape in an lto9 drive):
> 
> For my NVME datastore it did not matter much how many threads were used
> so i guess the bottleneck was either in the hba/drive or cable rather
> than the disks/cpu. (Always got around ~300MB/s from the task log)
> 
> For a datastore on a single HDD, the results are much more interesting:
> 
> 1 Thread:  ~55MB/s
> 2 Threads: ~70MB/s
> 4 Threads: ~80MB/s
> 8 Threads: ~95MB/s
> 
> So the fact that multiple IO request are done in parallel does speed up
> the tape backup in general.
> 

eh that sentence might be misleading, what i meant was not 'in general'
but for the case of spinning disks

could be amended before applying or in a v2


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


  reply	other threads:[~2024-04-30 11:01 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-04-30  9:39 [pbs-devel] [PATCH proxmox-backup 1/2] datastore: add tuning options for the number of reading threads Dominik Csapak
2024-04-30  9:39 ` [pbs-devel] [PATCH proxmox-backup 2/2] tape: use datastores 'read-thread' for tape backup Dominik Csapak
2024-04-30 11:00   ` Dominik Csapak [this message]
2024-05-02 12:22 ` [pbs-devel] [PATCH proxmox-backup 1/2] datastore: add tuning options for the number of reading threads Thomas Lamprecht
2024-05-02 13:14   ` Dominik Csapak

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=14667c59-bea8-43a3-998c-a13a7ae31927@proxmox.com \
    --to=d.csapak@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal