public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: applied: [PATCH] PVE backup: vma reader: merge contiguous writes on restore
Date: Thu, 12 Nov 2026 23:35:59 +0100	[thread overview]
Message-ID: <2abe2bb3-30f5-4105-80e2-4b420802f1c0@proxmox.com> (raw)
In-Reply-To: <20261109160210.3380582-1-t.lamprecht@proxmox.com>

Am 09.11.26 um 17:02 schrieb Thomas Lamprecht:
> Restoring wrote every 64 KiB cluster, or every 4 KiB block of a
> cluster with zero blocks, as a separate synchronous request. With the
> page cache this mostly costs extra system calls, as writeback merges the
> data again there. Without page cache – like cache=none that qemu-server
> uses for restores to ZFS pools – each request goes to the storage on its
> own and has to complete before vma continues reading the archive.
> 
> To improve this, queue writes that continue the previous one on the same
> target and issue them as a single request once the target changes, a gap
> occurs, or the extent is done, as the queued data points into the extent
> buffer. The vma writer puts up to 58 of the 64 KiB clusters into an
> extent, so a request can now cover about 3.6 MiB. Writes to each target
> stay synchronous and in order, and a successful restore produces the
> same data.
> 
> One small drawback: QEMU admits a throttled request as a whole and
> delays the next one, so a bandwidth limit now takes effect in steps of
> one such request instead of one cluster. Over a longer restore the
> average rate stays the same, but it can be "burstier" if the limit is
> less than ~30 Mbps (one 3.6 MiB batched write submitted). Might be
> improvable, but not sure if its really worth the complexity.
> 
> A simple benchmark of restoring a 3 GiB image from a zstd-compressed
> archive like qemu-server does (inside a VM with ext4 on a virtual disk)
> took on average 45s instead of 66s for a sparse target and 38s instead
> of 83s when also writing zeroes, both with cache=none; so a rather
> noticable improvement. With the page cache, the before/after times
> stayed within the measurement noise.
> 
> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
> ---
> 
> Patch on top of the current pve downstream series. This fell out of
> reviewing the impact that Elias noticed in Markus' "{pbs-,vma-}restore:
> disable caching for LVM-thin" series [0], where vma restore performed
> worse after over PBS restore.
> 
> [0]: https://lore.proxmox.com/pve-devel/20260908123727.285848-1-m.frank@proxmox.com/
> 
>  vma-reader.c | 77 +++++++++++++++++++++++++++++++++++++++++++++++-----
>  1 file changed, 70 insertions(+), 7 deletions(-)
> 
>

applied this one 




  reply	other threads:[~2026-11-12 22:36 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-09 16:02 [PATCH] PVE backup: vma reader: merge contiguous writes on restore Thomas Lamprecht
2026-11-12 22:35 ` Thomas Lamprecht [this message]
2026-11-13 11:46 ` Elias Huhsovitz

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2abe2bb3-30f5-4105-80e2-4b420802f1c0@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal