From: Wolfgang Bumiller <w.bumiller@proxmox.com>
To: Andreas Rogge <andreas.rogge@bareos.com>
Cc: Proxmox VE development discussion <pve-devel@lists.proxmox.com>,
Thomas Lamprecht <t.lamprecht@proxmox.com>,
Philipp Storz <philipp.storz@bareos.com>
Subject: Re: [pve-devel] [PATCH storage v4 09/27] plugin: introduce new_backup_provider() method
Date: Thu, 3 Apr 2025 09:24:41 +0200 [thread overview]
Message-ID: <ey5nze36xsp7iaipishhlsvxo5xyllyvze2sevxutbcttofyye@bxzkiqxcm2no> (raw)
In-Reply-To: <c79787cf-4cc7-4ac6-9e8f-f864f77c1812@bareos.com>
On Wed, Apr 02, 2025 at 06:16:57PM +0200, Andreas Rogge wrote:
> Am 02.04.25 um 10:30 schrieb Wolfgang Bumiller:
> > On Tue, Apr 01, 2025 at 08:21:30PM +0200, Thomas Lamprecht wrote:
> > > > This sounds pretty inefficient - especially when
> > > > comparing with qmrestore's ability to just read read from stdin.
> >
> > The reading from stdin is quite limited, does not support sparse files
> > efficiently, and does not support our live-restore feature.
(PS: I may have had a different case in mind when thinking of sparse
files - the restore from stdin uses the VMA format after all)
> >
> > If we can *pull* data out-of-order from the backup provider via a better
> > protocol (like NBD which Thomas mentioned), holes in the disks don't
> > need to be transferred over the network, and we could probably support
> > live-restore, where the VMs immediately start running *during* the
> > restore process. (qemu would simply treat read-requests from the guest
> > which have not yet been restored with a higher priority while otherwise
> > continuing to copy the data in-order in the background)
> Neither pulling nor out-of-order is an option in the Bareos architecture.
Fair enough.
While this is not currently done, I'd still like to know: what about
*backing up* out of order? PVE's original "VMA" format was specifically
designed to allow what is essentially a "copy-before-write" style backup
while a VM is running.
The current provider-API uses a fleecing-approach to support running VMS
while the provider plugin can choose to perform backups in its desired
order.
But I do wonder if - to reduce space-requirements for backing up running
VMs - at some point we might also add the ability for qemu to provide
some kind of queue containing the offset-length pairs of blocks which
have been stored in the temporary fleecing image. The provider could
consume this queue to keep the temporary storage at a minimum by doing
out-of-order backups.
This only makes sense if there are backup systems which can benefit from
it. (And it should be a simple-enough api extension to add this in the
future, from what I can tell)
>
> > Some alternatives btw. are providing a fuse or ublk device on the PVE
> > side which pulls data from bareos (or to which bareos can push data
> > which, like qemu's "fleecing" mode, could store the not-yet restored
> > portions in temporary files, discarding them as they are read by/moved
> > to qemu).
> That's basically like staging the image just that you can start reading
> before we finish writing. I'll keep it in mind, even tough I don't think it
> is really feasible.
>
> > *Technically* we could have a mode where we allocate the disks and "map"
> > them onto the system (potentially via nbd, or `rbd map` for ceph etc.)
> Yes, please do.
I guess it makes sense for us to not expect/require random access, as
any feature like that already imposes limitations on how the data can be
stored. I'd expect different backup solutions to have different
limitations in that regard.
The API design in this series (or at least the current version of it)
wouldn't prevent pure sequential restores, just not make them "easy"
right now, but that's something we should be able to accommodate with
another mechanism.
I *believe* `qemu-nbd` should be able to bind all the storage types we
want to restore to to /dev/nbdXY devices, which would give the provider
a bunch of block devices to write to in whichever way they want to, so
the provider would then only need to figure out how to receive the data
and forward that to the devices.
We'll need to try.
> >
> > *But* it would make live-restore impossible with that plugin.
> There will be no live-restore with Bareos.
> If we would ever consider implementing this, on the Bareos side it would be
> pretty complicated and full of limitations. In that case we would probably
> just implement yet another plugin for PVE.
Fair enough.
>
> > Which is why the most flexible thing to do is to use a `qemu-img` call
> > and giving it the paths, or more precisely, the URLs to the disks.
>
> I understand how this makes sense. However, if you don't have the data in a
> format that qemu-img can consume, things become complicated.
It can also just read data from a stream (but without any smarter
protocol, this would make holes/unallocated blocks extremely
inefficient), where the provider would create that stream in whichever
way they want to.
(Although I'm not actually sure the current invocation can deal with a
stream - we do have patches on qemu-img's `dd` subcommand to
specifically allow this, though...)
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
next prev parent reply other threads:[~2025-04-03 7:25 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-11-14 15:07 [pve-devel] [PATCH-SERIES qemu/common/storage/qemu-server/container/manager v4 00/27] backup provider API Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu v4 01/27] PVE backup: add target ID in backup state Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu v4 02/27] PVE backup: get device info: allow caller to specify filter for which devices use fleecing Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu v4 03/27] PVE backup: implement backup access setup and teardown API for external providers Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu v4 04/27] PVE backup: implement bitmap support for external backup access Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH common v4 05/27] test: lock file: get rid of END block that made test always pass Fiona Ebner
2024-11-14 19:46 ` [pve-devel] applied: " Thomas Lamprecht
2024-11-14 15:07 ` [pve-devel] [PATCH common v4 06/27] tools: run fork: allow running code in parent after fork Fiona Ebner
2024-11-14 19:46 ` [pve-devel] applied: " Thomas Lamprecht
2024-11-14 15:07 ` [pve-devel] [PATCH common v4 07/27] test: have lock file test use run_fork() helper Fiona Ebner
2024-11-14 19:46 ` [pve-devel] applied: " Thomas Lamprecht
2024-11-14 15:07 ` [pve-devel] [PATCH storage v4 08/27] add storage_has_feature() helper function Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH storage v4 09/27] plugin: introduce new_backup_provider() method Fiona Ebner
2025-04-01 16:02 ` Andreas Rogge
2025-04-01 18:21 ` Thomas Lamprecht
2025-04-02 8:30 ` Wolfgang Bumiller
2025-04-02 16:16 ` Andreas Rogge
2025-04-03 7:24 ` Wolfgang Bumiller [this message]
2025-04-03 16:08 ` Andreas Rogge
2025-04-04 7:15 ` Fabian Grünbichler
2025-04-02 8:33 ` Fiona Ebner
2025-04-03 8:06 ` Andreas Rogge
2025-04-02 16:01 ` Andreas Rogge
2024-11-14 15:07 ` [pve-devel] [PATCH storage v4 10/27] extract backup config: delegate to backup provider for storages that support it Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [POC storage v4 11/27] add backup provider example Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [POC storage v4 12/27] WIP Borg plugin Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu-server v4 13/27] backup: keep track of block-node size for fleecing Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu-server v4 14/27] module config: load nbd module at boot Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu-server v4 15/27] backup: allow adding fleecing images also for EFI and TPM Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu-server v4 16/27] backup: implement backup for external providers Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu-server v4 17/27] backup: implement restore " Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu-server v4 18/27] backup restore: external: hardening check for untrusted source image Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 19/27] add LXC::Namespaces module Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 20/27] backup: implement backup for external providers Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 21/27] backup: implement restore " Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 22/27] external restore: don't use 'one-file-system' tar flag when restoring from a directory Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 23/27] create: factor out compression option helper Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 24/27] restore tar archive: check potentially untrusted archive Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 25/27] api: add early check against restoring privileged container from external source Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH manager v4 26/27] ui: backup: also check for backup subtype to classify archive Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH manager v4 27/27] backup: implement backup for external providers Fiona Ebner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ey5nze36xsp7iaipishhlsvxo5xyllyvze2sevxutbcttofyye@bxzkiqxcm2no \
--to=w.bumiller@proxmox.com \
--cc=andreas.rogge@bareos.com \
--cc=philipp.storz@bareos.com \
--cc=pve-devel@lists.proxmox.com \
--cc=t.lamprecht@proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal