Re: [pve-devel] [PATCH storage v4 09/27] plugin: introduce new_backup_provider() method

From: Andreas Rogge <andreas.rogge@bareos.com>
To: Wolfgang Bumiller <w.bumiller@proxmox.com>,
	Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Proxmox VE development discussion <pve-devel@lists.proxmox.com>,
	Philipp Storz <philipp.storz@bareos.com>
Subject: Re: [pve-devel] [PATCH storage v4 09/27] plugin: introduce new_backup_provider() method
Date: Wed, 2 Apr 2025 18:16:57 +0200	[thread overview]
Message-ID: <c79787cf-4cc7-4ac6-9e8f-f864f77c1812@bareos.com> (raw)
In-Reply-To: <dktagxdpeqlvvmqhv2hr5n3dxmdh7nfyctluo64mwqgsimread@jbjcwstbk5fn>

Am 02.04.25 um 10:30 schrieb Wolfgang Bumiller:
> On Tue, Apr 01, 2025 at 08:21:30PM +0200, Thomas Lamprecht wrote:
>>> This sounds pretty inefficient - especially when
>>> comparing with qmrestore's ability to just read read from stdin.
> 
> The reading from stdin is quite limited, does not support sparse files
> efficiently, and does not support our live-restore feature.
> 
> If we can *pull* data out-of-order from the backup provider via a better
> protocol (like NBD which Thomas mentioned), holes in the disks  don't
> need to be transferred over the network, and we could probably support
> live-restore, where the VMs immediately start running *during* the
> restore process. (qemu would simply treat read-requests from the guest
> which have not yet been restored with a higher priority while otherwise
> continuing to copy the data in-order in the background)
Neither pulling nor out-of-order is an option in the Bareos architecture.

> Some alternatives btw. are providing a fuse or ublk device on the PVE
> side which pulls data from bareos (or to which bareos can push data
> which, like qemu's "fleecing" mode, could store the not-yet restored
> portions in temporary files, discarding them as they are read by/moved
> to qemu).
That's basically like staging the image just that you can start reading 
before we finish writing. I'll keep it in mind, even tough I don't think 
it is really feasible.

> *Technically* we could have a mode where we allocate the disks and "map"
> them onto the system (potentially via nbd, or `rbd map` for ceph etc.)
Yes, please do.
> 
> *But* it would make live-restore impossible with that plugin.
There will be no live-restore with Bareos.
If we would ever consider implementing this, on the Bareos side it would 
be pretty complicated and full of limitations. In that case we would 
probably just implement yet another plugin for PVE.

> Which is why the most flexible thing to do is to use a `qemu-img` call
> and giving it the paths, or more precisely, the URLs to the disks.

I understand how this makes sense. However, if you don't have the data 
in a format that qemu-img can consume, things become complicated.

Best Regards,
Andreas

-- 
Andreas Rogge                             andreas.rogge@bareos.com
   Bareos GmbH & Co. KG                      Phone: +49 221-630693-86
   http://www.bareos.com

   Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
   Komplementär: Bareos Verwaltungs-GmbH
   Geschäftsführer: Stephan Dühr, Jörg Steffens, Philipp Storz

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel