Re: [pve-devel] [PATCH 00/11] live-restore for PBS snapshots

[aderumier@odiso.com]

>>What's planned is a fully featured file-restore, including a GUI in
PVE 
>>(similar to what's already available for CTs in the PBS interface).

Wonderful ! 

Thanks about the "proxmox-backup-client map", I'll try it.

Le lundi 11 janvier 2021 à 17:42 +0100, Stefan Reiter a écrit :
> On 1/11/21 4:50 PM, aderumier@odiso.com wrote:
> > Hi,
> > that's great !
> > 
> > I'm not sure it's related, but do you have any plan to add support
> > to
> > restore/extract files from the vm filesystem directly ?
> > if we could make some kind of qemu-nbd + mount loop of the backup
> > volume for example.
> > 
> 
> Unrelated, but yes, we're working on it.
> 
> At the moment you can already map snapshots via loop devices using
> the 
> map command, e.g.:
> proxmox-backup-client map vm/115/2020-10-13T15:03:12Z drive-scsi0.img
> # mapped as /dev/loopN
> proxmox-backup-client unmap ...
> 
> What's planned is a fully featured file-restore, including a GUI in
> PVE 
> (similar to what's already available for CTs in the PBS interface).
> 
> ~ Stefan
> 
> > 
> > This is the main blocking feature for me, instead using my rbd
> > import/export for backups.
> > 
> > Regards,
> > 
> > Alexandre
> >  > Le lundi 11 janvier 2021 à 12:13 +0100, Stefan Reiter a écrit :
> > > "live-restore" allows starting a VM immediately from a backup
> > > snapshot, no
> > > waiting for a long restore process. This is made possible with
> > > QEMU
> > > backing
> > > images, i.e. data is read from the backup which is attached to
> > > the VM
> > > as a
> > > drive, but new data is written to the destination, while a
> > > background
> > > process
> > > ('block-stream') copies over data in a linear fashion as well.
> > > 
> > > QEMU backing images are normally only supported for qcow2 images,
> > > but
> > > since the
> > > destination always starts out empty, we can use a dirty bitmap to
> > > achieve the
> > > same effect - this is implemented as the 'alloc-track' driver in
> > > the
> > > 'qemu' part
> > > of the series.
> > > 
> > > The Rust part of the equation is adjusted to provide (quiet a
> > > lot)
> > > more caching,
> > > as mixing random read/write from the guest with the linear reads
> > > from
> > > the
> > > background process (both of which may use read sizes smaller or
> > > bigger than a
> > > single chunk) would thrash performance without large buffers.
> > > 
> > > I've marked the feature as 'experimental' in the GUI for now, as
> > > I'm
> > > sure there
> > > are a lot of edge cases I've missed to test, and there's also the
> > > possibility of
> > > data loss, since anything the VM writes during the restore is
> > > removed
> > > if the
> > > restore fails.
> > > 
> > > 
> > > qemu: Stefan Reiter (3):
> > >    PVE: explicitly add libuuid as linking dependency
> > >    PVE: block/pbs: fast-path reads without allocation if possible
> > >    block: add alloc-track driver
> > > 
> > >   Makefile.objs       |   2 +
> > >   block/Makefile.objs |   1 +
> > >   block/alloc-track.c | 319
> > > ++++++++++++++++++++++++++++++++++++++++++++
> > >   block/pbs.c         |  17 ++-
> > >   4 files changed, 336 insertions(+), 3 deletions(-)
> > >   create mode 100644 block/alloc-track.c
> > > 
> > > proxmox-backup: Stefan Reiter (1):
> > >    RemoteChunkReader: add LRU cached variant
> > > 
> > >   src/bin/proxmox_backup_client/mount.rs |  4 +-
> > >   src/client/remote_chunk_reader.rs      | 77
> > > ++++++++++++++++++++----
> > > --
> > >   2 files changed, 62 insertions(+), 19 deletions(-)
> > > 
> > > proxmox-backup-qemu: Stefan Reiter (1):
> > >    access: use bigger cache and LRU chunk reader
> > > 
> > >   src/restore.rs | 5 +++--
> > >   1 file changed, 3 insertions(+), 2 deletions(-)
> > > 
> > > qemu-server: Stefan Reiter (5):
> > >    make qemu_drive_mirror_monitor more generic
> > >    cfg2cmd: allow PBS snapshots as backing files for drives
> > >    enable live-restore for PBS
> > >    extract register_qmeventd_handle to QemuServer.pm
> > >    live-restore: register qmeventd handle
> > > 
> > >   PVE/API2/Qemu.pm         |  15 ++-
> > >   PVE/QemuServer.pm        | 263
> > > ++++++++++++++++++++++++++++++++-----
> > > --
> > >   PVE/VZDump/QemuServer.pm |  32 +----
> > >   3 files changed, 233 insertions(+), 77 deletions(-)
> > > 
> > > manager: Stefan Reiter (1):
> > >    ui: restore: add live-restore checkbox
> > > 
> > >   www/manager6/grid/BackupView.js    |  8 +++++--
> > >   www/manager6/storage/BackupView.js |  7 ++++--
> > >   www/manager6/window/Restore.js     | 38
> > > +++++++++++++++++++++++++++++-
> > >   3 files changed, 48 insertions(+), 5 deletions(-)
> > > 
> > 
> > 
> > 
> > _______________________________________________
> > pve-devel mailing list
> > pve-devel@lists.proxmox.com
> > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> > 
>