From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id E05C467154 for ; Mon, 11 Jan 2021 17:42:46 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id D5D391D766 for ; Mon, 11 Jan 2021 17:42:46 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 0FB841D759 for ; Mon, 11 Jan 2021 17:42:45 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id D517E45F27; Mon, 11 Jan 2021 17:42:44 +0100 (CET) To: pve-devel@lists.proxmox.com References: <20210111111409.32385-1-s.reiter@proxmox.com> From: Stefan Reiter Message-ID: Date: Mon, 11 Jan 2021 17:42:43 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.039 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.001 Looks like a legit reply (A) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] [PATCH 00/11] live-restore for PBS snapshots X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2021 16:42:46 -0000 On 1/11/21 4:50 PM, aderumier@odiso.com wrote: > Hi, > that's great ! > > I'm not sure it's related, but do you have any plan to add support to > restore/extract files from the vm filesystem directly ? > if we could make some kind of qemu-nbd + mount loop of the backup > volume for example. > Unrelated, but yes, we're working on it. At the moment you can already map snapshots via loop devices using the map command, e.g.: proxmox-backup-client map vm/115/2020-10-13T15:03:12Z drive-scsi0.img # mapped as /dev/loopN proxmox-backup-client unmap ... What's planned is a fully featured file-restore, including a GUI in PVE (similar to what's already available for CTs in the PBS interface). ~ Stefan > > This is the main blocking feature for me, instead using my rbd > import/export for backups. > > Regards, > > Alexandre > > Le lundi 11 janvier 2021 à 12:13 +0100, Stefan Reiter a écrit : >> "live-restore" allows starting a VM immediately from a backup >> snapshot, no >> waiting for a long restore process. This is made possible with QEMU >> backing >> images, i.e. data is read from the backup which is attached to the VM >> as a >> drive, but new data is written to the destination, while a background >> process >> ('block-stream') copies over data in a linear fashion as well. >> >> QEMU backing images are normally only supported for qcow2 images, but >> since the >> destination always starts out empty, we can use a dirty bitmap to >> achieve the >> same effect - this is implemented as the 'alloc-track' driver in the >> 'qemu' part >> of the series. >> >> The Rust part of the equation is adjusted to provide (quiet a lot) >> more caching, >> as mixing random read/write from the guest with the linear reads from >> the >> background process (both of which may use read sizes smaller or >> bigger than a >> single chunk) would thrash performance without large buffers. >> >> I've marked the feature as 'experimental' in the GUI for now, as I'm >> sure there >> are a lot of edge cases I've missed to test, and there's also the >> possibility of >> data loss, since anything the VM writes during the restore is removed >> if the >> restore fails. >> >> >> qemu: Stefan Reiter (3): >>   PVE: explicitly add libuuid as linking dependency >>   PVE: block/pbs: fast-path reads without allocation if possible >>   block: add alloc-track driver >> >>  Makefile.objs       |   2 + >>  block/Makefile.objs |   1 + >>  block/alloc-track.c | 319 >> ++++++++++++++++++++++++++++++++++++++++++++ >>  block/pbs.c         |  17 ++- >>  4 files changed, 336 insertions(+), 3 deletions(-) >>  create mode 100644 block/alloc-track.c >> >> proxmox-backup: Stefan Reiter (1): >>   RemoteChunkReader: add LRU cached variant >> >>  src/bin/proxmox_backup_client/mount.rs |  4 +- >>  src/client/remote_chunk_reader.rs      | 77 ++++++++++++++++++++---- >> -- >>  2 files changed, 62 insertions(+), 19 deletions(-) >> >> proxmox-backup-qemu: Stefan Reiter (1): >>   access: use bigger cache and LRU chunk reader >> >>  src/restore.rs | 5 +++-- >>  1 file changed, 3 insertions(+), 2 deletions(-) >> >> qemu-server: Stefan Reiter (5): >>   make qemu_drive_mirror_monitor more generic >>   cfg2cmd: allow PBS snapshots as backing files for drives >>   enable live-restore for PBS >>   extract register_qmeventd_handle to QemuServer.pm >>   live-restore: register qmeventd handle >> >>  PVE/API2/Qemu.pm         |  15 ++- >>  PVE/QemuServer.pm        | 263 ++++++++++++++++++++++++++++++++----- >> -- >>  PVE/VZDump/QemuServer.pm |  32 +---- >>  3 files changed, 233 insertions(+), 77 deletions(-) >> >> manager: Stefan Reiter (1): >>   ui: restore: add live-restore checkbox >> >>  www/manager6/grid/BackupView.js    |  8 +++++-- >>  www/manager6/storage/BackupView.js |  7 ++++-- >>  www/manager6/window/Restore.js     | 38 >> +++++++++++++++++++++++++++++- >>  3 files changed, 48 insertions(+), 5 deletions(-) >> > > > > _______________________________________________ > pve-devel mailing list > pve-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel >