From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <s.reiter@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id E05C467154
for <pve-devel@lists.proxmox.com>; Mon, 11 Jan 2021 17:42:46 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id D5D391D766
for <pve-devel@lists.proxmox.com>; Mon, 11 Jan 2021 17:42:46 +0100 (CET)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[212.186.127.180])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS id 0FB841D759
for <pve-devel@lists.proxmox.com>; Mon, 11 Jan 2021 17:42:45 +0100 (CET)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id D517E45F27;
Mon, 11 Jan 2021 17:42:44 +0100 (CET)
To: pve-devel@lists.proxmox.com
References: <20210111111409.32385-1-s.reiter@proxmox.com>
<a91ee7b1e8094581fefa180f02b40a47f14eb722.camel@odiso.com>
From: Stefan Reiter <s.reiter@proxmox.com>
Message-ID: <ab4c7373-0c5f-fdf0-fcd1-f46c57422aa1@proxmox.com>
Date: Mon, 11 Jan 2021 17:42:43 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <a91ee7b1e8094581fefa180f02b40a47f14eb722.camel@odiso.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.039 Adjusted score from AWL reputation of From: address
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
NICE_REPLY_A -0.001 Looks like a legit reply (A)
RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/,
medium trust
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: Re: [pve-devel] [PATCH 00/11] live-restore for PBS snapshots
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Mon, 11 Jan 2021 16:42:46 -0000
On 1/11/21 4:50 PM, aderumier@odiso.com wrote:
> Hi,
> that's great !
>
> I'm not sure it's related, but do you have any plan to add support to
> restore/extract files from the vm filesystem directly ?
> if we could make some kind of qemu-nbd + mount loop of the backup
> volume for example.
>
Unrelated, but yes, we're working on it.
At the moment you can already map snapshots via loop devices using the
map command, e.g.:
proxmox-backup-client map vm/115/2020-10-13T15:03:12Z drive-scsi0.img
# mapped as /dev/loopN
proxmox-backup-client unmap ...
What's planned is a fully featured file-restore, including a GUI in PVE
(similar to what's already available for CTs in the PBS interface).
~ Stefan
>
> This is the main blocking feature for me, instead using my rbd
> import/export for backups.
>
> Regards,
>
> Alexandre
> > Le lundi 11 janvier 2021 à 12:13 +0100, Stefan Reiter a écrit :
>> "live-restore" allows starting a VM immediately from a backup
>> snapshot, no
>> waiting for a long restore process. This is made possible with QEMU
>> backing
>> images, i.e. data is read from the backup which is attached to the VM
>> as a
>> drive, but new data is written to the destination, while a background
>> process
>> ('block-stream') copies over data in a linear fashion as well.
>>
>> QEMU backing images are normally only supported for qcow2 images, but
>> since the
>> destination always starts out empty, we can use a dirty bitmap to
>> achieve the
>> same effect - this is implemented as the 'alloc-track' driver in the
>> 'qemu' part
>> of the series.
>>
>> The Rust part of the equation is adjusted to provide (quiet a lot)
>> more caching,
>> as mixing random read/write from the guest with the linear reads from
>> the
>> background process (both of which may use read sizes smaller or
>> bigger than a
>> single chunk) would thrash performance without large buffers.
>>
>> I've marked the feature as 'experimental' in the GUI for now, as I'm
>> sure there
>> are a lot of edge cases I've missed to test, and there's also the
>> possibility of
>> data loss, since anything the VM writes during the restore is removed
>> if the
>> restore fails.
>>
>>
>> qemu: Stefan Reiter (3):
>> PVE: explicitly add libuuid as linking dependency
>> PVE: block/pbs: fast-path reads without allocation if possible
>> block: add alloc-track driver
>>
>> Makefile.objs | 2 +
>> block/Makefile.objs | 1 +
>> block/alloc-track.c | 319
>> ++++++++++++++++++++++++++++++++++++++++++++
>> block/pbs.c | 17 ++-
>> 4 files changed, 336 insertions(+), 3 deletions(-)
>> create mode 100644 block/alloc-track.c
>>
>> proxmox-backup: Stefan Reiter (1):
>> RemoteChunkReader: add LRU cached variant
>>
>> src/bin/proxmox_backup_client/mount.rs | 4 +-
>> src/client/remote_chunk_reader.rs | 77 ++++++++++++++++++++----
>> --
>> 2 files changed, 62 insertions(+), 19 deletions(-)
>>
>> proxmox-backup-qemu: Stefan Reiter (1):
>> access: use bigger cache and LRU chunk reader
>>
>> src/restore.rs | 5 +++--
>> 1 file changed, 3 insertions(+), 2 deletions(-)
>>
>> qemu-server: Stefan Reiter (5):
>> make qemu_drive_mirror_monitor more generic
>> cfg2cmd: allow PBS snapshots as backing files for drives
>> enable live-restore for PBS
>> extract register_qmeventd_handle to QemuServer.pm
>> live-restore: register qmeventd handle
>>
>> PVE/API2/Qemu.pm | 15 ++-
>> PVE/QemuServer.pm | 263 ++++++++++++++++++++++++++++++++-----
>> --
>> PVE/VZDump/QemuServer.pm | 32 +----
>> 3 files changed, 233 insertions(+), 77 deletions(-)
>>
>> manager: Stefan Reiter (1):
>> ui: restore: add live-restore checkbox
>>
>> www/manager6/grid/BackupView.js | 8 +++++--
>> www/manager6/storage/BackupView.js | 7 ++++--
>> www/manager6/window/Restore.js | 38
>> +++++++++++++++++++++++++++++-
>> 3 files changed, 48 insertions(+), 5 deletions(-)
>>
>
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>