From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id E8343B99E5 for ; Fri, 15 Mar 2024 11:34:53 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id C4F6B1A553 for ; Fri, 15 Mar 2024 11:34:53 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Fri, 15 Mar 2024 11:34:52 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 1FAD748A1D for ; Fri, 15 Mar 2024 11:25:10 +0100 (CET) From: Fiona Ebner To: pve-devel@lists.proxmox.com Date: Fri, 15 Mar 2024 11:25:02 +0100 Message-Id: <20240315102502.84163-22-f.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240315102502.84163-1-f.ebner@proxmox.com> References: <20240315102502.84163-1-f.ebner@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.069 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: [pve-devel] [PATCH docs v2 21/21] vzdump: add section about backup fleecing X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2024 10:34:54 -0000 Signed-off-by: Fiona Ebner --- Changes in v2: * flesh out recommendations much more * adapt to changes in backend (i.e. using qcow2 if available, no storage default) vzdump.adoc | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/vzdump.adoc b/vzdump.adoc index b5bbac7..b01d3c5 100644 --- a/vzdump.adoc +++ b/vzdump.adoc @@ -136,6 +136,44 @@ not included in backups. For volume mount points you can set the *Backup* option to include the mount point in the backup. Device and bind mounts are never backed up as their content is managed outside the {pve} storage library. +VM Backup Fleecing +~~~~~~~~~~~~~~~~~~ + +WARNING: Backup fleecing is still being worked on (also in upstream QEMU) and is +currently only a technology preview. + +When a backup for a VM is started, QEMU will install a "copy-before-write" +filter in its block layer. This filter ensures that upon new guest writes, old +data still needed for the backup is sent to the backup target first. The guest +write blocks until this operation is finished so guest IO to not-yet-backed-up +sectors will be limited by the speed of the backup target. + +With backup fleecing, such old data is cached in a fleecing image rather than +sent directly to the backup target. This can help guest IO performance and even +prevent hangs in certain scenarios, at the cost of requiring more storage space. +Use e.g. `vzdump 123 --fleecing enabled=1,storage=local-lvm` to enable backup +fleecing, with fleecing images created on the storage `local-lvm`. + +The fleecing storage should be a fast local storage, with thin provisioning and +discard support. Examples are LVM-thin, RBD, ZFS with `sparse 1` in the storage +configuration, many file-based storages. Ideally, the fleecing storage is a +dedicated storage, so it running full will not affect other guests and just fail +the backup. Parts of the fleecing image that have been backed up will be +discarded to try and keep the space usage low. + +For file-based storages that do not support discard (e.g. NFS before version +4.2), you should set `preallocation off` in the storage configuration. In +combination with `qcow2` (used automatically as the format for the fleecing +image when the storage supports it), this has the advantage that already +allocated parts of the image can be re-used later, which can still help save +quite a bit of space. + +WARNING: On a storage that's not thinly provisioned, e.g. LVM or ZFS without the +`sparse` option, the full size of the original disk needs to be reserved for the +fleecing image up-front. On a thinly provisioned storage, the fleecing image can +grow to the same size as the original image only if the guest re-writes a whole +disk while the backup is busy with another disk. + Backup File Names ----------------- -- 2.39.2