From: Fiona Ebner <f.ebner@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH docs v2 21/21] vzdump: add section about backup fleecing
Date: Fri, 15 Mar 2024 11:25:02 +0100 [thread overview]
Message-ID: <20240315102502.84163-22-f.ebner@proxmox.com> (raw)
In-Reply-To: <20240315102502.84163-1-f.ebner@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
Changes in v2:
* flesh out recommendations much more
* adapt to changes in backend (i.e. using qcow2 if available, no
storage default)
vzdump.adoc | 38 ++++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)
diff --git a/vzdump.adoc b/vzdump.adoc
index b5bbac7..b01d3c5 100644
--- a/vzdump.adoc
+++ b/vzdump.adoc
@@ -136,6 +136,44 @@ not included in backups. For volume mount points you can set the *Backup* option
to include the mount point in the backup. Device and bind mounts are never
backed up as their content is managed outside the {pve} storage library.
+VM Backup Fleecing
+~~~~~~~~~~~~~~~~~~
+
+WARNING: Backup fleecing is still being worked on (also in upstream QEMU) and is
+currently only a technology preview.
+
+When a backup for a VM is started, QEMU will install a "copy-before-write"
+filter in its block layer. This filter ensures that upon new guest writes, old
+data still needed for the backup is sent to the backup target first. The guest
+write blocks until this operation is finished so guest IO to not-yet-backed-up
+sectors will be limited by the speed of the backup target.
+
+With backup fleecing, such old data is cached in a fleecing image rather than
+sent directly to the backup target. This can help guest IO performance and even
+prevent hangs in certain scenarios, at the cost of requiring more storage space.
+Use e.g. `vzdump 123 --fleecing enabled=1,storage=local-lvm` to enable backup
+fleecing, with fleecing images created on the storage `local-lvm`.
+
+The fleecing storage should be a fast local storage, with thin provisioning and
+discard support. Examples are LVM-thin, RBD, ZFS with `sparse 1` in the storage
+configuration, many file-based storages. Ideally, the fleecing storage is a
+dedicated storage, so it running full will not affect other guests and just fail
+the backup. Parts of the fleecing image that have been backed up will be
+discarded to try and keep the space usage low.
+
+For file-based storages that do not support discard (e.g. NFS before version
+4.2), you should set `preallocation off` in the storage configuration. In
+combination with `qcow2` (used automatically as the format for the fleecing
+image when the storage supports it), this has the advantage that already
+allocated parts of the image can be re-used later, which can still help save
+quite a bit of space.
+
+WARNING: On a storage that's not thinly provisioned, e.g. LVM or ZFS without the
+`sparse` option, the full size of the original disk needs to be reserved for the
+fleecing image up-front. On a thinly provisioned storage, the fleecing image can
+grow to the same size as the original image only if the guest re-writes a whole
+disk while the backup is busy with another disk.
+
Backup File Names
-----------------
--
2.39.2
prev parent reply other threads:[~2024-03-15 10:34 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-15 10:24 [pve-devel] [PATCH-SERIES v2] fix #4136: implement " Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [PATCH qemu v2 01/21] block/copy-before-write: fix permission Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [PATCH qemu v2 02/21] block/copy-before-write: support unligned snapshot-discard Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [PATCH qemu v2 03/21] block/copy-before-write: create block_copy bitmap in filter node Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [PATCH qemu v2 04/21] qapi: blockdev-backup: add discard-source parameter Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [PATCH qemu v2 05/21] copy-before-write: allow specifying minimum cluster size Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [PATCH qemu v2 06/21] backup: add minimum cluster size to performance options Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [PATCH qemu v2 07/21] PVE backup: add fleecing option Fiona Ebner
2024-04-08 12:45 ` Wolfgang Bumiller
2024-04-10 9:30 ` Fiona Ebner
2024-04-10 11:38 ` Wolfgang Bumiller
2024-03-15 10:24 ` [pve-devel] [PATCH common v2 08/21] json schema: add format description for pve-storage-id standard option Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [PATCH guest-common v2 09/21] vzdump: schema: add fleecing property string Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [PATCH guest-common v2 10/21] vzdump: schema: make storage for fleecing semi-optional Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [RFC guest-common v2 11/21] abstract config: do not copy fleecing images entry for snapshot Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [PATCH manager v2 12/21] vzdump: handle new 'fleecing' property string Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [PATCH manager v2 13/21] api: backup/vzdump: add permission check for fleecing storage Fiona Ebner
2024-04-08 8:47 ` Wolfgang Bumiller
2024-04-10 9:57 ` Fiona Ebner
2024-04-10 11:37 ` Wolfgang Bumiller
2024-03-15 10:24 ` [pve-devel] [PATCH qemu-server v2 14/21] backup: disk info: also keep track of size Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [PATCH qemu-server v2 15/21] backup: implement fleecing option Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [RFC qemu-server v2 16/21] parse config: allow config keys with minus sign Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [RFC qemu-server v2 17/21] schema: add fleecing-images config property Fiona Ebner
2024-03-15 10:24 ` [pve-devel] [RFC qemu-server v2 18/21] vzdump: better cleanup fleecing images after hard errors Fiona Ebner
2024-03-15 10:25 ` [pve-devel] [RFC qemu-server v2 19/21] migration: attempt to clean up potential left-over fleecing images Fiona Ebner
2024-03-15 10:25 ` [pve-devel] [RFC qemu-server v2 20/21] destroy vm: " Fiona Ebner
2024-03-15 10:25 ` Fiona Ebner [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240315102502.84163-22-f.ebner@proxmox.com \
--to=f.ebner@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal