From: Fiona Ebner <f.ebner@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH docs v3 22/22] vzdump: add section about backup fleecing
Date: Thu, 11 Apr 2024 11:29:43 +0200 [thread overview]
Message-ID: <20240411092943.57377-23-f.ebner@proxmox.com> (raw)
In-Reply-To: <20240411092943.57377-1-f.ebner@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
vzdump.adoc | 38 ++++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)
diff --git a/vzdump.adoc b/vzdump.adoc
index b5bbac7..b01d3c5 100644
--- a/vzdump.adoc
+++ b/vzdump.adoc
@@ -136,6 +136,44 @@ not included in backups. For volume mount points you can set the *Backup* option
to include the mount point in the backup. Device and bind mounts are never
backed up as their content is managed outside the {pve} storage library.
+VM Backup Fleecing
+~~~~~~~~~~~~~~~~~~
+
+WARNING: Backup fleecing is still being worked on (also in upstream QEMU) and is
+currently only a technology preview.
+
+When a backup for a VM is started, QEMU will install a "copy-before-write"
+filter in its block layer. This filter ensures that upon new guest writes, old
+data still needed for the backup is sent to the backup target first. The guest
+write blocks until this operation is finished so guest IO to not-yet-backed-up
+sectors will be limited by the speed of the backup target.
+
+With backup fleecing, such old data is cached in a fleecing image rather than
+sent directly to the backup target. This can help guest IO performance and even
+prevent hangs in certain scenarios, at the cost of requiring more storage space.
+Use e.g. `vzdump 123 --fleecing enabled=1,storage=local-lvm` to enable backup
+fleecing, with fleecing images created on the storage `local-lvm`.
+
+The fleecing storage should be a fast local storage, with thin provisioning and
+discard support. Examples are LVM-thin, RBD, ZFS with `sparse 1` in the storage
+configuration, many file-based storages. Ideally, the fleecing storage is a
+dedicated storage, so it running full will not affect other guests and just fail
+the backup. Parts of the fleecing image that have been backed up will be
+discarded to try and keep the space usage low.
+
+For file-based storages that do not support discard (e.g. NFS before version
+4.2), you should set `preallocation off` in the storage configuration. In
+combination with `qcow2` (used automatically as the format for the fleecing
+image when the storage supports it), this has the advantage that already
+allocated parts of the image can be re-used later, which can still help save
+quite a bit of space.
+
+WARNING: On a storage that's not thinly provisioned, e.g. LVM or ZFS without the
+`sparse` option, the full size of the original disk needs to be reserved for the
+fleecing image up-front. On a thinly provisioned storage, the fleecing image can
+grow to the same size as the original image only if the guest re-writes a whole
+disk while the backup is busy with another disk.
+
Backup File Names
-----------------
--
2.39.2
next prev parent reply other threads:[~2024-04-11 9:33 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-11 9:29 [pve-devel] [PATCH-SERIES v3] fix #4136: implement " Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [PATCH qemu v3 01/22] block/copy-before-write: fix permission Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [PATCH qemu v3 02/22] block/copy-before-write: support unligned snapshot-discard Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [PATCH qemu v3 03/22] block/copy-before-write: create block_copy bitmap in filter node Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [PATCH qemu v3 04/22] qapi: blockdev-backup: add discard-source parameter Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [PATCH qemu v3 05/22] copy-before-write: allow specifying minimum cluster size Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [PATCH qemu v3 06/22] backup: add minimum cluster size to performance options Fiona Ebner
2024-04-11 18:41 ` [pve-devel] partially-applied: " Thomas Lamprecht
2024-04-11 9:29 ` [pve-devel] [PATCH qemu v3 07/22] PVE backup: add fleecing option Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [PATCH common v3 08/22] json schema: add format description for pve-storage-id standard option Fiona Ebner
2024-04-11 17:58 ` [pve-devel] applied: " Thomas Lamprecht
2024-04-11 9:29 ` [pve-devel] [PATCH guest-common v3 09/22] vzdump: schema: add fleecing property string Fiona Ebner
2024-04-11 18:07 ` [pve-devel] applied: " Thomas Lamprecht
2024-04-12 8:38 ` Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [PATCH guest-common v3 10/22] vzdump: schema: make storage for fleecing semi-optional Fiona Ebner
2024-04-11 18:07 ` Thomas Lamprecht
2024-04-11 18:07 ` [pve-devel] applied: " Thomas Lamprecht
2024-04-11 9:29 ` [pve-devel] [RFC guest-common v3 11/22] abstract config: do not copy fleecing images entry for snapshot Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [PATCH manager v3 12/22] vzdump: have property string helpers always return the result Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [PATCH manager v3 13/22] vzdump: handle new 'fleecing' property string Fiona Ebner
2024-04-22 8:15 ` Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [PATCH manager v3 14/22] api: backup/vzdump: add permission check for fleecing storage Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [PATCH qemu-server v3 15/22] backup: disk info: also keep track of size Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [PATCH qemu-server v3 16/22] backup: implement fleecing option Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [RFC qemu-server v3 17/22] parse config: allow config keys with minus sign Fiona Ebner
2024-04-11 17:50 ` Thomas Lamprecht
2024-04-16 9:02 ` Fiona Ebner
2024-10-21 13:28 ` Thomas Lamprecht
2024-04-11 9:29 ` [pve-devel] [RFC qemu-server v3 18/22] schema: add fleecing-images config property Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [RFC qemu-server v3 19/22] vzdump: better cleanup fleecing images after hard errors Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [RFC qemu-server v3 20/22] migration: attempt to clean up potential left-over fleecing images Fiona Ebner
2024-04-11 9:29 ` [pve-devel] [RFC qemu-server v3 21/22] destroy vm: " Fiona Ebner
2024-04-11 9:29 ` Fiona Ebner [this message]
2024-04-19 15:23 ` [pve-devel] partially-applied: [PATCH-SERIES v3] fix #4136: implement backup fleecing Fiona Ebner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240411092943.57377-23-f.ebner@proxmox.com \
--to=f.ebner@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal