public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] [PATCH-SERIES v3] fix #4136: implement backup fleecing
@ 2024-04-11  9:29 Fiona Ebner
  2024-04-11  9:29 ` [pve-devel] [PATCH qemu v3 01/22] block/copy-before-write: fix permission Fiona Ebner
                   ` (22 more replies)
  0 siblings, 23 replies; 33+ messages in thread
From: Fiona Ebner @ 2024-04-11  9:29 UTC (permalink / raw)
  To: pve-devel

Changes in v3 (thanks to Wolfgang for feedback!):
    * Fix brittle code for permission check that only worked by
      chance.

Changes in v2 (thanks - not limited to - to Fabian and Alexandre for
feedback!):
    * Use v3 of "discard-source" upstream series (v4 was posted in the
      meantime but without any semantic change)
    * Add patches to specify minimum cluster size during backup, to
      allow discard to work even if fleecing image has larger cluster
      size than backup target.
    * Add permission check for fleecing storage.
    * Record fleecing image in config to be able to clean up after
      hard failure.
    * Do not use "same storage as image" as default fleecing storage.
    * Use qcow2 for fleecing image if storage supports it
    * Flesh out recommendations for fleecing storage in docs.

When a backup for a VM is started, QEMU will install a
"copy-before-write" filter in its block layer. This filter ensures
that upon new guest writes, old data still needed for the backup is
sent to the backup target first. The guest write blocks until this
operation is finished so guest IO to not-yet-backed-up sectors will be
limited by the speed of the backup target.

With backup fleecing, such old data is cached in a fleecing image
rather than sent directly to the backup target. This can help guest IO
performance and even prevent hangs in certain scenarios, at the cost
of requiring more storage space.

With this series it will be possible to enable backup-fleecing via
e.g. `vzdump 123 --fleecing enabled=1,storage=local-lvm` with fleecing
images created on the storage `local-lvm`. The fleecing storage should
be a fast local storage which supports thin-provisioning and discard.
If the storage supports qcow2, that is used as the fleecing image
format. If the underlying file system does not support discard, with
qcow2 and preallocation=off, at least already allocated parts of the
image can be re-used later.

Fleecing images are created by qemu-server via pve-storage and
attached to QEMU before the backup starts, and cleaned up after the
backup finished or failed. The naming schema for fleecing images is
'vm-ID-fleece-N(.FORMAT)'. The allocated images are recorded in the
guest configuration, so that even after a hard failure, clean-up can
be re-attempted. While not too bad, it's a non-trivial amount of code
and I'm not 100% sure about the cost-benefit, so sending those as RFC.

The fleecing image needs to be the exact same size as the source, but
luckily, an explicit size can be specified when attaching a raw image
to QEMU so there are no size issues when using storages that have
coarser allocation/round up. For qcow2, it seems that virtual size can
be nearly arbitrary (i.e. modulo 512 byte granularity) during
allocation.

While tests seem fine so far, most important part to review is the
setup of the backup job and bitmap handling inside QEMU.

QEMU patches are for the submodule for better reviewability. There are
two prerequisites (that are expected to be picked up by upstream at
some point):

1. For being able to discard the fleecing image, addition of a
discard-source parameter [0].

2. In combination with discard, cluster size issue when fleecing image
has a larger cluster size than backup target. Proposed workaround is
to be able to specify the minimum granularity for the backup job [1].


Dependencies:
pve-manager -> pve-guest-common -> pve-common
            \-> qemu-server

Plus new pve-qemu-kvm to actually be able to use the feature.

[0]: https://lore.kernel.org/qemu-devel/20240228141501.455989-1-vsementsov@yandex-team.ru/
[1]: https://lore.kernel.org/qemu-devel/20240308155158.830258-1-f.ebner@proxmox.com/


qemu:

Fiona Ebner (3):
  copy-before-write: allow specifying minimum cluster size
  backup: add minimum cluster size to performance options
  PVE backup: add fleecing option

Vladimir Sementsov-Ogievskiy (4):
  block/copy-before-write: fix permission
  block/copy-before-write: support unligned snapshot-discard
  block/copy-before-write: create block_copy bitmap in filter node
  qapi: blockdev-backup: add discard-source parameter

 block/backup.c                         |   5 +-
 block/block-copy.c                     |  29 ++++-
 block/copy-before-write.c              |  42 ++++++--
 block/copy-before-write.h              |   2 +
 block/monitor/block-hmp-cmds.c         |   1 +
 block/replication.c                    |   4 +-
 blockdev.c                             |   5 +-
 include/block/block-common.h           |   2 +
 include/block/block-copy.h             |   3 +
 include/block/block_int-global-state.h |   2 +-
 pve-backup.c                           | 143 ++++++++++++++++++++++++-
 qapi/block-core.json                   |  29 ++++-
 tests/qemu-iotests/257.out             | 112 +++++++++----------
 13 files changed, 298 insertions(+), 81 deletions(-)


common:

Fiona Ebner (1):
  json schema: add format description for pve-storage-id standard option

 src/PVE/JSONSchema.pm | 1 +
 1 file changed, 1 insertion(+)


guest-common:

Fiona Ebner (3):
  vzdump: schema: add fleecing property string
  vzdump: schema: make storage for fleecing semi-optional
  abstract config: do not copy fleecing images entry for snapshot

 src/PVE/AbstractConfig.pm |  1 +
 src/PVE/VZDump/Common.pm  | 37 +++++++++++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+)


manager:

Fiona Ebner (3):
  vzdump: have property string helpers always return the result
  vzdump: handle new 'fleecing' property string
  api: backup/vzdump: add permission check for fleecing storage

 PVE/API2/Backup.pm | 10 ++++++++--
 PVE/API2/VZDump.pm |  9 +++++----
 PVE/VZDump.pm      | 22 ++++++++++++++++++++--
 3 files changed, 33 insertions(+), 8 deletions(-)


qemu-server:

Fiona Ebner (7):
  backup: disk info: also keep track of size
  backup: implement fleecing option
  parse config: allow config keys with minus sign
  schema: add fleecing-images config property
  vzdump: better cleanup fleecing images after hard errors
  migration: attempt to clean up potential left-over fleecing images
  destroy vm: clean up potential left-over fleecing images

 PVE/API2/Qemu.pm         |   9 +++
 PVE/QemuConfig.pm        |  40 ++++++++++
 PVE/QemuMigrate.pm       |   3 +
 PVE/QemuServer.pm        |  12 ++-
 PVE/VZDump/QemuServer.pm | 163 ++++++++++++++++++++++++++++++++++++++-
 5 files changed, 224 insertions(+), 3 deletions(-)


docs:

Fiona Ebner (1):
  vzdump: add section about backup fleecing

 vzdump.adoc | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)


Summary over all repositories:
  25 files changed, 632 insertions(+), 92 deletions(-)

-- 
Generated by git-murpp 0.5.0




^ permalink raw reply	[flat|nested] 33+ messages in thread

end of thread, other threads:[~2024-04-22  8:16 UTC | newest]

Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-11  9:29 [pve-devel] [PATCH-SERIES v3] fix #4136: implement backup fleecing Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH qemu v3 01/22] block/copy-before-write: fix permission Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH qemu v3 02/22] block/copy-before-write: support unligned snapshot-discard Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH qemu v3 03/22] block/copy-before-write: create block_copy bitmap in filter node Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH qemu v3 04/22] qapi: blockdev-backup: add discard-source parameter Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH qemu v3 05/22] copy-before-write: allow specifying minimum cluster size Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH qemu v3 06/22] backup: add minimum cluster size to performance options Fiona Ebner
2024-04-11 18:41   ` [pve-devel] partially-applied: " Thomas Lamprecht
2024-04-11  9:29 ` [pve-devel] [PATCH qemu v3 07/22] PVE backup: add fleecing option Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH common v3 08/22] json schema: add format description for pve-storage-id standard option Fiona Ebner
2024-04-11 17:58   ` [pve-devel] applied: " Thomas Lamprecht
2024-04-11  9:29 ` [pve-devel] [PATCH guest-common v3 09/22] vzdump: schema: add fleecing property string Fiona Ebner
2024-04-11 18:07   ` [pve-devel] applied: " Thomas Lamprecht
2024-04-12  8:38     ` Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH guest-common v3 10/22] vzdump: schema: make storage for fleecing semi-optional Fiona Ebner
2024-04-11 18:07   ` Thomas Lamprecht
2024-04-11 18:07   ` [pve-devel] applied: " Thomas Lamprecht
2024-04-11  9:29 ` [pve-devel] [RFC guest-common v3 11/22] abstract config: do not copy fleecing images entry for snapshot Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH manager v3 12/22] vzdump: have property string helpers always return the result Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH manager v3 13/22] vzdump: handle new 'fleecing' property string Fiona Ebner
2024-04-22  8:15   ` Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH manager v3 14/22] api: backup/vzdump: add permission check for fleecing storage Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH qemu-server v3 15/22] backup: disk info: also keep track of size Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH qemu-server v3 16/22] backup: implement fleecing option Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [RFC qemu-server v3 17/22] parse config: allow config keys with minus sign Fiona Ebner
2024-04-11 17:50   ` Thomas Lamprecht
2024-04-16  9:02     ` Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [RFC qemu-server v3 18/22] schema: add fleecing-images config property Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [RFC qemu-server v3 19/22] vzdump: better cleanup fleecing images after hard errors Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [RFC qemu-server v3 20/22] migration: attempt to clean up potential left-over fleecing images Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [RFC qemu-server v3 21/22] destroy vm: " Fiona Ebner
2024-04-11  9:29 ` [pve-devel] [PATCH docs v3 22/22] vzdump: add section about backup fleecing Fiona Ebner
2024-04-19 15:23 ` [pve-devel] partially-applied: [PATCH-SERIES v3] fix #4136: implement " Fiona Ebner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal