From: Alexandre Derumier via pve-devel <pve-devel@lists.proxmox.com>
To: pve-devel@lists.proxmox.com
Cc: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
Subject: [pve-devel] [PATCH-SERIES v6 pve-storage/qemu-server/pve-qemu] add external qcow2 snapshot support
Date: Tue, 3 Jun 2025 09:55:35 +0200 [thread overview]
Message-ID: <mailman.197.1748937402.395.pve-devel@lists.proxmox.com> (raw)
[-- Attachment #1: Type: message/rfc822, Size: 13270 bytes --]
From: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH-SERIES v6 pve-storage/qemu-server/pve-qemu] add external qcow2 snapshot support
Date: Tue, 3 Jun 2025 09:55:35 +0200
Message-ID: <20250603075558.627850-1-alexandre.derumier@groupe-cyllene.com>
This patch series implement qcow2 external snapshot support for files && lvm volumes
The current internal qcow2 snapshots have bad write performance because no metadatas can be preallocated.
This is particulary visible on a shared filesystem like ocfs2 or gfs2.
Also other bugs are freeze/lock reported by users since years on snapshots delete on nfs
(The disk access seem to be frozen during all the delete duration)
This also open doors for remote snapshot export-import for storage replication.
The pve-storage patches applied on top of Fiona qemu_blockdev_options patch series
https://lists.proxmox.com/pipermail/pve-devel/2025-May/071204.html
This new patch series enable blockdev for qemu machine > 10.0 to avoid breaking efidisk and maybe
potential migrations bug
Changelog v6:
pve-storage:
- use fiona qemu_blockdev_options patch
- add helpers for qemu-img call
- use qemu-img measure for lvm size (including l2_extended=on)
- add a new helper volume_support_qemu_snapshot
- fix Fabian requested changes
qemu-server:
- enable blockdev for machine version > 10.0
- move blockdev code in QemuServer/Blockdev.pm
- use Fiona qemu_blockdev_options patch for storage plugins
- add raw size for efidisk
- fix drivedel with backing chain
- add tests for qemu-img convert, backing chain,....
- fix Fabian requested changes
pve-qemu:
- add a fake 10.0 qemu bump for testing
MAIN TODO:
- convert/implement all the backup code, dirty bitmap mirro, ...
- add snapshots tests
pve-storage:
Alexandre Derumier (9):
common: add qemu_img_create an preallocation_cmd_option
common: add qemu_img_create_qcow2_backed
common: add qemu_img_info helper
common: add qemu_img_measure
rename_volume: add source && target snap
storage: volume_snapshot: add $running param + api bump
qcow2: add external snapshot support
lvmplugin: add qcow2 snapshot
storage: add volume_support_qemu_snapshot
src/PVE/Storage.pm | 28 +-
src/PVE/Storage/BTRFSPlugin.pm | 2 +-
src/PVE/Storage/Common.pm | 175 ++++++++++
src/PVE/Storage/DirPlugin.pm | 11 +
src/PVE/Storage/ESXiPlugin.pm | 2 +-
src/PVE/Storage/GlusterfsPlugin.pm | 2 +-
src/PVE/Storage/ISCSIDirectPlugin.pm | 2 +-
src/PVE/Storage/LVMPlugin.pm | 467 +++++++++++++++++++++++----
src/PVE/Storage/LvmThinPlugin.pm | 2 +-
src/PVE/Storage/PBSPlugin.pm | 2 +-
src/PVE/Storage/Plugin.pm | 339 ++++++++++++++-----
src/PVE/Storage/RBDPlugin.pm | 8 +-
src/PVE/Storage/ZFSPoolPlugin.pm | 2 +-
13 files changed, 875 insertions(+), 167 deletions(-)
qemu-server:
Alexandre Derumier (13):
blockdev: cmdline: add blockdev syntax support
blockdev: add support for ovmf && efidisk
blockdev: vm_devices_list : fix block-query
blockdev: add support to qemu_driveadd && qemu_drivedel
blockdev: add support to qemu_block_set_io_throttle
blockdev: add support for cdrom media eject/insert
blockdev: add support for block_resize
blockdev: add support for nbd_export: block-export-add
blockdev: add blockdev_mirror
blockdev: change aio on target if io_uring is not default.
qemu_img convert : add external snapshot support
blockdev: add backing_chain support
qcow2: add external snapshot support
PVE/QemuConfig.pm | 4 +-
PVE/QemuMigrate.pm | 2 +-
PVE/QemuServer.pm | 761 +++++++++++-------
PVE/QemuServer/Blockdev.pm | 494 ++++++++++++
PVE/QemuServer/Drive.pm | 201 +++++
PVE/QemuServer/Makefile | 1 +
test/MigrationTest/QemuMigrateMock.pm | 10 +-
test/cfg2cmd/bootorder-blockdev.conf | 15 +
test/cfg2cmd/bootorder-blockdev.conf.cmd | 43 +
test/cfg2cmd/bootorder-empty-blockdev.conf | 15 +
.../cfg2cmd/bootorder-empty-blockdev.conf.cmd | 43 +
test/cfg2cmd/bootorder-empty.conf | 1 +
test/cfg2cmd/bootorder-empty.conf.cmd | 2 +-
test/cfg2cmd/bootorder-legacy.conf | 1 +
test/cfg2cmd/bootorder-legacy.conf.cmd | 2 +-
test/cfg2cmd/bootorder.conf | 1 +
test/cfg2cmd/bootorder.conf.cmd | 2 +-
.../cputype-icelake-client-deprecation.conf | 3 -
...putype-icelake-client-deprecation.conf.cmd | 7 +-
.../custom-cpu-model-defaults.conf.cmd | 2 +-
test/cfg2cmd/efi-raw-template.conf.cmd | 7 +-
test/cfg2cmd/efi-raw.conf | 1 +
test/cfg2cmd/efi-raw.conf.cmd | 2 +-
test/cfg2cmd/efi-secboot-and-tpm-q35.conf.cmd | 7 +-
test/cfg2cmd/efi-secboot-and-tpm.conf.cmd | 7 +-
test/cfg2cmd/efidisk-on-rbd.conf | 1 +
test/cfg2cmd/efidisk-on-rbd.conf.cmd | 2 +-
test/cfg2cmd/i440fx-viommu-virtio.conf.cmd | 2 +-
test/cfg2cmd/ide-blockdev.conf | 14 +
test/cfg2cmd/ide-blockdev.conf.cmd | 46 ++
test/cfg2cmd/ide.conf | 1 +
test/cfg2cmd/ide.conf.cmd | 2 +-
.../cfg2cmd/memory-hotplug-hugepages.conf.cmd | 2 +-
test/cfg2cmd/memory-hotplug.conf.cmd | 2 +-
test/cfg2cmd/memory-hugepages-1g.conf.cmd | 2 +-
test/cfg2cmd/memory-hugepages-2m.conf.cmd | 2 +-
test/cfg2cmd/minimal-defaults.conf.cmd | 2 +-
test/cfg2cmd/netdev-7.1-multiqueues.conf.cmd | 2 +-
test/cfg2cmd/netdev-7.1.conf.cmd | 2 +-
test/cfg2cmd/q35-ide-blockdev.conf | 15 +
test/cfg2cmd/q35-ide-blockdev.conf.cmd | 45 ++
test/cfg2cmd/q35-ide.conf | 2 +-
test/cfg2cmd/q35-ide.conf.cmd | 3 +-
.../q35-linux-hostpci-mapping.conf.cmd | 7 +-
.../q35-linux-hostpci-multifunction.conf.cmd | 7 +-
.../q35-linux-hostpci-template.conf.cmd | 10 +-
...q35-linux-hostpci-x-pci-overrides.conf.cmd | 9 +-
test/cfg2cmd/q35-linux-hostpci.conf.cmd | 7 +-
test/cfg2cmd/q35-simple.conf.cmd | 7 +-
test/cfg2cmd/q35-viommu-intel.conf.cmd | 2 +-
test/cfg2cmd/q35-viommu-virtio.conf.cmd | 2 +-
test/cfg2cmd/seabios_serial.conf | 4 -
test/cfg2cmd/seabios_serial.conf.cmd | 7 +-
test/cfg2cmd/sev-es.conf.cmd | 7 +-
test/cfg2cmd/sev-snp.conf.cmd | 2 +-
test/cfg2cmd/sev-std.conf.cmd | 7 +-
test/cfg2cmd/simple-backingchain.conf | 21 +
test/cfg2cmd/simple-backingchain.conf.cmd | 30 +
test/cfg2cmd/simple-btrfs.conf | 1 +
test/cfg2cmd/simple-btrfs.conf.cmd | 2 +-
.../simple-disk-passthrough-blockdev.conf | 15 +
.../simple-disk-passthrough-blockdev.conf.cmd | 40 +
test/cfg2cmd/simple-disk-passthrough.conf | 1 +
test/cfg2cmd/simple-disk-passthrough.conf.cmd | 2 +-
test/cfg2cmd/simple-rbd-blockdev.conf | 15 +
test/cfg2cmd/simple-rbd-blockdev.conf.cmd | 38 +
test/cfg2cmd/simple-rbd.conf | 1 +
test/cfg2cmd/simple-rbd.conf.cmd | 2 +-
test/cfg2cmd/simple-virtio-blk-blockdev.conf | 13 +
.../simple-virtio-blk-blockdev.conf.cmd | 35 +
test/cfg2cmd/simple-virtio-blk.conf | 1 +
test/cfg2cmd/simple-virtio-blk.conf.cmd | 2 +-
.../simple-zfs-over-iscsi-blockdev.conf | 14 +
.../simple-zfs-over-iscsi-blockdev.conf.cmd | 35 +
test/cfg2cmd/simple-zfs-over-iscsi.conf | 1 +
test/cfg2cmd/simple-zfs-over-iscsi.conf.cmd | 2 +-
test/cfg2cmd/simple1-blockdev.conf | 14 +
test/cfg2cmd/simple1-blockdev.conf.cmd | 35 +
test/cfg2cmd/simple1-template.conf.cmd | 13 +-
test/cfg2cmd/simple1.conf | 1 +
test/cfg2cmd/simple1.conf.cmd | 2 +-
test/cfg2cmd/vnc-clipboard-spice.conf.cmd | 2 +-
test/cfg2cmd/vnc-clipboard-std.conf.cmd | 2 +-
test/run_config2command_tests.pl | 52 ++
test/run_qemu_img_convert_tests.pl | 44 +
test/snapshot-test.pm | 4 +-
86 files changed, 1905 insertions(+), 394 deletions(-)
create mode 100644 PVE/QemuServer/Blockdev.pm
create mode 100644 test/cfg2cmd/bootorder-blockdev.conf
create mode 100644 test/cfg2cmd/bootorder-blockdev.conf.cmd
create mode 100644 test/cfg2cmd/bootorder-empty-blockdev.conf
create mode 100644 test/cfg2cmd/bootorder-empty-blockdev.conf.cmd
create mode 100644 test/cfg2cmd/ide-blockdev.conf
create mode 100644 test/cfg2cmd/ide-blockdev.conf.cmd
create mode 100644 test/cfg2cmd/q35-ide-blockdev.conf
create mode 100644 test/cfg2cmd/q35-ide-blockdev.conf.cmd
create mode 100644 test/cfg2cmd/simple-backingchain.conf
create mode 100644 test/cfg2cmd/simple-backingchain.conf.cmd
create mode 100644 test/cfg2cmd/simple-disk-passthrough-blockdev.conf
create mode 100644 test/cfg2cmd/simple-disk-passthrough-blockdev.conf.cmd
create mode 100644 test/cfg2cmd/simple-rbd-blockdev.conf
create mode 100644 test/cfg2cmd/simple-rbd-blockdev.conf.cmd
create mode 100644 test/cfg2cmd/simple-virtio-blk-blockdev.conf
create mode 100644 test/cfg2cmd/simple-virtio-blk-blockdev.conf.cmd
create mode 100644 test/cfg2cmd/simple-zfs-over-iscsi-blockdev.conf
create mode 100644 test/cfg2cmd/simple-zfs-over-iscsi-blockdev.conf.cmd
create mode 100644 test/cfg2cmd/simple1-blockdev.conf
create mode 100644 test/cfg2cmd/simple1-blockdev.conf.cmd
pve-qemu:
Alexandre Derumier (1):
add fake 10.0
debian/changelog | 6 +
.../extra/0001-bump-version-to-10.0.0.patch | 19 ++
...438a42f7a178504347456b9e50e3d2c50478.patch | 247 ++++++++++++++++++
...b0606879ddeddf04bcf8440aabcfe1393b9c.patch | 120 +++++++++
debian/patches/series | 3 +
5 files changed, 395 insertions(+)
create mode 100644 debian/patches/extra/0001-bump-version-to-10.0.0.patch
create mode 100644 debian/patches/extra/0a7c438a42f7a178504347456b9e50e3d2c50478.patch
create mode 100644 debian/patches/extra/7956b0606879ddeddf04bcf8440aabcfe1393b9c.patch
--
2.39.5
[-- Attachment #2: Type: text/plain, Size: 160 bytes --]
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
next reply other threads:[~2025-06-03 7:57 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-06-03 7:55 Alexandre Derumier via pve-devel [this message]
2025-06-05 13:53 ` Fiona Ebner
2025-06-05 14:08 ` DERUMIER, Alexandre via pve-devel
[not found] ` <0aa862ab988ff9df50f4637f3712f137f23c34c2.camel@groupe-cyllene.com>
2025-06-05 14:19 ` Fiona Ebner
2025-06-06 10:17 ` joao.sousa via pve-devel
2025-06-10 8:54 ` Fiona Ebner
2025-06-11 21:28 ` joao.sousa via pve-devel
[not found] ` <5sqpzsnyzmfvvoco3ilo6tg3pcpomhle7h5dsnmbsbdjobpr5a@rqfsbsas36e2>
2025-06-12 7:02 ` DERUMIER, Alexandre via pve-devel
2025-06-12 8:12 ` Fiona Ebner
2025-06-13 10:25 ` joao.sousa via pve-devel
[not found] ` <zwkygc6va6jmieho2h7gqsx4idrgrxvd7et2kfbi3vyddhjk7c@kfiwufb7auvv>
2025-06-16 8:34 ` Fiona Ebner
[not found] ` <8cbed06bbe3db00e96670dec1e240123b946a917.camel@groupe-cyllene.com>
2025-06-12 11:33 ` DERUMIER, Alexandre via pve-devel
[not found] ` <a7185aa585b414710be6e09729adda57ed3787b8.camel@groupe-cyllene.com>
2025-06-13 16:42 ` joao.sousa--- via pve-devel
[not found] ` <qss36oltvd4gz5tiss7dqono5h2zmsgwmodj4cqkroboizypwj@kbyz3mymh75a>
2025-06-16 5:50 ` DERUMIER, Alexandre via pve-devel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=mailman.197.1748937402.395.pve-devel@lists.proxmox.com \
--to=pve-devel@lists.proxmox.com \
--cc=alexandre.derumier@groupe-cyllene.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal