* [pve-devel] [PATCH-SERIES v7 pve-storage/qemu-server] add external qcow2 snapshot support
@ 2025-07-04 6:44 Alexandre Derumier via pve-devel
2025-07-04 11:58 ` Fabian Grünbichler
0 siblings, 1 reply; 4+ messages in thread
From: Alexandre Derumier via pve-devel @ 2025-07-04 6:44 UTC (permalink / raw)
To: pve-devel; +Cc: Alexandre Derumier
[-- Attachment #1: Type: message/rfc822, Size: 6359 bytes --]
From: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH-SERIES v7 pve-storage/qemu-server] add external qcow2 snapshot support
Date: Fri, 4 Jul 2025 08:44:54 +0200
Message-ID: <20250704064507.511884-1-alexandre.derumier@groupe-cyllene.com>
This patch series implement qcow2 external snapshot support for files && lvm volumes
The current internal qcow2 snapshots have bad write performance because no metadatas can be preallocated.
This is particulary visible on a shared filesystem like ocfs2 or gfs2.
Also other bugs are freeze/lock reported by users since years on snapshots delete on nfs
(The disk access seem to be frozen during all the delete duration)
This also open doors for remote snapshot export-import for storage replication.
Changelog v7:
- rebased on fiona blockdev work
MAIN TODO:
- add snapshots tests in both pve-storage && qemu-server
- better handle snapshot failure with multiple disks
pve-storage:
Alexandre Derumier (10):
tests: add lvmplugin test
common: add qemu_img_create an preallocation_cmd_option
common: qemu_img_create: add backing_file support
rename_volume: add source && target snap
common: add qemu_img_info helper
common: add qemu-img measure
storage: volume_snapshot: add $running param
qcow2: add external snapshot support
lvmplugin: add qcow2 snapshot
storage : add volume_support_qemu_snapshot
src/PVE/Storage.pm | 34 +-
src/PVE/Storage/Common.pm | 116 ++++++
src/PVE/Storage/DirPlugin.pm | 11 +
src/PVE/Storage/ESXiPlugin.pm | 2 +-
src/PVE/Storage/ISCSIDirectPlugin.pm | 2 +-
src/PVE/Storage/LVMPlugin.pm | 519 ++++++++++++++++++++----
src/PVE/Storage/LvmThinPlugin.pm | 2 +-
src/PVE/Storage/PBSPlugin.pm | 2 +-
src/PVE/Storage/Plugin.pm | 376 ++++++++++++-----
src/PVE/Storage/RBDPlugin.pm | 8 +-
src/PVE/Storage/ZFSPoolPlugin.pm | 2 +-
src/test/Makefile | 5 +-
src/test/run_test_lvmplugin.pl | 577 +++++++++++++++++++++++++++
13 files changed, 1465 insertions(+), 191 deletions(-)
create mode 100755 src/test/run_test_lvmplugin.pl
qemu-server:
Alexandre Derumier (3):
qemu_img convert : add external snapshot support
blockdev: add backing_chain support
qcow2: add external snapshot support
src/PVE/QemuConfig.pm | 4 +-
src/PVE/QemuServer.pm | 132 +++++--
src/PVE/QemuServer/Blockdev.pm | 345 +++++++++++++++++-
src/PVE/QemuServer/QemuImage.pm | 6 +-
src/test/cfg2cmd/simple-backingchain.conf | 25 ++
src/test/cfg2cmd/simple-backingchain.conf.cmd | 33 ++
src/test/run_config2command_tests.pl | 47 +++
src/test/run_qemu_img_convert_tests.pl | 59 +++
src/test/snapshot-test.pm | 4 +-
9 files changed, 620 insertions(+), 35 deletions(-)
create mode 100644 src/test/cfg2cmd/simple-backingchain.conf
create mode 100644 src/test/cfg2cmd/simple-backingchain.conf.cmd
--
2.39.5
[-- Attachment #2: Type: text/plain, Size: 160 bytes --]
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [pve-devel] [PATCH-SERIES v7 pve-storage/qemu-server] add external qcow2 snapshot support
2025-07-04 6:44 [pve-devel] [PATCH-SERIES v7 pve-storage/qemu-server] add external qcow2 snapshot support Alexandre Derumier via pve-devel
@ 2025-07-04 11:58 ` Fabian Grünbichler
2025-07-04 12:10 ` DERUMIER, Alexandre via pve-devel
[not found] ` <a1b7140a1c715ce9506f9956c296138b2373cf6b.camel@groupe-cyllene.com>
0 siblings, 2 replies; 4+ messages in thread
From: Fabian Grünbichler @ 2025-07-04 11:58 UTC (permalink / raw)
To: Proxmox VE development discussion; +Cc: Thomas Lamprecht
> Alexandre Derumier via pve-devel <pve-devel@lists.proxmox.com> hat am 04.07.2025 08:44 CEST geschrieben:
> This patch series implement qcow2 external snapshot support for files && lvm volumes
>
> The current internal qcow2 snapshots have bad write performance because no metadatas can be preallocated.
>
> This is particulary visible on a shared filesystem like ocfs2 or gfs2.
>
> Also other bugs are freeze/lock reported by users since years on snapshots delete on nfs
> (The disk access seem to be frozen during all the delete duration)
>
> This also open doors for remote snapshot export-import for storage replication.
sent some feedback, mostly for the storage and qemu-server <-> storage interface parts,
haven't yet done an in-depth pass over qemu-server or end-to-end.
in particular, the rename_volume vs rename_snapshot and the Plugin snapshot file name/..
questions should probably be solved before the next version
it would also be great if you could add a patch bumping API version and age and
documenting the changes/new parameters/interfaces - also so that we can ensure
we are on the same page and haven't overlooked anything. AFAICT it should all be
new opt-in or backwards compatible changes, once the feedback is incorporated.
>
> Changelog v7:
> - rebased on fiona blockdev work
>
> MAIN TODO:
> - add snapshots tests in both pve-storage && qemu-server
> - better handle snapshot failure with multiple disks
>
> pve-storage:
>
> Alexandre Derumier (10):
> tests: add lvmplugin test
> common: add qemu_img_create an preallocation_cmd_option
> common: qemu_img_create: add backing_file support
> rename_volume: add source && target snap
> common: add qemu_img_info helper
> common: add qemu-img measure
> storage: volume_snapshot: add $running param
> qcow2: add external snapshot support
> lvmplugin: add qcow2 snapshot
> storage : add volume_support_qemu_snapshot
>
> src/PVE/Storage.pm | 34 +-
> src/PVE/Storage/Common.pm | 116 ++++++
> src/PVE/Storage/DirPlugin.pm | 11 +
> src/PVE/Storage/ESXiPlugin.pm | 2 +-
> src/PVE/Storage/ISCSIDirectPlugin.pm | 2 +-
> src/PVE/Storage/LVMPlugin.pm | 519 ++++++++++++++++++++----
> src/PVE/Storage/LvmThinPlugin.pm | 2 +-
> src/PVE/Storage/PBSPlugin.pm | 2 +-
> src/PVE/Storage/Plugin.pm | 376 ++++++++++++-----
> src/PVE/Storage/RBDPlugin.pm | 8 +-
> src/PVE/Storage/ZFSPoolPlugin.pm | 2 +-
> src/test/Makefile | 5 +-
> src/test/run_test_lvmplugin.pl | 577 +++++++++++++++++++++++++++
> 13 files changed, 1465 insertions(+), 191 deletions(-)
> create mode 100755 src/test/run_test_lvmplugin.pl
>
> qemu-server:
>
> Alexandre Derumier (3):
> qemu_img convert : add external snapshot support
> blockdev: add backing_chain support
> qcow2: add external snapshot support
>
> src/PVE/QemuConfig.pm | 4 +-
> src/PVE/QemuServer.pm | 132 +++++--
> src/PVE/QemuServer/Blockdev.pm | 345 +++++++++++++++++-
> src/PVE/QemuServer/QemuImage.pm | 6 +-
> src/test/cfg2cmd/simple-backingchain.conf | 25 ++
> src/test/cfg2cmd/simple-backingchain.conf.cmd | 33 ++
> src/test/run_config2command_tests.pl | 47 +++
> src/test/run_qemu_img_convert_tests.pl | 59 +++
> src/test/snapshot-test.pm | 4 +-
> 9 files changed, 620 insertions(+), 35 deletions(-)
> create mode 100644 src/test/cfg2cmd/simple-backingchain.conf
> create mode 100644 src/test/cfg2cmd/simple-backingchain.conf.cmd
>
> --
> 2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [pve-devel] [PATCH-SERIES v7 pve-storage/qemu-server] add external qcow2 snapshot support
2025-07-04 11:58 ` Fabian Grünbichler
@ 2025-07-04 12:10 ` DERUMIER, Alexandre via pve-devel
[not found] ` <a1b7140a1c715ce9506f9956c296138b2373cf6b.camel@groupe-cyllene.com>
1 sibling, 0 replies; 4+ messages in thread
From: DERUMIER, Alexandre via pve-devel @ 2025-07-04 12:10 UTC (permalink / raw)
To: pve-devel, f.gruenbichler; +Cc: DERUMIER, Alexandre, t.lamprecht
[-- Attachment #1: Type: message/rfc822, Size: 19360 bytes --]
From: "DERUMIER, Alexandre" <alexandre.derumier@groupe-cyllene.com>
To: "pve-devel@lists.proxmox.com" <pve-devel@lists.proxmox.com>, "f.gruenbichler@proxmox.com" <f.gruenbichler@proxmox.com>
Cc: "t.lamprecht@proxmox.com" <t.lamprecht@proxmox.com>
Subject: Re: [pve-devel] [PATCH-SERIES v7 pve-storage/qemu-server] add external qcow2 snapshot support
Date: Fri, 4 Jul 2025 12:10:15 +0000
Message-ID: <a1b7140a1c715ce9506f9956c296138b2373cf6b.camel@groupe-cyllene.com>
-------- Message initial --------
De: Fabian Grünbichler <f.gruenbichler@proxmox.com>
À: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Cc: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>, Thomas
Lamprecht <t.lamprecht@proxmox.com>
Objet: Re: [pve-devel] [PATCH-SERIES v7 pve-storage/qemu-server] add
external qcow2 snapshot support
Date: 04/07/2025 13:58:38
> Alexandre Derumier via pve-devel <pve-devel@lists.proxmox.com> hat am
> 04.07.2025 08:44 CEST geschrieben:
> This patch series implement qcow2 external snapshot support for files
> && lvm volumes
>
> The current internal qcow2 snapshots have bad write performance
> because no metadatas can be preallocated.
>
> This is particulary visible on a shared filesystem like ocfs2 or
> gfs2.
>
> Also other bugs are freeze/lock reported by users since years on
> snapshots delete on nfs
> (The disk access seem to be frozen during all the delete duration)
>
> This also open doors for remote snapshot export-import for storage
> replication.
>>sent some feedback, mostly for the storage and qemu-server <->
>>storage interface parts,
>>haven't yet done an in-depth pass over qemu-server or end-to-end.
>>
>>in particular, the rename_volume vs rename_snapshot and the Plugin
>>snapshot file name/..
>>questions should probably be solved before the next version
Thanks ! I'll work on it next week
>>it would also be great if you could add a patch bumping API version
>>and age
I was not sure it was needed, as Fiona has just bumped it in last
patches and no .deb has been released ?
>> and
>>documenting the changes/new parameters/interfaces - also so that we
>>can ensure
>>we are on the same page and haven't overlooked anything. AFAICT it
>>should all be
>>new opt-in or backwards compatible changes, once the feedback is
>>incorporated.
ok, will do !
>
> Changelog v7:
> - rebased on fiona blockdev work
>
> MAIN TODO:
> - add snapshots tests in both pve-storage && qemu-server
> - better handle snapshot failure with multiple disks
>
> pve-storage:
>
> Alexandre Derumier (10):
> tests: add lvmplugin test
> common: add qemu_img_create an preallocation_cmd_option
> common: qemu_img_create: add backing_file support
> rename_volume: add source && target snap
> common: add qemu_img_info helper
> common: add qemu-img measure
> storage: volume_snapshot: add $running param
> qcow2: add external snapshot support
> lvmplugin: add qcow2 snapshot
> storage : add volume_support_qemu_snapshot
>
> src/PVE/Storage.pm | 34 +-
> src/PVE/Storage/Common.pm | 116 ++++++
> src/PVE/Storage/DirPlugin.pm | 11 +
> src/PVE/Storage/ESXiPlugin.pm | 2 +-
> src/PVE/Storage/ISCSIDirectPlugin.pm | 2 +-
> src/PVE/Storage/LVMPlugin.pm | 519 ++++++++++++++++++++----
> src/PVE/Storage/LvmThinPlugin.pm | 2 +-
> src/PVE/Storage/PBSPlugin.pm | 2 +-
> src/PVE/Storage/Plugin.pm | 376 ++++++++++++-----
> src/PVE/Storage/RBDPlugin.pm | 8 +-
> src/PVE/Storage/ZFSPoolPlugin.pm | 2 +-
> src/test/Makefile | 5 +-
> src/test/run_test_lvmplugin.pl | 577
> +++++++++++++++++++++++++++
> 13 files changed, 1465 insertions(+), 191 deletions(-)
> create mode 100755 src/test/run_test_lvmplugin.pl
>
> qemu-server:
>
> Alexandre Derumier (3):
> qemu_img convert : add external snapshot support
> blockdev: add backing_chain support
> qcow2: add external snapshot support
>
> src/PVE/QemuConfig.pm | 4 +-
> src/PVE/QemuServer.pm | 132 +++++--
> src/PVE/QemuServer/Blockdev.pm | 345
> +++++++++++++++++-
> src/PVE/QemuServer/QemuImage.pm | 6 +-
> src/test/cfg2cmd/simple-backingchain.conf | 25 ++
> src/test/cfg2cmd/simple-backingchain.conf.cmd | 33 ++
> src/test/run_config2command_tests.pl | 47 +++
> src/test/run_qemu_img_convert_tests.pl | 59 +++
> src/test/snapshot-test.pm | 4 +-
> 9 files changed, 620 insertions(+), 35 deletions(-)
> create mode 100644 src/test/cfg2cmd/simple-backingchain.conf
> create mode 100644 src/test/cfg2cmd/simple-backingchain.conf.cmd
>
> --
> 2.39.5
[-- Attachment #2: Type: text/plain, Size: 160 bytes --]
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 4+ messages in thread[parent not found: <a1b7140a1c715ce9506f9956c296138b2373cf6b.camel@groupe-cyllene.com>]
* Re: [pve-devel] [PATCH-SERIES v7 pve-storage/qemu-server] add external qcow2 snapshot support
[not found] ` <a1b7140a1c715ce9506f9956c296138b2373cf6b.camel@groupe-cyllene.com>
@ 2025-07-07 7:49 ` Fabian Grünbichler
0 siblings, 0 replies; 4+ messages in thread
From: Fabian Grünbichler @ 2025-07-07 7:49 UTC (permalink / raw)
To: DERUMIER, Alexandre, pve-devel; +Cc: t.lamprecht
> DERUMIER, Alexandre <alexandre.derumier@groupe-cyllene.com> hat am 04.07.2025 14:10 CEST geschrieben:
> >>it would also be great if you could add a patch bumping API version
> >>and age
> I was not sure it was needed, as Fiona has just bumped it in last
> patches and no .deb has been released ?
we could maybe skip bumping twice, that's true
> >> and
> >>documenting the changes/new parameters/interfaces - also so that we
> >>can ensure
> >>we are on the same page and haven't overlooked anything. AFAICT it
> >>should all be
> >>new opt-in or backwards compatible changes, once the feedback is
> >>incorporated.
>
> ok, will do !
but this is still needed in any case ;)
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2025-07-07 7:48 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-07-04 6:44 [pve-devel] [PATCH-SERIES v7 pve-storage/qemu-server] add external qcow2 snapshot support Alexandre Derumier via pve-devel
2025-07-04 11:58 ` Fabian Grünbichler
2025-07-04 12:10 ` DERUMIER, Alexandre via pve-devel
[not found] ` <a1b7140a1c715ce9506f9956c296138b2373cf6b.camel@groupe-cyllene.com>
2025-07-07 7:49 ` Fabian Grünbichler
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox