public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] [PATCH-SERIES v6 pve-storage/qemu-server/pve-qemu] add external qcow2 snapshot support
@ 2025-06-03  7:55 Alexandre Derumier via pve-devel
  2025-06-05 13:53 ` Fiona Ebner
  2025-06-06 10:17 ` joao.sousa via pve-devel
  0 siblings, 2 replies; 14+ messages in thread
From: Alexandre Derumier via pve-devel @ 2025-06-03  7:55 UTC (permalink / raw)
  To: pve-devel; +Cc: Alexandre Derumier

[-- Attachment #1: Type: message/rfc822, Size: 13270 bytes --]

From: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH-SERIES v6 pve-storage/qemu-server/pve-qemu] add external qcow2 snapshot support
Date: Tue,  3 Jun 2025 09:55:35 +0200
Message-ID: <20250603075558.627850-1-alexandre.derumier@groupe-cyllene.com>

This patch series implement qcow2 external snapshot support for files && lvm volumes

The current internal qcow2 snapshots have bad write performance because no metadatas can be preallocated.

This is particulary visible on a shared filesystem like ocfs2 or gfs2.

Also other bugs are freeze/lock reported by users since years on snapshots delete on nfs
(The disk access seem to be frozen during all the delete duration)

This also open doors for remote snapshot export-import for storage replication.

The pve-storage patches applied on top of Fiona qemu_blockdev_options patch series
https://lists.proxmox.com/pipermail/pve-devel/2025-May/071204.html

This new patch series enable blockdev for qemu machine > 10.0 to avoid breaking efidisk and maybe
potential migrations bug

Changelog v6:
     pve-storage:
      - use fiona qemu_blockdev_options patch
      - add helpers for qemu-img call
      - use qemu-img measure for lvm size (including l2_extended=on)
      - add a new helper volume_support_qemu_snapshot
      - fix Fabian requested changes

     qemu-server:
      - enable blockdev for machine version > 10.0
      - move blockdev code in QemuServer/Blockdev.pm
      - use Fiona qemu_blockdev_options patch for storage plugins
      - add raw size for efidisk
      - fix drivedel with backing chain
      - add tests for qemu-img convert, backing chain,....
      - fix Fabian requested changes
     pve-qemu:
      - add a fake 10.0 qemu bump for testing

MAIN TODO:
    - convert/implement all the backup code, dirty bitmap mirro, ...
    - add snapshots tests


pve-storage:

Alexandre Derumier (9):
  common: add qemu_img_create an preallocation_cmd_option
  common: add qemu_img_create_qcow2_backed
  common: add qemu_img_info helper
  common: add qemu_img_measure
  rename_volume: add source && target snap
  storage: volume_snapshot: add $running param + api bump
  qcow2: add external snapshot support
  lvmplugin: add qcow2 snapshot
  storage: add volume_support_qemu_snapshot

 src/PVE/Storage.pm                   |  28 +-
 src/PVE/Storage/BTRFSPlugin.pm       |   2 +-
 src/PVE/Storage/Common.pm            | 175 ++++++++++
 src/PVE/Storage/DirPlugin.pm         |  11 +
 src/PVE/Storage/ESXiPlugin.pm        |   2 +-
 src/PVE/Storage/GlusterfsPlugin.pm   |   2 +-
 src/PVE/Storage/ISCSIDirectPlugin.pm |   2 +-
 src/PVE/Storage/LVMPlugin.pm         | 467 +++++++++++++++++++++++----
 src/PVE/Storage/LvmThinPlugin.pm     |   2 +-
 src/PVE/Storage/PBSPlugin.pm         |   2 +-
 src/PVE/Storage/Plugin.pm            | 339 ++++++++++++++-----
 src/PVE/Storage/RBDPlugin.pm         |   8 +-
 src/PVE/Storage/ZFSPoolPlugin.pm     |   2 +-
 13 files changed, 875 insertions(+), 167 deletions(-)

qemu-server:

Alexandre Derumier (13):
  blockdev: cmdline: add blockdev syntax support
  blockdev: add support for ovmf && efidisk
  blockdev: vm_devices_list : fix block-query
  blockdev: add support to qemu_driveadd && qemu_drivedel
  blockdev: add support to qemu_block_set_io_throttle
  blockdev: add support for cdrom media eject/insert
  blockdev: add support for block_resize
  blockdev: add support for nbd_export: block-export-add
  blockdev: add blockdev_mirror
  blockdev: change aio on target if io_uring is not default.
  qemu_img convert : add external snapshot support
  blockdev: add backing_chain support
  qcow2: add external snapshot support

 PVE/QemuConfig.pm                             |   4 +-
 PVE/QemuMigrate.pm                            |   2 +-
 PVE/QemuServer.pm                             | 761 +++++++++++-------
 PVE/QemuServer/Blockdev.pm                    | 494 ++++++++++++
 PVE/QemuServer/Drive.pm                       | 201 +++++
 PVE/QemuServer/Makefile                       |   1 +
 test/MigrationTest/QemuMigrateMock.pm         |  10 +-
 test/cfg2cmd/bootorder-blockdev.conf          |  15 +
 test/cfg2cmd/bootorder-blockdev.conf.cmd      |  43 +
 test/cfg2cmd/bootorder-empty-blockdev.conf    |  15 +
 .../cfg2cmd/bootorder-empty-blockdev.conf.cmd |  43 +
 test/cfg2cmd/bootorder-empty.conf             |   1 +
 test/cfg2cmd/bootorder-empty.conf.cmd         |   2 +-
 test/cfg2cmd/bootorder-legacy.conf            |   1 +
 test/cfg2cmd/bootorder-legacy.conf.cmd        |   2 +-
 test/cfg2cmd/bootorder.conf                   |   1 +
 test/cfg2cmd/bootorder.conf.cmd               |   2 +-
 .../cputype-icelake-client-deprecation.conf   |   3 -
 ...putype-icelake-client-deprecation.conf.cmd |   7 +-
 .../custom-cpu-model-defaults.conf.cmd        |   2 +-
 test/cfg2cmd/efi-raw-template.conf.cmd        |   7 +-
 test/cfg2cmd/efi-raw.conf                     |   1 +
 test/cfg2cmd/efi-raw.conf.cmd                 |   2 +-
 test/cfg2cmd/efi-secboot-and-tpm-q35.conf.cmd |   7 +-
 test/cfg2cmd/efi-secboot-and-tpm.conf.cmd     |   7 +-
 test/cfg2cmd/efidisk-on-rbd.conf              |   1 +
 test/cfg2cmd/efidisk-on-rbd.conf.cmd          |   2 +-
 test/cfg2cmd/i440fx-viommu-virtio.conf.cmd    |   2 +-
 test/cfg2cmd/ide-blockdev.conf                |  14 +
 test/cfg2cmd/ide-blockdev.conf.cmd            |  46 ++
 test/cfg2cmd/ide.conf                         |   1 +
 test/cfg2cmd/ide.conf.cmd                     |   2 +-
 .../cfg2cmd/memory-hotplug-hugepages.conf.cmd |   2 +-
 test/cfg2cmd/memory-hotplug.conf.cmd          |   2 +-
 test/cfg2cmd/memory-hugepages-1g.conf.cmd     |   2 +-
 test/cfg2cmd/memory-hugepages-2m.conf.cmd     |   2 +-
 test/cfg2cmd/minimal-defaults.conf.cmd        |   2 +-
 test/cfg2cmd/netdev-7.1-multiqueues.conf.cmd  |   2 +-
 test/cfg2cmd/netdev-7.1.conf.cmd              |   2 +-
 test/cfg2cmd/q35-ide-blockdev.conf            |  15 +
 test/cfg2cmd/q35-ide-blockdev.conf.cmd        |  45 ++
 test/cfg2cmd/q35-ide.conf                     |   2 +-
 test/cfg2cmd/q35-ide.conf.cmd                 |   3 +-
 .../q35-linux-hostpci-mapping.conf.cmd        |   7 +-
 .../q35-linux-hostpci-multifunction.conf.cmd  |   7 +-
 .../q35-linux-hostpci-template.conf.cmd       |  10 +-
 ...q35-linux-hostpci-x-pci-overrides.conf.cmd |   9 +-
 test/cfg2cmd/q35-linux-hostpci.conf.cmd       |   7 +-
 test/cfg2cmd/q35-simple.conf.cmd              |   7 +-
 test/cfg2cmd/q35-viommu-intel.conf.cmd        |   2 +-
 test/cfg2cmd/q35-viommu-virtio.conf.cmd       |   2 +-
 test/cfg2cmd/seabios_serial.conf              |   4 -
 test/cfg2cmd/seabios_serial.conf.cmd          |   7 +-
 test/cfg2cmd/sev-es.conf.cmd                  |   7 +-
 test/cfg2cmd/sev-snp.conf.cmd                 |   2 +-
 test/cfg2cmd/sev-std.conf.cmd                 |   7 +-
 test/cfg2cmd/simple-backingchain.conf         |  21 +
 test/cfg2cmd/simple-backingchain.conf.cmd     |  30 +
 test/cfg2cmd/simple-btrfs.conf                |   1 +
 test/cfg2cmd/simple-btrfs.conf.cmd            |   2 +-
 .../simple-disk-passthrough-blockdev.conf     |  15 +
 .../simple-disk-passthrough-blockdev.conf.cmd |  40 +
 test/cfg2cmd/simple-disk-passthrough.conf     |   1 +
 test/cfg2cmd/simple-disk-passthrough.conf.cmd |   2 +-
 test/cfg2cmd/simple-rbd-blockdev.conf         |  15 +
 test/cfg2cmd/simple-rbd-blockdev.conf.cmd     |  38 +
 test/cfg2cmd/simple-rbd.conf                  |   1 +
 test/cfg2cmd/simple-rbd.conf.cmd              |   2 +-
 test/cfg2cmd/simple-virtio-blk-blockdev.conf  |  13 +
 .../simple-virtio-blk-blockdev.conf.cmd       |  35 +
 test/cfg2cmd/simple-virtio-blk.conf           |   1 +
 test/cfg2cmd/simple-virtio-blk.conf.cmd       |   2 +-
 .../simple-zfs-over-iscsi-blockdev.conf       |  14 +
 .../simple-zfs-over-iscsi-blockdev.conf.cmd   |  35 +
 test/cfg2cmd/simple-zfs-over-iscsi.conf       |   1 +
 test/cfg2cmd/simple-zfs-over-iscsi.conf.cmd   |   2 +-
 test/cfg2cmd/simple1-blockdev.conf            |  14 +
 test/cfg2cmd/simple1-blockdev.conf.cmd        |  35 +
 test/cfg2cmd/simple1-template.conf.cmd        |  13 +-
 test/cfg2cmd/simple1.conf                     |   1 +
 test/cfg2cmd/simple1.conf.cmd                 |   2 +-
 test/cfg2cmd/vnc-clipboard-spice.conf.cmd     |   2 +-
 test/cfg2cmd/vnc-clipboard-std.conf.cmd       |   2 +-
 test/run_config2command_tests.pl              |  52 ++
 test/run_qemu_img_convert_tests.pl            |  44 +
 test/snapshot-test.pm                         |   4 +-
 86 files changed, 1905 insertions(+), 394 deletions(-)
 create mode 100644 PVE/QemuServer/Blockdev.pm
 create mode 100644 test/cfg2cmd/bootorder-blockdev.conf
 create mode 100644 test/cfg2cmd/bootorder-blockdev.conf.cmd
 create mode 100644 test/cfg2cmd/bootorder-empty-blockdev.conf
 create mode 100644 test/cfg2cmd/bootorder-empty-blockdev.conf.cmd
 create mode 100644 test/cfg2cmd/ide-blockdev.conf
 create mode 100644 test/cfg2cmd/ide-blockdev.conf.cmd
 create mode 100644 test/cfg2cmd/q35-ide-blockdev.conf
 create mode 100644 test/cfg2cmd/q35-ide-blockdev.conf.cmd
 create mode 100644 test/cfg2cmd/simple-backingchain.conf
 create mode 100644 test/cfg2cmd/simple-backingchain.conf.cmd
 create mode 100644 test/cfg2cmd/simple-disk-passthrough-blockdev.conf
 create mode 100644 test/cfg2cmd/simple-disk-passthrough-blockdev.conf.cmd
 create mode 100644 test/cfg2cmd/simple-rbd-blockdev.conf
 create mode 100644 test/cfg2cmd/simple-rbd-blockdev.conf.cmd
 create mode 100644 test/cfg2cmd/simple-virtio-blk-blockdev.conf
 create mode 100644 test/cfg2cmd/simple-virtio-blk-blockdev.conf.cmd
 create mode 100644 test/cfg2cmd/simple-zfs-over-iscsi-blockdev.conf
 create mode 100644 test/cfg2cmd/simple-zfs-over-iscsi-blockdev.conf.cmd
 create mode 100644 test/cfg2cmd/simple1-blockdev.conf
 create mode 100644 test/cfg2cmd/simple1-blockdev.conf.cmd

pve-qemu:

Alexandre Derumier (1):
  add fake 10.0

 debian/changelog                              |   6 +
 .../extra/0001-bump-version-to-10.0.0.patch   |  19 ++
 ...438a42f7a178504347456b9e50e3d2c50478.patch | 247 ++++++++++++++++++
 ...b0606879ddeddf04bcf8440aabcfe1393b9c.patch | 120 +++++++++
 debian/patches/series                         |   3 +
 5 files changed, 395 insertions(+)
 create mode 100644 debian/patches/extra/0001-bump-version-to-10.0.0.patch
 create mode 100644 debian/patches/extra/0a7c438a42f7a178504347456b9e50e3d2c50478.patch
 create mode 100644 debian/patches/extra/7956b0606879ddeddf04bcf8440aabcfe1393b9c.patch


-- 
2.39.5



[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2025-06-16  8:35 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-06-03  7:55 [pve-devel] [PATCH-SERIES v6 pve-storage/qemu-server/pve-qemu] add external qcow2 snapshot support Alexandre Derumier via pve-devel
2025-06-05 13:53 ` Fiona Ebner
2025-06-05 14:08   ` DERUMIER, Alexandre via pve-devel
     [not found]   ` <0aa862ab988ff9df50f4637f3712f137f23c34c2.camel@groupe-cyllene.com>
2025-06-05 14:19     ` Fiona Ebner
2025-06-06 10:17 ` joao.sousa via pve-devel
2025-06-10  8:54   ` Fiona Ebner
2025-06-11 21:28     ` joao.sousa via pve-devel
     [not found]     ` <5sqpzsnyzmfvvoco3ilo6tg3pcpomhle7h5dsnmbsbdjobpr5a@rqfsbsas36e2>
2025-06-12  7:02       ` DERUMIER, Alexandre via pve-devel
2025-06-12  8:12       ` Fiona Ebner
2025-06-13 10:25         ` joao.sousa via pve-devel
     [not found]         ` <zwkygc6va6jmieho2h7gqsx4idrgrxvd7et2kfbi3vyddhjk7c@kfiwufb7auvv>
2025-06-16  8:34           ` Fiona Ebner
     [not found]       ` <8cbed06bbe3db00e96670dec1e240123b946a917.camel@groupe-cyllene.com>
2025-06-12 11:33         ` DERUMIER, Alexandre via pve-devel
     [not found]         ` <a7185aa585b414710be6e09729adda57ed3787b8.camel@groupe-cyllene.com>
2025-06-13 16:42           ` joao.sousa--- via pve-devel
     [not found]           ` <qss36oltvd4gz5tiss7dqono5h2zmsgwmodj4cqkroboizypwj@kbyz3mymh75a>
2025-06-16  5:50             ` DERUMIER, Alexandre via pve-devel

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal