From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
by lore.proxmox.com (Postfix) with ESMTPS id 4C30E1FF15F
for <inbox@lore.proxmox.com>; Mon, 16 Dec 2024 10:13:37 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 9CB5A7398;
Mon, 16 Dec 2024 10:13:17 +0100 (CET)
To: pve-devel@lists.proxmox.com
Date: Mon, 16 Dec 2024 10:12:14 +0100
MIME-Version: 1.0
Message-ID: <mailman.214.1734340397.332.pve-devel@lists.proxmox.com>
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Post: <mailto:pve-devel@lists.proxmox.com>
From: Alexandre Derumier via pve-devel <pve-devel@lists.proxmox.com>
Precedence: list
Cc: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
X-Mailman-Version: 2.1.29
X-BeenThere: pve-devel@lists.proxmox.com
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
Subject: [pve-devel] [PATCH-SERIES v3 pve-storage/qemu-server/pve-qemu] add
external qcow2 snapshot support
Content-Type: multipart/mixed; boundary="===============2588629413616728112=="
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
--===============2588629413616728112==
Content-Type: message/rfc822
Content-Disposition: inline
Return-Path: <root@formationkvm1.odiso.net>
X-Original-To: pve-devel@lists.proxmox.com
Delivered-To: pve-devel@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id CDE55C30EA
for <pve-devel@lists.proxmox.com>; Mon, 16 Dec 2024 10:13:15 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 0987A6CF0
for <pve-devel@lists.proxmox.com>; Mon, 16 Dec 2024 10:12:44 +0100 (CET)
Received: from bastiontest.odiso.net (unknown [IPv6:2a0a:1580:2000:6700::14])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pve-devel@lists.proxmox.com>; Mon, 16 Dec 2024 10:12:39 +0100 (CET)
Received: from formationkvm1.odiso.net (unknown [10.11.201.57])
by bastiontest.odiso.net (Postfix) with ESMTP id 1579685A3FF;
Mon, 16 Dec 2024 10:12:32 +0100 (CET)
Received: by formationkvm1.odiso.net (Postfix, from userid 0)
id 480DF1084C68; Mon, 16 Dec 2024 10:12:31 +0100 (CET)
From: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH-SERIES v3 pve-storage/qemu-server/pve-qemu] add external qcow2 snapshot support
Date: Mon, 16 Dec 2024 10:12:14 +0100
Message-Id: <20241216091229.3142660-1-alexandre.derumier@groupe-cyllene.com>
X-Mailer: git-send-email 2.39.5
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.038 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_NONE 0.1 DMARC none policy
HEADER_FROM_DIFFERENT_DOMAINS 0.248 From and EnvelopeFrom 2nd level mail domains are different
KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods
RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_NONE 0.001 SPF: sender does not publish an SPF Record
This patch series implement qcow2 external snapshot support for files && lvm volumes
The current internal qcow2 snapshots have bad write performance because no metadatas can be preallocated.
This is particulary visible on a shared filesystem like ocfs2 or gfs2.
Also other bugs are freeze/lock reported by users since years on snapshots delete on nfs
(The disk access seem to be frozen during all the delete duration)
This also open doors for remote snapshot export-import for storage replication.
Changelog v3:
storage:
- snapshots files now have the name of the snapshot, and "current" snapshot is the vm volname
- allow only qcow2 format for base image for simplication
- merge snapshot code in lvmplugin (qcow2 format auto enable snapshot)
- the code is a lot more simple now
qemu-server:
- convertion -drive to modern -blockdev
This is needed for blockdev-reopen, where we need to create && switch to snapshot files with same cache,aio,...
This is also needed to live rename snapshot files (to keep volname for current snapshot)
I have implemented && tested:
- disk create,delete,resize,convert,drive_mirror, hotplug,unplug,nbd mirror,cdrom insert/eject
- block protocol: file,block_device,rbd,nbd,glusterfs
Note that it's currently incomplete:
- proxmox backup/restore code need to be converted to blockdev, help is needed
- iscsi:// path is not yet implemented (I'll look for the v4)
- efi still in drive format (not blocking, but I'll look for v4)
The live migration between -drive ---> --blockdev seem to work without breaking, and seem to be
transparent for the guest ok, so I think we could try to target pve9 ?
storage.cfg example:
dir: local2
path /var/liv/vz
content snippets,vztmpl,backup,images,iso,rootdir
snapext 1
lvm:test
vgname test
content images
pve-storage:
Alexandre Derumier (3):
qcow2: add external snapshot support
lvmplugin: add qcow2 snapshot
storage: vdisk_free: remove external snapshots
src/PVE/Storage.pm | 18 ++-
src/PVE/Storage/DirPlugin.pm | 1 +
src/PVE/Storage/LVMPlugin.pm | 231 ++++++++++++++++++++++++++---
src/PVE/Storage/Plugin.pm | 207 ++++++++++++++++++++++----
src/test/run_test_zfspoolplugin.pl | 18 +++
5 files changed, 424 insertions(+), 51 deletions(-)
Alexandre Derumier (11):
blockdev: cmdline: convert drive to blockdev syntax
blockdev: fix cfg2cmd tests
blockdev : convert qemu_driveadd && qemu_drivedel
blockdev: vm_devices_list : fix block-query
blockdev: convert cdrom media eject/insert
blockdev: block_resize: convert to blockdev
blockdev: nbd_export: block-export-add : use drive-$id for nodename
blockdev: convert drive_mirror to blockdev_mirror
blockdev: mirror: change aio on target if io_uring is not default.
blockdev: add backing_chain support
qcow2: add external snapshot support
PVE/QemuConfig.pm | 4 +-
PVE/QemuMigrate.pm | 2 +-
PVE/QemuServer.pm | 946 ++++++++++++++----
test/cfg2cmd/bootorder-empty.conf.cmd | 12 +-
test/cfg2cmd/bootorder-legacy.conf.cmd | 12 +-
test/cfg2cmd/bootorder.conf.cmd | 12 +-
...putype-icelake-client-deprecation.conf.cmd | 6 +-
test/cfg2cmd/ide.conf.cmd | 23 +-
test/cfg2cmd/pinned-version-pxe-pve.conf.cmd | 6 +-
test/cfg2cmd/pinned-version-pxe.conf.cmd | 6 +-
test/cfg2cmd/pinned-version.conf.cmd | 6 +-
test/cfg2cmd/q35-ide.conf.cmd | 23 +-
.../q35-linux-hostpci-template.conf.cmd | 3 +-
test/cfg2cmd/seabios_serial.conf.cmd | 6 +-
...imple-balloon-free-page-reporting.conf.cmd | 6 +-
test/cfg2cmd/simple-btrfs.conf.cmd | 6 +-
test/cfg2cmd/simple-virtio-blk.conf.cmd | 6 +-
test/cfg2cmd/simple1-template.conf.cmd | 11 +-
test/cfg2cmd/simple1.conf.cmd | 6 +-
19 files changed, 830 insertions(+), 272 deletions(-)
pve-qemu:
add block-commit-replaces option patch
...052-block-commit-add-replaces-option.patch | 137 ++++++++++++++++++
debian/patches/series | 1 +
2 files changed, 138 insertions(+)
create mode 100644 debian/patches/pve/0052-block-commit-add-replaces-option.patch
--
2.39.5
--===============2588629413616728112==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
--===============2588629413616728112==--