From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id D74091FF15E
for <inbox@lore.proxmox.com>; Tue, 11 Mar 2025 11:31:33 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id DF7DFC9CB;
Tue, 11 Mar 2025 11:29:48 +0100 (CET)
To: pve-devel@lists.proxmox.com
Date: Tue, 11 Mar 2025 11:28:48 +0100
MIME-Version: 1.0
Message-ID: <mailman.958.1741688988.293.pve-devel@lists.proxmox.com>
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Post: <mailto:pve-devel@lists.proxmox.com>
From: Alexandre Derumier via pve-devel <pve-devel@lists.proxmox.com>
Precedence: list
Cc: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
X-Mailman-Version: 2.1.29
X-BeenThere: pve-devel@lists.proxmox.com
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
Subject: [pve-devel] [PATCH-SERIES v4 pve-storage/qemu-server/pve-qemu] add
external qcow2 snapshot support
Content-Type: multipart/mixed; boundary="===============7866423736416695117=="
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
--===============7866423736416695117==
Content-Type: message/rfc822
Content-Disposition: inline
Return-Path: <root@formationkvm1.odiso.net>
X-Original-To: pve-devel@lists.proxmox.com
Delivered-To: pve-devel@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id EA347D586E
for <pve-devel@lists.proxmox.com>; Tue, 11 Mar 2025 11:29:47 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id CB6EFC027
for <pve-devel@lists.proxmox.com>; Tue, 11 Mar 2025 11:29:17 +0100 (CET)
Received: from bastiontest.odiso.net (unknown [IPv6:2a0a:1580:2000:6700::14])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pve-devel@lists.proxmox.com>; Tue, 11 Mar 2025 11:29:15 +0100 (CET)
Received: from formationkvm1.odiso.net (unknown [10.11.201.57])
by bastiontest.odiso.net (Postfix) with ESMTP id 041DE86087F;
Tue, 11 Mar 2025 11:29:09 +0100 (CET)
Received: by formationkvm1.odiso.net (Postfix, from userid 0)
id 44842119292E; Tue, 11 Mar 2025 11:29:09 +0100 (CET)
From: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH-SERIES v4 pve-storage/qemu-server/pve-qemu] add external qcow2 snapshot support
Date: Tue, 11 Mar 2025 11:28:48 +0100
Message-Id: <20250311102905.2680524-1-alexandre.derumier@groupe-cyllene.com>
X-Mailer: git-send-email 2.39.5
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.147 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_NONE 0.1 DMARC none policy
HEADER_FROM_DIFFERENT_DOMAINS 0.001 From and EnvelopeFrom 2nd level mail domains are different
KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods
RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_NONE 0.001 SPF: sender does not publish an SPF Record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [dirplugin.pm,qemumigrate.pm,plugin.pm,qemumigratemock.pm,lvmplugin.pm,drive.pm,qemuconfig.pm,qemuserver.pm,storage.pm]
This patch series implement qcow2 external snapshot support for files && lvm volumes
The current internal qcow2 snapshots have bad write performance because no metadatas can be preallocated.
This is particulary visible on a shared filesystem like ocfs2 or gfs2.
Also other bugs are freeze/lock reported by users since years on snapshots delete on nfs
(The disk access seem to be frozen during all the delete duration)
This also open doors for remote snapshot export-import for storage replication.
Changelog v4:
- use blocknode fixed names with sha1+base62
- use block-stream for snapshot delete
- qemu : use throttle qom object (no more x-throttle )
- various cleanup && refactoring from fabian comments
MAIN TODO:
- convert/implement all the backup code, dirty dirty bitmap, ...
- iscsi blockdev syntax
- ceph keyring as blockdev option
- unit tests
storage.cfg example:
dir: local2
path /var/liv/vz
content snippets,vztmpl,backup,images,iso,rootdir
snapext 1
lvm:test
vgname test
content images
pve-storage:
Alexandre Derumier (5):
qcow2: add external snapshot support
lvmplugin: add qcow2 snapshot
storage: vdisk_free: remove external snapshots
lvm: lvrename helper: allow path
lvm: add lvremove helper
src/PVE/Storage.pm | 22 ++-
src/PVE/Storage/DirPlugin.pm | 1 +
src/PVE/Storage/LVMPlugin.pm | 251 ++++++++++++++++++++++++++---
src/PVE/Storage/Plugin.pm | 232 +++++++++++++++++++++-----
src/test/run_test_zfspoolplugin.pl | 18 +++
5 files changed, 456 insertions(+), 68 deletions(-)
qemu-server:
Alexandre Derumier (11):
blockdev: cmdline: convert drive to blockdev syntax
blockdev : convert qemu_driveadd && qemu_drivedel
replace qemu_block_set_io_throttle with qom-set throttlegroup limits
blockdev: vm_devices_list : fix block-query
blockdev: convert cdrom media eject/insert
blockdev: block_resize: convert to blockdev
blockdev: nbd_export: block-export-add : use drive-$id for nodename
blockdev: convert drive_mirror to blockdev_mirror
blockdev: change aio on target if io_uring is not default.
blockdev: add backing_chain support
qcow2: add external snapshot support
PVE/QemuConfig.pm | 4 +-
PVE/QemuMigrate.pm | 2 +-
PVE/QemuServer.pm | 632 +++++++++---------
PVE/QemuServer/Drive.pm | 415 ++++++++++++
test/MigrationTest/QemuMigrateMock.pm | 10 +-
test/cfg2cmd/bootorder-empty.conf.cmd | 12 +-
test/cfg2cmd/bootorder-legacy.conf.cmd | 12 +-
test/cfg2cmd/bootorder.conf.cmd | 12 +-
...putype-icelake-client-deprecation.conf.cmd | 6 +-
test/cfg2cmd/ide.conf.cmd | 23 +-
test/cfg2cmd/pinned-version-pxe-pve.conf.cmd | 6 +-
test/cfg2cmd/pinned-version-pxe.conf.cmd | 6 +-
test/cfg2cmd/pinned-version.conf.cmd | 6 +-
test/cfg2cmd/q35-ide.conf.cmd | 23 +-
.../q35-linux-hostpci-template.conf.cmd | 3 +-
test/cfg2cmd/seabios_serial.conf.cmd | 6 +-
...imple-balloon-free-page-reporting.conf.cmd | 6 +-
test/cfg2cmd/simple-btrfs.conf.cmd | 6 +-
test/cfg2cmd/simple-virtio-blk.conf.cmd | 6 +-
test/cfg2cmd/simple1-template.conf.cmd | 11 +-
test/cfg2cmd/simple1-throttle.conf | 14 +
test/cfg2cmd/simple1-throttle.conf.cmd | 33 +
test/cfg2cmd/simple1.conf.cmd | 6 +-
23 files changed, 886 insertions(+), 374 deletions(-)
create mode 100644 test/cfg2cmd/simple1-throttle.conf
create mode 100644 test/cfg2cmd/simple1-throttle.conf.cmd
pve-qemu:
Alexandre Derumier (1):
add block-commit-replaces option patch
...051-block-commit-add-replaces-option.patch | 137 ++++++++++++++++++
debian/patches/series | 1 +
2 files changed, 138 insertions(+)
create mode 100644 debian/patches/pve/0051-block-commit-add-replaces-option.patch
--
2.39.5
--===============7866423736416695117==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
--===============7866423736416695117==--