From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 4DC111FF17C for ; Wed, 9 Jul 2025 18:22:42 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id DC9BE11185; Wed, 9 Jul 2025 18:22:44 +0200 (CEST) To: pve-devel@lists.proxmox.com Date: Wed, 9 Jul 2025 18:21:45 +0200 MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Alexandre Derumier via pve-devel Precedence: list Cc: Alexandre Derumier X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] [PATCH-SERIES v8 pve-storage/qemu-server] add external qcow2 snapshot support Content-Type: multipart/mixed; boundary="===============0522460355055785270==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============0522460355055785270== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 2B716D7BA5 for ; Wed, 9 Jul 2025 18:22:43 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 11CFF109EE for ; Wed, 9 Jul 2025 18:22:13 +0200 (CEST) Received: from bastiontest.odiso.net (unknown [185.151.190.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 9 Jul 2025 18:22:10 +0200 (CEST) Received: from formationkvm1.odiso.net (unknown [10.11.201.57]) by bastiontest.odiso.net (Postfix) with ESMTP id 20D6C860F12; Wed, 9 Jul 2025 18:22:04 +0200 (CEST) Received: by formationkvm1.odiso.net (Postfix, from userid 0) id 10B5910CBC9B; Wed, 9 Jul 2025 18:22:04 +0200 (CEST) From: Alexandre Derumier To: pve-devel@lists.proxmox.com Subject: [PATCH-SERIES v8 pve-storage/qemu-server] add external qcow2 snapshot support Date: Wed, 9 Jul 2025 18:21:45 +0200 Message-Id: <20250709162202.2952597-1-alexandre.derumier@groupe-cyllene.com> X-Mailer: git-send-email 2.39.5 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.012 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_NONE 0.1 DMARC none policy HEADER_FROM_DIFFERENT_DOMAINS 0.039 From and EnvelopeFrom 2nd level mail domains are different KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [esxiplugin.pm,iscsidirectplugin.pm,common.pm,qemuimage.pm,blockdev.pm,plugin.pm,cifsplugin.pm,snapshot-test.pm,dirplugin.pm,storage.pm,zfsplugin.pm,lvmplugin.pm,btrfsplugin.pm,pbsplugin.pm,nfsplugin.pm,rbdplugin.pm,zfspoolplugin.pm,qemuconfig.pm,lvmthinplugin.pm,qemuserver.pm] This patch series implement qcow2 external snapshot support for files && lvm volumes The current internal qcow2 snapshots have bad write performance because no metadatas can be preallocated. This is particulary visible on a shared filesystem like ocfs2 or gfs2. Also other bugs are freeze/lock reported by users since years on snapshots delete on nfs (The disk access seem to be frozen during all the delete duration) This also open doors for remote snapshot export-import for storage replication. Changelog v8: storage : - fix Fabian comments - add rename_snapshot - add qemu_resize - plugin: restrict volnames - plugin: use 'external-snapshots' instead 'snapext' qemu-server: - fix efi test with wrong volnames vm-disk-100-0.raw - use rename_snapshot MAIN TODO: - add snapshots tests in both pve-storage && qemu-server - better handle snapshot failure with multiple disks pve-storage: Alexandre Derumier (13): plugin: add qemu_img_create plugin: add qemu_img_create_qcow2_backed plugin: add qemu_img_info plugin: add qemu_img_measure plugin: add qemu_img_resize rbd && zfs : create_base : remove $running param from volume_snapshot storage: volume_snapshot: add $running param storage: add rename_snapshot method storage: add volume_support_qemu_snapshot plugin: fix volname parsing qcow2: add external snapshot support lvmplugin: add qcow2 snapshot tests: add lvmplugin test ApiChangeLog | 15 + src/PVE/Storage.pm | 45 ++- src/PVE/Storage/BTRFSPlugin.pm | 6 + src/PVE/Storage/CIFSPlugin.pm | 1 + src/PVE/Storage/Common.pm | 33 ++ src/PVE/Storage/DirPlugin.pm | 11 + src/PVE/Storage/ESXiPlugin.pm | 8 +- src/PVE/Storage/ISCSIDirectPlugin.pm | 2 +- src/PVE/Storage/LVMPlugin.pm | 571 +++++++++++++++++++++----- src/PVE/Storage/LvmThinPlugin.pm | 8 +- src/PVE/Storage/NFSPlugin.pm | 1 + src/PVE/Storage/PBSPlugin.pm | 2 +- src/PVE/Storage/Plugin.pm | 518 +++++++++++++++++++++--- src/PVE/Storage/RBDPlugin.pm | 18 +- src/PVE/Storage/ZFSPlugin.pm | 4 +- src/PVE/Storage/ZFSPoolPlugin.pm | 8 +- src/test/Makefile | 5 +- src/test/run_test_lvmplugin.pl | 577 +++++++++++++++++++++++++++ 18 files changed, 1662 insertions(+), 171 deletions(-) create mode 100755 src/test/run_test_lvmplugin.pl qemu-server : Alexandre Derumier (4): qemu_img convert : add external snapshot support blockdev: add backing_chain support qcow2: add external snapshot support tests: fix efi vm-disk-100-0.raw -> vm-100-disk-0.raw src/PVE/QemuConfig.pm | 4 +- src/PVE/QemuServer.pm | 132 +++++-- src/PVE/QemuServer/Blockdev.pm | 345 +++++++++++++++++- src/PVE/QemuServer/QemuImage.pm | 6 +- src/test/cfg2cmd/efi-raw-old.conf | 2 +- src/test/cfg2cmd/efi-raw-old.conf.cmd | 2 +- src/test/cfg2cmd/efi-raw-template.conf | 2 +- src/test/cfg2cmd/efi-raw-template.conf.cmd | 2 +- src/test/cfg2cmd/efi-raw.conf | 2 +- src/test/cfg2cmd/efi-raw.conf.cmd | 2 +- src/test/cfg2cmd/efi-secboot-and-tpm-q35.conf | 2 +- .../cfg2cmd/efi-secboot-and-tpm-q35.conf.cmd | 2 +- src/test/cfg2cmd/efi-secboot-and-tpm.conf | 2 +- src/test/cfg2cmd/efi-secboot-and-tpm.conf.cmd | 2 +- src/test/cfg2cmd/sev-es.conf | 2 +- src/test/cfg2cmd/sev-es.conf.cmd | 2 +- src/test/cfg2cmd/sev-std.conf | 2 +- src/test/cfg2cmd/sev-std.conf.cmd | 2 +- src/test/cfg2cmd/simple-backingchain.conf | 25 ++ src/test/cfg2cmd/simple-backingchain.conf.cmd | 33 ++ src/test/run_config2command_tests.pl | 47 +++ src/test/run_qemu_img_convert_tests.pl | 59 +++ src/test/snapshot-test.pm | 4 +- 23 files changed, 634 insertions(+), 49 deletions(-) create mode 100644 src/test/cfg2cmd/simple-backingchain.conf create mode 100644 src/test/cfg2cmd/simple-backingchain.conf.cmd -- 2.39.5 --===============0522460355055785270== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============0522460355055785270==--