From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 59C211FF164 for ; Fri, 4 Jul 2025 08:44:43 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 4171F1FE49; Fri, 4 Jul 2025 08:45:19 +0200 (CEST) To: pve-devel@lists.proxmox.com Date: Fri, 4 Jul 2025 08:44:54 +0200 MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Alexandre Derumier via pve-devel Precedence: list Cc: Alexandre Derumier X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] [PATCH-SERIES v7 pve-storage/qemu-server] add external qcow2 snapshot support Content-Type: multipart/mixed; boundary="===============1870919488679442350==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============1870919488679442350== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id CF0F5D5376 for ; Fri, 4 Jul 2025 08:45:16 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id AB4991FD77 for ; Fri, 4 Jul 2025 08:45:16 +0200 (CEST) Received: from bastiontest.odiso.net (unknown [185.151.190.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Fri, 4 Jul 2025 08:45:15 +0200 (CEST) Received: from formationkvm1.odiso.net (unknown [10.11.201.57]) by bastiontest.odiso.net (Postfix) with ESMTP id EB952860F12; Fri, 4 Jul 2025 08:45:08 +0200 (CEST) Received: by formationkvm1.odiso.net (Postfix, from userid 0) id D4BFA1159334; Fri, 4 Jul 2025 08:45:08 +0200 (CEST) From: Alexandre Derumier To: pve-devel@lists.proxmox.com Subject: [PATCH-SERIES v7 pve-storage/qemu-server] add external qcow2 snapshot support Date: Fri, 4 Jul 2025 08:44:54 +0200 Message-Id: <20250704064507.511884-1-alexandre.derumier@groupe-cyllene.com> X-Mailer: git-send-email 2.39.5 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.081 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_NONE 0.1 DMARC none policy HEADER_FROM_DIFFERENT_DOMAINS 0.001 From and EnvelopeFrom 2nd level mail domains are different KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.218 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [lvmthinplugin.pm,plugin.pm,iscsidirectplugin.pm,storage.pm,blockdev.pm,dirplugin.pm,rbdplugin.pm,lvmplugin.pm,snapshot-test.pm,qemuserver.pm,qemuconfig.pm,pbsplugin.pm,common.pm,zfspoolplugin.pm,esxiplugin.pm,qemuimage.pm] This patch series implement qcow2 external snapshot support for files && lvm volumes The current internal qcow2 snapshots have bad write performance because no metadatas can be preallocated. This is particulary visible on a shared filesystem like ocfs2 or gfs2. Also other bugs are freeze/lock reported by users since years on snapshots delete on nfs (The disk access seem to be frozen during all the delete duration) This also open doors for remote snapshot export-import for storage replication. Changelog v7: - rebased on fiona blockdev work MAIN TODO: - add snapshots tests in both pve-storage && qemu-server - better handle snapshot failure with multiple disks pve-storage: Alexandre Derumier (10): tests: add lvmplugin test common: add qemu_img_create an preallocation_cmd_option common: qemu_img_create: add backing_file support rename_volume: add source && target snap common: add qemu_img_info helper common: add qemu-img measure storage: volume_snapshot: add $running param qcow2: add external snapshot support lvmplugin: add qcow2 snapshot storage : add volume_support_qemu_snapshot src/PVE/Storage.pm | 34 +- src/PVE/Storage/Common.pm | 116 ++++++ src/PVE/Storage/DirPlugin.pm | 11 + src/PVE/Storage/ESXiPlugin.pm | 2 +- src/PVE/Storage/ISCSIDirectPlugin.pm | 2 +- src/PVE/Storage/LVMPlugin.pm | 519 ++++++++++++++++++++---- src/PVE/Storage/LvmThinPlugin.pm | 2 +- src/PVE/Storage/PBSPlugin.pm | 2 +- src/PVE/Storage/Plugin.pm | 376 ++++++++++++----- src/PVE/Storage/RBDPlugin.pm | 8 +- src/PVE/Storage/ZFSPoolPlugin.pm | 2 +- src/test/Makefile | 5 +- src/test/run_test_lvmplugin.pl | 577 +++++++++++++++++++++++++++ 13 files changed, 1465 insertions(+), 191 deletions(-) create mode 100755 src/test/run_test_lvmplugin.pl qemu-server: Alexandre Derumier (3): qemu_img convert : add external snapshot support blockdev: add backing_chain support qcow2: add external snapshot support src/PVE/QemuConfig.pm | 4 +- src/PVE/QemuServer.pm | 132 +++++-- src/PVE/QemuServer/Blockdev.pm | 345 +++++++++++++++++- src/PVE/QemuServer/QemuImage.pm | 6 +- src/test/cfg2cmd/simple-backingchain.conf | 25 ++ src/test/cfg2cmd/simple-backingchain.conf.cmd | 33 ++ src/test/run_config2command_tests.pl | 47 +++ src/test/run_qemu_img_convert_tests.pl | 59 +++ src/test/snapshot-test.pm | 4 +- 9 files changed, 620 insertions(+), 35 deletions(-) create mode 100644 src/test/cfg2cmd/simple-backingchain.conf create mode 100644 src/test/cfg2cmd/simple-backingchain.conf.cmd -- 2.39.5 --===============1870919488679442350== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============1870919488679442350==--