From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <pve-devel-bounces@lists.proxmox.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id EECEC1FF15D for <inbox@lore.proxmox.com>; Thu, 19 Sep 2024 15:19:29 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 5F7021653A; Thu, 19 Sep 2024 15:19:31 +0200 (CEST) To: pve-devel@lists.proxmox.com Date: Thu, 19 Sep 2024 15:19:13 +0200 MIME-Version: 1.0 Message-ID: <mailman.18.1726751969.332.pve-devel@lists.proxmox.com> List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com> List-Post: <mailto:pve-devel@lists.proxmox.com> From: Alexandre Derumier via pve-devel <pve-devel@lists.proxmox.com> Precedence: list Cc: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com> X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/> Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com> List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help> Subject: [pve-devel] [PATCH SERIES pve-storage/qemu-server] add external qcow2 snapshot support Content-Type: multipart/mixed; boundary="===============3372862063908586886==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com> --===============3372862063908586886== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: <root@formationkvm1.odiso.net> X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 2F03AC04D9 for <pve-devel@lists.proxmox.com>; Thu, 19 Sep 2024 15:19:29 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 180F3162F9 for <pve-devel@lists.proxmox.com>; Thu, 19 Sep 2024 15:19:29 +0200 (CEST) Received: from bastiontest.odiso.net (unknown [IPv6:2a0a:1580:2000:6700::14]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for <pve-devel@lists.proxmox.com>; Thu, 19 Sep 2024 15:19:27 +0200 (CEST) Received: from formationkvm1.odiso.net (unknown [10.11.201.57]) by bastiontest.odiso.net (Postfix) with ESMTP id F1DB682E01A; Thu, 19 Sep 2024 15:19:19 +0200 (CEST) Received: by formationkvm1.odiso.net (Postfix, from userid 0) id 61E7F1101725; Thu, 19 Sep 2024 15:19:17 +0200 (CEST) From: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com> To: pve-devel@lists.proxmox.com Subject: [PATCH SERIES pve-storage/qemu-server] add external qcow2 snapshot support Date: Thu, 19 Sep 2024 15:19:13 +0200 Message-Id: <20240919131915.2624831-1-alexandre.derumier@groupe-cyllene.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.045 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_NONE 0.1 DMARC none policy HEADER_FROM_DIFFERENT_DOMAINS 0.248 From and EnvelopeFrom 2nd level mail domains are different KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record This patch series implement qcow2 external snapshot support. (for directory plugin currently,but it can be extended to nfs, gluster,...) The current internal qcow2 snapshots have a lot of performance problems. I have tested through nfs and also local filesystem https://lore.proxmox.com/pve-devel/6a44716a-88bc-4523-b210-d67031917d8f@proxmox.com/t/ I see that Fiona don't have same result than me, but I got something like 200~300iops vs 20000 iops with 4k randwrite when a snapshot exist. The result is even worst on a shared filesystem like ocfs2 or gfs2. (around 80 iops) I think (I'm not 100% sure) this is mostly because metadatas are not preallocated anymore with qcow2 internal snap. With external snapshot, I almost don't have any performance impact when a snapshot exist. Also other bugs are freeze/lock reported by users since years on snapshots delete on nfs https://forum.proxmox.com/threads/snapshot-removal-jams-the-vm.111648/ (The disk access seem to be frozen during all the delete duration) External qcow2 snapshots also allow snapshot of raw devices. Currently working with : .raw files ,so 0 performance impact without snapshots. But this could be extended for lvm shared volumes. (I have another patch series for thick lvm dynamic extend, but if we could have at minimum snapshot working, it could great :) I have tested: snasphot, snap rollback, snap delete, clone, move disk, rename disk. (online && offline) pve-storage: Alexandre Derumier (1): add external snasphot support src/PVE/Storage/DirPlugin.pm | 1 + src/PVE/Storage/Plugin.pm | 220 +++++++++++++++++++++++++++++++---- 2 files changed, 196 insertions(+), 25 deletions(-) qemu-server: Alexandre Derumier (1): implement external snapshot PVE/QemuServer.pm | 74 +++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 62 insertions(+), 12 deletions(-) -- 2.39.2 --===============3372862063908586886== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============3372862063908586886==--