From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <pve-devel-bounces@lists.proxmox.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 5E0921FF165 for <inbox@lore.proxmox.com>; Thu, 22 May 2025 15:38:30 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id F3E9D363AA; Thu, 22 May 2025 15:38:30 +0200 (CEST) Date: Thu, 22 May 2025 15:37:56 +0200 (CEST) From: =?UTF-8?Q?Fabian_Gr=C3=BCnbichler?= <f.gruenbichler@proxmox.com> To: Proxmox VE development discussion <pve-devel@lists.proxmox.com> Message-ID: <1661381270.18235.1747921076530@webmail.proxmox.com> In-Reply-To: <mailman.456.1747650234.394.pve-devel@lists.proxmox.com> References: <mailman.456.1747650234.394.pve-devel@lists.proxmox.com> MIME-Version: 1.0 X-Priority: 3 Importance: Normal X-Mailer: Open-Xchange Mailer v7.10.6-Rev75 X-Originating-Client: open-xchange-appsuite X-SPAM-LEVEL: Spam detection results: 0 AWL 0.045 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [common.pm, proxmox.com, glusterfsplugin.pm, plugin.pm] Subject: Re: [pve-devel] [PATCH pve-storage 0/2] move qemu_img_create to common helpers and enable preallocation on backed images X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/> List-Post: <mailto:pve-devel@lists.proxmox.com> List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe> Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com> Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com> > Alexandre Derumier via pve-devel <pve-devel@lists.proxmox.com> hat am 19.05.2025 12:23 CEST geschrieben: > This is part of my work on qcow2 external snapshot, but could improve current qcow2 linked clone > > This patch serie move qemu_img_create to common helpers, > and enable preallocation on backed_image to increase performance did you maybe forget to send the actual patches? I can't find them in my inbox, and they aren't on lore either: https://lore.proxmox.com/pve-devel/mailman.556.1747917864.394.pve-devel@lists.proxmox.com/t/#u > > This require l2_extended=on on the backed image > > I have done some benchmarks on localssd with 100gb qcow2, the performance in randwrite 4k is 5x faster > > some presentation of l2_extended=on are available here > https://www.youtube.com/watch?v=zJetcfDVFNw > https://www.youtube.com/watch?v=NfgLCdtkRus > > I don't have enabled it for base image, as I think that Fabian see performance regression some month ago. > but I don't see performance difference in my bench. (can you could test on your side again ?) > > It could help to reduce qcow2 overhead on disk, > and allow to keep more metadatas in memory for bigger image, as qemu default memory l2_cache_size=1MB) > https://www.ibm.com/products/tutorials/how-to-tune-qemu-l2-cache-size-and-qcow2-cluster-size > Maybe more test with bigger image (>1TB) could be done too to see if it's help > > > > bench on 100G qcow2 file: > > fio --filename=/dev/sdb --direct=1 --rw=randwrite --bs=4k --iodepth=32 --ioengine=libaio --name=test > fio --filename=/dev/sdb --direct=1 --rw=randread --bs=4k --iodepth=32 --ioengine=libaio --name=test > > base image: > > randwrite 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 20215 > randread 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 22219 > randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 20217 > randread 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 21742 > randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 21599 > randread 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 22037 > > linked clone image with backing file: > > randwrite 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 3912 > randread 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 21476 > randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 20563 > randread 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 22265 > randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 18016 > randread 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 21611 > > Alexandre Derumier (2): > common: add qemu_img_create an preallocation_cmd_option > common: qemu_img_create: add backing_file support > > src/PVE/Storage/Common.pm | 62 ++++++++++++++++++++++++++++++ > src/PVE/Storage/GlusterfsPlugin.pm | 2 +- > src/PVE/Storage/Plugin.pm | 52 +------------------------ > 3 files changed, 65 insertions(+), 51 deletions(-) > > -- > 2.39.5 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel