From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 28ADD1FF17C for ; Wed, 9 Jul 2025 18:22:50 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 01E1D111F6; Wed, 9 Jul 2025 18:22:46 +0200 (CEST) To: pve-devel@lists.proxmox.com Date: Wed, 9 Jul 2025 18:21:49 +0200 In-Reply-To: <20250709162202.2952597-1-alexandre.derumier@groupe-cyllene.com> References: <20250709162202.2952597-1-alexandre.derumier@groupe-cyllene.com> MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Alexandre Derumier via pve-devel Precedence: list Cc: Alexandre Derumier X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] [PATCH pve-storage 02/13] plugin: add qemu_img_create_qcow2_backed Content-Type: multipart/mixed; boundary="===============5744482074745303473==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============5744482074745303473== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 3D8CDD7B21 for ; Wed, 9 Jul 2025 18:22:44 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 1A4AE109FF for ; Wed, 9 Jul 2025 18:22:14 +0200 (CEST) Received: from bastiontest.odiso.net (unknown [IPv6:2a0a:1580:2000:6700::14]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 9 Jul 2025 18:22:12 +0200 (CEST) Received: from formationkvm1.odiso.net (unknown [10.11.201.57]) by bastiontest.odiso.net (Postfix) with ESMTP id D6020862E44; Wed, 9 Jul 2025 18:22:04 +0200 (CEST) Received: by formationkvm1.odiso.net (Postfix, from userid 0) id D397010CBC9B; Wed, 9 Jul 2025 18:22:04 +0200 (CEST) From: Alexandre Derumier To: pve-devel@lists.proxmox.com Subject: [PATCH pve-storage 02/13] plugin: add qemu_img_create_qcow2_backed Date: Wed, 9 Jul 2025 18:21:49 +0200 Message-Id: <20250709162202.2952597-5-alexandre.derumier@groupe-cyllene.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250709162202.2952597-1-alexandre.derumier@groupe-cyllene.com> References: <20250709162202.2952597-1-alexandre.derumier@groupe-cyllene.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.092 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_NONE 0.1 DMARC none policy HEADER_FROM_DIFFERENT_DOMAINS 0.039 From and EnvelopeFrom 2nd level mail domains are different KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [plugin.pm] and use it for plugin linked clone This also enable extended_l2=on, as it's mandatory for backing file preallocation. Preallocation was missing previously, so it should increase performance for linked clone now (around x5 in randwrite 4k) cluster_size is set to 128k, as it reduce qcow2 overhead (reduce disk, but also memory needed to cache metadatas) l2_extended is not enabled yet on base image, but it could help too to reduce overhead without impacting performance bench on 100G qcow2 file: fio --filename=/dev/sdb --direct=1 --rw=randwrite --bs=4k --iodepth=32 --ioengine=libaio --name=test fio --filename=/dev/sdb --direct=1 --rw=randread --bs=4k --iodepth=32 --ioengine=libaio --name=test base image: randwrite 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 20215 randread 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 22219 randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 20217 randread 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 21742 randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 21599 randread 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 22037 clone image with backing file: randwrite 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 3912 randread 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 21476 randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 20563 randread 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 22265 randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 18016 randread 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 21611 Signed-off-by: Alexandre Derumier --- src/PVE/Storage/Plugin.pm | 52 ++++++++++++++++++++++++++++----------- 1 file changed, 38 insertions(+), 14 deletions(-) diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm index 65a34b1..c4b4cd3 100644 --- a/src/PVE/Storage/Plugin.pm +++ b/src/PVE/Storage/Plugin.pm @@ -51,6 +51,10 @@ our $RAW_PREALLOCATION = { full => 1, }; +my $QCOW2_CLUSTERS = { + backed => ['extended_l2=on', 'cluster_size=128k'], +}; + our $MAX_VOLUMES_PER_GUEST = 1024; cfs_register_file( @@ -654,6 +658,39 @@ sub qemu_img_create { run_command($cmd, errmsg => "unable to create image"); } +=pod + +=head3 qemu_img_create_qcow2_backed + + qemu_img_create_qcow2_backed($scfg, $path, $backing_path, $backing_format) + +Create a new qemu qcow2 image C<$path> using an existing backing image C<$backing_path> with backing_format C<$backing_format>. + +=cut + +sub qemu_img_create_qcow2_backed { + my ($scfg, $path, $backing_path, $backing_format) = @_; + + my $cmd = [ + '/usr/bin/qemu-img', + 'create', + '-F', + $backing_format, + '-b', + $backing_path, + '-f', + 'qcow2', + $path, + ]; + + my $options = $QCOW2_CLUSTERS->{backed}; + + push @$options, preallocation_cmd_option($scfg, 'qcow2'); + push @$cmd, '-o', join(',', @$options) if @$options > 0; + + run_command($cmd, errmsg => "unable to create image"); +} + # Storage implementation # called during addition of storage (before the new storage config got written) @@ -941,20 +978,7 @@ sub clone_image { # Note: we use relative paths, so we need to call chdir before qemu-img eval { local $CWD = $imagedir; - - my $cmd = [ - '/usr/bin/qemu-img', - 'create', - '-b', - "../$basevmid/$basename", - '-F', - $format, - '-f', - 'qcow2', - $path, - ]; - - run_command($cmd); + qemu_img_create_qcow2_backed($scfg, $path, "../$basevmid/$basename", $format); }; my $err = $@; -- 2.39.5 --===============5744482074745303473== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============5744482074745303473==--