From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id 894581FF16F
for <inbox@lore.proxmox.com>; Tue, 27 May 2025 10:49:19 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 20128F1FA;
Tue, 27 May 2025 10:49:33 +0200 (CEST)
Message-ID: <31ef5e5b-8994-49de-9634-eb59a60b78ff@proxmox.com>
Date: Tue, 27 May 2025 10:48:59 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
References: <20250522135304.2513284-1-alexandre.derumier@groupe-cyllene.com>
<mailman.560.1747921998.394.pve-devel@lists.proxmox.com>
Content-Language: en-US
From: Fiona Ebner <f.ebner@proxmox.com>
In-Reply-To: <mailman.560.1747921998.394.pve-devel@lists.proxmox.com>
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.034 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
information. [common.pm, plugin.pm]
Subject: Re: [pve-devel] [PATCH pve-storage 2/2] common: qemu_img_create:
add backing_file support
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
Am 22.05.25 um 15:53 schrieb Alexandre Derumier via pve-devel:
>
> and use it for plugin linked clone
>
> This also enable extended_l2=on, as it's mandatory for backing file
> preallocation.
>
> Preallocation was missing previously, so it should increase performance
> for linked clone now (around x5 in randwrite 4k)
>
> cluster_size is set to 128k, as it reduce qcow2 overhead (reduce disk,
> but also memory needed to cache metadatas)
>
> l2_extended is not enabled yet on base image, but it could help too
> to reduce overhead without impacting performance
>
> bench on 100G qcow2 file:
>
> fio --filename=/dev/sdb --direct=1 --rw=randwrite --bs=4k --iodepth=32 --ioengine=libaio --name=test
> fio --filename=/dev/sdb --direct=1 --rw=randread --bs=4k --iodepth=32 --ioengine=libaio --name=test
>
> base image:
>
> randwrite 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 20215
> randread 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 22219
> randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 20217
> randread 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 21742
> randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 21599
> randread 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 22037
>
> clone image with backing file:
>
> randwrite 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 3912
> randread 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 21476
> randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 20563
> randread 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 22265
> randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 18016
> randread 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 21611
>
> Signed-off-by: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
In general, the approach is fine by me :)
> ---
> src/PVE/Storage/Common.pm | 17 +++++++++++++----
> src/PVE/Storage/Plugin.pm | 5 +----
> 2 files changed, 14 insertions(+), 8 deletions(-)
>
> diff --git a/src/PVE/Storage/Common.pm b/src/PVE/Storage/Common.pm
> index 0770d70..01a995a 100644
> --- a/src/PVE/Storage/Common.pm
> +++ b/src/PVE/Storage/Common.pm
> @@ -148,14 +148,23 @@ sub preallocation_cmd_option {
> }
>
> sub qemu_img_create {
> - my ($scfg, $fmt, $size, $path) = @_;
> + my ($scfg, $fmt, $size, $path, $backing_path) = @_;
> +
> + die "size can't be specified if backing file is used" if $size && $backing_path;
I'd prefer having a second dedicated function
qemu_img_create_qcow2_with_backing() rather than an interface where some
parameters need to be undef depending on how you want to use it. It also
doesn't require a format parameter, because we only allow it for qcow2.
> my $cmd = ['/usr/bin/qemu-img', 'create'];
>
> - my $prealloc_opt = preallocation_cmd_option($scfg, $fmt);
> - push @$cmd, '-o', $prealloc_opt if defined($prealloc_opt);
> + my $options = [];
> +
> + if($backing_path) {
> + push @$cmd, '-b', $backing_path, '-F', 'qcow2';
You're now using "-F qcow2" and...(continued below)
> + push @$options, 'extended_l2=on','cluster_size=128k';
> + };
>
> - push @$cmd, '-f', $fmt, $path, "${size}K";
> + push @$options, preallocation_cmd_option($scfg, $fmt);
> + push @$cmd, '-o', join(',', @$options) if @$options > 0;
> + push @$cmd, '-f', $fmt, $path;
..."-f $fmt"...
> + push @$cmd, "${size}K" if $size;
>
> run_command($cmd, errmsg => "unable to create image");
> }
> diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
> index 7030e4e..a3856ca 100644
> --- a/src/PVE/Storage/Plugin.pm
> +++ b/src/PVE/Storage/Plugin.pm
> @@ -847,10 +847,7 @@ sub clone_image {
> eval {
> local $CWD = $imagedir;
>
> - my $cmd = ['/usr/bin/qemu-img', 'create', '-b', "../$basevmid/$basename",
> - '-F', $format, '-f', 'qcow2', $path];
...but the old command was using "-F $format -f qcow2"!
> -
> - run_command($cmd);
> + PVE::Storage::Common::qemu_img_create($scfg, $format, undef, $path, "../$basevmid/$basename");
> };
> my $err = $@;
>
> --
> 2.39.5
>
>
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel