From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <f.ebner@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 48B8D69A18
for <pve-devel@lists.proxmox.com>; Tue, 14 Sep 2021 11:45:00 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 39A78B856
for <pve-devel@lists.proxmox.com>; Tue, 14 Sep 2021 11:45:00 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[94.136.29.106])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS id 0F92DB848
for <pve-devel@lists.proxmox.com>; Tue, 14 Sep 2021 11:44:59 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id CC5574481B
for <pve-devel@lists.proxmox.com>; Tue, 14 Sep 2021 11:44:58 +0200 (CEST)
To: pve-devel@lists.proxmox.com, Lorenz Stechauner <l.stechauner@proxmox.com>
References: <20210906131542.178844-1-l.stechauner@proxmox.com>
<20210906131542.178844-2-l.stechauner@proxmox.com>
From: Fabian Ebner <f.ebner@proxmox.com>
Message-ID: <291d8cca-a5c2-0a95-ad30-d1999b6e42f4@proxmox.com>
Date: Tue, 14 Sep 2021 11:44:56 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210906131542.178844-2-l.stechauner@proxmox.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-SPAM-LEVEL: Spam detection results: 0
AWL 1.075 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
NICE_REPLY_A -1.969 Looks like a legit reply (A)
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
information. [glusterfsplugin.pm, btrfsplugin.pm, dirplugin.pm, cifsplugin.pm,
plugin.pm, readthedocs.io, nfsplugin.pm]
URI_NOVOWEL 0.5 URI hostname has long non-vowel sequence
Subject: Re: [pve-devel] [PATCH storage 1/1] fix #3580: plugins: make
preallocation mode selectable for qcow2 and raw images
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Tue, 14 Sep 2021 09:45:00 -0000
Am 06.09.21 um 15:15 schrieb Lorenz Stechauner:
> the plugins for file based storages
> * BTRFS
> * CIFS
> * Dir
> * Glusterfs
> * NFS
> now allow the option 'preallocation'.
>
> 'preallocation' can have four values:
> * default
> * off
> * metadata
> * falloc
> * full
> see man pages for `qemu-img` for what these mean exactly. [0]
>
> the defualt value was chosen to be
> * qcow2: metadata (as previously)
> * raw: off (I was unable to find any documentation on this, so
> could only test this and found, that 'off' was the most
> fitting.)
>
> when using 'metadata' as preallocation mode, for raw images 'off'
> is used.
>
> [0] https://qemu.readthedocs.io/en/latest/system/images.html#disk-image-file-formats
>
> Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
> ---
> PVE/Storage/BTRFSPlugin.pm | 1 +
> PVE/Storage/CIFSPlugin.pm | 1 +
> PVE/Storage/DirPlugin.pm | 1 +
> PVE/Storage/GlusterfsPlugin.pm | 4 ++-
> PVE/Storage/NFSPlugin.pm | 1 +
> PVE/Storage/Plugin.pm | 46 +++++++++++++++++++++++++++++++++-
> 6 files changed, 52 insertions(+), 2 deletions(-)
>
> diff --git a/PVE/Storage/BTRFSPlugin.pm b/PVE/Storage/BTRFSPlugin.pm
> index fe42082..31a2954 100644
> --- a/PVE/Storage/BTRFSPlugin.pm
> +++ b/PVE/Storage/BTRFSPlugin.pm
> @@ -73,6 +73,7 @@ sub options {
> is_mountpoint => { optional => 1 },
> nocow => { optional => 1 },
> mkdir => { optional => 1 },
> + preallocation => { optional => 1 },
> # TODO: The new variant of mkdir with `populate` vs `create`...
> };
> }
> diff --git a/PVE/Storage/CIFSPlugin.pm b/PVE/Storage/CIFSPlugin.pm
> index 0221069..2d94413 100644
> --- a/PVE/Storage/CIFSPlugin.pm
> +++ b/PVE/Storage/CIFSPlugin.pm
> @@ -140,6 +140,7 @@ sub options {
> smbversion => { optional => 1},
> mkdir => { optional => 1 },
> bwlimit => { optional => 1 },
> + preallocation => { optional => 1 },
> };
> }
>
> diff --git a/PVE/Storage/DirPlugin.pm b/PVE/Storage/DirPlugin.pm
> index 2267f11..3eeec98 100644
> --- a/PVE/Storage/DirPlugin.pm
> +++ b/PVE/Storage/DirPlugin.pm
> @@ -59,6 +59,7 @@ sub options {
> mkdir => { optional => 1 },
> is_mountpoint => { optional => 1 },
> bwlimit => { optional => 1 },
> + preallocation => { optional => 1 },
> };
> }
>
> diff --git a/PVE/Storage/GlusterfsPlugin.pm b/PVE/Storage/GlusterfsPlugin.pm
> index ea4df82..d8d2b88 100644
> --- a/PVE/Storage/GlusterfsPlugin.pm
> +++ b/PVE/Storage/GlusterfsPlugin.pm
> @@ -137,6 +137,7 @@ sub options {
> format => { optional => 1 },
> mkdir => { optional => 1 },
> bwlimit => { optional => 1 },
> + preallocation => { optional => 1 },
> };
> }
>
> @@ -260,7 +261,8 @@ sub alloc_image {
>
> my $cmd = ['/usr/bin/qemu-img', 'create'];
>
> - push @$cmd, '-o', 'preallocation=metadata' if $fmt eq 'qcow2';
> + my $prealloc_opt = PVE::Storage::Plugin::preallocation_cmd_option($scfg, $fmt);
> + push @$cmd, '-o', $prealloc_opt if defined($prealloc_opt);
>
> push @$cmd, '-f', $fmt, $volumepath, "${size}K";
>
> diff --git a/PVE/Storage/NFSPlugin.pm b/PVE/Storage/NFSPlugin.pm
> index 39bf15a..21b288a 100644
> --- a/PVE/Storage/NFSPlugin.pm
> +++ b/PVE/Storage/NFSPlugin.pm
> @@ -90,6 +90,7 @@ sub options {
> format => { optional => 1 },
> mkdir => { optional => 1 },
> bwlimit => { optional => 1 },
> + preallocation => { optional => 1 },
> };
> }
>
> diff --git a/PVE/Storage/Plugin.pm b/PVE/Storage/Plugin.pm
> index b1865cb..4924525 100644
> --- a/PVE/Storage/Plugin.pm
> +++ b/PVE/Storage/Plugin.pm
> @@ -41,6 +41,19 @@ our @SHARED_STORAGE = (
> 'pbs',
> );
>
> +our $QCOW2_PREALLOCATION = {
> + off => 1,
> + metadata => 1,
> + falloc => 1,
> + full => 1,
> +};
> +
> +our $RAW_PREALLOCATION = {
> + off => 1,
> + falloc => 1,
> + full => 1,
> +};
> +
> our $MAX_VOLUMES_PER_GUEST = 1024;
>
> cfs_register_file ('storage.cfg',
> @@ -150,6 +163,11 @@ my $defaultData = {
> type => 'string', format => 'pve-storage-format',
> optional => 1,
> },
> + preallocation => {
> + description => "Preallocation mode for raw and qcow2 images.",
> + type => 'string', enum => ['default', 'off', 'metadata', 'falloc', 'full'],
> + optional => 1,
It might be worth to mention that 'metadata' for raw means 'off' in the
description.
Any reason for an explicit 'default' rather then having 'option not set
at all' be the default? You could also write
default => 'metadata',
as part of the option's properties.
> + },
> },
> };
>
> @@ -762,7 +780,8 @@ sub alloc_image {
> } else {
> my $cmd = ['/usr/bin/qemu-img', 'create'];
>
> - push @$cmd, '-o', 'preallocation=metadata' if $fmt eq 'qcow2';
> + my $prealloc_opt = preallocation_cmd_option($scfg, $fmt);
> + push @$cmd, '-o', $prealloc_opt if defined($prealloc_opt);
>
> push @$cmd, '-f', $fmt, $path, "${size}K";
>
> @@ -1484,4 +1503,29 @@ sub volume_import_formats {
> return ();
> }
>
> +sub preallocation_cmd_option {
I suppose this function should not be part of the Plugin API, so it
should either be above the
# Storage implementation
comment or made into a private helper using
my $preallocation_cmd_option = sub
> + my ($scfg, $fmt) = @_;
> +
> + my $prealloc = $scfg->{preallocation};
> +
> + $prealloc = undef if $prealloc eq 'default';
Will lead to a warning when $prealloc is already undef:
Use of uninitialized value $prealloc in string eq
> +
> + if ($fmt eq 'qcow2') {
> + $prealloc = $prealloc // 'metadata';
> +
> + die "preallocation mode '$prealloc' not supported by format '$fmt'\n" if !$QCOW2_PREALLOCATION->{$prealloc};
Style nit: line too long
Nit: I'm wondering if the check is worth it. It should not be possible
to trigger with the current implementation, and even if there were a bad
value and no check here, qemu-img would still complain afterwards. But
I'm also fine with the check.
> +
> + return "preallocation=$prealloc";
> + } elsif ($fmt eq 'raw') {
> + $prealloc = $prealloc // 'off';
> + $prealloc = 'off' if $prealloc eq 'metadata';
> +
> + die "preallocation mode '$prealloc' not supported by format '$fmt'\n" if !$RAW_PREALLOCATION->{$prealloc};
Same here.
> +
> + return "preallocation=$prealloc";
> + }
> +
> + return undef;
Nit: 'return;' should be preferred over 'return undef;' (behaves
differently when called in list context).
> +}
> +
> 1;
>