From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 7EB8A710CD for ; Wed, 8 Sep 2021 10:11:57 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 6882E23022 for ; Wed, 8 Sep 2021 10:11:57 +0200 (CEST) Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 45F342300C for ; Wed, 8 Sep 2021 10:11:54 +0200 (CEST) Received: by mail-wr1-x42c.google.com with SMTP id u16so1913163wrn.5 for ; Wed, 08 Sep 2021 01:11:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=odiso-com.20150623.gappssmtp.com; s=20150623; h=message-id:subject:from:to:date:in-reply-to:references:user-agent :mime-version; bh=4QNB5QTLVcgzjRY1xo3dyBHNCsXHzp86TbOOp+sy1mw=; b=EqvtIaYiZg0LJUCs9r5iBqDkMb2q37QOluBvjcKmKAN+/+oqqvqhEj6rauR7guS9oe IQcyZArXUbs9XzTGuNNM4JNfWlYxV3XPsjFy2HiXJ0Wj1S+QLBrbKPCxqUKI5tQhOfls H6qfKJ/YP8JqGwFxeqJsZAR5woMuCqwLKAxnI6yRVdiT8FlAXHX6mgAIqd+cIkoyUfOj rGav466DyE1drfunoLBCKDUzOUlTEPRdTnc/h9JrwWtlt9a0uFseqho4tOAbhrah5Q8N 47KZpgAklnUWbg+r1VlWWnjflBpC4nqke0Q9KrskP/FWrJjy5rneDXtxet+2MEZpImc3 kCqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:date:in-reply-to :references:user-agent:mime-version; bh=4QNB5QTLVcgzjRY1xo3dyBHNCsXHzp86TbOOp+sy1mw=; b=Xf0vpFosenH9JpRG1A5tVRnYlFHaHWmHJZYtNTynv2xmFTzZlqpfuJlExm/XR6VeTl 1XheHGUVak+i7VkN0mUIlleOvRW0ZJNaRCXs9eT3T+Z3kzOkakeD7dHPca+qGMkmZHu4 fDakS+/R6NTBcaxd1bN9peeqHz48NBXdBrBQrsdWH/J6iJ8V9INA81ACi9K1sGY7dLOW 1PbDUFHSVkUPD8zcDasOP25h4F6j/g0nMaLHZYzM4gT0KPtl19peyxj/5gjKI9TcZXaT UxFk2gS7S3EEal4GrLOiU7wZcNmfwkiJaRMG2gadBI8Om7mRgfob+1sZ7Ya73BMa25Qq W4Iw== X-Gm-Message-State: AOAM531hh4OjBfApvH4qGywSvrlMuweSbfBPrgeLO8vg7vfuIu7Y2io5 urm6AQmOqEPlSpf3fjcxX+CY+xncRlCt6A== X-Google-Smtp-Source: ABdhPJz6hw9Bw+IK97teKEFKu66Yvjizpzmt+3cR+BN6er5HKhq3xIZwoL8574QYlZszqwqA/sSC7A== X-Received: by 2002:a05:6000:1566:: with SMTP id 6mr2564073wrz.274.1631088707615; Wed, 08 Sep 2021 01:11:47 -0700 (PDT) Received: from ?IPv6:2a0a:1580:0:1::100c? (ovpn1.odiso.net. [2a0a:1580:2000::3f]) by smtp.gmail.com with ESMTPSA id c14sm1369505wme.6.2021.09.08.01.11.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 01:11:47 -0700 (PDT) Message-ID: From: alexandre derumier To: Proxmox VE development discussion Date: Wed, 08 Sep 2021 10:11:45 +0200 In-Reply-To: <20210906131542.178844-2-l.stechauner@proxmox.com> References: <20210906131542.178844-1-l.stechauner@proxmox.com> <20210906131542.178844-2-l.stechauner@proxmox.com> User-Agent: Evolution 3.40.4 MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.463 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature HTML_MESSAGE 0.001 HTML included in message RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URI_NOVOWEL 0.5 URI hostname has long non-vowel sequence Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [pve-devel] [PATCH storage 1/1] fix #3580: plugins: make preallocation mode selectable for qcow2 and raw images X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Sep 2021 08:11:57 -0000 Hi, it can be done too with ceph rbd with "rbd create ... –thick-provision" Le lundi 06 septembre 2021 à 15:15 +0200, Lorenz Stechauner a écrit : > the plugins for file based storages >  * BTRFS >  * CIFS >  * Dir >  * Glusterfs >  * NFS > now allow the option 'preallocation'. > > 'preallocation' can have four values: >  * default >  * off >  * metadata >  * falloc >  * full > see man pages for `qemu-img` for what these mean exactly. [0] > > the defualt value was chosen to be >  * qcow2: metadata (as previously) >  * raw: off (I was unable to find any documentation on this, so >     could only test this and found, that 'off' was the most >     fitting.) > > when using 'metadata' as preallocation mode, for raw images 'off' > is used. > > [0] > https://qemu.readthedocs.io/en/latest/system/images.html#disk-image-file-formats > > Signed-off-by: Lorenz Stechauner > --- >  PVE/Storage/BTRFSPlugin.pm     |  1 + >  PVE/Storage/CIFSPlugin.pm      |  1 + >  PVE/Storage/DirPlugin.pm       |  1 + >  PVE/Storage/GlusterfsPlugin.pm |  4 ++- >  PVE/Storage/NFSPlugin.pm       |  1 + >  PVE/Storage/Plugin.pm          | 46 > +++++++++++++++++++++++++++++++++- >  6 files changed, 52 insertions(+), 2 deletions(-) > > diff --git a/PVE/Storage/BTRFSPlugin.pm b/PVE/Storage/BTRFSPlugin.pm > index fe42082..31a2954 100644 > --- a/PVE/Storage/BTRFSPlugin.pm > +++ b/PVE/Storage/BTRFSPlugin.pm > @@ -73,6 +73,7 @@ sub options { >         is_mountpoint => { optional => 1 }, >         nocow => { optional => 1 }, >         mkdir => { optional => 1 }, > +       preallocation => { optional => 1 }, >         # TODO: The new variant of mkdir with  `populate` vs > `create`... >      }; >  } > diff --git a/PVE/Storage/CIFSPlugin.pm b/PVE/Storage/CIFSPlugin.pm > index 0221069..2d94413 100644 > --- a/PVE/Storage/CIFSPlugin.pm > +++ b/PVE/Storage/CIFSPlugin.pm > @@ -140,6 +140,7 @@ sub options { >         smbversion => { optional => 1}, >         mkdir => { optional => 1 }, >         bwlimit => { optional => 1 }, > +       preallocation => { optional => 1 }, >      }; >  } >   > diff --git a/PVE/Storage/DirPlugin.pm b/PVE/Storage/DirPlugin.pm > index 2267f11..3eeec98 100644 > --- a/PVE/Storage/DirPlugin.pm > +++ b/PVE/Storage/DirPlugin.pm > @@ -59,6 +59,7 @@ sub options { >         mkdir => { optional => 1 }, >         is_mountpoint => { optional => 1 }, >         bwlimit => { optional => 1 }, > +       preallocation => { optional => 1 }, >     }; >  } >   > diff --git a/PVE/Storage/GlusterfsPlugin.pm > b/PVE/Storage/GlusterfsPlugin.pm > index ea4df82..d8d2b88 100644 > --- a/PVE/Storage/GlusterfsPlugin.pm > +++ b/PVE/Storage/GlusterfsPlugin.pm > @@ -137,6 +137,7 @@ sub options { >         format => { optional => 1 }, >         mkdir => { optional => 1 }, >         bwlimit => { optional => 1 }, > +       preallocation => { optional => 1 }, >      }; >  } >   > @@ -260,7 +261,8 @@ sub alloc_image { >   >      my $cmd = ['/usr/bin/qemu-img', 'create']; >   > -    push @$cmd, '-o', 'preallocation=metadata' if $fmt eq 'qcow2'; > +    my $prealloc_opt = > PVE::Storage::Plugin::preallocation_cmd_option($scfg, $fmt); > +    push @$cmd, '-o', $prealloc_opt if defined($prealloc_opt); >   >      push @$cmd, '-f', $fmt, $volumepath, "${size}K"; >   > diff --git a/PVE/Storage/NFSPlugin.pm b/PVE/Storage/NFSPlugin.pm > index 39bf15a..21b288a 100644 > --- a/PVE/Storage/NFSPlugin.pm > +++ b/PVE/Storage/NFSPlugin.pm > @@ -90,6 +90,7 @@ sub options { >         format => { optional => 1 }, >         mkdir => { optional => 1 }, >         bwlimit => { optional => 1 }, > +       preallocation => { optional => 1 }, >      }; >  } >   > diff --git a/PVE/Storage/Plugin.pm b/PVE/Storage/Plugin.pm > index b1865cb..4924525 100644 > --- a/PVE/Storage/Plugin.pm > +++ b/PVE/Storage/Plugin.pm > @@ -41,6 +41,19 @@ our @SHARED_STORAGE = ( >      'pbs', >  ); >   > +our $QCOW2_PREALLOCATION = { > +    off => 1, > +    metadata => 1, > +    falloc => 1, > +    full => 1, > +}; > + > +our $RAW_PREALLOCATION = { > +    off => 1, > +    falloc => 1, > +    full => 1, > +}; > + >  our $MAX_VOLUMES_PER_GUEST = 1024; >   >  cfs_register_file ('storage.cfg', > @@ -150,6 +163,11 @@ my $defaultData = { >             type => 'string', format => 'pve-storage-format', >             optional => 1, >         }, > +       preallocation => { > +           description => "Preallocation mode for raw and qcow2 > images.", > +           type => 'string', enum => ['default', 'off', 'metadata', > 'falloc', 'full'], > +           optional => 1, > +       }, >      }, >  }; >   > @@ -762,7 +780,8 @@ sub alloc_image { >      } else { >         my $cmd = ['/usr/bin/qemu-img', 'create']; >   > -       push @$cmd, '-o', 'preallocation=metadata' if $fmt eq > 'qcow2'; > +       my $prealloc_opt = preallocation_cmd_option($scfg, $fmt); > +       push @$cmd, '-o', $prealloc_opt if defined($prealloc_opt); >   >         push @$cmd, '-f', $fmt, $path, "${size}K"; >   > @@ -1484,4 +1503,29 @@ sub volume_import_formats { >      return (); >  } >   > +sub preallocation_cmd_option { > +    my ($scfg, $fmt) = @_; > + > +    my $prealloc = $scfg->{preallocation}; > + > +    $prealloc = undef if $prealloc eq 'default'; > + > +    if ($fmt eq 'qcow2') { > +       $prealloc = $prealloc // 'metadata'; > + > +       die "preallocation mode '$prealloc' not supported by format > '$fmt'\n" if !$QCOW2_PREALLOCATION->{$prealloc}; > + > +       return "preallocation=$prealloc"; > +    } elsif ($fmt eq 'raw') { > +       $prealloc = $prealloc // 'off'; > +       $prealloc = 'off' if $prealloc eq 'metadata'; > + > +       die "preallocation mode '$prealloc' not supported by format > '$fmt'\n" if !$RAW_PREALLOCATION->{$prealloc}; > + > +       return "preallocation=$prealloc"; > +    } > + > +    return undef; > +} > + >  1;