Re: [pve-devel] [PATCH pve-storage 03/10] common: qemu_img_create: add backing_file support

["Fabian Grünbichler" <f.gruenbichler@proxmox.com>]

> Alexandre Derumier via pve-devel <pve-devel@lists.proxmox.com> hat am 04.07.2025 08:44 CEST geschrieben:
> and use it for plugin linked clone
> 
> This also enable extended_l2=on, as it's mandatory for backing file
> preallocation.
> 
> Preallocation was missing previously, so it should increase performance
> for linked clone now (around x5 in randwrite 4k)
> 
> cluster_size is set to 128k, as it reduce qcow2 overhead (reduce disk,
> but also memory needed to cache metadatas)

should we make this configurable?

> l2_extended is not enabled yet on base image, but it could help too
> to reduce overhead without impacting performance
> 
> bench on 100G qcow2 file:
> 
> fio --filename=/dev/sdb --direct=1 --rw=randwrite --bs=4k --iodepth=32 --ioengine=libaio --name=test
> fio --filename=/dev/sdb --direct=1 --rw=randread --bs=4k --iodepth=32 --ioengine=libaio --name=test
> 
> base image:
> 
> randwrite 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 20215
> randread 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 22219
> randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 20217
> randread 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 21742
> randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 21599
> randread 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 22037
> 
> clone image with backing file:
> 
> randwrite 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 3912
> randread 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 21476
> randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 20563
> randread 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 22265
> randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 18016
> randread 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 21611
> 
> Signed-off-by: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
> ---
>  src/PVE/Storage/Common.pm | 17 +++++++++++++----
>  src/PVE/Storage/Plugin.pm | 17 +++--------------
>  2 files changed, 16 insertions(+), 18 deletions(-)
> 
> diff --git a/src/PVE/Storage/Common.pm b/src/PVE/Storage/Common.pm
> index 29f2e52..78e5320 100644
> --- a/src/PVE/Storage/Common.pm
> +++ b/src/PVE/Storage/Common.pm
> @@ -150,14 +150,23 @@ sub preallocation_cmd_option {
>  }
>  
>  sub qemu_img_create {
> -    my ($scfg, $fmt, $size, $path) = @_;
> +    my ($scfg, $fmt, $size, $path, $backing_path) = @_;
> +
> +    die "size can't be specified if backing file is used" if $size && $backing_path;

should we assert that $backing_path is only used with $fmt eq 'qcow2'?

>  
>      my $cmd = ['/usr/bin/qemu-img', 'create'];
>  
> -    my $prealloc_opt = preallocation_cmd_option($scfg, $fmt);
> -    push @$cmd, '-o', $prealloc_opt if defined($prealloc_opt);
> +    my $options = [];
> +
> +    if ($backing_path) {
> +        push @$cmd, '-b', $backing_path, '-F', 'qcow2';

and then use $fmt here as well?

> +        push @$options, 'extended_l2=on', 'cluster_size=128k';
> +    }
>  
> -    push @$cmd, '-f', $fmt, $path, "${size}K";
> +    push @$options, preallocation_cmd_option($scfg, $fmt);
> +    push @$cmd, '-o', join(',', @$options) if @$options > 0;
> +    push @$cmd, '-f', $fmt, $path;
> +    push @$cmd, "${size}K" if $size;
>  
>      run_command($cmd, errmsg => "unable to create image");
>  }
> diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
> index 80bb077..c35d5e5 100644
> --- a/src/PVE/Storage/Plugin.pm
> +++ b/src/PVE/Storage/Plugin.pm
> @@ -880,20 +880,9 @@ sub clone_image {
>      # Note: we use relative paths, so we need to call chdir before qemu-img
>      eval {
>          local $CWD = $imagedir;
> -
> -        my $cmd = [
> -            '/usr/bin/qemu-img',
> -            'create',
> -            '-b',
> -            "../$basevmid/$basename",
> -            '-F',
> -            $format,
> -            '-f',
> -            'qcow2',
> -            $path,
> -        ];
> -
> -        run_command($cmd);
> +        PVE::Storage::Common::qemu_img_create(
> +            $scfg, $format, undef, $path, "../$basevmid/$basename",
> +        );
>      };
>      my $err = $@;
>  
> -- 
> 2.39.5


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel