From: Alexandre Derumier via pve-devel <pve-devel@lists.proxmox.com>
To: pve-devel@lists.proxmox.com
Cc: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
Subject: [pve-devel] [PATCH pve-storage 03/10] common: qemu_img_create: add backing_file support
Date: Fri, 4 Jul 2025 08:44:59 +0200 [thread overview]
Message-ID: <mailman.964.1751611551.395.pve-devel@lists.proxmox.com> (raw)
In-Reply-To: <20250704064507.511884-1-alexandre.derumier@groupe-cyllene.com>
[-- Attachment #1: Type: message/rfc822, Size: 6464 bytes --]
From: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH pve-storage 03/10] common: qemu_img_create: add backing_file support
Date: Fri, 4 Jul 2025 08:44:59 +0200
Message-ID: <20250704064507.511884-6-alexandre.derumier@groupe-cyllene.com>
and use it for plugin linked clone
This also enable extended_l2=on, as it's mandatory for backing file
preallocation.
Preallocation was missing previously, so it should increase performance
for linked clone now (around x5 in randwrite 4k)
cluster_size is set to 128k, as it reduce qcow2 overhead (reduce disk,
but also memory needed to cache metadatas)
l2_extended is not enabled yet on base image, but it could help too
to reduce overhead without impacting performance
bench on 100G qcow2 file:
fio --filename=/dev/sdb --direct=1 --rw=randwrite --bs=4k --iodepth=32 --ioengine=libaio --name=test
fio --filename=/dev/sdb --direct=1 --rw=randread --bs=4k --iodepth=32 --ioengine=libaio --name=test
base image:
randwrite 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 20215
randread 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 22219
randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 20217
randread 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 21742
randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 21599
randread 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 22037
clone image with backing file:
randwrite 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 3912
randread 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 21476
randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 20563
randread 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 22265
randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 18016
randread 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 21611
Signed-off-by: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
---
src/PVE/Storage/Common.pm | 17 +++++++++++++----
src/PVE/Storage/Plugin.pm | 17 +++--------------
2 files changed, 16 insertions(+), 18 deletions(-)
diff --git a/src/PVE/Storage/Common.pm b/src/PVE/Storage/Common.pm
index 29f2e52..78e5320 100644
--- a/src/PVE/Storage/Common.pm
+++ b/src/PVE/Storage/Common.pm
@@ -150,14 +150,23 @@ sub preallocation_cmd_option {
}
sub qemu_img_create {
- my ($scfg, $fmt, $size, $path) = @_;
+ my ($scfg, $fmt, $size, $path, $backing_path) = @_;
+
+ die "size can't be specified if backing file is used" if $size && $backing_path;
my $cmd = ['/usr/bin/qemu-img', 'create'];
- my $prealloc_opt = preallocation_cmd_option($scfg, $fmt);
- push @$cmd, '-o', $prealloc_opt if defined($prealloc_opt);
+ my $options = [];
+
+ if ($backing_path) {
+ push @$cmd, '-b', $backing_path, '-F', 'qcow2';
+ push @$options, 'extended_l2=on', 'cluster_size=128k';
+ }
- push @$cmd, '-f', $fmt, $path, "${size}K";
+ push @$options, preallocation_cmd_option($scfg, $fmt);
+ push @$cmd, '-o', join(',', @$options) if @$options > 0;
+ push @$cmd, '-f', $fmt, $path;
+ push @$cmd, "${size}K" if $size;
run_command($cmd, errmsg => "unable to create image");
}
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 80bb077..c35d5e5 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -880,20 +880,9 @@ sub clone_image {
# Note: we use relative paths, so we need to call chdir before qemu-img
eval {
local $CWD = $imagedir;
-
- my $cmd = [
- '/usr/bin/qemu-img',
- 'create',
- '-b',
- "../$basevmid/$basename",
- '-F',
- $format,
- '-f',
- 'qcow2',
- $path,
- ];
-
- run_command($cmd);
+ PVE::Storage::Common::qemu_img_create(
+ $scfg, $format, undef, $path, "../$basevmid/$basename",
+ );
};
my $err = $@;
--
2.39.5
[-- Attachment #2: Type: text/plain, Size: 160 bytes --]
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
next prev parent reply other threads:[~2025-07-04 6:45 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20250704064507.511884-1-alexandre.derumier@groupe-cyllene.com>
2025-07-04 6:44 ` [pve-devel] [PATCH qemu-server 1/3] qemu_img convert : add external snapshot support Alexandre Derumier via pve-devel
2025-07-04 6:44 ` [pve-devel] [PATCH pve-storage 01/10] tests: add lvmplugin test Alexandre Derumier via pve-devel
2025-07-04 6:44 ` [pve-devel] [PATCH qemu-server 2/3] blockdev: add backing_chain support Alexandre Derumier via pve-devel
2025-07-04 6:44 ` [pve-devel] [PATCH pve-storage 02/10] common: add qemu_img_create an preallocation_cmd_option Alexandre Derumier via pve-devel
2025-07-04 11:53 ` Fabian Grünbichler
2025-07-04 12:33 ` DERUMIER, Alexandre via pve-devel
[not found] ` <51f988f11e60f9dfaa49658c1ed9ecf72fcfcde4.camel@groupe-cyllene.com>
2025-07-07 7:55 ` Fabian Grünbichler
2025-07-04 6:44 ` Alexandre Derumier via pve-devel [this message]
2025-07-04 11:52 ` [pve-devel] [PATCH pve-storage 03/10] common: qemu_img_create: add backing_file support Fabian Grünbichler
2025-07-04 12:31 ` DERUMIER, Alexandre via pve-devel
2025-07-07 7:16 ` DERUMIER, Alexandre via pve-devel
2025-07-04 6:45 ` [pve-devel] [PATCH qemu-server 3/3] qcow2: add external snapshot support Alexandre Derumier via pve-devel
2025-07-04 11:52 ` Fabian Grünbichler
2025-07-04 12:46 ` DERUMIER, Alexandre via pve-devel
2025-07-04 6:45 ` [pve-devel] [PATCH pve-storage 04/10] rename_volume: add source && target snap Alexandre Derumier via pve-devel
2025-07-04 11:52 ` Fabian Grünbichler
2025-07-04 12:04 ` Thomas Lamprecht
2025-07-07 10:34 ` DERUMIER, Alexandre via pve-devel
2025-07-04 6:45 ` [pve-devel] [PATCH pve-storage 05/10] common: add qemu_img_info helper Alexandre Derumier via pve-devel
2025-07-04 6:45 ` [pve-devel] [PATCH pve-storage 06/10] common: add qemu-img measure Alexandre Derumier via pve-devel
2025-07-04 11:51 ` Fabian Grünbichler
2025-07-04 6:45 ` [pve-devel] [PATCH pve-storage 07/10] storage: volume_snapshot: add $running param Alexandre Derumier via pve-devel
2025-07-04 11:52 ` Fabian Grünbichler
2025-07-04 6:45 ` [pve-devel] [PATCH pve-storage 08/10] qcow2: add external snapshot support Alexandre Derumier via pve-devel
2025-07-04 11:52 ` Fabian Grünbichler
2025-07-04 13:22 ` DERUMIER, Alexandre via pve-devel
[not found] ` <c38598bae6477dfa6af0db96da054b156698d41c.camel@groupe-cyllene.com>
2025-07-07 8:17 ` Fabian Grünbichler
2025-07-07 10:18 ` DERUMIER, Alexandre via pve-devel
[not found] ` <c671fe82a7cdab90a3691115a7132d0a35ae79b7.camel@groupe-cyllene.com>
2025-07-07 10:53 ` Fabian Grünbichler
2025-07-08 8:44 ` DERUMIER, Alexandre via pve-devel
[not found] ` <3d1d8516e3c68de370608033647a38e99ef50f23.camel@groupe-cyllene.com>
2025-07-08 8:56 ` Fabian Grünbichler
2025-07-08 11:37 ` DERUMIER, Alexandre via pve-devel
2025-07-08 10:04 ` DERUMIER, Alexandre via pve-devel
[not found] ` <27854af70a4fe3a7765d2760098e2f82f3475f17.camel@groupe-cyllene.com>
2025-07-08 10:59 ` Fabian Grünbichler
2025-07-08 11:35 ` DERUMIER, Alexandre via pve-devel
[not found] ` <0b2ba0c34d2c8c15d7cb642442b300a3180e1592.camel@groupe-cyllene.com>
2025-07-08 12:50 ` Thomas Lamprecht
2025-07-08 13:19 ` DERUMIER, Alexandre via pve-devel
2025-07-08 13:42 ` DERUMIER, Alexandre via pve-devel
[not found] ` <67627e7904281520e1f7152657ed00c7ba3c138b.camel@groupe-cyllene.com>
2025-07-08 14:18 ` Fabian Grünbichler
2025-07-09 12:52 ` DERUMIER, Alexandre via pve-devel
2025-07-04 6:45 ` [pve-devel] [PATCH pve-storage 09/10] lvmplugin: add qcow2 snapshot Alexandre Derumier via pve-devel
2025-07-04 11:51 ` Fabian Grünbichler
2025-07-09 7:24 ` DERUMIER, Alexandre via pve-devel
2025-07-09 8:06 ` DERUMIER, Alexandre via pve-devel
2025-07-04 6:45 ` [pve-devel] [PATCH pve-storage 10/10] storage : add volume_support_qemu_snapshot Alexandre Derumier via pve-devel
2025-07-04 11:51 ` Fabian Grünbichler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=mailman.964.1751611551.395.pve-devel@lists.proxmox.com \
--to=pve-devel@lists.proxmox.com \
--cc=alexandre.derumier@groupe-cyllene.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal