public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] [RFC storage] work-around #6543: do not use preallocation for qcow2 on top of LVM
@ 2025-07-22 15:31 Fiona Ebner
  2025-07-23 11:25 ` DERUMIER, Alexandre via pve-devel
                   ` (2 more replies)
  0 siblings, 3 replies; 5+ messages in thread
From: Fiona Ebner @ 2025-07-22 15:31 UTC (permalink / raw)
  To: pve-devel

With the default preallocation=metadata for qcow2, the image can grow
more than 'qemu-img measure' reports, see bug #6543. Currently, the
option does not seem to make a difference with 'qemu-img measure', so
that needs to be further investigated. Once it's safe to add
preallocation back here, it should also be re-added to the
qemu_img_resize() call. Also, the option should be passed along to the
qemu_img_measure() call, because otherwise, it wouldn't match the
actual creation options.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

RFC, because this stop-gap might not even be complete (but it's the
best I have right now). Needs to be furhter investigated in any case.

 src/PVE/Storage/LVMPlugin.pm | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm
index 8be2a10..fba592d 100644
--- a/src/PVE/Storage/LVMPlugin.pm
+++ b/src/PVE/Storage/LVMPlugin.pm
@@ -568,8 +568,14 @@ my sub lvm_qcow2_format {
     $class->activate_volume($storeid, $scfg, $name);
     my $path = $class->path($scfg, $name, $storeid);
 
+    # TODO with the default preallocation=metadata for qcow2, the image can grow more than
+    # 'qemu-img measure' reports, see bug #6543. Currently, the option does not seem to make a
+    # difference with 'qemu-img measure', so that needs to be further investigated. Once it's safe
+    # to add preallocation back here, it should also be re-added to the qemu_img_resize() call.
+    # Also, the option should be passed along to the qemu_img_measure() call, because otherwise,
+    # it wouldn't match the actual creation options.
     my $options = {
-        preallocation => PVE::Storage::Plugin::preallocation_cmd_opt($scfg, $fmt),
+        # preallocation => PVE::Storage::Plugin::preallocation_cmd_opt($scfg, $fmt),
     };
     if ($backing_snap) {
         my $backing_volname = get_snap_name($class, $name, $backing_snap);
@@ -927,8 +933,7 @@ sub volume_resize {
     );
 
     if (!$running && $format eq 'qcow2') {
-        my $preallocation = PVE::Storage::Plugin::preallocation_cmd_opt($scfg, $format);
-        PVE::Storage::Common::qemu_img_resize($path, $format, $size, $preallocation, 10);
+        PVE::Storage::Common::qemu_img_resize($path, $format, $size, undef, 10);
     }
 
     return 1;
-- 
2.47.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pve-devel] [RFC storage] work-around #6543: do not use preallocation for qcow2 on top of LVM
  2025-07-22 15:31 [pve-devel] [RFC storage] work-around #6543: do not use preallocation for qcow2 on top of LVM Fiona Ebner
@ 2025-07-23 11:25 ` DERUMIER, Alexandre via pve-devel
  2025-07-23 12:15 ` DERUMIER, Alexandre via pve-devel
  2025-07-24  9:41 ` Fiona Ebner
  2 siblings, 0 replies; 5+ messages in thread
From: DERUMIER, Alexandre via pve-devel @ 2025-07-23 11:25 UTC (permalink / raw)
  To: pve-devel; +Cc: DERUMIER, Alexandre

[-- Attachment #1: Type: message/rfc822, Size: 16531 bytes --]

From: "DERUMIER, Alexandre" <alexandre.derumier@groupe-cyllene.com>
To: "pve-devel@lists.proxmox.com" <pve-devel@lists.proxmox.com>
Subject: Re: [pve-devel] [RFC storage] work-around #6543: do not use preallocation for qcow2 on top of LVM
Date: Wed, 23 Jul 2025 11:25:38 +0000
Message-ID: <0bd1078bb6845675cd31a54bb07607d870922e85.camel@groupe-cyllene.com>

Hi Fiona, I'm on holiday, can't verify, but before using qemu-img
measure I had implemented compute of metadatas size (It was not 100%
perfect).

This is strange, because I thinked that "qemu-img measure" was working
correctly (we need to pass it blocksize && l2_extended option too). I
had tried with a 1TB qcow2 volume.


Note that I'm almost pretty sure that l2_extended=on need
preallocation. (If I remember it's failing without it, and performance
of backed volumes are pretty bad with l2_extended).

-------- Message initial --------
De: Fiona Ebner <f.ebner@proxmox.com>
Répondre à: Proxmox VE development discussion <pve-
devel@lists.proxmox.com>
À: pve-devel@lists.proxmox.com
Objet: [pve-devel] [RFC storage] work-around #6543: do not use
preallocation for qcow2 on top of LVM
Date: 22/07/2025 17:31:50

With the default preallocation=metadata for qcow2, the image can grow
more than 'qemu-img measure' reports, see bug #6543. Currently, the
option does not seem to make a difference with 'qemu-img measure', so
that needs to be further investigated. Once it's safe to add
preallocation back here, it should also be re-added to the
qemu_img_resize() call. Also, the option should be passed along to the
qemu_img_measure() call, because otherwise, it wouldn't match the
actual creation options.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

RFC, because this stop-gap might not even be complete (but it's the
best I have right now). Needs to be furhter investigated in any case.

 src/PVE/Storage/LVMPlugin.pm | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/src/PVE/Storage/LVMPlugin.pm
b/src/PVE/Storage/LVMPlugin.pm
index 8be2a10..fba592d 100644
--- a/src/PVE/Storage/LVMPlugin.pm
+++ b/src/PVE/Storage/LVMPlugin.pm
@@ -568,8 +568,14 @@ my sub lvm_qcow2_format {
     $class->activate_volume($storeid, $scfg, $name);
     my $path = $class->path($scfg, $name, $storeid);
 
+    # TODO with the default preallocation=metadata for qcow2, the
image can grow more than
+    # 'qemu-img measure' reports, see bug #6543. Currently, the option
does not seem to make a
+    # difference with 'qemu-img measure', so that needs to be further
investigated. Once it's safe
+    # to add preallocation back here, it should also be re-added to
the qemu_img_resize() call.
+    # Also, the option should be passed along to the
qemu_img_measure() call, because otherwise,
+    # it wouldn't match the actual creation options.
     my $options = {
-        preallocation =>
PVE::Storage::Plugin::preallocation_cmd_opt($scfg, $fmt),
+        # preallocation =>
PVE::Storage::Plugin::preallocation_cmd_opt($scfg, $fmt),
     };
     if ($backing_snap) {
         my $backing_volname = get_snap_name($class, $name,
$backing_snap);
@@ -927,8 +933,7 @@ sub volume_resize {
     );
 
     if (!$running && $format eq 'qcow2') {
-        my $preallocation =
PVE::Storage::Plugin::preallocation_cmd_opt($scfg, $format);
-        PVE::Storage::Common::qemu_img_resize($path, $format, $size,
$preallocation, 10);
+        PVE::Storage::Common::qemu_img_resize($path, $format, $size,
undef, 10);
     }
 
     return 1;

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pve-devel] [RFC storage] work-around #6543: do not use preallocation for qcow2 on top of LVM
  2025-07-22 15:31 [pve-devel] [RFC storage] work-around #6543: do not use preallocation for qcow2 on top of LVM Fiona Ebner
  2025-07-23 11:25 ` DERUMIER, Alexandre via pve-devel
@ 2025-07-23 12:15 ` DERUMIER, Alexandre via pve-devel
  2025-07-23 13:07   ` Fiona Ebner
  2025-07-24  9:41 ` Fiona Ebner
  2 siblings, 1 reply; 5+ messages in thread
From: DERUMIER, Alexandre via pve-devel @ 2025-07-23 12:15 UTC (permalink / raw)
  To: pve-devel; +Cc: DERUMIER, Alexandre

[-- Attachment #1: Type: message/rfc822, Size: 13330 bytes --]

From: "DERUMIER, Alexandre" <alexandre.derumier@groupe-cyllene.com>
To: "pve-devel@lists.proxmox.com" <pve-devel@lists.proxmox.com>
Subject: Re: [pve-devel] [RFC storage] work-around #6543: do not use preallocation for qcow2 on top of LVM
Date: Wed, 23 Jul 2025 12:15:14 +0000
Message-ID: <7bc6dccecb0e400bb2156cc4c64fe2734aec64ce.camel@groupe-cyllene.com>

>>Currently, the
>>option does not seem to make a difference with 'qemu-img measure', so
>>that needs to be further investigated. 

it seem to be normal that preallocation option don't make a difference
here. qemu-img measure always compute size of all metadatas.  (only
cluster_size && l2_extended change the result)


I don't understand why disabling preallocation could help here (until
it's bugged), because the same amount of metadatas need to be allocated
later on the fly  with disabled preallocation.

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pve-devel] [RFC storage] work-around #6543: do not use preallocation for qcow2 on top of LVM
  2025-07-23 12:15 ` DERUMIER, Alexandre via pve-devel
@ 2025-07-23 13:07   ` Fiona Ebner
  0 siblings, 0 replies; 5+ messages in thread
From: Fiona Ebner @ 2025-07-23 13:07 UTC (permalink / raw)
  To: Proxmox VE development discussion

Hi Alexandre,

Am 23.07.25 um 2:15 PM schrieb DERUMIER, Alexandre via pve-devel:
> Hi Fiona, I'm on holiday, can't verify, but before using qemu-img
> measure I had implemented compute of metadatas size (It was not 100%
> perfect).
> 
> This is strange, because I thinked that "qemu-img measure" was working
> correctly (we need to pass it blocksize && l2_extended option too). I
> had tried with a 1TB qcow2 volume.
> 
> 
> Note that I'm almost pretty sure that l2_extended=on need
> preallocation. (If I remember it's failing without it, and performance
> of backed volumes are pretty bad with l2_extended).

allocation seems to work fine:

Formatting '/dev/sharedlvm/vm-101-disk-0.qcow2', fmt=qcow2
cluster_size=131072 extended_l2=on compression_type=zlib size=1073741824
backing_file=snap_vm-101-disk-0_foobar.qcow2 backing_fmt=qcow2
lazy_refcounts=off refcount_bits=16

Do you mean for the performance benefits?

> it seem to be normal that preallocation option don't make a difference
> here. qemu-img measure always compute size of all metadatas.  (only
> cluster_size && l2_extended change the result)
> 
> 
> I don't understand why disabling preallocation could help here (until
> it's bugged), because the same amount of metadatas need to be allocated
> later on the fly  with disabled preallocation.

Yes, intuitively and telling from qemu-img measure, preallocation
shouldn't make a difference, but it seems to. The bug has not been fully
investigated, but I can clearly see qcow2 trying to write to an offset
beyond the LV size (which is even slightly larger than what qemu-img
measure reported, because it rounded up to full extent).

In practice, something is wrong, see the bug report. Friedrich and I
were both able to reproduce the issue, but only with
preallocation=metadata for now. I used a 4GiB disk on a ZFS storage that
first was filled with zeroes from within the VM (like in the bug
report). Then offline move it to qcow2-on-LVM and again fill it with
zeroes from within the VM. The error doesn't happen every time though,
still need to investigate further.

Best Regards,
Fiona


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pve-devel] [RFC storage] work-around #6543: do not use preallocation for qcow2 on top of LVM
  2025-07-22 15:31 [pve-devel] [RFC storage] work-around #6543: do not use preallocation for qcow2 on top of LVM Fiona Ebner
  2025-07-23 11:25 ` DERUMIER, Alexandre via pve-devel
  2025-07-23 12:15 ` DERUMIER, Alexandre via pve-devel
@ 2025-07-24  9:41 ` Fiona Ebner
  2 siblings, 0 replies; 5+ messages in thread
From: Fiona Ebner @ 2025-07-24  9:41 UTC (permalink / raw)
  To: pve-devel

This is obsolete, see the recent discussion in the bugzilla entry. I'll
work out other patches that use the discard-no-unref option to avoid the
issue.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2025-07-24  9:40 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-07-22 15:31 [pve-devel] [RFC storage] work-around #6543: do not use preallocation for qcow2 on top of LVM Fiona Ebner
2025-07-23 11:25 ` DERUMIER, Alexandre via pve-devel
2025-07-23 12:15 ` DERUMIER, Alexandre via pve-devel
2025-07-23 13:07   ` Fiona Ebner
2025-07-24  9:41 ` Fiona Ebner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal