Re: [pve-devel] [PATCH pve-storage 3/5] lvmplugin: add qcow2 snapshot

["DERUMIER, Alexandre via pve-devel" <pve-devel@lists.proxmox.com>]

[-- Attachment #1: Type: message/rfc822, Size: 17897 bytes --]

From: "DERUMIER, Alexandre" <alexandre.derumier@groupe-cyllene.com>
To: "pve-devel@lists.proxmox.com" <pve-devel@lists.proxmox.com>, "f.gruenbichler@proxmox.com" <f.gruenbichler@proxmox.com>
Subject: Re: [pve-devel] [PATCH pve-storage 3/5] lvmplugin: add qcow2 snapshot
Date: Wed, 14 May 2025 07:05:21 +0000
Message-ID: <ecc1612c51b8fab5efb238bc65c5f63aa9ab1cb3.camel@groupe-cyllene.com>

ok, I have found the problem

the
    eval { lvm_qcow2_format($class, $storeid, $scfg, $name, $fmt,
$backing_snap, $size) };

is expecting bytes for size,

but vdisk_alloc is sending kb.   (and volume_resize is sending bytes
for example...)



So, It's creating a really big qcow2, and it's fail of metadata
preallocation, because metadatas can be handled by the lvm size.
(That's explain why I didn't see it with smaller size)


    eval { lvm_qcow2_format($class, $storeid, $scfg, $name, $fmt,
$backing_snap, $size/1024) };   

is fixing it.


Bu they are also the l2_extended not enabled on base qcow2.
I think you tell me some month ago that you had performance regression
with this was enabled, so I only have keeped it for snapshot volumes
(but forgot to change the preallocation code, as the overhead is 16x
without suballocation clusters)

I need to check, because according the suballocation code developper, 
it should help too with base image without backing file
https://youtu.be/clmbHp81GAc?t=823
(developper said it could depend of the underlay filesystem, but here
we use raw lvm, so maybe performance different than with a qcow2 file
over xfs|ext|nfs...)



I'm really sorry about this, I really need to implement some unit tests
, but didn't have time yet :/


-------- Message initial --------
De: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
À: Fabian Grünbichler <f.gruenbichler@proxmox.com>, Proxmox VE
development discussion <pve-devel@lists.proxmox.com>
Objet: Re: [pve-devel] [PATCH pve-storage 3/5] lvmplugin: add qcow2
snapshot
Date: 13/05/2025 20:37:36

> > It's work for me if the vm if offline, I think this is the problem
> > with
> > the size defined in bytes or kb, depend of the code path if the vm
> > is
> > offline or online. (I don't remember exactly where, I need to
> > check)

mmmmm, I just have retested on my side, online or offline, and it's
working fine with 10G  (so the lvm size is ok vs qcow2 size)


online

#qm create 10000 && qm start 10000 && qm set 10000 --scsi0
lvmtest2:10,format=qcow2
update VM 10000: -scsi0 lvmtest2:10,format=qcow2
  Logical volume "vm-10000-disk-2.qcow2" created.
Formatting '/dev/test/vm-10000-disk-2.qcow2', fmt=qcow2
cluster_size=65536 extended_l2=off preallocation=metadata
compression_type=zlib size=10737418240 lazy_refcounts=off
refcount_bits=16
scsi0: successfully created disk 'lvmtest2:vm-10000-disk-
2.qcow2,size=10G'


offline
#qm create 10000 && qm set 10000 --scsi0 lvmtest2:10,format=qcow2
  Logical volume "vm-10000-disk-2.qcow2" created.
Formatting '/dev/test/vm-10000-disk-2.qcow2', fmt=qcow2
cluster_size=65536 extended_l2=off preallocation=metadata
compression_type=zlib size=10737418240 lazy_refcounts=off
refcount_bits=16
scsi0: successfully created disk 'lvmtest2:vm-10000-disk-
2.qcow2,size=10G'



but failed with 28G (online or offline)

qm create 10000 && qm set 10000 --scsi0 lvmtest2:28,format=qcow2
  Logical volume "vm-10000-disk-2.qcow2" created.
Formatting '/dev/test/vm-10000-disk-2.qcow2', fmt=qcow2
cluster_size=65536 extended_l2=off preallocation=metadata
compression_type=zlib size=30064771072 lazy_refcounts=off
refcount_bits=16
qemu-img: Failed to flush the refcount block cache: No space left on
device
  Logical volume "vm-10000-disk-2.qcow2" successfully removed.
unable to create image: qemu-img: /dev/test/vm-10000-disk-2.qcow2:
Could not resize image: Allocating clusters failed: No space left on
device


So,It's something with the overhead calc...

(I need to check as extended_l2=off with base image, vs extended_l2=on
with snapshot. I don't rember if I have set extended_l2=off explicitly
to off for the base image for performance or not)






[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel