From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 5F3841FF15C for ; Fri, 25 Jul 2025 09:37:44 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id EF1E011631; Fri, 25 Jul 2025 09:39:02 +0200 (CEST) Date: Fri, 25 Jul 2025 09:38:54 +0200 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= To: Proxmox VE development discussion References: <20250724135956.112138-1-f.ebner@proxmox.com> <20250724135956.112138-7-f.ebner@proxmox.com> In-Reply-To: <20250724135956.112138-7-f.ebner@proxmox.com> MIME-Version: 1.0 User-Agent: astroid/0.16.0 (https://github.com/astroidmail/astroid) Message-Id: <1753428793.n5phbfd5ll.astroid@yuna.none> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1753429133893 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.045 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [blockdev.pm, qemuimage.pm] Subject: Re: [pve-devel] [PATCH qemu-server 6/6] fix #6543: use qcow2 'discard-no-unref' option when using snapshot-as-volume-chain X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" On July 24, 2025 3:59 pm, Fiona Ebner wrote: > Without the 'discard-no-unref', a qcow2 file can grow beyond what > 'qemu-img measure' reports, because of fragmentation. This can lead to > IO errors with qcow2 on top of LVM storages, where the containing LV > is allocated with that size. Guard enabling the option with > having 'snapshot-as-volume-chain' in the storage configuration for > now. Enabling it always should be evaluated a bit more and tested on > different storages. It is a runtime-only option just affecting how > referencing clusters is handled during discard in qcow2 and nothing > else, so it is also fine for existing images and migration streams. > > While 'snapshot-as-volume-chain' is not the perfect proxy, as that's > not only for LVM, it's an experimental feature that covers the LVM > case and it seems like a nice fit to try out the new option on > file-based storages too. > > Suggested-by: Alexandre Derumier > Signed-off-by: Fiona Ebner > --- > src/PVE/QemuServer/Blockdev.pm | 7 ++++++ > src/PVE/QemuServer/QemuImage.pm | 19 +++++++++++++++ > src/test/cfg2cmd/simple-backingchain.conf.cmd | 2 +- > src/test/run_qemu_img_convert_tests.pl | 24 ++++++++++++++----- > 4 files changed, 45 insertions(+), 7 deletions(-) > > diff --git a/src/PVE/QemuServer/Blockdev.pm b/src/PVE/QemuServer/Blockdev.pm > index 8528a587..1487bc99 100644 > --- a/src/PVE/QemuServer/Blockdev.pm > +++ b/src/PVE/QemuServer/Blockdev.pm > @@ -372,6 +372,13 @@ my sub generate_format_blockdev { > $blockdev->{size} = int($options->{size}); > } > > + # see bug #6543: without this option, fragmentation can lead to the qcow2 file growing larger > + # than what qemu-img measure reports, which is problematic for qcow2-on-top-of-LVM > + # TODO test and consider enabling this in general > + if ($scfg && $scfg->{'snapshot-as-volume-chain'}) { > + $blockdev->{'discard-no-unref'} = JSON::true if $format eq 'qcow2'; > + } > + > return $blockdev; > } > > diff --git a/src/PVE/QemuServer/QemuImage.pm b/src/PVE/QemuServer/QemuImage.pm > index 026c24e9..7f6d5f01 100644 > --- a/src/PVE/QemuServer/QemuImage.pm > +++ b/src/PVE/QemuServer/QemuImage.pm > @@ -3,6 +3,9 @@ package PVE::QemuServer::QemuImage; > use strict; > use warnings; > > +use Fcntl qw(S_ISBLK); > +use File::stat; > + > use PVE::Format qw(render_bytes); > use PVE::Storage; > use PVE::Tools; > @@ -27,6 +30,18 @@ sub convert_iscsi_path { > die "cannot convert iscsi path '$path', unknown format\n"; > } > > +my sub qcow2_target_image_opts { > + my ($path, @qcow2_opts) = @_; > + > + my $st = File::stat::stat($path) or die "stat for '$path' failed - $!\n"; right now this is only called for PVE-managed volumes.. so we could actually call qemu_blockdev_options instead in `convert` below, and use the driver (and possibly other things?) from there? > + > + my $driver = S_ISBLK($st->mode) ? 'host_device' : 'file'; > + > + my $qcow2_opts_str = ',' . join(',', @qcow2_opts); > + > + return "driver=qcow2$qcow2_opts_str,file.driver=$driver,file.filename=$path"; > +} > + > # The possible options are: > # bwlimit - The bandwidth limit in KiB/s. > # is-zero-initialized - If the destination image is zero-initialized. > @@ -71,6 +86,7 @@ sub convert { > my $dst_format = checked_volume_format($storecfg, $dst_volid); > my $dst_path = PVE::Storage::path($storecfg, $dst_volid); > my $dst_is_iscsi = ($dst_path =~ m|^iscsi://|); > + my $dst_needs_discard_no_unref = $dst_scfg->{'snapshot-as-volume-chain'}; && $dst_format eq 'qcow2' as above in Blockdev.pm? > my $support_qemu_snapshots = PVE::Storage::volume_qemu_snapshot_method($storecfg, $src_volid); > > my $cmd = []; > @@ -94,6 +110,9 @@ sub convert { > if ($dst_is_iscsi) { > push @$cmd, '--target-image-opts'; > $dst_path = convert_iscsi_path($dst_path); > + } elsif ($dst_needs_discard_no_unref) { > + push @$cmd, '--target-image-opts'; > + $dst_path = qcow2_target_image_opts($dst_path, 'discard-no-unref=true'); > } else { > push @$cmd, '-O', $dst_format; > } _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel