public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH v8 qemu-server 01/11] backup: keep track of block-node size for fleecing
Date: Thu,  3 Apr 2025 14:30:59 +0200	[thread overview]
Message-ID: <20250403123118.264974-20-w.bumiller@proxmox.com> (raw)
In-Reply-To: <20250403123118.264974-1-w.bumiller@proxmox.com>

From: Fiona Ebner <f.ebner@proxmox.com>

For fleecing, the size needs to match exactly what QEMU sees. In
particular, EFI disks might be attached with a 'size=' option, meaning
that size can be different from the volume's size. Commit 36377acf
("backup: disk info: also keep track of size") introduced size
tracking and it was used for fleecing since then, but the accurate
size information needs to be queried via QMP.

Should also help with the following issue reported in the community
forum:
https://forum.proxmox.com/threads/152202

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
No changes to v7.

 PVE/VZDump/QemuServer.pm | 31 +++++++++++++++++++++++++++++--
 1 file changed, 29 insertions(+), 2 deletions(-)

diff --git a/PVE/VZDump/QemuServer.pm b/PVE/VZDump/QemuServer.pm
index 3238e34..4a25889 100644
--- a/PVE/VZDump/QemuServer.pm
+++ b/PVE/VZDump/QemuServer.pm
@@ -558,7 +558,7 @@ my sub allocate_fleecing_images {
 		my $name = "vm-$vmid-fleece-$n";
 		$name .= ".$format" if $scfg->{path};
 
-		my $size = PVE::Tools::convert_size($di->{size}, 'b' => 'kb');
+		my $size = PVE::Tools::convert_size($di->{'block-node-size'}, 'b' => 'kb');
 
 		$di->{'fleece-volid'} = PVE::Storage::vdisk_alloc(
 		    $self->{storecfg}, $fleecing_storeid, $vmid, $format, $name, $size);
@@ -607,7 +607,7 @@ my sub attach_fleecing_images {
 	    my $drive = "file=$path,if=none,id=$devid,format=$format,discard=unmap";
 	    # Specify size explicitly, to make it work if storage backend rounded up size for
 	    # fleecing image when allocating.
-	    $drive .= ",size=$di->{size}" if $format eq 'raw';
+	    $drive .= ",size=$di->{'block-node-size'}" if $format eq 'raw';
 	    $drive =~ s/\\/\\\\/g;
 	    my $ret = PVE::QemuServer::Monitor::hmp_cmd($vmid, "drive_add auto \"$drive\"", 60);
 	    die "attaching fleecing image $volid failed - $ret\n" if $ret !~ m/OK/s;
@@ -633,6 +633,8 @@ my sub check_and_prepare_fleecing {
     }
 
     if ($use_fleecing) {
+	$self->query_block_node_sizes($vmid, $disks);
+
 	my ($default_format, $valid_formats) = PVE::Storage::storage_default_format(
 	    $self->{storecfg}, $fleecing_opts->{storage});
 	my $format = scalar(grep { $_ eq 'qcow2' } $valid_formats->@*) ? 'qcow2' : 'raw';
@@ -1038,6 +1040,31 @@ sub qga_fs_thaw {
     $self->logerr($@) if $@;
 }
 
+# The size for fleecing images needs to be exactly the same size as QEMU sees. E.g. EFI disk can bex
+# attached with a smaller size then the underyling image on the storage.
+sub query_block_node_sizes {
+    my ($self, $vmid, $disks) = @_;
+
+    my $block_info = mon_cmd($vmid, "query-block");
+    $block_info = { map { $_->{device} => $_ } $block_info->@* };
+
+    for my $diskinfo ($disks->@*) {
+	my $drive_key = $diskinfo->{virtdev};
+	$drive_key .= "-backup" if $drive_key eq 'tpmstate0';
+	my $block_node_size =
+	    eval { $block_info->{"drive-$drive_key"}->{inserted}->{image}->{'virtual-size'}; };
+	if (!$block_node_size) {
+	    $self->loginfo(
+		"could not determine block node size of drive '$drive_key' - using fallback");
+	    $block_node_size = $diskinfo->{size}
+		or die "could not determine size of drive '$drive_key'\n";
+	}
+	$diskinfo->{'block-node-size'} = $block_node_size;
+    }
+
+    return;
+}
+
 # we need a running QEMU/KVM process for backup, starts a paused (prelaunch)
 # one if VM isn't already running
 sub enforce_vm_running_for_backup {
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


  parent reply	other threads:[~2025-04-03 12:35 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-04-03 12:30 [pve-devel] [PATCH v8 storage 0/9] backup provider API Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 qemu 01/10] PVE backup: clean up directly in setup_snapshot_access() when it fails Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 qemu 02/10] PVE backup: factor out helper to clear backup state's bitmap list Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 qemu 03/10] PVE backup: factor out helper to initialize backup state stat struct Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 qemu 04/10] PVE backup: add target ID in backup state Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 qemu 05/10] PVE backup: get device info: allow caller to specify filter for which devices use fleecing Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 qemu 06/10] PVE backup: implement backup access setup and teardown API for external providers Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 qemu 07/10] PVE backup: factor out get_single_device_info() helper Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 qemu 08/10] PVE backup: implement bitmap support for external backup access Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 qemu 09/10] PVE backup: backup-access api: indicate situation where a bitmap was recreated Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 qemu 10/10] PVE backup: backup-access-api: explicit bitmap-mode parameter Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 storage 1/8] add storage_has_feature() helper function Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 storage 2/8] common: add deallocate " Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 storage 3/8] plugin: introduce new_backup_provider() method Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 storage 4/8] config api/plugins: let plugins define sensitive properties themselves Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 storage 5/8] plugin api: bump api version and age Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [PATCH v8 storage 6/8] extract backup config: delegate to backup provider for storages that support it Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [POC v8 storage 7/8] add backup provider example Wolfgang Bumiller
2025-04-04  6:58   ` Wolfgang Bumiller
2025-04-03 12:30 ` [pve-devel] [POC v8 storage 8/8] Borg example plugin Wolfgang Bumiller
2025-04-03 12:30 ` Wolfgang Bumiller [this message]
2025-04-03 12:31 ` [pve-devel] [PATCH v8 qemu-server 02/11] backup: fleecing: use exact size when allocating non-raw fleecing images Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 qemu-server 03/11] backup: allow adding fleecing images also for EFI and TPM Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 qemu-server 04/11] backup: implement backup for external providers Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 qemu-server 05/11] test: qemu img convert: add test cases for snapshots Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 qemu-server 06/11] image convert: collect options in hash argument Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 qemu-server 07/11] image convert: allow caller to specify the format of the source path Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 qemu-server 08/11] backup: implement restore for external providers Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 qemu-server 09/11] backup: future-proof checks for QEMU feature support Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 qemu-server 10/11] backup: support 'missing-recreated' bitmap action Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 qemu-server 11/11] backup: bitmap action to human: lie about TPM state Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 container 1/7] add LXC::Namespaces module Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 container 2/7] backup: implement backup for external providers Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 container 3/7] backup: implement restore " Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 container 4/7] external restore: don't use 'one-file-system' tar flag when restoring from a directory Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 container 5/7] create: factor out compression option helper Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 container 6/7] restore tar archive: check potentially untrusted archive Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 container 7/7] api: add early check against restoring privileged container from external source Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 manager 1/2] ui: backup: also check for backup subtype to classify archive Wolfgang Bumiller
2025-04-03 12:31 ` [pve-devel] [PATCH v8 manager 2/2] backup: implement backup for external providers Wolfgang Bumiller
2025-04-03 16:10 ` [pve-devel] partially-applied-series: [PATCH v8 storage 0/9] backup provider API Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250403123118.264974-20-w.bumiller@proxmox.com \
    --to=w.bumiller@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal