public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Fiona Ebner <f.ebner@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH qemu-server v4 13/27] backup: keep track of block-node size for fleecing
Date: Thu, 14 Nov 2024 16:07:40 +0100	[thread overview]
Message-ID: <20241114150754.374376-14-f.ebner@proxmox.com> (raw)
In-Reply-To: <20241114150754.374376-1-f.ebner@proxmox.com>

For fleecing, the size needs to match exactly what QEMU sees. In
particular, EFI disks might be attached with a 'size=' option, meaning
that size can be different from the volume's size. Commit 36377acf
("backup: disk info: also keep track of size") introduced size
tracking and it was used for fleecing since then, but the accurate
size information needs to be queried via QMP.

Should also help with the following issue reported in the community
forum:
https://forum.proxmox.com/threads/152202

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

Changes in v4:
* avoid needlessly passing in $task ($disks is enough)

 PVE/VZDump/QemuServer.pm | 31 +++++++++++++++++++++++++++++--
 1 file changed, 29 insertions(+), 2 deletions(-)

diff --git a/PVE/VZDump/QemuServer.pm b/PVE/VZDump/QemuServer.pm
index c46e607c..2157d411 100644
--- a/PVE/VZDump/QemuServer.pm
+++ b/PVE/VZDump/QemuServer.pm
@@ -551,7 +551,7 @@ my sub allocate_fleecing_images {
 		my $name = "vm-$vmid-fleece-$n";
 		$name .= ".$format" if $scfg->{path};
 
-		my $size = PVE::Tools::convert_size($di->{size}, 'b' => 'kb');
+		my $size = PVE::Tools::convert_size($di->{'block-node-size'}, 'b' => 'kb');
 
 		$di->{'fleece-volid'} = PVE::Storage::vdisk_alloc(
 		    $self->{storecfg}, $fleecing_storeid, $vmid, $format, $name, $size);
@@ -600,7 +600,7 @@ my sub attach_fleecing_images {
 	    my $drive = "file=$path,if=none,id=$devid,format=$format,discard=unmap";
 	    # Specify size explicitly, to make it work if storage backend rounded up size for
 	    # fleecing image when allocating.
-	    $drive .= ",size=$di->{size}" if $format eq 'raw';
+	    $drive .= ",size=$di->{'block-node-size'}" if $format eq 'raw';
 	    $drive =~ s/\\/\\\\/g;
 	    my $ret = PVE::QemuServer::Monitor::hmp_cmd($vmid, "drive_add auto \"$drive\"", 60);
 	    die "attaching fleecing image $volid failed - $ret\n" if $ret !~ m/OK/s;
@@ -626,6 +626,8 @@ my sub check_and_prepare_fleecing {
     }
 
     if ($use_fleecing) {
+	$self->query_block_node_sizes($vmid, $disks);
+
 	my ($default_format, $valid_formats) = PVE::Storage::storage_default_format(
 	    $self->{storecfg}, $fleecing_opts->{storage});
 	my $format = scalar(grep { $_ eq 'qcow2' } $valid_formats->@*) ? 'qcow2' : 'raw';
@@ -1042,6 +1044,31 @@ sub qga_fs_thaw {
     $self->logerr($@) if $@;
 }
 
+# The size for fleecing images needs to be exactly the same size as QEMU sees. E.g. EFI disk can bex
+# attached with a smaller size then the underyling image on the storage.
+sub query_block_node_sizes {
+    my ($self, $vmid, $disks) = @_;
+
+    my $block_info = mon_cmd($vmid, "query-block");
+    $block_info = { map { $_->{device} => $_ } $block_info->@* };
+
+    for my $diskinfo ($disks->@*) {
+	my $drive_key = $diskinfo->{virtdev};
+	$drive_key .= "-backup" if $drive_key eq 'tpmstate0';
+	my $block_node_size =
+	    eval { $block_info->{"drive-$drive_key"}->{inserted}->{image}->{'virtual-size'}; };
+	if (!$block_node_size) {
+	    $self->loginfo(
+		"could not determine block node size of drive '$drive_key' - using fallback");
+	    $block_node_size = $diskinfo->{size}
+		or die "could not determine size of drive '$drive_key'\n";
+	}
+	$diskinfo->{'block-node-size'} = $block_node_size;
+    }
+
+    return;
+}
+
 # we need a running QEMU/KVM process for backup, starts a paused (prelaunch)
 # one if VM isn't already running
 sub enforce_vm_running_for_backup {
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


  parent reply	other threads:[~2024-11-14 15:10 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-11-14 15:07 [pve-devel] [PATCH-SERIES qemu/common/storage/qemu-server/container/manager v4 00/27] backup provider API Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu v4 01/27] PVE backup: add target ID in backup state Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu v4 02/27] PVE backup: get device info: allow caller to specify filter for which devices use fleecing Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu v4 03/27] PVE backup: implement backup access setup and teardown API for external providers Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu v4 04/27] PVE backup: implement bitmap support for external backup access Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH common v4 05/27] test: lock file: get rid of END block that made test always pass Fiona Ebner
2024-11-14 19:46   ` [pve-devel] applied: " Thomas Lamprecht
2024-11-14 15:07 ` [pve-devel] [PATCH common v4 06/27] tools: run fork: allow running code in parent after fork Fiona Ebner
2024-11-14 19:46   ` [pve-devel] applied: " Thomas Lamprecht
2024-11-14 15:07 ` [pve-devel] [PATCH common v4 07/27] test: have lock file test use run_fork() helper Fiona Ebner
2024-11-14 19:46   ` [pve-devel] applied: " Thomas Lamprecht
2024-11-14 15:07 ` [pve-devel] [PATCH storage v4 08/27] add storage_has_feature() helper function Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH storage v4 09/27] plugin: introduce new_backup_provider() method Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH storage v4 10/27] extract backup config: delegate to backup provider for storages that support it Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [POC storage v4 11/27] add backup provider example Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [POC storage v4 12/27] WIP Borg plugin Fiona Ebner
2024-11-14 15:07 ` Fiona Ebner [this message]
2024-11-14 15:07 ` [pve-devel] [PATCH qemu-server v4 14/27] module config: load nbd module at boot Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu-server v4 15/27] backup: allow adding fleecing images also for EFI and TPM Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu-server v4 16/27] backup: implement backup for external providers Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu-server v4 17/27] backup: implement restore " Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH qemu-server v4 18/27] backup restore: external: hardening check for untrusted source image Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 19/27] add LXC::Namespaces module Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 20/27] backup: implement backup for external providers Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 21/27] backup: implement restore " Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 22/27] external restore: don't use 'one-file-system' tar flag when restoring from a directory Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 23/27] create: factor out compression option helper Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 24/27] restore tar archive: check potentially untrusted archive Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH container v4 25/27] api: add early check against restoring privileged container from external source Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH manager v4 26/27] ui: backup: also check for backup subtype to classify archive Fiona Ebner
2024-11-14 15:07 ` [pve-devel] [PATCH manager v4 27/27] backup: implement backup for external providers Fiona Ebner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20241114150754.374376-14-f.ebner@proxmox.com \
    --to=f.ebner@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal