From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 8BBBDAF96 for ; Thu, 28 Apr 2022 13:37:56 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 8103DA77 for ; Thu, 28 Apr 2022 13:37:56 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 164D2A6A for ; Thu, 28 Apr 2022 13:37:55 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id E3BC6428B9 for ; Thu, 28 Apr 2022 13:37:54 +0200 (CEST) From: Fabian Ebner To: pve-devel@lists.proxmox.com Date: Thu, 28 Apr 2022 13:37:37 +0200 Message-Id: <20220428113737.105842-1-f.ebner@proxmox.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.078 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [qemuserver.pm, qemumigrate.pm, qemu.pm] Subject: [pve-devel] [PATCH qemu-server] fix #3861: migrate: fix live migration when cloud-init changes storage X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Apr 2022 11:37:56 -0000 Generalizes fd95d780 ("migrate: send updated TPM state volid to target node") to also handle other offline migrated disks appearing in the VM config, which currently should only be cloud-init. Breaks migration new -> old under similar (edge-case-)conditions as fd95d780 did. Keep sending the 'tpmstate0' STDIN parameter to avoid breaking new -> old in the scenario fd95d780 fixed. Keep parsing the vm_start 'tpmstate0' STDIN parameter to avoid breaking old -> new, and to be able to keep sending it. Signed-off-by: Fabian Ebner --- PVE/API2/Qemu.pm | 10 ++++++---- PVE/QemuMigrate.pm | 19 ++++++++++++++----- PVE/QemuServer.pm | 15 +++++++++------ 3 files changed, 29 insertions(+), 15 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 54955ee2..1d09cab3 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -2542,7 +2542,7 @@ __PACKAGE__->register_method({ my $spice_ticket; my $nbd_protocol_version = 0; my $replicated_volumes = {}; - my $tpmstate_vol; + my $offline_volumes = {}; if ($stateuri && ($stateuri eq 'tcp' || $stateuri eq 'unix') && $migratedfrom && ($rpcenv->{type} eq 'cli')) { while (defined(my $line = )) { chomp $line; @@ -2552,8 +2552,10 @@ __PACKAGE__->register_method({ $nbd_protocol_version = $1; } elsif ($line =~ m/^replicated_volume: (.*)$/) { $replicated_volumes->{$1} = 1; - } elsif ($line =~ m/^tpmstate0: (.*)$/) { - $tpmstate_vol = $1; + } elsif ($line =~ m/^tpmstate0: (.*)$/) { # Deprecated, use offline_volume instead + $offline_volumes->{tpmstate0} = $1; + } elsif ($line =~ m/^offline_volume: ([^:]+): (.*)$/) { + $offline_volumes->{$1} = $2; } elsif (!$spice_ticket) { # fallback for old source node $spice_ticket = $line; @@ -2595,7 +2597,7 @@ __PACKAGE__->register_method({ storagemap => $storagemap, nbd_proto_version => $nbd_protocol_version, replicated_volumes => $replicated_volumes, - tpmstate_vol => $tpmstate_vol, + offline_volumes => $offline_volumes, }; my $params = { diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index dfe92325..d52dc8db 100644 --- a/PVE/QemuMigrate.pm +++ b/PVE/QemuMigrate.pm @@ -710,11 +710,20 @@ sub phase2 { my $nbd_protocol_version = 1; my $input = "nbd_protocol_version: $nbd_protocol_version\n"; - if ($conf->{tpmstate0}) { - my $tpmdrive = PVE::QemuServer::parse_drive('tpmstate0', $conf->{tpmstate0}); - my $tpmvol = $tpmdrive->{file}; - $input .= "tpmstate0: $self->{volume_map}->{$tpmvol}" - if $self->{volume_map}->{$tpmvol} && $tpmvol ne $self->{volume_map}->{$tpmvol}; + my @offline_local_volumes = $self->filter_local_volumes('offline'); + for my $volid (@offline_local_volumes) { + my $drivename = $local_volumes->{$volid}->{drivename}; + next if !$drivename || !$conf->{$drivename}; + + my $new_volid = $self->{volume_map}->{$volid}; + next if !$new_volid || $volid eq $new_volid; + + # FIXME PVE 8.x only use offline_volume variant once all targets can handle it + if ($drivename eq 'tpmstate0') { + $input .= "$drivename: $new_volid\n" + } else { + $input .= "offline_volume: $drivename: $new_volid\n" + } } $input .= "spice_ticket: $spice_ticket\n" if $spice_ticket; diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 7f983904..629b64fe 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -5432,7 +5432,8 @@ sub vm_start { # type => secure/insecure - tunnel over encrypted connection or plain-text # nbd_proto_version => int, 0 for TCP, 1 for UNIX # replicated_volumes => which volids should be re-used with bitmaps for nbd migration -# tpmstate_vol => new volid of tpmstate0, not yet contained in config +# offline_volumes => new volids of offline migrated disks like tpmstate and cloudinit, not yet +# contained in config sub vm_start_nolock { my ($storecfg, $vmid, $conf, $params, $migrate_opts) = @_; @@ -5457,11 +5458,13 @@ sub vm_start_nolock { # this way we can reuse the old ISO with the correct config PVE::QemuServer::Cloudinit::generate_cloudinitconfig($conf, $vmid) if !$migratedfrom; - # override TPM state vol if migrated, conf is out of date still - if (my $tpmvol = $migrate_opts->{tpmstate_vol}) { - my $parsed = parse_drive("tpmstate0", $conf->{tpmstate0}); - $parsed->{file} = $tpmvol; - $conf->{tpmstate0} = print_drive($parsed); + # override offline migrated volumes, conf is out of date still + if (my $offline_volumes = $migrate_opts->{offline_volumes}) { + for my $key (sort keys $offline_volumes->%*) { + my $parsed = parse_drive($key, $conf->{$key}); + $parsed->{file} = $offline_volumes->{$key}; + $conf->{$key} = print_drive($parsed); + } } my $defaults = load_defaults(); -- 2.30.2