public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Jonas Theisen <j.theisen@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH qemu-server 08/15] migrate: block migration if the target cannot run the kept virtual CPU
Date: Wed, 18 Nov 2026 16:40:39 +0100	[thread overview]
Message-ID: <20261118154207.424260-9-j.theisen@proxmox.com> (raw)
In-Reply-To: <20261118154207.424260-1-j.theisen@proxmox.com>

For VMs with an abstracted CPU configuration, for example with a custom
CPU model or the 'nested-virt' flag, the target is started with the
QEMU CPU of the running instance, instead of generating it again from
the configuration. That QEMU CPU can contain flags that KVM on the
target does not support, for example the Intel-only 'hv-evmcs'
enlightenment that is enabled for Windows 11 guests on Intel hosts.
QEMU on an AMD target then refuses to start:

  kvm: Hyper-V enlightened VMCS (hv-evmcs) is not supported by kernel

and the migration fails only after its preparation.

Check upfront whether KVM on the target supports all enabled Hyper-V
enlightenments of the kept CPU, which QEMU always requires, and all of
its other enabled flags if 'enforce' is set. This uses the CPU flags
that pvestatd already broadcasts for each node. Only flags that KVM
supports on the source node are considered, so that QEMU CPU
properties that are not CPU flags are not reported as missing.

Signed-off-by: Jonas Theisen <j.theisen@proxmox.com>
---
 src/PVE/API2/Qemu.pm               |  3 +-
 src/PVE/QemuMigrate/Helpers.pm     | 39 +++++++++++++-
 src/test/run_qemu_migrate_tests.pl | 85 ++++++++++++++++++++++++++++++
 3 files changed, 125 insertions(+), 2 deletions(-)

diff --git a/src/PVE/API2/Qemu.pm b/src/PVE/API2/Qemu.pm
index f55cb641..5e14a1bb 100644
--- a/src/PVE/API2/Qemu.pm
+++ b/src/PVE/API2/Qemu.pm
@@ -5528,7 +5528,8 @@ __PACKAGE__->register_method({
                                     type => {
                                         type => 'string',
                                         description => "The check that failed.",
-                                        enum => ['cpu-vendor', 'host-cpu-flags'],
+                                        enum =>
+                                            ['cpu-vendor', 'host-cpu-flags', 'kept-cpu-flags'],
                                     },
                                     message => {
                                         type => 'string',
diff --git a/src/PVE/QemuMigrate/Helpers.pm b/src/PVE/QemuMigrate/Helpers.pm
index 67f3c0e3..37fa10d4 100644
--- a/src/PVE/QemuMigrate/Helpers.pm
+++ b/src/PVE/QemuMigrate/Helpers.pm
@@ -186,7 +186,8 @@ my sub format_flag_list {
 # Checks whether the CPU of a target node can run a running VM after a live migration.
 #
 # 'applies' returns true if the check is relevant for the VM's CPU, as parsed by
-# parse_qemu_cpu_string(), with 'kvm' set if the VM uses KVM. 'needs' lists the CPU information
+# parse_qemu_cpu_string(), with 'kvm' set if the VM uses KVM and 'kept' set if the target is
+# started with the same QEMU CPU, see is_running_cpu_kept(). 'needs' lists the CPU information
 # of the target node the check requires, nodes lacking any of it are skipped. The CPU information
 # is in the format of get_host_cpu_info(), plus the CPU flags KVM supports as 'kvm-flags' hash.
 # 'check' returns why the target node is incompatible, if it is. With the 'error' severity, an
@@ -287,6 +288,41 @@ my $target_cpu_checks = [
                 . " identical CPUs, but the node has '$target_model' instead of '$source_model'";
         },
     },
+    {
+        type => 'kept-cpu-flags',
+        severity => 'error',
+        needs => ['kvm-flags'],
+        applies => sub {
+            my ($cpu) = @_;
+            return $cpu->{kept};
+        },
+        check => sub {
+            my ($cpu, $source, $target) = @_;
+
+            my ($source_flags, $target_flags) =
+                ($source->{'kvm-flags'}, $target->{'kvm-flags'});
+            return if !$source_flags;
+
+            my $props = $cpu->{props};
+            my $enforce = PVE::JSONSchema::parse_boolean($props->{enforce} // 'off');
+
+            # QEMU refuses to start with unsupported Hyper-V enlightenments, and with any other
+            # unsupported flag only if 'enforce' is set. Requiring support on the source skips
+            # properties that are no CPU flags.
+            my @missing = sort grep {
+                PVE::JSONSchema::parse_boolean($props->{$_})
+                    && ($enforce || m/^hv-/)
+                    && $source_flags->{$_}
+                    && !$target_flags->{$_}
+            } keys $props->%*;
+            return if !scalar(@missing);
+
+            return
+                "the VM keeps its current virtual CPU when migrated, but KVM on the node does"
+                . " not support: "
+                . format_flag_list(\@missing);
+        },
+    },
 ];
 
 # Returns the issues of the target node, and whether all checks that can prevent the migration
@@ -351,6 +387,7 @@ sub check_target_cpu_compatibility {
 
     my $cpu = parse_qemu_cpu_string(PVE::QemuServer::CPUConfig::get_cpu_from_running_vm($pid));
     $cpu->{kvm} = $conf->{kvm} // 1;
+    $cpu->{kept} = is_running_cpu_kept($conf);
 
     my $checks = [grep { $_->{applies}->($cpu) } $target_cpu_checks->@*];
     return ({}, []) if !scalar($checks->@*);
diff --git a/src/test/run_qemu_migrate_tests.pl b/src/test/run_qemu_migrate_tests.pl
index 50f38388..57aa59b1 100755
--- a/src/test/run_qemu_migrate_tests.pl
+++ b/src/test/run_qemu_migrate_tests.pl
@@ -1859,6 +1859,91 @@ my $tests = [
             },
         },
     },
+    {
+        name => '1033_running_kept_cpu_supported_flags',
+        target => 'pve1',
+        vmid => 1033,
+        vm_status => {
+            running => 1,
+            runningmachine => 'pc-q35-5.0+pve0',
+            runningcpu =>
+                'kvm64,enforce,+vmx,hv_evmcs,hv_relaxed,hv_spinlocks=0x1fff,+kvm_pv_eoi',
+        },
+        cpu_info => {
+            source => { vendor => 'GenuineIntel' },
+            nodes => {
+                pve1 => { vendor => 'GenuineIntel' },
+            },
+            'kvm-flags' => {
+                pve0 => 'fpu hv-evmcs hv-relaxed hv-spinlocks kvm-pv-eoi sse vmx',
+                pve1 => 'fpu hv-evmcs hv-relaxed kvm-pv-eoi sse vmx',
+            },
+        },
+        opts => {
+            online => 1,
+        },
+        config_patch => {
+            cpu => 'kvm64,flags=+nested-virt',
+        },
+        expected_calls => $default_expected_calls_online,
+        expected => {
+            source_volids => {},
+            target_volids => {},
+            vm_config => get_patched_config(1033, {
+                    cpu => 'kvm64,flags=+nested-virt',
+            }),
+            vm_status => {
+                running => 1,
+                runningmachine => 'pc-q35-5.0+pve0',
+                runningcpu =>
+                    'kvm64,enforce,+vmx,hv_evmcs,hv_relaxed,hv_spinlocks=0x1fff,+kvm_pv_eoi',
+            },
+        },
+    },
+    {
+        name => '1033_running_kept_cpu_unsupported_flags',
+        target => 'pve1',
+        vmid => 1033,
+        vm_status => {
+            running => 1,
+            runningmachine => 'pc-q35-5.0+pve0',
+            runningcpu =>
+                'kvm64,enforce,+vmx,hv_evmcs,hv_relaxed,hv_spinlocks=0x1fff,+kvm_pv_eoi',
+        },
+        cpu_info => {
+            source => { vendor => 'GenuineIntel' },
+            nodes => {
+                pve1 => { vendor => 'AuthenticAMD' },
+            },
+            'kvm-flags' => {
+                pve0 => 'fpu hv-evmcs hv-relaxed hv-spinlocks kvm-pv-eoi sse vmx',
+                pve1 => 'fpu hv-relaxed kvm-pv-eoi sse svm',
+            },
+        },
+        opts => {
+            online => 1,
+        },
+        config_patch => {
+            cpu => 'kvm64,flags=+nested-virt',
+        },
+        expected_calls => {},
+        expect_die =>
+            "can't migrate VM to 'pve1' - incompatible CPU: the VM keeps its current virtual"
+            . " CPU when migrated, but KVM on the node does not support: hv-evmcs, vmx",
+        expected => {
+            source_volids => {},
+            target_volids => {},
+            vm_config => get_patched_config(1033, {
+                    cpu => 'kvm64,flags=+nested-virt',
+            }),
+            vm_status => {
+                running => 1,
+                runningmachine => 'pc-q35-5.0+pve0',
+                runningcpu =>
+                    'kvm64,enforce,+vmx,hv_evmcs,hv_relaxed,hv_spinlocks=0x1fff,+kvm_pv_eoi',
+            },
+        },
+    },
     {
         name => '105_replicated_to_non_replication_target',
         target => 'pve1',
-- 
2.47.3





  parent reply	other threads:[~2026-11-18 15:43 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 15:40 [PATCH manager/qemu-server 00/15] check the target node's CPU before live migrating a VM Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 01/15] test: migration: allow checking for warnings and errors in the log Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 02/15] migrate: block live migration of host CPU VMs between CPU vendors Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 03/15] fix #6535: migrate: warn about live migration " Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 04/15] cpu flags: factor out parsing the broadcast CPU flags of a node Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 05/15] fix #2924: migrate: block live migration of host CPU VMs to nodes lacking flags Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 06/15] migrate: warn about live migration of host CPU VMs between CPU models Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 07/15] migrate: factor out whether the running virtual CPU is kept Jonas Theisen
2026-11-18 15:40 ` Jonas Theisen [this message]
2026-11-18 15:40 ` [PATCH qemu-server 09/15] helpers: add helper to query the host's TSC frequency and scaling support Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 10/15] cpu config: add hv-reenlightenment flag and set TSC frequency for it Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 11/15] migrate: block migration to nodes with incompatible TSC for re-enlightenment Jonas Theisen
2026-11-18 15:40 ` [PATCH manager 12/15] pvestatd: broadcast host CPU information relevant for live migration Jonas Theisen
2026-11-18 15:40 ` [PATCH manager 13/15] ui: migrate: show why the CPU of the target node is incompatible Jonas Theisen
2026-11-18 15:40 ` [PATCH manager 14/15] fix #6535: ui: migrate: show CPU warnings for the target node Jonas Theisen
2026-11-18 15:40 ` [PATCH manager 15/15] api: nodes: migrateall: skip VMs whose CPU is incompatible with the target Jonas Theisen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118154207.424260-9-j.theisen@proxmox.com \
    --to=j.theisen@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal