[PATCH qemu/qemu-server 0/3] include Hyper-V enlightenment flags in CPU flags lists

[PATCH qemu/qemu-server 0/3] include Hyper-V enlightenment flags in CPU flags lists

[Arthur Bied-Charreton]

This series addresses two issues with CPU flag support detection.

First, as reported on the forum [0], the Hyper-V enlightenment flags
(hv-*) are always reported as unsupported. The temporary VM that
query_supported_cpu_flags [1] queries is spawned with `-cpu host`,
which by itself does not advertise those flags. The qemu-server
commit fixes this by additionally enabling hv-passthrough on that VM
when querying flags for KVM.

Second, in the context of custom CPU models, the hv-* flags are not
part of the base list at all, not even as unsupported, making it
impossible to set them without editing cpu-models.conf manually (the
VM-specific flags in the create wizard and processor edit window are
unaffected, since they use a different base list [2]). That base list
is generated from the `-cpu help` output at pve-qemu-kvm build time [3].
The first pve-qemu patch extends the parsing script to additionally
query the hv-* flags via the qom-list-properties QMP command.

The final pve-qemu patch pins the resulting flag list into the
repository and fails the build on any divergence, as already done for
CPU models. This makes sure flag additions are surfaced for review on
QEMU bumps.

The fixes can be applied independently. The qemu-server one is the
most pressing, as it fixes the regression reported in the forum.

[0] https://forum.proxmox.com/threads/incorrect-supported-cpu-flags.183722/#post-853902
[1] https://git.proxmox.com/?p=qemu-server.git;a=blob;f=src/PVE/QemuServer.pm;h=118f26bc94d9ee8e8c4c39a3d710e67c14f61bc0;hb=refs/heads/master#l2941
[2] https://git.proxmox.com/?p=qemu-server.git;a=blob;f=src/PVE/QemuServer/CPUFlags.pm;h=0d3206b2d3119c44a9dd9dd817b838767afbefd7;hb=refs/heads/master#l19
[3] https://git.proxmox.com/?p=pve-qemu.git;a=blob;f=debian/rules;h=c90db29b0f03568224b1c79431bce2b753283a4d;hb=refs/heads/master#l126


qemu-server:

Arthur Bied-Charreton (1):
  cpu flags: include Hyper-V enlightenment flags in supported flags

 src/PVE/QemuServer.pm | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)


pve-qemu:

Arthur Bied-Charreton (2):
  build: query Hyper-V enlightenment flags for CPU flags list
  build: fail when recognized CPU flags list changes

 debian/parse-cpu-flags.pl            |  42 +++
 debian/recognized-CPUID-flags-x86_64 | 412 +++++++++++++++++++++++++++
 debian/rules                         |   6 +-
 3 files changed, 459 insertions(+), 1 deletion(-)
 create mode 100644 debian/recognized-CPUID-flags-x86_64


Summary over all repositories:
  4 files changed, 463 insertions(+), 2 deletions(-)

-- 
Generated by murpp 0.11.0

[PATCH qemu-server 1/3] cpu flags: include Hyper-V enlightenment flags in supported flags

[Arthur Bied-Charreton]

QEMU advertises the Hyper-V enlightenment flags [0] through a different
CPUID range, which query-cpu-model-expansion does not cover by default.

This leads to query_supported_cpu_flags reporting the hv-* flags as
unsupported [1] even though some of them are in the VM-specific flags
and QEMU would accept them when starting a VM.

Enable hv-passthrough in the temporary VM used to query supported flags
for KVM, making it advertise support for the hv-* flags as well.

[0] https://www.qemu.org/docs/master/system/i386/hyperv.html
[1] https://forum.proxmox.com/threads/incorrect-supported-cpu-flags.183722/#post-853902

Signed-off-by: Arthur Bied-Charreton <a.bied-charreton@proxmox.com>
---
 src/PVE/QemuServer.pm | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 118f26bc..239b0cab 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -2982,12 +2982,15 @@ sub query_supported_cpu_flags {
         my $rc = run_command($cmd, noerr => 1, quiet => 0);
         die "QEMU flag querying VM exited with code " . $rc if $rc;
 
+        my $model = { name => $kvm ? 'host' : 'max' };
+        $model->{props} = { 'hv-passthrough' => JSON::true } if $kvm;
+
         eval {
             my $cmd_result = mon_cmd(
                 $fakevmid,
                 'query-cpu-model-expansion',
                 type => 'full',
-                model => { name => $kvm ? 'host' : 'max' },
+                model => $model,
             );
 
             my $props = $cmd_result->{model}->{props};
-- 
2.47.3

[PATCH pve-qemu 2/3] build: query Hyper-V enlightenment flags for CPU flags list

[Arthur Bied-Charreton]

The recognized CPU flags list shipped with pve-qemu-kvm must include
Hyper-V enlightenment flags so the custom CPU models API can offer them.

Query them from the 'host-x86_64-cpu' type via QMP (qom-list-properties)
in parse-cpu-flags.pl, filtering for properties with the 'hv-' prefix,
and include them in the recognized flags list.

Filter out non-boolean enlightenments, since the custom CPU model config
only supports enable/disable.

Suggested-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Arthur Bied-Charreton <a.bied-charreton@proxmox.com>
---
 debian/parse-cpu-flags.pl | 42 +++++++++++++++++++++++++++++++++++++++
 debian/rules              |  2 +-
 2 files changed, 43 insertions(+), 1 deletion(-)

diff --git a/debian/parse-cpu-flags.pl b/debian/parse-cpu-flags.pl
index 1847b3e..2b5d32a 100755
--- a/debian/parse-cpu-flags.pl
+++ b/debian/parse-cpu-flags.pl
@@ -2,7 +2,10 @@
 
 use warnings;
 use strict;
+use IPC::Open2;
+use JSON;
 
+my ($qemu_bin) = @ARGV;
 my @flags = ();
 my $got_flags_section;
 
@@ -20,4 +23,43 @@ while (<STDIN>) {
 
 die "no QEMU/KVM CPU flags detected from STDIN input" if scalar (@flags) <= 0;
 
+my $pid = open2(
+    my $out,
+    my $in,
+    $qemu_bin,
+    '-machine',
+    'none',
+    '-display',
+    'none',
+    '-S',
+    '-qmp',
+    'stdio',
+);
+
+sub qmp {
+    my ($cmd, %args) = @_;
+    print $in encode_json({ execute => $cmd, %args ? (arguments => \%args) : () }), "\n";
+    while (my $line = <$out>) {
+        my $msg = decode_json($line);
+        next if $msg->{event};
+        return $msg->{return} if exists $msg->{return};
+        die "QMP error: " . encode_json($msg->{error}) if $msg->{error};
+    }
+}
+
+<$out>;
+qmp('qmp_capabilities');
+my $props = qmp('qom-list-properties', typename => 'host-x86_64-cpu');
+for my $qo ($props->@*) {
+    # Filter out non-boolean flags, our custom CPU model config only supports
+    # enable/disable.
+    if (index($qo->{name}, 'hv-') == 0 && $qo->{type} eq 'bool') {
+        push @flags, $qo->{name};
+    }
+}
+qmp('quit');
+waitpid($pid, 0);
+
+@flags = sort @flags;
+
 print join("\n", @flags) or die "$!\n";
diff --git a/debian/rules b/debian/rules
index c90db29..e3eebe8 100755
--- a/debian/rules
+++ b/debian/rules
@@ -123,7 +123,7 @@ install: build
 	rm -f $(destdir)/usr/lib/kvm/virtfs-proxy-helper
 
 	# CPU flags are static for QEMU version, allows avoiding more costly checks
-	$(destdir)/usr/bin/qemu-system-x86_64 -cpu help | ./debian/parse-cpu-flags.pl > $(flagfile)
+	$(destdir)/usr/bin/qemu-system-x86_64 -cpu help | ./debian/parse-cpu-flags.pl $(destdir)/usr/bin/qemu-system-x86_64 > $(flagfile)
 
 	# Supported machine versions are static for a given QEMU binary.
 	$(destdir)/usr/bin/qemu-system-x86_64 -machine help | ./debian/parse-machines.pl > $(machine_file_x86_64)
-- 
2.47.3

[PATCH pve-qemu 3/3] build: fail when recognized CPU flags list changes

[Arthur Bied-Charreton]

Check in the current recognized CPU flags list and fail the build if the
list newly generated by parse-cpu-flags.pl differs from it.

This forces new or removed flags to be manually reviewed instead of
being silently picked up on QEMU bumps.

This mirrors the existing approach for CPU models.

Suggested-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Arthur Bied-Charreton <a.bied-charreton@proxmox.com>
---
 debian/recognized-CPUID-flags-x86_64 | 412 +++++++++++++++++++++++++++
 debian/rules                         |   4 +
 2 files changed, 416 insertions(+)
 create mode 100644 debian/recognized-CPUID-flags-x86_64

diff --git a/debian/recognized-CPUID-flags-x86_64 b/debian/recognized-CPUID-flags-x86_64
new file mode 100644
index 0000000..4458eb2
--- /dev/null
+++ b/debian/recognized-CPUID-flags-x86_64
@@ -0,0 +1,412 @@
+3dnow
+3dnowext
+3dnowprefetch
+abm
+ace2
+ace2-en
+acpi
+adx
+aes
+amd-no-ssb
+amd-psfd
+amd-ssbd
+amd-stibp
+amx-bf16
+amx-complex
+amx-fp16
+amx-int8
+amx-tile
+apic
+arat
+arch-capabilities
+arch-lbr
+auto-ibrs
+avic
+avx
+avx-ifma
+avx-ne-convert
+avx-vnni
+avx-vnni-int16
+avx-vnni-int8
+avx10
+avx10-128
+avx10-256
+avx10-512
+avx2
+avx512-4fmaps
+avx512-4vnniw
+avx512-bf16
+avx512-fp16
+avx512-vp2intersect
+avx512-vpopcntdq
+avx512bitalg
+avx512bw
+avx512cd
+avx512dq
+avx512er
+avx512f
+avx512ifma
+avx512pf
+avx512vbmi
+avx512vbmi2
+avx512vl
+avx512vnni
+bhi-ctrl
+bhi-no
+bmi1
+bmi2
+bus-lock-detect
+cid
+cldemote
+clflush
+clflushopt
+clwb
+clzero
+cmov
+cmp-legacy
+cmpccxadd
+core-capability
+cr8legacy
+cx16
+cx8
+dca
+ddpd-u
+de
+decodeassists
+ds
+ds-cpl
+dtes64
+eraps
+erms
+est
+extapic
+f16c
+fb-clear
+fbsdp-no
+fdp-excptn-only
+flush-l1d
+flushbyasid
+fma
+fma4
+fpu
+fred
+fs-gs-base-ns
+fsgsbase
+fsrc
+fsrm
+fsrs
+full-width-write
+fxsr
+fxsr-opt
+fzrm
+gds-no
+gfni
+hle
+ht
+hv-apicv
+hv-avic
+hv-crash
+hv-emsr-bitmap
+hv-enforce-cpuid
+hv-evmcs
+hv-frequencies
+hv-ipi
+hv-passthrough
+hv-reenlightenment
+hv-relaxed
+hv-reset
+hv-runtime
+hv-stimer
+hv-stimer-direct
+hv-syndbg
+hv-synic
+hv-time
+hv-tlbflush
+hv-tlbflush-direct
+hv-tlbflush-ext
+hv-vapic
+hv-vpindex
+hv-xmm-input
+hypervisor
+ia64
+ibpb
+ibpb-brtype
+ibrs
+ibrs-all
+ibs
+intel-psfd
+intel-pt
+intel-pt-lip
+invpcid
+invtsc
+ipred-ctrl
+its-no
+kvm-asyncpf
+kvm-asyncpf-int
+kvm-asyncpf-vmexit
+kvm-hint-dedicated
+kvm-mmu
+kvm-msi-ext-dest-id
+kvm-nopiodelay
+kvm-poll-control
+kvm-pv-eoi
+kvm-pv-ipi
+kvm-pv-sched-yield
+kvm-pv-tlb-flush
+kvm-pv-unhalt
+kvm-steal-time
+kvmclock
+kvmclock
+kvmclock-stable-bit
+la57
+lahf-lm
+lam
+lbrv
+lfence-always-serializing
+lkgs
+lm
+lwp
+mca
+mcdt-no
+mce
+md-clear
+mds-no
+misalignsse
+mmx
+mmxext
+monitor
+movbe
+movdir64b
+movdiri
+mpx
+msr
+msr-imm
+mtrr
+no-nested-data-bp
+nodeid-msr
+npt
+nrip-save
+null-sel-clr-base
+nx
+osvw
+overflow-recov
+pae
+pat
+pause-filter
+pbe
+pbrsb-no
+pcid
+pclmulqdq
+pcommit
+pdcm
+pdpe1gb
+perfctr-core
+perfctr-nb
+perfmon-v2
+pfthreshold
+pge
+phe
+phe-en
+pks
+pku
+pmm
+pmm-en
+pn
+pni
+popcnt
+prefetchi
+prefetchiti
+pschange-mc-no
+psdp-no
+pse
+pse36
+rdctl-no
+rdpid
+rdrand
+rdseed
+rdtscp
+rfds-clear
+rfds-no
+rrsba-ctrl
+rsba
+rtm
+sbdr-ssdp-no
+sbpb
+sep
+serialize
+sgx
+sgx-aex-notify
+sgx-debug
+sgx-edeccssa
+sgx-exinfo
+sgx-kss
+sgx-mode64
+sgx-provisionkey
+sgx-tokenkey
+sgx1
+sgx2
+sgxlc
+sha-ni
+sha512
+skinit
+skip-l1dfl-vmentry
+sm3
+sm4
+smap
+smep
+smx
+spec-ctrl
+split-lock-detect
+srso-no
+srso-user-kernel-no
+ss
+ssb-no
+ssbd
+sse
+sse2
+sse4.1
+sse4.2
+sse4a
+ssse3
+stibp
+stibp-always-on
+succor
+svm
+svm-lock
+svme-addr-chk
+syscall
+taa-no
+tbm
+tce
+tm
+tm2
+topoext
+tsc
+tsc-adjust
+tsc-deadline
+tsc-scale
+tsx-ctrl
+tsx-ldtrk
+umip
+v-vmsave-vmload
+vaes
+vgif
+virt-ssbd
+vmcb-clean
+vme
+vmx
+vmx-activity-hlt
+vmx-activity-shutdown
+vmx-activity-wait-sipi
+vmx-any-errcode
+vmx-apicv-register
+vmx-apicv-vid
+vmx-apicv-x2apic
+vmx-apicv-xapic
+vmx-cr3-load-noexit
+vmx-cr3-store-noexit
+vmx-cr8-load-exit
+vmx-cr8-store-exit
+vmx-desc-exit
+vmx-enable-user-wait-pause
+vmx-encls-exit
+vmx-entry-ia32e-mode
+vmx-entry-load-bndcfgs
+vmx-entry-load-efer
+vmx-entry-load-fred
+vmx-entry-load-pat
+vmx-entry-load-perf-global-ctrl
+vmx-entry-load-pkrs
+vmx-entry-load-rtit-ctl
+vmx-entry-noload-debugctl
+vmx-ept
+vmx-ept-1gb
+vmx-ept-2mb
+vmx-ept-advanced-exitinfo
+vmx-ept-execonly
+vmx-eptad
+vmx-eptp-switching
+vmx-exit-ack-intr
+vmx-exit-clear-bndcfgs
+vmx-exit-clear-rtit-ctl
+vmx-exit-load-efer
+vmx-exit-load-pat
+vmx-exit-load-perf-global-ctrl
+vmx-exit-load-pkrs
+vmx-exit-nosave-debugctl
+vmx-exit-save-efer
+vmx-exit-save-pat
+vmx-exit-save-preemption-timer
+vmx-exit-secondary-ctls
+vmx-flexpriority
+vmx-hlt-exit
+vmx-ins-outs
+vmx-intr-exit
+vmx-invept
+vmx-invept-all-context
+vmx-invept-single-context
+vmx-invept-single-context
+vmx-invept-single-context-noglobals
+vmx-invlpg-exit
+vmx-invpcid-exit
+vmx-invvpid
+vmx-invvpid-all-context
+vmx-invvpid-single-addr
+vmx-io-bitmap
+vmx-io-exit
+vmx-monitor-exit
+vmx-movdr-exit
+vmx-msr-bitmap
+vmx-mtf
+vmx-mwait-exit
+vmx-nested-exception
+vmx-nmi-exit
+vmx-page-walk-4
+vmx-page-walk-5
+vmx-pause-exit
+vmx-ple
+vmx-pml
+vmx-posted-intr
+vmx-preemption-timer
+vmx-rdpmc-exit
+vmx-rdrand-exit
+vmx-rdseed-exit
+vmx-rdtsc-exit
+vmx-rdtscp-exit
+vmx-secondary-ctls
+vmx-shadow-vmcs
+vmx-store-lma
+vmx-true-ctls
+vmx-tsc-offset
+vmx-tsc-scaling
+vmx-unrestricted-guest
+vmx-vintr-pending
+vmx-vmfunc
+vmx-vmwrite-vmexit-fields
+vmx-vnmi
+vmx-vnmi-pending
+vmx-vpid
+vmx-wbinvd-exit
+vmx-xsaves
+vmx-zero-len-inject
+vnmi
+vpclmulqdq
+waitpkg
+wbnoinvd
+wdt
+wrmsrns
+x2apic
+xcrypt
+xcrypt-en
+xfd
+xgetbv1
+xop
+xsave
+xsavec
+xsaveerptr
+xsaveopt
+xsaves
+xstore
+xstore-en
+xtpr
+zero-fcs-fds
\ No newline at end of file
diff --git a/debian/rules b/debian/rules
index e3eebe8..d2bb593 100755
--- a/debian/rules
+++ b/debian/rules
@@ -124,6 +124,10 @@ install: build
 
 	# CPU flags are static for QEMU version, allows avoiding more costly checks
 	$(destdir)/usr/bin/qemu-system-x86_64 -cpu help | ./debian/parse-cpu-flags.pl $(destdir)/usr/bin/qemu-system-x86_64 > $(flagfile)
+	# NOTE: If the diff fails here after upgrading the QEMU submodule, check which new flags
+	# are to be picked up and which are to be excluded, adapt to other changes and commit the
+	# new expected file (and parse-cpu-flags.pl script if that changed as well).
+	diff -u $(flagfile) ./debian/recognized-CPUID-flags-x86_64
 
 	# Supported machine versions are static for a given QEMU binary.
 	$(destdir)/usr/bin/qemu-system-x86_64 -machine help | ./debian/parse-machines.pl > $(machine_file_x86_64)
-- 
2.47.3