[PATCH-SERIES qemu-server v2 0/8] cpu config: support aarch64 CPU models

[PATCH-SERIES qemu-server v2 0/8] cpu config: support aarch64 CPU models

[Fiona Ebner]

Changes in v2:
* also support querying flags for a specific arch
* style: fix include ordering in CPU API module

Previously, only the default 'cortex-a57' CPU model would be used
implicitly. Group models and built-in models by architecture, since
that is what (most) use sites are interested in. Note that the 'host'
model only exists if the host arch matches the emulator/vCPU arch.

Some use sites do require a list of all possible ones, e.g. custom
types, because they are not namespaced by arch.

qemu-server:

Fiona Ebner (8):
  cpu config: introduce pve-qm-cpu-arch standard option for virtual CPU
    architecture
  cpu config: guard adding hyperv enlightenments by arch
  cpu config: 'hidden' option only applies to vCPUs with x86_64 arch
  cpu config: introduce module-wide $host_arch variable
  cpu config: support aarch64 CPU models
  api: cpu: allow querying CPU models for a given architecture
  cpu config: support aarch64 CPU flags
  api: cpu flags: allow querying CPU flags for a given architecture

 src/PVE/API2/Qemu/CPU.pm        |   8 +-
 src/PVE/API2/Qemu/CPUFlags.pm   |  11 +-
 src/PVE/QemuServer.pm           |   7 +-
 src/PVE/QemuServer/CPUConfig.pm | 453 ++++++++++++++++++--------------
 4 files changed, 279 insertions(+), 200 deletions(-)


Summary over all repositories:
  4 files changed, 279 insertions(+), 200 deletions(-)

-- 
Generated by git-murpp 0.5.0

[PATCH qemu-server v2 1/8] cpu config: introduce pve-qm-cpu-arch standard option for virtual CPU architecture

[Fiona Ebner]

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 src/PVE/QemuServer.pm           | 7 +------
 src/PVE/QemuServer/CPUConfig.pm | 7 +++++++
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 341b4321..dae72c40 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -637,12 +637,7 @@ EODESCR
             . ' This is used internally for snapshots.',
     },
     machine => get_standard_option('pve-qemu-machine'),
-    arch => {
-        description => "Virtual processor architecture. Defaults to the host.",
-        optional => 1,
-        type => 'string',
-        enum => [qw(x86_64 aarch64)],
-    },
+    arch => get_standard_option('pve-qm-cpu-arch', { optional => 1 }),
     smbios1 => {
         description => "Specify SMBIOS type 1 fields.",
         type => 'string',
diff --git a/src/PVE/QemuServer/CPUConfig.pm b/src/PVE/QemuServer/CPUConfig.pm
index ec418e73..6240807b 100644
--- a/src/PVE/QemuServer/CPUConfig.pm
+++ b/src/PVE/QemuServer/CPUConfig.pm
@@ -25,6 +25,13 @@ our @EXPORT_OK = qw(
     get_cvm_type
 );
 
+my $arch_desc = {
+    description => "Virtual processor architecture. Defaults to the host architecture.",
+    type => 'string',
+    enum => [qw(x86_64 aarch64)],
+};
+PVE::JSONSchema::register_standard_option("pve-qm-cpu-arch", $arch_desc);
+
 # under certain race-conditions, this module might be loaded before pve-cluster
 # has started completely, so ensure we don't prevent the FUSE mount with our dir
 if (PVE::Cluster::check_cfs_is_mounted(1)) {
-- 
2.47.3

[PATCH qemu-server v2 2/8] cpu config: guard adding hyperv enlightenments by arch

[Fiona Ebner]

These CPU flags are not available for non-x86 archs like aarch64.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 src/PVE/QemuServer/CPUConfig.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/PVE/QemuServer/CPUConfig.pm b/src/PVE/QemuServer/CPUConfig.pm
index 6240807b..dfbe4546 100644
--- a/src/PVE/QemuServer/CPUConfig.pm
+++ b/src/PVE/QemuServer/CPUConfig.pm
@@ -811,7 +811,7 @@ sub get_cpu_options {
     my $pve_flags = get_pve_cpu_flags($conf, $kvm, $cputype, $arch, $machine_version);
 
     my $hv_flags;
-    if ($kvm) {
+    if ($kvm && $arch eq 'x86_64') {
         $hv_flags = get_hyperv_enlightenments(
             $winversion,
             $machine_version,
-- 
2.47.3

[PATCH qemu-server v2 3/8] cpu config: 'hidden' option only applies to vCPUs with x86_64 arch

[Fiona Ebner]

There is no 'kvm' CPU flag for aarch64 CPU types.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 src/PVE/QemuServer/CPUConfig.pm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/src/PVE/QemuServer/CPUConfig.pm b/src/PVE/QemuServer/CPUConfig.pm
index dfbe4546..825e691a 100644
--- a/src/PVE/QemuServer/CPUConfig.pm
+++ b/src/PVE/QemuServer/CPUConfig.pm
@@ -277,7 +277,9 @@ my $cpu_fmt = {
         optional => 1,
     },
     hidden => {
-        description => "Do not identify as a KVM virtual machine.",
+        description =>
+            "Do not identify as a KVM virtual machine. Only affects vCPUs with x86-64"
+            . " architecture.",
         type => 'boolean',
         optional => 1,
         default => 0,
@@ -835,7 +837,7 @@ sub get_cpu_options {
             reason => "error if requested CPU settings not available",
         };
     }
-    if ($kvm_off) {
+    if ($kvm_off && $arch eq 'x86_64') {
         $pve_forced_flags->{'kvm'} = {
             value => "off",
             reason => "hide KVM virtualization from guest",
-- 
2.47.3

[PATCH qemu-server v2 4/8] cpu config: introduce module-wide $host_arch variable

[Fiona Ebner]

It cannot change while the module is loaded. Also, commit "cpu config:
support aarch64 CPU models" will use the host arch as a hash key,
which is cleaner if no additional function call is required.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 src/PVE/QemuServer/CPUConfig.pm | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/src/PVE/QemuServer/CPUConfig.pm b/src/PVE/QemuServer/CPUConfig.pm
index 825e691a..2825da46 100644
--- a/src/PVE/QemuServer/CPUConfig.pm
+++ b/src/PVE/QemuServer/CPUConfig.pm
@@ -9,7 +9,7 @@ use PVE::JSONSchema qw(json_bool);
 use PVE::Cluster qw(cfs_register_file cfs_read_file);
 use PVE::ProcFSTools;
 use PVE::RESTEnvironment qw(log_warn);
-use PVE::Tools qw(run_command get_host_arch);
+use PVE::Tools qw(run_command);
 
 use PVE::QemuServer::Helpers qw(min_version);
 
@@ -25,6 +25,8 @@ our @EXPORT_OK = qw(
     get_cvm_type
 );
 
+my $host_arch = PVE::Tools::get_host_arch();
+
 my $arch_desc = {
     description => "Virtual processor architecture. Defaults to the host architecture.",
     type => 'string',
@@ -1016,13 +1018,13 @@ sub get_default_cpu_type {
 
 sub is_native_arch($) {
     my ($arch) = @_;
-    return get_host_arch() eq $arch;
+    return $host_arch eq $arch;
 }
 
 sub get_cpu_bitness {
     my ($cpu_prop_str, $arch) = @_;
 
-    $arch //= get_host_arch();
+    $arch //= $host_arch;
 
     my $cputype = get_default_cpu_type($arch, 0);
 
-- 
2.47.3

[PATCH qemu-server v2 5/8] cpu config: support aarch64 CPU models

[Fiona Ebner]

Previously, only the default 'cortex-a57' CPU model would be used
implicitly. Group models and built-in models by architecture, since
that is what (most) use sites are interested in. Note that the 'host'
model only exists if the host arch matches the emulator/vCPU arch.

Some use sites do require a list of all possible ones, e.g. custom
types, because they are not namespaced by arch.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 src/PVE/QemuServer/CPUConfig.pm | 289 ++++++++++++++++++--------------
 1 file changed, 167 insertions(+), 122 deletions(-)

diff --git a/src/PVE/QemuServer/CPUConfig.pm b/src/PVE/QemuServer/CPUConfig.pm
index 2825da46..00d56b22 100644
--- a/src/PVE/QemuServer/CPUConfig.pm
+++ b/src/PVE/QemuServer/CPUConfig.pm
@@ -52,27 +52,37 @@ sub load_custom_model_conf {
 }
 
 #builtin models : reported-model is mandatory
-my $builtin_models = {
-    'x86-64-v2' => {
-        'reported-model' => 'qemu64',
-        flags => "+popcnt;+pni;+sse4.1;+sse4.2;+ssse3",
-    },
-    'x86-64-v2-AES' => {
-        'reported-model' => 'qemu64',
-        flags => "+aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3",
-    },
-    'x86-64-v3' => {
-        'reported-model' => 'qemu64',
-        flags =>
-            "+aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe;+xsave",
-    },
-    'x86-64-v4' => {
-        'reported-model' => 'qemu64',
-        flags =>
-            "+aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe;+xsave;+avx512f;+avx512bw;+avx512cd;+avx512dq;+avx512vl",
+my $builtin_models_by_arch = {
+    x86_64 => {
+        'x86-64-v2' => {
+            'reported-model' => 'qemu64',
+            flags => "+popcnt;+pni;+sse4.1;+sse4.2;+ssse3",
+        },
+        'x86-64-v2-AES' => {
+            'reported-model' => 'qemu64',
+            flags => "+aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3",
+        },
+        'x86-64-v3' => {
+            'reported-model' => 'qemu64',
+            flags =>
+                "+aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe;+xsave",
+        },
+        'x86-64-v4' => {
+            'reported-model' => 'qemu64',
+            flags =>
+                "+aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe;+xsave;+avx512f;+avx512bw;+avx512cd;+avx512dq;+avx512vl",
+        },
     },
+    aarch64 => {},
 };
 
+my $all_builtin_models;
+for my $arch (keys $builtin_models_by_arch->%*) {
+    for my $model (keys $builtin_models_by_arch->{$arch}->%*) {
+        $all_builtin_models->{$model} = $builtin_models_by_arch->{$arch}->{$model};
+    }
+}
+
 my $depreacated_cpu_map = {
     # there never was such a client CPU, so map it to the server one for backward compat
     'Icelake-Client' => 'Icelake-Server',
@@ -90,104 +100,130 @@ my $cputypes_32bit = {
     'qemu32' => 1,
 };
 
-my $cpu_vendor_list = {
-    # Intel CPUs
-    486 => 'GenuineIntel',
-    pentium => 'GenuineIntel',
-    pentium2 => 'GenuineIntel',
-    pentium3 => 'GenuineIntel',
-    coreduo => 'GenuineIntel',
-    core2duo => 'GenuineIntel',
-    Conroe => 'GenuineIntel',
-    Penryn => 'GenuineIntel',
-    Nehalem => 'GenuineIntel',
-    'Nehalem-IBRS' => 'GenuineIntel',
-    Westmere => 'GenuineIntel',
-    'Westmere-IBRS' => 'GenuineIntel',
-    SandyBridge => 'GenuineIntel',
-    'SandyBridge-IBRS' => 'GenuineIntel',
-    IvyBridge => 'GenuineIntel',
-    'IvyBridge-IBRS' => 'GenuineIntel',
-    Haswell => 'GenuineIntel',
-    'Haswell-IBRS' => 'GenuineIntel',
-    'Haswell-noTSX' => 'GenuineIntel',
-    'Haswell-noTSX-IBRS' => 'GenuineIntel',
-    Broadwell => 'GenuineIntel',
-    'Broadwell-IBRS' => 'GenuineIntel',
-    'Broadwell-noTSX' => 'GenuineIntel',
-    'Broadwell-noTSX-IBRS' => 'GenuineIntel',
-    'Skylake-Client' => 'GenuineIntel',
-    'Skylake-Client-IBRS' => 'GenuineIntel',
-    'Skylake-Client-noTSX-IBRS' => 'GenuineIntel',
-    'Skylake-Client-v4' => 'GenuineIntel',
-    'Skylake-Server' => 'GenuineIntel',
-    'Skylake-Server-IBRS' => 'GenuineIntel',
-    'Skylake-Server-noTSX-IBRS' => 'GenuineIntel',
-    'Skylake-Server-v4' => 'GenuineIntel',
-    'Skylake-Server-v5' => 'GenuineIntel',
-    'Cascadelake-Server' => 'GenuineIntel',
-    'Cascadelake-Server-v2' => 'GenuineIntel',
-    'Cascadelake-Server-noTSX' => 'GenuineIntel',
-    'Cascadelake-Server-v4' => 'GenuineIntel',
-    'Cascadelake-Server-v5' => 'GenuineIntel',
-    'Cooperlake' => 'GenuineIntel',
-    'Cooperlake-v2' => 'GenuineIntel',
-    KnightsMill => 'GenuineIntel',
-    'Icelake-Client' => 'GenuineIntel', # depreacated, removed with QEMU 7.1
-    'Icelake-Client-noTSX' => 'GenuineIntel', # depreacated, removed with QEMU 7.1
-    'Icelake-Server' => 'GenuineIntel',
-    'Icelake-Server-noTSX' => 'GenuineIntel',
-    'Icelake-Server-v3' => 'GenuineIntel',
-    'Icelake-Server-v4' => 'GenuineIntel',
-    'Icelake-Server-v5' => 'GenuineIntel',
-    'Icelake-Server-v6' => 'GenuineIntel',
-    'Icelake-Server-v7' => 'GenuineIntel',
-    'SapphireRapids' => 'GenuineIntel',
-    'SapphireRapids-v2' => 'GenuineIntel',
-    'SapphireRapids-v3' => 'GenuineIntel',
-    'SapphireRapids-v4' => 'GenuineIntel',
-    'GraniteRapids' => 'GenuineIntel',
-    'GraniteRapids-v2' => 'GenuineIntel',
-    'GraniteRapids-v3' => 'GenuineIntel',
-    'SierraForest' => 'GenuineIntel',
-    'SierraForest-v2' => 'GenuineIntel',
-    'SierraForest-v3' => 'GenuineIntel',
-    'ClearwaterForest' => 'GenuineIntel',
+my $cpu_models_by_arch = {
+    x86_64 => {
+        # Intel CPUs
+        486 => 'GenuineIntel',
+        pentium => 'GenuineIntel',
+        pentium2 => 'GenuineIntel',
+        pentium3 => 'GenuineIntel',
+        coreduo => 'GenuineIntel',
+        core2duo => 'GenuineIntel',
+        Conroe => 'GenuineIntel',
+        Penryn => 'GenuineIntel',
+        Nehalem => 'GenuineIntel',
+        'Nehalem-IBRS' => 'GenuineIntel',
+        Westmere => 'GenuineIntel',
+        'Westmere-IBRS' => 'GenuineIntel',
+        SandyBridge => 'GenuineIntel',
+        'SandyBridge-IBRS' => 'GenuineIntel',
+        IvyBridge => 'GenuineIntel',
+        'IvyBridge-IBRS' => 'GenuineIntel',
+        Haswell => 'GenuineIntel',
+        'Haswell-IBRS' => 'GenuineIntel',
+        'Haswell-noTSX' => 'GenuineIntel',
+        'Haswell-noTSX-IBRS' => 'GenuineIntel',
+        Broadwell => 'GenuineIntel',
+        'Broadwell-IBRS' => 'GenuineIntel',
+        'Broadwell-noTSX' => 'GenuineIntel',
+        'Broadwell-noTSX-IBRS' => 'GenuineIntel',
+        'Skylake-Client' => 'GenuineIntel',
+        'Skylake-Client-IBRS' => 'GenuineIntel',
+        'Skylake-Client-noTSX-IBRS' => 'GenuineIntel',
+        'Skylake-Client-v4' => 'GenuineIntel',
+        'Skylake-Server' => 'GenuineIntel',
+        'Skylake-Server-IBRS' => 'GenuineIntel',
+        'Skylake-Server-noTSX-IBRS' => 'GenuineIntel',
+        'Skylake-Server-v4' => 'GenuineIntel',
+        'Skylake-Server-v5' => 'GenuineIntel',
+        'Cascadelake-Server' => 'GenuineIntel',
+        'Cascadelake-Server-v2' => 'GenuineIntel',
+        'Cascadelake-Server-noTSX' => 'GenuineIntel',
+        'Cascadelake-Server-v4' => 'GenuineIntel',
+        'Cascadelake-Server-v5' => 'GenuineIntel',
+        'Cooperlake' => 'GenuineIntel',
+        'Cooperlake-v2' => 'GenuineIntel',
+        KnightsMill => 'GenuineIntel',
+        'Icelake-Client' => 'GenuineIntel', # depreacated, removed with QEMU 7.1
+        'Icelake-Client-noTSX' => 'GenuineIntel', # depreacated, removed with QEMU 7.1
+        'Icelake-Server' => 'GenuineIntel',
+        'Icelake-Server-noTSX' => 'GenuineIntel',
+        'Icelake-Server-v3' => 'GenuineIntel',
+        'Icelake-Server-v4' => 'GenuineIntel',
+        'Icelake-Server-v5' => 'GenuineIntel',
+        'Icelake-Server-v6' => 'GenuineIntel',
+        'Icelake-Server-v7' => 'GenuineIntel',
+        'SapphireRapids' => 'GenuineIntel',
+        'SapphireRapids-v2' => 'GenuineIntel',
+        'SapphireRapids-v3' => 'GenuineIntel',
+        'SapphireRapids-v4' => 'GenuineIntel',
+        'GraniteRapids' => 'GenuineIntel',
+        'GraniteRapids-v2' => 'GenuineIntel',
+        'GraniteRapids-v3' => 'GenuineIntel',
+        'SierraForest' => 'GenuineIntel',
+        'SierraForest-v2' => 'GenuineIntel',
+        'SierraForest-v3' => 'GenuineIntel',
+        'ClearwaterForest' => 'GenuineIntel',
 
-    # AMD CPUs
-    athlon => 'AuthenticAMD',
-    phenom => 'AuthenticAMD',
-    Opteron_G1 => 'AuthenticAMD',
-    Opteron_G2 => 'AuthenticAMD',
-    Opteron_G3 => 'AuthenticAMD',
-    Opteron_G4 => 'AuthenticAMD',
-    Opteron_G5 => 'AuthenticAMD',
-    EPYC => 'AuthenticAMD',
-    'EPYC-IBPB' => 'AuthenticAMD',
-    'EPYC-v3' => 'AuthenticAMD',
-    'EPYC-v4' => 'AuthenticAMD',
-    'EPYC-v5' => 'AuthenticAMD',
-    'EPYC-Rome' => 'AuthenticAMD',
-    'EPYC-Rome-v2' => 'AuthenticAMD',
-    'EPYC-Rome-v3' => 'AuthenticAMD',
-    'EPYC-Rome-v4' => 'AuthenticAMD',
-    'EPYC-Rome-v5' => 'AuthenticAMD',
-    'EPYC-Milan' => 'AuthenticAMD',
-    'EPYC-Milan-v2' => 'AuthenticAMD',
-    'EPYC-Milan-v3' => 'AuthenticAMD',
-    'EPYC-Genoa' => 'AuthenticAMD',
-    'EPYC-Genoa-v2' => 'AuthenticAMD',
-    'EPYC-Turin' => 'AuthenticAMD',
+        # AMD CPUs
+        athlon => 'AuthenticAMD',
+        phenom => 'AuthenticAMD',
+        Opteron_G1 => 'AuthenticAMD',
+        Opteron_G2 => 'AuthenticAMD',
+        Opteron_G3 => 'AuthenticAMD',
+        Opteron_G4 => 'AuthenticAMD',
+        Opteron_G5 => 'AuthenticAMD',
+        EPYC => 'AuthenticAMD',
+        'EPYC-IBPB' => 'AuthenticAMD',
+        'EPYC-v3' => 'AuthenticAMD',
+        'EPYC-v4' => 'AuthenticAMD',
+        'EPYC-v5' => 'AuthenticAMD',
+        'EPYC-Rome' => 'AuthenticAMD',
+        'EPYC-Rome-v2' => 'AuthenticAMD',
+        'EPYC-Rome-v3' => 'AuthenticAMD',
+        'EPYC-Rome-v4' => 'AuthenticAMD',
+        'EPYC-Rome-v5' => 'AuthenticAMD',
+        'EPYC-Milan' => 'AuthenticAMD',
+        'EPYC-Milan-v2' => 'AuthenticAMD',
+        'EPYC-Milan-v3' => 'AuthenticAMD',
+        'EPYC-Genoa' => 'AuthenticAMD',
+        'EPYC-Genoa-v2' => 'AuthenticAMD',
+        'EPYC-Turin' => 'AuthenticAMD',
 
-    # generic types, use vendor from host node
-    host => 'default',
-    kvm32 => 'default',
-    kvm64 => 'default',
-    qemu32 => 'default',
-    qemu64 => 'default',
-    max => 'default',
+        # generic types, use vendor from host node
+        kvm32 => 'default',
+        kvm64 => 'default',
+        qemu32 => 'default',
+        qemu64 => 'default',
+        max => 'default',
+    },
+    aarch64 => {
+        'a64fx' => 'ARM',
+        'cortex-a35' => 'ARM',
+        'cortex-a53' => 'ARM',
+        'cortex-a55' => 'ARM',
+        'cortex-a57' => 'ARM',
+        'cortex-a710' => 'ARM',
+        'cortex-a72' => 'ARM',
+        'cortex-a76' => 'ARM',
+        'neoverse-n1' => 'ARM',
+        'neoverse-n2' => 'ARM',
+        'neoverse-v1' => 'ARM',
+        # 32 bit and deprecated models were not added
+        max => 'default',
+    },
 };
 
+# The host CPU model only exists if the arch matches
+$cpu_models_by_arch->{$host_arch}->{host} = 'default';
+
+my $all_cpu_models;
+for my $arch (keys $cpu_models_by_arch->%*) {
+    for my $model (keys $cpu_models_by_arch->{$arch}->%*) {
+        $all_cpu_models->{$model} = $cpu_models_by_arch->{$arch}->{$model};
+    }
+}
+
 my $supported_cpu_flags = [
     {
         name => 'nested-virt',
@@ -274,7 +310,7 @@ my $cpu_fmt = {
             "CPU model and vendor to report to the guest. Must be a QEMU/KVM supported model."
             . " Only valid for custom CPU model definitions, default models will always report themselves to the guest OS.",
         type => 'string',
-        enum => [sort { lc("$a") cmp lc("$b") } keys %$cpu_vendor_list],
+        enum => [sort { lc("$a") cmp lc("$b") } keys $all_cpu_models->%*],
         default => 'kvm64',
         optional => 1,
     },
@@ -439,7 +475,7 @@ sub validate_vm_cpu_conf {
     if (is_custom_model($cputype)) {
         # dies on unknown model
         get_custom_model($cputype);
-    } elsif (!defined($cpu_vendor_list->{$cputype}) && !defined($builtin_models->{$cputype})) {
+    } elsif (!defined($all_cpu_models->{$cputype}) && !defined($all_builtin_models->{$cputype})) {
         die "Built-in cputype '$cputype' is not defined (missing 'custom-' prefix?)\n";
     }
 
@@ -529,7 +565,10 @@ sub add_cpu_json_properties {
 }
 
 sub get_cpu_models {
-    my ($include_custom) = @_;
+    my ($include_custom, $arch) = @_;
+
+    $arch = $host_arch if !defined($arch);
+    my $cpu_vendor_list = $cpu_models_by_arch->{$arch};
 
     my $models = [];
 
@@ -542,6 +581,7 @@ sub get_cpu_models {
             };
     }
 
+    my $builtin_models = $builtin_models_by_arch->{$arch};
     for my $model (keys %{$builtin_models}) {
         my $reported_model = $builtin_models->{$model}->{'reported-model'};
         my $vendor = $cpu_vendor_list->{$reported_model};
@@ -559,7 +599,7 @@ sub get_cpu_models {
     for my $custom_model (keys %{ $conf->{ids} }) {
         my $reported_model = $conf->{ids}->{$custom_model}->{'reported-model'};
         $reported_model //= $cpu_fmt->{'reported-model'}->{default};
-        my $vendor = $cpu_vendor_list->{$reported_model};
+        my $vendor = $all_cpu_models->{$reported_model};
         push @$models,
             {
                 name => "custom-$custom_model",
@@ -614,6 +654,7 @@ sub print_cpu_device {
             or die "Cannot parse cpu description: $cputype\n";
         $cpu = $cpuconf->{cputype};
 
+        my $builtin_models = $builtin_models_by_arch->{$arch};
         if (my $model = $builtin_models->{$cpu}) {
             $cpu = $model->{'reported-model'};
         } elsif (is_custom_model($cputype)) {
@@ -792,6 +833,7 @@ sub get_cpu_options {
             or die "Cannot parse cpu description: $cpu_prop_str\n";
 
         $cputype = $cpu->{cputype};
+        my $builtin_models = $builtin_models_by_arch->{$arch};
         if (my $model = $builtin_models->{$cputype}) {
             $cputype = $model->{'reported-model'};
             $builtin_cpu->{flags} = $model->{'flags'};
@@ -812,6 +854,9 @@ sub get_cpu_options {
         $hv_vendor_id = $cpu->{'hv-vendor-id'} if defined($cpu->{'hv-vendor-id'});
     }
 
+    die "CPU model '$cputype' does not exist for configured vCPU architecture '$arch'\n"
+        if !defined($cpu_models_by_arch->{$arch}->{$cputype});
+
     my $pve_flags = get_pve_cpu_flags($conf, $kvm, $cputype, $arch, $machine_version);
 
     my $hv_flags;
@@ -846,13 +891,12 @@ sub get_cpu_options {
         };
     }
 
-    # $cputype is the "reported-model" for custom types, so we can just look up
-    # the vendor in the default list
-    my $cpu_vendor = $cpu_vendor_list->{$cputype};
-    if ($cpu_vendor) {
+    # For aarch64, QEMU does not have a vendor property for the -cpu commandline.
+    if ($arch eq 'x86_64') {
+        # $cputype is the "reported-model" for custom types, so we can just look up
+        # the vendor in the default list
+        my $cpu_vendor = $cpu_models_by_arch->{$arch}->{$cputype} or die "internal error";
         $pve_forced_flags->{'vendor'} = { value => $cpu_vendor } if $cpu_vendor ne 'default';
-    } elsif ($arch ne 'aarch64') {
-        die "internal error"; # should not happen
     }
 
     my $cpu_str = $cputype;
@@ -1034,6 +1078,7 @@ sub get_cpu_bitness {
 
         $cputype = $cpu->{cputype};
 
+        my $builtin_models = $builtin_models_by_arch->{$arch};
         if (my $model = $builtin_models->{$cputype}) {
             $cputype = $model->{'reported-model'};
         } elsif (is_custom_model($cputype)) {
-- 
2.47.3

[PATCH qemu-server v2 6/8] api: cpu: allow querying CPU models for a given architecture

[Fiona Ebner]

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

Changes in v2:
* fix include ordering.

 src/PVE/API2/Qemu/CPU.pm | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/src/PVE/API2/Qemu/CPU.pm b/src/PVE/API2/Qemu/CPU.pm
index 812ef4f7..f8a7e11d 100644
--- a/src/PVE/API2/Qemu/CPU.pm
+++ b/src/PVE/API2/Qemu/CPU.pm
@@ -6,6 +6,7 @@ use warnings;
 use PVE::JSONSchema qw(get_standard_option);
 use PVE::RPCEnvironment;
 use PVE::RESTHandler;
+use PVE::Tools qw(extract_param);
 
 use PVE::QemuServer::CPUConfig;
 
@@ -25,6 +26,7 @@ __PACKAGE__->register_method({
         additionalProperties => 0,
         properties => {
             node => get_standard_option('pve-node'),
+            arch => get_standard_option('pve-qm-cpu-arch', { optional => 1 }),
         },
     },
     returns => {
@@ -53,11 +55,15 @@ __PACKAGE__->register_method({
         links => [{ rel => 'child', href => '{name}' }],
     },
     code => sub {
+        my ($param) = @_;
+
         my $rpcenv = PVE::RPCEnvironment::get();
         my $authuser = $rpcenv->get_user();
         my $include_custom = $rpcenv->check($authuser, "/nodes", ['Sys.Audit'], 1);
 
-        return PVE::QemuServer::CPUConfig::get_cpu_models($include_custom);
+        my $arch = extract_param($param, 'arch');
+
+        return PVE::QemuServer::CPUConfig::get_cpu_models($include_custom, $arch);
     },
 });
 
-- 
2.47.3

[PATCH qemu-server v2 7/8] cpu config: support aarch64 CPU flags

[Fiona Ebner]

Do not add any flags for now and wait until requested. Available would
be (for kvm and tcg):

"aarch64"
"kvm-no-adjvtime" (kvm only)
"kvm-steal-time" (kvm only)
"pauth"
"pmu"
"sve"
"sve1024"
"sve1152"
"sve128"
"sve1280"
"sve1408"
"sve1536"
"sve1664"
"sve1792"
"sve1920"
"sve2048"
"sve256"
"sve384"
"sve512"
"sve640"
"sve768"
"sve896"

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

New in v2.

 src/PVE/QemuServer/CPUConfig.pm | 139 ++++++++++++++++++--------------
 1 file changed, 77 insertions(+), 62 deletions(-)

diff --git a/src/PVE/QemuServer/CPUConfig.pm b/src/PVE/QemuServer/CPUConfig.pm
index 00d56b22..32ec4954 100644
--- a/src/PVE/QemuServer/CPUConfig.pm
+++ b/src/PVE/QemuServer/CPUConfig.pm
@@ -224,72 +224,87 @@ for my $arch (keys $cpu_models_by_arch->%*) {
     }
 }
 
-my $supported_cpu_flags = [
-    {
-        name => 'nested-virt',
-        description => "Controls nested virtualization, namely 'svm' for AMD CPUs and 'vmx' for"
-            . " Intel CPUs. Live migration still only works if it's the same flag on both sides."
-            . " Use a CPU model similar to the host, with the same vendor, not x86-64-vX!",
-    },
-    {
-        name => 'md-clear',
-        description => "Required to let the guest OS know if MDS is mitigated correctly.",
-    },
-    {
-        name => 'pcid',
-        description =>
-            "Meltdown fix cost reduction on Westmere, Sandy-, and IvyBridge Intel CPUs.",
-    },
-    {
-        name => 'spec-ctrl',
-        description => "Allows improved Spectre mitigation with Intel CPUs.",
-    },
-    {
-        name => 'ssbd',
-        description => "Protection for 'Speculative Store Bypass' for Intel models.",
-    },
-    {
-        name => 'ibpb',
-        description => "Allows improved Spectre mitigation with AMD CPUs.",
-    },
-    {
-        name => 'virt-ssbd',
-        description => "Basis for 'Speculative Store Bypass' protection for AMD models.",
-    },
-    {
-        name => 'amd-ssbd',
-        description => "Improves Spectre mitigation performance with AMD CPUs, best used with"
-            . " 'virt-ssbd'.",
-    },
-    {
-        name => 'amd-no-ssb',
-        description => "Notifies guest OS that host is not vulnerable for Spectre on AMD CPUs.",
-    },
-    {
-        name => 'pdpe1gb',
-        description => "Allow guest OS to use 1GB size pages, if host HW supports it.",
-    },
-    {
-        name => 'hv-tlbflush',
-        description => "Improve performance in overcommitted Windows guests. May lead to guest"
-            . " bluescreens on old CPUs.",
-    },
-    {
-        name => 'hv-evmcs',
-        description => "Improve performance for nested virtualization. Only supported on Intel"
-            . " CPUs.",
-    },
-    {
-        name => 'aes',
-        description => "Activate AES instruction set for HW acceleration.",
-    },
-];
+my $supported_cpu_flags_by_arch = {
+    x86_64 => [
+        {
+            name => 'nested-virt',
+            description =>
+                "Controls nested virtualization, namely 'svm' for AMD CPUs and 'vmx' for"
+                . " Intel CPUs. Live migration still only works if it's the same flag on both sides."
+                . " Use a CPU model similar to the host, with the same vendor, not x86-64-vX!",
+        },
+        {
+            name => 'md-clear',
+            description => "Required to let the guest OS know if MDS is mitigated correctly.",
+        },
+        {
+            name => 'pcid',
+            description =>
+                "Meltdown fix cost reduction on Westmere, Sandy-, and IvyBridge Intel CPUs.",
+        },
+        {
+            name => 'spec-ctrl',
+            description => "Allows improved Spectre mitigation with Intel CPUs.",
+        },
+        {
+            name => 'ssbd',
+            description => "Protection for 'Speculative Store Bypass' for Intel models.",
+        },
+        {
+            name => 'ibpb',
+            description => "Allows improved Spectre mitigation with AMD CPUs.",
+        },
+        {
+            name => 'virt-ssbd',
+            description => "Basis for 'Speculative Store Bypass' protection for AMD models.",
+        },
+        {
+            name => 'amd-ssbd',
+            description =>
+                "Improves Spectre mitigation performance with AMD CPUs, best used with"
+                . " 'virt-ssbd'.",
+        },
+        {
+            name => 'amd-no-ssb',
+            description =>
+                "Notifies guest OS that host is not vulnerable for Spectre on AMD CPUs.",
+        },
+        {
+            name => 'pdpe1gb',
+            description => "Allow guest OS to use 1GB size pages, if host HW supports it.",
+        },
+        {
+            name => 'hv-tlbflush',
+            description =>
+                "Improve performance in overcommitted Windows guests. May lead to guest"
+                . " bluescreens on old CPUs.",
+        },
+        {
+            name => 'hv-evmcs',
+            description =>
+                "Improve performance for nested virtualization. Only supported on Intel" . " CPUs.",
+        },
+        {
+            name => 'aes',
+            description => "Activate AES instruction set for HW acceleration.",
+        },
+    ],
+    aarch64 => [],
+};
 
 sub get_supported_cpu_flags {
-    return $supported_cpu_flags;
+    my ($arch) = @_;
+    $arch = $host_arch if !defined($arch);
+    return $supported_cpu_flags_by_arch->{$arch};
 }
 
-my @supported_cpu_flags_names = map { $_->{name} } $supported_cpu_flags->@*;
+my $all_supported_cpu_flags = {};
+for my $arch ($supported_cpu_flags_by_arch->%*) {
+    for my $flag ($supported_cpu_flags_by_arch->{$arch}->@*) {
+        $all_supported_cpu_flags->{ $flag->{name} } = 1;
+    }
+}
+my @supported_cpu_flags_names = sort keys $all_supported_cpu_flags->%*;
 my $cpu_flag_supported_re = qr/([+-])(@{[join('|', @supported_cpu_flags_names)]})/;
 my $cpu_flag_any_re = qr/([+-])([a-zA-Z0-9\-_\.]+)/;
 
-- 
2.47.3

[PATCH qemu-server v2 8/8] api: cpu flags: allow querying CPU flags for a given architecture

[Fiona Ebner]

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

New in v2.

 src/PVE/API2/Qemu/CPUFlags.pm | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/src/PVE/API2/Qemu/CPUFlags.pm b/src/PVE/API2/Qemu/CPUFlags.pm
index 6201654f..672bd2d2 100644
--- a/src/PVE/API2/Qemu/CPUFlags.pm
+++ b/src/PVE/API2/Qemu/CPUFlags.pm
@@ -2,8 +2,10 @@ package PVE::API2::Qemu::CPUFlags;
 
 use v5.36;
 
-use PVE::RESTHandler;
 use PVE::JSONSchema qw(get_standard_option);
+use PVE::RESTHandler;
+use PVE::Tools qw(extract_param);
+
 use PVE::QemuServer::CPUConfig;
 
 use base qw(PVE::RESTHandler);
@@ -18,6 +20,7 @@ __PACKAGE__->register_method({
         additionalProperties => 0,
         properties => {
             node => get_standard_option('pve-node'),
+            arch => get_standard_option('pve-qm-cpu-arch', { optional => 1 }),
         },
     },
     returns => {
@@ -37,7 +40,11 @@ __PACKAGE__->register_method({
         },
     },
     code => sub {
-        return PVE::QemuServer::CPUConfig::get_supported_cpu_flags();
+        my ($param) = @_;
+
+        my $arch = extract_param($param, 'arch');
+
+        return PVE::QemuServer::CPUConfig::get_supported_cpu_flags($arch);
     },
 });
 
-- 
2.47.3