[pve-devel] [PATCH-SERIES v6 qemu-server/manager] add and set x86-64-v2-AES as default model for new vms

[pve-devel] [PATCH-SERIES v6 qemu-server/manager] add and set x86-64-v2-AES as default model for new vms

[Alexandre Derumier]

Hi,

we used kvm64 as default cpumodel since the begin of proxmox. (basically, it's like a pentium4 cpu flags).

New distros like rhel9 are compiled to use more modern cpu flags.
(and windows already use new flags since year, and we already add some extra cpu flags)

"
In 2020, AMD, Intel, Red Hat, and SUSE worked together to define
three microarchitecture levels on top of the historical x86-64
baseline:

  * x86-64-v1: original x86_64 baseline instruction set
  * x86-64-v2: vector instructions up to Streaming SIMD
               Extensions 4.2 (SSE4.2)  and Supplemental
               Streaming SIMD Extensions 3 (SSSE3), the
               POPCNT instruction, and CMPXCHG16B
  * x86-64-v3: vector instructions up to AVX2, MOVBE,
               and additional bit-manipulation instructions.
  * x86-64-v4: vector instructions from some of the
               AVX-512 variants.

"


This patch series add new models inspired from a patch was found on qemu mailing, but never appplied

https://lore.kernel.org/all/20210526144038.278899-1-berrange@redhat.com/T/


In addition to theses model, I have enabled aes too.
I think it's really important, because a lot of users use default values and have
bad performance with ssl and other crypto stuffs.


This was discussed on the qemu mailing

"
Crypto accelerator caveats
==========================
Similarly I'm not a huge fan of leaving out the "aes"
instruction for accelerated crypto, as missing "aes" is
also one of the key factors in making qemu64 a bad choice.

If we include 'aes' in x86-64-v2, then we loose support
for Nehalem hosts.

If we include 'aes' in x86-64-v3 then we further loose
support for Dhyana hosts (an EPYC derived CPU).
"


Nahelemn is a 2008 cpu, so I think it's ok, we are in 2013 ;)
(and user can disable aes flag in gui too)

Dhyana is a chinese fork of epyc, so we don't support the vendor


I still think than enable aes by default is really the more easy,
but for x86-64-v2, just do 1 model without aes (for nehalem), and a
model with aes.

Like this, users don't need to play manually with flags.

This patch series add new models, and set x86-64-v2-AES model as default in pve-manager
wizard only.
(to not break current vm, where kvm64 is the default when cputype is not defined
in configuration)

Here the new builtin models:


x86-64-v1 : not implemented, as it's basicaly qemu64|kvm64 -vme,-cx16 for compat Opteron_G1 from 2004
            so will use it as qemu64|kvm64 is higher are not working on opteron_g1 anyway

x86-64-v2 : Derived from qemu, +popcnt;+pni;+sse4.1;+sse4.2;+ssse3

min intel: Nehalem
min amd : Opteron_G3

x86-64-v2-AES : Derived from qemu, +aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3

min intel: Westmere
min amd : Opteron_G3

x86-64-v3 : Derived from qemu64 +aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe;+xsave

min intel: Haswell
min amd : EPYC_v1

x86-64-v4 : Derived from qemu64 +aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe;+xsave;+avx512f;+avx512bw;+avx512cd;+avx512dq;+avx512vl

min intel: Skylake
min amd : EPYC_v4



changelog v4:
 - remove patches for best cpu detection (maybe do a standalone tool later)
 - use qemu64 as base model and add extra flags
 - add x64-64-v2-AES (default)
 - remove x64-64-v1
 - add x64-64-v4
 - fix fiona comments

changelog v5:
 - fix fiona comments

changelog v6:
 - add missing "xsave" on v3/v4 models


qemu-server:

Alexandre Derumier (1):
  cpuconfig: add new x86-64-vX models

 PVE/QemuServer/CPUConfig.pm | 48 +++++++++++++++++++++++++++++++++----
 1 file changed, 43 insertions(+), 5 deletions(-)

pve-manager:

Alexandre Derumier (1):
  qemu: processor : set x86-64-v2-AES as default cputype for create
    wizard

 www/manager6/qemu/OSDefaults.js | 1 +
 www/manager6/qemu/OSTypeEdit.js | 1 +
 2 files changed, 2 insertions(+)

-- 
2.30.2

[pve-devel] [PATCH v6 qemu-server 1/1] cpuconfig: add new x86-64-vX models

[Alexandre Derumier]

https://gitlab.com/x86-psABIs/x86-64-ABI/
https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg01592.html
"
In 2020, AMD, Intel, Red Hat, and SUSE worked together to define
three microarchitecture levels on top of the historical x86-64
baseline:

  * x86-64:    original x86_64 baseline instruction set
  * x86-64-v2: vector instructions up to Streaming SIMD
               Extensions 4.2 (SSE4.2)  and Supplemental
               Streaming SIMD Extensions 3 (SSSE3), the
               POPCNT instruction, and CMPXCHG16B
  * x86-64-v3: vector instructions up to AVX2, MOVBE,
               and additional bit-manipulation instructions.
  * x86-64-v4: vector instructions from some of the
               AVX-512 variants.
"

This patch add new builtin model derivated from qemu64 model,
to be compatible between intel/amd.

mandatory flags from qemu-doc generator:
https://gitlab.com/qemu/qemu/-/blob/master/scripts/cpu-x86-uarch-abi.py

levels = [
    [ # x86-64 baseline
        "cmov",
        "cx8",
        "fpu",
        "fxsr",
        "mmx",
        "syscall",
        "sse",
        "sse2",
    ],
    [ # x86-64-v2
        "cx16",
        "lahf-lm",
        "popcnt",
        "pni",
        "sse4.1",
        "sse4.2",
        "ssse3",
    ],
    [ # x86-64-v3
        "avx",
        "avx2",
        "bmi1",
        "bmi2",
        "f16c",
        "fma",
        "abm",
        "movbe",
	"xsave"  #missing from qemu doc currently
    ],
    [ # x86-64-v4
        "avx512f",
        "avx512bw",
        "avx512cd",
        "avx512dq",
        "avx512vl",
    ],
]

x86-64-v1 : I'm skipping it, as it's basicaly qemu64|kvm64 -vme,-cx16 for compat Opteron_G1 from 2004
            so will use it as qemu64|kvm64 is higher are not working on opteron_g1 anyway

x86-64-v2 : Derived from qemu, +popcnt;+pni;+sse4.1;+sse4.2;+ssse3

min intel: Nehalem
min amd : Opteron_G3

x86-64-v2-AES : Derived from qemu, +aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3

min intel: Westmere
min amd : Opteron_G3

x86-64-v3 : Derived from qemu64 +aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe+xsave

min intel: Haswell
min amd : EPYC_v1

x86-64-v4 : Derived from qemu64 +aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe;+xsave;+avx512f;+avx512bw;+avx512cd;+avx512dq;+avx512vl

min intel: Skylake
min amd : EPYC_v4

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
---
 PVE/QemuServer/CPUConfig.pm | 48 +++++++++++++++++++++++++++++++++----
 1 file changed, 43 insertions(+), 5 deletions(-)

diff --git a/PVE/QemuServer/CPUConfig.pm b/PVE/QemuServer/CPUConfig.pm
index fb0861b..6ec0356 100644
--- a/PVE/QemuServer/CPUConfig.pm
+++ b/PVE/QemuServer/CPUConfig.pm
@@ -31,6 +31,26 @@ sub load_custom_model_conf {
     return cfs_read_file($default_filename);
 }
 
+#builtin models : reported-model is mandatory
+my $builtin_models = {
+    'x86-64-v2' => {
+	'reported-model' => 'qemu64',
+	flags => "+popcnt;+pni;+sse4.1;+sse4.2;+ssse3",
+    },
+    'x86-64-v2-AES' => {
+	'reported-model' => 'qemu64',
+	flags => "+aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3",
+    },
+    'x86-64-v3' => {
+	'reported-model' => 'qemu64',
+	flags => "+aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe;+xsave",
+    },
+    'x86-64-v4' => {
+	'reported-model' => 'qemu64',
+	flags => "+aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe;+xsave;+avx512f;+avx512bw;+avx512cd;+avx512dq;+avx512vl",
+    },
+};
+
 my $depreacated_cpu_map = {
     # there never was such a client CPU, so map it to the server one for backward compat
     'Icelake-Client' => 'Icelake-Server',
@@ -212,7 +232,7 @@ sub validate_vm_cpu_conf {
 	get_custom_model($cputype);
     } else {
 	die "Built-in cputype '$cputype' is not defined (missing 'custom-' prefix?)\n"
-	    if !defined($cpu_vendor_list->{$cputype});
+	    if !defined($cpu_vendor_list->{$cputype}) && !defined($builtin_models->{$cputype});
     }
 
     # in a VM-specific config, certain properties are limited/forbidden
@@ -302,6 +322,16 @@ sub get_cpu_models {
 	};
     }
 
+    for my $model (keys %{$builtin_models}) {
+	my $reported_model = $builtin_models->{$model}->{'reported-model'};
+	my $vendor = $cpu_vendor_list->{$reported_model};
+	push @$models, {
+	    name => $model,
+	    custom => 0,
+	    vendor => $vendor,
+	};
+    }
+
     return $models if !$include_custom;
 
     my $conf = load_custom_model_conf();
@@ -359,7 +389,9 @@ sub print_cpu_device {
 	    or die "Cannot parse cpu description: $cputype\n";
 	$cpu = $cpuconf->{cputype};
 
-	if (is_custom_model($cpu)) {
+	if (my $model = $builtin_models->{$cpu}) {
+	    $cpu = $model->{'reported-model'};
+	} elsif (is_custom_model($cputype)) {
 	    my $custom_cpu = get_custom_model($cpu);
 
 	    $cpu = $custom_cpu->{'reported-model'} // $cpu_fmt->{'reported-model'}->{default};
@@ -468,14 +500,17 @@ sub get_cpu_options {
 
     my $cpu = {};
     my $custom_cpu;
+    my $builtin_cpu;
     my $hv_vendor_id;
     if (my $cpu_prop_str = $conf->{cpu}) {
 	$cpu = PVE::JSONSchema::parse_property_string('pve-vm-cpu-conf', $cpu_prop_str)
 	    or die "Cannot parse cpu description: $cpu_prop_str\n";
 
 	$cputype = $cpu->{cputype};
-
-	if (is_custom_model($cputype)) {
+	if (my $model = $builtin_models->{$cputype}) {
+	    $cputype = $model->{'reported-model'};
+	    $builtin_cpu->{flags} = $model->{'flags'};
+	} elsif (is_custom_model($cputype)) {
 	    $custom_cpu = get_custom_model($cputype);
 
 	    $cputype = $custom_cpu->{'reported-model'} // $cpu_fmt->{'reported-model'}->{default};
@@ -504,6 +539,9 @@ sub get_cpu_options {
 	)
 	: undef;
 
+    my $builtin_cputype_flags = parse_cpuflag_list(
+	$cpu_flag_any_re, "set by builtin CPU model", $builtin_cpu->{flags});
+
     my $custom_cputype_flags = parse_cpuflag_list(
 	$cpu_flag_any_re, "set by custom CPU model", $custom_cpu->{flags});
 
@@ -534,7 +572,7 @@ sub get_cpu_options {
 
     # will be resolved in parameter order
     $cpu_str .= resolve_cpu_flags(
-	$pve_flags, $hv_flags, $custom_cputype_flags, $vm_flags, $pve_forced_flags);
+	$pve_flags, $hv_flags, $builtin_cputype_flags, $custom_cputype_flags, $vm_flags, $pve_forced_flags);
 
     my $phys_bits = '';
     foreach my $conf ($custom_cpu, $cpu) {
-- 
2.30.2

[pve-devel] [PATCH v3 pve-manager 1/1] qemu: processor : set x86-64-v2-AES as default cputype for create wizard

[Alexandre Derumier]

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
---
 www/manager6/qemu/OSDefaults.js | 1 +
 www/manager6/qemu/OSTypeEdit.js | 1 +
 2 files changed, 2 insertions(+)

diff --git a/www/manager6/qemu/OSDefaults.js b/www/manager6/qemu/OSDefaults.js
index 5e588a58..58bc76ff 100644
--- a/www/manager6/qemu/OSDefaults.js
+++ b/www/manager6/qemu/OSDefaults.js
@@ -43,6 +43,7 @@ Ext.define('PVE.qemu.OSDefaults', {
 		    virtio: 1,
 	    },
 	    scsihw: 'virtio-scsi-single',
+	    cputype: 'x86-64-v2-AES',
 	};
 
        // virtio-net is in kernel since 2.6.25
diff --git a/www/manager6/qemu/OSTypeEdit.js b/www/manager6/qemu/OSTypeEdit.js
index d9a0988e..3332a0bc 100644
--- a/www/manager6/qemu/OSTypeEdit.js
+++ b/www/manager6/qemu/OSTypeEdit.js
@@ -27,6 +27,7 @@ Ext.define('PVE.qemu.OSTypeInputPanel', {
 
 	    me.setWidget('pveBusSelector', targetValues.busType);
 	    me.setWidget('pveNetworkCardSelector', targetValues.networkCard);
+	    me.setWidget('CPUModelSelector', targetValues.cputype);
 	    var scsihw = targetValues.scsihw || '__default__';
 	    this.getViewModel().set('current.scsihw', scsihw);
 	    this.getViewModel().set('current.ostype', ostype);
-- 
2.30.2

[pve-devel] applied-series: [PATCH-SERIES v6 qemu-server/manager] add and set x86-64-v2-AES as default model for new vms

[Fiona Ebner]

Am 08.06.23 um 11:51 schrieb Alexandre Derumier:
> 
> Hi,
> 
> we used kvm64 as default cpumodel since the begin of proxmox. (basically, it's like a pentium4 cpu flags).
> 
> New distros like rhel9 are compiled to use more modern cpu flags.
> (and windows already use new flags since year, and we already add some extra cpu flags)
> 
> "
> In 2020, AMD, Intel, Red Hat, and SUSE worked together to define
> three microarchitecture levels on top of the historical x86-64
> baseline:
> 
>   * x86-64-v1: original x86_64 baseline instruction set
>   * x86-64-v2: vector instructions up to Streaming SIMD
>                Extensions 4.2 (SSE4.2)  and Supplemental
>                Streaming SIMD Extensions 3 (SSSE3), the
>                POPCNT instruction, and CMPXCHG16B
>   * x86-64-v3: vector instructions up to AVX2, MOVBE,
>                and additional bit-manipulation instructions.
>   * x86-64-v4: vector instructions from some of the
>                AVX-512 variants.
> 
> "
> 
> 
> This patch series add new models inspired from a patch was found on qemu mailing, but never appplied
> 
> https://lore.kernel.org/all/20210526144038.278899-1-berrange@redhat.com/T/
> 
> 
> In addition to theses model, I have enabled aes too.
> I think it's really important, because a lot of users use default values and have
> bad performance with ssl and other crypto stuffs.
>

applied series, thanks!

We could bump the dependency pve-manager -> qemu-server, but when having
different versions across nodes, one would still run into an issue if
the backend doesn't know the new model, so not sure if worth it.

Re: [pve-devel] [PATCH-SERIES v6 qemu-server/manager] add and set x86-64-v2-AES as default model for new vms

[Thomas Lamprecht]

Am 08/06/2023 um 11:51 schrieb Alexandre Derumier:
> Here the new builtin models:
> 
> 
> x86-64-v1 : not implemented, as it's basicaly qemu64|kvm64 -vme,-cx16 for compat Opteron_G1 from 2004
>             so will use it as qemu64|kvm64 is higher are not working on opteron_g1 anyway
> 
> x86-64-v2 : Derived from qemu, +popcnt;+pni;+sse4.1;+sse4.2;+ssse3
> 
> min intel: Nehalem
> min amd : Opteron_G3
> 
> x86-64-v2-AES : Derived from qemu, +aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3
> 
> min intel: Westmere

Could confirm this.

> min amd : Opteron_G3

Could not confirm this, QEMU mentions that for this model it means AMD Opteron 23xx
(Gen 3 Class Opteron) CPUs, but I couldn't find any mentioning that those had already
AES support.

The first model that mentioned support for this were the Bulldozer based ones, i.e.,
Opteron 6200-series "Interlagos" and consorts, which QEMU lists under Opteron_G4.

Did I overlook something or is here the minimum really G4? Not that it's a big Problem,
but we need to know exactly what models can work with the new default for support
reasons.

> 
> x86-64-v3 : Derived from qemu64 +aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe;+xsave
> 
> min intel: Haswell
> min amd : EPYC_v1
> 
> x86-64-v4 : Derived from qemu64 +aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe;+xsave;+avx512f;+avx512bw;+avx512cd;+avx512dq;+avx512vl
> 
> min intel: Skylake
> min amd : EPYC_v4

Re: [pve-devel] [PATCH-SERIES v6 qemu-server/manager] add and set x86-64-v2-AES as default model for new vms

[DERUMIER, Alexandre]

Le lundi 19 juin 2023 à 09:11 +0200, Thomas Lamprecht a écrit :
> Am 08/06/2023 um 11:51 schrieb Alexandre Derumier:
> > Here the new builtin models:
> > 
> > 
> > x86-64-v1 : not implemented, as it's basicaly qemu64|kvm64 -vme,-
> > cx16 for compat Opteron_G1 from 2004
> >             so will use it as qemu64|kvm64 is higher are not
> > working on opteron_g1 anyway
> > 
> > x86-64-v2 : Derived from qemu, +popcnt;+pni;+sse4.1;+sse4.2;+ssse3
> > 
> > min intel: Nehalem
> > min amd : Opteron_G3
> > 
> > x86-64-v2-AES : Derived from qemu,
> > +aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3
> > 
> > min intel: Westmere
> 
> Could confirm this.
> 
> > min amd : Opteron_G3
> 
> Could not confirm this, QEMU mentions that for this model it means
> AMD Opteron 23xx
> (Gen 3 Class Opteron) CPUs, but I couldn't find any mentioning that
> those had already
> AES support.

mmm, you are right, AES is only available since opteron g4.

https://github.com/qemu/qemu/blob/master/target/i386/cpu.c