* [pve-devel] [PATCH qemu-server v10 1/4] add C program to get hardware capabilities from CPUID
2024-05-10 11:47 [pve-devel] [PATCH qemu-server/docs/manager v10 0/4] AMD SEV Markus Frank
@ 2024-05-10 11:47 ` Markus Frank
2024-05-17 11:21 ` Dominik Csapak
2024-05-10 11:47 ` [pve-devel] [PATCH qemu-server v10 2/4] config: add AMD SEV support Markus Frank
` (3 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Markus Frank @ 2024-05-10 11:47 UTC (permalink / raw)
To: pve-devel
Implement a systemd service that runs a C program that extracts AMD
SEV hardware information such as reduced-phys-bios and cbitpos from
CPUID at boot time, looks if SEV, SEV-ES & SEV-SNP are enabled, and
outputs these details as JSON to /run/qemu-server/hw-params.json.
This programm can also be used to read and save other hardware
information at boot time.
Signed-off-by: Markus Frank <m.frank@proxmox.com>
Co-authored-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-by: Filip Schauer <f.schauer@proxmox.com>
---
changes v10:
* removed include of sys/types.h
Makefile | 1 +
query-machine-capabilities/Makefile | 21 ++++++
.../query-machine-capabilities.c | 70 +++++++++++++++++++
.../query-machine-capabilities.service | 12 ++++
4 files changed, 104 insertions(+)
create mode 100644 query-machine-capabilities/Makefile
create mode 100644 query-machine-capabilities/query-machine-capabilities.c
create mode 100644 query-machine-capabilities/query-machine-capabilities.service
diff --git a/Makefile b/Makefile
index 133468d..ed67fe0 100644
--- a/Makefile
+++ b/Makefile
@@ -65,6 +65,7 @@ install: $(PKGSOURCES)
install -m 0644 -D bootsplash.jpg $(DESTDIR)/usr/share/$(PACKAGE)
$(MAKE) -C PVE install
$(MAKE) -C qmeventd install
+ $(MAKE) -C query-machine-capabilities install
$(MAKE) -C qemu-configs install
$(MAKE) -C vm-network-scripts install
install -m 0755 qm $(DESTDIR)$(SBINDIR)
diff --git a/query-machine-capabilities/Makefile b/query-machine-capabilities/Makefile
new file mode 100644
index 0000000..c5f6348
--- /dev/null
+++ b/query-machine-capabilities/Makefile
@@ -0,0 +1,21 @@
+DESTDIR=
+PREFIX=/usr
+SBINDIR=${PREFIX}/libexec/qemu-server
+SERVICEDIR=/lib/systemd/system
+
+CC ?= gcc
+CFLAGS += -O2 -fanalyzer -Werror -Wall -Wextra -Wpedantic -Wtype-limits -Wl,-z,relro -std=gnu11
+
+query-machine-capabilities: query-machine-capabilities.c
+ $(CC) $(CFLAGS) -o $@ $< $(LDFLAGS)
+
+.PHONY: install
+install: query-machine-capabilities
+ install -d ${DESTDIR}/${SBINDIR}
+ install -d ${DESTDIR}${SERVICEDIR}
+ install -m 0644 query-machine-capabilities.service ${DESTDIR}${SERVICEDIR}
+ install -m 0755 query-machine-capabilities ${DESTDIR}${SBINDIR}
+
+.PHONY: clean
+clean:
+ rm -f query-machine-capabilities
diff --git a/query-machine-capabilities/query-machine-capabilities.c b/query-machine-capabilities/query-machine-capabilities.c
new file mode 100644
index 0000000..4f18cde
--- /dev/null
+++ b/query-machine-capabilities/query-machine-capabilities.c
@@ -0,0 +1,70 @@
+#include <stdio.h>
+#include <stdint.h>
+#include <stdbool.h>
+#include <sys/stat.h>
+#include <errno.h>
+#include <string.h>
+
+int main() {
+ uint32_t eax, ebx, ecx, edx;
+
+ // query Encrypted Memory Capabilities, see:
+ // https://en.wikipedia.org/wiki/CPUID#EAX=8000001Fh:_Encrypted_Memory_Capabilities
+ uint32_t query_function = 0x8000001F;
+ asm volatile("cpuid"
+ : "=a"(eax), "=b"(ebx), "=c"(ecx), "=d"(edx)
+ : "0"(query_function)
+ );
+
+ bool sev_support = (eax & (1<<1)) != 0;
+ bool sev_es_support = (eax & (1<<3)) != 0;
+ bool sev_snp_support = (eax & (1<<4)) != 0;
+
+ uint8_t cbitpos = ebx & 0x3f;
+ uint8_t reduced_phys_bits = (ebx >> 6) & 0x3f;
+
+ const char *path = "/run/qemu-server/";
+ // Check that the directory exists and create it if it does not.
+ struct stat statbuf;
+ int stats = stat(path, &statbuf);
+ if (stats == 0 && S_ISDIR(statbuf.st_mode)) {
+ printf("Directory %s already exists.\n", path);
+ } else if (errno == ENOENT) {
+ printf("%s does not exist. Creating directory.\n", path);
+ if (mkdir(path, 0755) != 0) {
+ printf("Error creating directory %s: %s\n", path, strerror(errno));
+ return 1;
+ }
+ } else {
+ printf("Error checking path %s: %s\n", path, strerror(errno));
+ return 1;
+ }
+
+ FILE *file;
+ const char *filename = "/run/qemu-server/host-hw-capabilities.json";
+ file = fopen(filename, "w");
+ if (file == NULL) {
+ perror("Error opening file");
+ return 1;
+ }
+
+ fprintf(file,
+ "{"
+ " \"amd-sev\": {"
+ " \"cbitpos\": %u,"
+ " \"reduced-phys-bits\": %u,"
+ " \"sev-support\": %s,"
+ " \"sev-support-es\": %s,"
+ " \"sev-support-snp\": %s"
+ " }"
+ " }\n",
+ cbitpos,
+ reduced_phys_bits,
+ sev_support ? "true" : "false",
+ sev_es_support ? "true" : "false",
+ sev_snp_support ? "true" : "false"
+ );
+
+ fclose(file);
+ return 0;
+}
diff --git a/query-machine-capabilities/query-machine-capabilities.service b/query-machine-capabilities/query-machine-capabilities.service
new file mode 100644
index 0000000..d5bf756
--- /dev/null
+++ b/query-machine-capabilities/query-machine-capabilities.service
@@ -0,0 +1,12 @@
+[Unit]
+Description=PVE Query Machine Capabilities
+RequiresMountsFor=/run
+Before=pve-ha-lrm.service
+Before=pve-guests.service
+
+[Service]
+ExecStart=/usr/libexec/qemu-server/query-machine-capabilities
+Type=oneshot
+
+[Install]
+WantedBy=multi-user.target
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [pve-devel] [PATCH qemu-server v10 1/4] add C program to get hardware capabilities from CPUID
2024-05-10 11:47 ` [pve-devel] [PATCH qemu-server v10 1/4] add C program to get hardware capabilities from CPUID Markus Frank
@ 2024-05-17 11:21 ` Dominik Csapak
2024-05-21 9:51 ` Thomas Lamprecht
0 siblings, 1 reply; 10+ messages in thread
From: Dominik Csapak @ 2024-05-17 11:21 UTC (permalink / raw)
To: Proxmox VE development discussion, Markus Frank
one small nit inline:
On 5/10/24 13:47, Markus Frank wrote:
> diff --git a/query-machine-capabilities/Makefile b/query-machine-capabilities/Makefile
> new file mode 100644
> index 0000000..c5f6348
> --- /dev/null
> +++ b/query-machine-capabilities/Makefile
> @@ -0,0 +1,21 @@
> +DESTDIR=
> +PREFIX=/usr
> +SBINDIR=${PREFIX}/libexec/qemu-server
> +SERVICEDIR=/lib/systemd/system
> +
PREFIX is only used once here, so it's probably better inlining the value
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [pve-devel] [PATCH qemu-server v10 1/4] add C program to get hardware capabilities from CPUID
2024-05-17 11:21 ` Dominik Csapak
@ 2024-05-21 9:51 ` Thomas Lamprecht
0 siblings, 0 replies; 10+ messages in thread
From: Thomas Lamprecht @ 2024-05-21 9:51 UTC (permalink / raw)
To: Proxmox VE development discussion, Dominik Csapak, Markus Frank
Am 17/05/2024 um 13:21 schrieb Dominik Csapak:
> one small nit inline:
>
> On 5/10/24 13:47, Markus Frank wrote:
>> diff --git a/query-machine-capabilities/Makefile b/query-machine-capabilities/Makefile
>> new file mode 100644
>> index 0000000..c5f6348
>> --- /dev/null
>> +++ b/query-machine-capabilities/Makefile
>> @@ -0,0 +1,21 @@
>> +DESTDIR=
>> +PREFIX=/usr
>> +SBINDIR=${PREFIX}/libexec/qemu-server
>> +SERVICEDIR=/lib/systemd/system
>> +
>
> PREFIX is only used once here, so it's probably better inlining the value
No, having the PREFIX variable separate is a common pattern
that allows customizing installation.
Even if we do not need that ourselves, it's still not costing us really
anything to keep following that here and could make comparing changes
between two packages with binaries installed in different paths easier.
So while I wouldn't go through all our build systems and introduce this
variable if missing, I'd also not recommend developers to drop it, as it
is not better to do so.
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* [pve-devel] [PATCH qemu-server v10 2/4] config: add AMD SEV support
2024-05-10 11:47 [pve-devel] [PATCH qemu-server/docs/manager v10 0/4] AMD SEV Markus Frank
2024-05-10 11:47 ` [pve-devel] [PATCH qemu-server v10 1/4] add C program to get hardware capabilities from CPUID Markus Frank
@ 2024-05-10 11:47 ` Markus Frank
2024-05-17 11:21 ` Dominik Csapak
2024-05-10 11:47 ` [pve-devel] [PATCH docs v10 3/4] add AMD SEV documentation Markus Frank
` (2 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Markus Frank @ 2024-05-10 11:47 UTC (permalink / raw)
To: pve-devel
This patch is for enabling AMD SEV (Secure Encrypted Virtualization)
support in QEMU.
VM-Config-Examples:
amd_sev: type=std,no-debug=1,no-key-sharing=1
amd_sev: es,no-debug=1,kernel-hashes=1
kernel-hashes, reduced-phys-bios & cbitpos correspond to the variables
with the same name in QEMU.
kernel-hashes=1 adds kernel-hashes to enable measured linux kernel
launch since it is per default off for backward compatibility.
reduced-phys-bios and cbitpos are system specific and are read out by
the query-machine-capabilities.service on boot and saved to the
/run/qemu-server/host-hw-capabilities.json file. This file is parsed
and than used by qemu-server to correctly start a AMD SEV VM.
type=std stands for standard sev to differentiate it from sev-es (es)
or sev-snp (snp) when support is upstream.
QEMU's sev-guest policy gets calculated with the parameters nodbg
& noks. These parameters correspond to policy-bits 0 & 1. If type is
'es' than policy-bit 2 gets set to 1 to activate SEV-ES. Policy bit 3
(nosend) is always set to 1, because migration features for sev are
not upstream yet and are attackable.
SEV-ES is highly experimental since it could not be tested.
see coherent doc patch
Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
changes v10:
* also die if the BIOS is not set, since the default is SeaBIOS
PVE/API2/Qemu.pm | 11 +++++++
PVE/QemuMigrate.pm | 4 +++
PVE/QemuServer.pm | 79 ++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 94 insertions(+)
diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 2a349c8..c29809d 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -4512,6 +4512,11 @@ __PACKAGE__->register_method({
push $local_resources->@*, "clipboard=vnc";
}
+ # do not allow live migration with AMD SEV enabled
+ if ($res->{running} && $vmconf->{amd_sev}) {
+ push $local_resources->@*, "amd_sev";
+ }
+
# if vm is not running, return target nodes where local storage/mapped devices are available
# for offline migration
if (!$res->{running}) {
@@ -5192,6 +5197,12 @@ __PACKAGE__->register_method({
die "unable to use snapshot name 'pending' (reserved name)\n"
if lc($snapname) eq 'pending';
+ my $conf = PVE::QemuConfig->load_config($vmid);
+ if ($param->{vmstate} && $conf->{amd_sev}) {
+ die "Snapshots that include memory are not supported while memory"
+ ." is encrypted by AMD SEV.\n"
+ }
+
my $realcmd = sub {
PVE::Cluster::log_msg('info', $authuser, "snapshot VM $vmid: $snapname");
PVE::QemuConfig->snapshot_create($vmid, $snapname, $param->{vmstate},
diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
index 8d9b35a..340402a 100644
--- a/PVE/QemuMigrate.pm
+++ b/PVE/QemuMigrate.pm
@@ -260,6 +260,10 @@ sub prepare {
die "VMs with 'clipboard' set to 'vnc' are not live migratable!\n";
}
+ if ($running && $conf->{'amd_sev'}) {
+ die "cannot live-migrate VM when AMD SEV is enabled.\n";
+ }
+
my $vollist = PVE::QemuServer::get_vm_volumes($conf);
my $storages = {};
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 82e7d6a..92960c5 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -177,6 +177,37 @@ my $agent_fmt = {
},
};
+my $sev_fmt = {
+ type => {
+ description => "Enable standard SEV with type='std' or enable"
+ ." experimental SEV-ES with the 'es' option.",
+ type => 'string',
+ default_key => 1,
+ format_description => "sev-type",
+ enum => ['std', 'es'],
+ maxLength => 3,
+ },
+ 'no-debug' => {
+ description => "Sets policy bit 0 to 1 to disallow debugging of guest",
+ type => 'boolean',
+ default => 0,
+ optional => 1,
+ },
+ 'no-key-sharing' => {
+ description => "Sets policy bit 1 to 1 to disallow key sharing with other guests",
+ type => 'boolean',
+ default => 0,
+ optional => 1,
+ },
+ "kernel-hashes" => {
+ description => "Add kernel hashes to guest firmware for measured linux kernel launch",
+ type => 'boolean',
+ default => 0,
+ optional => 1,
+ },
+};
+PVE::JSONSchema::register_format('pve-qemu-sev-fmt', $sev_fmt);
+
my $vga_fmt = {
type => {
description => "Select the VGA type.",
@@ -358,6 +389,12 @@ my $confdesc = {
description => "Memory properties.",
format => $PVE::QemuServer::Memory::memory_fmt
},
+ amd_sev => {
+ description => "Secure Encrypted Virtualization (SEV) features by AMD CPUs",
+ optional => 1,
+ format => 'pve-qemu-sev-fmt',
+ type => 'string',
+ },
balloon => {
optional => 1,
type => 'integer',
@@ -4091,6 +4128,39 @@ sub config_to_command {
}
}
+ if ($conf->{amd_sev}) {
+ if (!$conf->{bios} || ($conf->{bios} && $conf->{bios} ne 'ovmf')) {
+ die "For using SEV you need to change your guest bios to ovmf.\n";
+ }
+
+ my $amd_sev_conf = parse_property_string($sev_fmt, $conf->{amd_sev});
+ my $sev_hw_caps = get_hw_capabilities()->{'amd-sev'};
+
+ if (!$sev_hw_caps->{'sev-support'}) {
+ die "Your CPU does not support AMD SEV!\n";
+ }
+ if ($amd_sev_conf->{type} eq 'es' && !$sev_hw_caps->{'sev-support-es'}) {
+ die "Your CPU does not support AMD SEV-ES!\n";
+ }
+
+ my $sev_mem_object = 'sev-guest,id=sev0'
+ .',cbitpos='.$sev_hw_caps->{cbitpos}
+ .',reduced-phys-bits='.$sev_hw_caps->{'reduced-phys-bits'};
+
+ my $policy = 0b0;
+ $policy += 0b1 if ($amd_sev_conf->{'no-debug'});
+ $policy += 0b10 if ($amd_sev_conf->{'no-key-sharing'});
+ $policy += 0b100 if ($amd_sev_conf->{type} eq 'es');
+ # disable migration with bit 3 nosend to prevent amd-sev-migration-attack
+ $policy += 0b1000;
+
+ $sev_mem_object .= ',policy='.sprintf("%#x", $policy);
+ $sev_mem_object .= ',kernel-hashes=on' if ($amd_sev_conf->{'kernel-hashes'});
+
+ push @$devices, '-object' , $sev_mem_object;
+ push @$machineFlags, 'confidential-guest-support=sev0';
+ }
+
push @$cmd, @$devices;
push @$cmd, '-rtc', join(',', @$rtcFlags) if scalar(@$rtcFlags);
push @$cmd, '-machine', join(',', @$machineFlags) if scalar(@$machineFlags);
@@ -4134,6 +4204,15 @@ sub check_rng_source {
}
}
+sub get_hw_capabilities {
+ # Get reduced-phys-bits & cbitpos from host-hw-capabilities.json
+ my $filename = '/run/qemu-server/host-hw-capabilities.json';
+ my $json_text = PVE::Tools::file_get_contents($filename);
+ ($json_text) = $json_text =~ /(.*)/; # untaint json text
+ my $hw_capabilities = decode_json($json_text);
+ return $hw_capabilities;
+}
+
sub spice_port {
my ($vmid) = @_;
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [pve-devel] [PATCH qemu-server v10 2/4] config: add AMD SEV support
2024-05-10 11:47 ` [pve-devel] [PATCH qemu-server v10 2/4] config: add AMD SEV support Markus Frank
@ 2024-05-17 11:21 ` Dominik Csapak
0 siblings, 0 replies; 10+ messages in thread
From: Dominik Csapak @ 2024-05-17 11:21 UTC (permalink / raw)
To: Proxmox VE development discussion, Markus Frank
comments inline:
On 5/10/24 13:47, Markus Frank wrote:
> This patch is for enabling AMD SEV (Secure Encrypted Virtualization)
> support in QEMU.
>
> VM-Config-Examples:
> amd_sev: type=std,no-debug=1,no-key-sharing=1
> amd_sev: es,no-debug=1,kernel-hashes=1
>
> kernel-hashes, reduced-phys-bios & cbitpos correspond to the variables
> with the same name in QEMU.
>
> kernel-hashes=1 adds kernel-hashes to enable measured linux kernel
> launch since it is per default off for backward compatibility.
>
> reduced-phys-bios and cbitpos are system specific and are read out by
> the query-machine-capabilities.service on boot and saved to the
> /run/qemu-server/host-hw-capabilities.json file. This file is parsed
> and than used by qemu-server to correctly start a AMD SEV VM.
>
> type=std stands for standard sev to differentiate it from sev-es (es)
> or sev-snp (snp) when support is upstream.
>
> QEMU's sev-guest policy gets calculated with the parameters nodbg
> & noks. These parameters correspond to policy-bits 0 & 1. If type is
> 'es' than policy-bit 2 gets set to 1 to activate SEV-ES. Policy bit 3
> (nosend) is always set to 1, because migration features for sev are
> not upstream yet and are attackable.
>
> SEV-ES is highly experimental since it could not be tested.
>
> see coherent doc patch
>
> Signed-off-by: Markus Frank <m.frank@proxmox.com>
> ---
> changes v10:
> * also die if the BIOS is not set, since the default is SeaBIOS
>
> PVE/API2/Qemu.pm | 11 +++++++
> PVE/QemuMigrate.pm | 4 +++
> PVE/QemuServer.pm | 79 ++++++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 94 insertions(+)
>
> diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
> index 2a349c8..c29809d 100644
> --- a/PVE/API2/Qemu.pm
> +++ b/PVE/API2/Qemu.pm
> @@ -4512,6 +4512,11 @@ __PACKAGE__->register_method({
> push $local_resources->@*, "clipboard=vnc";
> }
>
> + # do not allow live migration with AMD SEV enabled
> + if ($res->{running} && $vmconf->{amd_sev}) {
> + push $local_resources->@*, "amd_sev";
> + }
> +
> # if vm is not running, return target nodes where local storage/mapped devices are available
> # for offline migration
> if (!$res->{running}) {
> @@ -5192,6 +5197,12 @@ __PACKAGE__->register_method({
> die "unable to use snapshot name 'pending' (reserved name)\n"
> if lc($snapname) eq 'pending';
>
> + my $conf = PVE::QemuConfig->load_config($vmid);
> + if ($param->{vmstate} && $conf->{amd_sev}) {
> + die "Snapshots that include memory are not supported while memory"
> + ." is encrypted by AMD SEV.\n"
> + }
> +
you do it for snapshots, but it's missing for suspend to disk, where we
basically migrate into a file
> my $realcmd = sub {
> PVE::Cluster::log_msg('info', $authuser, "snapshot VM $vmid: $snapname");
> PVE::QemuConfig->snapshot_create($vmid, $snapname, $param->{vmstate},
> diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
> index 8d9b35a..340402a 100644
> --- a/PVE/QemuMigrate.pm
> +++ b/PVE/QemuMigrate.pm
> @@ -260,6 +260,10 @@ sub prepare {
> die "VMs with 'clipboard' set to 'vnc' are not live migratable!\n";
> }
>
> + if ($running && $conf->{'amd_sev'}) {
> + die "cannot live-migrate VM when AMD SEV is enabled.\n";
> + }
> +
> my $vollist = PVE::QemuServer::get_vm_volumes($conf);
>
> my $storages = {};
> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
> index 82e7d6a..92960c5 100644
> --- a/PVE/QemuServer.pm
> +++ b/PVE/QemuServer.pm
> @@ -177,6 +177,37 @@ my $agent_fmt = {
> },
> };
>
> +my $sev_fmt = {
> + type => {
> + description => "Enable standard SEV with type='std' or enable"
> + ." experimental SEV-ES with the 'es' option.",
> + type => 'string',
> + default_key => 1,
> + format_description => "sev-type",
> + enum => ['std', 'es'],
> + maxLength => 3,
> + },
> + 'no-debug' => {
> + description => "Sets policy bit 0 to 1 to disallow debugging of guest",
> + type => 'boolean',
> + default => 0,
> + optional => 1,
> + },
> + 'no-key-sharing' => {
> + description => "Sets policy bit 1 to 1 to disallow key sharing with other guests",
> + type => 'boolean',
> + default => 0,
> + optional => 1,
> + },
> + "kernel-hashes" => {
> + description => "Add kernel hashes to guest firmware for measured linux kernel launch",
> + type => 'boolean',
> + default => 0,
> + optional => 1,
> + },
> +};
> +PVE::JSONSchema::register_format('pve-qemu-sev-fmt', $sev_fmt);
> +
> my $vga_fmt = {
> type => {
> description => "Select the VGA type.",
> @@ -358,6 +389,12 @@ my $confdesc = {
> description => "Memory properties.",
> format => $PVE::QemuServer::Memory::memory_fmt
> },
> + amd_sev => {
> + description => "Secure Encrypted Virtualization (SEV) features by AMD CPUs",
> + optional => 1,
> + format => 'pve-qemu-sev-fmt',
> + type => 'string',
> + },
> balloon => {
> optional => 1,
> type => 'integer',
> @@ -4091,6 +4128,39 @@ sub config_to_command {
> }
> }
>
> + if ($conf->{amd_sev}) {
> + if (!$conf->{bios} || ($conf->{bios} && $conf->{bios} ne 'ovmf')) {
> + die "For using SEV you need to change your guest bios to ovmf.\n";
> + }
> +
> + my $amd_sev_conf = parse_property_string($sev_fmt, $conf->{amd_sev});
> + my $sev_hw_caps = get_hw_capabilities()->{'amd-sev'};
> +
> + if (!$sev_hw_caps->{'sev-support'}) {
> + die "Your CPU does not support AMD SEV!\n";
> + }
> + if ($amd_sev_conf->{type} eq 'es' && !$sev_hw_caps->{'sev-support-es'}) {
> + die "Your CPU does not support AMD SEV-ES!\n";
> + }
> +
> + my $sev_mem_object = 'sev-guest,id=sev0'
> + .',cbitpos='.$sev_hw_caps->{cbitpos}
> + .',reduced-phys-bits='.$sev_hw_caps->{'reduced-phys-bits'};
> +
> + my $policy = 0b0;
> + $policy += 0b1 if ($amd_sev_conf->{'no-debug'});
> + $policy += 0b10 if ($amd_sev_conf->{'no-key-sharing'});
> + $policy += 0b100 if ($amd_sev_conf->{type} eq 'es');
> + # disable migration with bit 3 nosend to prevent amd-sev-migration-attack
> + $policy += 0b1000;
isn't it possible to keep the bitlength identically? makes it easier to compare
e.g. like this:
my $policy = 0b0000;
$policy += 0b0001 if ...
$policy += 0b0010 if ...
etc..
> +
> + $sev_mem_object .= ',policy='.sprintf("%#x", $policy);
> + $sev_mem_object .= ',kernel-hashes=on' if ($amd_sev_conf->{'kernel-hashes'});
> +
> + push @$devices, '-object' , $sev_mem_object;
> + push @$machineFlags, 'confidential-guest-support=sev0';
> + }
> +
also i'd prefer to put this whole block into e.g. PVE/QemuServer/CPUConfig
so 'config_to_command' does not get more bloated than it already is
> push @$cmd, @$devices;
> push @$cmd, '-rtc', join(',', @$rtcFlags) if scalar(@$rtcFlags);
> push @$cmd, '-machine', join(',', @$machineFlags) if scalar(@$machineFlags);
> @@ -4134,6 +4204,15 @@ sub check_rng_source {
> }
> }
>
> +sub get_hw_capabilities {
> + # Get reduced-phys-bits & cbitpos from host-hw-capabilities.json
> + my $filename = '/run/qemu-server/host-hw-capabilities.json';
> + my $json_text = PVE::Tools::file_get_contents($filename);
> + ($json_text) = $json_text =~ /(.*)/; # untaint json text
> + my $hw_capabilities = decode_json($json_text);
> + return $hw_capabilities;
> +}
> +
also this maybe? though it could also live in 'Helpers'
> sub spice_port {
> my ($vmid) = @_;
>
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* [pve-devel] [PATCH docs v10 3/4] add AMD SEV documentation
2024-05-10 11:47 [pve-devel] [PATCH qemu-server/docs/manager v10 0/4] AMD SEV Markus Frank
2024-05-10 11:47 ` [pve-devel] [PATCH qemu-server v10 1/4] add C program to get hardware capabilities from CPUID Markus Frank
2024-05-10 11:47 ` [pve-devel] [PATCH qemu-server v10 2/4] config: add AMD SEV support Markus Frank
@ 2024-05-10 11:47 ` Markus Frank
2024-05-10 11:47 ` [pve-devel] [PATCH manager v10 4/4] ui: add AMD SEV configuration to Options Markus Frank
2024-05-17 11:21 ` [pve-devel] [PATCH qemu-server/docs/manager v10 0/4] AMD SEV Dominik Csapak
4 siblings, 0 replies; 10+ messages in thread
From: Markus Frank @ 2024-05-10 11:47 UTC (permalink / raw)
To: pve-devel
add documentation for the "[PATCH qemu-server] config: QEMU AMD SEV
enable" patch.
Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
changes v10:
* none
qm.adoc | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 103 insertions(+)
diff --git a/qm.adoc b/qm.adoc
index 42c26db..2001bd4 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -715,6 +715,109 @@ systems.
When allocating RAM to your VMs, a good rule of thumb is always to leave 1GB
of RAM available to the host.
+[[qm_memory_encryption]]
+Memory Encryption
+~~~~~~~~~~~~~~~~~
+
+[[qm_memory_encryption_sev]]
+AMD SEV
+^^^^^^^
+
+SEV (Secure Encrypted Virtualization) enables memory encryption per VM using
+AES-128 encryption and the AMD Secure Processor.
+
+SEV-ES (Secure Encrypted Virtualization-Encrypted State) in addition encrypts
+all CPU register contents when a VM stops running, to prevent leakage of
+information to the hypervisor. This feature is very experimental.
+
+*Host Requirements:*
+
+* AMD EPYC CPU
+* SEV-ES is only supported on AMD EPYC 7xx2 and newer
+* configure AMD memory encryption in the BIOS settings of the host machine
+* add "kvm_amd.sev=1" to kernel parameters if not enabled by default
+* add "mem_encrypt=on" to kernel parameters if you want to encrypt memory on the
+host (SME) see https://www.kernel.org/doc/Documentation/x86/amd-memory-encryption.txt
+* maybe increase SWIOTLB see https://github.com/AMDESE/AMDSEV#faq-4
+
+To check if SEV is enabled on the host search for `sev` in dmesg and print out
+the SEV kernel parameter of kvm_amd:
+
+----
+# dmesg | grep -i sev
+[...] ccp 0000:45:00.1: sev enabled
+[...] ccp 0000:45:00.1: SEV API: <buildversion>
+[...] SEV supported: <number> ASIDs
+[...] SEV-ES supported: <number> ASIDs
+# cat /sys/module/kvm_amd/parameters/sev
+Y
+----
+
+*Guest Requirements:*
+
+* edk2-OVMF
+* advisable to use Q35
+* The guest operating system must contain SEV-support.
+
+*Limitations:*
+
+* Because the memory is encrypted the memory usage on host is always wrong.
+* Operations that involve saving or restoring memory like snapshots
+& live migration do not work yet or are attackable.
+https://github.com/PSPReverse/amd-sev-migration-attack
+* PCI passthrough is not supported.
+* SEV-ES is very experimental.
+* QEMU & AMD-SEV documentation is very limited.
+
+Example Configuration:
+
+----
+# qm set <vmid> -amd_sev type=std,no-debug=1,no-key-sharing=1,kernel-hashes=1
+----
+
+The *type* defines the encryption technology ("type=" is not necessary).
+Available options are std & es.
+
+The QEMU *policy* parameter gets calculated with the *no-debug* and
+*no-key-sharing* parameters. These parameters correspond to policy-bit 0 and 1.
+If *type* is *es* the policy-bit 2 is set to 1 so that SEV-ES is enabled.
+Policy-bit 3 (nosend) is always set to 1 to prevent migration-attacks. For more
+information on how to calculate the policy see:
+https://www.amd.com/system/files/TechDocs/55766_SEV-KM_API_Specification.pdf[AMD SEV API Specification Chapter 3]
+
+The *kernel-hashes* is per default off for backward compatibility with older
+OVMF images and guests that do not measure the kernel/initrd.
+See https://lists.gnu.org/archive/html/qemu-devel/2021-11/msg02598.html
+
+*Check if SEV is working on the guest*
+
+Method 1 - dmesg:
+
+Output should look like this.
+
+----
+# dmesg | grep -i sev
+AMD Memory Encryption Features active: SEV
+----
+
+Method 2 - MSR 0xc0010131 (MSR_AMD64_SEV):
+
+Output should be 1.
+
+----
+# apt install msr-tools
+# modprobe msr
+# rdmsr -a 0xc0010131
+1
+----
+
+Links:
+
+* https://developer.amd.com/sev/
+* https://github.com/AMDESE/AMDSEV
+* https://www.qemu.org/docs/master/system/i386/amd-memory-encryption.html
+* https://www.amd.com/system/files/TechDocs/55766_SEV-KM_API_Specification.pdf
+* https://documentation.suse.com/sles/15-SP1/html/SLES-amd-sev/index.html
[[qm_network_device]]
Network Device
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 10+ messages in thread* [pve-devel] [PATCH manager v10 4/4] ui: add AMD SEV configuration to Options
2024-05-10 11:47 [pve-devel] [PATCH qemu-server/docs/manager v10 0/4] AMD SEV Markus Frank
` (2 preceding siblings ...)
2024-05-10 11:47 ` [pve-devel] [PATCH docs v10 3/4] add AMD SEV documentation Markus Frank
@ 2024-05-10 11:47 ` Markus Frank
2024-05-17 11:21 ` Dominik Csapak
2024-05-17 11:21 ` [pve-devel] [PATCH qemu-server/docs/manager v10 0/4] AMD SEV Dominik Csapak
4 siblings, 1 reply; 10+ messages in thread
From: Markus Frank @ 2024-05-10 11:47 UTC (permalink / raw)
To: pve-devel
By adding a new input panel with an AMD SEV technology selection combo
box and checkboxes for the optional parameters in an advanced section,
the user can configure the amd_sev option via the WebUI's Options tab.
Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
changes v10:
* this patch is new to v10
www/manager6/Makefile | 1 +
www/manager6/qemu/Options.js | 11 ++++
www/manager6/qemu/SevEdit.js | 98 ++++++++++++++++++++++++++++++++++++
3 files changed, 110 insertions(+)
create mode 100644 www/manager6/qemu/SevEdit.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 2c3a822b..801683a3 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -264,6 +264,7 @@ JSSRC= \
qemu/SSHKey.js \
qemu/ScsiHwEdit.js \
qemu/SerialEdit.js \
+ qemu/SevEdit.js \
qemu/Smbios1Edit.js \
qemu/SystemEdit.js \
qemu/USBEdit.js \
diff --git a/www/manager6/qemu/Options.js b/www/manager6/qemu/Options.js
index 7b112400..6907699c 100644
--- a/www/manager6/qemu/Options.js
+++ b/www/manager6/qemu/Options.js
@@ -338,6 +338,17 @@ Ext.define('PVE.qemu.Options', {
},
} : undefined,
},
+ amd_sev: {
+ header: gettext('AMD SEV'),
+ editor: caps.vms['VM.Config.HWType'] ? 'PVE.qemu.SevEdit' : undefined,
+ defaultValue: Proxmox.Utils.defaultText + ' (' + Proxmox.Utils.disabledText + ')',
+ renderer: function(value, metaData, record, ri, ci, store, pending) {
+ let amd_sev = PVE.Parser.parsePropertyString(value, "type");
+ if (amd_sev.type === 'std') return 'AMD SEV (' + value + ')';
+ if (amd_sev.type === 'es') return 'AMD SEV-ES (' + value + ')';
+ return value;
+ },
+ },
hookscript: {
header: gettext('Hookscript'),
},
diff --git a/www/manager6/qemu/SevEdit.js b/www/manager6/qemu/SevEdit.js
new file mode 100644
index 00000000..f0187cde
--- /dev/null
+++ b/www/manager6/qemu/SevEdit.js
@@ -0,0 +1,98 @@
+Ext.define('PVE.qemu.SevInputPanel', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pveSevInputPanel',
+ onlineHelp: 'qm_memory_encryption',
+
+ viewModel: {
+ data: {
+ type: '__default__',
+ },
+ formulas: {
+ sevEnabled: get => get('type') === 'std' || get('type') === 'es',
+ },
+ },
+
+ onGetValues: function(values) {
+ if (values.delete === 'type') {
+ values.delete = 'amd_sev';
+ return values;
+ }
+ let ret = {};
+ ret.amd_sev = PVE.Parser.printPropertyString(values, 'type');
+ return ret;
+ },
+
+ items: {
+ xtype: 'proxmoxKVComboBox',
+ fieldLabel: gettext('AMD Secure Encrypted Virtualization (SEV)'),
+ name: 'type',
+ value: '__default__',
+ comboItems: [
+ ['__default__', Proxmox.Utils.defaultText + ' (' + Proxmox.Utils.disabledText + ')'],
+ ['std', 'AMD SEV'],
+ ['es', 'AMD SEV-ES (highly experimental)'],
+ ],
+ bind: {
+ value: '{type}',
+ },
+ },
+
+ advancedItems: [
+ {
+ xtype: 'proxmoxcheckbox',
+ fieldLabel: gettext('no-debug'),
+ name: 'no-debug',
+ deleteDefaultValue: false,
+ bind: {
+ hidden: '{!sevEnabled}',
+ disabled: '{!sevEnabled}',
+ },
+ },
+ {
+ xtype: 'proxmoxcheckbox',
+ fieldLabel: gettext('no-key-sharing'),
+ name: 'no-key-sharing',
+ deleteDefaultValue: false,
+ bind: {
+ hidden: '{!sevEnabled}',
+ disabled: '{!sevEnabled}',
+ },
+ },
+ {
+ xtype: 'proxmoxcheckbox',
+ fieldLabel: gettext('kernel-hashes'),
+ name: 'kernel-hashes',
+ deleteDefaultValue: false,
+ bind: {
+ hidden: '{!sevEnabled}',
+ disabled: '{!sevEnabled}',
+ },
+ },
+ ],
+});
+
+Ext.define('PVE.qemu.SevEdit', {
+ extend: 'Proxmox.window.Edit',
+
+ subject: gettext('SEV'),
+
+ items: {
+ xtype: 'pveSevInputPanel',
+ },
+
+ width: 400,
+
+ initComponent: function() {
+ let me = this;
+
+ me.callParent();
+
+ me.load({
+ success: function(response) {
+ let conf = response.result.data;
+ let amd_sev = conf.amd_sev || '__default__';
+ me.setValues(PVE.Parser.parsePropertyString(amd_sev, 'type'));
+ },
+ });
+ },
+});
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [pve-devel] [PATCH manager v10 4/4] ui: add AMD SEV configuration to Options
2024-05-10 11:47 ` [pve-devel] [PATCH manager v10 4/4] ui: add AMD SEV configuration to Options Markus Frank
@ 2024-05-17 11:21 ` Dominik Csapak
0 siblings, 0 replies; 10+ messages in thread
From: Dominik Csapak @ 2024-05-17 11:21 UTC (permalink / raw)
To: Proxmox VE development discussion, Markus Frank
comments inline
On 5/10/24 13:47, Markus Frank wrote:
> By adding a new input panel with an AMD SEV technology selection combo
> box and checkboxes for the optional parameters in an advanced section,
> the user can configure the amd_sev option via the WebUI's Options tab.
>
> Signed-off-by: Markus Frank <m.frank@proxmox.com>
> ---
> changes v10:
> * this patch is new to v10
>
> www/manager6/Makefile | 1 +
> www/manager6/qemu/Options.js | 11 ++++
> www/manager6/qemu/SevEdit.js | 98 ++++++++++++++++++++++++++++++++++++
> 3 files changed, 110 insertions(+)
> create mode 100644 www/manager6/qemu/SevEdit.js
>
> diff --git a/www/manager6/Makefile b/www/manager6/Makefile
> index 2c3a822b..801683a3 100644
> --- a/www/manager6/Makefile
> +++ b/www/manager6/Makefile
> @@ -264,6 +264,7 @@ JSSRC= \
> qemu/SSHKey.js \
> qemu/ScsiHwEdit.js \
> qemu/SerialEdit.js \
> + qemu/SevEdit.js \
> qemu/Smbios1Edit.js \
> qemu/SystemEdit.js \
> qemu/USBEdit.js \
> diff --git a/www/manager6/qemu/Options.js b/www/manager6/qemu/Options.js
> index 7b112400..6907699c 100644
> --- a/www/manager6/qemu/Options.js
> +++ b/www/manager6/qemu/Options.js
> @@ -338,6 +338,17 @@ Ext.define('PVE.qemu.Options', {
> },
> } : undefined,
> },
> + amd_sev: {
> + header: gettext('AMD SEV'),
> + editor: caps.vms['VM.Config.HWType'] ? 'PVE.qemu.SevEdit' : undefined,
> + defaultValue: Proxmox.Utils.defaultText + ' (' + Proxmox.Utils.disabledText + ')',
> + renderer: function(value, metaData, record, ri, ci, store, pending) {
> + let amd_sev = PVE.Parser.parsePropertyString(value, "type");
> + if (amd_sev.type === 'std') return 'AMD SEV (' + value + ')';
> + if (amd_sev.type === 'es') return 'AMD SEV-ES (' + value + ')';
> + return value;
> + },
> + },
> hookscript: {
> header: gettext('Hookscript'),
> },
> diff --git a/www/manager6/qemu/SevEdit.js b/www/manager6/qemu/SevEdit.js
> new file mode 100644
> index 00000000..f0187cde
> --- /dev/null
> +++ b/www/manager6/qemu/SevEdit.js
> @@ -0,0 +1,98 @@
> +Ext.define('PVE.qemu.SevInputPanel', {
> + extend: 'Proxmox.panel.InputPanel',
> + xtype: 'pveSevInputPanel',
> + onlineHelp: 'qm_memory_encryption',
> +
> + viewModel: {
> + data: {
> + type: '__default__',
> + },
> + formulas: {
> + sevEnabled: get => get('type') === 'std' || get('type') === 'es',
would'nt that be `get('type') !== '__default__'` ?
makes it shorter and more future proof should we add some other type there
> + },
> + },
> +
> + onGetValues: function(values) {
> + if (values.delete === 'type') {
> + values.delete = 'amd_sev';
> + return values;
> + }
> + let ret = {};
> + ret.amd_sev = PVE.Parser.printPropertyString(values, 'type');
> + return ret;
> + },
> +
> + items: {
> + xtype: 'proxmoxKVComboBox',
> + fieldLabel: gettext('AMD Secure Encrypted Virtualization (SEV)'),
> + name: 'type',
> + value: '__default__',
> + comboItems: [
> + ['__default__', Proxmox.Utils.defaultText + ' (' + Proxmox.Utils.disabledText + ')'],
> + ['std', 'AMD SEV'],
> + ['es', 'AMD SEV-ES (highly experimental)'],
> + ],
> + bind: {
> + value: '{type}',
> + },
> + },
> +
> + advancedItems: [
> + {
> + xtype: 'proxmoxcheckbox',
> + fieldLabel: gettext('no-debug'),
nit: i guess it probably make sense to expose the policy settings with their names
but i really disklike 'no-something' options that one have to enable
we could in the ui reverse it and make the default true?
also IMHO this text is a bit too short
e.g. 'allow debugging' would be nicer?
> + name: 'no-debug',
> + deleteDefaultValue: false,
> + bind: {
> + hidden: '{!sevEnabled}',
> + disabled: '{!sevEnabled}',
> + },
> + },
> + {
> + xtype: 'proxmoxcheckbox',
> + fieldLabel: gettext('no-key-sharing'),
same here...
> + name: 'no-key-sharing',
> + deleteDefaultValue: false,
> + bind: {
> + hidden: '{!sevEnabled}',
> + disabled: '{!sevEnabled}',
> + },
> + },
> + {
> + xtype: 'proxmoxcheckbox',
> + fieldLabel: gettext('kernel-hashes'),
> + name: 'kernel-hashes',
> + deleteDefaultValue: false,
> + bind: {
> + hidden: '{!sevEnabled}',
> + disabled: '{!sevEnabled}',
> + },
> + },
> + ],
> +});
> +
> +Ext.define('PVE.qemu.SevEdit', {
> + extend: 'Proxmox.window.Edit',
> +
> + subject: gettext('SEV'),
> +
> + items: {
> + xtype: 'pveSevInputPanel',
> + },
> +
> + width: 400,
> +
> + initComponent: function() {
> + let me = this;
> +
> + me.callParent();
> +
> + me.load({
> + success: function(response) {
> + let conf = response.result.data;
> + let amd_sev = conf.amd_sev || '__default__';
> + me.setValues(PVE.Parser.parsePropertyString(amd_sev, 'type'));
> + },
> + });
> + },
> +});
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [pve-devel] [PATCH qemu-server/docs/manager v10 0/4] AMD SEV
2024-05-10 11:47 [pve-devel] [PATCH qemu-server/docs/manager v10 0/4] AMD SEV Markus Frank
` (3 preceding siblings ...)
2024-05-10 11:47 ` [pve-devel] [PATCH manager v10 4/4] ui: add AMD SEV configuration to Options Markus Frank
@ 2024-05-17 11:21 ` Dominik Csapak
4 siblings, 0 replies; 10+ messages in thread
From: Dominik Csapak @ 2024-05-17 11:21 UTC (permalink / raw)
To: Proxmox VE development discussion, Markus Frank
all in all the series looks mostly good to me with some minor comments
(see comments on the individual patches)
i could not test it since i don't have an amd epyc system here ;)
one high level thing though (but no hard feelings)
is the systemd service really necessary ?
we could simply call the binary the first time it's needed?
so in qemu-server:
---
if (! -e $cap_path) {
# executing the binary
}
---
before we try to parse it ?
we can still make it a service should we need, but IMHO this seems
overkill for the current use case
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 10+ messages in thread