* [pve-devel] [PATCH qemu-server v8 2/3] config: add AMD SEV support
2024-04-25 11:24 [pve-devel] [PATCH qemu-server v8 1/3] add C program to get hardware capabilities from CPUID Markus Frank
@ 2024-04-25 11:24 ` Markus Frank
2024-04-25 11:24 ` [pve-devel] [PATCH docs v8 3/3] add AMD SEV documentation Markus Frank
2024-04-25 11:48 ` [pve-devel] [PATCH qemu-server v8 1/3] add C program to get hardware capabilities from CPUID Stefan Sterz
2 siblings, 0 replies; 4+ messages in thread
From: Markus Frank @ 2024-04-25 11:24 UTC (permalink / raw)
To: pve-devel
This patch is for enabling AMD SEV (Secure Encrypted Virtualization)
support in QEMU.
VM-Config-Examples:
amd_sev: type=std,no-debug=1,no-key-sharing=1
amd_sev: es,no-debug=1,kernel-hashes=1
kernel-hashes, reduced-phys-bios & cbitpos correspond to the
variables with the same name in QEMU.
kernel-hashes=1 adds kernel-hashes to enable measured linux kernel
launch since it is per default off for backward compatibility.
reduced-phys-bios and cbitpos are system specific and are read out by
the query-machine-capabilities.service on boot and saved to the
/run/qemu-server/host-hw-capabilities.json file. This file is parsed
and than used by qemu-server to correctly start a AMD SEV VM.
type=std stands for standard sev to differentiate it from sev-es (es)
or sev-snp (snp) when support is upstream.
QEMU's sev-guest policy gets calculated with the parameters nodbg
& noks. These parameters correspond to policy-bits 0 & 1. If type is
'es' than policy-bit 2 gets set to 1 to activate SEV-ES. Policy bit 3
(nosend) is always set to 1, because migration features for sev are
not upstream yet and are attackable.
SEV-ES is highly experimental since it could not be tested.
see coherent doc patch
Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
v8:
* renamed "params" to "capabilities" or "caps"
* renamed "nodbg" to "no-debug" and "noks" to "no-key-sharing"
* untainted json_text as it prevented a SEV VM from starting via GUI
v7:
* adjustments for the changes made in the query-machine-params C program
v6:
* rebase on master
* removed unused $sev_node_fmt object
v5:
* parse /run/amd-sev-params for hardware parameters
* removed NodeConfig dependency
* only disallow live-migration and snapshots with vmstate
-> allow offline migration and snapshots without vmstate
v4:
* reduced lines of code
* added text that SEV-ES is experimental
PVE/API2/Qemu.pm | 11 +++++++
PVE/QemuMigrate.pm | 4 +++
PVE/QemuServer.pm | 79 ++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 94 insertions(+)
diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 2a349c8..c29809d 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -4512,6 +4512,11 @@ __PACKAGE__->register_method({
push $local_resources->@*, "clipboard=vnc";
}
+ # do not allow live migration with AMD SEV enabled
+ if ($res->{running} && $vmconf->{amd_sev}) {
+ push $local_resources->@*, "amd_sev";
+ }
+
# if vm is not running, return target nodes where local storage/mapped devices are available
# for offline migration
if (!$res->{running}) {
@@ -5192,6 +5197,12 @@ __PACKAGE__->register_method({
die "unable to use snapshot name 'pending' (reserved name)\n"
if lc($snapname) eq 'pending';
+ my $conf = PVE::QemuConfig->load_config($vmid);
+ if ($param->{vmstate} && $conf->{amd_sev}) {
+ die "Snapshots that include memory are not supported while memory"
+ ." is encrypted by AMD SEV.\n"
+ }
+
my $realcmd = sub {
PVE::Cluster::log_msg('info', $authuser, "snapshot VM $vmid: $snapname");
PVE::QemuConfig->snapshot_create($vmid, $snapname, $param->{vmstate},
diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
index 8d9b35a..340402a 100644
--- a/PVE/QemuMigrate.pm
+++ b/PVE/QemuMigrate.pm
@@ -260,6 +260,10 @@ sub prepare {
die "VMs with 'clipboard' set to 'vnc' are not live migratable!\n";
}
+ if ($running && $conf->{'amd_sev'}) {
+ die "cannot live-migrate VM when AMD SEV is enabled.\n";
+ }
+
my $vollist = PVE::QemuServer::get_vm_volumes($conf);
my $storages = {};
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 82e7d6a..3417a86 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -177,6 +177,37 @@ my $agent_fmt = {
},
};
+my $sev_fmt = {
+ type => {
+ description => "Enable standard SEV with type='std' or enable"
+ ." experimental SEV-ES with the 'es' option.",
+ type => 'string',
+ default_key => 1,
+ format_description => "sev-type",
+ enum => ['std', 'es'],
+ maxLength => 3,
+ },
+ 'no-debug' => {
+ description => "Sets policy bit 0 to 1 to disallow debugging of guest",
+ type => 'boolean',
+ default => 0,
+ optional => 1,
+ },
+ 'no-key-sharing' => {
+ description => "Sets policy bit 1 to 1 to disallow key sharing with other guests",
+ type => 'boolean',
+ default => 0,
+ optional => 1,
+ },
+ "kernel-hashes" => {
+ description => "Add kernel hashes to guest firmware for measured linux kernel launch",
+ type => 'boolean',
+ default => 0,
+ optional => 1,
+ },
+};
+PVE::JSONSchema::register_format('pve-qemu-sev-fmt', $sev_fmt);
+
my $vga_fmt = {
type => {
description => "Select the VGA type.",
@@ -358,6 +389,12 @@ my $confdesc = {
description => "Memory properties.",
format => $PVE::QemuServer::Memory::memory_fmt
},
+ amd_sev => {
+ description => "Secure Encrypted Virtualization (SEV) features by AMD CPUs",
+ optional => 1,
+ format => 'pve-qemu-sev-fmt',
+ type => 'string',
+ },
balloon => {
optional => 1,
type => 'integer',
@@ -4091,6 +4128,39 @@ sub config_to_command {
}
}
+ if ($conf->{amd_sev}) {
+ if ($conf->{bios} && $conf->{bios} ne 'ovmf') {
+ die "For using SEV you need to change your guest bios to ovmf.\n";
+ }
+
+ my $amd_sev_conf = parse_property_string($sev_fmt, $conf->{amd_sev});
+ my $sev_hw_caps = get_hw_capabilities()->{'amd-sev'};
+
+ if (!$sev_hw_caps->{'sev-support'}) {
+ die "Your CPU does not support AMD SEV!\n";
+ }
+ if ($amd_sev_conf->{type} eq 'es' && !$sev_hw_caps->{'sev-support-es'}) {
+ die "Your CPU does not support AMD SEV-ES!\n";
+ }
+
+ my $sev_mem_object = 'sev-guest,id=sev0'
+ .',cbitpos='.$sev_hw_caps->{cbitpos}
+ .',reduced-phys-bits='.$sev_hw_caps->{'reduced-phys-bits'};
+
+ my $policy = 0b0;
+ $policy += 0b1 if ($amd_sev_conf->{'no-debug'});
+ $policy += 0b10 if ($amd_sev_conf->{'no-key-sharing'});
+ $policy += 0b100 if ($amd_sev_conf->{type} eq 'es');
+ # disable migration with bit 3 nosend to prevent amd-sev-migration-attack
+ $policy += 0b1000;
+
+ $sev_mem_object .= ',policy='.sprintf("%#x", $policy);
+ $sev_mem_object .= ',kernel-hashes=on' if ($amd_sev_conf->{'kernel-hashes'});
+
+ push @$devices, '-object' , $sev_mem_object;
+ push @$machineFlags, 'confidential-guest-support=sev0';
+ }
+
push @$cmd, @$devices;
push @$cmd, '-rtc', join(',', @$rtcFlags) if scalar(@$rtcFlags);
push @$cmd, '-machine', join(',', @$machineFlags) if scalar(@$machineFlags);
@@ -4134,6 +4204,15 @@ sub check_rng_source {
}
}
+sub get_hw_capabilities {
+ # Get reduced-phys-bits & cbitpos from host-hw-capabilities.json
+ my $filename = '/run/qemu-server/host-hw-capabilities.json';
+ my $json_text = PVE::Tools::file_get_contents($filename);
+ ($json_text) = $json_text =~ /(.*)/; # untaint json text
+ my $hw_capabilities = decode_json($json_text);
+ return $hw_capabilities;
+}
+
sub spice_port {
my ($vmid) = @_;
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [pve-devel] [PATCH qemu-server v8 1/3] add C program to get hardware capabilities from CPUID
2024-04-25 11:24 [pve-devel] [PATCH qemu-server v8 1/3] add C program to get hardware capabilities from CPUID Markus Frank
2024-04-25 11:24 ` [pve-devel] [PATCH qemu-server v8 2/3] config: add AMD SEV support Markus Frank
2024-04-25 11:24 ` [pve-devel] [PATCH docs v8 3/3] add AMD SEV documentation Markus Frank
@ 2024-04-25 11:48 ` Stefan Sterz
2 siblings, 0 replies; 4+ messages in thread
From: Stefan Sterz @ 2024-04-25 11:48 UTC (permalink / raw)
To: Proxmox VE development discussion
On Thu Apr 25, 2024 at 1:24 PM CEST, Markus Frank wrote:
> Implement a systemd service that runs a C program that extracts AMD
> SEV hardware information such as reduced-phys-bios and cbitpos from
> CPUID at boot time, looks if SEV, SEV-ES & SEV-SNP are enabled, and
> outputs these details as JSON to /run/qemu-server/hw-params.json.
>
> This programm can also be used to read and save other hardware
> information at boot time.
>
> Signed-off-by: Markus Frank <m.frank@proxmox.com>
> Co-authored-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
> ---
> v8:
> * renamed query-machine-params to query-machine-capabilities
>
> v7:
> * renamed amd-sev-support to query-machine-params
> * mv /run/amd-sev-params to /run/qemu-server/hw-params.json
> * add "mkdir /run/qemu-server" to ensure that the directory exists
> * moved json content to amd-sev property inside a bigger json
> so that other hardware parameters could also be read at boot time and
> included in this json file.
>
> Makefile | 1 +
> query-machine-capabilities/Makefile | 21 +++++++
> .../query-machine-capabilities.c | 55 +++++++++++++++++++
> .../query-machine-capabilities.service | 12 ++++
> 4 files changed, 89 insertions(+)
> create mode 100644 query-machine-capabilities/Makefile
> create mode 100644 query-machine-capabilities/query-machine-capabilities.c
> create mode 100644 query-machine-capabilities/query-machine-capabilities.service
>
> diff --git a/Makefile b/Makefile
> index 133468d..ed67fe0 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -65,6 +65,7 @@ install: $(PKGSOURCES)
> install -m 0644 -D bootsplash.jpg $(DESTDIR)/usr/share/$(PACKAGE)
> $(MAKE) -C PVE install
> $(MAKE) -C qmeventd install
> + $(MAKE) -C query-machine-capabilities install
> $(MAKE) -C qemu-configs install
> $(MAKE) -C vm-network-scripts install
> install -m 0755 qm $(DESTDIR)$(SBINDIR)
> diff --git a/query-machine-capabilities/Makefile b/query-machine-capabilities/Makefile
> new file mode 100644
> index 0000000..c5f6348
> --- /dev/null
> +++ b/query-machine-capabilities/Makefile
> @@ -0,0 +1,21 @@
> +DESTDIR=
> +PREFIX=/usr
> +SBINDIR=${PREFIX}/libexec/qemu-server
> +SERVICEDIR=/lib/systemd/system
> +
> +CC ?= gcc
> +CFLAGS += -O2 -fanalyzer -Werror -Wall -Wextra -Wpedantic -Wtype-limits -Wl,-z,relro -std=gnu11
> +
> +query-machine-capabilities: query-machine-capabilities.c
> + $(CC) $(CFLAGS) -o $@ $< $(LDFLAGS)
> +
> +.PHONY: install
> +install: query-machine-capabilities
> + install -d ${DESTDIR}/${SBINDIR}
> + install -d ${DESTDIR}${SERVICEDIR}
> + install -m 0644 query-machine-capabilities.service ${DESTDIR}${SERVICEDIR}
> + install -m 0755 query-machine-capabilities ${DESTDIR}${SBINDIR}
> +
> +.PHONY: clean
> +clean:
> + rm -f query-machine-capabilities
> diff --git a/query-machine-capabilities/query-machine-capabilities.c b/query-machine-capabilities/query-machine-capabilities.c
> new file mode 100644
> index 0000000..f4a9f9f
> --- /dev/null
> +++ b/query-machine-capabilities/query-machine-capabilities.c
> @@ -0,0 +1,55 @@
> +#include <stdio.h>
> +#include <stdint.h>
> +#include <stdbool.h>
> +#include <sys/stat.h>
> +#include <sys/types.h>
> +
> +int main() {
> + uint32_t eax, ebx, ecx, edx;
> +
> + // query Encrypted Memory Capabilities, see:
> + // https://en.wikipedia.org/wiki/CPUID#EAX=8000001Fh:_Encrypted_Memory_Capabilities
> + uint32_t query_function = 0x8000001F;
> + asm volatile("cpuid"
> + : "=a"(eax), "=b"(ebx), "=c"(ecx), "=d"(edx)
> + : "0"(query_function)
> + );
> +
> + bool sev_support = (eax & (1<<1)) != 0;
> + bool sev_es_support = (eax & (1<<3)) != 0;
> + bool sev_snp_support = (eax & (1<<4)) != 0;
> +
> + uint8_t cbitpos = ebx & 0x3f;
> + uint8_t reduced_phys_bits = (ebx >> 6) & 0x3f;
> +
> + FILE *file;
> + char filename[] = "/run/qemu-server/host-hw-capabilities.json";
> +
> + mkdir("/run/qemu-server/", 0755);
> +
wouldn't it make sense to check whether this call succeeded too like you
do for the `fopen` below? also might be nice to use `strerror` and
handle `errno` in those cases too.
> + file = fopen(filename, "w");
> + if (file == NULL) {
> + perror("Error opening file");
> + return 1;
> + }
> +
> + fprintf(file,
> + "{"
> + " \"amd-sev\": {"
> + " \"cbitpos\": %u,"
> + " \"reduced-phys-bits\": %u,"
> + " \"sev-support\": %s,"
> + " \"sev-support-es\": %s,"
> + " \"sev-support-snp\": %s"
> + " }"
> + " }\n",
> + cbitpos,
> + reduced_phys_bits,
> + sev_support ? "true" : "false",
> + sev_es_support ? "true" : "false",
> + sev_snp_support ? "true" : "false"
> + );
> +
> + fclose(file);
> + return 0;
> +}
> diff --git a/query-machine-capabilities/query-machine-capabilities.service b/query-machine-capabilities/query-machine-capabilities.service
> new file mode 100644
> index 0000000..f926074
> --- /dev/null
> +++ b/query-machine-capabilities/query-machine-capabilities.service
> @@ -0,0 +1,12 @@
> +[Unit]
> +Description=read AMD SEV parameters
> +RequiresMountsFor=/run
> +Before=pve-ha-lrm.service
> +Before=pve-guests.service
> +
> +[Service]
> +ExecStart=/usr/libexec/qemu-server/query-machine-capabilities
> +Type=oneshot
> +
> +[Install]
> +WantedBy=multi-user.target
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 4+ messages in thread