* [pve-devel] [PATCH qemu-server v5 1/3] add C program to get AMD SEV hardware parameters from CPUID @ 2024-04-19 10:10 Markus Frank 2024-04-19 10:10 ` [pve-devel] [PATCH qemu-server v5 2/3] config: QEMU AMD SEV enable Markus Frank 2024-04-19 10:10 ` [pve-devel] [PATCH docs v5 3/3] add AMD SEV documentation Markus Frank 0 siblings, 2 replies; 3+ messages in thread From: Markus Frank @ 2024-04-19 10:10 UTC (permalink / raw) To: pve-devel Implement a systemd service that runs a C program that extracts AMD SEV hardware parameters such as reduced-phys-bios and cbitpos from CPUID at boot time, verifies that SEV, SEV-ES & SEV-SNP are enabled, and outputs these details as JSON to /run/amd-sev-params. Signed-off-by: Markus Frank <m.frank@proxmox.com> --- Makefile | 1 + amd-sev-support/Makefile | 21 +++++++++++ amd-sev-support/amd-sev-support.c | 48 +++++++++++++++++++++++++ amd-sev-support/amd-sev-support.service | 12 +++++++ 4 files changed, 82 insertions(+) create mode 100644 amd-sev-support/Makefile create mode 100644 amd-sev-support/amd-sev-support.c create mode 100644 amd-sev-support/amd-sev-support.service diff --git a/Makefile b/Makefile index 133468d..ccd12a1 100644 --- a/Makefile +++ b/Makefile @@ -65,6 +65,7 @@ install: $(PKGSOURCES) install -m 0644 -D bootsplash.jpg $(DESTDIR)/usr/share/$(PACKAGE) $(MAKE) -C PVE install $(MAKE) -C qmeventd install + $(MAKE) -C amd-sev-support install $(MAKE) -C qemu-configs install $(MAKE) -C vm-network-scripts install install -m 0755 qm $(DESTDIR)$(SBINDIR) diff --git a/amd-sev-support/Makefile b/amd-sev-support/Makefile new file mode 100644 index 0000000..022ed94 --- /dev/null +++ b/amd-sev-support/Makefile @@ -0,0 +1,21 @@ +DESTDIR= +PREFIX=/usr +SBINDIR=${PREFIX}/libexec/qemu-server +SERVICEDIR=/lib/systemd/system + +CC ?= gcc +CFLAGS += -O2 -fanalyzer -Werror -Wall -Wextra -Wpedantic -Wtype-limits -Wl,-z,relro -std=gnu11 + +amd-sev-support: amd-sev-support.c + $(CC) $(CFLAGS) -o $@ $< $(LDFLAGS) + +.PHONY: install +install: amd-sev-support + install -d ${DESTDIR}/${SBINDIR} + install -d ${DESTDIR}${SERVICEDIR} + install -m 0644 amd-sev-support.service ${DESTDIR}${SERVICEDIR} + install -m 0755 amd-sev-support ${DESTDIR}${SBINDIR} + +.PHONY: clean +clean: + rm -f amd-sev-support diff --git a/amd-sev-support/amd-sev-support.c b/amd-sev-support/amd-sev-support.c new file mode 100644 index 0000000..73a7bd8 --- /dev/null +++ b/amd-sev-support/amd-sev-support.c @@ -0,0 +1,48 @@ +#include <stdio.h> +#include <stdint.h> +#include <stdbool.h> + +int main() { + uint32_t eax, ebx, ecx, edx; + + // query Encrypted Memory Capabilities, see: + // https://en.wikipedia.org/wiki/CPUID#EAX=8000001Fh:_Encrypted_Memory_Capabilities + uint32_t query_function = 0x8000001F; + asm volatile("cpuid" + : "=a"(eax), "=b"(ebx), "=c"(ecx), "=d"(edx) + : "0"(query_function) + ); + + bool sev_support = (eax & (1<<1)) != 0; + bool sev_es_support = (eax & (1<<3)) != 0; + bool sev_snp_support = (eax & (1<<4)) != 0; + + uint8_t cbitpos = ebx & 0x3f; + uint8_t reduced_phys_bits = (ebx >> 6) & 0x3f; + + FILE *file; + char *filename = "/run/amd-sev-params"; + + file = fopen(filename, "w"); + if (file == NULL) { + perror("Error opening file"); + return 1; + } + + fprintf(file, "{" + " \"cbitpos\": %u," + " \"reduced-phys-bits\": %u," + " \"sev\": %s," + " \"sev-es\": %s," + " \"sev-snp\": %s" + " }\n", + cbitpos, + reduced_phys_bits, + sev_support ? "true" : "false", + sev_es_support ? "true" : "false", + sev_snp_support ? "true" : "false" + ); + + fclose(file); + return 0; +} diff --git a/amd-sev-support/amd-sev-support.service b/amd-sev-support/amd-sev-support.service new file mode 100644 index 0000000..466dd0a --- /dev/null +++ b/amd-sev-support/amd-sev-support.service @@ -0,0 +1,12 @@ +[Unit] +Description=Read AMD SEV Parameters +RequiresMountsFor=/run +Before=pve-ha-lrm.service +Before=pve-guests.service + +[Service] +ExecStart=/usr/libexec/qemu-server/amd-sev-support +Type=forking + +[Install] +WantedBy=multi-user.target -- 2.39.2 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel ^ permalink raw reply [flat|nested] 3+ messages in thread
* [pve-devel] [PATCH qemu-server v5 2/3] config: QEMU AMD SEV enable 2024-04-19 10:10 [pve-devel] [PATCH qemu-server v5 1/3] add C program to get AMD SEV hardware parameters from CPUID Markus Frank @ 2024-04-19 10:10 ` Markus Frank 2024-04-19 10:10 ` [pve-devel] [PATCH docs v5 3/3] add AMD SEV documentation Markus Frank 1 sibling, 0 replies; 3+ messages in thread From: Markus Frank @ 2024-04-19 10:10 UTC (permalink / raw) To: pve-devel This patch is for enabling AMD SEV (Secure Encrypted Virtualization) support in QEMU VM-Config-Examples: amd_sev: type=std,nodbg=1,noks=1 amd_sev: es,nodbg=1,kernel-hashes=1 Node-Config-Example (gets generated automatically): amd_sev: cbitpos=47,reduced-phys-bios=1 kernel-hashes, reduced-phys-bios & cbitpos correspond to the varibles with the same name in qemu. kernel-hashes=1 adds kernel-hashes to enable measured linux kernel launch since it is per default off for backward compatibility. reduced-phys-bios and cbitpos are system specific and are read out by the amd-sev-support.service on boot and saved to the /run/amd-sev-params file. This file is parsed and than used by qemu-server to correctly start a AMD SEV VM. type=std stands for standard sev to differentiate it from sev-es (es) or sev-snp (snp) when support is upstream. QEMU's sev-guest policy gets calculated with the parameters nodbg & noks These parameters correspond to policy-bits 0 & 1. If type is 'es' than policy-bit 2 gets set to 1 to activate SEV-ES. Policy bit 3 (nosend) is always set to 1, because migration features for sev are not upstream yet and are attackable. SEV-ES is highly experimental since it could not be tested. see coherent doc patch Signed-off-by: Markus Frank <m.frank@proxmox.com> --- v5: * parse /run/amd-sev-params for hardware parameters * removed NodeConfig dependency * only disallow live-migration and snapshots with vmstate -> allow offline migration and snapshots without vmstate v4: * reduced lines of code * added text that SEV-ES is experimental PVE/API2/Qemu.pm | 10 +++++ PVE/QemuMigrate.pm | 4 ++ PVE/QemuServer.pm | 102 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 116 insertions(+) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 497987f..b35ab69 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -4490,6 +4490,10 @@ __PACKAGE__->register_method({ push $local_resources->@*, "clipboard=vnc"; } + if ($res->{running} && $vmconf->{amd_sev}) { + push $local_resources->@*, "amd_sev"; + } + # if vm is not running, return target nodes where local storage/mapped devices are available # for offline migration if (!$res->{running}) { @@ -5170,6 +5174,12 @@ __PACKAGE__->register_method({ die "unable to use snapshot name 'pending' (reserved name)\n" if lc($snapname) eq 'pending'; + my $conf = PVE::QemuConfig->load_config($vmid); + if ($param->{vmstate} && $conf->{amd_sev}) { + die "Snapshots that include memory are not supported while memory" + ." is encrypted by AMD SEV.\n" + } + my $realcmd = sub { PVE::Cluster::log_msg('info', $authuser, "snapshot VM $vmid: $snapname"); PVE::QemuConfig->snapshot_create($vmid, $snapname, $param->{vmstate}, diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index 8d9b35a..7db18b2 100644 --- a/PVE/QemuMigrate.pm +++ b/PVE/QemuMigrate.pm @@ -260,6 +260,10 @@ sub prepare { die "VMs with 'clipboard' set to 'vnc' are not live migratable!\n"; } + if ($running && $conf->{'amd_sev'}) { + die "VMs with AMD SEV are not live migratable!\n"; + } + my $vollist = PVE::QemuServer::get_vm_volumes($conf); my $storages = {}; diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 6e2c805..e1ad190 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -185,6 +185,59 @@ my $agent_fmt = { }, }; +my $sev_fmt = { + type => { + description => "Enable standard SEV with type='std' or enable experimental SEV-ES" + ." with the 'es' option.", + type => 'string', + default_key => 1, + format_description => "qemu-sev-type", + enum => ['std', 'es'], + maxLength => 3, + }, + nodbg => { + description => "Sets policy bit 0 to 1 to disallow debugging of guest", + type => 'boolean', + format_description => "qemu-sev-nodbg", + default => 0, + optional => 1, + }, + noks => { + description => "Sets policy bit 1 to 1 to disallow key sharing with other guests", + type => 'boolean', + format_description => "qemu-sev-noks", + default => 0, + optional => 1, + }, + "kernel-hashes" => { + description => "Add kernel hashes to guest firmware for measured linux kernel launch", + type => 'boolean', + format_description => "qemu-sev-kernel-hashes", + default => 0, + optional => 1, + }, +}; +PVE::JSONSchema::register_format('pve-qemu-sev-fmt', $sev_fmt); + +my $sev_node_fmt = { + cbitpos => { + description => "C-bit: marks if a memory page is protected. System dependent", + type => 'integer', + default => 47, + optional => 1, + minimum => 0, + maximum => 100, + }, + 'reduced-phys-bits' => { + description => "Number of bits the physical address space is reduced by. System dependent", + type => 'integer', + default => 1, + optional => 1, + minimum => 0, + maximum => 100, + }, +}; + my $vga_fmt = { type => { description => "Select the VGA type.", @@ -366,6 +419,12 @@ my $confdesc = { description => "Memory properties.", format => $PVE::QemuServer::Memory::memory_fmt }, + amd_sev => { + description => "Secure Encrypted Virtualization (SEV) features by AMD CPUs", + optional => 1, + format => 'pve-qemu-sev-fmt', + type => 'string', + }, balloon => { optional => 1, type => 'integer', @@ -4084,6 +4143,39 @@ sub config_to_command { } push @$machineFlags, "type=${machine_type_min}"; + if ($conf->{amd_sev}) { + if ($conf->{bios} && $conf->{bios} ne 'ovmf') { + die "For using SEV you need to change your guest bios to ovmf.\n"; + } + + my $amd_sev_conf = parse_property_string($sev_fmt, $conf->{amd_sev}); + my $sev_hw_params = get_sev_hw_parameters(); + + if (!$sev_hw_params->{sev}) { + die "Your CPU does not support AMD SEV!\n"; + } + if ($amd_sev_conf->{type} eq 'es' && !$sev_hw_params->{'sev-es'}) { + die "Your CPU does not support AMD SEV-ES!\n"; + } + + my $sev_mem_object = 'sev-guest,id=sev0' + .',cbitpos='.$sev_hw_params->{cbitpos} + .',reduced-phys-bits='.$sev_hw_params->{'reduced-phys-bits'}; + + my $policy = 0b0; + $policy += 0b1 if ($amd_sev_conf->{nodbg}); + $policy += 0b10 if ($amd_sev_conf->{noks}); + $policy += 0b100 if ($amd_sev_conf->{type} eq 'es'); + # disable migration with bit 3 nosend to prevent amd-sev-migration-attack + $policy += 0b1000; + + $sev_mem_object .= ',policy='.sprintf("%#x", $policy); + $sev_mem_object .= ',kernel-hashes=on' if ($amd_sev_conf->{'kernel-hashes'}); + + push @$devices, '-object' , $sev_mem_object; + push @$machineFlags, 'confidential-guest-support=sev0'; + } + push @$cmd, @$devices; push @$cmd, '-rtc', join(',', @$rtcFlags) if scalar(@$rtcFlags); push @$cmd, '-machine', join(',', @$machineFlags) if scalar(@$machineFlags); @@ -4127,6 +4219,16 @@ sub check_rng_source { } } +sub get_sev_hw_parameters { + # Get reduced-phys-bits & cbitpos from /run/amd-sev-params + my $filename = '/run/amd-sev-params'; + open my $fh, '<', $filename or die "Could not open '$filename' for reading: $!"; + my $json_text = do { local $/; <$fh> }; + close $fh; + my $sev_params = JSON->new->decode($json_text); + return $sev_params; +} + sub spice_port { my ($vmid) = @_; -- 2.39.2 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel ^ permalink raw reply [flat|nested] 3+ messages in thread
* [pve-devel] [PATCH docs v5 3/3] add AMD SEV documentation 2024-04-19 10:10 [pve-devel] [PATCH qemu-server v5 1/3] add C program to get AMD SEV hardware parameters from CPUID Markus Frank 2024-04-19 10:10 ` [pve-devel] [PATCH qemu-server v5 2/3] config: QEMU AMD SEV enable Markus Frank @ 2024-04-19 10:10 ` Markus Frank 1 sibling, 0 replies; 3+ messages in thread From: Markus Frank @ 2024-04-19 10:10 UTC (permalink / raw) To: pve-devel add documentation for the "[PATCH qemu-server] config: QEMU AMD SEV enable" patch. Signed-off-by: Markus Frank <m.frank@proxmox.com> --- v5: * removed NodeConfig part v4: * added text that SEV-ES is experimental qm.adoc | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) diff --git a/qm.adoc b/qm.adoc index 45e3a57..6590373 100644 --- a/qm.adoc +++ b/qm.adoc @@ -715,6 +715,109 @@ systems. When allocating RAM to your VMs, a good rule of thumb is always to leave 1GB of RAM available to the host. +[[qm_memory_encryption]] +Memory Encryption +~~~~~~~~~~~~~~~~~ + +[[qm_memory_encryption_sev]] +AMD SEV +^^^^^^^ + +SEV (Secure Encrypted Virtualization) enables memory encryption per VM using +AES-128 encryption and the AMD Secure Processor. + +SEV-ES (Secure Encrypted Virtualization-Encrypted State) in addition encrypts +all CPU register contents when a VM stops running, to prevent leakage of +information to the hypervisor. This feature is very experimental. + +*Host Requirements:* + +* AMD EPYC CPU +* SEV-ES is only supported on AMD EPYC 7xx2 and newer +* configure AMD memory encryption in the BIOS settings of the host machine +* add "kvm_amd.sev=1" to kernel parameters if not enabled by default +* add "mem_encrypt=on" to kernel parameters if you want to encrypt memory on the +host (SME) see https://www.kernel.org/doc/Documentation/x86/amd-memory-encryption.txt +* maybe increase SWIOTLB see https://github.com/AMDESE/AMDSEV#faq-4 + +To check if SEV is enabled on the host search for `sev` in dmesg and print out +the SEV kernel parameter of kvm_amd: + +---- +# dmesg | grep -i sev +[...] ccp 0000:45:00.1: sev enabled +[...] ccp 0000:45:00.1: SEV API: <buildversion> +[...] SEV supported: <number> ASIDs +[...] SEV-ES supported: <number> ASIDs +# cat /sys/module/kvm_amd/parameters/sev +Y +---- + +*Guest Requirements:* + +* edk2-OVMF +* advisable to use Q35 +* The guest operating system must contain SEV-support. + +*Limitations:* + +* Because the memory is encrypted the memory usage on host is always wrong. +* Operations that involve saving or restoring memory like snapshots +& live migration do not work yet or are attackable. +https://github.com/PSPReverse/amd-sev-migration-attack +* PCI passthrough is not supported. +* SEV-ES is very experimental. +* QEMU & AMD-SEV documentation is very limited. + +Example Configuration: + +---- +# qm set <vmid> -amd_sev type=std,nodbg=1,noks=1,kernel-hashes=1 +---- + +The *type* defines the encryption technology ("type=" is not necessary). +Available options are std & es. + +The QEMU *policy* parameter gets calculated with the *nodbg* and *noks* +parameters. These parameters correspond to policy-bit 0 and 1. If *type* is *es* +the policy-bit 2 is set to 1 so that SEV-ES is enabled. Policy-bit 3 (nosend) is +always set to 1 to prevent migration-attacks. For more information on how to +calculate the policy see: +https://www.amd.com/system/files/TechDocs/55766_SEV-KM_API_Specification.pdf[AMD SEV API Specification Chapter 3] + +The *kernel-hashes* is per default off for backward compatibility with older +OVMF images and guests that do not measure the kernel/initrd. +See https://lists.gnu.org/archive/html/qemu-devel/2021-11/msg02598.html + +*Check if SEV is working on the guest* + +Method 1 - dmesg: + +Output should look like this. + +---- +# dmesg | grep -i sev +AMD Memory Encryption Features active: SEV +---- + +Method 2 - MSR 0xc0010131 (MSR_AMD64_SEV): + +Output should be 1. + +---- +# apt install msr-tools +# modprobe msr +# rdmsr -a 0xc0010131 +1 +---- + +Links: + +* https://developer.amd.com/sev/ +* https://github.com/AMDESE/AMDSEV +* https://www.qemu.org/docs/master/system/i386/amd-memory-encryption.html +* https://www.amd.com/system/files/TechDocs/55766_SEV-KM_API_Specification.pdf +* https://documentation.suse.com/sles/15-SP1/html/SLES-amd-sev/index.html [[qm_network_device]] Network Device -- 2.39.2 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2024-04-19 10:12 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-04-19 10:10 [pve-devel] [PATCH qemu-server v5 1/3] add C program to get AMD SEV hardware parameters from CPUID Markus Frank 2024-04-19 10:10 ` [pve-devel] [PATCH qemu-server v5 2/3] config: QEMU AMD SEV enable Markus Frank 2024-04-19 10:10 ` [pve-devel] [PATCH docs v5 3/3] add AMD SEV documentation Markus Frank
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox