* [pve-devel] [PATCH qemu-server v5 1/3] add C program to get AMD SEV hardware parameters from CPUID @ 2024-04-19 10:10 Markus Frank 2024-04-19 10:10 ` [pve-devel] [PATCH qemu-server v5 2/3] config: QEMU AMD SEV enable Markus Frank 2024-04-19 10:10 ` [pve-devel] [PATCH docs v5 3/3] add AMD SEV documentation Markus Frank 0 siblings, 2 replies; 3+ messages in thread From: Markus Frank @ 2024-04-19 10:10 UTC (permalink / raw) To: pve-devel Implement a systemd service that runs a C program that extracts AMD SEV hardware parameters such as reduced-phys-bios and cbitpos from CPUID at boot time, verifies that SEV, SEV-ES & SEV-SNP are enabled, and outputs these details as JSON to /run/amd-sev-params. Signed-off-by: Markus Frank <m.frank@proxmox.com> --- Makefile | 1 + amd-sev-support/Makefile | 21 +++++++++++ amd-sev-support/amd-sev-support.c | 48 +++++++++++++++++++++++++ amd-sev-support/amd-sev-support.service | 12 +++++++ 4 files changed, 82 insertions(+) create mode 100644 amd-sev-support/Makefile create mode 100644 amd-sev-support/amd-sev-support.c create mode 100644 amd-sev-support/amd-sev-support.service diff --git a/Makefile b/Makefile index 133468d..ccd12a1 100644 --- a/Makefile +++ b/Makefile @@ -65,6 +65,7 @@ install: $(PKGSOURCES) install -m 0644 -D bootsplash.jpg $(DESTDIR)/usr/share/$(PACKAGE) $(MAKE) -C PVE install $(MAKE) -C qmeventd install + $(MAKE) -C amd-sev-support install $(MAKE) -C qemu-configs install $(MAKE) -C vm-network-scripts install install -m 0755 qm $(DESTDIR)$(SBINDIR) diff --git a/amd-sev-support/Makefile b/amd-sev-support/Makefile new file mode 100644 index 0000000..022ed94 --- /dev/null +++ b/amd-sev-support/Makefile @@ -0,0 +1,21 @@ +DESTDIR= +PREFIX=/usr +SBINDIR=${PREFIX}/libexec/qemu-server +SERVICEDIR=/lib/systemd/system + +CC ?= gcc +CFLAGS += -O2 -fanalyzer -Werror -Wall -Wextra -Wpedantic -Wtype-limits -Wl,-z,relro -std=gnu11 + +amd-sev-support: amd-sev-support.c + $(CC) $(CFLAGS) -o $@ $< $(LDFLAGS) + +.PHONY: install +install: amd-sev-support + install -d ${DESTDIR}/${SBINDIR} + install -d ${DESTDIR}${SERVICEDIR} + install -m 0644 amd-sev-support.service ${DESTDIR}${SERVICEDIR} + install -m 0755 amd-sev-support ${DESTDIR}${SBINDIR} + +.PHONY: clean +clean: + rm -f amd-sev-support diff --git a/amd-sev-support/amd-sev-support.c b/amd-sev-support/amd-sev-support.c new file mode 100644 index 0000000..73a7bd8 --- /dev/null +++ b/amd-sev-support/amd-sev-support.c @@ -0,0 +1,48 @@ +#include <stdio.h> +#include <stdint.h> +#include <stdbool.h> + +int main() { + uint32_t eax, ebx, ecx, edx; + + // query Encrypted Memory Capabilities, see: + // https://en.wikipedia.org/wiki/CPUID#EAX=8000001Fh:_Encrypted_Memory_Capabilities + uint32_t query_function = 0x8000001F; + asm volatile("cpuid" + : "=a"(eax), "=b"(ebx), "=c"(ecx), "=d"(edx) + : "0"(query_function) + ); + + bool sev_support = (eax & (1<<1)) != 0; + bool sev_es_support = (eax & (1<<3)) != 0; + bool sev_snp_support = (eax & (1<<4)) != 0; + + uint8_t cbitpos = ebx & 0x3f; + uint8_t reduced_phys_bits = (ebx >> 6) & 0x3f; + + FILE *file; + char *filename = "/run/amd-sev-params"; + + file = fopen(filename, "w"); + if (file == NULL) { + perror("Error opening file"); + return 1; + } + + fprintf(file, "{" + " \"cbitpos\": %u," + " \"reduced-phys-bits\": %u," + " \"sev\": %s," + " \"sev-es\": %s," + " \"sev-snp\": %s" + " }\n", + cbitpos, + reduced_phys_bits, + sev_support ? "true" : "false", + sev_es_support ? "true" : "false", + sev_snp_support ? "true" : "false" + ); + + fclose(file); + return 0; +} diff --git a/amd-sev-support/amd-sev-support.service b/amd-sev-support/amd-sev-support.service new file mode 100644 index 0000000..466dd0a --- /dev/null +++ b/amd-sev-support/amd-sev-support.service @@ -0,0 +1,12 @@ +[Unit] +Description=Read AMD SEV Parameters +RequiresMountsFor=/run +Before=pve-ha-lrm.service +Before=pve-guests.service + +[Service] +ExecStart=/usr/libexec/qemu-server/amd-sev-support +Type=forking + +[Install] +WantedBy=multi-user.target -- 2.39.2 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel ^ permalink raw reply [flat|nested] 3+ messages in thread
* [pve-devel] [PATCH qemu-server v5 2/3] config: QEMU AMD SEV enable 2024-04-19 10:10 [pve-devel] [PATCH qemu-server v5 1/3] add C program to get AMD SEV hardware parameters from CPUID Markus Frank @ 2024-04-19 10:10 ` Markus Frank 2024-04-19 10:10 ` [pve-devel] [PATCH docs v5 3/3] add AMD SEV documentation Markus Frank 1 sibling, 0 replies; 3+ messages in thread From: Markus Frank @ 2024-04-19 10:10 UTC (permalink / raw) To: pve-devel This patch is for enabling AMD SEV (Secure Encrypted Virtualization) support in QEMU VM-Config-Examples: amd_sev: type=std,nodbg=1,noks=1 amd_sev: es,nodbg=1,kernel-hashes=1 Node-Config-Example (gets generated automatically): amd_sev: cbitpos=47,reduced-phys-bios=1 kernel-hashes, reduced-phys-bios & cbitpos correspond to the varibles with the same name in qemu. kernel-hashes=1 adds kernel-hashes to enable measured linux kernel launch since it is per default off for backward compatibility. reduced-phys-bios and cbitpos are system specific and are read out by the amd-sev-support.service on boot and saved to the /run/amd-sev-params file. This file is parsed and than used by qemu-server to correctly start a AMD SEV VM. type=std stands for standard sev to differentiate it from sev-es (es) or sev-snp (snp) when support is upstream. QEMU's sev-guest policy gets calculated with the parameters nodbg & noks These parameters correspond to policy-bits 0 & 1. If type is 'es' than policy-bit 2 gets set to 1 to activate SEV-ES. Policy bit 3 (nosend) is always set to 1, because migration features for sev are not upstream yet and are attackable. SEV-ES is highly experimental since it could not be tested. see coherent doc patch Signed-off-by: Markus Frank <m.frank@proxmox.com> --- v5: * parse /run/amd-sev-params for hardware parameters * removed NodeConfig dependency * only disallow live-migration and snapshots with vmstate -> allow offline migration and snapshots without vmstate v4: * reduced lines of code * added text that SEV-ES is experimental PVE/API2/Qemu.pm | 10 +++++ PVE/QemuMigrate.pm | 4 ++ PVE/QemuServer.pm | 102 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 116 insertions(+) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 497987f..b35ab69 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -4490,6 +4490,10 @@ __PACKAGE__->register_method({ push $local_resources->@*, "clipboard=vnc"; } + if ($res->{running} && $vmconf->{amd_sev}) { + push $local_resources->@*, "amd_sev"; + } + # if vm is not running, return target nodes where local storage/mapped devices are available # for offline migration if (!$res->{running}) { @@ -5170,6 +5174,12 @@ __PACKAGE__->register_method({ die "unable to use snapshot name 'pending' (reserved name)\n" if lc($snapname) eq 'pending'; + my $conf = PVE::QemuConfig->load_config($vmid); + if ($param->{vmstate} && $conf->{amd_sev}) { + die "Snapshots that include memory are not supported while memory" + ." is encrypted by AMD SEV.\n" + } + my $realcmd = sub { PVE::Cluster::log_msg('info', $authuser, "snapshot VM $vmid: $snapname"); PVE::QemuConfig->snapshot_create($vmid, $snapname, $param->{vmstate}, diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index 8d9b35a..7db18b2 100644 --- a/PVE/QemuMigrate.pm +++ b/PVE/QemuMigrate.pm @@ -260,6 +260,10 @@ sub prepare { die "VMs with 'clipboard' set to 'vnc' are not live migratable!\n"; } + if ($running && $conf->{'amd_sev'}) { + die "VMs with AMD SEV are not live migratable!\n"; + } + my $vollist = PVE::QemuServer::get_vm_volumes($conf); my $storages = {}; diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 6e2c805..e1ad190 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -185,6 +185,59 @@ my $agent_fmt = { }, }; +my $sev_fmt = { + type => { + description => "Enable standard SEV with type='std' or enable experimental SEV-ES" + ." with the 'es' option.", + type => 'string', + default_key => 1, + format_description => "qemu-sev-type", + enum => ['std', 'es'], + maxLength => 3, + }, + nodbg => { + description => "Sets policy bit 0 to 1 to disallow debugging of guest", + type => 'boolean', + format_description => "qemu-sev-nodbg", + default => 0, + optional => 1, + }, + noks => { + description => "Sets policy bit 1 to 1 to disallow key sharing with other guests", + type => 'boolean', + format_description => "qemu-sev-noks", + default => 0, + optional => 1, + }, + "kernel-hashes" => { + description => "Add kernel hashes to guest firmware for measured linux kernel launch", + type => 'boolean', + format_description => "qemu-sev-kernel-hashes", + default => 0, + optional => 1, + }, +}; +PVE::JSONSchema::register_format('pve-qemu-sev-fmt', $sev_fmt); + +my $sev_node_fmt = { + cbitpos => { + description => "C-bit: marks if a memory page is protected. System dependent", + type => 'integer', + default => 47, + optional => 1, + minimum => 0, + maximum => 100, + }, + 'reduced-phys-bits' => { + description => "Number of bits the physical address space is reduced by. System dependent", + type => 'integer', + default => 1, + optional => 1, + minimum => 0, + maximum => 100, + }, +}; + my $vga_fmt = { type => { description => "Select the VGA type.", @@ -366,6 +419,12 @@ my $confdesc = { description => "Memory properties.", format => $PVE::QemuServer::Memory::memory_fmt }, + amd_sev => { + description => "Secure Encrypted Virtualization (SEV) features by AMD CPUs", + optional => 1, + format => 'pve-qemu-sev-fmt', + type => 'string', + }, balloon => { optional => 1, type => 'integer', @@ -4084,6 +4143,39 @@ sub config_to_command { } push @$machineFlags, "type=${machine_type_min}"; + if ($conf->{amd_sev}) { + if ($conf->{bios} && $conf->{bios} ne 'ovmf') { + die "For using SEV you need to change your guest bios to ovmf.\n"; + } + + my $amd_sev_conf = parse_property_string($sev_fmt, $conf->{amd_sev}); + my $sev_hw_params = get_sev_hw_parameters(); + + if (!$sev_hw_params->{sev}) { + die "Your CPU does not support AMD SEV!\n"; + } + if ($amd_sev_conf->{type} eq 'es' && !$sev_hw_params->{'sev-es'}) { + die "Your CPU does not support AMD SEV-ES!\n"; + } + + my $sev_mem_object = 'sev-guest,id=sev0' + .',cbitpos='.$sev_hw_params->{cbitpos} + .',reduced-phys-bits='.$sev_hw_params->{'reduced-phys-bits'}; + + my $policy = 0b0; + $policy += 0b1 if ($amd_sev_conf->{nodbg}); + $policy += 0b10 if ($amd_sev_conf->{noks}); + $policy += 0b100 if ($amd_sev_conf->{type} eq 'es'); + # disable migration with bit 3 nosend to prevent amd-sev-migration-attack + $policy += 0b1000; + + $sev_mem_object .= ',policy='.sprintf("%#x", $policy); + $sev_mem_object .= ',kernel-hashes=on' if ($amd_sev_conf->{'kernel-hashes'}); + + push @$devices, '-object' , $sev_mem_object; + push @$machineFlags, 'confidential-guest-support=sev0'; + } + push @$cmd, @$devices; push @$cmd, '-rtc', join(',', @$rtcFlags) if scalar(@$rtcFlags); push @$cmd, '-machine', join(',', @$machineFlags) if scalar(@$machineFlags); @@ -4127,6 +4219,16 @@ sub check_rng_source { } } +sub get_sev_hw_parameters { + # Get reduced-phys-bits & cbitpos from /run/amd-sev-params + my $filename = '/run/amd-sev-params'; + open my $fh, '<', $filename or die "Could not open '$filename' for reading: $!"; + my $json_text = do { local $/; <$fh> }; + close $fh; + my $sev_params = JSON->new->decode($json_text); + return $sev_params; +} + sub spice_port { my ($vmid) = @_; -- 2.39.2 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel ^ permalink raw reply [flat|nested] 3+ messages in thread
* [pve-devel] [PATCH docs v5 3/3] add AMD SEV documentation 2024-04-19 10:10 [pve-devel] [PATCH qemu-server v5 1/3] add C program to get AMD SEV hardware parameters from CPUID Markus Frank 2024-04-19 10:10 ` [pve-devel] [PATCH qemu-server v5 2/3] config: QEMU AMD SEV enable Markus Frank @ 2024-04-19 10:10 ` Markus Frank 1 sibling, 0 replies; 3+ messages in thread From: Markus Frank @ 2024-04-19 10:10 UTC (permalink / raw) To: pve-devel add documentation for the "[PATCH qemu-server] config: QEMU AMD SEV enable" patch. Signed-off-by: Markus Frank <m.frank@proxmox.com> --- v5: * removed NodeConfig part v4: * added text that SEV-ES is experimental qm.adoc | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) diff --git a/qm.adoc b/qm.adoc index 45e3a57..6590373 100644 --- a/qm.adoc +++ b/qm.adoc @@ -715,6 +715,109 @@ systems. When allocating RAM to your VMs, a good rule of thumb is always to leave 1GB of RAM available to the host. +[[qm_memory_encryption]] +Memory Encryption +~~~~~~~~~~~~~~~~~ + +[[qm_memory_encryption_sev]] +AMD SEV +^^^^^^^ + +SEV (Secure Encrypted Virtualization) enables memory encryption per VM using +AES-128 encryption and the AMD Secure Processor. + +SEV-ES (Secure Encrypted Virtualization-Encrypted State) in addition encrypts +all CPU register contents when a VM stops running, to prevent leakage of +information to the hypervisor. This feature is very experimental. + +*Host Requirements:* + +* AMD EPYC CPU +* SEV-ES is only supported on AMD EPYC 7xx2 and newer +* configure AMD memory encryption in the BIOS settings of the host machine +* add "kvm_amd.sev=1" to kernel parameters if not enabled by default +* add "mem_encrypt=on" to kernel parameters if you want to encrypt memory on the +host (SME) see https://www.kernel.org/doc/Documentation/x86/amd-memory-encryption.txt +* maybe increase SWIOTLB see https://github.com/AMDESE/AMDSEV#faq-4 + +To check if SEV is enabled on the host search for `sev` in dmesg and print out +the SEV kernel parameter of kvm_amd: + +---- +# dmesg | grep -i sev +[...] ccp 0000:45:00.1: sev enabled +[...] ccp 0000:45:00.1: SEV API: <buildversion> +[...] SEV supported: <number> ASIDs +[...] SEV-ES supported: <number> ASIDs +# cat /sys/module/kvm_amd/parameters/sev +Y +---- + +*Guest Requirements:* + +* edk2-OVMF +* advisable to use Q35 +* The guest operating system must contain SEV-support. + +*Limitations:* + +* Because the memory is encrypted the memory usage on host is always wrong. +* Operations that involve saving or restoring memory like snapshots +& live migration do not work yet or are attackable. +https://github.com/PSPReverse/amd-sev-migration-attack +* PCI passthrough is not supported. +* SEV-ES is very experimental. +* QEMU & AMD-SEV documentation is very limited. + +Example Configuration: + +---- +# qm set <vmid> -amd_sev type=std,nodbg=1,noks=1,kernel-hashes=1 +---- + +The *type* defines the encryption technology ("type=" is not necessary). +Available options are std & es. + +The QEMU *policy* parameter gets calculated with the *nodbg* and *noks* +parameters. These parameters correspond to policy-bit 0 and 1. If *type* is *es* +the policy-bit 2 is set to 1 so that SEV-ES is enabled. Policy-bit 3 (nosend) is +always set to 1 to prevent migration-attacks. For more information on how to +calculate the policy see: +https://www.amd.com/system/files/TechDocs/55766_SEV-KM_API_Specification.pdf[AMD SEV API Specification Chapter 3] + +The *kernel-hashes* is per default off for backward compatibility with older +OVMF images and guests that do not measure the kernel/initrd. +See https://lists.gnu.org/archive/html/qemu-devel/2021-11/msg02598.html + +*Check if SEV is working on the guest* + +Method 1 - dmesg: + +Output should look like this. + +---- +# dmesg | grep -i sev +AMD Memory Encryption Features active: SEV +---- + +Method 2 - MSR 0xc0010131 (MSR_AMD64_SEV): + +Output should be 1. + +---- +# apt install msr-tools +# modprobe msr +# rdmsr -a 0xc0010131 +1 +---- + +Links: + +* https://developer.amd.com/sev/ +* https://github.com/AMDESE/AMDSEV +* https://www.qemu.org/docs/master/system/i386/amd-memory-encryption.html +* https://www.amd.com/system/files/TechDocs/55766_SEV-KM_API_Specification.pdf +* https://documentation.suse.com/sles/15-SP1/html/SLES-amd-sev/index.html [[qm_network_device]] Network Device -- 2.39.2 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2024-04-19 10:12 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-04-19 10:10 [pve-devel] [PATCH qemu-server v5 1/3] add C program to get AMD SEV hardware parameters from CPUID Markus Frank 2024-04-19 10:10 ` [pve-devel] [PATCH qemu-server v5 2/3] config: QEMU AMD SEV enable Markus Frank 2024-04-19 10:10 ` [pve-devel] [PATCH docs v5 3/3] add AMD SEV documentation Markus Frank
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.Service provided by Proxmox Server Solutions GmbH | Privacy | Legal