From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 9A4A4E5EF for ; Fri, 9 Dec 2022 15:25:53 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 79B4825206 for ; Fri, 9 Dec 2022 15:25:53 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Fri, 9 Dec 2022 15:25:51 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 308AC43E54 for ; Fri, 9 Dec 2022 15:25:51 +0100 (CET) From: Markus Frank To: pve-devel@lists.proxmox.com Date: Fri, 9 Dec 2022 15:25:21 +0100 Message-Id: <20221209142522.236233-1-m.frank@proxmox.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.040 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pve-devel] [PATCH qemu-server v3] QEMU AMD SEV enable X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Dec 2022 14:25:53 -0000 This Patch is for enabling AMD SEV (Secure Encrypted Virtualization) support in QEMU VM-Config-Examples: amd_sev: type=std,nodbg=1,noks=1 amd_sev: es,nodbg=1,kernel-hashes=1 Node-Config-Example (gets generated automatically): amd_sev: cbitpos=47,reduced-phys-bios=1 kernel-hashes, reduced-phys-bios & cbitpos correspond to the varibles with the same name in qemu. kernel-hashes=1 adds kernel-hashes to enable measured linux kernel launch since it is per default off for backward compatibility. reduced-phys-bios and cbitpos are system specific and can be read out with QMP. If not set by the user, a dummy-vm gets started to read QMP for these variables out and save them to the node config. Afterwards the dummy-vm gets stopped. type=std stands for standard sev to differentiate it from sev-es (es) or sev-snp (snp) when support is upstream. Qemu's sev-guest policy gets calculated with the parameters nodbg & noks These parameters correspond to policy-bits 0 & 1. If type=es than policy-bit 2 gets set to 1 to activate SEV-ES. Policy bit 3 (nosend) is always set to 1, because migration features for sev are not upstream yet and are attackable. see coherent doc patch Signed-off-by: Markus Frank --- I still could not get SEV-ES to work. After a firmware update I got the same error like Daniel in his testing: kvm: ../softmmu/vl.c:2568: qemu_machine_creation_done: Assertion `machine->cgs->ready' failed. v3: * moved parameters to node config * created get_sev_parameters_from_node function * added policy calculation v2: * spelling of minimum * !$conf->{bios} eq 'ovmf' changed to $conf->{bios} ne 'ovmf' PVE/API2/Qemu.pm | 9 +++ PVE/QemuServer.pm | 140 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 149 insertions(+) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index badfc37..82b53d0 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -4358,6 +4358,10 @@ __PACKAGE__->register_method({ # test if VM exists my $conf = PVE::QemuConfig->load_config($vmid); + my $amd_sev_conf = PVE::QemuServer::parse_amd_sev($conf->{amd_sev}); + die "AMD SEV does not support migration\n" + if ($amd_sev_conf->{type} eq 'std'); + # try to detect errors early PVE::QemuConfig->check_lock($conf); @@ -4909,6 +4913,11 @@ __PACKAGE__->register_method({ die "unable to use snapshot name 'pending' (reserved name)\n" if lc($snapname) eq 'pending'; + my $conf = PVE::QemuConfig->load_config($vmid); + my $amd_sev_conf = PVE::QemuServer::parse_amd_sev($conf->{amd_sev}); + die "AMD SEV does not support snapshots\n" + if ($amd_sev_conf->{type} eq 'std'); + my $realcmd = sub { PVE::Cluster::log_msg('info', $authuser, "snapshot VM $vmid: $snapname"); PVE::QemuConfig->snapshot_create($vmid, $snapname, $param->{vmstate}, diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index a52a883..9e1267d 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -56,6 +56,7 @@ use PVE::QemuServer::Memory; use PVE::QemuServer::Monitor qw(mon_cmd); use PVE::QemuServer::PCI qw(print_pci_addr print_pcie_addr print_pcie_root_port parse_hostpci); use PVE::QemuServer::USB qw(parse_usb_device); +use PVE::NodeConfig; my $have_sdn; eval { @@ -170,6 +171,59 @@ my $agent_fmt = { }, }; +my $sev_fmt = { + type => { + description => "Enable standard SEV with type='std' or enable SEV-ES" + ." with the 'es' option.", + type => 'string', + default_key => 1, + format_description => "qemu-sev-type", + enum => ['std', 'es'], + maxLength => 3, + }, + nodbg => { + description => "Sets policy bit 0 to 1 to disallow debugging of guest", + type => 'boolean', + format_description => "qemu-sev-nodbg", + default => 0, + optional => 1, + }, + noks => { + description => "Sets policy bit 1 to 1 to disallow key sharing with other guests", + type => 'boolean', + format_description => "qemu-sev-noks", + default => 0, + optional => 1, + }, + "kernel-hashes" => { + description => "Add kernel hashes to guest firmware for measured linux kernel launch", + type => 'boolean', + format_description => "qemu-sev-kernel-hashes", + default => 0, + optional => 1, + }, +}; +PVE::JSONSchema::register_format('pve-qemu-sev-fmt', $sev_fmt); + +my $sev_node_fmt = { + cbitpos => { + description => "C-bit: marks if a memory page is protected. System dependent", + type => 'integer', + default => 47, + optional => 1, + minimum => 0, + maximum => 100, + }, + 'reduced-phys-bits' => { + description => "Number of bits the physical address space is reduced by. System dependent", + type => 'integer', + default => 1, + optional => 1, + minimum => 0, + maximum => 100, + }, +}; + my $vga_fmt = { type => { description => "Select the VGA type.", @@ -346,6 +400,12 @@ my $confdesc = { minimum => 16, default => 512, }, + amd_sev => { + description => "Secure Encrypted Virtualization (SEV) features by AMD CPUs", + optional => 1, + format => 'pve-qemu-sev-fmt', + type => 'string', + }, balloon => { optional => 1, type => 'integer', @@ -2141,6 +2201,15 @@ sub parse_guest_agent { return $res; } +sub parse_amd_sev { + my ($value) = @_; + + return if !$value; + + my $res = parse_property_string($sev_fmt, $value); + return $res; +} + sub get_qga_key { my ($conf, $key) = @_; return undef if !defined($conf->{agent}); @@ -4132,6 +4201,40 @@ sub config_to_command { } push @$machineFlags, "type=${machine_type_min}"; + my $amd_sev_conf = parse_amd_sev($conf->{amd_sev}); + + if ( + $amd_sev_conf->{'type'} + && ($amd_sev_conf->{type} eq 'std' || $amd_sev_conf->{type} eq 'es') + && $conf->{bios} + && $conf->{bios} ne 'ovmf' + ) { + die "For using SEV you need to change your guest bios to ovmf.\n"; + } + + if ($amd_sev_conf->{'type'} && $amd_sev_conf->{type} eq 'es' && $kvm) { + die "SEV-ES does not work with kvm. Disable kvm to use tcg.\n"; + } + + if ( + $amd_sev_conf->{'type'} + && ($amd_sev_conf->{type} eq 'std' || $amd_sev_conf->{type} eq 'es') + ) { + my $node_config = get_sev_parameters_from_node($nodename, $arch); + my $memobjcmd = 'sev-guest,id=sev0,cbitpos='.$node_config->{cbitpos} + .',reduced-phys-bits='.$node_config->{'reduced-phys-bits'}; + my $policy = 0b0; + $policy += 0b1 if ($amd_sev_conf->{nodbg}); + $policy += 0b10 if ($amd_sev_conf->{noks}); + $policy += 0b100 if ($amd_sev_conf->{type} eq 'es'); + # disable migration with bit 3 nosend to prevent amd-sev-migration-attack + $policy += 0b1000; + $memobjcmd .= ',policy='.sprintf("%#x", $policy); + $memobjcmd .= ',kernel-hashes=on' if ($amd_sev_conf->{'kernel-hashes'}); + push @$devices, '-object' , $memobjcmd; + push @$machineFlags, 'confidential-guest-support=sev0'; + } + push @$cmd, @$devices; push @$cmd, '-rtc', join(',', @$rtcFlags) if scalar(@$rtcFlags); push @$cmd, '-machine', join(',', @$machineFlags) if scalar(@$machineFlags); @@ -4175,6 +4278,43 @@ sub check_rng_source { } } +sub get_sev_parameters_from_node { + my ($nodename, $arch) = @_; + # Get reduced-phys-bits & cbitpos from QMP, if not set + my $node_config = PVE::NodeConfig::load_config($nodename); + my $sev_node_config; + if ($node_config->{amd_sev}) { + $sev_node_config = parse_property_string($sev_node_fmt, $node_config->{amd_sev}); + } + if ( + !$sev_node_config->{'reduced-phys-bits'} + || !$sev_node_config->{cbitpos} + ) { + my $fakevmid = -1; + my $qemu_cmd = get_command_for_arch($arch); + my $pidfile = PVE::QemuServer::Helpers::pidfile_name($fakevmid); + my $default_machine = $default_machines->{$arch}; + my $cmd = [ + $qemu_cmd, + '-machine', $default_machine, + '-display', 'none', + '-chardev', "socket,id=qmp,path=/var/run/qemu-server/$fakevmid.qmp,server=on,wait=off", + '-mon', 'chardev=qmp,mode=control', + '-pidfile', $pidfile, + '-S', '-daemonize' + ]; + my $rc = run_command($cmd, noerr => 1, quiet => 0); + die "QEMU flag querying VM exited with code " . $rc . "\n" if $rc; + my $res = mon_cmd($fakevmid, 'query-sev-capabilities'); + vm_stop(undef, $fakevmid, 1, 1, 10, 0, 1); + $sev_node_config->{'reduced-phys-bits'} = $res->{'reduced-phys-bits'}; + $sev_node_config->{cbitpos} = $res->{cbitpos}; + $node_config->{amd_sev} = PVE::JSONSchema::print_property_string($sev_node_config, $sev_node_fmt); + PVE::NodeConfig::write_config($nodename, $node_config); + } + return $sev_node_config; +} + sub spice_port { my ($vmid) = @_; -- 2.30.2