From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 8814F1FF38C for ; Fri, 17 May 2024 16:50:53 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id C195B14E90; Fri, 17 May 2024 16:51:04 +0200 (CEST) Message-ID: Date: Fri, 17 May 2024 16:50:29 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: Proxmox VE development discussion , Markus Frank References: <20240426095829.746663-1-m.frank@proxmox.com> <20240426095829.746663-2-m.frank@proxmox.com> Content-Language: en-US From: Fiona Ebner In-Reply-To: <20240426095829.746663-2-m.frank@proxmox.com> X-SPAM-LEVEL: Spam detection results: 0 AWL -0.065 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com, qemuserver.pm, qemu.pm, qemumigrate.pm] Subject: Re: [pve-devel] [PATCH qemu-server v9 2/3] config: add AMD SEV support X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" Am 26.04.24 um 11:58 schrieb Markus Frank: > This patch is for enabling AMD SEV (Secure Encrypted Virtualization) > support in QEMU. > > VM-Config-Examples: > amd_sev: type=std,no-debug=1,no-key-sharing=1 > amd_sev: es,no-debug=1,kernel-hashes=1 > > kernel-hashes, reduced-phys-bios & cbitpos correspond to the variables > with the same name in qemu. s/reduced-phys-bios/reduced-phys-bits/ s/qemu/QEMU/ > > kernel-hashes=1 adds kernel-hashes to enable measured linux kernel The second time it should be "kernel hashes" instead of "kernel-hashes" > launch since it is per default off for backward compatibility. > ---snip--- > diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm > index 2a349c8..c29809d 100644 > --- a/PVE/API2/Qemu.pm > +++ b/PVE/API2/Qemu.pm > @@ -4512,6 +4512,11 @@ __PACKAGE__->register_method({ > push $local_resources->@*, "clipboard=vnc"; > } > > + # do not allow live migration with AMD SEV enabled The comment does not add any information (it's already obvious from the code). Maybe mention the migration attack instead? > + if ($res->{running} && $vmconf->{amd_sev}) { > + push $local_resources->@*, "amd_sev"; > + } > + > # if vm is not running, return target nodes where local storage/mapped devices are available > # for offline migration > if (!$res->{running}) { > @@ -5192,6 +5197,12 @@ __PACKAGE__->register_method({ > die "unable to use snapshot name 'pending' (reserved name)\n" > if lc($snapname) eq 'pending'; > > + my $conf = PVE::QemuConfig->load_config($vmid); > + if ($param->{vmstate} && $conf->{amd_sev}) { > + die "Snapshots that include memory are not supported while memory" > + ." is encrypted by AMD SEV.\n" > + } > + > my $realcmd = sub { > PVE::Cluster::log_msg('info', $authuser, "snapshot VM $vmid: $snapname"); > PVE::QemuConfig->snapshot_create($vmid, $snapname, $param->{vmstate}, What about hibernate? That uses the very same mechanism under the hood (savevm-start QMP command), so that should be prevented as well, right? A helper would be good for the common checks. Could be called check_non_migratable_resources() and start out with the checks for clipboard and hostpci devices (currently present for hibernation). Your series could then add the AMD-SEV check. The helper can then be called by check_local_resources() (although then we should avoid adding hostpci devices twice to the list of local resources), as well as the snapshot and hibernation API calls (when state is included). Off-topic: I noticed the clipboard check is also missing from snapshot and hibernate API calls, but it's not 100% clear to me if they should be added right now or if we should wait for a (minor) release, see: https://lists.proxmox.com/pipermail/pve-devel/2024-May/063896.html So you could start out with just the AMD-SEV check until we decide to enforce the VNC clipboard check for snapshot/hibernate and how to properly avoid duplicates with the hostpci check. (A TODO comment for those would be good). > diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm > index 8d9b35a..340402a 100644 > --- a/PVE/QemuMigrate.pm > +++ b/PVE/QemuMigrate.pm > @@ -260,6 +260,10 @@ sub prepare { > die "VMs with 'clipboard' set to 'vnc' are not live migratable!\n"; > } > > + if ($running && $conf->{'amd_sev'}) { > + die "cannot live-migrate VM when AMD SEV is enabled.\n"; > + } Then you could also re-use the helper here :) > + > my $vollist = PVE::QemuServer::get_vm_volumes($conf); > > my $storages = {}; > diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm > index 82e7d6a..3417a86 100644 > --- a/PVE/QemuServer.pm > +++ b/PVE/QemuServer.pm > @@ -177,6 +177,37 @@ my $agent_fmt = { > }, > }; > > +my $sev_fmt = { > + type => { > + description => "Enable standard SEV with type='std' or enable" > + ." experimental SEV-ES with the 'es' option.", > + type => 'string', > + default_key => 1, > + format_description => "sev-type", You don't need a format_description if there is an enum. > + enum => ['std', 'es'], > + maxLength => 3, You don't need a maxLenght if there is an enum. > + }, > + 'no-debug' => { > + description => "Sets policy bit 0 to 1 to disallow debugging of guest", > + type => 'boolean', > + default => 0, > + optional => 1, > + }, > + 'no-key-sharing' => { > + description => "Sets policy bit 1 to 1 to disallow key sharing with other guests", > + type => 'boolean', > + default => 0, > + optional => 1, > + }, > + "kernel-hashes" => { > + description => "Add kernel hashes to guest firmware for measured linux kernel launch", > + type => 'boolean', > + default => 0, > + optional => 1, > + }, > +}; > +PVE::JSONSchema::register_format('pve-qemu-sev-fmt', $sev_fmt); > + > my $vga_fmt = { > type => { > description => "Select the VGA type.", > @@ -358,6 +389,12 @@ my $confdesc = { > description => "Memory properties.", > format => $PVE::QemuServer::Memory::memory_fmt > }, > + amd_sev => { > + description => "Secure Encrypted Virtualization (SEV) features by AMD CPUs", > + optional => 1, > + format => 'pve-qemu-sev-fmt', > + type => 'string', > + }, > balloon => { > optional => 1, > type => 'integer', > @@ -4091,6 +4128,39 @@ sub config_to_command { > } > } > > + if ($conf->{amd_sev}) { Please factor this out into a helper function instead of adding a big block to config_to_command. It's already huge enough. > + if ($conf->{bios} && $conf->{bios} ne 'ovmf') { > + die "For using SEV you need to change your guest bios to ovmf.\n"; s/bios/BIOS/ > + } > + > + my $amd_sev_conf = parse_property_string($sev_fmt, $conf->{amd_sev}); > + my $sev_hw_caps = get_hw_capabilities()->{'amd-sev'}; Maybe error out if the parsed caps are not a hash like we expect (before accessing 'amd-sev')? And error out if the keys we expect do not exist in the result. > + > + if (!$sev_hw_caps->{'sev-support'}) { > + die "Your CPU does not support AMD SEV!\n"; > + } > + if ($amd_sev_conf->{type} eq 'es' && !$sev_hw_caps->{'sev-support-es'}) { > + die "Your CPU does not support AMD SEV-ES!\n"; > + } > + > + my $sev_mem_object = 'sev-guest,id=sev0' > + .',cbitpos='.$sev_hw_caps->{cbitpos} > + .',reduced-phys-bits='.$sev_hw_caps->{'reduced-phys-bits'}; > + > + my $policy = 0b0; > + $policy += 0b1 if ($amd_sev_conf->{'no-debug'}); > + $policy += 0b10 if ($amd_sev_conf->{'no-key-sharing'}); > + $policy += 0b100 if ($amd_sev_conf->{type} eq 'es'); > + # disable migration with bit 3 nosend to prevent amd-sev-migration-attack > + $policy += 0b1000; > + > + $sev_mem_object .= ',policy='.sprintf("%#x", $policy); > + $sev_mem_object .= ',kernel-hashes=on' if ($amd_sev_conf->{'kernel-hashes'}); Style nit: superfluous parentheses for post-if > + > + push @$devices, '-object' , $sev_mem_object; > + push @$machineFlags, 'confidential-guest-support=sev0'; > + } > + > push @$cmd, @$devices; > push @$cmd, '-rtc', join(',', @$rtcFlags) if scalar(@$rtcFlags); > push @$cmd, '-machine', join(',', @$machineFlags) if scalar(@$machineFlags); > @@ -4134,6 +4204,15 @@ sub check_rng_source { > } > } > > +sub get_hw_capabilities { > + # Get reduced-phys-bits & cbitpos from host-hw-capabilities.json > + my $filename = '/run/qemu-server/host-hw-capabilities.json'; > + my $json_text = PVE::Tools::file_get_contents($filename); > + ($json_text) = $json_text =~ /(.*)/; # untaint json text> + my $hw_capabilities = decode_json($json_text); Maybe eval and use a nice error message/prefix if the decoding dies here? > + return $hw_capabilities; > +} > + > sub spice_port { > my ($vmid) = @_; > _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel