[pve-devel] [PATCH qemu-server/docs/manager v10 0/4] AMD SEV

[pve-devel] [PATCH qemu-server/docs/manager v10 0/4] AMD SEV

[Markus Frank]

Patch series to enable AMD Secure Encrypted Virtualization (SEV)

changes v10:
* removed include of sys/types.h in C Program
* also die if the BIOS is not set, since the default is SeaBIOS
* added pve-manager patch

apply/compile order:
1. qemu-server: add C program to get hardware capabilities from CPUID
2. qemu-server: config: add AMD SEV support
3. pve-docs: add AMD SEV documentation
4. pve-manager: ui: add AMD SEV configuration to Options


qemu-server:

Markus Frank (2):
  add C program to get hardware capabilities from CPUID
  config: add AMD SEV support

 Makefile                                      |  1 +
 PVE/API2/Qemu.pm                              | 11 +++
 PVE/QemuMigrate.pm                            |  4 +
 PVE/QemuServer.pm                             | 79 +++++++++++++++++++
 query-machine-capabilities/Makefile           | 21 +++++
 .../query-machine-capabilities.c              | 70 ++++++++++++++++
 .../query-machine-capabilities.service        | 12 +++
 7 files changed, 198 insertions(+)
 create mode 100644 query-machine-capabilities/Makefile
 create mode 100644 query-machine-capabilities/query-machine-capabilities.c
 create mode 100644 query-machine-capabilities/query-machine-capabilities.service


docs:

Markus Frank (1):
  add AMD SEV documentation

 qm.adoc | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 103 insertions(+)


manager:

Markus Frank (1):
  ui: add AMD SEV configuration to Options

 www/manager6/Makefile        |  1 +
 www/manager6/qemu/Options.js | 11 ++++
 www/manager6/qemu/SevEdit.js | 98 ++++++++++++++++++++++++++++++++++++
 3 files changed, 110 insertions(+)
 create mode 100644 www/manager6/qemu/SevEdit.js

-- 
2.39.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v10 1/4] add C program to get hardware capabilities from CPUID

[Markus Frank]

Implement a systemd service that runs a C program that extracts AMD
SEV hardware information such as reduced-phys-bios and cbitpos from
CPUID at boot time, looks if SEV, SEV-ES & SEV-SNP are enabled, and
outputs these details as JSON to /run/qemu-server/hw-params.json.

This programm can also be used to read and save other hardware
information at boot time.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
Co-authored-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-by: Filip Schauer <f.schauer@proxmox.com>
---
changes v10:
* removed include of sys/types.h

 Makefile                                      |  1 +
 query-machine-capabilities/Makefile           | 21 ++++++
 .../query-machine-capabilities.c              | 70 +++++++++++++++++++
 .../query-machine-capabilities.service        | 12 ++++
 4 files changed, 104 insertions(+)
 create mode 100644 query-machine-capabilities/Makefile
 create mode 100644 query-machine-capabilities/query-machine-capabilities.c
 create mode 100644 query-machine-capabilities/query-machine-capabilities.service

diff --git a/Makefile b/Makefile
index 133468d..ed67fe0 100644
--- a/Makefile
+++ b/Makefile
@@ -65,6 +65,7 @@ install: $(PKGSOURCES)
 	install -m 0644 -D bootsplash.jpg $(DESTDIR)/usr/share/$(PACKAGE)
 	$(MAKE) -C PVE install
 	$(MAKE) -C qmeventd install
+	$(MAKE) -C query-machine-capabilities install
 	$(MAKE) -C qemu-configs install
 	$(MAKE) -C vm-network-scripts install
 	install -m 0755 qm $(DESTDIR)$(SBINDIR)
diff --git a/query-machine-capabilities/Makefile b/query-machine-capabilities/Makefile
new file mode 100644
index 0000000..c5f6348
--- /dev/null
+++ b/query-machine-capabilities/Makefile
@@ -0,0 +1,21 @@
+DESTDIR=
+PREFIX=/usr
+SBINDIR=${PREFIX}/libexec/qemu-server
+SERVICEDIR=/lib/systemd/system
+
+CC ?= gcc
+CFLAGS += -O2 -fanalyzer -Werror -Wall -Wextra -Wpedantic -Wtype-limits -Wl,-z,relro -std=gnu11
+
+query-machine-capabilities: query-machine-capabilities.c
+	$(CC) $(CFLAGS) -o $@ $< $(LDFLAGS)
+
+.PHONY: install
+install: query-machine-capabilities
+	install -d ${DESTDIR}/${SBINDIR}
+	install -d ${DESTDIR}${SERVICEDIR}
+	install -m 0644 query-machine-capabilities.service ${DESTDIR}${SERVICEDIR}
+	install -m 0755 query-machine-capabilities ${DESTDIR}${SBINDIR}
+
+.PHONY: clean
+clean:
+	rm -f query-machine-capabilities
diff --git a/query-machine-capabilities/query-machine-capabilities.c b/query-machine-capabilities/query-machine-capabilities.c
new file mode 100644
index 0000000..4f18cde
--- /dev/null
+++ b/query-machine-capabilities/query-machine-capabilities.c
@@ -0,0 +1,70 @@
+#include <stdio.h>
+#include <stdint.h>
+#include <stdbool.h>
+#include <sys/stat.h>
+#include <errno.h>
+#include <string.h>
+
+int main() {
+    uint32_t eax, ebx, ecx, edx;
+
+    // query Encrypted Memory Capabilities, see:
+    // https://en.wikipedia.org/wiki/CPUID#EAX=8000001Fh:_Encrypted_Memory_Capabilities
+    uint32_t query_function = 0x8000001F;
+    asm volatile("cpuid"
+	 : "=a"(eax), "=b"(ebx), "=c"(ecx), "=d"(edx)
+	 : "0"(query_function)
+    );
+
+    bool sev_support = (eax & (1<<1)) != 0;
+    bool sev_es_support = (eax & (1<<3)) != 0;
+    bool sev_snp_support = (eax & (1<<4)) != 0;
+
+    uint8_t cbitpos = ebx & 0x3f;
+    uint8_t reduced_phys_bits = (ebx >> 6) & 0x3f;
+
+    const char *path = "/run/qemu-server/";
+    // Check that the directory exists and create it if it does not.
+    struct stat statbuf;
+    int stats = stat(path, &statbuf);
+    if (stats == 0 && S_ISDIR(statbuf.st_mode)) {
+	printf("Directory %s already exists.\n", path);
+    } else if (errno == ENOENT) {
+	printf("%s does not exist. Creating directory.\n", path);
+	if (mkdir(path, 0755) != 0) {
+	    printf("Error creating directory %s: %s\n", path, strerror(errno));
+	    return 1;
+	}
+    } else {
+	printf("Error checking path %s: %s\n", path, strerror(errno));
+	return 1;
+    }
+
+    FILE *file;
+    const char *filename = "/run/qemu-server/host-hw-capabilities.json";
+    file = fopen(filename, "w");
+    if (file == NULL) {
+	perror("Error opening file");
+	return 1;
+    }
+
+    fprintf(file,
+	"{"
+	" \"amd-sev\": {"
+	" \"cbitpos\": %u,"
+	" \"reduced-phys-bits\": %u,"
+	" \"sev-support\": %s,"
+	" \"sev-support-es\": %s,"
+	" \"sev-support-snp\": %s"
+	" }"
+	" }\n",
+	cbitpos,
+	reduced_phys_bits,
+	sev_support ? "true" : "false",
+	sev_es_support ? "true" : "false",
+	sev_snp_support ? "true" : "false"
+    );
+
+    fclose(file);
+    return 0;
+}
diff --git a/query-machine-capabilities/query-machine-capabilities.service b/query-machine-capabilities/query-machine-capabilities.service
new file mode 100644
index 0000000..d5bf756
--- /dev/null
+++ b/query-machine-capabilities/query-machine-capabilities.service
@@ -0,0 +1,12 @@
+[Unit]
+Description=PVE Query Machine Capabilities
+RequiresMountsFor=/run
+Before=pve-ha-lrm.service
+Before=pve-guests.service
+
+[Service]
+ExecStart=/usr/libexec/qemu-server/query-machine-capabilities
+Type=oneshot
+
+[Install]
+WantedBy=multi-user.target
-- 
2.39.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v10 2/4] config: add AMD SEV support

[Markus Frank]

This patch is for enabling AMD SEV (Secure Encrypted Virtualization)
support in QEMU.

VM-Config-Examples:
amd_sev: type=std,no-debug=1,no-key-sharing=1
amd_sev: es,no-debug=1,kernel-hashes=1

kernel-hashes, reduced-phys-bios & cbitpos correspond to the variables
with the same name in QEMU.

kernel-hashes=1 adds kernel-hashes to enable measured linux kernel
launch since it is per default off for backward compatibility.

reduced-phys-bios and cbitpos are system specific and are read out by
the query-machine-capabilities.service on boot and saved to the
/run/qemu-server/host-hw-capabilities.json file. This file is parsed
and than used by qemu-server to correctly start a AMD SEV VM.

type=std stands for standard sev to differentiate it from sev-es (es)
or sev-snp (snp) when support is upstream.

QEMU's sev-guest policy gets calculated with the parameters nodbg
& noks. These parameters correspond to policy-bits 0 & 1. If type is
'es' than policy-bit 2 gets set to 1 to activate SEV-ES. Policy bit 3
(nosend) is always set to 1, because migration features for sev are
not upstream yet and are attackable.

SEV-ES is highly experimental since it could not be tested.

see coherent doc patch

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
changes v10:
* also die if the BIOS is not set, since the default is SeaBIOS

 PVE/API2/Qemu.pm   | 11 +++++++
 PVE/QemuMigrate.pm |  4 +++
 PVE/QemuServer.pm  | 79 ++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 94 insertions(+)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 2a349c8..c29809d 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -4512,6 +4512,11 @@ __PACKAGE__->register_method({
 	    push $local_resources->@*, "clipboard=vnc";
 	}
 
+	# do not allow live migration with AMD SEV enabled
+	if ($res->{running} && $vmconf->{amd_sev}) {
+	    push $local_resources->@*, "amd_sev";
+	}
+
 	# if vm is not running, return target nodes where local storage/mapped devices are available
 	# for offline migration
 	if (!$res->{running}) {
@@ -5192,6 +5197,12 @@ __PACKAGE__->register_method({
 	die "unable to use snapshot name 'pending' (reserved name)\n"
 	    if lc($snapname) eq 'pending';
 
+	my $conf = PVE::QemuConfig->load_config($vmid);
+	if ($param->{vmstate} && $conf->{amd_sev}) {
+	    die "Snapshots that include memory are not supported while memory"
+		." is encrypted by AMD SEV.\n"
+	}
+
 	my $realcmd = sub {
 	    PVE::Cluster::log_msg('info', $authuser, "snapshot VM $vmid: $snapname");
 	    PVE::QemuConfig->snapshot_create($vmid, $snapname, $param->{vmstate},
diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
index 8d9b35a..340402a 100644
--- a/PVE/QemuMigrate.pm
+++ b/PVE/QemuMigrate.pm
@@ -260,6 +260,10 @@ sub prepare {
 	die "VMs with 'clipboard' set to 'vnc' are not live migratable!\n";
     }
 
+    if ($running && $conf->{'amd_sev'}) {
+	die "cannot live-migrate VM when AMD SEV is enabled.\n";
+    }
+
     my $vollist = PVE::QemuServer::get_vm_volumes($conf);
 
     my $storages = {};
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 82e7d6a..92960c5 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -177,6 +177,37 @@ my $agent_fmt = {
     },
 };
 
+my $sev_fmt = {
+    type => {
+	description => "Enable standard SEV with type='std' or enable"
+	    ." experimental SEV-ES with the 'es' option.",
+	type => 'string',
+	default_key => 1,
+	format_description => "sev-type",
+	enum => ['std', 'es'],
+	maxLength => 3,
+    },
+    'no-debug' => {
+	description => "Sets policy bit 0 to 1 to disallow debugging of guest",
+	type => 'boolean',
+	default => 0,
+	optional => 1,
+    },
+    'no-key-sharing' => {
+	description => "Sets policy bit 1 to 1 to disallow key sharing with other guests",
+	type => 'boolean',
+	default => 0,
+	optional => 1,
+    },
+    "kernel-hashes" => {
+	description => "Add kernel hashes to guest firmware for measured linux kernel launch",
+	type => 'boolean',
+	default => 0,
+	optional => 1,
+    },
+};
+PVE::JSONSchema::register_format('pve-qemu-sev-fmt', $sev_fmt);
+
 my $vga_fmt = {
     type => {
 	description => "Select the VGA type.",
@@ -358,6 +389,12 @@ my $confdesc = {
 	description => "Memory properties.",
 	format => $PVE::QemuServer::Memory::memory_fmt
     },
+    amd_sev => {
+	description => "Secure Encrypted Virtualization (SEV) features by AMD CPUs",
+	optional => 1,
+	format => 'pve-qemu-sev-fmt',
+	type => 'string',
+    },
     balloon => {
 	optional => 1,
 	type => 'integer',
@@ -4091,6 +4128,39 @@ sub config_to_command {
 	}
     }
 
+    if ($conf->{amd_sev}) {
+	if (!$conf->{bios} || ($conf->{bios} && $conf->{bios} ne 'ovmf')) {
+	    die "For using SEV you need to change your guest bios to ovmf.\n";
+	}
+
+	my $amd_sev_conf = parse_property_string($sev_fmt, $conf->{amd_sev});
+	my $sev_hw_caps = get_hw_capabilities()->{'amd-sev'};
+
+	if (!$sev_hw_caps->{'sev-support'}) {
+	    die "Your CPU does not support AMD SEV!\n";
+	}
+	if ($amd_sev_conf->{type} eq 'es' && !$sev_hw_caps->{'sev-support-es'}) {
+	    die "Your CPU does not support AMD SEV-ES!\n";
+	}
+
+	my $sev_mem_object = 'sev-guest,id=sev0'
+	    .',cbitpos='.$sev_hw_caps->{cbitpos}
+	    .',reduced-phys-bits='.$sev_hw_caps->{'reduced-phys-bits'};
+
+	my $policy = 0b0;
+	$policy += 0b1 if ($amd_sev_conf->{'no-debug'});
+	$policy += 0b10 if ($amd_sev_conf->{'no-key-sharing'});
+	$policy += 0b100 if ($amd_sev_conf->{type} eq 'es');
+	# disable migration with bit 3 nosend to prevent amd-sev-migration-attack
+	$policy += 0b1000;
+
+	$sev_mem_object .= ',policy='.sprintf("%#x", $policy);
+	$sev_mem_object .= ',kernel-hashes=on' if ($amd_sev_conf->{'kernel-hashes'});
+
+	push @$devices, '-object' , $sev_mem_object;
+	push @$machineFlags, 'confidential-guest-support=sev0';
+    }
+
     push @$cmd, @$devices;
     push @$cmd, '-rtc', join(',', @$rtcFlags) if scalar(@$rtcFlags);
     push @$cmd, '-machine', join(',', @$machineFlags) if scalar(@$machineFlags);
@@ -4134,6 +4204,15 @@ sub check_rng_source {
     }
 }
 
+sub get_hw_capabilities {
+    # Get reduced-phys-bits & cbitpos from host-hw-capabilities.json
+    my $filename = '/run/qemu-server/host-hw-capabilities.json';
+    my $json_text = PVE::Tools::file_get_contents($filename);
+    ($json_text) = $json_text =~ /(.*)/; # untaint json text
+    my $hw_capabilities = decode_json($json_text);
+    return $hw_capabilities;
+}
+
 sub spice_port {
     my ($vmid) = @_;
 
-- 
2.39.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH docs v10 3/4] add AMD SEV documentation

[Markus Frank]

add documentation for the "[PATCH qemu-server] config: QEMU AMD SEV
enable" patch.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
changes v10:
* none

 qm.adoc | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 103 insertions(+)

diff --git a/qm.adoc b/qm.adoc
index 42c26db..2001bd4 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -715,6 +715,109 @@ systems.
 When allocating RAM to your VMs, a good rule of thumb is always to leave 1GB
 of RAM available to the host.
 
+[[qm_memory_encryption]]
+Memory Encryption
+~~~~~~~~~~~~~~~~~
+
+[[qm_memory_encryption_sev]]
+AMD SEV
+^^^^^^^
+
+SEV (Secure Encrypted Virtualization) enables memory encryption per VM using
+AES-128 encryption and the AMD Secure Processor.
+
+SEV-ES (Secure Encrypted Virtualization-Encrypted State) in addition encrypts
+all CPU register contents when a VM stops running, to prevent leakage of
+information to the hypervisor. This feature is very experimental.
+
+*Host Requirements:*
+
+* AMD EPYC CPU
+* SEV-ES is only supported on AMD EPYC 7xx2 and newer
+* configure AMD memory encryption in the BIOS settings of the host machine
+* add "kvm_amd.sev=1" to kernel parameters if not enabled by default
+* add "mem_encrypt=on" to kernel parameters if you want to encrypt memory on the
+host (SME) see https://www.kernel.org/doc/Documentation/x86/amd-memory-encryption.txt
+* maybe increase SWIOTLB see https://github.com/AMDESE/AMDSEV#faq-4
+
+To check if SEV is enabled on the host search for `sev` in dmesg and print out
+the SEV kernel parameter of kvm_amd:
+
+----
+# dmesg | grep -i sev
+[...] ccp 0000:45:00.1: sev enabled
+[...] ccp 0000:45:00.1: SEV API: <buildversion>
+[...] SEV supported: <number> ASIDs
+[...] SEV-ES supported: <number> ASIDs
+# cat /sys/module/kvm_amd/parameters/sev
+Y
+----
+
+*Guest Requirements:*
+
+* edk2-OVMF
+* advisable to use Q35
+* The guest operating system must contain SEV-support.
+
+*Limitations:*
+
+* Because the memory is encrypted the memory usage on host is always wrong.
+* Operations that involve saving or restoring memory like snapshots
+& live migration do not work yet or are attackable.
+https://github.com/PSPReverse/amd-sev-migration-attack
+* PCI passthrough is not supported.
+* SEV-ES is very experimental.
+* QEMU & AMD-SEV documentation is very limited.
+
+Example Configuration:
+
+----
+# qm set <vmid> -amd_sev type=std,no-debug=1,no-key-sharing=1,kernel-hashes=1
+----
+
+The *type* defines the encryption technology ("type=" is not necessary).
+Available options are std & es.
+
+The QEMU *policy* parameter gets calculated with the *no-debug* and
+*no-key-sharing* parameters. These parameters correspond to policy-bit 0 and 1.
+If *type* is *es* the policy-bit 2 is set to 1 so that SEV-ES is enabled.
+Policy-bit 3 (nosend) is always set to 1 to prevent migration-attacks. For more
+information on how to calculate the policy see:
+https://www.amd.com/system/files/TechDocs/55766_SEV-KM_API_Specification.pdf[AMD SEV API Specification Chapter 3]
+
+The *kernel-hashes* is per default off for backward compatibility with older
+OVMF images and guests that do not measure the kernel/initrd.
+See https://lists.gnu.org/archive/html/qemu-devel/2021-11/msg02598.html
+
+*Check if SEV is working on the guest*
+
+Method 1 - dmesg:
+
+Output should look like this.
+
+----
+# dmesg | grep -i sev
+AMD Memory Encryption Features active: SEV
+----
+
+Method 2 - MSR 0xc0010131 (MSR_AMD64_SEV):
+
+Output should be 1.
+
+----
+# apt install msr-tools
+# modprobe msr
+# rdmsr -a 0xc0010131
+1
+----
+
+Links:
+
+* https://developer.amd.com/sev/
+* https://github.com/AMDESE/AMDSEV
+* https://www.qemu.org/docs/master/system/i386/amd-memory-encryption.html
+* https://www.amd.com/system/files/TechDocs/55766_SEV-KM_API_Specification.pdf
+* https://documentation.suse.com/sles/15-SP1/html/SLES-amd-sev/index.html
 
 [[qm_network_device]]
 Network Device
-- 
2.39.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH manager v10 4/4] ui: add AMD SEV configuration to Options

[Markus Frank]

By adding a new input panel with an AMD SEV technology selection combo
box and checkboxes for the optional parameters in an advanced section,
the user can configure the amd_sev option via the WebUI's Options tab.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
changes v10:
* this patch is new to v10

 www/manager6/Makefile        |  1 +
 www/manager6/qemu/Options.js | 11 ++++
 www/manager6/qemu/SevEdit.js | 98 ++++++++++++++++++++++++++++++++++++
 3 files changed, 110 insertions(+)
 create mode 100644 www/manager6/qemu/SevEdit.js

diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 2c3a822b..801683a3 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -264,6 +264,7 @@ JSSRC= 							\
 	qemu/SSHKey.js					\
 	qemu/ScsiHwEdit.js				\
 	qemu/SerialEdit.js				\
+	qemu/SevEdit.js					\
 	qemu/Smbios1Edit.js				\
 	qemu/SystemEdit.js				\
 	qemu/USBEdit.js					\
diff --git a/www/manager6/qemu/Options.js b/www/manager6/qemu/Options.js
index 7b112400..6907699c 100644
--- a/www/manager6/qemu/Options.js
+++ b/www/manager6/qemu/Options.js
@@ -338,6 +338,17 @@ Ext.define('PVE.qemu.Options', {
 		    },
 		} : undefined,
 	    },
+	    amd_sev: {
+		header: gettext('AMD SEV'),
+		editor: caps.vms['VM.Config.HWType'] ? 'PVE.qemu.SevEdit' : undefined,
+		defaultValue: Proxmox.Utils.defaultText + ' (' + Proxmox.Utils.disabledText + ')',
+		renderer: function(value, metaData, record, ri, ci, store, pending) {
+		    let amd_sev = PVE.Parser.parsePropertyString(value, "type");
+		    if (amd_sev.type === 'std') return 'AMD SEV (' + value + ')';
+		    if (amd_sev.type === 'es') return 'AMD SEV-ES (' + value + ')';
+		    return value;
+		},
+	    },
 	    hookscript: {
 		header: gettext('Hookscript'),
 	    },
diff --git a/www/manager6/qemu/SevEdit.js b/www/manager6/qemu/SevEdit.js
new file mode 100644
index 00000000..f0187cde
--- /dev/null
+++ b/www/manager6/qemu/SevEdit.js
@@ -0,0 +1,98 @@
+Ext.define('PVE.qemu.SevInputPanel', {
+    extend: 'Proxmox.panel.InputPanel',
+    xtype: 'pveSevInputPanel',
+    onlineHelp: 'qm_memory_encryption',
+
+    viewModel: {
+	data: {
+	    type: '__default__',
+	},
+	formulas: {
+	    sevEnabled: get => get('type') === 'std' || get('type') === 'es',
+	},
+    },
+
+    onGetValues: function(values) {
+	if (values.delete === 'type') {
+	    values.delete = 'amd_sev';
+	    return values;
+	}
+	let ret = {};
+	ret.amd_sev = PVE.Parser.printPropertyString(values, 'type');
+	return ret;
+    },
+
+    items: {
+	xtype: 'proxmoxKVComboBox',
+	fieldLabel: gettext('AMD Secure Encrypted Virtualization (SEV)'),
+	name: 'type',
+	value: '__default__',
+	comboItems: [
+	    ['__default__', Proxmox.Utils.defaultText + ' (' + Proxmox.Utils.disabledText + ')'],
+	    ['std', 'AMD SEV'],
+	    ['es', 'AMD SEV-ES (highly experimental)'],
+	],
+	bind: {
+	    value: '{type}',
+	},
+    },
+
+    advancedItems: [
+	{
+	    xtype: 'proxmoxcheckbox',
+	    fieldLabel: gettext('no-debug'),
+	    name: 'no-debug',
+	    deleteDefaultValue: false,
+	    bind: {
+		hidden: '{!sevEnabled}',
+		disabled: '{!sevEnabled}',
+	    },
+	},
+	{
+	    xtype: 'proxmoxcheckbox',
+	    fieldLabel: gettext('no-key-sharing'),
+	    name: 'no-key-sharing',
+	    deleteDefaultValue: false,
+	    bind: {
+		hidden: '{!sevEnabled}',
+		disabled: '{!sevEnabled}',
+	    },
+	},
+	{
+	    xtype: 'proxmoxcheckbox',
+	    fieldLabel: gettext('kernel-hashes'),
+	    name: 'kernel-hashes',
+	    deleteDefaultValue: false,
+	    bind: {
+		hidden: '{!sevEnabled}',
+		disabled: '{!sevEnabled}',
+	    },
+	},
+    ],
+});
+
+Ext.define('PVE.qemu.SevEdit', {
+    extend: 'Proxmox.window.Edit',
+
+    subject: gettext('SEV'),
+
+    items: {
+	xtype: 'pveSevInputPanel',
+    },
+
+    width: 400,
+
+    initComponent: function() {
+	let me = this;
+
+	me.callParent();
+
+	me.load({
+	    success: function(response) {
+		let conf = response.result.data;
+		let amd_sev = conf.amd_sev || '__default__';
+		me.setValues(PVE.Parser.parsePropertyString(amd_sev, 'type'));
+	    },
+	});
+    },
+});
-- 
2.39.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH qemu-server/docs/manager v10 0/4] AMD SEV

[Dominik Csapak]

all in all the series looks mostly good to me with some minor comments
(see comments on the individual patches)
i could not test it since i don't have an amd epyc system here ;)


one high level thing though (but no hard feelings)

is the systemd service really necessary ?

we could simply call the binary the first time it's needed?

so in qemu-server:

---
if (! -e $cap_path) {
    # executing the binary
}
---

before we try to parse it ?

we can still make it a service should we need, but IMHO this seems
overkill for the current use case


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH qemu-server v10 1/4] add C program to get hardware capabilities from CPUID

[Dominik Csapak]

one small nit inline:

On 5/10/24 13:47, Markus Frank wrote:
> diff --git a/query-machine-capabilities/Makefile b/query-machine-capabilities/Makefile
> new file mode 100644
> index 0000000..c5f6348
> --- /dev/null
> +++ b/query-machine-capabilities/Makefile
> @@ -0,0 +1,21 @@
> +DESTDIR=
> +PREFIX=/usr
> +SBINDIR=${PREFIX}/libexec/qemu-server
> +SERVICEDIR=/lib/systemd/system
> +

PREFIX is only used once here, so it's probably better inlining the value


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH qemu-server v10 2/4] config: add AMD SEV support

[Dominik Csapak]

comments inline:

On 5/10/24 13:47, Markus Frank wrote:
> This patch is for enabling AMD SEV (Secure Encrypted Virtualization)
> support in QEMU.
> 
> VM-Config-Examples:
> amd_sev: type=std,no-debug=1,no-key-sharing=1
> amd_sev: es,no-debug=1,kernel-hashes=1
> 
> kernel-hashes, reduced-phys-bios & cbitpos correspond to the variables
> with the same name in QEMU.
> 
> kernel-hashes=1 adds kernel-hashes to enable measured linux kernel
> launch since it is per default off for backward compatibility.
> 
> reduced-phys-bios and cbitpos are system specific and are read out by
> the query-machine-capabilities.service on boot and saved to the
> /run/qemu-server/host-hw-capabilities.json file. This file is parsed
> and than used by qemu-server to correctly start a AMD SEV VM.
> 
> type=std stands for standard sev to differentiate it from sev-es (es)
> or sev-snp (snp) when support is upstream.
> 
> QEMU's sev-guest policy gets calculated with the parameters nodbg
> & noks. These parameters correspond to policy-bits 0 & 1. If type is
> 'es' than policy-bit 2 gets set to 1 to activate SEV-ES. Policy bit 3
> (nosend) is always set to 1, because migration features for sev are
> not upstream yet and are attackable.
> 
> SEV-ES is highly experimental since it could not be tested.
> 
> see coherent doc patch
> 
> Signed-off-by: Markus Frank <m.frank@proxmox.com>
> ---
> changes v10:
> * also die if the BIOS is not set, since the default is SeaBIOS
> 
>   PVE/API2/Qemu.pm   | 11 +++++++
>   PVE/QemuMigrate.pm |  4 +++
>   PVE/QemuServer.pm  | 79 ++++++++++++++++++++++++++++++++++++++++++++++
>   3 files changed, 94 insertions(+)
> 
> diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
> index 2a349c8..c29809d 100644
> --- a/PVE/API2/Qemu.pm
> +++ b/PVE/API2/Qemu.pm
> @@ -4512,6 +4512,11 @@ __PACKAGE__->register_method({
>   	    push $local_resources->@*, "clipboard=vnc";
>   	}
>   
> +	# do not allow live migration with AMD SEV enabled
> +	if ($res->{running} && $vmconf->{amd_sev}) {
> +	    push $local_resources->@*, "amd_sev";
> +	}
> +
>   	# if vm is not running, return target nodes where local storage/mapped devices are available
>   	# for offline migration
>   	if (!$res->{running}) {
> @@ -5192,6 +5197,12 @@ __PACKAGE__->register_method({
>   	die "unable to use snapshot name 'pending' (reserved name)\n"
>   	    if lc($snapname) eq 'pending';
>   
> +	my $conf = PVE::QemuConfig->load_config($vmid);
> +	if ($param->{vmstate} && $conf->{amd_sev}) {
> +	    die "Snapshots that include memory are not supported while memory"
> +		." is encrypted by AMD SEV.\n"
> +	}
> +

you do it for snapshots, but it's missing for suspend to disk, where we
basically migrate into a file

>   	my $realcmd = sub {
>   	    PVE::Cluster::log_msg('info', $authuser, "snapshot VM $vmid: $snapname");
>   	    PVE::QemuConfig->snapshot_create($vmid, $snapname, $param->{vmstate},
> diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
> index 8d9b35a..340402a 100644
> --- a/PVE/QemuMigrate.pm
> +++ b/PVE/QemuMigrate.pm
> @@ -260,6 +260,10 @@ sub prepare {
>   	die "VMs with 'clipboard' set to 'vnc' are not live migratable!\n";
>       }
>   
> +    if ($running && $conf->{'amd_sev'}) {
> +	die "cannot live-migrate VM when AMD SEV is enabled.\n";
> +    }
> +
>       my $vollist = PVE::QemuServer::get_vm_volumes($conf);
>   
>       my $storages = {};
> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
> index 82e7d6a..92960c5 100644
> --- a/PVE/QemuServer.pm
> +++ b/PVE/QemuServer.pm
> @@ -177,6 +177,37 @@ my $agent_fmt = {
>       },
>   };
>   
> +my $sev_fmt = {
> +    type => {
> +	description => "Enable standard SEV with type='std' or enable"
> +	    ." experimental SEV-ES with the 'es' option.",
> +	type => 'string',
> +	default_key => 1,
> +	format_description => "sev-type",
> +	enum => ['std', 'es'],
> +	maxLength => 3,
> +    },
> +    'no-debug' => {
> +	description => "Sets policy bit 0 to 1 to disallow debugging of guest",
> +	type => 'boolean',
> +	default => 0,
> +	optional => 1,
> +    },
> +    'no-key-sharing' => {
> +	description => "Sets policy bit 1 to 1 to disallow key sharing with other guests",
> +	type => 'boolean',
> +	default => 0,
> +	optional => 1,
> +    },
> +    "kernel-hashes" => {
> +	description => "Add kernel hashes to guest firmware for measured linux kernel launch",
> +	type => 'boolean',
> +	default => 0,
> +	optional => 1,
> +    },
> +};
> +PVE::JSONSchema::register_format('pve-qemu-sev-fmt', $sev_fmt);
> +
>   my $vga_fmt = {
>       type => {
>   	description => "Select the VGA type.",
> @@ -358,6 +389,12 @@ my $confdesc = {
>   	description => "Memory properties.",
>   	format => $PVE::QemuServer::Memory::memory_fmt
>       },
> +    amd_sev => {
> +	description => "Secure Encrypted Virtualization (SEV) features by AMD CPUs",
> +	optional => 1,
> +	format => 'pve-qemu-sev-fmt',
> +	type => 'string',
> +    },
>       balloon => {
>   	optional => 1,
>   	type => 'integer',
> @@ -4091,6 +4128,39 @@ sub config_to_command {
>   	}
>       }
>   
> +    if ($conf->{amd_sev}) {
> +	if (!$conf->{bios} || ($conf->{bios} && $conf->{bios} ne 'ovmf')) {
> +	    die "For using SEV you need to change your guest bios to ovmf.\n";
> +	}
> +
> +	my $amd_sev_conf = parse_property_string($sev_fmt, $conf->{amd_sev});
> +	my $sev_hw_caps = get_hw_capabilities()->{'amd-sev'};
> +
> +	if (!$sev_hw_caps->{'sev-support'}) {
> +	    die "Your CPU does not support AMD SEV!\n";
> +	}
> +	if ($amd_sev_conf->{type} eq 'es' && !$sev_hw_caps->{'sev-support-es'}) {
> +	    die "Your CPU does not support AMD SEV-ES!\n";
> +	}
> +
> +	my $sev_mem_object = 'sev-guest,id=sev0'
> +	    .',cbitpos='.$sev_hw_caps->{cbitpos}
> +	    .',reduced-phys-bits='.$sev_hw_caps->{'reduced-phys-bits'};
> +
> +	my $policy = 0b0;
> +	$policy += 0b1 if ($amd_sev_conf->{'no-debug'});
> +	$policy += 0b10 if ($amd_sev_conf->{'no-key-sharing'});
> +	$policy += 0b100 if ($amd_sev_conf->{type} eq 'es');
> +	# disable migration with bit 3 nosend to prevent amd-sev-migration-attack
> +	$policy += 0b1000;

isn't it possible to keep the bitlength identically? makes it easier to compare
e.g. like this:

my $policy = 0b0000;
$policy += 0b0001 if ...
$policy += 0b0010 if ...

etc..

> +
> +	$sev_mem_object .= ',policy='.sprintf("%#x", $policy);
> +	$sev_mem_object .= ',kernel-hashes=on' if ($amd_sev_conf->{'kernel-hashes'});
> +
> +	push @$devices, '-object' , $sev_mem_object;
> +	push @$machineFlags, 'confidential-guest-support=sev0';
> +    }
> +

also i'd prefer to put this whole block into e.g. PVE/QemuServer/CPUConfig

so 'config_to_command' does not get more bloated than it already is

>       push @$cmd, @$devices;
>       push @$cmd, '-rtc', join(',', @$rtcFlags) if scalar(@$rtcFlags);
>       push @$cmd, '-machine', join(',', @$machineFlags) if scalar(@$machineFlags);
> @@ -4134,6 +4204,15 @@ sub check_rng_source {
>       }
>   }
>   
> +sub get_hw_capabilities {
> +    # Get reduced-phys-bits & cbitpos from host-hw-capabilities.json
> +    my $filename = '/run/qemu-server/host-hw-capabilities.json';
> +    my $json_text = PVE::Tools::file_get_contents($filename);
> +    ($json_text) = $json_text =~ /(.*)/; # untaint json text
> +    my $hw_capabilities = decode_json($json_text);
> +    return $hw_capabilities;
> +}
> +

also this maybe? though it could also live in 'Helpers'

>   sub spice_port {
>       my ($vmid) = @_;
>   



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH manager v10 4/4] ui: add AMD SEV configuration to Options

[Dominik Csapak]

comments inline

On 5/10/24 13:47, Markus Frank wrote:
> By adding a new input panel with an AMD SEV technology selection combo
> box and checkboxes for the optional parameters in an advanced section,
> the user can configure the amd_sev option via the WebUI's Options tab.
> 
> Signed-off-by: Markus Frank <m.frank@proxmox.com>
> ---
> changes v10:
> * this patch is new to v10
> 
>   www/manager6/Makefile        |  1 +
>   www/manager6/qemu/Options.js | 11 ++++
>   www/manager6/qemu/SevEdit.js | 98 ++++++++++++++++++++++++++++++++++++
>   3 files changed, 110 insertions(+)
>   create mode 100644 www/manager6/qemu/SevEdit.js
> 
> diff --git a/www/manager6/Makefile b/www/manager6/Makefile
> index 2c3a822b..801683a3 100644
> --- a/www/manager6/Makefile
> +++ b/www/manager6/Makefile
> @@ -264,6 +264,7 @@ JSSRC= 							\
>   	qemu/SSHKey.js					\
>   	qemu/ScsiHwEdit.js				\
>   	qemu/SerialEdit.js				\
> +	qemu/SevEdit.js					\
>   	qemu/Smbios1Edit.js				\
>   	qemu/SystemEdit.js				\
>   	qemu/USBEdit.js					\
> diff --git a/www/manager6/qemu/Options.js b/www/manager6/qemu/Options.js
> index 7b112400..6907699c 100644
> --- a/www/manager6/qemu/Options.js
> +++ b/www/manager6/qemu/Options.js
> @@ -338,6 +338,17 @@ Ext.define('PVE.qemu.Options', {
>   		    },
>   		} : undefined,
>   	    },
> +	    amd_sev: {
> +		header: gettext('AMD SEV'),
> +		editor: caps.vms['VM.Config.HWType'] ? 'PVE.qemu.SevEdit' : undefined,
> +		defaultValue: Proxmox.Utils.defaultText + ' (' + Proxmox.Utils.disabledText + ')',
> +		renderer: function(value, metaData, record, ri, ci, store, pending) {
> +		    let amd_sev = PVE.Parser.parsePropertyString(value, "type");
> +		    if (amd_sev.type === 'std') return 'AMD SEV (' + value + ')';
> +		    if (amd_sev.type === 'es') return 'AMD SEV-ES (' + value + ')';
> +		    return value;
> +		},
> +	    },
>   	    hookscript: {
>   		header: gettext('Hookscript'),
>   	    },
> diff --git a/www/manager6/qemu/SevEdit.js b/www/manager6/qemu/SevEdit.js
> new file mode 100644
> index 00000000..f0187cde
> --- /dev/null
> +++ b/www/manager6/qemu/SevEdit.js
> @@ -0,0 +1,98 @@
> +Ext.define('PVE.qemu.SevInputPanel', {
> +    extend: 'Proxmox.panel.InputPanel',
> +    xtype: 'pveSevInputPanel',
> +    onlineHelp: 'qm_memory_encryption',
> +
> +    viewModel: {
> +	data: {
> +	    type: '__default__',
> +	},
> +	formulas: {
> +	    sevEnabled: get => get('type') === 'std' || get('type') === 'es',

would'nt that be `get('type') !== '__default__'` ?

makes it shorter and more future proof should we add some other type there

> +	},
> +    },
> +
> +    onGetValues: function(values) {
> +	if (values.delete === 'type') {
> +	    values.delete = 'amd_sev';
> +	    return values;
> +	}
> +	let ret = {};
> +	ret.amd_sev = PVE.Parser.printPropertyString(values, 'type');
> +	return ret;
> +    },
> +
> +    items: {
> +	xtype: 'proxmoxKVComboBox',
> +	fieldLabel: gettext('AMD Secure Encrypted Virtualization (SEV)'),
> +	name: 'type',
> +	value: '__default__',
> +	comboItems: [
> +	    ['__default__', Proxmox.Utils.defaultText + ' (' + Proxmox.Utils.disabledText + ')'],
> +	    ['std', 'AMD SEV'],
> +	    ['es', 'AMD SEV-ES (highly experimental)'],
> +	],
> +	bind: {
> +	    value: '{type}',
> +	},
> +    },
> +
> +    advancedItems: [
> +	{
> +	    xtype: 'proxmoxcheckbox',
> +	    fieldLabel: gettext('no-debug'),

nit: i guess it probably make sense to expose the policy settings with their names
but i really disklike 'no-something' options that one have to enable

we could in the ui reverse it and make the default true?

also IMHO this text is a bit too short

e.g. 'allow debugging' would be nicer?

> +	    name: 'no-debug',
> +	    deleteDefaultValue: false,
> +	    bind: {
> +		hidden: '{!sevEnabled}',
> +		disabled: '{!sevEnabled}',
> +	    },
> +	},
> +	{
> +	    xtype: 'proxmoxcheckbox',
> +	    fieldLabel: gettext('no-key-sharing'),

same here...

> +	    name: 'no-key-sharing',
> +	    deleteDefaultValue: false,
> +	    bind: {
> +		hidden: '{!sevEnabled}',
> +		disabled: '{!sevEnabled}',
> +	    },
> +	},
> +	{
> +	    xtype: 'proxmoxcheckbox',
> +	    fieldLabel: gettext('kernel-hashes'),
> +	    name: 'kernel-hashes',
> +	    deleteDefaultValue: false,
> +	    bind: {
> +		hidden: '{!sevEnabled}',
> +		disabled: '{!sevEnabled}',
> +	    },
> +	},
> +    ],
> +});
> +
> +Ext.define('PVE.qemu.SevEdit', {
> +    extend: 'Proxmox.window.Edit',
> +
> +    subject: gettext('SEV'),
> +
> +    items: {
> +	xtype: 'pveSevInputPanel',
> +    },
> +
> +    width: 400,
> +
> +    initComponent: function() {
> +	let me = this;
> +
> +	me.callParent();
> +
> +	me.load({
> +	    success: function(response) {
> +		let conf = response.result.data;
> +		let amd_sev = conf.amd_sev || '__default__';
> +		me.setValues(PVE.Parser.parsePropertyString(amd_sev, 'type'));
> +	    },
> +	});
> +    },
> +});



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH qemu-server v10 1/4] add C program to get hardware capabilities from CPUID

[Thomas Lamprecht]

Am 17/05/2024 um 13:21 schrieb Dominik Csapak:
> one small nit inline:
> 
> On 5/10/24 13:47, Markus Frank wrote:
>> diff --git a/query-machine-capabilities/Makefile b/query-machine-capabilities/Makefile
>> new file mode 100644
>> index 0000000..c5f6348
>> --- /dev/null
>> +++ b/query-machine-capabilities/Makefile
>> @@ -0,0 +1,21 @@
>> +DESTDIR=
>> +PREFIX=/usr
>> +SBINDIR=${PREFIX}/libexec/qemu-server
>> +SERVICEDIR=/lib/systemd/system
>> +
> 
> PREFIX is only used once here, so it's probably better inlining the value

No, having the PREFIX variable separate is a common pattern
that allows customizing installation.
Even if we do not need that ourselves, it's still not costing us really
anything to keep following that here and could make comparing changes
between two packages with binaries installed in different paths easier.
So while I wouldn't go through all our build systems and introduce this
variable if missing, I'd also not recommend developers to drop it, as it
is not better to do so.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel