From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 024E394DF1 for ; Thu, 11 Apr 2024 12:49:44 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id D96AC1FC29 for ; Thu, 11 Apr 2024 12:49:13 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Thu, 11 Apr 2024 12:49:12 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 91E5944AF5 for ; Thu, 11 Apr 2024 12:49:12 +0200 (CEST) From: Markus Frank To: pve-devel@lists.proxmox.com Date: Thu, 11 Apr 2024 12:48:20 +0200 Message-Id: <20240411104822.832432-2-m.frank@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240411104822.832432-1-m.frank@proxmox.com> References: <20240411104822.832432-1-m.frank@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.032 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [machine.pm, 8006.pid, qemuserver.pm, qemu.pm] Subject: [pve-devel] [PATCH qemu-server v9 1/3] fix #3784: config: Parameter for guest vIOMMU + test-cases X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Apr 2024 10:49:44 -0000 vIOMMU enables the option to passthrough pci devices to L2 VMs in L1 VMs via Nested Virtualisation and adds an extra isolation. Uses the new property-string from the "config: define machine schema as property-string"-commit to add the viommu option to the machine parameter. Currently there are two vIOMMU implementation in QEMU to choose: intel or virtio Virtio-iommu is more recent but less used in production than intel-iommu. The assert_valid_machine_property function prevents using intel-iommu with i440fx. Signed-off-by: Markus Frank --- PVE/API2/Qemu.pm | 2 ++ PVE/QemuServer.pm | 12 +++++++++++ PVE/QemuServer/Machine.pm | 17 ++++++++++++++- test/cfg2cmd/i440fx-viommu-intel.conf | 2 ++ test/cfg2cmd/i440fx-viommu-virtio.conf | 1 + test/cfg2cmd/i440fx-viommu-virtio.conf.cmd | 25 ++++++++++++++++++++++ test/cfg2cmd/q35-viommu-intel.conf | 1 + test/cfg2cmd/q35-viommu-intel.conf.cmd | 23 ++++++++++++++++++++ test/cfg2cmd/q35-viommu-virtio.conf | 1 + test/cfg2cmd/q35-viommu-virtio.conf.cmd | 23 ++++++++++++++++++++ 10 files changed, 106 insertions(+), 1 deletion(-) create mode 100644 test/cfg2cmd/i440fx-viommu-intel.conf create mode 100644 test/cfg2cmd/i440fx-viommu-virtio.conf create mode 100644 test/cfg2cmd/i440fx-viommu-virtio.conf.cmd create mode 100644 test/cfg2cmd/q35-viommu-intel.conf create mode 100644 test/cfg2cmd/q35-viommu-intel.conf.cmd create mode 100644 test/cfg2cmd/q35-viommu-virtio.conf create mode 100644 test/cfg2cmd/q35-viommu-virtio.conf.cmd diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index f3ce83d..3eabddd 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -1136,6 +1136,7 @@ __PACKAGE__->register_method({ $conf->{machine} = PVE::QemuServer::Machine::print_machine($machine_conf); } } + PVE::QemuServer::Machine::assert_valid_machine_property($conf, $machine_conf); $conf->{lock} = 'import' if $live_import_mapping; @@ -2000,6 +2001,7 @@ my $update_vm_api = sub { $conf->{pending}->{$opt} = $param->{$opt}; } elsif ($opt eq 'machine') { my $machine_conf = PVE::QemuServer::Machine::parse_machine($param->{$opt}); + PVE::QemuServer::Machine::assert_valid_machine_property($conf, $machine_conf); $conf->{pending}->{$opt} = $param->{$opt}; } else { $conf->{pending}->{$opt} = $param->{$opt}; diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index abe175a..c94c90d 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -4080,6 +4080,18 @@ sub config_to_command { } push @$machineFlags, "type=${machine_type_min}"; + PVE::QemuServer::Machine::assert_valid_machine_property($conf, $machine_conf); + + my $viommu = $machine_conf->{viommu}; + if ($viommu) { + if ($viommu eq 'intel') { + unshift @$devices, '-device', 'intel-iommu,intremap=on,caching-mode=on'; + push @$machineFlags, 'kernel-irqchip=split'; + } elsif ($viommu eq 'virtio') { + push @$devices, '-device', 'virtio-iommu-pci'; + } + } + push @$cmd, @$devices; push @$cmd, '-rtc', join(',', @$rtcFlags) if scalar(@$rtcFlags); push @$cmd, '-machine', join(',', @$machineFlags) if scalar(@$machineFlags); diff --git a/PVE/QemuServer/Machine.pm b/PVE/QemuServer/Machine.pm index 5e3a75c..3d92c96 100644 --- a/PVE/QemuServer/Machine.pm +++ b/PVE/QemuServer/Machine.pm @@ -23,12 +23,19 @@ my $machine_fmt = { format_description => 'machine type', optional => 1, }, + viommu => { + type => 'string', + description => "Enable and set guest vIOMMU variant (Intel vIOMMU needs q35 to be set as" + ." machine type).", + enum => ['intel', 'virtio'], + optional => 1, + }, }; PVE::JSONSchema::register_format('pve-qemu-machine-fmt', $machine_fmt); PVE::JSONSchema::register_standard_option('pve-qemu-machine', { - description => "Specify the QEMU machine type.", + description => "Specify the QEMU machine.", type => 'string', optional => 1, format => PVE::JSONSchema::get_format('pve-qemu-machine-fmt'), @@ -48,6 +55,14 @@ sub print_machine { return print_property_string($machine_conf, $machine_fmt); } +sub assert_valid_machine_property { + my ($conf, $machine_conf) = @_; + my $q35 = $machine_conf->{type} && ($machine_conf->{type} =~ m/q35/) ? 1 : 0; + if ($machine_conf->{viommu} && $machine_conf->{viommu} eq "intel" && !$q35) { + die "to use Intel vIOMMU please set the machine type to q35\n"; + } +} + sub machine_type_is_q35 { my ($conf) = @_; diff --git a/test/cfg2cmd/i440fx-viommu-intel.conf b/test/cfg2cmd/i440fx-viommu-intel.conf new file mode 100644 index 0000000..bc1eb95 --- /dev/null +++ b/test/cfg2cmd/i440fx-viommu-intel.conf @@ -0,0 +1,2 @@ +# EXPECT_ERROR: to use Intel vIOMMU please set the machine type to q35 +machine: pc,viommu=intel diff --git a/test/cfg2cmd/i440fx-viommu-virtio.conf b/test/cfg2cmd/i440fx-viommu-virtio.conf new file mode 100644 index 0000000..fe7b514 --- /dev/null +++ b/test/cfg2cmd/i440fx-viommu-virtio.conf @@ -0,0 +1 @@ +machine: pc,viommu=virtio diff --git a/test/cfg2cmd/i440fx-viommu-virtio.conf.cmd b/test/cfg2cmd/i440fx-viommu-virtio.conf.cmd new file mode 100644 index 0000000..0352354 --- /dev/null +++ b/test/cfg2cmd/i440fx-viommu-virtio.conf.cmd @@ -0,0 +1,25 @@ +/usr/bin/kvm \ + -id 8006 \ + -name 'vm8006,debug-threads=on' \ + -no-shutdown \ + -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server=on,wait=off' \ + -mon 'chardev=qmp,mode=control' \ + -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \ + -mon 'chardev=qmp-event,mode=control' \ + -pidfile /var/run/qemu-server/8006.pid \ + -daemonize \ + -smp '1,sockets=1,cores=1,maxcpus=1' \ + -nodefaults \ + -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \ + -vnc 'unix:/var/run/qemu-server/8006.vnc,password=on' \ + -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \ + -m 512 \ + -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \ + -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \ + -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \ + -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \ + -device 'VGA,id=vga,bus=pci.0,addr=0x2' \ + -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \ + -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \ + -device virtio-iommu-pci \ + -machine 'type=pc+pve0' diff --git a/test/cfg2cmd/q35-viommu-intel.conf b/test/cfg2cmd/q35-viommu-intel.conf new file mode 100644 index 0000000..e500ab0 --- /dev/null +++ b/test/cfg2cmd/q35-viommu-intel.conf @@ -0,0 +1 @@ +machine: q35,viommu=intel diff --git a/test/cfg2cmd/q35-viommu-intel.conf.cmd b/test/cfg2cmd/q35-viommu-intel.conf.cmd new file mode 100644 index 0000000..24e873d --- /dev/null +++ b/test/cfg2cmd/q35-viommu-intel.conf.cmd @@ -0,0 +1,23 @@ +/usr/bin/kvm \ + -id 8006 \ + -name 'vm8006,debug-threads=on' \ + -no-shutdown \ + -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server=on,wait=off' \ + -mon 'chardev=qmp,mode=control' \ + -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \ + -mon 'chardev=qmp-event,mode=control' \ + -pidfile /var/run/qemu-server/8006.pid \ + -daemonize \ + -smp '1,sockets=1,cores=1,maxcpus=1' \ + -nodefaults \ + -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \ + -vnc 'unix:/var/run/qemu-server/8006.vnc,password=on' \ + -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \ + -m 512 \ + -device 'intel-iommu,intremap=on,caching-mode=on' \ + -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg \ + -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \ + -device 'VGA,id=vga,bus=pcie.0,addr=0x1' \ + -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \ + -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \ + -machine 'type=q35+pve0,kernel-irqchip=split' diff --git a/test/cfg2cmd/q35-viommu-virtio.conf b/test/cfg2cmd/q35-viommu-virtio.conf new file mode 100644 index 0000000..d31b339 --- /dev/null +++ b/test/cfg2cmd/q35-viommu-virtio.conf @@ -0,0 +1 @@ +machine: type=q35,viommu=virtio diff --git a/test/cfg2cmd/q35-viommu-virtio.conf.cmd b/test/cfg2cmd/q35-viommu-virtio.conf.cmd new file mode 100644 index 0000000..294c353 --- /dev/null +++ b/test/cfg2cmd/q35-viommu-virtio.conf.cmd @@ -0,0 +1,23 @@ +/usr/bin/kvm \ + -id 8006 \ + -name 'vm8006,debug-threads=on' \ + -no-shutdown \ + -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server=on,wait=off' \ + -mon 'chardev=qmp,mode=control' \ + -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \ + -mon 'chardev=qmp-event,mode=control' \ + -pidfile /var/run/qemu-server/8006.pid \ + -daemonize \ + -smp '1,sockets=1,cores=1,maxcpus=1' \ + -nodefaults \ + -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \ + -vnc 'unix:/var/run/qemu-server/8006.vnc,password=on' \ + -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \ + -m 512 \ + -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg \ + -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \ + -device 'VGA,id=vga,bus=pcie.0,addr=0x1' \ + -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \ + -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \ + -device virtio-iommu-pci \ + -machine 'type=q35+pve0' -- 2.39.2