From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 828521FF16F for ; Tue, 2 Sep 2025 13:23:38 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B546711085; Tue, 2 Sep 2025 13:23:45 +0200 (CEST) From: Daniel Kral To: pve-devel@lists.proxmox.com Date: Tue, 2 Sep 2025 13:21:58 +0200 Message-ID: <20250902112307.124706-3-d.kral@proxmox.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250902112307.124706-1-d.kral@proxmox.com> References: <20250902112307.124706-1-d.kral@proxmox.com> MIME-Version: 1.0 X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1756812176998 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.014 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pve-devel] [PATCH qemu-server v2 1/4] fix #6608: expose viommu driver aw-bits option X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" Since QEMU 9.2 [0], the default I/O address space bit width was raised from 39 bits to 48 bits for the Intel vIOMMU driver, which makes the aw-bits check introduced in [1] to trip for host CPUs with less than 48 bits physical address width from QEMU 9.2 onwards: vfio 0000:XX:YY.Z: Failed to set vIOMMU: aw-bits 48 > host aw-bits 39 For VFIO devices where a vIOMMU is in-use, QEMU fetches the IOVA ranges with the iommufd ioctl IOMMU_IOAS_IOVA_RANGES or the vfio_iommu_type1's VFIO_IOMMU_TYPE1_INFO_CAP_IOVA_RANGE info, so 'phys-bits' doesn't change the behavior of the check. Therefore, expose the 'aw-bits' option of the intel-iommu and virtio-iommu QEMU drivers to allow users to set the value. [0] https://lore.kernel.org/qemu-devel/20241212083757.605022-17-zhenzhong.duan@intel.com/ [1] https://lore.kernel.org/qemu-devel/20240605083043.317831-18-zhenzhong.duan@intel.com/ Signed-off-by: Daniel Kral --- changes from v1: - add test names - add virtio-iommu test as suggested by @Fiona off-list src/PVE/QemuServer.pm | 9 +++++-- src/PVE/QemuServer/Machine.pm | 21 +++++++++++++--- .../cfg2cmd/q35-viommu-intel-aw-bits.conf | 2 ++ .../cfg2cmd/q35-viommu-intel-aw-bits.conf.cmd | 25 +++++++++++++++++++ .../cfg2cmd/q35-viommu-virtio-aw-bits.conf | 2 ++ .../q35-viommu-virtio-aw-bits.conf.cmd | 25 +++++++++++++++++++ 6 files changed, 79 insertions(+), 5 deletions(-) create mode 100644 src/test/cfg2cmd/q35-viommu-intel-aw-bits.conf create mode 100644 src/test/cfg2cmd/q35-viommu-intel-aw-bits.conf.cmd create mode 100644 src/test/cfg2cmd/q35-viommu-virtio-aw-bits.conf create mode 100644 src/test/cfg2cmd/q35-viommu-virtio-aw-bits.conf.cmd diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm index 9597d316..04e988c7 100644 --- a/src/PVE/QemuServer.pm +++ b/src/PVE/QemuServer.pm @@ -3903,11 +3903,16 @@ sub config_to_command { PVE::QemuServer::Machine::assert_valid_machine_property($machine_conf); if (my $viommu = $machine_conf->{viommu}) { + my $viommu_devstr = ''; + $viommu_devstr .= ",aw-bits=$machine_conf->{'aw-bits'}" if $machine_conf->{'aw-bits'}; + if ($viommu eq 'intel') { - unshift @$devices, '-device', 'intel-iommu,intremap=on,caching-mode=on'; + $viommu_devstr = "intel-iommu,intremap=on,caching-mode=on$viommu_devstr"; + unshift @$devices, '-device', $viommu_devstr; push @$machineFlags, 'kernel-irqchip=split'; } elsif ($viommu eq 'virtio') { - push @$devices, '-device', 'virtio-iommu-pci'; + $viommu_devstr = "virtio-iommu-pci$viommu_devstr"; + push @$devices, '-device', $viommu_devstr; } } diff --git a/src/PVE/QemuServer/Machine.pm b/src/PVE/QemuServer/Machine.pm index b61667e0..57d583c2 100644 --- a/src/PVE/QemuServer/Machine.pm +++ b/src/PVE/QemuServer/Machine.pm @@ -58,6 +58,16 @@ my $machine_fmt = { enum => ['intel', 'virtio'], optional => 1, }, + 'aw-bits' => { + type => 'number', + description => "Specifies the vIOMMU address space bit width.", + verbose_description => "Specifies the vIOMMU address space bit width.\n\n" + . "Intel vIOMMU supports a bit width of either 39 or 48 bits and" + . " VirtIO vIOMMU supports any bit width between 32 and 64 bits.", + minimum => 32, + maximum => 64, + optional => 1, + }, 'enable-s3' => { type => 'boolean', description => @@ -112,9 +122,14 @@ sub default_machine_for_arch { sub assert_valid_machine_property { my ($machine_conf) = @_; - my $q35 = $machine_conf->{type} && ($machine_conf->{type} =~ m/q35/) ? 1 : 0; - if ($machine_conf->{viommu} && $machine_conf->{viommu} eq "intel" && !$q35) { - die "to use Intel vIOMMU please set the machine type to q35\n"; + if ($machine_conf->{viommu} && $machine_conf->{viommu} eq "intel") { + my $q35 = $machine_conf->{type} && ($machine_conf->{type} =~ m/q35/) ? 1 : 0; + die "to use Intel vIOMMU please set the machine type to q35\n" if !$q35; + + die "Intel vIOMMU supports only 39 or 48 bits as address width\n" + if $machine_conf->{'aw-bits'} + && $machine_conf->{'aw-bits'} != 39 + && $machine_conf->{'aw-bits'} != 48; } } diff --git a/src/test/cfg2cmd/q35-viommu-intel-aw-bits.conf b/src/test/cfg2cmd/q35-viommu-intel-aw-bits.conf new file mode 100644 index 00000000..9e84e42e --- /dev/null +++ b/src/test/cfg2cmd/q35-viommu-intel-aw-bits.conf @@ -0,0 +1,2 @@ +# TEST: Check if aw-bits are propagated correctly to intel-iommu device +machine: q35,viommu=intel,aw-bits=39 diff --git a/src/test/cfg2cmd/q35-viommu-intel-aw-bits.conf.cmd b/src/test/cfg2cmd/q35-viommu-intel-aw-bits.conf.cmd new file mode 100644 index 00000000..030ccaa5 --- /dev/null +++ b/src/test/cfg2cmd/q35-viommu-intel-aw-bits.conf.cmd @@ -0,0 +1,25 @@ +/usr/bin/kvm \ + -id 8006 \ + -name 'vm8006,debug-threads=on' \ + -no-shutdown \ + -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server=on,wait=off' \ + -mon 'chardev=qmp,mode=control' \ + -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect-ms=5000' \ + -mon 'chardev=qmp-event,mode=control' \ + -pidfile /var/run/qemu-server/8006.pid \ + -daemonize \ + -smp '1,sockets=1,cores=1,maxcpus=1' \ + -nodefaults \ + -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \ + -vnc 'unix:/var/run/qemu-server/8006.vnc,password=on' \ + -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \ + -m 512 \ + -global 'ICH9-LPC.disable_s3=1' \ + -global 'ICH9-LPC.disable_s4=1' \ + -device 'intel-iommu,intremap=on,caching-mode=on,aw-bits=39' \ + -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg \ + -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \ + -device 'VGA,id=vga,bus=pcie.0,addr=0x1' \ + -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \ + -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \ + -machine 'type=q35+pve0,kernel-irqchip=split' diff --git a/src/test/cfg2cmd/q35-viommu-virtio-aw-bits.conf b/src/test/cfg2cmd/q35-viommu-virtio-aw-bits.conf new file mode 100644 index 00000000..dd8ef1fd --- /dev/null +++ b/src/test/cfg2cmd/q35-viommu-virtio-aw-bits.conf @@ -0,0 +1,2 @@ +# TEST: Check if aw-bits are propagated correctly to virtio-iommu-pci device +machine: q35,viommu=virtio,aw-bits=39 diff --git a/src/test/cfg2cmd/q35-viommu-virtio-aw-bits.conf.cmd b/src/test/cfg2cmd/q35-viommu-virtio-aw-bits.conf.cmd new file mode 100644 index 00000000..c3b12eee --- /dev/null +++ b/src/test/cfg2cmd/q35-viommu-virtio-aw-bits.conf.cmd @@ -0,0 +1,25 @@ +/usr/bin/kvm \ + -id 8006 \ + -name 'vm8006,debug-threads=on' \ + -no-shutdown \ + -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server=on,wait=off' \ + -mon 'chardev=qmp,mode=control' \ + -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect-ms=5000' \ + -mon 'chardev=qmp-event,mode=control' \ + -pidfile /var/run/qemu-server/8006.pid \ + -daemonize \ + -smp '1,sockets=1,cores=1,maxcpus=1' \ + -nodefaults \ + -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \ + -vnc 'unix:/var/run/qemu-server/8006.vnc,password=on' \ + -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \ + -m 512 \ + -global 'ICH9-LPC.disable_s3=1' \ + -global 'ICH9-LPC.disable_s4=1' \ + -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg \ + -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \ + -device 'VGA,id=vga,bus=pcie.0,addr=0x1' \ + -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \ + -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \ + -device 'virtio-iommu-pci,aw-bits=39' \ + -machine 'type=q35+pve0' -- 2.47.2 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel