Re: [pve-devel] [PATCH qemu-server v3] fix #6608: expose viommu driver aw-bits option

[Fiona Ebner <f.ebner@proxmox.com>]

Am 05.09.25 um 4:15 PM schrieb Daniel Kral:
> Since QEMU 9.2 [0], the default I/O address space bit width was raised
> from 39 bits to 48 bits for the Intel vIOMMU driver, which makes the
> aw-bits check introduced in [1] to trip for host CPUs with less than 48
> bits physical address width from QEMU 9.2 onwards:
> 
> vfio 0000:XX:YY.Z: Failed to set vIOMMU: aw-bits 48 > host aw-bits 39
> 
> For VFIO devices where a vIOMMU is in-use, QEMU fetches the IOVA ranges
> with the iommufd ioctl IOMMU_IOAS_IOVA_RANGES or the vfio_iommu_type1's
> VFIO_IOMMU_TYPE1_INFO_CAP_IOVA_RANGE info, so 'phys-bits' doesn't change
> the behavior of the check.
> 
> Therefore, expose the 'aw-bits' option of the intel-iommu and
> virtio-iommu QEMU drivers to allow users to set the value.
> 
> [0] qemu ddd84fd0c1 ("intel_iommu: Set default aw_bits to 48 starting from QEMU 9.2")
> [1] qemu 77f6efc0ab ("intel_iommu: Check compatibility with host IOMMU capabilities")
> 
> Signed-off-by: Daniel Kral <d.kral@proxmox.com>

I'll go ahead and apply this and the below, if that addition is fine by you?

> commit 05eb8e6394ca83e53cbf6a01e1a8848ff3d4d3e8
> Author: Fiona Ebner <f.ebner@proxmox.com>
> Date:   Mon Sep 8 10:37:24 2025 +0200
> 
>     cfg2cmd: inform users that setting guest-phys-bits might be necessary when setting aw-bits
>     
>     Until QEMU warns about this itself, inform the users here. Commit
>     message below copied from [1].
>     
>     If a virtual machine is setup with an intel-iommu device, QEMU
>     allocates and maps the (virtual) I/O address space (IOAS) for a VFIO
>     passthrough device with iommufd.
>     
>     In case of a mismatch of the address width of the host CPU and IOMMU
>     CPU, the guest physical address space (GPAS) and memory-type range
>     registers (MTRRs) are setup to the host CPU's address width, which
>     causes IOAS to be allocated and mapped outside of the IOMMU's maximum
>     guest address width (MGAW) and causes the following error from QEMU
>     (the error message is copied from the user forum [0]):
>     
>         kvm: vfio_container_dma_map(0x5c9222494280, 0x380000000000, 0x10000, 0x78075ee70000) = -22 (Invalid argument)
>     
>     [0]: https://forum.proxmox.com/threads/169586/page-3#post-795717
>     [1]: https://lore.proxmox.com/pve-devel/20250902112307.124706-5-d.kral@proxmox.com/
>     
>     Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
> ---
>  src/PVE/QemuServer.pm | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
> index c428e2d7..bf229610 100644
> --- a/src/PVE/QemuServer.pm
> +++ b/src/PVE/QemuServer.pm
> @@ -3944,7 +3944,13 @@ sub config_to_command {
>  
>      if (my $viommu = $machine_conf->{viommu}) {
>          my $viommu_devstr = '';
> -        $viommu_devstr .= ",aw-bits=$machine_conf->{'aw-bits'}" if $machine_conf->{'aw-bits'};
> +        if ($machine_conf->{'aw-bits'}) {
> +            $viommu_devstr .= ",aw-bits=$machine_conf->{'aw-bits'}";
> +
> +            # TODO remove message once this gets properly checked/warned about in QEMU itself.
> +            print "vIOMMU 'aw-bits' set to $machine_conf->{'aw-bits'}. Sometimes it is necessary to"
> +                . " set the CPU's 'guest-phys-bits' to the same value.\n";
> +        }
>  
>          if ($viommu eq 'intel') {
>              $viommu_devstr = "intel-iommu,intremap=on,caching-mode=on$viommu_devstr";



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel