Re: [pve-devel] [RFC qemu-server v2 3/4] fix #6378 (continued): warn intel-iommu users about iommu and host aw bits mismatch

[Fiona Ebner <f.ebner@proxmox.com>]

Am 02.09.25 um 1:23 PM schrieb Daniel Kral:
> For certain host CPUs, such as Intel consumer-grade CPUs, there is a
> frequent mismatch between the CPU's physical address width and the

What do you mean by "frequent"? You already conditionalized with "For
certain host CPUs". Do you mean "the default IOMMU's address witdth"?

> IOMMU's address width.
> 
> If a virtual machine is setup with an intel-iommu device, qemu allocates
> and maps the (virtual) I/O address space (IOAS) for a VFIO passthrough
> device with iommufd.
> 
> In case of a mismatch of the address width of the host CPU and IOMMU
> CPU, the guest physical address space (GPAS) and memory-type range
> registers (MTRRs) are setup to the host CPU's address width, which
> causes IOAS to be allocated and mapped outside of the IOMMU's maximum
> guest address width (MGAW) and causes the following error from qemu (the
> error message is copied from the user forum [0]):
> 
>     kvm: vfio_container_dma_map(0x5c9222494280, 0x380000000000, 0x10000, 0x78075ee70000) = -22 (Invalid argument)
> 
> This error is rather confusing and unhelpful to users, so warn them
> about a CPU physical address width that exceeds the IOMMU address width.
> 
> [0] https://forum.proxmox.com/threads/vm-wont-start-with-pci-passthrough-after-upgrade-to-9-0.169586/page-3#post-795717
> 

After this commit, the test added by qemu-server 1/4 fails on my system:
not ok 51 - 'q35-viommu-intel-aw-bits.conf' - Check if aw-bits are
propagated correctly to intel-iommu device
#   Failed test ''q35-viommu-intel-aw-bits.conf' - Check if aw-bits are
propagated correctly to intel-iommu device'
#   at ./run_config2command_tests.pl line 599.
# got unexpected warning 'guest address width exceeds vIOMMU address
width: 40 > 39'

You'd need to mock the relevant parts to avoid querying the real host.

> Signed-off-by: Daniel Kral <d.kral@proxmox.com>
> ---
> I already talked about this with @Fiona off-list, but the code this
> adds to qemu-server only for a warning is quite a lot, but is more
> readable than the above error that is only issued when the VM is already
> run.
> 
> Particularily, I don't like the logic duplication of
> get_cpu_address_width(...), which tries to copy what
> target/i386/{,host-,kvm/kvm-}cpu.c do to retrieve the {,guest_}phys_bits
> value, where I'd rather see this implemented in pve-qemu as in [0].
> 
> There are two qemu and edk2 discussion threads that might help in
> deciding how to go with this patch [0] [1]. It could also be better to
> implement this downstream in pve-qemu for now similar to [0], or of
> course contribute to upstream with an actual fix.
> 
> [0] https://lore.kernel.org/qemu-devel/20250130115800.60b7cbe6.alex.williamson@redhat.com/
> [1] https://edk2.groups.io/g/devel/topic/patch_v1/102359124

To avoid all the complexity and maintainability burden to stay
compatible with how QEMU calculates, can we simply notify/warn users who
set aw-bits that they might need to set guest-phys-bits to the same
value too?

> @@ -133,6 +133,17 @@ sub assert_valid_machine_property {
>      }
>  }
>  
> +sub check_valid_iommu_address_width {
> +    my ($machine_conf, $machine_version, $cpu_aw_bits) = @_;
> +    if ($machine_conf->{viommu} && $machine_conf->{viommu} eq 'intel') {
> +        my $iommu_aw_bits_default = min_version($machine_version, 9, 2) ? 48 : 39;
> +        my $iommu_aw_bits = $machine_conf->{'aw-bits'} // $iommu_aw_bits_default;
> +
> +        warn "guest address width exceeds vIOMMU address width: $cpu_aw_bits > $iommu_aw_bits\n"
> +            if $cpu_aw_bits && $iommu_aw_bits && $cpu_aw_bits > $iommu_aw_bits;

Should mention that it can be fixed by setting the guest-phys-bits
accordingly.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel