[pve-devel] [PATCH docs 0/3] pci-passthrough: update section for newer kernels, add mdev driver note

[pve-devel] [PATCH docs 0/3] pci-passthrough: update section for newer kernels, add mdev driver note

[Christoph Heiss]

#1 and #2 update the section w.r.t. to the current 6.8 kernels, dropping
some now-obsolete requirements for PCI passthrough.

#3 adds two small notes when passing through mediated devices, as these
work a bit differently than normal passthrough.

Noticed these things while testing out Dominik's Nvidia vGPU series [0].

[0] https://lore.proxmox.com/pve-devel/20240806122203.2266054-1-d.csapak@proxmox.com/

Christoph Heiss (3):
  pci-passthrough: drop obsolete `vfio_virqfd` kernel module requirement
  pci-passthrough: mention that `intel_iommu=on` is the default for 6.8+
  pci-passthrough: add note about mediated devices w.r.t kernel modules

 qm-pci-passthrough.adoc | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

-- 
2.46.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH docs 1/3] pci-passthrough: drop obsolete `vfio_virqfd` kernel module requirement

[Christoph Heiss]

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
 qm-pci-passthrough.adoc | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/qm-pci-passthrough.adoc b/qm-pci-passthrough.adoc
index 4cbaadd..c3dd184 100644
--- a/qm-pci-passthrough.adoc
+++ b/qm-pci-passthrough.adoc
@@ -92,17 +92,13 @@ to the xref:sysboot_edit_kernel_cmdline[kernel commandline].
 
 .Kernel Modules
 
-//TODO: remove `vfio_virqfd` stuff with eol of pve 7
 You have to make sure the following modules are loaded. This can be achieved by
-adding them to `'/etc/modules''. In kernels newer than 6.2 ({pve} 8 and onward)
-the 'vfio_virqfd' module is part of the 'vfio' module, therefore loading
-'vfio_virqfd' in {pve} 8 and newer is not necessary.
+adding them to `'/etc/modules''.
 
 ----
  vfio
  vfio_iommu_type1
  vfio_pci
- vfio_virqfd #not needed if on kernel 6.2 or newer
 ----
 
 [[qm_pci_passthrough_update_initramfs]]
-- 
2.46.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH docs 2/3] pci-passthrough: mention that `intel_iommu=on` is the default for 6.8+

[Christoph Heiss]

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
 qm-pci-passthrough.adoc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/qm-pci-passthrough.adoc b/qm-pci-passthrough.adoc
index c3dd184..70e33f2 100644
--- a/qm-pci-passthrough.adoc
+++ b/qm-pci-passthrough.adoc
@@ -74,7 +74,8 @@ xref:sysboot_edit_kernel_cmdline[kernel command line] kernels by adding:
  intel_iommu=on
 ----
 
-For AMD CPUs it should be enabled automatically.
+For AMD CPUs it should be enabled automatically. For Intel CPUs this is also
+the default since the 6.8 kernel series.
 
 .IOMMU Passthrough Mode
 
-- 
2.46.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH docs 2/3] pci-passthrough: mention that `intel_iommu=on` is the default for 6.8+

[Aaron Lauterer]

Looking at the surrounding context, I would rework this section even 
more. Before I just apply my approach, what do you think about the 
following change?
---
diff --git a/qm-pci-passthrough.adoc b/qm-pci-passthrough.adoc
index 4f19f2a..ca07692 100644
--- a/qm-pci-passthrough.adoc
+++ b/qm-pci-passthrough.adoc
@@ -63,19 +63,18 @@ some configuration to enable PCI(e) passthrough.

  .IOMMU

-First, you will have to enable IOMMU support in your BIOS/UEFI. Usually the
+You will have to enable IOMMU support in your BIOS/UEFI. Usually the
  corresponding setting is called `IOMMU` or `VT-d`, but you should find 
the exact
  option name in the manual of your motherboard.

-For Intel CPUs, you also need to enable the IOMMU on the
-xref:sysboot_edit_kernel_cmdline[kernel command line] kernels by adding:
+With recent kernels (6.8 or newer), `IOMMU` should be enabled 
automatically.
+On older kernels, it was necessary to enable it on Intel CPUs via the
+xref:sysboot_edit_kernel_cmdline[kernel command line] by adding:

  ----
   intel_iommu=on
  ----

-For AMD CPUs it should be enabled automatically.
-
  .IOMMU Passthrough Mode

  If your hardware supports IOMMU passthrough mode, enabling this mode might



On  2024-10-22  12:19, Christoph Heiss wrote:
> Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
> ---
>   qm-pci-passthrough.adoc | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/qm-pci-passthrough.adoc b/qm-pci-passthrough.adoc
> index c3dd184..70e33f2 100644
> --- a/qm-pci-passthrough.adoc
> +++ b/qm-pci-passthrough.adoc
> @@ -74,7 +74,8 @@ xref:sysboot_edit_kernel_cmdline[kernel command line] kernels by adding:
>    intel_iommu=on
>   ----
>   
> -For AMD CPUs it should be enabled automatically.
> +For AMD CPUs it should be enabled automatically. For Intel CPUs this is also
> +the default since the 6.8 kernel series.
>   
>   .IOMMU Passthrough Mode
>   



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH docs 2/3] pci-passthrough: mention that `intel_iommu=on` is the default for 6.8+

[Christoph Heiss]

Thanks for taking a look!

On Tue, Oct 29, 2024 at 02:17:59PM GMT, Aaron Lauterer wrote:
> Looking at the surrounding context, I would rework this section even more.
> Before I just apply my approach, what do you think about the following
> change?
> ---
> diff --git a/qm-pci-passthrough.adoc b/qm-pci-passthrough.adoc
> index 4f19f2a..ca07692 100644
> --- a/qm-pci-passthrough.adoc
> +++ b/qm-pci-passthrough.adoc
> @@ -63,19 +63,18 @@ some configuration to enable PCI(e) passthrough.
>
>  .IOMMU
>
> -First, you will have to enable IOMMU support in your BIOS/UEFI. Usually the
> +You will have to enable IOMMU support in your BIOS/UEFI. Usually the
>  corresponding setting is called `IOMMU` or `VT-d`, but you should find the
> exact
>  option name in the manual of your motherboard.
>
> -For Intel CPUs, you also need to enable the IOMMU on the
> -xref:sysboot_edit_kernel_cmdline[kernel command line] kernels by adding:
> +With recent kernels (6.8 or newer), `IOMMU` should be enabled

I'd drop the code tags for IOMMU here (and add a "the") -- like in the
first sentence -- since this is referring to the IOMMU in a more general
sense rather than a specific setting as above such, IMO.

> automatically.

Since you drop the note about AMD CPUs below, I'd either keep that one
below or explicitly mention here that this automatic enablement is for
both Intel and AMD CPUs.

Otherwise users/admin might be left wondering what needs to be done for
AMD CPUs, if only Intel are mentioned here. Just to be thorough.

> +On older kernels, it was necessary to enable it on Intel CPUs via the
> +xref:sysboot_edit_kernel_cmdline[kernel command line] by adding:
>
>  ----
>   intel_iommu=on
>  ----
>
> -For AMD CPUs it should be enabled automatically.
> -
>  .IOMMU Passthrough Mode
>
>  If your hardware supports IOMMU passthrough mode, enabling this mode might
> [..]


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH docs 3/3] pci-passthrough: add note about mediated devices w.r.t kernel modules

[Christoph Heiss]

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
 qm-pci-passthrough.adoc | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/qm-pci-passthrough.adoc b/qm-pci-passthrough.adoc
index 70e33f2..ae0bdde 100644
--- a/qm-pci-passthrough.adoc
+++ b/qm-pci-passthrough.adoc
@@ -96,6 +96,14 @@ to the xref:sysboot_edit_kernel_cmdline[kernel commandline].
 You have to make sure the following modules are loaded. This can be achieved by
 adding them to `'/etc/modules''.
 
+.Mediated devices passthrough
+[NOTE]
+====
+If passing through mediated devices (e.g. vGPUs), the following is not needed.
+In these cases, the device will be owned by the appropriate host-driver
+directly.
+====
+
 ----
  vfio
  vfio_iommu_type1
@@ -274,6 +282,13 @@ Kernel driver in use: vfio-pci
 or the 'in use' line is missing entirely, the device is ready to be used for
 passthrough.
 
+.Mediated devices
+[NOTE]
+====
+For mediated devices this line will differ as the device will be owned as the
+host driver directly, not 'vfio-pci'.
+====
+
 [[qm_pci_passthrough_vm_config]]
 VM Configuration
 ^^^^^^^^^^^^^^^^
-- 
2.46.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] applied: [PATCH docs 0/3] pci-passthrough: update section for newer kernels, add mdev driver note

[Aaron Lauterer]

On  2024-10-22  12:19, Christoph Heiss wrote:
> #1 and #2 update the section w.r.t. to the current 6.8 kernels, dropping
> some now-obsolete requirements for PCI passthrough.
> 
> #3 adds two small notes when passing through mediated devices, as these
> work a bit differently than normal passthrough.
> 
> Noticed these things while testing out Dominik's Nvidia vGPU series [0].
> 
> [0] https://lore.proxmox.com/pve-devel/20240806122203.2266054-1-d.csapak@proxmox.com/
> 
> Christoph Heiss (3):
>    pci-passthrough: drop obsolete `vfio_virqfd` kernel module requirement
>    pci-passthrough: mention that `intel_iommu=on` is the default for 6.8+
>    pci-passthrough: add note about mediated devices w.r.t kernel modules
> 
>   qm-pci-passthrough.adoc | 24 ++++++++++++++++++------
>   1 file changed, 18 insertions(+), 6 deletions(-)
> 


applied patches 1 & 3.
I left patch 2 out and created a new one that rewords and restructures 
the whole section for better readability.

Thanks!


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel