From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <m.frank@proxmox.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 7896094DE6 for <pve-devel@lists.proxmox.com>; Thu, 11 Apr 2024 12:49:14 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 569AF1FBAF for <pve-devel@lists.proxmox.com>; Thu, 11 Apr 2024 12:49:14 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for <pve-devel@lists.proxmox.com>; Thu, 11 Apr 2024 12:49:12 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 2799644B79 for <pve-devel@lists.proxmox.com>; Thu, 11 Apr 2024 12:49:12 +0200 (CEST) From: Markus Frank <m.frank@proxmox.com> To: pve-devel@lists.proxmox.com Date: Thu, 11 Apr 2024 12:48:21 +0200 Message-Id: <20240411104822.832432-3-m.frank@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240411104822.832432-1-m.frank@proxmox.com> References: <20240411104822.832432-1-m.frank@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.032 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [archive.org, proxmox.com, intel.com, qemu.org, wikipedia.org, medium.com] Subject: [pve-devel] [PATCH docs v9 2/3] add vIOMMU documentation X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/> List-Post: <mailto:pve-devel@lists.proxmox.com> List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe> X-List-Received-Date: Thu, 11 Apr 2024 10:49:14 -0000 Signed-off-by: Markus Frank <m.frank@proxmox.com> --- qm-pci-passthrough.adoc | 50 +++++++++++++++++++++++++++++++++++++++++ qm.adoc | 1 + 2 files changed, 51 insertions(+) diff --git a/qm-pci-passthrough.adoc b/qm-pci-passthrough.adoc index 3abeb50..bbd6b85 100644 --- a/qm-pci-passthrough.adoc +++ b/qm-pci-passthrough.adoc @@ -499,6 +499,56 @@ properly used with HA and hardware changes are detected and non root users can configure them. See xref:resource_mapping[Resource Mapping] for details on that. +[[qm_pci_viommu]] +vIOMMU (emulated IOMMU) +~~~~~~~~~~~~~~~~~~~~~~~ + +vIOMMU is the emulation of a hardware IOMMU within a virtual machine, providing +improved memory access control and security for virtualized I/O devices. Using +the vIOMMU option also allows you to pass through PCI devices to level-2 VMs in +level-1 VMs via https://pve.proxmox.com/wiki/Nested_Virtualization[Nested Virtualization]. +There are currently two vIOMMU implementations available: Intel and VirtIO. + +Host requirement: + +* Add `intel_iommu=on` or `amd_iommu=on` depending on your CPU to your kernel +command line. + +Intel vIOMMU +^^^^^^^^^^^^ + +Intel vIOMMU specific VM requirements: + +* Whether you are using an Intel or AMD CPU on your host, it is important to set +`intel_iommu=on` in the VMs kernel parameters. + +* To use Intel vIOMMU you need to set *q35* as the machine type. + +If all requirements are met, you can add `viommu=intel` to the machine parameter +in the configuration of the VM that should be able to pass through PCI devices. + +---- +# qm set VMID -machine q35,viommu=intel +---- + +https://wiki.qemu.org/Features/VT-d[QEMU documentation for VT-d] + +VirtIO vIOMMU +^^^^^^^^^^^^^ + +This vIOMMU implementation is more recent and does not have as many limitations +as Intel vIOMMU but is currently less used in production and less documentated. + +With VirtIO vIOMMU there is *no* need to set any kernel parameters. It is also +*not* necessary to use q35 as the machine type, but it is advisable if you want +to use PCIe. + +---- +# qm set VMID -machine q35,viommu=virtio +---- + +https://web.archive.org/web/20230804075844/https://michael2012z.medium.com/virtio-iommu-789369049443[Blog-Post by Michael Zhao explaining virtio-iommu] + ifdef::wiki[] See Also diff --git a/qm.adoc b/qm.adoc index 91f3617..45e3a57 100644 --- a/qm.adoc +++ b/qm.adoc @@ -152,6 +152,7 @@ https://en.wikipedia.org/wiki/Intel_440FX[Intel 440FX] or the https://ark.intel.com/content/www/us/en/ark/products/31918/intel-82q35-graphics-and-memory-controller.html[Q35] chipset, which also provides a virtual PCIe bus, and thus may be desired if you want to pass through PCIe hardware. +Additionally, you can select a xref:qm_pci_viommu[vIOMMU] implementation. Machine Version +++++++++++++++ -- 2.39.2