all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Dominik Csapak <d.csapak@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH docs v6 1/1] qemu: add documentation about cluster device mapping
Date: Wed, 14 Jun 2023 10:46:22 +0200	[thread overview]
Message-ID: <20230614084622.1446211-23-d.csapak@proxmox.com> (raw)
In-Reply-To: <20230614084622.1446211-1-d.csapak@proxmox.com>

explain why someone would want it, how to configure and which privileges
are necessary

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
new in v6
 qm-pci-passthrough.adoc |  8 ++++
 qm.adoc                 | 86 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 94 insertions(+)

diff --git a/qm-pci-passthrough.adoc b/qm-pci-passthrough.adoc
index df6cf21..b90a0b9 100644
--- a/qm-pci-passthrough.adoc
+++ b/qm-pci-passthrough.adoc
@@ -400,6 +400,14 @@ Example configuration with an `Intel GVT-g vGPU` (`Intel Skylake 6700k`):
 With this set, {pve} automatically creates such a device on VM start, and
 cleans it up again when the VM stops.
 
+Use in Clusters
+~~~~~~~~~~~~~~~
+
+It is also possible to map devices on a cluster level, so that they can be
+properly used with HA and hardware changes are detected and non root users
+can configure them. See xref:resource_mapping[Resource Mapping]
+for details on that.
+
 ifdef::wiki[]
 
 See Also
diff --git a/qm.adoc b/qm.adoc
index c6dc652..53f6450 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -753,6 +753,10 @@ if you use a SPICE client which supports it. If you add a SPICE USB port
 to your VM, you can passthrough a USB device from where your SPICE client is,
 directly to the VM (for example an input device or hardware dongle).
 
+It is also possible to map devices on a cluster level, so that they can be
+properly used with HA and hardware changes are detected and non root users
+can configure them. See xref:resource_mapping[Resource Mapping]
+for details on that.
 
 [[qm_bios_and_uefi]]
 BIOS and UEFI
@@ -1511,6 +1515,88 @@ chosen, the first of:
 3. The first non-shared storage from any VM disk.
 4. The storage `local` as a fallback.
 
+[[resource_mapping]]
+Resource Mapping
+~~~~~~~~~~~~~~~~
+
+When using or referencing local resources (e.g. address of a pci device), using
+the raw address or id is sometimes problematic, for example:
+
+* when using HA, a different device with the same id or path may exist on the
+  target node, and if one is not careful when assigning such guests to HA
+  groups, the wrong device could be used, breaking configurations.
+
+* changing hardware can change ids and paths, so one would have to check all
+  assigned devices and see if the path or id is still correct.
+
+To handle this better, one can define cluster wide resource mappings, such that
+a resource has a cluster unique, user selected identifier which can correspond
+to different devices on different hosts. With this, HA won't start a guest with
+a wrong device, and hardware changes can be detected.
+
+Creating such a mapping can be done with the {pve} web GUI under `Datacenter`
+in the relevant tab in the `Resource Mappings` category, or on the cli with
+
+----
+# pvesh create /cluster/mapping/TYPE OPTIONS
+----
+
+Where `TYPE` is the hardware type (currently either `pci` or `usb`) and
+`OPTIONS` are the device mappings and other configuration parameters.
+
+Note that the options must include a map property with all identifying
+properties of that hardware, so that it's possible to verify the hardware did
+not change and the correct device is passed through.
+
+For example to add a PCI device as `device1` with the path `0000:01:00.0` that
+has the device id `0001` and the vendor id `0002` on the node `node1`, and
+`0000:02:00.0` on `node2` you can add it with:
+
+----
+# pvesh create /cluster/mapping/pci --id device1 \
+ --map node=node1,path=0000:01:00.0,id=0002:0001 \
+ --map node=node2,path=0000:02:00.0,id=0002:0001
+----
+
+You must repeat the `map` parameter for each node where that device should have
+a mapping.
+
+Using the GUI makes this much easier, as the correct properties are
+automatically picked up and sent to the API.
+
+It's also possible for PCI devices to provide multiple devices per node with
+multiple map properties for the nodes. If such a device is assigned to a guest,
+the first free one will be used when the guest is started. The order of the
+paths given is also the order in which they are tried, so arbitrary allocation
+policies can be implemented.
+
+This is useful for devices with SR-IOV, since some times it is not important
+which exact virtual function is passed through.
+
+You can assign such a device to a guest either with the GUI or with
+
+----
+# qm set ID -hostpci0 NAME
+----
+
+for PCI devices, or
+
+----
+# qm set ID -usb0 NAME
+----
+
+for USB devices.
+
+Where `ID` is the guests id and `NAME` is the chosen name for the created
+mapping. All usual options for passing through the devices are allowed, such as
+`mdev`.
+
+To create mappings `Mapping.Modify` on `/mapping/TYPE/NAME` is necessary
+(where `TYPE` is the device type and `NAME` is the name of the mapping).
+
+To use these mappings, `Mapping.Use` on `/mapping/TYPE/NAME` is necessary (in
+addition to the normal guest privileges to edit the configuration).
+
 Managing Virtual Machines with `qm`
 ------------------------------------
 
-- 
2.30.2





  parent reply	other threads:[~2023-06-14  8:47 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-06-14  8:46 [pve-devel] [PATCH qemu-server/manger/docs v6] cluster mapping Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH qemu-server v6 1/6] enable cluster mapped USB devices for guests Dominik Csapak
2023-06-16  7:50   ` Fabian Grünbichler
2023-06-14  8:46 ` [pve-devel] [PATCH qemu-server v6 2/6] enable cluster mapped PCI " Dominik Csapak
2023-06-16  7:49   ` Fabian Grünbichler
2023-06-14  8:46 ` [pve-devel] [PATCH qemu-server v6 3/6] check_local_resources: extend for mapped resources Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH qemu-server v6 4/6] api: migrate preconditions: use new check_local_resources info Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH qemu-server v6 5/6] migration: check for mapped resources Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH qemu-server v6 6/6] add test for mapped pci devices Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 01/15] pvesh: fix parameters for proxyto_callback Dominik Csapak
2023-06-16  9:27   ` [pve-devel] applied: " Wolfgang Bumiller
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 02/15] api: add resource map api endpoints for PCI and USB Dominik Csapak
2023-06-16  7:50   ` Fabian Grünbichler
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 03/15] ui: parser: add helper for lists of property strings Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 04/15] ui: form/USBSelector: make it more flexible with nodename Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 05/15] ui: form: add PCIMapSelector Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 06/15] ui: form: add USBMapSelector Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 07/15] ui: qemu/PCIEdit: rework panel to add a mapped configuration Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 08/15] ui: qemu/USBEdit: add 'mapped' device case Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 09/15] ui: form: add MultiPCISelector Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 10/15] ui: add edit window for pci mappings Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 11/15] ui: add edit window for usb mappings Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 12/15] ui: add ResourceMapTree Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 13/15] ui: allow configuring pci and usb mapping Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 14/15] ui: window/Migrate: allow mapped devices Dominik Csapak
2023-06-14  8:46 ` [pve-devel] [PATCH manager v6 15/15] ui: improve permission handling for hardware Dominik Csapak
2023-06-14  8:46 ` Dominik Csapak [this message]
2023-06-14 12:01 ` [pve-devel] [PATCH qemu-server/manger/docs v6] cluster mapping Markus Frank
2023-06-16  7:51 ` Fabian Grünbichler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230614084622.1446211-23-d.csapak@proxmox.com \
    --to=d.csapak@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal