From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id DF3C09E108 for ; Tue, 6 Jun 2023 15:52:26 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B900836387 for ; Tue, 6 Jun 2023 15:52:26 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Tue, 6 Jun 2023 15:52:24 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 98EFC48C8A for ; Tue, 6 Jun 2023 15:52:24 +0200 (CEST) From: Dominik Csapak To: pve-devel@lists.proxmox.com Date: Tue, 6 Jun 2023 15:51:59 +0200 Message-Id: <20230606135222.984747-1-d.csapak@proxmox.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.135 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: [pve-devel] [PATCH access-control/guest-common/qemu-server/manager v5] cluster mapping X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jun 2023 13:52:26 -0000 this series aims to add a cluster-wide device mapping for pci and usb devices. so that an admin can configure a device to be availble for migration and configuring for uses that are non-root (the existing pattern can be copied easily for other types, e.g. markus upcoming folder sharing) note that this series requires the array support in api/section config[0] but how the api is formed/where the data comes should not be hard to change this time the gui is included again, but since the api changed quite a bit, the gui had to follow suit the docs are not fully done yet, i'll send them as soon as i'm finished changes from v4: * gui now included again, changes from v3: - the usb/pci panels are now separate but displayed together, they share most of the code - api paths adapted to the api change - sends now the array of all maps on each edit, included new path wih helpers for that - api now has full arrays for multiple mappings of one node, so we don't have to extract the info of the first device anymore and attach it to the list of pci lists, but rather each selected device is it's own map entry - on the overview, error checking is fully done on the gui, by querying the pci list of each node - some rewordings/rename of commits/files and classes * we now use the 'mapping' terminology througout: - the api paths are /cluster/mapping - the config files are in /etc/pve/mapping/ - the privileges are now 'Mapping.*' - the acl paths are now in '/mapping/' - the perl modules are now *::Mapping::* * adapted to the changes in the array patches, namely the 'map' parameter does not have to be checked anymore if they are an arrayref * the hostpci/usb configs are now checked during clone/restore NOTE: having a hostpci/usb config in the backup/source config did not prevent non root@pam users from cloning/restoring before, so this is a breaking change. * Administrator now retains the Mapping.* privileges * Add a Mapping.Audit privilege changes from v3: * the configs are now split by type (for ease of use of the section config) and live in pve-guest-common, to avoid a cyclic dependcy * the configs are section configs now (with mentioned array support) * the api is now only defined in /cluster/resource/{TYPE} and has no nodespecific api anymore, besides a 'check-node' parameter (see the pve-manager patch for more details on that) * the internal structure of the pci parsing changed completely, making the structure more understandable * a single map entry now has the same semantic as the qemu-server hostpci config entry, meaning if you want multiple mappings per host, you have to add multiple map entries. this is a more flexible approach, and the parsing code gets a bit simpler * combined some properties in the config (e.g. vendor/device) so that we don't have too many * squashed some changes together, as they didn't make much sense separately anyway (e.g. api/config patches) and it didn't make reviewing easier * changed the ACL paths & privileges to be more general * surely some other changes i forgot.. changes from v2: * some bug fixes (e.g use of unitialized variable) * don't set mdev for multifunction devices -> this should fix alexandres issue, since it's not possible anymore to select a mediated device when having a multifunction device selected changes from v1: * dropped 'check_hw_perm' (just use 'check_full' now) * added some cleanups * renamed the buttons in the ui (hopefully better now) * added multi device mapping for each host this includes a new 'multi pci' selector for that window, which automatically adds entries for the whole slots which, when selected, disabled the selection of the individual functions * fixed some issues (e.g. missing entries in the 'caps' object, wrong usb config parsing, etc.) changes from the rfc: * new cluster wide gui instead of node-local one (removed that, since it's not necessary when we have a cluster-wide one) * uses json instead of a section config * api is quite different overall, i split the type into its own level for configuring, similar to what we do in pbs (e.g. /nodes/NODENAME/hardware/mapping/usb/) * fixed quite some bugs the rfc had * added patch for handling the gui with limited permissions better * added a 'comment' field for mappings dependencies are pretty straight forward this time around (if i'm not overlooking something): qemu-server/pve-manager -> new access-control/pve-guest-common -> new pve-cluster 0: https://lists.proxmox.com/pipermail/pve-devel/2023-June/057193.html pve-access-control: Dominik Csapak (1): add privileges and paths for cluster resource mapping src/PVE/AccessControl.pm | 19 +++++++++++++++++++ src/PVE/RPCEnvironment.pm | 3 ++- 2 files changed, 21 insertions(+), 1 deletion(-) pve-guest-common: Dominik Csapak (2): vzdump: change 'exclude-path' from alist to an array format add PCI/USB Mapping configs src/Makefile | 3 + src/PVE/Mapping/PCI.pm | 226 +++++++++++++++++++++++++++++++++++++++ src/PVE/Mapping/USB.pm | 183 +++++++++++++++++++++++++++++++ src/PVE/VZDump/Common.pm | 7 +- 4 files changed, 417 insertions(+), 2 deletions(-) create mode 100644 src/PVE/Mapping/PCI.pm create mode 100644 src/PVE/Mapping/USB.pm qemu-server: Dominik Csapak (7): api: switch agent api call to 'array' type enable cluster mapped USB devices for guests enable cluster mapped PCI devices for guests check_local_resources: extend for mapped resources api: migrate preconditions: use new check_local_resources info migration: check for mapped resources add test for mapped pci devices PVE/API2/Qemu.pm | 123 ++++++++- PVE/API2/Qemu/Agent.pm | 15 +- PVE/QemuMigrate.pm | 23 +- PVE/QemuServer.pm | 162 +++++++++--- PVE/QemuServer/PCI.pm | 243 +++++++++++++++--- PVE/QemuServer/USB.pm | 27 +- test/MigrationTest/Shared.pm | 14 + test/cfg2cmd/q35-linux-hostpci-mapping.conf | 17 ++ .../q35-linux-hostpci-mapping.conf.cmd | 36 +++ test/cfg2cmd/q35-linux-hostpci.conf | 2 +- test/cfg2cmd/q35-linux-hostpci.conf.cmd | 2 +- test/run_config2command_tests.pl | 83 ++++++ 12 files changed, 648 insertions(+), 99 deletions(-) create mode 100644 test/cfg2cmd/q35-linux-hostpci-mapping.conf create mode 100644 test/cfg2cmd/q35-linux-hostpci-mapping.conf.cmd pve-manager: Dominik Csapak (15): pvesh: fix parameters for proxyto_callback api: add resource map api endpoints for PCI and USB ui: parser: add helpers for lists of property strings ui: form/USBSelector: make it more flexible with nodename ui: form: add PCIMapSelector ui: form: add USBMapSelector ui: qemu/PCIEdit: rework panel to add a mapped configuration ui: qemu/USBEdit: add 'mapped' device case ui: form: add MultiPCISelector ui: add edit window for pci mappings ui: add edit window for usb mappings ui: add ResourceMapTree ui: allow configuring pci and usb mapping ui: window/Migrate: allow mapped devices ui: improve permission handling for hardware PVE/API2/Cluster.pm | 8 + PVE/API2/Cluster/Makefile | 5 + PVE/API2/Cluster/Mapping.pm | 53 +++++ PVE/API2/Cluster/Mapping/Makefile | 18 ++ PVE/API2/Cluster/Mapping/PCI.pm | 298 ++++++++++++++++++++++++ PVE/API2/Cluster/Mapping/USB.pm | 293 +++++++++++++++++++++++ PVE/API2/Hardware.pm | 1 - PVE/API2/Nodes.pm | 1 + PVE/CLI/pvesh.pm | 10 +- www/css/ext6-pve.css | 4 + www/manager6/Makefile | 8 + www/manager6/Parser.js | 14 ++ www/manager6/StateProvider.js | 1 + www/manager6/data/PermPathStore.js | 1 + www/manager6/dc/Config.js | 45 +++- www/manager6/dc/PCIMapView.js | 106 +++++++++ www/manager6/dc/USBMapView.js | 98 ++++++++ www/manager6/form/MultiPCISelector.js | 288 +++++++++++++++++++++++ www/manager6/form/PCIMapSelector.js | 112 +++++++++ www/manager6/form/PCISelector.js | 18 +- www/manager6/form/USBMapSelector.js | 98 ++++++++ www/manager6/form/USBSelector.js | 33 ++- www/manager6/qemu/HardwareView.js | 17 +- www/manager6/qemu/PCIEdit.js | 323 +++++++++++++++++--------- www/manager6/qemu/USBEdit.js | 36 ++- www/manager6/tree/ResourceMapTree.js | 316 +++++++++++++++++++++++++ www/manager6/window/Migrate.js | 50 +++- www/manager6/window/PCIMapEdit.js | 205 ++++++++++++++++ www/manager6/window/USBMapEdit.js | 216 +++++++++++++++++ 29 files changed, 2536 insertions(+), 140 deletions(-) create mode 100644 PVE/API2/Cluster/Mapping.pm create mode 100644 PVE/API2/Cluster/Mapping/Makefile create mode 100644 PVE/API2/Cluster/Mapping/PCI.pm create mode 100644 PVE/API2/Cluster/Mapping/USB.pm create mode 100644 www/manager6/dc/PCIMapView.js create mode 100644 www/manager6/dc/USBMapView.js create mode 100644 www/manager6/form/MultiPCISelector.js create mode 100644 www/manager6/form/PCIMapSelector.js create mode 100644 www/manager6/form/USBMapSelector.js create mode 100644 www/manager6/tree/ResourceMapTree.js create mode 100644 www/manager6/window/PCIMapEdit.js create mode 100644 www/manager6/window/USBMapEdit.js -- 2.30.2