[PATCH docs/manager/qemu-server v4 00/17] Add API and UI for custom CPU models

[Arthur Bied-Charreton <a.bied-charreton@proxmox.com>]

This picks up and extends an old series [0] by Stefan Reiter.

This series adds a full CRUD API and a UI editor for custom CPU models
which allows users to manage them in the Datacenter interface rather
than editing /etc/pve/virtual-guest/cpu-models.conf manually.

It also improves on the existing VM CPU flags selector by providing a
list of nodes supporting each flag to help gauge cluster compatibility.

qemu-server gets a new CPUFlags module grouping cpu-flags-related
helpers. A new helper (query_available_cpu_flags) is introduced, which
derives the set of flags accepted by QEMU as -cpu arguments and
annotates each with the cluster nodes that report supporting it. The
existing node-level endpoint is extended with an accel parameter to
filter flags by acceleration type. Its default behavior is preserved
modulo the added 'supported-on' field per flag in the return value.

pve-manager gets two new cluster-level endpoints:

1. /cluster/qemu/cpu-flags queries flags available across the cluster
   and reports which node supports each of them. It differs from the
   node-level endpoint in qemu-server in that it returns *all* flags,
   as opposed to the VM-specific ones.

2. /cluster/qemu/custom-cpu-models provides full CRUD for custom CPU
   model definitions.

The UI gets a "Custom CPU models" view in the Datacenter panel (at the
bottom under "Guest Resources/Hardware") with an editor for model
properties. The flags can now be filtered by acceleration type when
creating a new VM or custom CPU model, and it is pre-selected based on
the VM's KVM config key when editing a pre-existing VM's CPU. The CPU
flags selector is also improved with the option to filter out flags
that are not supported anywhere in the cluster.

Dependencies:
 pve-manager requires bumped pve-docs
 pve-manager requires bumped qemu-server

Changes since v1:
 - Reorder patches according to dependency directions

 qemu-server:
  - Rebase
  - Add preparatory commit renaming $default_filename to
    $cpu_models_filename
  - Squash helpers into commits with their first users
  - Drop unrelated formatting change
  - Move cluster-wide cpu-flags to pve-manager, related changes in
    pve-manager changelog, keep vm-specific cpu-flags endpoint
  - Return all flags from cpu-flags endpoint, not only the ones with at
    least one node supporting them - allow filtering for that in the UI
  - Return only VM-specific cpu-flags by default in node-specific endpoint
  - Add acceleration type filter to cpu-flags endpoint, defaulting to kvm
  - Document why no flags are currently returned for aarch64
  - Only add nested-virt to flags for hosts that support one of svm/vmx
  - Rename extract_flags to query_available_cpu_flags
  - Remove web references from doc comments
  - Refactor query_available_cpu_flags to a more imperative style
  - Move custom CPU models CRUD endpoints to pve-manager, related changes
    in pve-manager changelog
  - Introduce QemuServer::CPUFlags module grouping flags-related helpers
  - Add flag_is_aliased helper to identify flags covered by nested-virt

 pve-manager:
  - Rebase
  - Drop unneeded dirty state fix from original series
  - Update unknown flags behavior to
    1. Show unknown flags at the top of the list in the UI
    2. Add a description to unknown flags signaling that the flag is not
    recognized in the cluster/for the currently selected acceleration type
  - Rename allowCustom CPU model selector config field to showCustomModels
  - Change "Reported Model" to "Base Model" in the UI
  - Link directly to "CPU Type" section in onlineHelp refs
  - Improve message in remove confirm dialog for custom CPU models
  - Move "Custom CPU Models" further down in Datacenter menu, and group it
    with "{Directory,Resource} Mappings" under "Guest Resources/Hardware"
  - Default VM-specific CPU flags selection to KVM during creation, and
    pre-select/lock it based on the VM's configured acceleration when
    editing
  - Remove hardcoded list of VM-specific flags in frontend
  - Change 'Set' to 'Value' in VMCpuFlagSelector
  - Use "" instead of `` for user-facing strings
  - Add search bar for long lists of CPU flags
  - Return all flags from the backend, not only the ones with at least one
    node supporting them - allow filtering for that in the UI
  - Show svm/vmx in cpu-flags list regardless of overlap with nested-virt
  - Add endpoint for querying CPU flags cluster-wide
  (`/cluster/qemu/cpu-flags`)
  - Document missing supported flags for `aarch64`
  - Add CRUD endpoint for custom CPU models
  (`/cluster/qemu/custom-cpu-models`)

 pve-docs:
  - Add qm_cpu_type anchor to CPU Type section in qm.adoc

Changes since v2:
 - Rebase all repos
 - Remove JS component names from commit subjects

 pve-manager:
  - Avoid losing flags that were set while "Show only flags supported by
    at least one node" was not checked when checking it again
  - Avoid overwriting the whole extraParams in setArch

Changes since v3:
 - Fix custom CPU model create requests by cleaning empty optional  
   values in the input panel value hook and dropping the unsupported 
   delete param on create 

[0] https://lore.proxmox.com/pve-devel/20211028114150.3245864-1-s.reiter@proxmox.com/


pve-docs:

Arthur Bied-Charreton (1):
  qm: add anchor to "CPU Type" section

 qm.adoc | 1 +
 1 file changed, 1 insertion(+)


qemu-server:

Arthur Bied-Charreton (7):
  cpu config: rename CPU models config path variable
  cpu flags: move cpu flags-related utilities to their own module
  cpu flags: add helper querying CPU flags with nodes supporting them
  cpu config: add helpers to lock and write config
  cpu: register standard option for CPU format
  cpu config: set 'type' field before writing
  cpu flags: improve flags list returned by endpoint

 src/PVE/API2/Qemu/CPUFlags.pm   |  27 +++-
 src/PVE/QemuServer.pm           |  24 +---
 src/PVE/QemuServer/CPUConfig.pm | 110 +++------------
 src/PVE/QemuServer/CPUFlags.pm  | 230 ++++++++++++++++++++++++++++++++
 src/PVE/QemuServer/Makefile     |   1 +
 5 files changed, 280 insertions(+), 112 deletions(-)
 create mode 100644 src/PVE/QemuServer/CPUFlags.pm


pve-manager:

Arthur Bied-Charreton (9):
  api: add endpoint querying available CPU flags cluster-wide
  api: add CRUD handlers for custom CPU models
  ui: cpu model selector: allow filtering out custom models
  ui: add basic custom CPU model editor
  ui: cpu flags selector: add CPU flag editor for custom models
  ui: cpu flags selector: fix buffered rendering error
  ui: cpu flags selector: allow filtering out flags supported on 0 nodes
  ui: cpu flags selector: add search bar for large lists of flags
  RFC: ui: group custom CPU with resource mappings

 PVE/API2/Cluster.pm                      |   7 +
 PVE/API2/Cluster/Makefile                |   4 +-
 PVE/API2/Cluster/Qemu.pm                 |  47 ++++
 PVE/API2/Cluster/Qemu/CPUFlags.pm        |  68 +++++
 PVE/API2/Cluster/Qemu/CustomCPUModels.pm | 211 ++++++++++++++++
 PVE/API2/Cluster/Qemu/Makefile           |  18 ++
 www/css/ext6-pve.css                     |   4 +
 www/manager6/Makefile                    |   3 +
 www/manager6/dc/CPUTypeEdit.js           | 105 ++++++++
 www/manager6/dc/CPUTypeView.js           | 149 +++++++++++
 www/manager6/dc/Config.js                |  27 ++
 www/manager6/form/CPUModelSelector.js    |  11 +
 www/manager6/form/PhysBitsSelector.js    | 153 +++++++++++
 www/manager6/form/VMCPUFlagSelector.js   | 308 ++++++++++++++++++++---
 www/manager6/qemu/ProcessorEdit.js       |   6 +
 15 files changed, 1090 insertions(+), 31 deletions(-)
 create mode 100644 PVE/API2/Cluster/Qemu.pm
 create mode 100644 PVE/API2/Cluster/Qemu/CPUFlags.pm
 create mode 100644 PVE/API2/Cluster/Qemu/CustomCPUModels.pm
 create mode 100644 PVE/API2/Cluster/Qemu/Makefile
 create mode 100644 www/manager6/dc/CPUTypeEdit.js
 create mode 100644 www/manager6/dc/CPUTypeView.js
 create mode 100644 www/manager6/form/PhysBitsSelector.js


Summary over all repositories:
  21 files changed, 1371 insertions(+), 143 deletions(-)

-- 
Generated by murpp 0.11.0