[PATCH manager/qemu-server v6 00/21] Add API and UI for custom CPU models

[Arthur Bied-Charreton <a.bied-charreton@proxmox.com>]

This picks up and extends an old series [0] by Stefan Reiter.

This series adds a full CRUD API and a UI editor for custom CPU models
which allows users to manage them in the Datacenter interface rather
than editing /etc/pve/virtual-guest/cpu-models.conf manually.

It also improves on the existing VM CPU flags selector by providing a
list of nodes supporting each flag to help gauge cluster compatibility.

qemu-server gets a new CPUFlags module grouping cpu-flags-related
helpers. A new helper (query_available_cpu_flags) is introduced, which
derives the set of flags accepted by QEMU as -cpu arguments and
annotates each with the cluster nodes that report supporting it. The
existing node-level endpoint is extended with an accel parameter to
filter flags by acceleration type. Its default behavior is preserved
modulo the added 'supported-on' field per flag in the return value.

pve-manager gets two new cluster-level endpoints:

1. /cluster/qemu/cpu-flags queries flags available across the cluster
   and reports which node supports each of them. It differs from the
   node-level endpoint in qemu-server in that it returns *all* flags,
   as opposed to the VM-specific ones.

2. /cluster/qemu/custom-cpu-models provides full CRUD for custom CPU
   model definitions.

The UI gets a "Custom CPU models" view in the Datacenter panel (at the
bottom under "Guest Resources/Hardware") with an editor for model
properties. The flags can be filtered by acceleration type when
creating/editing a custom CPU model. In the ProcessorEdit window, a new
checkbox is added that allows setting the acceleration type in the VM
config.


Future work:

Future UX improvements could make acceleration-specific flags in custom
CPU models more explicit. Since custom CPU models have no configuration
field indicating whether they are meant to be used with KVM or TCG, the
UI always defaults to KVM, which would be quite annoying for a user
with a custom model meant to be used only with TCG.


Dependencies:

 Build:
  pve-manager requires bumped pve-doc-generator
  pve-manager requires bumped qemu-server

 Runtime:
  pve-manager requires bumped pve-docs
  pve-manager requires bumped qemu-server


Changes since v1 (thanks @Fiona):
https://lore.proxmox.com/pve-devel/20260401080028.62513-1-a.bied-charreton@proxmox.com/

Changes since v2:
https://lore.proxmox.com/pve-devel/20260430153505.527032-1-a.bied-charreton@proxmox.com/

Changes since v3:
https://lore.proxmox.com/pve-devel/20260430160109.565536-1-a.bied-charreton@proxmox.com/

Changes since v4 (thanks @David, @Fiona, @Thomas):

 pve-docs:
  - Change "CPU Type" anchor to _cpu_type

 qemu-server:
  - Use PVE::File instead of PVE::Tools for file_get_contents
  - Improve $understood_cpu_flag_dir doc comment in CPUFlags
  - Sort the cpu flag names only once at module load in CPUFlags
  - Normalize CPU flag names to QEMU's format
  - Compare booleans against JSON::true in query_supported_cpu_flags
  - Use postfix dereference style
  - Rename the *_custom_cpu_config helpers
  - Register the custom CPU config standard option as
    'pve-qm-custom-cpu-model'
  - Improve arch description in cpu-flags endpoint
  - Remove the added explicit type field in the CPU models config
  - Fix some broken formatting and perlcritic complaints
  - Avoid redundant config loads by get_custom_model
  - Rephrase doc comment of query_available_cpu_flags from "... flags
    that will be accepted by QEMU as -cpu arguments" to "... by PVE
    in processor configs", since the list includes 'nested-virt'

 pve-manager:
  - Change permissions for new cluster-wide endpoints to Sys.Audit/
    Sys.Modify
  - Use check_config in create and update custom CPU model CRUD
    operations
  - Use delete_from_config for deleting fields
  - Remove assert_if_modified from POST and DELETE endpoints
  - Require base model instead of defaulting to kvm64 for new custom
    models
  - Use plain panel for the new datacenter grouping
  - Avoid using private Ext.js APIs (BufferedRenderer.refreshView) when
    refreshing the CPU flag grid after editing
  - Replace the in-flag selector acceleration radio with a dedicated
    "KVM hardware virtualization" checkbox in VM ProcessorEdit, which
    now also persists in the VM config

Changes since v5 (thanks @Max):
 
 qemu-server:
  - Add 'builtin' flag to models returned by 
    /nodes/{node}/capabilities/qemu/cpu 

 pve-manager:
  - Filter builtin models out of the CPU model selector

[0] https://lore.proxmox.com/pve-devel/20211028114150.3245864-1-s.reiter@proxmox.com/


qemu-server:

Arthur Bied-Charreton (11):
  cpu config: rename CPU models config path variable
  cpu flags: move cpu flags-related utilities to their own module
  cpu flags: compare against JSON::true when querying supported flags
  cpu flags: normalize CPU flags to QEMU's format
  cpu flags: add helper querying CPU flags with nodes supporting them
  cpu config: rename custom CPU model config loader
  cpu config: add helpers to lock and write config
  cpu: register standard option for CPU format
  api: cpu flags: improve flags list returned by endpoint
  custom cpu models: avoid redundant config load
  cpu models: add 'builtin' flag to CPU model list endpoint

 src/PVE/API2/Qemu/CPU.pm             |   5 +
 src/PVE/API2/Qemu/CPUFlags.pm        |  26 ++-
 src/PVE/QemuServer.pm                |  33 +--
 src/PVE/QemuServer/CPUConfig.pm      | 120 +++--------
 src/PVE/QemuServer/CPUFlags.pm       | 308 +++++++++++++++++++++++++++
 src/PVE/QemuServer/Makefile          |   1 +
 src/test/run_config2command_tests.pl |   2 +-
 7 files changed, 373 insertions(+), 122 deletions(-)
 create mode 100644 src/PVE/QemuServer/CPUFlags.pm


pve-manager:

Arthur Bied-Charreton (10):
  cluster: reorder imports
  cluster: makefile: reorder perl sources and align backslashes
  api: add endpoint querying available CPU flags cluster-wide
  api: add CRUD handlers for custom CPU models
  ui: cpu model selector: allow filtering out custom and builtin models
  ui: add basic custom CPU model editor
  ui: cpu flags selector: add CPU flag editor for custom models
  ui: cpu flags selector: allow filtering out flags supported on 0 nodes
  ui: cpu flags selector: add search bar for large lists of flags
  ui: group custom CPU with resource mappings

 PVE/API2/Cluster.pm                      |   9 +-
 PVE/API2/Cluster/Makefile                |  12 +-
 PVE/API2/Cluster/Qemu.pm                 |  47 +++++
 PVE/API2/Cluster/Qemu/CPUFlags.pm        |  68 +++++++
 PVE/API2/Cluster/Qemu/CustomCPUModels.pm | 204 ++++++++++++++++++++
 PVE/API2/Cluster/Qemu/Makefile           |  18 ++
 www/manager6/Makefile                    |   3 +
 www/manager6/dc/CPUTypeEdit.js           | 153 +++++++++++++++
 www/manager6/dc/CPUTypeView.js           | 150 +++++++++++++++
 www/manager6/dc/Config.js                |  27 +++
 www/manager6/form/CPUModelSelector.js    |  11 ++
 www/manager6/form/PhysBitsSelector.js    | 153 +++++++++++++++
 www/manager6/form/VMCPUFlagSelector.js   | 231 +++++++++++++++++++----
 www/manager6/qemu/ProcessorEdit.js       |  42 +++++
 14 files changed, 1090 insertions(+), 38 deletions(-)
 create mode 100644 PVE/API2/Cluster/Qemu.pm
 create mode 100644 PVE/API2/Cluster/Qemu/CPUFlags.pm
 create mode 100644 PVE/API2/Cluster/Qemu/CustomCPUModels.pm
 create mode 100644 PVE/API2/Cluster/Qemu/Makefile
 create mode 100644 www/manager6/dc/CPUTypeEdit.js
 create mode 100644 www/manager6/dc/CPUTypeView.js
 create mode 100644 www/manager6/form/PhysBitsSelector.js


Summary over all repositories:
  21 files changed, 1463 insertions(+), 160 deletions(-)

-- 
Generated by murpp 0.11.0