[PATCH common/manager/proxmox-widget-toolkit/storage/storage-plugin-examples v4 00/17] GUI Support for Custom Storage Plugins

["Max R. Carrara" <m.carrara@proxmox.com>]

GUI Support for Custom Storage Plugins - v4
===========================================

This refresh should now *hopefully* cover the remaining issues that were
found during reviewing, and also some other things that were found
off-list.

Huge thanks to the reviewers & testers @Thomas E., @Jakob K., and
@Alfonso K. -- I appreciate your work a lot!

Since a lot of little and not-so-little things changed here and there, I
have not included any previous Reviewed-by / Tested-by git trailers.

I have smoke-tested the series myself using the SSHFS plugin, but any
additional testing would be much appreciated.


Maintainer Notice
-----------------

- pve-storage needs a version bump on its libpve-common-perl dependency,
  since third-party storage plugins might now start using the new
  `multiline` format and have pve-common as a transitive dependency

- pve-manager needs version bumps on its libpve-storage-perl and
  proxmox-widget-toolkit dependencies, due to the API changes and the
  new helper in Utils.js

- libpve-storage-plugin-sshfs-perl needs a version bump on
  libpve-storage-perl, since it now uses the `multiline` string format
  for one of its properties, but does not explicitly depend on
  libpve-common-perl


Building
--------

Note that building pve-manager directly depends on the changes made to
pve-storage, so you'll have to install the storage package in your build
env first before building pve-manager. There shouldn't be any other
issues otherwise, if I'm not mistaken.


Notable Changes Since v3
------------------------

- Rebase on latest master.

- pve-common: Keep the `multiline` format as it is.

  There was one last remaining design decision to make, namely whether
  to support the `multiline` format "natively" inside SectionConfig,
  perhaps encoding the multiline string in base64 or something similar,
  since SectionConfig file format syntax has no concept of multiline
  strings.

  Since multiline strings are rarely used wherever we use SectionConfig,
  I decided to keep the format as it is, and not support it in SC. Sites
  that use this format are instead expected to handle it themselves.

  The main reason why this format is added is to be able to let users
  upload secrets like private keys or other kinds of credentials, and
  those things should be stored in `/etc/pve/priv` anyway, not in
  world-readable configuration files.

- pve-storage: Ensure that the `optional` key for a property is *always*
  set to what's in the plugin's `options()` method. This avoids
  mismatches between what the UI allows and what the backend allows.

- pve-storage: Likewise, the `x-sensitive`, `x-advanced`, and `x-hidden`
  keys are *always* set for each property, even if their resp. hashes
  were not defined in `plugindata()`.

- pve-storage: Use `PVE::Storage::Plugin::sensitive_properties()` to
  determine which properties should be marked with the `x-sensitive`
  key, since that helper also takes the old, hard-coded sensitive
  properties into account.

- pve-storage: The `content-dirs` property is no longer marked as
  `x-advanced`, since it is `x-hidden` anyway.

- proxmox-widget-toolkit: Handle default values for booleans
  (checkboxes) and enums (comboboxes) correctly. Checkboxes in
  particular always set a value, even if left empty; that behavior is
  now accounted for.

- proxmox-widget-toolkit: For the ACME challenge plugins, always mark
  all properties as optional in order to restore original behavior.

  It went a bit under my radar that some providers support
  alternative credentials as well. In previous series, a lot of fields
  became required, since the ACME schema was used as is. This meant that
  the UI wouldn't accept certain configurations anymore.

  Therefore, all ACME fields are marked as optional now (again), which
  restores the previous behavior.

- pve-manager: Wrap missing translatable strings with `gettext()`.

- pve-manager: Instead of waiting for the request that fetches the
  custom storage plugin metadata to receive a response, *append* custom
  storage plugins to the "Add" droptown menu once the request succeeds.

  Built-in storage types therefore do not depend on this request now,
  and can still be edited right away, and also if the request fails for
  some reason.

  Editing a custom storage type before the request finishes opens the
  editor once it does.

- pve-manager: Handle the `none` content type in the UI as well, in the
  rare case where a custom plugin might actually use it.

- pve-manager: Ensure that defaults of checkboxes do not end up on the
  config if left unchanged.

- pve-storage-plugin-examples: Update to APIVER 16 instead of 15.

Previous Versions
-----------------

rfc-v1: https://lore.proxmox.com/pve-devel/20250908180058.530119-1-m.carrara@proxmox.com/
rfc-v2: https://lore.proxmox.com/pve-devel/20251121165858.818307-1-m.carrara@proxmox.com/
v1: https://lore.proxmox.com/pve-devel/20260623143402.772452-1-m.carrara@proxmox.com/
v2: https://lore.proxmox.com/pve-devel/20260717154943.696411-1-m.carrara@proxmox.com/
v3: https://lore.proxmox.com/pve-devel/20260727143445.513900-1-m.carrara@proxmox.com/

Summary of Changes
------------------


pve-common:

Max R. Carrara (1):
  json schema: add multiline string format

 src/PVE/JSONSchema.pm | 5 +++++
 1 file changed, 5 insertions(+)


pve-storage:

Max R. Carrara (8):
  api: plugins/storage: add initial routes and endpoints
  api: plugins/storage/plugin: include schema in plugin metadata
  api: plugins/storage/plugin: mark sensitive properties in schema
  api: plugins/storage/plugin: factor plugin metadata code into helper
  api: plugins/storage/plugin: add plugins' 'content' to their metadata
  all plugins: add 'title' to properties, adapt 'description's
  all plugins, api: mark certain properties as advanced in plugindata()
  all plugins, api: mark certain properties as hidden in plugindata()

 src/PVE/API2/Makefile                  |   1 +
 src/PVE/API2/Plugins/Makefile          |  18 +++
 src/PVE/API2/Plugins/Storage.pm        |  52 +++++++
 src/PVE/API2/Plugins/Storage/Makefile  |  17 +++
 src/PVE/API2/Plugins/Storage/Plugin.pm | 188 +++++++++++++++++++++++++
 src/PVE/Storage/BTRFSPlugin.pm         |   9 ++
 src/PVE/Storage/CIFSPlugin.pm          |  21 ++-
 src/PVE/Storage/CephFSPlugin.pm        |  14 +-
 src/PVE/Storage/DirPlugin.pm           |  22 ++-
 src/PVE/Storage/ESXiPlugin.pm          |   3 +
 src/PVE/Storage/ISCSIDirectPlugin.pm   |   4 +
 src/PVE/Storage/ISCSIPlugin.pm         |  10 +-
 src/PVE/Storage/LVMPlugin.pm           |  17 ++-
 src/PVE/Storage/LvmThinPlugin.pm       |   7 +-
 src/PVE/Storage/NFSPlugin.pm           |  17 ++-
 src/PVE/Storage/PBSPlugin.pm           |   7 +-
 src/PVE/Storage/Plugin.pm              |  46 ++++--
 src/PVE/Storage/RBDPlugin.pm           |  20 ++-
 src/PVE/Storage/ZFSPlugin.pm           |  20 ++-
 src/PVE/Storage/ZFSPoolPlugin.pm       |  13 +-
 20 files changed, 469 insertions(+), 37 deletions(-)
 create mode 100644 src/PVE/API2/Plugins/Makefile
 create mode 100644 src/PVE/API2/Plugins/Storage.pm
 create mode 100644 src/PVE/API2/Plugins/Storage/Makefile
 create mode 100644 src/PVE/API2/Plugins/Storage/Plugin.pm


proxmox-widget-toolkit:

Max R. Carrara (3):
  form: introduce new 'proxmoxtextarea' field
  utils: introduce helper function getFieldDefFromPropertySchema
  acme: use helper to construct ExtJS fields from property schemas

 src/Utils.js                 | 202 +++++++++++++++++++++++++++++++++++
 src/form/TextAreaField.js    |  47 ++++++++
 src/window/ACMEPluginEdit.js |  47 ++++----
 3 files changed, 267 insertions(+), 29 deletions(-)


pve-manager:

Max R. Carrara (4):
  api: sort modules and API method registrations
  api: add API routes 'plugins' and 'plugins/storage'
  ui: storage view: display error when no editor for storage type exists
  ui: storage: add basic UI integration for custom storage plugins

 PVE/API2.pm                        |  26 +--
 PVE/API2/Makefile                  |   1 +
 PVE/API2/Plugins.pm                |  61 +++++++
 www/manager6/Makefile              |   1 +
 www/manager6/dc/StorageView.js     | 102 +++++++++++-
 www/manager6/storage/Base.js       |   7 +-
 www/manager6/storage/CustomEdit.js | 254 +++++++++++++++++++++++++++++
 7 files changed, 433 insertions(+), 19 deletions(-)
 create mode 100644 PVE/API2/Plugins.pm
 create mode 100644 www/manager6/storage/CustomEdit.js


pve-storage-plugin-examples:

Max R. Carrara (1):
  sshfs: Update SSHFS plugin

 .../src/PVE/Storage/Custom/SSHFSPlugin.pm     | 26 ++++++++++++++-----
 1 file changed, 20 insertions(+), 6 deletions(-)


Summary over all repositories:
  32 files changed, 1194 insertions(+), 91 deletions(-)

-- 
Generated by murpp 0.12.0