[pve-devel] [RFC pve-storage, pve-manager master v1 00/12] GUI Support for Custom Storage Plugins

["Max R. Carrara" <m.carrara@proxmox.com>]

GUI Support for Custom Storage Plugins
======================================

tl;dr:

Add an API method to PVE::Storage::Plugin that returns the definition
for the form view of custom storage plugins. This definition is used by
the frontend to build the form view for creating / editing the storage
config entry of the plugin. The ultimate goal here is that custom
storage plugin devs don't have to (and also *must not*) ever touch
JavaScript to make their plugins show up in the GUI.

Overview
--------

This RFC implements GUI support for custom storage plugins.

To achieve this, four new paths are added to the API:

  - plugins/storage

      Returns the metadata of all plugins.

      Plugin metadata includes supported content types and formats (as well
      as their defaults), the plugin's short name, the views a plugin
      defines, and more.

  - plugins/storage/{plugin}

      Returns the metadata of a single plugin.

  - plugins/storage/{plugin}/views

      Returns a list of all view declarations of a plugin. If the plugin
      defines no views, the list is empty.

  - plugins/storage/{plugin}/views/form

      Returns the form view definition of a plugin.

How Custom Views Work
---------------------

A view is what defines how data should be displayed to users. Views are
specified via a nested hash in Perl and converted to JSON via the API.
The view definition inside the JSON object is then taken by the frontend
and built into a view for the user.

In particular, this RFC adds support for custom form views for storage
plugin config entries (Datacenter > Storage). A plugin may define a form
view by implementing the new `get_form_view()` plugin API method and
specifying in `plugindata()` that it has declared such a view.

Additionally, the JSON schema for form views is versioned to make
forward- and backward-compat easier.

The form view currently only allows customization of the "General" tab.
However, if a config entry has the "backup" content type selected, the
"Backup Retention" tab becomes unmasked, just like with inbuilt plugins.

The JSON schema for the form view mainly consists of the columns of the
"General" tab and the fields those columns may include.

The supported columns reflect those that are currently used in our
frontend:
  - A list of "regular" columns
  - A "bottom" column (the wide column below the regular ones)
  - Columns in the advanced subsection
  - A "bottom" column in the advanced subsection

Every column may contain a list of fields. Fields are typed and
correspond to a SectionConfig property of the storage plugin returning
the view. This means that defining the form view is enough, no
additional API methods need to be implemented otherwise. Ext.JS will
then use the property names in the field definitions for the regular
storage API calls.

This means that custom storage plugin authors don't have to write a
single line of JavaScript when implementing GUI support for their
plugin.

The currently supported field types in the form view schema are:
  - boolean
  - integer
  - number
  - string
  - selection

Fields have common attributes as well as attributes specific to its
particular type. For example, the 'string' field may have the additional
'display-mode' attribute, with which the field may be displayed as
regular text field (the default), as text area, or as a password field.

How these field definitions are interpreted depends on the frontend.
In this RFC, the properties of the corresponding Ext.JS field are
determined and stitched together dynamically.

The four fields for the storage ID, nodes, content types and enable /
disable checkbox are always added by default and cannot be declared in
the form view.

Example Implementation
----------------------

To show the custom form view in action, the whole thing described here
is implemented for the ZFS pool plugin. You should notice only minor
differences from the original form view.

Current Limitations
-------------------

- The "default text" is currently not set. Didn't want to give in to the
  ever-lingering feature creep surrounding this RFC.
  - The same probably goes for other minor particularities that Ext.JS
    supports. If the reader has any additional ideas, please send them
    my way.

- There is no support for cluster setups yet.
  - This is only *really* an issue for node-local storages. The example
    implementation for the ZFS pool storage in this RFC works for single
    node setups, but there's no node selector or anything of the sort
    for custom selections. Would highly appreciate any ideas in that
    regard, as we might have to deviate from the "standard look" that
    our storage config forms currently have when it comes to that.

- For some reason the checkbox for the advanced section doesn't show up
  even if fields exist inside its columns.
  - The fields still show up as expected, it's just that they can't be
    hidden with the "Advanced [ ]" checkbox.
  - No idea why that happens. Would appreciate any Ext.JS lore / help in
    that regard.

- Docstrings for the new stuff will be added once this becomes a proper
  series.

Further Ideas
-------------

- While this only aims to implement GUI support for custom storage
  plugins, there's nothing that's really stopping us from using this for
  our own plugins once all the rough edges have been smoothed out.
  - The only thing that might hinder us from *fully* switching over to
    declaring our inbuilt plugins' form views according to this series
    is the fact that some plugins define custom dialogues and such.
  - E.g. the PBS plugin has a custom "Encryption" tab with whole
    dialogues for auto-generating / uploading encryption keys.

- As of right now, the already existing field types in Ext.JS
  (meaning 'xtypes' here) are used.
  - What we could do is add custom field types in Ext.JS that correspond
    to the five types that the form view schema allows, in order to
    provide a more uniform way of building the fields and columns in
    Ext.JS. Right now the Ext.JS fields are just made up on the spot,
    which is a bit convoluted.
  - Not sure if this is strictly necessary though, but might be nice to
    have.

- This whole concept in the RFC can theoretically be generalized so that
  it may be used throughout other places in PVE as well.
  - The JSON schemas for columns and fields in particular technically
    aren't really limited to storage plugin stuff.
  - It might therefore be beneficial overall to pull the smaller pieces
    out and define them in a separate module (debian package) so that
    the rest of the backend can also benefit from this.
  - I don't know of any other use cases as of right now though, which
    is why I confined the schemas to PVE::Storage::Plugin::Views at the
    moment.
  - If we do want to generalize field / column / row / etc. schemas 
    eventually, we can just add a new schema version for the storage
    plugin form view that uses the altered schemas when that happens.
    :^)

Closing Thoughts
----------------

If you read this far, thanks a lot for your attention. 🙏 I hope I
haven't missed anything. I'd appreciate any feedback.

Also, thanks a lot to Aaron L. for brainstorming this through with me in
the beginning!

Summary of Changes
------------------

pve-storage:

Max R. Carrara (8):
  plugin: meta: add package PVE::Storage::Plugin::Meta
  api: Add 'plugins/storage' and 'plugins/storage/{plugin}' paths
  plugin: meta: introduce 'short-name'
  plugin: views: add package PVE::Storage::Plugin::Views
  plugin: add new plugin API method `get_form_view()`
  plugin: meta: add metadata regarding views in API
  api: views: add paths regarding storage plugin views
  plugin: zfspool: add 'short-name' and form view for ZFS pool plugin

 src/PVE/API2/Makefile                  |   1 +
 src/PVE/API2/Plugins/Makefile          |  18 ++
 src/PVE/API2/Plugins/Storage/Config.pm | 188 +++++++++++++++++++
 src/PVE/API2/Plugins/Storage/Makefile  |  18 ++
 src/PVE/API2/Plugins/Storage/Views.pm  | 172 ++++++++++++++++++
 src/PVE/Storage/Makefile               |   1 +
 src/PVE/Storage/Plugin.pm              |   8 +
 src/PVE/Storage/Plugin/Makefile        |  11 ++
 src/PVE/Storage/Plugin/Meta.pm         | 211 +++++++++++++++++++++
 src/PVE/Storage/Plugin/Views.pm        | 242 +++++++++++++++++++++++++
 src/PVE/Storage/ZFSPoolPlugin.pm       |  67 +++++++
 11 files changed, 937 insertions(+)
 create mode 100644 src/PVE/API2/Plugins/Makefile
 create mode 100644 src/PVE/API2/Plugins/Storage/Config.pm
 create mode 100644 src/PVE/API2/Plugins/Storage/Makefile
 create mode 100644 src/PVE/API2/Plugins/Storage/Views.pm
 create mode 100644 src/PVE/Storage/Plugin/Makefile
 create mode 100644 src/PVE/Storage/Plugin/Meta.pm
 create mode 100644 src/PVE/Storage/Plugin/Views.pm

pve-manager:

Max R. Carrara (4):
  api: handle path 'plugins/storage' through its package
  ui: storage: add CustomBase.js
  ui: storage: support custom storage plugins in Datacenter > Storage
  ui: storage: use `Ext.Msg.alert()` instead of throwing an exception

 PVE/API2.pm                        |   6 +
 www/manager6/Makefile              |   1 +
 www/manager6/dc/StorageView.js     | 137 ++++++++--
 www/manager6/storage/CustomBase.js | 402 +++++++++++++++++++++++++++++
 4 files changed, 524 insertions(+), 22 deletions(-)
 create mode 100644 www/manager6/storage/CustomBase.js

-- 
2.47.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel