public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: "Max R. Carrara" <m.carrara@proxmox.com>
To: pve-devel@lists.proxmox.com
Cc: Alfonso Kuen <gerencia@idkmanager.com>,
	Thomas Ellmenreich <t.ellmenreich@proxmox.com>,
	Jakob Klocker <j.klocker@proxmox.com>
Subject: [PATCH pve-storage v5 03/19] api: plugins/storage/plugin: include schema in plugin metadata
Date: Fri, 13 Nov 2026 21:33:07 +0100	[thread overview]
Message-ID: <20261113203330.932813-4-m.carrara@proxmox.com> (raw)
In-Reply-To: <20261113203330.932813-1-m.carrara@proxmox.com>

Return a schema that describes a given plugin as part of its metadata.

This schema is simply a hash consisting of a given plugin's
properties' schemas. Each property schema is obtained by calling the
`get_property_schema()` method of `PVE::SectionConfig` (which
`PVE::Storage::Plugin` inherits) for every property that a given
plugin uses. This method also handles properties from isolated
plugins, so no additional handling is required here.

To determine which properties a plugin uses, use the plugin's
*initialized* options (from its private data), *not* the `options()`
method. The initialized options also include properties that have been
implicitly / automatically marked as required by the SectionConfig
machinery.

(For context, if a property is defined in `properties()` but not in
`options()`, it is added to the plugin's initialized options
automatically and marked as required.)

Whether a property is optional or not is also taken from the plugin's
initialized options, which is also what is enforced when the config is
checked. The property schemas themselves cannot be used for that, as
the global property list marks every plugin property as optional.

The globally available properties are also taken into account here.

The schemas are all memoized as well.

In other words, this means that each returned schema is completely
direved from a given plugin's section config. At the same time, this
should be adaptable enough to include extra hints, such as whether a
property is sensitive or not, for example.

Deriving a schema for each plugin like this is preferable over
exposing `createSchema()` and `updateSchema()` of
`PVE::Storage::Plugin` directly. In particular, these two schemas come
with some drawbacks when it comes to describing an *individual*
plugin's data:

- Neither schema contains any information on which properties are used
  by which plugins.

- It is not possible to determine whether a property is actually
  optional (or not) for a given plugin. If one plugin declares a
  property as optional somewhere, the property will be optional in the
  schema in most cases, despite being non-optional for other plugins.
  (This is generally true; skipping over a bunch of SectionConfig
  implementation details here for the sake of brevity).

- The same is true even more so for fixed properties. While the
  `updateSchema()` will oftentimes *not* contain fixed properties,
  meaning that you can compare it with `createSchema()` and figure out
  which properties *may* be fixed, this only holds if the property is
  fixed in *every* plugin's `options()`. As soon as a plugin declares
  a usually fixed property as non-fixed, the property is included in
  the `updateSchema()`.

- Even when switching to property isolation for
  `PVE::Storage::Plugin`, which properties are fixed / optional for
  individual plugins is still not consistently determinable for the
  previous two reasons.

This means that in addition to exposing `createSchema()` and
`updateSchema()`, we would also have to expose the `options()` of each
plugin, and then stitch all of that information together again just to
obtain the hash that the `get_schema_for_plugin()` helper being added
here returns.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/API2/Plugins/Storage/Plugin.pm | 72 +++++++++++++++++++++++++-
 1 file changed, 71 insertions(+), 1 deletion(-)

diff --git a/src/PVE/API2/Plugins/Storage/Plugin.pm b/src/PVE/API2/Plugins/Storage/Plugin.pm
index 4e98b6b5..7d697d06 100644
--- a/src/PVE/API2/Plugins/Storage/Plugin.pm
+++ b/src/PVE/API2/Plugins/Storage/Plugin.pm
@@ -29,19 +29,75 @@ my $PLUGIN_METADATA_SCHEMA = {
                 . " installed separately ('custom').",
             type => 'string',
             enum => $PLUGIN_ORIGINS,
             optional => 0,
         },
+        schema => {
+            type => 'object',
+            optional => 0,
+        },
         type => {
             description => "The storage type of the plugin.",
             type => 'string',
             optional => 0,
             enum => $ALL_PLUGIN_TYPES,
         },
     },
 };
 
+my $PLUGIN_SCHEMAS = {};
+
+my sub get_schema_for_plugin : prototype($) ($plugin) {
+    my $type = $plugin->type();
+
+    return $PLUGIN_SCHEMAS->{$type} if defined($PLUGIN_SCHEMAS->{$type});
+
+    # Derive options from the initialized data instead of using `options()`,
+    # since `init_isolated()` automatically adds properties to options that
+    # were not specified explicitly
+    my $pdata = $plugin->private();
+    my $options = $pdata->{options}->{$type};
+
+    my $schema = {};
+
+    for my $option (keys $options->%*) {
+        my $prop_schema = PVE::RESTHandler::api_dump_remove_refs(
+            PVE::Storage::Plugin->get_property_schema($type, $option));
+
+        # Deep clone to prevent any accidental Perl-esque side effects, because
+        # we are memoizing each schema through $PLUGIN_SCHEMAS
+        my $property = dclone($prop_schema);
+        $schema->{$option} = $property;
+
+        for my $opt_key (keys $options->{$option}->%*) {
+            $property->{$opt_key} = $options->{$option}->{$opt_key};
+        }
+
+        # NOTE: The global property list in SectionConfig marks every plugin
+        # property as optional. This means that we cannot rely on the
+        # `optional` key that the `get_property_schema` above returns for
+        # validation in the front end; it would mark required values as
+        # optional.
+        #
+        # Note that we also cannot rely on `createSchema()` / `updateSchema()`
+        # here for the same reason.
+        #
+        # Therefore, we override this key below with what the finalized set of
+        # options (from the private data) contain directly. This makes it so that the
+        # front end validation matches the back end's.
+        #
+        # See the docstring for PVE::SectionConfig::properties() for more
+        # information.
+        my $is_optional = $options->{$option}->{optional};
+        $property->{optional} = $is_optional ? 1 : 0;
+    }
+
+    $PLUGIN_SCHEMAS->{$type} = $schema;
+
+    return $schema;
+}
+
 # plugins/storage/plugin
 
 __PACKAGE__->register_method({
     name => 'index',
     path => '',
@@ -70,17 +126,25 @@ __PACKAGE__->register_method({
         my $origin = extract_param($param, 'origin');
 
         my $result = [];
 
         for my $type ($ALL_PLUGIN_TYPES->@*) {
+            my $plugin = PVE::Storage::Plugin->lookup($type);
+
             if (defined($origin)) {
-                my $plugin = PVE::Storage::Plugin->lookup($type);
                 next if plugin_origin($plugin) ne $origin;
             }
 
+            my $schema = eval { get_schema_for_plugin($plugin) };
+            if (my $err = $@) {
+                warn "skipping storage plugin '$type' - cannot build its schema: $err\n";
+                next;
+            }
+
             my $item = {
                 origin => $origin,
+                schema => $schema,
                 type => $type,
             };
 
             push($result->@*, $item);
         }
@@ -114,12 +178,18 @@ __PACKAGE__->register_method({
         my $param_type = extract_param($param, 'type');
 
         # Does not fail since `type` is validated first
         my $plugin = PVE::Storage::Plugin->lookup($param_type);
 
+        my $schema = eval { get_schema_for_plugin($plugin) };
+        if (my $err = $@) {
+            raise("Cannot build metadata of storage plugin '$param_type' - $err");
+        }
+
         my $result = {
             origin => plugin_origin($plugin),
+            schema => $schema,
             type => $param_type,
         };
 
         return $result;
     },
-- 
2.47.3





  parent reply	other threads:[~2026-11-13 20:34 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-13 20:33 [PATCH common/manager/proxmox-widget-toolkit/storage/storage-plugin-examples v5 00/19] GUI Support for Custom Storage Plugins Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-common v5 01/19] json schema: add multiline validation keyword Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-storage v5 02/19] api: plugins/storage: add initial routes and endpoints Max R. Carrara
2026-11-13 20:33 ` Max R. Carrara [this message]
2026-11-13 20:33 ` [PATCH pve-storage v5 04/19] api: plugins/storage/plugin: handle sensitive properties in schema Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-storage v5 05/19] api: plugins/storage/plugin: factor plugin metadata code into helper Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-storage v5 06/19] api: plugins/storage/plugin: add plugins' 'content' to their metadata Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-storage v5 07/19] all plugins: add 'title' to properties, adapt 'description's Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-storage v5 08/19] all plugins, api: mark certain properties as advanced in plugindata() Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-storage v5 09/19] all plugins, api: mark certain properties as hidden " Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-storage v5 10/19] api: plugins/storage/plugin: allow plugins to have display names Max R. Carrara
2026-11-13 20:33 ` [PATCH proxmox-widget-toolkit v5 11/19] form: introduce new 'proxmoxtextarea' field Max R. Carrara
2026-11-13 20:33 ` [PATCH proxmox-widget-toolkit v5 12/19] utils: introduce helper function getFieldDefFromPropertySchema Max R. Carrara
2026-11-13 20:33 ` [PATCH proxmox-widget-toolkit v5 13/19] acme: use helper to construct ExtJS fields from property schemas Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-manager v5 14/19] api: sort modules and API method registrations Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-manager v5 15/19] ui: storage: remove entry of the removed DRBD plugin Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-manager v5 16/19] api: add API routes 'plugins' and 'plugins/storage' Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-manager v5 17/19] ui: storage view: display error when no editor for storage type exists Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-manager v5 18/19] ui: storage: add basic UI integration for custom storage plugins Max R. Carrara
2026-11-13 20:33 ` [PATCH pve-storage-plugin-examples v5 19/19] sshfs: Update SSHFS plugin Max R. Carrara
2026-11-16  2:20 ` applied: [PATCH common/manager/proxmox-widget-toolkit/storage/storage-plugin-examples v5 00/19] GUI Support for Custom Storage Plugins Thomas Lamprecht
2026-11-17  8:07 ` Roland Kammerer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261113203330.932813-4-m.carrara@proxmox.com \
    --to=m.carrara@proxmox.com \
    --cc=gerencia@idkmanager.com \
    --cc=j.klocker@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    --cc=t.ellmenreich@proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal