From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id C23651FF0B7 for ; Fri, 13 Nov 2026 21:34:10 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 959BE2163F; Fri, 13 Nov 2026 21:33:56 +0100 (CET) From: "Max R. Carrara" To: pve-devel@lists.proxmox.com Subject: [PATCH pve-storage v5 03/19] api: plugins/storage/plugin: include schema in plugin metadata Date: Fri, 13 Nov 2026 21:33:07 +0100 Message-ID: <20261113203330.932813-4-m.carrara@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261113203330.932813-1-m.carrara@proxmox.com> References: <20261113203330.932813-1-m.carrara@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1794602016771 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.298 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: PLBGGOREVO2WOTSTNS73KVFHEL3Q57XZ X-Message-ID-Hash: PLBGGOREVO2WOTSTNS73KVFHEL3Q57XZ X-MailFrom: m.carrara@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Alfonso Kuen , Thomas Ellmenreich , Jakob Klocker X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Return a schema that describes a given plugin as part of its metadata. This schema is simply a hash consisting of a given plugin's properties' schemas. Each property schema is obtained by calling the `get_property_schema()` method of `PVE::SectionConfig` (which `PVE::Storage::Plugin` inherits) for every property that a given plugin uses. This method also handles properties from isolated plugins, so no additional handling is required here. To determine which properties a plugin uses, use the plugin's *initialized* options (from its private data), *not* the `options()` method. The initialized options also include properties that have been implicitly / automatically marked as required by the SectionConfig machinery. (For context, if a property is defined in `properties()` but not in `options()`, it is added to the plugin's initialized options automatically and marked as required.) Whether a property is optional or not is also taken from the plugin's initialized options, which is also what is enforced when the config is checked. The property schemas themselves cannot be used for that, as the global property list marks every plugin property as optional. The globally available properties are also taken into account here. The schemas are all memoized as well. In other words, this means that each returned schema is completely direved from a given plugin's section config. At the same time, this should be adaptable enough to include extra hints, such as whether a property is sensitive or not, for example. Deriving a schema for each plugin like this is preferable over exposing `createSchema()` and `updateSchema()` of `PVE::Storage::Plugin` directly. In particular, these two schemas come with some drawbacks when it comes to describing an *individual* plugin's data: - Neither schema contains any information on which properties are used by which plugins. - It is not possible to determine whether a property is actually optional (or not) for a given plugin. If one plugin declares a property as optional somewhere, the property will be optional in the schema in most cases, despite being non-optional for other plugins. (This is generally true; skipping over a bunch of SectionConfig implementation details here for the sake of brevity). - The same is true even more so for fixed properties. While the `updateSchema()` will oftentimes *not* contain fixed properties, meaning that you can compare it with `createSchema()` and figure out which properties *may* be fixed, this only holds if the property is fixed in *every* plugin's `options()`. As soon as a plugin declares a usually fixed property as non-fixed, the property is included in the `updateSchema()`. - Even when switching to property isolation for `PVE::Storage::Plugin`, which properties are fixed / optional for individual plugins is still not consistently determinable for the previous two reasons. This means that in addition to exposing `createSchema()` and `updateSchema()`, we would also have to expose the `options()` of each plugin, and then stitch all of that information together again just to obtain the hash that the `get_schema_for_plugin()` helper being added here returns. Signed-off-by: Max R. Carrara --- src/PVE/API2/Plugins/Storage/Plugin.pm | 72 +++++++++++++++++++++++++- 1 file changed, 71 insertions(+), 1 deletion(-) diff --git a/src/PVE/API2/Plugins/Storage/Plugin.pm b/src/PVE/API2/Plugins/Storage/Plugin.pm index 4e98b6b5..7d697d06 100644 --- a/src/PVE/API2/Plugins/Storage/Plugin.pm +++ b/src/PVE/API2/Plugins/Storage/Plugin.pm @@ -29,19 +29,75 @@ my $PLUGIN_METADATA_SCHEMA = { . " installed separately ('custom').", type => 'string', enum => $PLUGIN_ORIGINS, optional => 0, }, + schema => { + type => 'object', + optional => 0, + }, type => { description => "The storage type of the plugin.", type => 'string', optional => 0, enum => $ALL_PLUGIN_TYPES, }, }, }; +my $PLUGIN_SCHEMAS = {}; + +my sub get_schema_for_plugin : prototype($) ($plugin) { + my $type = $plugin->type(); + + return $PLUGIN_SCHEMAS->{$type} if defined($PLUGIN_SCHEMAS->{$type}); + + # Derive options from the initialized data instead of using `options()`, + # since `init_isolated()` automatically adds properties to options that + # were not specified explicitly + my $pdata = $plugin->private(); + my $options = $pdata->{options}->{$type}; + + my $schema = {}; + + for my $option (keys $options->%*) { + my $prop_schema = PVE::RESTHandler::api_dump_remove_refs( + PVE::Storage::Plugin->get_property_schema($type, $option)); + + # Deep clone to prevent any accidental Perl-esque side effects, because + # we are memoizing each schema through $PLUGIN_SCHEMAS + my $property = dclone($prop_schema); + $schema->{$option} = $property; + + for my $opt_key (keys $options->{$option}->%*) { + $property->{$opt_key} = $options->{$option}->{$opt_key}; + } + + # NOTE: The global property list in SectionConfig marks every plugin + # property as optional. This means that we cannot rely on the + # `optional` key that the `get_property_schema` above returns for + # validation in the front end; it would mark required values as + # optional. + # + # Note that we also cannot rely on `createSchema()` / `updateSchema()` + # here for the same reason. + # + # Therefore, we override this key below with what the finalized set of + # options (from the private data) contain directly. This makes it so that the + # front end validation matches the back end's. + # + # See the docstring for PVE::SectionConfig::properties() for more + # information. + my $is_optional = $options->{$option}->{optional}; + $property->{optional} = $is_optional ? 1 : 0; + } + + $PLUGIN_SCHEMAS->{$type} = $schema; + + return $schema; +} + # plugins/storage/plugin __PACKAGE__->register_method({ name => 'index', path => '', @@ -70,17 +126,25 @@ __PACKAGE__->register_method({ my $origin = extract_param($param, 'origin'); my $result = []; for my $type ($ALL_PLUGIN_TYPES->@*) { + my $plugin = PVE::Storage::Plugin->lookup($type); + if (defined($origin)) { - my $plugin = PVE::Storage::Plugin->lookup($type); next if plugin_origin($plugin) ne $origin; } + my $schema = eval { get_schema_for_plugin($plugin) }; + if (my $err = $@) { + warn "skipping storage plugin '$type' - cannot build its schema: $err\n"; + next; + } + my $item = { origin => $origin, + schema => $schema, type => $type, }; push($result->@*, $item); } @@ -114,12 +178,18 @@ __PACKAGE__->register_method({ my $param_type = extract_param($param, 'type'); # Does not fail since `type` is validated first my $plugin = PVE::Storage::Plugin->lookup($param_type); + my $schema = eval { get_schema_for_plugin($plugin) }; + if (my $err = $@) { + raise("Cannot build metadata of storage plugin '$param_type' - $err"); + } + my $result = { origin => plugin_origin($plugin), + schema => $schema, type => $param_type, }; return $result; }, -- 2.47.3