[pve-devel] [PATCH v1 pve-storage 0/8] Base Module + Documentation for PVE::Storage::Plugin API

[pve-devel] [PATCH v1 pve-storage 0/8] Base Module + Documentation for PVE::Storage::Plugin API

[Max Carrara]

Abstract Base Module + Documentation for PVE::Storage::Plugin API - v1
======================================================================

This series adds a base module (interface) for PVE::Storage::Plugin,
named PVE::Storage::PluginBase, which lists all methods that are part of
the Storage Plugin API. Additionally, docstrings for every method are
added in order to provide context for (third-party) developers, as
sometimes a complete roundtrip through our code is otherwise necessary
in order to (really) understand what a method does.

Special thanks go to @Maximilano, who has helped me out a lot with this
series!

Essentially, the base module is similar to PVE::LXC::Plugin [1], with
the exception that docstrings are added.

Instead of documenting every single parameter separately, common /
recurring method parameters are listed and explained at the very top of
the file. The caching mechanism of a few methods is also described in
the top-level documentation.

Any feedback regarding the docs is highly appreciated -- I hope that we
haven't overlooked anything. We tried to keep things concise unless it's
absolutely necessary to add additional context.

Also, most (but not all) of the methods `croak` right now if they're
unimplemented, but I feel like default return values for certain methods
could be returned instead. Then again, ::Plugin already provides those
defaults, so I'm not sure if it matters either way.

References
----------

[1]: https://git.proxmox.com/?p=pve-container.git;a=blob;f=src/PVE/LXC/Setup/Plugin.pm;h=b9d9c2d9d19e086438faeda6cfce8f854dddc7ea;hb=refs/heads/master

Summary of Changes
------------------

Max Carrara (8):
  pluginbase: introduce PVE::Storage::PluginBase with doc scaffold
  pluginbase: add high-level plugin API description
  pluginbase: document SectionConfig methods
  pluginbase: document general plugin methods
  pluginbase: document hooks
  pluginbase: document image operation methods
  pluginbase: document volume operations
  pluginbase: document import and export methods

 src/PVE/Storage/Makefile      |    1 +
 src/PVE/Storage/Plugin.pm     |    2 +-
 src/PVE/Storage/PluginBase.pm | 1475 +++++++++++++++++++++++++++++++++
 3 files changed, 1477 insertions(+), 1 deletion(-)
 create mode 100644 src/PVE/Storage/PluginBase.pm

-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v1 pve-storage 1/8] pluginbase: introduce PVE::Storage::PluginBase with doc scaffold

[Max Carrara]

Add PVE::Storage::PluginBase, which defines stubs for all methods that
storage plugins should implement in order to conform to our plugin
API. This makes it much easier for (third-party) developers to see
which methods should be implemented.

PluginBase is inserted into the inheritance chain between
PVE::Storage::Plugin and PVE::SectionConfig instead of letting the
Plugin module inherit from SectionConfig directly. This keeps the
inheritance chain linear, avoiding multiple inheritance.

Also provide a scaffold for documentation. Preserve pre-existing
comments for context's sake.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Makefile      |   1 +
 src/PVE/Storage/Plugin.pm     |   2 +-
 src/PVE/Storage/PluginBase.pm | 328 ++++++++++++++++++++++++++++++++++
 3 files changed, 330 insertions(+), 1 deletion(-)
 create mode 100644 src/PVE/Storage/PluginBase.pm

diff --git a/src/PVE/Storage/Makefile b/src/PVE/Storage/Makefile
index ce3fd68..f2cdb66 100644
--- a/src/PVE/Storage/Makefile
+++ b/src/PVE/Storage/Makefile
@@ -1,6 +1,7 @@
 SOURCES= \
 	Common.pm \
 	Plugin.pm \
+	PluginBase.pm \
 	DirPlugin.pm \
 	LVMPlugin.pm \
 	NFSPlugin.pm \
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 65cf43f..df6882a 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -19,7 +19,7 @@ use PVE::Storage::Common;
 
 use JSON;
 
-use base qw(PVE::SectionConfig);
+use parent qw(PVE::Storage::PluginBase);
 
 use constant KNOWN_COMPRESSION_FORMATS =>  ('gz', 'lzo', 'zst', 'bz2');
 use constant COMPRESSOR_RE => join('|', KNOWN_COMPRESSION_FORMATS);
diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
new file mode 100644
index 0000000..e56aa72
--- /dev/null
+++ b/src/PVE/Storage/PluginBase.pm
@@ -0,0 +1,328 @@
+=head1 NAME
+
+C<PVE::Storage::PluginBase> - Storage Plugin API Interface
+
+=head1 DESCRIPTION
+
+=cut
+
+package PVE::Storage::PluginBase;
+
+use strict;
+use warnings;
+
+use Carp qw(croak);
+
+use parent qw(PVE::SectionConfig);
+
+=head1 PLUGIN INTERFACE METHODS
+
+=cut
+
+=head2 PLUGIN DEFINITION
+
+=cut
+
+sub type {
+    croak "implement me in sub-class\n";
+}
+
+sub properties {
+    my ($class) = @_;
+    return $class->SUPER::properties();
+}
+
+sub options {
+    my ($class) = @_;
+    return $class->SUPER::options();
+}
+
+sub plugindata {
+    my ($class) = @_;
+    return $class->SUPER::plugindata();
+}
+
+sub private {
+    croak "implement me in sub-class\n";
+}
+
+=head2 GENERAL
+
+=cut
+
+sub check_connection {
+    my ($class, $storeid, $scfg) = @_;
+
+    return 1;
+}
+
+sub activate_storage {
+    my ($class, $storeid, $scfg, $cache) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub deactivate_storage {
+    my ($class, $storeid, $scfg, $cache) = @_;
+
+    return;
+}
+
+sub status {
+    my ($class, $storeid, $scfg, $cache) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub cluster_lock_storage {
+    my ($class, $storeid, $shared, $timeout, $func, @param) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub parse_volname {
+    my ($class, $volname) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub get_subdir {
+    my ($class, $scfg, $vtype) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub filesystem_path {
+    my ($class, $scfg, $volname, $snapname) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub path {
+    my ($class, $scfg, $volname, $storeid, $snapname) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub find_free_diskname {
+    my ($class, $storeid, $scfg, $vmid, $fmt, $add_fmt_suffix) = @_;
+    croak "implement me in sub-class\n";
+}
+
+=head2 HOOKS
+
+=cut
+
+# called during addition of storage (before the new storage config got written)
+# die to abort addition if there are (grave) problems
+# NOTE: runs in a storage config *locked* context
+sub on_add_hook {
+    my ($class, $storeid, $scfg, %param) = @_;
+    return undef;
+}
+
+# called during storage configuration update (before the updated storage config got written)
+# die to abort the update if there are (grave) problems
+# NOTE: runs in a storage config *locked* context
+sub on_update_hook {
+    my ($class, $storeid, $scfg, %param) = @_;
+    return undef;
+}
+
+# called during deletion of storage (before the new storage config got written)
+# and if the activate check on addition fails, to cleanup all storage traces
+# which on_add_hook may have created.
+# die to abort deletion if there are (very grave) problems
+# NOTE: runs in a storage config *locked* context
+sub on_delete_hook {
+    my ($class, $storeid, $scfg) = @_;
+    return undef;
+}
+
+=head2 IMAGE OPERATIONS
+
+=cut
+
+sub list_images {
+    my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub create_base {
+    my ($class, $storeid, $scfg, $volname) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub clone_image {
+    my ($class, $scfg, $storeid, $volname, $vmid, $snap) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub alloc_image {
+    my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub free_image {
+    my ($class, $storeid, $scfg, $volname, $isBase, $format) = @_;
+    croak "implement me in sub-class\n";
+}
+
+=head2 VOLUME OPERATIONS
+
+=cut
+
+sub list_volumes {
+    my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
+    croak "implement me in sub-class\n";
+}
+
+# Returns undef if the attribute is not supported for the volume.
+# Should die if there is an error fetching the attribute.
+# Possible attributes:
+# notes     - user-provided comments/notes.
+# protected - not to be removed by free_image, and for backups, ignored when pruning.
+sub get_volume_attribute {
+    my ($class, $scfg, $storeid, $volname, $attribute) = @_;
+    croak "implement me in sub-class\n";
+}
+
+# Dies if the attribute is not supported for the volume.
+sub update_volume_attribute {
+    my ($class, $scfg, $storeid, $volname, $attribute, $value) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub volume_size_info {
+    my ($class, $scfg, $storeid, $volname, $timeout) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub volume_resize {
+    my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub volume_snapshot {
+    my ($class, $scfg, $storeid, $volname, $snap) = @_;
+    croak "implement me in sub-class\n";
+}
+
+# Returns a hash with the snapshot names as keys and the following data:
+# id        - Unique id to distinguish different snapshots even if the have the same name.
+# timestamp - Creation time of the snapshot (seconds since epoch).
+# Returns an empty hash if the volume does not exist.
+sub volume_snapshot_info {
+    my ($class, $scfg, $storeid, $volname) = @_;
+    croak "implement me in sub-class\n";
+}
+
+# Asserts that a rollback to $snap on $volname is possible.
+# If certain snapshots are preventing the rollback and $blockers is an array
+# reference, the snapshot names can be pushed onto $blockers prior to dying.
+sub volume_rollback_is_possible {
+    my ($class, $scfg, $storeid, $volname, $snap, $blockers) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub volume_snapshot_rollback {
+    my ($class, $scfg, $storeid, $volname, $snap) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub volume_snapshot_delete {
+    my ($class, $scfg, $storeid, $volname, $snap, $running) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub volume_snapshot_needs_fsfreeze {
+    croak "implement me in sub-class\n";
+}
+
+sub storage_can_replicate {
+    my ($class, $scfg, $storeid, $format) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub volume_has_feature {
+    my ($class, $scfg, $feature, $storeid, $volname, $snapname, $running, $opts) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub map_volume {
+    my ($class, $storeid, $scfg, $volname, $snapname) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub unmap_volume {
+    my ($class, $storeid, $scfg, $volname, $snapname) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub activate_volume {
+    my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub deactivate_volume {
+    my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub rename_volume {
+    my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub prune_backups {
+    my ($class, $scfg, $storeid, $keep, $vmid, $type, $dryrun, $logfunc) = @_;
+    croak "implement me in sub-class\n";
+}
+
+=head2 IMPORTS AND EXPORTS
+
+=cut
+
+# Import/Export interface:
+#   Any path based storage is assumed to support 'raw' and 'tar' streams, so
+#   the default implementations will return this if $scfg->{path} is set,
+#   mimicking the old PVE::Storage::storage_migrate() function.
+#
+# Plugins may fall back to PVE::Storage::Plugin::volume_{export,import}...
+#   functions in case the format doesn't match their specialized
+#   implementations to reuse the raw/tar code.
+#
+# Format specification:
+#   The following formats are all prefixed with image information in the form
+#   of a 64 bit little endian unsigned integer (pack('Q<')) in order to be able
+#   to preallocate the image on storages which require it.
+#
+#   raw+size: (image files only)
+#     A raw binary data stream such as produced via `dd if=TheImageFile`.
+#   qcow2+size, vmdk: (image files only)
+#     A raw qcow2/vmdk/... file such as produced via `dd if=some.qcow2` for
+#     files which are already in qcow2 format, or via `qemu-img convert`.
+#     Note that these formats are only valid with $with_snapshots being true.
+#   tar+size: (subvolumes only)
+#     A GNU tar stream containing just the inner contents of the subvolume.
+#     This does not distinguish between the contents of a privileged or
+#     unprivileged container. In other words, this is from the root user
+#     namespace's point of view with no uid-mapping in effect.
+#     As produced via `tar -C vm-100-disk-1.subvol -cpf TheOutputFile.dat .`
+
+# Export a volume into a file handle as a stream of desired format.
+sub volume_export {
+    my ($class, $scfg, $storeid, $fh, $volname, $format, $snapshot, $base_snapshot, $with_snapshots) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub volume_export_formats {
+    my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
+    croak "implement me in sub-class\n";
+}
+
+# Import data from a stream, creating a new or replacing or adding to an existing volume.
+sub volume_import {
+    my ($class, $scfg, $storeid, $fh, $volname, $format, $snapshot, $base_snapshot, $with_snapshots, $allow_rename) = @_;
+    croak "implement me in sub-class\n";
+}
+
+sub volume_import_formats {
+    my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
+    croak "implement me in sub-class\n";
+}
+
+1;
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v1 pve-storage 2/8] pluginbase: add high-level plugin API description

[Max Carrara]

Add a short paragraph in DESCRIPTION serving as an introduction as
well as the GENERAL PARAMETERS and CACHING EXPENSIVE OPERATIONS
sections.

These sections are added in order to avoid repeatedly describing the
same parameters as well as to elaborate on / clarify a couple terms,
e.g. what the $cache parameter does or what a volume in our case is.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/PluginBase.pm | 77 +++++++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)

diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
index e56aa72..16977f3 100644
--- a/src/PVE/Storage/PluginBase.pm
+++ b/src/PVE/Storage/PluginBase.pm
@@ -4,6 +4,83 @@ C<PVE::Storage::PluginBase> - Storage Plugin API Interface
 
 =head1 DESCRIPTION
 
+This module documents the public Storage Plugin API of PVE and serves
+as a base for C<L<PVE::Storage::Plugin>>. Plugins must B<always> inherit from
+C<L<PVE::Storage::Plugin>>, as this module is for documentation purposes
+only.
+
+=head2 DEFAULT IMPLEMENTATIONS
+
+C<L<PVE::Storage::Plugin>> implements most of the methods listed in
+L</PLUGIN INTERFACE METHODS> by default. These provided implementations are
+tailored towards file-based storages and can therefore be used as-is in that
+case. Plugins for other kinds of storages will most likely have to adapt each
+method for their individual use cases.
+
+=head2 GENERAL PARAMETERS
+
+The parameter naming throughout the code is kept as consistent as possible.
+Therefore, common reappearing subroutine parameters are listed here for
+convenience:
+
+=over
+
+=item * C<< \%scfg >>
+
+The storage configuration associated with the given C<$storeid>. This is a
+reference to a hash that represents the section associated with C<$storeid> in
+C</etc/pve/storage.cfg>.
+
+=item * C<< $storeid >>
+
+The ID of the storage. This ID is user-provided; the IDs for existing
+storages can be found in the UI via B<< Datacenter > Storage >>.
+
+=item * C<< $volname >>
+
+The name of a volume. The term I<volume> can refer to a disk image, an ISO
+image, a backup, etc. depending on the content type of the volume.
+
+=item * C<< $volid >>
+
+The ID of a volume, which is essentially C<"${storeid}:${volname}">. Less used
+within the plugin API, but nevertheless relevant.
+
+=item * C<< $snapname >> (or C<< $snap >>)
+
+The name of a snapshot associated with the given C<$volname>.
+
+=item * C<< \%cache >>
+
+See L<CACHING EXPENSIVE OPERATIONS>.
+
+=item * C<< $vmid >>
+
+The ID of a guest (so, either of a VM or a container).
+
+=back
+
+=head2 CACHING EXPENSIVE OPERATIONS
+
+Certain methods take a C<\%cache> as parameter, which is used to store the
+results of time-consuming / expensive operations specific to certain plugins.
+The exact lifetime of the cached data is B<unspecified>, since it depends on
+the exact usage of the C<L<PVE::Storage>> API, but can generally be assumed to
+be B<short-lived>.
+
+For example, the C<L<PVE::Storage::LVMPlugin>> uses this cache to store the
+parsed output of the C<lvs> command. Since the number and precise information
+about LVM's logical volumes is unlikely to change within a short time, other
+API methods can then use this data in order to avoid repeatedly calling and
+parsing the output of C<lvs>.
+
+Third-party plugin developers should ensure that the data stored and retrieved
+is specific to their plugin, and not rely on the data that other plugins might
+store in C<\%cache>. Furthermore, the names of keys should be rather unique in
+the sense that they're unlikely to conflict with any future keys that may be
+introduced internally. To illustrate, e.g. C<myplugin_mounts> should be used
+instead of a plain C<mounts> key.
+
 =cut
 
 package PVE::Storage::PluginBase;
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v1 pve-storage 3/8] pluginbase: document SectionConfig methods

[Max Carrara]

This commit adds docstrings for the relevant PVE::SectionConfig
methods in the context of the storage plugin API.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/PluginBase.pm | 194 +++++++++++++++++++++++++++++++++-
 1 file changed, 192 insertions(+), 2 deletions(-)

diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
index 16977f3..5f7e6fd 100644
--- a/src/PVE/Storage/PluginBase.pm
+++ b/src/PVE/Storage/PluginBase.pm
@@ -88,7 +88,7 @@ package PVE::Storage::PluginBase;
 use strict;
 use warnings;
 
-use Carp qw(croak);
+use Carp qw(croak confess);
 
 use parent qw(PVE::SectionConfig);
 
@@ -100,27 +100,217 @@ use parent qw(PVE::SectionConfig);
 
 =cut
 
+=head3 $plugin->type()
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+This should return a string with which the plugin can be uniquely identified.
+
+Any string is acceptable, as long as it's descriptive and you're sure it won't
+conflict with another plugin. In the cases of built-in plugins, you will often
+find the filesystem name or something similar being used.
+
+See C<L<PVE::SectionConfig/type>> for more information.
+
+=cut
+
 sub type {
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->properties()
+
+B<OPTIONAL:> May be implemented in a storage plugin.
+
+This method should be implemented if there are additional properties to be used
+by your plugin. Since properties are global and may be reused across plugins,
+the names of properties must not collide with one another.
+
+When implementing a third-party plugin, it is recommended to prefix properties
+with some kind of identifier, like so:
+
+    sub properties {
+	return {
+	    'example-storage-address' => {
+		description => 'Host address of the ExampleStorage to connect to.',
+		type => 'string',
+	    },
+	    'example-storage-pool' => {
+		description => 'Name of the ExampleStorage pool to use.',
+		type => 'string',
+	    },
+	    # [...]
+	};
+    }
+
+However, it is encouraged to reuse properties of inbuilt plugins whenever
+possible. There are a few provided properties that are regarded as I<sensitive>
+and will be treated differently in order to not expose them or write them as
+plain text into configuration files. One such property fit for external use is
+C<password>, which you can use to provide a password, API secret, or similar.
+
+See C<L<PVE::SectionConfig/properties>> for more information.
+
+=cut
+
 sub properties {
     my ($class) = @_;
     return $class->SUPER::properties();
 }
 
+=head3 $plugin->options()
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+This method returns a hash of the properties used by the plugin. Because
+properties are shared among plugins, it is recommended to reuse any existing
+ones of inbuilt plugins and only define custom properties via
+C<L<< properties()|/"$plugin->properties()" >>> if necessary.
+
+The properties a plugin uses are then declared as follows:
+
+    sub options {
+	return {
+	    nodes => { optional => 1 },
+	    content => { optional => 1 },
+	    disable => { optional => 1 },
+	    'example-storage-pool' => { fixed => 1 },
+	    'example-storage-address' => { fixed => 1 },
+	    password => { optional => 1 },
+	};
+    }
+
+C<optional> properties are not required to be set. It is recommended to set
+most properties optional by default, unless it I<really> is required to always
+exist.
+
+C<fixed> properties can only be set when creating a new storage via the plugin
+and cannot be changed afterwards.
+
+See C<L<PVE::SectionConfig/options>> for more information.
+
+=cut
+
 sub options {
     my ($class) = @_;
     return $class->SUPER::options();
 }
 
+=head3 $plugin->plugindata()
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+This method returns a hash that specifies additional capabilities of the storage
+plugin, such as what kinds of data may be stored on it or what VM disk formats
+the storage supports. Additionally, defaults may also be set. This is done
+through the C<content> and C<format> keys.
+
+The C<content> key is used to declare B<supported content types> and their
+defaults, while the C<format> key declares B<supported disk formats> and the
+default disk format of the storage:
+
+    sub plugindata {
+	return {
+	    content => [
+		# possible content types
+		{
+		    images => 1,   # disk images
+		    rootdir => 1,  # container root directories
+		    vztmpl => 1,   # container templates
+		    iso => 1,      # iso images
+		    backup => 1,   # vzdump backup files
+		    snippets => 1, # snippets
+		    import => 1,   # imports; see explanation below
+		    none => 1,     # no content; see explanation below
+		},
+		# defaults if 'content' isn't explicitly set
+		{
+		    images => 1,   # store disk images by default
+		    rootdir => 1   # store containers by default
+		    # possible to add more or have no defaults
+		}
+	    ],
+	    format => [
+		# possible disk formats
+		{
+		    raw => 1,   # raw disk image
+		    qcow2 => 1, # QEMU image format
+		    vmdk => 1,  # VMware image format
+		    subvol => 1 # subvolumes; see explanation below
+		},
+		"qcow2" # default if 'format' isn't explicitly set
+	    ]
+	    # [...]
+	};
+    }
+
+While the example above depicts a rather capable storage, the following
+shows a simpler storage that can only be used for VM disks:
+
+    sub plugindata {
+	return {
+	    content => [
+		{ images => 1 },
+	    ],
+	    format => [
+		{ raw => 1 },
+		"raw",
+	    ]
+	};
+    }
+
+Which content types and formats are supported depends on the underlying storage
+implementation.
+
+B<Regarding C<import>:> The C<import> content type is used internally to
+interface with virtual guests of foreign sources or formats. The corresponding
+functionality has not yet been published to the public parts of the storage
+API. Third-party plugins therefore should not declare this content type.
+
+B<Regarding C<none>:> The C<none> content type denotes the I<absence> of other
+types of content, i.e. this content type may only be set on a storage if no
+other content type is set. This is used internally for storages that support
+adding another storage "on top" of them; at the moment, this makes it possible
+to set up an LVM (thin) pool on top of an iSCSI LUN. The corresponding
+functionality has not yet been published to the public parts of the storage
+API. Third-party plugins therefore should not declare this content type.
+
+B<Regarding C<subvol>:> The C<subvol> format is used internally to allow the
+root directories of containers to use ZFS subvolumes (also known as
+I<ZFS datasets>, not to be confused with I<ZVOLs>). Third-party plugins should
+not declare this format type.
+
+There is one more key, C<select_existing>, which is used internally for
+ISCSI-related GUI functionality. Third-party plugins should not declare this
+key.
+
+=cut
+
 sub plugindata {
     my ($class) = @_;
     return $class->SUPER::plugindata();
 }
 
+=head3 $plugin->private()
+
+B<WARNING:> This method is provided by C<L<PVE::Storage::Plugin>> and
+must be used as-is. It is merely documented here for informal purposes
+and B<must not be overridden.>
+
+Returns a hash containing the tracked plugin metadata, most notably the
+C<propertyList>, which contains all known properties of all plugins.
+
+C<L<PVE::Storage::Plugin>> uses this to predefine a lot of useful properties
+that are relevant for all plugins. Core functionality such as defining
+whether a storage is shared, which nodes may use it, whether a storage
+is enabled or not, etc. are implemented via these properties.
+
+See C<L<PVE::SectionConfig/private>> for more information.
+
+=cut
+
 sub private {
-    croak "implement me in sub-class\n";
+    confess "private() is provided by PVE::Storage::Plugin and must not be overridden";
 }
 
 =head2 GENERAL
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v1 pve-storage 4/8] pluginbase: document general plugin methods

[Max Carrara]

Add docstrings for the following methods:
- check_connection
- activate_storage
- deactivate_storage
- status
- cluster_lock_storage
- parse_volname
- get_subdir
- filesystem_path
- path
- find_free_diskname

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/PluginBase.pm | 255 ++++++++++++++++++++++++++++++++++
 1 file changed, 255 insertions(+)

diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
index 5f7e6fd..8a61dc3 100644
--- a/src/PVE/Storage/PluginBase.pm
+++ b/src/PVE/Storage/PluginBase.pm
@@ -317,53 +317,308 @@ sub private {
 
 =cut
 
+=head3 $plugin->check_connection($storeid, \%scfg)
+
+B<OPTIONAL:> May be implemented in a storage plugin.
+
+Performs a connection check.
+
+This method is useful for plugins that require some kind of network connection
+or similar and is called before C<L<< activate_storage()|/"$plugin->activate_storage($storeid, \%scfg, \%cache)" >>>.
+
+Returns C<1> by default and C<0> if the storage isn't online / reachable.
+
+C<die>s if an error occurs while performing the connection check.
+
+Note that this method's purpose is to simply verify that the storage is
+I<reachable>, and not necessarily that authentication etc. succeeds.
+
+For example: If the storage is mainly accessed via TCP, you can try to simply
+open a TCP connection to see if it's online:
+
+    # In custom module:
+
+    use PVE::Network;
+    use parent qw(PVE::Storage::Plugin)
+
+    # [...]
+
+    sub check_connection {
+	my ($class, $storeid, $scfg) = @_;
+
+	my $port = $scfg->{port} || 5432;
+	return PVE::Network::tcp_ping($scfg->{server}, $port, 5);
+    }
+
+=cut
+
 sub check_connection {
     my ($class, $storeid, $scfg) = @_;
 
     return 1;
 }
 
+=head3 $plugin->activate_storage($storeid, \%scfg, \%cache)
+
+=head3 $plugin->activate_storage(...)
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Activates the storage, making it ready for further use.
+
+In essence, this method performs the steps necessary so that the storage can be
+used by remaining parts of the system.
+
+In the case of file-based storages, this usually entails creating the directory
+of the mountpoint, mounting the storage and then creating the directories for
+the different content types that the storage has enabled. See
+C<L<PVE::Storage::NFSPlugin>> and C<L<PVE::Storage::CIFSPlugin>> for examples
+in that regard.
+
+Other types of storages would use this method for establishing a connection to
+the storage and authenticating with it or similar. See C<L<PVE::Storage::ISCSIPlugin>>
+for an example.
+
+If the storage doesn't need to be activated in some way, this method can be a
+no-op.
+
+C<die>s in case of errors.
+
+This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
+
+=cut
+
 sub activate_storage {
     my ($class, $storeid, $scfg, $cache) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->deactivate_storage($storeid, \%scfg, \%cache)
+
+=head3 $plugin->deactivate_storage(...)
+
+B<OPTIONAL:> May be implemented in a storage plugin.
+
+Deactivates the storage. Counterpart to C<L<< activate_storage()|/"$plugin->activate_storage(...)" >>>.
+
+This method is used to make the storage unavailable to the rest of the system,
+which usually entails unmounting the storage, closing an established
+connection, or similar.
+
+Does nothing by default.
+
+C<die>s in case of errors.
+
+This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
+
+B<NOTE:> This method is currently not used by our API due to historical
+reasons.
+
+=cut
+
 sub deactivate_storage {
     my ($class, $storeid, $scfg, $cache) = @_;
 
     return;
 }
 
+=head3 $plugin->status($storeid, \%scfg, \%cache)
+
+=head3 $plugin->status(...)
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Returns a list of scalars in the following form:
+
+    my ($total, $available, $used, $active) = $plugin->status($storeid, $scfg, $cache)
+
+This list contains the C<$total>, C<$available> and C<$used> storage capacity,
+respectively, as well as whether the storage is C<$active> or not.
+
+This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
+
+=cut
+
 sub status {
     my ($class, $storeid, $scfg, $cache) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->cluster_lock_storage($storeid, $shared, $timeout, \&func, @param)
+
+=head3 $plugin->cluster_lock_storage(...)
+
+B<WARNING:> This method is provided by C<L<PVE::Storage::Plugin>> and
+must be used as-is. It is merely documented here for informal purposes
+and B<must not be overridden.>
+
+Locks the storage with the given C<$storeid> for the entire host and runs the
+given C<\&func> with the given C<@param>s, unlocking the storage once finished.
+If the storage is C<$shared>, it is instead locked on the entire cluster. If
+the storage is already locked, wait for at most the number of seconds given by
+C<$timeout>.
+
+This method is used to synchronize access to the given storage, preventing
+simultaneous modification from multiple workers. This is necessary for
+operations that modify data on the storage directly, like cloning and
+allocating images.
+
+=cut
+
 sub cluster_lock_storage {
     my ($class, $storeid, $shared, $timeout, $func, @param) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->parse_volname($volname)
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Parses C<$volname>, returning a list representing the parts encoded in
+the volume name:
+
+    my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format)
+	= $plugin->parse_volname($volname);
+
+Not all parts need to be included in the list. Those marked as I<optional>
+in the list below may be set to C<undef> if not applicable.
+
+This method may C<die> in case of errors.
+
+=over
+
+=item * C<< $vtype >>
+
+The content type ("volume type") of the volume, e.g. C<"images">, C<"iso">,
+etc.
+
+See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all content types.
+
+=item * C<< $name >>
+
+The display name of the volume. This is usually what the underlying storage
+itself uses to address the volume.
+
+For example, disks for virtual machines that are stored on LVM thin pools are
+named C<vm-100-disk-0>, C<vm-1337-disk-1>, etc. That would be the C<$name> in
+this case.
+
+=item * C<< $vmid >> (optional)
+
+The ID of the guest that owns the volume.
+
+=item * C<< $basename >> (optional)
+
+The C<$name> of the volume this volume is based on. Whether this part
+is returned or not depends on the plugin and underlying storage.
+Only applies to disk images.
+
+For example, on ZFS, if the VM is a linked clone, C<$basename> refers
+to the C<$name> of the original disk volume that the parsed disk volume
+corresponds to.
+
+=item * C<< $basevmid >> (optional)
+
+Equivalent to C<$basename>, except that C<$basevmid> refers to the
+C<$vmid> of the original disk volume instead.
+
+=item * C<< $isBase >> (optional)
+
+Whether the volume is a base disk image.
+
+=item * C<< $format >>
+
+The format of the volume. If the volume is a VM disk (C<$vtype> is
+C<"images">), this should be whatever format the disk is in. For most
+other content types C<"raw"> should be used.
+
+See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all formats.
+
+=back
+
+=cut
+
 sub parse_volname {
     my ($class, $volname) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->get_subdir(\%scfg, $vtype)
+
+B<SITUATIONAL:> This method must be implemented for file-based storages.
+
+Returns the path to the sub-directory associated with the given content type
+(C<$vtype>). See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all
+content types.
+
+The default directory layout is predefined and must not be altered:
+
+    my $vtype_subdirs = {
+	images   => 'images',
+	rootdir  => 'private',
+	iso      => 'template/iso',
+	vztmpl   => 'template/cache',
+	backup   => 'dump',
+	snippets => 'snippets',
+	import   => 'import',
+    };
+
+See also: L<Storage: Directory|"https://pve.proxmox.com/wiki/Storage:_Directory">
+
+=cut
+
 sub get_subdir {
     my ($class, $scfg, $vtype) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->filesystem_path(\%scfg, $volname [, $snapname])
+
+=head3 $plugin->filesystem_path(...)
+
+B<SITUATIONAL:> This method must be implemented for file-based storages.
+
+Returns the absolute path on the filesystem for the given volume or snapshot.
+In list context, returns path, guest ID and content type:
+
+    my $path = $plugin->filesystem_path($scfg, $volname, $snapname)
+    my ($path, $vmid, $vtype) = $plugin->filesystem_path($scfg, $volname, $snapname)
+
+=cut
+
 sub filesystem_path {
     my ($class, $scfg, $volname, $snapname) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->path(\%scfg, $volname, $storeid [, $snapname])
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Returns a unique path that points to the given volume or snapshot depending
+on the underlying storage implementation. For file-based storages, this
+method should return the same as C<L<< filesystem_path()|/"$plugin->filesystem_path(...)" >>>.
+
+=cut
+
 sub path {
     my ($class, $scfg, $volname, $storeid, $snapname) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->find_free_diskname($storeid, \%scfg, $vmid [, $fmt, $add_fmt_suffix])
+
+=head3 $plugin->find_free_diskname(...)
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Finds and returns the next available disk name, that is, the volume name
+(C<$volname>) for a new disk image. Optionally, C<$fmt> specifies the
+format of the disk image and C<$add_fmt_suffix> denotes whether C<$fmt>
+should be added as suffix to the resulting name.
+
+=cut
+
 sub find_free_diskname {
     my ($class, $storeid, $scfg, $vmid, $fmt, $add_fmt_suffix) = @_;
     croak "implement me in sub-class\n";
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v1 pve-storage 5/8] pluginbase: document hooks

[Max Carrara]

Add docstrings for the following methods:
- on_add_hook
- on_update_hook
- on_delete_hook

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/PluginBase.pm | 85 ++++++++++++++++++++++++++++++-----
 1 file changed, 74 insertions(+), 11 deletions(-)

diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
index 8a61dc3..b3ce684 100644
--- a/src/PVE/Storage/PluginBase.pm
+++ b/src/PVE/Storage/PluginBase.pm
@@ -626,29 +626,92 @@ sub find_free_diskname {
 
 =head2 HOOKS
 
+The following methods are called whenever a storage associated with the given
+plugin is added, updated, or deleted. These methods are useful for:
+
+=over
+
+=item * Setting up certain prerequisites when adding the storage (and then
+tearing them down again when the storage is deleted)
+
+=item * Handling sensitive parameters that shouldn't be written directly
+to C</etc/pve/storage.cfg> and ought to be stored elsewhere
+
+=item * Ensuring that certain conditions in the configuration are being upheld
+that cannot be done via the remaining API otherwise
+
+=back
+
+and more.
+
+=cut
+
+=head3 $plugin->on_add_hook($storeid, \%scfg, %param)
+
+=head3 $plugin->on_add_hook(...)
+
+B<OPTIONAL:> May be implemented in a storage plugin.
+
+Called during the addition of a storage, before the new storage configuration
+gets written.
+
+C<%param> contains additional key-value arguments, usually sensitive keys that
+have been extracted from C<\%scfg> in order not to write them to the storage
+configuration.
+
+C<die>s in order to abort the addition if there are (grave) problems.
+
+C</etc/pve/storage.cfg> is B<locked> when this method is called.
+
 =cut
 
-# called during addition of storage (before the new storage config got written)
-# die to abort addition if there are (grave) problems
-# NOTE: runs in a storage config *locked* context
 sub on_add_hook {
     my ($class, $storeid, $scfg, %param) = @_;
     return undef;
 }
 
-# called during storage configuration update (before the updated storage config got written)
-# die to abort the update if there are (grave) problems
-# NOTE: runs in a storage config *locked* context
+=head3 $plugin->on_update_hook($storeid, \%scfg, %param)
+
+=head3 $plugin->on_update_hook(...)
+
+B<OPTIONAL:> May be implemented in a storage plugin.
+
+Called during the update of a storage configuration, before the new
+configuration gets written.
+
+C<%param> contains additional key-value arguments, usually sensitive keys that
+have been extracted from C<\%scfg> in order not to write them to the storage
+configuration.
+
+C<die>s in order to abort the config update if there are (grave) problems.
+
+C</etc/pve/storage.cfg> is B<locked> when this method is called.
+
+=cut
+
 sub on_update_hook {
     my ($class, $storeid, $scfg, %param) = @_;
     return undef;
 }
 
-# called during deletion of storage (before the new storage config got written)
-# and if the activate check on addition fails, to cleanup all storage traces
-# which on_add_hook may have created.
-# die to abort deletion if there are (very grave) problems
-# NOTE: runs in a storage config *locked* context
+=head3 $plugin->on_delete_hook($storeid, \%scfg)
+
+=head3 $plugin->on_delete_hook(...)
+
+B<OPTIONAL:> May be implemented in a storage plugin.
+
+Called during the deletion of a storage, before the new storage configuration
+gets written. Also gets called if the activation check during storage
+addition fails in order to clean up all traces which
+C<L<< on_add_hook()|/"$plugin->on_add_hook($storeid, \%scfg, %param)" >>>
+may have created.
+
+C<die>s in order to abort the deletion of there are (very grave) problems.
+
+C</etc/pve/storage.cfg> is B<locked> when this method is called.
+
+=cut
+
 sub on_delete_hook {
     my ($class, $storeid, $scfg) = @_;
     return undef;
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v1 pve-storage 6/8] pluginbase: document image operation methods

[Max Carrara]

Add documentation for the following methods:
- list_images
- create_base
- clone_image
- alloc_image
- free_image

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
Co-authored-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
---
 src/PVE/Storage/PluginBase.pm | 111 ++++++++++++++++++++++++++++++++++
 1 file changed, 111 insertions(+)

diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
index b3ce684..37b1471 100644
--- a/src/PVE/Storage/PluginBase.pm
+++ b/src/PVE/Storage/PluginBase.pm
@@ -721,26 +721,137 @@ sub on_delete_hook {
 
 =cut
 
+=head3 $plugin->list_images($storeid, \%scfg [, $vmid, \@vollist, \%cache])
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Returns a listref of all disk images of a storage. If the storage does not
+support storing disk images, returns an empty listref.
+
+Optionally, if C<\@vollist> is provided, return only disks whose volume ID is
+within C<\@vollist>. Note that this usually has higher precedence than
+C<$vmid>.
+
+C<die>s in case of errors.
+
+This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
+
+The returned listref has the following structure:
+
+    [
+	{
+	    ctime => "1689163322", # creation time as unix timestamp
+	    format => "raw",
+	    size => 8589934592, # in bytes!
+	    vmid => 101,
+	    volid => "local-lvm:base-101-disk-0", # volume ID, storage-specific
+	},
+	# [...]
+    ]
+
+=cut
+
 sub list_images {
     my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->create_base($storeid, \%scfg, $volname)
+
+B<OPTIONAL:> May be implemented in a storage plugin.
+
+Creates a base volume from an existing volume, allowing the volume to be
+L<< cloned|/"$plugin->clone_image(...)" >>. This cloned volume (usually
+a disk image) may then be used as a base for the purpose of creating linked
+clones. See L<C<PVE::Storage::LvmThinPlugin>> and
+L<C<PVE::Storage::ZFSPoolPlugin>> for example implementations.
+
+On completion, returns the name of the new base volume (the new C<$volname>).
+
+This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
+i.e. when the storage is B<locked>.
+
+=cut
+
 sub create_base {
     my ($class, $storeid, $scfg, $volname) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->clone_image($scfg, $storeid, $volname, $vmid [, $snap])
+
+=head3 $plugin->clone_image(...)
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Clones a disk image or a snapshot of an image, returning the name of the new
+image (the new C<$volname>). Note that I<cloning> here means to create a linked
+clone and not duplicating an image. See L<C<PVE::Storage::LvmThinPlugin>> and
+L<C<PVE::Storage::ZFSPoolPlugin>> for example implementations.
+
+C<die>s in case of an error of if the underlying storage doesn't support
+cloning images.
+
+This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
+i.e. when the storage is B<locked>.
+
+=cut
+
 sub clone_image {
     my ($class, $scfg, $storeid, $volname, $vmid, $snap) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->alloc_image($storeid, $scfg, $vmid, $fmt, $name, $size)
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Allocates a disk image with the given format C<$fmt> and C<$size> in bytes,
+returning the name of the new image (the new C<$volname>). See
+C<L<< plugindata()|/"$plugin->plugindata()" >>> for all disk formats.
+
+Optionally, if given, set the name of the image to C<$name>. If C<$name> isn't
+provided, the next name should be determined via C<L<< find_free_diskname()|/"$plugin->find_free_diskname(...)" >>>.
+
+C<die>s in case of an error of if the underlying storage doesn't support
+allocating images.
+
+This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
+i.e. when the storage is B<locked>.
+
+=cut
+
 sub alloc_image {
     my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->free_image($storeid, $scfg, $volname [, $isBase, $format])
+
+=head3 $plugin->free_image(...)
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Frees (deletes) the disk image given by C<$volname>. Optionally, the image may
+be a base image (C<$isBase>) and have a certain C<$format>. See
+C<L<< plugindata()|/"$plugin->plugindata()" >>> for all disk formats.
+
+If a cleanup is required after freeing the image, returns a reference to a
+subroutine that performs the cleanup, and C<undef> otherwise.
+
+C<die>s in case of errors, if the image has a L<< C<protected> attribute|/"$plugin->get_volume_attribute(...)" >>
+(and may thus not be freed), or if the underlying storage implementation
+doesn't support freeing images.
+
+This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
+i.e. when the storage is B<locked>.
+
+B<NOTE:> The returned cleanup subroutine is called in a separate worker and
+should L<< lock|/"$plugin->cluster_lock_storage(...)" >> the underlying storage
+separately.
+
+=cut
+
 sub free_image {
     my ($class, $storeid, $scfg, $volname, $isBase, $format) = @_;
     croak "implement me in sub-class\n";
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v1 pve-storage 7/8] pluginbase: document volume operations

[Max Carrara]

Add docstrings for the following methods:
- list_volumes
- get_volume_attribute
- update_volume_attribute
- volume_size_info
- volume_resize
- volume_snapshot
- volume_snapshot_info
- volume_rollback_is_possible
- volume_snapshot_rollback
- volume_snapshot_delete
- volume_snapshot_needs_fsfreeze
- storage_can_replicate
- volume_has_feature
- map_volume
- unmap_volume
- activate_volume
- deactivate_volume
- rename_volume
- prune_backups

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
Co-authored-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
---
 src/PVE/Storage/PluginBase.pm | 401 ++++++++++++++++++++++++++++++++--
 1 file changed, 388 insertions(+), 13 deletions(-)

diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
index 37b1471..a1bfc8d 100644
--- a/src/PVE/Storage/PluginBase.pm
+++ b/src/PVE/Storage/PluginBase.pm
@@ -861,108 +861,483 @@ sub free_image {
 
 =cut
 
+=head3 $plugin->list_volumes($storeid, \%scfg, $vmid, \@content_types)
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Returns a list of volumes for the given guest whose content type is within the
+given C<\@content_types>. If C<\@content_types> is empty, no volumes will be
+returned. See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all content types.
+
+    # List all backups for a guest
+    my $backups = $plugin->list_volumes($storeid, $scfg, $vmid, ['backup']);
+
+    # List all containers and virtual machines on the storage
+    my $guests = $plugin->list_volumes($storeid, $scfg, undef, ['rootdir', 'images'])
+
+The returned listref of hashrefs has the following structure:
+
+    [
+	{
+	    content => "images",
+	    ctime => "1702298038", # creation time as unix timestamp
+	    format => "raw",
+	    size => 9663676416, # in bytes!
+	    vmid => 102,
+	    volid => "local-lvm:vm-102-disk-0",
+	},
+	# [...]
+    ]
+
+Backups will also contain additional keys:
+
+    [
+	{
+	    content => "backup",
+	    ctime => 1742405070, # creation time as unix timestamp
+	    format => "tar.zst",
+	    notes => "...", # comment that was entered when backup was created
+	    size => 328906840, # in bytes!
+	    subtype => "lxc", # "lxc" for containers, "qemu" for VMs
+	    vmid => 106,
+	    volid => "local:backup/vzdump-lxc-106-2025_03_19-18_24_30.tar.zst",
+	},
+	# [...]
+    ]
+
+=cut
+
 sub list_volumes {
     my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
     croak "implement me in sub-class\n";
 }
 
-# Returns undef if the attribute is not supported for the volume.
-# Should die if there is an error fetching the attribute.
-# Possible attributes:
-# notes     - user-provided comments/notes.
-# protected - not to be removed by free_image, and for backups, ignored when pruning.
+=head3 $plugin->get_volume_attribute(\%scfg, $storeid, $volname, $attribute)
+
+=head3 $plugin->get_volume_attribute(...)
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Returns the value of the given C<$attribute> for a volume. If the attribute isn't
+supported for the volume, returns C<undef>.
+
+C<die>s if there is an error fetching the attribute.
+
+B<Possible attributes:>
+
+=over
+
+=item * C<notes> (returns scalar)
+
+User-provided comments / notes.
+
+=item * C<protected> (returns scalar)
+
+When set to C<1>, the volume must not be removed by C<L<< free_image()|/"$plugin->free_image(...)" >>>.
+Backups with C<protected> set to C<1> are ignored when pruning.
+
+=back
+
+=cut
+
 sub get_volume_attribute {
     my ($class, $scfg, $storeid, $volname, $attribute) = @_;
     croak "implement me in sub-class\n";
 }
 
-# Dies if the attribute is not supported for the volume.
+=head3 $plugin->update_volume_attribute(\%scfg, $storeid, $volname, $attribute, $value)
+
+=head3 $plugin->update_volume_attribute(...)
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Sets the value of the given C<$attribute> for a volume to C<$value>.
+
+C<die>s if the attribute isn't supported for the volume (or storage).
+
+For a list of supported attributes, see C<L<< get_volume_attribute()|/"$plugin->get_volume_attribute(...)" >>>.
+
+=cut
+
 sub update_volume_attribute {
     my ($class, $scfg, $storeid, $volname, $attribute, $value) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->volume_size_info(\%scfg, $storeid, $volname [, $timeout])
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Returns information about the given volume's size. In scalar context, this returns
+just the volume's size in bytes:
+
+    my $size = $plugin->volume_size_info($scfg, $storeid, $volname, $timeout)
+
+In list context, returns an array with the following structure:
+
+    my ($size, $format, $used, $parent, $ctime) = $plugin->volume_size_info(
+	$scfg, $storeid, $volname, $timeout
+    )
+
+where C<$size> is the size of the volume in bytes, C<$format> one of the possible
+L<< formats listed in C<plugindata()>|/"$plugin->plugindata()" >>, C<$used> the
+amount of space used in bytes, C<$parent> the (optional) name of the base volume
+and C<$ctime> the creation time as unix timestamp.
+
+Optionally, a C<$timeout> may be provided after which the method should C<die>.
+This timeout is best passed to other helpers which already support timeouts,
+such as C<L<< PVE::Tools::run_command|PVE::Tools/run_command >>>.
+
+See also the C<L<< PVE::Storage::Plugin::file_size_info|PVE::Storage::Plugin/file_size_info >>> helper.
+
+=cut
+
 sub volume_size_info {
     my ($class, $scfg, $storeid, $volname, $timeout) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->volume_resize(\%scfg, $storeid, $volname, $size [, $running])
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Resizes a volume to the new C<$size> in bytes. Optionally, the guest that owns
+the volume may be C<$running> (= C<1>).
+
+C<die>s in case of errors, or if the underlying storage implementation or the
+volume's format doesn't support resizing.
+
+This function should not return any value. In previous versions the returned
+value would be used to determine the new size of the volume or whether the
+operation succeeded.
+
+=cut
+
 sub volume_resize {
     my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->volume_snapshot(\%scfg, $storeid, $volname, $snap)
+
+B<OPTIONAL:> May be implemented if the underlying storage supports snapshots.
+
+Takes a snapshot of a volume and gives it the name provided by C<$snap>.
+
+C<die>s if the underlying storrage doesn't support snapshots or an error
+occurs while taking a snapshot.
+
+=cut
+
 sub volume_snapshot {
     my ($class, $scfg, $storeid, $volname, $snap) = @_;
     croak "implement me in sub-class\n";
 }
 
-# Returns a hash with the snapshot names as keys and the following data:
-# id        - Unique id to distinguish different snapshots even if the have the same name.
-# timestamp - Creation time of the snapshot (seconds since epoch).
-# Returns an empty hash if the volume does not exist.
+=head3 $plugin->volume_snapshot_info(\%scfg, $storeid, $volname)
+
+Returns a hashref with the snapshot names as keys and the following structure:
+
+    {
+        my_snapshot => {
+            id => "01b7e668-58b4-6f42-9a5e-1944c2855c84",  # Unique id to distinguish different snapshots with the same name.
+            timestamp => "1729841807",  # Creation time of the snapshot (seconds since epoch).
+        },
+	# [...]
+    }
+
+Returns an empty hashref if the volume does not exist.
+
+=cut
+
 sub volume_snapshot_info {
     my ($class, $scfg, $storeid, $volname) = @_;
     croak "implement me in sub-class\n";
 }
 
-# Asserts that a rollback to $snap on $volname is possible.
-# If certain snapshots are preventing the rollback and $blockers is an array
-# reference, the snapshot names can be pushed onto $blockers prior to dying.
+=head3 $plugin->volume_rollback_is_possible(\%scfg, $storeid, $volname, $snap [, \@blockers])
+
+=head3 $plugin->volume_rollback_is_possible(...)
+
+B<OPTIONAL:> May be implemented if the underlying storage supports snapshots.
+
+Asserts whether a rollback to C<$snap> on C<$volname> is possible, C<die>s if
+it isn't.
+
+Optionally, C<\@blockers> may be provided, which may contain the names of
+snapshots that are preventing the rollback. Should any such snapshots exist,
+they should be pushed to this listref pior to C<die>-ing. The caller may then
+use this listref when handling errors.
+
+=cut
+
 sub volume_rollback_is_possible {
     my ($class, $scfg, $storeid, $volname, $snap, $blockers) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->volume_snapshot_rollback(\%scfg, $storeid, $volname, $snap)
+
+Performs a rollback to the given C<$snap>shot on C<$volname>.
+
+C<die>s in case of errors.
+
+=cut
+
 sub volume_snapshot_rollback {
     my ($class, $scfg, $storeid, $volname, $snap) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->volume_snapshot_delete(\%scfg, $storeid, $volname, $snap [, $running])
+
+Deletes the C<$snap>shot of C<$volname>.
+
+C<die>s in case of errors.
+
+Optionally, the guest that owns the given volume may be C<$running> (= C<1>).
+
+B<Deprecated:> The C<$running> parameter is deprecated and will be removed on the
+next C<APIAGE> reset.
+
+=cut
+
 sub volume_snapshot_delete {
     my ($class, $scfg, $storeid, $volname, $snap, $running) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->volume_snapshot_needs_fsfreeze()
+
+Returns whether filesystems on top of the volume need to flush their journal for
+consistency before a snapshot is taken. See L<fsfreeze(8)>.
+
+This is needed for container mountpoints.
+
+=cut
+
 sub volume_snapshot_needs_fsfreeze {
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->storage_can_replicate(\%scfg, $storeid, $format)
+
+Returns whether volumes in a given C<$format> support replication.
+
+See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all disk formats.
+
+=cut
+
 sub storage_can_replicate {
     my ($class, $scfg, $storeid, $format) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->volume_has_feature(\%scfg, $feature, $storeid, $volname, $snapname [, $running, \%opts])
+
+=head3 $plugin->volume_has_feature(...)
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Checks whether a volume C<$volname> or its snapshot C<$snapname> supports the
+given C<$feature>, returning C<1> if it does and C<undef> otherwise. The guest
+owning the volume may optionally be C<$running>.
+
+C<$feature> may be one of the following:
+
+    clone      # linked clone is possible
+    copy       # full clone is possible
+    replicate  # replication is possible
+    snapshot   # taking a snapshot is possible
+    sparseinit # volume is sparsely initialized
+    template   # conversion to base image is possible
+    rename     # renaming volumes is possible
+
+Which features are available under which circumstances depends on multiple
+factors, such as the underlying storage implementation, the format used, etc.
+It's best to check out C<L<PVE::Storage::Plugin>> or C<L<PVE::Storage::ZFSPoolPlugin>>
+for examples on how to handle features.
+
+Additional keys are given in C<\%opts>:
+
+=over
+
+=item * C<< valid_target_formats => [...] >>
+
+Listref of formats for the target of a copy/clone operation that the caller
+could work with. The format of the given volume is always considered valid and
+if no list is specified, all formats are considered valid.
+
+=back
+
+=cut
+
 sub volume_has_feature {
     my ($class, $scfg, $feature, $storeid, $volname, $snapname, $running, $opts) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->map_volume($storeid, \%scfg, $volname [, $snapname])
+
+B<OPTIONAL:> May be implemented in a storage plugin.
+
+Maps the device asscoiated with a volume or a volume's snapshot to a filesystem
+path, returning the path on completion. This method is used by containers.
+
+C<die>s in case of errors.
+
+C<L<< unmap_volume()|/"$plugin->unmap_volume(...)" >>> can be used to declare
+how the device should be unmapped.
+
+=cut
+
 sub map_volume {
     my ($class, $storeid, $scfg, $volname, $snapname) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->unmap_volume($storeid, \%scfg, $volname [, $snapname])
+
+B<OPTIONAL:> May be implemented in a storage plugin.
+
+Unmaps the device associated to a volume or a volume's snapshot.
+
+C<die>s in case of errors.
+
+=cut
+
 sub unmap_volume {
     my ($class, $storeid, $scfg, $volname, $snapname) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->activate_volume($storeid, \%scfg, $volname, $snapname [, \%cache])
+
+=head3 $plugin->activate_volume(...)
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Activates a volume or its associated snapshot, making it available to the
+system for further use. For example, this could mean activating an LVM volume,
+mounting a ZFS dataset, checking whether the volume's file path exists, etc.
+
+C<die>s in case of errors or if an operation is not supported.
+
+If this isn't needed, the method should simply be a no-op.
+
+This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
+
+=cut
+
 sub activate_volume {
     my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->deactivate_volume($storeid, \%scfg, $volname, $snapname [, \%cache])
+
+=head3 deactivate_volume(...)
+
+B<REQUIRED:> Must be implemented in every storage plugin.
+
+Deactivates a volume or its associated snapshot, making it unavailable to
+the system. For example, this could mean deactivating an LVM volume,
+unmapping a Ceph/RBD device, etc.
+
+If this isn't needed, the method should simply be a no-op.
+
+This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
+
+=cut
+
 sub deactivate_volume {
     my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->rename_volume(...)
+
+=head3 $plugin->rename_volume(\%scfg, $storeid, $source_volname, $target_vmid, $target_volname)
+
+B<OPTIONAL:> May be implemented in a storage plugin.
+
+Renames the volume given by C<$source_volname> to C<$target_volname> and assigns
+it to the guest C<$target_vmid>. Returns the volume ID of the renamed volume.
+
+This method is needed for the I<Change Owner> feature.
+
+C<die>s if the rename failed.
+
+=cut
+
 sub rename_volume {
     my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->prune_backups(\%scfg, $storeid [, \%keep, $vmid, $type, $dryrun, \&logfunc])
+
+=head3 $plugin->prune_backups(...)
+
+Export a volume into a file handle as a stream with a desired format.
+
+C<die>s if there are (grave) problems while pruning.
+
+This method may take several optional parameters:
+
+=over
+
+=item * C<< \%keep >>
+
+A hashref containing backup retention policies. It has the following structure:
+
+    {
+	'keep-all'     => 1 # (optional) Whether to keep all backups.
+	                    # Conflicts with the other options when true.
+	'keep-last'    => N # (optional) Keep the last N backups.
+	'keep-hourly'  => N # (optional) Keep backups for the last N different hours.
+			    # If there is more than one backup for a single
+			    # hour, only the latest one is kept.
+	'keep-daily'   => N # (optional) Keep backups for the last N different days.
+			    # If there is more than one backup for a single
+			    # day, only the latest one is kept.
+	'keep-weekly'  => N # (optional) Keep backups for the last N different weeks.
+			    # If there is more than one backup for a single
+			    # week, only the latest one is kept.
+	'keep-monthly' => N # (optional) Keep backups for the last N different months.
+			    # If there is more than one backup for a single
+			    # month, only the latest one is kept.
+	'keep-yearly'  => N # (optional) Keep backups for the last N different years.
+			    # If there is more than one backup for a single
+			    # year, only the latest one is kept.
+    }
+
+=item * C<< $vmid >>
+
+The guest's ID.
+
+=item * C<< $type >>
+
+The type of guest. When C<defined>, it can be one of C<"qemu"> or C<"lxc">.
+If C<undefined>, both types of backups will be pruned.
+
+=item * C<< $dry_run >>
+
+Whether this is a dry run. If set to C<1> there won't be any change on the
+underlying storage.
+
+=item * C<< \&logfunc >>
+
+A subroutine ref that can be used to log messages with the following signature:
+
+    $logfunc->($severity, $message)
+
+where $severity can be one of C<"info">, C<"err">, or C<"warn">.
+
+=back
+
+=cut
+
 sub prune_backups {
     my ($class, $scfg, $storeid, $keep, $vmid, $type, $dryrun, $logfunc) = @_;
     croak "implement me in sub-class\n";
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v1 pve-storage 8/8] pluginbase: document import and export methods

[Max Carrara]

Adapt the previous description, slightly rewording it and formatting
it for POD under the IMPORTS AND EXPORTS section.

Also add docstrings for the following methods:
- volume_export
- volume_export_formats
- volume_import
- volume_import_formats

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
Co-authored-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
---
 src/PVE/Storage/PluginBase.pm | 134 ++++++++++++++++++++++++++--------
 1 file changed, 105 insertions(+), 29 deletions(-)

diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
index a1bfc8d..cfa5087 100644
--- a/src/PVE/Storage/PluginBase.pm
+++ b/src/PVE/Storage/PluginBase.pm
@@ -1345,55 +1345,131 @@ sub prune_backups {
 
 =head2 IMPORTS AND EXPORTS
 
+Any path-based storage is assumed to support C<raw> and C<tar> streams, so
+the default implementations in C<L<PVE::Storage::Plugin>> will return this if
+C<< $scfg->{path} >> is set (thereby mimicking the old C<< PVE::Storage::storage_migrate() >>
+function).
+
+Plugins may fall back to methods like C<volume_export>, C<volume_import>, etc.
+of C<L<PVE::Storage::Plugin>> in case the format doesn't match their
+specialized implementations to reuse the C<raw>/C<tar> code.
+
+=head3 FORMATS
+
+The following formats are all prefixed with image information in the form of a
+64 bit little endian unsigned integer (C<< pack('Q\<') >>) in order to be able
+to preallocate the image on storages which require it.
+
+=over
+
+=item * C<< raw+size >> (image files only)
+
+A raw binary data stream as produced via C<< dd if=$IMAGE_FILE >>.
+
+=item * C<< qcow2+size >>, C<< vmdk >> (image files only)
+
+A raw C<qcow2>/C<vmdk>/... file as produced via C<< dd if=some_file.qcow2 >>
+for files which are already in C<qcow2> format, or via C<qemu-img convert>.
+
+B<NOTE:> These formats are only valid with C<$with_snapshots> being true (C<1>).
+
+=item * C<< tar+size >> (subvolumes only)
+
+A GNU C<tar> stream containing just the inner contents of the subvolume. This
+does not distinguish between the contents of a privileged or unprivileged
+container. In other words, this is from the root user namespace's point of view
+with no uid-mapping in effect. As produced via e.g.
+C<< tar -C vm-100-disk-1.subvol -cpf output_file.dat . >>
+
+=back
+
 =cut
 
-# Import/Export interface:
-#   Any path based storage is assumed to support 'raw' and 'tar' streams, so
-#   the default implementations will return this if $scfg->{path} is set,
-#   mimicking the old PVE::Storage::storage_migrate() function.
-#
-# Plugins may fall back to PVE::Storage::Plugin::volume_{export,import}...
-#   functions in case the format doesn't match their specialized
-#   implementations to reuse the raw/tar code.
-#
-# Format specification:
-#   The following formats are all prefixed with image information in the form
-#   of a 64 bit little endian unsigned integer (pack('Q<')) in order to be able
-#   to preallocate the image on storages which require it.
-#
-#   raw+size: (image files only)
-#     A raw binary data stream such as produced via `dd if=TheImageFile`.
-#   qcow2+size, vmdk: (image files only)
-#     A raw qcow2/vmdk/... file such as produced via `dd if=some.qcow2` for
-#     files which are already in qcow2 format, or via `qemu-img convert`.
-#     Note that these formats are only valid with $with_snapshots being true.
-#   tar+size: (subvolumes only)
-#     A GNU tar stream containing just the inner contents of the subvolume.
-#     This does not distinguish between the contents of a privileged or
-#     unprivileged container. In other words, this is from the root user
-#     namespace's point of view with no uid-mapping in effect.
-#     As produced via `tar -C vm-100-disk-1.subvol -cpf TheOutputFile.dat .`
+=head3 $plugin->volume_export(\%scfg, $storeid, $fh, $volname, $format [, $snapshot, $base_snapshot, $with_snapshots])
+
+=head3 $plugin->volume_export(...)
+
+Exports a volume or a volume's C<$snapshot> into a file handle C<$fh> as a
+stream with a desired export C<$format>. See L<FORMATS> for all import/export
+formats.
+
+Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
+C<$with_snapshots> states whether the volume has snapshots overall.
+
+C<die>s in order to abort the export if there are (grave) problems, if the
+given C<$format> is not supported, or if exporting volumes is not supported as
+a whole.
+
+=cut
 
-# Export a volume into a file handle as a stream of desired format.
 sub volume_export {
     my ($class, $scfg, $storeid, $fh, $volname, $format, $snapshot, $base_snapshot, $with_snapshots) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->volume_export_formats(\%scfg, $storeid, $volname [, $snapshot, $base_snapshot, $with_snapshot])
+
+=head3 $plugin->volume_export_formats(...)
+
+B<OPTIONAL:> May be implemented in a storage plugin.
+
+Returns a list of supported export formats for the given volume or snapshot.
+
+Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
+C<$with_snapshots> states whether the volume has snapshots overall.
+
+If the storage does not support exporting volumes at all, and empty list should
+be returned.
+
+=cut
+
 sub volume_export_formats {
     my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
     croak "implement me in sub-class\n";
 }
 
-# Import data from a stream, creating a new or replacing or adding to an existing volume.
+=head3 $plugin->volume_import(\%scfg, $storeid, $fh, $volname, $format [, $snapshot, $base_snapshot, $with_snapshots, $allow_rename])
+
+=head3 $plugin->volume_import(...)
+
+B<OPTIONAL:> May be implemented in a storage plugin.
+
+Imports data with the given C<$format> from a stream / filehandle C<$fh>,
+creating a new volume, or replacing or adding to an existing one. Returns the
+volume ID of the imported volume.
+
+Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
+C<$with_snapshots> states whether the volume has snapshots overall. Renaming an
+existing volume may also optionally be allowed via C<$allow_rename>
+
+C<die>s if the import fails, if the given C<$format> is not supported, or if
+importing volumes is not supported as a whole.
+
+=cut
+
 sub volume_import {
     my ($class, $scfg, $storeid, $fh, $volname, $format, $snapshot, $base_snapshot, $with_snapshots, $allow_rename) = @_;
     croak "implement me in sub-class\n";
 }
 
+=head3 $plugin->volume_import_formats(\%scfg, $storeid, $volname [, $snapshot, $base_snapshot, $with_snapshot])
+
+=head3 $plugin->volume_import_formats(...)
+
+B<OPTIONAL:> May be implemented in a storage plugin.
+
+Returns a list of supported import formats for the given volume or snapshot.
+
+Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
+C<$with_snapshots> states whether the volume has snapshots overall.
+
+If the storage does not support importing volumes at all, and empty list should
+be returned.
+
+=cut
+
 sub volume_import_formats {
     my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
     croak "implement me in sub-class\n";
 }
-
 1;
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 4/8] pluginbase: document general plugin methods

[Maximiliano Sandoval]

Max Carrara <m.carrara@proxmox.com> writes:

> Add docstrings for the following methods:
> - check_connection
> - activate_storage
> - deactivate_storage
> - status
> - cluster_lock_storage
> - parse_volname
> - get_subdir
> - filesystem_path
> - path
> - find_free_diskname
>
> Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> ---
>  src/PVE/Storage/PluginBase.pm | 255 ++++++++++++++++++++++++++++++++++
>  1 file changed, 255 insertions(+)
>
> diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> index 5f7e6fd..8a61dc3 100644
> --- a/src/PVE/Storage/PluginBase.pm
> +++ b/src/PVE/Storage/PluginBase.pm
> @@ -317,53 +317,308 @@ sub private {
>
>  =cut
>
> +=head3 $plugin->check_connection($storeid, \%scfg)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Performs a connection check.
> +
> +This method is useful for plugins that require some kind of network connection
> +or similar and is called before C<L<< activate_storage()|/"$plugin->activate_storage($storeid, \%scfg, \%cache)" >>>.
> +
> +Returns C<1> by default and C<0> if the storage isn't online / reachable.

small nit: I think this might read a bit better if it was e.g.

```
Returns whether the storage is online and/or reachable.
```

or
```
Returns C<1> when the storage is online and/or reachable, and C<0> otherwise.
```

> +Returns C<1> by default and C<0> if the storage isn't online / reachable.

> +
> +C<die>s if an error occurs while performing the connection check.
> +
> +Note that this method's purpose is to simply verify that the storage is
> +I<reachable>, and not necessarily that authentication etc. succeeds.
> +
> +For example: If the storage is mainly accessed via TCP, you can try to simply
> +open a TCP connection to see if it's online:
> +
> +    # In custom module:
> +
> +    use PVE::Network;
> +    use parent qw(PVE::Storage::Plugin)
> +
> +    # [...]
> +
> +    sub check_connection {
> +	my ($class, $storeid, $scfg) = @_;
> +
> +	my $port = $scfg->{port} || 5432;
> +	return PVE::Network::tcp_ping($scfg->{server}, $port, 5);
> +    }
> +
> +=cut
> +
>  sub check_connection {
>      my ($class, $storeid, $scfg) = @_;
>
>      return 1;
>  }
>
> +=head3 $plugin->activate_storage($storeid, \%scfg, \%cache)
> +
> +=head3 $plugin->activate_storage(...)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Activates the storage, making it ready for further use.
> +
> +In essence, this method performs the steps necessary so that the storage can be
> +used by remaining parts of the system.
> +
> +In the case of file-based storages, this usually entails creating the directory
> +of the mountpoint, mounting the storage and then creating the directories for
> +the different content types that the storage has enabled. See
> +C<L<PVE::Storage::NFSPlugin>> and C<L<PVE::Storage::CIFSPlugin>> for examples
> +in that regard.
> +
> +Other types of storages would use this method for establishing a connection to
> +the storage and authenticating with it or similar. See C<L<PVE::Storage::ISCSIPlugin>>
> +for an example.
> +
> +If the storage doesn't need to be activated in some way, this method can be a
> +no-op.
> +
> +C<die>s in case of errors.
> +
> +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> +
> +=cut
> +
>  sub activate_storage {
>      my ($class, $storeid, $scfg, $cache) = @_;
>      croak "implement me in sub-class\n";
>  }
>
> +=head3 $plugin->deactivate_storage($storeid, \%scfg, \%cache)
> +
> +=head3 $plugin->deactivate_storage(...)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Deactivates the storage. Counterpart to C<L<< activate_storage()|/"$plugin->activate_storage(...)" >>>.
> +
> +This method is used to make the storage unavailable to the rest of the system,
> +which usually entails unmounting the storage, closing an established
> +connection, or similar.
> +
> +Does nothing by default.
> +
> +C<die>s in case of errors.
> +
> +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> +
> +B<NOTE:> This method is currently not used by our API due to historical
> +reasons.
> +
> +=cut
> +
>  sub deactivate_storage {
>      my ($class, $storeid, $scfg, $cache) = @_;
>
>      return;
>  }
>
> +=head3 $plugin->status($storeid, \%scfg, \%cache)
> +
> +=head3 $plugin->status(...)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Returns a list of scalars in the following form:
> +
> +    my ($total, $available, $used, $active) = $plugin->status($storeid, $scfg, $cache)
> +
> +This list contains the C<$total>, C<$available> and C<$used> storage capacity,
> +respectively, as well as whether the storage is C<$active> or not.
> +
> +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> +
> +=cut
> +
>  sub status {
>      my ($class, $storeid, $scfg, $cache) = @_;
>      croak "implement me in sub-class\n";
>  }
>
> +=head3 $plugin->cluster_lock_storage($storeid, $shared, $timeout, \&func, @param)
> +
> +=head3 $plugin->cluster_lock_storage(...)
> +
> +B<WARNING:> This method is provided by C<L<PVE::Storage::Plugin>> and
> +must be used as-is. It is merely documented here for informal purposes
> +and B<must not be overridden.>
> +
> +Locks the storage with the given C<$storeid> for the entire host and runs the
> +given C<\&func> with the given C<@param>s, unlocking the storage once finished.
> +If the storage is C<$shared>, it is instead locked on the entire cluster. If
> +the storage is already locked, wait for at most the number of seconds given by
> +C<$timeout>.
> +
> +This method is used to synchronize access to the given storage, preventing
> +simultaneous modification from multiple workers. This is necessary for
> +operations that modify data on the storage directly, like cloning and
> +allocating images.
> +
> +=cut
> +
>  sub cluster_lock_storage {
>      my ($class, $storeid, $shared, $timeout, $func, @param) = @_;
>      croak "implement me in sub-class\n";
>  }
>
> +=head3 $plugin->parse_volname($volname)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Parses C<$volname>, returning a list representing the parts encoded in
> +the volume name:
> +
> +    my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format)
> +	= $plugin->parse_volname($volname);
> +
> +Not all parts need to be included in the list. Those marked as I<optional>
> +in the list below may be set to C<undef> if not applicable.
> +
> +This method may C<die> in case of errors.
> +
> +=over
> +
> +=item * C<< $vtype >>
> +
> +The content type ("volume type") of the volume, e.g. C<"images">, C<"iso">,
> +etc.
> +
> +See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all content types.
> +
> +=item * C<< $name >>
> +
> +The display name of the volume. This is usually what the underlying storage
> +itself uses to address the volume.
> +
> +For example, disks for virtual machines that are stored on LVM thin pools are
> +named C<vm-100-disk-0>, C<vm-1337-disk-1>, etc. That would be the C<$name> in
> +this case.
> +
> +=item * C<< $vmid >> (optional)
> +
> +The ID of the guest that owns the volume.
> +
> +=item * C<< $basename >> (optional)
> +
> +The C<$name> of the volume this volume is based on. Whether this part
> +is returned or not depends on the plugin and underlying storage.
>

I would personally use something along the lines of:

```
Whether the parameter is defined or not depends...
```


> +Only applies to disk images.
> +
> +For example, on ZFS, if the VM is a linked clone, C<$basename> refers
> +to the C<$name> of the original disk volume that the parsed disk volume
> +corresponds to.
> +
> +=item * C<< $basevmid >> (optional)
> +
> +Equivalent to C<$basename>, except that C<$basevmid> refers to the
> +C<$vmid> of the original disk volume instead.
> +
> +=item * C<< $isBase >> (optional)
> +
> +Whether the volume is a base disk image.
> +
> +=item * C<< $format >>
> +
> +The format of the volume. If the volume is a VM disk (C<$vtype> is
> +C<"images">), this should be whatever format the disk is in. For most
> +other content types C<"raw"> should be used.
> +
> +See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all formats.
> +
> +=back
> +
> +=cut
> +
>  sub parse_volname {
>      my ($class, $volname) = @_;
>      croak "implement me in sub-class\n";
>  }
>
> +=head3 $plugin->get_subdir(\%scfg, $vtype)
> +
> +B<SITUATIONAL:> This method must be implemented for file-based storages.
> +
> +Returns the path to the sub-directory associated with the given content type
> +(C<$vtype>). See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all
> +content types.
> +
> +The default directory layout is predefined and must not be altered:
> +
> +    my $vtype_subdirs = {
> +	images   => 'images',
> +	rootdir  => 'private',
> +	iso      => 'template/iso',
> +	vztmpl   => 'template/cache',
> +	backup   => 'dump',
> +	snippets => 'snippets',
> +	import   => 'import',
> +    };
> +
> +See also: L<Storage: Directory|"https://pve.proxmox.com/wiki/Storage:_Directory">
> +
> +=cut
> +
>  sub get_subdir {
>      my ($class, $scfg, $vtype) = @_;
>      croak "implement me in sub-class\n";
>  }
>
> +=head3 $plugin->filesystem_path(\%scfg, $volname [, $snapname])
> +
> +=head3 $plugin->filesystem_path(...)
> +
> +B<SITUATIONAL:> This method must be implemented for file-based storages.
> +
> +Returns the absolute path on the filesystem for the given volume or snapshot.
> +In list context, returns path, guest ID and content type:
> +
> +    my $path = $plugin->filesystem_path($scfg, $volname, $snapname)
> +    my ($path, $vmid, $vtype) = $plugin->filesystem_path($scfg, $volname, $snapname)
> +
> +=cut
> +
>  sub filesystem_path {
>      my ($class, $scfg, $volname, $snapname) = @_;
>      croak "implement me in sub-class\n";
>  }
>
> +=head3 $plugin->path(\%scfg, $volname, $storeid [, $snapname])
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Returns a unique path that points to the given volume or snapshot depending
> +on the underlying storage implementation. For file-based storages, this
> +method should return the same as C<L<< filesystem_path()|/"$plugin->filesystem_path(...)" >>>.
> +
> +=cut
> +
>  sub path {
>      my ($class, $scfg, $volname, $storeid, $snapname) = @_;
>      croak "implement me in sub-class\n";
>  }
>
> +=head3 $plugin->find_free_diskname($storeid, \%scfg, $vmid [, $fmt, $add_fmt_suffix])
> +
> +=head3 $plugin->find_free_diskname(...)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Finds and returns the next available disk name, that is, the volume name
> +(C<$volname>) for a new disk image. Optionally, C<$fmt> specifies the
> +format of the disk image and C<$add_fmt_suffix> denotes whether C<$fmt>
> +should be added as suffix to the resulting name.
> +
> +=cut
> +
>  sub find_free_diskname {
>      my ($class, $storeid, $scfg, $vmid, $fmt, $add_fmt_suffix) = @_;
>      croak "implement me in sub-class\n";



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 5/8] pluginbase: document hooks

[Maximiliano Sandoval]

Max Carrara <m.carrara@proxmox.com> writes:

> Add docstrings for the following methods:
> - on_add_hook
> - on_update_hook
> - on_delete_hook
>
> Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> ---
>  src/PVE/Storage/PluginBase.pm | 85 ++++++++++++++++++++++++++++++-----
>  1 file changed, 74 insertions(+), 11 deletions(-)
>
> diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> index 8a61dc3..b3ce684 100644
> --- a/src/PVE/Storage/PluginBase.pm
> +++ b/src/PVE/Storage/PluginBase.pm
> @@ -626,29 +626,92 @@ sub find_free_diskname {
>
>  =head2 HOOKS
>
> +The following methods are called whenever a storage associated with the given
> +plugin is added, updated, or deleted. These methods are useful for:
> +
> +=over
> +
> +=item * Setting up certain prerequisites when adding the storage (and then
> +tearing them down again when the storage is deleted)
> +
> +=item * Handling sensitive parameters that shouldn't be written directly
> +to C</etc/pve/storage.cfg> and ought to be stored elsewhere
> +
> +=item * Ensuring that certain conditions in the configuration are being upheld
> +that cannot be done via the remaining API otherwise
> +
> +=back
> +
> +and more.
> +
> +=cut
> +
> +=head3 $plugin->on_add_hook($storeid, \%scfg, %param)
> +
> +=head3 $plugin->on_add_hook(...)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Called during the addition of a storage, before the new storage configuration
> +gets written.
> +
> +C<%param> contains additional key-value arguments, usually sensitive keys that
> +have been extracted from C<\%scfg> in order not to write them to the storage
> +configuration.
> +
> +C<die>s in order to abort the addition if there are (grave) problems.
> +
> +C</etc/pve/storage.cfg> is B<locked> when this method is called.

nit: Perhaps it would be more precise to say something like

```
C</etc/pve/storage.cfg> is B<locked> while this method is called.
```

to make it clear the lock will, at the very least, outlive whatever
happens inside on_add_hook. Similar for the rest.

> +
>  =cut
>
> -# called during addition of storage (before the new storage config got written)
> -# die to abort addition if there are (grave) problems
> -# NOTE: runs in a storage config *locked* context
>  sub on_add_hook {
>      my ($class, $storeid, $scfg, %param) = @_;
>      return undef;
>  }
>
> -# called during storage configuration update (before the updated storage config got written)
> -# die to abort the update if there are (grave) problems
> -# NOTE: runs in a storage config *locked* context
> +=head3 $plugin->on_update_hook($storeid, \%scfg, %param)
> +
> +=head3 $plugin->on_update_hook(...)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Called during the update of a storage configuration, before the new
> +configuration gets written.
> +
> +C<%param> contains additional key-value arguments, usually sensitive keys that
> +have been extracted from C<\%scfg> in order not to write them to the storage
> +configuration.
> +
> +C<die>s in order to abort the config update if there are (grave) problems.
> +
> +C</etc/pve/storage.cfg> is B<locked> when this method is called.
> +
> +=cut
> +
>  sub on_update_hook {
>      my ($class, $storeid, $scfg, %param) = @_;
>      return undef;
>  }
>
> -# called during deletion of storage (before the new storage config got written)
> -# and if the activate check on addition fails, to cleanup all storage traces
> -# which on_add_hook may have created.
> -# die to abort deletion if there are (very grave) problems
> -# NOTE: runs in a storage config *locked* context
> +=head3 $plugin->on_delete_hook($storeid, \%scfg)
> +
> +=head3 $plugin->on_delete_hook(...)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Called during the deletion of a storage, before the new storage configuration
> +gets written. Also gets called if the activation check during storage
> +addition fails in order to clean up all traces which
> +C<L<< on_add_hook()|/"$plugin->on_add_hook($storeid, \%scfg, %param)" >>>
> +may have created.
> +
> +C<die>s in order to abort the deletion of there are (very grave) problems.
> +
> +C</etc/pve/storage.cfg> is B<locked> when this method is called.
> +
> +=cut
> +
>  sub on_delete_hook {
>      my ($class, $storeid, $scfg) = @_;
>      return undef;



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 7/8] pluginbase: document volume operations

[Fabian Grünbichler]

On March 26, 2025 3:20 pm, Max Carrara wrote:
> Add docstrings for the following methods:
> - list_volumes
> - get_volume_attribute
> - update_volume_attribute
> - volume_size_info
> - volume_resize
> - volume_snapshot
> - volume_snapshot_info
> - volume_rollback_is_possible
> - volume_snapshot_rollback
> - volume_snapshot_delete
> - volume_snapshot_needs_fsfreeze
> - storage_can_replicate
> - volume_has_feature
> - map_volume
> - unmap_volume
> - activate_volume
> - deactivate_volume
> - rename_volume
> - prune_backups
> 
> Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> Co-authored-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
> ---
>  src/PVE/Storage/PluginBase.pm | 401 ++++++++++++++++++++++++++++++++--
>  1 file changed, 388 insertions(+), 13 deletions(-)
> 
> diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> index 37b1471..a1bfc8d 100644
> --- a/src/PVE/Storage/PluginBase.pm
> +++ b/src/PVE/Storage/PluginBase.pm
> @@ -861,108 +861,483 @@ sub free_image {
>  
>  =cut
>  
> +=head3 $plugin->list_volumes($storeid, \%scfg, $vmid, \@content_types)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Returns a list of volumes for the given guest whose content type is within the

this is wrong - what if $vmid is not set? or if content type is snippets
or one of other ones that don't have an associated guest/owner? or what
if there is no $vmid given, as in the example below?

> +given C<\@content_types>. If C<\@content_types> is empty, no volumes will be
> +returned. See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all content types.

here we are actually talking about content types for once (and this also
translates from content type "images" and "rootdir" to volume type
"images"! in PVE::Plugin!).

> +
> +    # List all backups for a guest
> +    my $backups = $plugin->list_volumes($storeid, $scfg, $vmid, ['backup']);
> +
> +    # List all containers and virtual machines on the storage
> +    my $guests = $plugin->list_volumes($storeid, $scfg, undef, ['rootdir', 'images'])

eh, this is a bad example? it doesn't list the guests, it lists their
volumes..

> +
> +The returned listref of hashrefs has the following structure:
> +
> +    [
> +	{
> +	    content => "images",
> +	    ctime => "1702298038", # creation time as unix timestamp
> +	    format => "raw",
> +	    size => 9663676416, # in bytes!
> +	    vmid => 102,
> +	    volid => "local-lvm:vm-102-disk-0",
> +	},
> +	# [...]
> +    ]
> +
> +Backups will also contain additional keys:
> +
> +    [
> +	{
> +	    content => "backup",
> +	    ctime => 1742405070, # creation time as unix timestamp
> +	    format => "tar.zst",
> +	    notes => "...", # comment that was entered when backup was created
> +	    size => 328906840, # in bytes!
> +	    subtype => "lxc", # "lxc" for containers, "qemu" for VMs
> +	    vmid => 106,
> +	    volid => "local:backup/vzdump-lxc-106-2025_03_19-18_24_30.tar.zst",
> +	},
> +	# [...]
> +    ]

soooo.. what is the interface here again? -> needs a (complete) schema
for the returned type, else how am I supposed to implement this?

> +
> +=cut
> +
>  sub list_volumes {
>      my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> -# Returns undef if the attribute is not supported for the volume.
> -# Should die if there is an error fetching the attribute.
> -# Possible attributes:
> -# notes     - user-provided comments/notes.
> -# protected - not to be removed by free_image, and for backups, ignored when pruning.
> +=head3 $plugin->get_volume_attribute(\%scfg, $storeid, $volname, $attribute)
> +
> +=head3 $plugin->get_volume_attribute(...)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Returns the value of the given C<$attribute> for a volume. If the attribute isn't
> +supported for the volume, returns C<undef>.
> +
> +C<die>s if there is an error fetching the attribute.
> +
> +B<Possible attributes:>
> +
> +=over
> +
> +=item * C<notes> (returns scalar)

scalar *string* ?
> +
> +User-provided comments / notes.
> +
> +=item * C<protected> (returns scalar)

scalar *boolean* ?

> +
> +When set to C<1>, the volume must not be removed by C<L<< free_image()|/"$plugin->free_image(...)" >>>.
> +Backups with C<protected> set to C<1> are ignored when pruning.

Backup volumes .. when calculating which backup volumes to prune.

> +
> +=back
> +
> +=cut
> +
>  sub get_volume_attribute {
>      my ($class, $scfg, $storeid, $volname, $attribute) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> -# Dies if the attribute is not supported for the volume.
> +=head3 $plugin->update_volume_attribute(\%scfg, $storeid, $volname, $attribute, $value)
> +
> +=head3 $plugin->update_volume_attribute(...)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Sets the value of the given C<$attribute> for a volume to C<$value>.
> +
> +C<die>s if the attribute isn't supported for the volume (or storage).
> +
> +For a list of supported attributes, see C<L<< get_volume_attribute()|/"$plugin->get_volume_attribute(...)" >>>.
> +
> +=cut
> +
>  sub update_volume_attribute {
>      my ($class, $scfg, $storeid, $volname, $attribute, $value) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->volume_size_info(\%scfg, $storeid, $volname [, $timeout])
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Returns information about the given volume's size. In scalar context, this returns
> +just the volume's size in bytes:
> +
> +    my $size = $plugin->volume_size_info($scfg, $storeid, $volname, $timeout)
> +
> +In list context, returns an array with the following structure:
> +
> +    my ($size, $format, $used, $parent, $ctime) = $plugin->volume_size_info(
> +	$scfg, $storeid, $volname, $timeout
> +    )
> +
> +where C<$size> is the size of the volume in bytes, C<$format> one of the possible
> +L<< formats listed in C<plugindata()>|/"$plugin->plugindata()" >>, C<$used> the
> +amount of space used in bytes, C<$parent> the (optional) name of the base volume
> +and C<$ctime> the creation time as unix timestamp.
> +
> +Optionally, a C<$timeout> may be provided after which the method should C<die>.
> +This timeout is best passed to other helpers which already support timeouts,
> +such as C<L<< PVE::Tools::run_command|PVE::Tools/run_command >>>.
> +
> +See also the C<L<< PVE::Storage::Plugin::file_size_info|PVE::Storage::Plugin/file_size_info >>> helper.

PVE::Storage::file_size_info (without Plugin::)!

> +
> +=cut
> +
>  sub volume_size_info {
>      my ($class, $scfg, $storeid, $volname, $timeout) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->volume_resize(\%scfg, $storeid, $volname, $size [, $running])
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Resizes a volume to the new C<$size> in bytes. Optionally, the guest that owns
> +the volume may be C<$running> (= C<1>).

the second sentence doesn't make any sense.

C<$running> indicates the guest is currently running.

but what does that mean/why should a plugin care?

> +
> +C<die>s in case of errors, or if the underlying storage implementation or the
> +volume's format doesn't support resizing.
> +
> +This function should not return any value. In previous versions the returned
> +value would be used to determine the new size of the volume or whether the
> +operation succeeded.

so since this documents the new version.. shouldn't we just drop the
last part?

> +
> +=cut
> +
>  sub volume_resize {
>      my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->volume_snapshot(\%scfg, $storeid, $volname, $snap)
> +
> +B<OPTIONAL:> May be implemented if the underlying storage supports snapshots.
> +
> +Takes a snapshot of a volume and gives it the name provided by C<$snap>.

this sounds like this is a two-step process..

Takes a snapshot called C<$snap> of the volume C<$volname>.

> +
> +C<die>s if the underlying storrage doesn't support snapshots or an error
> +occurs while taking a snapshot.

s/a/the
s/storrage/storage

> +
> +=cut
> +
>  sub volume_snapshot {
>      my ($class, $scfg, $storeid, $volname, $snap) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> -# Returns a hash with the snapshot names as keys and the following data:
> -# id        - Unique id to distinguish different snapshots even if the have the same name.
> -# timestamp - Creation time of the snapshot (seconds since epoch).
> -# Returns an empty hash if the volume does not exist.
> +=head3 $plugin->volume_snapshot_info(\%scfg, $storeid, $volname)
> +
> +Returns a hashref with the snapshot names as keys and the following structure:
> +
> +    {
> +        my_snapshot => {
> +            id => "01b7e668-58b4-6f42-9a5e-1944c2855c84",  # Unique id to distinguish different snapshots with the same name.
> +            timestamp => "1729841807",  # Creation time of the snapshot (seconds since epoch).
> +        },
> +	# [...]
> +    }
> +
> +Returns an empty hashref if the volume does not exist.

or dies if retrieving the snapshot information for the given volume
failed. ?

> +
> +=cut
> +
>  sub volume_snapshot_info {
>      my ($class, $scfg, $storeid, $volname) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> -# Asserts that a rollback to $snap on $volname is possible.
> -# If certain snapshots are preventing the rollback and $blockers is an array
> -# reference, the snapshot names can be pushed onto $blockers prior to dying.
> +=head3 $plugin->volume_rollback_is_possible(\%scfg, $storeid, $volname, $snap [, \@blockers])
> +
> +=head3 $plugin->volume_rollback_is_possible(...)
> +
> +B<OPTIONAL:> May be implemented if the underlying storage supports snapshots.
> +
> +Asserts whether a rollback to C<$snap> on C<$volname> is possible, C<die>s if
> +it isn't.
> +
> +Optionally, C<\@blockers> may be provided, which may contain the names of
> +snapshots that are preventing the rollback. Should any such snapshots exist,
> +they should be pushed to this listref pior to C<die>-ing. The caller may then
> +use this listref when handling errors.

If the optional paramater C<\@blockers> is provided, the names of any
snapshots blocking the rollback should be added to this listref prior to
C<die>-ing.

> +
> +=cut
> +
>  sub volume_rollback_is_possible {
>      my ($class, $scfg, $storeid, $volname, $snap, $blockers) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->volume_snapshot_rollback(\%scfg, $storeid, $volname, $snap)
> +
> +Performs a rollback to the given C<$snap>shot on C<$volname>.
> +
> +C<die>s in case of errors.
> +
> +=cut
> +
>  sub volume_snapshot_rollback {
>      my ($class, $scfg, $storeid, $volname, $snap) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->volume_snapshot_delete(\%scfg, $storeid, $volname, $snap [, $running])
> +
> +Deletes the C<$snap>shot of C<$volname>.
> +
> +C<die>s in case of errors.
> +
> +Optionally, the guest that owns the given volume may be C<$running> (= C<1>).

Again, this is phrased weird *and* gives no information to a potential
implementor of this API.. why should a plugin care?

> +
> +B<Deprecated:> The C<$running> parameter is deprecated and will be removed on the
> +next C<APIAGE> reset.

and this makes it even more confusing!

> +
> +=cut
> +
>  sub volume_snapshot_delete {
>      my ($class, $scfg, $storeid, $volname, $snap, $running) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->volume_snapshot_needs_fsfreeze()
> +
> +Returns whether filesystems on top of the volume need to flush their journal for
> +consistency before a snapshot is taken. See L<fsfreeze(8)>.
> +
> +This is needed for container mountpoints.

this doesn't really tell me what I should do here either.. (and it's
also a really ugly interace that we seemingly introduced because RBD
snapshots cannot be mounted RO otherwise?? does this really need to be
part of the plugin interface? Oo)

> +
> +=cut
> +
>  sub volume_snapshot_needs_fsfreeze {
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->storage_can_replicate(\%scfg, $storeid, $format)
> +
> +Returns whether volumes in a given C<$format> support replication.
> +
> +See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all disk formats.
> +

should probably note that replication is limited to ZFS at the moment in
any case?

> +=cut
> +
>  sub storage_can_replicate {
>      my ($class, $scfg, $storeid, $format) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->volume_has_feature(\%scfg, $feature, $storeid, $volname, $snapname [, $running, \%opts])
> +
> +=head3 $plugin->volume_has_feature(...)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Checks whether a volume C<$volname> or its snapshot C<$snapname> supports the
> +given C<$feature>, returning C<1> if it does and C<undef> otherwise. The guest
> +owning the volume may optionally be C<$running>.

that last sentence again doesn't make any sense the way it is phrased..

> +
> +C<$feature> may be one of the following:
> +
> +    clone      # linked clone is possible
> +    copy       # full clone is possible

actually, this is "bit-wise copying from" is possible, right?

> +    replicate  # replication is possible
> +    snapshot   # taking a snapshot is possible
> +    sparseinit # volume is sparsely initialized
> +    template   # conversion to base image is possible
> +    rename     # renaming volumes is possible
> +
> +Which features are available under which circumstances depends on multiple
> +factors, such as the underlying storage implementation, the format used, etc.

*and whether the corresponding guest is currently running*, which is the
whole point of that parameter ;)

this really needs to explain what the values for those feature keys
mean..

> +It's best to check out C<L<PVE::Storage::Plugin>> or C<L<PVE::Storage::ZFSPoolPlugin>>
> +for examples on how to handle features.
> +
> +Additional keys are given in C<\%opts>:
> +
> +=over
> +
> +=item * C<< valid_target_formats => [...] >>
> +
> +Listref of formats for the target of a copy/clone operation that the caller
> +could work with. The format of the given volume is always considered valid and
> +if no list is specified, all formats are considered valid.
> +
> +=back
> +
> +=cut
> +
>  sub volume_has_feature {
>      my ($class, $scfg, $feature, $storeid, $volname, $snapname, $running, $opts) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->map_volume($storeid, \%scfg, $volname [, $snapname])
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Maps the device asscoiated with a volume or a volume's snapshot to a filesystem

associated

> +path, returning the path on completion. This method is used by containers.

returning the path. (obviously on completion, how else?)

it's not only used by containers, so maybe drop that part and instead
note that it only needs to be implemented if `path` doesn't (always)
return such a path anyway.

> +
> +C<die>s in case of errors.
> +
> +C<L<< unmap_volume()|/"$plugin->unmap_volume(...)" >>> can be used to declare
> +how the device should be unmapped.

maybe just say that if you implement map, you should also implement
unmap?

> +
> +=cut
> +
>  sub map_volume {
>      my ($class, $storeid, $scfg, $volname, $snapname) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->unmap_volume($storeid, \%scfg, $volname [, $snapname])
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Unmaps the device associated to a volume or a volume's snapshot.
> +
> +C<die>s in case of errors.
> +
> +=cut
> +
>  sub unmap_volume {
>      my ($class, $storeid, $scfg, $volname, $snapname) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->activate_volume($storeid, \%scfg, $volname, $snapname [, \%cache])
> +
> +=head3 $plugin->activate_volume(...)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Activates a volume or its associated snapshot, making it available to the
> +system for further use. For example, this could mean activating an LVM volume,
> +mounting a ZFS dataset, checking whether the volume's file path exists, etc.
> +
> +C<die>s in case of errors or if an operation is not supported.
> +
> +If this isn't needed, the method should simply be a no-op.

If no activation is needed, 

> +
> +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> +
> +=cut
> +
>  sub activate_volume {
>      my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->deactivate_volume($storeid, \%scfg, $volname, $snapname [, \%cache])
> +
> +=head3 deactivate_volume(...)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Deactivates a volume or its associated snapshot, making it unavailable to
> +the system. For example, this could mean deactivating an LVM volume,
> +unmapping a Ceph/RBD device, etc.
> +
> +If this isn't needed, the method should simply be a no-op.

If deactivation is not needed/not possible,

should we note here that deactivation will not happen for every
activation, but only when we know for sure that the volume is no longer
needed, such as before it is removed or when its owner is migrated to
another node?

> +
> +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> +
> +=cut
> +
>  sub deactivate_volume {
>      my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->rename_volume(...)
> +
> +=head3 $plugin->rename_volume(\%scfg, $storeid, $source_volname, $target_vmid, $target_volname)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Renames the volume given by C<$source_volname> to C<$target_volname> and assigns
> +it to the guest C<$target_vmid>. Returns the volume ID of the renamed volume.
> +
> +This method is needed for the I<Change Owner> feature.
> +
> +C<die>s if the rename failed.
> +
> +=cut
> +
>  sub rename_volume {
>      my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->prune_backups(\%scfg, $storeid [, \%keep, $vmid, $type, $dryrun, \&logfunc])
> +
> +=head3 $plugin->prune_backups(...)
> +
> +Export a volume into a file handle as a stream with a desired format.

copy-paste mistake? ;)

> +
> +C<die>s if there are (grave) problems while pruning.
> +
> +This method may take several optional parameters:
> +
> +=over
> +
> +=item * C<< \%keep >>
> +
> +A hashref containing backup retention policies. It has the following structure:
> +
> +    {
> +	'keep-all'     => 1 # (optional) Whether to keep all backups.
> +	                    # Conflicts with the other options when true.
> +	'keep-last'    => N # (optional) Keep the last N backups.
> +	'keep-hourly'  => N # (optional) Keep backups for the last N different hours.
> +			    # If there is more than one backup for a single
> +			    # hour, only the latest one is kept.
> +	'keep-daily'   => N # (optional) Keep backups for the last N different days.
> +			    # If there is more than one backup for a single
> +			    # day, only the latest one is kept.
> +	'keep-weekly'  => N # (optional) Keep backups for the last N different weeks.
> +			    # If there is more than one backup for a single
> +			    # week, only the latest one is kept.
> +	'keep-monthly' => N # (optional) Keep backups for the last N different months.
> +			    # If there is more than one backup for a single
> +			    # month, only the latest one is kept.
> +	'keep-yearly'  => N # (optional) Keep backups for the last N different years.
> +			    # If there is more than one backup for a single
> +			    # year, only the latest one is kept.
> +    }

should we add a pointer to docs here? or to
PVE::Storage::prune_mark_backup_group? and should that or its body be
moved somewhere else?

> +
> +=item * C<< $vmid >>
> +
> +The guest's ID.
> +
> +=item * C<< $type >>
> +
> +The type of guest. When C<defined>, it can be one of C<"qemu"> or C<"lxc">.
> +If C<undefined>, both types of backups will be pruned.
> +
> +=item * C<< $dry_run >>
> +
> +Whether this is a dry run. If set to C<1> there won't be any change on the
> +underlying storage.
> +
> +=item * C<< \&logfunc >>
> +
> +A subroutine ref that can be used to log messages with the following signature:
> +
> +    $logfunc->($severity, $message)
> +
> +where $severity can be one of C<"info">, C<"err">, or C<"warn">.
> +
> +=back
> +
> +=cut
> +
>  sub prune_backups {
>      my ($class, $scfg, $storeid, $keep, $vmid, $type, $dryrun, $logfunc) = @_;
>      croak "implement me in sub-class\n";
> -- 
> 2.39.5
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
> 


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 6/8] pluginbase: document image operation methods

[Fabian Grünbichler]

On March 26, 2025 3:20 pm, Max Carrara wrote:
> Add documentation for the following methods:
> - list_images
> - create_base
> - clone_image
> - alloc_image
> - free_image
> 
> Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> Co-authored-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
> ---
>  src/PVE/Storage/PluginBase.pm | 111 ++++++++++++++++++++++++++++++++++
>  1 file changed, 111 insertions(+)
> 
> diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> index b3ce684..37b1471 100644
> --- a/src/PVE/Storage/PluginBase.pm
> +++ b/src/PVE/Storage/PluginBase.pm
> @@ -721,26 +721,137 @@ sub on_delete_hook {
>  
>  =cut
>  
> +=head3 $plugin->list_images($storeid, \%scfg [, $vmid, \@vollist, \%cache])
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Returns a listref of all disk images of a storage. If the storage does not
> +support storing disk images, returns an empty listref.
> +
> +Optionally, if C<\@vollist> is provided, return only disks whose volume ID is
> +within C<\@vollist>. Note that this usually has higher precedence than
> +C<$vmid>.

what does usually mean? what does $vmid do?

> +
> +C<die>s in case of errors.
> +
> +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> +
> +The returned listref has the following structure:
> +
> +    [
> +	{
> +	    ctime => "1689163322", # creation time as unix timestamp
> +	    format => "raw",
> +	    size => 8589934592, # in bytes!
> +	    vmid => 101,
> +	    volid => "local-lvm:base-101-disk-0", # volume ID, storage-specific
> +	},
> +	# [...]
> +    ]
> +
> +=cut
> +
>  sub list_images {
>      my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->create_base($storeid, \%scfg, $volname)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Creates a base volume from an existing volume, allowing the volume to be

this is misleading - it doesn't create a base volume from an existing
volume, it *converts* an existing volume into a base volume!

> +L<< cloned|/"$plugin->clone_image(...)" >>. This cloned volume (usually
> +a disk image) may then be used as a base for the purpose of creating linked

this is wrong? there is no cloned volume yet? and all of this only
applies to images and not other volumes?

> +clones. See L<C<PVE::Storage::LvmThinPlugin>> and
> +L<C<PVE::Storage::ZFSPoolPlugin>> for example implementations.
> +
> +On completion, returns the name of the new base volume (the new C<$volname>).
> +
> +This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
> +i.e. when the storage is B<locked>.

Shouldn't this say that it *should* be called in a locked context?

> +
> +=cut
> +
>  sub create_base {
>      my ($class, $storeid, $scfg, $volname) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->clone_image($scfg, $storeid, $volname, $vmid [, $snap])
> +
> +=head3 $plugin->clone_image(...)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.

not really? there's a feature guard for it, so only storage plugins that
support it should ever get it called anyway..

what does $vmid mean?

> +
> +Clones a disk image or a snapshot of an image, returning the name of the new
> +image (the new C<$volname>). Note that I<cloning> here means to create a linked
> +clone and not duplicating an image. See L<C<PVE::Storage::LvmThinPlugin>> and
> +L<C<PVE::Storage::ZFSPoolPlugin>> for example implementations.

then why not start with

"Creates a linked clone of an existing disk image or snapshot of an
image"

? maybe also include that unless the linked clone is 100% independent
from the base (only true for LVM thin atm?), the new volid should encode
the base->clone relation in the volname?

> +
> +C<die>s in case of an error of if the underlying storage doesn't support

s/of/or/

> +cloning images.
> +
> +This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
> +i.e. when the storage is B<locked>.

same as above

> +
> +=cut
> +
>  sub clone_image {
>      my ($class, $scfg, $storeid, $volname, $vmid, $snap) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->alloc_image($storeid, $scfg, $vmid, $fmt, $name, $size)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Allocates a disk image with the given format C<$fmt> and C<$size> in bytes,
> +returning the name of the new image (the new C<$volname>). See
> +C<L<< plugindata()|/"$plugin->plugindata()" >>> for all disk formats.
> +
> +Optionally, if given, set the name of the image to C<$name>. If C<$name> isn't
> +provided, the next name should be determined via C<L<< find_free_diskname()|/"$plugin->find_free_diskname(...)" >>>.

a suitable name should be automatically generated, unless we really want
to stabilize find_free_diskname as part of the API..

what does $vmid mean?

> +
> +C<die>s in case of an error of if the underlying storage doesn't support
> +allocating images.
> +
> +This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
> +i.e. when the storage is B<locked>.
> +
> +=cut
> +
>  sub alloc_image {
>      my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->free_image($storeid, $scfg, $volname [, $isBase, $format])
> +
> +=head3 $plugin->free_image(...)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Frees (deletes) the disk image given by C<$volname>. Optionally, the image may
> +be a base image (C<$isBase>) and have a certain C<$format>. See
> +C<L<< plugindata()|/"$plugin->plugindata()" >>> for all disk formats.
> +
> +If a cleanup is required after freeing the image, returns a reference to a
> +subroutine that performs the cleanup, and C<undef> otherwise.

might be a good idea to explain why? also, what do $isBase and $format
mean?

> +
> +C<die>s in case of errors, if the image has a L<< C<protected> attribute|/"$plugin->get_volume_attribute(...)" >>
> +(and may thus not be freed), or if the underlying storage implementation
> +doesn't support freeing images.
> +
> +This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
> +i.e. when the storage is B<locked>.
> +
> +B<NOTE:> The returned cleanup subroutine is called in a separate worker and
> +should L<< lock|/"$plugin->cluster_lock_storage(...)" >> the underlying storage
> +separately.

it should? or it must?

> +
> +=cut
> +
>  sub free_image {
>      my ($class, $storeid, $scfg, $volname, $isBase, $format) = @_;
>      croak "implement me in sub-class\n";
> -- 
> 2.39.5
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
> 


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 5/8] pluginbase: document hooks

[Fabian Grünbichler]

On March 26, 2025 3:20 pm, Max Carrara wrote:
> Add docstrings for the following methods:
> - on_add_hook
> - on_update_hook
> - on_delete_hook
> 
> Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> ---
>  src/PVE/Storage/PluginBase.pm | 85 ++++++++++++++++++++++++++++++-----
>  1 file changed, 74 insertions(+), 11 deletions(-)
> 
> diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> index 8a61dc3..b3ce684 100644
> --- a/src/PVE/Storage/PluginBase.pm
> +++ b/src/PVE/Storage/PluginBase.pm
> @@ -626,29 +626,92 @@ sub find_free_diskname {
>  
>  =head2 HOOKS
>  
> +The following methods are called whenever a storage associated with the given
> +plugin is added, updated, or deleted. These methods are useful for:
> +
> +=over
> +
> +=item * Setting up certain prerequisites when adding the storage (and then
> +tearing them down again when the storage is deleted)
> +
> +=item * Handling sensitive parameters that shouldn't be written directly
> +to C</etc/pve/storage.cfg> and ought to be stored elsewhere
> +
> +=item * Ensuring that certain conditions in the configuration are being upheld
> +that cannot be done via the remaining API otherwise
> +
> +=back
> +
> +and more.
> +
> +=cut
> +
> +=head3 $plugin->on_add_hook($storeid, \%scfg, %param)
> +
> +=head3 $plugin->on_add_hook(...)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Called during the addition of a storage, before the new storage configuration
> +gets written.

see comments on first patch ;)

> +
> +C<%param> contains additional key-value arguments, usually sensitive keys that
> +have been extracted from C<\%scfg> in order not to write them to the storage
> +configuration.
> +
> +C<die>s in order to abort the addition if there are (grave) problems.
> +
> +C</etc/pve/storage.cfg> is B<locked> when this method is called.

This method is called while C</etc/pve/storage.cfg> is locked.

Although I am not sure what extra information this provides?

same applies to the rest below..

> +
>  =cut
>  
> -# called during addition of storage (before the new storage config got written)
> -# die to abort addition if there are (grave) problems
> -# NOTE: runs in a storage config *locked* context
>  sub on_add_hook {
>      my ($class, $storeid, $scfg, %param) = @_;
>      return undef;
>  }
>  
> -# called during storage configuration update (before the updated storage config got written)
> -# die to abort the update if there are (grave) problems
> -# NOTE: runs in a storage config *locked* context
> +=head3 $plugin->on_update_hook($storeid, \%scfg, %param)
> +
> +=head3 $plugin->on_update_hook(...)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Called during the update of a storage configuration, before the new
> +configuration gets written.
> +
> +C<%param> contains additional key-value arguments, usually sensitive keys that
> +have been extracted from C<\%scfg> in order not to write them to the storage
> +configuration.
> +
> +C<die>s in order to abort the config update if there are (grave) problems.
> +
> +C</etc/pve/storage.cfg> is B<locked> when this method is called.
> +
> +=cut
> +
>  sub on_update_hook {
>      my ($class, $storeid, $scfg, %param) = @_;
>      return undef;
>  }
>  
> -# called during deletion of storage (before the new storage config got written)
> -# and if the activate check on addition fails, to cleanup all storage traces
> -# which on_add_hook may have created.
> -# die to abort deletion if there are (very grave) problems
> -# NOTE: runs in a storage config *locked* context
> +=head3 $plugin->on_delete_hook($storeid, \%scfg)
> +
> +=head3 $plugin->on_delete_hook(...)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Called during the deletion of a storage, before the new storage configuration
> +gets written. Also gets called if the activation check during storage
> +addition fails in order to clean up all traces which
> +C<L<< on_add_hook()|/"$plugin->on_add_hook($storeid, \%scfg, %param)" >>>
> +may have created.
> +
> +C<die>s in order to abort the deletion of there are (very grave) problems.
> +
> +C</etc/pve/storage.cfg> is B<locked> when this method is called.
> +
> +=cut
> +
>  sub on_delete_hook {
>      my ($class, $storeid, $scfg) = @_;
>      return undef;
> -- 
> 2.39.5
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
> 


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 4/8] pluginbase: document general plugin methods

[Fabian Grünbichler]

On March 26, 2025 3:20 pm, Max Carrara wrote:
> Add docstrings for the following methods:
> - check_connection
> - activate_storage
> - deactivate_storage
> - status
> - cluster_lock_storage
> - parse_volname
> - get_subdir
> - filesystem_path
> - path
> - find_free_diskname
> 
> Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> ---
>  src/PVE/Storage/PluginBase.pm | 255 ++++++++++++++++++++++++++++++++++
>  1 file changed, 255 insertions(+)
> 
> diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> index 5f7e6fd..8a61dc3 100644
> --- a/src/PVE/Storage/PluginBase.pm
> +++ b/src/PVE/Storage/PluginBase.pm
> @@ -317,53 +317,308 @@ sub private {
>  
>  =cut
>  
> +=head3 $plugin->check_connection($storeid, \%scfg)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Performs a connection check.
> +
> +This method is useful for plugins that require some kind of network connection
> +or similar and is called before C<L<< activate_storage()|/"$plugin->activate_storage($storeid, \%scfg, \%cache)" >>>.

This method can be implemented by network-based storages. It will be
called before storage activation attempts. Non-network storages should
not implement it.

> +
> +Returns C<1> by default and C<0> if the storage isn't online / reachable.

by default doesn't make any sense, even though I know what you mean.

> +
> +C<die>s if an error occurs while performing the connection check.
> +
> +Note that this method's purpose is to simply verify that the storage is
> +I<reachable>, and not necessarily that authentication etc. succeeds.
> +
> +For example: If the storage is mainly accessed via TCP, you can try to simply
> +open a TCP connection to see if it's online:
> +
> +    # In custom module:
> +
> +    use PVE::Network;
> +    use parent qw(PVE::Storage::Plugin)
> +
> +    # [...]
> +
> +    sub check_connection {
> +	my ($class, $storeid, $scfg) = @_;
> +
> +	my $port = $scfg->{port} || 5432;
> +	return PVE::Network::tcp_ping($scfg->{server}, $port, 5);
> +    }
> +
> +=cut
> +
>  sub check_connection {
>      my ($class, $storeid, $scfg) = @_;
>  
>      return 1;
>  }
>  
> +=head3 $plugin->activate_storage($storeid, \%scfg, \%cache)
> +
> +=head3 $plugin->activate_storage(...)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Activates the storage, making it ready for further use.
> +
> +In essence, this method performs the steps necessary so that the storage can be
> +used by remaining parts of the system.

this is superfluous (all of that information is already contained in the
previous sentence).

> +
> +In the case of file-based storages, this usually entails creating the directory
> +of the mountpoint, mounting the storage and then creating the directories for
> +the different content types that the storage has enabled. See
> +C<L<PVE::Storage::NFSPlugin>> and C<L<PVE::Storage::CIFSPlugin>> for examples
> +in that regard.
> +
> +Other types of storages would use this method for establishing a connection to
> +the storage and authenticating with it or similar. See C<L<PVE::Storage::ISCSIPlugin>>
> +for an example.
> +

I am not sure this examples provide much benefit in this verbosity..
maybe just a single sentence like

Common examples of activation might include mounting filesystems,
preparing directory trees or establishing sessions with network
storages.

> +If the storage doesn't need to be activated in some way, this method can be a
> +no-op.
> +
> +C<die>s in case of errors.

*Should* die?

> +
> +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> +
> +=cut
> +
>  sub activate_storage {
>      my ($class, $storeid, $scfg, $cache) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->deactivate_storage($storeid, \%scfg, \%cache)
> +
> +=head3 $plugin->deactivate_storage(...)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Deactivates the storage. Counterpart to C<L<< activate_storage()|/"$plugin->activate_storage(...)" >>>.
> +
> +This method is used to make the storage unavailable to the rest of the system,
> +which usually entails unmounting the storage, closing an established
> +connection, or similar.
> +
> +Does nothing by default.
> +
> +C<die>s in case of errors.

*Should*

> +
> +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> +
> +B<NOTE:> This method is currently not used by our API due to historical
> +reasons.
> +
> +=cut
> +
>  sub deactivate_storage {
>      my ($class, $storeid, $scfg, $cache) = @_;
>  
>      return;
>  }
>  
> +=head3 $plugin->status($storeid, \%scfg, \%cache)
> +
> +=head3 $plugin->status(...)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Returns a list of scalars in the following form:
> +
> +    my ($total, $available, $used, $active) = $plugin->status($storeid, $scfg, $cache)
> +
> +This list contains the C<$total>, C<$available> and C<$used> storage capacity,
> +respectively, as well as whether the storage is C<$active> or not.

might make sense to include the information which unit the returned
numbers should be in?

> +
> +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> +
> +=cut
> +
>  sub status {
>      my ($class, $storeid, $scfg, $cache) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->cluster_lock_storage($storeid, $shared, $timeout, \&func, @param)
> +
> +=head3 $plugin->cluster_lock_storage(...)
> +
> +B<WARNING:> This method is provided by C<L<PVE::Storage::Plugin>> and
> +must be used as-is. It is merely documented here for informal purposes
> +and B<must not be overridden.>

see my comment on the first patch.. this is not actually part of the
plugin interface, it is a helper that is implemented by
PVE::Storage::Plugin for historical reasons and should be moved
somewhere else and then removed from PVE::Storage::Plugin when we do the
next break of that inheritance hierarchy..

> +
> +Locks the storage with the given C<$storeid> for the entire host and runs the
> +given C<\&func> with the given C<@param>s, unlocking the storage once finished.
> +If the storage is C<$shared>, it is instead locked on the entire cluster. If
> +the storage is already locked, wait for at most the number of seconds given by
> +C<$timeout>.

Attempts to acquire a lock on C<$storeid>, waiting at most C<$timeout>
seconds before giving up. If successful, runs C<\&func> with the given
C<@param>s while holding the lock. If C<$shared> is set, the lock
operation will be cluster-wide (with a timeout for the execution of
C<\&func> of 60 seconds).

This method must be used to guard against concurrent modifications of
the storage by multiple tasks running at the same time, in particular
for actions such as:
- allocating new volumes (including clones) or snapshots
- deleting existing volumes or snapshots
- renaming existing volumes or snapshots

> +
> +This method is used to synchronize access to the given storage, preventing
> +simultaneous modification from multiple workers. This is necessary for
> +operations that modify data on the storage directly, like cloning and
> +allocating images.


> +
> +=cut
> +
>  sub cluster_lock_storage {
>      my ($class, $storeid, $shared, $timeout, $func, @param) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->parse_volname($volname)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Parses C<$volname>, returning a list representing the parts encoded in
> +the volume name:

s/parts/volume properties/ ?

should we incorporate the outcome of this discussion:

https://lore.proxmox.com/pve-devel/qdnb3vypbucbcf7ch2hsjbeo3hqb5bh4whoinl5xglggpt7b7t@igryfncdupdp/

?

> +
> +    my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format)
> +	= $plugin->parse_volname($volname);
> +
> +Not all parts need to be included in the list. Those marked as I<optional>
> +in the list below may be set to C<undef> if not applicable.
> +
> +This method may C<die> in case of errors.

s/may/should/ ?

> +
> +=over
> +
> +=item * C<< $vtype >>
> +
> +The content type ("volume type") of the volume, e.g. C<"images">, C<"iso">,
> +etc.

content type != volume type! the two are related obviously, but the
values are subtly different for images for historic reasons.

> +
> +See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all content types.
> +
> +=item * C<< $name >>
> +
> +The display name of the volume. This is usually what the underlying storage
> +itself uses to address the volume.

Ideally, this maps directly to some entity on the underlying storage.

> +
> +For example, disks for virtual machines that are stored on LVM thin pools are
> +named C<vm-100-disk-0>, C<vm-1337-disk-1>, etc. That would be the C<$name> in
> +this case.

This is rather imprecise, maybe just say

For example, a guest volume named C<vm-100-disk-0> stored on an LVM thin
pool will refer to a thin LV of the same name.

> +
> +=item * C<< $vmid >> (optional)
> +
> +The ID of the guest that owns the volume.

should maybe note that this must be set for `images` or `backup`, and
must not be set for the other vtypes?

> +
> +=item * C<< $basename >> (optional)
> +
> +The C<$name> of the volume this volume is based on. Whether this part
> +is returned or not depends on the plugin and underlying storage.

The C<$name> of this volume's base volume, if it is a linked clone.

?

> +Only applies to disk images.
> +
> +For example, on ZFS, if the VM is a linked clone, C<$basename> refers
> +to the C<$name> of the original disk volume that the parsed disk volume
> +corresponds to.

it is only used for that, and has this semantic across all storages,
right?

> +
> +=item * C<< $basevmid >> (optional)
> +
> +Equivalent to C<$basename>, except that C<$basevmid> refers to the
> +C<$vmid> of the original disk volume instead.

s/original/base

> +
> +=item * C<< $isBase >> (optional)
> +
> +Whether the volume is a base disk image.
> +
> +=item * C<< $format >>
> +
> +The format of the volume. If the volume is a VM disk (C<$vtype> is
> +C<"images">), this should be whatever format the disk is in. For most
> +other content types C<"raw"> should be used.

"images" is currently also for container volumes.. import volumes also
have a format..

> +
> +See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all formats.
> +
> +=back
> +
> +=cut
> +
>  sub parse_volname {
>      my ($class, $volname) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->get_subdir(\%scfg, $vtype)

I am not sure about this one long-term.. while the interface is generic
enough, it is only used for the helpers in PVE::Storage which are in
turn used for
- uploading/download things (templates, iso files, ..)
- creating backup archives (or rather, getting the path to the dump dir
  to create them)

> +
> +B<SITUATIONAL:> This method must be implemented for file-based storages.

s/file-based/dir-based

?

> +
> +Returns the path to the sub-directory associated with the given content type
> +(C<$vtype>). See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all
> +content types.

again, this is vtype not content type..

> +
> +The default directory layout is predefined and must not be altered:
> +
> +    my $vtype_subdirs = {
> +	images   => 'images',
> +	rootdir  => 'private',
> +	iso      => 'template/iso',
> +	vztmpl   => 'template/cache',
> +	backup   => 'dump',
> +	snippets => 'snippets',
> +	import   => 'import',
> +    };
> +
> +See also: L<Storage: Directory|"https://pve.proxmox.com/wiki/Storage:_Directory">
> +
> +=cut
> +
>  sub get_subdir {
>      my ($class, $scfg, $vtype) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->filesystem_path(\%scfg, $volname [, $snapname])
> +
> +=head3 $plugin->filesystem_path(...)
> +
> +B<SITUATIONAL:> This method must be implemented for file-based storages.

file/dir

> +
> +Returns the absolute path on the filesystem for the given volume or snapshot.
> +In list context, returns path, guest ID and content type:

content/volume

but - this is not actually part of the plugin API. it just looks like it
because Plugin's implementation of path (which is the actual API) calls
it, and thus plugins derived from that base implement or call it as
well.

the public interface if you want a file or blockdev path for a given
volid is PVE::Storage::abs_filesystem_path, which internally also calls
the plugin's `path` method, not the non-public `filesystem_path`.

> +
> +    my $path = $plugin->filesystem_path($scfg, $volname, $snapname)
> +    my ($path, $vmid, $vtype) = $plugin->filesystem_path($scfg, $volname, $snapname)
> +
> +=cut
> +
>  sub filesystem_path {
>      my ($class, $scfg, $volname, $snapname) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->path(\%scfg, $volname, $storeid [, $snapname])
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Returns a unique path that points to the given volume or snapshot depending
> +on the underlying storage implementation. For file-based storages, this
> +method should return the same as C<L<< filesystem_path()|/"$plugin->filesystem_path(...)" >>>.

this is inverted (see above). it should also note that if there is an
actual path corresponding to a volume, it should return that, but that
it can also return something else that Qemu understands in lieu of a
real path.

if multiple paths exist, it should return the most specific/unique one
to avoid collisions between multiple plugin instances of the same
storage type.

> +
> +=cut
> +
>  sub path {
>      my ($class, $scfg, $volname, $storeid, $snapname) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->find_free_diskname($storeid, \%scfg, $vmid [, $fmt, $add_fmt_suffix])
> +
> +=head3 $plugin->find_free_diskname(...)
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +Finds and returns the next available disk name, that is, the volume name
> +(C<$volname>) for a new disk image. Optionally, C<$fmt> specifies the
> +format of the disk image and C<$add_fmt_suffix> denotes whether C<$fmt>
> +should be added as suffix to the resulting name.

similarly, this is not actually part of the plugin API, it is an
implementation detail of the existing plugin hierarchy. it is currently
called by Plugin's clone, allocate and rename functionality, so if you
don't implement/override it you need to also override those (which you
likely need to do in any case..).

these two are actually a good example of what I meant with the messiness
of Plugin.pm leaking if we continue deriving *all* plugins from it,
including new external ones..

> +
> +=cut
> +
>  sub find_free_diskname {
>      my ($class, $storeid, $scfg, $vmid, $fmt, $add_fmt_suffix) = @_;
>      croak "implement me in sub-class\n";
> -- 
> 2.39.5
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
> 


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 3/8] pluginbase: document SectionConfig methods

[Fabian Grünbichler]

On March 26, 2025 3:20 pm, Max Carrara wrote:
> This commit adds docstrings for the relevant PVE::SectionConfig
> methods in the context of the storage plugin API.
> 
> Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> ---
>  src/PVE/Storage/PluginBase.pm | 194 +++++++++++++++++++++++++++++++++-
>  1 file changed, 192 insertions(+), 2 deletions(-)
> 
> diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> index 16977f3..5f7e6fd 100644
> --- a/src/PVE/Storage/PluginBase.pm
> +++ b/src/PVE/Storage/PluginBase.pm
> @@ -88,7 +88,7 @@ package PVE::Storage::PluginBase;
>  use strict;
>  use warnings;
>  
> -use Carp qw(croak);
> +use Carp qw(croak confess);
>  
>  use parent qw(PVE::SectionConfig);
>  
> @@ -100,27 +100,217 @@ use parent qw(PVE::SectionConfig);
>  
>  =cut
>  
> +=head3 $plugin->type()
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.

I am not sure whether we should do

s/implemented in/implemented by

for this whole thing and SectionConfig, but I won't note it for every
instance ;)

> +
> +This should return a string with which the plugin can be uniquely identified.

a string which uniquely identifies the plugin.

> +Any string is acceptable, as long as it's descriptive and you're sure it won't
> +conflict with another plugin. In the cases of built-in plugins, you will often
> +find the filesystem name or something similar being used.

I'd replace that with something like:

Commonly, the name of the storage technology or filesystem supported by
the plugin is used. If it is necessary to differentiate multiple
plugins for the same storage technology/filesystem, add a suffix
denoting the supported variant or feature set.

> +
> +See C<L<PVE::SectionConfig/type>> for more information.
> +
> +=cut
> +
>  sub type {
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->properties()
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +This method should be implemented if there are additional properties to be used
> +by your plugin. Since properties are global and may be reused across plugins,

s/used/registered

IMHO that conveys better what the difference between properties and
options are..

"global and may be reused across plugins" is kinda superfluous, maybe

"Since properties share a global namespace across all plugins"?

> +the names of properties must not collide with one another.

each property can only be registered once by a single plugin, but used
as an option by many.

> +
> +When implementing a third-party plugin, it is recommended to prefix properties
> +with some kind of identifier, like so:

Third-party plugins should therefore prefix any properties they register
with their plugin type, for example:

> +
> +    sub properties {
> +	return {
> +	    'example-storage-address' => {
> +		description => 'Host address of the ExampleStorage to connect to.',
> +		type => 'string',
> +	    },
> +	    'example-storage-pool' => {
> +		description => 'Name of the ExampleStorage pool to use.',
> +		type => 'string',
> +	    },
> +	    # [...]
> +	};
> +    }
> +
> +However, it is encouraged to reuse properties of inbuilt plugins whenever
> +possible. There are a few provided properties that are regarded as I<sensitive>
> +and will be treated differently in order to not expose them or write them as
> +plain text into configuration files. One such property fit for external use is
> +C<password>, which you can use to provide a password, API secret, or similar.

If possible, generic properties registered by built-in plugins should be
reused.

(inbuilt is a very weird, very British word ;))

the rest of that is kinda misplaced here and should be its own section
somewhere (it affects options just as much as properties). we should
probably allow registering sensitive properties for plugins?

> +
> +See C<L<PVE::SectionConfig/properties>> for more information.
> +
> +=cut
> +
>  sub properties {
>      my ($class) = @_;
>      return $class->SUPER::properties();
>  }
>  
> +=head3 $plugin->options()
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +This method returns a hash of the properties used by the plugin. Because
> +properties are shared among plugins, it is recommended to reuse any existing
> +ones of inbuilt plugins and only define custom properties via
> +C<L<< properties()|/"$plugin->properties()" >>> if necessary.

Everything but the first sentence can be a reference to the properties
part above..

> +
> +The properties a plugin uses are then declared as follows:

or just, "For example:" ? ;)

> +
> +    sub options {
> +	return {
> +	    nodes => { optional => 1 },
> +	    content => { optional => 1 },
> +	    disable => { optional => 1 },
> +	    'example-storage-pool' => { fixed => 1 },
> +	    'example-storage-address' => { fixed => 1 },
> +	    password => { optional => 1 },
> +	};
> +    }
> +
> +C<optional> properties are not required to be set. It is recommended to set

s/set/make or "declare"?

> +most properties optional by default, unless it I<really> is required to always
> +exist.

s/it/they
s/exist/be set

> +
> +C<fixed> properties can only be set when creating a new storage via the plugin
> +and cannot be changed afterwards.

s/creating a new storage/adding a new storage config entry/ ? a bit more
precise to differentiate it from actually "creating a storage" as in
formatting a disk/..

I am not sure what "via the plugin" is supposed to mean there, I think
it can just be dropped.

> +
> +See C<L<PVE::SectionConfig/options>> for more information.
> +
> +=cut
> +
>  sub options {
>      my ($class) = @_;
>      return $class->SUPER::options();
>  }
>  
> +=head3 $plugin->plugindata()
> +
> +B<REQUIRED:> Must be implemented in every storage plugin.
> +
> +This method returns a hash that specifies additional capabilities of the storage
> +plugin, such as what kinds of data may be stored on it or what VM disk formats

s/data/content or s/data/volumes

> +the storage supports. Additionally, defaults may also be set. This is done
> +through the C<content> and C<format> keys.

but how about re-organizing it a bit?

This method returns a hash declaring which content types and image
formats this plugin (or the underlying storage) supports.

and then describe each key?

> +
> +The C<content> key is used to declare B<supported content types> and their
> +defaults, while the C<format> key declares B<supported disk formats> and the
> +default disk format of the storage:

"and their defaults" sounds weird.

The 'content' key stores a list containing two hashes. The first list
element is a hash declaring which content types are supported by the
plugin. The second list element is a hash declaring which content types
are used by default if no explicit 'content' option is set on a plugin
instance.

and maybe link somewhere else for a description of the content type
values, so that there is a single place that various parts that need it
can link to?

and then

The 'format' key stores a list containing two hashes. The first list
element is a hash declaring which image formats are supported by the
storage plugin. The second list element is the default image format that
should be used for images on the storage if non is explicitly provided.

> +
> +    sub plugindata {
> +	return {
> +	    content => [
> +		# possible content types
> +		{
> +		    images => 1,   # disk images
> +		    rootdir => 1,  # container root directories
> +		    vztmpl => 1,   # container templates
> +		    iso => 1,      # iso images
> +		    backup => 1,   # vzdump backup files
> +		    snippets => 1, # snippets
> +		    import => 1,   # imports; see explanation below
> +		    none => 1,     # no content; see explanation below
> +		},
> +		# defaults if 'content' isn't explicitly set
> +		{
> +		    images => 1,   # store disk images by default
> +		    rootdir => 1   # store containers by default
> +		    # possible to add more or have no defaults
> +		}
> +	    ],
> +	    format => [
> +		# possible disk formats
> +		{
> +		    raw => 1,   # raw disk image
> +		    qcow2 => 1, # QEMU image format
> +		    vmdk => 1,  # VMware image format
> +		    subvol => 1 # subvolumes; see explanation below
> +		},
> +		"qcow2" # default if 'format' isn't explicitly set
> +	    ]
> +	    # [...]
> +	};
> +    }
> +
> +While the example above depicts a rather capable storage, the following
> +shows a simpler storage that can only be used for VM disks:

raw-formatted VM disks

> +
> +    sub plugindata {
> +	return {
> +	    content => [
> +		{ images => 1 },
> +	    ],
> +	    format => [
> +		{ raw => 1 },
> +		"raw",
> +	    ]
> +	};
> +    }
> +
> +Which content types and formats are supported depends on the underlying storage
> +implementation.
> +
> +B<Regarding C<import>:> The C<import> content type is used internally to
> +interface with virtual guests of foreign sources or formats. The corresponding
> +functionality has not yet been published to the public parts of the storage
> +API. Third-party plugins therefore should not declare this content type.

this should live somewhere where content types are described

> +
> +B<Regarding C<none>:> The C<none> content type denotes the I<absence> of other
> +types of content, i.e. this content type may only be set on a storage if no
> +other content type is set. This is used internally for storages that support
> +adding another storage "on top" of them; at the moment, this makes it possible
> +to set up an LVM (thin) pool on top of an iSCSI LUN. The corresponding
> +functionality has not yet been published to the public parts of the storage
> +API. Third-party plugins therefore should not declare this content type.

same

> +
> +B<Regarding C<subvol>:> The C<subvol> format is used internally to allow the
> +root directories of containers to use ZFS subvolumes (also known as
> +I<ZFS datasets>, not to be confused with I<ZVOLs>). Third-party plugins should
> +not declare this format type.

this is incomplete, but same.

> +
> +There is one more key, C<select_existing>, which is used internally for
> +ISCSI-related GUI functionality. Third-party plugins should not declare this
> +key.

this is okay here :)

> +
> +=cut
> +
>  sub plugindata {
>      my ($class) = @_;
>      return $class->SUPER::plugindata();
>  }
>  
> +=head3 $plugin->private()
> +
> +B<WARNING:> This method is provided by C<L<PVE::Storage::Plugin>> and
> +must be used as-is. It is merely documented here for informal purposes
> +and B<must not be overridden.>
> +
> +Returns a hash containing the tracked plugin metadata, most notably the
> +C<propertyList>, which contains all known properties of all plugins.
> +
> +C<L<PVE::Storage::Plugin>> uses this to predefine a lot of useful properties
> +that are relevant for all plugins. Core functionality such as defining
> +whether a storage is shared, which nodes may use it, whether a storage
> +is enabled or not, etc. are implemented via these properties.

this is not true and sounds like it is very interesting to look at/mess
with ;) in reality, it is part of SectionConfig machinery, so just say
that and that it must not be overridden and be done with it?

if PluginBase becomes the real base plugin then it and some other
helpers would need to move here anyway..

> +
> +See C<L<PVE::SectionConfig/private>> for more information.
> +
> +=cut
> +
>  sub private {
> -    croak "implement me in sub-class\n";
> +    confess "private() is provided by PVE::Storage::Plugin and must not be overridden";
>  }
>  
>  =head2 GENERAL
> -- 
> 2.39.5
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
> 


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 2/8] pluginbase: add high-level plugin API description

[Fabian Grünbichler]

On March 26, 2025 3:20 pm, Max Carrara wrote:
> Add a short paragraph in DESCRIPTION serving as an introduction as
> well as the GENERAL PARAMETERS and CACHING EXPENSIVE OPERATIONS
> sections.
> 
> These sections are added in order to avoid repeatedly describing the
> same parameters as well as to elaborate on / clarify a couple terms,
> e.g. what the $cache parameter does or what a volume in our case is.
> 
> Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> ---
>  src/PVE/Storage/PluginBase.pm | 77 +++++++++++++++++++++++++++++++++++
>  1 file changed, 77 insertions(+)
> 
> diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> index e56aa72..16977f3 100644
> --- a/src/PVE/Storage/PluginBase.pm
> +++ b/src/PVE/Storage/PluginBase.pm
> @@ -4,6 +4,83 @@ C<PVE::Storage::PluginBase> - Storage Plugin API Interface
>  
>  =head1 DESCRIPTION
>  
> +This module documents the public Storage Plugin API of PVE and serves
> +as a base for C<L<PVE::Storage::Plugin>>. Plugins must B<always> inherit from
> +C<L<PVE::Storage::Plugin>>, as this module is for documentation purposes
> +only.

does this make sense? if we now provide a clean base for the structure
of plugins, why shouldn't plugins be able to use that, but instead have
to inherit from PVE::Storage::Plugin which has a lot of extra stuff that
makes things messy?

granted, switching over to load from PluginBase could be done as a
follow up or with 9.0 (or not at all, if there is a rationale)..

after this series we have:

SectionConfig
-> PluginBase (not an actual base plugin w.r.t. SectionConfig, and not
something you base plugins on as a result)
--> Plugin (a combination of base plugin and base of all our
directory-based plugins)
---> other plugins, including third party ones

which seems unfortunate, even if the contents of PluginBase are helpful
when implementing your own..

IMHO this should either be

SectionConfig
-> PluginBase (actual base plugin, with SectionConfig implementations
moved over from current Plugin.pm as needed)
--> PluginTemplate (what's currently PluginBase in this series - nicely
documented parent of third party plugins, not actually registered, just
contains the storage.cfg interface without the low-level SectionConfig
things)
---> NewThirdPartyPlugin (clean slate implementation just using the
nicely documented interfaces, guaranteed to not use any helpers from
Plugin since it's not a (grand)parent)
--> Plugin (base of built-in plugins, probably base of existing third
party plugins, should ideally have a different name in the future!)
---> DirPlugin
---> ...
---> ExistingThirdPartyPlugin (this might rely on helpers from Plugin,
so we can't just rename that one unless we wait for 9.0)

or

SectionConfig
-> PluginBase (actual base plugin + docs of Plugin API)
--> Plugin (base of our plugins and existing third party ones, dir-related helpers, ..)
---> other plugins, including third party ones
--> NewThirdPartyPlugin (clean slate as above)

side-note: should we mention somewhere that plugin code is not called
directly (pre-existing exceptions that we want to get rid off ignored),
but that PVE::Storage is the "gateway"/interface for it?

> +
> +=head2 DEFAULT IMPLEMENTATIONS
> +
> +C<L<PVE::Storage::Plugin>> implements most of the methods listed in
> +L</PLUGIN INTERFACE METHODS> by default. These provided implementations are
> +tailored towards file-based storages and can therefore be used as-is in that
> +case. Plugins for other kinds of storages will most likely have to adapt each
> +method for their individual use cases.

see above

> +
> +=head2 GENERAL PARAMETERS
> +
> +The parameter naming throughout the code is kept as consistent as possible.
> +Therefore, common reappearing subroutine parameters are listed here for
> +convenience:
> +
> +=over
> +
> +=item * C<< \%scfg >>
> +
> +The storage configuration associated with the given C<$storeid>. This is a
> +reference to a hash that represents the section associated with C<$storeid> in
> +C</etc/pve/storage.cfg>.
> +
> +=item * C<< $storeid >>
> +
> +The ID of the storage. This ID is user-provided; the IDs for existing
> +storages can be found in the UI via B<< Datacenter > Storage >>.

The unique ID of the storage, as used in C</etc/pve/storage.cfg> and as
part of every volid.

this is not really user-provided in most cases (that makes it sound like
you have to be super careful when handling it to prevent exploits),
although the user of course initially named the section like that ;)

> +
> +=item * C<< $volname >>
> +
> +The name of a volume. The term I<volume> can refer to a disk image, an ISO
> +image, a backup, etc. depending on the content type of the volume.

backup archive/snapshot ?

should we list all types here?

> +
> +=item * C<< $volid >>
> +
> +The ID of a volume, which is essentially C<"${storeid}:${volname}">. Less used
> +within the plugin API, but nevertheless relevant.

s/essentially// (it is exactly that, not essentially)

the second sentence doesn't really add much, if we want to keep it, then
I suggest replacing it with something like

Frequently used in guest-specific API calls and passed to
PVE::Storage::parse_volume_id to split it into storeid and volname parts
before calling into storage plugin code.

> +
> +=item * C<< $snapname >> (or C<< $snap >>)
> +
> +The name of a snapshot associated with the given C<$volname>.

what is "given" here? the phrasing doesn't really make sense IMHO ;)

The name of a snapshot of a volume.

> +
> +=item * C<< \%cache >>
> +
> +See L<CACHING EXPENSIVE OPERATIONS>.
> +
> +=item * C<< $vmid >>
> +
> +The ID of a guest (so, either of a VM or a container).
> +
> +=back
> +
> +=head2 CACHING EXPENSIVE OPERATIONS
> +
> +Certain methods take a C<\%cache> as parameter, which is used to store the
> +results of time-consuming / expensive operations specific to certain plugins.

this is not really accurate. the cache is used to store different
information, currently (according to a quick grep through the code):

for storages (activate_storage/activate_storage_list):
- parsed /proc/mounts for various dir-based storages where the plugin
  handles mounting, to speed up "is storage already mounted" checks
- udev sequence numbers (to know whether udev needs settling after
  activating a storage)
- a flag that a storage was activated (as an optimization to skip
  "is it already active" checks)

this already mixes state of PVE::Storage with plugin-specific data,
which is a mess.

when listing images across multiple storages:
- vgs/lvs output (LVM plugins)

that one we've eliminated from most plugins, since it rarely makes
sense. LVM is a bit of an exception there, as we query global LVM state
that is valid for multiple storage instances. this maybe could be
changed to drop the usage as well, and instead query VG-specific
information only?

> +The exact lifetime of the cached data is B<unspecified>, since it depends on
> +the exact usage of the C<L<PVE::Storage>> API, but can generally be assumed to
> +be B<short-lived>.

this is not really something that helps me as a plugin developer - what
kind of information can/should/must I store in the cache (or not)? how
is it structured?

> +
> +For example, the C<L<PVE::Storage::LVMPlugin>> uses this cache to store the
> +parsed output of the C<lvs> command. Since the number and precise information
> +about LVM's logical volumes is unlikely to change within a short time, other
> +API methods can then use this data in order to avoid repeatedly calling and
> +parsing the output of C<lvs>.

this reads like the cache is used across PVE API calls (although maybe
you meant "storage plugin API"?). the original (flawed) reason was/is
that if we activate 20 volumes on a single storage for a certain task,
then it is enough to check with LVM once and re-use the (slightly stale)
data. this since got eliminated from most plugins as it was not working.
the other use case (see above) is if we (potentially) activate 10
storages, we can check once what is already mounted and re-use that for
the subsequent 9 activations. I am not sure this is worth it either to
be honest.

> +
> +Third-party plugin developers should ensure that the data stored and retrieved
> +is specific to their plugin, and not rely on the data that other plugins might
> +store in C<\%cache>. Furthermore, the names of keys should be rather unique in
> +the sense that they're unlikely to conflict with any future keys that may be
> +introduced internally. To illustrate, e.g. C<myplugin_mounts> should be used
> +instead of a plain C<mounts> key.

and this here clearly shows that the current interface is bogus and
under-specified in any case. so until that is fixed, this here should
read "ignore the cache parameter it is for internal use only". if a
plugin needs to cache things internally, it can do so anyway based on
its own criteria..


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 1/8] pluginbase: introduce PVE::Storage::PluginBase with doc scaffold

[Fabian Grünbichler]

On March 26, 2025 3:20 pm, Max Carrara wrote:
> Add PVE::Storage::PluginBase, which defines stubs for all methods that
> storage plugins should implement in order to conform to our plugin
> API. This makes it much easier for (third-party) developers to see
> which methods should be implemented.
> 
> PluginBase is inserted into the inheritance chain between
> PVE::Storage::Plugin and PVE::SectionConfig instead of letting the
> Plugin module inherit from SectionConfig directly. This keeps the
> inheritance chain linear, avoiding multiple inheritance.
> 
> Also provide a scaffold for documentation. Preserve pre-existing
> comments for context's sake.
> 
> Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> ---
>  src/PVE/Storage/Makefile      |   1 +
>  src/PVE/Storage/Plugin.pm     |   2 +-
>  src/PVE/Storage/PluginBase.pm | 328 ++++++++++++++++++++++++++++++++++
>  3 files changed, 330 insertions(+), 1 deletion(-)
>  create mode 100644 src/PVE/Storage/PluginBase.pm
> 
> diff --git a/src/PVE/Storage/Makefile b/src/PVE/Storage/Makefile
> index ce3fd68..f2cdb66 100644
> --- a/src/PVE/Storage/Makefile
> +++ b/src/PVE/Storage/Makefile
> @@ -1,6 +1,7 @@
>  SOURCES= \
>  	Common.pm \
>  	Plugin.pm \
> +	PluginBase.pm \
>  	DirPlugin.pm \
>  	LVMPlugin.pm \
>  	NFSPlugin.pm \
> diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
> index 65cf43f..df6882a 100644
> --- a/src/PVE/Storage/Plugin.pm
> +++ b/src/PVE/Storage/Plugin.pm
> @@ -19,7 +19,7 @@ use PVE::Storage::Common;
>  
>  use JSON;
>  
> -use base qw(PVE::SectionConfig);
> +use parent qw(PVE::Storage::PluginBase);
>  
>  use constant KNOWN_COMPRESSION_FORMATS =>  ('gz', 'lzo', 'zst', 'bz2');
>  use constant COMPRESSOR_RE => join('|', KNOWN_COMPRESSION_FORMATS);
> diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> new file mode 100644
> index 0000000..e56aa72
> --- /dev/null
> +++ b/src/PVE/Storage/PluginBase.pm
> @@ -0,0 +1,328 @@
> +=head1 NAME
> +
> +C<PVE::Storage::PluginBase> - Storage Plugin API Interface
> +
> +=head1 DESCRIPTION
> +
> +=cut
> +
> +package PVE::Storage::PluginBase;
> +
> +use strict;
> +use warnings;
> +
> +use Carp qw(croak);
> +
> +use parent qw(PVE::SectionConfig);
> +
> +=head1 PLUGIN INTERFACE METHODS
> +
> +=cut
> +
> +=head2 PLUGIN DEFINITION
> +
> +=cut
> +
> +sub type {
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub properties {
> +    my ($class) = @_;
> +    return $class->SUPER::properties();
> +}
> +
> +sub options {
> +    my ($class) = @_;
> +    return $class->SUPER::options();
> +}
> +
> +sub plugindata {
> +    my ($class) = @_;
> +    return $class->SUPER::plugindata();
> +}
> +
> +sub private {
> +    croak "implement me in sub-class\n";
> +}
> +
> +=head2 GENERAL
> +
> +=cut
> +
> +sub check_connection {
> +    my ($class, $storeid, $scfg) = @_;
> +
> +    return 1;
> +}
> +
> +sub activate_storage {
> +    my ($class, $storeid, $scfg, $cache) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub deactivate_storage {
> +    my ($class, $storeid, $scfg, $cache) = @_;
> +
> +    return;
> +}
> +
> +sub status {
> +    my ($class, $storeid, $scfg, $cache) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub cluster_lock_storage {
> +    my ($class, $storeid, $shared, $timeout, $func, @param) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub parse_volname {
> +    my ($class, $volname) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub get_subdir {
> +    my ($class, $scfg, $vtype) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub filesystem_path {
> +    my ($class, $scfg, $volname, $snapname) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub path {
> +    my ($class, $scfg, $volname, $storeid, $snapname) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub find_free_diskname {
> +    my ($class, $storeid, $scfg, $vmid, $fmt, $add_fmt_suffix) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +=head2 HOOKS
> +
> +=cut
> +
> +# called during addition of storage (before the new storage config got written)

called when adding a storage config entry, before the new config gets written

> +# die to abort addition if there are (grave) problems
> +# NOTE: runs in a storage config *locked* context
> +sub on_add_hook {
> +    my ($class, $storeid, $scfg, %param) = @_;
> +    return undef;
> +}
> +
> +# called during storage configuration update (before the updated storage config got written)

called when updating a storage config entry, before the updated config
gets written

> +# die to abort the update if there are (grave) problems
> +# NOTE: runs in a storage config *locked* context
> +sub on_update_hook {
> +    my ($class, $storeid, $scfg, %param) = @_;
> +    return undef;
> +}
> +
> +# called during deletion of storage (before the new storage config got written)
> +# and if the activate check on addition fails, to cleanup all storage traces
> +# which on_add_hook may have created.

called when deleting a storage config entry, before the new storage
config gets written.

also called as part of error handling when undoing the addition of a new
storage config entry.

> +# die to abort deletion if there are (very grave) problems
> +# NOTE: runs in a storage config *locked* context
> +sub on_delete_hook {
> +    my ($class, $storeid, $scfg) = @_;
> +    return undef;
> +}
> +
> +=head2 IMAGE OPERATIONS
> +

should this describe what IMAGES are in the context of PVE? else as a
newcomer the difference between IMAGE here and VOLUME below might not
be clear..

> +=cut
> +
> +sub list_images {
> +    my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub create_base {
> +    my ($class, $storeid, $scfg, $volname) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub clone_image {
> +    my ($class, $scfg, $storeid, $volname, $vmid, $snap) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub alloc_image {
> +    my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub free_image {
> +    my ($class, $storeid, $scfg, $volname, $isBase, $format) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +=head2 VOLUME OPERATIONS

see above

> +
> +=cut
> +
> +sub list_volumes {
> +    my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +# Returns undef if the attribute is not supported for the volume.
> +# Should die if there is an error fetching the attribute.
> +# Possible attributes:
> +# notes     - user-provided comments/notes.
> +# protected - not to be removed by free_image, and for backups, ignored when pruning.
> +sub get_volume_attribute {
> +    my ($class, $scfg, $storeid, $volname, $attribute) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +# Dies if the attribute is not supported for the volume.
> +sub update_volume_attribute {
> +    my ($class, $scfg, $storeid, $volname, $attribute, $value) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub volume_size_info {
> +    my ($class, $scfg, $storeid, $volname, $timeout) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub volume_resize {
> +    my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub volume_snapshot {
> +    my ($class, $scfg, $storeid, $volname, $snap) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +# Returns a hash with the snapshot names as keys and the following data:
> +# id        - Unique id to distinguish different snapshots even if the have the same name.
> +# timestamp - Creation time of the snapshot (seconds since epoch).
> +# Returns an empty hash if the volume does not exist.
> +sub volume_snapshot_info {
> +    my ($class, $scfg, $storeid, $volname) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +# Asserts that a rollback to $snap on $volname is possible.
> +# If certain snapshots are preventing the rollback and $blockers is an array
> +# reference, the snapshot names can be pushed onto $blockers prior to dying.
> +sub volume_rollback_is_possible {
> +    my ($class, $scfg, $storeid, $volname, $snap, $blockers) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub volume_snapshot_rollback {
> +    my ($class, $scfg, $storeid, $volname, $snap) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub volume_snapshot_delete {
> +    my ($class, $scfg, $storeid, $volname, $snap, $running) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub volume_snapshot_needs_fsfreeze {
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub storage_can_replicate {
> +    my ($class, $scfg, $storeid, $format) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub volume_has_feature {
> +    my ($class, $scfg, $feature, $storeid, $volname, $snapname, $running, $opts) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub map_volume {
> +    my ($class, $storeid, $scfg, $volname, $snapname) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub unmap_volume {
> +    my ($class, $storeid, $scfg, $volname, $snapname) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub activate_volume {
> +    my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub deactivate_volume {
> +    my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub rename_volume {
> +    my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub prune_backups {
> +    my ($class, $scfg, $storeid, $keep, $vmid, $type, $dryrun, $logfunc) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +=head2 IMPORTS AND EXPORTS
> +
> +=cut
> +
> +# Import/Export interface:
> +#   Any path based storage is assumed to support 'raw' and 'tar' streams, so
> +#   the default implementations will return this if $scfg->{path} is set,
> +#   mimicking the old PVE::Storage::storage_migrate() function.
> +#
> +# Plugins may fall back to PVE::Storage::Plugin::volume_{export,import}...
> +#   functions in case the format doesn't match their specialized
> +#   implementations to reuse the raw/tar code.
> +#
> +# Format specification:
> +#   The following formats are all prefixed with image information in the form
> +#   of a 64 bit little endian unsigned integer (pack('Q<')) in order to be able
> +#   to preallocate the image on storages which require it.
> +#
> +#   raw+size: (image files only)
> +#     A raw binary data stream such as produced via `dd if=TheImageFile`.
> +#   qcow2+size, vmdk: (image files only)
> +#     A raw qcow2/vmdk/... file such as produced via `dd if=some.qcow2` for
> +#     files which are already in qcow2 format, or via `qemu-img convert`.
> +#     Note that these formats are only valid with $with_snapshots being true.
> +#   tar+size: (subvolumes only)
> +#     A GNU tar stream containing just the inner contents of the subvolume.
> +#     This does not distinguish between the contents of a privileged or
> +#     unprivileged container. In other words, this is from the root user
> +#     namespace's point of view with no uid-mapping in effect.
> +#     As produced via `tar -C vm-100-disk-1.subvol -cpf TheOutputFile.dat .`
> +
> +# Export a volume into a file handle as a stream of desired format.
> +sub volume_export {
> +    my ($class, $scfg, $storeid, $fh, $volname, $format, $snapshot, $base_snapshot, $with_snapshots) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub volume_export_formats {
> +    my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +# Import data from a stream, creating a new or replacing or adding to an existing volume.
> +sub volume_import {
> +    my ($class, $scfg, $storeid, $fh, $volname, $format, $snapshot, $base_snapshot, $with_snapshots, $allow_rename) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +sub volume_import_formats {
> +    my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
> +    croak "implement me in sub-class\n";
> +}
> +
> +1;
> -- 
> 2.39.5
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
> 


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 8/8] pluginbase: document import and export methods

[Fabian Grünbichler]

CCing Fiona in case of further input w.r.t. export/import things

On March 26, 2025 3:20 pm, Max Carrara wrote:
> Adapt the previous description, slightly rewording it and formatting
> it for POD under the IMPORTS AND EXPORTS section.
> 
> Also add docstrings for the following methods:
> - volume_export
> - volume_export_formats
> - volume_import
> - volume_import_formats
> 
> Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> Co-authored-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
> ---
>  src/PVE/Storage/PluginBase.pm | 134 ++++++++++++++++++++++++++--------
>  1 file changed, 105 insertions(+), 29 deletions(-)
> 
> diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> index a1bfc8d..cfa5087 100644
> --- a/src/PVE/Storage/PluginBase.pm
> +++ b/src/PVE/Storage/PluginBase.pm
> @@ -1345,55 +1345,131 @@ sub prune_backups {
>  
>  =head2 IMPORTS AND EXPORTS
>  
> +Any path-based storage is assumed to support C<raw> and C<tar> streams, so
> +the default implementations in C<L<PVE::Storage::Plugin>> will return this if
> +C<< $scfg->{path} >> is set (thereby mimicking the old C<< PVE::Storage::storage_migrate() >>
> +function).

meh, I don't think this makes sense if we want to document the
interface, we should document the interface, and not the implementation
of our plugin hierarchy..

> +
> +Plugins may fall back to methods like C<volume_export>, C<volume_import>, etc.
> +of C<L<PVE::Storage::Plugin>> in case the format doesn't match their
> +specialized implementations to reuse the C<raw>/C<tar> code.

and these should move to some helper module and be used by
PVE::Storage::Plugin, if we want to allow external plugins to re-use
them as basic implementation..

> +
> +=head3 FORMATS
> +
> +The following formats are all prefixed with image information in the form of a
> +64 bit little endian unsigned integer (C<< pack('Q\<') >>) in order to be able
> +to preallocate the image on storages which require it.

"image information" should maybe be a bit more precise, it's easy to
guess from the name that information==size, but why not spell it out?

> +
> +=over
> +
> +=item * C<< raw+size >> (image files only)
> +
> +A raw binary data stream as produced via C<< dd if=$IMAGE_FILE >>.

A binary data stream containing a volume's logical raw data, for example
as produced via .. if the image is already in raw format, *or via
qemu-img convert* if not.

> +
> +=item * C<< qcow2+size >>, C<< vmdk >> (image files only)

missing +size for vmdk

> +
> +A raw C<qcow2>/C<vmdk>/... file as produced via C<< dd if=some_file.qcow2 >>
> +for files which are already in C<qcow2> format, or via C<qemu-img convert>.

"A raw qcow2/vmdk/.. file" is confusing..

A binary data stream containing the qcow2/vmdk-formatted contents of a
qcow2/vmdk file as produced via ..

the qemu-img convert part got moved to the wrong format, it's not needed
to produce a qcow2+size stream for raw files (we don't do that), but to
produce a raw+size stream from a qcow2 file, see above.

> +
> +B<NOTE:> These formats are only valid with C<$with_snapshots> being true (C<1>).

that's not strictly speaking true, but an implementation detail of the
current implementation. what is true is that raw+size can not contain
snapshots for obvious reasons.

> +
> +=item * C<< tar+size >> (subvolumes only)
> +
> +A GNU C<tar> stream containing just the inner contents of the subvolume. This
> +does not distinguish between the contents of a privileged or unprivileged
> +container. In other words, this is from the root user namespace's point of view
> +with no uid-mapping in effect. As produced via e.g.
> +C<< tar -C vm-100-disk-1.subvol -cpf output_file.dat . >>

what is "inner"? should/must the content be relative or absolute?
anchored? ...

> +
> +=back
> +
>  =cut
>  
> -# Import/Export interface:
> -#   Any path based storage is assumed to support 'raw' and 'tar' streams, so
> -#   the default implementations will return this if $scfg->{path} is set,
> -#   mimicking the old PVE::Storage::storage_migrate() function.
> -#
> -# Plugins may fall back to PVE::Storage::Plugin::volume_{export,import}...
> -#   functions in case the format doesn't match their specialized
> -#   implementations to reuse the raw/tar code.
> -#
> -# Format specification:
> -#   The following formats are all prefixed with image information in the form
> -#   of a 64 bit little endian unsigned integer (pack('Q<')) in order to be able
> -#   to preallocate the image on storages which require it.
> -#
> -#   raw+size: (image files only)
> -#     A raw binary data stream such as produced via `dd if=TheImageFile`.
> -#   qcow2+size, vmdk: (image files only)
> -#     A raw qcow2/vmdk/... file such as produced via `dd if=some.qcow2` for
> -#     files which are already in qcow2 format, or via `qemu-img convert`.
> -#     Note that these formats are only valid with $with_snapshots being true.
> -#   tar+size: (subvolumes only)
> -#     A GNU tar stream containing just the inner contents of the subvolume.
> -#     This does not distinguish between the contents of a privileged or
> -#     unprivileged container. In other words, this is from the root user
> -#     namespace's point of view with no uid-mapping in effect.
> -#     As produced via `tar -C vm-100-disk-1.subvol -cpf TheOutputFile.dat .`
> +=head3 $plugin->volume_export(\%scfg, $storeid, $fh, $volname, $format [, $snapshot, $base_snapshot, $with_snapshots])
> +
> +=head3 $plugin->volume_export(...)
> +
> +Exports a volume or a volume's C<$snapshot> into a file handle C<$fh> as a
> +stream with a desired export C<$format>. See L<FORMATS> for all import/export
> +formats.
> +
> +Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
> +C<$with_snapshots> states whether the volume has snapshots overall.

this is incomplete/wrong

$with_snapshots means the export should include snapshots, not whether
the volume has snapshots..
$snapshot means "this is the snapshot to export" if only exporting the
snapshot ($with_snapshots == 0), or "this is the *last* snapshot export"
if exporting a stream of snapshots ($with_snapshots == 1)
$base_snapshot means "this is the start of the snapshot range to export"
(i.e., do an incremental export on top of this base)

this is mostly only relevant for zfs at the moment (other storages can
either export a volume including its snapshots, or just the volume, but
no complicated incremental streams of snapshots), but will change once
we implement replication export/import for other storages..

> +
> +C<die>s in order to abort the export if there are (grave) problems, if the
> +given C<$format> is not supported, or if exporting volumes is not supported as
> +a whole.
> +
> +=cut
>  
> -# Export a volume into a file handle as a stream of desired format.
>  sub volume_export {
>      my ($class, $scfg, $storeid, $fh, $volname, $format, $snapshot, $base_snapshot, $with_snapshots) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->volume_export_formats(\%scfg, $storeid, $volname [, $snapshot, $base_snapshot, $with_snapshot])
> +
> +=head3 $plugin->volume_export_formats(...)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Returns a list of supported export formats for the given volume or snapshot.
> +
> +Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
> +C<$with_snapshots> states whether the volume has snapshots overall.

see above.. these parameters are used to affect the returned list of
formats

> +
> +If the storage does not support exporting volumes at all, and empty list should
> +be returned.
> +
> +=cut
> +
>  sub volume_export_formats {
>      my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> -# Import data from a stream, creating a new or replacing or adding to an existing volume.
> +=head3 $plugin->volume_import(\%scfg, $storeid, $fh, $volname, $format [, $snapshot, $base_snapshot, $with_snapshots, $allow_rename])
> +
> +=head3 $plugin->volume_import(...)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Imports data with the given C<$format> from a stream / filehandle C<$fh>,
> +creating a new volume, or replacing or adding to an existing one. Returns the
> +volume ID of the imported volume.

I don't think we ever replace an existing volume? what we might do is
import new snapshots on top of base_snapshot in case of an incremental
import..

maybe

creating a new volume or importing additional snapshots on top of an
existing one.

> +
> +Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
> +C<$with_snapshots> states whether the volume has snapshots overall. Renaming an
> +existing volume may also optionally be allowed via C<$allow_rename>

see above, but here $snapshot is mainly there to have the same
arguments for volume_import_formats so a plugin can have a single
implementation, not because it is used anywhere IIRC..

> +
> +C<die>s if the import fails, if the given C<$format> is not supported, or if
> +importing volumes is not supported as a whole.
> +
> +=cut
> +
>  sub volume_import {
>      my ($class, $scfg, $storeid, $fh, $volname, $format, $snapshot, $base_snapshot, $with_snapshots, $allow_rename) = @_;
>      croak "implement me in sub-class\n";
>  }
>  
> +=head3 $plugin->volume_import_formats(\%scfg, $storeid, $volname [, $snapshot, $base_snapshot, $with_snapshot])
> +
> +=head3 $plugin->volume_import_formats(...)
> +
> +B<OPTIONAL:> May be implemented in a storage plugin.
> +
> +Returns a list of supported import formats for the given volume or snapshot.
> +
> +Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
> +C<$with_snapshots> states whether the volume has snapshots overall.

see above, these parameters serve the same purpose as for export_formats
- to affect the return value / inform the plugin about the intended
export/import

it might make sense to note somewhere that the order of returned formats
matters for both, since the first element of the intersection of both
return values will be used to do a storage migration?

> +
> +If the storage does not support importing volumes at all, and empty list should
> +be returned.
> +
> +=cut
> +
>  sub volume_import_formats {
>      my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
>      croak "implement me in sub-class\n";
>  }
> -
>  1;


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 8/8] pluginbase: document import and export methods

[Fiona Ebner]

Am 01.04.25 um 10:40 schrieb Fabian Grünbichler:
> On March 26, 2025 3:20 pm, Max Carrara wrote:
>> +=head3 $plugin->volume_export(\%scfg, $storeid, $fh, $volname, $format [, $snapshot, $base_snapshot, $with_snapshots])
>> +
>> +=head3 $plugin->volume_export(...)
>> +
>> +Exports a volume or a volume's C<$snapshot> into a file handle C<$fh> as a
>> +stream with a desired export C<$format>. See L<FORMATS> for all import/export
>> +formats.
>> +
>> +Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
>> +C<$with_snapshots> states whether the volume has snapshots overall.
> 
> this is incomplete/wrong
> 
> $with_snapshots means the export should include snapshots, not whether
> the volume has snapshots..
> $snapshot means "this is the snapshot to export" if only exporting the
> snapshot ($with_snapshots == 0), or "this is the *last* snapshot export"
> if exporting a stream of snapshots ($with_snapshots == 1)
> $base_snapshot means "this is the start of the snapshot range to export"
> (i.e., do an incremental export on top of this base)
> 
> this is mostly only relevant for zfs at the moment (other storages can
> either export a volume including its snapshots, or just the volume, but
> no complicated incremental streams of snapshots), but will change once
> we implement replication export/import for other storages..

There are already ideas floating around to change this and add proper
format negotiation. We'll also need to ask the target what it supports
like for remote migration, the sending side cannot really know that. And
as part of that change from the confusing set of snapshot-related
parameters to having an actual "what kind of transport" enum:
1. current data (of an image or a snapshot)
2. full sync with all snapshots
3. incremental stream
No details worked out yet though and not really relevant for documenting
the status quo.

>> +Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
>> +C<$with_snapshots> states whether the volume has snapshots overall. Renaming an
>> +existing volume may also optionally be allowed via C<$allow_rename>
> 
> see above, but here $snapshot is mainly there to have the same
> arguments for volume_import_formats so a plugin can have a single
> implementation, not because it is used anywhere IIRC..

The LVMPlugin.pm and Plugin.pm do have different implementations of the
volume_{export,import}_formats() methods, precisely because they need to
ignore the $snapshot parameter for the import case. Yes, we do pass the
same arguments, but it can only matter for the "incremental stream"
scenario. Otherwise, the parameter has nothing to do with the import
side. Here it would also be much nicer to have an actual "what kind of
transport" parameter.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 1/8] pluginbase: introduce PVE::Storage::PluginBase with doc scaffold

[Max Carrara]

On Mon Mar 31, 2025 at 5:13 PM CEST, Fabian Grünbichler wrote:
> On March 26, 2025 3:20 pm, Max Carrara wrote:
> > Add PVE::Storage::PluginBase, which defines stubs for all methods that
> > storage plugins should implement in order to conform to our plugin
> > API. This makes it much easier for (third-party) developers to see
> > which methods should be implemented.
> > 
> > PluginBase is inserted into the inheritance chain between
> > PVE::Storage::Plugin and PVE::SectionConfig instead of letting the
> > Plugin module inherit from SectionConfig directly. This keeps the
> > inheritance chain linear, avoiding multiple inheritance.
> > 
> > Also provide a scaffold for documentation. Preserve pre-existing
> > comments for context's sake.
> > 
> > Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> > ---
> >  src/PVE/Storage/Makefile      |   1 +
> >  src/PVE/Storage/Plugin.pm     |   2 +-
> >  src/PVE/Storage/PluginBase.pm | 328 ++++++++++++++++++++++++++++++++++
> >  3 files changed, 330 insertions(+), 1 deletion(-)
> >  create mode 100644 src/PVE/Storage/PluginBase.pm
> > 
> > diff --git a/src/PVE/Storage/Makefile b/src/PVE/Storage/Makefile
> > index ce3fd68..f2cdb66 100644
> > --- a/src/PVE/Storage/Makefile
> > +++ b/src/PVE/Storage/Makefile
> > @@ -1,6 +1,7 @@
> >  SOURCES= \
> >  	Common.pm \
> >  	Plugin.pm \
> > +	PluginBase.pm \
> >  	DirPlugin.pm \
> >  	LVMPlugin.pm \
> >  	NFSPlugin.pm \
> > diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
> > index 65cf43f..df6882a 100644
> > --- a/src/PVE/Storage/Plugin.pm
> > +++ b/src/PVE/Storage/Plugin.pm
> > @@ -19,7 +19,7 @@ use PVE::Storage::Common;
> >  
> >  use JSON;
> >  
> > -use base qw(PVE::SectionConfig);
> > +use parent qw(PVE::Storage::PluginBase);
> >  
> >  use constant KNOWN_COMPRESSION_FORMATS =>  ('gz', 'lzo', 'zst', 'bz2');
> >  use constant COMPRESSOR_RE => join('|', KNOWN_COMPRESSION_FORMATS);
> > diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> > new file mode 100644
> > index 0000000..e56aa72
> > --- /dev/null
> > +++ b/src/PVE/Storage/PluginBase.pm
> > @@ -0,0 +1,328 @@
> > +=head1 NAME
> > +
> > +C<PVE::Storage::PluginBase> - Storage Plugin API Interface
> > +
> > +=head1 DESCRIPTION
> > +
> > +=cut
> > +
> > +package PVE::Storage::PluginBase;
> > +
> > +use strict;
> > +use warnings;
> > +
> > +use Carp qw(croak);
> > +
> > +use parent qw(PVE::SectionConfig);
> > +
> > +=head1 PLUGIN INTERFACE METHODS
> > +
> > +=cut
> > +
> > +=head2 PLUGIN DEFINITION
> > +
> > +=cut
> > +
> > +sub type {
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub properties {
> > +    my ($class) = @_;
> > +    return $class->SUPER::properties();
> > +}
> > +
> > +sub options {
> > +    my ($class) = @_;
> > +    return $class->SUPER::options();
> > +}
> > +
> > +sub plugindata {
> > +    my ($class) = @_;
> > +    return $class->SUPER::plugindata();
> > +}
> > +
> > +sub private {
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +=head2 GENERAL
> > +
> > +=cut
> > +
> > +sub check_connection {
> > +    my ($class, $storeid, $scfg) = @_;
> > +
> > +    return 1;
> > +}
> > +
> > +sub activate_storage {
> > +    my ($class, $storeid, $scfg, $cache) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub deactivate_storage {
> > +    my ($class, $storeid, $scfg, $cache) = @_;
> > +
> > +    return;
> > +}
> > +
> > +sub status {
> > +    my ($class, $storeid, $scfg, $cache) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub cluster_lock_storage {
> > +    my ($class, $storeid, $shared, $timeout, $func, @param) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub parse_volname {
> > +    my ($class, $volname) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub get_subdir {
> > +    my ($class, $scfg, $vtype) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub filesystem_path {
> > +    my ($class, $scfg, $volname, $snapname) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub path {
> > +    my ($class, $scfg, $volname, $storeid, $snapname) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub find_free_diskname {
> > +    my ($class, $storeid, $scfg, $vmid, $fmt, $add_fmt_suffix) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +=head2 HOOKS
> > +
> > +=cut
> > +
> > +# called during addition of storage (before the new storage config got written)
>
> called when adding a storage config entry, before the new config gets written
>
> > +# die to abort addition if there are (grave) problems
> > +# NOTE: runs in a storage config *locked* context
> > +sub on_add_hook {
> > +    my ($class, $storeid, $scfg, %param) = @_;
> > +    return undef;
> > +}
> > +
> > +# called during storage configuration update (before the updated storage config got written)
>
> called when updating a storage config entry, before the updated config
> gets written
>
> > +# die to abort the update if there are (grave) problems
> > +# NOTE: runs in a storage config *locked* context
> > +sub on_update_hook {
> > +    my ($class, $storeid, $scfg, %param) = @_;
> > +    return undef;
> > +}
> > +
> > +# called during deletion of storage (before the new storage config got written)
> > +# and if the activate check on addition fails, to cleanup all storage traces
> > +# which on_add_hook may have created.
>
> called when deleting a storage config entry, before the new storage
> config gets written.
>
> also called as part of error handling when undoing the addition of a new
> storage config entry.

Regarding your three responses above: The comments here were preserved
from `::Plugin` for context's sake. But tbh, on second thought, they can
probably just be removed, as they'll be replaced by POD anyways.

>
> > +# die to abort deletion if there are (very grave) problems
> > +# NOTE: runs in a storage config *locked* context
> > +sub on_delete_hook {
> > +    my ($class, $storeid, $scfg) = @_;
> > +    return undef;
> > +}
> > +
> > +=head2 IMAGE OPERATIONS
> > +
>
> should this describe what IMAGES are in the context of PVE? else as a
> newcomer the difference between IMAGE here and VOLUME below might not
> be clear..

Yeah, that's a good idea. Maximiliano and I were also thinking about
maybe adding a GLOSSARY section at the bottom of the file where certain
terms could be explained / defined in more detail in general.
What do you think?

Alternatively, we could also have the top-level description define the
most basic of terms, but I don't want to load the docs here with too
much information up front.

>
> > +=cut
> > +
> > +sub list_images {
> > +    my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub create_base {
> > +    my ($class, $storeid, $scfg, $volname) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub clone_image {
> > +    my ($class, $scfg, $storeid, $volname, $vmid, $snap) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub alloc_image {
> > +    my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub free_image {
> > +    my ($class, $storeid, $scfg, $volname, $isBase, $format) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +=head2 VOLUME OPERATIONS
>
> see above
>
> > +
> > +=cut
> > +
> > +sub list_volumes {
> > +    my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +# Returns undef if the attribute is not supported for the volume.
> > +# Should die if there is an error fetching the attribute.
> > +# Possible attributes:
> > +# notes     - user-provided comments/notes.
> > +# protected - not to be removed by free_image, and for backups, ignored when pruning.
> > +sub get_volume_attribute {
> > +    my ($class, $scfg, $storeid, $volname, $attribute) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +# Dies if the attribute is not supported for the volume.
> > +sub update_volume_attribute {
> > +    my ($class, $scfg, $storeid, $volname, $attribute, $value) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub volume_size_info {
> > +    my ($class, $scfg, $storeid, $volname, $timeout) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub volume_resize {
> > +    my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub volume_snapshot {
> > +    my ($class, $scfg, $storeid, $volname, $snap) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +# Returns a hash with the snapshot names as keys and the following data:
> > +# id        - Unique id to distinguish different snapshots even if the have the same name.
> > +# timestamp - Creation time of the snapshot (seconds since epoch).
> > +# Returns an empty hash if the volume does not exist.
> > +sub volume_snapshot_info {
> > +    my ($class, $scfg, $storeid, $volname) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +# Asserts that a rollback to $snap on $volname is possible.
> > +# If certain snapshots are preventing the rollback and $blockers is an array
> > +# reference, the snapshot names can be pushed onto $blockers prior to dying.
> > +sub volume_rollback_is_possible {
> > +    my ($class, $scfg, $storeid, $volname, $snap, $blockers) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub volume_snapshot_rollback {
> > +    my ($class, $scfg, $storeid, $volname, $snap) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub volume_snapshot_delete {
> > +    my ($class, $scfg, $storeid, $volname, $snap, $running) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub volume_snapshot_needs_fsfreeze {
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub storage_can_replicate {
> > +    my ($class, $scfg, $storeid, $format) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub volume_has_feature {
> > +    my ($class, $scfg, $feature, $storeid, $volname, $snapname, $running, $opts) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub map_volume {
> > +    my ($class, $storeid, $scfg, $volname, $snapname) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub unmap_volume {
> > +    my ($class, $storeid, $scfg, $volname, $snapname) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub activate_volume {
> > +    my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub deactivate_volume {
> > +    my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub rename_volume {
> > +    my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub prune_backups {
> > +    my ($class, $scfg, $storeid, $keep, $vmid, $type, $dryrun, $logfunc) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +=head2 IMPORTS AND EXPORTS
> > +
> > +=cut
> > +
> > +# Import/Export interface:
> > +#   Any path based storage is assumed to support 'raw' and 'tar' streams, so
> > +#   the default implementations will return this if $scfg->{path} is set,
> > +#   mimicking the old PVE::Storage::storage_migrate() function.
> > +#
> > +# Plugins may fall back to PVE::Storage::Plugin::volume_{export,import}...
> > +#   functions in case the format doesn't match their specialized
> > +#   implementations to reuse the raw/tar code.
> > +#
> > +# Format specification:
> > +#   The following formats are all prefixed with image information in the form
> > +#   of a 64 bit little endian unsigned integer (pack('Q<')) in order to be able
> > +#   to preallocate the image on storages which require it.
> > +#
> > +#   raw+size: (image files only)
> > +#     A raw binary data stream such as produced via `dd if=TheImageFile`.
> > +#   qcow2+size, vmdk: (image files only)
> > +#     A raw qcow2/vmdk/... file such as produced via `dd if=some.qcow2` for
> > +#     files which are already in qcow2 format, or via `qemu-img convert`.
> > +#     Note that these formats are only valid with $with_snapshots being true.
> > +#   tar+size: (subvolumes only)
> > +#     A GNU tar stream containing just the inner contents of the subvolume.
> > +#     This does not distinguish between the contents of a privileged or
> > +#     unprivileged container. In other words, this is from the root user
> > +#     namespace's point of view with no uid-mapping in effect.
> > +#     As produced via `tar -C vm-100-disk-1.subvol -cpf TheOutputFile.dat .`
> > +
> > +# Export a volume into a file handle as a stream of desired format.
> > +sub volume_export {
> > +    my ($class, $scfg, $storeid, $fh, $volname, $format, $snapshot, $base_snapshot, $with_snapshots) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub volume_export_formats {
> > +    my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +# Import data from a stream, creating a new or replacing or adding to an existing volume.
> > +sub volume_import {
> > +    my ($class, $scfg, $storeid, $fh, $volname, $format, $snapshot, $base_snapshot, $with_snapshots, $allow_rename) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +sub volume_import_formats {
> > +    my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
> > +    croak "implement me in sub-class\n";
> > +}
> > +
> > +1;
> > -- 
> > 2.39.5
> > 
> > 
> > 
> > _______________________________________________
> > pve-devel mailing list
> > pve-devel@lists.proxmox.com
> > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> > 
> > 
> > 
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 2/8] pluginbase: add high-level plugin API description

[Max Carrara]

On Mon Mar 31, 2025 at 5:13 PM CEST, Fabian Grünbichler wrote:
> On March 26, 2025 3:20 pm, Max Carrara wrote:
> > Add a short paragraph in DESCRIPTION serving as an introduction as
> > well as the GENERAL PARAMETERS and CACHING EXPENSIVE OPERATIONS
> > sections.
> > 
> > These sections are added in order to avoid repeatedly describing the
> > same parameters as well as to elaborate on / clarify a couple terms,
> > e.g. what the $cache parameter does or what a volume in our case is.
> > 
> > Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> > ---
> >  src/PVE/Storage/PluginBase.pm | 77 +++++++++++++++++++++++++++++++++++
> >  1 file changed, 77 insertions(+)
> > 
> > diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> > index e56aa72..16977f3 100644
> > --- a/src/PVE/Storage/PluginBase.pm
> > +++ b/src/PVE/Storage/PluginBase.pm
> > @@ -4,6 +4,83 @@ C<PVE::Storage::PluginBase> - Storage Plugin API Interface
> >  
> >  =head1 DESCRIPTION
> >  
> > +This module documents the public Storage Plugin API of PVE and serves
> > +as a base for C<L<PVE::Storage::Plugin>>. Plugins must B<always> inherit from
> > +C<L<PVE::Storage::Plugin>>, as this module is for documentation purposes
> > +only.
>
> does this make sense? if we now provide a clean base for the structure
> of plugins, why shouldn't plugins be able to use that, but instead have
> to inherit from PVE::Storage::Plugin which has a lot of extra stuff that
> makes things messy?
>
> granted, switching over to load from PluginBase could be done as a
> follow up or with 9.0 (or not at all, if there is a rationale)..
>
> after this series we have:
>
> SectionConfig
> -> PluginBase (not an actual base plugin w.r.t. SectionConfig, and not
> something you base plugins on as a result)
> --> Plugin (a combination of base plugin and base of all our
> directory-based plugins)
> ---> other plugins, including third party ones
>
> which seems unfortunate, even if the contents of PluginBase are helpful
> when implementing your own..
>
> IMHO this should either be
>
> SectionConfig
> -> PluginBase (actual base plugin, with SectionConfig implementations
> moved over from current Plugin.pm as needed)
> --> PluginTemplate (what's currently PluginBase in this series - nicely
> documented parent of third party plugins, not actually registered, just
> contains the storage.cfg interface without the low-level SectionConfig
> things)
> ---> NewThirdPartyPlugin (clean slate implementation just using the
> nicely documented interfaces, guaranteed to not use any helpers from
> Plugin since it's not a (grand)parent)
> --> Plugin (base of built-in plugins, probably base of existing third
> party plugins, should ideally have a different name in the future!)
> ---> DirPlugin
> ---> ...
> ---> ExistingThirdPartyPlugin (this might rely on helpers from Plugin,
> so we can't just rename that one unless we wait for 9.0)
>
> or
>
> SectionConfig
> -> PluginBase (actual base plugin + docs of Plugin API)
> --> Plugin (base of our plugins and existing third party ones, dir-related helpers, ..)
> ---> other plugins, including third party ones
> --> NewThirdPartyPlugin (clean slate as above)

I agree with your point here -- for now, `::PluginBase` should IMO still
only be about documentation and enumerating what's part of the Plugin
API, *but* adding more layers inbetween can still be done eventually.

However, I think we shouldn't have two different parents for internal
and external plugins, as it would introduce yet another thing we'd have
to track wrt. APIAGE resets etc. Maybe it's not actually an issue in
this case, but ...

That actually gives me an idea that's similar to your first suggestion:

As of this series, the hierarchy would be as follows:

PluginBase
└── Plugin
    ├── ExistingThirdPartyPlugin
    ├── DirPlugin
    └── ...

We could keep `PluginBase` as it is, since IMO having the docs and the
SectionConfig-related stuff in one place is fine, unless we really want
to keep the docs separate from the rest of the code. (Would seem a bit
redundant to introduce another inheritance layer in that case, but I
personally don't mind.)

Then, we eventually introduce `PluginTemplate` on the same layer as
`Plugin`. `PluginTemplate` would only contain implementations for the
most basic methods (and not provide defaults for file-based storages).

PluginBase
├── Plugin
│   ├── ExistingThirdPartyPlugin
│   ├── DirPlugin
│   └── ...
└── PluginTemplate

The idea behind this is that we could then "migrate" each plugin and
base it off `PluginTemplate` instead. Helpers that are shared between
plugins could go into `PVE::Storage::Common::*` instead of being
implicitly becoming part of plugins' modules due to inheritance.

PluginBase
├── Plugin
│   ├── ...
│   └── ExistingThirdPartyPlugin
└── PluginTemplate
    ├── ...
    └── DirPlugin (cleaned up)

That way we could step by step "disentangle" the existing plugins from
each other without having to constantly keep the original behaviour(s)
of `Plugin` in the back of one's head and account for them. Instead,
each plugin would implement precisely what it itself needs.

Since both the old `Plugin` and the new `PluginTemplate` share the same
interface, namely `PluginBase`, we could still support the old `Plugin`
until everything's been moved over and third-party devs have had enough
time to adapt their own code, too.

While doing all this, we could also rework parts of the API that didn't
age that gracefully, perhaps deprecate certain old methods, introduce
new methods, etc. as we'd have a "clean" break, sotospeak.


So, to summarize my idea:
- Keep `PluginBase` as it is right now, but also include
  SectionConfig-related code
- Introduce `PluginTemplate` with minimal implementation later down the
  line on the same inheritance layer as `Plugin`
- Slowly migrate our plugins, basing them off of `PluginTemplate` while
  tossing out old code, making them independent from one another, and
  collecting shared helpers in `PVE::Storage::Common::*`


I'd really like to hear your thoughts on this, because I'm not sure if
this is *actually* feasible or provides any ROI down the line. One
alternative that I can think of is to just keep the inheritance
hierarchy as it is (as in this series) and disentangle the plugins as
they are right now, without changing their parent (so, almost the same
as your second idea). I did start breaking apart our plugins like that
last year, but that was too much all at once [1].

[1]: https://lore.proxmox.com/pve-devel/20240717094034.124857-1-m.carrara@proxmox.com/

>
> side-note: should we mention somewhere that plugin code is not called
> directly (pre-existing exceptions that we want to get rid off ignored),
> but that PVE::Storage is the "gateway"/interface for it?

Yeah, good point; we should definitely mention that. Will include it in v2!

>
> > +
> > +=head2 DEFAULT IMPLEMENTATIONS
> > +
> > +C<L<PVE::Storage::Plugin>> implements most of the methods listed in
> > +L</PLUGIN INTERFACE METHODS> by default. These provided implementations are
> > +tailored towards file-based storages and can therefore be used as-is in that
> > +case. Plugins for other kinds of storages will most likely have to adapt each
> > +method for their individual use cases.
>
> see above
>
> > +
> > +=head2 GENERAL PARAMETERS
> > +
> > +The parameter naming throughout the code is kept as consistent as possible.
> > +Therefore, common reappearing subroutine parameters are listed here for
> > +convenience:
> > +
> > +=over
> > +
> > +=item * C<< \%scfg >>
> > +
> > +The storage configuration associated with the given C<$storeid>. This is a
> > +reference to a hash that represents the section associated with C<$storeid> in
> > +C</etc/pve/storage.cfg>.
> > +
> > +=item * C<< $storeid >>
> > +
> > +The ID of the storage. This ID is user-provided; the IDs for existing
> > +storages can be found in the UI via B<< Datacenter > Storage >>.
>
> The unique ID of the storage, as used in C</etc/pve/storage.cfg> and as
> part of every volid.
>
> this is not really user-provided in most cases (that makes it sound like
> you have to be super careful when handling it to prevent exploits),
> although the user of course initially named the section like that ;)

ACK! Thanks for the suggestion!

>
> > +
> > +=item * C<< $volname >>
> > +
> > +The name of a volume. The term I<volume> can refer to a disk image, an ISO
> > +image, a backup, etc. depending on the content type of the volume.
>
> backup archive/snapshot ?
>
> should we list all types here?

Perhaps not here, but maybe in a separate section instead to which we
can refer to? Then again, they're listed in the docstring for
`plugindata()`. Perhaps its docs could benefit from such a section, too.

I'll see what I can do.

>
> > +
> > +=item * C<< $volid >>
> > +
> > +The ID of a volume, which is essentially C<"${storeid}:${volname}">. Less used
> > +within the plugin API, but nevertheless relevant.
>
> s/essentially// (it is exactly that, not essentially)
>
> the second sentence doesn't really add much, if we want to keep it, then
> I suggest replacing it with something like
>
> Frequently used in guest-specific API calls and passed to
> PVE::Storage::parse_volume_id to split it into storeid and volname parts
> before calling into storage plugin code.

Agree, this is much better.

>
> > +
> > +=item * C<< $snapname >> (or C<< $snap >>)
> > +
> > +The name of a snapshot associated with the given C<$volname>.
>
> what is "given" here? the phrasing doesn't really make sense IMHO ;)

Woops, that was a remnant of when I was moving this around. Thanks for
spotting!

>
> The name of a snapshot of a volume.
>
> > +
> > +=item * C<< \%cache >>
> > +
> > +See L<CACHING EXPENSIVE OPERATIONS>.
> > +
> > +=item * C<< $vmid >>
> > +
> > +The ID of a guest (so, either of a VM or a container).
> > +
> > +=back
> > +
> > +=head2 CACHING EXPENSIVE OPERATIONS
> > +
> > +Certain methods take a C<\%cache> as parameter, which is used to store the
> > +results of time-consuming / expensive operations specific to certain plugins.
>
> this is not really accurate. the cache is used to store different
> information, currently (according to a quick grep through the code):
>
> for storages (activate_storage/activate_storage_list):
> - parsed /proc/mounts for various dir-based storages where the plugin
>   handles mounting, to speed up "is storage already mounted" checks
> - udev sequence numbers (to know whether udev needs settling after
>   activating a storage)
> - a flag that a storage was activated (as an optimization to skip
>   "is it already active" checks)
>
> this already mixes state of PVE::Storage with plugin-specific data,
> which is a mess.
>
> when listing images across multiple storages:
> - vgs/lvs output (LVM plugins)
>
> that one we've eliminated from most plugins, since it rarely makes
> sense. LVM is a bit of an exception there, as we query global LVM state
> that is valid for multiple storage instances. this maybe could be
> changed to drop the usage as well, and instead query VG-specific
> information only?
>
> > +The exact lifetime of the cached data is B<unspecified>, since it depends on
> > +the exact usage of the C<L<PVE::Storage>> API, but can generally be assumed to
> > +be B<short-lived>.
>
> this is not really something that helps me as a plugin developer - what
> kind of information can/should/must I store in the cache (or not)? how
> is it structured?
>
> > +
> > +For example, the C<L<PVE::Storage::LVMPlugin>> uses this cache to store the
> > +parsed output of the C<lvs> command. Since the number and precise information
> > +about LVM's logical volumes is unlikely to change within a short time, other
> > +API methods can then use this data in order to avoid repeatedly calling and
> > +parsing the output of C<lvs>.
>
> this reads like the cache is used across PVE API calls (although maybe
> you meant "storage plugin API"?). the original (flawed) reason was/is
> that if we activate 20 volumes on a single storage for a certain task,
> then it is enough to check with LVM once and re-use the (slightly stale)
> data. this since got eliminated from most plugins as it was not working.
> the other use case (see above) is if we (potentially) activate 10
> storages, we can check once what is already mounted and re-use that for
> the subsequent 9 activations. I am not sure this is worth it either to
> be honest.
>
> > +
> > +Third-party plugin developers should ensure that the data stored and retrieved
> > +is specific to their plugin, and not rely on the data that other plugins might
> > +store in C<\%cache>. Furthermore, the names of keys should be rather unique in
> > +the sense that they're unlikely to conflict with any future keys that may be
> > +introduced internally. To illustrate, e.g. C<myplugin_mounts> should be used
> > +instead of a plain C<mounts> key.
>
> and this here clearly shows that the current interface is bogus and
> under-specified in any case. so until that is fixed, this here should
> read "ignore the cache parameter it is for internal use only". if a
> plugin needs to cache things internally, it can do so anyway based on
> its own criteria..

In the initial description I tried to simplify / generalize its
description and not really document every single implementation detail
so that third-party devs could use it for their own purposes in order to
avoid running the same time-consuming code in short succession.

I hadn't really thought of the fact that plugins can just keep an
internal cache if they want to cache stuff. 🤦

So yeah, after reading all this, I agree that we should just deter third
parties from using the parameter altogether.

Side-note: I'll also refrain from using it in the SSHFS example plugin I
recently cooked up [2].

[2]: https://lore.proxmox.com/pve-devel/20250328171209.503132-1-m.carrara@proxmox.com/

>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 3/8] pluginbase: document SectionConfig methods

[Max Carrara]

On Mon Mar 31, 2025 at 5:13 PM CEST, Fabian Grünbichler wrote:
> On March 26, 2025 3:20 pm, Max Carrara wrote:
> > This commit adds docstrings for the relevant PVE::SectionConfig
> > methods in the context of the storage plugin API.
> > 
> > Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> > ---
> >  src/PVE/Storage/PluginBase.pm | 194 +++++++++++++++++++++++++++++++++-
> >  1 file changed, 192 insertions(+), 2 deletions(-)
> > 
> > diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> > index 16977f3..5f7e6fd 100644
> > --- a/src/PVE/Storage/PluginBase.pm
> > +++ b/src/PVE/Storage/PluginBase.pm
> > @@ -88,7 +88,7 @@ package PVE::Storage::PluginBase;
> >  use strict;
> >  use warnings;
> >  
> > -use Carp qw(croak);
> > +use Carp qw(croak confess);
> >  
> >  use parent qw(PVE::SectionConfig);
> >  
> > @@ -100,27 +100,217 @@ use parent qw(PVE::SectionConfig);
> >  
> >  =cut
> >  
> > +=head3 $plugin->type()
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.

Upfront note: Unless I otherwise comment something, I agree with you.
Just sparing myself from writing and you from reading too many ACKs :P

>
> I am not sure whether we should do
>
> s/implemented in/implemented by
>
> for this whole thing and SectionConfig, but I won't note it for every
> instance ;)
>
> > +
> > +This should return a string with which the plugin can be uniquely identified.
>
> a string which uniquely identifies the plugin.
>
> > +Any string is acceptable, as long as it's descriptive and you're sure it won't
> > +conflict with another plugin. In the cases of built-in plugins, you will often
> > +find the filesystem name or something similar being used.
>
> I'd replace that with something like:
>
> Commonly, the name of the storage technology or filesystem supported by
> the plugin is used. If it is necessary to differentiate multiple
> plugins for the same storage technology/filesystem, add a suffix
> denoting the supported variant or feature set.
>
> > +
> > +See C<L<PVE::SectionConfig/type>> for more information.
> > +
> > +=cut
> > +
> >  sub type {
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->properties()
> > +
> > +B<OPTIONAL:> May be implemented in a storage plugin.
> > +
> > +This method should be implemented if there are additional properties to be used
> > +by your plugin. Since properties are global and may be reused across plugins,
>
> s/used/registered
>
> IMHO that conveys better what the difference between properties and
> options are..
>
> "global and may be reused across plugins" is kinda superfluous, maybe
>
> "Since properties share a global namespace across all plugins"?
>
> > +the names of properties must not collide with one another.
>
> each property can only be registered once by a single plugin, but used
> as an option by many.
>
> > +
> > +When implementing a third-party plugin, it is recommended to prefix properties
> > +with some kind of identifier, like so:
>
> Third-party plugins should therefore prefix any properties they register
> with their plugin type, for example:
>
> > +
> > +    sub properties {
> > +	return {
> > +	    'example-storage-address' => {
> > +		description => 'Host address of the ExampleStorage to connect to.',
> > +		type => 'string',
> > +	    },
> > +	    'example-storage-pool' => {
> > +		description => 'Name of the ExampleStorage pool to use.',
> > +		type => 'string',
> > +	    },
> > +	    # [...]
> > +	};
> > +    }
> > +
> > +However, it is encouraged to reuse properties of inbuilt plugins whenever
> > +possible. There are a few provided properties that are regarded as I<sensitive>
> > +and will be treated differently in order to not expose them or write them as
> > +plain text into configuration files. One such property fit for external use is
> > +C<password>, which you can use to provide a password, API secret, or similar.
>
> If possible, generic properties registered by built-in plugins should be
> reused.
>
> (inbuilt is a very weird, very British word ;))
>
> the rest of that is kinda misplaced here and should be its own section
> somewhere (it affects options just as much as properties). we should
> probably allow registering sensitive properties for plugins?

Yes, we should! And Fiona's already on it 🎉 [0]

[0]: https://lore.proxmox.com/pve-devel/20250321134852.103871-11-f.ebner@proxmox.com/

>
> > +
> > +See C<L<PVE::SectionConfig/properties>> for more information.
> > +
> > +=cut
> > +
> >  sub properties {
> >      my ($class) = @_;
> >      return $class->SUPER::properties();
> >  }
> >  
> > +=head3 $plugin->options()
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +This method returns a hash of the properties used by the plugin. Because
> > +properties are shared among plugins, it is recommended to reuse any existing
> > +ones of inbuilt plugins and only define custom properties via
> > +C<L<< properties()|/"$plugin->properties()" >>> if necessary.
>
> Everything but the first sentence can be a reference to the properties
> part above..
>
> > +
> > +The properties a plugin uses are then declared as follows:
>
> or just, "For example:" ? ;)
>
> > +
> > +    sub options {
> > +	return {
> > +	    nodes => { optional => 1 },
> > +	    content => { optional => 1 },
> > +	    disable => { optional => 1 },
> > +	    'example-storage-pool' => { fixed => 1 },
> > +	    'example-storage-address' => { fixed => 1 },
> > +	    password => { optional => 1 },
> > +	};
> > +    }
> > +
> > +C<optional> properties are not required to be set. It is recommended to set
>
> s/set/make or "declare"?
>
> > +most properties optional by default, unless it I<really> is required to always
> > +exist.
>
> s/it/they
> s/exist/be set
>
> > +
> > +C<fixed> properties can only be set when creating a new storage via the plugin
> > +and cannot be changed afterwards.
>
> s/creating a new storage/adding a new storage config entry/ ? a bit more
> precise to differentiate it from actually "creating a storage" as in
> formatting a disk/..
>
> I am not sure what "via the plugin" is supposed to mean there, I think
> it can just be dropped.
>
> > +
> > +See C<L<PVE::SectionConfig/options>> for more information.
> > +
> > +=cut
> > +
> >  sub options {
> >      my ($class) = @_;
> >      return $class->SUPER::options();
> >  }
> >  
> > +=head3 $plugin->plugindata()
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +This method returns a hash that specifies additional capabilities of the storage
> > +plugin, such as what kinds of data may be stored on it or what VM disk formats
>
> s/data/content or s/data/volumes
>
> > +the storage supports. Additionally, defaults may also be set. This is done
> > +through the C<content> and C<format> keys.
>
> but how about re-organizing it a bit?
>
> This method returns a hash declaring which content types and image
> formats this plugin (or the underlying storage) supports.
>
> and then describe each key?
>
> > +
> > +The C<content> key is used to declare B<supported content types> and their
> > +defaults, while the C<format> key declares B<supported disk formats> and the
> > +default disk format of the storage:
>
> "and their defaults" sounds weird.
>
> The 'content' key stores a list containing two hashes. The first list
> element is a hash declaring which content types are supported by the
> plugin. The second list element is a hash declaring which content types
> are used by default if no explicit 'content' option is set on a plugin
> instance.
>
> and maybe link somewhere else for a description of the content type
> values, so that there is a single place that various parts that need it
> can link to?
>
> and then
>
> The 'format' key stores a list containing two hashes. The first list
> element is a hash declaring which image formats are supported by the
> storage plugin. The second list element is the default image format that
> should be used for images on the storage if non is explicitly provided.
>
> > +
> > +    sub plugindata {
> > +	return {
> > +	    content => [
> > +		# possible content types
> > +		{
> > +		    images => 1,   # disk images
> > +		    rootdir => 1,  # container root directories
> > +		    vztmpl => 1,   # container templates
> > +		    iso => 1,      # iso images
> > +		    backup => 1,   # vzdump backup files
> > +		    snippets => 1, # snippets
> > +		    import => 1,   # imports; see explanation below
> > +		    none => 1,     # no content; see explanation below
> > +		},
> > +		# defaults if 'content' isn't explicitly set
> > +		{
> > +		    images => 1,   # store disk images by default
> > +		    rootdir => 1   # store containers by default
> > +		    # possible to add more or have no defaults
> > +		}
> > +	    ],
> > +	    format => [
> > +		# possible disk formats
> > +		{
> > +		    raw => 1,   # raw disk image
> > +		    qcow2 => 1, # QEMU image format
> > +		    vmdk => 1,  # VMware image format
> > +		    subvol => 1 # subvolumes; see explanation below
> > +		},
> > +		"qcow2" # default if 'format' isn't explicitly set
> > +	    ]
> > +	    # [...]
> > +	};
> > +    }
> > +
> > +While the example above depicts a rather capable storage, the following
> > +shows a simpler storage that can only be used for VM disks:
>
> raw-formatted VM disks
>
> > +
> > +    sub plugindata {
> > +	return {
> > +	    content => [
> > +		{ images => 1 },
> > +	    ],
> > +	    format => [
> > +		{ raw => 1 },
> > +		"raw",
> > +	    ]
> > +	};
> > +    }
> > +
> > +Which content types and formats are supported depends on the underlying storage
> > +implementation.
> > +
> > +B<Regarding C<import>:> The C<import> content type is used internally to
> > +interface with virtual guests of foreign sources or formats. The corresponding
> > +functionality has not yet been published to the public parts of the storage
> > +API. Third-party plugins therefore should not declare this content type.
>
> this should live somewhere where content types are described
>
> > +
> > +B<Regarding C<none>:> The C<none> content type denotes the I<absence> of other
> > +types of content, i.e. this content type may only be set on a storage if no
> > +other content type is set. This is used internally for storages that support
> > +adding another storage "on top" of them; at the moment, this makes it possible
> > +to set up an LVM (thin) pool on top of an iSCSI LUN. The corresponding
> > +functionality has not yet been published to the public parts of the storage
> > +API. Third-party plugins therefore should not declare this content type.
>
> same
>
> > +
> > +B<Regarding C<subvol>:> The C<subvol> format is used internally to allow the
> > +root directories of containers to use ZFS subvolumes (also known as
> > +I<ZFS datasets>, not to be confused with I<ZVOLs>). Third-party plugins should
> > +not declare this format type.
>
> this is incomplete, but same.
>
> > +
> > +There is one more key, C<select_existing>, which is used internally for
> > +ISCSI-related GUI functionality. Third-party plugins should not declare this
> > +key.
>
> this is okay here :)
>
> > +
> > +=cut
> > +
> >  sub plugindata {
> >      my ($class) = @_;
> >      return $class->SUPER::plugindata();
> >  }
> >  
> > +=head3 $plugin->private()
> > +
> > +B<WARNING:> This method is provided by C<L<PVE::Storage::Plugin>> and
> > +must be used as-is. It is merely documented here for informal purposes
> > +and B<must not be overridden.>
> > +
> > +Returns a hash containing the tracked plugin metadata, most notably the
> > +C<propertyList>, which contains all known properties of all plugins.
> > +
> > +C<L<PVE::Storage::Plugin>> uses this to predefine a lot of useful properties
> > +that are relevant for all plugins. Core functionality such as defining
> > +whether a storage is shared, which nodes may use it, whether a storage
> > +is enabled or not, etc. are implemented via these properties.
>
> this is not true and sounds like it is very interesting to look at/mess
> with ;) in reality, it is part of SectionConfig machinery, so just say
> that and that it must not be overridden and be done with it?
>
> if PluginBase becomes the real base plugin then it and some other
> helpers would need to move here anyway..
>
> > +
> > +See C<L<PVE::SectionConfig/private>> for more information.
> > +
> > +=cut
> > +
> >  sub private {
> > -    croak "implement me in sub-class\n";
> > +    confess "private() is provided by PVE::Storage::Plugin and must not be overridden";
> >  }
> >  
> >  =head2 GENERAL
> > -- 
> > 2.39.5
> > 
> > 
> > 
> > _______________________________________________
> > pve-devel mailing list
> > pve-devel@lists.proxmox.com
> > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> > 
> > 
> > 
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 4/8] pluginbase: document general plugin methods

[Max Carrara]

On Mon Mar 31, 2025 at 5:12 PM CEST, Fabian Grünbichler wrote:
> On March 26, 2025 3:20 pm, Max Carrara wrote:
> > Add docstrings for the following methods:
> > - check_connection
> > - activate_storage
> > - deactivate_storage
> > - status
> > - cluster_lock_storage
> > - parse_volname
> > - get_subdir
> > - filesystem_path
> > - path
> > - find_free_diskname
> > 
> > Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> > ---
> >  src/PVE/Storage/PluginBase.pm | 255 ++++++++++++++++++++++++++++++++++
> >  1 file changed, 255 insertions(+)
> > 
> > diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> > index 5f7e6fd..8a61dc3 100644
> > --- a/src/PVE/Storage/PluginBase.pm
> > +++ b/src/PVE/Storage/PluginBase.pm
> > @@ -317,53 +317,308 @@ sub private {
> >  
> >  =cut
> >  
> > +=head3 $plugin->check_connection($storeid, \%scfg)
> > +
> > +B<OPTIONAL:> May be implemented in a storage plugin.
> > +
> > +Performs a connection check.
> > +
> > +This method is useful for plugins that require some kind of network connection
> > +or similar and is called before C<L<< activate_storage()|/"$plugin->activate_storage($storeid, \%scfg, \%cache)" >>>.

Upfront note: Unless I otherwise comment something, I agree with you.
Just sparing myself from writing and you from reading too many ACKs :P

>
> This method can be implemented by network-based storages. It will be
> called before storage activation attempts. Non-network storages should
> not implement it.
>
> > +
> > +Returns C<1> by default and C<0> if the storage isn't online / reachable.
>
> by default doesn't make any sense, even though I know what you mean.
>
> > +
> > +C<die>s if an error occurs while performing the connection check.
> > +
> > +Note that this method's purpose is to simply verify that the storage is
> > +I<reachable>, and not necessarily that authentication etc. succeeds.
> > +
> > +For example: If the storage is mainly accessed via TCP, you can try to simply
> > +open a TCP connection to see if it's online:
> > +
> > +    # In custom module:
> > +
> > +    use PVE::Network;
> > +    use parent qw(PVE::Storage::Plugin)
> > +
> > +    # [...]
> > +
> > +    sub check_connection {
> > +	my ($class, $storeid, $scfg) = @_;
> > +
> > +	my $port = $scfg->{port} || 5432;
> > +	return PVE::Network::tcp_ping($scfg->{server}, $port, 5);
> > +    }
> > +
> > +=cut
> > +
> >  sub check_connection {
> >      my ($class, $storeid, $scfg) = @_;
> >  
> >      return 1;
> >  }
> >  
> > +=head3 $plugin->activate_storage($storeid, \%scfg, \%cache)
> > +
> > +=head3 $plugin->activate_storage(...)
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Activates the storage, making it ready for further use.
> > +
> > +In essence, this method performs the steps necessary so that the storage can be
> > +used by remaining parts of the system.
>
> this is superfluous (all of that information is already contained in the
> previous sentence).
>
> > +
> > +In the case of file-based storages, this usually entails creating the directory
> > +of the mountpoint, mounting the storage and then creating the directories for
> > +the different content types that the storage has enabled. See
> > +C<L<PVE::Storage::NFSPlugin>> and C<L<PVE::Storage::CIFSPlugin>> for examples
> > +in that regard.
> > +
> > +Other types of storages would use this method for establishing a connection to
> > +the storage and authenticating with it or similar. See C<L<PVE::Storage::ISCSIPlugin>>
> > +for an example.
> > +
>
> I am not sure this examples provide much benefit in this verbosity..
> maybe just a single sentence like
>
> Common examples of activation might include mounting filesystems,
> preparing directory trees or establishing sessions with network
> storages.
>
> > +If the storage doesn't need to be activated in some way, this method can be a
> > +no-op.
> > +
> > +C<die>s in case of errors.
>
> *Should* die?
>
> > +
> > +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> > +
> > +=cut
> > +
> >  sub activate_storage {
> >      my ($class, $storeid, $scfg, $cache) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->deactivate_storage($storeid, \%scfg, \%cache)
> > +
> > +=head3 $plugin->deactivate_storage(...)
> > +
> > +B<OPTIONAL:> May be implemented in a storage plugin.
> > +
> > +Deactivates the storage. Counterpart to C<L<< activate_storage()|/"$plugin->activate_storage(...)" >>>.
> > +
> > +This method is used to make the storage unavailable to the rest of the system,
> > +which usually entails unmounting the storage, closing an established
> > +connection, or similar.
> > +
> > +Does nothing by default.
> > +
> > +C<die>s in case of errors.
>
> *Should*
>
> > +
> > +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> > +
> > +B<NOTE:> This method is currently not used by our API due to historical
> > +reasons.
> > +
> > +=cut
> > +
> >  sub deactivate_storage {
> >      my ($class, $storeid, $scfg, $cache) = @_;
> >  
> >      return;
> >  }
> >  
> > +=head3 $plugin->status($storeid, \%scfg, \%cache)
> > +
> > +=head3 $plugin->status(...)
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Returns a list of scalars in the following form:
> > +
> > +    my ($total, $available, $used, $active) = $plugin->status($storeid, $scfg, $cache)
> > +
> > +This list contains the C<$total>, C<$available> and C<$used> storage capacity,
> > +respectively, as well as whether the storage is C<$active> or not.
>
> might make sense to include the information which unit the returned
> numbers should be in?
>
> > +
> > +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> > +
> > +=cut
> > +
> >  sub status {
> >      my ($class, $storeid, $scfg, $cache) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->cluster_lock_storage($storeid, $shared, $timeout, \&func, @param)
> > +
> > +=head3 $plugin->cluster_lock_storage(...)
> > +
> > +B<WARNING:> This method is provided by C<L<PVE::Storage::Plugin>> and
> > +must be used as-is. It is merely documented here for informal purposes
> > +and B<must not be overridden.>
>
> see my comment on the first patch.. this is not actually part of the
> plugin interface, it is a helper that is implemented by
> PVE::Storage::Plugin for historical reasons and should be moved
> somewhere else and then removed from PVE::Storage::Plugin when we do the
> next break of that inheritance hierarchy..
>
> > +
> > +Locks the storage with the given C<$storeid> for the entire host and runs the
> > +given C<\&func> with the given C<@param>s, unlocking the storage once finished.
> > +If the storage is C<$shared>, it is instead locked on the entire cluster. If
> > +the storage is already locked, wait for at most the number of seconds given by
> > +C<$timeout>.
>
> Attempts to acquire a lock on C<$storeid>, waiting at most C<$timeout>
> seconds before giving up. If successful, runs C<\&func> with the given
> C<@param>s while holding the lock. If C<$shared> is set, the lock
> operation will be cluster-wide (with a timeout for the execution of
> C<\&func> of 60 seconds).
>
> This method must be used to guard against concurrent modifications of
> the storage by multiple tasks running at the same time, in particular
> for actions such as:
> - allocating new volumes (including clones) or snapshots
> - deleting existing volumes or snapshots
> - renaming existing volumes or snapshots
>
> > +
> > +This method is used to synchronize access to the given storage, preventing
> > +simultaneous modification from multiple workers. This is necessary for
> > +operations that modify data on the storage directly, like cloning and
> > +allocating images.
>
>
> > +
> > +=cut
> > +
> >  sub cluster_lock_storage {
> >      my ($class, $storeid, $shared, $timeout, $func, @param) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->parse_volname($volname)
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Parses C<$volname>, returning a list representing the parts encoded in
> > +the volume name:
>
> s/parts/volume properties/ ?
>
> should we incorporate the outcome of this discussion:
>
> https://lore.proxmox.com/pve-devel/qdnb3vypbucbcf7ch2hsjbeo3hqb5bh4whoinl5xglggpt7b7t@igryfncdupdp/

Hmm, I'd say so, yeah. The original message isn't on lore apparently, so
I'll see if I can dig this up in my mailbox later.

>
> ?
>
> > +
> > +    my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format)
> > +	= $plugin->parse_volname($volname);
> > +
> > +Not all parts need to be included in the list. Those marked as I<optional>
> > +in the list below may be set to C<undef> if not applicable.
> > +
> > +This method may C<die> in case of errors.
>
> s/may/should/ ?
>
> > +
> > +=over
> > +
> > +=item * C<< $vtype >>
> > +
> > +The content type ("volume type") of the volume, e.g. C<"images">, C<"iso">,
> > +etc.
>
> content type != volume type! the two are related obviously, but the
> values are subtly different for images for historic reasons.

Oh, right. Thanks for pointing this out! I'll see if I can note down
those differences somewhere.

>
> > +
> > +See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all content types.
> > +
> > +=item * C<< $name >>
> > +
> > +The display name of the volume. This is usually what the underlying storage
> > +itself uses to address the volume.
>
> Ideally, this maps directly to some entity on the underlying storage.
>
> > +
> > +For example, disks for virtual machines that are stored on LVM thin pools are
> > +named C<vm-100-disk-0>, C<vm-1337-disk-1>, etc. That would be the C<$name> in
> > +this case.
>
> This is rather imprecise, maybe just say
>
> For example, a guest volume named C<vm-100-disk-0> stored on an LVM thin
> pool will refer to a thin LV of the same name.
>
> > +
> > +=item * C<< $vmid >> (optional)
> > +
> > +The ID of the guest that owns the volume.
>
> should maybe note that this must be set for `images` or `backup`, and
> must not be set for the other vtypes?
>
> > +
> > +=item * C<< $basename >> (optional)
> > +
> > +The C<$name> of the volume this volume is based on. Whether this part
> > +is returned or not depends on the plugin and underlying storage.
>
> The C<$name> of this volume's base volume, if it is a linked clone.
>
> ?
>
> > +Only applies to disk images.
> > +
> > +For example, on ZFS, if the VM is a linked clone, C<$basename> refers
> > +to the C<$name> of the original disk volume that the parsed disk volume
> > +corresponds to.
>
> it is only used for that, and has this semantic across all storages,
> right?

AFAIR yes. But now you made me check again soon. :P

>
> > +
> > +=item * C<< $basevmid >> (optional)
> > +
> > +Equivalent to C<$basename>, except that C<$basevmid> refers to the
> > +C<$vmid> of the original disk volume instead.
>
> s/original/base
>
> > +
> > +=item * C<< $isBase >> (optional)
> > +
> > +Whether the volume is a base disk image.
> > +
> > +=item * C<< $format >>
> > +
> > +The format of the volume. If the volume is a VM disk (C<$vtype> is
> > +C<"images">), this should be whatever format the disk is in. For most
> > +other content types C<"raw"> should be used.
>
> "images" is currently also for container volumes.. import volumes also
> have a format..

Yeah, Wolfgang also told me this last week a little after I sent in this
series. Will correct the docs.

>
> > +
> > +See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all formats.
> > +
> > +=back
> > +
> > +=cut
> > +
> >  sub parse_volname {
> >      my ($class, $volname) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->get_subdir(\%scfg, $vtype)
>
> I am not sure about this one long-term.. while the interface is generic
> enough, it is only used for the helpers in PVE::Storage which are in
> turn used for
> - uploading/download things (templates, iso files, ..)
> - creating backup archives (or rather, getting the path to the dump dir
>   to create them)
>
> > +
> > +B<SITUATIONAL:> This method must be implemented for file-based storages.
>
> s/file-based/dir-based
>
> ?
>
> > +
> > +Returns the path to the sub-directory associated with the given content type
> > +(C<$vtype>). See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all
> > +content types.
>
> again, this is vtype not content type..
>
> > +
> > +The default directory layout is predefined and must not be altered:
> > +
> > +    my $vtype_subdirs = {
> > +	images   => 'images',
> > +	rootdir  => 'private',
> > +	iso      => 'template/iso',
> > +	vztmpl   => 'template/cache',
> > +	backup   => 'dump',
> > +	snippets => 'snippets',
> > +	import   => 'import',
> > +    };
> > +
> > +See also: L<Storage: Directory|"https://pve.proxmox.com/wiki/Storage:_Directory">
> > +
> > +=cut
> > +
> >  sub get_subdir {
> >      my ($class, $scfg, $vtype) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->filesystem_path(\%scfg, $volname [, $snapname])
> > +
> > +=head3 $plugin->filesystem_path(...)
> > +
> > +B<SITUATIONAL:> This method must be implemented for file-based storages.
>
> file/dir
>
> > +
> > +Returns the absolute path on the filesystem for the given volume or snapshot.
> > +In list context, returns path, guest ID and content type:
>
> content/volume
>
> but - this is not actually part of the plugin API. it just looks like it
> because Plugin's implementation of path (which is the actual API) calls
> it, and thus plugins derived from that base implement or call it as
> well.
>
> the public interface if you want a file or blockdev path for a given
> volid is PVE::Storage::abs_filesystem_path, which internally also calls
> the plugin's `path` method, not the non-public `filesystem_path`.
>
> > +
> > +    my $path = $plugin->filesystem_path($scfg, $volname, $snapname)
> > +    my ($path, $vmid, $vtype) = $plugin->filesystem_path($scfg, $volname, $snapname)
> > +
> > +=cut
> > +
> >  sub filesystem_path {
> >      my ($class, $scfg, $volname, $snapname) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->path(\%scfg, $volname, $storeid [, $snapname])
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Returns a unique path that points to the given volume or snapshot depending
> > +on the underlying storage implementation. For file-based storages, this
> > +method should return the same as C<L<< filesystem_path()|/"$plugin->filesystem_path(...)" >>>.
>
> this is inverted (see above). it should also note that if there is an
> actual path corresponding to a volume, it should return that, but that
> it can also return something else that Qemu understands in lieu of a
> real path.
>
> if multiple paths exist, it should return the most specific/unique one
> to avoid collisions between multiple plugin instances of the same
> storage type.
>
> > +
> > +=cut
> > +
> >  sub path {
> >      my ($class, $scfg, $volname, $storeid, $snapname) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->find_free_diskname($storeid, \%scfg, $vmid [, $fmt, $add_fmt_suffix])
> > +
> > +=head3 $plugin->find_free_diskname(...)
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Finds and returns the next available disk name, that is, the volume name
> > +(C<$volname>) for a new disk image. Optionally, C<$fmt> specifies the
> > +format of the disk image and C<$add_fmt_suffix> denotes whether C<$fmt>
> > +should be added as suffix to the resulting name.
>
> similarly, this is not actually part of the plugin API, it is an
> implementation detail of the existing plugin hierarchy. it is currently
> called by Plugin's clone, allocate and rename functionality, so if you
> don't implement/override it you need to also override those (which you
> likely need to do in any case..).
>
> these two are actually a good example of what I meant with the messiness
> of Plugin.pm leaking if we continue deriving *all* plugins from it,
> including new external ones..

... Yeah, this just reaffirms my stance that a proper investigative look
and documentation for all of this *really* necessary. Even after this
digital archaeology tour we've apparently missed a lot of details, or in
the case of the two methods above, included details that aren't actually
part of the plugin API.

Thanks for going through all of this, by the way -- it's really highly
appreciated!

>
> > +
> > +=cut
> > +
> >  sub find_free_diskname {
> >      my ($class, $storeid, $scfg, $vmid, $fmt, $add_fmt_suffix) = @_;
> >      croak "implement me in sub-class\n";
> > -- 
> > 2.39.5
> > 
> > 
> > 
> > _______________________________________________
> > pve-devel mailing list
> > pve-devel@lists.proxmox.com
> > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> > 
> > 
> > 
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 6/8] pluginbase: document image operation methods

[Max Carrara]

On Mon Mar 31, 2025 at 5:12 PM CEST, Fabian Grünbichler wrote:
> On March 26, 2025 3:20 pm, Max Carrara wrote:
> > Add documentation for the following methods:
> > - list_images
> > - create_base
> > - clone_image
> > - alloc_image
> > - free_image
> > 
> > Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> > Co-authored-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
> > ---
> >  src/PVE/Storage/PluginBase.pm | 111 ++++++++++++++++++++++++++++++++++
> >  1 file changed, 111 insertions(+)
> > 
> > diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> > index b3ce684..37b1471 100644
> > --- a/src/PVE/Storage/PluginBase.pm
> > +++ b/src/PVE/Storage/PluginBase.pm
> > @@ -721,26 +721,137 @@ sub on_delete_hook {
> >  
> >  =cut
> >  
> > +=head3 $plugin->list_images($storeid, \%scfg [, $vmid, \@vollist, \%cache])
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Returns a listref of all disk images of a storage. If the storage does not
> > +support storing disk images, returns an empty listref.
> > +
> > +Optionally, if C<\@vollist> is provided, return only disks whose volume ID is
> > +within C<\@vollist>. Note that this usually has higher precedence than
> > +C<$vmid>.

Upfront note: Unless I otherwise comment something, I agree with you.
Just sparing myself from writing and you from reading too many ACKs :P

>
> what does usually mean? what does $vmid do?

Sorry, this got lost when tossing out some redundant paragraphs here.

The "usually" can be dropped; C<$vmid> is the guest's ID (mentioned in
the DESCRIPTION in patch 01) and unless C<\@vollist> is provided, the
images that the given C<$vmid> owns will be returned.

>
> > +
> > +C<die>s in case of errors.
> > +
> > +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> > +
> > +The returned listref has the following structure:
> > +
> > +    [
> > +	{
> > +	    ctime => "1689163322", # creation time as unix timestamp
> > +	    format => "raw",
> > +	    size => 8589934592, # in bytes!
> > +	    vmid => 101,
> > +	    volid => "local-lvm:base-101-disk-0", # volume ID, storage-specific
> > +	},
> > +	# [...]
> > +    ]
> > +
> > +=cut
> > +
> >  sub list_images {
> >      my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->create_base($storeid, \%scfg, $volname)
> > +
> > +B<OPTIONAL:> May be implemented in a storage plugin.
> > +
> > +Creates a base volume from an existing volume, allowing the volume to be
>
> this is misleading - it doesn't create a base volume from an existing
> volume, it *converts* an existing volume into a base volume!

Oh dang, thanks for pointing this out! Sorry for the oversight.

>
> > +L<< cloned|/"$plugin->clone_image(...)" >>. This cloned volume (usually
> > +a disk image) may then be used as a base for the purpose of creating linked
>
> this is wrong? there is no cloned volume yet? and all of this only
> applies to images and not other volumes?
>
> > +clones. See L<C<PVE::Storage::LvmThinPlugin>> and
> > +L<C<PVE::Storage::ZFSPoolPlugin>> for example implementations.
> > +
> > +On completion, returns the name of the new base volume (the new C<$volname>).
> > +
> > +This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
> > +i.e. when the storage is B<locked>.
>
> Shouldn't this say that it *should* be called in a locked context?
>
> > +
> > +=cut
> > +
> >  sub create_base {
> >      my ($class, $storeid, $scfg, $volname) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->clone_image($scfg, $storeid, $volname, $vmid [, $snap])
> > +
> > +=head3 $plugin->clone_image(...)
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
>
> not really? there's a feature guard for it, so only storage plugins that
> support it should ever get it called anyway..
>
> what does $vmid mean?

See above; the guest's ID. Since it was unclear, the parameter should
probably be explained here (and also above) instead of (just) in the
DESCRIPTION section.

>
> > +
> > +Clones a disk image or a snapshot of an image, returning the name of the new
> > +image (the new C<$volname>). Note that I<cloning> here means to create a linked
> > +clone and not duplicating an image. See L<C<PVE::Storage::LvmThinPlugin>> and
> > +L<C<PVE::Storage::ZFSPoolPlugin>> for example implementations.
>
> then why not start with
>
> "Creates a linked clone of an existing disk image or snapshot of an
> image"
>
> ? maybe also include that unless the linked clone is 100% independent
> from the base (only true for LVM thin atm?), the new volid should encode
> the base->clone relation in the volname?
>
> > +
> > +C<die>s in case of an error of if the underlying storage doesn't support
>
> s/of/or/
>
> > +cloning images.
> > +
> > +This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
> > +i.e. when the storage is B<locked>.
>
> same as above
>
> > +
> > +=cut
> > +
> >  sub clone_image {
> >      my ($class, $scfg, $storeid, $volname, $vmid, $snap) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->alloc_image($storeid, $scfg, $vmid, $fmt, $name, $size)
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Allocates a disk image with the given format C<$fmt> and C<$size> in bytes,
> > +returning the name of the new image (the new C<$volname>). See
> > +C<L<< plugindata()|/"$plugin->plugindata()" >>> for all disk formats.
> > +
> > +Optionally, if given, set the name of the image to C<$name>. If C<$name> isn't
> > +provided, the next name should be determined via C<L<< find_free_diskname()|/"$plugin->find_free_diskname(...)" >>>.
>
> a suitable name should be automatically generated, unless we really want
> to stabilize find_free_diskname as part of the API..
>
> what does $vmid mean?
>
> > +
> > +C<die>s in case of an error of if the underlying storage doesn't support
> > +allocating images.
> > +
> > +This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
> > +i.e. when the storage is B<locked>.
> > +
> > +=cut
> > +
> >  sub alloc_image {
> >      my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->free_image($storeid, $scfg, $volname [, $isBase, $format])
> > +
> > +=head3 $plugin->free_image(...)
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Frees (deletes) the disk image given by C<$volname>. Optionally, the image may
> > +be a base image (C<$isBase>) and have a certain C<$format>. See
> > +C<L<< plugindata()|/"$plugin->plugindata()" >>> for all disk formats.
> > +
> > +If a cleanup is required after freeing the image, returns a reference to a
> > +subroutine that performs the cleanup, and C<undef> otherwise.
>
> might be a good idea to explain why? also, what do $isBase and $format
> mean?
>
> > +
> > +C<die>s in case of errors, if the image has a L<< C<protected> attribute|/"$plugin->get_volume_attribute(...)" >>
> > +(and may thus not be freed), or if the underlying storage implementation
> > +doesn't support freeing images.
> > +
> > +This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
> > +i.e. when the storage is B<locked>.
> > +
> > +B<NOTE:> The returned cleanup subroutine is called in a separate worker and
> > +should L<< lock|/"$plugin->cluster_lock_storage(...)" >> the underlying storage
> > +separately.
>
> it should? or it must?
>
> > +
> > +=cut
> > +
> >  sub free_image {
> >      my ($class, $storeid, $scfg, $volname, $isBase, $format) = @_;
> >      croak "implement me in sub-class\n";
> > -- 
> > 2.39.5
> > 
> > 
> > 
> > _______________________________________________
> > pve-devel mailing list
> > pve-devel@lists.proxmox.com
> > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> > 
> > 
> > 
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 7/8] pluginbase: document volume operations

[Max Carrara]

On Mon Mar 31, 2025 at 5:12 PM CEST, Fabian Grünbichler wrote:
> On March 26, 2025 3:20 pm, Max Carrara wrote:
> > Add docstrings for the following methods:
> > - list_volumes
> > - get_volume_attribute
> > - update_volume_attribute
> > - volume_size_info
> > - volume_resize
> > - volume_snapshot
> > - volume_snapshot_info
> > - volume_rollback_is_possible
> > - volume_snapshot_rollback
> > - volume_snapshot_delete
> > - volume_snapshot_needs_fsfreeze
> > - storage_can_replicate
> > - volume_has_feature
> > - map_volume
> > - unmap_volume
> > - activate_volume
> > - deactivate_volume
> > - rename_volume
> > - prune_backups
> > 
> > Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> > Co-authored-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
> > ---
> >  src/PVE/Storage/PluginBase.pm | 401 ++++++++++++++++++++++++++++++++--
> >  1 file changed, 388 insertions(+), 13 deletions(-)
> > 
> > diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> > index 37b1471..a1bfc8d 100644
> > --- a/src/PVE/Storage/PluginBase.pm
> > +++ b/src/PVE/Storage/PluginBase.pm
> > @@ -861,108 +861,483 @@ sub free_image {
> >  
> >  =cut
> >  
> > +=head3 $plugin->list_volumes($storeid, \%scfg, $vmid, \@content_types)
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Returns a list of volumes for the given guest whose content type is within the

Upfront note: Unless I otherwise comment something, I agree with you.
Just sparing myself from writing and you from reading too many ACKs :P

>
> this is wrong - what if $vmid is not set? or if content type is snippets
> or one of other ones that don't have an associated guest/owner? or what
> if there is no $vmid given, as in the example below?

Right, thanks for spotting this too.

>
> > +given C<\@content_types>. If C<\@content_types> is empty, no volumes will be
> > +returned. See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all content types.
>
> here we are actually talking about content types for once (and this also
> translates from content type "images" and "rootdir" to volume type
> "images"! in PVE::Plugin!).
>
> > +
> > +    # List all backups for a guest
> > +    my $backups = $plugin->list_volumes($storeid, $scfg, $vmid, ['backup']);
> > +
> > +    # List all containers and virtual machines on the storage
> > +    my $guests = $plugin->list_volumes($storeid, $scfg, undef, ['rootdir', 'images'])
>
> eh, this is a bad example? it doesn't list the guests, it lists their
> volumes..
>
> > +
> > +The returned listref of hashrefs has the following structure:
> > +
> > +    [
> > +	{
> > +	    content => "images",
> > +	    ctime => "1702298038", # creation time as unix timestamp
> > +	    format => "raw",
> > +	    size => 9663676416, # in bytes!
> > +	    vmid => 102,
> > +	    volid => "local-lvm:vm-102-disk-0",
> > +	},
> > +	# [...]
> > +    ]
> > +
> > +Backups will also contain additional keys:
> > +
> > +    [
> > +	{
> > +	    content => "backup",
> > +	    ctime => 1742405070, # creation time as unix timestamp
> > +	    format => "tar.zst",
> > +	    notes => "...", # comment that was entered when backup was created
> > +	    size => 328906840, # in bytes!
> > +	    subtype => "lxc", # "lxc" for containers, "qemu" for VMs
> > +	    vmid => 106,
> > +	    volid => "local:backup/vzdump-lxc-106-2025_03_19-18_24_30.tar.zst",
> > +	},
> > +	# [...]
> > +    ]
>
> soooo.. what is the interface here again? -> needs a (complete) schema
> for the returned type, else how am I supposed to implement this?

I mean, we could define one. I didn't want to clobber the docstring with
too many examples / too much text here, but I agree.

>
> > +
> > +=cut
> > +
> >  sub list_volumes {
> >      my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > -# Returns undef if the attribute is not supported for the volume.
> > -# Should die if there is an error fetching the attribute.
> > -# Possible attributes:
> > -# notes     - user-provided comments/notes.
> > -# protected - not to be removed by free_image, and for backups, ignored when pruning.
> > +=head3 $plugin->get_volume_attribute(\%scfg, $storeid, $volname, $attribute)
> > +
> > +=head3 $plugin->get_volume_attribute(...)
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Returns the value of the given C<$attribute> for a volume. If the attribute isn't
> > +supported for the volume, returns C<undef>.
> > +
> > +C<die>s if there is an error fetching the attribute.
> > +
> > +B<Possible attributes:>
> > +
> > +=over
> > +
> > +=item * C<notes> (returns scalar)
>
> scalar *string* ?
> > +
> > +User-provided comments / notes.
> > +
> > +=item * C<protected> (returns scalar)
>
> scalar *boolean* ?
>
> > +
> > +When set to C<1>, the volume must not be removed by C<L<< free_image()|/"$plugin->free_image(...)" >>>.
> > +Backups with C<protected> set to C<1> are ignored when pruning.
>
> Backup volumes .. when calculating which backup volumes to prune.
>
> > +
> > +=back
> > +
> > +=cut
> > +
> >  sub get_volume_attribute {
> >      my ($class, $scfg, $storeid, $volname, $attribute) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > -# Dies if the attribute is not supported for the volume.
> > +=head3 $plugin->update_volume_attribute(\%scfg, $storeid, $volname, $attribute, $value)
> > +
> > +=head3 $plugin->update_volume_attribute(...)
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Sets the value of the given C<$attribute> for a volume to C<$value>.
> > +
> > +C<die>s if the attribute isn't supported for the volume (or storage).
> > +
> > +For a list of supported attributes, see C<L<< get_volume_attribute()|/"$plugin->get_volume_attribute(...)" >>>.
> > +
> > +=cut
> > +
> >  sub update_volume_attribute {
> >      my ($class, $scfg, $storeid, $volname, $attribute, $value) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->volume_size_info(\%scfg, $storeid, $volname [, $timeout])
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Returns information about the given volume's size. In scalar context, this returns
> > +just the volume's size in bytes:
> > +
> > +    my $size = $plugin->volume_size_info($scfg, $storeid, $volname, $timeout)
> > +
> > +In list context, returns an array with the following structure:
> > +
> > +    my ($size, $format, $used, $parent, $ctime) = $plugin->volume_size_info(
> > +	$scfg, $storeid, $volname, $timeout
> > +    )
> > +
> > +where C<$size> is the size of the volume in bytes, C<$format> one of the possible
> > +L<< formats listed in C<plugindata()>|/"$plugin->plugindata()" >>, C<$used> the
> > +amount of space used in bytes, C<$parent> the (optional) name of the base volume
> > +and C<$ctime> the creation time as unix timestamp.
> > +
> > +Optionally, a C<$timeout> may be provided after which the method should C<die>.
> > +This timeout is best passed to other helpers which already support timeouts,
> > +such as C<L<< PVE::Tools::run_command|PVE::Tools/run_command >>>.
> > +
> > +See also the C<L<< PVE::Storage::Plugin::file_size_info|PVE::Storage::Plugin/file_size_info >>> helper.
>
> PVE::Storage::file_size_info (without Plugin::)!
>
> > +
> > +=cut
> > +
> >  sub volume_size_info {
> >      my ($class, $scfg, $storeid, $volname, $timeout) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->volume_resize(\%scfg, $storeid, $volname, $size [, $running])
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Resizes a volume to the new C<$size> in bytes. Optionally, the guest that owns
> > +the volume may be C<$running> (= C<1>).
>
> the second sentence doesn't make any sense.
>
> C<$running> indicates the guest is currently running.
>
> but what does that mean/why should a plugin care?
>
> > +
> > +C<die>s in case of errors, or if the underlying storage implementation or the
> > +volume's format doesn't support resizing.
> > +
> > +This function should not return any value. In previous versions the returned
> > +value would be used to determine the new size of the volume or whether the
> > +operation succeeded.
>
> so since this documents the new version.. shouldn't we just drop the
> last part?
>
> > +
> > +=cut
> > +
> >  sub volume_resize {
> >      my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->volume_snapshot(\%scfg, $storeid, $volname, $snap)
> > +
> > +B<OPTIONAL:> May be implemented if the underlying storage supports snapshots.
> > +
> > +Takes a snapshot of a volume and gives it the name provided by C<$snap>.
>
> this sounds like this is a two-step process..
>
> Takes a snapshot called C<$snap> of the volume C<$volname>.
>
> > +
> > +C<die>s if the underlying storrage doesn't support snapshots or an error
> > +occurs while taking a snapshot.
>
> s/a/the
> s/storrage/storage
>
> > +
> > +=cut
> > +
> >  sub volume_snapshot {
> >      my ($class, $scfg, $storeid, $volname, $snap) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > -# Returns a hash with the snapshot names as keys and the following data:
> > -# id        - Unique id to distinguish different snapshots even if the have the same name.
> > -# timestamp - Creation time of the snapshot (seconds since epoch).
> > -# Returns an empty hash if the volume does not exist.
> > +=head3 $plugin->volume_snapshot_info(\%scfg, $storeid, $volname)
> > +
> > +Returns a hashref with the snapshot names as keys and the following structure:
> > +
> > +    {
> > +        my_snapshot => {
> > +            id => "01b7e668-58b4-6f42-9a5e-1944c2855c84",  # Unique id to distinguish different snapshots with the same name.
> > +            timestamp => "1729841807",  # Creation time of the snapshot (seconds since epoch).
> > +        },
> > +	# [...]
> > +    }
> > +
> > +Returns an empty hashref if the volume does not exist.
>
> or dies if retrieving the snapshot information for the given volume
> failed. ?
>
> > +
> > +=cut
> > +
> >  sub volume_snapshot_info {
> >      my ($class, $scfg, $storeid, $volname) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > -# Asserts that a rollback to $snap on $volname is possible.
> > -# If certain snapshots are preventing the rollback and $blockers is an array
> > -# reference, the snapshot names can be pushed onto $blockers prior to dying.
> > +=head3 $plugin->volume_rollback_is_possible(\%scfg, $storeid, $volname, $snap [, \@blockers])
> > +
> > +=head3 $plugin->volume_rollback_is_possible(...)
> > +
> > +B<OPTIONAL:> May be implemented if the underlying storage supports snapshots.
> > +
> > +Asserts whether a rollback to C<$snap> on C<$volname> is possible, C<die>s if
> > +it isn't.
> > +
> > +Optionally, C<\@blockers> may be provided, which may contain the names of
> > +snapshots that are preventing the rollback. Should any such snapshots exist,
> > +they should be pushed to this listref pior to C<die>-ing. The caller may then
> > +use this listref when handling errors.
>
> If the optional paramater C<\@blockers> is provided, the names of any
> snapshots blocking the rollback should be added to this listref prior to
> C<die>-ing.
>
> > +
> > +=cut
> > +
> >  sub volume_rollback_is_possible {
> >      my ($class, $scfg, $storeid, $volname, $snap, $blockers) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->volume_snapshot_rollback(\%scfg, $storeid, $volname, $snap)
> > +
> > +Performs a rollback to the given C<$snap>shot on C<$volname>.
> > +
> > +C<die>s in case of errors.
> > +
> > +=cut
> > +
> >  sub volume_snapshot_rollback {
> >      my ($class, $scfg, $storeid, $volname, $snap) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->volume_snapshot_delete(\%scfg, $storeid, $volname, $snap [, $running])
> > +
> > +Deletes the C<$snap>shot of C<$volname>.
> > +
> > +C<die>s in case of errors.
> > +
> > +Optionally, the guest that owns the given volume may be C<$running> (= C<1>).
>
> Again, this is phrased weird *and* gives no information to a potential
> implementor of this API.. why should a plugin care?
>
> > +
> > +B<Deprecated:> The C<$running> parameter is deprecated and will be removed on the
> > +next C<APIAGE> reset.
>
> and this makes it even more confusing!
>
> > +
> > +=cut
> > +
> >  sub volume_snapshot_delete {
> >      my ($class, $scfg, $storeid, $volname, $snap, $running) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->volume_snapshot_needs_fsfreeze()
> > +
> > +Returns whether filesystems on top of the volume need to flush their journal for
> > +consistency before a snapshot is taken. See L<fsfreeze(8)>.
> > +
> > +This is needed for container mountpoints.
>
> this doesn't really tell me what I should do here either.. (and it's
> also a really ugly interace that we seemingly introduced because RBD
> snapshots cannot be mounted RO otherwise?? does this really need to be
> part of the plugin interface? Oo)
>
> > +
> > +=cut
> > +
> >  sub volume_snapshot_needs_fsfreeze {
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->storage_can_replicate(\%scfg, $storeid, $format)
> > +
> > +Returns whether volumes in a given C<$format> support replication.
> > +
> > +See C<L<< plugindata()|/"$plugin->plugindata()" >>> for all disk formats.
> > +
>
> should probably note that replication is limited to ZFS at the moment in
> any case?
>
> > +=cut
> > +
> >  sub storage_can_replicate {
> >      my ($class, $scfg, $storeid, $format) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->volume_has_feature(\%scfg, $feature, $storeid, $volname, $snapname [, $running, \%opts])
> > +
> > +=head3 $plugin->volume_has_feature(...)
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Checks whether a volume C<$volname> or its snapshot C<$snapname> supports the
> > +given C<$feature>, returning C<1> if it does and C<undef> otherwise. The guest
> > +owning the volume may optionally be C<$running>.
>
> that last sentence again doesn't make any sense the way it is phrased..
>
> > +
> > +C<$feature> may be one of the following:
> > +
> > +    clone      # linked clone is possible
> > +    copy       # full clone is possible
>
> actually, this is "bit-wise copying from" is possible, right?
>
> > +    replicate  # replication is possible
> > +    snapshot   # taking a snapshot is possible
> > +    sparseinit # volume is sparsely initialized
> > +    template   # conversion to base image is possible
> > +    rename     # renaming volumes is possible
> > +
> > +Which features are available under which circumstances depends on multiple
> > +factors, such as the underlying storage implementation, the format used, etc.
>
> *and whether the corresponding guest is currently running*, which is the
> whole point of that parameter ;)
>
> this really needs to explain what the values for those feature keys
> mean..

Could add a separate section for those keys as well, as with content
types. Otherwise, this docstring would probably blow up.

>
> > +It's best to check out C<L<PVE::Storage::Plugin>> or C<L<PVE::Storage::ZFSPoolPlugin>>
> > +for examples on how to handle features.
> > +
> > +Additional keys are given in C<\%opts>:
> > +
> > +=over
> > +
> > +=item * C<< valid_target_formats => [...] >>
> > +
> > +Listref of formats for the target of a copy/clone operation that the caller
> > +could work with. The format of the given volume is always considered valid and
> > +if no list is specified, all formats are considered valid.
> > +
> > +=back
> > +
> > +=cut
> > +
> >  sub volume_has_feature {
> >      my ($class, $scfg, $feature, $storeid, $volname, $snapname, $running, $opts) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->map_volume($storeid, \%scfg, $volname [, $snapname])
> > +
> > +B<OPTIONAL:> May be implemented in a storage plugin.
> > +
> > +Maps the device asscoiated with a volume or a volume's snapshot to a filesystem
>
> associated
>
> > +path, returning the path on completion. This method is used by containers.
>
> returning the path. (obviously on completion, how else?)
>
> it's not only used by containers, so maybe drop that part and instead
> note that it only needs to be implemented if `path` doesn't (always)
> return such a path anyway.
>
> > +
> > +C<die>s in case of errors.
> > +
> > +C<L<< unmap_volume()|/"$plugin->unmap_volume(...)" >>> can be used to declare
> > +how the device should be unmapped.
>
> maybe just say that if you implement map, you should also implement
> unmap?
>
> > +
> > +=cut
> > +
> >  sub map_volume {
> >      my ($class, $storeid, $scfg, $volname, $snapname) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->unmap_volume($storeid, \%scfg, $volname [, $snapname])
> > +
> > +B<OPTIONAL:> May be implemented in a storage plugin.
> > +
> > +Unmaps the device associated to a volume or a volume's snapshot.
> > +
> > +C<die>s in case of errors.
> > +
> > +=cut
> > +
> >  sub unmap_volume {
> >      my ($class, $storeid, $scfg, $volname, $snapname) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->activate_volume($storeid, \%scfg, $volname, $snapname [, \%cache])
> > +
> > +=head3 $plugin->activate_volume(...)
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Activates a volume or its associated snapshot, making it available to the
> > +system for further use. For example, this could mean activating an LVM volume,
> > +mounting a ZFS dataset, checking whether the volume's file path exists, etc.
> > +
> > +C<die>s in case of errors or if an operation is not supported.
> > +
> > +If this isn't needed, the method should simply be a no-op.
>
> If no activation is needed, 
>
> > +
> > +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> > +
> > +=cut
> > +
> >  sub activate_volume {
> >      my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->deactivate_volume($storeid, \%scfg, $volname, $snapname [, \%cache])
> > +
> > +=head3 deactivate_volume(...)
> > +
> > +B<REQUIRED:> Must be implemented in every storage plugin.
> > +
> > +Deactivates a volume or its associated snapshot, making it unavailable to
> > +the system. For example, this could mean deactivating an LVM volume,
> > +unmapping a Ceph/RBD device, etc.
> > +
> > +If this isn't needed, the method should simply be a no-op.
>
> If deactivation is not needed/not possible,
>
> should we note here that deactivation will not happen for every
> activation, but only when we know for sure that the volume is no longer
> needed, such as before it is removed or when its owner is migrated to
> another node?

Oh, yes. Very good point!

>
> > +
> > +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> > +
> > +=cut
> > +
> >  sub deactivate_volume {
> >      my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->rename_volume(...)
> > +
> > +=head3 $plugin->rename_volume(\%scfg, $storeid, $source_volname, $target_vmid, $target_volname)
> > +
> > +B<OPTIONAL:> May be implemented in a storage plugin.
> > +
> > +Renames the volume given by C<$source_volname> to C<$target_volname> and assigns
> > +it to the guest C<$target_vmid>. Returns the volume ID of the renamed volume.
> > +
> > +This method is needed for the I<Change Owner> feature.
> > +
> > +C<die>s if the rename failed.
> > +
> > +=cut
> > +
> >  sub rename_volume {
> >      my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->prune_backups(\%scfg, $storeid [, \%keep, $vmid, $type, $dryrun, \&logfunc])
> > +
> > +=head3 $plugin->prune_backups(...)
> > +
> > +Export a volume into a file handle as a stream with a desired format.
>
> copy-paste mistake? ;)
>
> > +
> > +C<die>s if there are (grave) problems while pruning.
> > +
> > +This method may take several optional parameters:
> > +
> > +=over
> > +
> > +=item * C<< \%keep >>
> > +
> > +A hashref containing backup retention policies. It has the following structure:
> > +
> > +    {
> > +	'keep-all'     => 1 # (optional) Whether to keep all backups.
> > +	                    # Conflicts with the other options when true.
> > +	'keep-last'    => N # (optional) Keep the last N backups.
> > +	'keep-hourly'  => N # (optional) Keep backups for the last N different hours.
> > +			    # If there is more than one backup for a single
> > +			    # hour, only the latest one is kept.
> > +	'keep-daily'   => N # (optional) Keep backups for the last N different days.
> > +			    # If there is more than one backup for a single
> > +			    # day, only the latest one is kept.
> > +	'keep-weekly'  => N # (optional) Keep backups for the last N different weeks.
> > +			    # If there is more than one backup for a single
> > +			    # week, only the latest one is kept.
> > +	'keep-monthly' => N # (optional) Keep backups for the last N different months.
> > +			    # If there is more than one backup for a single
> > +			    # month, only the latest one is kept.
> > +	'keep-yearly'  => N # (optional) Keep backups for the last N different years.
> > +			    # If there is more than one backup for a single
> > +			    # year, only the latest one is kept.
> > +    }
>
> should we add a pointer to docs here? or to
> PVE::Storage::prune_mark_backup_group? and should that or its body be
> moved somewhere else?

Yeah, I'll see what I can do here!

>
> > +
> > +=item * C<< $vmid >>
> > +
> > +The guest's ID.
> > +
> > +=item * C<< $type >>
> > +
> > +The type of guest. When C<defined>, it can be one of C<"qemu"> or C<"lxc">.
> > +If C<undefined>, both types of backups will be pruned.
> > +
> > +=item * C<< $dry_run >>
> > +
> > +Whether this is a dry run. If set to C<1> there won't be any change on the
> > +underlying storage.
> > +
> > +=item * C<< \&logfunc >>
> > +
> > +A subroutine ref that can be used to log messages with the following signature:
> > +
> > +    $logfunc->($severity, $message)
> > +
> > +where $severity can be one of C<"info">, C<"err">, or C<"warn">.
> > +
> > +=back
> > +
> > +=cut
> > +
> >  sub prune_backups {
> >      my ($class, $scfg, $storeid, $keep, $vmid, $type, $dryrun, $logfunc) = @_;
> >      croak "implement me in sub-class\n";
> > -- 
> > 2.39.5
> > 
> > 
> > 
> > _______________________________________________
> > pve-devel mailing list
> > pve-devel@lists.proxmox.com
> > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> > 
> > 
> > 
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 8/8] pluginbase: document import and export methods

[Max Carrara]

On Tue Apr 1, 2025 at 10:40 AM CEST, Fabian Grünbichler wrote:
> CCing Fiona in case of further input w.r.t. export/import things
>
> On March 26, 2025 3:20 pm, Max Carrara wrote:
> > Adapt the previous description, slightly rewording it and formatting
> > it for POD under the IMPORTS AND EXPORTS section.
> > 
> > Also add docstrings for the following methods:
> > - volume_export
> > - volume_export_formats
> > - volume_import
> > - volume_import_formats
> > 
> > Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> > Co-authored-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
> > ---
> >  src/PVE/Storage/PluginBase.pm | 134 ++++++++++++++++++++++++++--------
> >  1 file changed, 105 insertions(+), 29 deletions(-)
> > 
> > diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> > index a1bfc8d..cfa5087 100644
> > --- a/src/PVE/Storage/PluginBase.pm
> > +++ b/src/PVE/Storage/PluginBase.pm
> > @@ -1345,55 +1345,131 @@ sub prune_backups {
> >  
> >  =head2 IMPORTS AND EXPORTS
> >  
> > +Any path-based storage is assumed to support C<raw> and C<tar> streams, so
> > +the default implementations in C<L<PVE::Storage::Plugin>> will return this if
> > +C<< $scfg->{path} >> is set (thereby mimicking the old C<< PVE::Storage::storage_migrate() >>
> > +function).

Replying up here to address all of the comments below: Yeah, we'll
rework this entire section here, I think, and expand upon everything
more. Seems like we missed a lot of details. :s

>
> meh, I don't think this makes sense if we want to document the
> interface, we should document the interface, and not the implementation
> of our plugin hierarchy..
>
> > +
> > +Plugins may fall back to methods like C<volume_export>, C<volume_import>, etc.
> > +of C<L<PVE::Storage::Plugin>> in case the format doesn't match their
> > +specialized implementations to reuse the C<raw>/C<tar> code.
>
> and these should move to some helper module and be used by
> PVE::Storage::Plugin, if we want to allow external plugins to re-use
> them as basic implementation..
>
> > +
> > +=head3 FORMATS
> > +
> > +The following formats are all prefixed with image information in the form of a
> > +64 bit little endian unsigned integer (C<< pack('Q\<') >>) in order to be able
> > +to preallocate the image on storages which require it.
>
> "image information" should maybe be a bit more precise, it's easy to
> guess from the name that information==size, but why not spell it out?
>
> > +
> > +=over
> > +
> > +=item * C<< raw+size >> (image files only)
> > +
> > +A raw binary data stream as produced via C<< dd if=$IMAGE_FILE >>.
>
> A binary data stream containing a volume's logical raw data, for example
> as produced via .. if the image is already in raw format, *or via
> qemu-img convert* if not.
>
> > +
> > +=item * C<< qcow2+size >>, C<< vmdk >> (image files only)
>
> missing +size for vmdk
>
> > +
> > +A raw C<qcow2>/C<vmdk>/... file as produced via C<< dd if=some_file.qcow2 >>
> > +for files which are already in C<qcow2> format, or via C<qemu-img convert>.
>
> "A raw qcow2/vmdk/.. file" is confusing..
>
> A binary data stream containing the qcow2/vmdk-formatted contents of a
> qcow2/vmdk file as produced via ..
>
> the qemu-img convert part got moved to the wrong format, it's not needed
> to produce a qcow2+size stream for raw files (we don't do that), but to
> produce a raw+size stream from a qcow2 file, see above.
>
> > +
> > +B<NOTE:> These formats are only valid with C<$with_snapshots> being true (C<1>).
>
> that's not strictly speaking true, but an implementation detail of the
> current implementation. what is true is that raw+size can not contain
> snapshots for obvious reasons.
>
> > +
> > +=item * C<< tar+size >> (subvolumes only)
> > +
> > +A GNU C<tar> stream containing just the inner contents of the subvolume. This
> > +does not distinguish between the contents of a privileged or unprivileged
> > +container. In other words, this is from the root user namespace's point of view
> > +with no uid-mapping in effect. As produced via e.g.
> > +C<< tar -C vm-100-disk-1.subvol -cpf output_file.dat . >>
>
> what is "inner"? should/must the content be relative or absolute?
> anchored? ...
>
> > +
> > +=back
> > +
> >  =cut
> >  
> > -# Import/Export interface:
> > -#   Any path based storage is assumed to support 'raw' and 'tar' streams, so
> > -#   the default implementations will return this if $scfg->{path} is set,
> > -#   mimicking the old PVE::Storage::storage_migrate() function.
> > -#
> > -# Plugins may fall back to PVE::Storage::Plugin::volume_{export,import}...
> > -#   functions in case the format doesn't match their specialized
> > -#   implementations to reuse the raw/tar code.
> > -#
> > -# Format specification:
> > -#   The following formats are all prefixed with image information in the form
> > -#   of a 64 bit little endian unsigned integer (pack('Q<')) in order to be able
> > -#   to preallocate the image on storages which require it.
> > -#
> > -#   raw+size: (image files only)
> > -#     A raw binary data stream such as produced via `dd if=TheImageFile`.
> > -#   qcow2+size, vmdk: (image files only)
> > -#     A raw qcow2/vmdk/... file such as produced via `dd if=some.qcow2` for
> > -#     files which are already in qcow2 format, or via `qemu-img convert`.
> > -#     Note that these formats are only valid with $with_snapshots being true.
> > -#   tar+size: (subvolumes only)
> > -#     A GNU tar stream containing just the inner contents of the subvolume.
> > -#     This does not distinguish between the contents of a privileged or
> > -#     unprivileged container. In other words, this is from the root user
> > -#     namespace's point of view with no uid-mapping in effect.
> > -#     As produced via `tar -C vm-100-disk-1.subvol -cpf TheOutputFile.dat .`
> > +=head3 $plugin->volume_export(\%scfg, $storeid, $fh, $volname, $format [, $snapshot, $base_snapshot, $with_snapshots])
> > +
> > +=head3 $plugin->volume_export(...)
> > +
> > +Exports a volume or a volume's C<$snapshot> into a file handle C<$fh> as a
> > +stream with a desired export C<$format>. See L<FORMATS> for all import/export
> > +formats.
> > +
> > +Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
> > +C<$with_snapshots> states whether the volume has snapshots overall.
>
> this is incomplete/wrong
>
> $with_snapshots means the export should include snapshots, not whether
> the volume has snapshots..
> $snapshot means "this is the snapshot to export" if only exporting the
> snapshot ($with_snapshots == 0), or "this is the *last* snapshot export"
> if exporting a stream of snapshots ($with_snapshots == 1)
> $base_snapshot means "this is the start of the snapshot range to export"
> (i.e., do an incremental export on top of this base)
>
> this is mostly only relevant for zfs at the moment (other storages can
> either export a volume including its snapshots, or just the volume, but
> no complicated incremental streams of snapshots), but will change once
> we implement replication export/import for other storages..
>
> > +
> > +C<die>s in order to abort the export if there are (grave) problems, if the
> > +given C<$format> is not supported, or if exporting volumes is not supported as
> > +a whole.
> > +
> > +=cut
> >  
> > -# Export a volume into a file handle as a stream of desired format.
> >  sub volume_export {
> >      my ($class, $scfg, $storeid, $fh, $volname, $format, $snapshot, $base_snapshot, $with_snapshots) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->volume_export_formats(\%scfg, $storeid, $volname [, $snapshot, $base_snapshot, $with_snapshot])
> > +
> > +=head3 $plugin->volume_export_formats(...)
> > +
> > +B<OPTIONAL:> May be implemented in a storage plugin.
> > +
> > +Returns a list of supported export formats for the given volume or snapshot.
> > +
> > +Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
> > +C<$with_snapshots> states whether the volume has snapshots overall.
>
> see above.. these parameters are used to affect the returned list of
> formats
>
> > +
> > +If the storage does not support exporting volumes at all, and empty list should
> > +be returned.
> > +
> > +=cut
> > +
> >  sub volume_export_formats {
> >      my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > -# Import data from a stream, creating a new or replacing or adding to an existing volume.
> > +=head3 $plugin->volume_import(\%scfg, $storeid, $fh, $volname, $format [, $snapshot, $base_snapshot, $with_snapshots, $allow_rename])
> > +
> > +=head3 $plugin->volume_import(...)
> > +
> > +B<OPTIONAL:> May be implemented in a storage plugin.
> > +
> > +Imports data with the given C<$format> from a stream / filehandle C<$fh>,
> > +creating a new volume, or replacing or adding to an existing one. Returns the
> > +volume ID of the imported volume.
>
> I don't think we ever replace an existing volume? what we might do is
> import new snapshots on top of base_snapshot in case of an incremental
> import..
>
> maybe
>
> creating a new volume or importing additional snapshots on top of an
> existing one.
>
> > +
> > +Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
> > +C<$with_snapshots> states whether the volume has snapshots overall. Renaming an
> > +existing volume may also optionally be allowed via C<$allow_rename>
>
> see above, but here $snapshot is mainly there to have the same
> arguments for volume_import_formats so a plugin can have a single
> implementation, not because it is used anywhere IIRC..
>
> > +
> > +C<die>s if the import fails, if the given C<$format> is not supported, or if
> > +importing volumes is not supported as a whole.
> > +
> > +=cut
> > +
> >  sub volume_import {
> >      my ($class, $scfg, $storeid, $fh, $volname, $format, $snapshot, $base_snapshot, $with_snapshots, $allow_rename) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> >  
> > +=head3 $plugin->volume_import_formats(\%scfg, $storeid, $volname [, $snapshot, $base_snapshot, $with_snapshot])
> > +
> > +=head3 $plugin->volume_import_formats(...)
> > +
> > +B<OPTIONAL:> May be implemented in a storage plugin.
> > +
> > +Returns a list of supported import formats for the given volume or snapshot.
> > +
> > +Optionally, C<$snapshot> (if provided) may have a C<$base_snapshot>, and
> > +C<$with_snapshots> states whether the volume has snapshots overall.
>
> see above, these parameters serve the same purpose as for export_formats
> - to affect the return value / inform the plugin about the intended
> export/import
>
> it might make sense to note somewhere that the order of returned formats
> matters for both, since the first element of the intersection of both
> return values will be used to do a storage migration?
>
> > +
> > +If the storage does not support importing volumes at all, and empty list should
> > +be returned.
> > +
> > +=cut
> > +
> >  sub volume_import_formats {
> >      my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
> >      croak "implement me in sub-class\n";
> >  }
> > -
> >  1;
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 2/8] pluginbase: add high-level plugin API description

[Fabian Grünbichler]

On April 2, 2025 6:31 pm, Max Carrara wrote:
> On Mon Mar 31, 2025 at 5:13 PM CEST, Fabian Grünbichler wrote:
>> On March 26, 2025 3:20 pm, Max Carrara wrote:
>> > Add a short paragraph in DESCRIPTION serving as an introduction as
>> > well as the GENERAL PARAMETERS and CACHING EXPENSIVE OPERATIONS
>> > sections.
>> > 
>> > These sections are added in order to avoid repeatedly describing the
>> > same parameters as well as to elaborate on / clarify a couple terms,
>> > e.g. what the $cache parameter does or what a volume in our case is.
>> > 
>> > Signed-off-by: Max Carrara <m.carrara@proxmox.com>
>> > ---
>> >  src/PVE/Storage/PluginBase.pm | 77 +++++++++++++++++++++++++++++++++++
>> >  1 file changed, 77 insertions(+)
>> > 
>> > diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
>> > index e56aa72..16977f3 100644
>> > --- a/src/PVE/Storage/PluginBase.pm
>> > +++ b/src/PVE/Storage/PluginBase.pm
>> > @@ -4,6 +4,83 @@ C<PVE::Storage::PluginBase> - Storage Plugin API Interface
>> >  
>> >  =head1 DESCRIPTION
>> >  
>> > +This module documents the public Storage Plugin API of PVE and serves
>> > +as a base for C<L<PVE::Storage::Plugin>>. Plugins must B<always> inherit from
>> > +C<L<PVE::Storage::Plugin>>, as this module is for documentation purposes
>> > +only.
>>
>> does this make sense? if we now provide a clean base for the structure
>> of plugins, why shouldn't plugins be able to use that, but instead have
>> to inherit from PVE::Storage::Plugin which has a lot of extra stuff that
>> makes things messy?
>>
>> granted, switching over to load from PluginBase could be done as a
>> follow up or with 9.0 (or not at all, if there is a rationale)..
>>
>> after this series we have:
>>
>> SectionConfig
>> -> PluginBase (not an actual base plugin w.r.t. SectionConfig, and not
>> something you base plugins on as a result)
>> --> Plugin (a combination of base plugin and base of all our
>> directory-based plugins)
>> ---> other plugins, including third party ones
>>
>> which seems unfortunate, even if the contents of PluginBase are helpful
>> when implementing your own..
>>
>> IMHO this should either be
>>
>> SectionConfig
>> -> PluginBase (actual base plugin, with SectionConfig implementations
>> moved over from current Plugin.pm as needed)
>> --> PluginTemplate (what's currently PluginBase in this series - nicely
>> documented parent of third party plugins, not actually registered, just
>> contains the storage.cfg interface without the low-level SectionConfig
>> things)
>> ---> NewThirdPartyPlugin (clean slate implementation just using the
>> nicely documented interfaces, guaranteed to not use any helpers from
>> Plugin since it's not a (grand)parent)
>> --> Plugin (base of built-in plugins, probably base of existing third
>> party plugins, should ideally have a different name in the future!)
>> ---> DirPlugin
>> ---> ...
>> ---> ExistingThirdPartyPlugin (this might rely on helpers from Plugin,
>> so we can't just rename that one unless we wait for 9.0)
>>
>> or
>>
>> SectionConfig
>> -> PluginBase (actual base plugin + docs of Plugin API)
>> --> Plugin (base of our plugins and existing third party ones, dir-related helpers, ..)
>> ---> other plugins, including third party ones
>> --> NewThirdPartyPlugin (clean slate as above)
> 
> I agree with your point here -- for now, `::PluginBase` should IMO still
> only be about documentation and enumerating what's part of the Plugin
> API, *but* adding more layers inbetween can still be done eventually.
> 
> However, I think we shouldn't have two different parents for internal
> and external plugins, as it would introduce yet another thing we'd have
> to track wrt. APIAGE resets etc. Maybe it's not actually an issue in
> this case, but ...

the API would be defined by the "public"/external/top-level base
obviously, not by our intermediate shared base ;)

> That actually gives me an idea that's similar to your first suggestion:
> 
> As of this series, the hierarchy would be as follows:
> 
> PluginBase
> └── Plugin
>     ├── ExistingThirdPartyPlugin
>     ├── DirPlugin
>     └── ...
> 
> We could keep `PluginBase` as it is, since IMO having the docs and the
> SectionConfig-related stuff in one place is fine, unless we really want
> to keep the docs separate from the rest of the code. (Would seem a bit
> redundant to introduce another inheritance layer in that case, but I
> personally don't mind.)

I don't mind having the SectionConfig and the "skeleton" in one module,
provided they are clearly separated. the SectionConfig methods are
inherited anyway and any third party plugin must uphold the invariant of
not messing with them..

> Then, we eventually introduce `PluginTemplate` on the same layer as
> `Plugin`. `PluginTemplate` would only contain implementations for the
> most basic methods (and not provide defaults for file-based storages).

do you have something in mind for this "Template"? and if those methods
were so generic and basic at the same time, why shouldn't they live in
PluginBase itself? ;) I am not yet convinced an extra layer like this
makes much sense, but I am happy to listen to (concrete) arguments why
it should exist. IMHO the less code inherited by external plugins the
better, else we always have to not touch that code for ages to avoid
breaking them..

> PluginBase
> ├── Plugin
> │   ├── ExistingThirdPartyPlugin
> │   ├── DirPlugin
> │   └── ...
> └── PluginTemplate
> 
> The idea behind this is that we could then "migrate" each plugin and
> base it off `PluginTemplate` instead. Helpers that are shared between
> plugins could go into `PVE::Storage::Common::*` instead of being
> implicitly becoming part of plugins' modules due to inheritance.

the issue with that is that we've now just moved the problem - what are
the stability guarantees for PVE::Storage::Common::* ? are external
plugins allowed/supposed to use them? I do think we want to keep some
amount of "PVE-specific, internal" helper code (whether in the plugins
themselves or in some standalone helper module) that are off-limits for
external plugins, if just to avoid writing us into a corner. the easiest
way to achieve that is to migrate external plugins away from Plugin.pm
(which we can enforce at plugin loading time at some point after a
deprecation period).

obviously for helpers that we deem stable enough moving them to Common
and wrapping them in their old location in Plugin would be a viable
migration strategy.

> PluginBase
> ├── Plugin
> │   ├── ...
> │   └── ExistingThirdPartyPlugin
> └── PluginTemplate
>     ├── ...
>     └── DirPlugin (cleaned up)
> 
> That way we could step by step "disentangle" the existing plugins from
> each other without having to constantly keep the original behaviour(s)
> of `Plugin` in the back of one's head and account for them. Instead,
> each plugin would implement precisely what it itself needs.
> 
> Since both the old `Plugin` and the new `PluginTemplate` share the same
> interface, namely `PluginBase`, we could still support the old `Plugin`
> until everything's been moved over and third-party devs have had enough
> time to adapt their own code, too.

see above - we can do all of that without introducing PluginTemplate as
well, including at some point no longer allowing third party plugins to
re-use Plugin but force them to be based of PluginBase.

> While doing all this, we could also rework parts of the API that didn't
> age that gracefully, perhaps deprecate certain old methods, introduce
> new methods, etc. as we'd have a "clean" break, sotospeak.

this could also be done without moving all plugins over to
PluginTemplate, although it might be a bit more messy/dangerous.

> So, to summarize my idea:
> - Keep `PluginBase` as it is right now, but also include
>   SectionConfig-related code
> - Introduce `PluginTemplate` with minimal implementation later down the
>   line on the same inheritance layer as `Plugin`
> - Slowly migrate our plugins, basing them off of `PluginTemplate` while
>   tossing out old code, making them independent from one another, and
>   collecting shared helpers in `PVE::Storage::Common::*`
> 
> I'd really like to hear your thoughts on this, because I'm not sure if
> this is *actually* feasible or provides any ROI down the line. One
> alternative that I can think of is to just keep the inheritance
> hierarchy as it is (as in this series) and disentangle the plugins as
> they are right now, without changing their parent (so, almost the same
> as your second idea). I did start breaking apart our plugins like that
> last year, but that was too much all at once [1].
> 
> [1]: https://lore.proxmox.com/pve-devel/20240717094034.124857-1-m.carrara@proxmox.com/


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 1/8] pluginbase: introduce PVE::Storage::PluginBase with doc scaffold

[Fabian Grünbichler]

On April 2, 2025 6:31 pm, Max Carrara wrote:
> On Mon Mar 31, 2025 at 5:13 PM CEST, Fabian Grünbichler wrote:
>> On March 26, 2025 3:20 pm, Max Carrara wrote:
>> > +=head2 HOOKS
>> > +
>> > +=cut
>> > +
>> > +# called during addition of storage (before the new storage config got written)
>>
>> called when adding a storage config entry, before the new config gets written
>>
>> > +# die to abort addition if there are (grave) problems
>> > +# NOTE: runs in a storage config *locked* context
>> > +sub on_add_hook {
>> > +    my ($class, $storeid, $scfg, %param) = @_;
>> > +    return undef;
>> > +}
>> > +
>> > +# called during storage configuration update (before the updated storage config got written)
>>
>> called when updating a storage config entry, before the updated config
>> gets written
>>
>> > +# die to abort the update if there are (grave) problems
>> > +# NOTE: runs in a storage config *locked* context
>> > +sub on_update_hook {
>> > +    my ($class, $storeid, $scfg, %param) = @_;
>> > +    return undef;
>> > +}
>> > +
>> > +# called during deletion of storage (before the new storage config got written)
>> > +# and if the activate check on addition fails, to cleanup all storage traces
>> > +# which on_add_hook may have created.
>>
>> called when deleting a storage config entry, before the new storage
>> config gets written.
>>
>> also called as part of error handling when undoing the addition of a new
>> storage config entry.
> 
> Regarding your three responses above: The comments here were preserved
> from `::Plugin` for context's sake. But tbh, on second thought, they can
> probably just be removed, as they'll be replaced by POD anyways.
> 

yes, but those parts mor or less made it to the POD as well slightly
rephrased, hence I called them out here already :-P

>>
>> > +# die to abort deletion if there are (very grave) problems
>> > +# NOTE: runs in a storage config *locked* context
>> > +sub on_delete_hook {
>> > +    my ($class, $storeid, $scfg) = @_;
>> > +    return undef;
>> > +}
>> > +
>> > +=head2 IMAGE OPERATIONS
>> > +
>>
>> should this describe what IMAGES are in the context of PVE? else as a
>> newcomer the difference between IMAGE here and VOLUME below might not
>> be clear..
> 
> Yeah, that's a good idea. Maximiliano and I were also thinking about
> maybe adding a GLOSSARY section at the bottom of the file where certain
> terms could be explained / defined in more detail in general.
> What do you think?
> 
> Alternatively, we could also have the top-level description define the
> most basic of terms, but I don't want to load the docs here with too
> much information up front.

I think it depends - things that are not only interesting for plugin
devs should probably go into docs proper and just be referenced here. I
don't think the distinction between volume and image is very important
outside of development, so that might be explained here?

the POD documentation here is targeted at us and plugin devs, so that
should be the guideline when deciding what goes where - and what goes in
in the first place - I think a focus on "what do I need to know/be aware
of when writing a plugin" (or adapting the interface) should be the
guiding principle for that.

e.g. for that knowing that this parameter is optional doesn't buy me
much. knowing what it means if it is set does (or that I can ignore it
if my storage can't do X, or should die if set in that case)


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 6/8] pluginbase: document image operation methods

[Fabian Grünbichler]

On April 2, 2025 6:32 pm, Max Carrara wrote:
> On Mon Mar 31, 2025 at 5:12 PM CEST, Fabian Grünbichler wrote:
>> On March 26, 2025 3:20 pm, Max Carrara wrote:
>> > Add documentation for the following methods:
>> > - list_images
>> > - create_base
>> > - clone_image
>> > - alloc_image
>> > - free_image
>> > 
>> > Signed-off-by: Max Carrara <m.carrara@proxmox.com>
>> > Co-authored-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
>> > ---
>> >  src/PVE/Storage/PluginBase.pm | 111 ++++++++++++++++++++++++++++++++++
>> >  1 file changed, 111 insertions(+)
>> > 
>> > diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
>> > index b3ce684..37b1471 100644
>> > --- a/src/PVE/Storage/PluginBase.pm
>> > +++ b/src/PVE/Storage/PluginBase.pm
>> > @@ -721,26 +721,137 @@ sub on_delete_hook {
>> >  
>> >  =cut
>> >  
>> > +=head3 $plugin->list_images($storeid, \%scfg [, $vmid, \@vollist, \%cache])
>> > +
>> > +B<REQUIRED:> Must be implemented in every storage plugin.
>> > +
>> > +Returns a listref of all disk images of a storage. If the storage does not
>> > +support storing disk images, returns an empty listref.
>> > +
>> > +Optionally, if C<\@vollist> is provided, return only disks whose volume ID is
>> > +within C<\@vollist>. Note that this usually has higher precedence than
>> > +C<$vmid>.
> 
> Upfront note: Unless I otherwise comment something, I agree with you.
> Just sparing myself from writing and you from reading too many ACKs :P
> 
>>
>> what does usually mean? what does $vmid do?
> 
> Sorry, this got lost when tossing out some redundant paragraphs here.
> 
> The "usually" can be dropped; C<$vmid> is the guest's ID (mentioned in
> the DESCRIPTION in patch 01) and unless C<\@vollist> is provided, the
> images that the given C<$vmid> owns will be returned.

in case that wasn't obvious - I know what $vmid is (and also what it
does) - it should be explained to readers that don't (yet).

So I think this should say something like:

If C<$vmid> is set, only images owned by that VMID must be included in
the return value.

If C<\@vollist> is set and not empty, the return value must be filtered
using the contained volids.

If both C<$vmid> and C<\@vollist> are set, only C<\@vollist> should be
honored.

and about that last part - do we actually have callers that set both? if
not, should we instead recommend treating it as an error and document it
as such?

>> > +
>> > +C<die>s in case of errors.
>> > +
>> > +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
>> > +
>> > +The returned listref has the following structure:
>> > +
>> > +    [
>> > +	{
>> > +	    ctime => "1689163322", # creation time as unix timestamp
>> > +	    format => "raw",
>> > +	    size => 8589934592, # in bytes!
>> > +	    vmid => 101,
>> > +	    volid => "local-lvm:base-101-disk-0", # volume ID, storage-specific
>> > +	},
>> > +	# [...]
>> > +    ]
>> > +
>> > +=cut
>> > +
>> >  sub list_images {
>> >      my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
>> >      croak "implement me in sub-class\n";
>> >  }
>> >  
>> > +=head3 $plugin->create_base($storeid, \%scfg, $volname)
>> > +
>> > +B<OPTIONAL:> May be implemented in a storage plugin.
>> > +
>> > +Creates a base volume from an existing volume, allowing the volume to be
>>
>> this is misleading - it doesn't create a base volume from an existing
>> volume, it *converts* an existing volume into a base volume!
> 
> Oh dang, thanks for pointing this out! Sorry for the oversight.
> 
>>
>> > +L<< cloned|/"$plugin->clone_image(...)" >>. This cloned volume (usually
>> > +a disk image) may then be used as a base for the purpose of creating linked
>>
>> this is wrong? there is no cloned volume yet? and all of this only
>> applies to images and not other volumes?
>>
>> > +clones. See L<C<PVE::Storage::LvmThinPlugin>> and
>> > +L<C<PVE::Storage::ZFSPoolPlugin>> for example implementations.
>> > +
>> > +On completion, returns the name of the new base volume (the new C<$volname>).
>> > +
>> > +This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
>> > +i.e. when the storage is B<locked>.
>>
>> Shouldn't this say that it *should* be called in a locked context?
>>
>> > +
>> > +=cut
>> > +
>> >  sub create_base {
>> >      my ($class, $storeid, $scfg, $volname) = @_;
>> >      croak "implement me in sub-class\n";
>> >  }
>> >  
>> > +=head3 $plugin->clone_image($scfg, $storeid, $volname, $vmid [, $snap])
>> > +
>> > +=head3 $plugin->clone_image(...)
>> > +
>> > +B<REQUIRED:> Must be implemented in every storage plugin.
>>
>> not really? there's a feature guard for it, so only storage plugins that
>> support it should ever get it called anyway..
>>
>> what does $vmid mean?
> 
> See above; the guest's ID. Since it was unclear, the parameter should
> probably be explained here (and also above) instead of (just) in the
> DESCRIPTION section.

I know what a vmid is ;) the question is what does this parameter *mean*
for this particular method?

C<$vmid> contains the owner of the new volume, which must be encoded in
the returned volid.

or something like that?

>> > +
>> > +Clones a disk image or a snapshot of an image, returning the name of the new
>> > +image (the new C<$volname>). Note that I<cloning> here means to create a linked
>> > +clone and not duplicating an image. See L<C<PVE::Storage::LvmThinPlugin>> and
>> > +L<C<PVE::Storage::ZFSPoolPlugin>> for example implementations.
>>
>> then why not start with
>>
>> "Creates a linked clone of an existing disk image or snapshot of an
>> image"
>>
>> ? maybe also include that unless the linked clone is 100% independent
>> from the base (only true for LVM thin atm?), the new volid should encode
>> the base->clone relation in the volname?
>>
>> > +
>> > +C<die>s in case of an error of if the underlying storage doesn't support
>>
>> s/of/or/
>>
>> > +cloning images.
>> > +
>> > +This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
>> > +i.e. when the storage is B<locked>.
>>
>> same as above
>>
>> > +
>> > +=cut
>> > +
>> >  sub clone_image {
>> >      my ($class, $scfg, $storeid, $volname, $vmid, $snap) = @_;
>> >      croak "implement me in sub-class\n";
>> >  }
>> >  
>> > +=head3 $plugin->alloc_image($storeid, $scfg, $vmid, $fmt, $name, $size)
>> > +
>> > +B<REQUIRED:> Must be implemented in every storage plugin.
>> > +
>> > +Allocates a disk image with the given format C<$fmt> and C<$size> in bytes,
>> > +returning the name of the new image (the new C<$volname>). See
>> > +C<L<< plugindata()|/"$plugin->plugindata()" >>> for all disk formats.
>> > +
>> > +Optionally, if given, set the name of the image to C<$name>. If C<$name> isn't
>> > +provided, the next name should be determined via C<L<< find_free_diskname()|/"$plugin->find_free_diskname(...)" >>>.
>>
>> a suitable name should be automatically generated, unless we really want
>> to stabilize find_free_diskname as part of the API..
>>
>> what does $vmid mean?

here as well, similar to clone_image above


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 1/8] pluginbase: introduce PVE::Storage::PluginBase with doc scaffold

[Max Carrara]

On Thu Apr 3, 2025 at 9:12 AM CEST, Fabian Grünbichler wrote:
> On April 2, 2025 6:31 pm, Max Carrara wrote:
> > On Mon Mar 31, 2025 at 5:13 PM CEST, Fabian Grünbichler wrote:
> >> On March 26, 2025 3:20 pm, Max Carrara wrote:
> >> > +=head2 HOOKS
> >> > +
> >> > +=cut
> >> > +
> >> > +# called during addition of storage (before the new storage config got written)
> >>
> >> called when adding a storage config entry, before the new config gets written
> >>
> >> > +# die to abort addition if there are (grave) problems
> >> > +# NOTE: runs in a storage config *locked* context
> >> > +sub on_add_hook {
> >> > +    my ($class, $storeid, $scfg, %param) = @_;
> >> > +    return undef;
> >> > +}
> >> > +
> >> > +# called during storage configuration update (before the updated storage config got written)
> >>
> >> called when updating a storage config entry, before the updated config
> >> gets written
> >>
> >> > +# die to abort the update if there are (grave) problems
> >> > +# NOTE: runs in a storage config *locked* context
> >> > +sub on_update_hook {
> >> > +    my ($class, $storeid, $scfg, %param) = @_;
> >> > +    return undef;
> >> > +}
> >> > +
> >> > +# called during deletion of storage (before the new storage config got written)
> >> > +# and if the activate check on addition fails, to cleanup all storage traces
> >> > +# which on_add_hook may have created.
> >>
> >> called when deleting a storage config entry, before the new storage
> >> config gets written.
> >>
> >> also called as part of error handling when undoing the addition of a new
> >> storage config entry.
> > 
> > Regarding your three responses above: The comments here were preserved
> > from `::Plugin` for context's sake. But tbh, on second thought, they can
> > probably just be removed, as they'll be replaced by POD anyways.
> > 
>
> yes, but those parts mor or less made it to the POD as well slightly
> rephrased, hence I called them out here already :-P

Fair :P

>
> >>
> >> > +# die to abort deletion if there are (very grave) problems
> >> > +# NOTE: runs in a storage config *locked* context
> >> > +sub on_delete_hook {
> >> > +    my ($class, $storeid, $scfg) = @_;
> >> > +    return undef;
> >> > +}
> >> > +
> >> > +=head2 IMAGE OPERATIONS
> >> > +
> >>
> >> should this describe what IMAGES are in the context of PVE? else as a
> >> newcomer the difference between IMAGE here and VOLUME below might not
> >> be clear..
> > 
> > Yeah, that's a good idea. Maximiliano and I were also thinking about
> > maybe adding a GLOSSARY section at the bottom of the file where certain
> > terms could be explained / defined in more detail in general.
> > What do you think?
> > 
> > Alternatively, we could also have the top-level description define the
> > most basic of terms, but I don't want to load the docs here with too
> > much information up front.
>
> I think it depends - things that are not only interesting for plugin
> devs should probably go into docs proper and just be referenced here. I
> don't think the distinction between volume and image is very important
> outside of development, so that might be explained here?
>
> the POD documentation here is targeted at us and plugin devs, so that
> should be the guideline when deciding what goes where - and what goes in
> in the first place - I think a focus on "what do I need to know/be aware
> of when writing a plugin" (or adapting the interface) should be the
> guiding principle for that.
>
> e.g. for that knowing that this parameter is optional doesn't buy me
> much. knowing what it means if it is set does (or that I can ignore it
> if my storage can't do X, or should die if set in that case)

Yeah that's basically what we've tried to follow as much as possible
here; just wasn't sure *how much* information is okay. For now I'll keep
it slim and just exlpain the differences between images and volumes and
*perhaps* any other terms that might come up, but only if they're of
immediate relevancy to the plugin API.

Though, I'll add the glossary idea to my backlog; might be something
that could be useful for the wiki / docs in general.

>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 2/8] pluginbase: add high-level plugin API description

[Max Carrara]

On Thu Apr 3, 2025 at 9:12 AM CEST, Fabian Grünbichler wrote:
> On April 2, 2025 6:31 pm, Max Carrara wrote:
> > On Mon Mar 31, 2025 at 5:13 PM CEST, Fabian Grünbichler wrote:
> >> On March 26, 2025 3:20 pm, Max Carrara wrote:
> >> > Add a short paragraph in DESCRIPTION serving as an introduction as
> >> > well as the GENERAL PARAMETERS and CACHING EXPENSIVE OPERATIONS
> >> > sections.
> >> > 
> >> > These sections are added in order to avoid repeatedly describing the
> >> > same parameters as well as to elaborate on / clarify a couple terms,
> >> > e.g. what the $cache parameter does or what a volume in our case is.
> >> > 
> >> > Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> >> > ---
> >> >  src/PVE/Storage/PluginBase.pm | 77 +++++++++++++++++++++++++++++++++++
> >> >  1 file changed, 77 insertions(+)
> >> > 
> >> > diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> >> > index e56aa72..16977f3 100644
> >> > --- a/src/PVE/Storage/PluginBase.pm
> >> > +++ b/src/PVE/Storage/PluginBase.pm
> >> > @@ -4,6 +4,83 @@ C<PVE::Storage::PluginBase> - Storage Plugin API Interface
> >> >  
> >> >  =head1 DESCRIPTION
> >> >  
> >> > +This module documents the public Storage Plugin API of PVE and serves
> >> > +as a base for C<L<PVE::Storage::Plugin>>. Plugins must B<always> inherit from
> >> > +C<L<PVE::Storage::Plugin>>, as this module is for documentation purposes
> >> > +only.
> >>
> >> does this make sense? if we now provide a clean base for the structure
> >> of plugins, why shouldn't plugins be able to use that, but instead have
> >> to inherit from PVE::Storage::Plugin which has a lot of extra stuff that
> >> makes things messy?
> >>
> >> granted, switching over to load from PluginBase could be done as a
> >> follow up or with 9.0 (or not at all, if there is a rationale)..
> >>
> >> after this series we have:
> >>
> >> SectionConfig
> >> -> PluginBase (not an actual base plugin w.r.t. SectionConfig, and not
> >> something you base plugins on as a result)
> >> --> Plugin (a combination of base plugin and base of all our
> >> directory-based plugins)
> >> ---> other plugins, including third party ones
> >>
> >> which seems unfortunate, even if the contents of PluginBase are helpful
> >> when implementing your own..
> >>
> >> IMHO this should either be
> >>
> >> SectionConfig
> >> -> PluginBase (actual base plugin, with SectionConfig implementations
> >> moved over from current Plugin.pm as needed)
> >> --> PluginTemplate (what's currently PluginBase in this series - nicely
> >> documented parent of third party plugins, not actually registered, just
> >> contains the storage.cfg interface without the low-level SectionConfig
> >> things)
> >> ---> NewThirdPartyPlugin (clean slate implementation just using the
> >> nicely documented interfaces, guaranteed to not use any helpers from
> >> Plugin since it's not a (grand)parent)
> >> --> Plugin (base of built-in plugins, probably base of existing third
> >> party plugins, should ideally have a different name in the future!)
> >> ---> DirPlugin
> >> ---> ...
> >> ---> ExistingThirdPartyPlugin (this might rely on helpers from Plugin,
> >> so we can't just rename that one unless we wait for 9.0)
> >>
> >> or
> >>
> >> SectionConfig
> >> -> PluginBase (actual base plugin + docs of Plugin API)
> >> --> Plugin (base of our plugins and existing third party ones, dir-related helpers, ..)
> >> ---> other plugins, including third party ones
> >> --> NewThirdPartyPlugin (clean slate as above)
> > 
> > I agree with your point here -- for now, `::PluginBase` should IMO still
> > only be about documentation and enumerating what's part of the Plugin
> > API, *but* adding more layers inbetween can still be done eventually.
> > 
> > However, I think we shouldn't have two different parents for internal
> > and external plugins, as it would introduce yet another thing we'd have
> > to track wrt. APIAGE resets etc. Maybe it's not actually an issue in
> > this case, but ...
>
> the API would be defined by the "public"/external/top-level base
> obviously, not by our intermediate shared base ;)

Fair point :P

>
> > That actually gives me an idea that's similar to your first suggestion:
> > 
> > As of this series, the hierarchy would be as follows:
> > 
> > PluginBase
> > └── Plugin
> >     ├── ExistingThirdPartyPlugin
> >     ├── DirPlugin
> >     └── ...
> > 
> > We could keep `PluginBase` as it is, since IMO having the docs and the
> > SectionConfig-related stuff in one place is fine, unless we really want
> > to keep the docs separate from the rest of the code. (Would seem a bit
> > redundant to introduce another inheritance layer in that case, but I
> > personally don't mind.)
>
> I don't mind having the SectionConfig and the "skeleton" in one module,
> provided they are clearly separated. the SectionConfig methods are
> inherited anyway and any third party plugin must uphold the invariant of
> not messing with them..
>
> > Then, we eventually introduce `PluginTemplate` on the same layer as
> > `Plugin`. `PluginTemplate` would only contain implementations for the
> > most basic methods (and not provide defaults for file-based storages).
>
> do you have something in mind for this "Template"? and if those methods
> were so generic and basic at the same time, why shouldn't they live in
> PluginBase itself? ;) I am not yet convinced an extra layer like this
> makes much sense, but I am happy to listen to (concrete) arguments why
> it should exist. IMHO the less code inherited by external plugins the
> better, else we always have to not touch that code for ages to avoid
> breaking them..

No that's a good point; this entire idea here could actually also be
done without introducing `PluginTemplate`.

>
> > PluginBase
> > ├── Plugin
> > │   ├── ExistingThirdPartyPlugin
> > │   ├── DirPlugin
> > │   └── ...
> > └── PluginTemplate
> > 
> > The idea behind this is that we could then "migrate" each plugin and
> > base it off `PluginTemplate` instead. Helpers that are shared between
> > plugins could go into `PVE::Storage::Common::*` instead of being
> > implicitly becoming part of plugins' modules due to inheritance.
>
> the issue with that is that we've now just moved the problem - what are
> the stability guarantees for PVE::Storage::Common::* ? are external
> plugins allowed/supposed to use them? I do think we want to keep some
> amount of "PVE-specific, internal" helper code (whether in the plugins
> themselves or in some standalone helper module) that are off-limits for
> external plugins, if just to avoid writing us into a corner. the easiest
> way to achieve that is to migrate external plugins away from Plugin.pm
> (which we can enforce at plugin loading time at some point after a
> deprecation period).

Yeah I agree, I'm just not entirely sure yet which *exact* approach here
would be the best.

If you'd like, we can revisit this topic once it becomes relevant; for
now I think we can move forward with the inheritance structure that this
series proposes, that is:

SectionConfig
└── PluginBase
    └── Plugin
        ├── DirPlugin
        ├── ...
        ├── ExistingThirdPartyPlugin
        └── ...

In a future series, I'd then proceed with moving the
SectionConfig-related stuff to PluginBase (and perhaps other
"fundamental" stuff; yet to be decided what). When that happens, I'll
also see what "migration approach" would be the best, what internal /
external helpers to have, stability guarantees, etc.

The only thing I would definitely want to avoid is introducing another
inheritance layer somewhere, as it's already somewhat hard to keep track
of things.

>
> obviously for helpers that we deem stable enough moving them to Common
> and wrapping them in their old location in Plugin would be a viable
> migration strategy.

I agree here as well; that's something I attempted a long while ago.
I'm looking forward to revisiting that 👀

>
> > PluginBase
> > ├── Plugin
> > │   ├── ...
> > │   └── ExistingThirdPartyPlugin
> > └── PluginTemplate
> >     ├── ...
> >     └── DirPlugin (cleaned up)
> > 
> > That way we could step by step "disentangle" the existing plugins from
> > each other without having to constantly keep the original behaviour(s)
> > of `Plugin` in the back of one's head and account for them. Instead,
> > each plugin would implement precisely what it itself needs.
> > 
> > Since both the old `Plugin` and the new `PluginTemplate` share the same
> > interface, namely `PluginBase`, we could still support the old `Plugin`
> > until everything's been moved over and third-party devs have had enough
> > time to adapt their own code, too.
>
> see above - we can do all of that without introducing PluginTemplate as
> well, including at some point no longer allowing third party plugins to
> re-use Plugin but force them to be based of PluginBase.
>
> > While doing all this, we could also rework parts of the API that didn't
> > age that gracefully, perhaps deprecate certain old methods, introduce
> > new methods, etc. as we'd have a "clean" break, sotospeak.
>
> this could also be done without moving all plugins over to
> PluginTemplate, although it might be a bit more messy/dangerous.
>
> > So, to summarize my idea:
> > - Keep `PluginBase` as it is right now, but also include
> >   SectionConfig-related code
> > - Introduce `PluginTemplate` with minimal implementation later down the
> >   line on the same inheritance layer as `Plugin`
> > - Slowly migrate our plugins, basing them off of `PluginTemplate` while
> >   tossing out old code, making them independent from one another, and
> >   collecting shared helpers in `PVE::Storage::Common::*`
> > 
> > I'd really like to hear your thoughts on this, because I'm not sure if
> > this is *actually* feasible or provides any ROI down the line. One
> > alternative that I can think of is to just keep the inheritance
> > hierarchy as it is (as in this series) and disentangle the plugins as
> > they are right now, without changing their parent (so, almost the same
> > as your second idea). I did start breaking apart our plugins like that
> > last year, but that was too much all at once [1].
> > 
> > [1]: https://lore.proxmox.com/pve-devel/20240717094034.124857-1-m.carrara@proxmox.com/
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v1 pve-storage 6/8] pluginbase: document image operation methods

[Max Carrara]

On Thu Apr 3, 2025 at 9:23 AM CEST, Fabian Grünbichler wrote:
> On April 2, 2025 6:32 pm, Max Carrara wrote:
> > On Mon Mar 31, 2025 at 5:12 PM CEST, Fabian Grünbichler wrote:
> >> On March 26, 2025 3:20 pm, Max Carrara wrote:
> >> > Add documentation for the following methods:
> >> > - list_images
> >> > - create_base
> >> > - clone_image
> >> > - alloc_image
> >> > - free_image
> >> > 
> >> > Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> >> > Co-authored-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
> >> > ---
> >> >  src/PVE/Storage/PluginBase.pm | 111 ++++++++++++++++++++++++++++++++++
> >> >  1 file changed, 111 insertions(+)
> >> > 
> >> > diff --git a/src/PVE/Storage/PluginBase.pm b/src/PVE/Storage/PluginBase.pm
> >> > index b3ce684..37b1471 100644
> >> > --- a/src/PVE/Storage/PluginBase.pm
> >> > +++ b/src/PVE/Storage/PluginBase.pm
> >> > @@ -721,26 +721,137 @@ sub on_delete_hook {
> >> >  
> >> >  =cut
> >> >  
> >> > +=head3 $plugin->list_images($storeid, \%scfg [, $vmid, \@vollist, \%cache])
> >> > +
> >> > +B<REQUIRED:> Must be implemented in every storage plugin.
> >> > +
> >> > +Returns a listref of all disk images of a storage. If the storage does not
> >> > +support storing disk images, returns an empty listref.
> >> > +
> >> > +Optionally, if C<\@vollist> is provided, return only disks whose volume ID is
> >> > +within C<\@vollist>. Note that this usually has higher precedence than
> >> > +C<$vmid>.
> > 
> > Upfront note: Unless I otherwise comment something, I agree with you.
> > Just sparing myself from writing and you from reading too many ACKs :P
> > 
> >>
> >> what does usually mean? what does $vmid do?
> > 
> > Sorry, this got lost when tossing out some redundant paragraphs here.
> > 
> > The "usually" can be dropped; C<$vmid> is the guest's ID (mentioned in
> > the DESCRIPTION in patch 01) and unless C<\@vollist> is provided, the
> > images that the given C<$vmid> owns will be returned.
>
> in case that wasn't obvious - I know what $vmid is (and also what it
> does) - it should be explained to readers that don't (yet).
>
> So I think this should say something like:
>
> If C<$vmid> is set, only images owned by that VMID must be included in
> the return value.
>
> If C<\@vollist> is set and not empty, the return value must be filtered
> using the contained volids.
>
> If both C<$vmid> and C<\@vollist> are set, only C<\@vollist> should be
> honored.

Thanks for the suggestion, that's actually much better. I also agree
with the other suggestions further below.

>
> and about that last part - do we actually have callers that set both? if
> not, should we instead recommend treating it as an error and document it
> as such?

I'll revisit the call sites again and see whether we actually do, just
to be sure. The current code at least doesn't treat it as an error.

>
> >> > +
> >> > +C<die>s in case of errors.
> >> > +
> >> > +This method may reuse L<< cached information via C<\%cache>|/"CACHING EXPENSIVE OPERATIONS" >>.
> >> > +
> >> > +The returned listref has the following structure:
> >> > +
> >> > +    [
> >> > +	{
> >> > +	    ctime => "1689163322", # creation time as unix timestamp
> >> > +	    format => "raw",
> >> > +	    size => 8589934592, # in bytes!
> >> > +	    vmid => 101,
> >> > +	    volid => "local-lvm:base-101-disk-0", # volume ID, storage-specific
> >> > +	},
> >> > +	# [...]
> >> > +    ]
> >> > +
> >> > +=cut
> >> > +
> >> >  sub list_images {
> >> >      my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
> >> >      croak "implement me in sub-class\n";
> >> >  }
> >> >  
> >> > +=head3 $plugin->create_base($storeid, \%scfg, $volname)
> >> > +
> >> > +B<OPTIONAL:> May be implemented in a storage plugin.
> >> > +
> >> > +Creates a base volume from an existing volume, allowing the volume to be
> >>
> >> this is misleading - it doesn't create a base volume from an existing
> >> volume, it *converts* an existing volume into a base volume!
> > 
> > Oh dang, thanks for pointing this out! Sorry for the oversight.
> > 
> >>
> >> > +L<< cloned|/"$plugin->clone_image(...)" >>. This cloned volume (usually
> >> > +a disk image) may then be used as a base for the purpose of creating linked
> >>
> >> this is wrong? there is no cloned volume yet? and all of this only
> >> applies to images and not other volumes?
> >>
> >> > +clones. See L<C<PVE::Storage::LvmThinPlugin>> and
> >> > +L<C<PVE::Storage::ZFSPoolPlugin>> for example implementations.
> >> > +
> >> > +On completion, returns the name of the new base volume (the new C<$volname>).
> >> > +
> >> > +This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
> >> > +i.e. when the storage is B<locked>.
> >>
> >> Shouldn't this say that it *should* be called in a locked context?
> >>
> >> > +
> >> > +=cut
> >> > +
> >> >  sub create_base {
> >> >      my ($class, $storeid, $scfg, $volname) = @_;
> >> >      croak "implement me in sub-class\n";
> >> >  }
> >> >  
> >> > +=head3 $plugin->clone_image($scfg, $storeid, $volname, $vmid [, $snap])
> >> > +
> >> > +=head3 $plugin->clone_image(...)
> >> > +
> >> > +B<REQUIRED:> Must be implemented in every storage plugin.
> >>
> >> not really? there's a feature guard for it, so only storage plugins that
> >> support it should ever get it called anyway..
> >>
> >> what does $vmid mean?
> > 
> > See above; the guest's ID. Since it was unclear, the parameter should
> > probably be explained here (and also above) instead of (just) in the
> > DESCRIPTION section.
>
> I know what a vmid is ;) the question is what does this parameter *mean*
> for this particular method?
>
> C<$vmid> contains the owner of the new volume, which must be encoded in
> the returned volid.
>
> or something like that?
>
> >> > +
> >> > +Clones a disk image or a snapshot of an image, returning the name of the new
> >> > +image (the new C<$volname>). Note that I<cloning> here means to create a linked
> >> > +clone and not duplicating an image. See L<C<PVE::Storage::LvmThinPlugin>> and
> >> > +L<C<PVE::Storage::ZFSPoolPlugin>> for example implementations.
> >>
> >> then why not start with
> >>
> >> "Creates a linked clone of an existing disk image or snapshot of an
> >> image"
> >>
> >> ? maybe also include that unless the linked clone is 100% independent
> >> from the base (only true for LVM thin atm?), the new volid should encode
> >> the base->clone relation in the volname?
> >>
> >> > +
> >> > +C<die>s in case of an error of if the underlying storage doesn't support
> >>
> >> s/of/or/
> >>
> >> > +cloning images.
> >> > +
> >> > +This method is called in the context of C<L<< cluster_lock_storage()|/"cluster_lock_storage(...)" >>>,
> >> > +i.e. when the storage is B<locked>.
> >>
> >> same as above
> >>
> >> > +
> >> > +=cut
> >> > +
> >> >  sub clone_image {
> >> >      my ($class, $scfg, $storeid, $volname, $vmid, $snap) = @_;
> >> >      croak "implement me in sub-class\n";
> >> >  }
> >> >  
> >> > +=head3 $plugin->alloc_image($storeid, $scfg, $vmid, $fmt, $name, $size)
> >> > +
> >> > +B<REQUIRED:> Must be implemented in every storage plugin.
> >> > +
> >> > +Allocates a disk image with the given format C<$fmt> and C<$size> in bytes,
> >> > +returning the name of the new image (the new C<$volname>). See
> >> > +C<L<< plugindata()|/"$plugin->plugindata()" >>> for all disk formats.
> >> > +
> >> > +Optionally, if given, set the name of the image to C<$name>. If C<$name> isn't
> >> > +provided, the next name should be determined via C<L<< find_free_diskname()|/"$plugin->find_free_diskname(...)" >>>.
> >>
> >> a suitable name should be automatically generated, unless we really want
> >> to stabilize find_free_diskname as part of the API..
> >>
> >> what does $vmid mean?
>
> here as well, similar to clone_image above
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel