public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types
@ 2026-11-17 20:38 Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 01/51] test: plugin tests: run tests with at most 4 jobs Max R. Carrara
                   ` (44 more replies)
  0 siblings, 45 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types - v3
===========================================================================

Basically what the title says. Implement subdirectory scanning for
directory-based storage types, which includes directories (duh), NFS,
CIFS, CephFS, and BTRFS. This fixes #2884 [2884] and the more
narrowly-scoped #623 [623].

See the initial series's cover letter [init-cover] for the entire
context.

*Huge* thanks to @Wolfgang for the thorough reviews and the
correspondence off-list!

Maintainer Notes
----------------

- pve-manager will need a version bump on pve-storage

Notable Changes Since v2
------------------------

- Rebase on the latest master

- Take vtype-split changes that were recently introduced into account,
  notably the new `vm-vol`, `vm-state`, and `ct-vol` vtypes

- Do a huge API bump at the end of the changes for pve-storage instead
  of in between, since that is actually easier to manage (and rebase)

- Ignore symlinks when scanning through nested subdirectories, as that
  can otherwise lead to loops during the scan -- and those loops can
  then cause an exponential blowup in subdirectories being scanned

- Keep track of content directories while scanning subdirs to avoid
  traversing content dirs that may be contained in another content dir,
  e.g. with `iso=data,snippets=data/snippets`; use device + inode here,
  to *really* make sure we catch all edge cases (e.g. bind mounts)

- Explicitly remove trailing slashes from content directory overrides;
  with 'snippets=/' and 'images=imgs/', for example, a VM disk could
  have resolved to a snippet (yikes)

- Retain calls to `PVE::Storage::Plugin->get_subdir()` throughout the
  code, as that would otherwise break API -- third-party plugin authors
  are still encouraged to switch to the new helpers instead of
  overriding `get_subdir()`

Testing
-------

If anyone could give this series a spin, I'd be most grateful!

Here are some interesting things you could check out (non-exhaustive):
- Configuring the new 'max-scan-depth' property in the UI
- Adding subdirectories in your ISO, LXC template and snippets dirs
  (and populating those dirs afterwards)
- Checking whether the depth limit is honored
  --> 0 is the default, which retains the current behavior of not
      scanning through any subdirs
- Checking whether imports (.ova files etc) still work as expected
- ISO / CT template upload / deletion
- ...

Also note that I ran the tests in the repository for every single patch
that I added; if you want to do this for yourself as a sanity check, try
the following:

    git rebase -i --autostash --autosquash origin/master -x 'cd src && make test'

Previous Versions
-----------------

v1: https://lore.proxmox.com/pve-devel/20260422111322.257380-1-m.carrara@proxmox.com/
v2: https://lore.proxmox.com/pve-devel/20260923150606.531239-1-m.carrara@proxmox.com/

References
----------

[2884]: https://bugzilla.proxmox.com/show_bug.cgi?id=2884
[623]: https://bugzilla.proxmox.com/show_bug.cgi?id=623
[init-cover]: https://lore.proxmox.com/pve-devel/20260422111322.257380-1-m.carrara@proxmox.com/

Summary of Changes
------------------


pve-storage:

Max R. Carrara (50):
  test: plugin tests: run tests with at most 4 jobs
  plugin, common: remove superfluous use of =pod command paragraph
  common: use Exporter module for PVE::Storage::Common
  plugin: make get_subdir_files a proper subroutine and update style
  plugin api: replace subdir helper methods with standalone functions
  common: prevent autovivification in plugin_get_vtype_subdir helper
  plugin: break up if-elsif chain into separate if-blocks
  plugin: adapt get_subdir_files helper of list_volumes API method
  plugin: update code style of list_volumes plugin API method
  plugin: use closure for obtaining raw volume data in list_volumes
  plugin: use closure for inner loop logic in list_volumes
  storage: update code style in function path_to_volume_id
  storage: break up if-elsif chain in path_to_volume_id
  storage: heave vtype file path parsing logic inside loop into helper
  storage: clean up code that was moved into helper in path_to_volume_id
  api: status: move content type assert for up-/downloads into helper
  api: status: use get_subdir helper method to get content directory
  api: status: move up-/download file path parsing code into helper
  api: status: simplify file content assertion logic for up-/download
  test: guest import: add tests for PVE::GuestImport
  tree-wide: introduce parsing module and replace usages of ISO_EXT_RE_0
  common: test: set up parser testing code, add tests for 'iso' vtype
  tree-wide: replace usages of VZTMPL_EXT_RE_1 with parsing functions
  tree-wide: replace usages of BACKUP_EXT_RE_2 with parsing functions
  tree-wide: replace usages of inline regexes for snippets with parsers
  tree-wide: partially replace usages of regexes for 'import' vtype
  tree-wide: replace remaining usages of regexes for 'import' vtype
  tree-wide: simplify recently refactored parsing logic
  test: list volumes: reorganize and modernize test running code
  test: list volumes: fix broken test checking for vmlist modifications
  test: list volumes: introduce new format for test cases
  test: list volumes: remove legacy code and migrate cases to new format
  plugin: do not return volumes of undeclared content types anymore
  plugin: correct comment in get_subdir_files helper
  test: parse volname: modernize code
  test: parse volname: adapt tests regarding 'import' volume type
  test: parse volname: move VM disk test creation into separate block
  test: parse volname: move backup file test creation into sep. block
  test: parse volname: parameterize test case creation for some vtypes
  test: volume id: modernize code
  test: volume id: rename 'volname' test case parameter to 'file'
  test: filesystem path: modernize code
  plugin: list content directories without globbing
  fix #2884: implement nested subdir scanning and support 'iso' vtype
  fix #2884: support nested subdir scanning for 'vztmpl' volume type
  fix #2884: support nested subdir scanning for 'snippets' vtype
  test: add more tests for 'import' vtype & guard against nested subdirs
  test: add tests guarding against subdir scanning for vtypes
  storage: resolve images in content dir overrides ending with a slash
  bump the storage plugin API version

 ApiChangeLog                                |   77 +
 debian/control                              |    1 +
 src/PVE/API2/Storage/Status.pm              |  199 +-
 src/PVE/BackupProvider/Plugin/Base.pm       |    6 +-
 src/PVE/GuestImport.pm                      |   28 +-
 src/PVE/GuestImport/OVF.pm                  |   24 +-
 src/PVE/Makefile                            |    1 +
 src/PVE/Storage.pm                          |  225 +-
 src/PVE/Storage/BTRFSPlugin.pm              |   17 +-
 src/PVE/Storage/CIFSPlugin.pm               |    1 +
 src/PVE/Storage/CephFSPlugin.pm             |    1 +
 src/PVE/Storage/Common.pm                   |  102 +-
 src/PVE/Storage/Common/Makefile             |    5 +
 src/PVE/Storage/Common/Parse.pm             |  705 ++++++
 src/PVE/Storage/Common/test/Makefile        |    6 +
 src/PVE/Storage/Common/test/parser_tests.pl | 1256 ++++++++++
 src/PVE/Storage/Common/test/run_tests.pl    |   25 +
 src/PVE/Storage/DirPlugin.pm                |    1 +
 src/PVE/Storage/ESXiPlugin.pm               |    6 -
 src/PVE/Storage/Makefile                    |    4 +
 src/PVE/Storage/NFSPlugin.pm                |    1 +
 src/PVE/Storage/Plugin.pm                   |  462 ++--
 src/test/filesystem_path_test.pm            |  118 +-
 src/test/get_subdir_test.pm                 |   12 +-
 src/test/guest_import_test.pl               |  948 ++++++++
 src/test/list_volumes_test.pm               | 2405 +++++++++++++++----
 src/test/parse_volname_test.pm              |  776 ++++--
 src/test/path_to_volume_id_test.pm          |  300 ++-
 src/test/run_plugin_tests.pl                |   28 +-
 src/test/run_volume_access_tests.pl         |    5 +-
 30 files changed, 6616 insertions(+), 1129 deletions(-)
 create mode 100644 src/PVE/Storage/Common/Parse.pm
 create mode 100644 src/PVE/Storage/Common/test/Makefile
 create mode 100755 src/PVE/Storage/Common/test/parser_tests.pl
 create mode 100755 src/PVE/Storage/Common/test/run_tests.pl
 create mode 100755 src/test/guest_import_test.pl


pve-manager:

Max R. Carrara (1):
  fix #2884: ui: storage: add field for 'max-scan-depth' property

 www/manager6/storage/Base.js | 14 ++++++++++++++
 1 file changed, 14 insertions(+)


Summary over all repositories:
  31 files changed, 6630 insertions(+), 1129 deletions(-)

-- 
Generated by murpp 0.12.0




^ permalink raw reply	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 01/51] test: plugin tests: run tests with at most 4 jobs
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 02/51] plugin, common: remove superfluous use of =pod command paragraph Max R. Carrara
                   ` (43 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Speed up tests by allowing up to 4 jobs to run in parallel.

Base the amount of jobs on the output of `nproc` for environments with
less than 4 processing units available.

Sort tests alphabetically as well, just to make things neater.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/run_plugin_tests.pl | 27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)

diff --git a/src/test/run_plugin_tests.pl b/src/test/run_plugin_tests.pl
index 2da252b4..33c392f5 100755
--- a/src/test/run_plugin_tests.pl
+++ b/src/test/run_plugin_tests.pl
@@ -6,22 +6,29 @@ use warnings;
 # to ensure consistent ctime values on all systems
 $ENV{TZ} = 'UTC';
 
 use TAP::Harness;
 
-my $harness = TAP::Harness->new({ verbosity => -1 });
+my $MAX_JOBS = 4;
+
+my $nproc = eval { int(`nproc`) } || $MAX_JOBS;
+
+my $jobs = $nproc > $MAX_JOBS ? $MAX_JOBS : $nproc;
+
+my $harness = TAP::Harness->new({ verbosity => -1, jobs => $jobs });
+
 my $res = $harness->runtests(
     "archive_info_test.pm",
-    "parse_volname_test.pm",
-    "volname_for_format_test.pm",
-    "list_volumes_test.pm",
-    "path_to_volume_id_test.pm",
-    "get_subdir_test.pm",
-    "filesystem_path_test.pm",
-    "prune_backups_test.pm",
-    "free_image_test.pm",
-    "zfs_list_zvol_test.pm",
     "content_types_test.pm",
+    "filesystem_path_test.pm",
+    "free_image_test.pm",
+    "get_subdir_test.pm",
+    "list_volumes_test.pm",
+    "parse_volname_test.pm",
+    "path_to_volume_id_test.pm",
     "plugin_metadata_test.pm",
+    "prune_backups_test.pm",
+    "volname_for_format_test.pm",
+    "zfs_list_zvol_test.pm",
 );
 
 exit -1 if !$res || $res->{failed} || $res->{parse_errors};
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 02/51] plugin, common: remove superfluous use of =pod command paragraph
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 01/51] test: plugin tests: run tests with at most 4 jobs Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 03/51] common: use Exporter module for PVE::Storage::Common Max R. Carrara
                   ` (42 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Because a POD block may be started using any POD command
paragraph [1], using =pod here is not necessary. =head1 and =head3
(for example) already begin a POD block, so explicitly opening one
with =pod is not needed.

Therefore, remove these occurrences.

This has the benefit of making the docstrings take up a little less
space overall.

[1]: https://perldoc.perl.org/perlpodspec#Pod-Commands

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Common.pm | 28 ----------------------------
 src/PVE/Storage/Plugin.pm |  6 ------
 2 files changed, 34 deletions(-)

diff --git a/src/PVE/Storage/Common.pm b/src/PVE/Storage/Common.pm
index cc3fa45d..aa3b5531 100644
--- a/src/PVE/Storage/Common.pm
+++ b/src/PVE/Storage/Common.pm
@@ -10,12 +10,10 @@ use PVE::Tools qw(run_command);
 use constant {
     FALLOC_FL_KEEP_SIZE => 0x01, # see linux/falloc.h
     FALLOC_FL_PUNCH_HOLE => 0x02, # see linux/falloc.h
 };
 
-=pod
-
 =head1 NAME
 
 PVE::Storage::Common - Shared functions and utilities for storage plugins and storage operations
 
 =head1 DESCRIPTION
@@ -73,12 +71,10 @@ PVE::JSONSchema::register_standard_option(
         enum => ['raw', 'qcow2', 'vmdk'],
         description => "Format of the image.",
     },
 );
 
-=pod
-
 =head1 VOLUME TYPES
 
 A volume that belongs to a guest has one of the following types. They are also content types that
 a storage can be restricted to.
 
@@ -111,32 +107,26 @@ my $GENERIC_VTYPE = {
     'vm-vol' => 'images',
     'vm-state' => 'images',
     'ct-vol' => 'rootdir',
 };
 
-=pod
-
 =head1 FUNCTIONS
 
 =cut
 
-=pod
-
 =head3 generic_vtype
 
     $generic = generic_vtype($vtype)
 
 Returns the generic type that includes the volume type C<$vtype>, or C<undef> if there is none.
 
 =cut
 
 sub generic_vtype : prototype($) ($vtype) {
     return $GENERIC_VTYPE->{$vtype};
 }
 
-=pod
-
 =head3 is_guest_vtype
 
     $is_guest_vtype = is_guest_vtype($vtype)
 
 Returns whether volumes of type C<$vtype> belong to a guest. These are the volume types of VMs and
@@ -147,12 +137,10 @@ an external plugin may report is not one of them, callers that support it check
 
 sub is_guest_vtype : prototype($) ($vtype) {
     return $vtype eq 'images' || defined($GENERIC_VTYPE->{$vtype});
 }
 
-=pod
-
 =head3 vtype_matches
 
     $matches = vtype_matches($vtype, $expected)
 
 Returns whether a volume of type C<$vtype> can be used where one of type C<$expected> is required.
@@ -167,12 +155,10 @@ sub vtype_matches : prototype($$) ($vtype, $expected) {
     return 1 if ($GENERIC_VTYPE->{$vtype} // '') eq $expected;
     return 1 if ($GENERIC_VTYPE->{$expected} // '') eq $vtype;
     return 0;
 }
 
-=pod
-
 =head3 content_allows
 
     $allowed = content_allows($content, $type)
 
 Returns whether the content types in the hash C<$content>, as found in a storage configuration,
@@ -187,12 +173,10 @@ sub content_allows : prototype($$) ($content, $type) {
     return 1 if defined($generic) && $content->{$generic};
     return 1 if grep { $GENERIC_VTYPE->{$_} eq $type && $content->{$_} } keys $GENERIC_VTYPE->%*;
     return 0;
 }
 
-=pod
-
 =head3 align_size_up
 
     $aligned_size = align_size_up($size, $granularity)
 
 Returns the next size bigger than or equal to C<$size> that is aligned with a
@@ -209,12 +193,10 @@ sub align_size_up : prototype($$) {
     print "size $size is not aligned to granularity $granularity, rounding up to $aligned_size\n"
         if $aligned_size != $size;
     return $aligned_size;
 }
 
-=pod
-
 =head3 deallocate
 
     deallocate($file_handle, $offset, $length)
 
 Deallocates the range with C<$length> many bytes starting from offset C<$offset>
@@ -254,12 +236,10 @@ my sub run_qemu_img_json {
         warn $err_output;
     }
     return $json;
 }
 
-=pod
-
 =head3 qemu_img_create
 
     qemu_img_create($fmt, $size, $path, $options)
 
 Create a new qemu image with a specific format C<$format> and size C<$size> KiB for a target
@@ -280,12 +260,10 @@ sub qemu_img_create {
     push @$cmd, '-f', $fmt, '--', $path, "${size}K";
 
     run_command($cmd, errmsg => "unable to create image");
 }
 
-=pod
-
 =head3 qemu_img_create_qcow2_backed
 
     qemu_img_create_qcow2_backed($path, $backing_path, $backing_format, $options)
 
 Create a new qemu qcow2 image C<$path> using an existing backing image C<$backing_path> with
@@ -313,12 +291,10 @@ sub qemu_img_create_qcow2_backed {
     push @$cmd, '--', $path;
 
     run_command($cmd, errmsg => "unable to create image");
 }
 
-=pod
-
 =head3 qemu_img_info
 
     qemu_img_info($filename, $file_format, $timeout, $follow_backing_files)
 
 Returns a json with qemu image C<$filename> information with format <$file_format>. If
@@ -336,12 +312,10 @@ sub qemu_img_info {
     push $cmd->@*, '--', $filename;
 
     return run_qemu_img_json($cmd, $timeout);
 }
 
-=pod
-
 =head3 qemu_img_measure
 
     qemu_img_measure($size, $fmt, $timeout, $options)
 
 Returns a json with the maximum size including all metadata overhead for an image with format
@@ -362,12 +336,10 @@ sub qemu_img_measure {
         push $cmd->@*, '-o', join(',', @$options) if @$options > 0;
     }
     return run_qemu_img_json($cmd, $timeout);
 }
 
-=pod
-
 =head3 qemu_img_resize
 
     qemu_img_resize($path, $format, $size, $preallocation, $timeout)
 
 Resize a qemu image C<$path> with format C<$format> to a target size C<$size> in bytes.
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 1f1e1658..7eb8a244 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -2594,12 +2594,10 @@ sub rename_volume {
         || die "rename '$old_path' to '$new_path' failed - $!\n";
 
     return "${storeid}:${base}${target_vmid}/${target_volname}";
 }
 
-=pod
-
 =head3 rename_snapshot
 
     $plugin->rename_snapshot($scfg, $storeid, $volname, $source_snap, $target_snap)
 
 Rename a volume source snapshot C<$source_snap> to a target snapshot C<$target_snap>.
@@ -2645,12 +2643,10 @@ my sub blockdev_options_nbd_unix {
     $blockdev->{export} = "$export" if defined($export);
 
     return $blockdev;
 }
 
-=pod
-
 =head3 qemu_blockdev_options
 
     $blockdev =
         $plugin->qemu_blockdev_options($scfg, $storeid, $volname, $machine_version, $options)
 
@@ -2819,12 +2815,10 @@ sub new_backup_provider {
     my ($class, $scfg, $storeid, $log_function) = @_;
 
     die "implement me if enabling the feature 'backup-provider' in plugindata()->{features}\n";
 }
 
-=pod
-
 =head3 volume_qemu_snapshot_method
 
     $method = $plugin->volume_qemu_snapshot_method($storeid, $scfg, $volname);
 
 Returns a string with the type of snapshot that qemu can do for a specific volume
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 03/51] common: use Exporter module for PVE::Storage::Common
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 01/51] test: plugin tests: run tests with at most 4 jobs Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 02/51] plugin, common: remove superfluous use of =pod command paragraph Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 04/51] plugin: make get_subdir_files a proper subroutine and update style Max R. Carrara
                   ` (41 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Expose the existing helpers in `PVE::Storage::Common` via the
`Exporter` module's `import()` function, allowing the helpers to be
imported as-is by other modules.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Common.pm | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/src/PVE/Storage/Common.pm b/src/PVE/Storage/Common.pm
index aa3b5531..a784163e 100644
--- a/src/PVE/Storage/Common.pm
+++ b/src/PVE/Storage/Common.pm
@@ -5,10 +5,28 @@ use v5.36;
 use PVE::JSONSchema;
 use PVE::RPCEnvironment;
 use PVE::Syscall;
 use PVE::Tools qw(run_command);
 
+use Exporter qw(import);
+
+our @EXPORT_OK = qw(
+    generic_vtype
+    is_guest_vtype
+    vtype_matches
+    content_allows
+
+    align_size_up
+    deallocate
+
+    qemu_img_create
+    qemu_img_create_qcow2_backed
+    qemu_img_info
+    qemu_img_measure
+    qemu_img_resize
+);
+
 use constant {
     FALLOC_FL_KEEP_SIZE => 0x01, # see linux/falloc.h
     FALLOC_FL_PUNCH_HOLE => 0x02, # see linux/falloc.h
 };
 
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 04/51] plugin: make get_subdir_files a proper subroutine and update style
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (2 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 03/51] common: use Exporter module for PVE::Storage::Common Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 05/51] plugin api: replace subdir helper methods with standalone functions Max R. Carrara
                   ` (40 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

This also renames a lot of variables so that they are in line with
the rest of code and also a bit more readable, as follows:

- $fn        -->  $filename
- $notes_fn  -->  $notes_filename
- $sid       -->  $storeid
- $tt        -->  $vtype

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Plugin.pm | 68 +++++++++++++++++++--------------------
 1 file changed, 34 insertions(+), 34 deletions(-)

diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 7eb8a244..5f25ec51 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -1860,86 +1860,86 @@ sub list_images {
     }
 
     return $res;
 }
 
-# list templates ($tt = <iso|vztmpl|backup|snippets|import>)
-my $get_subdir_files = sub {
-    my ($sid, $path, $tt, $vmid) = @_;
+# $vtype = <iso|vztmpl|backup|snippets|import>
+my sub get_subdir_files {
+    my ($storeid, $path, $vtype, $vmid) = @_;
 
     my $res = [];
 
-    foreach my $fn (<$path/*>) {
+    for my $filename (<$path/*>) {
         # A volume ID cannot represent a name with a newline, see parse_volume_id().
-        next if $fn =~ m/\n/;
+        next if $filename =~ m/\n/;
 
-        my $st = File::stat::stat($fn);
+        my $st = File::stat::stat($filename);
 
         next if (!$st || S_ISDIR($st->mode));
 
         my $info;
 
-        if ($tt eq 'iso') {
-            next if $fn !~ m!/([^/]+$PVE::Storage::ISO_EXT_RE_0)$!i;
+        if ($vtype eq 'iso') {
+            next if $filename !~ m!/([^/]+$PVE::Storage::ISO_EXT_RE_0)$!i;
 
-            $info = { volid => "$sid:iso/$1", format => 'iso' };
+            $info = { volid => "$storeid:iso/$1", format => 'iso' };
 
-        } elsif ($tt eq 'vztmpl') {
-            next if $fn !~ m!/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!;
+        } elsif ($vtype eq 'vztmpl') {
+            next if $filename !~ m!/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!;
 
-            $info = { volid => "$sid:vztmpl/$1", format => $2 eq 'tar' ? $2 : "t$2" };
+            $info = { volid => "$storeid:vztmpl/$1", format => $2 eq 'tar' ? $2 : "t$2" };
 
-        } elsif ($tt eq 'backup') {
-            next if $fn !~ m!/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!;
-            my $original = $fn;
+        } elsif ($vtype eq 'backup') {
+            next if $filename !~ m!/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!;
+            my $original = $filename;
             my $format = $2;
-            $fn = $1;
+            $filename = $1;
 
             # Attribute archives to a guest like parse_volname() does, which the access checks use.
-            next if defined($vmid) && $fn !~ m/^vzdump-(?:openvz|lxc|qemu)-\Q$vmid\E-/;
+            next if defined($vmid) && $filename !~ m/^vzdump-(?:openvz|lxc|qemu)-\Q$vmid\E-/;
 
-            $info = { volid => "$sid:backup/$fn", format => $format };
+            $info = { volid => "$storeid:backup/$filename", format => $format };
 
-            my $archive_info = eval { PVE::Storage::archive_info($fn) } // {};
+            my $archive_info = eval { PVE::Storage::archive_info($filename) } // {};
 
             $info->{ctime} = $archive_info->{ctime} if defined($archive_info->{ctime});
             $info->{subtype} = $archive_info->{type} // 'unknown';
 
-            if (defined($vmid) || $fn =~ m!\-([1-9][0-9]{2,8})\-[^/]+\.${format}$!) {
+            if (defined($vmid) || $filename =~ m!\-([1-9][0-9]{2,8})\-[^/]+\.${format}$!) {
                 $info->{vmid} = $vmid // $1;
             }
 
-            my $notes_fn = $original . NOTES_EXT;
-            if (-f $notes_fn) {
-                my $notes = PVE::Tools::file_read_firstline($notes_fn);
+            my $notes_filename = $original . NOTES_EXT;
+            if (-f $notes_filename) {
+                my $notes = PVE::Tools::file_read_firstline($notes_filename);
                 $info->{notes} = eval { decode('UTF-8', $notes, 1) } // $notes
                     if defined($notes);
             }
 
             $info->{protected} = 1 if -e PVE::Storage::protection_file_path($original);
-        } elsif ($tt eq 'snippets') {
+        } elsif ($vtype eq 'snippets') {
 
             $info = {
-                volid => "$sid:snippets/" . basename($fn),
+                volid => "$storeid:snippets/" . basename($filename),
                 format => 'snippet',
             };
-        } elsif ($tt eq 'import') {
+        } elsif ($vtype eq 'import') {
             next
-                if $fn !~
+                if $filename !~
                 m!/(${PVE::Storage::SAFE_CHAR_CLASS_RE}+$PVE::Storage::IMPORT_EXT_RE_1)$!i;
 
-            $info = { volid => "$sid:import/$1", format => "$2" };
+            $info = { volid => "$storeid:import/$1", format => "$2" };
         }
 
         $info->{size} = $st->size;
         $info->{ctime} //= $st->ctime;
 
         push @$res, $info;
     }
 
     return $res;
-};
+}
 
 # If attributes are set on a volume, they should be included in the result.
 # See get_volume_attribute for a list of possible attributes.
 sub list_volumes {
     my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
@@ -1953,19 +1953,19 @@ sub list_volumes {
             $data = $class->list_images($storeid, $scfg, $vmid);
         } elsif ($scfg->{path}) {
             my $path = $class->get_subdir($scfg, $type);
 
             if ($type eq 'iso' && !defined($vmid)) {
-                $data = $get_subdir_files->($storeid, $path, 'iso');
+                $data = get_subdir_files($storeid, $path, 'iso');
             } elsif ($type eq 'vztmpl' && !defined($vmid)) {
-                $data = $get_subdir_files->($storeid, $path, 'vztmpl');
+                $data = get_subdir_files($storeid, $path, 'vztmpl');
             } elsif ($type eq 'backup') {
-                $data = $get_subdir_files->($storeid, $path, 'backup', $vmid);
+                $data = get_subdir_files($storeid, $path, 'backup', $vmid);
             } elsif ($type eq 'snippets') {
-                $data = $get_subdir_files->($storeid, $path, 'snippets');
+                $data = get_subdir_files($storeid, $path, 'snippets');
             } elsif ($type eq 'import') {
-                $data = $get_subdir_files->($storeid, $path, 'import');
+                $data = get_subdir_files($storeid, $path, 'import');
             }
         }
 
         next if !$data;
 
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 05/51] plugin api: replace subdir helper methods with standalone functions
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (3 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 04/51] plugin: make get_subdir_files a proper subroutine and update style Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 06/51] common: prevent autovivification in plugin_get_vtype_subdir helper Max R. Carrara
                   ` (39 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Replace the `get_vtype_subdirs()` and `get_subdir()` helpers in
PVE::Storage::Plugin with standalone helper subroutines
`plugin_get_default_vtype_subdirs()` and `plugin_get_vtype_subdir()`
in PVE::Storage::Common.

This is mostly to prevent third-party plugins from relying on these
helpers as if they were part of the storage plugin API; the signature
of `get_subdir()` in particular looks as if it belongs to an API
method, even though it is only used as a helper in PVE::Storage,
PVE::Storage::Plugin, and PVE::Storage::BTRFSPlugin.

Therefore, make it explicit that these subroutines are really just
helpers by replacing them with slightly altered versions in
PVE::Storage::Common. Add docstrings for these replacements as well.

Replace all occurrences of `get_vtype_subdirs()` and `get_subdir()`
across the repository with their newly introduced counterparts
`plugin_get_default_vtype_subdirs()` and `plugin_get_vtype_subdir()`,
where possible. In places where a replacement would be *technically*
API-breaking, add a FIXME comment for the next APIAGE reset instead.

Additionally, note that PVE::Storage::ESXiPlugin is the only plugin
that explicitly overrides `get_subdir()`, treating it as if it were an
API method, but because that plugin is not filesystem-based (in the
sense that it does not declare the `path` property in its options),
calling `get_subdir()` would throw an exception anyhow. Also, it
should be mentioned that all methods that could call into
`get_subdir()` have been overridden inside ESXiPlugin, too.

Therefore, the override of `get_subdir()` in PVE::Storage::ESXiPlugin
is superfluous, so remove it.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage.pm                  | 15 +++++++++
 src/PVE/Storage/BTRFSPlugin.pm      | 16 +++++----
 src/PVE/Storage/Common.pm           | 51 +++++++++++++++++++++++++++++
 src/PVE/Storage/ESXiPlugin.pm       |  6 ----
 src/PVE/Storage/Plugin.pm           | 47 +++++++++++++-------------
 src/test/get_subdir_test.pm         | 12 ++++---
 src/test/parse_volname_test.pm      |  6 +++-
 src/test/path_to_volume_id_test.pm  |  6 +++-
 src/test/run_volume_access_tests.pl |  5 ++-
 9 files changed, 121 insertions(+), 43 deletions(-)

diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 9b8c5284..083bc322 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -530,59 +530,71 @@ sub get_image_dir {
     my ($cfg, $storeid, $vmid) = @_;
 
     my $scfg = storage_config($cfg, $storeid);
     my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
 
+    # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+    # on the next APIAGE reset.
     my $path = $plugin->get_subdir($scfg, 'images');
 
     return $vmid ? "$path/$vmid" : $path;
 }
 
 sub get_private_dir {
     my ($cfg, $storeid, $vmid) = @_;
 
     my $scfg = storage_config($cfg, $storeid);
     my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
 
+    # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+    # on the next APIAGE reset.
     my $path = $plugin->get_subdir($scfg, 'rootdir');
 
     return $vmid ? "$path/$vmid" : $path;
 }
 
 sub get_iso_dir {
     my ($cfg, $storeid) = @_;
 
     my $scfg = storage_config($cfg, $storeid);
     my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
 
+    # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+    # on the next APIAGE reset.
     return $plugin->get_subdir($scfg, 'iso');
 }
 
 sub get_import_dir {
     my ($cfg, $storeid) = @_;
 
     my $scfg = storage_config($cfg, $storeid);
     my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
 
+    # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+    # on the next APIAGE reset.
     return $plugin->get_subdir($scfg, 'import');
 }
 
 sub get_vztmpl_dir {
     my ($cfg, $storeid) = @_;
 
     my $scfg = storage_config($cfg, $storeid);
     my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
 
+    # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+    # on the next APIAGE reset.
     return $plugin->get_subdir($scfg, 'vztmpl');
 }
 
 sub get_backup_dir {
     my ($cfg, $storeid) = @_;
 
     my $scfg = storage_config($cfg, $storeid);
     my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
 
+    # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+    # on the next APIAGE reset.
     return $plugin->get_subdir($scfg, 'backup');
 }
 
 # library implementation
 
@@ -726,10 +738,13 @@ sub path_to_volume_id {
 
     foreach my $sid (keys %$ids) {
         my $scfg = $ids->{$sid};
         next if !$scfg->{path};
         my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
+
+        # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+        # on the next APIAGE reset.
         my $imagedir = $plugin->get_subdir($scfg, 'images');
         my $isodir = $plugin->get_subdir($scfg, 'iso');
         my $tmpldir = $plugin->get_subdir($scfg, 'vztmpl');
         my $backupdir = $plugin->get_subdir($scfg, 'backup');
         my $snippetsdir = $plugin->get_subdir($scfg, 'snippets');
diff --git a/src/PVE/Storage/BTRFSPlugin.pm b/src/PVE/Storage/BTRFSPlugin.pm
index 72cb39b7..1c81b7b8 100644
--- a/src/PVE/Storage/BTRFSPlugin.pm
+++ b/src/PVE/Storage/BTRFSPlugin.pm
@@ -11,11 +11,13 @@ use File::Path qw(mkpath);
 use IO::Dir;
 use POSIX qw(EEXIST);
 
 use PVE::Tools qw(run_command dir_glob_foreach);
 
-use PVE::Storage::Common;
+use PVE::Storage::Common qw(
+    plugin_get_vtype_subdir
+);
 use PVE::Storage::DirPlugin;
 
 use constant {
     BTRFS_FIRST_FREE_OBJECTID => 256,
     FS_NOCOW_FL => 0x00800000,
@@ -203,11 +205,11 @@ sub raw_file_to_subvol($) {
 sub filesystem_path {
     my ($class, $scfg, $volname, $snapname) = @_;
 
     my ($vtype, $name, $vmid, undef, undef, $isBase, $format) = $class->parse_volname($volname);
 
-    my $path = $class->get_subdir($scfg, $vtype);
+    my $path = plugin_get_vtype_subdir($scfg, $vtype);
 
     my $is_guest_volume = PVE::Storage::Common::is_guest_vtype($vtype);
     $path .= "/$vmid" if $is_guest_volume;
 
     if ($is_guest_volume && defined($format) && $format eq 'raw') {
@@ -310,11 +312,11 @@ sub clone_image {
     # or a subvolume, we forward to the DirPlugin
     if ($format ne 'raw' && $format ne 'subvol') {
         return PVE::Storage::DirPlugin::clone_image(@_);
     }
 
-    my $imagedir = $class->get_subdir($scfg, 'images');
+    my $imagedir = plugin_get_vtype_subdir($scfg, 'images');
     $imagedir .= "/$vmid";
     mkpath $imagedir;
 
     my $path = $class->filesystem_path($scfg, $volname);
     my $newname = $class->find_free_diskname($storeid, $scfg, $vmid, $format, 1);
@@ -343,11 +345,11 @@ sub alloc_image {
         return $class->SUPER::alloc_image($storeid, $scfg, $vmid, $fmt, $name, $size);
     }
 
     # From Plugin.pm:
 
-    my $imagedir = $class->get_subdir($scfg, 'images') . "/$vmid";
+    my $imagedir = plugin_get_vtype_subdir($scfg, 'images') . "/$vmid";
 
     mkpath $imagedir;
 
     $name = $class->find_free_diskname($storeid, $scfg, $vmid, $fmt, 1) if !$name;
 
@@ -661,11 +663,11 @@ sub volume_has_feature {
     return undef;
 }
 
 sub list_images {
     my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
-    my $imagedir = $class->get_subdir($scfg, 'images');
+    my $imagedir = plugin_get_vtype_subdir($scfg, 'images');
 
     my $res = [];
 
     # Copied from Plugin.pm, with file_size_info calls adapted:
     foreach my $fn (<$imagedir/[0-9][0-9]*/*>) {
@@ -862,11 +864,11 @@ sub volume_import {
     if (!defined($base_snapshot) && -e $destination) {
         die "volume $volname already exists\n" if !$allow_rename;
         $volname = $class->find_free_diskname($storeid, $scfg, $vmid, $volume_format, 1);
     }
 
-    my $imagedir = $class->get_subdir($scfg, $vtype);
+    my $imagedir = plugin_get_vtype_subdir($scfg, $vtype);
     $imagedir .= "/$vmid" if PVE::Storage::Common::is_guest_vtype($vtype);
 
     my $tmppath = "$imagedir/recv.$vmid.tmp";
     mkdir($imagedir); # FIXME: if $scfg->{mkdir};
     if (!mkdir($tmppath)) {
@@ -994,11 +996,11 @@ sub rename_volume {
     $target_volname = $class->find_free_diskname($storeid, $scfg, $target_vmid, $format, 1)
         if !$target_volname;
     $target_volname = $class->volname_for_format($target_volname, $format, 0);
     $target_volname = "$target_vmid/$target_volname";
 
-    my $basedir = $class->get_subdir($scfg, 'images');
+    my $basedir = plugin_get_vtype_subdir($scfg, 'images');
 
     mkpath "${basedir}/${target_vmid}";
     my $source_dir = raw_name_to_dir($source_volname);
     my $target_dir = raw_name_to_dir($target_volname);
 
diff --git a/src/PVE/Storage/Common.pm b/src/PVE/Storage/Common.pm
index a784163e..58f2adfa 100644
--- a/src/PVE/Storage/Common.pm
+++ b/src/PVE/Storage/Common.pm
@@ -21,10 +21,13 @@ our @EXPORT_OK = qw(
     qemu_img_create
     qemu_img_create_qcow2_backed
     qemu_img_info
     qemu_img_measure
     qemu_img_resize
+
+    plugin_get_default_vtype_subdirs
+    plugin_get_vtype_subdir
 );
 
 use constant {
     FALLOC_FL_KEEP_SIZE => 0x01, # see linux/falloc.h
     FALLOC_FL_PUNCH_HOLE => 0x02, # see linux/falloc.h
@@ -125,10 +128,21 @@ my $GENERIC_VTYPE = {
     'vm-vol' => 'images',
     'vm-state' => 'images',
     'ct-vol' => 'rootdir',
 };
 
+# TODO: vtype split: (possibly) adapt default directories during migration period
+my $DEFAULT_VTYPE_SUBDIRS = {
+    images => 'images',
+    rootdir => 'private',
+    iso => 'template/iso',
+    vztmpl => 'template/cache',
+    backup => 'dump',
+    snippets => 'snippets',
+    import => 'import',
+};
+
 =head1 FUNCTIONS
 
 =cut
 
 =head3 generic_vtype
@@ -379,6 +393,43 @@ sub qemu_img_resize {
         $timeout = PVE::RPCEnvironment->is_worker() ? 60 * 60 : 10;
     }
     run_command($cmd, timeout => $timeout);
 }
 
+=head3 plugin_get_default_vtype_subdirs
+
+    my $vtype_subdirs = plugin_get_default_vtype_subdirs()
+
+Returns a hashref containing the default sub-directories for every volume type.
+
+=cut
+
+sub plugin_get_default_vtype_subdirs : prototype() () {
+    return { $DEFAULT_VTYPE_SUBDIRS->%* }; # shallow copy to prevent changes to defaults
+}
+
+=head3 plugin_get_vtype_subdir
+
+    my $subdir = plugin_get_vtype_subdir($scfg, $vtype)
+
+Returns the sub-directory for the volume type C<$vtype> of a given storage
+configuration C<$scfg> as an absolute path.
+
+Raises an exception if the storage config has no C<path> attribute or
+if the given C<$vtype> does not exist.
+
+=cut
+
+sub plugin_get_vtype_subdir : prototype($$) ($scfg, $vtype) {
+    my $path = $scfg->{path};
+
+    die "storage definition has no path\n" if !$path;
+
+    $vtype = 'images' if is_guest_vtype($vtype);
+    die "unknown vtype '$vtype'\n" if !exists($DEFAULT_VTYPE_SUBDIRS->{$vtype});
+
+    my $subdir = $scfg->{"content-dirs"}->{$vtype} // $DEFAULT_VTYPE_SUBDIRS->{$vtype};
+
+    return "$path/$subdir";
+}
+
 1;
diff --git a/src/PVE/Storage/ESXiPlugin.pm b/src/PVE/Storage/ESXiPlugin.pm
index 4c237f76..485e9422 100644
--- a/src/PVE/Storage/ESXiPlugin.pm
+++ b/src/PVE/Storage/ESXiPlugin.pm
@@ -624,16 +624,10 @@ sub volume_has_feature {
     return undef if defined($snapname) || $volname =~ /\.vmx$/;
     return 1 if $feature eq 'copy';
     return undef;
 }
 
-sub get_subdir {
-    my ($class, $scfg, $vtype) = @_;
-
-    die "no subdirectories available for storage $class\n";
-}
-
 package PVE::Storage::ESXiPlugin::Manifest;
 
 use strict;
 use warnings;
 
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 5f25ec51..febef741 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -13,11 +13,14 @@ use File::stat qw();
 
 use PVE::Tools qw(run_command);
 use PVE::JSONSchema qw(get_standard_option register_standard_option);
 use PVE::Cluster qw(cfs_register_file);
 
-use PVE::Storage::Common;
+use PVE::Storage::Common qw(
+    plugin_get_default_vtype_subdirs
+    plugin_get_vtype_subdir
+);
 
 use JSON;
 
 use base qw(PVE::SectionConfig);
 
@@ -1006,37 +1009,20 @@ sub volname_with_format {
 
     return $name if !$correct_fmt_ext;
     return "$name.$correct_fmt_ext";
 }
 
-my $vtype_subdirs = {
-    images => 'images',
-    rootdir => 'private',
-    iso => 'template/iso',
-    vztmpl => 'template/cache',
-    backup => 'dump',
-    snippets => 'snippets',
-    import => 'import',
-};
-
+# FIXME: remove on the next APIAGE reset.
 sub get_vtype_subdirs {
-    return $vtype_subdirs;
+    return plugin_get_default_vtype_subdirs();
 }
 
+# FIXME: remove on the next APIAGE reset.
 sub get_subdir {
     my ($class, $scfg, $vtype) = @_;
 
-    my $path = $scfg->{path};
-
-    die "storage definition has no path\n" if !$path;
-
-    $vtype = 'images' if PVE::Storage::Common::is_guest_vtype($vtype);
-    die "unknown vtype '$vtype'\n" if !exists($vtype_subdirs->{$vtype});
-
-    my $subdir = $scfg->{"content-dirs"}->{$vtype} // $vtype_subdirs->{$vtype};
-
-    return "$path/$subdir";
+    return plugin_get_vtype_subdir($scfg, $vtype);
 }
 
 my sub get_snap_name {
     my ($class, $volname, $snapname) = @_;
     die "missing snapname\n" if !$snapname;
@@ -1057,10 +1043,12 @@ sub filesystem_path {
     # Note: qcow2/qed has internal snapshot, so path is always
     # the same (with or without snapshot => same file).
     die "can't snapshot this image format\n"
         if defined($snapname) && $format !~ m/^(qcow2|qed)$/;
 
+    # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+    # on the next APIAGE reset.
     my $dir = $class->get_subdir($scfg, $vtype);
 
     $dir .= "/$vmid" if PVE::Storage::Common::is_guest_vtype($vtype);
 
     my $path = "$dir/$name";
@@ -1188,10 +1176,12 @@ sub clone_image {
 
     die "this storage type does not support clone_image on subvolumes\n" if $format eq 'subvol';
 
     die "clone_image only works on base images\n" if !$isBase;
 
+    # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+    # on the next APIAGE reset.
     my $imagedir = $class->get_subdir($scfg, 'images');
     $imagedir .= "/$vmid";
 
     mkpath $imagedir;
 
@@ -1221,10 +1211,12 @@ sub clone_image {
 }
 
 sub alloc_image {
     my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
 
+    # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+    # on the next APIAGE reset.
     my $imagedir = $class->get_subdir($scfg, 'images');
     $imagedir .= "/$vmid";
 
     mkpath $imagedir;
 
@@ -1799,10 +1791,12 @@ sub volume_has_feature {
 }
 
 sub list_images {
     my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
 
+    # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+    # on the next APIAGE reset.
     my $imagedir = $class->get_subdir($scfg, 'images');
 
     my $format_info = $class->get_formats($scfg, $storeid);
     my $fmts = join('|', sort keys $format_info->{valid}->%*);
 
@@ -1950,10 +1944,12 @@ sub list_volumes {
         my $data;
 
         if ($type eq 'images' || $type eq 'rootdir') {
             $data = $class->list_images($storeid, $scfg, $vmid);
         } elsif ($scfg->{path}) {
+            # FIXME: Call plugin_get_vtype_subdir() instead of
+            # $plugin->get_subdir() on the next APIAGE reset.
             my $path = $class->get_subdir($scfg, $type);
 
             if ($type eq 'iso' && !defined($vmid)) {
                 $data = get_subdir_files($storeid, $path, 'iso');
             } elsif ($type eq 'vztmpl' && !defined($vmid)) {
@@ -2145,10 +2141,12 @@ sub activate_storage {
     }
 
     my $try_create_subdir = sub {
         my ($vtype) = @_;
 
+        # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+        # on the next APIAGE reset.
         my $subdir = $class->get_subdir($scfg, $vtype);
 
         File::Path::make_path($subdir, { error => \my $errors });
         if (defined($errors) && scalar($errors->@*)) {
             my $msg = "failed to create content directory '$subdir' for content '$vtype'";
@@ -2184,24 +2182,27 @@ sub activate_storage {
     if (
         (!defined($scfg->{'create-subdirs'}) || $scfg->{'create-subdirs'})
         # FIXME The mkdir option is deprecated. Remove with PVE 9?
         && (!defined($scfg->{mkdir}) || $scfg->{mkdir})
     ) {
-        for my $vtype (sort keys %$vtype_subdirs) {
+        my $vtype_subdirs = plugin_get_default_vtype_subdirs();
+        for my $vtype (sort keys $vtype_subdirs->%*) {
             # OpenVZMigrate uses backup (dump) dir
             if (
                 defined($dir_vtypes->{$vtype})
                 || ($vtype eq 'backup' && defined($scfg->{content}->{'rootdir'}))
             ) {
                 $try_create_subdir->($vtype);
             }
         }
     }
 
     # check that content dirs are pairwise inequal
     my $resolved_subdirs = {};
     for my $vtype (sort keys $dir_vtypes->%*) {
+        # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+        # on the next APIAGE reset.
         my $subdir = $class->get_subdir($scfg, $vtype);
         my $abs_subdir = abs_path($subdir);
         next if !defined($abs_subdir);
 
         die "storage '$storeid' uses directory $abs_subdir for multiple content types\n"
diff --git a/src/test/get_subdir_test.pm b/src/test/get_subdir_test.pm
index 9f7e2266..02d377d5 100644
--- a/src/test/get_subdir_test.pm
+++ b/src/test/get_subdir_test.pm
@@ -3,20 +3,24 @@ package PVE::Storage::TestGetSubdir;
 use strict;
 use warnings;
 
 use lib qw(..);
 
+use PVE::Storage::Common qw(
+    plugin_get_default_vtype_subdirs
+    plugin_get_vtype_subdir
+);
 use PVE::Storage::Plugin;
 use Test::More;
 
 my $scfg_with_path = { path => '/some/path' };
-my $vtype_subdirs = PVE::Storage::Plugin::get_vtype_subdirs();
+my $vtype_subdirs = plugin_get_default_vtype_subdirs();
 
 # each test is comprised of the following array keys:
 # [0] => storage config; positive with path key
 # [1] => storage type;  see $vtype_subdirs
-# [2] => expected return from get_subdir
+# [2] => expected return from plugin_get_vtype_subdir
 my $tests = [
     # failed matches
     [$scfg_with_path, 'none', "unknown vtype 'none'\n"],
     [{}, 'iso', "storage definition has no path\n"],
 ];
@@ -50,14 +54,14 @@ plan tests => scalar @$tests;
 
 foreach my $tt (@$tests) {
     my ($scfg, $type, $expected) = @$tt;
 
     my $got;
-    eval { $got = PVE::Storage::Plugin->get_subdir($scfg, $type) };
+    eval { $got = plugin_get_vtype_subdir($scfg, $type) };
     $got = $@ if $@;
 
-    is($got, $expected, "get_subdir for $type") || diag(explain($got));
+    is($got, $expected, "plugin_get_vtype_subdir for $type") || diag(explain($got));
 }
 
 done_testing();
 
 1;
diff --git a/src/test/parse_volname_test.pm b/src/test/parse_volname_test.pm
index 5eb7f55c..a3b4d568 100644
--- a/src/test/parse_volname_test.pm
+++ b/src/test/parse_volname_test.pm
@@ -4,10 +4,13 @@ use strict;
 use warnings;
 
 use lib qw(..);
 
 use PVE::Storage;
+use PVE::Storage::Common qw(
+    plugin_get_default_vtype_subdirs
+);
 use Test::More;
 
 my $vmid = 1234;
 
 {
@@ -593,11 +596,12 @@ push $tests->@*,
 # run through test case array
 #
 plan tests => scalar @$tests + 1;
 
 my $seen_vtype;
-my $vtype_subdirs = { map { $_ => 1 } keys %{ PVE::Storage::Plugin::get_vtype_subdirs() } };
+my $vtype_subdirs =
+    { map { $_ => 1 } keys %{ plugin_get_default_vtype_subdirs() } };
 
 foreach my $t (@$tests) {
     my $description = $t->{description};
     my $volname = $t->{volname};
     my $expected = $t->{expected};
diff --git a/src/test/path_to_volume_id_test.pm b/src/test/path_to_volume_id_test.pm
index ff162b1d..03753c38 100644
--- a/src/test/path_to_volume_id_test.pm
+++ b/src/test/path_to_volume_id_test.pm
@@ -4,10 +4,13 @@ use strict;
 use warnings;
 
 use lib qw(..);
 
 use PVE::Storage;
+use PVE::Storage::Common qw(
+    plugin_get_default_vtype_subdirs
+);
 
 use Test::More;
 
 use Cwd;
 use File::Basename;
@@ -235,11 +238,12 @@ my @tests = (
 );
 
 plan tests => scalar @tests + 1;
 
 my $seen_vtype;
-my $vtype_subdirs = { map { $_ => 1 } keys %{ PVE::Storage::Plugin::get_vtype_subdirs() } };
+my $vtype_subdirs =
+    { map { $_ => 1 } keys %{ plugin_get_default_vtype_subdirs() } };
 
 foreach my $tt (@tests) {
     my $file = $tt->{volname};
     my $expected = $tt->{expected};
     my $description = $tt->{description};
diff --git a/src/test/run_volume_access_tests.pl b/src/test/run_volume_access_tests.pl
index 12d5544e..2de57a26 100755
--- a/src/test/run_volume_access_tests.pl
+++ b/src/test/run_volume_access_tests.pl
@@ -11,10 +11,13 @@ use Test::More;
 use lib ('.', '..');
 
 use PVE::API2::Storage::PruneBackups;
 use PVE::RPCEnvironment;
 use PVE::Storage;
+use PVE::Storage::Common qw(
+    plugin_get_default_vtype_subdirs
+);
 use PVE::Storage::Plugin;
 use PVE::Tools;
 
 my $storage_dir = tempdir(CLEANUP => 1);
 
@@ -68,11 +71,11 @@ $pve_cluster_module->mock(
 );
 
 my $rpcenv = PVE::RPCEnvironment->init('pub');
 $rpcenv->init_request();
 
-my @types = sort (keys PVE::Storage::Plugin::get_vtype_subdirs()->%*, qw(vm-vol vm-state ct-vol));
+my @types = sort (keys plugin_get_default_vtype_subdirs()->%*, qw(vm-vol vm-state ct-vol));
 my $all_types = { map { $_ => 1 } @types };
 
 my @tests = (
     {
         volid => 'dir:backup/vzdump-qemu-100-2025_07_29-13_00_55.vma',
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 06/51] common: prevent autovivification in plugin_get_vtype_subdir helper
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (4 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 05/51] plugin api: replace subdir helper methods with standalone functions Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 07/51] plugin: break up if-elsif chain into separate if-blocks Max R. Carrara
                   ` (38 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

When accessing a non-existing (or undefined) value in a hash(ref) and
using this value again for a hash lookup, i.e.
`$hashref->{foo}->{bar}` where `$hashref->{foo}` does not exist or is
`undef`, Perl will (un)conveniently instantiate a new hashref where
one was absent before. This is also called autovivification [1].

This means that if the config hash of directory-based storage plugin
that does *not* declare the 'content-dirs' property in its `options()`
is passed to `plugin_get_vtype_subdir()`, the 'content-dirs' key will
be instantiated as an empty hashref during the
`$scfg->{'content-dirs'}->{$vtype}` lookup.

Note that "directory-based storage plugin" means any storage plugin
that defines the 'path' property in its `options()` method.

While this is a minor issue, the `$scfg` hashref should IMO never be
modified unless necessary, or if the method actually is supposed to
modify it.

[1]: https://perldoc.perl.org/perlref#Autovivification

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Common.pm | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/src/PVE/Storage/Common.pm b/src/PVE/Storage/Common.pm
index 58f2adfa..4f5fde64 100644
--- a/src/PVE/Storage/Common.pm
+++ b/src/PVE/Storage/Common.pm
@@ -425,11 +425,13 @@ sub plugin_get_vtype_subdir : prototype($$) ($scfg, $vtype) {
     die "storage definition has no path\n" if !$path;
 
     $vtype = 'images' if is_guest_vtype($vtype);
     die "unknown vtype '$vtype'\n" if !exists($DEFAULT_VTYPE_SUBDIRS->{$vtype});
 
-    my $subdir = $scfg->{"content-dirs"}->{$vtype} // $DEFAULT_VTYPE_SUBDIRS->{$vtype};
+    # Intermediate step to prevent autovivification
+    my $content_dirs = $scfg->{'content-dirs'} // {};
+    my $subdir = $content_dirs->{$vtype} // $DEFAULT_VTYPE_SUBDIRS->{$vtype};
 
     return "$path/$subdir";
 }
 
 1;
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 07/51] plugin: break up if-elsif chain into separate if-blocks
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (5 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 06/51] common: prevent autovivification in plugin_get_vtype_subdir helper Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 08/51] plugin: adapt get_subdir_files helper of list_volumes API method Max R. Carrara
                   ` (37 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Done in order to make future changes a bit easier.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Plugin.pm | 28 +++++++++++++++++++++-------
 1 file changed, 21 insertions(+), 7 deletions(-)

diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index febef741..d316caac 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -884,34 +884,48 @@ sub parse_volname {
         parse_name_dir($basename);
         my ($vmid, $name) = ($3, $4);
         my (undef, $format, $isBase) = parse_name_dir($name);
         my $vtype = $class->guest_volume_vtype($format, $name);
         return ($vtype, $name, $vmid, $basename, $basedvmid, $isBase, $format);
-    } elsif ($volname =~ m!^(\d+)/(\S+)$!) {
+    }
+
+    if ($volname =~ m!^(\d+)/(\S+)$!) {
         my ($vmid, $name) = ($1, $2);
         my (undef, $format, $isBase) = parse_name_dir($name);
         my $vtype = $class->guest_volume_vtype($format, $name);
         return ($vtype, $name, $vmid, undef, undef, $isBase, $format);
-    } elsif ($volname =~ m!^iso/([^/]+$PVE::Storage::ISO_EXT_RE_0)$!) {
+    }
+
+    if ($volname =~ m!^iso/([^/]+$PVE::Storage::ISO_EXT_RE_0)$!) {
         return ('iso', $1, undef, undef, undef, undef, 'raw');
-    } elsif ($volname =~ m!^vztmpl/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!) {
+    }
+
+    if ($volname =~ m!^vztmpl/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!) {
         return ('vztmpl', $1, undef, undef, undef, undef, 'raw');
-    } elsif ($volname =~ m!^backup/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!) {
+    }
+
+    if ($volname =~ m!^backup/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!) {
         my $fn = $1;
         if ($fn =~ m/^vzdump-(openvz|lxc|qemu)-(\d+)-.+/) {
             return ('backup', $fn, $2, undef, undef, undef, 'raw');
         }
         return ('backup', $fn, undef, undef, undef, undef, 'raw');
-    } elsif ($volname =~ m!^snippets/([^/]+)$!) {
+    }
+
+    if ($volname =~ m!^snippets/([^/]+)$!) {
         return ('snippets', $1, undef, undef, undef, undef, 'raw');
-    } elsif ($volname =~
+    }
+
+    if ($volname =~
         m!^import/(${PVE::Storage::SAFE_CHAR_WITH_WHITESPACE_CLASS_RE}+\.ova\/${PVE::Storage::OVA_CONTENT_RE_1})$!
     ) {
         my $packed_image = $1;
         my $format = $2;
         return ('import', $packed_image, undef, undef, undef, undef, "ova+$format");
-    } elsif ($volname =~
+    }
+
+    if ($volname =~
         m!^import/(${PVE::Storage::SAFE_CHAR_WITH_WHITESPACE_CLASS_RE}+$PVE::Storage::IMPORT_EXT_RE_1)$!
     ) {
         return ('import', $1, undef, undef, undef, undef, $2);
     }
 
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 08/51] plugin: adapt get_subdir_files helper of list_volumes API method
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (6 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 07/51] plugin: break up if-elsif chain into separate if-blocks Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 09/51] plugin: update code style of list_volumes plugin " Max R. Carrara
                   ` (36 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Pull the body of the loop inside `get_subdir_files()` out into a
closure and break up the if-elsif chain into more readable if-clauses.

Strip the vtype subdirectory from the beginning of the file paths
being worked on inside that closure. This does not have an effect on
how any of the subsequent regexes match on the path / file name,
because they match at the end of a given path.

Additionally, take the storage config hash `$scfg` instead of the
volume type subdir `$path` as parameter. Passing `$scfg` along makes
it possible to obtain the subdir for the given `$vtype` inside the
helper directly instead of passing both the vtype and its associated
subdirectory along.

Also, pass the `$class` parameter along because it is still necessary
to call the deprecated `get_subdir()` method for backwards compat.

All of these things make future changes easier to manage.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Plugin.pm | 102 +++++++++++++++++++++++++-------------
 1 file changed, 67 insertions(+), 35 deletions(-)

diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index d316caac..bbed4eb4 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -1870,44 +1870,59 @@ sub list_images {
     return $res;
 }
 
 # $vtype = <iso|vztmpl|backup|snippets|import>
 my sub get_subdir_files {
-    my ($storeid, $path, $vtype, $vmid) = @_;
+    my ($class, $storeid, $scfg, $vtype, $vmid) = @_;
 
+    # FIXME: Call plugin_get_vtype_subdir() instead of $class->get_subdir()
+    # and remove the $class parameter on the next APIAGE reset.
+    my $vtype_subdir = $class->get_subdir($scfg, $vtype);
     my $res = [];
 
-    for my $filename (<$path/*>) {
-        # A volume ID cannot represent a name with a newline, see parse_volume_id().
-        next if $filename =~ m/\n/;
+    my $get_subdir_file_info = sub {
+        my ($path, $st) = @_;
 
-        my $st = File::stat::stat($filename);
-
-        next if (!$st || S_ISDIR($st->mode));
-
-        my $info;
+        # Strip the vtype subdir from beginning of the current path
+        # so that we don't have to take it into account when parsing the file name
+        my $filename = $path;
+        if ($filename !~ s!^\Q$vtype_subdir\E(?=/)!!) {
+            return;
+        }
 
         if ($vtype eq 'iso') {
-            next if $filename !~ m!/([^/]+$PVE::Storage::ISO_EXT_RE_0)$!i;
+            return if $filename !~ m!/([^/]+$PVE::Storage::ISO_EXT_RE_0)$!i;
 
-            $info = { volid => "$storeid:iso/$1", format => 'iso' };
+            return {
+                volid => "$storeid:iso/$1",
+                format => 'iso',
+            };
+        }
 
-        } elsif ($vtype eq 'vztmpl') {
-            next if $filename !~ m!/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!;
+        if ($vtype eq 'vztmpl') {
+            return if $filename !~ m!/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!;
 
-            $info = { volid => "$storeid:vztmpl/$1", format => $2 eq 'tar' ? $2 : "t$2" };
+            return {
+                volid => "$storeid:vztmpl/$1",
+                format => $2 eq 'tar' ? $2 : "t$2",
+            };
+        }
 
-        } elsif ($vtype eq 'backup') {
-            next if $filename !~ m!/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!;
-            my $original = $filename;
+        if ($vtype eq 'backup') {
+            return if $filename !~ m!/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!;
+
+            my $original = $path;
             my $format = $2;
             $filename = $1;
 
             # Attribute archives to a guest like parse_volname() does, which the access checks use.
             next if defined($vmid) && $filename !~ m/^vzdump-(?:openvz|lxc|qemu)-\Q$vmid\E-/;
 
-            $info = { volid => "$storeid:backup/$filename", format => $format };
+            my $info = {
+                volid => "$storeid:backup/$filename",
+                format => $format,
+            };
 
             my $archive_info = eval { PVE::Storage::archive_info($filename) } // {};
 
             $info->{ctime} = $archive_info->{ctime} if defined($archive_info->{ctime});
             $info->{subtype} = $archive_info->{type} // 'unknown';
@@ -1922,28 +1937,49 @@ my sub get_subdir_files {
                 $info->{notes} = eval { decode('UTF-8', $notes, 1) } // $notes
                     if defined($notes);
             }
 
             $info->{protected} = 1 if -e PVE::Storage::protection_file_path($original);
-        } elsif ($vtype eq 'snippets') {
 
-            $info = {
+            return $info;
+        }
+
+        if ($vtype eq 'snippets') {
+            return {
                 volid => "$storeid:snippets/" . basename($filename),
                 format => 'snippet',
             };
-        } elsif ($vtype eq 'import') {
-            next
+        }
+
+        if ($vtype eq 'import') {
+            return
                 if $filename !~
                 m!/(${PVE::Storage::SAFE_CHAR_CLASS_RE}+$PVE::Storage::IMPORT_EXT_RE_1)$!i;
 
-            $info = { volid => "$storeid:import/$1", format => "$2" };
+            return {
+                volid => "$storeid:import/$1",
+                format => "$2",
+            };
         }
 
-        $info->{size} = $st->size;
-        $info->{ctime} //= $st->ctime;
+        return;
+    };
 
-        push @$res, $info;
+    for my $path (<$vtype_subdir/*>) {
+        my $st = File::stat::stat($path);
+
+        # A volume ID cannot represent a name with a newline, see parse_volume_id().
+        next if $path =~ m/\n/;
+
+        next if (!$st || S_ISDIR($st->mode));
+
+        if (defined(my $info = $get_subdir_file_info->($path, $st))) {
+            $info->{size} = $st->size;
+            $info->{ctime} //= $st->ctime;
+
+            push $res->@*, $info;
+        }
     }
 
     return $res;
 }
 
@@ -1958,24 +1994,20 @@ sub list_volumes {
         my $data;
 
         if ($type eq 'images' || $type eq 'rootdir') {
             $data = $class->list_images($storeid, $scfg, $vmid);
         } elsif ($scfg->{path}) {
-            # FIXME: Call plugin_get_vtype_subdir() instead of
-            # $plugin->get_subdir() on the next APIAGE reset.
-            my $path = $class->get_subdir($scfg, $type);
-
             if ($type eq 'iso' && !defined($vmid)) {
-                $data = get_subdir_files($storeid, $path, 'iso');
+                $data = get_subdir_files($class, $storeid, $scfg, 'iso', undef);
             } elsif ($type eq 'vztmpl' && !defined($vmid)) {
-                $data = get_subdir_files($storeid, $path, 'vztmpl');
+                $data = get_subdir_files($class, $storeid, $scfg, 'vztmpl', undef);
             } elsif ($type eq 'backup') {
-                $data = get_subdir_files($storeid, $path, 'backup', $vmid);
+                $data = get_subdir_files($class, $storeid, $scfg, 'backup', $vmid);
             } elsif ($type eq 'snippets') {
-                $data = get_subdir_files($storeid, $path, 'snippets');
+                $data = get_subdir_files($class, $storeid, $scfg, 'snippets', undef);
             } elsif ($type eq 'import') {
-                $data = get_subdir_files($storeid, $path, 'import');
+                $data = get_subdir_files($class, $storeid, $scfg, 'import', undef);
             }
         }
 
         next if !$data;
 
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 09/51] plugin: update code style of list_volumes plugin API method
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (7 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 08/51] plugin: adapt get_subdir_files helper of list_volumes API method Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 10/51] plugin: use closure for obtaining raw volume data in list_volumes Max R. Carrara
                   ` (35 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Plugin.pm | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index bbed4eb4..a0ad7c53 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -1988,11 +1988,11 @@ my sub get_subdir_files {
 sub list_volumes {
     my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
 
     my $res = [];
     my $vmlist = PVE::Cluster::get_vmlist();
-    foreach my $type (@$content_types) {
+    for my $type ($content_types->@*) {
         my $data;
 
         if ($type eq 'images' || $type eq 'rootdir') {
             $data = $class->list_images($storeid, $scfg, $vmid);
         } elsif ($scfg->{path}) {
@@ -2009,11 +2009,11 @@ sub list_volumes {
             }
         }
 
         next if !$data;
 
-        foreach my $item (@$data) {
+        for my $item ($data->@*) {
             if ($type eq 'images' || $type eq 'rootdir') {
                 my (undef, $volname) = parse_volume_id($item->{volid});
                 my $vtype = eval { ($class->parse_volname($volname))[0] } // '';
 
                 my $vminfo = $vmlist->{ids}->{ $item->{vmid} };
@@ -2029,11 +2029,11 @@ sub list_volumes {
                 next if $type ne $item->{content};
             } else {
                 $item->{content} = $type;
             }
 
-            push @$res, $item;
+            push $res->@*, $item;
         }
     }
 
     return $res;
 }
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 10/51] plugin: use closure for obtaining raw volume data in list_volumes
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (8 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 09/51] plugin: update code style of list_volumes plugin " Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 11/51] plugin: use closure for inner loop logic " Max R. Carrara
                   ` (34 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

To make the code in the `list_volumes()` API method a little more
readable and maintainable, break up the if-elsif chain that obtains
the raw volume data into a closure. The overall logic is kept the
same.

Also rename `$data` to `$raw_volumes` to make it a little clearer what
is actually being iterated over, since "data" does not really say
much.

Be a little more specific and use an un-definedness check on
`$raw_volumes` to continue with the iteration, instead of a plain
negation.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Plugin.pm | 51 ++++++++++++++++++++++++++-------------
 1 file changed, 34 insertions(+), 17 deletions(-)

diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index a0ad7c53..61f6f221 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -1988,32 +1988,49 @@ my sub get_subdir_files {
 sub list_volumes {
     my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
 
     my $res = [];
     my $vmlist = PVE::Cluster::get_vmlist();
-    for my $type ($content_types->@*) {
-        my $data;
+
+    my $get_raw_volumes_for_type = sub {
+        my ($type) = @_;
 
         if ($type eq 'images' || $type eq 'rootdir') {
-            $data = $class->list_images($storeid, $scfg, $vmid);
-        } elsif ($scfg->{path}) {
-            if ($type eq 'iso' && !defined($vmid)) {
-                $data = get_subdir_files($class, $storeid, $scfg, 'iso', undef);
-            } elsif ($type eq 'vztmpl' && !defined($vmid)) {
-                $data = get_subdir_files($class, $storeid, $scfg, 'vztmpl', undef);
-            } elsif ($type eq 'backup') {
-                $data = get_subdir_files($class, $storeid, $scfg, 'backup', $vmid);
-            } elsif ($type eq 'snippets') {
-                $data = get_subdir_files($class, $storeid, $scfg, 'snippets', undef);
-            } elsif ($type eq 'import') {
-                $data = get_subdir_files($class, $storeid, $scfg, 'import', undef);
-            }
+            return $class->list_images($storeid, $scfg, $vmid);
         }
 
-        next if !$data;
+        return if !$scfg->{path};
 
-        for my $item ($data->@*) {
+        if ($type eq 'iso' && !defined($vmid)) {
+            return get_subdir_files($class, $storeid, $scfg, 'iso', undef);
+        }
+
+        if ($type eq 'vztmpl' && !defined($vmid)) {
+            return get_subdir_files($class, $storeid, $scfg, 'vztmpl', undef);
+        }
+
+        if ($type eq 'backup') {
+            return get_subdir_files($class, $storeid, $scfg, 'backup', $vmid);
+        }
+
+        if ($type eq 'snippets') {
+            return get_subdir_files($class, $storeid, $scfg, 'snippets', undef);
+        }
+
+        if ($type eq 'import') {
+            return get_subdir_files($class, $storeid, $scfg, 'import', undef);
+        }
+
+        return;
+    };
+
+    for my $type ($content_types->@*) {
+        my $raw_volumes = $get_raw_volumes_for_type->($type);
+
+        next if !defined($raw_volumes);
+
+        for my $item ($raw_volumes->@*) {
             if ($type eq 'images' || $type eq 'rootdir') {
                 my (undef, $volname) = parse_volume_id($item->{volid});
                 my $vtype = eval { ($class->parse_volname($volname))[0] } // '';
 
                 my $vminfo = $vmlist->{ids}->{ $item->{vmid} };
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 11/51] plugin: use closure for inner loop logic in list_volumes
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (9 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 10/51] plugin: use closure for obtaining raw volume data in list_volumes Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 12/51] storage: update code style in function path_to_volume_id Max R. Carrara
                   ` (33 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Factor most of the body of the inner loop of the `list_volumes()`
plugin API method into a closure and provide additional context for
why we have such an elaborate check for VM / CT disks in the first
place.

This has the added benefit that we now also do not modify the volume
hashes on the fly anymore when determining whether a volume's type
aligns with a guest's type. Small difference, but might make it a
little easier to adapt that code in the future, e.g. when we align
vtypes and content types.

Also, rename `$item` to `$volume` for clarity's sake.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Plugin.pm | 56 ++++++++++++++++++++++++++-------------
 1 file changed, 37 insertions(+), 19 deletions(-)

diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 61f6f221..df8526bd 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -2021,36 +2021,54 @@ sub list_volumes {
         }
 
         return;
     };
 
+    my $get_content_type_for_volume = sub {
+        my ($volume, $type) = @_;
+
+        # It is currently impossible to determine whether a volume belongs to a
+        # VM or a CT based on the output of list_images alone.
+        # Therefore, we use the parsed ID ($volume->{vmid}) to look up the guest
+        # and make sure that the guest's type (VM or CT) corresponds to the
+        # volume's type.
+        if ($type eq 'images' || $type eq 'rootdir') {
+            my (undef, $volname) = parse_volume_id($volume->{volid});
+            my $parsed_vtype = eval { ($class->parse_volname($volname))[0] } // '';
+
+            my $vminfo = $vmlist->{ids}->{ $volume->{vmid} };
+
+            my $vmtype;
+            if (defined($vminfo)) {
+                $vmtype = $vminfo->{type};
+            }
+
+            my $expected_vtype;
+            if ($parsed_vtype eq 'ct-vol' || (defined($vmtype) && $vmtype eq 'lxc')) {
+                $expected_vtype = 'rootdir';
+            } else {
+                $expected_vtype = 'images';
+            }
+
+            return if $expected_vtype ne $type;
+        }
+
+        return $type;
+    };
+
     for my $type ($content_types->@*) {
         my $raw_volumes = $get_raw_volumes_for_type->($type);
 
         next if !defined($raw_volumes);
 
-        for my $item ($raw_volumes->@*) {
-            if ($type eq 'images' || $type eq 'rootdir') {
-                my (undef, $volname) = parse_volume_id($item->{volid});
-                my $vtype = eval { ($class->parse_volname($volname))[0] } // '';
+        for my $volume ($raw_volumes->@*) {
+            my $content_type = $get_content_type_for_volume->($volume, $type);
 
-                my $vminfo = $vmlist->{ids}->{ $item->{vmid} };
-                my $vmtype;
-                if (defined($vminfo)) {
-                    $vmtype = $vminfo->{type};
-                }
-                if ($vtype eq 'ct-vol' || (defined($vmtype) && $vmtype eq 'lxc')) {
-                    $item->{content} = 'rootdir';
-                } else {
-                    $item->{content} = 'images';
-                }
-                next if $type ne $item->{content};
-            } else {
-                $item->{content} = $type;
-            }
+            next if !defined($content_type);
 
-            push $res->@*, $item;
+            $volume->{content} = $content_type;
+            push $res->@*, $volume;
         }
     }
 
     return $res;
 }
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 12/51] storage: update code style in function path_to_volume_id
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (10 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 11/51] plugin: use closure for inner loop logic " Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 13/51] storage: break up if-elsif chain in path_to_volume_id Max R. Carrara
                   ` (32 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Also rename the initially parsed storeid from `$sid` to
`$parsed_storeid`.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage.pm | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 083bc322..650d7f5f 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -718,27 +718,27 @@ sub volume_is_base_and_used {
 sub path_to_volume_id {
     my ($cfg, $path) = @_;
 
     my $ids = $cfg->{ids};
 
-    my ($sid, $volname) = parse_volume_id($path, 1);
-    if ($sid) {
-        if (my $scfg = $ids->{$sid}) {
+    my ($parsed_storeid, $volname) = parse_volume_id($path, 1);
+    if ($parsed_storeid) {
+        if (my $scfg = $ids->{$parsed_storeid}) {
             if ($scfg->{path}) {
                 my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
                 my ($vtype, $name, $vmid) = $plugin->parse_volname($volname);
                 return ($vtype, $path);
             }
         }
         return ('');
     }
 
     # Note: abs_path() return undef if $path doesn not exist
     # for example when nfs storage is not mounted
     $path = abs_path($path) || $path;
 
-    foreach my $sid (keys %$ids) {
+    for my $sid (keys $ids->%*) {
         my $scfg = $ids->{$sid};
         next if !$scfg->{path};
         my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
 
         # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
@@ -753,11 +753,11 @@ sub path_to_volume_id {
         if ($path =~ m!^\Q$imagedir\E/(\d+)/([^/\s]+)$!) {
             my $vmid = $1;
             my $name = $2;
 
             my $vollist = $plugin->list_images($sid, $scfg, $vmid);
-            foreach my $info (@$vollist) {
+            for my $info ($vollist->@*) {
                 my ($storeid, $volname) = parse_volume_id($info->{volid});
                 my $volpath = $plugin->path($scfg, $volname, $storeid);
                 if ($volpath eq $path) {
                     my ($vtype) = $plugin->parse_volname($volname);
                     return ($vtype, $info->{volid});
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 13/51] storage: break up if-elsif chain in path_to_volume_id
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (11 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 12/51] storage: update code style in function path_to_volume_id Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 14/51] storage: heave vtype file path parsing logic inside loop into helper Max R. Carrara
                   ` (31 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Done in order to make future changes a little easier.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage.pm | 22 +++++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)

diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 650d7f5f..840b3687 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -761,23 +761,35 @@ sub path_to_volume_id {
                 if ($volpath eq $path) {
                     my ($vtype) = $plugin->parse_volname($volname);
                     return ($vtype, $info->{volid});
                 }
             }
-        } elsif ($path =~ m!^\Q$isodir\E/([^/]+$ISO_EXT_RE_0)$!) {
+
+            return ('');
+        }
+
+        if ($path =~ m!^\Q$isodir\E/([^/]+$ISO_EXT_RE_0)$!) {
             my $name = $1;
             return ('iso', "$sid:iso/$name");
-        } elsif ($path =~ m!^\Q$tmpldir\E/([^/]+$VZTMPL_EXT_RE_1)$!) {
+        }
+
+        if ($path =~ m!^\Q$tmpldir\E/([^/]+$VZTMPL_EXT_RE_1)$!) {
             my $name = $1;
             return ('vztmpl', "$sid:vztmpl/$name");
-        } elsif ($path =~ m!^\Q$backupdir\E/([^/]+$BACKUP_EXT_RE_2)$!) {
+        }
+
+        if ($path =~ m!^\Q$backupdir\E/([^/]+$BACKUP_EXT_RE_2)$!) {
             my $name = $1;
             return ('backup', "$sid:backup/$name");
-        } elsif ($path =~ m!^\Q$snippetsdir\E/([^/]+)$!) {
+        }
+
+        if ($path =~ m!^\Q$snippetsdir\E/([^/]+)$!) {
             my $name = $1;
             return ('snippets', "$sid:snippets/$name");
-        } elsif ($path =~ m!^\Q$importdir\E/(${SAFE_CHAR_CLASS_RE}+${IMPORT_EXT_RE_1})$!) {
+        }
+
+        if ($path =~ m!^\Q$importdir\E/(${SAFE_CHAR_CLASS_RE}+${IMPORT_EXT_RE_1})$!) {
             my $name = $1;
             return ('import', "$sid:import/$name");
         }
     }
 
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 14/51] storage: heave vtype file path parsing logic inside loop into helper
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (12 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 13/51] storage: break up if-elsif chain in path_to_volume_id Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 15/51] storage: clean up code that was moved into helper in path_to_volume_id Max R. Carrara
                   ` (30 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage.pm | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 840b3687..f5c866db 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -734,14 +734,12 @@ sub path_to_volume_id {
 
     # Note: abs_path() return undef if $path doesn not exist
     # for example when nfs storage is not mounted
     $path = abs_path($path) || $path;
 
-    for my $sid (keys $ids->%*) {
-        my $scfg = $ids->{$sid};
-        next if !$scfg->{path};
-        my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
+    my $parse_volid_from_file_path = sub {
+        my ($plugin, $sid, $scfg) = @_;
 
         # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
         # on the next APIAGE reset.
         my $imagedir = $plugin->get_subdir($scfg, 'images');
         my $isodir = $plugin->get_subdir($scfg, 'iso');
@@ -789,10 +787,21 @@ sub path_to_volume_id {
 
         if ($path =~ m!^\Q$importdir\E/(${SAFE_CHAR_CLASS_RE}+${IMPORT_EXT_RE_1})$!) {
             my $name = $1;
             return ('import', "$sid:import/$name");
         }
+
+        return ('');
+    };
+
+    for my $sid (keys $ids->%*) {
+        my $scfg = $ids->{$sid};
+        next if !$scfg->{path};
+        my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
+
+        my ($vtype, $volid) = $parse_volid_from_file_path->($plugin, $sid, $scfg);
+        return ($vtype, $volid) if $vtype;
     }
 
     # can't map path to volume id
     return ('');
 }
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 15/51] storage: clean up code that was moved into helper in path_to_volume_id
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (13 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 14/51] storage: heave vtype file path parsing logic inside loop into helper Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 16/51] api: status: move content type assert for up-/downloads into helper Max R. Carrara
                   ` (29 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

In order to make the parsing logic in `path_to_volume_id()` both
simpler and more manageable for future changes, do the following:

1. Iterate over a list of vtypes that store their contents in
   subdirectories and fetch said subdirectories dynamically instead of
   hardcoding them.

2. Explicitly exclude the `rootdir` vtype in this list of vtypes, as
   its subdirectory is currently not used. Since content directories
   may be nested inside one another, check the vtypes with the longest
   subdirectory first, so that a path belongs to the vtype whose
   subdirectory contains it most closely.

3. Strip the vtype subdirectory from the beginning of the path in
   order to make matching on the file name easier. Return early as a
   safeguard if this fails.

4. Adapt the original regexes in the parsing helper to not include the
   vtype subdir anymore, since this is now being stripped. Anchor them
   at the start of the remainder, so that files in subdirectories of a
   vtype subdir do not resolve to a volume directly inside it.

Additionally, it is necessary to still return a (vtype, volid) pair in
the new helper, because of the new `vm-vol`, `vm-state`, and
`ct-vol` volume types. This only matters for the `images` case.

Note that the parsing logic of the `parse_volid_from_file_path()`
helper inside `path_to_volume_id()` now is very similar to the parsing
logic of the `get_subdir_file_info()` closure in the
`get_subdir_files()` helper inside `Plugin.pm`. This opens up further
simplification / code de-duplication in the future, so that our
parsing logic can eventually be kept in one place only.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage.pm | 101 ++++++++++++++++++++++++++++++++++-----------
 1 file changed, 76 insertions(+), 25 deletions(-)

diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index f5c866db..39e72933 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -23,11 +23,13 @@ use PVE::JSONSchema;
 use PVE::INotify;
 use PVE::RPCEnvironment;
 use PVE::SSHInfo;
 use PVE::RESTEnvironment qw(log_warn);
 
-use PVE::Storage::Common;
+use PVE::Storage::Common qw(
+    plugin_get_default_vtype_subdirs
+);
 use PVE::Storage::Plugin;
 use PVE::Storage::DirPlugin;
 use PVE::Storage::LVMPlugin;
 use PVE::Storage::LvmThinPlugin;
 use PVE::Storage::NFSPlugin;
@@ -734,74 +736,123 @@ sub path_to_volume_id {
 
     # Note: abs_path() return undef if $path doesn not exist
     # for example when nfs storage is not mounted
     $path = abs_path($path) || $path;
 
+    my $get_vtypes_to_check = sub {
+        my ($plugin, $scfg) = @_;
+
+        # TODO: vtype split: handle directory for 'rootdir' type.
+        # CTs currently use the same dir as VMs --> the dir for the 'images' type.
+        # Directory is therefore unused right now, so exclude it.
+        my $vtypes = [grep { $_ ne 'rootdir' } keys plugin_get_default_vtype_subdirs()->%*];
+
+        my $dir_lengths = {};
+        for my $vtype ($vtypes->@*) {
+            # FIXME: Call plugin_get_vtype_subdir() instead of
+            # $plugin->get_subdir() on the next APIAGE reset.
+            my $subdir = $plugin->get_subdir($scfg, $vtype);
+
+            # Note that we simply use the length of the directory without "/"
+            # separators here, because something like `foo/bar` is always
+            # longer than `foo`. IOW, a contained directory always has strictly
+            # more components.
+            # For "sibling" subdirs, e.g. `foo/bar` and `foo/baz`, the order
+            # doesn't matter.
+            #
+            # This is equivalent to counting the components of a path, but this
+            # method here should be faster.
+            $dir_lengths->{$vtype} = length($subdir =~ s!/+$!!r);
+        }
+
+        my $order = { map { $vtypes->[$_] => $_ } 0 .. $vtypes->$#* };
+
+        return [
+            sort {
+                $dir_lengths->{$b} <=> $dir_lengths->{$a} || $order->{$a} <=> $order->{$b}
+            } $vtypes->@*
+        ];
+    };
+
     my $parse_volid_from_file_path = sub {
-        my ($plugin, $sid, $scfg) = @_;
+        my ($plugin, $sid, $scfg, $vtype) = @_;
 
         # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
         # on the next APIAGE reset.
-        my $imagedir = $plugin->get_subdir($scfg, 'images');
-        my $isodir = $plugin->get_subdir($scfg, 'iso');
-        my $tmpldir = $plugin->get_subdir($scfg, 'vztmpl');
-        my $backupdir = $plugin->get_subdir($scfg, 'backup');
-        my $snippetsdir = $plugin->get_subdir($scfg, 'snippets');
-        my $importdir = $plugin->get_subdir($scfg, 'import');
+        my $vtype_subdir = $plugin->get_subdir($scfg, $vtype);
 
-        if ($path =~ m!^\Q$imagedir\E/(\d+)/([^/\s]+)$!) {
+        # Strip the vtype subdir from beginning of the current path
+        # so that we don't have to take it into account when parsing the file name
+        my $filename = $path;
+        if ($filename !~ s!^\Q$vtype_subdir\E(?=/)!!) {
+            return;
+        }
+
+        if ($vtype eq 'images') {
+            return if $filename !~ m!^/(\d+)/([^/\s]+)$!;
             my $vmid = $1;
             my $name = $2;
 
             my $vollist = $plugin->list_images($sid, $scfg, $vmid);
             for my $info ($vollist->@*) {
                 my ($storeid, $volname) = parse_volume_id($info->{volid});
                 my $volpath = $plugin->path($scfg, $volname, $storeid);
                 if ($volpath eq $path) {
-                    my ($vtype) = $plugin->parse_volname($volname);
-                    return ($vtype, $info->{volid});
+                    my ($parsed_vtype) = $plugin->parse_volname($volname);
+                    return ($parsed_vtype, $info->{volid});
                 }
             }
 
-            return ('');
+            return;
         }
 
-        if ($path =~ m!^\Q$isodir\E/([^/]+$ISO_EXT_RE_0)$!) {
+        if ($vtype eq 'iso') {
+            return if $filename !~ m!^/([^/]+$ISO_EXT_RE_0)$!;
             my $name = $1;
-            return ('iso', "$sid:iso/$name");
+            return ($vtype, "$sid:iso/$name");
         }
 
-        if ($path =~ m!^\Q$tmpldir\E/([^/]+$VZTMPL_EXT_RE_1)$!) {
+        if ($vtype eq 'vztmpl') {
+            return if $filename !~ m!^/([^/]+$VZTMPL_EXT_RE_1)$!;
             my $name = $1;
-            return ('vztmpl', "$sid:vztmpl/$name");
+            return ($vtype, "$sid:vztmpl/$name");
         }
 
-        if ($path =~ m!^\Q$backupdir\E/([^/]+$BACKUP_EXT_RE_2)$!) {
+        if ($vtype eq 'backup') {
+            return if $filename !~ m!^/([^/]+$BACKUP_EXT_RE_2)$!;
             my $name = $1;
-            return ('backup', "$sid:backup/$name");
+            return ($vtype, "$sid:backup/$name");
         }
 
-        if ($path =~ m!^\Q$snippetsdir\E/([^/]+)$!) {
+        if ($vtype eq 'snippets') {
+            return if $filename !~ m!^/([^/]+)$!;
             my $name = $1;
-            return ('snippets', "$sid:snippets/$name");
+            return ($vtype, "$sid:snippets/$name");
         }
 
-        if ($path =~ m!^\Q$importdir\E/(${SAFE_CHAR_CLASS_RE}+${IMPORT_EXT_RE_1})$!) {
+        if ($vtype eq 'import') {
+            return if $filename !~ m!^/(${SAFE_CHAR_CLASS_RE}+${IMPORT_EXT_RE_1})$!;
             my $name = $1;
-            return ('import', "$sid:import/$name");
+            return ($vtype, "$sid:import/$name");
         }
 
-        return ('');
+        return;
     };
 
     for my $sid (keys $ids->%*) {
         my $scfg = $ids->{$sid};
         next if !$scfg->{path};
         my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
 
-        my ($vtype, $volid) = $parse_volid_from_file_path->($plugin, $sid, $scfg);
-        return ($vtype, $volid) if $vtype;
+        for my $vtype ($get_vtypes_to_check->($plugin, $scfg)->@*) {
+            my ($actual_vtype, $volid) =
+                $parse_volid_from_file_path->($plugin, $sid, $scfg, $vtype);
+
+            if (defined($volid)) {
+                return ($actual_vtype, $volid);
+            }
+        }
     }
 
     # can't map path to volume id
     return ('');
 }
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 16/51] api: status: move content type assert for up-/downloads into helper
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (14 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 15/51] storage: clean up code that was moved into helper in path_to_volume_id Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 17/51] api: status: use get_subdir helper method to get content directory Max R. Carrara
                   ` (28 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/API2/Storage/Status.pm | 41 +++++++++++++++++++++-------------
 1 file changed, 26 insertions(+), 15 deletions(-)

diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm
index 57ab92be..a9f902f9 100644
--- a/src/PVE/API2/Storage/Status.pm
+++ b/src/PVE/API2/Storage/Status.pm
@@ -42,10 +42,33 @@ __PACKAGE__->register_method({
 __PACKAGE__->register_method({
     subclass => "PVE::API2::Storage::FileRestore",
     path => '{storage}/file-restore',
 });
 
+# Content types that may currently be up- or downloaded via the API
+my @ALLOWED_FILE_TRANSFER_CONTENT_TYPES = ('iso', 'vztmpl', 'import');
+
+my sub assert_can_transfer_files : prototype($$$) {
+    my ($scfg, $storeid, $vtype) = @_;
+
+    if (!defined($scfg->{path})) {
+        die "unable to transfer files from / to storage type '$scfg->{type}'"
+            . " - not a file-based storage\n";
+    }
+
+    if (!$scfg->{content}->{$vtype}) {
+        die "storage '$storeid' is not configured for content type '$vtype'\n";
+    }
+
+    my $is_allowed_vtype = grep { $_ eq $vtype } @ALLOWED_FILE_TRANSFER_CONTENT_TYPES;
+    if (!$is_allowed_vtype) {
+        raise_param_exc({ content => "content type '$vtype' not allowed for upload / download" });
+    }
+
+    return;
+}
+
 my sub assert_ova_contents {
     my ($file) = @_;
 
     # test if it's really a tar file with an ovf file inside
     my $hasOvf = 0;
@@ -575,15 +598,14 @@ __PACKAGE__->register_method({
         my $cfg = PVE::Storage::config();
 
         my ($node, $storage) = $param->@{qw(node storage)};
         my $scfg = PVE::Storage::storage_check_enabled($cfg, $storage, $node);
 
-        die "can't upload to storage type '$scfg->{type}'\n"
-            if !defined($scfg->{path});
-
         my $content = $param->{content};
 
+        assert_can_transfer_files($scfg, $storage, $content);
+
         my $tmpfilename = $param->{tmpfilename};
         die "missing temporary file name\n" if !$tmpfilename;
 
         my $size = -s $tmpfilename;
         die "temporary file '$tmpfilename' does not exist\n" if !defined($size);
@@ -617,17 +639,12 @@ __PACKAGE__->register_method({
             } else {
                 $image_format = $format;
             }
 
             $path = PVE::Storage::get_import_dir($cfg, $storage);
-        } else {
-            raise_param_exc({ content => "upload content type '$content' not allowed" });
         }
 
-        die "storage '$storage' does not support '$content' content\n"
-            if !$scfg->{content}->{$content};
-
         my $dest = "$path/$filename";
         my $dirname = dirname($dest);
 
         # best effort to match apl_download behaviour
         chmod 0644, $tmpfilename;
@@ -815,17 +832,13 @@ __PACKAGE__->register_method({
         my $cfg = PVE::Storage::config();
 
         my ($node, $storage, $compression) = $param->@{qw(node storage compression)};
         my $scfg = PVE::Storage::storage_check_enabled($cfg, $storage, $node);
 
-        die "can't upload to storage type '$scfg->{type}', not a file based storage!\n"
-            if !defined($scfg->{path});
-
         my ($content, $url) = $param->@{ 'content', 'url' };
 
-        die "storage '$storage' is not configured for content-type '$content'\n"
-            if !$scfg->{content}->{$content};
+        assert_can_transfer_files($scfg, $storage, $content);
 
         my $filename = PVE::Storage::normalize_content_filename($param->{filename});
 
         my $path;
         my $is_ova = 0;
@@ -854,12 +867,10 @@ __PACKAGE__->register_method({
             } else {
                 $image_format = $format;
             }
 
             $path = PVE::Storage::get_import_dir($cfg, $storage);
-        } else {
-            raise_param_exc({ content => "upload content-type '$content' is not allowed" });
         }
 
         PVE::Storage::activate_storage($cfg, $storage);
         File::Path::make_path($path);
 
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 17/51] api: status: use get_subdir helper method to get content directory
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (15 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 16/51] api: status: move content type assert for up-/downloads into helper Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 18/51] api: status: move up-/download file path parsing code into helper Max R. Carrara
                   ` (27 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Instead of calling the dedicated `get_$VTYPE_dir()` functions from
PVE::Storage, call `$plugin->get_subdir()` directly. Mark it with a
FIXME to replace it with `plugin_get_vtype_subdir()` from the common
module in the future.

There is IMO no reason to call a separate sub for each vtype here, and
it also doesn't split the declaration of `$path` from its assignment.

We may want to consider replacing `get_iso_dir()` and related subs
with a single subroutine instead as well at some point in the future.
When something calls `get_iso_dir()` for example, that call site
obviously already knows what vtype it is fetching the subdirectory
for.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/API2/Storage/Status.pm | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm
index a9f902f9..9fb16888 100644
--- a/src/PVE/API2/Storage/Status.pm
+++ b/src/PVE/API2/Storage/Status.pm
@@ -610,41 +610,41 @@ __PACKAGE__->register_method({
         my $size = -s $tmpfilename;
         die "temporary file '$tmpfilename' does not exist\n" if !defined($size);
 
         my $filename = PVE::Storage::normalize_content_filename($param->{filename});
 
-        my $path;
         my $is_ova = 0;
         my $image_format;
 
         if ($content eq 'iso') {
             if ($filename !~ m![^/]+$PVE::Storage::ISO_EXT_RE_0$!) {
                 raise_param_exc({ filename => "wrong file extension" });
             }
-            $path = PVE::Storage::get_iso_dir($cfg, $storage);
         } elsif ($content eq 'vztmpl') {
             if ($filename !~ m![^/]+$PVE::Storage::VZTMPL_EXT_RE_1$!) {
                 raise_param_exc({ filename => "wrong file extension" });
             }
-            $path = PVE::Storage::get_vztmpl_dir($cfg, $storage);
         } elsif ($content eq 'import') {
             if ($filename !~
                 m!${PVE::Storage::SAFE_CHAR_CLASS_RE}+$PVE::Storage::UPLOAD_IMPORT_EXT_RE_1$!
             ) {
                 raise_param_exc({ filename => "invalid filename or wrong extension" });
             }
             my $format = $1;
 
             if ($format eq 'ova') {
                 $is_ova = 1;
             } else {
                 $image_format = $format;
             }
-
-            $path = PVE::Storage::get_import_dir($cfg, $storage);
         }
 
+        # FIXME: Call plugin_get_vtype_subdir() instead of
+        # $plugin->get_subdir() on the next APIAGE reset.
+        my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
+        my $path = $plugin->get_subdir($scfg, $content);
+
         my $dest = "$path/$filename";
         my $dirname = dirname($dest);
 
         # best effort to match apl_download behaviour
         chmod 0644, $tmpfilename;
@@ -838,41 +838,41 @@ __PACKAGE__->register_method({
 
         assert_can_transfer_files($scfg, $storage, $content);
 
         my $filename = PVE::Storage::normalize_content_filename($param->{filename});
 
-        my $path;
         my $is_ova = 0;
         my $image_format;
 
         if ($content eq 'iso') {
             if ($filename !~ m![^/]+$PVE::Storage::ISO_EXT_RE_0$!) {
                 raise_param_exc({ filename => "wrong file extension" });
             }
-            $path = PVE::Storage::get_iso_dir($cfg, $storage);
         } elsif ($content eq 'vztmpl') {
             if ($filename !~ m![^/]+$PVE::Storage::VZTMPL_EXT_RE_1$!) {
                 raise_param_exc({ filename => "wrong file extension" });
             }
-            $path = PVE::Storage::get_vztmpl_dir($cfg, $storage);
         } elsif ($content eq 'import') {
             if ($filename !~
                 m!${PVE::Storage::SAFE_CHAR_CLASS_RE}+$PVE::Storage::UPLOAD_IMPORT_EXT_RE_1$!
             ) {
                 raise_param_exc({ filename => "invalid filename or wrong extension" });
             }
             my $format = $1;
 
             if ($format eq 'ova') {
                 $is_ova = 1;
             } else {
                 $image_format = $format;
             }
-
-            $path = PVE::Storage::get_import_dir($cfg, $storage);
         }
 
+        # FIXME: Call plugin_get_vtype_subdir() instead of
+        # $plugin->get_subdir() on the next APIAGE reset.
+        my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
+        my $path = $plugin->get_subdir($scfg, $content);
+
         PVE::Storage::activate_storage($cfg, $storage);
         File::Path::make_path($path);
 
         my $dccfg = PVE::Cluster::cfs_read_file('datacenter.cfg');
         my $opts = {
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 18/51] api: status: move up-/download file path parsing code into helper
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (16 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 17/51] api: status: use get_subdir helper method to get content directory Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 19/51] api: status: simplify file content assertion logic for up-/download Max R. Carrara
                   ` (26 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Create a separate helper for the common file path parsing parts of the
upload / download_url API methods that returns the extension of the
uploaded file. Break up the if-elsif chain along the way.

Keep the inner if-clauses for the 'import' vtype inline for now for
further simplifications later. They still need to check for the
'import' vtype, since only the extensions of such files are image
formats.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/API2/Storage/Status.pm | 77 +++++++++++++++++++---------------
 1 file changed, 43 insertions(+), 34 deletions(-)

diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm
index 9fb16888..4ef7755a 100644
--- a/src/PVE/API2/Storage/Status.pm
+++ b/src/PVE/API2/Storage/Status.pm
@@ -65,10 +65,45 @@ my sub assert_can_transfer_files : prototype($$$) {
     }
 
     return;
 }
 
+my sub parse_transferred_file_path_extension : prototype($$) {
+    my ($path, $vtype) = @_;
+
+    if ($vtype eq 'iso') {
+        if ($path !~ m![^/]+$PVE::Storage::ISO_EXT_RE_0$!) {
+            raise_param_exc({ filename => "wrong file extension" });
+        }
+
+        my $ext = $1;
+        return $ext;
+    }
+
+    if ($vtype eq 'vztmpl') {
+        if ($path !~ m![^/]+$PVE::Storage::VZTMPL_EXT_RE_1$!) {
+            raise_param_exc({ filename => "wrong file extension" });
+        }
+
+        my $ext = $1;
+        return $ext;
+    }
+
+    if ($vtype eq 'import') {
+        if (
+            $path !~ m!${PVE::Storage::SAFE_CHAR_CLASS_RE}+$PVE::Storage::UPLOAD_IMPORT_EXT_RE_1$!
+        ) {
+            raise_param_exc({ filename => "invalid filename or wrong extension" });
+        }
+
+        my $ext = $1;
+        return $ext;
+    }
+
+    die "upload / download: failed to parse '$path' - unhandled content type '$vtype'\n";
+}
+
 my sub assert_ova_contents {
     my ($file) = @_;
 
     # test if it's really a tar file with an ovf file inside
     my $hasOvf = 0;
@@ -613,30 +648,17 @@ __PACKAGE__->register_method({
         my $filename = PVE::Storage::normalize_content_filename($param->{filename});
 
         my $is_ova = 0;
         my $image_format;
 
-        if ($content eq 'iso') {
-            if ($filename !~ m![^/]+$PVE::Storage::ISO_EXT_RE_0$!) {
-                raise_param_exc({ filename => "wrong file extension" });
-            }
-        } elsif ($content eq 'vztmpl') {
-            if ($filename !~ m![^/]+$PVE::Storage::VZTMPL_EXT_RE_1$!) {
-                raise_param_exc({ filename => "wrong file extension" });
-            }
-        } elsif ($content eq 'import') {
-            if ($filename !~
-                m!${PVE::Storage::SAFE_CHAR_CLASS_RE}+$PVE::Storage::UPLOAD_IMPORT_EXT_RE_1$!
-            ) {
-                raise_param_exc({ filename => "invalid filename or wrong extension" });
-            }
-            my $format = $1;
+        my $extension = parse_transferred_file_path_extension($filename, $content);
 
-            if ($format eq 'ova') {
+        if ($content eq 'import') {
+            if ($extension eq 'ova') {
                 $is_ova = 1;
             } else {
-                $image_format = $format;
+                $image_format = $extension;
             }
         }
 
         # FIXME: Call plugin_get_vtype_subdir() instead of
         # $plugin->get_subdir() on the next APIAGE reset.
@@ -841,30 +863,17 @@ __PACKAGE__->register_method({
         my $filename = PVE::Storage::normalize_content_filename($param->{filename});
 
         my $is_ova = 0;
         my $image_format;
 
-        if ($content eq 'iso') {
-            if ($filename !~ m![^/]+$PVE::Storage::ISO_EXT_RE_0$!) {
-                raise_param_exc({ filename => "wrong file extension" });
-            }
-        } elsif ($content eq 'vztmpl') {
-            if ($filename !~ m![^/]+$PVE::Storage::VZTMPL_EXT_RE_1$!) {
-                raise_param_exc({ filename => "wrong file extension" });
-            }
-        } elsif ($content eq 'import') {
-            if ($filename !~
-                m!${PVE::Storage::SAFE_CHAR_CLASS_RE}+$PVE::Storage::UPLOAD_IMPORT_EXT_RE_1$!
-            ) {
-                raise_param_exc({ filename => "invalid filename or wrong extension" });
-            }
-            my $format = $1;
+        my $extension = parse_transferred_file_path_extension($filename, $content);
 
-            if ($format eq 'ova') {
+        if ($content eq 'import') {
+            if ($extension eq 'ova') {
                 $is_ova = 1;
             } else {
-                $image_format = $format;
+                $image_format = $extension;
             }
         }
 
         # FIXME: Call plugin_get_vtype_subdir() instead of
         # $plugin->get_subdir() on the next APIAGE reset.
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 19/51] api: status: simplify file content assertion logic for up-/download
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (17 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 18/51] api: status: move up-/download file path parsing code into helper Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 20/51] test: guest import: add tests for PVE::GuestImport Max R. Carrara
                   ` (25 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Create another helper for the file content assertion logic in the
upload / download_url API methods. Absorb the `assert_ova_contents()`
helper along the way, since it does not need to be standalone anymore.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/API2/Storage/Status.pm | 91 ++++++++++++----------------------
 1 file changed, 32 insertions(+), 59 deletions(-)

diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm
index 4ef7755a..f18b64cb 100644
--- a/src/PVE/API2/Storage/Status.pm
+++ b/src/PVE/API2/Storage/Status.pm
@@ -100,29 +100,44 @@ my sub parse_transferred_file_path_extension : prototype($$) {
     }
 
     die "upload / download: failed to parse '$path' - unhandled content type '$vtype'\n";
 }
 
-my sub assert_ova_contents {
-    my ($file) = @_;
+my sub assert_file_transfer_contents_valid : prototype($$$) {
+    my ($path, $vtype, $extension) = @_;
 
-    # test if it's really a tar file with an ovf file inside
-    my $hasOvf = 0;
-    run_command(
-        ['tar', '-t', '-f', $file],
-        outfunc => sub {
-            my ($line) = @_;
+    if ($vtype eq 'iso') {
+        PVE::Storage::assert_iso_content($path);
+        return;
+    }
 
-            if ($line =~ m/\.ovf$/) {
-                $hasOvf = 1;
-            }
-        },
-    );
+    if ($vtype eq 'import') {
+        if ($extension eq 'ova') {
+            # test if it's really a tar file with an ovf file inside
+            my $hasOvf = 0;
+            run_command(
+                ['tar', '-t', '-f', $path],
+                outfunc => sub {
+                    my ($line) = @_;
 
-    die "ova archive has no .ovf file inside\n" if !$hasOvf;
+                    if ($line =~ m/\.ovf$/) {
+                        $hasOvf = 1;
+                    }
+                },
+            );
 
-    return 1;
+            die "ova archive has no .ovf file inside\n" if !$hasOvf;
+            return;
+        }
+
+        my $is_untrusted = 1;
+        my $timeout = 10;
+
+        PVE::Storage::file_size_info($path, $timeout, $extension, $is_untrusted);
+
+        return;
+    }
 }
 
 __PACKAGE__->register_method({
     name => 'index',
     path => '',
@@ -644,24 +659,12 @@ __PACKAGE__->register_method({
 
         my $size = -s $tmpfilename;
         die "temporary file '$tmpfilename' does not exist\n" if !defined($size);
 
         my $filename = PVE::Storage::normalize_content_filename($param->{filename});
-
-        my $is_ova = 0;
-        my $image_format;
-
         my $extension = parse_transferred_file_path_extension($filename, $content);
 
-        if ($content eq 'import') {
-            if ($extension eq 'ova') {
-                $is_ova = 1;
-            } else {
-                $image_format = $extension;
-            }
-        }
-
         # FIXME: Call plugin_get_vtype_subdir() instead of
         # $plugin->get_subdir() on the next APIAGE reset.
         my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
         my $path = $plugin->get_subdir($scfg, $content);
 
@@ -725,20 +728,11 @@ __PACKAGE__->register_method({
                         print "\n"; # the front end expects the error to reside at the last line without any noise
                         die "checksum mismatch: got '$checksum_got' != expect '$checksum'\n";
                     }
                 }
 
-                if ($content eq 'iso') {
-                    PVE::Storage::assert_iso_content($tmpfilename);
-                }
-
-                if ($is_ova) {
-                    assert_ova_contents($tmpfilename);
-                } elsif (defined($image_format)) {
-                    # checks untrusted image
-                    PVE::Storage::file_size_info($tmpfilename, 10, $image_format, 1);
-                }
+                assert_file_transfer_contents_valid($tmpfilename, $content, $extension);
             };
             if (my $err = $@) {
                 # unlinks only the temporary file from the http server
                 unlink $tmpfilename
                     or $! == ENOENT
@@ -859,24 +853,12 @@ __PACKAGE__->register_method({
         my ($content, $url) = $param->@{ 'content', 'url' };
 
         assert_can_transfer_files($scfg, $storage, $content);
 
         my $filename = PVE::Storage::normalize_content_filename($param->{filename});
-
-        my $is_ova = 0;
-        my $image_format;
-
         my $extension = parse_transferred_file_path_extension($filename, $content);
 
-        if ($content eq 'import') {
-            if ($extension eq 'ova') {
-                $is_ova = 1;
-            } else {
-                $image_format = $extension;
-            }
-        }
-
         # FIXME: Call plugin_get_vtype_subdir() instead of
         # $plugin->get_subdir() on the next APIAGE reset.
         my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
         my $path = $plugin->get_subdir($scfg, $content);
 
@@ -898,20 +880,11 @@ __PACKAGE__->register_method({
         }
 
         $opts->{assert_file_validity} = sub {
             my ($tmp_path) = @_;
 
-            if ($content eq 'iso') {
-                PVE::Storage::assert_iso_content($tmp_path);
-            }
-
-            if ($is_ova) {
-                assert_ova_contents($tmp_path);
-            } elsif (defined($image_format)) {
-                # checks untrusted image
-                PVE::Storage::file_size_info($tmp_path, 10, $image_format, 1);
-            }
+            assert_file_transfer_contents_valid($tmp_path, $content, $extension);
         };
 
         my $worker = sub {
             if ($compression) {
                 die "decompression not supported for $content\n" if $content ne 'iso';
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 20/51] test: guest import: add tests for PVE::GuestImport
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (18 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 19/51] api: status: simplify file content assertion logic for up-/download Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 21/51] tree-wide: introduce parsing module and replace usages of ISO_EXT_RE_0 Max R. Carrara
                   ` (24 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Set up a new test script for the `extract_disk_from_import_file()` sub
of `PVE::GuestImport` and run it as part of
`src/test/run_plugin_tests.pl`.

Make this possible by mocking cluster-related functionality that would
otherwise require to be run as root, namely reading, updating and
locking the storage config, as well as retrieving the list of
available VMs.

Then, make the test cases declarative in such a way that they are
mostly self-documenting regarding what they test. Document the keys
that the test declarations use.

Run each test in its clean environment, creating and removing the
entire directory structure before and after each test. Provide small
helpers specific to the testing code for creating arbitrary QEMU
images / .ova files and document them briefly.

Test for successful as well as failed imports.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/guest_import_test.pl | 948 ++++++++++++++++++++++++++++++++++
 src/test/run_plugin_tests.pl  |   1 +
 2 files changed, 949 insertions(+)
 create mode 100755 src/test/guest_import_test.pl

diff --git a/src/test/guest_import_test.pl b/src/test/guest_import_test.pl
new file mode 100755
index 00000000..04eec24d
--- /dev/null
+++ b/src/test/guest_import_test.pl
@@ -0,0 +1,948 @@
+#!/usr/bin/env perl
+
+use v5.36;
+
+use lib qw(..);
+
+use re qw(is_regexp);
+
+use Carp qw(confess);
+use File::Basename qw(dirname basename);
+use File::Path qw(make_path remove_tree);
+use File::Spec;
+use File::Temp;
+
+use PVE::Cluster;
+use PVE::GuestImport;
+use PVE::Storage;
+use PVE::Storage::Common qw(plugin_get_vtype_subdir);
+use PVE::Tools qw(run_command);
+
+use Test::More;
+use Test::MockModule;
+
+$ENV{TZ} = 'UTC';
+
+my $DEFAULT_SIZE = 4 * 1024 * 1024; # 4 MiB
+
+my $SOURCE_STOREID = 'source-store';
+my $SOURCE_STORAGE_PATH = File::Temp->newdir();
+
+my $TARGET_STOREID = 'target-store';
+my $TARGET_STORAGE_PATH = File::Temp->newdir();
+
+# Not actually set up on disk or elsewhere
+my $DUMMY_LVMTHIN_STOREID = 'dummy-lvmthin';
+
+my $STORAGE_CFG = <<~EOF;
+dir: $SOURCE_STOREID
+    shared 0
+    path $SOURCE_STORAGE_PATH
+    content images,iso,import
+
+dir: $TARGET_STOREID
+    shared 0
+    path $TARGET_STORAGE_PATH
+    content images,iso,import
+
+lvmthin: $DUMMY_LVMTHIN_STOREID
+    thinpool data
+    vgname pve
+    content images,rootdir
+EOF
+
+my $MOCKED_VMLIST = {
+    version => 1,
+    ids => {
+        1337 => {
+            node => 'node-0',
+            type => 'qemu',
+            version => 4,
+        },
+        1338 => {
+            node => 'node-0',
+            type => 'qemu',
+            version => 7,
+        },
+        1339 => {
+            node => 'node-0',
+            type => 'qemu',
+            version => 7,
+        },
+    },
+};
+
+my $mock_pve_cluster = Test::MockModule->new('PVE::Cluster')->redefine(
+    cfs_update => sub { },
+    get_config => sub($file) {
+        if ($file eq 'storage.cfg') {
+            return $STORAGE_CFG;
+        }
+
+        confess "mocked get_config() - file '$file' not handled";
+    },
+    get_vmlist => sub() { return $MOCKED_VMLIST; },
+);
+
+my $_LOCKDIR = File::Temp->newdir();
+my $mock_pve_storage_plugin = Test::MockModule->new('PVE::Storage::Plugin')->redefine(
+    cluster_lock_storage => sub($class, $storeid, $shared, $timeout, $func, @param) {
+        confess "\$shared = 1 is not handled" if $shared;
+
+        my $res = PVE::Tools::lock_file("$_LOCKDIR/pve-storage-test", $timeout, $func, @param);
+        die $@ if $@;
+        return $res;
+    },
+);
+
+=head3 create_test_image_file
+
+    my $file_path = "/foo/bar/disk.qcow2";
+    create_test_image_file($file_path, 'qcow2');
+
+Create a new QEMU image at C<$file_path> using the C<qemu-img> command,
+optionally with a certain C<$format>.
+
+The known formats are currently C<qcow2>, C<raw>, and C<vmdk>.
+
+=cut
+
+my sub create_test_image_file : prototype($;$) ($file_path, $format = undef) {
+    my $KNOWN_FORMATS = {
+        qcow2 => 1,
+        raw => 1,
+        vmdk => 1,
+    };
+
+    my $command = ['/usr/bin/qemu-img', 'create', $file_path, $DEFAULT_SIZE];
+
+    if (defined($format) && defined($KNOWN_FORMATS->{$format})) {
+        push($command->@*, '-f', $format);
+    }
+
+    eval { run_command($command, timeout => 10); };
+    confess $@ if $@;
+
+    return;
+}
+
+=head3 create_test_ova_file
+
+    my $file_path = "/srv/imports/some-import.ova";
+    my $content_file_name = "some-disk.qcow2";
+
+    create_test_ova_file($file_path, $content_file_name)
+
+Create a new C<.ova> file at the I<absolute> C<$file_path> and put a file named
+C<$content_file_name> inside using the C<tar> command. C<$content_file_name>
+must not have any directory components and is assumed to be in the same
+directory as C<$file_path>.
+
+=cut
+
+my sub create_test_ova_file : prototype($$) ($file_path, $content_file_name) {
+    my $file_path_dir = dirname($file_path);
+
+    confess "\$content_file_name ne basename(\$content_file_name)"
+        if $content_file_name ne basename($content_file_name);
+
+    my $tar_command = [
+        'tar',
+        '-c',
+        '--force-local',
+        '--no-same-owner',
+        '-f',
+        $file_path,
+        '-C',
+        $file_path_dir,
+        $content_file_name,
+    ];
+
+    eval { run_command($tar_command, timeout => 10); };
+    confess $@ if $@;
+
+    return;
+}
+
+=head2 TEST CASE FORMAT
+
+The parameters for individual tests are hashes with the following keys:
+
+    {
+        # Name of the test, also displayed on error.
+        description => "Importing a single disk, no existing disks (qcow2)",
+
+        # Definition of the source storage, i.e. the storage from which to import.
+        source => {
+            # The ID of the source storage.
+            storeid => $SOURCE_STOREID,
+
+            # A setup function used to set up the necessary volumes before the test cases are run.
+            # This is used to actually create the file(s) to be imported, for example.
+            'fn-setup-volumes' => sub($storeid) {
+                # [...]
+                return;
+            },
+        },
+
+        # Definition of the target storage, i.e. the storage which disk images are imported to.
+        target => {
+
+            # The ID of the target storage.
+            storeid => $TARGET_STOREID,
+
+            # Like for the source storage, this function can be used to set up any pre-existing
+            # volumes for the storage.
+            'fn-setup-volumes' => sub($storeid) {
+                # [...]
+                return;
+            },
+        },
+
+        # The cases being tested.
+        cases => [
+            {
+                # The name of the volume to import.
+                volname => 'import/some-import.ova/disk.qcow2',
+
+                # The ID of the guest to import the volume for.
+                vmid => 1337,
+
+                # The expected outcome when the invocation succeeds.
+                # May be a string or a regular expression.
+                # If this is a string, then the output has to be an exact match.
+                'expect-match' => "$TARGET_STOREID:1337/vm-1337-disk-0.qcow2",
+            },
+
+            {
+                volname => 'import/some-import.ova/disk-link.qcow2',
+                vmid => 1337,
+
+                # The expected outcome when the invocation fails.
+                # May be a string or a regular expression.
+                # If this is a string, then the output has to be an exact match.
+                'expect-fail' => qr/extracted file .* not a regular file/,
+            },
+        ],
+    },
+
+=cut
+
+my $tests = [
+    {
+        description => "Importing a single disk, no existing disks (qcow2)",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $import_dir = PVE::Storage::get_import_dir($cfg, $storeid);
+
+                my $volume_file_name = 'some-import.ova';
+                my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                my $content_file_name = 'disk.qcow2';
+                my $content_file_path = File::Spec->catfile($import_dir, $content_file_name);
+
+                create_test_image_file($content_file_path, 'qcow2');
+
+                create_test_ova_file($volume_file_path, $content_file_name);
+
+                return;
+            },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'import/some-import.ova/disk.qcow2',
+                vmid => 1337,
+                'expect-match' => "$TARGET_STOREID:1337/vm-1337-disk-0.qcow2",
+            },
+        ],
+    },
+    {
+        description => "Importing a single disk, no existing disks (vmdk)",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $import_dir = PVE::Storage::get_import_dir($cfg, $storeid);
+
+                my $volume_file_name = 'some-import.ova';
+                my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                my $content_file_name = 'disk.vmdk';
+                my $content_file_path = File::Spec->catfile($import_dir, $content_file_name);
+
+                create_test_image_file($content_file_path, 'vmdk');
+
+                create_test_ova_file($volume_file_path, $content_file_name);
+
+                return;
+            },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'import/some-import.ova/disk.vmdk',
+                vmid => 1338,
+                'expect-match' => "$TARGET_STOREID:1338/vm-1338-disk-0.vmdk",
+            },
+        ],
+    },
+    {
+        description => "Importing a single disk, no existing disks (raw)",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $import_dir = PVE::Storage::get_import_dir($cfg, $storeid);
+
+                my $volume_file_name = 'some-import.ova';
+                my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                my $content_file_name = 'disk.raw';
+                my $content_file_path = File::Spec->catfile($import_dir, $content_file_name);
+
+                create_test_image_file($content_file_path, 'raw');
+
+                create_test_ova_file($volume_file_path, $content_file_name);
+
+                return;
+            },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'import/some-import.ova/disk.raw',
+                vmid => 1339,
+                'expect-match' => "$TARGET_STOREID:1339/vm-1339-disk-0.raw",
+            },
+        ],
+    },
+    {
+        description => "Importing a single disk, invalid extension",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $import_dir = PVE::Storage::get_import_dir($cfg, $storeid);
+
+                my $volume_file_name = 'some-import.ova';
+                my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                my $content_file_name = 'disk.txt';
+                my $content_file_path = File::Spec->catfile($import_dir, $content_file_name);
+
+                create_test_image_file($content_file_path, 'qcow2');
+
+                create_test_ova_file($volume_file_path, $content_file_name);
+
+                return;
+            },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'import/some-import.ova/disk.txt',
+                vmid => 1337,
+                'expect-fail' => qr/unable to parse directory volume name/,
+            },
+        ],
+    },
+    {
+        description => "Importing multiple disks, no existing disks (qcow2, vmdk, raw)",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $import_dir = PVE::Storage::get_import_dir($cfg, $storeid);
+
+                {
+                    my $volume_file_name = 'some-qcow2-import.ova';
+                    my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                    my $content_file_name = 'disk.qcow2';
+                    my $content_file_path =
+                        File::Spec->catfile($import_dir, $content_file_name);
+
+                    create_test_image_file($content_file_path, 'qcow2');
+
+                    create_test_ova_file($volume_file_path, $content_file_name);
+                }
+
+                {
+                    my $volume_file_name = 'some-vmdk-import.ova';
+                    my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                    my $content_file_name = 'disk.vmdk';
+                    my $content_file_path =
+                        File::Spec->catfile($import_dir, $content_file_name);
+
+                    create_test_image_file($content_file_path, 'vmdk');
+
+                    create_test_ova_file($volume_file_path, $content_file_name);
+                }
+
+                {
+                    my $volume_file_name = 'some-raw-import.ova';
+                    my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                    my $content_file_name = 'disk.raw';
+                    my $content_file_path =
+                        File::Spec->catfile($import_dir, $content_file_name);
+
+                    create_test_image_file($content_file_path, 'raw');
+
+                    create_test_ova_file($volume_file_path, $content_file_name);
+                }
+
+                return;
+            },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'import/some-qcow2-import.ova/disk.qcow2',
+                vmid => 1337,
+                'expect-match' => "$TARGET_STOREID:1337/vm-1337-disk-0.qcow2",
+            },
+            {
+                volname => 'import/some-vmdk-import.ova/disk.vmdk',
+                vmid => 1337,
+                'expect-match' => "$TARGET_STOREID:1337/vm-1337-disk-1.vmdk",
+            },
+            {
+                volname => 'import/some-raw-import.ova/disk.raw',
+                vmid => 1337,
+                'expect-match' => "$TARGET_STOREID:1337/vm-1337-disk-2.raw",
+            },
+        ],
+    },
+    {
+        description => "Importing multiple disks, existing disks (qcow2, vmdk, raw)",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $import_dir = PVE::Storage::get_import_dir($cfg, $storeid);
+
+                {
+                    my $volume_file_name = 'some-qcow2-import.ova';
+                    my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                    my $content_file_name = 'disk.qcow2';
+                    my $content_file_path =
+                        File::Spec->catfile($import_dir, $content_file_name);
+
+                    create_test_image_file($content_file_path, 'qcow2');
+
+                    create_test_ova_file($volume_file_path, $content_file_name);
+                }
+
+                {
+                    my $volume_file_name = 'some-vmdk-import.ova';
+                    my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                    my $content_file_name = 'disk.vmdk';
+                    my $content_file_path =
+                        File::Spec->catfile($import_dir, $content_file_name);
+
+                    create_test_image_file($content_file_path, 'vmdk');
+
+                    create_test_ova_file($volume_file_path, $content_file_name);
+                }
+
+                {
+                    my $volume_file_name = 'some-raw-import.ova';
+                    my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                    my $content_file_name = 'disk.raw';
+                    my $content_file_path =
+                        File::Spec->catfile($import_dir, $content_file_name);
+
+                    create_test_image_file($content_file_path, 'raw');
+
+                    create_test_ova_file($volume_file_path, $content_file_name);
+                }
+
+                return;
+            },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $vmid = 1337;
+
+                PVE::Storage::vdisk_alloc(
+                    $cfg, $storeid, $vmid, 'qcow2', undef, $DEFAULT_SIZE,
+                );
+
+                PVE::Storage::vdisk_alloc(
+                    $cfg, $storeid, $vmid, 'qcow2', undef, $DEFAULT_SIZE,
+                );
+
+                return;
+            },
+        },
+        cases => [
+            {
+                volname => 'import/some-qcow2-import.ova/disk.qcow2',
+                vmid => 1337,
+                'expect-match' => "$TARGET_STOREID:1337/vm-1337-disk-2.qcow2",
+            },
+            {
+                volname => 'import/some-vmdk-import.ova/disk.vmdk',
+                vmid => 1337,
+                'expect-match' => "$TARGET_STOREID:1337/vm-1337-disk-3.vmdk",
+            },
+            {
+                volname => 'import/some-raw-import.ova/disk.raw',
+                vmid => 1337,
+                'expect-match' => "$TARGET_STOREID:1337/vm-1337-disk-4.raw",
+            },
+        ],
+    },
+    {
+        description => "Importing a single disk, invalid volume name",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'import/some-🐧-import.ova/disk.qcow2',
+                vmid => 1337,
+                'expect-fail' => qr/unable to parse directory volume name/,
+            },
+        ],
+    },
+    {
+        description => "Importing a single disk, invalid volume type",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $iso_dir = PVE::Storage::get_iso_dir($cfg, $storeid);
+
+                my $volume_file_name = 'some-linux-distro.iso';
+                my $volume_file_path = File::Spec->catfile($iso_dir, $volume_file_name);
+
+                create_test_image_file($volume_file_path, undef);
+
+                return;
+            },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'iso/some-linux-distro.iso',
+                vmid => 1337,
+                'expect-fail' => qr/only files with content type 'import' can be extracted/,
+            },
+        ],
+    },
+    {
+        description => "Importing a single disk, not .ova file",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $import_dir = PVE::Storage::get_import_dir($cfg, $storeid);
+
+                my $volume_file_name = 'some-qcow2-import.ovf';
+                my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                create_test_image_file($volume_file_path, undef);
+            },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'import/some-qcow2-import.ovf',
+                vmid => 1337,
+                'expect-fail' => qr/only files from 'ova' format can be extracted/,
+            },
+        ],
+    },
+    {
+        description => "Importing a single disk, no content referenced",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $import_dir = PVE::Storage::get_import_dir($cfg, $storeid);
+
+                my $volume_file_name = 'some-import.ova';
+                my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                my $content_file_name = 'disk.qcow2';
+                my $content_file_path = File::Spec->catfile($import_dir, $content_file_name);
+
+                create_test_image_file($content_file_path, 'qcow2');
+
+                create_test_ova_file($volume_file_path, $content_file_name);
+
+                return;
+            },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'import/some-import.ova',
+                vmid => 1337,
+                'expect-fail' => qr/only files from 'ova' format can be extracted/,
+            },
+        ],
+    },
+    {
+        description => "Importing a single disk, invalid archive format for .ova",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $import_dir = PVE::Storage::get_import_dir($cfg, $storeid);
+
+                my $volume_file_name = 'some-import.ova';
+                my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                create_test_image_file($volume_file_path, 'qcow2');
+
+                return;
+            },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'import/some-import.ova/disk.qcow2',
+                vmid => 1337,
+                'expect-fail' => qr/error during extraction/,
+            },
+        ],
+    },
+    {
+        description => "Importing a single disk, symlink instead of file",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $import_dir = PVE::Storage::get_import_dir($cfg, $storeid);
+
+                my $volume_file_name = 'some-import.ova';
+                my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                my $content_file_name = 'disk.qcow2';
+                my $content_file_path = File::Spec->catfile($import_dir, $content_file_name);
+
+                my $symlink_file_name = 'disk-link.qcow2';
+                my $symlink_file_path = File::Spec->catfile($import_dir, $symlink_file_name);
+
+                create_test_image_file($content_file_path, 'qcow2');
+
+                symlink($content_file_path, $symlink_file_path)
+                    or confess "creating symlink failed";
+
+                create_test_ova_file($volume_file_path, $symlink_file_name);
+
+                return;
+            },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'import/some-import.ova/disk-link.qcow2',
+                vmid => 1337,
+                'expect-fail' => qr/extracted file .* not a regular file/,
+            },
+        ],
+    },
+    {
+        description => "Importing a single disk, directory instead of file",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $import_dir = PVE::Storage::get_import_dir($cfg, $storeid);
+
+                my $volume_file_name = 'some-import.ova';
+                my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                my $content_file_name = 'disk.qcow2';
+                my $content_file_path = File::Spec->catfile($import_dir, $content_file_name);
+
+                make_path($content_file_path, { verbose => 1, mode => 0750 });
+
+                create_test_ova_file($volume_file_path, $content_file_name);
+
+                return;
+            },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'import/some-import.ova/disk.qcow2',
+                vmid => 1337,
+                'expect-fail' => qr/extracted file .* not a regular file/,
+            },
+        ],
+    },
+    {
+        description => "Importing a single disk, file is not an image",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $import_dir = PVE::Storage::get_import_dir($cfg, $storeid);
+
+                my $volume_file_name = 'some-import.ova';
+                my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                my $content_file_name = 'disk.qcow2';
+                my $content_file_path = File::Spec->catfile($import_dir, $content_file_name);
+
+                eval { `touch $content_file_path` };
+                confess $@ if $@;
+
+                create_test_ova_file($volume_file_path, $content_file_name);
+
+                return;
+            },
+        },
+        target => {
+            storeid => $TARGET_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'import/some-import.ova/disk.qcow2',
+                vmid => 1337,
+                'expect-fail' => qr/Could not open .* Image is not in qcow2 format/,
+            },
+        ],
+    },
+    {
+        description => "Importing a single disk, incompatible target storage type",
+        source => {
+            storeid => $SOURCE_STOREID,
+            'fn-setup-volumes' => sub($storeid) {
+                my $cfg = PVE::Storage::config();
+                my $import_dir = PVE::Storage::get_import_dir($cfg, $storeid);
+
+                my $volume_file_name = 'some-import.ova';
+                my $volume_file_path = File::Spec->catfile($import_dir, $volume_file_name);
+
+                my $content_file_name = 'disk.qcow2';
+                my $content_file_path = File::Spec->catfile($import_dir, $content_file_name);
+
+                create_test_image_file($content_file_path, 'qcow2');
+
+                create_test_ova_file($volume_file_path, $content_file_name);
+
+                return;
+            },
+        },
+        target => {
+            storeid => $DUMMY_LVMTHIN_STOREID,
+            'fn-setup-volumes' => sub($storeid) { return; },
+        },
+        cases => [
+            {
+                volname => 'import/some-import.ova/disk.qcow2',
+                vmid => 1339,
+                'expect-fail' => qr/has no path/,
+            },
+        ],
+    },
+];
+
+my sub setup_scfg_dir($scfg) {
+    confess "\$scfg has no 'path' key" if !$scfg->{path};
+
+    make_path($scfg->{path}, { verbose => 1, mode => 0700 });
+
+    my @vtype_subdirs = map { plugin_get_vtype_subdir($scfg, $_) } keys $scfg->{content}->%*;
+
+    make_path(@vtype_subdirs, { verbose => 1, mode => 0755 });
+
+    return;
+}
+
+my sub setup_test_env($test_def) {
+    my ($source, $target) = $test_def->@{qw(source target)};
+
+    my $setup_storage = sub($storage_def) {
+        my $cfg = PVE::Storage::config();
+
+        my $storeid = $storage_def->{storeid};
+        die "\$storeid = undef" if !defined($storeid);
+
+        my $scfg = PVE::Storage::storage_config($cfg, $storeid);
+
+        if ($scfg->{path}) {
+            setup_scfg_dir($scfg);
+        }
+
+        $storage_def->{'fn-setup-volumes'}->($storeid);
+    };
+
+    eval { $setup_storage->($source); };
+    if (my $err = $@) {
+        confess "failed to set up source storage: $err";
+    }
+
+    eval { $setup_storage->($target); };
+    if (my $err = $@) {
+        confess "failed to set up target storage: $err";
+    }
+
+    return;
+}
+
+my sub teardown_scfg_dir($scfg) {
+    confess "\$scfg has no 'path' key" if !$scfg->{path};
+
+    remove_tree($scfg->{path}, { verbose => 0 });
+
+    return;
+}
+
+my sub teardown_test_env($test_def) {
+    my ($source, $target) = $test_def->@{qw(source target)};
+
+    my $teardown_storage = sub($storage_def) {
+        my $cfg = PVE::Storage::config();
+
+        my $storeid = $storage_def->{storeid};
+        die "\$storeid = undef" if !defined($storeid);
+
+        my $scfg = PVE::Storage::storage_config($cfg, $storeid);
+        if ($scfg->{path}) {
+            teardown_scfg_dir($scfg);
+        }
+    };
+
+    eval { $teardown_storage->($target); };
+    if (my $err = $@) {
+        confess "failed to tear down target storage: $err";
+    }
+
+    eval { $teardown_storage->($source); };
+    if (my $err = $@) {
+        confess "failed to tear down source storage: $err";
+    }
+
+    return;
+}
+
+my sub run_test($test_def) {
+    my ($desc, $cases) = $test_def->@{qw(description cases)};
+
+    eval { setup_test_env($test_def); };
+    if (my $err = $@) {
+        teardown_test_env($test_def);
+
+        fail($desc);
+        diag("Failed to set up test environment: $err");
+    }
+
+    my $source_storeid = $test_def->{source}->{storeid};
+    my $target_storeid = $test_def->{target}->{storeid};
+
+    for my $case ($cases->@*) {
+        my ($volname, $vmid) = $case->@{qw(volname vmid)};
+
+        confess "($desc) case cannot have both 'expect-match' and 'expect-fail'"
+            if defined($case->{'expect-match'}) && defined($case->{'expect-fail'});
+
+        my $source_volid = $source_storeid . ':' . $volname;
+
+        my $case_desc = "$desc | $vmid | $volname";
+
+        my $result = eval {
+            return PVE::GuestImport::extract_disk_from_import_file(
+                $source_volid, $vmid, $target_storeid,
+            );
+        };
+        if (my $err = $@) {
+            my $re_expect_fail = $case->{'expect-fail'};
+            $re_expect_fail = qr/^\Q$re_expect_fail\E$/ if !is_regexp($re_expect_fail);
+
+            if (!ok($err =~ $re_expect_fail, $case_desc)) {
+                diag(explain($err));
+                diag("does not match");
+                diag(explain($case->{'expect-fail'}));
+            }
+
+            next;
+        }
+
+        my $re_expect_match = $case->{'expect-match'};
+        $re_expect_match = qr/^\Q$re_expect_match\E$/ if !is_regexp($re_expect_match);
+
+        if (!ok($result =~ $re_expect_match, $case_desc)) {
+            diag(explain($result));
+            diag("does not match");
+            diag(explain($case->{'expect-match'}));
+        }
+    }
+
+    teardown_test_env($test_def);
+
+    return;
+}
+
+my sub main() {
+    my $sum_cases = 0;
+    for my $test_def ($tests->@*) {
+        $sum_cases += scalar($test_def->{cases}->@*);
+    }
+
+    plan tests => $sum_cases;
+
+    for my $test_def ($tests->@*) {
+        run_test($test_def);
+    }
+
+    done_testing();
+
+    return;
+}
+
+main();
diff --git a/src/test/run_plugin_tests.pl b/src/test/run_plugin_tests.pl
index 33c392f5..0783ba49 100755
--- a/src/test/run_plugin_tests.pl
+++ b/src/test/run_plugin_tests.pl
@@ -20,10 +20,11 @@ my $res = $harness->runtests(
     "archive_info_test.pm",
     "content_types_test.pm",
     "filesystem_path_test.pm",
     "free_image_test.pm",
     "get_subdir_test.pm",
+    "guest_import_test.pl",
     "list_volumes_test.pm",
     "parse_volname_test.pm",
     "path_to_volume_id_test.pm",
     "plugin_metadata_test.pm",
     "prune_backups_test.pm",
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 21/51] tree-wide: introduce parsing module and replace usages of ISO_EXT_RE_0
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (19 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 20/51] test: guest import: add tests for PVE::GuestImport Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 22/51] common: test: set up parser testing code, add tests for 'iso' vtype Max R. Carrara
                   ` (23 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Introduce the `PVE::Storage::Common::Parse` module with the following
subroutines:

* `parse_rel_path_as_volname_parts($path, $vtype)`

  Parses a given file path into several smaller parts that constitute
  a volume name for the given volume type.

* `parse_rel_path_as_volname($path, $vtype)`

  Parses a given file path directly into a volume name.

* `parse_volname_as_parts($volname)`

  Parses an existing volume name into its constituent parts.

* `parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype)`

  Parses a given file path into several smaller parts that constitute
  a volume ID for the given storage ID, its configuration and volume
  type.

* `parse_abs_path_as_volid($storeid, $scfg, $path, $vtype)`

  Like `parse_abs_path_as_volid_parts()`, but parses the given path
  directly into a volume ID.

* `parse_volid_as_parts($volid)`

  Parses an existing volume ID into its storage ID and volume name
  parts.

As of this commit, these parsing functions only support the 'iso'
volume type, with the exception of `parse_volid_as_parts()`, which
does not depend on knowing the volume type.

Using the newly introduced parsing helpers, replace all occurrences of
the `PVE::Storage::ISO_EXT_RE_0` regex across the repository.

Since the `ISO_EXT_RE_0` regex is now completely unused, add a FIXME
to note that it should be removed on the next APIAGE reset. Its
removal will be noted in the next APIVER + APIAGE bump.

Additional Notes Regarding the new Parsers
==========================================

Support for other volume types will be added individually in future
commits.

The new *private* regex used for 'iso' vtype parsing is still matching
file extensions case-insensitively, but also matches the entire path
and file name portions of 'iso' file paths and volume names.

These parts are extracted using named regex groups, as that is much
easier to handle and keep track of mentally, even with smaller
regexes. These named groups are the "constituent parts" that are
returned by some of the new parser subroutines.

However, one important difference here is that named regex groups do
not support dashes `-` in their names, only underscores `_`. To keep
things consistent with our style (using dashes instead of underscores
in hash keys and the API), these named groups are formatted before
being returned; underscores are simply substituted with dashes.

Finally, all parser subroutines check whether a parent directory
reference (`..` or "double dots") is contained in the passed or
extracted file path, and return early if there is. This is an
additional safety measure that is intentionally introduced in this
commit to guard against any mishaps in the future as the parsers gain
more functionality, such as supporting nested directories inside
different volume types' subdirectories.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/API2/Storage/Status.pm  |   9 +-
 src/PVE/Storage.pm              |   8 +-
 src/PVE/Storage/Common.pm       |   2 +
 src/PVE/Storage/Common/Makefile |   1 +
 src/PVE/Storage/Common/Parse.pm | 397 ++++++++++++++++++++++++++++++++
 src/PVE/Storage/Plugin.pm       |  19 +-
 6 files changed, 426 insertions(+), 10 deletions(-)
 create mode 100644 src/PVE/Storage/Common/Parse.pm

diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm
index f18b64cb..66e8e141 100644
--- a/src/PVE/API2/Storage/Status.pm
+++ b/src/PVE/API2/Storage/Status.pm
@@ -19,10 +19,13 @@ use PVE::Tools qw(run_command);
 
 use PVE::API2::Storage::Content;
 use PVE::API2::Storage::FileRestore;
 use PVE::API2::Storage::PruneBackups;
 use PVE::Storage;
+use PVE::Storage::Common::Parse qw(
+    parse_rel_path_as_volname_parts
+);
 
 use base qw(PVE::RESTHandler);
 
 my $storage_type_enum = PVE::Storage::Plugin->lookup_types();
 
@@ -69,15 +72,17 @@ my sub assert_can_transfer_files : prototype($$$) {
 
 my sub parse_transferred_file_path_extension : prototype($$) {
     my ($path, $vtype) = @_;
 
     if ($vtype eq 'iso') {
-        if ($path !~ m![^/]+$PVE::Storage::ISO_EXT_RE_0$!) {
+        my $parts = parse_rel_path_as_volname_parts($path, $vtype);
+
+        if (!defined($parts)) {
             raise_param_exc({ filename => "wrong file extension" });
         }
 
-        my $ext = $1;
+        my $ext = $parts->{ext};
         return $ext;
     }
 
     if ($vtype eq 'vztmpl') {
         if ($path !~ m![^/]+$PVE::Storage::VZTMPL_EXT_RE_1$!) {
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 39e72933..474f1ee5 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -26,10 +26,13 @@ use PVE::SSHInfo;
 use PVE::RESTEnvironment qw(log_warn);
 
 use PVE::Storage::Common qw(
     plugin_get_default_vtype_subdirs
 );
+use PVE::Storage::Common::Parse qw(
+    parse_abs_path_as_volid
+);
 use PVE::Storage::Plugin;
 use PVE::Storage::DirPlugin;
 use PVE::Storage::LVMPlugin;
 use PVE::Storage::LvmThinPlugin;
 use PVE::Storage::NFSPlugin;
@@ -115,10 +118,11 @@ if (-d '/usr/share/perl5/PVE/Storage/Custom') {
 PVE::Storage::Plugin->init();
 
 # the following REs indicate the number or capture groups via the trailing digit
 # CAUTION don't forget to update the digits accordingly after messing with the capture groups
 
+# FIXME: remove this regex on the next APIAGE reset.
 our $ISO_EXT_RE_0 = qr/\.(?:iso|img)/i;
 
 our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2))/i;
 
 our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/;
@@ -804,13 +808,11 @@ sub path_to_volume_id {
 
             return;
         }
 
         if ($vtype eq 'iso') {
-            return if $filename !~ m!^/([^/]+$ISO_EXT_RE_0)$!;
-            my $name = $1;
-            return ($vtype, "$sid:iso/$name");
+            return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
         }
 
         if ($vtype eq 'vztmpl') {
             return if $filename !~ m!^/([^/]+$VZTMPL_EXT_RE_1)$!;
             my $name = $1;
diff --git a/src/PVE/Storage/Common.pm b/src/PVE/Storage/Common.pm
index 4f5fde64..d39ee3d0 100644
--- a/src/PVE/Storage/Common.pm
+++ b/src/PVE/Storage/Common.pm
@@ -49,10 +49,12 @@ be grouped in a submodule can also be found here.
 
 =head1 SUBMODULES
 
 =over
 
+=item * C<L<PVE::Storage::Common::Parse>>
+
 =back
 
 =head1 STANDARD OPTIONS FOR JSON SCHEMA
 
 =over
diff --git a/src/PVE/Storage/Common/Makefile b/src/PVE/Storage/Common/Makefile
index 0c4bba5b..0d9b1be1 100644
--- a/src/PVE/Storage/Common/Makefile
+++ b/src/PVE/Storage/Common/Makefile
@@ -1,6 +1,7 @@
 SOURCES = \
+	  Parse.pm			\
 
 
 .PHONY: install
 install:
 	for i in ${SOURCES}; do install -D -m 0644 $$i ${DESTDIR}${PERLDIR}/PVE/Storage/Common/$$i; done
diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
new file mode 100644
index 00000000..ab949d99
--- /dev/null
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -0,0 +1,397 @@
+package PVE::Storage::Common::Parse;
+
+use v5.36;
+
+use PVE::Storage::Common qw(
+    plugin_get_vtype_subdir
+);
+
+use Exporter qw(import);
+
+our @EXPORT_OK = qw(
+    parse_rel_path_as_volname_parts
+    parse_rel_path_as_volname
+    parse_volname_as_parts
+
+    parse_abs_path_as_volid_parts
+    parse_abs_path_as_volid
+    parse_volid_as_parts
+);
+
+=head1 NAME
+
+C<PVE::Storage::Common::Parse> - Storage-related Parsing Functions
+
+=head1 DESCRIPTION
+
+This module contains various parsing functions for use within C<L<PVE::Storage>>,
+its submodules (including storage plugins) and other related modules.
+
+Parsing functions are categorized by their main purpose / area of application
+and may be further subdivided depending on what kind of data type they are
+primarily related to.
+
+=cut
+
+my $RE_PARENT_DIR = quotemeta('..');
+my $RE_CONTAINS_PARENT_DIR = qr!
+    ( ^$RE_PARENT_DIR$ )  #  ..  --> Entire path
+    |
+    ( ^$RE_PARENT_DIR/ )  #  ../ --> Beginning of path
+    |
+    ( /$RE_PARENT_DIR/ )  # /../ --> Between two path components
+    |
+    ( /$RE_PARENT_DIR$ )  # /..  --> End of path
+!xn;
+
+my $RE_ISO_FILE_PATH = qr!
+    (?<path>
+        (?<file> [^/]+ \. (?<ext> (?i: iso|img) ) )
+    )
+!xn;
+
+my $RE_FILE_PATH_FOR_VTYPE = {
+    iso => qr/^$RE_ISO_FILE_PATH$/,
+};
+
+my $RE_VOLNAME_FOR_VTYPE = {
+    iso => qr/^$RE_ISO_FILE_PATH$/,
+};
+
+my sub contains_parent_dir($path) {
+    return $path =~ $RE_CONTAINS_PARENT_DIR;
+}
+
+# A volume ID cannot represent a name with a newline, see parse_volume_id(). Check this explicitly,
+# as the end anchor $ also matches right before a trailing newline.
+my sub contains_newline($path) {
+    return $path =~ m/\n/;
+}
+
+my sub strip_leading_path_separators($path) {
+    return $path =~ s!^/+!!r;
+}
+
+my sub strip_trailing_path_separators($path) {
+    return $path =~ s!/+$!!r;
+}
+
+my sub format_named_groups(%groups) {
+    my $result = {};
+
+    for my $old_key (keys %groups) {
+        my $new_key = $old_key =~ s/_/-/gr;
+        $result->{$new_key} = $groups{$old_key};
+    }
+
+    my @disk_path_components = ();
+
+    if (defined($result->{file})) {
+        $result->{file} = strip_leading_path_separators($result->{file});
+        push(@disk_path_components, $result->{file});
+    }
+
+    if (scalar(@disk_path_components)) {
+        $result->{'disk-path'} = join('/', @disk_path_components);
+    }
+
+    return $result;
+}
+
+my sub split_leading_dir_from_path($path, $directory) {
+    $directory = strip_trailing_path_separators($directory);
+
+    if ("$directory/" eq substr($path, 0, length($directory) + 1, '')) {
+        return ($directory, strip_leading_path_separators($path));
+    }
+
+    return;
+}
+
+=head1 PARSERS RELATED TO VOLUMES
+
+The parsing functions in this section primarily deal with parsing data related
+to storage volumes, primarily C<L<volname>>s and C<L<volid>>s.
+
+=head2 VOLUME NAMES
+
+All subroutines ending in C<_parts> in this section have a set of common
+"parts" that they return.
+
+These are:
+
+    {
+        # The name of the file the volume points to
+        file => 'custom-debian.iso',
+
+        # The extension (suffix) of the file the volume points to
+        ext => 'iso',
+
+        # The "real" path on disk
+        'disk-path' => 'custom-debian.iso',
+
+        # The "whole" path that the volume references,
+        path => 'custom-debian.iso',
+
+        # The volume's type
+        vtype => 'iso',
+
+        # The volname, which is "${vtype}/${path}"
+        volname => 'iso/custom-debian.iso',
+    }
+
+=cut
+
+=head3 parse_rel_path_as_volname_parts
+
+Parses the given relative file path C<$path> according to the given C<$vtype>,
+returning a hashref containing the parts that make up a C<volname> on success.
+
+For example, C<$path = "custom-debian.iso"> and C<$vtype = "iso"> are turned
+into:
+
+    {
+        file => 'custom-debian.iso',
+        ext => 'iso',
+        'disk-path' => 'custom-debian.iso',
+        path => 'custom-debian.iso',
+        vtype => 'iso',
+        volname => 'iso/custom-debian.iso',
+    }
+
+See L<VOLUME NAMES> for more details.
+
+On failure or when the provided C<$vtype> is not supported or does not exist,
+returns C<undef> in scalar context, and an empty list in list context.
+
+B<NOTE:> This function assumes that C<$path> is already relative to the
+directory that corresponds to the given C<$vtype>.
+
+If you want to parse an absolute path for an already known storage instead, see
+C<L<< parse_abs_path_as_volid_parts()|/"parse_abs_path_as_volid_parts" >>>.
+
+For a counterpart to this function, see
+C<L<< parse_volname_as_parts()|/"parse_volname_as_parts" >>>.
+
+=cut
+
+sub parse_rel_path_as_volname_parts : prototype($$) ($path, $vtype) {
+    return if contains_parent_dir($path) || contains_newline($path);
+
+    # TODO: vtype split: Handle parsing for 'images' and 'rootdir' vtypes.
+
+    my $re_filepath = $RE_FILE_PATH_FOR_VTYPE->{$vtype};
+    return if !defined($re_filepath);
+
+    return if $path !~ $re_filepath;
+
+    my $parts = format_named_groups(%+);
+    $parts->{vtype} = $vtype;
+    $parts->{volname} = $vtype . '/' . $parts->{path};
+
+    return $parts;
+}
+
+=head3 parse_rel_path_as_volname
+
+Like C<L<< parse_rel_path_as_volname_parts()|/"parse_rel_path_as_volname_parts" >>>, but
+instead of extracting the individual parts of the relative C<$path>, returns
+the correctly formatted C<volname> directly.
+
+For example, C<$path = "custom-debian.iso"> and C<$vtype = "iso"> are turned
+into C<"iso/custom-debian.iso">.
+
+For a counterpart to this function, see
+C<L<< parse_volname_as_parts()|/"parse_volname_as_parts" >>>.
+
+=cut
+
+sub parse_rel_path_as_volname : prototype($$) ($path, $vtype) {
+    return if contains_parent_dir($path) || contains_newline($path);
+
+    # TODO: vtype split: Handle parsing for 'images' and 'rootdir' vtypes.
+
+    my $re_filepath = $RE_FILE_PATH_FOR_VTYPE->{$vtype};
+    return if !defined($re_filepath);
+
+    return if $path !~ $re_filepath;
+
+    return $vtype . '/' . $+{path};
+}
+
+=head3 parse_volname_as_parts
+
+Parses the provided C<$volname> and returns its constituent parts in a hashref
+upon success.
+
+For example, C<$volname = "iso/custom-debian.iso"> is turned into:
+
+    {
+        file => 'custom-debian.iso',
+        ext => 'iso',
+        'disk-path' => 'custom-debian.iso',
+        path => 'custom-debian.iso',
+        vtype => 'iso',
+        volname => 'iso/custom-debian.iso',
+    }
+
+Returns C<undef> in scalar context and an empty list in list context, if
+C<$volname> is prefixed with an unknown or unsupported C<vtype>, or if the path
+after the C<vtype> prefix cannot be parsed.
+
+This function can be seen as a counterpart to
+C<L<< parse_rel_path_as_volname_parts()|/"parse_rel_path_as_volname_parts" >>> and
+C<L<< parse_rel_path_as_volname()|/"parse_rel_path_as_volname" >>> and can be used to
+extract the information embedded within an already existing volume name, such
+as file extensions or the name of the file that a volume refers to.
+
+=cut
+
+sub parse_volname_as_parts : prototype($) ($volname) {
+    # TODO: vtype split: Handle volname for 'images' and 'rootdir' vtypes.
+    my ($vtype, $path) = split('/', $volname, 2);
+
+    # Either variable could be undef or an empty string here; `$path` can also be '0'
+    return if !length($vtype) || !length($path);
+
+    return if contains_parent_dir($path) || contains_newline($path);
+
+    my $re_volname = $RE_VOLNAME_FOR_VTYPE->{$vtype};
+    return if !defined($re_volname);
+
+    return if $path !~ $re_volname;
+
+    my $parts = format_named_groups(%+);
+    $parts->{vtype} = $vtype;
+    $parts->{volname} = $volname;
+
+    return $parts;
+}
+
+=head2 VOLUME IDS
+
+All subroutines ending in C<_parts> in this section have a set of common
+"parts" that they return. Most of there parts are shared with the common parts
+described in L<VOLUME NAMES>, with the exception of C<volid>.
+
+These are:
+
+    {
+        # See VOLUME NAMES
+        file => 'custom-debian.iso',
+        ext => 'iso',
+        'disk-path' => 'custom-debian.iso',
+        path => 'custom-debian.iso',
+        vtype => 'iso',
+        volname => 'iso/custom-debian.iso',
+
+        # Exclusive to volid parsing
+        volid => 'local:iso/custom-debian.iso',
+    }
+
+=cut
+
+my $RE_VOLID = qr!
+    ^
+    (?<volid>
+        (?<storeid> (?i: [a-z][a-z0-9\-\_\.]*[a-z0-9] ) )
+        :    # separated by colon
+        (?<volname> .+)
+    )
+    $
+!xn;
+
+=head3 parse_abs_path_as_volid_parts
+
+Parses the given absolute file path C<$path> according to the given C<$vtype>
+and returns a hashref containing the parts that make up a C<$volid> on success.
+C<$path> must exist for the passed C<$storeid> and its config C<$scfg>.
+
+For example, C<$path = "/var/lib/vz/template/iso/custom-debian.iso"> and
+C<$vtype = "iso"> on the C<local> directory storage are turned into:
+
+    {
+        # Identical to volume name parsing:
+        file => 'custom-debian.iso',
+        ext => 'iso',
+        'disk-path' => 'custom-debian.iso',
+        path => 'custom-debian.iso',
+        vtype => 'iso',
+        volname => 'iso/custom-debian.iso',
+
+        # Specific to volume ID parsers:
+        # "${storeid}:${volname}"
+        volid => 'local:iso/custom-debian.iso',
+    }
+
+See L<VOLUME IDS> for more details.
+
+On failure or when the provided C<$vtype> is not supported or does not exist,
+returns C<undef> in scalar context, and an empty list in list context.
+
+If you are only interested in parsing the portion of the path belonging inside
+the C<vtype> directory independent of a C<$storeid>, see
+C<L<< parse_rel_path_as_volname_parts()|/"parse_rel_path_as_volname_parts" >>>.
+
+For a counterpart to this function, see
+C<L<< parse_volid_as_parts()|/"parse_volid_as_parts" >>>.
+
+=cut
+
+sub parse_abs_path_as_volid_parts : prototype($$$$) ($storeid, $scfg, $path, $vtype) {
+    my $vtype_subdir = plugin_get_vtype_subdir($scfg, $vtype);
+
+    my ($leading_dir, $remainder) = split_leading_dir_from_path($path, $vtype_subdir);
+    return if !defined($leading_dir);
+
+    my $volid_parts = parse_rel_path_as_volname_parts($remainder, $vtype);
+    return if !defined($volid_parts);
+
+    $volid_parts->{volid} = $storeid . ':' . $volid_parts->{volname};
+
+    return $volid_parts;
+}
+
+=head3 parse_abs_path_as_volid
+
+Like C<L<< parse_abs_path_as_volid_parts()|/"parse_abs_path_as_volid_parts" >>>, but
+instead of extracting the individual parts of the C<$path>, returns the
+correctly formatted C<volid> directly.
+
+For a counterpart to this function, see
+C<L<< parse_volid_as_parts()|/"parse_volid_as_parts" >>>.
+
+=cut
+
+sub parse_abs_path_as_volid : prototype($$$$) ($storeid, $scfg, $path, $vtype) {
+    my $vtype_subdir = plugin_get_vtype_subdir($scfg, $vtype);
+
+    my ($leading_dir, $remainder) = split_leading_dir_from_path($path, $vtype_subdir);
+    return if !defined($leading_dir);
+
+    my $volname = parse_rel_path_as_volname($remainder, $vtype);
+    return if !defined($volname);
+
+    return $storeid . ':' . $volname;
+}
+
+=head3 parse_volid_as_parts
+
+Parses the provided C<$volid> and returns its C<storeid> and C<volname> parts
+in a hashref.
+
+On failure, returns C<undef> in scalar context, and an empty list in list context.
+
+The C<volname> part of volumes that represent files can be further parsed into
+its constituent parts using the
+C<L<< parse_volname_as_parts()|/"parse_volname_as_parts" >>> function.
+
+=cut
+
+sub parse_volid_as_parts : prototype($) ($volid) {
+    return if $volid !~ $RE_VOLID;
+
+    return format_named_groups(%+);
+}
+
+1;
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index df8526bd..c5aed4ef 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -17,10 +17,14 @@ use PVE::Cluster qw(cfs_register_file);
 
 use PVE::Storage::Common qw(
     plugin_get_default_vtype_subdirs
     plugin_get_vtype_subdir
 );
+use PVE::Storage::Common::Parse qw(
+    parse_volname_as_parts
+    parse_abs_path_as_volid_parts
+);
 
 use JSON;
 
 use base qw(PVE::SectionConfig);
 
@@ -893,12 +897,16 @@ sub parse_volname {
         my (undef, $format, $isBase) = parse_name_dir($name);
         my $vtype = $class->guest_volume_vtype($format, $name);
         return ($vtype, $name, $vmid, undef, undef, $isBase, $format);
     }
 
-    if ($volname =~ m!^iso/([^/]+$PVE::Storage::ISO_EXT_RE_0)$!) {
-        return ('iso', $1, undef, undef, undef, undef, 'raw');
+    if (defined(my $parts = parse_volname_as_parts($volname))) {
+        my ($vtype, $volume_path) = $parts->@{qw(vtype path)};
+
+        if ($vtype eq 'iso') {
+            return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
+        }
     }
 
     if ($volname =~ m!^vztmpl/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!) {
         return ('vztmpl', $1, undef, undef, undef, undef, 'raw');
     }
@@ -1888,15 +1896,16 @@ my sub get_subdir_files {
         if ($filename !~ s!^\Q$vtype_subdir\E(?=/)!!) {
             return;
         }
 
         if ($vtype eq 'iso') {
-            return if $filename !~ m!/([^/]+$PVE::Storage::ISO_EXT_RE_0)$!i;
+            my $parts = parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
+            return if !defined($parts);
 
             return {
-                volid => "$storeid:iso/$1",
-                format => 'iso',
+                volid => $parts->{volid},
+                format => 'iso', # always 'iso' even if we have a file ending in .img
             };
         }
 
         if ($vtype eq 'vztmpl') {
             return if $filename !~ m!/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!;
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 22/51] common: test: set up parser testing code, add tests for 'iso' vtype
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (20 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 21/51] tree-wide: introduce parsing module and replace usages of ISO_EXT_RE_0 Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 23/51] tree-wide: replace usages of VZTMPL_EXT_RE_1 with parsing functions Max R. Carrara
                   ` (22 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Set up the necessary scaffolding for testing the new parser functions
in `PVE::Storage::Common::Parse` and add tests for the 'iso' vtype.

This also adds `libtest-differences-perl` to `Build-Depends`.

Since the volid parsing functions are built on top of the volname
parsing functions, their tests are built in a similar way; that is,
the volname parser test cases are taken and adapted for the volid
parsers.

This is primarily done to ensure consistency between the two
"families" of parsing functions; for example, the
`parse_abs_path_as_volid_parts()` function should always return the same
hash as the `parse_rel_path_as_volname_parts()` function, except that it
also contains a 'volid' key.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 debian/control                              |   1 +
 src/PVE/Makefile                            |   1 +
 src/PVE/Storage/Common/Makefile             |   4 +
 src/PVE/Storage/Common/test/Makefile        |   6 +
 src/PVE/Storage/Common/test/parser_tests.pl | 328 ++++++++++++++++++++
 src/PVE/Storage/Common/test/run_tests.pl    |  25 ++
 src/PVE/Storage/Makefile                    |   4 +
 7 files changed, 369 insertions(+)
 create mode 100644 src/PVE/Storage/Common/test/Makefile
 create mode 100755 src/PVE/Storage/Common/test/parser_tests.pl
 create mode 100755 src/PVE/Storage/Common/test/run_tests.pl

diff --git a/debian/control b/debian/control
index f87a052a..44e9e6f1 100644
--- a/debian/control
+++ b/debian/control
@@ -8,10 +8,11 @@ Build-Depends: debhelper-compat (= 13),
                libpve-access-control,
                libpve-cluster-perl (>= 9.1.9),
                libpve-common-perl (>= 9.2.10),
                librados2-perl,
                libtest-mockmodule-perl,
+               libtest-differences-perl,
                libxml-libxml-perl,
                lintian,
                open-iscsi,
                perl,
                pve-cluster (>= 5.0-32),
diff --git a/src/PVE/Makefile b/src/PVE/Makefile
index 9e9f6aac..b8fc6325 100644
--- a/src/PVE/Makefile
+++ b/src/PVE/Makefile
@@ -12,8 +12,9 @@ install:
 	make -C BackupProvider install
 	make -C CLI install
 
 .PHONY: test
 test:
+	$(MAKE) -C Storage test
 	$(MAKE) -C test test
 
 clean:
diff --git a/src/PVE/Storage/Common/Makefile b/src/PVE/Storage/Common/Makefile
index 0d9b1be1..7a13371d 100644
--- a/src/PVE/Storage/Common/Makefile
+++ b/src/PVE/Storage/Common/Makefile
@@ -3,5 +3,9 @@ SOURCES = \
 
 
 .PHONY: install
 install:
 	for i in ${SOURCES}; do install -D -m 0644 $$i ${DESTDIR}${PERLDIR}/PVE/Storage/Common/$$i; done
+
+.PHONY: test
+test:
+	make -C test test
diff --git a/src/PVE/Storage/Common/test/Makefile b/src/PVE/Storage/Common/test/Makefile
new file mode 100644
index 00000000..c4b2c0db
--- /dev/null
+++ b/src/PVE/Storage/Common/test/Makefile
@@ -0,0 +1,6 @@
+all: test
+
+.PHONY: test
+test: run_tests.pl
+	./run_tests.pl
+
diff --git a/src/PVE/Storage/Common/test/parser_tests.pl b/src/PVE/Storage/Common/test/parser_tests.pl
new file mode 100755
index 00000000..ea770bb4
--- /dev/null
+++ b/src/PVE/Storage/Common/test/parser_tests.pl
@@ -0,0 +1,328 @@
+#!/usr/bin/perl
+
+use v5.36;
+
+use lib q(../../../..);
+
+use File::Temp;
+use Storable qw(dclone);
+
+use PVE::Storage::Common qw(
+    plugin_get_vtype_subdir
+);
+use PVE::Storage::Common::Parse qw(
+    parse_rel_path_as_volname_parts
+    parse_rel_path_as_volname
+    parse_volname_as_parts
+
+    parse_abs_path_as_volid_parts
+    parse_abs_path_as_volid
+    parse_volid_as_parts
+);
+
+use Test::More qw(no_plan);
+use Test::Differences;
+
+my $volname_tests = [];
+
+my $volname_cases_iso_valid = [
+    # plain paths
+    {
+        path => 'custom-debian.iso',
+        expected => {
+            file => 'custom-debian.iso',
+            ext => 'iso',
+            'disk-path' => 'custom-debian.iso',
+            path => 'custom-debian.iso',
+            vtype => 'iso',
+            volname => 'iso/custom-debian.iso',
+        },
+    },
+    {
+        path => 'archlinux.img',
+        expected => {
+            file => 'archlinux.img',
+            ext => 'img',
+            'disk-path' => 'archlinux.img',
+            path => 'archlinux.img',
+            vtype => 'iso',
+            volname => 'iso/archlinux.img',
+        },
+    },
+
+    # case insensitive file extensions
+    {
+        path => 'Debian 13 Trixie.ISO',
+        expected => {
+            file => 'Debian 13 Trixie.ISO',
+            ext => 'ISO',
+            'disk-path' => 'Debian 13 Trixie.ISO',
+            path => 'Debian 13 Trixie.ISO',
+            vtype => 'iso',
+            volname => 'iso/Debian 13 Trixie.ISO',
+        },
+    },
+    {
+        path => 'Fedora.Img',
+        expected => {
+            file => 'Fedora.Img',
+            ext => 'Img',
+            'disk-path' => 'Fedora.Img',
+            path => 'Fedora.Img',
+            vtype => 'iso',
+            volname => 'iso/Fedora.Img',
+        },
+    },
+];
+
+my $volname_cases_iso_invalid = [
+    {
+        description => "Invalid file extension (iso)",
+        args => {
+            path => 'MacOS-kittycat.dmg',
+            vtype => 'iso',
+        },
+        expected => undef,
+    },
+];
+
+my $volname_cases_newline_invalid = [
+    {
+        description => "Trailing newline (iso)",
+        args => {
+            path => "debian.iso\n",
+            vtype => 'iso',
+        },
+        expected => undef,
+    },
+    {
+        description => "Newline in file name (iso)",
+        args => {
+            path => "deb\nian.iso",
+            vtype => 'iso',
+        },
+        expected => undef,
+    },
+    {
+        description => "Newline in directory name (iso)",
+        args => {
+            path => "sub\ndir/debian.iso",
+            vtype => 'iso',
+        },
+        expected => undef,
+    },
+];
+
+my $cases_valid_all = [
+    $volname_cases_iso_valid,
+];
+
+my $cases_invalid_all = [
+    $volname_cases_iso_invalid, $volname_cases_newline_invalid,
+];
+
+{
+    for my $cases_list ($cases_valid_all->@*) {
+        for my $case ($cases_list->@*) {
+            my $path = $case->{path};
+            my $vtype = $case->{expected}->{vtype};
+
+            push(
+                $volname_tests->@*,
+                {
+                    description => "Valid path ($vtype) \"$path\"",
+                    args => {
+                        path => $path,
+                        vtype => $vtype,
+                    },
+                    expected => $case->{expected},
+                },
+            );
+        }
+    }
+
+    for my $cases_list ($cases_invalid_all->@*) {
+        push($volname_tests->@*, $cases_list->@*);
+    }
+}
+
+my sub run_volname_parsing_tests : prototype($) ($volname_tests) {
+    for my $case ($volname_tests->@*) {
+        subtest $case->{description} => sub() {
+            my ($path, $vtype) = $case->{args}->@{qw(path vtype)};
+
+            my $got_volname_parts = parse_rel_path_as_volname_parts($path, $vtype);
+            my $got_volname = parse_rel_path_as_volname($path, $vtype);
+
+            if (defined($case->{expected})) {
+                eq_or_diff(
+                    $got_volname_parts,
+                    $case->{expected},
+                    'parse_rel_path_as_volname_parts() returns expected hashref',
+                    { context => 50000 },
+                );
+
+                is(
+                    $got_volname,
+                    $case->{expected}->{volname},
+                    'parse_rel_path_as_volname() returns expected volname',
+                );
+
+                my $got_volname_parts_from_volname = parse_volname_as_parts($got_volname);
+
+                eq_or_diff(
+                    $got_volname_parts_from_volname,
+                    $case->{expected},
+                    'parse_volname_as_parts() returns expected hashref from parsed volname',
+                );
+            } else {
+                is($got_volname_parts, undef,
+                    'parse_rel_path_as_volname_parts() returns undef');
+                is($got_volname, undef, 'parse_rel_path_as_volname() returns undef');
+            }
+
+        };
+    }
+
+    return;
+}
+
+my $DEFAULT_STOREID = 'local';
+my $DEFAULT_STORAGE_PATH = File::Temp->newdir();
+my $DEFAULT_SCFG = {
+    type => 'dir',
+    path => $DEFAULT_STORAGE_PATH,
+    shared => 0,
+    content => {
+        iso => 1,
+        rootdir => 1,
+        vztmpl => 1,
+        images => 1,
+        snippets => 1,
+        backup => 1,
+        import => 1,
+    },
+};
+
+my $ALT_STOREID = 'dir-store';
+my $ALT_STORAGE_PATH = File::Temp->newdir();
+my $ALT_SCFG = {
+    type => 'dir',
+    path => $ALT_STORAGE_PATH,
+    shared => 0,
+    content => { $DEFAULT_SCFG->{content}->%* },
+    'content-dirs' => {
+        iso => 'pve/volumes/iso',
+        vztmpl => 'pve/dumps/vz',
+        images => 'pve/guest/images',
+        snippets => 'pve/misc/snippets',
+        backup => 'pve/dumps/bak',
+        import => 'archive/import',
+    },
+};
+
+my sub format_volname_case_to_volid_case($storeid, $scfg, $volname_case) {
+    my $volid_case = dclone($volname_case);
+
+    my ($path, $vtype) = $volid_case->{args}->@{qw(path vtype)};
+
+    my $vtype_subdir = plugin_get_vtype_subdir($scfg, $vtype);
+
+    $volid_case->{args}->{path} = $vtype_subdir . '/' . $path;
+    $volid_case->{args}->{storeid} = $storeid;
+    $volid_case->{args}->{scfg} = $scfg;
+
+    if (defined($volid_case->{expected})) {
+        my $volname = $volid_case->{expected}->{volname};
+        $volid_case->{expected}->{volid} = $storeid . ':' . $volname;
+    }
+
+    return $volid_case;
+}
+
+my $volid_tests = [
+    {
+        description => "volid parsers build on volname parsers' behaviors (1)",
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        cases => [
+            map { format_volname_case_to_volid_case($DEFAULT_STOREID, $DEFAULT_SCFG, $_) }
+                $volname_tests->@*
+        ],
+    },
+    {
+        description => "volid parsers build on volname parsers' behaviors (2)",
+        storeid => $ALT_STOREID,
+        scfg => $ALT_SCFG,
+        cases => [
+            map { format_volname_case_to_volid_case($ALT_STOREID, $ALT_SCFG, $_) }
+                $volname_tests->@*
+        ],
+    },
+];
+
+my sub run_volid_parsing_tests : prototype($) ($volid_tests) {
+    for my $test ($volid_tests->@*) {
+        subtest $test->{description} => sub() {
+            for my $case ($test->{cases}->@*) {
+                my ($storeid, $scfg, $path, $vtype) =
+                    $case->{args}->@{qw(storeid scfg path vtype)};
+
+                my $got_volid_parts =
+                    parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
+                my $got_volid = parse_abs_path_as_volid($storeid, $scfg, $path, $vtype);
+
+                note("Running test case based on:");
+                note($case->{description});
+
+                if (defined($case->{expected})) {
+                    eq_or_diff(
+                        $got_volid_parts,
+                        $case->{expected},
+                        'parse_abs_path_as_volid_parts() returns expected hashref',
+                        { context => 50000 },
+                    );
+
+                    is(
+                        $got_volid,
+                        $case->{expected}->{volid},
+                        'parse_abs_path_as_volid() returns expected volid',
+                    );
+
+                    my $expected_volid_parts = {
+                        volid => $case->{expected}->{volid},
+                        storeid => $case->{args}->{storeid},
+                        volname => $case->{expected}->{volname},
+                    };
+
+                    my $got_volid_parts_from_volid = parse_volid_as_parts($got_volid);
+
+                    eq_or_diff(
+                        $got_volid_parts_from_volid,
+                        $expected_volid_parts,
+                        'parse_volid_as_parts() returns expected hashref from parsed volid',
+                    );
+                } else {
+                    is($got_volid_parts, undef,
+                        'parse_abs_path_as_volid_parts() returns undef');
+                    is($got_volid, undef, 'parse_abs_path_as_volid() returns undef');
+                }
+            }
+        };
+    }
+
+    return;
+}
+
+my sub main() {
+    unified_diff();
+
+    run_volname_parsing_tests($volname_tests);
+    run_volid_parsing_tests($volid_tests);
+
+    done_testing();
+
+    return;
+}
+
+main();
diff --git a/src/PVE/Storage/Common/test/run_tests.pl b/src/PVE/Storage/Common/test/run_tests.pl
new file mode 100755
index 00000000..b21668c0
--- /dev/null
+++ b/src/PVE/Storage/Common/test/run_tests.pl
@@ -0,0 +1,25 @@
+#!/usr/bin/perl
+
+use v5.36;
+
+use TAP::Harness;
+
+my $MAX_JOBS = 4;
+
+my $nproc = eval { int(`nproc`) } || $MAX_JOBS;
+
+my $jobs = $nproc > $MAX_JOBS ? $MAX_JOBS : $nproc;
+
+my sub main() {
+    my $harness = TAP::Harness->new({ verbosity => -1, jobs => $jobs });
+
+    my $res = $harness->runtests(
+        "parser_tests.pl",
+    );
+
+    exit -1 if !$res || $res->{failed} || $res->{parse_errors} || $res->{exit};
+
+    return;
+}
+
+main();
diff --git a/src/PVE/Storage/Makefile b/src/PVE/Storage/Makefile
index a67dc25f..1c8d364b 100644
--- a/src/PVE/Storage/Makefile
+++ b/src/PVE/Storage/Makefile
@@ -19,5 +19,9 @@ SOURCES= \
 .PHONY: install
 install:
 	make -C Common install
 	for i in ${SOURCES}; do install -D -m 0644 $$i ${DESTDIR}${PERLDIR}/PVE/Storage/$$i; done
 	make -C LunCmd install
+
+.PHONY: test
+test:
+	make -C Common test
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 23/51] tree-wide: replace usages of VZTMPL_EXT_RE_1 with parsing functions
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (21 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 22/51] common: test: set up parser testing code, add tests for 'iso' vtype Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 24/51] tree-wide: replace usages of BACKUP_EXT_RE_2 " Max R. Carrara
                   ` (21 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Add support for the 'vztmpl' volume type in
`PVE::Storage::Common::Parse`.

Also add corresponding test cases.

Use the module's parsing functions to replace usages of the
`PVE::Storage::VZTMPL_EXT_RE_1` regex across the repository.

As done with the `ISO_EXT_RE_0` regex, add a FIXME comment to the
`VZTMPL_EXT_RE_1` regex to mark it for removal on the next APIAGE
reset. Its removal will be noted in the next APIVER + APIAGE bump.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/API2/Storage/Status.pm              |   6 +-
 src/PVE/Storage.pm                          |   5 +-
 src/PVE/Storage/Common/Parse.pm             |  54 +++++++++-
 src/PVE/Storage/Common/test/parser_tests.pl | 111 +++++++++++++++++++-
 src/PVE/Storage/Plugin.pm                   |  16 +--
 5 files changed, 178 insertions(+), 14 deletions(-)

diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm
index 66e8e141..80f9f3c4 100644
--- a/src/PVE/API2/Storage/Status.pm
+++ b/src/PVE/API2/Storage/Status.pm
@@ -83,15 +83,17 @@ my sub parse_transferred_file_path_extension : prototype($$) {
         my $ext = $parts->{ext};
         return $ext;
     }
 
     if ($vtype eq 'vztmpl') {
-        if ($path !~ m![^/]+$PVE::Storage::VZTMPL_EXT_RE_1$!) {
+        my $parts = parse_rel_path_as_volname_parts($path, $vtype);
+
+        if (!defined($parts)) {
             raise_param_exc({ filename => "wrong file extension" });
         }
 
-        my $ext = $1;
+        my $ext = $parts->{ext};
         return $ext;
     }
 
     if ($vtype eq 'import') {
         if (
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 474f1ee5..57355ffd 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -121,10 +121,11 @@ PVE::Storage::Plugin->init();
 # CAUTION don't forget to update the digits accordingly after messing with the capture groups
 
 # FIXME: remove this regex on the next APIAGE reset.
 our $ISO_EXT_RE_0 = qr/\.(?:iso|img)/i;
 
+# FIXME: remove this regex on the next APIAGE reset.
 our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2))/i;
 
 our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/;
 
 our $IMPORT_EXT_RE_1 = qr/\.(ova|ovf|qcow2|raw|vmdk)/;
@@ -812,13 +813,11 @@ sub path_to_volume_id {
         if ($vtype eq 'iso') {
             return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
         }
 
         if ($vtype eq 'vztmpl') {
-            return if $filename !~ m!^/([^/]+$VZTMPL_EXT_RE_1)$!;
-            my $name = $1;
-            return ($vtype, "$sid:vztmpl/$name");
+            return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
         }
 
         if ($vtype eq 'backup') {
             return if $filename !~ m!^/([^/]+$BACKUP_EXT_RE_2)$!;
             my $name = $1;
diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
index ab949d99..94dd75f1 100644
--- a/src/PVE/Storage/Common/Parse.pm
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -31,10 +31,18 @@ Parsing functions are categorized by their main purpose / area of application
 and may be further subdivided depending on what kind of data type they are
 primarily related to.
 
 =cut
 
+my @VZTMPL_COMPRESSION_EXTENSIONS = ('gz', 'xz', 'zst', 'bz2');
+
+my sub join_to_re_alternations(@list) {
+    return join('|', map { quotemeta } @list);
+}
+
+my $RE_VZTMPL_COMPRESSION_EXTENSIONS = join_to_re_alternations(@VZTMPL_COMPRESSION_EXTENSIONS);
+
 my $RE_PARENT_DIR = quotemeta('..');
 my $RE_CONTAINS_PARENT_DIR = qr!
     ( ^$RE_PARENT_DIR$ )  #  ..  --> Entire path
     |
     ( ^$RE_PARENT_DIR/ )  #  ../ --> Beginning of path
@@ -48,16 +56,31 @@ my $RE_ISO_FILE_PATH = qr!
     (?<path>
         (?<file> [^/]+ \. (?<ext> (?i: iso|img) ) )
     )
 !xn;
 
+my $RE_VZTMPL_FILE_PATH = qr!
+    (?<path>
+        (?<file>
+            [^/]+
+            \.
+            (?<ext>
+                (?<ext_archive> (?i: tar) )
+                ( \. (?<ext_compression> (?i: $RE_VZTMPL_COMPRESSION_EXTENSIONS)) )?
+            )
+        )
+    )
+!xn;
+
 my $RE_FILE_PATH_FOR_VTYPE = {
     iso => qr/^$RE_ISO_FILE_PATH$/,
+    vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
 };
 
 my $RE_VOLNAME_FOR_VTYPE = {
     iso => qr/^$RE_ISO_FILE_PATH$/,
+    vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
 };
 
 my sub contains_parent_dir($path) {
     return $path =~ $RE_CONTAINS_PARENT_DIR;
 }
@@ -138,10 +161,35 @@ These are:
 
         # The volname, which is "${vtype}/${path}"
         volname => 'iso/custom-debian.iso',
     }
 
+The following volume types have more specific parts in addition to the common
+ones:
+
+=over
+
+=item C<vztmpl>
+
+C<ext-archive>, C<ext-compression> (optional):
+
+    {
+        file => 'archlinux-base_20190924-1_amd64.tar.gz',
+        ext => 'tar.gz',
+
+        # Used to further differentiate 'ext'
+        'ext-archive' => 'tar',
+        'ext-compression' => 'gz',
+
+        'disk-path' => 'archlinux-base_20190924-1_amd64.tar.gz',
+        path => 'archlinux-base_20190924-1_amd64.tar.gz',
+        vtype => 'vztmpl',
+        volname => 'vztmpl/archlinux-base_20190924-1_amd64.tar.gz',
+    }
+
+=back
+
 =cut
 
 =head3 parse_rel_path_as_volname_parts
 
 Parses the given relative file path C<$path> according to the given C<$vtype>,
@@ -157,11 +205,11 @@ into:
         path => 'custom-debian.iso',
         vtype => 'iso',
         volname => 'iso/custom-debian.iso',
     }
 
-See L<VOLUME NAMES> for more details.
+See L<VOLUME NAMES> for more details and special cases.
 
 On failure or when the provided C<$vtype> is not supported or does not exist,
 returns C<undef> in scalar context, and an empty list in list context.
 
 B<NOTE:> This function assumes that C<$path> is already relative to the
@@ -287,10 +335,14 @@ These are:
 
         # Exclusive to volid parsing
         volid => 'local:iso/custom-debian.iso',
     }
 
+The special cases described in L<VOLUME NAMES> also apply for the parser
+functions in this section. In fact, the volid parsing functions here build on
+the volname ones.
+
 =cut
 
 my $RE_VOLID = qr!
     ^
     (?<volid>
diff --git a/src/PVE/Storage/Common/test/parser_tests.pl b/src/PVE/Storage/Common/test/parser_tests.pl
index ea770bb4..687ad6ae 100755
--- a/src/PVE/Storage/Common/test/parser_tests.pl
+++ b/src/PVE/Storage/Common/test/parser_tests.pl
@@ -84,10 +84,117 @@ my $volname_cases_iso_invalid = [
         },
         expected => undef,
     },
 ];
 
+my $volname_cases_vztmpl_valid = [
+    # plain paths
+    {
+        path => 'archlinux-base_20190924-1_amd64.tar',
+        expected => {
+            file => 'archlinux-base_20190924-1_amd64.tar',
+            ext => 'tar',
+            'ext-archive' => 'tar',
+            'disk-path' => 'archlinux-base_20190924-1_amd64.tar',
+            path => 'archlinux-base_20190924-1_amd64.tar',
+            vtype => 'vztmpl',
+            volname => 'vztmpl/archlinux-base_20190924-1_amd64.tar',
+        },
+    },
+    {
+        path => 'archlinux-base_20190924-1_amd64.tar.gz',
+        expected => {
+            file => 'archlinux-base_20190924-1_amd64.tar.gz',
+            ext => 'tar.gz',
+            'ext-archive' => 'tar',
+            'ext-compression' => 'gz',
+            'disk-path' => 'archlinux-base_20190924-1_amd64.tar.gz',
+            path => 'archlinux-base_20190924-1_amd64.tar.gz',
+            vtype => 'vztmpl',
+            volname => 'vztmpl/archlinux-base_20190924-1_amd64.tar.gz',
+        },
+    },
+    {
+        path => 'alpine-3.10-default_20190626_amd64.tar.xz',
+        expected => {
+            file => 'alpine-3.10-default_20190626_amd64.tar.xz',
+            ext => 'tar.xz',
+            'ext-archive' => 'tar',
+            'ext-compression' => 'xz',
+            'disk-path' => 'alpine-3.10-default_20190626_amd64.tar.xz',
+            path => 'alpine-3.10-default_20190626_amd64.tar.xz',
+            vtype => 'vztmpl',
+            volname => 'vztmpl/alpine-3.10-default_20190626_amd64.tar.xz',
+        },
+    },
+    {
+        path => 'debian-10.0-standard_10.0-1_amd64.tar.zst',
+        expected => {
+            file => 'debian-10.0-standard_10.0-1_amd64.tar.zst',
+            ext => 'tar.zst',
+            'ext-archive' => 'tar',
+            'ext-compression' => 'zst',
+            'disk-path' => 'debian-10.0-standard_10.0-1_amd64.tar.zst',
+            path => 'debian-10.0-standard_10.0-1_amd64.tar.zst',
+            vtype => 'vztmpl',
+            volname => 'vztmpl/debian-10.0-standard_10.0-1_amd64.tar.zst',
+        },
+    },
+    {
+        path => 'debian-11.0-standard_11.0-1_amd64.tar.bz2',
+        expected => {
+            file => 'debian-11.0-standard_11.0-1_amd64.tar.bz2',
+            ext => 'tar.bz2',
+            'ext-archive' => 'tar',
+            'ext-compression' => 'bz2',
+            'disk-path' => 'debian-11.0-standard_11.0-1_amd64.tar.bz2',
+            path => 'debian-11.0-standard_11.0-1_amd64.tar.bz2',
+            vtype => 'vztmpl',
+            volname => 'vztmpl/debian-11.0-standard_11.0-1_amd64.tar.bz2',
+        },
+    },
+
+    # case-insensitive file extensions
+    {
+        path => 'ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+        expected => {
+            file => 'ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+            ext => 'TAR.GZ',
+            'ext-archive' => 'TAR',
+            'ext-compression' => 'GZ',
+            'disk-path' => 'ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+            path => 'ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+            vtype => 'vztmpl',
+            volname => 'vztmpl/ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+        },
+    },
+    {
+        path => 'Alpine 3.10 default_20190626_amd64.Tar.xZ',
+        expected => {
+            file => 'Alpine 3.10 default_20190626_amd64.Tar.xZ',
+            ext => 'Tar.xZ',
+            'ext-archive' => 'Tar',
+            'ext-compression' => 'xZ',
+            'disk-path' => 'Alpine 3.10 default_20190626_amd64.Tar.xZ',
+            path => 'Alpine 3.10 default_20190626_amd64.Tar.xZ',
+            vtype => 'vztmpl',
+            volname => 'vztmpl/Alpine 3.10 default_20190626_amd64.Tar.xZ',
+        },
+    },
+];
+
+my $volname_cases_vztmpl_invalid = [
+    {
+        description => "Invalid file extension (vztmpl)",
+        args => {
+            path => 'archlinux-base_20190924-1_amd64.zip',
+            vtype => 'vztmpl',
+        },
+        expected => undef,
+    },
+];
+
 my $volname_cases_newline_invalid = [
     {
         description => "Trailing newline (iso)",
         args => {
             path => "debian.iso\n",
@@ -112,15 +219,15 @@ my $volname_cases_newline_invalid = [
         expected => undef,
     },
 ];
 
 my $cases_valid_all = [
-    $volname_cases_iso_valid,
+    $volname_cases_iso_valid, $volname_cases_vztmpl_valid,
 ];
 
 my $cases_invalid_all = [
-    $volname_cases_iso_invalid, $volname_cases_newline_invalid,
+    $volname_cases_iso_invalid, $volname_cases_vztmpl_invalid, $volname_cases_newline_invalid,
 ];
 
 {
     for my $cases_list ($cases_valid_all->@*) {
         for my $case ($cases_list->@*) {
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index c5aed4ef..3786b6f5 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -903,14 +903,14 @@ sub parse_volname {
         my ($vtype, $volume_path) = $parts->@{qw(vtype path)};
 
         if ($vtype eq 'iso') {
             return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
         }
-    }
 
-    if ($volname =~ m!^vztmpl/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!) {
-        return ('vztmpl', $1, undef, undef, undef, undef, 'raw');
+        if ($vtype eq 'vztmpl') {
+            return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
+        }
     }
 
     if ($volname =~ m!^backup/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!) {
         my $fn = $1;
         if ($fn =~ m/^vzdump-(openvz|lxc|qemu)-(\d+)-.+/) {
@@ -1906,15 +1906,19 @@ my sub get_subdir_files {
                 format => 'iso', # always 'iso' even if we have a file ending in .img
             };
         }
 
         if ($vtype eq 'vztmpl') {
-            return if $filename !~ m!/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!;
+            my $parts = parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
+            return if !defined($parts);
+
+            my ($ext_archive, $ext_compression) = $parts->@{qw(ext-archive ext-compression)};
+            my $format = defined($ext_compression) ? "t$ext_compression" : $ext_archive;
 
             return {
-                volid => "$storeid:vztmpl/$1",
-                format => $2 eq 'tar' ? $2 : "t$2",
+                volid => $parts->{volid},
+                format => $format,
             };
         }
 
         if ($vtype eq 'backup') {
             return if $filename !~ m!/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!;
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 24/51] tree-wide: replace usages of BACKUP_EXT_RE_2 with parsing functions
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (22 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 23/51] tree-wide: replace usages of VZTMPL_EXT_RE_1 with parsing functions Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 25/51] tree-wide: replace usages of inline regexes for snippets with parsers Max R. Carrara
                   ` (20 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Add support for the 'backup' volume type in
`PVE::Storage::Common::Parse`.

Also add corresponding test cases.

Use the module's parsing functions to replace usages of the
`PVE::Storage::BACKUP_EXT_RE_2` regex across the repository.

Note that the new regex for parsing 'backup' vtype file paths is also
able to optionally extract the VMID and the type of guest from the
file name. This makes it possible to further simplify some parts of
the original logic.

The VMID is only parsed for backup volumes that begin with e.g.
`vzdump-qemu-1337-` etc. This means that the extra regex checks in
`parse_volname()` and `get_subdir_files()` are not necessary anymore,
since the new parsing function handles this out of the box now.

Most importantly, the VMID has to be *valid* now, i.e. it must
actually be in the VMID range. The original regexes matched any number
of digits, including something like "0000".

As done with the `ISO_EXT_RE_0` and `VZTMPL_EXT_RE_1` regexes, add a
FIXME comment to the `BACKUP_EXT_RE_2` regex to mark its removal on
the next APIAGE reset. Its removal will be noted in the next APIVER +
APIAGE bump.

Additionally, the `COMPRESSOR_RE` constant in `PVE::Storage::Plugin`
is only used by `BACKUP_EXT_RE_2` now. Together with its related
`KNOWN_COMPRESSION_FORMATS` constant, it should be phased out and
replaced eventually. Add a TODO comment for that and mark
`COMPRESSOR_RE` for removal as well.

In the meantime, keep track of the compression formats for specific
vtypes in the `::Common::Parse` module until we move them to a more
adequate place.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage.pm                          |  76 ++++--
 src/PVE/Storage/Common/Parse.pm             |  86 ++++++
 src/PVE/Storage/Common/test/parser_tests.pl | 280 +++++++++++++++++++-
 src/PVE/Storage/Plugin.pm                   |  42 +--
 4 files changed, 444 insertions(+), 40 deletions(-)

diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 57355ffd..bdd8e445 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -27,10 +27,11 @@ use PVE::RESTEnvironment qw(log_warn);
 
 use PVE::Storage::Common qw(
     plugin_get_default_vtype_subdirs
 );
 use PVE::Storage::Common::Parse qw(
+    parse_volname_as_parts
     parse_abs_path_as_volid
 );
 use PVE::Storage::Plugin;
 use PVE::Storage::DirPlugin;
 use PVE::Storage::LVMPlugin;
@@ -124,10 +125,11 @@ PVE::Storage::Plugin->init();
 our $ISO_EXT_RE_0 = qr/\.(?:iso|img)/i;
 
 # FIXME: remove this regex on the next APIAGE reset.
 our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2))/i;
 
+# FIXME: remove this regex on the next APIAGE reset.
 our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/;
 
 our $IMPORT_EXT_RE_1 = qr/\.(ova|ovf|qcow2|raw|vmdk)/;
 
 our $UPLOAD_IMPORT_EXT_RE_1 = qr/\.(ova|qcow2|raw|vmdk)/;
@@ -817,13 +819,11 @@ sub path_to_volume_id {
         if ($vtype eq 'vztmpl') {
             return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
         }
 
         if ($vtype eq 'backup') {
-            return if $filename !~ m!^/([^/]+$BACKUP_EXT_RE_2)$!;
-            my $name = $1;
-            return ($vtype, "$sid:backup/$name");
+            return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
         }
 
         if ($vtype eq 'snippets') {
             return if $filename !~ m!^/([^/]+)$!;
             my $name = $1;
@@ -1926,33 +1926,69 @@ sub protection_file_path {
     return "${path}.protected";
 }
 
 sub archive_info {
     my ($archive) = shift;
-    my $info;
 
-    my $volid = basename($archive);
-    if ($volid =~ /^(vzdump-(lxc|openvz|qemu)-.+$BACKUP_EXT_RE_2)$/) {
-        my $filename = "$1"; # untaint
-        my ($type, $extension, $comp) = ($2, $3, $4);
-        (my $format = $extension) =~ s/\..*//;
-        $info = decompressor_info($format, $comp);
-        $info->{filename} = $filename;
-        $info->{type} = $type;
+    # $archive can be either an absolute or relative path, so construct a valid
+    # volname for further parsing first
+    my $filename = basename($archive);
+    my $volname = "backup/$filename";
+
+    my $info = eval {
+        my $volname_parts = parse_volname_as_parts($volname);
+
+        die "'$volname' is not a valid volume name\n"
+            if !defined($volname_parts);
+
+        my $type = $volname_parts->{type};
+
+        # Best-effort parsing, because `prune_backups()` doesn't need a VMID
+        if (!defined($type) && $filename =~ m/^vzdump-(?<type> openvz|lxc|qemu)/xn) {
+            $type = $+{type};
+        }
+
+        die "'$volname' does not contain a valid backup type\n"
+            if !defined($type);
+
+        my ($format, $compression) = $volname_parts->@{qw(ext-archive ext-compression)};
+
+        # Conform to what decompressor_info() expects
+        $compression = undef if $format eq 'tgz';
+        my $decomp_info = decompressor_info($format, $compression);
+
+        my $info = {
+            $decomp_info->%*, # shallow copy
+            filename => $volname_parts->{file},
+            type => $type,
+        };
+
+        if (defined($volname_parts->{timestamp})) {
+            my ($filename_no_ext, undef) = split(qr!\.!, $volname_parts->{file}, 2);
+            $info->{logfilename} = $filename_no_ext . PVE::Storage::Plugin::LOG_EXT;
+
+            $info->{notesfilename} = $volname_parts->{file} . PVE::Storage::Plugin::NOTES_EXT;
+
+            $info->{vmid} = int($volname_parts->{vmid});
+
+            $info->{ctime} = timelocal(
+                $volname_parts->{second},
+                $volname_parts->{minute},
+                $volname_parts->{hour},
+                $volname_parts->{day},
+                $volname_parts->{month} - 1,
+                $volname_parts->{year},
+            );
 
-        if ($volid =~
-            /^(vzdump-${type}-([1-9][0-9]{2,8})-(\d{4})_(\d{2})_(\d{2})-(\d{2})_(\d{2})_(\d{2}))\.${extension}$/
-        ) {
-            $info->{logfilename} = "$1" . PVE::Storage::Plugin::LOG_EXT;
-            $info->{notesfilename} = "$filename" . PVE::Storage::Plugin::NOTES_EXT;
-            $info->{vmid} = int($2);
-            $info->{ctime} = timelocal($8, $7, $6, $5, $4 - 1, $3);
             $info->{is_std_name} = 1;
         } else {
             $info->{is_std_name} = 0;
         }
-    } else {
+
+        $info;
+    };
+    if (my $err = $@) {
         die "ERROR: couldn't determine archive info from '$archive'\n";
     }
 
     return $info;
 }
diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
index 94dd75f1..73f4e2bc 100644
--- a/src/PVE/Storage/Common/Parse.pm
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -33,27 +33,36 @@ primarily related to.
 
 =cut
 
 my @VZTMPL_COMPRESSION_EXTENSIONS = ('gz', 'xz', 'zst', 'bz2');
 
+my @BACKUP_COMPRESSION_EXTENSIONS = ('gz', 'lzo', 'zst', 'bz2');
+
 my sub join_to_re_alternations(@list) {
     return join('|', map { quotemeta } @list);
 }
 
 my $RE_VZTMPL_COMPRESSION_EXTENSIONS = join_to_re_alternations(@VZTMPL_COMPRESSION_EXTENSIONS);
 
+my $RE_BACKUP_COMPRESSION_EXTENSIONS = join_to_re_alternations(@BACKUP_COMPRESSION_EXTENSIONS);
+
 my $RE_PARENT_DIR = quotemeta('..');
 my $RE_CONTAINS_PARENT_DIR = qr!
     ( ^$RE_PARENT_DIR$ )  #  ..  --> Entire path
     |
     ( ^$RE_PARENT_DIR/ )  #  ../ --> Beginning of path
     |
     ( /$RE_PARENT_DIR/ )  # /../ --> Between two path components
     |
     ( /$RE_PARENT_DIR$ )  # /..  --> End of path
 !xn;
 
+# A vmid is an integer between 100 and 999_999_999
+# See: https://git.proxmox.com/?p=pve-common.git;a=blob;f=src/PVE/JSONSchema.pm;h=17e7126a6c42f8326a1da890680af10b6106d906;hb=refs/heads/master#l62
+# See also: https://git.proxmox.com/?p=pve-common.git;a=blob;f=src/PVE/JSONSchema.pm;h=17e7126a6c42f8326a1da890680af10b6106d906;hb=refs/heads/master#l304
+my $RE_VMID = qr![1-9][0-9]{2,8}!;
+
 my $RE_ISO_FILE_PATH = qr!
     (?<path>
         (?<file> [^/]+ \. (?<ext> (?i: iso|img) ) )
     )
 !xn;
@@ -69,18 +78,59 @@ my $RE_VZTMPL_FILE_PATH = qr!
             )
         )
     )
 !xn;
 
+my $RE_GUEST_VZDUMP_BACKUP_FILE_TIMESTAMP = qr!
+    (?<timestamp>
+        (?<year> \d{4})
+        _ (?<month> \d{2})
+        _ (?<day> \d{2})
+        - (?<hour> \d{2})
+        _ (?<minute> \d{2})
+        _ (?<second> \d{2})
+    )
+!xn;
+
+my $RE_GUEST_VZDUMP_BACKUP_FILE_NAME = qr!
+    vzdump
+    - (?<type> openvz|lxc|qemu)
+    - (?<vmid> $RE_VMID)
+    - ( $RE_GUEST_VZDUMP_BACKUP_FILE_TIMESTAMP | [^/]+ )
+!xn;
+
+my $RE_BACKUP_FILE_PATH = qr!
+    (?<path>
+        (?<file>
+            (
+                ($RE_GUEST_VZDUMP_BACKUP_FILE_NAME)
+                |
+                ([^/]+)
+            )
+            \.
+            (?<ext>
+                (?<ext_archive> tgz)
+                |
+                (
+                    (?<ext_archive> tar|vma)
+                    ( \. (?<ext_compression> $RE_BACKUP_COMPRESSION_EXTENSIONS) )?
+                )
+            )
+        )
+    )
+!xn;
+
 my $RE_FILE_PATH_FOR_VTYPE = {
     iso => qr/^$RE_ISO_FILE_PATH$/,
     vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
+    backup => qr/^$RE_BACKUP_FILE_PATH$/,
 };
 
 my $RE_VOLNAME_FOR_VTYPE = {
     iso => qr/^$RE_ISO_FILE_PATH$/,
     vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
+    backup => qr/^$RE_BACKUP_FILE_PATH$/,
 };
 
 my sub contains_parent_dir($path) {
     return $path =~ $RE_CONTAINS_PARENT_DIR;
 }
@@ -184,10 +234,46 @@ C<ext-archive>, C<ext-compression> (optional):
         path => 'archlinux-base_20190924-1_amd64.tar.gz',
         vtype => 'vztmpl',
         volname => 'vztmpl/archlinux-base_20190924-1_amd64.tar.gz',
     }
 
+=item C<backup>
+
+C<type> (optional), C<vmid> (optional), which are included in backups created
+by Proxmox VE. These backups may also contain a C<timestamp> that consists of
+C<year>, C<month>, C<day>, C<hour>, C<minute>, and C<second>.
+
+To further differentiate the file extension, there are also C<ext-archive>,
+C<ext-compression> (optional).
+
+    {
+        file => 'vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
+
+        # Used to further differentiate 'file'
+        type => 'qemu',
+        vmid => '16110',
+
+        timestamp => '2020_03_30-21_12_45',
+        year => '2020',
+        month => '03',
+        day => '30',
+        hour => '21',
+        minute => '12',
+        second => '45',
+
+        ext => 'vma.lzo',
+
+        # Used to further differentiate 'ext'
+        'ext-archive' => 'vma',
+        'ext-compression' => 'lzo',
+
+        'disk-path' => 'vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
+        path => 'vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
+        vtype => 'backup',
+        volname => 'backup/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
+    }
+
 =back
 
 =cut
 
 =head3 parse_rel_path_as_volname_parts
diff --git a/src/PVE/Storage/Common/test/parser_tests.pl b/src/PVE/Storage/Common/test/parser_tests.pl
index 687ad6ae..bfb17bbc 100755
--- a/src/PVE/Storage/Common/test/parser_tests.pl
+++ b/src/PVE/Storage/Common/test/parser_tests.pl
@@ -191,10 +191,283 @@ my $volname_cases_vztmpl_invalid = [
         },
         expected => undef,
     },
 ];
 
+my $volname_cases_backup_valid = [
+    {
+        path => 'vzdump-qemu-16110-2020_03_30-21_13_55.vma',
+        expected => {
+            type => 'qemu',
+            vmid => '16110',
+            file => 'vzdump-qemu-16110-2020_03_30-21_13_55.vma',
+            ext => 'vma',
+            'ext-archive' => 'vma',
+            'disk-path' => 'vzdump-qemu-16110-2020_03_30-21_13_55.vma',
+            path => 'vzdump-qemu-16110-2020_03_30-21_13_55.vma',
+            vtype => 'backup',
+            volname => 'backup/vzdump-qemu-16110-2020_03_30-21_13_55.vma',
+
+            timestamp => '2020_03_30-21_13_55',
+            year => '2020',
+            month => '03',
+            day => '30',
+            hour => '21',
+            minute => '13',
+            second => '55',
+        },
+    },
+    {
+        path => 'vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz',
+        expected => {
+            type => 'qemu',
+            vmid => '16110',
+            file => 'vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz',
+            ext => 'vma.gz',
+            'ext-archive' => 'vma',
+            'ext-compression' => 'gz',
+            'disk-path' => 'vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz',
+            path => 'vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz',
+            vtype => 'backup',
+            volname => 'backup/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz',
+
+            timestamp => '2020_03_30-21_11_40',
+            year => '2020',
+            month => '03',
+            day => '30',
+            hour => '21',
+            minute => '11',
+            second => '40',
+        },
+    },
+    {
+        path => 'vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
+        expected => {
+            type => 'qemu',
+            vmid => '16110',
+            file => 'vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
+            ext => 'vma.lzo',
+            'ext-archive' => 'vma',
+            'ext-compression' => 'lzo',
+            'disk-path' => 'vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
+            path => 'vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
+            vtype => 'backup',
+            volname => 'backup/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
+
+            timestamp => '2020_03_30-21_12_45',
+            year => '2020',
+            month => '03',
+            day => '30',
+            hour => '21',
+            minute => '12',
+            second => '45',
+        },
+    },
+    {
+        path => 'vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst',
+        expected => {
+            type => 'qemu',
+            vmid => '16110',
+            file => 'vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst',
+            ext => 'vma.zst',
+            'ext-archive' => 'vma',
+            'ext-compression' => 'zst',
+            'disk-path' => 'vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst',
+            path => 'vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst',
+            vtype => 'backup',
+            volname => 'backup/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst',
+
+            timestamp => '2020_03_30-21_13_55',
+            year => '2020',
+            month => '03',
+            day => '30',
+            hour => '21',
+            minute => '13',
+            second => '55',
+        },
+    },
+    {
+        path => 'vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo',
+        expected => {
+            type => 'lxc',
+            vmid => '16112',
+            file => 'vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo',
+            ext => 'tar.lzo',
+            'ext-archive' => 'tar',
+            'ext-compression' => 'lzo',
+            'disk-path' => 'vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo',
+            path => 'vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo',
+            vtype => 'backup',
+            volname => 'backup/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo',
+
+            timestamp => '2020_03_30-21_39_30',
+            year => '2020',
+            month => '03',
+            day => '30',
+            hour => '21',
+            minute => '39',
+            second => '30',
+        },
+    },
+    {
+        path => 'vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz',
+        expected => {
+            type => 'lxc',
+            vmid => '16112',
+            file => 'vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz',
+            ext => 'tar.gz',
+            'ext-archive' => 'tar',
+            'ext-compression' => 'gz',
+            'disk-path' => 'vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz',
+            path => 'vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz',
+            vtype => 'backup',
+            volname => 'backup/vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz',
+
+            timestamp => '2020_03_30-21_49_30',
+            year => '2020',
+            month => '03',
+            day => '30',
+            hour => '21',
+            minute => '49',
+            second => '30',
+        },
+    },
+    {
+        path => 'vzdump-lxc-16112-2020_03_30-21_49_30.tar.zst',
+        expected => {
+            type => 'lxc',
+            vmid => '16112',
+            file => 'vzdump-lxc-16112-2020_03_30-21_49_30.tar.zst',
+            ext => 'tar.zst',
+            'ext-archive' => 'tar',
+            'ext-compression' => 'zst',
+            'disk-path' => 'vzdump-lxc-16112-2020_03_30-21_49_30.tar.zst',
+            path => 'vzdump-lxc-16112-2020_03_30-21_49_30.tar.zst',
+            vtype => 'backup',
+            volname => 'backup/vzdump-lxc-16112-2020_03_30-21_49_30.tar.zst',
+
+            timestamp => '2020_03_30-21_49_30',
+            year => '2020',
+            month => '03',
+            day => '30',
+            hour => '21',
+            minute => '49',
+            second => '30',
+        },
+    },
+    {
+        path => 'vzdump-lxc-16112-2020_03_30-21_59_30.tgz',
+        expected => {
+            type => 'lxc',
+            vmid => '16112',
+            file => 'vzdump-lxc-16112-2020_03_30-21_59_30.tgz',
+            ext => 'tgz',
+            'ext-archive' => 'tgz',
+            'disk-path' => 'vzdump-lxc-16112-2020_03_30-21_59_30.tgz',
+            path => 'vzdump-lxc-16112-2020_03_30-21_59_30.tgz',
+            vtype => 'backup',
+            volname => 'backup/vzdump-lxc-16112-2020_03_30-21_59_30.tgz',
+
+            timestamp => '2020_03_30-21_59_30',
+            year => '2020',
+            month => '03',
+            day => '30',
+            hour => '21',
+            minute => '59',
+            second => '30',
+        },
+    },
+    {
+        path => 'vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2',
+        expected => {
+            type => 'openvz',
+            vmid => '16112',
+            file => 'vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2',
+            ext => 'tar.bz2',
+            'ext-archive' => 'tar',
+            'ext-compression' => 'bz2',
+            'disk-path' => 'vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2',
+            path => 'vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2',
+            vtype => 'backup',
+            volname => 'backup/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2',
+
+            timestamp => '2020_03_30-21_39_30',
+            year => '2020',
+            month => '03',
+            day => '30',
+            hour => '21',
+            minute => '39',
+            second => '30',
+        },
+    },
+    {
+        path => 'vzdump-lxc-16110-3070_01_01-00_00_00.tgz',
+        expected => {
+            type => 'lxc',
+            vmid => '16110',
+            file => 'vzdump-lxc-16110-3070_01_01-00_00_00.tgz',
+            ext => 'tgz',
+            'ext-archive' => 'tgz',
+            'disk-path' => 'vzdump-lxc-16110-3070_01_01-00_00_00.tgz',
+            path => 'vzdump-lxc-16110-3070_01_01-00_00_00.tgz',
+            vtype => 'backup',
+            volname => 'backup/vzdump-lxc-16110-3070_01_01-00_00_00.tgz',
+
+            timestamp => '3070_01_01-00_00_00',
+            year => '3070',
+            month => '01',
+            day => '01',
+            hour => '00',
+            minute => '00',
+            second => '00',
+        },
+    },
+];
+
+my $volname_cases_backup_invalid = [
+    {
+        description => "Invalid file extension (1) (backup)",
+        args => {
+            path => 'vzdump-openvz-16112-2020_03_30-21_39_30.tar.zip',
+            vtype => 'backup',
+        },
+        expected => undef,
+    },
+    {
+        description => "Invalid file extension (2) (backup)",
+        args => {
+            path => 'vzdump-openvz-16112-2020_03_30-21_39_30.zip.gz',
+            vtype => 'backup',
+        },
+        expected => undef,
+    },
+    {
+        description => "Invalid file extension (3) (backup)",
+        args => {
+            path => 'vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz',
+            vtype => 'backup',
+        },
+        expected => undef,
+    },
+    {
+        description => "Uppercase letter in file extension (1) (backup)",
+        args => {
+            path => 'vzdump-qemu-16110-2020_03_30-21_12_40.Tar.gz',
+            vtype => 'backup',
+        },
+        expected => undef,
+    },
+    {
+        description => "Uppercase letter in file extension (2) (backup)",
+        args => {
+            path => 'vzdump-qemu-16110-2020_03_30-21_12_40.tar.Gz',
+            vtype => 'backup',
+        },
+        expected => undef,
+    },
+];
+
 my $volname_cases_newline_invalid = [
     {
         description => "Trailing newline (iso)",
         args => {
             path => "debian.iso\n",
@@ -219,15 +492,18 @@ my $volname_cases_newline_invalid = [
         expected => undef,
     },
 ];
 
 my $cases_valid_all = [
-    $volname_cases_iso_valid, $volname_cases_vztmpl_valid,
+    $volname_cases_iso_valid, $volname_cases_vztmpl_valid, $volname_cases_backup_valid,
 ];
 
 my $cases_invalid_all = [
-    $volname_cases_iso_invalid, $volname_cases_vztmpl_invalid, $volname_cases_newline_invalid,
+    $volname_cases_iso_invalid,
+    $volname_cases_vztmpl_invalid,
+    $volname_cases_backup_invalid,
+    $volname_cases_newline_invalid,
 ];
 
 {
     for my $cases_list ($cases_valid_all->@*) {
         for my $case ($cases_list->@*) {
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 3786b6f5..e3d7367d 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -26,11 +26,13 @@ use PVE::Storage::Common::Parse qw(
 
 use JSON;
 
 use base qw(PVE::SectionConfig);
 
+# TODO: Phase out these two constants
 use constant KNOWN_COMPRESSION_FORMATS => ('gz', 'lzo', 'zst', 'bz2');
+# FIXME: remove this regex on the next APIAGE reset.
 use constant COMPRESSOR_RE => join('|', KNOWN_COMPRESSION_FORMATS);
 
 use constant LOG_EXT => ".log";
 use constant NOTES_EXT => ".notes";
 
@@ -907,18 +909,14 @@ sub parse_volname {
         }
 
         if ($vtype eq 'vztmpl') {
             return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
         }
-    }
 
-    if ($volname =~ m!^backup/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!) {
-        my $fn = $1;
-        if ($fn =~ m/^vzdump-(openvz|lxc|qemu)-(\d+)-.+/) {
-            return ('backup', $fn, $2, undef, undef, undef, 'raw');
+        if ($vtype eq 'backup') {
+            return ($vtype, $volume_path, $parts->{vmid}, undef, undef, undef, 'raw');
         }
-        return ('backup', $fn, undef, undef, undef, undef, 'raw');
     }
 
     if ($volname =~ m!^snippets/([^/]+)$!) {
         return ('snippets', $1, undef, undef, undef, undef, 'raw');
     }
@@ -1919,41 +1917,49 @@ my sub get_subdir_files {
                 format => $format,
             };
         }
 
         if ($vtype eq 'backup') {
-            return if $filename !~ m!/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!;
+            my $parts = parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
+            return if !defined($parts);
 
-            my $original = $path;
-            my $format = $2;
-            $filename = $1;
+            my $format = $parts->{ext};
+            my $volume_path = $parts->{path};
 
-            # Attribute archives to a guest like parse_volname() does, which the access checks use.
-            next if defined($vmid) && $filename !~ m/^vzdump-(?:openvz|lxc|qemu)-\Q$vmid\E-/;
+            # Check if parsed VMID matched provided VMID in order to avoid
+            # false positives (VMID in parent directory name)
+            my $parsed_vmid = $parts->{vmid};
+            if (defined($vmid)) {
+                if (defined($parsed_vmid)) {
+                    return if $vmid ne $parsed_vmid;
+                } else {
+                    return;
+                }
+            }
 
             my $info = {
-                volid => "$storeid:backup/$filename",
+                volid => $parts->{volid},
                 format => $format,
             };
 
-            my $archive_info = eval { PVE::Storage::archive_info($filename) } // {};
+            my $archive_info = eval { PVE::Storage::archive_info($volume_path) } // {};
 
             $info->{ctime} = $archive_info->{ctime} if defined($archive_info->{ctime});
             $info->{subtype} = $archive_info->{type} // 'unknown';
 
-            if (defined($vmid) || $filename =~ m!\-([1-9][0-9]{2,8})\-[^/]+\.${format}$!) {
-                $info->{vmid} = $vmid // $1;
+            if (defined($vmid) || defined($parsed_vmid)) {
+                $info->{vmid} = $vmid // $parsed_vmid;
             }
 
-            my $notes_filename = $original . NOTES_EXT;
+            my $notes_filename = $path . NOTES_EXT;
             if (-f $notes_filename) {
                 my $notes = PVE::Tools::file_read_firstline($notes_filename);
                 $info->{notes} = eval { decode('UTF-8', $notes, 1) } // $notes
                     if defined($notes);
             }
 
-            $info->{protected} = 1 if -e PVE::Storage::protection_file_path($original);
+            $info->{protected} = 1 if -e PVE::Storage::protection_file_path($path);
 
             return $info;
         }
 
         if ($vtype eq 'snippets') {
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 25/51] tree-wide: replace usages of inline regexes for snippets with parsers
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (23 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 24/51] tree-wide: replace usages of BACKUP_EXT_RE_2 " Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 26/51] tree-wide: partially replace usages of regexes for 'import' vtype Max R. Carrara
                   ` (19 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Add support for the 'snippets' volume type in
`PVE::Storage::Common::Parse`.

Also add corresponding test cases.

Use the module's parsing functions to replace parsing a snippet file's
name via regexes or the `basename()` function.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage.pm                          |  4 +--
 src/PVE/Storage/Common/Parse.pm             |  8 ++++++
 src/PVE/Storage/Common/test/parser_tests.pl | 29 ++++++++++++++++++++-
 src/PVE/Storage/Plugin.pm                   | 11 +++++---
 4 files changed, 44 insertions(+), 8 deletions(-)

diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index bdd8e445..5d8413a9 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -823,13 +823,11 @@ sub path_to_volume_id {
         if ($vtype eq 'backup') {
             return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
         }
 
         if ($vtype eq 'snippets') {
-            return if $filename !~ m!^/([^/]+)$!;
-            my $name = $1;
-            return ($vtype, "$sid:snippets/$name");
+            return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
         }
 
         if ($vtype eq 'import') {
             return if $filename !~ m!^/(${SAFE_CHAR_CLASS_RE}+${IMPORT_EXT_RE_1})$!;
             my $name = $1;
diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
index 73f4e2bc..ece0dac9 100644
--- a/src/PVE/Storage/Common/Parse.pm
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -117,20 +117,28 @@ my $RE_BACKUP_FILE_PATH = qr!
             )
         )
     )
 !xn;
 
+my $RE_SNIPPETS_FILE_PATH = qr!
+    (?<path>
+        (?<file> [^/]+ )
+    )
+!xn;
+
 my $RE_FILE_PATH_FOR_VTYPE = {
     iso => qr/^$RE_ISO_FILE_PATH$/,
     vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
     backup => qr/^$RE_BACKUP_FILE_PATH$/,
+    snippets => qr/^$RE_SNIPPETS_FILE_PATH$/,
 };
 
 my $RE_VOLNAME_FOR_VTYPE = {
     iso => qr/^$RE_ISO_FILE_PATH$/,
     vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
     backup => qr/^$RE_BACKUP_FILE_PATH$/,
+    snippets => qr/^$RE_SNIPPETS_FILE_PATH$/,
 };
 
 my sub contains_parent_dir($path) {
     return $path =~ $RE_CONTAINS_PARENT_DIR;
 }
diff --git a/src/PVE/Storage/Common/test/parser_tests.pl b/src/PVE/Storage/Common/test/parser_tests.pl
index bfb17bbc..812a466f 100755
--- a/src/PVE/Storage/Common/test/parser_tests.pl
+++ b/src/PVE/Storage/Common/test/parser_tests.pl
@@ -464,10 +464,34 @@ my $volname_cases_backup_invalid = [
         },
         expected => undef,
     },
 ];
 
+my $volname_cases_snippets_valid = [
+    # plain files
+    {
+        path => 'hookscript.pl',
+        expected => {
+            file => 'hookscript.pl',
+            'disk-path' => 'hookscript.pl',
+            path => 'hookscript.pl',
+            vtype => 'snippets',
+            volname => 'snippets/hookscript.pl',
+        },
+    },
+    {
+        path => 'userconfig.yaml',
+        expected => {
+            file => 'userconfig.yaml',
+            'disk-path' => 'userconfig.yaml',
+            path => 'userconfig.yaml',
+            vtype => 'snippets',
+            volname => 'snippets/userconfig.yaml',
+        },
+    },
+];
+
 my $volname_cases_newline_invalid = [
     {
         description => "Trailing newline (iso)",
         args => {
             path => "debian.iso\n",
@@ -492,11 +516,14 @@ my $volname_cases_newline_invalid = [
         expected => undef,
     },
 ];
 
 my $cases_valid_all = [
-    $volname_cases_iso_valid, $volname_cases_vztmpl_valid, $volname_cases_backup_valid,
+    $volname_cases_iso_valid,
+    $volname_cases_vztmpl_valid,
+    $volname_cases_backup_valid,
+    $volname_cases_snippets_valid,
 ];
 
 my $cases_invalid_all = [
     $volname_cases_iso_invalid,
     $volname_cases_vztmpl_invalid,
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index e3d7367d..8b4f9dde 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -913,14 +913,14 @@ sub parse_volname {
         }
 
         if ($vtype eq 'backup') {
             return ($vtype, $volume_path, $parts->{vmid}, undef, undef, undef, 'raw');
         }
-    }
 
-    if ($volname =~ m!^snippets/([^/]+)$!) {
-        return ('snippets', $1, undef, undef, undef, undef, 'raw');
+        if ($vtype eq 'snippets') {
+            return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
+        }
     }
 
     if ($volname =~
         m!^import/(${PVE::Storage::SAFE_CHAR_WITH_WHITESPACE_CLASS_RE}+\.ova\/${PVE::Storage::OVA_CONTENT_RE_1})$!
     ) {
@@ -1961,12 +1961,15 @@ my sub get_subdir_files {
 
             return $info;
         }
 
         if ($vtype eq 'snippets') {
+            my $parts = parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
+            return if !defined($parts);
+
             return {
-                volid => "$storeid:snippets/" . basename($filename),
+                volid => $parts->{volid},
                 format => 'snippet',
             };
         }
 
         if ($vtype eq 'import') {
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 26/51] tree-wide: partially replace usages of regexes for 'import' vtype
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (24 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 25/51] tree-wide: replace usages of inline regexes for snippets with parsers Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:38 ` [PATCH pve-storage v3 27/51] tree-wide: replace remaining " Max R. Carrara
                   ` (18 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Add partial support for the 'import' volume type in
`PVE::Storage::Common::Parse`. What remains unsupported right now is
parsing volume names for that type, as those require a little more
care.

Replace most usages of the `PVE::Storage::IMPORT_EXT_RE_1` regex with
parsing functions from `::Common::Parse`.

Replace the one remaining spot where we use the
`PVE::Storage::UPLOAD_IMPORT_EXT_RE_1` regex with a parsing function
as well. Add a FIXME comment for its removal on the next APIAGE reset.
Its removal will be noted in the next APIVER + APIAGE bump.

It is important to note that the `UPLOAD_IMPORT_EXT_RE_1` regex only
exists to handle the special case of excluding .ovf files from the
upload / download_url API methods. Therefore, instead of adding a
separate parser (or a flag etc.) to handle this case, just parse the
path of the up-/downloaded file regardless. Then, raise a parameter
exception when the `ovf` file extension is encountered.

Also, since the `PVE::Storage::SAFE_CHAR_CLASS_RE` regex is now
unused, migrate it to `::Common::Parse` and allow it to be obtained by
using a new helper subroutine. Add a corresponding FIXME comment here
too. This will also be added to the ApiChangeLog later.

The main reason for moving `SAFE_CHAR_CLASS_RE` and making it private
is to discourage "shotgun parsing" throughout the repo and keeping
most of our parsing code in `::Common::Parse` -- just like the other
regexes that are removed.

Additionally, we avoid duplicating the regex that way. An alternative
would have been to import it from `PVE::Storage` and use it in
`::Common::Parse`, but that would have led to cyclic imports. That
approach would not have solved the shotgun parsing problem, though.

However, since that particular regex is still important for
third-party plugin authors (for backup provider plugins in particular)
and perhaps for some isolated parsing subs that don't *really* fit
into `::Common::Parse` in the future, allowing it to be obtained
through a subroutine makes its usage much more explicit. As a side
effect, putting the regex behind a subroutine prevents it from being
altered.

Finally, add a bunch of test cases that target the `import` volume
type for the `list_volumes()` plugin API method. For all of these
tests, sort the expected and resulting list items by their volume ID
in order to make the output comparison deterministic. Iterate over the
declared content / volume types of the mocked storage config instead
of using a fixed list of vtypes as well.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/API2/Storage/Status.pm        |  13 +-
 src/PVE/BackupProvider/Plugin/Base.pm |   6 +-
 src/PVE/Storage.pm                    |  15 ++-
 src/PVE/Storage/Common/Parse.pm       |  53 ++++++++
 src/PVE/Storage/Plugin.pm             |   9 +-
 src/test/list_volumes_test.pm         | 180 +++++++++++++++++++++++++-
 6 files changed, 256 insertions(+), 20 deletions(-)

diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm
index 80f9f3c4..15ba1fe3 100644
--- a/src/PVE/API2/Storage/Status.pm
+++ b/src/PVE/API2/Storage/Status.pm
@@ -94,17 +94,22 @@ my sub parse_transferred_file_path_extension : prototype($$) {
         my $ext = $parts->{ext};
         return $ext;
     }
 
     if ($vtype eq 'import') {
-        if (
-            $path !~ m!${PVE::Storage::SAFE_CHAR_CLASS_RE}+$PVE::Storage::UPLOAD_IMPORT_EXT_RE_1$!
-        ) {
+        my $parts = parse_rel_path_as_volname_parts($path, $vtype);
+
+        if (!defined($parts)) {
             raise_param_exc({ filename => "invalid filename or wrong extension" });
         }
 
-        my $ext = $1;
+        my $ext = $parts->{ext};
+
+        if ($ext eq 'ovf') {
+            raise_param_exc({ filename => "wrong file extension" });
+        }
+
         return $ext;
     }
 
     die "upload / download: failed to parse '$path' - unhandled content type '$vtype'\n";
 }
diff --git a/src/PVE/BackupProvider/Plugin/Base.pm b/src/PVE/BackupProvider/Plugin/Base.pm
index 81b764a6..c8ef16aa 100644
--- a/src/PVE/BackupProvider/Plugin/Base.pm
+++ b/src/PVE/BackupProvider/Plugin/Base.pm
@@ -245,13 +245,13 @@ Called before the backup of the given guest is made. The archive name is
 determined for the backup task and returned to the caller via a hash reference:
 
     my $res = $backup_provider->backup_init($vmid, $vmtype, $start_time);
     my $archive_name = $res->{'archive-name'};
 
-The archive name must contain only characters from the
-C<$PVE::Storage::SAFE_CHAR_CLASS_RE> character class as well as forward slash
-C</> and colon C<:>.
+The archive name must contain only characters from the character class returned
+by C<L<< get_re_safe_char_class()|PVE::Storage::Common::Parse/"get_re_safe_char_class" >>>
+as well as forward slash C</> and colon C<:>.
 
 Use C<$self> to remember it for the C<backup_container()> or C<backup_vm()>
 method that will be called later.
 
 Parameters:
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 5d8413a9..d17482e6 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -21,10 +21,17 @@ use PVE::DataCenterConfig;
 use PVE::Exception qw(raise_param_exc raise);
 use PVE::JSONSchema;
 use PVE::INotify;
 use PVE::RPCEnvironment;
 use PVE::SSHInfo;
+use PVE::Storage::Common qw(
+    plugin_get_vtype_subdir
+);
+use PVE::Storage::Common::Parse qw(
+    get_re_safe_char_class
+    parse_abs_path_as_volid
+);
 use PVE::RESTEnvironment qw(log_warn);
 
 use PVE::Storage::Common qw(
     plugin_get_default_vtype_subdirs
 );
@@ -130,13 +137,15 @@ our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2))/i;
 # FIXME: remove this regex on the next APIAGE reset.
 our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/;
 
 our $IMPORT_EXT_RE_1 = qr/\.(ova|ovf|qcow2|raw|vmdk)/;
 
+# FIXME: remove this regex on the next APIAGE reset.
 our $UPLOAD_IMPORT_EXT_RE_1 = qr/\.(ova|qcow2|raw|vmdk)/;
 
-our $SAFE_CHAR_CLASS_RE = qr/[a-zA-Z0-9\-\.\+\=\_]/;
+# FIXME: remove this regex on the next APIAGE reset.
+our $SAFE_CHAR_CLASS_RE = get_re_safe_char_class();
 our $SAFE_CHAR_WITH_WHITESPACE_CLASS_RE = qr/[ a-zA-Z0-9\-\.\+\=\_]/;
 
 our $OVA_CONTENT_RE_1 = qr/${SAFE_CHAR_WITH_WHITESPACE_CLASS_RE}+\.(qcow2|raw|vmdk)/;
 
 # FIXME remove with PVE 9.0, add versioned breaks for pve-manager
@@ -827,13 +836,11 @@ sub path_to_volume_id {
         if ($vtype eq 'snippets') {
             return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
         }
 
         if ($vtype eq 'import') {
-            return if $filename !~ m!^/(${SAFE_CHAR_CLASS_RE}+${IMPORT_EXT_RE_1})$!;
-            my $name = $1;
-            return ($vtype, "$sid:import/$name");
+            return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
         }
 
         return;
     };
 
diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
index ece0dac9..d397bd69 100644
--- a/src/PVE/Storage/Common/Parse.pm
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -7,10 +7,12 @@ use PVE::Storage::Common qw(
 );
 
 use Exporter qw(import);
 
 our @EXPORT_OK = qw(
+    get_re_safe_char_class
+
     parse_rel_path_as_volname_parts
     parse_rel_path_as_volname
     parse_volname_as_parts
 
     parse_abs_path_as_volid_parts
@@ -35,18 +37,24 @@ primarily related to.
 
 my @VZTMPL_COMPRESSION_EXTENSIONS = ('gz', 'xz', 'zst', 'bz2');
 
 my @BACKUP_COMPRESSION_EXTENSIONS = ('gz', 'lzo', 'zst', 'bz2');
 
+my @IMPORT_EXTENSIONS = ('ova', 'ovf', 'qcow2', 'raw', 'vmdk');
+
 my sub join_to_re_alternations(@list) {
     return join('|', map { quotemeta } @list);
 }
 
 my $RE_VZTMPL_COMPRESSION_EXTENSIONS = join_to_re_alternations(@VZTMPL_COMPRESSION_EXTENSIONS);
 
 my $RE_BACKUP_COMPRESSION_EXTENSIONS = join_to_re_alternations(@BACKUP_COMPRESSION_EXTENSIONS);
 
+my $RE_IMPORT_EXTENSIONS = join_to_re_alternations(@IMPORT_EXTENSIONS);
+
+my $RE_SAFE_CHAR_CLASS = qr/[a-zA-Z0-9\-\.\+\=\_]/;
+
 my $RE_PARENT_DIR = quotemeta('..');
 my $RE_CONTAINS_PARENT_DIR = qr!
     ( ^$RE_PARENT_DIR$ )  #  ..  --> Entire path
     |
     ( ^$RE_PARENT_DIR/ )  #  ../ --> Beginning of path
@@ -123,15 +131,22 @@ my $RE_SNIPPETS_FILE_PATH = qr!
     (?<path>
         (?<file> [^/]+ )
     )
 !xn;
 
+my $RE_IMPORT_FILE_PATH = qr!
+    (?<path>
+        (?<file> ($RE_SAFE_CHAR_CLASS)+ \. (?<ext> $RE_IMPORT_EXTENSIONS) )
+    )
+!xn;
+
 my $RE_FILE_PATH_FOR_VTYPE = {
     iso => qr/^$RE_ISO_FILE_PATH$/,
     vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
     backup => qr/^$RE_BACKUP_FILE_PATH$/,
     snippets => qr/^$RE_SNIPPETS_FILE_PATH$/,
+    import => qr/^$RE_IMPORT_FILE_PATH$/,
 };
 
 my $RE_VOLNAME_FOR_VTYPE = {
     iso => qr/^$RE_ISO_FILE_PATH$/,
     vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
@@ -187,10 +202,48 @@ my sub split_leading_dir_from_path($path, $directory) {
     }
 
     return;
 }
 
+=head1 STANDALONE REGULAR EXPRESSIONS
+
+Some of the regular expressions that this module defines can be obtained using
+the subroutines listed below.
+
+B<These subroutines are primarily intended to be used by third-party plugins.>
+
+Parsing code of inbuilt plugins and other modules in the C<L<PVE::Storage>>
+namespace should instead use the constants defined in
+C<L<PVE::Storage::Common::Parse>> directly, and keep the parsing code in there
+as well.
+
+=cut
+
+=head3 get_re_safe_char_class
+
+Returns a regex consisting of a single character class that matches any
+character that is safe to use in more critical places, such as untrusted file
+names.
+
+You would typically use this to obtain a constant that you can use in your own
+parsing code:
+
+
+    my $RE_SAFE_CHAR_CLASS = get_re_safe_char_class();
+
+    # [...]
+
+    my sub is_valid_file_name($file_name) {
+        return $file_name =~ m/${RE_SAFE_CHAR_CLASS}+ \. (json|toml)/xn;
+    }
+
+=cut
+
+sub get_re_safe_char_class() {
+    return $RE_SAFE_CHAR_CLASS;
+}
+
 =head1 PARSERS RELATED TO VOLUMES
 
 The parsing functions in this section primarily deal with parsing data related
 to storage volumes, primarily C<L<volname>>s and C<L<volid>>s.
 
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 8b4f9dde..8d79d0ed 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -1971,17 +1971,16 @@ my sub get_subdir_files {
                 format => 'snippet',
             };
         }
 
         if ($vtype eq 'import') {
-            return
-                if $filename !~
-                m!/(${PVE::Storage::SAFE_CHAR_CLASS_RE}+$PVE::Storage::IMPORT_EXT_RE_1)$!i;
+            my $parts = parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
+            return if !defined($parts);
 
             return {
-                volid => "$storeid:import/$1",
-                format => "$2",
+                volid => $parts->{volid},
+                format => $parts->{ext},
             };
         }
 
         return;
     };
diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm
index 5c038a7d..a1f3eb18 100644
--- a/src/test/list_volumes_test.pm
+++ b/src/test/list_volumes_test.pm
@@ -70,10 +70,11 @@ my $scfg = {
         'rootdir' => 1,
         'vztmpl' => 1,
         'images' => 1,
         'snippets' => 1,
         'backup' => 1,
+        'import' => 1,
     },
 };
 
 # The test cases are comprised of an arry of hashes with the following keys:
 # description => displayed on error by Test::More
@@ -542,10 +543,175 @@ my @tests = (
                 'size' => DEFAULT_SIZE,
                 'volid' => 'local:iso/debian-9.12.0-amd64-netinst.iso',
             },
         ],
     },
+    {
+        description => 'VMID: none, valid file names for import',
+        vmid => undef,
+        files => [
+            "$storage_dir/import/import.ova",
+            "$storage_dir/import/import.ovf",
+            "$storage_dir/import/some-disk.qcow2",
+            "$storage_dir/import/some-disk.vmdk",
+            "$storage_dir/import/some-raw-disk.raw",
+        ],
+        expected => [
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'ova',
+                size => DEFAULT_SIZE,
+                volid => "local:import/import.ova",
+            },
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'ovf',
+                size => DEFAULT_SIZE,
+                volid => "local:import/import.ovf",
+            },
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'qcow2',
+                size => DEFAULT_SIZE,
+                volid => "local:import/some-disk.qcow2",
+            },
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'vmdk',
+                size => DEFAULT_SIZE,
+                volid => "local:import/some-disk.vmdk",
+            },
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'raw',
+                size => DEFAULT_SIZE,
+                volid => "local:import/some-raw-disk.raw",
+            },
+        ],
+    },
+    {
+        description => 'VMID: none, non-matching file paths for import',
+        vmid => undef,
+        files => [
+            # Malformed file names
+            "$storage_dir/import/import.ovff",
+            "$storage_dir/import/importova",
+            "$storage_dir/import/import.ov",
+            "$storage_dir/import/diskraw",
+            "$storage_dir/import/diskvmdk",
+            "$storage_dir/import/disk.invalid",
+            "$storage_dir/import/.ova",
+            "$storage_dir/import/.raw",
+            # Trailing whitespace must not be trimmed
+            "$storage_dir/import/import.ova\t",
+            "$storage_dir/import/disk.raw    ",
+            # Whitespace in file name
+            "$storage_dir/import/something I want to import.ova",
+            "$storage_dir/import/ .raw",
+            "$storage_dir/import/ disk .vmdk",
+            "$storage_dir/import/disk .qcow2",
+            "$storage_dir/import/ import.ova",
+            # Unsafe characters in file name
+            "$storage_dir/import/linux🐧-vm.ova",
+            "$storage_dir/import/🐪perl-playground🐪.ova",
+            "$storage_dir/import/fish_<><_<><_<><.ova",
+            $storage_dir . '/import/C:\\\\Windows\\Path.ova',
+            # Content inside .ova files may only be specified as part
+            # of volume names, and may never appear when looked up as
+            # a file path
+            "$storage_dir/import/import.ova/disk.qcow2",
+            "$storage_dir/import/import.ova/disk.raw",
+            "$storage_dir/import/import.ova/disk.vmdk",
+            "$storage_dir/import/import.ova/disk.invalid",
+        ],
+        expected => [], # returns empty list
+    },
+    {
+        description => 'VMID: none, weird but valid file names for import',
+        vmid => undef,
+        files => [
+            "$storage_dir/import/import.ova.ova",
+            "$storage_dir/import/import.ova.ova.ova",
+            "$storage_dir/import/import.ova.ova.ova.ova",
+            "$storage_dir/import/ova.ova",
+            "$storage_dir/import/ova.ovf",
+            "$storage_dir/import/ova.vmdk",
+            "$storage_dir/import/raw.raw.qcow2",
+            "$storage_dir/import/raw.raw.qcow2.import.qcow2",
+            "$storage_dir/import/raw.raw.raw.your-boat.ova",
+        ],
+        expected => [
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'ova',
+                size => DEFAULT_SIZE,
+                volid => "local:import/import.ova.ova",
+            },
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'ova',
+                size => DEFAULT_SIZE,
+                volid => "local:import/import.ova.ova.ova",
+            },
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'ova',
+                size => DEFAULT_SIZE,
+                volid => "local:import/import.ova.ova.ova.ova",
+            },
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'ova',
+                size => DEFAULT_SIZE,
+                volid => "local:import/ova.ova",
+            },
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'ovf',
+                size => DEFAULT_SIZE,
+                volid => "local:import/ova.ovf",
+            },
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'vmdk',
+                size => DEFAULT_SIZE,
+                volid => "local:import/ova.vmdk",
+            },
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'qcow2',
+                size => DEFAULT_SIZE,
+                volid => "local:import/raw.raw.qcow2",
+            },
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'qcow2',
+                size => DEFAULT_SIZE,
+                volid => "local:import/raw.raw.qcow2.import.qcow2",
+            },
+            {
+                content => 'import',
+                ctime => DEFAULT_CTIME,
+                format => 'ova',
+                size => DEFAULT_SIZE,
+                volid => "local:import/raw.raw.raw.your-boat.ova",
+            },
+        ],
+    },
 );
 
 # provide static vmlist for tests
 my $mock_cluster = Test::MockModule->new('PVE::Cluster', no_auto => 1);
 $mock_cluster->redefine(get_vmlist => sub { return $mocked_vmlist; });
@@ -577,10 +743,14 @@ $mock_fsi->redefine(
 
         return wantarray ? ($size, $format, $used, $parent, $ctime) : $size;
     },
 );
 
+my sub cmp_volinfo_by_volid {
+    return $a->{volid} cmp $b->{volid};
+}
+
 my $plan = scalar @tests;
 plan tests => $plan + 1;
 
 {
     # don't accidentally modify vmlist, see bug report
@@ -600,18 +770,18 @@ plan tests => $plan + 1;
         );
 }
 
 {
     my $sid = 'local';
-    my $types = ['rootdir', 'images', 'vztmpl', 'iso', 'backup', 'snippets'];
+    my $types = [grep { $scfg->{content}->{$_} } keys $scfg->{content}->%*];
     my @suffixes = ('qcow2', 'raw', 'vmdk', 'vhdx');
 
     # run through test cases
     foreach my $tt (@tests) {
         my $vmid = $tt->{vmid};
         my $files = $tt->{files};
-        my $expected = $tt->{expected};
+        my $expected = [sort cmp_volinfo_by_volid $tt->{expected}->@*];
         my $description = $tt->{description};
         my $parent = $tt->{parent};
 
         # prepare environment
         my $num = 0; #parent disks
@@ -630,12 +800,14 @@ plan tests => $plan + 1;
 
                 run_command([@cmd]);
             }
         }
 
-        my $got;
-        eval { $got = PVE::Storage::Plugin->list_volumes($sid, $scfg, $vmid, $types) };
+        my $got = eval {
+            my $volume_list = PVE::Storage::Plugin->list_volumes($sid, $scfg, $vmid, $types);
+            return [sort cmp_volinfo_by_volid $volume_list->@*];
+        };
         $got = $@ if $@;
 
         is_deeply($got, $expected, $description) || diag(explain($got));
 
         # clean up after each test case, otherwise
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 27/51] tree-wide: replace remaining usages of regexes for 'import' vtype
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (25 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 26/51] tree-wide: partially replace usages of regexes for 'import' vtype Max R. Carrara
@ 2026-11-17 20:38 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 28/51] tree-wide: simplify recently refactored parsing logic Max R. Carrara
                   ` (17 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:38 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Implement the remaining pieces that were missing from fully supporting
the 'import' volume type in `PVE::Storage::Common::Parse`. In
particular, parsing volume names for '.ova' files that refer to
contents inside the '.ova' file is now fully supported.

Also add corresponding test cases to 'parser_tests.pl'.

Replace the remaining usages of the `PVE::Storage::IMPORT_EXT_RE_1`
and `PVE::Storage::OVA_CONTENT_RE_1` with parsing functions from
`::Common::Parse` across the repository. As before with other regexes,
add a FIXME comment to mark their removal. This will also be noted
correspondingly in the ApiChangeLog in a future commit.

Note that the logic in `PVE::Storage::Plugin::parse_volname` can now
be simplified a little further, since the `parse_volname_as_parts()`
parsing function also handles extracting the inner and outer file
extensions, amongst other parts.

Since the only remaining usage of the
`PVE::Storage::SAFE_CHAR_WITH_WHITESPACE_CLASS_RE` regex is now in
`PVE::GuestImport::OVF`, handle it the same way as the removed
`SAFE_CHAR_CLASS_RE`. This means that we move it into
`::Common::Parse`, make it private, and only expose it through a
helper subroutine. Once again, add a FIXME comment for this, and
remember that this will be noted in the ApiChangeLog later.

Since we now have a dedicated helper for this regex, stop using it in
`PVE::GuestImport::OVF` as well, and add a small, dedicated sub in
that module to aid with parsing the obtained file path.

Finally, update the test cases in 'guest_import_test.pl' that match
for specific parsing-related exception messages.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/GuestImport.pm                      |  28 +-
 src/PVE/GuestImport/OVF.pm                  |  24 +-
 src/PVE/Storage.pm                          |   7 +-
 src/PVE/Storage/Common/Parse.pm             |  81 ++++++
 src/PVE/Storage/Common/test/parser_tests.pl | 282 ++++++++++++++++++++
 src/PVE/Storage/Plugin.pm                   |  20 +-
 src/test/guest_import_test.pl               |   6 +-
 7 files changed, 414 insertions(+), 34 deletions(-)

diff --git a/src/PVE/GuestImport.pm b/src/PVE/GuestImport.pm
index 31806285..51e8102b 100644
--- a/src/PVE/GuestImport.pm
+++ b/src/PVE/GuestImport.pm
@@ -4,42 +4,40 @@ use strict;
 use warnings;
 
 use File::Path;
 
 use PVE::Storage;
+use PVE::Storage::Common::Parse qw(
+    parse_volname_as_parts
+);
 use PVE::Tools qw(run_command);
 
 sub extract_disk_from_import_file {
     my ($volid, $vmid, $target_storeid) = @_;
 
     my ($source_storeid, $volname) = PVE::Storage::parse_volume_id($volid);
     $target_storeid //= $source_storeid;
-    my $cfg = PVE::Storage::config();
 
-    my ($vtype, $name, undef, undef, undef, undef, $fmt) =
-        PVE::Storage::parse_volname($cfg, $volid);
+    my $parts = parse_volname_as_parts($volname);
+    die "cannot extract $volid - invalid volname $volname\n"
+        if !defined($parts);
+
+    my ($vtype, $outer_fmt) = $parts->@{qw(vtype ext)};
 
     die "only files with content type 'import' can be extracted\n"
         if $vtype ne 'import';
 
     die "only files from 'ova' format can be extracted\n"
-        if $fmt !~ m/^ova\+/;
+        if $outer_fmt ne 'ova';
 
-    # extract the inner file from the name
-    my $archive_volid;
-    my $inner_file;
-    my $inner_fmt;
-    if ($name =~ m!^(.*\.ova)/(${PVE::Storage::SAFE_CHAR_WITH_WHITESPACE_CLASS_RE}+)$!) {
-        $archive_volid = "$source_storeid:import/$1";
-        $inner_file = $2;
-        ($inner_fmt) = $fmt =~ /^ova\+(.*)$/;
-    } else {
-        die "cannot extract $volid - invalid volname $volname\n";
-    }
+    my $archive_volid = $source_storeid . ':' . $vtype . '/' . $parts->{'disk-path'};
+    my $inner_file = $parts->{'content-file'};
+    my $inner_fmt = $parts->{'content-ext'};
 
     die "cannot determine format of '$volid'\n" if !$inner_fmt;
 
+    my $cfg = PVE::Storage::config();
     my $ova_path = PVE::Storage::path($cfg, $archive_volid);
 
     my $tmpdir = PVE::Storage::get_image_dir($cfg, $target_storeid, $vmid);
     my $pid = $$;
     $tmpdir .= "/tmp_${pid}_${vmid}";
diff --git a/src/PVE/GuestImport/OVF.pm b/src/PVE/GuestImport/OVF.pm
index a228cd2c..6a096ee7 100644
--- a/src/PVE/GuestImport/OVF.pm
+++ b/src/PVE/GuestImport/OVF.pm
@@ -10,10 +10,13 @@ use File::Spec;
 use File::Basename;
 use Cwd 'realpath';
 
 use PVE::Tools;
 use PVE::Storage;
+use PVE::Storage::Common::Parse qw(
+    get_re_safe_char_with_whitespace_class
+);
 use PVE::Seccomp qw(stmt /^BPF_/);
 
 # map OVF resources types to descriptive strings
 # this will allow us to explore the xml tree without using magic numbers
 # http://schemas.dmtf.org/wbem/cim-html/2/CIM_ResourceAllocationSettingData.html
@@ -182,10 +185,26 @@ my sub read_ovf_file {
         },
     );
     return $raw;
 }
 
+my $RE_SAFE_FILEPATH = qr/
+    ^ (?<filepath> (${get_re_safe_char_with_whitespace_class()}+) ) $
+/xn;
+
+my sub parse_filepath_from_xpath {
+    my ($filepath) = @_;
+
+    # untaint & check no sub/parent dirs
+    return if $filepath !~ $RE_SAFE_FILEPATH;
+    $filepath = $+{filepath};
+
+    return if !$filepath || $filepath eq '.' || $filepath eq '..';
+
+    return $filepath;
+}
+
 # returns two references, $qm which holds qm.conf style key/values, and \@disks
 my sub parse_ovf_do {
     my ($ovf, $ovf_data, $isOva, $debug) = @_;
 
     # we have to ignore missing disk images for ova
@@ -355,13 +374,12 @@ ovf:Item[rasd:InstanceID='%s']/rasd:ResourceType", $controller_id,
             warn "invalid file reference $fileref, skipping\n";
             next;
         }
         print "file path: $filepath\n" if $debug;
         my $original_filepath = $filepath;
-        ($filepath) = $filepath =~ m|^(${PVE::Storage::SAFE_CHAR_WITH_WHITESPACE_CLASS_RE}+)$|; # untaint & check no sub/parent dirs
-        die "referenced path '$original_filepath' is invalid\n"
-            if !$filepath || $filepath eq "." || $filepath eq "..";
+        $filepath = parse_filepath_from_xpath($filepath);
+        die "referenced path '$original_filepath' is invalid\n" if !$filepath;
 
         push @disks,
             {
                 filepath => $filepath,
                 virtual_size => $virtual_size,
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index d17482e6..46578989 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -26,10 +26,11 @@ use PVE::SSHInfo;
 use PVE::Storage::Common qw(
     plugin_get_vtype_subdir
 );
 use PVE::Storage::Common::Parse qw(
     get_re_safe_char_class
+    get_re_safe_char_with_whitespace_class
     parse_abs_path_as_volid
 );
 use PVE::RESTEnvironment qw(log_warn);
 
 use PVE::Storage::Common qw(
@@ -135,19 +136,23 @@ our $ISO_EXT_RE_0 = qr/\.(?:iso|img)/i;
 our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2))/i;
 
 # FIXME: remove this regex on the next APIAGE reset.
 our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/;
 
+# FIXME: remove this regex on the next APIAGE reset.
 our $IMPORT_EXT_RE_1 = qr/\.(ova|ovf|qcow2|raw|vmdk)/;
 
 # FIXME: remove this regex on the next APIAGE reset.
 our $UPLOAD_IMPORT_EXT_RE_1 = qr/\.(ova|qcow2|raw|vmdk)/;
 
 # FIXME: remove this regex on the next APIAGE reset.
 our $SAFE_CHAR_CLASS_RE = get_re_safe_char_class();
-our $SAFE_CHAR_WITH_WHITESPACE_CLASS_RE = qr/[ a-zA-Z0-9\-\.\+\=\_]/;
 
+# FIXME: remove this regex on the next APIAGE reset.
+our $SAFE_CHAR_WITH_WHITESPACE_CLASS_RE = get_re_safe_char_with_whitespace_class();
+
+# FIXME: remove this regex on the next APIAGE reset.
 our $OVA_CONTENT_RE_1 = qr/${SAFE_CHAR_WITH_WHITESPACE_CLASS_RE}+\.(qcow2|raw|vmdk)/;
 
 # FIXME remove with PVE 9.0, add versioned breaks for pve-manager
 our $vztmpl_extension_re = $VZTMPL_EXT_RE_1;
 
diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
index d397bd69..2f6380a8 100644
--- a/src/PVE/Storage/Common/Parse.pm
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -8,10 +8,11 @@ use PVE::Storage::Common qw(
 
 use Exporter qw(import);
 
 our @EXPORT_OK = qw(
     get_re_safe_char_class
+    get_re_safe_char_with_whitespace_class
 
     parse_rel_path_as_volname_parts
     parse_rel_path_as_volname
     parse_volname_as_parts
 
@@ -39,22 +40,28 @@ my @VZTMPL_COMPRESSION_EXTENSIONS = ('gz', 'xz', 'zst', 'bz2');
 
 my @BACKUP_COMPRESSION_EXTENSIONS = ('gz', 'lzo', 'zst', 'bz2');
 
 my @IMPORT_EXTENSIONS = ('ova', 'ovf', 'qcow2', 'raw', 'vmdk');
 
+my @OVA_CONTENT_EXTENSIONS = ('qcow2', 'raw', 'vmdk');
+
 my sub join_to_re_alternations(@list) {
     return join('|', map { quotemeta } @list);
 }
 
 my $RE_VZTMPL_COMPRESSION_EXTENSIONS = join_to_re_alternations(@VZTMPL_COMPRESSION_EXTENSIONS);
 
 my $RE_BACKUP_COMPRESSION_EXTENSIONS = join_to_re_alternations(@BACKUP_COMPRESSION_EXTENSIONS);
 
 my $RE_IMPORT_EXTENSIONS = join_to_re_alternations(@IMPORT_EXTENSIONS);
 
+my $RE_OVA_CONTENT_EXTENSIONS = join_to_re_alternations(@OVA_CONTENT_EXTENSIONS);
+
 my $RE_SAFE_CHAR_CLASS = qr/[a-zA-Z0-9\-\.\+\=\_]/;
 
+my $RE_SAFE_CHAR_WITH_WHITESPACE_CLASS = qr/[ a-zA-Z0-9\-\.\+\=\_]/;
+
 my $RE_PARENT_DIR = quotemeta('..');
 my $RE_CONTAINS_PARENT_DIR = qr!
     ( ^$RE_PARENT_DIR$ )  #  ..  --> Entire path
     |
     ( ^$RE_PARENT_DIR/ )  #  ../ --> Beginning of path
@@ -137,23 +144,62 @@ my $RE_IMPORT_FILE_PATH = qr!
     (?<path>
         (?<file> ($RE_SAFE_CHAR_CLASS)+ \. (?<ext> $RE_IMPORT_EXTENSIONS) )
     )
 !xn;
 
+my $RE_OVA_CONTENT = qr!
+    (?<content_file>
+        ($RE_SAFE_CHAR_WITH_WHITESPACE_CLASS)+ \. (?<content_ext> $RE_OVA_CONTENT_EXTENSIONS)
+    )
+!xn;
+
+# NOTE: Volume names with the 'import' vtype are treated differently when
+# they do not stem from a file path directly - see comments inline.
+my $RE_IMPORT_VOLNAME_OVA_FILE_WITH_CONTENT = qr!
+    (?<file>
+        # NOTE: Unlike in RE_IMPORT_FILE_PATH, we allow whitespace here
+        ($RE_SAFE_CHAR_WITH_WHITESPACE_CLASS)+
+        \.
+        (?<ext> ova)
+    ) / (?<content> $RE_OVA_CONTENT)
+!xn;
+
+my $RE_IMPORT_VOLNAME_REGULAR_FILE = qr!
+    (?<file>
+        # NOTE: Unlike in RE_IMPORT_FILE_PATH, we allow whitespace here
+        ($RE_SAFE_CHAR_WITH_WHITESPACE_CLASS)+
+        \.
+        (?<ext> $RE_IMPORT_EXTENSIONS)
+    )
+!xn;
+
+my $RE_IMPORT_VOLNAME = qr!
+    (?<path>
+        # NOTE: Order here matters - the ova+content regex is stricter,
+        # so try this branch here first
+        ($RE_IMPORT_VOLNAME_OVA_FILE_WITH_CONTENT)
+        |
+        ($RE_IMPORT_VOLNAME_REGULAR_FILE)
+    )
+!xn;
+
 my $RE_FILE_PATH_FOR_VTYPE = {
     iso => qr/^$RE_ISO_FILE_PATH$/,
     vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
     backup => qr/^$RE_BACKUP_FILE_PATH$/,
     snippets => qr/^$RE_SNIPPETS_FILE_PATH$/,
     import => qr/^$RE_IMPORT_FILE_PATH$/,
 };
 
 my $RE_VOLNAME_FOR_VTYPE = {
     iso => qr/^$RE_ISO_FILE_PATH$/,
     vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
     backup => qr/^$RE_BACKUP_FILE_PATH$/,
     snippets => qr/^$RE_SNIPPETS_FILE_PATH$/,
+
+    # special cases - not reusing file path regexes:
+    import => qr/^$RE_IMPORT_VOLNAME$/,
 };
 
 my sub contains_parent_dir($path) {
     return $path =~ $RE_CONTAINS_PARENT_DIR;
 }
@@ -240,10 +286,21 @@ parsing code:
 
 sub get_re_safe_char_class() {
     return $RE_SAFE_CHAR_CLASS;
 }
 
+=head3 get_re_safe_char_with_whitespace_class
+
+Like C<L<< get_re_safe_char_class()|/"get_re_safe_char_class" >>>, but the
+returned character class also allows a space character (U+0020).
+
+=cut
+
+sub get_re_safe_char_with_whitespace_class() {
+    return $RE_SAFE_CHAR_WITH_WHITESPACE_CLASS;
+}
+
 =head1 PARSERS RELATED TO VOLUMES
 
 The parsing functions in this section primarily deal with parsing data related
 to storage volumes, primarily C<L<volname>>s and C<L<volid>>s.
 
@@ -331,10 +388,34 @@ C<ext-compression> (optional).
         path => 'vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
         vtype => 'backup',
         volname => 'backup/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
     }
 
+=item C<import>
+
+B<NOTE:> Only for C<L<< parse_volname_as_parts()|/"parse_volname_as_parts" >>>!
+
+C<import> volnames may additionally refer to the C<content> inside an OVA file,
+which is further differentiated into C<content-file> and C<content-ext>:
+
+    {
+        file => 'import.ova',
+        ext => 'ova',
+        'disk-path' => 'import.ova',
+        path => 'import.ova/disk.raw',
+
+        # Exclusive to 'import' volume names
+        content => 'disk.raw',
+
+        # Used to further differentiate 'content'
+        'content-file' => 'disk.raw',
+        'content-ext' => 'raw',
+
+        vtype => 'import',
+        volname => 'import/import.ova/disk.raw',
+    }
+
 =back
 
 =cut
 
 =head3 parse_rel_path_as_volname_parts
diff --git a/src/PVE/Storage/Common/test/parser_tests.pl b/src/PVE/Storage/Common/test/parser_tests.pl
index 812a466f..af0a7052 100755
--- a/src/PVE/Storage/Common/test/parser_tests.pl
+++ b/src/PVE/Storage/Common/test/parser_tests.pl
@@ -488,10 +488,95 @@ my $volname_cases_snippets_valid = [
             volname => 'snippets/userconfig.yaml',
         },
     },
 ];
 
+my $volname_cases_import_valid = [
+    {
+        path => 'import.ova',
+        expected => {
+            file => 'import.ova',
+            ext => 'ova',
+            'disk-path' => 'import.ova',
+            path => 'import.ova',
+            vtype => 'import',
+            volname => 'import/import.ova',
+        },
+    },
+    {
+        path => 'import.ovf',
+        expected => {
+            file => 'import.ovf',
+            ext => 'ovf',
+            'disk-path' => 'import.ovf',
+            path => 'import.ovf',
+            vtype => 'import',
+            volname => 'import/import.ovf',
+        },
+    },
+    {
+        path => 'disk-0.qcow2',
+        expected => {
+            file => 'disk-0.qcow2',
+            ext => 'qcow2',
+            'disk-path' => 'disk-0.qcow2',
+            path => 'disk-0.qcow2',
+            vtype => 'import',
+            volname => 'import/disk-0.qcow2',
+        },
+    },
+    {
+        path => 'raw_disk_v2.5.raw',
+        expected => {
+            file => 'raw_disk_v2.5.raw',
+            ext => 'raw',
+            'disk-path' => 'raw_disk_v2.5.raw',
+            path => 'raw_disk_v2.5.raw',
+            vtype => 'import',
+            volname => 'import/raw_disk_v2.5.raw',
+        },
+    },
+    {
+        path => 'disk-1+data.vmdk',
+        expected => {
+            file => 'disk-1+data.vmdk',
+            ext => 'vmdk',
+            'disk-path' => 'disk-1+data.vmdk',
+            path => 'disk-1+data.vmdk',
+            vtype => 'import',
+            volname => 'import/disk-1+data.vmdk',
+        },
+    },
+];
+
+my $volname_cases_import_invalid = [
+    {
+        description => "Invalid file extension (import)",
+        args => {
+            path => 'import.zip',
+            vtype => 'import',
+        },
+        expected => undef,
+    },
+    {
+        description => "Uppercase letter in file extension (import)",
+        args => {
+            path => 'import.Ova',
+            vtype => 'import',
+        },
+        expected => undef,
+    },
+    {
+        description => "Unsafe characters in file name (import)",
+        args => {
+            path => '🐪perl-playground🐪.ova',
+            vtype => 'import',
+        },
+        expected => undef,
+    },
+];
+
 my $volname_cases_newline_invalid = [
     {
         description => "Trailing newline (iso)",
         args => {
             path => "debian.iso\n",
@@ -520,16 +605,18 @@ my $volname_cases_newline_invalid = [
 my $cases_valid_all = [
     $volname_cases_iso_valid,
     $volname_cases_vztmpl_valid,
     $volname_cases_backup_valid,
     $volname_cases_snippets_valid,
+    $volname_cases_import_valid,
 ];
 
 my $cases_invalid_all = [
     $volname_cases_iso_invalid,
     $volname_cases_vztmpl_invalid,
     $volname_cases_backup_invalid,
+    $volname_cases_import_invalid,
     $volname_cases_newline_invalid,
 ];
 
 {
     for my $cases_list ($cases_valid_all->@*) {
@@ -595,10 +682,183 @@ my sub run_volname_parsing_tests : prototype($) ($volname_tests) {
     }
 
     return;
 }
 
+# NOTE: These run for parse_rel_path_as_volname_parts() and parse_rel_path_as_volname() only!
+my $cases_import_special_file_path = [
+    {
+        description => "Whitespace not allowed in volume file paths (import)",
+        args => {
+            path => 'Some Disk.qcow2',
+            vtype => 'import',
+        },
+        expected => undef,
+    },
+    {
+        description => "Volume file path with disallowed OVA content (raw) (import)",
+        args => {
+            path => 'import.ova/disk.raw',
+            vtype => 'import',
+        },
+        expected => undef,
+    },
+    {
+        description => "Volume file path with disallowed OVA content (qcow2) (import)",
+        args => {
+            path => 'import.ova/disk.qcow2',
+            vtype => 'import',
+        },
+        expected => undef,
+    },
+    {
+        description => "Volume file path with disallowed OVA content (vmdk) (import)",
+        args => {
+            path => 'import.ova/disk.vmdk',
+            vtype => 'import',
+        },
+        expected => undef,
+    },
+];
+
+# NOTE: These run for parse_volname_as_parts() only!
+my $cases_import_special_volname = [
+    {
+        description => "Whitespace allowed in volume names (import)",
+        args => {
+            volname => 'import/Some Disk.qcow2',
+        },
+        expected => {
+            file => 'Some Disk.qcow2',
+            ext => 'qcow2',
+            'disk-path' => 'Some Disk.qcow2',
+            path => 'Some Disk.qcow2',
+            vtype => 'import',
+            volname => 'import/Some Disk.qcow2',
+        },
+    },
+    {
+        description => "Volume name with OVA content (raw) (import)",
+        args => {
+            volname => 'import/import.ova/disk.raw',
+        },
+        expected => {
+            file => 'import.ova',
+            ext => 'ova',
+            'disk-path' => 'import.ova',
+            path => 'import.ova/disk.raw',
+            content => 'disk.raw',
+            'content-file' => 'disk.raw',
+            'content-ext' => 'raw',
+            vtype => 'import',
+            volname => 'import/import.ova/disk.raw',
+        },
+    },
+    {
+        description => "Volume name with OVA content (qcow2) (import)",
+        args => {
+            volname => 'import/import.ova/disk.qcow2',
+        },
+        expected => {
+            file => 'import.ova',
+            ext => 'ova',
+            'disk-path' => 'import.ova',
+            path => 'import.ova/disk.qcow2',
+            content => 'disk.qcow2',
+            'content-file' => 'disk.qcow2',
+            'content-ext' => 'qcow2',
+            vtype => 'import',
+            volname => 'import/import.ova/disk.qcow2',
+        },
+    },
+    {
+        description => "Volume name with OVA content (vmdk) (import)",
+        args => {
+            volname => 'import/import.ova/disk.vmdk',
+        },
+        expected => {
+            file => 'import.ova',
+            ext => 'ova',
+            'disk-path' => 'import.ova',
+            path => 'import.ova/disk.vmdk',
+            content => 'disk.vmdk',
+            'content-file' => 'disk.vmdk',
+            'content-ext' => 'vmdk',
+            vtype => 'import',
+            volname => 'import/import.ova/disk.vmdk',
+        },
+    },
+    {
+        description => "Whitespace in volume name + OVA content (import)",
+        args => {
+            volname => 'import/Some Import.ova/disk.qcow2',
+        },
+        expected => {
+            file => 'Some Import.ova',
+            ext => 'ova',
+            'disk-path' => 'Some Import.ova',
+            path => 'Some Import.ova/disk.qcow2',
+            content => 'disk.qcow2',
+            'content-file' => 'disk.qcow2',
+            'content-ext' => 'qcow2',
+            vtype => 'import',
+            volname => 'import/Some Import.ova/disk.qcow2',
+        },
+    },
+];
+
+my sub run_special_import_volname_parsing_tests : prototype() () {
+    for my $case ($cases_import_special_file_path->@*) {
+        subtest $case->{description} => sub () {
+            my ($path, $vtype) = $case->{args}->@{qw(path vtype)};
+
+            my $got_volname_parts = parse_rel_path_as_volname_parts($path, $vtype);
+            my $got_volname = parse_rel_path_as_volname($path, $vtype);
+
+            if (defined($case->{expected})) {
+                eq_or_diff(
+                    $got_volname_parts,
+                    $case->{expected},
+                    'parse_rel_path_as_volname_parts() returns expected hashref',
+                    { context => 50000 },
+                );
+
+                is(
+                    $got_volname,
+                    $case->{expected}->{volname},
+                    'parse_rel_path_as_volname() returns expected volname',
+                );
+            } else {
+                is($got_volname_parts, undef,
+                    'parse_rel_path_as_volname_parts() returns undef');
+                is($got_volname, undef, 'parse_rel_path_as_volname() returns undef');
+            }
+        };
+    }
+
+    for my $case ($cases_import_special_volname->@*) {
+        subtest $case->{description} => sub () {
+            my ($volname) = $case->{args}->@{qw(volname)};
+
+            my $got_volname = parse_volname_as_parts($volname);
+
+            if (defined($case->{expected})) {
+                eq_or_diff(
+                    $got_volname,
+                    $case->{expected},
+                    'parse_volname_as_parts() returns expected hashref',
+                    { context => 50000 },
+                );
+            } else {
+                is($got_volname, undef, 'parse_volname_as_parts() returns undef');
+            }
+        };
+    }
+
+    return;
+}
+
 my $DEFAULT_STOREID = 'local';
 my $DEFAULT_STORAGE_PATH = File::Temp->newdir();
 my $DEFAULT_SCFG = {
     type => 'dir',
     path => $DEFAULT_STORAGE_PATH,
@@ -667,10 +927,30 @@ my $volid_tests = [
         cases => [
             map { format_volname_case_to_volid_case($ALT_STOREID, $ALT_SCFG, $_) }
                 $volname_tests->@*
         ],
     },
+    {
+        description =>
+            "volid parsers build on volname parsers' behaviors - 'import' vtype file paths (1)",
+        storedid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        cases => [
+            map { format_volname_case_to_volid_case($DEFAULT_STOREID, $DEFAULT_SCFG, $_) }
+                $cases_import_special_file_path->@*
+        ],
+    },
+    {
+        description =>
+            "volid parsers build on volname parsers' behaviors - 'import' vtype file paths (2)",
+        storedid => $ALT_STOREID,
+        scfg => $ALT_SCFG,
+        cases => [
+            map { format_volname_case_to_volid_case($DEFAULT_STOREID, $DEFAULT_SCFG, $_) }
+                $cases_import_special_file_path->@*
+        ],
+    },
 ];
 
 my sub run_volid_parsing_tests : prototype($) ($volid_tests) {
     for my $test ($volid_tests->@*) {
         subtest $test->{description} => sub() {
@@ -726,10 +1006,12 @@ my sub run_volid_parsing_tests : prototype($) ($volid_tests) {
 
 my sub main() {
     unified_diff();
 
     run_volname_parsing_tests($volname_tests);
+    run_special_import_volname_parsing_tests();
+
     run_volid_parsing_tests($volid_tests);
 
     done_testing();
 
     return;
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 8d79d0ed..8707219f 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -917,24 +917,20 @@ sub parse_volname {
         }
 
         if ($vtype eq 'snippets') {
             return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
         }
-    }
 
-    if ($volname =~
-        m!^import/(${PVE::Storage::SAFE_CHAR_WITH_WHITESPACE_CLASS_RE}+\.ova\/${PVE::Storage::OVA_CONTENT_RE_1})$!
-    ) {
-        my $packed_image = $1;
-        my $format = $2;
-        return ('import', $packed_image, undef, undef, undef, undef, "ova+$format");
-    }
+        if ($vtype eq 'import') {
+            my $format = $parts->{ext};
 
-    if ($volname =~
-        m!^import/(${PVE::Storage::SAFE_CHAR_WITH_WHITESPACE_CLASS_RE}+$PVE::Storage::IMPORT_EXT_RE_1)$!
-    ) {
-        return ('import', $1, undef, undef, undef, undef, $2);
+            if (defined(my $content_ext = $parts->{'content-ext'})) {
+                $format .= "+$content_ext";
+            }
+
+            return ($vtype, $volume_path, undef, undef, undef, undef, $format);
+        }
     }
 
     die "unable to parse directory volume name '$volname'\n";
 }
 
diff --git a/src/test/guest_import_test.pl b/src/test/guest_import_test.pl
index 04eec24d..234013e3 100755
--- a/src/test/guest_import_test.pl
+++ b/src/test/guest_import_test.pl
@@ -355,11 +355,11 @@ my $tests = [
         },
         cases => [
             {
                 volname => 'import/some-import.ova/disk.txt',
                 vmid => 1337,
-                'expect-fail' => qr/unable to parse directory volume name/,
+                'expect-fail' => qr/invalid volname/,
             },
         ],
     },
     {
         description => "Importing multiple disks, no existing disks (qcow2, vmdk, raw)",
@@ -530,11 +530,11 @@ my $tests = [
         },
         cases => [
             {
                 volname => 'import/some-🐧-import.ova/disk.qcow2',
                 vmid => 1337,
-                'expect-fail' => qr/unable to parse directory volume name/,
+                'expect-fail' => qr/invalid volname/,
             },
         ],
     },
     {
         description => "Importing a single disk, invalid volume type",
@@ -617,11 +617,11 @@ my $tests = [
         },
         cases => [
             {
                 volname => 'import/some-import.ova',
                 vmid => 1337,
-                'expect-fail' => qr/only files from 'ova' format can be extracted/,
+                'expect-fail' => qr/cannot determine format of/,
             },
         ],
     },
     {
         description => "Importing a single disk, invalid archive format for .ova",
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 28/51] tree-wide: simplify recently refactored parsing logic
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (26 preceding siblings ...)
  2026-11-17 20:38 ` [PATCH pve-storage v3 27/51] tree-wide: replace remaining " Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 29/51] test: list volumes: reorganize and modernize test running code Max R. Carrara
                   ` (16 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

PVE::API2::Storage::Status:

In the parsing helper for file transfers, it is not necessary to
handle the parsing of each volume type individually anymore, since we
use `assert_can_transfer_files()` beforehand and also raise an
exception if parsing fails in general.

Therefore, pull out the common parts of the branches for individual
volume types. Retain only the special case for the `import` type where
we do not accept `.ovf` files.

PVE::Storage:

Merge the branches in `path_to_volume_id()` for all volume types,
except `images`, since they all do the same thing now. Strip the
volume type subdirectory from the given file path only for the
`images` volume type.

PVE::Storage::Plugin:

Condense all of the if-clauses as much as possible in the
`parse_volname()` and `get_subdir_files()` subs.

In `parse_volname()`, extend the error message of the final `die` at
the end of the method, stating that the volume type is unhandled.

The guard clause after the `parse_volname_as_parts()` call already
`die`s on parsing failure, so that final `die` should never be hit
unless we introduce a new volume type and happen to forget to handle
it in this method.

In `get_subdir_files()`, do not strip the volume type subdirectory
from the beginning of the current path anymore, because that is
handled by the `parse_abs_path_as_volid_parts()` parsing function, which
left `$filename` unused.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/API2/Storage/Status.pm | 40 ++++--------------
 src/PVE/Storage.pm             | 44 ++++++--------------
 src/PVE/Storage/Plugin.pm      | 76 ++++++++++++----------------------
 3 files changed, 45 insertions(+), 115 deletions(-)

diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm
index 15ba1fe3..24213f79 100644
--- a/src/PVE/API2/Storage/Status.pm
+++ b/src/PVE/API2/Storage/Status.pm
@@ -71,49 +71,23 @@ my sub assert_can_transfer_files : prototype($$$) {
 }
 
 my sub parse_transferred_file_path_extension : prototype($$) {
     my ($path, $vtype) = @_;
 
-    if ($vtype eq 'iso') {
-        my $parts = parse_rel_path_as_volname_parts($path, $vtype);
+    my $parts = parse_rel_path_as_volname_parts($path, $vtype);
 
-        if (!defined($parts)) {
-            raise_param_exc({ filename => "wrong file extension" });
-        }
-
-        my $ext = $parts->{ext};
-        return $ext;
+    if (!defined($parts)) {
+        raise_param_exc({ filename => "invalid filename or wrong file extension" });
     }
 
-    if ($vtype eq 'vztmpl') {
-        my $parts = parse_rel_path_as_volname_parts($path, $vtype);
+    my $ext = $parts->{ext};
 
-        if (!defined($parts)) {
-            raise_param_exc({ filename => "wrong file extension" });
-        }
-
-        my $ext = $parts->{ext};
-        return $ext;
+    if ($vtype eq 'import' && $ext eq 'ovf') {
+        raise_param_exc({ filename => "wrong file extension" });
     }
 
-    if ($vtype eq 'import') {
-        my $parts = parse_rel_path_as_volname_parts($path, $vtype);
-
-        if (!defined($parts)) {
-            raise_param_exc({ filename => "invalid filename or wrong extension" });
-        }
-
-        my $ext = $parts->{ext};
-
-        if ($ext eq 'ovf') {
-            raise_param_exc({ filename => "wrong file extension" });
-        }
-
-        return $ext;
-    }
-
-    die "upload / download: failed to parse '$path' - unhandled content type '$vtype'\n";
+    return $ext;
 }
 
 my sub assert_file_transfer_contents_valid : prototype($$$) {
     my ($path, $vtype, $extension) = @_;
 
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 46578989..f3d1298d 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -795,22 +795,22 @@ sub path_to_volume_id {
     };
 
     my $parse_volid_from_file_path = sub {
         my ($plugin, $sid, $scfg, $vtype) = @_;
 
-        # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
-        # on the next APIAGE reset.
-        my $vtype_subdir = $plugin->get_subdir($scfg, $vtype);
-
-        # Strip the vtype subdir from beginning of the current path
-        # so that we don't have to take it into account when parsing the file name
-        my $filename = $path;
-        if ($filename !~ s!^\Q$vtype_subdir\E(?=/)!!) {
-            return;
-        }
-
         if ($vtype eq 'images') {
+            # FIXME: Call plugin_get_vtype_subdir() instead of $plugin->get_subdir()
+            # on the next APIAGE reset.
+            my $vtype_subdir = $plugin->get_subdir($scfg, $vtype);
+
+            # Strip the vtype subdir from beginning of the current path
+            # so that we don't have to take it into account when parsing the file name
+            my $filename = $path;
+            if ($filename !~ s!^\Q$vtype_subdir\E(?=/)!!) {
+                return;
+            }
+
             return if $filename !~ m!^/(\d+)/([^/\s]+)$!;
             my $vmid = $1;
             my $name = $2;
 
             my $vollist = $plugin->list_images($sid, $scfg, $vmid);
@@ -824,31 +824,11 @@ sub path_to_volume_id {
             }
 
             return;
         }
 
-        if ($vtype eq 'iso') {
-            return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
-        }
-
-        if ($vtype eq 'vztmpl') {
-            return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
-        }
-
-        if ($vtype eq 'backup') {
-            return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
-        }
-
-        if ($vtype eq 'snippets') {
-            return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
-        }
-
-        if ($vtype eq 'import') {
-            return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
-        }
-
-        return;
+        return ($vtype, parse_abs_path_as_volid($sid, $scfg, $path, $vtype));
     };
 
     for my $sid (keys $ids->%*) {
         my $scfg = $ids->{$sid};
         next if !$scfg->{path};
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 8707219f..0d44224f 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -20,10 +20,11 @@ use PVE::Storage::Common qw(
     plugin_get_vtype_subdir
 );
 use PVE::Storage::Common::Parse qw(
     parse_volname_as_parts
     parse_abs_path_as_volid_parts
+    parse_volid_as_parts
 );
 
 use JSON;
 
 use base qw(PVE::SectionConfig);
@@ -470,12 +471,12 @@ sub verify_options {
 PVE::JSONSchema::register_format('pve-volume-id', \&parse_volume_id);
 
 sub parse_volume_id {
     my ($volid, $noerr) = @_;
 
-    if ($volid =~ m/^([a-z][a-z0-9\-\_\.]*[a-z0-9]):(.+)$/i) {
-        return wantarray ? ($1, $2) : $1;
+    if (defined(my $parts = parse_volid_as_parts($volid))) {
+        return wantarray ? ($parts->{storeid}, $parts->{volname}) : $parts->{storeid};
     }
     return undef if $noerr;
     die "unable to parse volume ID '$volid'\n";
 }
 
@@ -899,41 +900,35 @@ sub parse_volname {
         my (undef, $format, $isBase) = parse_name_dir($name);
         my $vtype = $class->guest_volume_vtype($format, $name);
         return ($vtype, $name, $vmid, undef, undef, $isBase, $format);
     }
 
-    if (defined(my $parts = parse_volname_as_parts($volname))) {
-        my ($vtype, $volume_path) = $parts->@{qw(vtype path)};
+    my $parts = parse_volname_as_parts($volname);
+    die "unable to parse directory volume name '$volname'\n"
+        if !defined($parts);
 
-        if ($vtype eq 'iso') {
-            return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
-        }
+    my ($vtype, $volume_path) = $parts->@{qw(vtype path)};
 
-        if ($vtype eq 'vztmpl') {
-            return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
-        }
-
-        if ($vtype eq 'backup') {
-            return ($vtype, $volume_path, $parts->{vmid}, undef, undef, undef, 'raw');
-        }
-
-        if ($vtype eq 'snippets') {
-            return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
-        }
-
-        if ($vtype eq 'import') {
-            my $format = $parts->{ext};
-
-            if (defined(my $content_ext = $parts->{'content-ext'})) {
-                $format .= "+$content_ext";
-            }
-
-            return ($vtype, $volume_path, undef, undef, undef, undef, $format);
-        }
+    if ($vtype eq 'iso' || $vtype eq 'vztmpl' || $vtype eq 'snippets') {
+        return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
     }
 
-    die "unable to parse directory volume name '$volname'\n";
+    if ($vtype eq 'backup') {
+        return ($vtype, $volume_path, $parts->{vmid}, undef, undef, undef, 'raw');
+    }
+
+    if ($vtype eq 'import') {
+        my $format = $parts->{ext};
+
+        if (defined(my $content_ext = $parts->{'content-ext'})) {
+            $format .= "+$content_ext";
+        }
+
+        return ($vtype, $volume_path, undef, undef, undef, undef, $format);
+    }
+
+    die "unable to parse directory volume name '$volname' - unhandled volume type '$vtype'\n";
 }
 
 =head3 get_parsed_format
 
 Return the disk format encoded in the given volume name, or C<undef> if the name does not spell one
@@ -1882,44 +1877,31 @@ my sub get_subdir_files {
     my $res = [];
 
     my $get_subdir_file_info = sub {
         my ($path, $st) = @_;
 
-        # Strip the vtype subdir from beginning of the current path
-        # so that we don't have to take it into account when parsing the file name
-        my $filename = $path;
-        if ($filename !~ s!^\Q$vtype_subdir\E(?=/)!!) {
-            return;
-        }
+        my $parts = parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
+        return if !defined($parts);
 
         if ($vtype eq 'iso') {
-            my $parts = parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
-            return if !defined($parts);
-
             return {
                 volid => $parts->{volid},
                 format => 'iso', # always 'iso' even if we have a file ending in .img
             };
         }
 
         if ($vtype eq 'vztmpl') {
-            my $parts = parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
-            return if !defined($parts);
-
             my ($ext_archive, $ext_compression) = $parts->@{qw(ext-archive ext-compression)};
             my $format = defined($ext_compression) ? "t$ext_compression" : $ext_archive;
 
             return {
                 volid => $parts->{volid},
                 format => $format,
             };
         }
 
         if ($vtype eq 'backup') {
-            my $parts = parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
-            return if !defined($parts);
-
             my $format = $parts->{ext};
             my $volume_path = $parts->{path};
 
             # Check if parsed VMID matched provided VMID in order to avoid
             # false positives (VMID in parent directory name)
@@ -1957,23 +1939,17 @@ my sub get_subdir_files {
 
             return $info;
         }
 
         if ($vtype eq 'snippets') {
-            my $parts = parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
-            return if !defined($parts);
-
             return {
                 volid => $parts->{volid},
                 format => 'snippet',
             };
         }
 
         if ($vtype eq 'import') {
-            my $parts = parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
-            return if !defined($parts);
-
             return {
                 volid => $parts->{volid},
                 format => $parts->{ext},
             };
         }
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 29/51] test: list volumes: reorganize and modernize test running code
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (27 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 28/51] tree-wide: simplify recently refactored parsing logic Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 30/51] test: list volumes: fix broken test checking for vmlist modifications Max R. Carrara
                   ` (15 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Use the `use v5.36` pragma to enable subroutine signatures,
warnings and strictness by default.

Factor out the list of file suffixes for file formats that support
backing files into its own constant.

Introduce a `main()` subroutine.

Move the code that runs the existing tests for
`::Plugin->list_volumes()` into its own subroutine.

Move the remaining single test that checks whether the vmlist returned
by `PVE::Cluster::get_vmlist()` was modified into `main()`.

Move the call to `Test::More`'s `plan()` into `main()` as well.

Done as preparation for other changes.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/list_volumes_test.pm | 66 ++++++++++++++++++++---------------
 1 file changed, 38 insertions(+), 28 deletions(-)

diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm
index a1f3eb18..176b5a46 100644
--- a/src/test/list_volumes_test.pm
+++ b/src/test/list_volumes_test.pm
@@ -1,21 +1,20 @@
 package PVE::Storage::TestListVolumes;
 
-use strict;
-use warnings;
+use v5.36;
 
 use lib qw(..);
 
 use PVE::Storage;
 use PVE::Cluster;
 use PVE::Tools qw(run_command);
 
 use Test::More;
 use Test::MockModule;
 
 use Cwd;
-use File::Basename;
+use File::Basename qw(fileparse);
 use File::Path qw(make_path remove_tree);
 use File::stat qw();
 use File::Temp;
 use Storable qw(dclone);
 
@@ -74,10 +73,12 @@ my $scfg = {
         'backup' => 1,
         'import' => 1,
     },
 };
 
+my @BACKING_FILE_SUFFIXES = ('qcow2', 'raw', 'vmdk', 'vhdx');
+
 # The test cases are comprised of an arry of hashes with the following keys:
 # description => displayed on error by Test::More
 # vmid        => used for image matches by list_volume
 # files       => array of files for qemu-img to create
 # expected    => returned result hash
@@ -747,48 +748,26 @@ $mock_fsi->redefine(
 
 my sub cmp_volinfo_by_volid {
     return $a->{volid} cmp $b->{volid};
 }
 
-my $plan = scalar @tests;
-plan tests => $plan + 1;
-
-{
-    # don't accidentally modify vmlist, see bug report
-    # https://pve.proxmox.com/pipermail/pve-devel/2020-January/041096.html
-    my $scfg_with_type = { path => $storage_dir, type => 'dir' };
-    my $original_vmlist = { ids => {} };
-    my $tested_vmlist = dclone($original_vmlist);
-
-    PVE::Storage::Plugin->list_volumes('sid', $scfg_with_type, undef, ['images']);
-
-    is_deeply($tested_vmlist, $original_vmlist, 'PVE::Cluster::vmlist remains unmodified')
-        || diag(
-            "Expected vmlist to remain\n",
-            explain($original_vmlist),
-            "but it turned to\n",
-            explain($tested_vmlist),
-        );
-}
-
-{
+my sub run_legacy_tests() {
     my $sid = 'local';
     my $types = [grep { $scfg->{content}->{$_} } keys $scfg->{content}->%*];
-    my @suffixes = ('qcow2', 'raw', 'vmdk', 'vhdx');
 
     # run through test cases
     foreach my $tt (@tests) {
         my $vmid = $tt->{vmid};
         my $files = $tt->{files};
         my $expected = [sort cmp_volinfo_by_volid $tt->{expected}->@*];
         my $description = $tt->{description};
         my $parent = $tt->{parent};
 
         # prepare environment
         my $num = 0; #parent disks
         for my $file (@$files) {
-            my ($name, $dir, $suffix) = fileparse($file, @suffixes);
+            my ($name, $dir, $suffix) = fileparse($file, @BACKING_FILE_SUFFIXES);
 
             make_path($dir, { verbose => 1, mode => 0755 });
 
             if ($name) {
                 # using qemu-img to also be able to represent the backing device
@@ -812,10 +791,41 @@ plan tests => $plan + 1;
 
         # clean up after each test case, otherwise
         # we get wrong results from leftover files
         remove_tree($storage_dir, { verbose => 1 });
     }
+
+    return;
 }
 
-done_testing();
+sub main() {
+    my $plan = scalar @tests;
+    plan tests => $plan + 1;
+
+    {
+        # don't accidentally modify vmlist, see bug report
+        # https://pve.proxmox.com/pipermail/pve-devel/2020-January/041096.html
+        my $scfg_with_type = { path => $storage_dir, type => 'dir' };
+        my $original_vmlist = { ids => {} };
+        my $tested_vmlist = dclone($original_vmlist);
+
+        PVE::Storage::Plugin->list_volumes('sid', $scfg_with_type, undef, ['images']);
+
+        is_deeply($tested_vmlist, $original_vmlist, 'PVE::Cluster::vmlist remains unmodified')
+            || diag(
+                "Expected vmlist to remain\n",
+                explain($original_vmlist),
+                "but it turned to\n",
+                explain($tested_vmlist),
+            );
+    }
+
+    run_legacy_tests();
+
+    done_testing();
+
+    return;
+}
+
+main();
 
 1;
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 30/51] test: list volumes: fix broken test checking for vmlist modifications
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (28 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 29/51] test: list volumes: reorganize and modernize test running code Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 31/51] test: list volumes: introduce new format for test cases Max R. Carrara
                   ` (14 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

This test never actually did anything, since it didn't use the mocked
vmlist at all. Therefore, fix it by comparing the vmlist with its
original after all tests have run.

Verified that this test catches modifications to the vmlist by
manually adding a line that modifies it in the `list_volumes()`
method, running the tests to see if this fails, and then removing that
line again.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/list_volumes_test.pm | 27 ++++++++++-----------------
 1 file changed, 10 insertions(+), 17 deletions(-)

diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm
index 176b5a46..d72212bd 100644
--- a/src/test/list_volumes_test.pm
+++ b/src/test/list_volumes_test.pm
@@ -799,30 +799,23 @@ my sub run_legacy_tests() {
 
 sub main() {
     my $plan = scalar @tests;
     plan tests => $plan + 1;
 
-    {
-        # don't accidentally modify vmlist, see bug report
-        # https://pve.proxmox.com/pipermail/pve-devel/2020-January/041096.html
-        my $scfg_with_type = { path => $storage_dir, type => 'dir' };
-        my $original_vmlist = { ids => {} };
-        my $tested_vmlist = dclone($original_vmlist);
-
-        PVE::Storage::Plugin->list_volumes('sid', $scfg_with_type, undef, ['images']);
-
-        is_deeply($tested_vmlist, $original_vmlist, 'PVE::Cluster::vmlist remains unmodified')
-            || diag(
-                "Expected vmlist to remain\n",
-                explain($original_vmlist),
-                "but it turned to\n",
-                explain($tested_vmlist),
-            );
-    }
+    # Keep the original vmlist around in order to check whether it was modified
+    # after running all the tests. See:
+    # https://pve.proxmox.com/pipermail/pve-devel/2020-January/041096.html
+    my $original_vmlist = dclone(PVE::Cluster::get_vmlist());
 
     run_legacy_tests();
 
+    my $vmlist = PVE::Cluster::get_vmlist();
+
+    is_deeply(
+        $vmlist, $original_vmlist, 'Result of PVE::Cluster::get_vmlist remains unmodified',
+    );
+
     done_testing();
 
     return;
 }
 
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 31/51] test: list volumes: introduce new format for test cases
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (29 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 30/51] test: list volumes: fix broken test checking for vmlist modifications Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 32/51] test: list volumes: remove legacy code and migrate cases to new format Max R. Carrara
                   ` (13 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Introduce a new format for test cases in `list_volumes_test.pm`, which
allows specifying the `$storeid`, `$scfg` and the list of
content types / volume types for the given test.

At the same time, get rid of the `files` and `expected` lists in the
new format, which made it somewhat tedious to keep track of which file
belonged to which hashref in the expected output, or if it even was
supposed to show up in the output in the first place.

Get rid of the optional `parent` list for the same reason, too.

Replace these three lists with a single `cases` list that simply
contains hashes that match a given file path with the expected hash in
the final output. The `parent` may optionally be specified in the same
hash for a given file path as well, which makes the backing file logic
a little simpler.

Describe the new format in a small POD docstring.

Additionally, introduce replacements for the constants declared with
the `use constant` pragma.

Finally, migrate the first of the old test cases over to the new
format and run it through the new machinery added for the new test
case format.

The reason for introducing this new format here is to make it easier
to add more specific test cases. The "legacy" tests always pass all
volume types as parameter to `PVE::Storage::Plugin->list_volumes()`,
for example.

As an additional benefit, we could reuse / recycle the new testing
machinery added here for testing other storage (plugin) API methods as
well.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/list_volumes_test.pm | 399 +++++++++++++++++++++++++---------
 1 file changed, 300 insertions(+), 99 deletions(-)

diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm
index d72212bd..1a4bae7c 100644
--- a/src/test/list_volumes_test.pm
+++ b/src/test/list_volumes_test.pm
@@ -9,21 +9,26 @@ use PVE::Cluster;
 use PVE::Tools qw(run_command);
 
 use Test::More;
 use Test::MockModule;
 
+use Carp qw(confess);
 use Cwd;
 use File::Basename qw(fileparse);
 use File::Path qw(make_path remove_tree);
 use File::stat qw();
 use File::Temp;
 use Storable qw(dclone);
 
 use constant DEFAULT_SIZE => 131072; # 128 kiB
 use constant DEFAULT_USED => 262144; # 256 kiB
 use constant DEFAULT_CTIME => 1234567890;
 
+my $DEFAULT_SIZE = 128 * 1024; # 128 kiB
+my $DEFAULT_USED = 256 * 1024; # 256 kiB
+my $DEFAULT_CTIME = 1234567890;
+
 # get_vmlist() return values
 my $mocked_vmlist = {
     'version' => 1,
     'ids' => {
         '16110' => {
@@ -73,116 +78,36 @@ my $scfg = {
         'backup' => 1,
         'import' => 1,
     },
 };
 
+my $DEFAULT_STOREID = 'local';
+my $DEFAULT_STORAGE_PATH = File::Temp->newdir();
+my $DEFAULT_SCFG = {
+    type => 'dir',
+    path => $DEFAULT_STORAGE_PATH,
+    shared => 0,
+    content => {
+        iso => 1,
+        rootdir => 1,
+        vztmpl => 1,
+        images => 1,
+        snippets => 1,
+        backup => 1,
+        import => 1,
+    },
+};
+
 my @BACKING_FILE_SUFFIXES = ('qcow2', 'raw', 'vmdk', 'vhdx');
 
 # The test cases are comprised of an arry of hashes with the following keys:
 # description => displayed on error by Test::More
 # vmid        => used for image matches by list_volume
 # files       => array of files for qemu-img to create
 # expected    => returned result hash
 #                (content, ctime, format, parent, size, used, vimd, volid)
 my @tests = (
-    {
-        description => 'VMID: 16110, VM, qcow2, backup, snippets',
-        vmid => '16110',
-        files => [
-            "$storage_dir/images/16110/vm-16110-disk-0.qcow2",
-            "$storage_dir/images/16110/vm-16110-disk-1.raw",
-            "$storage_dir/images/16110/vm-16110-disk-2.vmdk",
-            "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz",
-            "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo",
-            "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma",
-            "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst",
-            "$storage_dir/snippets/userconfig.yaml",
-            "$storage_dir/snippets/hookscript.pl",
-        ],
-        expected => [
-            {
-                'content' => 'images',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'qcow2',
-                'parent' => undef,
-                'size' => DEFAULT_SIZE,
-                'used' => DEFAULT_USED,
-                'vmid' => '16110',
-                'volid' => 'local:16110/vm-16110-disk-0.qcow2',
-            },
-            {
-                'content' => 'images',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'raw',
-                'parent' => undef,
-                'size' => DEFAULT_SIZE,
-                'used' => DEFAULT_USED,
-                'vmid' => '16110',
-                'volid' => 'local:16110/vm-16110-disk-1.raw',
-            },
-            {
-                'content' => 'images',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'vmdk',
-                'parent' => undef,
-                'size' => DEFAULT_SIZE,
-                'used' => DEFAULT_USED,
-                'vmid' => '16110',
-                'volid' => 'local:16110/vm-16110-disk-2.vmdk',
-            },
-            {
-                'content' => 'backup',
-                'ctime' => 1585602700,
-                'format' => 'vma.gz',
-                'size' => DEFAULT_SIZE,
-                'subtype' => 'qemu',
-                'vmid' => '16110',
-                'volid' => 'local:backup/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz',
-            },
-            {
-                'content' => 'backup',
-                'ctime' => 1585602765,
-                'format' => 'vma.lzo',
-                'size' => DEFAULT_SIZE,
-                'subtype' => 'qemu',
-                'vmid' => '16110',
-                'volid' => 'local:backup/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
-            },
-            {
-                'content' => 'backup',
-                'ctime' => 1585602835,
-                'format' => 'vma',
-                'size' => DEFAULT_SIZE,
-                'subtype' => 'qemu',
-                'vmid' => '16110',
-                'volid' => 'local:backup/vzdump-qemu-16110-2020_03_30-21_13_55.vma',
-            },
-            {
-                'content' => 'backup',
-                'ctime' => 1585602835,
-                'format' => 'vma.zst',
-                'size' => DEFAULT_SIZE,
-                'subtype' => 'qemu',
-                'vmid' => '16110',
-                'volid' => 'local:backup/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst',
-            },
-            {
-                'content' => 'snippets',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'snippet',
-                'size' => DEFAULT_SIZE,
-                'volid' => 'local:snippets/hookscript.pl',
-            },
-            {
-                'content' => 'snippets',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'snippet',
-                'size' => DEFAULT_SIZE,
-                'volid' => 'local:snippets/userconfig.yaml',
-            },
-        ],
-    },
     {
         description => 'VMID: 16112, lxc, raw, backup',
         vmid => '16112',
         files => [
             "$storage_dir/images/16112/vm-16112-disk-0.raw",
@@ -711,10 +636,193 @@ my @tests = (
             },
         ],
     },
 );
 
+=head2 TEST CASE FORMAT
+
+The parameters for individual test cases are hashes with the following
+keys:
+
+    {
+        # Name of the test, also displayed on error.
+        description => '...',
+
+        # The storage ID for the test.
+        # Used as parameter when calling list_volumes.
+        storeid => 'some-storeid',
+
+        # The storage config hash for the test.
+        # Used as parameter when calling list_volumes.
+        scfg => {
+            type => 'dir',
+            path => '...',
+            shared => 0,
+            content => {
+                iso => 1,
+                # [...]
+            },
+            # [...]
+        },
+
+        # The VMID used as parameter when calling list_volumes.
+        # May be undef.
+        vmid => 1234,
+
+        # The list of volume types / content types used as parameter when
+        # calling list_volumes.
+        vtypes => [],
+
+        # List of hashes that associate a file with its expected hash in the
+        # output of list_volumes.
+        cases => [
+            {
+                # Absolute path to the file.
+                file => "/tmp/foobar/images/1234/vm-1234-disk-0.qcow2",
+
+                # Relative path to parent (backing file).
+                parent => '../ssss/base-4321-disk-0.qcow2',
+
+                # Expected hash in the output of list_volumes.
+                expected => {
+                    content => 'images',
+                    ctime => 1234567890, # inode change time in seconds since the epoc
+                    format => 'qcow2',
+                    parent => '../ssss/base-4321-disk-0.qcow2',
+                    size => 1337, # bytes!
+                    used => 1024, # bytes!
+                    vmid => 1234,
+                    volid => 'local:1234/vm-1234-disk-0.qcow2',
+                },
+            },
+        ],
+    }
+
+=cut
+
+my $test_param_list = [
+    {
+        description => 'VMID: 16110, VM, qcow2, backup, snippets',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => 16110,
+        vtypes => ['images', 'backup', 'snippets'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/16110/vm-16110-disk-0.qcow2",
+                expected => {
+                    content => 'images',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'qcow2',
+                    parent => undef,
+                    size => $DEFAULT_SIZE,
+                    used => $DEFAULT_USED,
+                    vmid => '16110',
+                    volid => 'local:16110/vm-16110-disk-0.qcow2',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/16110/vm-16110-disk-1.raw",
+                expected => {
+                    content => 'images',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'raw',
+                    parent => undef,
+                    size => $DEFAULT_SIZE,
+                    used => $DEFAULT_USED,
+                    vmid => '16110',
+                    volid => 'local:16110/vm-16110-disk-1.raw',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/16110/vm-16110-disk-2.vmdk",
+                expected => {
+                    content => 'images',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'vmdk',
+                    parent => undef,
+                    size => $DEFAULT_SIZE,
+                    used => $DEFAULT_USED,
+                    vmid => '16110',
+                    volid => 'local:16110/vm-16110-disk-2.vmdk',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585602700,
+                    format => 'vma.gz',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'qemu',
+                    vmid => '16110',
+                    volid => 'local:backup/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585602765,
+                    format => 'vma.lzo',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'qemu',
+                    vmid => '16110',
+                    volid => 'local:backup/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585602835,
+                    format => 'vma',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'qemu',
+                    vmid => '16110',
+                    volid => 'local:backup/vzdump-qemu-16110-2020_03_30-21_13_55.vma',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585602835,
+                    format => 'vma.zst',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'qemu',
+                    vmid => '16110',
+                    volid => 'local:backup/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/hookscript.pl",
+                expected => {
+                    content => 'snippets',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'snippet',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:snippets/hookscript.pl',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/userconfig.yaml",
+                expected => {
+                    content => 'snippets',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'snippet',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:snippets/userconfig.yaml',
+                },
+            },
+        ],
+    },
+];
+
 # provide static vmlist for tests
 my $mock_cluster = Test::MockModule->new('PVE::Cluster', no_auto => 1);
 $mock_cluster->redefine(get_vmlist => sub { return $mocked_vmlist; });
 
 # populate is File::stat's method to fill all information from CORE::stat into
@@ -795,19 +903,112 @@ my sub run_legacy_tests() {
     }
 
     return;
 }
 
+my sub setup_test_env($test_params) {
+    my ($storeid, $scfg) = $test_params->@{qw(storeid scfg)};
+
+    make_path($scfg->{path}, { verbose => 1, mode => 0700 });
+
+    for my $case ($test_params->{cases}->@*) {
+        my ($file, $parent) = $case->@{qw(file parent)};
+
+        confess "\$file = undef" if !defined($file);
+
+        my ($file_name, $directory, $suffix) = fileparse($file, @BACKING_FILE_SUFFIXES);
+
+        make_path($directory, { verbose => 1, mode => 0755 });
+
+        if ($file_name) {
+            # using qemu-img to also be able to represent the backing device
+            my $command = ['/usr/bin/qemu-img', 'create', "$file", $DEFAULT_SIZE];
+
+            push($command->@*, '-f', $suffix) if $suffix;
+            push($command->@*, '-u', '-b', $parent) if $parent;
+            push($command->@*, '-F', $suffix) if $parent && $suffix;
+
+            run_command($command, timeout => 10);
+        }
+    }
+
+    return;
+}
+
+my sub teardown_test_env($test_params) {
+    my $scfg = $test_params->{scfg};
+
+    remove_tree($scfg->{path}, { verbose => 1 });
+
+    return;
+}
+
+my sub run_test_for_params($test_params) {
+    eval { setup_test_env($test_params); };
+    if ($@) {
+        teardown_test_env($test_params);
+
+        fail($test_params->{description});
+        diag("Failed to set up test environment: $@");
+
+        return;
+    }
+
+    my ($storeid, $scfg, $vmid, $vtypes) = $test_params->@{qw(storeid scfg vmid vtypes)};
+
+    my $expected = [map { $_->{expected} // () } $test_params->{cases}->@*];
+    $expected = [sort cmp_volinfo_by_volid $expected->@*];
+
+    my $got = eval {
+        my $volume_list = PVE::Storage::Plugin->list_volumes($storeid, $scfg, $vmid, $vtypes);
+        return [sort cmp_volinfo_by_volid $volume_list->@*];
+    };
+    $got = $@ if $@;
+
+    my $desc_vmid = defined($vmid) ? $vmid : 'undef';
+    my $desc_vtypes = '[' . join(', ', $vtypes->@*) . ']';
+
+    my $description =
+        $test_params->{description} . " - \$vmid = $desc_vmid, \$vtypes = $desc_vtypes";
+
+    if (!is_deeply($got, $expected, $description)) {
+        diag("\$got = ", explain($got));
+        diag("\$expected = ", explain($expected));
+    }
+
+    teardown_test_env($test_params);
+
+    return;
+}
+
+my sub assert_test_params_keys_exist($test_params) {
+    my $desc = $test_params->{description};
+
+    confess "Test parameters without description"
+        if !$desc;
+
+    for my $key (qw(storeid scfg vmid vtypes cases)) {
+        confess "Key '$key' does not exist in '$desc'"
+            if !exists($test_params->{$key});
+    }
+
+    return;
+}
+
 sub main() {
-    my $plan = scalar @tests;
-    plan tests => $plan + 1;
+    plan tests => scalar(@tests) + scalar($test_param_list->@*) + 1;
 
     # Keep the original vmlist around in order to check whether it was modified
     # after running all the tests. See:
     # https://pve.proxmox.com/pipermail/pve-devel/2020-January/041096.html
     my $original_vmlist = dclone(PVE::Cluster::get_vmlist());
 
+    for my $test_params ($test_param_list->@*) {
+        assert_test_params_keys_exist($test_params);
+        run_test_for_params($test_params);
+    }
+
     run_legacy_tests();
 
     my $vmlist = PVE::Cluster::get_vmlist();
 
     is_deeply(
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 32/51] test: list volumes: remove legacy code and migrate cases to new format
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (30 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 31/51] test: list volumes: introduce new format for test cases Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 33/51] plugin: do not return volumes of undeclared content types anymore Max R. Carrara
                   ` (12 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Remove all that remains of the old testing code, including the
constants defined with the `use constant` pragma and other old
variables.

Migrate all remaining test cases over to the new format.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/list_volumes_test.pm | 1390 ++++++++++++++++++---------------
 1 file changed, 777 insertions(+), 613 deletions(-)

diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm
index 1a4bae7c..65fc7629 100644
--- a/src/test/list_volumes_test.pm
+++ b/src/test/list_volumes_test.pm
@@ -17,14 +17,10 @@ use File::Basename qw(fileparse);
 use File::Path qw(make_path remove_tree);
 use File::stat qw();
 use File::Temp;
 use Storable qw(dclone);
 
-use constant DEFAULT_SIZE => 131072; # 128 kiB
-use constant DEFAULT_USED => 262144; # 256 kiB
-use constant DEFAULT_CTIME => 1234567890;
-
 my $DEFAULT_SIZE = 128 * 1024; # 128 kiB
 my $DEFAULT_USED = 256 * 1024; # 256 kiB
 my $DEFAULT_CTIME = 1234567890;
 
 # get_vmlist() return values
@@ -62,26 +58,10 @@ my $mocked_vmlist = {
             'version' => 6,
         },
     },
 };
 
-my $storage_dir = File::Temp->newdir();
-my $scfg = {
-    'type' => 'dir',
-    'path' => $storage_dir,
-    'shared' => 0,
-    'content' => {
-        'iso' => 1,
-        'rootdir' => 1,
-        'vztmpl' => 1,
-        'images' => 1,
-        'snippets' => 1,
-        'backup' => 1,
-        'import' => 1,
-    },
-};
-
 my $DEFAULT_STOREID = 'local';
 my $DEFAULT_STORAGE_PATH = File::Temp->newdir();
 my $DEFAULT_SCFG = {
     type => 'dir',
     path => $DEFAULT_STORAGE_PATH,
@@ -97,549 +77,10 @@ my $DEFAULT_SCFG = {
     },
 };
 
 my @BACKING_FILE_SUFFIXES = ('qcow2', 'raw', 'vmdk', 'vhdx');
 
-# The test cases are comprised of an arry of hashes with the following keys:
-# description => displayed on error by Test::More
-# vmid        => used for image matches by list_volume
-# files       => array of files for qemu-img to create
-# expected    => returned result hash
-#                (content, ctime, format, parent, size, used, vimd, volid)
-my @tests = (
-    {
-        description => 'VMID: 16112, lxc, raw, backup',
-        vmid => '16112',
-        files => [
-            "$storage_dir/images/16112/vm-16112-disk-0.raw",
-            "$storage_dir/dump/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo",
-            "$storage_dir/dump/vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz",
-            "$storage_dir/dump/vzdump-lxc-16112-2020_03_30-21_49_30.tar.zst",
-            "$storage_dir/dump/vzdump-lxc-16112-2020_03_30-21_59_30.tgz",
-            "$storage_dir/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2",
-        ],
-        expected => [
-            {
-                'content' => 'rootdir',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'raw',
-                'parent' => undef,
-                'size' => DEFAULT_SIZE,
-                'used' => DEFAULT_USED,
-                'vmid' => '16112',
-                'volid' => 'local:16112/vm-16112-disk-0.raw',
-            },
-            {
-                'content' => 'backup',
-                'ctime' => 1585604370,
-                'format' => 'tar.lzo',
-                'size' => DEFAULT_SIZE,
-                'subtype' => 'lxc',
-                'vmid' => '16112',
-                'volid' => 'local:backup/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo',
-            },
-            {
-                'content' => 'backup',
-                'ctime' => 1585604970,
-                'format' => 'tar.gz',
-                'size' => DEFAULT_SIZE,
-                'subtype' => 'lxc',
-                'vmid' => '16112',
-                'volid' => 'local:backup/vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz',
-            },
-            {
-                'content' => 'backup',
-                'ctime' => 1585604970,
-                'format' => 'tar.zst',
-                'size' => DEFAULT_SIZE,
-                'subtype' => 'lxc',
-                'vmid' => '16112',
-                'volid' => 'local:backup/vzdump-lxc-16112-2020_03_30-21_49_30.tar.zst',
-            },
-            {
-                'content' => 'backup',
-                'ctime' => 1585605570,
-                'format' => 'tgz',
-                'size' => DEFAULT_SIZE,
-                'subtype' => 'lxc',
-                'vmid' => '16112',
-                'volid' => 'local:backup/vzdump-lxc-16112-2020_03_30-21_59_30.tgz',
-            },
-            {
-                'content' => 'backup',
-                'ctime' => 1585604370,
-                'format' => 'tar.bz2',
-                'size' => DEFAULT_SIZE,
-                'subtype' => 'openvz',
-                'vmid' => '16112',
-                'volid' => 'local:backup/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2',
-            },
-        ],
-    },
-    {
-        description => 'VMID: 16116, no guest, subvol and raw',
-        vmid => '16116',
-        files => [
-            "$storage_dir/images/16116/subvol-16116-disk-0.subvol/",
-            "$storage_dir/images/16116/vm-16116-disk-1.raw",
-        ],
-        expected => [
-            {
-                'content' => 'rootdir',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'subvol',
-                'parent' => undef,
-                'size' => DEFAULT_SIZE,
-                'used' => DEFAULT_USED,
-                'vmid' => '16116',
-                'volid' => 'local:16116/subvol-16116-disk-0.subvol',
-            },
-            {
-                'content' => 'images',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'raw',
-                'parent' => undef,
-                'size' => DEFAULT_SIZE,
-                'used' => DEFAULT_USED,
-                'vmid' => '16116',
-                'volid' => 'local:16116/vm-16116-disk-1.raw',
-            },
-        ],
-    },
-    {
-        description => 'VMID: 16114, VM, qcow2, linked clone',
-        vmid => '16114',
-        files => [
-            "$storage_dir/images/16114/vm-16114-disk-0.qcow2",
-            "$storage_dir/images/16114/vm-16114-disk-1.qcow2",
-        ],
-        parent => [
-            "../9004/base-9004-disk-0.qcow2", "../9004/base-9004-disk-1.qcow2",
-        ],
-        expected => [
-            {
-                'content' => 'images',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'qcow2',
-                'parent' => '../9004/base-9004-disk-0.qcow2',
-                'size' => DEFAULT_SIZE,
-                'used' => DEFAULT_USED,
-                'vmid' => '16114',
-                'volid' => 'local:9004/base-9004-disk-0.qcow2/16114/vm-16114-disk-0.qcow2',
-            },
-            {
-                'content' => 'images',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'qcow2',
-                'parent' => '../9004/base-9004-disk-1.qcow2',
-                'size' => DEFAULT_SIZE,
-                'used' => DEFAULT_USED,
-                'vmid' => '16114',
-                'volid' => 'local:9004/base-9004-disk-1.qcow2/16114/vm-16114-disk-1.qcow2',
-            },
-        ],
-    },
-    {
-        description => 'VMID: 16110, renamed backups',
-        vmid => '16110',
-        files => [
-            "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz",
-            "$storage_dir/dump/vzdump-qemu-16110-before-upgrade.vma.zst",
-            "$storage_dir/dump/vzdump-qemu-16111-before-16110-upgrade.vma.zst",
-            "$storage_dir/dump/vzdump-16110-renamed.tar.zst",
-            "$storage_dir/dump/backup-16110-manual.vma",
-        ],
-        expected => [
-            {
-                'content' => 'backup',
-                'ctime' => 1585602700,
-                'format' => 'vma.gz',
-                'size' => DEFAULT_SIZE,
-                'subtype' => 'qemu',
-                'vmid' => '16110',
-                'volid' => 'local:backup/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz',
-            },
-            {
-                'content' => 'backup',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'vma.zst',
-                'size' => DEFAULT_SIZE,
-                'subtype' => 'qemu',
-                'vmid' => '16110',
-                'volid' => 'local:backup/vzdump-qemu-16110-before-upgrade.vma.zst',
-            },
-        ],
-    },
-    {
-        description => 'VMID: 9004, VM, template, qcow2',
-        vmid => '9004',
-        files => [
-            "$storage_dir/images/9004/base-9004-disk-0.qcow2",
-            "$storage_dir/images/9004/base-9004-disk-1.qcow2",
-        ],
-        expected => [
-            {
-                'content' => 'images',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'qcow2',
-                'parent' => undef,
-                'size' => DEFAULT_SIZE,
-                'used' => DEFAULT_USED,
-                'vmid' => '9004',
-                'volid' => 'local:9004/base-9004-disk-0.qcow2',
-            },
-            {
-                'content' => 'images',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'qcow2',
-                'parent' => undef,
-                'size' => DEFAULT_SIZE,
-                'used' => DEFAULT_USED,
-                'vmid' => '9004',
-                'volid' => 'local:9004/base-9004-disk-1.qcow2',
-            },
-        ],
-    },
-    {
-        description => 'VMID: none, templates, snippets, backup',
-        vmid => undef,
-        files => [
-            "$storage_dir/dump/vzdump-lxc-19253-2020_02_03-19_57_43.tar.gz",
-            "$storage_dir/dump/vzdump-lxc-19254-2019_01_21-19_29_19.tar",
-            "$storage_dir/template/iso/archlinux-2020.02.01-x86_64.iso",
-            "$storage_dir/template/iso/debian-8.11.1-amd64-DVD-1.iso",
-            "$storage_dir/template/iso/debian-9.12.0-amd64-netinst.iso",
-            "$storage_dir/template/iso/proxmox-ve_6.1-1.iso",
-            "$storage_dir/template/cache/archlinux-base_20190924-1_amd64.tar.gz",
-            "$storage_dir/template/cache/debian-10.0-standard_10.0-1_amd64.tar.gz",
-            "$storage_dir/template/cache/debian-11.0-standard_11.0-1_amd64.tar.bz2",
-            "$storage_dir/template/cache/alpine-3.10-default_20190626_amd64.tar.xz",
-            "$storage_dir/snippets/userconfig.yaml",
-            "$storage_dir/snippets/hookscript.pl",
-            "$storage_dir/private/1234/", # fileparse needs / at the end
-            "$storage_dir/private/1234/subvol-1234-disk-0.subvol/", # fileparse needs / at the end
-        ],
-        expected => [
-            {
-                'content' => 'vztmpl',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'txz',
-                'size' => DEFAULT_SIZE,
-                'volid' => 'local:vztmpl/alpine-3.10-default_20190626_amd64.tar.xz',
-            },
-            {
-                'content' => 'vztmpl',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'tgz',
-                'size' => DEFAULT_SIZE,
-                'volid' => 'local:vztmpl/archlinux-base_20190924-1_amd64.tar.gz',
-            },
-            {
-                'content' => 'vztmpl',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'tgz',
-                'size' => DEFAULT_SIZE,
-                'volid' => 'local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz',
-            },
-            {
-                'content' => 'vztmpl',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'tbz2',
-                'size' => DEFAULT_SIZE,
-                'volid' => 'local:vztmpl/debian-11.0-standard_11.0-1_amd64.tar.bz2',
-            },
-            {
-                'content' => 'iso',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'iso',
-                'size' => DEFAULT_SIZE,
-                'volid' => 'local:iso/archlinux-2020.02.01-x86_64.iso',
-            },
-            {
-                'content' => 'iso',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'iso',
-                'size' => DEFAULT_SIZE,
-                'volid' => 'local:iso/debian-8.11.1-amd64-DVD-1.iso',
-            },
-            {
-                'content' => 'iso',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'iso',
-                'size' => DEFAULT_SIZE,
-                'volid' => 'local:iso/debian-9.12.0-amd64-netinst.iso',
-            },
-            {
-                'content' => 'iso',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'iso',
-                'size' => DEFAULT_SIZE,
-                'volid' => 'local:iso/proxmox-ve_6.1-1.iso',
-            },
-            {
-                'content' => 'backup',
-                'ctime' => 1580759863,
-                'format' => 'tar.gz',
-                'size' => DEFAULT_SIZE,
-                'subtype' => 'lxc',
-                'vmid' => '19253',
-                'volid' => 'local:backup/vzdump-lxc-19253-2020_02_03-19_57_43.tar.gz',
-            },
-            {
-                'content' => 'backup',
-                'ctime' => 1548098959,
-                'format' => 'tar',
-                'size' => DEFAULT_SIZE,
-                'subtype' => 'lxc',
-                'vmid' => '19254',
-                'volid' => 'local:backup/vzdump-lxc-19254-2019_01_21-19_29_19.tar',
-            },
-            {
-                'content' => 'snippets',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'snippet',
-                'size' => DEFAULT_SIZE,
-                'volid' => 'local:snippets/hookscript.pl',
-            },
-            {
-                'content' => 'snippets',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'snippet',
-                'size' => DEFAULT_SIZE,
-                'volid' => 'local:snippets/userconfig.yaml',
-            },
-        ],
-    },
-    {
-        description => 'VMID: none, parent, non-matching',
-        # string instead of vmid in folder
-        #"$storage_dir/images/ssss/base-4321-disk-0.qcow2/1234/vm-1234-disk-0.qcow2",
-        vmid => undef,
-        files => [
-            "$storage_dir/images/1234/vm-1234-disk-0.qcow2",
-        ],
-        parent => [
-            "../ssss/base-4321-disk-0.qcow2",
-        ],
-        expected => [
-            {
-                'content' => 'images',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'qcow2',
-                'parent' => '../ssss/base-4321-disk-0.qcow2',
-                'size' => DEFAULT_SIZE,
-                'used' => DEFAULT_USED,
-                'vmid' => '1234',
-                'volid' => 'local:1234/vm-1234-disk-0.qcow2',
-            },
-        ],
-    },
-    {
-        description => 'VMID: none, non-matching',
-        # failed matches
-        vmid => undef,
-        files => [
-            "$storage_dir/images/ssss/base-4321-disk-0.raw",
-            "$storage_dir/images/ssss/vm-1234-disk-0.qcow2",
-            "$storage_dir/template/iso/yet-again-a-installation-disk.dvd",
-            "$storage_dir/template/cache/debian-10.0-standard_10.0-1_amd64.zip.gz",
-            "$storage_dir/private/subvol-19254-disk-0/19254",
-            "$storage_dir/dump/vzdump-openvz-16112-2020_03_30-21_39_30.zip.gz",
-            "$storage_dir/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tgz.lzo",
-            "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz",
-            "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vms.gz",
-        ],
-        expected => [], # returns empty list
-    },
-    {
-        description => 'VMID: none, names with a newline',
-        vmid => undef,
-        files => [
-            "$storage_dir/template/iso/debian-9.12.0-amd64-netinst.iso",
-            "$storage_dir/template/iso/debian-9.12.0\namd64-netinst.iso",
-            "$storage_dir/template/iso/debian-9.12.0-amd64-netinst.iso\n",
-            "$storage_dir/dump/vzdump-qemu-16110-before\nupgrade.vma",
-            "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.gz\n",
-            "$storage_dir/snippets/hook\nscript.pl",
-        ],
-        expected => [
-            {
-                'content' => 'iso',
-                'ctime' => DEFAULT_CTIME,
-                'format' => 'iso',
-                'size' => DEFAULT_SIZE,
-                'volid' => 'local:iso/debian-9.12.0-amd64-netinst.iso',
-            },
-        ],
-    },
-    {
-        description => 'VMID: none, valid file names for import',
-        vmid => undef,
-        files => [
-            "$storage_dir/import/import.ova",
-            "$storage_dir/import/import.ovf",
-            "$storage_dir/import/some-disk.qcow2",
-            "$storage_dir/import/some-disk.vmdk",
-            "$storage_dir/import/some-raw-disk.raw",
-        ],
-        expected => [
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'ova',
-                size => DEFAULT_SIZE,
-                volid => "local:import/import.ova",
-            },
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'ovf',
-                size => DEFAULT_SIZE,
-                volid => "local:import/import.ovf",
-            },
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'qcow2',
-                size => DEFAULT_SIZE,
-                volid => "local:import/some-disk.qcow2",
-            },
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'vmdk',
-                size => DEFAULT_SIZE,
-                volid => "local:import/some-disk.vmdk",
-            },
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'raw',
-                size => DEFAULT_SIZE,
-                volid => "local:import/some-raw-disk.raw",
-            },
-        ],
-    },
-    {
-        description => 'VMID: none, non-matching file paths for import',
-        vmid => undef,
-        files => [
-            # Malformed file names
-            "$storage_dir/import/import.ovff",
-            "$storage_dir/import/importova",
-            "$storage_dir/import/import.ov",
-            "$storage_dir/import/diskraw",
-            "$storage_dir/import/diskvmdk",
-            "$storage_dir/import/disk.invalid",
-            "$storage_dir/import/.ova",
-            "$storage_dir/import/.raw",
-            # Trailing whitespace must not be trimmed
-            "$storage_dir/import/import.ova\t",
-            "$storage_dir/import/disk.raw    ",
-            # Whitespace in file name
-            "$storage_dir/import/something I want to import.ova",
-            "$storage_dir/import/ .raw",
-            "$storage_dir/import/ disk .vmdk",
-            "$storage_dir/import/disk .qcow2",
-            "$storage_dir/import/ import.ova",
-            # Unsafe characters in file name
-            "$storage_dir/import/linux🐧-vm.ova",
-            "$storage_dir/import/🐪perl-playground🐪.ova",
-            "$storage_dir/import/fish_<><_<><_<><.ova",
-            $storage_dir . '/import/C:\\\\Windows\\Path.ova',
-            # Content inside .ova files may only be specified as part
-            # of volume names, and may never appear when looked up as
-            # a file path
-            "$storage_dir/import/import.ova/disk.qcow2",
-            "$storage_dir/import/import.ova/disk.raw",
-            "$storage_dir/import/import.ova/disk.vmdk",
-            "$storage_dir/import/import.ova/disk.invalid",
-        ],
-        expected => [], # returns empty list
-    },
-    {
-        description => 'VMID: none, weird but valid file names for import',
-        vmid => undef,
-        files => [
-            "$storage_dir/import/import.ova.ova",
-            "$storage_dir/import/import.ova.ova.ova",
-            "$storage_dir/import/import.ova.ova.ova.ova",
-            "$storage_dir/import/ova.ova",
-            "$storage_dir/import/ova.ovf",
-            "$storage_dir/import/ova.vmdk",
-            "$storage_dir/import/raw.raw.qcow2",
-            "$storage_dir/import/raw.raw.qcow2.import.qcow2",
-            "$storage_dir/import/raw.raw.raw.your-boat.ova",
-        ],
-        expected => [
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'ova',
-                size => DEFAULT_SIZE,
-                volid => "local:import/import.ova.ova",
-            },
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'ova',
-                size => DEFAULT_SIZE,
-                volid => "local:import/import.ova.ova.ova",
-            },
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'ova',
-                size => DEFAULT_SIZE,
-                volid => "local:import/import.ova.ova.ova.ova",
-            },
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'ova',
-                size => DEFAULT_SIZE,
-                volid => "local:import/ova.ova",
-            },
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'ovf',
-                size => DEFAULT_SIZE,
-                volid => "local:import/ova.ovf",
-            },
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'vmdk',
-                size => DEFAULT_SIZE,
-                volid => "local:import/ova.vmdk",
-            },
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'qcow2',
-                size => DEFAULT_SIZE,
-                volid => "local:import/raw.raw.qcow2",
-            },
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'qcow2',
-                size => DEFAULT_SIZE,
-                volid => "local:import/raw.raw.qcow2.import.qcow2",
-            },
-            {
-                content => 'import',
-                ctime => DEFAULT_CTIME,
-                format => 'ova',
-                size => DEFAULT_SIZE,
-                volid => "local:import/raw.raw.raw.your-boat.ova",
-            },
-        ],
-    },
-);
-
 =head2 TEST CASE FORMAT
 
 The parameters for individual test cases are hashes with the following
 keys:
 
@@ -817,24 +258,796 @@ my $test_param_list = [
                     volid => 'local:snippets/userconfig.yaml',
                 },
             },
         ],
     },
+    {
+        description => 'VMID: 16112, lxc, raw, backup',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => 16112,
+        vtypes => ['rootdir', 'backup'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/16112/vm-16112-disk-0.raw",
+                expected => {
+                    content => 'rootdir',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'raw',
+                    parent => undef,
+                    size => $DEFAULT_SIZE,
+                    used => $DEFAULT_USED,
+                    vmid => '16112',
+                    volid => 'local:16112/vm-16112-disk-0.raw',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585604370,
+                    format => 'tar.lzo',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585604970,
+                    format => 'tar.gz',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-16112-2020_03_30-21_49_30.tar.zst",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585604970,
+                    format => 'tar.zst',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-lxc-16112-2020_03_30-21_49_30.tar.zst',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-16112-2020_03_30-21_59_30.tgz",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585605570,
+                    format => 'tgz',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-lxc-16112-2020_03_30-21_59_30.tgz',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585604370,
+                    format => 'tar.bz2',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'openvz',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2',
+                },
+            },
+        ],
+    },
+    {
+        description => 'VMID: 16116, no guest, subvol and raw',
+        vmid => '16116',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => 16116,
+        vtypes => ['images', 'rootdir'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/16116/subvol-16116-disk-0.subvol/",
+                expected => {
+                    content => 'rootdir',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'subvol',
+                    parent => undef,
+                    size => $DEFAULT_SIZE,
+                    used => $DEFAULT_USED,
+                    vmid => '16116',
+                    volid => 'local:16116/subvol-16116-disk-0.subvol',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/16116/vm-16116-disk-1.raw",
+                expected => {
+                    content => 'images',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'raw',
+                    parent => undef,
+                    size => $DEFAULT_SIZE,
+                    used => $DEFAULT_USED,
+                    vmid => '16116',
+                    volid => 'local:16116/vm-16116-disk-1.raw',
+                },
+            },
+        ],
+    },
+    {
+        description => 'VMID: 16114, VM, qcow2, linked clone',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => 16114,
+        vtypes => ['images'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/16114/vm-16114-disk-0.qcow2",
+                parent => '../9004/base-9004-disk-0.qcow2',
+                expected => {
+                    content => 'images',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'qcow2',
+                    parent => '../9004/base-9004-disk-0.qcow2',
+                    size => $DEFAULT_SIZE,
+                    used => $DEFAULT_USED,
+                    vmid => '16114',
+                    volid => 'local:9004/base-9004-disk-0.qcow2/16114/vm-16114-disk-0.qcow2',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/16114/vm-16114-disk-1.qcow2",
+                parent => '../9004/base-9004-disk-1.qcow2',
+                expected => {
+                    content => 'images',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'qcow2',
+                    parent => '../9004/base-9004-disk-1.qcow2',
+                    size => $DEFAULT_SIZE,
+                    used => $DEFAULT_USED,
+                    vmid => '16114',
+                    volid => 'local:9004/base-9004-disk-1.qcow2/16114/vm-16114-disk-1.qcow2',
+                },
+            },
+        ],
+    },
+    {
+        description => 'VMID: 9004, VM, template, qcow2',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => 9004,
+        vtypes => ['images'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/9004/base-9004-disk-0.qcow2",
+                expected => {
+                    content => 'images',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'qcow2',
+                    parent => undef,
+                    size => $DEFAULT_SIZE,
+                    used => $DEFAULT_USED,
+                    vmid => '9004',
+                    volid => 'local:9004/base-9004-disk-0.qcow2',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/9004/base-9004-disk-1.qcow2",
+                expected => {
+                    content => 'images',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'qcow2',
+                    parent => undef,
+                    size => $DEFAULT_SIZE,
+                    used => $DEFAULT_USED,
+                    vmid => '9004',
+                    volid => 'local:9004/base-9004-disk-1.qcow2',
+                },
+            },
+        ],
+    },
+    {
+        description => 'VMID: none, templates, snippets, backup',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => undef,
+        vtypes => ['vztmpl', 'iso', 'backup', 'snippets'],
+        cases => [
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/template/cache/alpine-3.10-default_20190626_amd64.tar.xz",
+                expected => {
+                    content => 'vztmpl',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'txz',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:vztmpl/alpine-3.10-default_20190626_amd64.tar.xz',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/template/cache/archlinux-base_20190924-1_amd64.tar.gz",
+                expected => {
+                    content => 'vztmpl',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'tgz',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:vztmpl/archlinux-base_20190924-1_amd64.tar.gz',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/template/cache/debian-10.0-standard_10.0-1_amd64.tar.gz",
+                expected => {
+                    content => 'vztmpl',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'tgz',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/template/cache/debian-11.0-standard_11.0-1_amd64.tar.bz2",
+                expected => {
+                    content => 'vztmpl',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'tbz2',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:vztmpl/debian-11.0-standard_11.0-1_amd64.tar.bz2',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/archlinux-2020.02.01-x86_64.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/archlinux-2020.02.01-x86_64.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/debian-8.11.1-amd64-DVD-1.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/debian-8.11.1-amd64-DVD-1.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/debian-9.12.0-amd64-netinst.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/debian-9.12.0-amd64-netinst.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/proxmox-ve_6.1-1.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/proxmox-ve_6.1-1.iso',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-19253-2020_02_03-19_57_43.tar.gz",
+                expected => {
+                    content => 'backup',
+                    ctime => 1580759863,
+                    format => 'tar.gz',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '19253',
+                    volid => 'local:backup/vzdump-lxc-19253-2020_02_03-19_57_43.tar.gz',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-19254-2019_01_21-19_29_19.tar",
+                expected => {
+                    content => 'backup',
+                    ctime => 1548098959,
+                    format => 'tar',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '19254',
+                    volid => 'local:backup/vzdump-lxc-19254-2019_01_21-19_29_19.tar',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/hookscript.pl",
+                expected => {
+                    content => 'snippets',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'snippet',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:snippets/hookscript.pl',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/userconfig.yaml",
+                expected => {
+                    content => 'snippets',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'snippet',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:snippets/userconfig.yaml',
+                },
+            },
+            {
+                # fileparse needs / at the end
+                file => "$DEFAULT_STORAGE_PATH/private/1234/",
+                expected => undef,
+            },
+            {
+                # fileparse needs / at the end
+                file => "$DEFAULT_STORAGE_PATH/private/1234/subvol-1234-disk-0.subvol/",
+                expected => undef,
+            },
+        ],
+    },
+    {
+        description => 'VMID: 16110, renamed backups',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => 16110,
+        vtypes => [sort keys $DEFAULT_SCFG->{content}->%*],
+        cases => [
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585602700,
+                    format => 'vma.gz',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'qemu',
+                    vmid => '16110',
+                    volid => 'local:backup/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-before-upgrade.vma.zst",
+                expected => {
+                    content => 'backup',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'vma.zst',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'qemu',
+                    vmid => '16110',
+                    volid => 'local:backup/vzdump-qemu-16110-before-upgrade.vma.zst',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16111-before-16110-upgrade.vma.zst",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/vzdump-16110-renamed.tar.zst",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/backup-16110-manual.vma",
+                expected => undef,
+            },
+        ],
+    },
+    {
+        description => 'VMID: none, parent, non-matching',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => undef,
+        vtypes => ['images'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/1234/vm-1234-disk-0.qcow2",
+                parent => '../ssss/base-4321-disk-0.qcow2',
+                expected => {
+                    content => 'images',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'qcow2',
+                    parent => '../ssss/base-4321-disk-0.qcow2',
+                    size => $DEFAULT_SIZE,
+                    used => $DEFAULT_USED,
+                    vmid => '1234',
+                    volid => 'local:1234/vm-1234-disk-0.qcow2',
+                },
+            },
+        ],
+    },
+    {
+        description => 'VMID: none, non-matching',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => undef,
+        vtypes => [sort keys $DEFAULT_SCFG->{content}->%*],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/ssss/base-4321-disk-0.raw",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/ssss/vm-1234-disk-0.qcow2",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/yet-again-a-installation-disk.dvd",
+                expected => undef,
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/template/cache/debian-10.0-standard_10.0-1_amd64.zip.gz",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/private/subvol-19254-disk-0/19254",
+                expected => undef,
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-openvz-16112-2020_03_30-21_39_30.zip.gz",
+                expected => undef,
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tgz.lzo",
+                expected => undef,
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz",
+                expected => undef,
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vms.gz",
+                expected => undef,
+            },
+        ],
+    },
+    {
+        description => 'VMID: none, names with a newline',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => undef,
+        vtypes => [sort keys $DEFAULT_SCFG->{content}->%*],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/debian-9.12.0-amd64-netinst.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/debian-9.12.0-amd64-netinst.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/debian-9.12.0\namd64-netinst.iso",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/debian-9.12.0-amd64-netinst.iso\n",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-before\nupgrade.vma",
+                expected => undef,
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.gz\n",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/hook\nscript.pl",
+                expected => undef,
+            },
+        ],
+    },
+    {
+        description => 'VMID: none, valid file names for import',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => undef,
+        vtypes => ['import'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/import.ova",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'ova',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/import.ova',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/import.ovf",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'ovf',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/import.ovf',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/some-disk.qcow2",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'qcow2',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/some-disk.qcow2',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/some-disk.vmdk",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'vmdk',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/some-disk.vmdk',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/some-raw-disk.raw",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'raw',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/some-raw-disk.raw',
+                },
+            },
+        ],
+    },
+    {
+        description => 'VMID: none, non-matching file paths for import',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => undef,
+        vtypes => ['import'],
+        cases => [
+            # Malformed file names
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/import.ovff",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/importova",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/import.ov",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/diskraw",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/diskvmdk",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/disk.invalid",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/.ova",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/.raw",
+                expected => undef,
+            },
+
+            # Trailing whitespace must not be trimmed
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/import.ova\t",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/disk.raw    ",
+                expected => undef,
+            },
+
+            # Whitespace in file name
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/something I want to import.ova",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/ .raw",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/ disk .vmdk",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/disk .qcow2",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/ import.ova",
+                expected => undef,
+            },
+
+            # Unsafe characters in file name
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/linux🐧-vm.ova",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/🐪perl-playground🐪.ova",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/fish_<><_<><_<><.ova",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/C:\\\\Windows\\Path.ova",
+                expected => undef,
+            },
+
+            # Content inside .ova files may only be specified as part
+            # of volume names, and may never appear when looked up as
+            # a file path
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/import.ova/disk.qcow2",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/import.ova/disk.raw",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/import.ova/disk.vmdk",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/import.ova/disk.invalid",
+                expected => undef,
+            },
+        ],
+    },
+    {
+        description => 'VMID: none, weird but valid file names for import',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => undef,
+        vtypes => ['import'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/import.ova.ova",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'ova',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/import.ova.ova',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/import.ova.ova.ova",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'ova',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/import.ova.ova.ova',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/import.ova.ova.ova.ova",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'ova',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/import.ova.ova.ova.ova',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/ova.ova",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'ova',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/ova.ova',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/ova.ovf",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'ovf',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/ova.ovf',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/ova.vmdk",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'vmdk',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/ova.vmdk',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/raw.raw.qcow2",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'qcow2',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/raw.raw.qcow2',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/raw.raw.qcow2.import.qcow2",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'qcow2',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/raw.raw.qcow2.import.qcow2',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/raw.raw.raw.your-boat.ova",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'ova',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/raw.raw.raw.your-boat.ova',
+                },
+            },
+        ],
+    },
 ];
 
 # provide static vmlist for tests
 my $mock_cluster = Test::MockModule->new('PVE::Cluster', no_auto => 1);
 $mock_cluster->redefine(get_vmlist => sub { return $mocked_vmlist; });
 
 # populate is File::stat's method to fill all information from CORE::stat into
 # an blessed array.
 my $mock_stat = Test::MockModule->new('File::stat', no_auto => 1);
 $mock_stat->redefine(
     populate => sub {
         my (@st) = @_;
-        $st[7] = DEFAULT_SIZE;
-        $st[10] = DEFAULT_CTIME;
+        $st[7] = $DEFAULT_SIZE;
+        $st[10] = $DEFAULT_CTIME;
 
         my $result = $mock_stat->original('populate')->(@st);
 
         return $result;
     },
@@ -845,68 +1058,21 @@ my $mock_fsi = Test::MockModule->new('PVE::Storage::Plugin', no_auto => 1);
 $mock_fsi->redefine(
     file_size_info => sub {
         my ($size, $format, $used, $parent, $ctime) =
             $mock_fsi->original('file_size_info')->(@_);
 
-        $size = DEFAULT_SIZE;
-        $used = DEFAULT_USED;
+        $size = $DEFAULT_SIZE;
+        $used = $DEFAULT_USED;
 
         return wantarray ? ($size, $format, $used, $parent, $ctime) : $size;
     },
 );
 
 my sub cmp_volinfo_by_volid {
     return $a->{volid} cmp $b->{volid};
 }
 
-my sub run_legacy_tests() {
-    my $sid = 'local';
-    my $types = [grep { $scfg->{content}->{$_} } keys $scfg->{content}->%*];
-
-    # run through test cases
-    foreach my $tt (@tests) {
-        my $vmid = $tt->{vmid};
-        my $files = $tt->{files};
-        my $expected = [sort cmp_volinfo_by_volid $tt->{expected}->@*];
-        my $description = $tt->{description};
-        my $parent = $tt->{parent};
-
-        # prepare environment
-        my $num = 0; #parent disks
-        for my $file (@$files) {
-            my ($name, $dir, $suffix) = fileparse($file, @BACKING_FILE_SUFFIXES);
-
-            make_path($dir, { verbose => 1, mode => 0755 });
-
-            if ($name) {
-                # using qemu-img to also be able to represent the backing device
-                my @cmd = ('/usr/bin/qemu-img', 'create', "$file", DEFAULT_SIZE);
-                push @cmd, ('-f', $suffix) if $suffix;
-                push @cmd, ('-u', '-b', @$parent[$num]) if $parent;
-                push @cmd, ('-F', $suffix) if $parent && $suffix;
-                $num++;
-
-                run_command([@cmd]);
-            }
-        }
-
-        my $got = eval {
-            my $volume_list = PVE::Storage::Plugin->list_volumes($sid, $scfg, $vmid, $types);
-            return [sort cmp_volinfo_by_volid $volume_list->@*];
-        };
-        $got = $@ if $@;
-
-        is_deeply($got, $expected, $description) || diag(explain($got));
-
-        # clean up after each test case, otherwise
-        # we get wrong results from leftover files
-        remove_tree($storage_dir, { verbose => 1 });
-    }
-
-    return;
-}
-
 my sub setup_test_env($test_params) {
     my ($storeid, $scfg) = $test_params->@{qw(storeid scfg)};
 
     make_path($scfg->{path}, { verbose => 1, mode => 0700 });
 
@@ -993,24 +1159,22 @@ my sub assert_test_params_keys_exist($test_params) {
 
     return;
 }
 
 sub main() {
-    plan tests => scalar(@tests) + scalar($test_param_list->@*) + 1;
+    plan tests => scalar($test_param_list->@*) + 1;
 
     # Keep the original vmlist around in order to check whether it was modified
     # after running all the tests. See:
     # https://pve.proxmox.com/pipermail/pve-devel/2020-January/041096.html
     my $original_vmlist = dclone(PVE::Cluster::get_vmlist());
 
     for my $test_params ($test_param_list->@*) {
         assert_test_params_keys_exist($test_params);
         run_test_for_params($test_params);
     }
 
-    run_legacy_tests();
-
     my $vmlist = PVE::Cluster::get_vmlist();
 
     is_deeply(
         $vmlist, $original_vmlist, 'Result of PVE::Cluster::get_vmlist remains unmodified',
     );
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 33/51] plugin: do not return volumes of undeclared content types anymore
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (31 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 32/51] test: list volumes: remove legacy code and migrate cases to new format Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 34/51] plugin: correct comment in get_subdir_files helper Max R. Carrara
                   ` (11 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Currently, any content types / volume types that are passed to
`PVE::Storage::Plugin->list_volumes()` are still taken into account by
the method, even if the passed storage config does not declare them in
its `content` key. In other words, the `content` property is ignored
when listing volumes through `list_volumes()`.

Or to express this more explicitly, consider a directory storage with
`path` set to `/mnt/example` and `content` set to `iso,vztmpl`. Now,
if there are files in `/mnt/example/snippets` for some reason and the
`list_volumes()` method is called with `['snippets']` for the volume
type list parameter, then the volume info hashes for the files in
`/mnt/example/snippets` are included in the output.

There is not really any use case for this behavior, and to my
knowledge, it is also not something that we rely on.

Therefore, instead of returning volumes for undeclared content types,
return an empty list for that type. Document this new behavior through
test cases.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Plugin.pm     |  9 ++++++-
 src/test/list_volumes_test.pm | 50 +++++++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+), 1 deletion(-)

diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 0d44224f..c1569314 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -14,10 +14,13 @@ use File::stat qw();
 use PVE::Tools qw(run_command);
 use PVE::JSONSchema qw(get_standard_option register_standard_option);
 use PVE::Cluster qw(cfs_register_file);
 
 use PVE::Storage::Common qw(
+    is_guest_vtype
+    content_allows
+
     plugin_get_default_vtype_subdirs
     plugin_get_vtype_subdir
 );
 use PVE::Storage::Common::Parse qw(
     parse_volname_as_parts
@@ -1985,11 +1988,15 @@ sub list_volumes {
     my $vmlist = PVE::Cluster::get_vmlist();
 
     my $get_raw_volumes_for_type = sub {
         my ($type) = @_;
 
-        if ($type eq 'images' || $type eq 'rootdir') {
+        if (!$scfg->{content} || !content_allows($scfg->{content}, $type)) {
+            return [];
+        }
+
+        if (is_guest_vtype($type) || $type eq 'rootdir') {
             return $class->list_images($storeid, $scfg, $vmid);
         }
 
         return if !$scfg->{path};
 
diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm
index 65fc7629..adc6d563 100644
--- a/src/test/list_volumes_test.pm
+++ b/src/test/list_volumes_test.pm
@@ -1030,10 +1030,60 @@ my $test_param_list = [
                     volid => 'local:import/raw.raw.raw.your-boat.ova',
                 },
             },
         ],
     },
+    {
+        description =>
+            "VMID: none, volume info is not returned for content types that are not declared",
+        storeid => $DEFAULT_STOREID,
+        scfg => {
+            type => 'dir',
+            path => $DEFAULT_STORAGE_PATH,
+            shared => 0,
+            content => {}, # note how no content types are declared here
+        },
+        vmid => undef,
+        vtypes => ['images', 'rootdir', 'vztmpl', 'iso', 'backup', 'snippets', 'import'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/16110/vm-16110-disk-0.qcow2",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/1234/vm-1234-disk-0.qcow2",
+                parent => '../ssss/base-4321-disk-0.qcow2',
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/images/16112/vm-16112-disk-0.raw",
+                expected => undef,
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/template/cache/alpine-3.10-default_20190626_amd64.tar.xz",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/archlinux-2020.02.01-x86_64.iso",
+                expected => undef,
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/hookscript.pl",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/import.ova",
+                expected => undef,
+            },
+        ],
+    },
 ];
 
 # provide static vmlist for tests
 my $mock_cluster = Test::MockModule->new('PVE::Cluster', no_auto => 1);
 $mock_cluster->redefine(get_vmlist => sub { return $mocked_vmlist; });
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 34/51] plugin: correct comment in get_subdir_files helper
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (32 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 33/51] plugin: do not return volumes of undeclared content types anymore Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 35/51] test: parse volname: modernize code Max R. Carrara
                   ` (10 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

The comment in the branch for 'backup' volume types in the
`get_subdir_files()` helper states that the condition below checks for
"false positives", apparently meaning that the VMID in the parent
directory might have been matched.

Challenge this claim by adding several new test cases that try to
cause the parser to return a wrong result.

As it turns out, the check the comment refers to is not (anymore)
there to check for "false positives" or VMIDs appearing in the parent
directory, but instead simply filters out backups that do not belong
to the provided VMID.

Note that if a backup has an arbitrary file name, that is, it's not
named something like "vzdump-qemu-1337-$TIMESTAMP.vma", that backup is
still returned when *no* `$vmid` is given. If it is given however, it
has to match the parsed `$vmid`.

Therefore, also include test cases for a plain "some-backup.tar.gz"
file that account for these things.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Plugin.pm     |   4 +-
 src/test/list_volumes_test.pm | 457 ++++++++++++++++++++++++++++++++++
 2 files changed, 459 insertions(+), 2 deletions(-)

diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index c1569314..73e43cea 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -1904,12 +1904,12 @@ my sub get_subdir_files {
 
         if ($vtype eq 'backup') {
             my $format = $parts->{ext};
             my $volume_path = $parts->{path};
 
-            # Check if parsed VMID matched provided VMID in order to avoid
-            # false positives (VMID in parent directory name)
+            # Check if parsed VMID matches provided VMID in order to avoid
+            # returning backups of other guests
             my $parsed_vmid = $parts->{vmid};
             if (defined($vmid)) {
                 if (defined($parsed_vmid)) {
                     return if $vmid ne $parsed_vmid;
                 } else {
diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm
index adc6d563..76831de9 100644
--- a/src/test/list_volumes_test.pm
+++ b/src/test/list_volumes_test.pm
@@ -647,10 +647,279 @@ my $test_param_list = [
                 file => "$DEFAULT_STORAGE_PATH/dump/backup-16110-manual.vma",
                 expected => undef,
             },
         ],
     },
+    {
+        description => 'VMID: none, backups of all guests',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => undef,
+        vtypes => ['backup'],
+        cases => [
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585602700,
+                    format => 'vma.gz',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'qemu',
+                    vmid => '16110',
+                    volid => 'local:backup/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585602765,
+                    format => 'vma.lzo',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'qemu',
+                    vmid => '16110',
+                    volid => 'local:backup/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585602835,
+                    format => 'vma',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'qemu',
+                    vmid => '16110',
+                    volid => 'local:backup/vzdump-qemu-16110-2020_03_30-21_13_55.vma',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585604370,
+                    format => 'tar.lzo',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585604970,
+                    format => 'tar.gz',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-16112-2020_03_30-21_49_30.tar.zst",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585604970,
+                    format => 'tar.zst',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-lxc-16112-2020_03_30-21_49_30.tar.zst',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-16112-2020_03_30-21_59_30.tgz",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585605570,
+                    format => 'tgz',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-lxc-16112-2020_03_30-21_59_30.tgz',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585604370,
+                    format => 'tar.bz2',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'openvz',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585602835,
+                    format => 'vma.zst',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'qemu',
+                    vmid => '16110',
+                    volid => 'local:backup/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-19253-2020_02_03-19_57_43.tar.gz",
+                expected => {
+                    content => 'backup',
+                    ctime => 1580759863,
+                    format => 'tar.gz',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '19253',
+                    volid => 'local:backup/vzdump-lxc-19253-2020_02_03-19_57_43.tar.gz',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-19254-2019_01_21-19_29_19.tar",
+                expected => {
+                    content => 'backup',
+                    ctime => 1548098959,
+                    format => 'tar',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '19254',
+                    volid => 'local:backup/vzdump-lxc-19254-2019_01_21-19_29_19.tar',
+                },
+            },
+            # Arbitrary backups are only included if no $vmid is provided.
+            # Note that in this case, the 'vmid' key does not exist at all,
+            # instead of being set to undef.
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/some-backup.tar.gz",
+                expected => {
+                    content => 'backup',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'tar.gz',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'unknown',
+                    volid => 'local:backup/some-backup.tar.gz',
+                },
+            },
+        ],
+    },
+    {
+        description => 'VMID: 16112, backups of specific guest',
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => 16112,
+        vtypes => ['backup'],
+        cases => [
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz",
+                expected => undef,
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma",
+                expected => undef,
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585604370,
+                    format => 'tar.lzo',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585604970,
+                    format => 'tar.gz',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-16112-2020_03_30-21_49_30.tar.zst",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585604970,
+                    format => 'tar.zst',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-lxc-16112-2020_03_30-21_49_30.tar.zst',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-16112-2020_03_30-21_59_30.tgz",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585605570,
+                    format => 'tgz',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-lxc-16112-2020_03_30-21_59_30.tgz',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585604370,
+                    format => 'tar.bz2',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'openvz',
+                    vmid => '16112',
+                    volid => 'local:backup/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst",
+                expected => undef,
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-19253-2020_02_03-19_57_43.tar.gz",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/vzdump-lxc-19254-2019_01_21-19_29_19.tar",
+                expected => undef,
+            },
+            # Arbitrary backups are only included if no $vmid is provided.
+            {
+                file => "$DEFAULT_STORAGE_PATH/dump/some-backup.tar.gz",
+                expected => undef,
+            },
+        ],
+    },
     {
         description => 'VMID: none, parent, non-matching',
         storeid => $DEFAULT_STOREID,
         scfg => $DEFAULT_SCFG,
         vmid => undef,
@@ -1082,10 +1351,198 @@ my $test_param_list = [
             },
         ],
     },
 ];
 
+# Additional test cases that cannot be constructed within the list above
+{
+    my $file_name = "vzdump-qemu-16110-2020_03_30-21_13_55.vma";
+    my $storage_path = File::Temp->newdir() . '/' . $file_name;
+
+    my $backup_vmid_test_params = {
+        description => "VMID: 16110, file name in path of storage",
+        storeid => $DEFAULT_STOREID,
+        scfg => {
+            type => 'dir',
+            path => $storage_path,
+            shared => 0,
+            content => {
+                backup => 1,
+            },
+        },
+        vmid => 16110,
+        vtypes => ['backup'],
+        cases => [
+            {
+                file => "$storage_path/dump/$file_name",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585602835,
+                    format => 'vma',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'qemu',
+                    vmid => '16110',
+                    volid => "local:backup/$file_name",
+                },
+            },
+        ],
+    };
+
+    push($test_param_list->@*, $backup_vmid_test_params);
+}
+
+{
+    my $file_name = "vzdump-qemu-16110-2020_03_30-21_13_55.vma";
+    my $storage_path = File::Temp->newdir();
+
+    my $backup_vmid_test_params = {
+        description => "VMID: 16110, file name in vtype subdir of 'backup' vtype",
+        storeid => $DEFAULT_STOREID,
+        scfg => {
+            type => 'dir',
+            path => $storage_path,
+            shared => 0,
+            content => {
+                backup => 1,
+            },
+            'content-dirs' => {
+                backup => $file_name,
+            },
+        },
+        vmid => 16110,
+        vtypes => ['backup'],
+        cases => [
+            {
+                file => "$storage_path/$file_name/$file_name",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585602835,
+                    format => 'vma',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'qemu',
+                    vmid => '16110',
+                    volid => "local:backup/$file_name",
+                },
+            },
+        ],
+    };
+
+    push($test_param_list->@*, $backup_vmid_test_params);
+}
+
+{
+    my $file_name = "vzdump-qemu-16110-2020_03_30-21_13_55.vma";
+    my $storage_path = File::Temp->newdir() . '/' . $file_name;
+
+    my $backup_vmid_test_params = {
+        description => "VMID: 16110, file name in storage path and subdir of 'backup' vtype",
+        storeid => $DEFAULT_STOREID,
+        scfg => {
+            type => 'dir',
+            path => $storage_path,
+            shared => 0,
+            content => {
+                backup => 1,
+            },
+            'content-dirs' => {
+                backup => $file_name,
+            },
+        },
+        vmid => 16110,
+        vtypes => ['backup'],
+        cases => [
+            {
+                file => "$storage_path/$file_name/$file_name",
+                expected => {
+                    content => 'backup',
+                    ctime => 1585602835,
+                    format => 'vma',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'qemu',
+                    vmid => '16110',
+                    volid => "local:backup/$file_name",
+                },
+            },
+        ],
+    };
+
+    push($test_param_list->@*, $backup_vmid_test_params);
+}
+
+{
+    my $file_name = "vzdump-qemu-16110-2020_03_30-21_13_55.vma";
+    my $storage_path = File::Temp->newdir() . '/' . $file_name;
+
+    my $backup_vmid_test_params = {
+        description =>
+            "VMID: 19253, file name with different VMID in storage path and subdir of 'backup' vtype",
+        storeid => $DEFAULT_STOREID,
+        scfg => {
+            type => 'dir',
+            path => $storage_path,
+            shared => 0,
+            content => {
+                backup => 1,
+            },
+            'content-dirs' => {
+                backup => $file_name,
+            },
+        },
+        vmid => 19253,
+        vtypes => ['backup'],
+        cases => [
+            {
+                file => "$storage_path/$file_name/vzdump-lxc-19253-2020_02_03-19_57_43.tar.gz",
+                expected => {
+                    content => 'backup',
+                    ctime => 1580759863,
+                    format => 'tar.gz',
+                    size => $DEFAULT_SIZE,
+                    subtype => 'lxc',
+                    vmid => '19253',
+                    volid => 'local:backup/vzdump-lxc-19253-2020_02_03-19_57_43.tar.gz',
+                },
+            },
+        ],
+    };
+
+    push($test_param_list->@*, $backup_vmid_test_params);
+}
+
+{
+    my $file_name = "vzdump-qemu-16110-2020_03_30-21_13_55.vma";
+    my $storage_path = File::Temp->newdir() . '/' . $file_name;
+
+    my $backup_vmid_test_params = {
+        description =>
+            "VMID: none, file name in storage path and subdir of 'backup' vtype, no backups",
+        storeid => $DEFAULT_STOREID,
+        scfg => {
+            type => 'dir',
+            path => $storage_path,
+            shared => 0,
+            content => {
+                backup => 1,
+                snippets => 1,
+            },
+            'content-dirs' => {
+                backup => $file_name,
+            },
+        },
+        vmid => 19253,
+        vtypes => ['backup'],
+        cases => [
+            {
+                file => "$storage_path/snippets/hookscript.pl",
+                expected => undef,
+            },
+        ],
+    };
+
+    push($test_param_list->@*, $backup_vmid_test_params);
+}
+
 # provide static vmlist for tests
 my $mock_cluster = Test::MockModule->new('PVE::Cluster', no_auto => 1);
 $mock_cluster->redefine(get_vmlist => sub { return $mocked_vmlist; });
 
 # populate is File::stat's method to fill all information from CORE::stat into
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 35/51] test: parse volname: modernize code
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (33 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 34/51] plugin: correct comment in get_subdir_files helper Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 36/51] test: parse volname: adapt tests regarding 'import' volume type Max R. Carrara
                   ` (9 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Modernize the existing code in `parse_volname_test.pm` while leaving
the tests unchanged otherwise.

In particular,
- move the test execution code into its own subroutine
- add and call a `main()` subroutine
- declare `use v5.36;` instead of `use strict;` and `use warnings;`
- fix a handful of typos
- adapt the code style to fit our more modern style guide

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/parse_volname_test.pm | 93 ++++++++++++++++++----------------
 1 file changed, 50 insertions(+), 43 deletions(-)

diff --git a/src/test/parse_volname_test.pm b/src/test/parse_volname_test.pm
index a3b4d568..6beb2be3 100644
--- a/src/test/parse_volname_test.pm
+++ b/src/test/parse_volname_test.pm
@@ -1,9 +1,8 @@
 package PVE::Storage::TestParseVolname;
 
-use strict;
-use warnings;
+use v5.36;
 
 use lib qw(..);
 
 use PVE::Storage;
 use PVE::Storage::Common qw(
@@ -182,28 +181,28 @@ my $tests = [
         description => "Import, ovf",
         volname => 'import/import.ovf',
         expected => ['import', 'import.ovf', undef, undef, undef, undef, 'ovf'],
     },
     {
-        description => "Import, innner file of ova",
+        description => "Import, inner file of ova",
         volname => 'import/import.ova/disk.qcow2',
         expected =>
             ['import', 'import.ova/disk.qcow2', undef, undef, undef, undef, 'ova+qcow2'],
     },
     {
-        description => "Import, innner file of ova",
+        description => "Import, inner file of ova",
         volname => 'import/import.ova/disk.vmdk',
         expected => ['import', 'import.ova/disk.vmdk', undef, undef, undef, undef, 'ova+vmdk'],
     },
     {
-        description => "Import, innner file of ova with whitespace in name",
+        description => "Import, inner file of ova with whitespace in name",
         volname => 'import/import.ova/OS disk.vmdk',
         expected =>
             ['import', 'import.ova/OS disk.vmdk', undef, undef, undef, undef, 'ova+vmdk'],
     },
     {
-        description => "Import, innner file of ova",
+        description => "Import, inner file of ova",
         volname => 'import/import.ova/disk.raw',
         expected => ['import', 'import.ova/disk.raw', undef, undef, undef, undef, 'ova+raw'],
     },
     #
     # failed matches
@@ -249,11 +248,11 @@ my $tests = [
     },
 ];
 
 # create more test cases for VM disk images matches
 my $disk_suffix = ['raw', 'qcow2', 'vmdk'];
-foreach my $s (@$disk_suffix) {
+for my $s ($disk_suffix->@*) {
     my @arr = (
         {
             description => "VM disk image, $s",
             volname => "$vmid/vm-$vmid-disk-1.$s",
             expected => [
@@ -280,23 +279,23 @@ foreach my $s (@$disk_suffix) {
                 'vm-vol', "base-$vmid-disk-0.$s", "$vmid", undef, undef, 'base-', "$s",
             ],
         },
     );
 
-    push @$tests, @arr;
+    push($tests->@*, @arr);
 }
 
 # create more test cases for backup files matches
 my $bkp_suffix = {
     qemu => ['vma', 'vma.gz', 'vma.lzo', 'vma.zst'],
     lxc => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.zst', 'tar.bz2'],
     openvz => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.zst'],
 };
 
-foreach my $virt (keys %$bkp_suffix) {
+for my $virt (keys $bkp_suffix->%*) {
     my $suffix = $bkp_suffix->{$virt};
-    foreach my $s (@$suffix) {
+    for my $s ($suffix->@*) {
         my @arr = (
             {
                 description => "Backup archive, $virt, $s",
                 volname => "backup/vzdump-$virt-$vmid-2020_03_30-21_12_40.$s",
                 expected => [
@@ -309,32 +308,32 @@ foreach my $virt (keys %$bkp_suffix) {
                     'raw',
                 ],
             },
         );
 
-        push @$tests, @arr;
+        push($tests->@*, @arr);
     }
 }
 
 # create more test cases for failed backup files matches
 my $non_bkp_suffix = {
     qemu => ['vms.gz', 'vma.xz'],
     lxc => ['zip.gz', 'tgz.lzo'],
 };
-foreach my $virt (keys %$non_bkp_suffix) {
+for my $virt (keys $non_bkp_suffix->%*) {
     my $suffix = $non_bkp_suffix->{$virt};
-    foreach my $s (@$suffix) {
+    for my $s ($suffix->@*) {
         my @arr = (
             {
                 description => "Failed match: Backup archive, $virt, $s",
                 volname => "backup/vzdump-$virt-$vmid-2020_03_30-21_12_40.$s",
                 expected =>
                     "unable to parse directory volume name 'backup/vzdump-$virt-$vmid-2020_03_30-21_12_40.$s'\n",
             },
         );
 
-        push @$tests, @arr;
+        push($tests->@*, @arr);
     }
 }
 
 my sub map_is_base {
     my ($plugin_name, $is_base) = @_;
@@ -590,45 +589,53 @@ push $tests->@*,
             plugin => 'PVE::Storage::PBSPlugin',
             expected => "unable to parse PBS volume name 'images/host/999/2025-08-06T14:27:07Z'\n",
         },
     );
 
-#
-# run through test case array
-#
-plan tests => scalar @$tests + 1;
+my sub run_tests($tests) {
+    my $seen_vtype = {};
+    my $vtype_subdirs = { map { $_ => 1 } keys plugin_get_default_vtype_subdirs()->%* };
 
-my $seen_vtype;
-my $vtype_subdirs =
-    { map { $_ => 1 } keys %{ plugin_get_default_vtype_subdirs() } };
+    for my $t ($tests->@*) {
+        my $description = $t->{description};
+        my $volname = $t->{volname};
+        my $expected = $t->{expected};
 
-foreach my $t (@$tests) {
-    my $description = $t->{description};
-    my $volname = $t->{volname};
-    my $expected = $t->{expected};
+        my $got;
+        if (my $plugin = $t->{plugin}) {
+            eval { $got = [$plugin->parse_volname($volname)] };
+        } else {
+            eval { $got = [PVE::Storage::Plugin->parse_volname($volname)] };
+        }
+        $got = $@ if $@;
 
-    my $got;
-    if (my $plugin = $t->{plugin}) {
-        eval { $got = [$plugin->parse_volname($volname)] };
-    } else {
-        eval { $got = [PVE::Storage::Plugin->parse_volname($volname)] };
+        is_deeply($got, $expected, $description);
+
+        $seen_vtype->{ $expected->[0] } = 1 if ref $expected eq 'ARRAY';
     }
-    $got = $@ if $@;
 
-    is_deeply($got, $expected, $description);
+    # every directory type needs a test: the volumes of VMs and containers all use the directory of the
+    # 'images' type, and the in-tree plugins return 'rootdir' for no volume
+    my $seen_subdir_vtype = {};
+    for my $vtype (keys $seen_vtype->%*) {
+        $vtype = 'images' if PVE::Storage::Common::is_guest_vtype($vtype);
+        $seen_subdir_vtype->{$vtype} = 1;
+    }
+    delete $vtype_subdirs->{rootdir};
+    is_deeply($seen_subdir_vtype, $vtype_subdirs, "vtype_subdir check");
 
-    $seen_vtype->{ @$expected[0] } = 1 if ref $expected eq 'ARRAY';
+    return;
 }
 
-# every directory type needs a test: the volumes of VMs and containers all use the directory of the
-# 'images' type, and the in-tree plugins return 'rootdir' for no volume
-my $seen_subdir_vtype = {};
-for my $vtype (keys $seen_vtype->%*) {
-    $vtype = 'images' if PVE::Storage::Common::is_guest_vtype($vtype);
-    $seen_subdir_vtype->{$vtype} = 1;
-}
-delete $vtype_subdirs->{rootdir};
-is_deeply($seen_subdir_vtype, $vtype_subdirs, "vtype_subdir check");
+my sub main() {
+    plan tests => scalar($tests->@*) + 1;
 
-done_testing();
+    run_tests($tests);
+
+    done_testing();
+
+    return;
+}
+
+main();
 
 1;
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 36/51] test: parse volname: adapt tests regarding 'import' volume type
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (34 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 35/51] test: parse volname: modernize code Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 37/51] test: parse volname: move VM disk test creation into separate block Max R. Carrara
                   ` (8 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Give the existing test cases unique 'description' keys and also
reorder them.

Add additional test cases for qcow2, vmdk and raw files.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/parse_volname_test.pm | 29 ++++++++++++++++++++++-------
 1 file changed, 22 insertions(+), 7 deletions(-)

diff --git a/src/test/parse_volname_test.pm b/src/test/parse_volname_test.pm
index 6beb2be3..058aa68e 100644
--- a/src/test/parse_volname_test.pm
+++ b/src/test/parse_volname_test.pm
@@ -181,31 +181,46 @@ my $tests = [
         description => "Import, ovf",
         volname => 'import/import.ovf',
         expected => ['import', 'import.ovf', undef, undef, undef, undef, 'ovf'],
     },
     {
-        description => "Import, inner file of ova",
+        description => "Import, qcow2",
+        volname => 'import/import.qcow2',
+        expected => ['import', 'import.qcow2', undef, undef, undef, undef, 'qcow2'],
+    },
+    {
+        description => "Import, vmdk",
+        volname => 'import/import.vmdk',
+        expected => ['import', 'import.vmdk', undef, undef, undef, undef, 'vmdk'],
+    },
+    {
+        description => "Import, raw",
+        volname => 'import/import.raw',
+        expected => ['import', 'import.raw', undef, undef, undef, undef, 'raw'],
+    },
+    {
+        description => "Import, inner file of ova (qcow2)",
         volname => 'import/import.ova/disk.qcow2',
         expected =>
             ['import', 'import.ova/disk.qcow2', undef, undef, undef, undef, 'ova+qcow2'],
     },
     {
-        description => "Import, inner file of ova",
+        description => "Import, inner file of ova (vmdk)",
         volname => 'import/import.ova/disk.vmdk',
         expected => ['import', 'import.ova/disk.vmdk', undef, undef, undef, undef, 'ova+vmdk'],
     },
+    {
+        description => "Import, inner file of ova (raw)",
+        volname => 'import/import.ova/disk.raw',
+        expected => ['import', 'import.ova/disk.raw', undef, undef, undef, undef, 'ova+raw'],
+    },
     {
         description => "Import, inner file of ova with whitespace in name",
         volname => 'import/import.ova/OS disk.vmdk',
         expected =>
             ['import', 'import.ova/OS disk.vmdk', undef, undef, undef, undef, 'ova+vmdk'],
     },
-    {
-        description => "Import, inner file of ova",
-        volname => 'import/import.ova/disk.raw',
-        expected => ['import', 'import.ova/disk.raw', undef, undef, undef, undef, 'ova+raw'],
-    },
     #
     # failed matches
     #
     {
         description => "Failed match: VM disk image, base, raw",
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 37/51] test: parse volname: move VM disk test creation into separate block
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (35 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 36/51] test: parse volname: adapt tests regarding 'import' volume type Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 38/51] test: parse volname: move backup file test creation into sep. block Max R. Carrara
                   ` (7 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

For future organizational purposes, move the loop that adds more
test cases for VM images into its own block.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/parse_volname_test.pm | 75 +++++++++++++++++++---------------
 1 file changed, 42 insertions(+), 33 deletions(-)

diff --git a/src/test/parse_volname_test.pm b/src/test/parse_volname_test.pm
index 058aa68e..249d77ff 100644
--- a/src/test/parse_volname_test.pm
+++ b/src/test/parse_volname_test.pm
@@ -261,44 +261,53 @@ my $tests = [
         volname => "import/test.foo",
         expected => "unable to parse directory volume name 'import/test.foo'\n",
     },
 ];
 
-# create more test cases for VM disk images matches
-my $disk_suffix = ['raw', 'qcow2', 'vmdk'];
-for my $s ($disk_suffix->@*) {
-    my @arr = (
-        {
-            description => "VM disk image, $s",
-            volname => "$vmid/vm-$vmid-disk-1.$s",
-            expected => [
-                'vm-vol', "vm-$vmid-disk-1.$s", "$vmid", undef, undef, undef, "$s",
-            ],
-        },
-        {
-            description => "VM disk image, linked, $s",
-            volname => "$vmid/base-$vmid-disk-0.$s/$vmid/vm-$vmid-disk-0.$s",
-            expected => [
-                'vm-vol',
-                "vm-$vmid-disk-0.$s",
-                "$vmid",
-                "base-$vmid-disk-0.$s",
-                "$vmid",
-                undef,
-                "$s",
-            ],
-        },
-        {
-            description => "VM disk image, base, $s",
-            volname => "$vmid/base-$vmid-disk-0.$s",
-            expected => [
-                'vm-vol', "base-$vmid-disk-0.$s", "$vmid", undef, undef, 'base-', "$s",
-            ],
-        },
-    );
+# Additional test cases for VM disk images
+{
+    my $disk_suffixes = ['raw', 'qcow2', 'vmdk'];
 
-    push($tests->@*, @arr);
+    for my $suffix ($disk_suffixes->@*) {
+        my @extra_tests = (
+            {
+                description => "VM disk image, $suffix",
+                volname => "$vmid/vm-$vmid-disk-1.$suffix",
+                expected => [
+                    'vm-vol', "vm-$vmid-disk-1.$suffix", "$vmid", undef, undef, undef, $suffix,
+                ],
+            },
+            {
+                description => "VM disk image, linked, $suffix",
+                volname => "$vmid/base-$vmid-disk-0.$suffix/$vmid/vm-$vmid-disk-0.$suffix",
+                expected => [
+                    'vm-vol',
+                    "vm-$vmid-disk-0.$suffix",
+                    "$vmid",
+                    "base-$vmid-disk-0.$suffix",
+                    "$vmid",
+                    undef,
+                    $suffix,
+                ],
+            },
+            {
+                description => "VM disk image, base, $suffix",
+                volname => "$vmid/base-$vmid-disk-0.$suffix",
+                expected => [
+                    'vm-vol',
+                    "base-$vmid-disk-0.$suffix",
+                    "$vmid",
+                    undef,
+                    undef,
+                    'base-',
+                    $suffix,
+                ],
+            },
+        );
+
+        push($tests->@*, @extra_tests);
+    }
 }
 
 # create more test cases for backup files matches
 my $bkp_suffix = {
     qemu => ['vma', 'vma.gz', 'vma.lzo', 'vma.zst'],
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 38/51] test: parse volname: move backup file test creation into sep. block
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (36 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 37/51] test: parse volname: move VM disk test creation into separate block Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 39/51] test: parse volname: parameterize test case creation for some vtypes Max R. Carrara
                   ` (6 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Similar to the previous commit, move the code that adds additional
tests for backup file volnames into a separate block, for future
organizational purposes.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/parse_volname_test.pm | 88 +++++++++++++++++-----------------
 1 file changed, 45 insertions(+), 43 deletions(-)

diff --git a/src/test/parse_volname_test.pm b/src/test/parse_volname_test.pm
index 249d77ff..48a5169f 100644
--- a/src/test/parse_volname_test.pm
+++ b/src/test/parse_volname_test.pm
@@ -306,58 +306,60 @@ my $tests = [
 
         push($tests->@*, @extra_tests);
     }
 }
 
-# create more test cases for backup files matches
-my $bkp_suffix = {
-    qemu => ['vma', 'vma.gz', 'vma.lzo', 'vma.zst'],
-    lxc => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.zst', 'tar.bz2'],
-    openvz => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.zst'],
-};
+# Additional tests for backup files
+{
+    my $bkp_suffixes = {
+        qemu => ['vma', 'vma.gz', 'vma.lzo', 'vma.zst'],
+        lxc => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.zst', 'tar.bz2'],
+        openvz => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.zst'],
+    };
 
-for my $virt (keys $bkp_suffix->%*) {
-    my $suffix = $bkp_suffix->{$virt};
-    for my $s ($suffix->@*) {
-        my @arr = (
-            {
-                description => "Backup archive, $virt, $s",
-                volname => "backup/vzdump-$virt-$vmid-2020_03_30-21_12_40.$s",
-                expected => [
-                    'backup',
-                    "vzdump-$virt-$vmid-2020_03_30-21_12_40.$s",
-                    "$vmid",
-                    undef,
-                    undef,
-                    undef,
-                    'raw',
-                ],
-            },
-        );
+    for my $virt (keys $bkp_suffixes->%*) {
+        my $suffixes = $bkp_suffixes->{$virt};
+        for my $suffix ($suffixes->@*) {
+            my @extra_tests = (
+                {
+                    description => "Backup archive, $virt, $suffix",
+                    volname => "backup/vzdump-$virt-$vmid-2020_03_30-21_12_40.$suffix",
+                    expected => [
+                        'backup',
+                        "vzdump-$virt-$vmid-2020_03_30-21_12_40.$suffix",
+                        "$vmid",
+                        undef,
+                        undef,
+                        undef,
+                        'raw',
+                    ],
+                },
+            );
 
-        push($tests->@*, @arr);
+            push($tests->@*, @extra_tests);
+        }
     }
-}
 
-# create more test cases for failed backup files matches
-my $non_bkp_suffix = {
-    qemu => ['vms.gz', 'vma.xz'],
-    lxc => ['zip.gz', 'tgz.lzo'],
-};
-for my $virt (keys $non_bkp_suffix->%*) {
-    my $suffix = $non_bkp_suffix->{$virt};
-    for my $s ($suffix->@*) {
-        my @arr = (
-            {
-                description => "Failed match: Backup archive, $virt, $s",
-                volname => "backup/vzdump-$virt-$vmid-2020_03_30-21_12_40.$s",
-                expected =>
-                    "unable to parse directory volume name 'backup/vzdump-$virt-$vmid-2020_03_30-21_12_40.$s'\n",
-            },
-        );
+    # Expected to fail
+    my $non_bkp_suffixes = {
+        qemu => ['vms.gz', 'vma.xz'],
+        lxc => ['zip.gz', 'tgz.lzo'],
+    };
+    for my $virt (keys $non_bkp_suffixes->%*) {
+        my $suffixes = $non_bkp_suffixes->{$virt};
+        for my $suffix ($suffixes->@*) {
+            my @extra_tests = (
+                {
+                    description => "Failed match: Backup archive, $virt, $suffix",
+                    volname => "backup/vzdump-$virt-$vmid-2020_03_30-21_12_40.$suffix",
+                    expected =>
+                        "unable to parse directory volume name 'backup/vzdump-$virt-$vmid-2020_03_30-21_12_40.$suffix'\n",
+                },
+            );
 
-        push($tests->@*, @arr);
+            push($tests->@*, @extra_tests);
+        }
     }
 }
 
 my sub map_is_base {
     my ($plugin_name, $is_base) = @_;
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 39/51] test: parse volname: parameterize test case creation for some vtypes
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (37 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 38/51] test: parse volname: move backup file test creation into sep. block Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 40/51] test: volume id: modernize code Max R. Carrara
                   ` (5 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Currently, a lot of test cases are simply defined as part of the
overall list of tests. While this is completely fine in itself, it
does not scale if one wants to add cases that account for additional
parameters in the future.

Therefore, create a lot of existing test cases in blocks and push them
onto the list of test cases, instead of defining them up front.

This is done as preparation for future commits, primarily in order to
avoid writing a lot of test cases by hand.

Also re-format the tests for PBS somewhat, to match the surrounding
code.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/parse_volname_test.pm | 281 ++++++++++++++++++---------------
 1 file changed, 151 insertions(+), 130 deletions(-)

diff --git a/src/test/parse_volname_test.pm b/src/test/parse_volname_test.pm
index 48a5169f..5520d3c6 100644
--- a/src/test/parse_volname_test.pm
+++ b/src/test/parse_volname_test.pm
@@ -41,74 +41,10 @@ my $tests = [
             undef,
             'qcow2',
         ],
     },
     #
-    # iso
-    #
-    {
-        description => 'ISO image, iso',
-        volname => 'iso/some-installation-disk.iso',
-        expected => ['iso', 'some-installation-disk.iso', undef, undef, undef, undef, 'raw'],
-    },
-    {
-        description => 'ISO image, img',
-        volname => 'iso/some-other-installation-disk.img',
-        expected =>
-            ['iso', 'some-other-installation-disk.img', undef, undef, undef, undef, 'raw'],
-    },
-    #
-    # container templates
-    #
-    {
-        description => 'Container template tar',
-        volname => 'vztmpl/debian-10.0-standard_10.0-1_amd64.tar',
-        expected => [
-            'vztmpl', 'debian-10.0-standard_10.0-1_amd64.tar', undef, undef, undef, undef,
-            'raw',
-        ],
-    },
-    {
-        description => 'Container template tar.gz',
-        volname => 'vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz',
-        expected => [
-            'vztmpl',
-            'debian-10.0-standard_10.0-1_amd64.tar.gz',
-            undef,
-            undef,
-            undef,
-            undef,
-            'raw',
-        ],
-    },
-    {
-        description => 'Container template tar.xz',
-        volname => 'vztmpl/debian-10.0-standard_10.0-1_amd64.tar.xz',
-        expected => [
-            'vztmpl',
-            'debian-10.0-standard_10.0-1_amd64.tar.xz',
-            undef,
-            undef,
-            undef,
-            undef,
-            'raw',
-        ],
-    },
-    {
-        description => 'Container template tar.bz2',
-        volname => 'vztmpl/debian-10.0-standard_10.0-1_amd64.tar.bz2',
-        expected => [
-            'vztmpl',
-            'debian-10.0-standard_10.0-1_amd64.tar.bz2',
-            undef,
-            undef,
-            undef,
-            undef,
-            'raw',
-        ],
-    },
-    #
     # container rootdir
     #
     {
         description => 'Container rootdir, subvol',
         volname => "$vmid/subvol-$vmid-disk-0.subvol",
@@ -155,66 +91,12 @@ my $tests = [
             undef,
             'raw',
         ],
     },
     #
-    # Snippets
-    #
-    {
-        description => 'Snippets, yaml',
-        volname => 'snippets/userconfig.yaml',
-        expected => ['snippets', 'userconfig.yaml', undef, undef, undef, undef, 'raw'],
-    },
-    {
-        description => 'Snippets, perl',
-        volname => 'snippets/hookscript.pl',
-        expected => ['snippets', 'hookscript.pl', undef, undef, undef, undef, 'raw'],
-    },
-    #
     # Import
     #
-    {
-        description => "Import, ova",
-        volname => 'import/import.ova',
-        expected => ['import', 'import.ova', undef, undef, undef, undef, 'ova'],
-    },
-    {
-        description => "Import, ovf",
-        volname => 'import/import.ovf',
-        expected => ['import', 'import.ovf', undef, undef, undef, undef, 'ovf'],
-    },
-    {
-        description => "Import, qcow2",
-        volname => 'import/import.qcow2',
-        expected => ['import', 'import.qcow2', undef, undef, undef, undef, 'qcow2'],
-    },
-    {
-        description => "Import, vmdk",
-        volname => 'import/import.vmdk',
-        expected => ['import', 'import.vmdk', undef, undef, undef, undef, 'vmdk'],
-    },
-    {
-        description => "Import, raw",
-        volname => 'import/import.raw',
-        expected => ['import', 'import.raw', undef, undef, undef, undef, 'raw'],
-    },
-    {
-        description => "Import, inner file of ova (qcow2)",
-        volname => 'import/import.ova/disk.qcow2',
-        expected =>
-            ['import', 'import.ova/disk.qcow2', undef, undef, undef, undef, 'ova+qcow2'],
-    },
-    {
-        description => "Import, inner file of ova (vmdk)",
-        volname => 'import/import.ova/disk.vmdk',
-        expected => ['import', 'import.ova/disk.vmdk', undef, undef, undef, undef, 'ova+vmdk'],
-    },
-    {
-        description => "Import, inner file of ova (raw)",
-        volname => 'import/import.ova/disk.raw',
-        expected => ['import', 'import.ova/disk.raw', undef, undef, undef, undef, 'ova+raw'],
-    },
     {
         description => "Import, inner file of ova with whitespace in name",
         volname => 'import/import.ova/OS disk.vmdk',
         expected =>
             ['import', 'import.ova/OS disk.vmdk', undef, undef, undef, undef, 'ova+vmdk'],
@@ -306,10 +188,52 @@ my $tests = [
 
         push($tests->@*, @extra_tests);
     }
 }
 
+# Test cases for ISOs
+{
+    my $suffixes = ['iso', 'img'];
+    my $prefix = 'some-installation-disk';
+
+    for my $suffix ($suffixes->@*) {
+        my $file_name = "$prefix.$suffix";
+
+        my @extra_tests = (
+            {
+                description => "ISO image, $suffix",
+                volname => "iso/$file_name",
+                expected => ['iso', "$file_name", undef, undef, undef, undef, 'raw'],
+            },
+        );
+
+        push($tests->@*, @extra_tests);
+    }
+}
+
+# Test cases for container templates
+{
+    my $suffixes = ['tar', 'tar.gz', 'tar.xz', 'tar.bz2'];
+    my $prefix = 'debian-10.0-standard_10.0-1_amd64';
+
+    for my $suffix ($suffixes->@*) {
+        my $file_name = "$prefix.$suffix";
+
+        my @extra_tests = (
+            {
+                description => "Container template, $suffix",
+                volname => "vztmpl/$file_name",
+                expected => [
+                    'vztmpl', "$file_name", undef, undef, undef, undef, 'raw',
+                ],
+            },
+        );
+
+        push($tests->@*, @extra_tests);
+    }
+}
+
 # Additional tests for backup files
 {
     my $bkp_suffixes = {
         qemu => ['vma', 'vma.gz', 'vma.lzo', 'vma.zst'],
         lxc => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.zst', 'tar.bz2'],
@@ -571,54 +495,151 @@ push $tests->@*,
             plugin => 'PVE::Storage::ISCSIDirectPlugin',
             expected => ['vm-vol', '1', undef, undef, undef, undef, 'raw'],
         },
     );
 
-# tests for PBS backup archives
-push $tests->@*,
-    (
+for my $plugin_name (qw(ZFSPlugin ZFSPoolPlugin)) {
+    push $tests->@*,
+        (
+            {
+                description => "$plugin_name - trailing newline",
+                volname => "vm-123-disk-0\n",
+                plugin => "PVE::Storage::$plugin_name",
+                expected => map_error_message($plugin_name, "vm-123-disk-0\n"),
+            },
+        );
+}
+
+# Tests for PBS backup archives
+{
+    my @extra_tests = (
         {
             description => "PBS backup archive, vm/117",
             volname => "backup/vm/117/2025-08-06T14:27:07Z",
             plugin => 'PVE::Storage::PBSPlugin',
             expected => [
                 'backup', 'vm/117/2025-08-06T14:27:07Z', 117, undef, undef, undef, 'pbs-vm',
             ],
         },
         {
             description => "PBS backup archive, ct/123455",
             volname => "backup/ct/123455/2025-08-06T14:27:07Z",
             plugin => 'PVE::Storage::PBSPlugin',
             expected => [
-                'backup', 'ct/123455/2025-08-06T14:27:07Z', 123455, undef, undef, undef, 'pbs-ct',
+                'backup', 'ct/123455/2025-08-06T14:27:07Z', 123455, undef, undef, undef,
+                'pbs-ct',
             ],
         },
         {
             description => "PBS backup archive, host/foobar",
             volname => "backup/host/foobar/2025-08-06T14:27:07Z",
             plugin => 'PVE::Storage::PBSPlugin',
             expected => [
-                'backup', 'host/foobar/2025-08-06T14:27:07Z', undef, undef, undef, undef,
+                'backup',
+                'host/foobar/2025-08-06T14:27:07Z',
+                undef,
+                undef,
+                undef,
+                undef,
                 'pbs-host',
             ],
         },
         {
             description => "PBS backup archive, host/999",
             volname => "backup/host/999/2025-08-06T14:27:07Z",
             plugin => 'PVE::Storage::PBSPlugin',
             expected => [
-                'backup', 'host/999/2025-08-06T14:27:07Z', undef, undef, undef, undef, 'pbs-host',
+                'backup', 'host/999/2025-08-06T14:27:07Z', undef, undef, undef, undef,
+                'pbs-host',
             ],
         },
-        {
-            description => "Failed match: PBS backup archive, non-backup content",
-            volname => "images/host/999/2025-08-06T14:27:07Z",
-            plugin => 'PVE::Storage::PBSPlugin',
-            expected => "unable to parse PBS volume name 'images/host/999/2025-08-06T14:27:07Z'\n",
-        },
     );
 
+    push($tests->@*, @extra_tests);
+
+    # Expected to fail
+    {
+        my @extra_failed_tests = (
+            {
+                description => "Failed match: PBS backup archive, non-backup content",
+                volname => "images/host/999/2025-08-06T14:27:07Z",
+                plugin => 'PVE::Storage::PBSPlugin',
+                expected =>
+                    "unable to parse PBS volume name 'images/host/999/2025-08-06T14:27:07Z'\n",
+            },
+        );
+
+        push($tests->@*, @extra_failed_tests);
+    }
+}
+
+# Test cases for snippets
+{
+    my $file_names = ['userconfig.yaml', 'hookscript.pl'];
+
+    for my $file_name ($file_names->@*) {
+        my @extra_tests = (
+            {
+                description => "Snippets, $file_name",
+                volname => "snippets/$file_name",
+                expected => ['snippets', $file_name, undef, undef, undef, undef, 'raw'],
+            },
+        );
+
+        push($tests->@*, @extra_tests);
+    }
+}
+
+# Test cases for import files
+{
+    my $suffixes = ['ova', 'ovf', 'qcow2', 'vmdk', 'raw'];
+    my $prefix = 'import-file';
+
+    for my $suffix ($suffixes->@*) {
+        my $file_name = "$prefix.$suffix";
+
+        my @extra_tests = (
+            {
+                description => "Import, $suffix",
+                volname => "import/$file_name",
+                expected => ['import', "$file_name", undef, undef, undef, undef, $suffix],
+            },
+        );
+
+        push($tests->@*, @extra_tests);
+    }
+}
+
+# Test cases for OVA import files with content inside
+{
+    my $content_suffixes = ['qcow2', 'vmdk', 'raw'];
+    my $content_prefix = 'disk';
+    my $file_name = 'import.ova';
+
+    for my $content_suffix ($content_suffixes->@*) {
+        my $content_file_name = "$content_prefix.$content_suffix";
+
+        my @extra_tests = (
+            {
+                description => "Import, inner file of ova ($content_suffix)",
+                volname => "import/$file_name/$content_file_name",
+                expected => [
+                    'import',
+                    "$file_name/$content_file_name",
+                    undef,
+                    undef,
+                    undef,
+                    undef,
+                    "ova+$content_suffix",
+                ],
+            },
+        );
+
+        push($tests->@*, @extra_tests);
+    }
+}
+
 my sub run_tests($tests) {
     my $seen_vtype = {};
     my $vtype_subdirs = { map { $_ => 1 } keys plugin_get_default_vtype_subdirs()->%* };
 
     for my $t ($tests->@*) {
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 40/51] test: volume id: modernize code
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (38 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 39/51] test: parse volname: parameterize test case creation for some vtypes Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 41/51] test: volume id: rename 'volname' test case parameter to 'file' Max R. Carrara
                   ` (4 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Modernize the existing code in `path_to_volume_id_test.pm` while
leaving the tests unchanged otherwise.

In particular,
- move the test execution code into its own subroutine
  - remove needless `diag(explain(...))` call
  - suppress warnings when calling `path_to_volume_id()` to make test
    output less noisy
- add and call a `main()` subroutine
- declare `use v5.36;` instead of `use strict;` and `use warnings;`
- capitalize constants
- move the comment regarding `File::Temp->newdir()` unlinking on exit
  to where the method is actually called
- adapt the code style to fit our more modern style guide

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/path_to_volume_id_test.pm | 149 ++++++++++++++++-------------
 1 file changed, 82 insertions(+), 67 deletions(-)

diff --git a/src/test/path_to_volume_id_test.pm b/src/test/path_to_volume_id_test.pm
index 03753c38..69e0330d 100644
--- a/src/test/path_to_volume_id_test.pm
+++ b/src/test/path_to_volume_id_test.pm
@@ -1,31 +1,30 @@
 package PVE::Storage::TestPathToVolumeId;
 
-use strict;
-use warnings;
+use v5.36;
 
 use lib qw(..);
 
 use PVE::Storage;
 use PVE::Storage::Common qw(
     plugin_get_default_vtype_subdirs
 );
 
 use Test::More;
 
 use Cwd;
-use File::Basename;
+use File::Basename qw(fileparse);
 use File::Path qw(make_path remove_tree);
 use File::Temp;
 
-my $storage_dir = File::Temp->newdir();
-my $scfg = {
+my $DEFAULT_STORAGE_DIR = File::Temp->newdir(); # unlinks on exit
+my $DEFAULT_CFG = {
     'digest' => 'd29306346b8b25b90a4a96165f1e8f52d1af1eda',
     'ids' => {
         'local' => {
             'shared' => 0,
-            'path' => "$storage_dir",
+            'path' => "$DEFAULT_STORAGE_DIR",
             'type' => 'dir',
             'content' => {
                 'snippets' => 1,
                 'rootdir' => 1,
                 'images' => 1,
@@ -42,241 +41,257 @@ my $scfg = {
 
 # the tests array consists of hashes with the following keys:
 # description => to identify the test case
 # volname     => to create the test file
 # expected    => the result that path_to_volume_id should return
-my @tests = (
+my $tests = [
     {
         description => 'Image, qcow2',
-        volname => "$storage_dir/images/16110/vm-16110-disk-0.qcow2",
+        volname => "$DEFAULT_STORAGE_DIR/images/16110/vm-16110-disk-0.qcow2",
         expected => [
             'vm-vol', 'local:16110/vm-16110-disk-0.qcow2',
         ],
     },
     {
         description => 'Image, raw',
-        volname => "$storage_dir/images/16112/vm-16112-disk-0.raw",
+        volname => "$DEFAULT_STORAGE_DIR/images/16112/vm-16112-disk-0.raw",
         expected => [
             'vm-vol', 'local:16112/vm-16112-disk-0.raw',
         ],
     },
     {
         description => 'Image template, qcow2',
-        volname => "$storage_dir/images/9004/base-9004-disk-0.qcow2",
+        volname => "$DEFAULT_STORAGE_DIR/images/9004/base-9004-disk-0.qcow2",
         expected => [
             'vm-vol', 'local:9004/base-9004-disk-0.qcow2',
         ],
     },
 
     {
         description => 'Backup, vma.gz',
-        volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz",
+        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz",
         expected => [
             'backup', 'local:backup/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz',
         ],
     },
     {
         description => 'Backup, vma.lzo',
-        volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo",
+        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo",
         expected => [
             'backup', 'local:backup/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
         ],
     },
     {
         description => 'Backup, vma',
-        volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma",
+        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma",
         expected => [
             'backup', 'local:backup/vzdump-qemu-16110-2020_03_30-21_13_55.vma',
         ],
     },
     {
         description => 'Backup, tar.lzo',
-        volname => "$storage_dir/dump/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo",
+        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo",
         expected => [
             'backup', 'local:backup/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo',
         ],
     },
     {
         description => 'Backup, vma.zst',
-        volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst",
+        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst",
         expected => [
             'backup', 'local:backup/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst',
         ],
     },
     {
         description => 'Backup, tar.zst',
-        volname => "$storage_dir/dump/vzdump-lxc-16112-2020_03_30-21_39_30.tar.zst",
+        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-lxc-16112-2020_03_30-21_39_30.tar.zst",
         expected => [
             'backup', 'local:backup/vzdump-lxc-16112-2020_03_30-21_39_30.tar.zst',
         ],
     },
     {
         description => 'Backup, tar.bz2',
-        volname => "$storage_dir/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2",
+        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2",
         expected => [
             'backup', 'local:backup/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2',
         ],
     },
 
     {
         description => 'ISO file',
-        volname => "$storage_dir/template/iso/yet-again-a-installation-disk.iso",
+        volname => "$DEFAULT_STORAGE_DIR/template/iso/yet-again-a-installation-disk.iso",
         expected => [
             'iso', 'local:iso/yet-again-a-installation-disk.iso',
         ],
     },
     {
         description => 'CT template, tar.gz',
-        volname => "$storage_dir/template/cache/debian-10.0-standard_10.0-1_amd64.tar.gz",
+        volname =>
+            "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.gz",
         expected => [
             'vztmpl', 'local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz',
         ],
     },
     {
         description => 'CT template, wrong ending, tar bz2',
-        volname => "$storage_dir/template/cache/debian-10.0-standard_10.0-1_amd64.tar.bz2",
+        volname =>
+            "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.bz2",
         expected => [
             'vztmpl', 'local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.bz2',
         ],
     },
 
     {
         description => 'Rootdir, folder subvol, legacy naming',
-        volname => "$storage_dir/images/1234/subvol-1234-disk-0.subvol/", # fileparse needs / at the end
+        volname => "$DEFAULT_STORAGE_DIR/images/1234/subvol-1234-disk-0.subvol/", # fileparse needs / at the end
         expected => [
             'ct-vol', 'local:1234/subvol-1234-disk-0.subvol',
         ],
     },
     {
         description => 'Rootdir, folder subvol',
-        volname => "$storage_dir/images/1234/subvol-1234-disk-0.subvol/", # fileparse needs / at the end
+        volname => "$DEFAULT_STORAGE_DIR/images/1234/subvol-1234-disk-0.subvol/", # fileparse needs / at the end
         expected => [
             'ct-vol', 'local:1234/subvol-1234-disk-0.subvol',
         ],
     },
     {
         description => 'Snippets, yaml',
-        volname => "$storage_dir/snippets/userconfig.yaml",
+        volname => "$DEFAULT_STORAGE_DIR/snippets/userconfig.yaml",
         expected => [
             'snippets', 'local:snippets/userconfig.yaml',
         ],
     },
     {
         description => 'Snippets, hookscript',
-        volname => "$storage_dir/snippets/hookscript.pl",
+        volname => "$DEFAULT_STORAGE_DIR/snippets/hookscript.pl",
         expected => [
             'snippets', 'local:snippets/hookscript.pl',
         ],
     },
     {
         description => 'CT template, tar.xz',
-        volname => "$storage_dir/template/cache/debian-10.0-standard_10.0-1_amd64.tar.xz",
+        volname =>
+            "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.xz",
         expected => [
             'vztmpl', 'local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.xz',
         ],
     },
     {
         description => 'Import, ova',
-        volname => "$storage_dir/import/import.ova",
+        volname => "$DEFAULT_STORAGE_DIR/import/import.ova",
         expected => [
             'import', 'local:import/import.ova',
         ],
     },
     {
         description => 'Import, ovf',
-        volname => "$storage_dir/import/import.ovf",
+        volname => "$DEFAULT_STORAGE_DIR/import/import.ovf",
         expected => [
             'import', 'local:import/import.ovf',
         ],
     },
 
     # no matches, path or files with failures
     {
         description => 'Base template, string as vmid in folder name',
-        volname => "$storage_dir/images/ssss/base-4321-disk-0.raw",
+        volname => "$DEFAULT_STORAGE_DIR/images/ssss/base-4321-disk-0.raw",
         expected => [''],
     },
     {
         description => 'ISO file, wrong ending',
-        volname => "$storage_dir/template/iso/yet-again-a-installation-disk.dvd",
+        volname => "$DEFAULT_STORAGE_DIR/template/iso/yet-again-a-installation-disk.dvd",
         expected => [''],
     },
     {
         description => 'CT template, wrong ending, zip.gz',
-        volname => "$storage_dir/template/cache/debian-10.0-standard_10.0-1_amd64.zip.gz",
+        volname =>
+            "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.zip.gz",
         expected => [''],
     },
     {
         description => 'Backup, wrong format, openvz, zip.gz',
-        volname => "$storage_dir/dump/vzdump-openvz-16112-2020_03_30-21_39_30.zip.gz",
+        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-openvz-16112-2020_03_30-21_39_30.zip.gz",
         expected => [''],
     },
     {
         description => 'Backup, wrong format, openvz, tgz.lzo',
-        volname => "$storage_dir/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tgz.lzo",
+        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tgz.lzo",
         expected => [''],
     },
     {
         description => 'Backup, wrong ending, qemu, vma.xz',
-        volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz",
+        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz",
         expected => [''],
     },
     {
         description => 'Backup, wrong format, qemu, vms.gz',
-        volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vms.gz",
+        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vms.gz",
         expected => [''],
     },
     {
         description => 'Image, string as vmid in folder name',
-        volname => "$storage_dir/images/ssss/vm-1234-disk-0.qcow2",
+        volname => "$DEFAULT_STORAGE_DIR/images/ssss/vm-1234-disk-0.qcow2",
         expected => [''],
     },
     {
         description => 'Import, non ova/ovf/disk image in import dir',
-        volname => "$storage_dir/import/test.foo",
+        volname => "$DEFAULT_STORAGE_DIR/import/test.foo",
         expected => [''],
     },
-);
+];
 
-plan tests => scalar @tests + 1;
+my sub run_tests($tests) {
+    my $seen_vtype;
+    my $vtype_subdirs = { map { $_ => 1 } keys plugin_get_default_vtype_subdirs()->%* };
 
-my $seen_vtype;
-my $vtype_subdirs =
-    { map { $_ => 1 } keys %{ plugin_get_default_vtype_subdirs() } };
+    for my $t ($tests->@*) {
+        my $file = $t->{volname};
+        my $expected = $t->{expected};
+        my $description = $t->{description};
 
-foreach my $tt (@tests) {
-    my $file = $tt->{volname};
-    my $expected = $tt->{expected};
-    my $description = $tt->{description};
+        # prepare environment
+        my ($name, $dir, $suffix) = fileparse($file);
+        make_path($dir, { verbose => 1, mode => 0755 });
 
-    # prepare environment
-    my ($name, $dir, $suffix) = fileparse($file);
-    make_path($dir, { verbose => 1, mode => 0755 });
+        if ($name) {
+            open(my $fh, ">>", "$file") || die "Error open file: $!";
+            close($fh);
+        }
 
-    if ($name) {
-        open(my $fh, ">>", "$file") || die "Error open file: $!";
-        close($fh);
+        my $got;
+        eval {
+            # Suppress warnings here to make output less noisy for certain tests
+            local $SIG{__WARN__} = sub { };
+
+            $got = [PVE::Storage::path_to_volume_id($DEFAULT_CFG, $file)];
+        };
+        $got = $@ if $@;
+
+        is_deeply($got, $expected, $description);
+
+        $seen_vtype->{ $expected->[0] } = 1
+            if ($expected->[0] ne '' && scalar($expected->@*) > 1);
     }
 
-    # run tests
-    my $got;
-    eval { $got = [PVE::Storage::path_to_volume_id($scfg, $file)] };
-    $got = $@ if $@;
+    # to check if all $vtype_subdirs are defined in path_to_volume_id
+    # or have a test
+    # FIXME re-enable after vtype split changes
+    #is_deeply($seen_vtype, $vtype_subdirs, "vtype_subdir check");
+    is_deeply({}, {}, "vtype_subdir check");
 
-    is_deeply($got, $expected, $description) || diag(explain($got));
-
-    $seen_vtype->{ @$expected[0] } = 1
-        if (@$expected[0] ne '' && scalar @$expected > 1);
+    return;
 }
 
-# to check if all $vtype_subdirs are defined in path_to_volume_id
-# or have a test
-# FIXME re-enable after vtype split changes
-#is_deeply($seen_vtype, $vtype_subdirs, "vtype_subdir check");
-is_deeply({}, {}, "vtype_subdir check");
+my sub main() {
+    plan tests => scalar($tests->@*) + 1;
 
-#cleanup
-# File::Temp unlinks tempdir on exit
+    run_tests($tests);
 
-done_testing();
+    done_testing();
+
+    return;
+}
+
+main();
 
 1;
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 41/51] test: volume id: rename 'volname' test case parameter to 'file'
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (39 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 40/51] test: volume id: modernize code Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 42/51] test: filesystem path: modernize code Max R. Carrara
                   ` (3 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

... since it's for file paths, not volume names.

Improve the comment describing the test case format along the way.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/path_to_volume_id_test.pm | 72 ++++++++++++++----------------
 1 file changed, 34 insertions(+), 38 deletions(-)

diff --git a/src/test/path_to_volume_id_test.pm b/src/test/path_to_volume_id_test.pm
index 69e0330d..6bc7cc07 100644
--- a/src/test/path_to_volume_id_test.pm
+++ b/src/test/path_to_volume_id_test.pm
@@ -37,217 +37,213 @@ my $DEFAULT_CFG = {
     'order' => {
         'local' => 1,
     },
 };
 
-# the tests array consists of hashes with the following keys:
-# description => to identify the test case
-# volname     => to create the test file
-# expected    => the result that path_to_volume_id should return
+# Each test is comprised of the following keys:
+# description => to identify a single test
+# file        => the absolute path to the volume's file that should be parsed
+# expected    => expected output of path_to_volume_id(), as listref
 my $tests = [
     {
         description => 'Image, qcow2',
-        volname => "$DEFAULT_STORAGE_DIR/images/16110/vm-16110-disk-0.qcow2",
+        file => "$DEFAULT_STORAGE_DIR/images/16110/vm-16110-disk-0.qcow2",
         expected => [
             'vm-vol', 'local:16110/vm-16110-disk-0.qcow2',
         ],
     },
     {
         description => 'Image, raw',
-        volname => "$DEFAULT_STORAGE_DIR/images/16112/vm-16112-disk-0.raw",
+        file => "$DEFAULT_STORAGE_DIR/images/16112/vm-16112-disk-0.raw",
         expected => [
             'vm-vol', 'local:16112/vm-16112-disk-0.raw',
         ],
     },
     {
         description => 'Image template, qcow2',
-        volname => "$DEFAULT_STORAGE_DIR/images/9004/base-9004-disk-0.qcow2",
+        file => "$DEFAULT_STORAGE_DIR/images/9004/base-9004-disk-0.qcow2",
         expected => [
             'vm-vol', 'local:9004/base-9004-disk-0.qcow2',
         ],
     },
 
     {
         description => 'Backup, vma.gz',
-        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz",
+        file => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz",
         expected => [
             'backup', 'local:backup/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz',
         ],
     },
     {
         description => 'Backup, vma.lzo',
-        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo",
+        file => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo",
         expected => [
             'backup', 'local:backup/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo',
         ],
     },
     {
         description => 'Backup, vma',
-        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma",
+        file => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma",
         expected => [
             'backup', 'local:backup/vzdump-qemu-16110-2020_03_30-21_13_55.vma',
         ],
     },
     {
         description => 'Backup, tar.lzo',
-        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo",
+        file => "$DEFAULT_STORAGE_DIR/dump/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo",
         expected => [
             'backup', 'local:backup/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo',
         ],
     },
     {
         description => 'Backup, vma.zst',
-        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst",
+        file => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst",
         expected => [
             'backup', 'local:backup/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst',
         ],
     },
     {
         description => 'Backup, tar.zst',
-        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-lxc-16112-2020_03_30-21_39_30.tar.zst",
+        file => "$DEFAULT_STORAGE_DIR/dump/vzdump-lxc-16112-2020_03_30-21_39_30.tar.zst",
         expected => [
             'backup', 'local:backup/vzdump-lxc-16112-2020_03_30-21_39_30.tar.zst',
         ],
     },
     {
         description => 'Backup, tar.bz2',
-        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2",
+        file => "$DEFAULT_STORAGE_DIR/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2",
         expected => [
             'backup', 'local:backup/vzdump-openvz-16112-2020_03_30-21_39_30.tar.bz2',
         ],
     },
 
     {
         description => 'ISO file',
-        volname => "$DEFAULT_STORAGE_DIR/template/iso/yet-again-a-installation-disk.iso",
+        file => "$DEFAULT_STORAGE_DIR/template/iso/yet-again-a-installation-disk.iso",
         expected => [
             'iso', 'local:iso/yet-again-a-installation-disk.iso',
         ],
     },
     {
         description => 'CT template, tar.gz',
-        volname =>
-            "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.gz",
+        file => "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.gz",
         expected => [
             'vztmpl', 'local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz',
         ],
     },
     {
         description => 'CT template, wrong ending, tar bz2',
-        volname =>
-            "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.bz2",
+        file => "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.bz2",
         expected => [
             'vztmpl', 'local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.bz2',
         ],
     },
 
     {
         description => 'Rootdir, folder subvol, legacy naming',
-        volname => "$DEFAULT_STORAGE_DIR/images/1234/subvol-1234-disk-0.subvol/", # fileparse needs / at the end
+        file => "$DEFAULT_STORAGE_DIR/images/1234/subvol-1234-disk-0.subvol/", # fileparse needs / at the end
         expected => [
             'ct-vol', 'local:1234/subvol-1234-disk-0.subvol',
         ],
     },
     {
         description => 'Rootdir, folder subvol',
-        volname => "$DEFAULT_STORAGE_DIR/images/1234/subvol-1234-disk-0.subvol/", # fileparse needs / at the end
+        file => "$DEFAULT_STORAGE_DIR/images/1234/subvol-1234-disk-0.subvol/", # fileparse needs / at the end
         expected => [
             'ct-vol', 'local:1234/subvol-1234-disk-0.subvol',
         ],
     },
     {
         description => 'Snippets, yaml',
-        volname => "$DEFAULT_STORAGE_DIR/snippets/userconfig.yaml",
+        file => "$DEFAULT_STORAGE_DIR/snippets/userconfig.yaml",
         expected => [
             'snippets', 'local:snippets/userconfig.yaml',
         ],
     },
     {
         description => 'Snippets, hookscript',
-        volname => "$DEFAULT_STORAGE_DIR/snippets/hookscript.pl",
+        file => "$DEFAULT_STORAGE_DIR/snippets/hookscript.pl",
         expected => [
             'snippets', 'local:snippets/hookscript.pl',
         ],
     },
     {
         description => 'CT template, tar.xz',
-        volname =>
-            "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.xz",
+        file => "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.xz",
         expected => [
             'vztmpl', 'local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.xz',
         ],
     },
     {
         description => 'Import, ova',
-        volname => "$DEFAULT_STORAGE_DIR/import/import.ova",
+        file => "$DEFAULT_STORAGE_DIR/import/import.ova",
         expected => [
             'import', 'local:import/import.ova',
         ],
     },
     {
         description => 'Import, ovf',
-        volname => "$DEFAULT_STORAGE_DIR/import/import.ovf",
+        file => "$DEFAULT_STORAGE_DIR/import/import.ovf",
         expected => [
             'import', 'local:import/import.ovf',
         ],
     },
 
     # no matches, path or files with failures
     {
         description => 'Base template, string as vmid in folder name',
-        volname => "$DEFAULT_STORAGE_DIR/images/ssss/base-4321-disk-0.raw",
+        file => "$DEFAULT_STORAGE_DIR/images/ssss/base-4321-disk-0.raw",
         expected => [''],
     },
     {
         description => 'ISO file, wrong ending',
-        volname => "$DEFAULT_STORAGE_DIR/template/iso/yet-again-a-installation-disk.dvd",
+        file => "$DEFAULT_STORAGE_DIR/template/iso/yet-again-a-installation-disk.dvd",
         expected => [''],
     },
     {
         description => 'CT template, wrong ending, zip.gz',
-        volname =>
-            "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.zip.gz",
+        file => "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.zip.gz",
         expected => [''],
     },
     {
         description => 'Backup, wrong format, openvz, zip.gz',
-        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-openvz-16112-2020_03_30-21_39_30.zip.gz",
+        file => "$DEFAULT_STORAGE_DIR/dump/vzdump-openvz-16112-2020_03_30-21_39_30.zip.gz",
         expected => [''],
     },
     {
         description => 'Backup, wrong format, openvz, tgz.lzo',
-        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tgz.lzo",
+        file => "$DEFAULT_STORAGE_DIR/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tgz.lzo",
         expected => [''],
     },
     {
         description => 'Backup, wrong ending, qemu, vma.xz',
-        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz",
+        file => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz",
         expected => [''],
     },
     {
         description => 'Backup, wrong format, qemu, vms.gz',
-        volname => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vms.gz",
+        file => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vms.gz",
         expected => [''],
     },
     {
         description => 'Image, string as vmid in folder name',
-        volname => "$DEFAULT_STORAGE_DIR/images/ssss/vm-1234-disk-0.qcow2",
+        file => "$DEFAULT_STORAGE_DIR/images/ssss/vm-1234-disk-0.qcow2",
         expected => [''],
     },
     {
         description => 'Import, non ova/ovf/disk image in import dir',
-        volname => "$DEFAULT_STORAGE_DIR/import/test.foo",
+        file => "$DEFAULT_STORAGE_DIR/import/test.foo",
         expected => [''],
     },
 ];
 
 my sub run_tests($tests) {
     my $seen_vtype;
     my $vtype_subdirs = { map { $_ => 1 } keys plugin_get_default_vtype_subdirs()->%* };
 
     for my $t ($tests->@*) {
-        my $file = $t->{volname};
+        my $file = $t->{file};
         my $expected = $t->{expected};
         my $description = $t->{description};
 
         # prepare environment
         my ($name, $dir, $suffix) = fileparse($file);
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 42/51] test: filesystem path: modernize code
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (40 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 41/51] test: volume id: rename 'volname' test case parameter to 'file' Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-17 20:39 ` [PATCH pve-storage v3 43/51] plugin: list content directories without globbing Max R. Carrara
                   ` (2 subsequent siblings)
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Modernize the existing code in `filesystem_path_test.pm` while leaving
the tests unchanged otherwise.

In particular,
- move the test execution code into its own subroutine
- add and call a `main()` subroutine
- declare `use v5.36;` instead of `use strict;` and `use warnings;`
- rename / capitalize the sole constant
- adapt the code style to fit our more modern style guide

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/test/filesystem_path_test.pm | 62 +++++++++++++++++++-------------
 1 file changed, 37 insertions(+), 25 deletions(-)

diff --git a/src/test/filesystem_path_test.pm b/src/test/filesystem_path_test.pm
index dd7e4fa7..f5574c65 100644
--- a/src/test/filesystem_path_test.pm
+++ b/src/test/filesystem_path_test.pm
@@ -1,16 +1,16 @@
 package PVE::Storage::TestFilesystemPath;
 
-use strict;
-use warnings;
+use v5.36;
 
 use lib qw(..);
 
 use PVE::Storage;
+
 use Test::More;
 
-my $path = '/some/path';
+my $DEFAULT_STORAGE_DIR = '/some/path';
 
 {
 
     package PVE::Storage::TestFilesystemPath::RootdirPlugin;
 
@@ -30,87 +30,99 @@ my $path = '/some/path';
 my $tests = [
     {
         volname => '1234/vm-1234-disk-0.raw',
         snapname => undef,
         expected => [
-            "$path/images/1234/vm-1234-disk-0.raw", '1234', 'vm-vol',
+            "$DEFAULT_STORAGE_DIR/images/1234/vm-1234-disk-0.raw", '1234', 'vm-vol',
         ],
     },
     {
         volname => '1234/vm-1234-disk-0.raw',
         snapname => 'my_snap',
         expected => "can't snapshot this image format\n",
     },
     {
         volname => '1234/vm-1234-disk-0.qcow2',
         snapname => undef,
         expected => [
-            "$path/images/1234/vm-1234-disk-0.qcow2", '1234', 'vm-vol',
+            "$DEFAULT_STORAGE_DIR/images/1234/vm-1234-disk-0.qcow2", '1234', 'vm-vol',
         ],
     },
     {
         volname => '1234/vm-1234-disk-0.qcow2',
         snapname => 'my_snap',
         expected => [
-            "$path/images/1234/vm-1234-disk-0.qcow2", '1234', 'vm-vol',
+            "$DEFAULT_STORAGE_DIR/images/1234/vm-1234-disk-0.qcow2", '1234', 'vm-vol',
         ],
     },
     {
         volname => '1234/vm-1234-state-my_snap.raw',
         snapname => undef,
         expected => [
-            "$path/images/1234/vm-1234-state-my_snap.raw", '1234', 'vm-state',
+            "$DEFAULT_STORAGE_DIR/images/1234/vm-1234-state-my_snap.raw", '1234', 'vm-state',
         ],
     },
     {
         volname => '1234/subvol-1234-disk-0.subvol',
         snapname => undef,
         expected => [
-            "$path/images/1234/subvol-1234-disk-0.subvol", '1234', 'ct-vol',
+            "$DEFAULT_STORAGE_DIR/images/1234/subvol-1234-disk-0.subvol", '1234', 'ct-vol',
         ],
     },
     {
         volname => 'iso/my-awesome-proxmox.iso',
         snapname => undef,
         expected => [
-            "$path/template/iso/my-awesome-proxmox.iso", undef, 'iso',
+            "$DEFAULT_STORAGE_DIR/template/iso/my-awesome-proxmox.iso", undef, 'iso',
         ],
     },
     {
         volname => "backup/vzdump-qemu-1234-2020_03_30-21_12_40.vma",
         snapname => undef,
         expected => [
-            "$path/dump/vzdump-qemu-1234-2020_03_30-21_12_40.vma", 1234, 'backup',
+            "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-1234-2020_03_30-21_12_40.vma", 1234,
+            'backup',
         ],
     },
     # an external plugin may return 'rootdir', whose volumes are in 'private' instead of 'images'
     {
         plugin => 'PVE::Storage::TestFilesystemPath::RootdirPlugin',
         volname => 'subvol-1234-disk-0',
         snapname => undef,
         expected => [
-            "$path/private/subvol-1234-disk-0", 1234, 'rootdir',
+            "$DEFAULT_STORAGE_DIR/private/subvol-1234-disk-0", 1234, 'rootdir',
         ],
     },
 ];
 
-plan tests => scalar @$tests;
+my sub run_tests($tests) {
+    for my $t ($tests->@*) {
+        my $volname = $t->{volname};
+        my $snapname = $t->{snapname};
+        my $expected = $t->{expected};
+        my $plugin = $t->{plugin} // 'PVE::Storage::Plugin';
+        my $scfg = { path => $DEFAULT_STORAGE_DIR };
+        my $got;
 
-foreach my $tt (@$tests) {
-    my $volname = $tt->{volname};
-    my $snapname = $tt->{snapname};
-    my $expected = $tt->{expected};
-    my $plugin = $tt->{plugin} // 'PVE::Storage::Plugin';
-    my $scfg = { path => $path };
-    my $got;
+        eval { $got = [$plugin->filesystem_path($scfg, $volname, $snapname)]; };
+        $got = $@ if $@;
 
-    eval { $got = [$plugin->filesystem_path($scfg, $volname, $snapname)]; };
-    $got = $@ if $@;
-
-    is_deeply($got, $expected, "wantarray: filesystem_path for $volname")
-        || diag(explain($got));
+        is_deeply($got, $expected, "wantarray: filesystem_path for $volname")
+            || diag(explain($got));
+    }
 
+    return;
 }
 
-done_testing();
+my sub main() {
+    plan tests => scalar($tests->@*);
+
+    run_tests($tests);
+
+    done_testing();
+
+    return;
+}
+
+main();
 
 1;
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 43/51] plugin: list content directories without globbing
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (41 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 42/51] test: filesystem path: modernize code Max R. Carrara
@ 2026-11-17 20:39 ` Max R. Carrara
  2026-11-18  9:12 ` [PATCH pve-storage v3 44/51] fix #2884: implement nested subdir scanning and support 'iso' vtype Max R. Carrara
  2026-11-18  9:14 ` [PATCH pve-storage v3 45/51] fix #2884: support nested subdir scanning for 'vztmpl' volume type Max R. Carrara
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-17 20:39 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Files of the iso, vztmpl, backup, snippets and import content types
were found by globbing the content directory with `<$dir/*>`. Perl's
glob splits its pattern on whitespace and expands braces and the tilde
in it, so for a storage path containing such characters the listing
silently came up empty.

Only the listing of these content types changes. VM and container
images are still found by globbing in list_images(), so storage paths
containing such characters remain unsupported for them.

Read the directory entries with readdir instead, skipping hidden
entries and sorting them like the glob did, so the result is otherwise
unchanged.

This also prepares for scanning nested subdirectories, whose names may
contain whitespace as well.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Plugin.pm     | 14 +++++++++++++-
 src/test/list_volumes_test.pm | 31 +++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+), 1 deletion(-)

diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 73e43cea..b8f415a8 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -1868,10 +1868,22 @@ sub list_images {
     }
 
     return $res;
 }
 
+# Returns the paths of all non-hidden entries in $dir, sorted like a `<$dir/*>` glob would. Unlike
+# such a glob, this does not split $dir on whitespace or expand any glob metacharacters in it.
+my sub list_dir_entries {
+    my ($dir) = @_;
+
+    opendir(my $dh, $dir) or return ();
+    my @names = grep { !m/^\./ } readdir($dh);
+    closedir($dh);
+
+    return map { "$dir/$_" } sort { lc($a) cmp lc($b) || $a cmp $b } @names;
+}
+
 # $vtype = <iso|vztmpl|backup|snippets|import>
 my sub get_subdir_files {
     my ($class, $storeid, $scfg, $vtype, $vmid) = @_;
 
     # FIXME: Call plugin_get_vtype_subdir() instead of $class->get_subdir()
@@ -1958,11 +1970,11 @@ my sub get_subdir_files {
         }
 
         return;
     };
 
-    for my $path (<$vtype_subdir/*>) {
+    for my $path (list_dir_entries($vtype_subdir)) {
         my $st = File::stat::stat($path);
 
         # A volume ID cannot represent a name with a newline, see parse_volume_id().
         next if $path =~ m/\n/;
 
diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm
index 76831de9..4932dbbf 100644
--- a/src/test/list_volumes_test.pm
+++ b/src/test/list_volumes_test.pm
@@ -75,10 +75,12 @@ my $DEFAULT_SCFG = {
         backup => 1,
         import => 1,
     },
 };
 
+my $SPECIAL_CHARS_STORAGE_PATH = "$DEFAULT_STORAGE_PATH/path with {glob} chars";
+
 my @BACKING_FILE_SUFFIXES = ('qcow2', 'raw', 'vmdk', 'vhdx');
 
 =head2 TEST CASE FORMAT
 
 The parameters for individual test cases are hashes with the following
@@ -1349,10 +1351,39 @@ my $test_param_list = [
                 file => "$DEFAULT_STORAGE_PATH/import/import.ova",
                 expected => undef,
             },
         ],
     },
+    {
+        description => 'Storage path containing whitespace and glob metacharacters',
+        storeid => $DEFAULT_STOREID,
+        scfg => { $DEFAULT_SCFG->%*, path => $SPECIAL_CHARS_STORAGE_PATH },
+        vmid => undef,
+        vtypes => ['iso', 'snippets'],
+        cases => [
+            {
+                file => "$SPECIAL_CHARS_STORAGE_PATH/template/iso/debian.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/debian.iso',
+                },
+            },
+            {
+                file => "$SPECIAL_CHARS_STORAGE_PATH/snippets/userconfig.yaml",
+                expected => {
+                    content => 'snippets',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'snippet',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:snippets/userconfig.yaml',
+                },
+            },
+        ],
+    },
 ];
 
 # Additional test cases that cannot be constructed within the list above
 {
     my $file_name = "vzdump-qemu-16110-2020_03_30-21_13_55.vma";
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 44/51] fix #2884: implement nested subdir scanning and support 'iso' vtype
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (42 preceding siblings ...)
  2026-11-17 20:39 ` [PATCH pve-storage v3 43/51] plugin: list content directories without globbing Max R. Carrara
@ 2026-11-18  9:12 ` Max R. Carrara
  2026-11-18  9:14 ` [PATCH pve-storage v3 45/51] fix #2884: support nested subdir scanning for 'vztmpl' volume type Max R. Carrara
  44 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-18  9:12 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Introduce a new 'max-scan-depth' property for directory-based
storages. This property makes it so that files for specific volume
types may also be put into (nested) subdirectories inside the volume
type's directory.

The maximum allowed depth is between 0 and 50, with 0 being the
default, which corresponds to the current behavior. The property is
available for directory, NFS, CIFS, CephFS and BTRFS storages.

Support nested subdirectories for ISOs (the 'iso' vtype) only for now.

Achieve all of this by calling the `get_subdir_files()` helper in
`PVE::Storage::Plugin` recursively. Note that Perl only warns about
deep recursion from a depth of 100 on [0], way above the maximum value
of 50 for the 'max-scan-depth' property. Should we need deeper
nesting, we can always make the subroutine iterative instead of
recursive later.

Symbolic links to directories are never followed. They may lead
outside of the volume type's directory, or back into a directory that
is already being scanned. Such loops would list the same files over
and over until the maximum depth is reached, with the number of
results growing exponentially with the number of looping links.

Do not descend into the content directory of another content type of
the storage either. With 'content-dirs' overrides, such a directory
may be nested inside the scanned one, for example with
'iso=data,snippets=data/snippets'. Its files belong to the other
content type and would otherwise be listed twice. Directories are
compared by device and inode number so that extreme edge cases (like
bind mounts e.g.) are handled. Directories of content types the
storage does not declare are scanned like any other.

Give the property a title and mark it as advanced in the plugin data
of the plugins that support it, like the other properties shown in
the advanced section of the storage edit window.

Add additional test cases wherever applicable to account for nested
subdirectories, including cases that check whether the limit set by
the property is upheld, and also cases that check for the existence of
parent directory references ('..') in volume names.

[0]: https://perldoc.perl.org/perl5101delta#Deep-recursion-on-subroutine-%22%25s%22

Originally-by: Noel Ullreich <n.ullreich@proxmox.com>
Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/BTRFSPlugin.pm              |   1 +
 src/PVE/Storage/CIFSPlugin.pm               |   1 +
 src/PVE/Storage/CephFSPlugin.pm             |   1 +
 src/PVE/Storage/Common/Parse.pm             |  18 ++
 src/PVE/Storage/Common/test/parser_tests.pl |  50 +++++
 src/PVE/Storage/DirPlugin.pm                |   1 +
 src/PVE/Storage/NFSPlugin.pm                |   1 +
 src/PVE/Storage/Plugin.pm                   | 104 ++++++++-
 src/test/filesystem_path_test.pm            |   8 +
 src/test/list_volumes_test.pm               | 225 +++++++++++++++++++-
 src/test/parse_volname_test.pm              |  56 +++++
 src/test/path_to_volume_id_test.pm          |   7 +
 12 files changed, 468 insertions(+), 5 deletions(-)

diff --git a/src/PVE/Storage/BTRFSPlugin.pm b/src/PVE/Storage/BTRFSPlugin.pm
index 1c81b7b8..518df37b 100644
--- a/src/PVE/Storage/BTRFSPlugin.pm
+++ b/src/PVE/Storage/BTRFSPlugin.pm
@@ -82,10 +82,11 @@ sub properties {
 }
 
 sub options {
     return {
         path => { fixed => 1 },
+        'max-scan-depth' => { optional => 1 },
         nodes => { optional => 1 },
         shared => { optional => 1 },
         disable => { optional => 1 },
         'prune-backups' => { optional => 1 },
         'max-protected-backups' => { optional => 1 },
diff --git a/src/PVE/Storage/CIFSPlugin.pm b/src/PVE/Storage/CIFSPlugin.pm
index 15d5a22d..45a0c5ac 100644
--- a/src/PVE/Storage/CIFSPlugin.pm
+++ b/src/PVE/Storage/CIFSPlugin.pm
@@ -183,10 +183,11 @@ sub properties {
 
 sub options {
     return {
         path => { fixed => 1 },
         'content-dirs' => { optional => 1 },
+        'max-scan-depth' => { optional => 1 },
         server => { fixed => 1 },
         share => { fixed => 1 },
         subdir => { optional => 1 },
         nodes => { optional => 1 },
         disable => { optional => 1 },
diff --git a/src/PVE/Storage/CephFSPlugin.pm b/src/PVE/Storage/CephFSPlugin.pm
index 82a4351f..4b751945 100644
--- a/src/PVE/Storage/CephFSPlugin.pm
+++ b/src/PVE/Storage/CephFSPlugin.pm
@@ -153,10 +153,11 @@ sub properties {
 
 sub options {
     return {
         path => { fixed => 1 },
         'content-dirs' => { optional => 1 },
+        'max-scan-depth' => { optional => 1 },
         monhost => { optional => 1 },
         nodes => { optional => 1 },
         subdir => { optional => 1 },
         disable => { optional => 1 },
         options => { optional => 1 },
diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
index 2f6380a8..2b6ae70e 100644
--- a/src/PVE/Storage/Common/Parse.pm
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -58,10 +58,14 @@ my $RE_OVA_CONTENT_EXTENSIONS = join_to_re_alternations(@OVA_CONTENT_EXTENSIONS)
 
 my $RE_SAFE_CHAR_CLASS = qr/[a-zA-Z0-9\-\.\+\=\_]/;
 
 my $RE_SAFE_CHAR_WITH_WHITESPACE_CLASS = qr/[ a-zA-Z0-9\-\.\+\=\_]/;
 
+my $RE_DIRECTORY_COMPONENTS = qr!
+    ( ($RE_SAFE_CHAR_WITH_WHITESPACE_CLASS)+ / )*
+!xn;
+
 my $RE_PARENT_DIR = quotemeta('..');
 my $RE_CONTAINS_PARENT_DIR = qr!
     ( ^$RE_PARENT_DIR$ )  #  ..  --> Entire path
     |
     ( ^$RE_PARENT_DIR/ )  #  ../ --> Beginning of path
@@ -76,10 +80,11 @@ my $RE_CONTAINS_PARENT_DIR = qr!
 # See also: https://git.proxmox.com/?p=pve-common.git;a=blob;f=src/PVE/JSONSchema.pm;h=17e7126a6c42f8326a1da890680af10b6106d906;hb=refs/heads/master#l304
 my $RE_VMID = qr![1-9][0-9]{2,8}!;
 
 my $RE_ISO_FILE_PATH = qr!
     (?<path>
+        (?<dir> $RE_DIRECTORY_COMPONENTS )?
         (?<file> [^/]+ \. (?<ext> (?i: iso|img) ) )
     )
 !xn;
 
 my $RE_VZTMPL_FILE_PATH = qr!
@@ -226,10 +231,19 @@ my sub format_named_groups(%groups) {
         $result->{$new_key} = $groups{$old_key};
     }
 
     my @disk_path_components = ();
 
+    if (defined($result->{dir})) {
+        if ($result->{dir} eq '') {
+            delete $result->{dir};
+        } else {
+            $result->{dir} = strip_trailing_path_separators($result->{dir});
+            push(@disk_path_components, $result->{dir});
+        }
+    }
+
     if (defined($result->{file})) {
         $result->{file} = strip_leading_path_separators($result->{file});
         push(@disk_path_components, $result->{file});
     }
 
@@ -329,10 +343,14 @@ These are:
 
         # The volname, which is "${vtype}/${path}"
         volname => 'iso/custom-debian.iso',
     }
 
+Volume types that support subdirectories in their C<path> part also have a
+C<dir> part if a subdirectory is specified. This is currently limited to the
+C<iso> volume type.
+
 The following volume types have more specific parts in addition to the common
 ones:
 
 =over
 
diff --git a/src/PVE/Storage/Common/test/parser_tests.pl b/src/PVE/Storage/Common/test/parser_tests.pl
index af0a7052..cb8d706f 100755
--- a/src/PVE/Storage/Common/test/parser_tests.pl
+++ b/src/PVE/Storage/Common/test/parser_tests.pl
@@ -71,21 +71,71 @@ my $volname_cases_iso_valid = [
             path => 'Fedora.Img',
             vtype => 'iso',
             volname => 'iso/Fedora.Img',
         },
     },
+
+    # subdirectories
+    {
+        path => 'subdir/custom-debian.iso',
+        expected => {
+            file => 'custom-debian.iso',
+            ext => 'iso',
+            dir => 'subdir',
+            'disk-path' => 'subdir/custom-debian.iso',
+            path => 'subdir/custom-debian.iso',
+            vtype => 'iso',
+            volname => 'iso/subdir/custom-debian.iso',
+        },
+    },
+    {
+        path => 'deeply/nested/dir/hannah-montana-linux.IMG',
+        expected => {
+            file => 'hannah-montana-linux.IMG',
+            ext => 'IMG',
+            dir => 'deeply/nested/dir',
+            'disk-path' => 'deeply/nested/dir/hannah-montana-linux.IMG',
+            path => 'deeply/nested/dir/hannah-montana-linux.IMG',
+            vtype => 'iso',
+            volname => 'iso/deeply/nested/dir/hannah-montana-linux.IMG',
+        },
+    },
 ];
 
 my $volname_cases_iso_invalid = [
     {
         description => "Invalid file extension (iso)",
         args => {
             path => 'MacOS-kittycat.dmg',
             vtype => 'iso',
         },
         expected => undef,
     },
+    {
+        description => "Parent dir reference in path (beginning) (iso)",
+        args => {
+            path => '../custom-debian.iso',
+            vtype => 'iso',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (middle) (iso)",
+        args => {
+            path => 'subdir/../custom-debian.iso',
+            vtype => 'iso',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (end) (iso)",
+        args => {
+            path => 'subdir/custom-debian.iso/..',
+            vtype => 'iso',
+        },
+        expected => undef,
+    },
 ];
 
 my $volname_cases_vztmpl_valid = [
     # plain paths
     {
diff --git a/src/PVE/Storage/DirPlugin.pm b/src/PVE/Storage/DirPlugin.pm
index 7df7a849..1cf2fbe9 100644
--- a/src/PVE/Storage/DirPlugin.pm
+++ b/src/PVE/Storage/DirPlugin.pm
@@ -100,10 +100,11 @@ sub properties {
 
 sub options {
     return {
         path => { fixed => 1 },
         'content-dirs' => { optional => 1 },
+        'max-scan-depth' => { optional => 1 },
         nodes => { optional => 1 },
         shared => { optional => 1 },
         disable => { optional => 1 },
         'prune-backups' => { optional => 1 },
         'max-protected-backups' => { optional => 1 },
diff --git a/src/PVE/Storage/NFSPlugin.pm b/src/PVE/Storage/NFSPlugin.pm
index 18b470c4..82cd2806 100644
--- a/src/PVE/Storage/NFSPlugin.pm
+++ b/src/PVE/Storage/NFSPlugin.pm
@@ -106,10 +106,11 @@ sub properties {
 
 sub options {
     return {
         path => { fixed => 1 },
         'content-dirs' => { optional => 1 },
+        'max-scan-depth' => { optional => 1 },
         server => { fixed => 1 },
         export => { fixed => 1 },
         nodes => { optional => 1 },
         disable => { optional => 1 },
         'prune-backups' => { optional => 1 },
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index b8f415a8..37fdc921 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -1,10 +1,12 @@
 package PVE::Storage::Plugin;
 
 use strict;
 use warnings;
 
+use Carp qw(confess);
+
 use Cwd qw(abs_path);
 use Encode qw(decode);
 use Fcntl ':mode';
 use File::chdir;
 use File::Path;
@@ -269,10 +271,20 @@ my $defaultData = {
             description => "Overrides for default content type directories.",
             type => "string",
             format => "pve-dir-override-list",
             optional => 1,
         },
+        'max-scan-depth' => {
+            title => "Maximum Directory Scan Depth",
+            description => "Maximum depth of subdirectories to traverse when searching for"
+                . " ISOs in directories.",
+            type => 'integer',
+            default => 0,
+            minimum => 0,
+            maximum => 50,
+            optional => 1,
+        },
         options => {
             title => "Mount Options",
             description => "Comma-separated list of mount options."
                 . " See 'man nfs', 'man mount.cifs', or 'man mount.ceph' for more information.",
             type => 'string',
@@ -1882,15 +1894,22 @@ my sub list_dir_entries {
     return map { "$dir/$_" } sort { lc($a) cmp lc($b) || $a cmp $b } @names;
 }
 
 # $vtype = <iso|vztmpl|backup|snippets|import>
 my sub get_subdir_files {
-    my ($class, $storeid, $scfg, $vtype, $vmid) = @_;
+    use feature 'current_sub'; # Needed for the __SUB__ token further below
+
+    my ($class, $storeid, $scfg, $vtype, $vmid, $remaining_depth, $current_path, $skip_dirs) = @_;
 
     # FIXME: Call plugin_get_vtype_subdir() instead of $class->get_subdir()
     # and remove the $class parameter on the next APIAGE reset.
     my $vtype_subdir = $class->get_subdir($scfg, $vtype);
+
+    if (!defined($current_path)) {
+        $current_path = $vtype_subdir;
+    }
+
     my $res = [];
 
     my $get_subdir_file_info = sub {
         my ($path, $st) = @_;
 
@@ -1970,17 +1989,90 @@ my sub get_subdir_files {
         }
 
         return;
     };
 
-    for my $path (list_dir_entries($vtype_subdir)) {
+    my $get_dir_id_from_stat = sub {
+        my ($stat_res) = @_;
+        return $stat_res->dev . ':' . $stat_res->ino;
+    };
+
+    # The content directories of the other content types of the storage
+    # may be nested inside this one, for example with 'content-dirs'
+    # overrides. Their files belong to the other content type, so we
+    # gotta make sure to not descend into them.
+    #
+    # Note that we only look the content dirs up once there is a
+    # directory to descend into.
+    if (!defined($skip_dirs)) {
+        $skip_dirs = {};
+
+        # NOTE: In the loop below we do not need to handle `images`,
+        # `rootdir`, `vm-state`, etc. types, because those are fetched
+        # using `list_images()`.
+
+        my $declared_vtypes = $scfg->{content} // {};
+        my $known_vtype_subdirs = plugin_get_default_vtype_subdirs();
+
+        for my $declared_vtype (keys $declared_vtypes->%*) {
+            # The dir of the vtype being listed is the one we are scanning.
+            next if $declared_vtype eq $vtype;
+
+            # Ignore vtypes without a directory of their own, e.g. 'none'
+            next if !exists($known_vtype_subdirs->{$declared_vtype});
+
+            # FIXME: Call plugin_get_vtype_subdir() instead of
+            # $class->get_subdir() on the next APIAGE reset.
+            my $declared_vtype_subdir = $class->get_subdir($scfg, $declared_vtype);
+
+            # Skip directories that do not exist (yet).
+            my $vtype_subdir_stat = File::stat::stat($declared_vtype_subdir);
+            next if !defined($vtype_subdir_stat);
+
+            # Identify the directory by device and inode, so that we
+            # don't iterate through the same dir more than once in
+            # extreme edge cases, like bind mounts and symlinks in the
+            # configured paths themselves.
+            my $dir_id = $get_dir_id_from_stat->($vtype_subdir_stat);
+            $skip_dirs->{$dir_id} = 1;
+        }
+    }
+
+    for my $path (list_dir_entries($current_path)) {
         my $st = File::stat::stat($path);
 
         # A volume ID cannot represent a name with a newline, see parse_volume_id().
         next if $path =~ m/\n/;
 
-        next if (!$st || S_ISDIR($st->mode));
+        next if !$st;
+
+        if (S_ISDIR($st->mode)) {
+            # Never follow symlinks to directories. They may lead outside of
+            # the vtype subdir or back into a directory that is already being
+            # scanned, listing the same files over and over until the maximum
+            # depth is reached.
+            next if -l $path;
+
+            next if !defined($remaining_depth) || $remaining_depth <= 0;
+
+            next if $skip_dirs->{ $get_dir_id_from_stat->($st) };
+
+            my $inner_res = __SUB__->(
+                $class,
+                $storeid,
+                $scfg,
+                $vtype,
+                $vmid,
+                $remaining_depth - 1,
+                $path,
+                $skip_dirs,
+            );
+
+            push($res->@*, $inner_res->@*);
+
+            next;
+        }
 
         if (defined(my $info = $get_subdir_file_info->($path, $st))) {
             $info->{size} = $st->size;
             $info->{ctime} //= $st->ctime;
 
@@ -1994,10 +2086,14 @@ my sub get_subdir_files {
 # If attributes are set on a volume, they should be included in the result.
 # See get_volume_attribute for a list of possible attributes.
 sub list_volumes {
     my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
 
+    my $depth = $scfg->{'max-scan-depth'} // 0;
+    $depth = 0 if $depth < 0;
+    $depth = 50 if $depth > 50;
+
     my $res = [];
     my $vmlist = PVE::Cluster::get_vmlist();
 
     my $get_raw_volumes_for_type = sub {
         my ($type) = @_;
@@ -2011,11 +2107,11 @@ sub list_volumes {
         }
 
         return if !$scfg->{path};
 
         if ($type eq 'iso' && !defined($vmid)) {
-            return get_subdir_files($class, $storeid, $scfg, 'iso', undef);
+            return get_subdir_files($class, $storeid, $scfg, 'iso', undef, $depth);
         }
 
         if ($type eq 'vztmpl' && !defined($vmid)) {
             return get_subdir_files($class, $storeid, $scfg, 'vztmpl', undef);
         }
diff --git a/src/test/filesystem_path_test.pm b/src/test/filesystem_path_test.pm
index f5574c65..8b3b9fdf 100644
--- a/src/test/filesystem_path_test.pm
+++ b/src/test/filesystem_path_test.pm
@@ -73,10 +73,18 @@ my $tests = [
         snapname => undef,
         expected => [
             "$DEFAULT_STORAGE_DIR/template/iso/my-awesome-proxmox.iso", undef, 'iso',
         ],
     },
+    {
+        volname => 'iso/foo/bar/baz/my-awesome-proxmox.iso',
+        snapname => undef,
+        expected => [
+            "$DEFAULT_STORAGE_DIR/template/iso/foo/bar/baz/my-awesome-proxmox.iso", undef,
+            'iso',
+        ],
+    },
     {
         volname => "backup/vzdump-qemu-1234-2020_03_30-21_12_40.vma",
         snapname => undef,
         expected => [
             "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-1234-2020_03_30-21_12_40.vma", 1234,
diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm
index 4932dbbf..59d8a56e 100644
--- a/src/test/list_volumes_test.pm
+++ b/src/test/list_volumes_test.pm
@@ -11,11 +11,11 @@ use PVE::Tools qw(run_command);
 use Test::More;
 use Test::MockModule;
 
 use Carp qw(confess);
 use Cwd;
-use File::Basename qw(fileparse);
+use File::Basename qw(dirname fileparse);
 use File::Path qw(make_path remove_tree);
 use File::stat qw();
 use File::Temp;
 use Storable qw(dclone);
 
@@ -136,10 +136,21 @@ keys:
                     vmid => 1234,
                     volid => 'local:1234/vm-1234-disk-0.qcow2',
                 },
             },
         ],
+
+        # Optional list of symbolic links to create after the files above.
+        symlinks => [
+            {
+                # Absolute path of the symbolic link.
+                link => "/tmp/foobar/template/iso/latest",
+
+                # Target of the symbolic link, used verbatim.
+                target => 'debian-13',
+            },
+        ],
     }
 
 =cut
 
 my $test_param_list = [
@@ -1351,10 +1362,212 @@ my $test_param_list = [
                 file => "$DEFAULT_STORAGE_PATH/import/import.ova",
                 expected => undef,
             },
         ],
     },
+    {
+        description => "VMID: none, no nested subdirectories when using defaults",
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => undef,
+        vtypes => ['iso', 'vztmpl', 'snippets', 'import'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/some-installer.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/some-installer.iso",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/2/3/4/5/some-installer.iso",
+                expected => undef,
+            },
+        ],
+    },
+    {
+        description => "VMID: none, nested subdirectories allowed, max-scan-depth = 1",
+        storeid => $DEFAULT_STOREID,
+        scfg => {
+            type => 'dir',
+            path => $DEFAULT_STORAGE_PATH,
+            shared => 0,
+            'max-scan-depth' => 1,
+            content => {
+                iso => 1,
+                vztmpl => 1,
+                snippets => 1,
+                import => 1,
+            },
+        },
+        vmid => undef,
+        vtypes => ['iso', 'vztmpl', 'snippets', 'import'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/some-installer.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/1/some-installer.iso',
+                },
+            },
+            {
+                # Exceeds max-scan-depth
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/2/some-installer.iso",
+                expected => undef,
+            },
+        ],
+    },
+    {
+        description => "VMID: none, nested subdirectories allowed, max-scan-depth = 5",
+        storeid => $DEFAULT_STOREID,
+        scfg => {
+            type => 'dir',
+            path => $DEFAULT_STORAGE_PATH,
+            shared => 0,
+            'max-scan-depth' => 5,
+            content => {
+                iso => 1,
+                vztmpl => 1,
+                snippets => 1,
+                import => 1,
+            },
+        },
+        vmid => undef,
+        vtypes => ['iso', 'vztmpl', 'snippets', 'import'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/some-installer.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/1/some-installer.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/2/3/4/5/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/1/2/3/4/5/some-installer.iso',
+                },
+            },
+            {
+                # Exceeds max-scan-depth
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/2/3/4/5/6/some-installer.iso",
+                expected => undef,
+            },
+        ],
+    },
+    {
+        description => "VMID: none, max-scan-depth = 2, nested content dir of snippets",
+        storeid => $DEFAULT_STOREID,
+        scfg => {
+            $DEFAULT_SCFG->%*,
+            'max-scan-depth' => 2,
+            'content-dirs' => { iso => 'data', snippets => 'data/snippets' },
+        },
+        vmid => undef,
+        vtypes => ['iso'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/data/a.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/a.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/data/sub/b.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/sub/b.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/data/snippets/seed.iso",
+                expected => undef,
+            },
+        ],
+    },
+    {
+        description => "VMID: none, max-scan-depth = 8, symlinks to directories",
+        storeid => $DEFAULT_STOREID,
+        scfg => { $DEFAULT_SCFG->%*, 'max-scan-depth' => 8 },
+        vmid => undef,
+        vtypes => ['iso'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/some-installer.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/1/some-installer.iso',
+                },
+            },
+        ],
+        # None of these are followed
+        symlinks => [
+            { link => "$DEFAULT_STORAGE_PATH/template/iso/alias", target => '1' },
+            { link => "$DEFAULT_STORAGE_PATH/template/iso/outside", target => '/' },
+            { link => "$DEFAULT_STORAGE_PATH/template/iso/loop-a", target => '.' },
+            { link => "$DEFAULT_STORAGE_PATH/template/iso/loop-b", target => '.' },
+            { link => "$DEFAULT_STORAGE_PATH/template/iso/1/loop-c", target => '..' },
+            { link => "$DEFAULT_STORAGE_PATH/template/iso/1/loop-d", target => '../1' },
+        ],
+    },
     {
         description => 'Storage path containing whitespace and glob metacharacters',
         storeid => $DEFAULT_STOREID,
         scfg => { $DEFAULT_SCFG->%*, path => $SPECIAL_CHARS_STORAGE_PATH },
         vmid => undef,
@@ -1580,10 +1793,13 @@ $mock_cluster->redefine(get_vmlist => sub { return $mocked_vmlist; });
 # an blessed array.
 my $mock_stat = Test::MockModule->new('File::stat', no_auto => 1);
 $mock_stat->redefine(
     populate => sub {
         my (@st) = @_;
+
+        return $mock_stat->original('populate')->() if !@st; # stat failed
+
         $st[7] = $DEFAULT_SIZE;
         $st[10] = $DEFAULT_CTIME;
 
         my $result = $mock_stat->original('populate')->(@st);
 
@@ -1633,10 +1849,17 @@ my sub setup_test_env($test_params) {
 
             run_command($command, timeout => 10);
         }
     }
 
+    for my $symlink (($test_params->{symlinks} // [])->@*) {
+        my ($link, $target) = $symlink->@{qw(link target)};
+
+        make_path(dirname($link), { verbose => 1, mode => 0755 });
+        symlink($target, $link) or die "failed to create symlink '$link' - $!\n";
+    }
+
     return;
 }
 
 my sub teardown_test_env($test_params) {
     my $scfg = $test_params->{scfg};
diff --git a/src/test/parse_volname_test.pm b/src/test/parse_volname_test.pm
index 5520d3c6..a9900152 100644
--- a/src/test/parse_volname_test.pm
+++ b/src/test/parse_volname_test.pm
@@ -202,14 +202,70 @@ my $tests = [
             {
                 description => "ISO image, $suffix",
                 volname => "iso/$file_name",
                 expected => ['iso', "$file_name", undef, undef, undef, undef, 'raw'],
             },
+            {
+                description => "ISO image, $suffix, subdirectory",
+                volname => "iso/foo/$file_name",
+                expected => [
+                    'iso', "foo/$file_name", undef, undef, undef, undef, 'raw',
+                ],
+            },
+            {
+                description => "ISO image, $suffix, nested subdirectories",
+                volname => "iso/foo/bar/baz/$file_name",
+                expected => [
+                    'iso', "foo/bar/baz/$file_name", undef, undef, undef, undef, 'raw',
+                ],
+            },
+            {
+                description => "ISO image, $suffix, subdirectory with same name as file",
+                volname => "iso/$file_name/$file_name",
+                expected => [
+                    'iso', "$file_name/$file_name", undef, undef, undef, undef, 'raw',
+                ],
+            },
         );
 
         push($tests->@*, @extra_tests);
     }
+
+    # Expected to fail
+    {
+        my $file_name = "$prefix.iso";
+
+        my @extra_failed_tests = (
+            {
+                description =>
+                    "ISO image, iso, parent directory reference before volume type prefix",
+                volname => "../iso/$file_name",
+                expected => "unable to parse directory volume name '../iso/$file_name'\n",
+            },
+            {
+                description =>
+                    "ISO image, iso, parent directory reference at beginning of volume path",
+                volname => "iso/../$file_name",
+                expected => "unable to parse directory volume name 'iso/../$file_name'\n",
+            },
+            {
+                description =>
+                    "ISO image, iso, parent directory reference at end of volume path",
+                volname => "iso/$file_name/..",
+                expected => "unable to parse directory volume name 'iso/$file_name/..'\n",
+            },
+            {
+                description =>
+                    "ISO image, iso, parent directory reference between dir components of volume path",
+                volname => "iso/foo/../bar/$file_name",
+                expected =>
+                    "unable to parse directory volume name 'iso/foo/../bar/$file_name'\n",
+            },
+        );
+
+        push($tests->@*, @extra_failed_tests);
+    }
 }
 
 # Test cases for container templates
 {
     my $suffixes = ['tar', 'tar.gz', 'tar.xz', 'tar.bz2'];
diff --git a/src/test/path_to_volume_id_test.pm b/src/test/path_to_volume_id_test.pm
index 6bc7cc07..c6c1b2e1 100644
--- a/src/test/path_to_volume_id_test.pm
+++ b/src/test/path_to_volume_id_test.pm
@@ -121,10 +121,17 @@ my $tests = [
         file => "$DEFAULT_STORAGE_DIR/template/iso/yet-again-a-installation-disk.iso",
         expected => [
             'iso', 'local:iso/yet-again-a-installation-disk.iso',
         ],
     },
+    {
+        description => 'ISO file, nested subdirectories',
+        file => "$DEFAULT_STORAGE_DIR/template/iso/foo/bar/hannah-montana-linux-installer.iso",
+        expected => [
+            'iso', 'local:iso/foo/bar/hannah-montana-linux-installer.iso',
+        ],
+    },
     {
         description => 'CT template, tar.gz',
         file => "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.gz",
         expected => [
             'vztmpl', 'local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz',
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 45/51] fix #2884: support nested subdir scanning for 'vztmpl' volume type
  2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
                   ` (43 preceding siblings ...)
  2026-11-18  9:12 ` [PATCH pve-storage v3 44/51] fix #2884: implement nested subdir scanning and support 'iso' vtype Max R. Carrara
@ 2026-11-18  9:14 ` Max R. Carrara
  2026-11-18  9:14   ` [PATCH pve-storage v3 46/51] fix #2884: support nested subdir scanning for 'snippets' vtype Max R. Carrara
                     ` (5 more replies)
  44 siblings, 6 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-18  9:14 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Add support for parsing nested subdirectories for the 'vztmpl' volume
type by adapting its corresponding regex used by parsing helpers.

Add additional test cases wherever applicable to account for nested
subdirectories for 'vztmpl' volumes, as done for the 'iso' volume
type.

Originally-by: Noel Ullreich <n.ullreich@proxmox.com>
Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Common/Parse.pm             |  3 +-
 src/PVE/Storage/Common/test/parser_tests.pl | 54 ++++++++++++++
 src/PVE/Storage/Plugin.pm                   |  4 +-
 src/test/filesystem_path_test.pm            | 18 +++++
 src/test/list_volumes_test.pm               | 81 +++++++++++++++++++++
 src/test/parse_volname_test.pm              | 57 +++++++++++++++
 src/test/path_to_volume_id_test.pm          |  8 ++
 7 files changed, 222 insertions(+), 3 deletions(-)

diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
index 2b6ae70e..4137d09d 100644
--- a/src/PVE/Storage/Common/Parse.pm
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -87,10 +87,11 @@ my $RE_ISO_FILE_PATH = qr!
     )
 !xn;
 
 my $RE_VZTMPL_FILE_PATH = qr!
     (?<path>
+        (?<dir> $RE_DIRECTORY_COMPONENTS )?
         (?<file>
             [^/]+
             \.
             (?<ext>
                 (?<ext_archive> (?i: tar) )
@@ -345,11 +346,11 @@ These are:
         volname => 'iso/custom-debian.iso',
     }
 
 Volume types that support subdirectories in their C<path> part also have a
 C<dir> part if a subdirectory is specified. This is currently limited to the
-C<iso> volume type.
+C<iso> and C<vztmpl> volume types.
 
 The following volume types have more specific parts in addition to the common
 ones:
 
 =over
diff --git a/src/PVE/Storage/Common/test/parser_tests.pl b/src/PVE/Storage/Common/test/parser_tests.pl
index cb8d706f..20c57885 100755
--- a/src/PVE/Storage/Common/test/parser_tests.pl
+++ b/src/PVE/Storage/Common/test/parser_tests.pl
@@ -228,21 +228,75 @@ my $volname_cases_vztmpl_valid = [
             path => 'Alpine 3.10 default_20190626_amd64.Tar.xZ',
             vtype => 'vztmpl',
             volname => 'vztmpl/Alpine 3.10 default_20190626_amd64.Tar.xZ',
         },
     },
+
+    # subdirectories
+    {
+        path => 'subdir/debian-10.0-standard_10.0-1_amd64.tar.zst',
+        expected => {
+            file => 'debian-10.0-standard_10.0-1_amd64.tar.zst',
+            ext => 'tar.zst',
+            'ext-archive' => 'tar',
+            'ext-compression' => 'zst',
+            dir => 'subdir',
+            'disk-path' => 'subdir/debian-10.0-standard_10.0-1_amd64.tar.zst',
+            path => 'subdir/debian-10.0-standard_10.0-1_amd64.tar.zst',
+            vtype => 'vztmpl',
+            volname => 'vztmpl/subdir/debian-10.0-standard_10.0-1_amd64.tar.zst',
+        },
+    },
+    {
+        path => 'deeply/nested/dir/debian-11.0-standard_11.0-1_amd64.tar.bz2',
+        expected => {
+            file => 'debian-11.0-standard_11.0-1_amd64.tar.bz2',
+            ext => 'tar.bz2',
+            'ext-archive' => 'tar',
+            'ext-compression' => 'bz2',
+            dir => 'deeply/nested/dir',
+            'disk-path' => 'deeply/nested/dir/debian-11.0-standard_11.0-1_amd64.tar.bz2',
+            path => 'deeply/nested/dir/debian-11.0-standard_11.0-1_amd64.tar.bz2',
+            vtype => 'vztmpl',
+            volname => 'vztmpl/deeply/nested/dir/debian-11.0-standard_11.0-1_amd64.tar.bz2',
+        },
+    },
 ];
 
 my $volname_cases_vztmpl_invalid = [
     {
         description => "Invalid file extension (vztmpl)",
         args => {
             path => 'archlinux-base_20190924-1_amd64.zip',
             vtype => 'vztmpl',
         },
         expected => undef,
     },
+    {
+        description => "Parent dir reference in path (beginning) (vztmpl)",
+        args => {
+            path => '../archlinux-base_20190924-1_amd64.tar.gz',
+            vtype => 'vztmpl',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (middle) (vztmpl)",
+        args => {
+            path => 'subdir/../archlinux-base_20190924-1_amd64.tar.gz',
+            vtype => 'vztmpl',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (end) (vztmpl)",
+        args => {
+            path => 'subdir/archlinux-base_20190924-1_amd64.tar.gz/..',
+            vtype => 'vztmpl',
+        },
+        expected => undef,
+    },
 ];
 
 my $volname_cases_backup_valid = [
     {
         path => 'vzdump-qemu-16110-2020_03_30-21_13_55.vma',
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 37fdc921..c932ec16 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -274,11 +274,11 @@ my $defaultData = {
             optional => 1,
         },
         'max-scan-depth' => {
             title => "Maximum Directory Scan Depth",
             description => "Maximum depth of subdirectories to traverse when searching for"
-                . " ISOs in directories.",
+                . " ISOs and container templates in directories.",
             type => 'integer',
             default => 0,
             minimum => 0,
             maximum => 50,
             optional => 1,
@@ -2111,11 +2111,11 @@ sub list_volumes {
         if ($type eq 'iso' && !defined($vmid)) {
             return get_subdir_files($class, $storeid, $scfg, 'iso', undef, $depth);
         }
 
         if ($type eq 'vztmpl' && !defined($vmid)) {
-            return get_subdir_files($class, $storeid, $scfg, 'vztmpl', undef);
+            return get_subdir_files($class, $storeid, $scfg, 'vztmpl', undef, $depth);
         }
 
         if ($type eq 'backup') {
             return get_subdir_files($class, $storeid, $scfg, 'backup', $vmid);
         }
diff --git a/src/test/filesystem_path_test.pm b/src/test/filesystem_path_test.pm
index 8b3b9fdf..a511f4ef 100644
--- a/src/test/filesystem_path_test.pm
+++ b/src/test/filesystem_path_test.pm
@@ -81,10 +81,28 @@ my $tests = [
         expected => [
             "$DEFAULT_STORAGE_DIR/template/iso/foo/bar/baz/my-awesome-proxmox.iso", undef,
             'iso',
         ],
     },
+    {
+        volname => 'vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz',
+        snapname => undef,
+        expected => [
+            "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.gz",
+            undef,
+            'vztmpl',
+        ],
+    },
+    {
+        volname => 'vztmpl/foo/bar/baz/debian-10.0-standard_10.0-1_amd64.tar.gz',
+        snapname => undef,
+        expected => [
+            "$DEFAULT_STORAGE_DIR/template/cache/foo/bar/baz/debian-10.0-standard_10.0-1_amd64.tar.gz",
+            undef,
+            'vztmpl',
+        ],
+    },
     {
         volname => "backup/vzdump-qemu-1234-2020_03_30-21_12_40.vma",
         snapname => undef,
         expected => [
             "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-1234-2020_03_30-21_12_40.vma", 1234,
diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm
index 59d8a56e..6d8d4faf 100644
--- a/src/test/list_volumes_test.pm
+++ b/src/test/list_volumes_test.pm
@@ -1387,10 +1387,29 @@ my $test_param_list = [
             },
             {
                 file => "$DEFAULT_STORAGE_PATH/template/iso/1/2/3/4/5/some-installer.iso",
                 expected => undef,
             },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/cache/some-lxc-template.tar.gz",
+                expected => {
+                    content => 'vztmpl',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'tgz',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:vztmpl/some-lxc-template.tar.gz',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/cache/1/some-lxc-template.tar.gz",
+                expected => undef,
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/template/cache/1/2/3/4/5/some-lxc-template.tar.gz",
+                expected => undef,
+            },
         ],
     },
     {
         description => "VMID: none, nested subdirectories allowed, max-scan-depth = 1",
         storeid => $DEFAULT_STOREID,
@@ -1432,10 +1451,35 @@ my $test_param_list = [
             {
                 # Exceeds max-scan-depth
                 file => "$DEFAULT_STORAGE_PATH/template/iso/1/2/some-installer.iso",
                 expected => undef,
             },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/cache/some-lxc-template.tar.gz",
+                expected => {
+                    content => 'vztmpl',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'tgz',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:vztmpl/some-lxc-template.tar.gz',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/cache/1/some-lxc-template.tar.gz",
+                expected => {
+                    content => 'vztmpl',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'tgz',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:vztmpl/1/some-lxc-template.tar.gz',
+                },
+            },
+            {
+                # Exceeds max-scan-depth
+                file => "$DEFAULT_STORAGE_PATH/template/cache/1/2/some-lxc-template.tar.gz",
+                expected => undef,
+            },
         ],
     },
     {
         description => "VMID: none, nested subdirectories allowed, max-scan-depth = 5",
         storeid => $DEFAULT_STOREID,
@@ -1487,10 +1531,47 @@ my $test_param_list = [
             {
                 # Exceeds max-scan-depth
                 file => "$DEFAULT_STORAGE_PATH/template/iso/1/2/3/4/5/6/some-installer.iso",
                 expected => undef,
             },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/cache/some-lxc-template.tar.gz",
+                expected => {
+                    content => 'vztmpl',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'tgz',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:vztmpl/some-lxc-template.tar.gz',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/cache/1/some-lxc-template.tar.gz",
+                expected => {
+                    content => 'vztmpl',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'tgz',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:vztmpl/1/some-lxc-template.tar.gz',
+                },
+            },
+            {
+                file =>
+                    "$DEFAULT_STORAGE_PATH/template/cache/1/2/3/4/5/some-lxc-template.tar.gz",
+                expected => {
+                    content => 'vztmpl',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'tgz',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:vztmpl/1/2/3/4/5/some-lxc-template.tar.gz',
+                },
+            },
+            {
+                # Exceeds max-scan-depth
+                file =>
+                    "$DEFAULT_STORAGE_PATH/template/cache/1/2/3/4/5/6/some-lxc-template.tar.gz",
+                expected => undef,
+            },
         ],
     },
     {
         description => "VMID: none, max-scan-depth = 2, nested content dir of snippets",
         storeid => $DEFAULT_STOREID,
diff --git a/src/test/parse_volname_test.pm b/src/test/parse_volname_test.pm
index a9900152..37f2f705 100644
--- a/src/test/parse_volname_test.pm
+++ b/src/test/parse_volname_test.pm
@@ -280,14 +280,71 @@ my $tests = [
                 volname => "vztmpl/$file_name",
                 expected => [
                     'vztmpl', "$file_name", undef, undef, undef, undef, 'raw',
                 ],
             },
+            {
+                description => "Container template, $suffix, subdirectory",
+                volname => "vztmpl/foo/$file_name",
+                expected => [
+                    'vztmpl', "foo/$file_name", undef, undef, undef, undef, 'raw',
+                ],
+            },
+            {
+                description => "Container template, $suffix, nested subdirectories",
+                volname => "vztmpl/foo/bar/baz/$file_name",
+                expected => [
+                    'vztmpl', "foo/bar/baz/$file_name", undef, undef, undef, undef, 'raw',
+                ],
+            },
+            {
+                description =>
+                    "Container template, $suffix, subdirectory with same name as file",
+                volname => "vztmpl/$file_name/$file_name",
+                expected => [
+                    'vztmpl', "$file_name/$file_name", undef, undef, undef, undef, 'raw',
+                ],
+            },
         );
 
         push($tests->@*, @extra_tests);
     }
+
+    # Expected to fail
+    {
+        my $file_name = "$prefix.tar.gz";
+
+        my @extra_failed_tests = (
+            {
+                description =>
+                    "Container template, tar.gz, parent directory reference before volume type prefix",
+                volname => "../vztmpl/$file_name",
+                expected => "unable to parse directory volume name '../vztmpl/$file_name'\n",
+            },
+            {
+                description =>
+                    "Container template, tar.gz, parent directory reference at beginning of volume path",
+                volname => "vztmpl/../$file_name",
+                expected => "unable to parse directory volume name 'vztmpl/../$file_name'\n",
+            },
+            {
+                description =>
+                    "Container template, tar.gz, parent directory reference at end of volume path",
+                volname => "vztmpl/$file_name/..",
+                expected => "unable to parse directory volume name 'vztmpl/$file_name/..'\n",
+            },
+            {
+                description =>
+                    "Container template, tar.gz, parent directory reference between dir components of volume path",
+                volname => "vztmpl/foo/../bar/$file_name",
+                expected =>
+                    "unable to parse directory volume name 'vztmpl/foo/../bar/$file_name'\n",
+            },
+        );
+
+        push($tests->@*, @extra_failed_tests);
+    }
 }
 
 # Additional tests for backup files
 {
     my $bkp_suffixes = {
diff --git a/src/test/path_to_volume_id_test.pm b/src/test/path_to_volume_id_test.pm
index c6c1b2e1..2f45c9b4 100644
--- a/src/test/path_to_volume_id_test.pm
+++ b/src/test/path_to_volume_id_test.pm
@@ -135,10 +135,18 @@ my $tests = [
         file => "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.gz",
         expected => [
             'vztmpl', 'local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz',
         ],
     },
+    {
+        description => 'CT template, tar.gz, nested subdirectories',
+        file =>
+            "$DEFAULT_STORAGE_DIR/template/cache/foo/bar/debian-10.0-standard_10.0-1_amd64.tar.gz",
+        expected => [
+            'vztmpl', 'local:vztmpl/foo/bar/debian-10.0-standard_10.0-1_amd64.tar.gz',
+        ],
+    },
     {
         description => 'CT template, wrong ending, tar bz2',
         file => "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.bz2",
         expected => [
             'vztmpl', 'local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.bz2',
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 46/51] fix #2884: support nested subdir scanning for 'snippets' vtype
  2026-11-18  9:14 ` [PATCH pve-storage v3 45/51] fix #2884: support nested subdir scanning for 'vztmpl' volume type Max R. Carrara
@ 2026-11-18  9:14   ` Max R. Carrara
  2026-11-18  9:14   ` [PATCH pve-storage v3 47/51] test: add more tests for 'import' vtype & guard against nested subdirs Max R. Carrara
                     ` (4 subsequent siblings)
  5 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-18  9:14 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Add support for parsing nested subdirectories for the 'snippets'
volume type by adapting its corresponding regex used by parsing
helpers.

Add additional test cases wherever applicable to account for nested
subdirectories for 'snippets' volumes, as was done for the 'iso' and
'vztmpl' volume types.

Originally-by: Noel Ullreich <n.ullreich@proxmox.com>
Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Common/Parse.pm             |  3 +-
 src/PVE/Storage/Common/test/parser_tests.pl | 52 ++++++++++++++
 src/PVE/Storage/Plugin.pm                   |  4 +-
 src/test/filesystem_path_test.pm            | 16 +++++
 src/test/list_volumes_test.pm               | 78 +++++++++++++++++++++
 src/test/parse_volname_test.pm              | 57 +++++++++++++++
 src/test/path_to_volume_id_test.pm          |  7 ++
 7 files changed, 214 insertions(+), 3 deletions(-)

diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
index 4137d09d..8ec2236d 100644
--- a/src/PVE/Storage/Common/Parse.pm
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -140,10 +140,11 @@ my $RE_BACKUP_FILE_PATH = qr!
     )
 !xn;
 
 my $RE_SNIPPETS_FILE_PATH = qr!
     (?<path>
+        (?<dir> $RE_DIRECTORY_COMPONENTS )?
         (?<file> [^/]+ )
     )
 !xn;
 
 my $RE_IMPORT_FILE_PATH = qr!
@@ -346,11 +347,11 @@ These are:
         volname => 'iso/custom-debian.iso',
     }
 
 Volume types that support subdirectories in their C<path> part also have a
 C<dir> part if a subdirectory is specified. This is currently limited to the
-C<iso> and C<vztmpl> volume types.
+C<iso>, C<vztmpl> and C<snippets> volume types.
 
 The following volume types have more specific parts in addition to the common
 ones:
 
 =over
diff --git a/src/PVE/Storage/Common/test/parser_tests.pl b/src/PVE/Storage/Common/test/parser_tests.pl
index 20c57885..44aa7e88 100755
--- a/src/PVE/Storage/Common/test/parser_tests.pl
+++ b/src/PVE/Storage/Common/test/parser_tests.pl
@@ -590,10 +590,61 @@ my $volname_cases_snippets_valid = [
             path => 'userconfig.yaml',
             vtype => 'snippets',
             volname => 'snippets/userconfig.yaml',
         },
     },
+
+    # subdirectories
+    {
+        path => 'subdir/hookscript.pl',
+        expected => {
+            file => 'hookscript.pl',
+            dir => 'subdir',
+            'disk-path' => 'subdir/hookscript.pl',
+            path => 'subdir/hookscript.pl',
+            vtype => 'snippets',
+            volname => 'snippets/subdir/hookscript.pl',
+        },
+    },
+    {
+        path => 'deeply/nested/subdir/hookscript.pl',
+        expected => {
+            file => 'hookscript.pl',
+            dir => 'deeply/nested/subdir',
+            'disk-path' => 'deeply/nested/subdir/hookscript.pl',
+            path => 'deeply/nested/subdir/hookscript.pl',
+            vtype => 'snippets',
+            volname => 'snippets/deeply/nested/subdir/hookscript.pl',
+        },
+    },
+];
+
+my $volname_cases_snippets_invalid = [
+    {
+        description => "Parent dir reference in path (beginning) (snippets)",
+        args => {
+            path => '../hookscript.pl',
+            vtype => 'snippets',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (middle) (snippets)",
+        args => {
+            path => 'subdir/../hookscript.pl',
+            vtype => 'snippets',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (end) (snippets)",
+        args => {
+            path => 'subdir/hookscript.pl/..',
+            vtype => 'snippets',
+        },
+        expected => undef,
+    },
 ];
 
 my $volname_cases_import_valid = [
     {
         path => 'import.ova',
@@ -716,10 +767,11 @@ my $cases_valid_all = [
 
 my $cases_invalid_all = [
     $volname_cases_iso_invalid,
     $volname_cases_vztmpl_invalid,
     $volname_cases_backup_invalid,
+    $volname_cases_snippets_invalid,
     $volname_cases_import_invalid,
     $volname_cases_newline_invalid,
 ];
 
 {
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index c932ec16..9c57cf37 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -274,11 +274,11 @@ my $defaultData = {
             optional => 1,
         },
         'max-scan-depth' => {
             title => "Maximum Directory Scan Depth",
             description => "Maximum depth of subdirectories to traverse when searching for"
-                . " ISOs and container templates in directories.",
+                . " ISOs, container templates and snippets in directories.",
             type => 'integer',
             default => 0,
             minimum => 0,
             maximum => 50,
             optional => 1,
@@ -2119,11 +2119,11 @@ sub list_volumes {
         if ($type eq 'backup') {
             return get_subdir_files($class, $storeid, $scfg, 'backup', $vmid);
         }
 
         if ($type eq 'snippets') {
-            return get_subdir_files($class, $storeid, $scfg, 'snippets', undef);
+            return get_subdir_files($class, $storeid, $scfg, 'snippets', undef, $depth);
         }
 
         if ($type eq 'import') {
             return get_subdir_files($class, $storeid, $scfg, 'import', undef);
         }
diff --git a/src/test/filesystem_path_test.pm b/src/test/filesystem_path_test.pm
index a511f4ef..06789187 100644
--- a/src/test/filesystem_path_test.pm
+++ b/src/test/filesystem_path_test.pm
@@ -116,10 +116,26 @@ my $tests = [
         snapname => undef,
         expected => [
             "$DEFAULT_STORAGE_DIR/private/subvol-1234-disk-0", 1234, 'rootdir',
         ],
     },
+    {
+        volname => 'snippets/userconfig.yaml',
+        snapname => undef,
+        expected => ["$DEFAULT_STORAGE_DIR/snippets/userconfig.yaml", undef, 'snippets'],
+    },
+    {
+        volname => 'snippets/hookscript.pl',
+        snapname => undef,
+        expected => ["$DEFAULT_STORAGE_DIR/snippets/hookscript.pl", undef, 'snippets'],
+    },
+    {
+        volname => 'snippets/foo/bar/baz/something.txt',
+        snapname => undef,
+        expected =>
+            ["$DEFAULT_STORAGE_DIR/snippets/foo/bar/baz/something.txt", undef, 'snippets'],
+    },
 ];
 
 my sub run_tests($tests) {
     for my $t ($tests->@*) {
         my $volname = $t->{volname};
diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm
index 6d8d4faf..c93b1634 100644
--- a/src/test/list_volumes_test.pm
+++ b/src/test/list_volumes_test.pm
@@ -1406,10 +1406,28 @@ my $test_param_list = [
             {
                 file =>
                     "$DEFAULT_STORAGE_PATH/template/cache/1/2/3/4/5/some-lxc-template.tar.gz",
                 expected => undef,
             },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/some-hookscript.pl",
+                expected => {
+                    content => 'snippets',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'snippet',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:snippets/some-hookscript.pl',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/1/some-hookscript.pl",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/1/2/3/4/5/some-hookscript.pl",
+                expected => undef,
+            },
         ],
     },
     {
         description => "VMID: none, nested subdirectories allowed, max-scan-depth = 1",
         storeid => $DEFAULT_STOREID,
@@ -1476,10 +1494,35 @@ my $test_param_list = [
             {
                 # Exceeds max-scan-depth
                 file => "$DEFAULT_STORAGE_PATH/template/cache/1/2/some-lxc-template.tar.gz",
                 expected => undef,
             },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/some-hookscript.pl",
+                expected => {
+                    content => 'snippets',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'snippet',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:snippets/some-hookscript.pl',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/1/some-hookscript.pl",
+                expected => {
+                    content => 'snippets',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'snippet',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:snippets/1/some-hookscript.pl',
+                },
+            },
+            {
+                # Exceeds max-scan-depth
+                file => "$DEFAULT_STORAGE_PATH/snippets/1/2/some-hookscript.pl",
+                expected => undef,
+            },
         ],
     },
     {
         description => "VMID: none, nested subdirectories allowed, max-scan-depth = 5",
         storeid => $DEFAULT_STOREID,
@@ -1568,10 +1611,45 @@ my $test_param_list = [
                 # Exceeds max-scan-depth
                 file =>
                     "$DEFAULT_STORAGE_PATH/template/cache/1/2/3/4/5/6/some-lxc-template.tar.gz",
                 expected => undef,
             },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/some-hookscript.pl",
+                expected => {
+                    content => 'snippets',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'snippet',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:snippets/some-hookscript.pl',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/1/some-hookscript.pl",
+                expected => {
+                    content => 'snippets',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'snippet',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:snippets/1/some-hookscript.pl',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/snippets/1/2/3/4/5/some-hookscript.pl",
+                expected => {
+                    content => 'snippets',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'snippet',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:snippets/1/2/3/4/5/some-hookscript.pl',
+                },
+            },
+            {
+                # Exceeds max-scan-depth
+                file => "$DEFAULT_STORAGE_PATH/snippets/1/2/3/4/5/6/some-hookscript.pl",
+                expected => undef,
+            },
         ],
     },
     {
         description => "VMID: none, max-scan-depth = 2, nested content dir of snippets",
         storeid => $DEFAULT_STOREID,
diff --git a/src/test/parse_volname_test.pm b/src/test/parse_volname_test.pm
index 37f2f705..f2aa0344 100644
--- a/src/test/parse_volname_test.pm
+++ b/src/test/parse_volname_test.pm
@@ -694,14 +694,71 @@ for my $plugin_name (qw(ZFSPlugin ZFSPoolPlugin)) {
             {
                 description => "Snippets, $file_name",
                 volname => "snippets/$file_name",
                 expected => ['snippets', $file_name, undef, undef, undef, undef, 'raw'],
             },
+            {
+                description => "Snippets, $file_name, subdirectory",
+                volname => "snippets/foo/$file_name",
+                expected => ['snippets', "foo/$file_name", undef, undef, undef, undef, 'raw'],
+            },
+            {
+                description => "Snippets, $file_name, nested subdirectories",
+                volname => "snippets/foo/bar/baz/$file_name",
+                expected =>
+                    ['snippets', "foo/bar/baz/$file_name", undef, undef, undef, undef, 'raw'],
+            },
+            {
+                description => "Snippets, $file_name, subdirectory with same name as file",
+                volname => "snippets/$file_name/$file_name",
+                expected =>
+                    ['snippets', "$file_name/$file_name", undef, undef, undef, undef, 'raw'],
+            },
         );
 
         push($tests->@*, @extra_tests);
     }
+
+    # Expected to fail
+    {
+        my $file_name = "some-file.txt";
+
+        my @extra_failed_tests = (
+            {
+                description =>
+                    "Snippets, $file_name, parent directory reference before volume type prefix",
+                volname => "../snippets/$file_name",
+                expected => "unable to parse directory volume name '../snippets/$file_name'\n",
+            },
+            {
+                description =>
+                    "Snippets, $file_name, parent directory reference at beginning of volume path",
+                volname => "snippets/../$file_name",
+                expected => "unable to parse directory volume name 'snippets/../$file_name'\n",
+            },
+            {
+                description => "Snippets, parent directory reference as entire path",
+                volname => "snippets/..",
+                expected => "unable to parse directory volume name 'snippets/..'\n",
+            },
+            {
+                description =>
+                    "Snippets, $file_name, parent directory reference at end of volume path",
+                volname => "snippets/$file_name/..",
+                expected => "unable to parse directory volume name 'snippets/$file_name/..'\n",
+            },
+            {
+                description =>
+                    "Snippets, $file_name, parent directory reference between dir components of volume path",
+                volname => "snippets/foo/../bar/$file_name",
+                expected =>
+                    "unable to parse directory volume name 'snippets/foo/../bar/$file_name'\n",
+            },
+        );
+
+        push($tests->@*, @extra_failed_tests);
+    }
 }
 
 # Test cases for import files
 {
     my $suffixes = ['ova', 'ovf', 'qcow2', 'vmdk', 'raw'];
diff --git a/src/test/path_to_volume_id_test.pm b/src/test/path_to_volume_id_test.pm
index 2f45c9b4..dcd05111 100644
--- a/src/test/path_to_volume_id_test.pm
+++ b/src/test/path_to_volume_id_test.pm
@@ -179,10 +179,17 @@ my $tests = [
         file => "$DEFAULT_STORAGE_DIR/snippets/hookscript.pl",
         expected => [
             'snippets', 'local:snippets/hookscript.pl',
         ],
     },
+    {
+        description => 'Snippets, arbitrary file, nested subdirectories',
+        file => "$DEFAULT_STORAGE_DIR/snippets/foo/bar/baz/something.txt",
+        expected => [
+            'snippets', 'local:snippets/foo/bar/baz/something.txt',
+        ],
+    },
     {
         description => 'CT template, tar.xz',
         file => "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.xz",
         expected => [
             'vztmpl', 'local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.xz',
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 47/51] test: add more tests for 'import' vtype & guard against nested subdirs
  2026-11-18  9:14 ` [PATCH pve-storage v3 45/51] fix #2884: support nested subdir scanning for 'vztmpl' volume type Max R. Carrara
  2026-11-18  9:14   ` [PATCH pve-storage v3 46/51] fix #2884: support nested subdir scanning for 'snippets' vtype Max R. Carrara
@ 2026-11-18  9:14   ` Max R. Carrara
  2026-11-18  9:14   ` [PATCH pve-storage v3 48/51] test: add tests guarding against subdir scanning for vtypes Max R. Carrara
                     ` (3 subsequent siblings)
  5 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-18  9:14 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Supporting nested subdirectories for the 'import' volume type is
currently not possible, because of a parsing ambiguity that would be
introduced. In particular, it would not be possible to differentiate
whether 'foo.ova' in 'foo.ova/bar.qcow2' refers to a directory ending
with '.ova' or an actual '.ova' file through parsing alone. Note this
down in a comment as well, so it doesn't get overlooked.

Therefore, instead of supporting nested subdirectories for the
'import' volume type, add tests that explicitly guard against
subdirectory scanning support.

Additionally, toss in a few extra tests for the `filesystem_path()`
plugin API method and also for the differences in whitespace handling
for file paths and volume names for the 'import' volume type.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Common/Parse.pm             |   7 ++
 src/PVE/Storage/Common/test/parser_tests.pl |  40 +++++++
 src/test/filesystem_path_test.pm            |  14 +++
 src/test/list_volumes_test.pm               |  65 +++++++++++
 src/test/parse_volname_test.pm              | 120 ++++++++++++++++++++
 src/test/path_to_volume_id_test.pm          |  13 +++
 6 files changed, 259 insertions(+)

diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
index 8ec2236d..0ecf7378 100644
--- a/src/PVE/Storage/Common/Parse.pm
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -145,10 +145,17 @@ my $RE_SNIPPETS_FILE_PATH = qr!
         (?<dir> $RE_DIRECTORY_COMPONENTS )?
         (?<file> [^/]+ )
     )
 !xn;
 
+# NOTE: Supporting subdirectories would introduce a parsing ambiguity with
+# volume names; in particular, there would be no way to tell whether 'foo.ova'
+# in 'foo.ova/bar.qcow2' refers to a directory or a file just through parsing
+# alone.
+#
+# Therefore, neither RE_IMPORT_FILE_PATH nor RE_IMPORT_VOLNAME currently
+# accept subdirectory components.
 my $RE_IMPORT_FILE_PATH = qr!
     (?<path>
         (?<file> ($RE_SAFE_CHAR_CLASS)+ \. (?<ext> $RE_IMPORT_EXTENSIONS) )
     )
 !xn;
diff --git a/src/PVE/Storage/Common/test/parser_tests.pl b/src/PVE/Storage/Common/test/parser_tests.pl
index 44aa7e88..ad3651e0 100755
--- a/src/PVE/Storage/Common/test/parser_tests.pl
+++ b/src/PVE/Storage/Common/test/parser_tests.pl
@@ -726,10 +726,50 @@ my $volname_cases_import_invalid = [
             path => '🐪perl-playground🐪.ova',
             vtype => 'import',
         },
         expected => undef,
     },
+    {
+        description => "Subdirectory (import)",
+        args => {
+            path => 'subdir/disk.ovf',
+            vtype => 'import',
+        },
+        expected => undef,
+    },
+    {
+        description => "Nested subdirectory (import)",
+        args => {
+            path => 'deeply/nested/subdir/disk.raw',
+            vtype => 'import',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (beginning) (import)",
+        args => {
+            path => '../disk.ova',
+            vtype => 'import',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (middle) (import)",
+        args => {
+            path => 'subdir/../disk.raw',
+            vtype => 'import',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (end) (import)",
+        args => {
+            path => 'subdir/disk.qcow2/..',
+            vtype => 'import',
+        },
+        expected => undef,
+    },
 ];
 
 my $volname_cases_newline_invalid = [
     {
         description => "Trailing newline (iso)",
diff --git a/src/test/filesystem_path_test.pm b/src/test/filesystem_path_test.pm
index 06789187..c85ab554 100644
--- a/src/test/filesystem_path_test.pm
+++ b/src/test/filesystem_path_test.pm
@@ -132,10 +132,24 @@ my $tests = [
         volname => 'snippets/foo/bar/baz/something.txt',
         snapname => undef,
         expected =>
             ["$DEFAULT_STORAGE_DIR/snippets/foo/bar/baz/something.txt", undef, 'snippets'],
     },
+    {
+        volname => 'import/import.ova',
+        snapname => undef,
+        expected => [
+            "$DEFAULT_STORAGE_DIR/import/import.ova", undef, 'import',
+        ],
+    },
+    {
+        volname => 'import/import.ovf',
+        snapname => undef,
+        expected => [
+            "$DEFAULT_STORAGE_DIR/import/import.ovf", undef, 'import',
+        ],
+    },
 ];
 
 my sub run_tests($tests) {
     for my $t ($tests->@*) {
         my $volname = $t->{volname};
diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm
index c93b1634..d72dbd6e 100644
--- a/src/test/list_volumes_test.pm
+++ b/src/test/list_volumes_test.pm
@@ -1424,10 +1424,30 @@ my $test_param_list = [
             },
             {
                 file => "$DEFAULT_STORAGE_PATH/snippets/1/2/3/4/5/some-hookscript.pl",
                 expected => undef,
             },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/some-import.ova",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'ova',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/some-import.ova',
+                },
+            },
+            {
+                # Not allowed regardless
+                file => "$DEFAULT_STORAGE_PATH/import/1/some-import.ova",
+                expected => undef,
+            },
+            {
+                # Not allowed regardless
+                file => "$DEFAULT_STORAGE_PATH/import/1/2/3/4/5/some-import.ova",
+                expected => undef,
+            },
         ],
     },
     {
         description => "VMID: none, nested subdirectories allowed, max-scan-depth = 1",
         storeid => $DEFAULT_STOREID,
@@ -1519,10 +1539,30 @@ my $test_param_list = [
             {
                 # Exceeds max-scan-depth
                 file => "$DEFAULT_STORAGE_PATH/snippets/1/2/some-hookscript.pl",
                 expected => undef,
             },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/some-import.ova",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'ova',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/some-import.ova',
+                },
+            },
+            {
+                # Not allowed
+                file => "$DEFAULT_STORAGE_PATH/import/1/some-import.ova",
+                expected => undef,
+            },
+            {
+                # Not allowed + exceeds max-scan-depth
+                file => "$DEFAULT_STORAGE_PATH/import/1/2/some-import.ova",
+                expected => undef,
+            },
         ],
     },
     {
         description => "VMID: none, nested subdirectories allowed, max-scan-depth = 5",
         storeid => $DEFAULT_STOREID,
@@ -1646,10 +1686,35 @@ my $test_param_list = [
             {
                 # Exceeds max-scan-depth
                 file => "$DEFAULT_STORAGE_PATH/snippets/1/2/3/4/5/6/some-hookscript.pl",
                 expected => undef,
             },
+            {
+                file => "$DEFAULT_STORAGE_PATH/import/some-import.ova",
+                expected => {
+                    content => 'import',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'ova',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:import/some-import.ova',
+                },
+            },
+            {
+                # Not allowed
+                file => "$DEFAULT_STORAGE_PATH/import/1/some-import.ova",
+                expected => undef,
+            },
+            {
+                # Still not allowed
+                file => "$DEFAULT_STORAGE_PATH/import/1/2/3/4/5/some-import.ova",
+                expected => undef,
+            },
+            {
+                # Not allowed + exceeds max-scan-depth
+                file => "$DEFAULT_STORAGE_PATH/import/1/2/3/4/5/6/some-import.ova",
+                expected => undef,
+            },
         ],
     },
     {
         description => "VMID: none, max-scan-depth = 2, nested content dir of snippets",
         storeid => $DEFAULT_STOREID,
diff --git a/src/test/parse_volname_test.pm b/src/test/parse_volname_test.pm
index f2aa0344..8b8e3f8b 100644
--- a/src/test/parse_volname_test.pm
+++ b/src/test/parse_volname_test.pm
@@ -99,10 +99,22 @@ my $tests = [
         description => "Import, inner file of ova with whitespace in name",
         volname => 'import/import.ova/OS disk.vmdk',
         expected =>
             ['import', 'import.ova/OS disk.vmdk', undef, undef, undef, undef, 'ova+vmdk'],
     },
+    {
+        description => "Import, ova file on disk with whitespace in name",
+        volname => 'import/My neat import.ova/disk.vmdk',
+        expected =>
+            ['import', 'My neat import.ova/disk.vmdk', undef, undef, undef, undef, 'ova+vmdk'],
+    },
+    {
+        description => "Import, ova file on disk and inner file of ova with whitespace in name",
+        volname => 'import/My neat import.ova/OS disk.vmdk',
+        expected =>
+            ['import', 'My neat import.ova/OS disk.vmdk', undef, undef, undef, undef, 'ova+vmdk'],
+    },
     #
     # failed matches
     #
     {
         description => "Failed match: VM disk image, base, raw",
@@ -775,10 +787,61 @@ for my $plugin_name (qw(ZFSPlugin ZFSPoolPlugin)) {
             },
         );
 
         push($tests->@*, @extra_tests);
     }
+
+    # Expected to fail
+    {
+        my $file_name = "$prefix.ova";
+
+        my @extra_failed_tests = (
+            {
+                description => "Import, ova, directory",
+                volname => "import/foo/$file_name",
+                expected => "unable to parse directory volume name 'import/foo/$file_name'\n",
+            },
+            {
+                description => "Import, ova, subdirectories",
+                volname => "import/foo/bar/baz/$file_name",
+                expected =>
+                    "unable to parse directory volume name 'import/foo/bar/baz/$file_name'\n",
+            },
+            {
+                description => "Import, ova, directory with same name as file",
+                volname => "import/$file_name/$file_name",
+                expected =>
+                    "unable to parse directory volume name 'import/$file_name/$file_name'\n",
+            },
+            {
+                description =>
+                    "Import, ova, parent directory reference before volume type prefix",
+                volname => "../import/$file_name",
+                expected => "unable to parse directory volume name '../import/$file_name'\n",
+            },
+            {
+                description =>
+                    "Import, ova, parent directory reference at beginning of volume path",
+                volname => "import/../$file_name",
+                expected => "unable to parse directory volume name 'import/../$file_name'\n",
+            },
+            {
+                description => "Import, ova, parent directory reference at end of volume path",
+                volname => "import/$file_name/..",
+                expected => "unable to parse directory volume name 'import/$file_name/..'\n",
+            },
+            {
+                description =>
+                    "Import, ova, parent directory reference between dir components of volume path",
+                volname => "import/foo/../bar/$file_name",
+                expected =>
+                    "unable to parse directory volume name 'import/foo/../bar/$file_name'\n",
+            },
+        );
+
+        push($tests->@*, @extra_failed_tests);
+    }
 }
 
 # Test cases for OVA import files with content inside
 {
     my $content_suffixes = ['qcow2', 'vmdk', 'raw'];
@@ -804,10 +867,67 @@ for my $plugin_name (qw(ZFSPlugin ZFSPoolPlugin)) {
             },
         );
 
         push($tests->@*, @extra_tests);
     }
+
+    # Expected to fail
+    {
+        my $content_file_name = "disk.qcow2";
+
+        my @extra_failed_tests = (
+            {
+                description => "Import, inner file of ova (qcow2), directory",
+                volname => "import/foo/$file_name/$content_file_name",
+                expected =>
+                    "unable to parse directory volume name 'import/foo/$file_name/$content_file_name'\n",
+            },
+            {
+                description => "Import, inner file of ova (qcow2), nested directories",
+                volname => "import/foo/bar/baz/$file_name/$content_file_name",
+                expected =>
+                    "unable to parse directory volume name 'import/foo/bar/baz/$file_name/$content_file_name'\n",
+            },
+            {
+                description =>
+                    "Import, inner file of ova (qcow2), directory with same name as file",
+                volname => "import/$file_name/$file_name/$content_file_name",
+                expected =>
+                    "unable to parse directory volume name 'import/$file_name/$file_name/$content_file_name'\n",
+            },
+            {
+                description => "Import, inner file of ova (qcow2),"
+                    . " parent directory reference before volume type prefix",
+                volname => "../import/$file_name/$content_file_name",
+                expected =>
+                    "unable to parse directory volume name '../import/$file_name/$content_file_name'\n",
+            },
+            {
+                description => "Import, inner file of ova (qcow2),"
+                    . " parent directory reference at beginning of volume path",
+                volname => "import/../$file_name/$content_file_name",
+                expected =>
+                    "unable to parse directory volume name 'import/../$file_name/$content_file_name'\n",
+            },
+            {
+                description => "Import, inner file of ova (qcow2),"
+                    . " parent directory reference at end of disk path",
+                volname => "import/$file_name/../$content_file_name",
+                expected =>
+                    "unable to parse directory volume name 'import/$file_name/../$content_file_name'\n",
+            },
+            {
+                description => "Import, inner file of ova (qcow2),"
+                    . " parent directory reference at end of volume path",
+                volname => "import/$file_name/$content_file_name/..",
+                expected =>
+                    "unable to parse directory volume name 'import/$file_name/$content_file_name/..'\n",
+            },
+        );
+
+        push($tests->@*, @extra_failed_tests);
+    }
 }
 
 my sub run_tests($tests) {
     my $seen_vtype = {};
     my $vtype_subdirs = { map { $_ => 1 } keys plugin_get_default_vtype_subdirs()->%* };
diff --git a/src/test/path_to_volume_id_test.pm b/src/test/path_to_volume_id_test.pm
index dcd05111..3cdf23ae 100644
--- a/src/test/path_to_volume_id_test.pm
+++ b/src/test/path_to_volume_id_test.pm
@@ -254,10 +254,23 @@ my $tests = [
     {
         description => 'Import, non ova/ovf/disk image in import dir',
         file => "$DEFAULT_STORAGE_DIR/import/test.foo",
         expected => [''],
     },
+    {
+        # Allowed in volume names, but not in file paths
+        description => 'Import, ova, whitespace in file name',
+        file => "$DEFAULT_STORAGE_DIR/import/My Import.ova",
+        expected => [''],
+    },
+    {
+        # Supporting subdirectories for the 'import' volume type is
+        # currently not possible because it would introduce parsing ambiguities.
+        description => 'Import, ova, subdirectory',
+        file => "$DEFAULT_STORAGE_DIR/import/foo/import.ova",
+        expected => [''],
+    },
 ];
 
 my sub run_tests($tests) {
     my $seen_vtype;
     my $vtype_subdirs = { map { $_ => 1 } keys plugin_get_default_vtype_subdirs()->%* };
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 48/51] test: add tests guarding against subdir scanning for vtypes
  2026-11-18  9:14 ` [PATCH pve-storage v3 45/51] fix #2884: support nested subdir scanning for 'vztmpl' volume type Max R. Carrara
  2026-11-18  9:14   ` [PATCH pve-storage v3 46/51] fix #2884: support nested subdir scanning for 'snippets' vtype Max R. Carrara
  2026-11-18  9:14   ` [PATCH pve-storage v3 47/51] test: add more tests for 'import' vtype & guard against nested subdirs Max R. Carrara
@ 2026-11-18  9:14   ` Max R. Carrara
  2026-11-18  9:14   ` [PATCH pve-storage v3 49/51] storage: resolve images in content dir overrides ending with a slash Max R. Carrara
                     ` (2 subsequent siblings)
  5 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-18  9:14 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Add tests that guard against implementing nested subdirectory scanning
for the 'backup' volume type in `parser_tests.pl`, and for the
'images' and 'backup' volume types in `path_to_volume_id_test.pm`.

For the 'backup' vtype, also add a comment above the parser's
corresponding regex that we do not allow nested subdirs there.

This is done mainly to prevent implementing subdir scanning blindly in
the future for these two volume types.

While adding support for subdir scanning for the 'backup' volume type
would probably be fairly trivial, there is also no (known) need for it
currently.

For the 'images' type, there currently is no plan or need to support
subdir scanning, and it would probably not make much sense anyway.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/Common/Parse.pm             |  1 +
 src/PVE/Storage/Common/test/parser_tests.pl | 40 +++++++++++++++++++++
 src/test/path_to_volume_id_test.pm          | 11 ++++++
 3 files changed, 52 insertions(+)

diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
index 0ecf7378..35d2007c 100644
--- a/src/PVE/Storage/Common/Parse.pm
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -117,10 +117,11 @@ my $RE_GUEST_VZDUMP_BACKUP_FILE_NAME = qr!
     - (?<type> openvz|lxc|qemu)
     - (?<vmid> $RE_VMID)
     - ( $RE_GUEST_VZDUMP_BACKUP_FILE_TIMESTAMP | [^/]+ )
 !xn;
 
+# NOTE: Not supporting subdirectories here.
 my $RE_BACKUP_FILE_PATH = qr!
     (?<path>
         (?<file>
             (
                 ($RE_GUEST_VZDUMP_BACKUP_FILE_NAME)
diff --git a/src/PVE/Storage/Common/test/parser_tests.pl b/src/PVE/Storage/Common/test/parser_tests.pl
index ad3651e0..27280f13 100755
--- a/src/PVE/Storage/Common/test/parser_tests.pl
+++ b/src/PVE/Storage/Common/test/parser_tests.pl
@@ -566,10 +566,50 @@ my $volname_cases_backup_invalid = [
             path => 'vzdump-qemu-16110-2020_03_30-21_12_40.tar.Gz',
             vtype => 'backup',
         },
         expected => undef,
     },
+    {
+        description => "Subdirectory (backup)",
+        args => {
+            path => 'subdir/vzdump-qemu-16110-2020_03_30-21_12_40.tar.gz',
+            vtype => 'backup',
+        },
+        expected => undef,
+    },
+    {
+        description => "Nested subdirectory (backup)",
+        args => {
+            path => 'deeply/nested/subdir/vzdump-qemu-16110-2020_03_30-21_12_40.tar.gz',
+            vtype => 'backup',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (beginning) (backup)",
+        args => {
+            path => '../vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz',
+            vtype => 'backup',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (middle) (backup)",
+        args => {
+            path => 'subdir/../vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz',
+            vtype => 'backup',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (end) (backup)",
+        args => {
+            path => 'subdir/vzdump-lxc-16112-2020_03_30-21_49_30.tar.gz/..',
+            vtype => 'backup',
+        },
+        expected => undef,
+    },
 ];
 
 my $volname_cases_snippets_valid = [
     # plain files
     {
diff --git a/src/test/path_to_volume_id_test.pm b/src/test/path_to_volume_id_test.pm
index 3cdf23ae..bcd9db9c 100644
--- a/src/test/path_to_volume_id_test.pm
+++ b/src/test/path_to_volume_id_test.pm
@@ -244,15 +244,26 @@ my $tests = [
     {
         description => 'Backup, wrong format, qemu, vms.gz',
         file => "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vms.gz",
         expected => [''],
     },
+    {
+        description => 'Backup, nested subdirectories, vma.gz',
+        file =>
+            "$DEFAULT_STORAGE_DIR/dump/foo/bar/baz/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz",
+        expected => [''],
+    },
     {
         description => 'Image, string as vmid in folder name',
         file => "$DEFAULT_STORAGE_DIR/images/ssss/vm-1234-disk-0.qcow2",
         expected => [''],
     },
+    {
+        description => 'Image, additional nested subdirectories before vmid',
+        file => "$DEFAULT_STORAGE_DIR/images/foo/bar/baz/1234/vm-1234-disk-0.qcow2",
+        expected => [''],
+    },
     {
         description => 'Import, non ova/ovf/disk image in import dir',
         file => "$DEFAULT_STORAGE_DIR/import/test.foo",
         expected => [''],
     },
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 49/51] storage: resolve images in content dir overrides ending with a slash
  2026-11-18  9:14 ` [PATCH pve-storage v3 45/51] fix #2884: support nested subdir scanning for 'vztmpl' volume type Max R. Carrara
                     ` (2 preceding siblings ...)
  2026-11-18  9:14   ` [PATCH pve-storage v3 48/51] test: add tests guarding against subdir scanning for vtypes Max R. Carrara
@ 2026-11-18  9:14   ` Max R. Carrara
  2026-11-18  9:14   ` [PATCH pve-storage v3 50/51] bump the storage plugin API version Max R. Carrara
  2026-11-18  9:14   ` [PATCH pve-manager v3 51/51] fix #2884: ui: storage: add field for 'max-scan-depth' property Max R. Carrara
  5 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-18  9:14 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

The content dir option accepts overrides like 'images=imgs/'. The
`images` branch of `path_to_volume_id()` strips the subdirectory only
if it is followed by a slash, and compares the path of each listed
image with the given path as strings. With a trailing slash, neither
matches, so such images were never resolved, also not before the
parsing rework.

Since the files of other volume types may now be found in nested
subdirectories, such a path falls through to the volume type of an
enclosing content directory instead. With 'snippets=/' and
'images=imgs/', for example, a VM disk can resolve to a snippet.

Strip trailing slashes from the subdirectory and compare the image
paths in canonical form.

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage.pm                 |  3 +-
 src/PVE/Storage/Common.pm          |  5 +-
 src/test/path_to_volume_id_test.pm | 99 ++++++++++++++++++++++++++++++
 3 files changed, 105 insertions(+), 2 deletions(-)

diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index f3d1298d..3cd17993 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -8,10 +8,11 @@ use IO::Select;
 use IO::File;
 use IO::Socket::IP;
 use IPC::Open3;
 use File::Basename;
 use File::Path;
+use File::Spec;
 use Cwd 'abs_path';
 use Socket;
 use Time::Local qw(timelocal);
 
 use PVE::Cmd qw(build_ssh_command);
@@ -815,11 +816,11 @@ sub path_to_volume_id {
 
             my $vollist = $plugin->list_images($sid, $scfg, $vmid);
             for my $info ($vollist->@*) {
                 my ($storeid, $volname) = parse_volume_id($info->{volid});
                 my $volpath = $plugin->path($scfg, $volname, $storeid);
-                if ($volpath eq $path) {
+                if (File::Spec->canonpath($volpath) eq $path) {
                     my ($parsed_vtype) = $plugin->parse_volname($volname);
                     return ($parsed_vtype, $info->{volid});
                 }
             }
 
diff --git a/src/PVE/Storage/Common.pm b/src/PVE/Storage/Common.pm
index d39ee3d0..68464a1a 100644
--- a/src/PVE/Storage/Common.pm
+++ b/src/PVE/Storage/Common.pm
@@ -431,9 +431,12 @@ sub plugin_get_vtype_subdir : prototype($$) ($scfg, $vtype) {
 
     # Intermediate step to prevent autovivification
     my $content_dirs = $scfg->{'content-dirs'} // {};
     my $subdir = $content_dirs->{$vtype} // $DEFAULT_VTYPE_SUBDIRS->{$vtype};
 
-    return "$path/$subdir";
+    # Content dirs may contain a trailing slash, but shouldn't!
+    my $without_trailing_slash = "$path/$subdir" =~ s!/+$!!r;
+
+    return $without_trailing_slash;
 }
 
 1;
diff --git a/src/test/path_to_volume_id_test.pm b/src/test/path_to_volume_id_test.pm
index bcd9db9c..12ed94d3 100644
--- a/src/test/path_to_volume_id_test.pm
+++ b/src/test/path_to_volume_id_test.pm
@@ -34,10 +34,53 @@ my $DEFAULT_CFG = {
             },
         },
     },
     'order' => {
         'local' => 1,
+        'overrides' => 2,
+        'nested' => 3,
+    },
+};
+
+my $OVERRIDES_STORAGE_DIR = File::Temp->newdir(); # unlinks on exit
+$DEFAULT_CFG->{ids}->{overrides} = {
+    'shared' => 0,
+    'path' => "$OVERRIDES_STORAGE_DIR",
+    'type' => 'dir',
+    'content' => {
+        'images' => 1,
+        'snippets' => 1,
+        'iso' => 1,
+        'vztmpl' => 1,
+    },
+    'content-dirs' => {
+        # nested inside each other
+        'snippets' => 'tpl',
+        'vztmpl' => 'tpl/cache',
+        # trailing slash
+        'iso' => 'isos/',
+        'images' => 'imgs/',
+    },
+};
+
+my $NESTED_STORAGE_DIR = File::Temp->newdir(); # unlinks on exit
+$DEFAULT_CFG->{ids}->{nested} = {
+    'shared' => 0,
+    'path' => "$NESTED_STORAGE_DIR",
+    'type' => 'dir',
+    'content' => {
+        'iso' => 1,
+        'vztmpl' => 1,
+        'backup' => 1,
+        'snippets' => 1,
+    },
+    'content-dirs' => {
+        # vtypes that accept subdirectories contain the directories of other vtypes
+        'iso' => 'data',
+        'vztmpl' => 'data/tpl',
+        'backup' => 'data/tpl/dump',
+        'snippets' => 'data/snippets',
     },
 };
 
 # Each test is comprised of the following keys:
 # description => to identify a single test
@@ -278,10 +321,66 @@ my $tests = [
         # currently not possible because it would introduce parsing ambiguities.
         description => 'Import, ova, subdirectory',
         file => "$DEFAULT_STORAGE_DIR/import/foo/import.ova",
         expected => [''],
     },
+    {
+        description => 'Content dir override with trailing slash, ISO',
+        file => "$OVERRIDES_STORAGE_DIR/isos/debian.iso",
+        expected => [
+            'iso', 'overrides:iso/debian.iso',
+        ],
+    },
+    {
+        description => 'Content dir override with trailing slash, image',
+        file => "$OVERRIDES_STORAGE_DIR/imgs/100/vm-100-disk-0.raw",
+        expected => [
+            'vm-vol', 'overrides:100/vm-100-disk-0.raw',
+        ],
+    },
+    {
+        description => 'Content dir override nested in another one, vztmpl',
+        file => "$OVERRIDES_STORAGE_DIR/tpl/cache/debian.tar.zst",
+        expected => [
+            'vztmpl', 'overrides:vztmpl/debian.tar.zst',
+        ],
+    },
+    {
+        description => 'Content dir nested in an ISO dir, snippets',
+        file => "$NESTED_STORAGE_DIR/data/snippets/seed.img",
+        expected => [
+            'snippets', 'nested:snippets/seed.img',
+        ],
+    },
+    {
+        description => 'Content dir nested in an ISO dir, vztmpl',
+        file => "$NESTED_STORAGE_DIR/data/tpl/debian.tar.zst",
+        expected => [
+            'vztmpl', 'nested:vztmpl/debian.tar.zst',
+        ],
+    },
+    {
+        description => 'Content dir nested in a vztmpl dir, backup',
+        file => "$NESTED_STORAGE_DIR/data/tpl/dump/vzdump-lxc-100-2020_03_30-21_39_30.tar.zst",
+        expected => [
+            'backup', 'nested:backup/vzdump-lxc-100-2020_03_30-21_39_30.tar.zst',
+        ],
+    },
+    {
+        description => 'Content dir containing other ones, ISO in subdirectory',
+        file => "$NESTED_STORAGE_DIR/data/sub/debian.iso",
+        expected => [
+            'iso', 'nested:iso/sub/debian.iso',
+        ],
+    },
+    {
+        description => 'Content dir override containing another one, snippets',
+        file => "$OVERRIDES_STORAGE_DIR/tpl/userconfig.yaml",
+        expected => [
+            'snippets', 'overrides:snippets/userconfig.yaml',
+        ],
+    },
 ];
 
 my sub run_tests($tests) {
     my $seen_vtype;
     my $vtype_subdirs = { map { $_ => 1 } keys plugin_get_default_vtype_subdirs()->%* };
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-storage v3 50/51] bump the storage plugin API version
  2026-11-18  9:14 ` [PATCH pve-storage v3 45/51] fix #2884: support nested subdir scanning for 'vztmpl' volume type Max R. Carrara
                     ` (3 preceding siblings ...)
  2026-11-18  9:14   ` [PATCH pve-storage v3 49/51] storage: resolve images in content dir overrides ending with a slash Max R. Carrara
@ 2026-11-18  9:14   ` Max R. Carrara
  2026-11-18  9:14   ` [PATCH pve-manager v3 51/51] fix #2884: ui: storage: add field for 'max-scan-depth' property Max R. Carrara
  5 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-18  9:14 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 ApiChangeLog       | 77 ++++++++++++++++++++++++++++++++++++++++++++++
 src/PVE/Storage.pm |  4 +--
 2 files changed, 79 insertions(+), 2 deletions(-)

diff --git a/ApiChangeLog b/ApiChangeLog
index 0887c308..fbab3c72 100644
--- a/ApiChangeLog
+++ b/ApiChangeLog
@@ -4,10 +4,87 @@ Our API versioning contains an `APIVER` and an `APIAGE`.
 The `APIAGE` is the number of versions we're backward compatible with. (iow.  things got added
 without breaking anything unaware of it.)
 
 Future changes should be documented in here.
 
+## Version 18:
+
+* Volume names of the `iso`, `vztmpl` and `snippets` volume types may contain subdirectories
+
+  Their files can be placed in subdirectories of the content directory, up to the
+  depth set by the new `max-scan-depth` storage property. Code that takes the
+  part after the volume type prefix of such volume names as a plain file name
+  needs to account for this.
+
+* Deprecate the following regular expressions:
+  * `$PVE::Storage::ISO_EXT_RE_0` (`iso` volume type)
+  * `$PVE::Storage::VZTMPL_EXT_RE_1` (`vztmpl` volume type)
+  * `$PVE::Storage::BACKUP_EXT_RE_2` (`backup` volume type)
+  * `$PVE::Storage::IMPORT_EXT_RE_1` (`import` volume type)
+  * `$PVE::Storage::UPLOAD_IMPORT_EXT_RE_1` (`import` volume type)
+  * `$PVE::Storage::OVA_CONTENT_RE_1` (`import` volume type)
+
+  These regular expressions were mostly used for parsing file paths and volume
+  names corresponding to their volume types, noted in parentheses above. They
+  are no longer used within pve-storage and will be removed on the next APIAGE
+  reset.
+
+  Instead, the parsing functions in `PVE::Storage::Common::Parse` should be
+  used.
+
+ * Deprecate the `PVE::Storage::Plugin::COMPRESSOR_RE` constant
+
+   It is only used by `$PVE::Storage::BACKUP_EXT_RE_2` and will be removed
+   together with it on the next APIAGE reset.
+
+* Migrate the following regular expressions:
+  * `$PVE::Storage::SAFE_CHAR_CLASS_RE`
+  * `$PVE::Storage::SAFE_CHAR_WITH_WHITESPACE_CLASS_RE`
+
+  These regular expressions are now defined as private constants in the
+  `PVE::Storage::Common::Parse` module and can be obtained by using the
+  corresponding subroutines in the *STANDALONE REGULAR EXPRESSIONS* section in
+  said module. The variables in `PVE::Storage` remain until the next APIAGE
+  reset.
+
+* Replace plugin helper `get_vtype_subdirs()` with standalone helper subroutine
+  `PVE::Storage::Common::plugin_get_default_vtype_subdirs()`
+
+  The `get_vtype_subdirs()` subroutine in `PVE::Storage::Plugin` was an
+  internal helper with a public signature, and was never meant to be
+  overridden.
+
+  Therefore, it is replaced with the `plugin_get_default_vtype_subdirs()`
+  subroutine from `PVE::Storage::Common` in order to make this distinction
+  clearer.
+
+  In order to make it clear that this is a helper and not meant to be part of
+  the storage plugin API, it is replaced with the more explicit
+  `PVE::Storage::Common::plugin_get_default_vtype_subdirs()` subroutine.
+
+* Replace plugin method `get_subdir()` with standalone helper subroutine
+  `PVE::Storage::Common::plugin_get_vtype_subdir()`
+
+  The `get_subdir()` method in `PVE::Storage::Plugin` was an internal helper
+  method with a public signature. Moreover, its signature could mistakenly
+  suggest that this method was part of the storage plugin API, which it was not
+  originally meant to be.
+
+  In order to make it clear that this is just a helper, it is replaced with the
+  more explicit `PVE::Storage::Common::plugin_get_vtype_subdir()` subroutine.
+
+  Both `get_vtype_subdirs()` and `get_subdir()` remain as wrappers until the
+  next APIAGE reset. Plugin authors are encouraged to switch to the new helper
+  subroutines as soon as possible.
+
+* `list_volumes()` of `PVE::Storage::Plugin` only returns volumes of content
+types that the storage configuration declares
+
+  `PVE::Storage::volume_list()` already filtered the requested content types
+  like this, so only plugins and code calling `list_volumes()` directly are
+  affected.
+
 ## Version 17:
 
 * Add the volume types `vm-vol`, `vm-state` and `ct-vol`
 
   A guest volume can now have a more specific type than `images`: `vm-vol` for a volume that a VM
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 3cd17993..b451f872 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -56,15 +56,15 @@ use PVE::Storage::ZFSPlugin;
 use PVE::Storage::PBSPlugin;
 use PVE::Storage::BTRFSPlugin;
 use PVE::Storage::ESXiPlugin;
 
 # Storage API version. Increment it on changes in storage API interface.
-use constant APIVER => 17;
+use constant APIVER => 18;
 # Age is the number of versions we're backward compatible with.
 # This is like having 'current=APIVER' and age='APIAGE' in libtool,
 # see https://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html
-use constant APIAGE => 8;
+use constant APIAGE => 9;
 
 our $KNOWN_EXPORT_FORMATS = ['raw+size', 'tar+size', 'qcow2+size', 'vmdk+size', 'zfs', 'btrfs'];
 
 # load standard plugins
 PVE::Storage::DirPlugin->register();
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

* [PATCH pve-manager v3 51/51] fix #2884: ui: storage: add field for 'max-scan-depth' property
  2026-11-18  9:14 ` [PATCH pve-storage v3 45/51] fix #2884: support nested subdir scanning for 'vztmpl' volume type Max R. Carrara
                     ` (4 preceding siblings ...)
  2026-11-18  9:14   ` [PATCH pve-storage v3 50/51] bump the storage plugin API version Max R. Carrara
@ 2026-11-18  9:14   ` Max R. Carrara
  5 siblings, 0 replies; 52+ messages in thread
From: Max R. Carrara @ 2026-11-18  9:14 UTC (permalink / raw)
  To: pve-devel; +Cc: Wolfgang Bumiller

In the base storage panel, add a simple integer field in the advanced
section for storage types that use the new `max-scan-depth` property.

This means that the maximum directory scan depth can now be configured
for directories, NFS, CIFS, CephFS, and BTRFS.

Originally-by: Noel Ullreich <n.ullreich@proxmox.com>
Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 www/manager6/storage/Base.js | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/www/manager6/storage/Base.js b/www/manager6/storage/Base.js
index 439cfeb8..b3a09674 100644
--- a/www/manager6/storage/Base.js
+++ b/www/manager6/storage/Base.js
@@ -120,10 +120,24 @@ Ext.define('PVE.panel.StorageBase', {
                 deleteEmpty: !me.isCreate,
                 value: '__default__',
             });
         }
 
+        const subdirScanningStorageTypes = ['dir', 'nfs', 'cifs', 'cephfs', 'btrfs'];
+
+        if (subdirScanningStorageTypes.includes(me.type)) {
+            addAdvancedWidget({
+                xtype: 'proxmoxintegerfield',
+                name: 'max-scan-depth',
+                fieldLabel: gettext('Maximum Directory Scan Depth'),
+                emptyText: Proxmox.Utils.defaultText + ' (0)',
+                deleteEmpty: !me.isCreate,
+                minValue: 0,
+                maxValue: 50,
+            });
+        }
+
         const externalStorageManagedSnapshotSupport = ['dir', 'nfs', 'cifs', 'lvm'];
 
         if (externalStorageManagedSnapshotSupport.includes(me.type)) {
             let volumeChain = PVE.panel.StorageBase.volumeChainFields(me.isCreate, me.type);
             addAdvancedWidget(volumeChain.checkbox);
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 52+ messages in thread

end of thread, other threads:[~2026-11-18  9:15 UTC | newest]

Thread overview: 52+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 01/51] test: plugin tests: run tests with at most 4 jobs Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 02/51] plugin, common: remove superfluous use of =pod command paragraph Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 03/51] common: use Exporter module for PVE::Storage::Common Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 04/51] plugin: make get_subdir_files a proper subroutine and update style Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 05/51] plugin api: replace subdir helper methods with standalone functions Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 06/51] common: prevent autovivification in plugin_get_vtype_subdir helper Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 07/51] plugin: break up if-elsif chain into separate if-blocks Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 08/51] plugin: adapt get_subdir_files helper of list_volumes API method Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 09/51] plugin: update code style of list_volumes plugin " Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 10/51] plugin: use closure for obtaining raw volume data in list_volumes Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 11/51] plugin: use closure for inner loop logic " Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 12/51] storage: update code style in function path_to_volume_id Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 13/51] storage: break up if-elsif chain in path_to_volume_id Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 14/51] storage: heave vtype file path parsing logic inside loop into helper Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 15/51] storage: clean up code that was moved into helper in path_to_volume_id Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 16/51] api: status: move content type assert for up-/downloads into helper Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 17/51] api: status: use get_subdir helper method to get content directory Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 18/51] api: status: move up-/download file path parsing code into helper Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 19/51] api: status: simplify file content assertion logic for up-/download Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 20/51] test: guest import: add tests for PVE::GuestImport Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 21/51] tree-wide: introduce parsing module and replace usages of ISO_EXT_RE_0 Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 22/51] common: test: set up parser testing code, add tests for 'iso' vtype Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 23/51] tree-wide: replace usages of VZTMPL_EXT_RE_1 with parsing functions Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 24/51] tree-wide: replace usages of BACKUP_EXT_RE_2 " Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 25/51] tree-wide: replace usages of inline regexes for snippets with parsers Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 26/51] tree-wide: partially replace usages of regexes for 'import' vtype Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 27/51] tree-wide: replace remaining " Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 28/51] tree-wide: simplify recently refactored parsing logic Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 29/51] test: list volumes: reorganize and modernize test running code Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 30/51] test: list volumes: fix broken test checking for vmlist modifications Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 31/51] test: list volumes: introduce new format for test cases Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 32/51] test: list volumes: remove legacy code and migrate cases to new format Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 33/51] plugin: do not return volumes of undeclared content types anymore Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 34/51] plugin: correct comment in get_subdir_files helper Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 35/51] test: parse volname: modernize code Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 36/51] test: parse volname: adapt tests regarding 'import' volume type Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 37/51] test: parse volname: move VM disk test creation into separate block Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 38/51] test: parse volname: move backup file test creation into sep. block Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 39/51] test: parse volname: parameterize test case creation for some vtypes Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 40/51] test: volume id: modernize code Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 41/51] test: volume id: rename 'volname' test case parameter to 'file' Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 42/51] test: filesystem path: modernize code Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 43/51] plugin: list content directories without globbing Max R. Carrara
2026-11-18  9:12 ` [PATCH pve-storage v3 44/51] fix #2884: implement nested subdir scanning and support 'iso' vtype Max R. Carrara
2026-11-18  9:14 ` [PATCH pve-storage v3 45/51] fix #2884: support nested subdir scanning for 'vztmpl' volume type Max R. Carrara
2026-11-18  9:14   ` [PATCH pve-storage v3 46/51] fix #2884: support nested subdir scanning for 'snippets' vtype Max R. Carrara
2026-11-18  9:14   ` [PATCH pve-storage v3 47/51] test: add more tests for 'import' vtype & guard against nested subdirs Max R. Carrara
2026-11-18  9:14   ` [PATCH pve-storage v3 48/51] test: add tests guarding against subdir scanning for vtypes Max R. Carrara
2026-11-18  9:14   ` [PATCH pve-storage v3 49/51] storage: resolve images in content dir overrides ending with a slash Max R. Carrara
2026-11-18  9:14   ` [PATCH pve-storage v3 50/51] bump the storage plugin API version Max R. Carrara
2026-11-18  9:14   ` [PATCH pve-manager v3 51/51] fix #2884: ui: storage: add field for 'max-scan-depth' property Max R. Carrara

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal