all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: "Max R. Carrara" <m.carrara@proxmox.com>
To: pve-devel@lists.proxmox.com
Cc: Wolfgang Bumiller <w.bumiller@proxmox.com>
Subject: [PATCH pve-storage v3 44/51] fix #2884: implement nested subdir scanning and support 'iso' vtype
Date: Wed, 18 Nov 2026 10:12:30 +0100	[thread overview]
Message-ID: <20261118091232.25103-1-m.carrara@proxmox.com> (raw)
In-Reply-To: <20261117203953.913035-1-m.carrara@proxmox.com>

Introduce a new 'max-scan-depth' property for directory-based
storages. This property makes it so that files for specific volume
types may also be put into (nested) subdirectories inside the volume
type's directory.

The maximum allowed depth is between 0 and 50, with 0 being the
default, which corresponds to the current behavior. The property is
available for directory, NFS, CIFS, CephFS and BTRFS storages.

Support nested subdirectories for ISOs (the 'iso' vtype) only for now.

Achieve all of this by calling the `get_subdir_files()` helper in
`PVE::Storage::Plugin` recursively. Note that Perl only warns about
deep recursion from a depth of 100 on [0], way above the maximum value
of 50 for the 'max-scan-depth' property. Should we need deeper
nesting, we can always make the subroutine iterative instead of
recursive later.

Symbolic links to directories are never followed. They may lead
outside of the volume type's directory, or back into a directory that
is already being scanned. Such loops would list the same files over
and over until the maximum depth is reached, with the number of
results growing exponentially with the number of looping links.

Do not descend into the content directory of another content type of
the storage either. With 'content-dirs' overrides, such a directory
may be nested inside the scanned one, for example with
'iso=data,snippets=data/snippets'. Its files belong to the other
content type and would otherwise be listed twice. Directories are
compared by device and inode number so that extreme edge cases (like
bind mounts e.g.) are handled. Directories of content types the
storage does not declare are scanned like any other.

Give the property a title and mark it as advanced in the plugin data
of the plugins that support it, like the other properties shown in
the advanced section of the storage edit window.

Add additional test cases wherever applicable to account for nested
subdirectories, including cases that check whether the limit set by
the property is upheld, and also cases that check for the existence of
parent directory references ('..') in volume names.

[0]: https://perldoc.perl.org/perl5101delta#Deep-recursion-on-subroutine-%22%25s%22

Originally-by: Noel Ullreich <n.ullreich@proxmox.com>
Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
 src/PVE/Storage/BTRFSPlugin.pm              |   1 +
 src/PVE/Storage/CIFSPlugin.pm               |   1 +
 src/PVE/Storage/CephFSPlugin.pm             |   1 +
 src/PVE/Storage/Common/Parse.pm             |  18 ++
 src/PVE/Storage/Common/test/parser_tests.pl |  50 +++++
 src/PVE/Storage/DirPlugin.pm                |   1 +
 src/PVE/Storage/NFSPlugin.pm                |   1 +
 src/PVE/Storage/Plugin.pm                   | 104 ++++++++-
 src/test/filesystem_path_test.pm            |   8 +
 src/test/list_volumes_test.pm               | 225 +++++++++++++++++++-
 src/test/parse_volname_test.pm              |  56 +++++
 src/test/path_to_volume_id_test.pm          |   7 +
 12 files changed, 468 insertions(+), 5 deletions(-)

diff --git a/src/PVE/Storage/BTRFSPlugin.pm b/src/PVE/Storage/BTRFSPlugin.pm
index 1c81b7b8..518df37b 100644
--- a/src/PVE/Storage/BTRFSPlugin.pm
+++ b/src/PVE/Storage/BTRFSPlugin.pm
@@ -82,10 +82,11 @@ sub properties {
 }
 
 sub options {
     return {
         path => { fixed => 1 },
+        'max-scan-depth' => { optional => 1 },
         nodes => { optional => 1 },
         shared => { optional => 1 },
         disable => { optional => 1 },
         'prune-backups' => { optional => 1 },
         'max-protected-backups' => { optional => 1 },
diff --git a/src/PVE/Storage/CIFSPlugin.pm b/src/PVE/Storage/CIFSPlugin.pm
index 15d5a22d..45a0c5ac 100644
--- a/src/PVE/Storage/CIFSPlugin.pm
+++ b/src/PVE/Storage/CIFSPlugin.pm
@@ -183,10 +183,11 @@ sub properties {
 
 sub options {
     return {
         path => { fixed => 1 },
         'content-dirs' => { optional => 1 },
+        'max-scan-depth' => { optional => 1 },
         server => { fixed => 1 },
         share => { fixed => 1 },
         subdir => { optional => 1 },
         nodes => { optional => 1 },
         disable => { optional => 1 },
diff --git a/src/PVE/Storage/CephFSPlugin.pm b/src/PVE/Storage/CephFSPlugin.pm
index 82a4351f..4b751945 100644
--- a/src/PVE/Storage/CephFSPlugin.pm
+++ b/src/PVE/Storage/CephFSPlugin.pm
@@ -153,10 +153,11 @@ sub properties {
 
 sub options {
     return {
         path => { fixed => 1 },
         'content-dirs' => { optional => 1 },
+        'max-scan-depth' => { optional => 1 },
         monhost => { optional => 1 },
         nodes => { optional => 1 },
         subdir => { optional => 1 },
         disable => { optional => 1 },
         options => { optional => 1 },
diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
index 2f6380a8..2b6ae70e 100644
--- a/src/PVE/Storage/Common/Parse.pm
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -58,10 +58,14 @@ my $RE_OVA_CONTENT_EXTENSIONS = join_to_re_alternations(@OVA_CONTENT_EXTENSIONS)
 
 my $RE_SAFE_CHAR_CLASS = qr/[a-zA-Z0-9\-\.\+\=\_]/;
 
 my $RE_SAFE_CHAR_WITH_WHITESPACE_CLASS = qr/[ a-zA-Z0-9\-\.\+\=\_]/;
 
+my $RE_DIRECTORY_COMPONENTS = qr!
+    ( ($RE_SAFE_CHAR_WITH_WHITESPACE_CLASS)+ / )*
+!xn;
+
 my $RE_PARENT_DIR = quotemeta('..');
 my $RE_CONTAINS_PARENT_DIR = qr!
     ( ^$RE_PARENT_DIR$ )  #  ..  --> Entire path
     |
     ( ^$RE_PARENT_DIR/ )  #  ../ --> Beginning of path
@@ -76,10 +80,11 @@ my $RE_CONTAINS_PARENT_DIR = qr!
 # See also: https://git.proxmox.com/?p=pve-common.git;a=blob;f=src/PVE/JSONSchema.pm;h=17e7126a6c42f8326a1da890680af10b6106d906;hb=refs/heads/master#l304
 my $RE_VMID = qr![1-9][0-9]{2,8}!;
 
 my $RE_ISO_FILE_PATH = qr!
     (?<path>
+        (?<dir> $RE_DIRECTORY_COMPONENTS )?
         (?<file> [^/]+ \. (?<ext> (?i: iso|img) ) )
     )
 !xn;
 
 my $RE_VZTMPL_FILE_PATH = qr!
@@ -226,10 +231,19 @@ my sub format_named_groups(%groups) {
         $result->{$new_key} = $groups{$old_key};
     }
 
     my @disk_path_components = ();
 
+    if (defined($result->{dir})) {
+        if ($result->{dir} eq '') {
+            delete $result->{dir};
+        } else {
+            $result->{dir} = strip_trailing_path_separators($result->{dir});
+            push(@disk_path_components, $result->{dir});
+        }
+    }
+
     if (defined($result->{file})) {
         $result->{file} = strip_leading_path_separators($result->{file});
         push(@disk_path_components, $result->{file});
     }
 
@@ -329,10 +343,14 @@ These are:
 
         # The volname, which is "${vtype}/${path}"
         volname => 'iso/custom-debian.iso',
     }
 
+Volume types that support subdirectories in their C<path> part also have a
+C<dir> part if a subdirectory is specified. This is currently limited to the
+C<iso> volume type.
+
 The following volume types have more specific parts in addition to the common
 ones:
 
 =over
 
diff --git a/src/PVE/Storage/Common/test/parser_tests.pl b/src/PVE/Storage/Common/test/parser_tests.pl
index af0a7052..cb8d706f 100755
--- a/src/PVE/Storage/Common/test/parser_tests.pl
+++ b/src/PVE/Storage/Common/test/parser_tests.pl
@@ -71,21 +71,71 @@ my $volname_cases_iso_valid = [
             path => 'Fedora.Img',
             vtype => 'iso',
             volname => 'iso/Fedora.Img',
         },
     },
+
+    # subdirectories
+    {
+        path => 'subdir/custom-debian.iso',
+        expected => {
+            file => 'custom-debian.iso',
+            ext => 'iso',
+            dir => 'subdir',
+            'disk-path' => 'subdir/custom-debian.iso',
+            path => 'subdir/custom-debian.iso',
+            vtype => 'iso',
+            volname => 'iso/subdir/custom-debian.iso',
+        },
+    },
+    {
+        path => 'deeply/nested/dir/hannah-montana-linux.IMG',
+        expected => {
+            file => 'hannah-montana-linux.IMG',
+            ext => 'IMG',
+            dir => 'deeply/nested/dir',
+            'disk-path' => 'deeply/nested/dir/hannah-montana-linux.IMG',
+            path => 'deeply/nested/dir/hannah-montana-linux.IMG',
+            vtype => 'iso',
+            volname => 'iso/deeply/nested/dir/hannah-montana-linux.IMG',
+        },
+    },
 ];
 
 my $volname_cases_iso_invalid = [
     {
         description => "Invalid file extension (iso)",
         args => {
             path => 'MacOS-kittycat.dmg',
             vtype => 'iso',
         },
         expected => undef,
     },
+    {
+        description => "Parent dir reference in path (beginning) (iso)",
+        args => {
+            path => '../custom-debian.iso',
+            vtype => 'iso',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (middle) (iso)",
+        args => {
+            path => 'subdir/../custom-debian.iso',
+            vtype => 'iso',
+        },
+        expected => undef,
+    },
+    {
+        description => "Parent dir reference in path (end) (iso)",
+        args => {
+            path => 'subdir/custom-debian.iso/..',
+            vtype => 'iso',
+        },
+        expected => undef,
+    },
 ];
 
 my $volname_cases_vztmpl_valid = [
     # plain paths
     {
diff --git a/src/PVE/Storage/DirPlugin.pm b/src/PVE/Storage/DirPlugin.pm
index 7df7a849..1cf2fbe9 100644
--- a/src/PVE/Storage/DirPlugin.pm
+++ b/src/PVE/Storage/DirPlugin.pm
@@ -100,10 +100,11 @@ sub properties {
 
 sub options {
     return {
         path => { fixed => 1 },
         'content-dirs' => { optional => 1 },
+        'max-scan-depth' => { optional => 1 },
         nodes => { optional => 1 },
         shared => { optional => 1 },
         disable => { optional => 1 },
         'prune-backups' => { optional => 1 },
         'max-protected-backups' => { optional => 1 },
diff --git a/src/PVE/Storage/NFSPlugin.pm b/src/PVE/Storage/NFSPlugin.pm
index 18b470c4..82cd2806 100644
--- a/src/PVE/Storage/NFSPlugin.pm
+++ b/src/PVE/Storage/NFSPlugin.pm
@@ -106,10 +106,11 @@ sub properties {
 
 sub options {
     return {
         path => { fixed => 1 },
         'content-dirs' => { optional => 1 },
+        'max-scan-depth' => { optional => 1 },
         server => { fixed => 1 },
         export => { fixed => 1 },
         nodes => { optional => 1 },
         disable => { optional => 1 },
         'prune-backups' => { optional => 1 },
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index b8f415a8..37fdc921 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -1,10 +1,12 @@
 package PVE::Storage::Plugin;
 
 use strict;
 use warnings;
 
+use Carp qw(confess);
+
 use Cwd qw(abs_path);
 use Encode qw(decode);
 use Fcntl ':mode';
 use File::chdir;
 use File::Path;
@@ -269,10 +271,20 @@ my $defaultData = {
             description => "Overrides for default content type directories.",
             type => "string",
             format => "pve-dir-override-list",
             optional => 1,
         },
+        'max-scan-depth' => {
+            title => "Maximum Directory Scan Depth",
+            description => "Maximum depth of subdirectories to traverse when searching for"
+                . " ISOs in directories.",
+            type => 'integer',
+            default => 0,
+            minimum => 0,
+            maximum => 50,
+            optional => 1,
+        },
         options => {
             title => "Mount Options",
             description => "Comma-separated list of mount options."
                 . " See 'man nfs', 'man mount.cifs', or 'man mount.ceph' for more information.",
             type => 'string',
@@ -1882,15 +1894,22 @@ my sub list_dir_entries {
     return map { "$dir/$_" } sort { lc($a) cmp lc($b) || $a cmp $b } @names;
 }
 
 # $vtype = <iso|vztmpl|backup|snippets|import>
 my sub get_subdir_files {
-    my ($class, $storeid, $scfg, $vtype, $vmid) = @_;
+    use feature 'current_sub'; # Needed for the __SUB__ token further below
+
+    my ($class, $storeid, $scfg, $vtype, $vmid, $remaining_depth, $current_path, $skip_dirs) = @_;
 
     # FIXME: Call plugin_get_vtype_subdir() instead of $class->get_subdir()
     # and remove the $class parameter on the next APIAGE reset.
     my $vtype_subdir = $class->get_subdir($scfg, $vtype);
+
+    if (!defined($current_path)) {
+        $current_path = $vtype_subdir;
+    }
+
     my $res = [];
 
     my $get_subdir_file_info = sub {
         my ($path, $st) = @_;
 
@@ -1970,17 +1989,90 @@ my sub get_subdir_files {
         }
 
         return;
     };
 
-    for my $path (list_dir_entries($vtype_subdir)) {
+    my $get_dir_id_from_stat = sub {
+        my ($stat_res) = @_;
+        return $stat_res->dev . ':' . $stat_res->ino;
+    };
+
+    # The content directories of the other content types of the storage
+    # may be nested inside this one, for example with 'content-dirs'
+    # overrides. Their files belong to the other content type, so we
+    # gotta make sure to not descend into them.
+    #
+    # Note that we only look the content dirs up once there is a
+    # directory to descend into.
+    if (!defined($skip_dirs)) {
+        $skip_dirs = {};
+
+        # NOTE: In the loop below we do not need to handle `images`,
+        # `rootdir`, `vm-state`, etc. types, because those are fetched
+        # using `list_images()`.
+
+        my $declared_vtypes = $scfg->{content} // {};
+        my $known_vtype_subdirs = plugin_get_default_vtype_subdirs();
+
+        for my $declared_vtype (keys $declared_vtypes->%*) {
+            # The dir of the vtype being listed is the one we are scanning.
+            next if $declared_vtype eq $vtype;
+
+            # Ignore vtypes without a directory of their own, e.g. 'none'
+            next if !exists($known_vtype_subdirs->{$declared_vtype});
+
+            # FIXME: Call plugin_get_vtype_subdir() instead of
+            # $class->get_subdir() on the next APIAGE reset.
+            my $declared_vtype_subdir = $class->get_subdir($scfg, $declared_vtype);
+
+            # Skip directories that do not exist (yet).
+            my $vtype_subdir_stat = File::stat::stat($declared_vtype_subdir);
+            next if !defined($vtype_subdir_stat);
+
+            # Identify the directory by device and inode, so that we
+            # don't iterate through the same dir more than once in
+            # extreme edge cases, like bind mounts and symlinks in the
+            # configured paths themselves.
+            my $dir_id = $get_dir_id_from_stat->($vtype_subdir_stat);
+            $skip_dirs->{$dir_id} = 1;
+        }
+    }
+
+    for my $path (list_dir_entries($current_path)) {
         my $st = File::stat::stat($path);
 
         # A volume ID cannot represent a name with a newline, see parse_volume_id().
         next if $path =~ m/\n/;
 
-        next if (!$st || S_ISDIR($st->mode));
+        next if !$st;
+
+        if (S_ISDIR($st->mode)) {
+            # Never follow symlinks to directories. They may lead outside of
+            # the vtype subdir or back into a directory that is already being
+            # scanned, listing the same files over and over until the maximum
+            # depth is reached.
+            next if -l $path;
+
+            next if !defined($remaining_depth) || $remaining_depth <= 0;
+
+            next if $skip_dirs->{ $get_dir_id_from_stat->($st) };
+
+            my $inner_res = __SUB__->(
+                $class,
+                $storeid,
+                $scfg,
+                $vtype,
+                $vmid,
+                $remaining_depth - 1,
+                $path,
+                $skip_dirs,
+            );
+
+            push($res->@*, $inner_res->@*);
+
+            next;
+        }
 
         if (defined(my $info = $get_subdir_file_info->($path, $st))) {
             $info->{size} = $st->size;
             $info->{ctime} //= $st->ctime;
 
@@ -1994,10 +2086,14 @@ my sub get_subdir_files {
 # If attributes are set on a volume, they should be included in the result.
 # See get_volume_attribute for a list of possible attributes.
 sub list_volumes {
     my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
 
+    my $depth = $scfg->{'max-scan-depth'} // 0;
+    $depth = 0 if $depth < 0;
+    $depth = 50 if $depth > 50;
+
     my $res = [];
     my $vmlist = PVE::Cluster::get_vmlist();
 
     my $get_raw_volumes_for_type = sub {
         my ($type) = @_;
@@ -2011,11 +2107,11 @@ sub list_volumes {
         }
 
         return if !$scfg->{path};
 
         if ($type eq 'iso' && !defined($vmid)) {
-            return get_subdir_files($class, $storeid, $scfg, 'iso', undef);
+            return get_subdir_files($class, $storeid, $scfg, 'iso', undef, $depth);
         }
 
         if ($type eq 'vztmpl' && !defined($vmid)) {
             return get_subdir_files($class, $storeid, $scfg, 'vztmpl', undef);
         }
diff --git a/src/test/filesystem_path_test.pm b/src/test/filesystem_path_test.pm
index f5574c65..8b3b9fdf 100644
--- a/src/test/filesystem_path_test.pm
+++ b/src/test/filesystem_path_test.pm
@@ -73,10 +73,18 @@ my $tests = [
         snapname => undef,
         expected => [
             "$DEFAULT_STORAGE_DIR/template/iso/my-awesome-proxmox.iso", undef, 'iso',
         ],
     },
+    {
+        volname => 'iso/foo/bar/baz/my-awesome-proxmox.iso',
+        snapname => undef,
+        expected => [
+            "$DEFAULT_STORAGE_DIR/template/iso/foo/bar/baz/my-awesome-proxmox.iso", undef,
+            'iso',
+        ],
+    },
     {
         volname => "backup/vzdump-qemu-1234-2020_03_30-21_12_40.vma",
         snapname => undef,
         expected => [
             "$DEFAULT_STORAGE_DIR/dump/vzdump-qemu-1234-2020_03_30-21_12_40.vma", 1234,
diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm
index 4932dbbf..59d8a56e 100644
--- a/src/test/list_volumes_test.pm
+++ b/src/test/list_volumes_test.pm
@@ -11,11 +11,11 @@ use PVE::Tools qw(run_command);
 use Test::More;
 use Test::MockModule;
 
 use Carp qw(confess);
 use Cwd;
-use File::Basename qw(fileparse);
+use File::Basename qw(dirname fileparse);
 use File::Path qw(make_path remove_tree);
 use File::stat qw();
 use File::Temp;
 use Storable qw(dclone);
 
@@ -136,10 +136,21 @@ keys:
                     vmid => 1234,
                     volid => 'local:1234/vm-1234-disk-0.qcow2',
                 },
             },
         ],
+
+        # Optional list of symbolic links to create after the files above.
+        symlinks => [
+            {
+                # Absolute path of the symbolic link.
+                link => "/tmp/foobar/template/iso/latest",
+
+                # Target of the symbolic link, used verbatim.
+                target => 'debian-13',
+            },
+        ],
     }
 
 =cut
 
 my $test_param_list = [
@@ -1351,10 +1362,212 @@ my $test_param_list = [
                 file => "$DEFAULT_STORAGE_PATH/import/import.ova",
                 expected => undef,
             },
         ],
     },
+    {
+        description => "VMID: none, no nested subdirectories when using defaults",
+        storeid => $DEFAULT_STOREID,
+        scfg => $DEFAULT_SCFG,
+        vmid => undef,
+        vtypes => ['iso', 'vztmpl', 'snippets', 'import'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/some-installer.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/some-installer.iso",
+                expected => undef,
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/2/3/4/5/some-installer.iso",
+                expected => undef,
+            },
+        ],
+    },
+    {
+        description => "VMID: none, nested subdirectories allowed, max-scan-depth = 1",
+        storeid => $DEFAULT_STOREID,
+        scfg => {
+            type => 'dir',
+            path => $DEFAULT_STORAGE_PATH,
+            shared => 0,
+            'max-scan-depth' => 1,
+            content => {
+                iso => 1,
+                vztmpl => 1,
+                snippets => 1,
+                import => 1,
+            },
+        },
+        vmid => undef,
+        vtypes => ['iso', 'vztmpl', 'snippets', 'import'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/some-installer.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/1/some-installer.iso',
+                },
+            },
+            {
+                # Exceeds max-scan-depth
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/2/some-installer.iso",
+                expected => undef,
+            },
+        ],
+    },
+    {
+        description => "VMID: none, nested subdirectories allowed, max-scan-depth = 5",
+        storeid => $DEFAULT_STOREID,
+        scfg => {
+            type => 'dir',
+            path => $DEFAULT_STORAGE_PATH,
+            shared => 0,
+            'max-scan-depth' => 5,
+            content => {
+                iso => 1,
+                vztmpl => 1,
+                snippets => 1,
+                import => 1,
+            },
+        },
+        vmid => undef,
+        vtypes => ['iso', 'vztmpl', 'snippets', 'import'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/some-installer.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/1/some-installer.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/2/3/4/5/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/1/2/3/4/5/some-installer.iso',
+                },
+            },
+            {
+                # Exceeds max-scan-depth
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/2/3/4/5/6/some-installer.iso",
+                expected => undef,
+            },
+        ],
+    },
+    {
+        description => "VMID: none, max-scan-depth = 2, nested content dir of snippets",
+        storeid => $DEFAULT_STOREID,
+        scfg => {
+            $DEFAULT_SCFG->%*,
+            'max-scan-depth' => 2,
+            'content-dirs' => { iso => 'data', snippets => 'data/snippets' },
+        },
+        vmid => undef,
+        vtypes => ['iso'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/data/a.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/a.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/data/sub/b.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/sub/b.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/data/snippets/seed.iso",
+                expected => undef,
+            },
+        ],
+    },
+    {
+        description => "VMID: none, max-scan-depth = 8, symlinks to directories",
+        storeid => $DEFAULT_STOREID,
+        scfg => { $DEFAULT_SCFG->%*, 'max-scan-depth' => 8 },
+        vmid => undef,
+        vtypes => ['iso'],
+        cases => [
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/some-installer.iso',
+                },
+            },
+            {
+                file => "$DEFAULT_STORAGE_PATH/template/iso/1/some-installer.iso",
+                expected => {
+                    content => 'iso',
+                    ctime => $DEFAULT_CTIME,
+                    format => 'iso',
+                    size => $DEFAULT_SIZE,
+                    volid => 'local:iso/1/some-installer.iso',
+                },
+            },
+        ],
+        # None of these are followed
+        symlinks => [
+            { link => "$DEFAULT_STORAGE_PATH/template/iso/alias", target => '1' },
+            { link => "$DEFAULT_STORAGE_PATH/template/iso/outside", target => '/' },
+            { link => "$DEFAULT_STORAGE_PATH/template/iso/loop-a", target => '.' },
+            { link => "$DEFAULT_STORAGE_PATH/template/iso/loop-b", target => '.' },
+            { link => "$DEFAULT_STORAGE_PATH/template/iso/1/loop-c", target => '..' },
+            { link => "$DEFAULT_STORAGE_PATH/template/iso/1/loop-d", target => '../1' },
+        ],
+    },
     {
         description => 'Storage path containing whitespace and glob metacharacters',
         storeid => $DEFAULT_STOREID,
         scfg => { $DEFAULT_SCFG->%*, path => $SPECIAL_CHARS_STORAGE_PATH },
         vmid => undef,
@@ -1580,10 +1793,13 @@ $mock_cluster->redefine(get_vmlist => sub { return $mocked_vmlist; });
 # an blessed array.
 my $mock_stat = Test::MockModule->new('File::stat', no_auto => 1);
 $mock_stat->redefine(
     populate => sub {
         my (@st) = @_;
+
+        return $mock_stat->original('populate')->() if !@st; # stat failed
+
         $st[7] = $DEFAULT_SIZE;
         $st[10] = $DEFAULT_CTIME;
 
         my $result = $mock_stat->original('populate')->(@st);
 
@@ -1633,10 +1849,17 @@ my sub setup_test_env($test_params) {
 
             run_command($command, timeout => 10);
         }
     }
 
+    for my $symlink (($test_params->{symlinks} // [])->@*) {
+        my ($link, $target) = $symlink->@{qw(link target)};
+
+        make_path(dirname($link), { verbose => 1, mode => 0755 });
+        symlink($target, $link) or die "failed to create symlink '$link' - $!\n";
+    }
+
     return;
 }
 
 my sub teardown_test_env($test_params) {
     my $scfg = $test_params->{scfg};
diff --git a/src/test/parse_volname_test.pm b/src/test/parse_volname_test.pm
index 5520d3c6..a9900152 100644
--- a/src/test/parse_volname_test.pm
+++ b/src/test/parse_volname_test.pm
@@ -202,14 +202,70 @@ my $tests = [
             {
                 description => "ISO image, $suffix",
                 volname => "iso/$file_name",
                 expected => ['iso', "$file_name", undef, undef, undef, undef, 'raw'],
             },
+            {
+                description => "ISO image, $suffix, subdirectory",
+                volname => "iso/foo/$file_name",
+                expected => [
+                    'iso', "foo/$file_name", undef, undef, undef, undef, 'raw',
+                ],
+            },
+            {
+                description => "ISO image, $suffix, nested subdirectories",
+                volname => "iso/foo/bar/baz/$file_name",
+                expected => [
+                    'iso', "foo/bar/baz/$file_name", undef, undef, undef, undef, 'raw',
+                ],
+            },
+            {
+                description => "ISO image, $suffix, subdirectory with same name as file",
+                volname => "iso/$file_name/$file_name",
+                expected => [
+                    'iso', "$file_name/$file_name", undef, undef, undef, undef, 'raw',
+                ],
+            },
         );
 
         push($tests->@*, @extra_tests);
     }
+
+    # Expected to fail
+    {
+        my $file_name = "$prefix.iso";
+
+        my @extra_failed_tests = (
+            {
+                description =>
+                    "ISO image, iso, parent directory reference before volume type prefix",
+                volname => "../iso/$file_name",
+                expected => "unable to parse directory volume name '../iso/$file_name'\n",
+            },
+            {
+                description =>
+                    "ISO image, iso, parent directory reference at beginning of volume path",
+                volname => "iso/../$file_name",
+                expected => "unable to parse directory volume name 'iso/../$file_name'\n",
+            },
+            {
+                description =>
+                    "ISO image, iso, parent directory reference at end of volume path",
+                volname => "iso/$file_name/..",
+                expected => "unable to parse directory volume name 'iso/$file_name/..'\n",
+            },
+            {
+                description =>
+                    "ISO image, iso, parent directory reference between dir components of volume path",
+                volname => "iso/foo/../bar/$file_name",
+                expected =>
+                    "unable to parse directory volume name 'iso/foo/../bar/$file_name'\n",
+            },
+        );
+
+        push($tests->@*, @extra_failed_tests);
+    }
 }
 
 # Test cases for container templates
 {
     my $suffixes = ['tar', 'tar.gz', 'tar.xz', 'tar.bz2'];
diff --git a/src/test/path_to_volume_id_test.pm b/src/test/path_to_volume_id_test.pm
index 6bc7cc07..c6c1b2e1 100644
--- a/src/test/path_to_volume_id_test.pm
+++ b/src/test/path_to_volume_id_test.pm
@@ -121,10 +121,17 @@ my $tests = [
         file => "$DEFAULT_STORAGE_DIR/template/iso/yet-again-a-installation-disk.iso",
         expected => [
             'iso', 'local:iso/yet-again-a-installation-disk.iso',
         ],
     },
+    {
+        description => 'ISO file, nested subdirectories',
+        file => "$DEFAULT_STORAGE_DIR/template/iso/foo/bar/hannah-montana-linux-installer.iso",
+        expected => [
+            'iso', 'local:iso/foo/bar/hannah-montana-linux-installer.iso',
+        ],
+    },
     {
         description => 'CT template, tar.gz',
         file => "$DEFAULT_STORAGE_DIR/template/cache/debian-10.0-standard_10.0-1_amd64.tar.gz",
         expected => [
             'vztmpl', 'local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz',
-- 
2.47.3





  parent reply	other threads:[~2026-11-18  9:12 UTC|newest]

Thread overview: 52+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17 20:38 [PATCH manager/storage v3 00/51] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 01/51] test: plugin tests: run tests with at most 4 jobs Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 02/51] plugin, common: remove superfluous use of =pod command paragraph Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 03/51] common: use Exporter module for PVE::Storage::Common Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 04/51] plugin: make get_subdir_files a proper subroutine and update style Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 05/51] plugin api: replace subdir helper methods with standalone functions Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 06/51] common: prevent autovivification in plugin_get_vtype_subdir helper Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 07/51] plugin: break up if-elsif chain into separate if-blocks Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 08/51] plugin: adapt get_subdir_files helper of list_volumes API method Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 09/51] plugin: update code style of list_volumes plugin " Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 10/51] plugin: use closure for obtaining raw volume data in list_volumes Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 11/51] plugin: use closure for inner loop logic " Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 12/51] storage: update code style in function path_to_volume_id Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 13/51] storage: break up if-elsif chain in path_to_volume_id Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 14/51] storage: heave vtype file path parsing logic inside loop into helper Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 15/51] storage: clean up code that was moved into helper in path_to_volume_id Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 16/51] api: status: move content type assert for up-/downloads into helper Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 17/51] api: status: use get_subdir helper method to get content directory Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 18/51] api: status: move up-/download file path parsing code into helper Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 19/51] api: status: simplify file content assertion logic for up-/download Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 20/51] test: guest import: add tests for PVE::GuestImport Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 21/51] tree-wide: introduce parsing module and replace usages of ISO_EXT_RE_0 Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 22/51] common: test: set up parser testing code, add tests for 'iso' vtype Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 23/51] tree-wide: replace usages of VZTMPL_EXT_RE_1 with parsing functions Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 24/51] tree-wide: replace usages of BACKUP_EXT_RE_2 " Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 25/51] tree-wide: replace usages of inline regexes for snippets with parsers Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 26/51] tree-wide: partially replace usages of regexes for 'import' vtype Max R. Carrara
2026-11-17 20:38 ` [PATCH pve-storage v3 27/51] tree-wide: replace remaining " Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 28/51] tree-wide: simplify recently refactored parsing logic Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 29/51] test: list volumes: reorganize and modernize test running code Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 30/51] test: list volumes: fix broken test checking for vmlist modifications Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 31/51] test: list volumes: introduce new format for test cases Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 32/51] test: list volumes: remove legacy code and migrate cases to new format Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 33/51] plugin: do not return volumes of undeclared content types anymore Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 34/51] plugin: correct comment in get_subdir_files helper Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 35/51] test: parse volname: modernize code Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 36/51] test: parse volname: adapt tests regarding 'import' volume type Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 37/51] test: parse volname: move VM disk test creation into separate block Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 38/51] test: parse volname: move backup file test creation into sep. block Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 39/51] test: parse volname: parameterize test case creation for some vtypes Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 40/51] test: volume id: modernize code Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 41/51] test: volume id: rename 'volname' test case parameter to 'file' Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 42/51] test: filesystem path: modernize code Max R. Carrara
2026-11-17 20:39 ` [PATCH pve-storage v3 43/51] plugin: list content directories without globbing Max R. Carrara
2026-11-18  9:12 ` Max R. Carrara [this message]
2026-11-18  9:14 ` [PATCH pve-storage v3 45/51] fix #2884: support nested subdir scanning for 'vztmpl' volume type Max R. Carrara
2026-11-18  9:14   ` [PATCH pve-storage v3 46/51] fix #2884: support nested subdir scanning for 'snippets' vtype Max R. Carrara
2026-11-18  9:14   ` [PATCH pve-storage v3 47/51] test: add more tests for 'import' vtype & guard against nested subdirs Max R. Carrara
2026-11-18  9:14   ` [PATCH pve-storage v3 48/51] test: add tests guarding against subdir scanning for vtypes Max R. Carrara
2026-11-18  9:14   ` [PATCH pve-storage v3 49/51] storage: resolve images in content dir overrides ending with a slash Max R. Carrara
2026-11-18  9:14   ` [PATCH pve-storage v3 50/51] bump the storage plugin API version Max R. Carrara
2026-11-18  9:14   ` [PATCH pve-manager v3 51/51] fix #2884: ui: storage: add field for 'max-scan-depth' property Max R. Carrara

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118091232.25103-1-m.carrara@proxmox.com \
    --to=m.carrara@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    --cc=w.bumiller@proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal