From: "Max R. Carrara" <m.carrara@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH pve-storage v2 24/50] tree-wide: replace usages of VZTMPL_EXT_RE_1 with parsing functions
Date: Wed, 23 Sep 2026 17:05:38 +0200 [thread overview]
Message-ID: <20260923150606.531239-25-m.carrara@proxmox.com> (raw)
In-Reply-To: <20260923150606.531239-1-m.carrara@proxmox.com>
Add support for the 'vztmpl' volume type in
`PVE::Storage::Common::Parse`.
Also add corresponding test cases.
Use the module's parsing functions to replace usages of the
`PVE::Storage::VZTMPL_EXT_RE_1` regex across the repository.
As done with the `ISO_EXT_RE_0` regex, note the removal of the
`VZTMPL_EXT_RE_1` regex in `ApiChangeLog`, since it's now completely
unused. Add a FIXME to note that it should be removed on the next
APIAGE reset.
Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
ApiChangeLog | 1 +
src/PVE/API2/Storage/Status.pm | 6 +-
src/PVE/Storage.pm | 5 +-
src/PVE/Storage/Common/Parse.pm | 54 +++++++++-
src/PVE/Storage/Common/test/parser_tests.pl | 109 ++++++++++++++++++++
src/PVE/Storage/Plugin.pm | 16 +--
6 files changed, 179 insertions(+), 12 deletions(-)
diff --git a/ApiChangeLog b/ApiChangeLog
index 7f79efb5..ffe3725c 100644
--- a/ApiChangeLog
+++ b/ApiChangeLog
@@ -30,6 +30,7 @@ Future changes should be documented in here.
* Remove the following regular expressions:
* `$PVE::Storage::ISO_EXT_RE_0` (`iso` volume type)
+ * `$PVE::Storage::VZTMPL_EXT_RE_1` (`vztmpl` volume type)
These regular expressions were mostly used for parsing file paths and volume
names corresponding to their volume types, noted in parentheses above.
diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm
index 310677d6..60f08628 100644
--- a/src/PVE/API2/Storage/Status.pm
+++ b/src/PVE/API2/Storage/Status.pm
@@ -87,11 +87,13 @@ my sub parse_transferred_file_path_extension : prototype($$) {
}
if ($vtype eq 'vztmpl') {
- if ($path !~ m![^/]+$PVE::Storage::VZTMPL_EXT_RE_1$!) {
+ my $parts = parse_rel_path_as_volname_parts($path, $vtype);
+
+ if (!defined($parts)) {
raise_param_exc({ filename => "wrong file extension" });
}
- my $ext = $1;
+ my $ext = $parts->{ext};
return $ext;
}
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 5358be3f..afe29178 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -123,6 +123,7 @@ PVE::Storage::Plugin->init();
# FIXME: remove this regex on the next APIAGE reset.
our $ISO_EXT_RE_0 = qr/\.(?:iso|img)/i;
+# FIXME: remove this regex on the next APIAGE reset.
our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2))/i;
our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/;
@@ -762,9 +763,7 @@ sub path_to_volume_id {
}
if ($vtype eq 'vztmpl') {
- return if $filename !~ m!/([^/]+$VZTMPL_EXT_RE_1)$!;
- my $name = $1;
- return "$sid:vztmpl/$name";
+ return parse_abs_path_as_volid($sid, $scfg, $path, $vtype);
}
if ($vtype eq 'backup') {
diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
index 1141a715..207f287b 100644
--- a/src/PVE/Storage/Common/Parse.pm
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -33,6 +33,14 @@ primarily related to.
=cut
+my @VZTMPL_COMPRESSION_EXTENSIONS = ('gz', 'xz', 'zst', 'bz2');
+
+my sub join_to_re_alternations(@list) {
+ return join('|', map { quotemeta } @list);
+}
+
+my $RE_VZTMPL_COMPRESSION_EXTENSIONS = join_to_re_alternations(@VZTMPL_COMPRESSION_EXTENSIONS);
+
my $RE_PARENT_DIR = quotemeta('..');
my $RE_CONTAINS_PARENT_DIR = qr!
( ^$RE_PARENT_DIR/ ) # ../ --> Beginning of path
@@ -48,12 +56,27 @@ my $RE_ISO_FILE_PATH = qr!
)
!xn;
+my $RE_VZTMPL_FILE_PATH = qr!
+ (?<path>
+ (?<file>
+ [^/]+
+ \.
+ (?<ext>
+ (?<ext_archive> (?i: tar) )
+ ( \. (?<ext_compression> (?i: $RE_VZTMPL_COMPRESSION_EXTENSIONS)) )?
+ )
+ )
+ )
+!xn;
+
my $RE_FILE_PATH_FOR_VTYPE = {
iso => qr/^$RE_ISO_FILE_PATH$/,
+ vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
};
my $RE_VOLNAME_FOR_VTYPE = {
iso => qr/^$RE_ISO_FILE_PATH$/,
+ vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
};
my sub contains_parent_dir($path) {
@@ -132,6 +155,31 @@ These are:
volname => 'iso/custom-debian.iso',
}
+The following volume types have more specific parts in addition to the common
+ones:
+
+=over
+
+=item C<vztmpl>
+
+C<ext-archive>, C<ext-compression> (optional):
+
+ {
+ file => 'archlinux-base_20190924-1_amd64.tar.gz',
+ ext => 'tar.gz',
+
+ # Used to further differentiate 'ext'
+ 'ext-archive' => 'tar',
+ 'ext-compression' => 'gz',
+
+ 'disk-path' => 'archlinux-base_20190924-1_amd64.tar.gz',
+ path => 'archlinux-base_20190924-1_amd64.tar.gz',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/archlinux-base_20190924-1_amd64.tar.gz',
+ }
+
+=back
+
=cut
=head3 parse_rel_path_as_volname_parts
@@ -151,7 +199,7 @@ into:
volname => 'iso/custom-debian.iso',
}
-See L<VOLUME NAMES> for more details.
+See L<VOLUME NAMES> for more details and special cases.
On failure or when the provided C<$vtype> is not supported or does not exist,
returns C<undef> in scalar context, and an empty list in list context.
@@ -281,6 +329,10 @@ These are:
volid => 'local:iso/custom-debian.iso',
}
+The special cases described in L<VOLUME NAMES> also apply for the parser
+functions in this section. In fact, the volid parsing functions here build on
+the volname ones.
+
=cut
my $RE_VOLID = qr!
diff --git a/src/PVE/Storage/Common/test/parser_tests.pl b/src/PVE/Storage/Common/test/parser_tests.pl
index c19456de..1ea8df1a 100755
--- a/src/PVE/Storage/Common/test/parser_tests.pl
+++ b/src/PVE/Storage/Common/test/parser_tests.pl
@@ -86,12 +86,121 @@ my $volname_cases_iso_invalid = [
},
];
+my $volname_cases_vztmpl_valid = [
+ # plain paths
+ {
+ path => 'archlinux-base_20190924-1_amd64.tar',
+ expected => {
+ file => 'archlinux-base_20190924-1_amd64.tar',
+ ext => 'tar',
+ 'ext-archive' => 'tar',
+ 'disk-path' => 'archlinux-base_20190924-1_amd64.tar',
+ path => 'archlinux-base_20190924-1_amd64.tar',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/archlinux-base_20190924-1_amd64.tar',
+ },
+ },
+ {
+ path => 'archlinux-base_20190924-1_amd64.tar.gz',
+ expected => {
+ file => 'archlinux-base_20190924-1_amd64.tar.gz',
+ ext => 'tar.gz',
+ 'ext-archive' => 'tar',
+ 'ext-compression' => 'gz',
+ 'disk-path' => 'archlinux-base_20190924-1_amd64.tar.gz',
+ path => 'archlinux-base_20190924-1_amd64.tar.gz',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/archlinux-base_20190924-1_amd64.tar.gz',
+ },
+ },
+ {
+ path => 'alpine-3.10-default_20190626_amd64.tar.xz',
+ expected => {
+ file => 'alpine-3.10-default_20190626_amd64.tar.xz',
+ ext => 'tar.xz',
+ 'ext-archive' => 'tar',
+ 'ext-compression' => 'xz',
+ 'disk-path' => 'alpine-3.10-default_20190626_amd64.tar.xz',
+ path => 'alpine-3.10-default_20190626_amd64.tar.xz',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/alpine-3.10-default_20190626_amd64.tar.xz',
+ },
+ },
+ {
+ path => 'debian-10.0-standard_10.0-1_amd64.tar.zst',
+ expected => {
+ file => 'debian-10.0-standard_10.0-1_amd64.tar.zst',
+ ext => 'tar.zst',
+ 'ext-archive' => 'tar',
+ 'ext-compression' => 'zst',
+ 'disk-path' => 'debian-10.0-standard_10.0-1_amd64.tar.zst',
+ path => 'debian-10.0-standard_10.0-1_amd64.tar.zst',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/debian-10.0-standard_10.0-1_amd64.tar.zst',
+ },
+ },
+ {
+ path => 'debian-11.0-standard_11.0-1_amd64.tar.bz2',
+ expected => {
+ file => 'debian-11.0-standard_11.0-1_amd64.tar.bz2',
+ ext => 'tar.bz2',
+ 'ext-archive' => 'tar',
+ 'ext-compression' => 'bz2',
+ 'disk-path' => 'debian-11.0-standard_11.0-1_amd64.tar.bz2',
+ path => 'debian-11.0-standard_11.0-1_amd64.tar.bz2',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/debian-11.0-standard_11.0-1_amd64.tar.bz2',
+ },
+ },
+
+ # case-insensitive file extensions
+ {
+ path => 'ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+ expected => {
+ file => 'ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+ ext => 'TAR.GZ',
+ 'ext-archive' => 'TAR',
+ 'ext-compression' => 'GZ',
+ 'disk-path' => 'ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+ path => 'ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+ },
+ },
+ {
+ path => 'Alpine 3.10 default_20190626_amd64.Tar.xZ',
+ expected => {
+ file => 'Alpine 3.10 default_20190626_amd64.Tar.xZ',
+ ext => 'Tar.xZ',
+ 'ext-archive' => 'Tar',
+ 'ext-compression' => 'xZ',
+ 'disk-path' => 'Alpine 3.10 default_20190626_amd64.Tar.xZ',
+ path => 'Alpine 3.10 default_20190626_amd64.Tar.xZ',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/Alpine 3.10 default_20190626_amd64.Tar.xZ',
+ },
+ },
+];
+
+my $volname_cases_vztmpl_invalid = [
+ {
+ description => "Invalid file extension (vztmpl)",
+ args => {
+ path => 'archlinux-base_20190924-1_amd64.zip',
+ vtype => 'vztmpl',
+ },
+ expected => undef,
+ },
+];
+
my $cases_valid_all = [
$volname_cases_iso_valid,
+ $volname_cases_vztmpl_valid,
];
my $cases_invalid_all = [
$volname_cases_iso_invalid,
+ $volname_cases_vztmpl_invalid,
];
{
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index d4860344..b539ef78 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -831,10 +831,10 @@ sub parse_volname {
if ($vtype eq 'iso') {
return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
}
- }
- if ($volname =~ m!^vztmpl/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!) {
- return ('vztmpl', $1, undef, undef, undef, undef, 'raw');
+ if ($vtype eq 'vztmpl') {
+ return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
+ }
}
if ($volname =~ m!^backup/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!) {
@@ -1734,11 +1734,15 @@ my sub get_subdir_files {
}
if ($vtype eq 'vztmpl') {
- return if $filename !~ m!/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!;
+ my $parts = parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
+ return if !defined($parts);
+
+ my ($ext, $ext_compression) = $parts->@{qw(ext ext-compression)};
+ my $format = $ext eq 'tar' ? $ext : ('t' . $ext_compression);
return {
- volid => "$storeid:vztmpl/$1",
- format => $2 eq 'tar' ? $2 : "t$2",
+ volid => $parts->{volid},
+ format => $format,
};
}
--
2.47.3
next prev parent reply other threads:[~2026-09-23 15:10 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-23 15:05 [PATCH manager/storage v2 00/50] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 01/50] test: plugin tests: run tests with at most 4 jobs Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 02/50] plugin, common: remove superfluous use of =pod command paragraph Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 03/50] common: add POD headings for groups of helpers Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 04/50] common: use Exporter module for PVE::Storage::Common Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 05/50] plugin: make get_subdir_files a proper subroutine and update style Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 06/50] plugin api: replace helpers w/ standalone subs, bump API version & age Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 07/50] common: prevent autovivification in plugin_get_vtype_subdir helper Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 08/50] plugin: break up if-elsif chain into separate if-blocks Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 09/50] plugin: adapt get_subdir_files helper of list_volumes API method Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 10/50] plugin: update code style of list_volumes plugin " Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 11/50] plugin: use closure for obtaining raw volume data in list_volumes Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 12/50] plugin: use closure for inner loop logic " Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 13/50] storage: update code style in function path_to_volume_id Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 14/50] storage: break up if-elsif chain in path_to_volume_id Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 15/50] storage: heave vtype file path parsing logic inside loop into helper Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 16/50] storage: clean up code that was moved into helper in path_to_volume_id Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 17/50] api: status: move content type assert for up-/downloads into helper Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 18/50] api: status: use helper from common module to get content directory Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 19/50] api: status: move up-/download file path parsing code into helper Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 20/50] api: status: simplify file content assertion logic for up-/download Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 21/50] test: guest import: add tests for PVE::GuestImport Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 22/50] tree-wide: introduce parsing module and replace usages of ISO_EXT_RE_0 Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 23/50] common: test: set up parser testing code, add tests for 'iso' vtype Max R. Carrara
2026-09-23 15:05 ` Max R. Carrara [this message]
2026-09-23 15:05 ` [PATCH pve-storage v2 25/50] tree-wide: replace usages of BACKUP_EXT_RE_2 with parsing functions Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 26/50] tree-wide: replace usages of inline regexes for snippets with parsers Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 27/50] tree-wide: partially replace usages of regexes for 'import' vtype Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 28/50] tree-wide: replace remaining " Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 29/50] tree-wide: simplify recently refactored parsing logic Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 30/50] test: list volumes: reorganize and modernize test running code Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 31/50] test: list volumes: fix broken test checking for vmlist modifications Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 32/50] test: list volumes: introduce new format for test cases Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 33/50] test: list volumes: remove legacy code and migrate cases to new format Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 34/50] plugin: do not return volumes of undeclared content types anymore Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 35/50] plugin: correct comment in get_subdir_files helper Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 36/50] test: parse volname: modernize code Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 37/50] test: parse volname: adapt tests regarding 'import' volume type Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 38/50] test: parse volname: move VM disk test creation into separate block Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 39/50] test: parse volname: move backup file test creation into sep. block Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 40/50] test: parse volname: parameterize test case creation for some vtypes Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 41/50] test: volume id: modernize code Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 42/50] test: volume id: rename 'volname' test case parameter to 'file' Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 43/50] test: filesystem path: modernize code Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-storage v2 44/50] fix #2884: implement nested subdir scanning and support 'iso' vtype Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-storage v2 45/50] fix #2884: support nested subdir scanning for 'vztmpl' volume type Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-storage v2 46/50] fix #2884: support nested subdir scanning for 'snippets' vtype Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-storage v2 47/50] test: add more tests for 'import' vtype & guard against nested subdirs Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-storage v2 48/50] test: add tests guarding against subdir scanning for vtypes Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-manager v2 49/50] fix #2884: ui: storage: add field for 'max-scan-depth' property Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-manager v2 50/50] pve8to9: use helper from common storage module to get vtype subdir Max R. Carrara
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260923150606.531239-25-m.carrara@proxmox.com \
--to=m.carrara@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.