From: "Max R. Carrara" <m.carrara@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH pve-storage v2 24/50] tree-wide: replace usages of VZTMPL_EXT_RE_1 with parsing functions
Date: Wed, 23 Sep 2026 17:05:38 +0200 [thread overview]
Message-ID: <20260923150606.531239-25-m.carrara@proxmox.com> (raw)
In-Reply-To: <20260923150606.531239-1-m.carrara@proxmox.com>
Add support for the 'vztmpl' volume type in
`PVE::Storage::Common::Parse`.
Also add corresponding test cases.
Use the module's parsing functions to replace usages of the
`PVE::Storage::VZTMPL_EXT_RE_1` regex across the repository.
As done with the `ISO_EXT_RE_0` regex, note the removal of the
`VZTMPL_EXT_RE_1` regex in `ApiChangeLog`, since it's now completely
unused. Add a FIXME to note that it should be removed on the next
APIAGE reset.
Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
ApiChangeLog | 1 +
src/PVE/API2/Storage/Status.pm | 6 +-
src/PVE/Storage.pm | 5 +-
src/PVE/Storage/Common/Parse.pm | 54 +++++++++-
src/PVE/Storage/Common/test/parser_tests.pl | 109 ++++++++++++++++++++
src/PVE/Storage/Plugin.pm | 16 +--
6 files changed, 179 insertions(+), 12 deletions(-)
diff --git a/ApiChangeLog b/ApiChangeLog
index 7f79efb5..ffe3725c 100644
--- a/ApiChangeLog
+++ b/ApiChangeLog
@@ -30,6 +30,7 @@ Future changes should be documented in here.
* Remove the following regular expressions:
* `$PVE::Storage::ISO_EXT_RE_0` (`iso` volume type)
+ * `$PVE::Storage::VZTMPL_EXT_RE_1` (`vztmpl` volume type)
These regular expressions were mostly used for parsing file paths and volume
names corresponding to their volume types, noted in parentheses above.
diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm
index 310677d6..60f08628 100644
--- a/src/PVE/API2/Storage/Status.pm
+++ b/src/PVE/API2/Storage/Status.pm
@@ -87,11 +87,13 @@ my sub parse_transferred_file_path_extension : prototype($$) {
}
if ($vtype eq 'vztmpl') {
- if ($path !~ m![^/]+$PVE::Storage::VZTMPL_EXT_RE_1$!) {
+ my $parts = parse_rel_path_as_volname_parts($path, $vtype);
+
+ if (!defined($parts)) {
raise_param_exc({ filename => "wrong file extension" });
}
- my $ext = $1;
+ my $ext = $parts->{ext};
return $ext;
}
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 5358be3f..afe29178 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -123,6 +123,7 @@ PVE::Storage::Plugin->init();
# FIXME: remove this regex on the next APIAGE reset.
our $ISO_EXT_RE_0 = qr/\.(?:iso|img)/i;
+# FIXME: remove this regex on the next APIAGE reset.
our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2))/i;
our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/;
@@ -762,9 +763,7 @@ sub path_to_volume_id {
}
if ($vtype eq 'vztmpl') {
- return if $filename !~ m!/([^/]+$VZTMPL_EXT_RE_1)$!;
- my $name = $1;
- return "$sid:vztmpl/$name";
+ return parse_abs_path_as_volid($sid, $scfg, $path, $vtype);
}
if ($vtype eq 'backup') {
diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm
index 1141a715..207f287b 100644
--- a/src/PVE/Storage/Common/Parse.pm
+++ b/src/PVE/Storage/Common/Parse.pm
@@ -33,6 +33,14 @@ primarily related to.
=cut
+my @VZTMPL_COMPRESSION_EXTENSIONS = ('gz', 'xz', 'zst', 'bz2');
+
+my sub join_to_re_alternations(@list) {
+ return join('|', map { quotemeta } @list);
+}
+
+my $RE_VZTMPL_COMPRESSION_EXTENSIONS = join_to_re_alternations(@VZTMPL_COMPRESSION_EXTENSIONS);
+
my $RE_PARENT_DIR = quotemeta('..');
my $RE_CONTAINS_PARENT_DIR = qr!
( ^$RE_PARENT_DIR/ ) # ../ --> Beginning of path
@@ -48,12 +56,27 @@ my $RE_ISO_FILE_PATH = qr!
)
!xn;
+my $RE_VZTMPL_FILE_PATH = qr!
+ (?<path>
+ (?<file>
+ [^/]+
+ \.
+ (?<ext>
+ (?<ext_archive> (?i: tar) )
+ ( \. (?<ext_compression> (?i: $RE_VZTMPL_COMPRESSION_EXTENSIONS)) )?
+ )
+ )
+ )
+!xn;
+
my $RE_FILE_PATH_FOR_VTYPE = {
iso => qr/^$RE_ISO_FILE_PATH$/,
+ vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
};
my $RE_VOLNAME_FOR_VTYPE = {
iso => qr/^$RE_ISO_FILE_PATH$/,
+ vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/,
};
my sub contains_parent_dir($path) {
@@ -132,6 +155,31 @@ These are:
volname => 'iso/custom-debian.iso',
}
+The following volume types have more specific parts in addition to the common
+ones:
+
+=over
+
+=item C<vztmpl>
+
+C<ext-archive>, C<ext-compression> (optional):
+
+ {
+ file => 'archlinux-base_20190924-1_amd64.tar.gz',
+ ext => 'tar.gz',
+
+ # Used to further differentiate 'ext'
+ 'ext-archive' => 'tar',
+ 'ext-compression' => 'gz',
+
+ 'disk-path' => 'archlinux-base_20190924-1_amd64.tar.gz',
+ path => 'archlinux-base_20190924-1_amd64.tar.gz',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/archlinux-base_20190924-1_amd64.tar.gz',
+ }
+
+=back
+
=cut
=head3 parse_rel_path_as_volname_parts
@@ -151,7 +199,7 @@ into:
volname => 'iso/custom-debian.iso',
}
-See L<VOLUME NAMES> for more details.
+See L<VOLUME NAMES> for more details and special cases.
On failure or when the provided C<$vtype> is not supported or does not exist,
returns C<undef> in scalar context, and an empty list in list context.
@@ -281,6 +329,10 @@ These are:
volid => 'local:iso/custom-debian.iso',
}
+The special cases described in L<VOLUME NAMES> also apply for the parser
+functions in this section. In fact, the volid parsing functions here build on
+the volname ones.
+
=cut
my $RE_VOLID = qr!
diff --git a/src/PVE/Storage/Common/test/parser_tests.pl b/src/PVE/Storage/Common/test/parser_tests.pl
index c19456de..1ea8df1a 100755
--- a/src/PVE/Storage/Common/test/parser_tests.pl
+++ b/src/PVE/Storage/Common/test/parser_tests.pl
@@ -86,12 +86,121 @@ my $volname_cases_iso_invalid = [
},
];
+my $volname_cases_vztmpl_valid = [
+ # plain paths
+ {
+ path => 'archlinux-base_20190924-1_amd64.tar',
+ expected => {
+ file => 'archlinux-base_20190924-1_amd64.tar',
+ ext => 'tar',
+ 'ext-archive' => 'tar',
+ 'disk-path' => 'archlinux-base_20190924-1_amd64.tar',
+ path => 'archlinux-base_20190924-1_amd64.tar',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/archlinux-base_20190924-1_amd64.tar',
+ },
+ },
+ {
+ path => 'archlinux-base_20190924-1_amd64.tar.gz',
+ expected => {
+ file => 'archlinux-base_20190924-1_amd64.tar.gz',
+ ext => 'tar.gz',
+ 'ext-archive' => 'tar',
+ 'ext-compression' => 'gz',
+ 'disk-path' => 'archlinux-base_20190924-1_amd64.tar.gz',
+ path => 'archlinux-base_20190924-1_amd64.tar.gz',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/archlinux-base_20190924-1_amd64.tar.gz',
+ },
+ },
+ {
+ path => 'alpine-3.10-default_20190626_amd64.tar.xz',
+ expected => {
+ file => 'alpine-3.10-default_20190626_amd64.tar.xz',
+ ext => 'tar.xz',
+ 'ext-archive' => 'tar',
+ 'ext-compression' => 'xz',
+ 'disk-path' => 'alpine-3.10-default_20190626_amd64.tar.xz',
+ path => 'alpine-3.10-default_20190626_amd64.tar.xz',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/alpine-3.10-default_20190626_amd64.tar.xz',
+ },
+ },
+ {
+ path => 'debian-10.0-standard_10.0-1_amd64.tar.zst',
+ expected => {
+ file => 'debian-10.0-standard_10.0-1_amd64.tar.zst',
+ ext => 'tar.zst',
+ 'ext-archive' => 'tar',
+ 'ext-compression' => 'zst',
+ 'disk-path' => 'debian-10.0-standard_10.0-1_amd64.tar.zst',
+ path => 'debian-10.0-standard_10.0-1_amd64.tar.zst',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/debian-10.0-standard_10.0-1_amd64.tar.zst',
+ },
+ },
+ {
+ path => 'debian-11.0-standard_11.0-1_amd64.tar.bz2',
+ expected => {
+ file => 'debian-11.0-standard_11.0-1_amd64.tar.bz2',
+ ext => 'tar.bz2',
+ 'ext-archive' => 'tar',
+ 'ext-compression' => 'bz2',
+ 'disk-path' => 'debian-11.0-standard_11.0-1_amd64.tar.bz2',
+ path => 'debian-11.0-standard_11.0-1_amd64.tar.bz2',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/debian-11.0-standard_11.0-1_amd64.tar.bz2',
+ },
+ },
+
+ # case-insensitive file extensions
+ {
+ path => 'ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+ expected => {
+ file => 'ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+ ext => 'TAR.GZ',
+ 'ext-archive' => 'TAR',
+ 'ext-compression' => 'GZ',
+ 'disk-path' => 'ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+ path => 'ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/ARCHLINUX-BASE 20190924-1 AMD64.TAR.GZ',
+ },
+ },
+ {
+ path => 'Alpine 3.10 default_20190626_amd64.Tar.xZ',
+ expected => {
+ file => 'Alpine 3.10 default_20190626_amd64.Tar.xZ',
+ ext => 'Tar.xZ',
+ 'ext-archive' => 'Tar',
+ 'ext-compression' => 'xZ',
+ 'disk-path' => 'Alpine 3.10 default_20190626_amd64.Tar.xZ',
+ path => 'Alpine 3.10 default_20190626_amd64.Tar.xZ',
+ vtype => 'vztmpl',
+ volname => 'vztmpl/Alpine 3.10 default_20190626_amd64.Tar.xZ',
+ },
+ },
+];
+
+my $volname_cases_vztmpl_invalid = [
+ {
+ description => "Invalid file extension (vztmpl)",
+ args => {
+ path => 'archlinux-base_20190924-1_amd64.zip',
+ vtype => 'vztmpl',
+ },
+ expected => undef,
+ },
+];
+
my $cases_valid_all = [
$volname_cases_iso_valid,
+ $volname_cases_vztmpl_valid,
];
my $cases_invalid_all = [
$volname_cases_iso_invalid,
+ $volname_cases_vztmpl_invalid,
];
{
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index d4860344..b539ef78 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -831,10 +831,10 @@ sub parse_volname {
if ($vtype eq 'iso') {
return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
}
- }
- if ($volname =~ m!^vztmpl/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!) {
- return ('vztmpl', $1, undef, undef, undef, undef, 'raw');
+ if ($vtype eq 'vztmpl') {
+ return ($vtype, $volume_path, undef, undef, undef, undef, 'raw');
+ }
}
if ($volname =~ m!^backup/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!) {
@@ -1734,11 +1734,15 @@ my sub get_subdir_files {
}
if ($vtype eq 'vztmpl') {
- return if $filename !~ m!/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!;
+ my $parts = parse_abs_path_as_volid_parts($storeid, $scfg, $path, $vtype);
+ return if !defined($parts);
+
+ my ($ext, $ext_compression) = $parts->@{qw(ext ext-compression)};
+ my $format = $ext eq 'tar' ? $ext : ('t' . $ext_compression);
return {
- volid => "$storeid:vztmpl/$1",
- format => $2 eq 'tar' ? $2 : "t$2",
+ volid => $parts->{volid},
+ format => $format,
};
}
--
2.47.3
next prev parent reply other threads:[~2026-09-23 15:10 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-23 15:05 [PATCH manager/storage v2 00/50] Fix #2884: Implement Subdirectory Scanning for Dir-Based Storage Types Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 01/50] test: plugin tests: run tests with at most 4 jobs Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 02/50] plugin, common: remove superfluous use of =pod command paragraph Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 03/50] common: add POD headings for groups of helpers Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 04/50] common: use Exporter module for PVE::Storage::Common Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 05/50] plugin: make get_subdir_files a proper subroutine and update style Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 06/50] plugin api: replace helpers w/ standalone subs, bump API version & age Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 07/50] common: prevent autovivification in plugin_get_vtype_subdir helper Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 08/50] plugin: break up if-elsif chain into separate if-blocks Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 09/50] plugin: adapt get_subdir_files helper of list_volumes API method Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 10/50] plugin: update code style of list_volumes plugin " Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 11/50] plugin: use closure for obtaining raw volume data in list_volumes Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 12/50] plugin: use closure for inner loop logic " Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 13/50] storage: update code style in function path_to_volume_id Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 14/50] storage: break up if-elsif chain in path_to_volume_id Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 15/50] storage: heave vtype file path parsing logic inside loop into helper Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 16/50] storage: clean up code that was moved into helper in path_to_volume_id Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 17/50] api: status: move content type assert for up-/downloads into helper Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 18/50] api: status: use helper from common module to get content directory Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 19/50] api: status: move up-/download file path parsing code into helper Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 20/50] api: status: simplify file content assertion logic for up-/download Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 21/50] test: guest import: add tests for PVE::GuestImport Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 22/50] tree-wide: introduce parsing module and replace usages of ISO_EXT_RE_0 Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 23/50] common: test: set up parser testing code, add tests for 'iso' vtype Max R. Carrara
2026-09-23 15:05 ` Max R. Carrara [this message]
2026-09-23 15:05 ` [PATCH pve-storage v2 25/50] tree-wide: replace usages of BACKUP_EXT_RE_2 with parsing functions Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 26/50] tree-wide: replace usages of inline regexes for snippets with parsers Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 27/50] tree-wide: partially replace usages of regexes for 'import' vtype Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 28/50] tree-wide: replace remaining " Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 29/50] tree-wide: simplify recently refactored parsing logic Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 30/50] test: list volumes: reorganize and modernize test running code Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 31/50] test: list volumes: fix broken test checking for vmlist modifications Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 32/50] test: list volumes: introduce new format for test cases Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 33/50] test: list volumes: remove legacy code and migrate cases to new format Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 34/50] plugin: do not return volumes of undeclared content types anymore Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 35/50] plugin: correct comment in get_subdir_files helper Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 36/50] test: parse volname: modernize code Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 37/50] test: parse volname: adapt tests regarding 'import' volume type Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 38/50] test: parse volname: move VM disk test creation into separate block Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 39/50] test: parse volname: move backup file test creation into sep. block Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 40/50] test: parse volname: parameterize test case creation for some vtypes Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 41/50] test: volume id: modernize code Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 42/50] test: volume id: rename 'volname' test case parameter to 'file' Max R. Carrara
2026-09-23 15:05 ` [PATCH pve-storage v2 43/50] test: filesystem path: modernize code Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-storage v2 44/50] fix #2884: implement nested subdir scanning and support 'iso' vtype Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-storage v2 45/50] fix #2884: support nested subdir scanning for 'vztmpl' volume type Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-storage v2 46/50] fix #2884: support nested subdir scanning for 'snippets' vtype Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-storage v2 47/50] test: add more tests for 'import' vtype & guard against nested subdirs Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-storage v2 48/50] test: add tests guarding against subdir scanning for vtypes Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-manager v2 49/50] fix #2884: ui: storage: add field for 'max-scan-depth' property Max R. Carrara
2026-09-23 15:16 ` [PATCH pve-manager v2 50/50] pve8to9: use helper from common storage module to get vtype subdir Max R. Carrara
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260923150606.531239-25-m.carrara@proxmox.com \
--to=m.carrara@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox