From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 320821FF141 for ; Tue, 30 Jun 2026 16:19:27 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 8554521414; Tue, 30 Jun 2026 16:19:26 +0200 (CEST) Date: Tue, 30 Jun 2026 16:19:21 +0200 From: Wolfgang Bumiller To: "Max R. Carrara" Subject: Re: [PATCH pve-storage v1 27/54] tree-wide: partially replace usages of regexes for 'import' vtype Message-ID: <77nuuuiwmuhe42lpwtfsdkbanjpdvnivyq6byak5y3i36xs6ca@53crwcf4r4y3> References: <20260422111322.257380-1-m.carrara@proxmox.com> <20260422111322.257380-28-m.carrara@proxmox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20260422111322.257380-28-m.carrara@proxmox.com> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1782829149036 X-SPAM-LEVEL: Spam detection results: 0 DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: EXWFZVWMHCQNSD56SZ5J5DGGK2RBSA6Y X-Message-ID-Hash: EXWFZVWMHCQNSD56SZ5J5DGGK2RBSA6Y X-MailFrom: w.bumiller@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: pve-devel@lists.proxmox.com X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Wed, Apr 22, 2026 at 01:12:53PM +0200, Max R. Carrara wrote: > Add partial support for the 'import' volume type in > `PVE::Storage::Common::Parse`. What remains unsupported right now is > parsing volume names for that type, as those require a little more > care. > > Replace most usages of the `PVE::Storage::IMPORT_EXT_RE_1` regex with > parsing functions from `::Common::Parse`. > > Replace the one remaining spot where we use the > `PVE::Storage::UPLOAD_IMPORT_EXT_RE_1` regex with a parsing function > as well. This regex should now be phased out in a future APIVER + > APIAGE bump, like the others in previous commits. > > Note that the `UPLOAD_IMPORT_EXT_RE_1` regex only exists to handle the > special case of excluding .ovf files from the upload / download_url > API methods. Instead of adding a separate parser (or a flag etc.) to > handle this case, just parse the path of the up-/downloaded file > regardless. Then, raise a parameter exception when the `ovf` file > extension is encountered. > > Also add a bunch of test cases that target the `import` volume type > for the `list_volumes()` plugin API method. For all of these tests, > sort the expected and resulting list items by their volume ID in order > to make the output comparison deterministic. Iterate over the declared > content / volume types of the mocked storage config instead of using a > fixed list of vtypes as well. > > Signed-off-by: Max R. Carrara > --- > src/PVE/API2/Storage/Status.pm | 13 ++- > src/PVE/Storage.pm | 4 +- > src/PVE/Storage/Common/Parse.pm | 13 +++ > src/PVE/Storage/Plugin.pm | 9 +- > src/test/list_volumes_test.pm | 180 +++++++++++++++++++++++++++++++- > 5 files changed, 203 insertions(+), 16 deletions(-) > > diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm > index 03929d26..b578a1ed 100644 > --- a/src/PVE/API2/Storage/Status.pm > +++ b/src/PVE/API2/Storage/Status.pm > @@ -96,13 +96,18 @@ my sub parse_transferred_file_path_extension : prototype($$) { > } > > if ($vtype eq 'import') { > - if ( > - $path !~ m!${PVE::Storage::SAFE_CHAR_CLASS_RE}+$PVE::Storage::UPLOAD_IMPORT_EXT_RE_1$! > - ) { > + my $parts = parse_path_as_volname_parts($path, $vtype); > + > + if (!defined($parts)) { > raise_param_exc({ filename => "invalid filename or wrong extension" }); > } > > - my $ext = $1; > + my $ext = $parts->{ext}; > + > + if ($ext eq 'ovf') { > + raise_param_exc({ filename => "wrong file extension" }); > + } > + > return $ext; > } > > diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm > index 20b7bafa..3a716894 100755 > --- a/src/PVE/Storage.pm > +++ b/src/PVE/Storage.pm > @@ -773,9 +773,7 @@ sub path_to_volume_id { > } > > if ($vtype eq 'import') { > - return if $filename !~ m!/(${SAFE_CHAR_CLASS_RE}+${IMPORT_EXT_RE_1})$!; > - my $name = $1; > - return "$sid:import/$name"; > + return parse_path_as_volid($sid, $scfg, $path, $vtype); > } > > return; > diff --git a/src/PVE/Storage/Common/Parse.pm b/src/PVE/Storage/Common/Parse.pm > index 33220758..bd7cec13 100644 > --- a/src/PVE/Storage/Common/Parse.pm > +++ b/src/PVE/Storage/Common/Parse.pm > @@ -37,6 +37,8 @@ my @VZTMPL_COMPRESSION_EXTENSIONS = ('gz', 'xz', 'zst', 'bz2'); > > my @BACKUP_COMPRESSION_EXTENSIONS = ('gz', 'lzo', 'zst', 'bz2'); > > +my @IMPORT_EXTENSIONS = ('ova', 'ovf', 'qcow2', 'raw', 'vmdk'); > + > my sub join_to_re_alternations(@list) { > return join('|', map { quotemeta } @list); > } > @@ -45,6 +47,10 @@ my $RE_VZTMPL_COMPRESSION_EXTENSIONS = join_to_re_alternations(@VZTMPL_COMPRESSI > > my $RE_BACKUP_COMPRESSION_EXTENSIONS = join_to_re_alternations(@BACKUP_COMPRESSION_EXTENSIONS); > > +my $RE_IMPORT_EXTENSIONS = join_to_re_alternations(@IMPORT_EXTENSIONS); > + > +my $RE_SAFE_CHAR_CLASS = qr/[a-zA-Z0-9\-\.\+\=\_]/; > + > my $RE_PARENT_DIR = quotemeta('..'); > my $RE_CONTAINS_PARENT_DIR = qr! > ( ^$RE_PARENT_DIR/ ) # ../ --> Beginning of path > @@ -112,11 +118,18 @@ my $RE_SNIPPETS_FILE_PATH = qr! > ) > !xn; > > +my $RE_IMPORT_FILE_PATH = qr! > + (? > + (? ($RE_SAFE_CHAR_CLASS)+ \. (? $RE_IMPORT_EXTENSIONS) ) This misses an `(?i)`. After more checking, so does `ext_compression` from the previous `BACKUP_EXT_RE_2` patch. Perhaps there should be a `join_to_insensitive_alterations()` helper which generally wraps the alterations inside `(?i:…)` used for all the file extension regexes? > + ) > +!xn; > + > my $RE_FILE_PATH_FOR_VTYPE = { > iso => qr/^$RE_ISO_FILE_PATH$/, > vztmpl => qr/^$RE_VZTMPL_FILE_PATH$/, > backup => qr/^$RE_BACKUP_FILE_PATH$/, > snippets => qr/^$RE_SNIPPETS_FILE_PATH$/, > + import => qr/^$RE_IMPORT_FILE_PATH$/, > }; > > my $RE_VOLNAME_FOR_VTYPE = { > diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm > index b08e038b..39e5cb0d 100644 > --- a/src/PVE/Storage/Plugin.pm > +++ b/src/PVE/Storage/Plugin.pm > @@ -1768,13 +1768,12 @@ my sub get_subdir_files { > } > > if ($vtype eq 'import') { > - return > - if $filename !~ > - m!/(${PVE::Storage::SAFE_CHAR_CLASS_RE}+$PVE::Storage::IMPORT_EXT_RE_1)$!i; (to-the-above-relevant /i (`m!…!i`) for context) > + my $parts = parse_path_as_volid_parts($storeid, $scfg, $path, $vtype); > + return if !defined($parts);