From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id A1FD68DA2 for ; Wed, 23 Aug 2023 11:04:30 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 8AD13144A4 for ; Wed, 23 Aug 2023 11:04:30 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 23 Aug 2023 11:04:29 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 93387437B3; Wed, 23 Aug 2023 11:04:29 +0200 (CEST) Message-ID: Date: Wed, 23 Aug 2023 11:04:28 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: Proxmox VE development discussion , Philipp Hufnagl References: <20230814144217.2082571-1-p.hufnagl@proxmox.com> <20230814144217.2082571-2-p.hufnagl@proxmox.com> Content-Language: en-US From: Dominik Csapak In-Reply-To: <20230814144217.2082571-2-p.hufnagl@proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.011 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [nodes.pm] Subject: Re: [pve-devel] [PATCH manager v6 1/2] fix #4849: api: download to storage: automatically dectect and configure compression X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Aug 2023 09:04:30 -0000 looks mostly good, one high level comment, and a few nits inline: do we really need the 'detect-compression' parameter? when wouldn't we want that? also the gui always enables that for isos anyway? if there is a good reason, that would be nice to have in a commit message if we can always use the detection, we don't have to give the parameter in the gui and save a few lines and an additional api parameter we have to bring with us for a long time ;) On 8/14/23 16:42, Philipp Hufnagl wrote: > extends the query_url_metadata callback with the functionallity to > detect used compressions. If a compression is used it tells the ui which > one > > Signed-off-by: Philipp Hufnagl > --- > PVE/API2/Nodes.pm | 20 ++++++++++++++++++++ > 1 file changed, 20 insertions(+) > > diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm > index 5a148d1d..66a8bb0b 100644 > --- a/PVE/API2/Nodes.pm > +++ b/PVE/API2/Nodes.pm > @@ -1564,6 +1564,12 @@ __PACKAGE__->register_method({ > type => 'boolean', > optional => 1, > default => 1, > + }, > + 'detect-compression' => { > + description => "If true an auto detect of used compression will be attempted", > + type => 'boolean', > + optional => 1, > + default => 0, > } > }, > }, > @@ -1583,6 +1589,11 @@ __PACKAGE__->register_method({ > type => 'string', > optional => 1, > }, > + compression => { > + type => 'string', > + enum => $PVE::Storage::Plugin::KNOWN_COMPRESSION_FORMATS, if you use this, you have to import it with a 'use' statement at the top of the package in this case it works because the storage part includes it already, but should that change it would not work anymore > + optional => 1, > + }, > }, > }, > code => sub { > @@ -1606,6 +1617,8 @@ __PACKAGE__->register_method({ > ); > } > > + my $detect_compression = $param->{'detect-compression'}; > + if we decide to leave the parameter in, i'd prefer to set the default here: my $detect_compression = $param->{'detect-compression'} // 0; (it works fine without that, but now i don't have to guess what the default should be without looking at the parameter description) > my $req = HTTP::Request->new(HEAD => $url); > my $res = $ua->request($req); > > @@ -1615,6 +1628,7 @@ __PACKAGE__->register_method({ > my $disposition = $res->header("Content-Disposition"); > my $type = $res->header("Content-Type"); > > + my $compression; > my $filename; > > if ($disposition && ($disposition =~ m/filename="([^"]*)"/ || $disposition =~ m/filename=([^;]*)/)) { > @@ -1628,10 +1642,16 @@ __PACKAGE__->register_method({ > $type = $1; > } > > + if ($detect_compression && $filename =~ m!^(.+)\.(${\PVE::Storage::Plugin::COMPRESSOR_RE})$!) { > + $filename = $1; > + $compression = $2; > + } > + > my $ret = {}; > $ret->{filename} = $filename if $filename; > $ret->{size} = $size + 0 if $size; > $ret->{mimetype} = $type if $type; > + $ret->{compression} = $compression if $compression; > > return $ret; > }});