From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <t.lamprecht@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id DB90BE7CF
for <pve-devel@lists.proxmox.com>; Tue, 26 Sep 2023 12:56:02 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id BBB173795B
for <pve-devel@lists.proxmox.com>; Tue, 26 Sep 2023 12:56:02 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[94.136.29.106])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pve-devel@lists.proxmox.com>; Tue, 26 Sep 2023 12:56:01 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 72D6748D54
for <pve-devel@lists.proxmox.com>; Tue, 26 Sep 2023 12:56:01 +0200 (CEST)
Message-ID: <6aab04ab-2d39-42ac-b389-8e563c7322d0@proxmox.com>
Date: Tue, 26 Sep 2023 12:56:00 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: de-AT, en-US
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>,
Philipp Hufnagl <p.hufnagl@proxmox.com>,
=?UTF-8?Q?Fabian_Gr=C3=BCnbichler?= <f.gruenbichler@proxmox.com>
References: <20230921130917.2000926-1-p.hufnagl@proxmox.com>
<20230921130917.2000926-2-p.hufnagl@proxmox.com>
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
Autocrypt: addr=t.lamprecht@proxmox.com; keydata=
xsFNBFsLjcYBEACsaQP6uTtw/xHTUCKF4VD4/Wfg7gGn47+OfCKJQAD+Oyb3HSBkjclopC5J
uXsB1vVOfqVYE6PO8FlD2L5nxgT3SWkc6Ka634G/yGDU3ZC3C/7NcDVKhSBI5E0ww4Qj8s9w
OQRloemb5LOBkJNEUshkWRTHHOmk6QqFB/qBPW2COpAx6oyxVUvBCgm/1S0dAZ9gfkvpqFSD
90B5j3bL6i9FIv3YGUCgz6Ue3f7u+HsEAew6TMtlt90XV3vT4M2IOuECG/pXwTy7NtmHaBQ7
UJBcwSOpDEweNob50+9B4KbnVn1ydx+K6UnEcGDvUWBkREccvuExvupYYYQ5dIhRFf3fkS4+
wMlyAFh8PQUgauod+vqs45FJaSgTqIALSBsEHKEs6IoTXtnnpbhu3p6XBin4hunwoBFiyYt6
YHLAM1yLfCyX510DFzX/Ze2hLqatqzY5Wa7NIXqYYelz7tXiuCLHP84+sV6JtEkeSUCuOiUY
virj6nT/nJK8m0BzdR6FgGtNxp7RVXFRz/+mwijJVLpFsyG1i0Hmv2zTn3h2nyGK/I6yhFNt
dX69y5hbo6LAsRjLUvZeHXpTU4TrpN/WiCjJblbj5um5eEr4yhcwhVmG102puTtuCECsDucZ
jpKpUqzXlpLbzG/dp9dXFH3MivvfuaHrg3MtjXY1i+/Oxyp5iwARAQABzTNUaG9tYXMgTGFt
cHJlY2h0IChBdXRoLTQpIDx0LmxhbXByZWNodEBwcm94bW94LmNvbT7CwY4EEwEIADgWIQQO
R4qbEl/pah9K6VrTZCM6gDZWBgUCWwuNxgIbAwULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAK
CRDTZCM6gDZWBm/jD/4+6JB2s67eaqoP6x9VGaXNGJPCscwzLuxDTCG90G9FYu29VcXtubH/
bPwsyBbNUQpqTm/s4XboU2qpS5ykCuTjqavrcP33tdkYfGcItj2xMipJ1i3TWvpikQVsX42R
G64wovLs/dvpTYphRZkg5DwhgTmy3mRkmofFCTa+//MOcNOORltemp984tWjpR3bUJETNWpF
sKGZHa3N4kCNxb7A+VMsJZ/1gN3jbQbQG7GkJtnHlWkw9rKCYqBtWrnrHa4UAvSa9M/XCIAB
FThFGqZI1ojdVlv5gd6b/nWxfOPrLlSxbUo5FZ1i/ycj7/24nznW1V4ykG9iUld4uYUY86bB
UGSjew1KYp9FmvKiwEoB+zxNnuEQfS7/Bj1X9nxizgweiHIyFsRqgogTvLh403QMSGNSoArk
tqkorf1U+VhEncIn4H3KksJF0njZKfilrieOO7Vuot1xKr9QnYrZzJ7m7ZxJ/JfKGaRHXkE1
feMmrvZD1AtdUATZkoeQtTOpMu4r6IQRfSdwm/CkppZXfDe50DJxAMDWwfK2rr2bVkNg/yZI
tKLBS0YgRTIynkvv0h8d9dIjiicw3RMeYXyqOnSWVva2r+tl+JBaenr8YTQw0zARrhC0mttu
cIZGnVEvQuDwib57QLqMjQaC1gazKHvhA15H5MNxUhwm229UmdH3KM7BTQRbC43GARAAyTkR
D6KRJ9Xa2fVMh+6f186q0M3ni+5tsaVhUiykxjsPgkuWXWW9MbLpYXkzX6h/RIEKlo2BGA95
QwG5+Ya2Bo3g7FGJHAkXY6loq7DgMp5/TVQ8phsSv3WxPTJLCBq6vNBamp5hda4cfXFUymsy
HsJy4dtgkrPQ/bnsdFDCRUuhJHopnAzKHN8APXpKU6xV5e3GE4LwFsDhNHfH/m9+2yO/trcD
txSFpyftbK2gaMERHgA8SKkzRhiwRTt9w5idOfpJVkYRsgvuSGZ0pcD4kLCOIFrer5xXudk6
NgJc36XkFRMnwqrL/bB4k6Pi2u5leyqcXSLyBgeHsZJxg6Lcr2LZ35+8RQGPOw9C0ItmRjtY
ZpGKPlSxjxA1WHT2YlF9CEt3nx7c4C3thHHtqBra6BGPyW8rvtq4zRqZRLPmZ0kt/kiMPhTM
8wZAlObbATVrUMcZ/uNjRv2vU9O5aTAD9E5r1B0dlqKgxyoImUWB0JgpILADaT3VybDd3C8X
s6Jt8MytUP+1cEWt9VKo4vY4Jh5vwrJUDLJvzpN+TsYCZPNVj18+jf9uGRaoK6W++DdMAr5l
gQiwsNgf9372dbMI7pt2gnT5/YdG+ZHnIIlXC6OUonA1Ro/Itg90Q7iQySnKKkqqnWVc+qO9
GJbzcGykxD6EQtCSlurt3/5IXTA7t6sAEQEAAcLBdgQYAQgAIBYhBA5HipsSX+lqH0rpWtNk
IzqANlYGBQJbC43GAhsMAAoJENNkIzqANlYGD1sP/ikKgHgcspEKqDED9gQrTBvipH85si0j
/Jwu/tBtnYjLgKLh2cjv1JkgYYjb3DyZa1pLsIv6rGnPX9bH9IN03nqirC/Q1Y1lnbNTynPk
IflgvsJjoTNZjgu1wUdQlBgL/JhUp1sIYID11jZphgzfDgp/E6ve/8xE2HMAnf4zAfJaKgD0
F+fL1DlcdYUditAiYEuN40Ns/abKs8I1MYx7Yglu3RzJfBzV4t86DAR+OvuF9v188WrFwXCS
RSf4DmJ8tntyNej+DVGUnmKHupLQJO7uqCKB/1HLlMKc5G3GLoGqJliHjUHUAXNzinlpE2Vj
C78pxpwxRNg2ilE3AhPoAXrY5qED5PLE9sLnmQ9AzRcMMJUXjTNEDxEYbF55SdGBHHOAcZtA
kEQKub86e+GHA+Z8oXQSGeSGOkqHi7zfgW1UexddTvaRwE6AyZ6FxTApm8wq8NT2cryWPWTF
BDSGB3ujWHMM8ERRYJPcBSjTvt0GcEqnd+OSGgxTkGOdufn51oz82zfpVo1t+J/FNz6MRMcg
8nEC+uKvgzH1nujxJ5pRCBOquFZaGn/p71Yr0oVitkttLKblFsqwa+10Lt6HBxm+2+VLp4Ja
0WZNncZciz3V3cuArpan/ZhhyiWYV5FD0pOXPCJIx7WS9PTtxiv0AOS4ScWEUmBxyhFeOpYa DrEx
In-Reply-To: <20230921130917.2000926-2-p.hufnagl@proxmox.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.078 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
information. [nodes.pm]
Subject: Re: [pve-devel] [PATCH manager v8 1/2] fix #4849: api: download to
storage: automatically dectect and configure compression
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Tue, 26 Sep 2023 10:56:02 -0000
while this is already applied, some comments inline, for a possible next
time, and also the big
question if this is even required, after all I can just check the few
compression algorithms easily in the frontend, i.e., offloading a simple
string regex match to the backend seems rather odd to me..
Am 21/09/2023 um 15:09 schrieb Philipp Hufnagl:
> extends the query_url_metadata callback with the functionallity to
s/functionallity/functionality/
> detect used compressions. If a compression is used it tells the ui whic=
h
> one
>=20
> Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
> ---
> PVE/API2/Nodes.pm | 22 +++++++++++++++++++++-
> 1 file changed, 21 insertions(+), 1 deletion(-)
>=20
> diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
> index 5a148d1d..1e8ed09e 100644
> --- a/PVE/API2/Nodes.pm
> +++ b/PVE/API2/Nodes.pm
> @@ -34,6 +34,7 @@ use PVE::RRD;
> use PVE::Report;
> use PVE::SafeSyslog;
> use PVE::Storage;
> +use PVE::Storage::Plugin;
> use PVE::Tools;
> use PVE::pvecfg;
> =20
> @@ -1564,7 +1565,13 @@ __PACKAGE__->register_method({
> type =3D> 'boolean',
> optional =3D> 1,
> default =3D> 1,
> - }
> + },
> + 'detect-compression' =3D> {
> + description =3D> "If true an auto detection of used compression will=
be attempted",
Grammatically and semantically slightly better would be something like:
"If true, the queried filename is checked for a compression extension."
> + type =3D> 'boolean',
> + optional =3D> 1,
> + default =3D> 0,
> + },
> },
> },
> returns =3D> {
> @@ -1583,6 +1590,11 @@ __PACKAGE__->register_method({
> type =3D> 'string',
> optional =3D> 1,
> },
> + compression =3D> {
> + type =3D> 'string',
> + enum =3D> $PVE::Storage::Plugin::KNOWN_COMPRESSION_FORMATS,> + opti=
onal =3D> 1,
> + },
> },
> },
> code =3D> sub {
> @@ -1605,6 +1617,7 @@ __PACKAGE__->register_method({
> SSL_verify_mode =3D> IO::Socket::SSL::SSL_VERIFY_NONE,
> );
> }
> + my $detect_compression =3D $param->{'detect-compression'} // 0;
It's often best to avoid such intermediate variables if there's just a
single use case and on doesn't needs to "jump through hoops" to get
to the value =E2=80=93 e.g., a simple hash access like here if 100% fine,=
if
the value would to be pulled out of some deeply nested structure, or
assembled or the like, then it might have its merit to use an
intermediate variable.
> =20
> my $req =3D HTTP::Request->new(HEAD =3D> $url);
> my $res =3D $ua->request($req);
> @@ -1615,6 +1628,7 @@ __PACKAGE__->register_method({
> my $disposition =3D $res->header("Content-Disposition");
> my $type =3D $res->header("Content-Type");
> =20
> + my $compression;
Keep definition and use closer together (I'd moved this down directly abo=
ve the if t that sets it)
> my $filename;
> =20
> if ($disposition && ($disposition =3D~ m/filename=3D"([^"]*)"/ || $di=
sposition =3D~ m/filename=3D([^;]*)/)) {
> @@ -1628,10 +1642,16 @@ __PACKAGE__->register_method({
> $type =3D $1;
> }
> =20
> + if ($detect_compression && $filename =3D~ m!^(.+)\.(${\PVE::Storage::=
Plugin::COMPRESSOR_RE})$!) {
There are code paths where $filename is not yet defined here, resulting
in a rather ugly warning =E2=80=93 so this needs upfront checking too =E2=
=80=93 always
check where the value code path is coming in (yeah, Rust would do that fo=
r
you, but most API endpoints are small enough to be able to do so quickly =
also manually)
> + $filename =3D $1;
> + $compression =3D $2;
> + }
> +
> my $ret =3D {};
> $ret->{filename} =3D $filename if $filename;
> $ret->{size} =3D $size + 0 if $size;
> $ret->{mimetype} =3D $type if $type;
> + $ret->{compression} =3D $compression if $compression;
> =20
> return $ret;
> }});