From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with UTF8SMTPS id 865087A0C9 for ; Thu, 6 May 2021 15:15:07 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with UTF8SMTP id 7B6B12129E for ; Thu, 6 May 2021 15:15:07 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with UTF8SMTPS id D7DF32128F for ; Thu, 6 May 2021 15:15:06 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with UTF8SMTP id 9EBBA464F9; Thu, 6 May 2021 15:15:06 +0200 (CEST) Message-ID: <5affaa5f-0506-bed2-56b8-485460e43b74@proxmox.com> Date: Thu, 6 May 2021 15:15:05 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:89.0) Gecko/20100101 Thunderbird/89.0 Content-Language: en-US To: Proxmox VE development discussion , Lorenz Stechauner References: <20210506091010.40737-1-l.stechauner@proxmox.com> <20210506091105.40976-1-l.stechauner@proxmox.com> From: Dominik Csapak In-Reply-To: <20210506091105.40976-1-l.stechauner@proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.018 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.001 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [nodes.pm] Subject: Re: [pve-devel] [PATCH v4 manager 1/7] api: nodes: add query_url_metadata method X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 May 2021 13:15:07 -0000 one comment inline On 5/6/21 11:10, Lorenz Stechauner wrote: > Signed-off-by: Lorenz Stechauner > --- > PVE/API2/Nodes.pm | 95 +++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 95 insertions(+) > > diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm > index e58d9c10..a6681ab3 100644 > --- a/PVE/API2/Nodes.pm > +++ b/PVE/API2/Nodes.pm > @@ -11,6 +11,7 @@ use JSON; > use POSIX qw(LONG_MAX); > use Time::Local qw(timegm_nocheck); > use Socket; > +use IO::Socket::SSL; > > use PVE::API2Tools; > use PVE::APLInfo; > @@ -238,6 +239,7 @@ __PACKAGE__->register_method ({ > { name => 'netstat' }, > { name => 'network' }, > { name => 'qemu' }, > + { name => 'query-url-metadata' }, > { name => 'replication' }, > { name => 'report' }, > { name => 'rrd' }, # fixme: remove? > @@ -1595,6 +1597,99 @@ __PACKAGE__->register_method({ > return $rpcenv->fork_worker('download', undef, $user, $worker); > }}); > > +__PACKAGE__->register_method({ > + name => 'query_url_metadata', > + path => 'query-url-metadata', > + method => 'GET', > + description => "Query metadata of an URL: file size, file name and mime type.", > + proxyto => 'node', > + permissions => { > + check => ['perm', '/', [ 'Sys.Audit', 'Sys.Modify' ]], > + }, > + parameters => { > + additionalProperties => 0, > + properties => { > + node => get_standard_option('pve-node'), > + url => { > + description => "The URL to query the metadata from.", > + type => 'string', > + pattern => 'https?://.*', > + }, > + 'verify-certificates' => { > + description => "If false, no SSL/TLS certificates will be verified.", > + type => 'boolean', > + optional => 1, > + default => 1, > + } > + }, > + }, > + returns => { > + type => "object", > + properties => { > + filename => { > + type => 'string', > + optional => 1, > + }, > + size => { > + type => 'integer', > + renderer => 'bytes', > + optional => 1, > + }, > + mimetype => { > + type => 'string', > + optional => 1, > + }, > + }, > + }, > + code => sub { > + my ($param) = @_; > + > + my $url = $param->{url}; > + > + my $ua = LWP::UserAgent->new(); > + > + my $dccfg = PVE::Cluster::cfs_read_file('datacenter.cfg'); > + if ($dccfg->{http_proxy}) { > + $ua->proxy('http', $dccfg->{http_proxy}); > + } > + > + if (!$param->{'verify-certificates'}) { this does not work sadly we do not fill in the defaults from the api schema into $param so when nothing is given $param->{'verify-certificates'} will be 'undef' if you want a 'truthy' default you have to do it yourself: my $verify = $param->{'verify-certificates'} // 1; (although there are some pieces of code where we *do* inject the default, e.g. pmg config parsing...) > + $ua->ssl_opts( > + verify_hostname => 0, > + SSL_verify_mode => IO::Socket::SSL::SSL_VERIFY_NONE, > + ); > + } > + > + my $req = HTTP::Request->new(HEAD => $url); > + my $res = $ua->request($req); > + > + die "invalid server response: '" . $res->status_line() . "'\n" if ($res->code() != 200); > + > + my $size = $res->header("Content-Length"); > + my $disposition = $res->header("Content-Disposition"); > + my $type = $res->header("Content-Type"); > + > + my $filename; > + > + if ($disposition && $disposition =~ m/filename=(.+)/) { > + $filename = $1; > + } elsif ($url =~ m!^[^?]+/([^?/]*)(?:\?.*)?$!) { > + $filename = $1; > + } > + > + # Content-Type: text/html; charset=utf-8 > + if ($type && $type =~ m/^([^;]+);/) { > + $type = $1; > + } > + > + my $ret = {}; > + $ret->{filename} = $filename if $filename; > + $ret->{size} = $size + 0 if $size; > + $ret->{mimetype} = $type if $type; > + > + return $ret; > + }}); > + > __PACKAGE__->register_method({ > name => 'report', > path => 'report', >