From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id E397B729BB for ; Wed, 16 Jun 2021 11:36:54 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 769CFFCA9 for ; Wed, 16 Jun 2021 11:36:24 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id DC40FFBC7 for ; Wed, 16 Jun 2021 11:36:20 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id B316543A70 for ; Wed, 16 Jun 2021 11:36:20 +0200 (CEST) From: Lorenz Stechauner To: pve-devel@lists.proxmox.com Date: Wed, 16 Jun 2021 11:36:00 +0200 Message-Id: <20210616093604.33668-5-l.stechauner@proxmox.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210616093604.33668-1-l.stechauner@proxmox.com> References: <20210616093604.33668-1-l.stechauner@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.845 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [nodes.pm] Subject: [pve-devel] [PATCH v9 manager 1/5] api: nodes: add query_url_metadata method X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jun 2021 09:36:54 -0000 metadata is gained using a HEAD request. Due to the ability of this api endpoint to request files on internal networks (which would not be visible/accessible from outside) it is restricted to users with permissions `Sys.Audit` and `Sys.Modify` on `/`. Users with these permissions are able to alter node (network) config anyway, so this should not create any further security risk. Signed-off-by: Lorenz Stechauner --- PVE/API2/Nodes.pm | 96 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm index e58d9c10..77fa710a 100644 --- a/PVE/API2/Nodes.pm +++ b/PVE/API2/Nodes.pm @@ -11,6 +11,7 @@ use JSON; use POSIX qw(LONG_MAX); use Time::Local qw(timegm_nocheck); use Socket; +use IO::Socket::SSL; use PVE::API2Tools; use PVE::APLInfo; @@ -238,6 +239,7 @@ __PACKAGE__->register_method ({ { name => 'netstat' }, { name => 'network' }, { name => 'qemu' }, + { name => 'query-url-metadata' }, { name => 'replication' }, { name => 'report' }, { name => 'rrd' }, # fixme: remove? @@ -1595,6 +1597,100 @@ __PACKAGE__->register_method({ return $rpcenv->fork_worker('download', undef, $user, $worker); }}); +__PACKAGE__->register_method({ + name => 'query_url_metadata', + path => 'query-url-metadata', + method => 'GET', + description => "Query metadata of an URL: file size, file name and mime type.", + proxyto => 'node', + permissions => { + check => ['perm', '/', [ 'Sys.Audit', 'Sys.Modify' ]], + }, + parameters => { + additionalProperties => 0, + properties => { + node => get_standard_option('pve-node'), + url => { + description => "The URL to query the metadata from.", + type => 'string', + pattern => 'https?://.*', + }, + 'verify-certificates' => { + description => "If false, no SSL/TLS certificates will be verified.", + type => 'boolean', + optional => 1, + default => 1, + } + }, + }, + returns => { + type => "object", + properties => { + filename => { + type => 'string', + optional => 1, + }, + size => { + type => 'integer', + renderer => 'bytes', + optional => 1, + }, + mimetype => { + type => 'string', + optional => 1, + }, + }, + }, + code => sub { + my ($param) = @_; + + my $url = $param->{url}; + + my $ua = LWP::UserAgent->new(); + + my $dccfg = PVE::Cluster::cfs_read_file('datacenter.cfg'); + if ($dccfg->{http_proxy}) { + $ua->proxy('http', $dccfg->{http_proxy}); + } + + my $verify = $param->{'verify-certificates'} // 1; + if (!$verify) { + $ua->ssl_opts( + verify_hostname => 0, + SSL_verify_mode => IO::Socket::SSL::SSL_VERIFY_NONE, + ); + } + + my $req = HTTP::Request->new(HEAD => $url); + my $res = $ua->request($req); + + die "invalid server response: '" . $res->status_line() . "'\n" if ($res->code() != 200); + + my $size = $res->header("Content-Length"); + my $disposition = $res->header("Content-Disposition"); + my $type = $res->header("Content-Type"); + + my $filename; + + if ($disposition && ($disposition =~ m/filename="([^"]*)"/ || $disposition =~ m/filename=([^;]*)/)) { + $filename = $1; + } elsif ($url =~ m!^[^?]+/([^?/]*)(?:\?.*)?$!) { + $filename = $1; + } + + # Content-Type: text/html; charset=utf-8 + if ($type && $type =~ m/^([^;]+);/) { + $type = $1; + } + + my $ret = {}; + $ret->{filename} = $filename if $filename; + $ret->{size} = $size + 0 if $size; + $ret->{mimetype} = $type if $type; + + return $ret; + }}); + __PACKAGE__->register_method({ name => 'report', path => 'report', -- 2.20.1