From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 7ED87604E5 for ; Wed, 2 Dec 2020 11:52:08 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 196BB1A4B7 for ; Wed, 2 Dec 2020 11:51:38 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 2DC0E1A417 for ; Wed, 2 Dec 2020 11:51:34 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id EC61C44226 for ; Wed, 2 Dec 2020 11:51:33 +0100 (CET) From: Thomas Lamprecht To: pve-devel@lists.proxmox.com Date: Wed, 2 Dec 2020 11:51:17 +0100 Message-Id: <20201202105122.1132809-2-t.lamprecht@proxmox.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201202105122.1132809-1-t.lamprecht@proxmox.com> References: <20201202105122.1132809-1-t.lamprecht@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.076 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [gluster.org, content.pm, status.pm, config.pm, scan.pm, pvesm.pm, prunebackups.pm] Subject: [pve-devel] applied: [PATCH storage 1/6] factor out scan CLI definition to real API module X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Dec 2020 10:52:08 -0000 we have a 1:1 copy of that code in pve-manager's PVE::API2::Scan, which we can avoid by using a common module form pvesm CLI and the API. This is the first basic step of dropping the code duplication in pve-manager. Signed-off-by: Thomas Lamprecht --- some very minor differences between both, CLI and pve-manager one, got already resolved here. PVE/API2/Storage/Makefile | 2 +- PVE/API2/Storage/Scan.pm | 341 +++++++++++++++++++++++++++++++++++++ PVE/CLI/pvesm.pm | 342 +------------------------------------- 3 files changed, 350 insertions(+), 335 deletions(-) create mode 100644 PVE/API2/Storage/Scan.pm diff --git a/PVE/API2/Storage/Makefile b/PVE/API2/Storage/Makefile index 3f667e8..690b437 100644 --- a/PVE/API2/Storage/Makefile +++ b/PVE/API2/Storage/Makefile @@ -1,5 +1,5 @@ -SOURCES= Content.pm Status.pm Config.pm PruneBackups.pm +SOURCES= Content.pm Status.pm Config.pm PruneBackups.pm Scan.pm .PHONY: install install: diff --git a/PVE/API2/Storage/Scan.pm b/PVE/API2/Storage/Scan.pm new file mode 100644 index 0000000..0e6fd9a --- /dev/null +++ b/PVE/API2/Storage/Scan.pm @@ -0,0 +1,341 @@ +package PVE::API2::Storage::Scan; + +use strict; +use warnings; + +# NOTE: This API endpoints are mounted by pve-manager's API2::Node module and pvesm CLI + +use PVE::JSONSchema qw(get_standard_option); +use PVE::RESTHandler; +use PVE::SafeSyslog; +use PVE::Storage::LVMPlugin; +use PVE::Storage; +use PVE::SysFSTools; + +use base qw(PVE::RESTHandler); + +__PACKAGE__->register_method({ + name => 'nfsscan', + path => 'nfs', + method => 'GET', + description => "Scan remote NFS server.", + protected => 1, + proxyto => "node", + permissions => { + check => ['perm', '/storage', ['Datastore.Allocate']], + }, + parameters => { + additionalProperties => 0, + properties => { + node => get_standard_option('pve-node'), + server => { + description => "The server address (name or IP).", + type => 'string', format => 'pve-storage-server', + }, + }, + }, + returns => { + type => 'array', + items => { + type => "object", + properties => { + path => { + description => "The exported path.", + type => 'string', + }, + options => { + description => "NFS export options.", + type => 'string', + }, + }, + }, + }, + code => sub { + my ($param) = @_; + + my $server = $param->{server}; + my $res = PVE::Storage::scan_nfs($server); + + my $data = []; + foreach my $k (sort keys %$res) { + push @$data, { path => $k, options => $res->{$k} }; + } + return $data; + }}); + +__PACKAGE__->register_method({ + name => 'cifsscan', + path => 'cifs', + method => 'GET', + description => "Scan remote CIFS server.", + protected => 1, + proxyto => "node", + permissions => { + check => ['perm', '/storage', ['Datastore.Allocate']], + }, + parameters => { + additionalProperties => 0, + properties => { + node => get_standard_option('pve-node'), + server => { + description => "The server address (name or IP).", + type => 'string', format => 'pve-storage-server', + }, + username => { + description => "User name.", + type => 'string', + optional => 1, + }, + password => { + description => "User password.", + type => 'string', + optional => 1, + }, + domain => { + description => "SMB domain (Workgroup).", + type => 'string', + optional => 1, + }, + }, + }, + returns => { + type => 'array', + items => { + type => "object", + properties => { + share => { + description => "The cifs share name.", + type => 'string', + }, + description => { + description => "Descriptive text from server.", + type => 'string', + }, + }, + }, + }, + code => sub { + my ($param) = @_; + + my $server = $param->{server}; + + my $username = $param->{username}; + my $password = $param->{password}; + my $domain = $param->{domain}; + + my $res = PVE::Storage::scan_cifs($server, $username, $password, $domain); + + my $data = []; + foreach my $k (sort keys %$res) { + push @$data, { share => $k, description => $res->{$k} }; + } + + return $data; + }}); + +# Note: GlusterFS currently does not have an equivalent of showmount. +# As workaround, we simply use nfs showmount. +# see http://www.gluster.org/category/volumes/ +__PACKAGE__->register_method({ + name => 'glusterfsscan', + path => 'glusterfs', + method => 'GET', + description => "Scan remote GlusterFS server.", + protected => 1, + proxyto => "node", + permissions => { + check => ['perm', '/storage', ['Datastore.Allocate']], + }, + parameters => { + additionalProperties => 0, + properties => { + node => get_standard_option('pve-node'), + server => { + description => "The server address (name or IP).", + type => 'string', format => 'pve-storage-server', + }, + }, + }, + returns => { + type => 'array', + items => { + type => "object", + properties => { + volname => { + description => "The volume name.", + type => 'string', + }, + }, + }, + }, + code => sub { + my ($param) = @_; + + my $server = $param->{server}; + my $res = PVE::Storage::scan_nfs($server); + + my $data = []; + foreach my $path (sort keys %$res) { + if ($path =~ m!^/([^\s/]+)$!) { + push @$data, { volname => $1 }; + } + } + return $data; + }}); + +__PACKAGE__->register_method({ + name => 'iscsiscan', + path => 'iscsi', + method => 'GET', + description => "Scan remote iSCSI server.", + protected => 1, + proxyto => "node", + permissions => { + check => ['perm', '/storage', ['Datastore.Allocate']], + }, + parameters => { + additionalProperties => 0, + properties => { + node => get_standard_option('pve-node'), + portal => { + description => "The iSCSI portal (IP or DNS name with optional port).", + type => 'string', format => 'pve-storage-portal-dns', + }, + }, + }, + returns => { + type => 'array', + items => { + type => "object", + properties => { + target => { + description => "The iSCSI target name.", + type => 'string', + }, + portal => { + description => "The iSCSI portal name.", + type => 'string', + }, + }, + }, + }, + code => sub { + my ($param) = @_; + + my $res = PVE::Storage::scan_iscsi($param->{portal}); + + my $data = []; + foreach my $k (sort keys %$res) { + push @$data, { target => $k, portal => join(',', @{$res->{$k}}) }; + } + + return $data; + }}); + +__PACKAGE__->register_method({ + name => 'lvmscan', + path => 'lvm', + method => 'GET', + description => "List local LVM volume groups.", + protected => 1, + proxyto => "node", + permissions => { + check => ['perm', '/storage', ['Datastore.Allocate']], + }, + parameters => { + additionalProperties => 0, + properties => { + node => get_standard_option('pve-node'), + }, + }, + returns => { + type => 'array', + items => { + type => "object", + properties => { + vg => { + description => "The LVM logical volume group name.", + type => 'string', + }, + }, + }, + }, + code => sub { + my ($param) = @_; + + my $res = PVE::Storage::LVMPlugin::lvm_vgs(); + return PVE::RESTHandler::hash_to_array($res, 'vg'); + }}); + +__PACKAGE__->register_method({ + name => 'lvmthinscan', + path => 'lvmthin', + method => 'GET', + description => "List local LVM Thin Pools.", + protected => 1, + proxyto => "node", + permissions => { + check => ['perm', '/storage', ['Datastore.Allocate']], + }, + parameters => { + additionalProperties => 0, + properties => { + node => get_standard_option('pve-node'), + vg => { + type => 'string', + pattern => '[a-zA-Z0-9\.\+\_][a-zA-Z0-9\.\+\_\-]+', # see lvm(8) manpage + maxLength => 100, + }, + }, + }, + returns => { + type => 'array', + items => { + type => "object", + properties => { + lv => { + description => "The LVM Thin Pool name (LVM logical volume).", + type => 'string', + }, + }, + }, + }, + code => sub { + my ($param) = @_; + + return PVE::Storage::LvmThinPlugin::list_thinpools($param->{vg}); + }}); + +__PACKAGE__->register_method({ + name => 'zfsscan', + path => 'zfs', + method => 'GET', + description => "Scan zfs pool list on local node.", + protected => 1, + proxyto => "node", + permissions => { + check => ['perm', '/storage', ['Datastore.Allocate']], + }, + parameters => { + additionalProperties => 0, + properties => { + node => get_standard_option('pve-node'), + }, + }, + returns => { + type => 'array', + items => { + type => "object", + properties => { + pool => { + description => "ZFS pool name.", + type => 'string', + }, + }, + }, + }, + code => sub { + my ($param) = @_; + + return PVE::Storage::scan_zfs(); + }}); diff --git a/PVE/CLI/pvesm.pm b/PVE/CLI/pvesm.pm index a2246cc..bc48617 100755 --- a/PVE/CLI/pvesm.pm +++ b/PVE/CLI/pvesm.pm @@ -16,6 +16,7 @@ use PVE::Tools qw(extract_param); use PVE::API2::Storage::Config; use PVE::API2::Storage::Content; use PVE::API2::Storage::PruneBackups; +use PVE::API2::Storage::Scan; use PVE::API2::Storage::Status; use PVE::JSONSchema qw(get_standard_option); use PVE::PTY; @@ -406,333 +407,6 @@ __PACKAGE__->register_method ({ } }); -__PACKAGE__->register_method ({ - name => 'nfsscan', - path => 'nfs', - method => 'GET', - description => "Scan remote NFS server.", - protected => 1, - proxyto => "node", - permissions => { - check => ['perm', '/storage', ['Datastore.Allocate']], - }, - parameters => { - additionalProperties => 0, - properties => { - node => get_standard_option('pve-node'), - server => { - description => "The server address (name or IP).", - type => 'string', format => 'pve-storage-server', - }, - }, - }, - returns => { - type => 'array', - items => { - type => "object", - properties => { - path => { - description => "The exported path.", - type => 'string', - }, - options => { - description => "NFS export options.", - type => 'string', - }, - }, - }, - }, - code => sub { - my ($param) = @_; - - my $server = $param->{server}; - my $res = PVE::Storage::scan_nfs($server); - - my $data = []; - foreach my $k (sort keys %$res) { - push @$data, { path => $k, options => $res->{$k} }; - } - return $data; - }}); - -__PACKAGE__->register_method ({ - name => 'cifsscan', - path => 'cifs', - method => 'GET', - description => "Scan remote CIFS server.", - protected => 1, - proxyto => "node", - permissions => { - check => ['perm', '/storage', ['Datastore.Allocate']], - }, - parameters => { - additionalProperties => 0, - properties => { - node => get_standard_option('pve-node'), - server => { - description => "The server address (name or IP).", - type => 'string', format => 'pve-storage-server', - }, - username => { - description => "User name.", - type => 'string', - optional => 1, - }, - password => { - description => "User password.", - type => 'string', - optional => 1, - }, - domain => { - description => "SMB domain (Workgroup).", - type => 'string', - optional => 1, - }, - }, - }, - returns => { - type => 'array', - items => { - type => "object", - properties => { - share => { - description => "The cifs share name.", - type => 'string', - }, - description => { - description => "Descriptive text from server.", - type => 'string', - }, - }, - }, - }, - code => sub { - my ($param) = @_; - - my $server = $param->{server}; - - my $username = $param->{username}; - my $password = $param->{password}; - my $domain = $param->{domain}; - - my $res = PVE::Storage::scan_cifs($server, $username, $password, $domain); - - my $data = []; - foreach my $k (sort keys %$res) { - push @$data, { share => $k, description => $res->{$k} }; - } - - return $data; - }}); - -# Note: GlusterFS currently does not have an equivalent of showmount. -# As workaround, we simply use nfs showmount. -# see http://www.gluster.org/category/volumes/ - -__PACKAGE__->register_method ({ - name => 'glusterfsscan', - path => 'glusterfs', - method => 'GET', - description => "Scan remote GlusterFS server.", - protected => 1, - proxyto => "node", - permissions => { - check => ['perm', '/storage', ['Datastore.Allocate']], - }, - parameters => { - additionalProperties => 0, - properties => { - node => get_standard_option('pve-node'), - server => { - description => "The server address (name or IP).", - type => 'string', format => 'pve-storage-server', - }, - }, - }, - returns => { - type => 'array', - items => { - type => "object", - properties => { - volname => { - description => "The volume name.", - type => 'string', - }, - }, - }, - }, - code => sub { - my ($param) = @_; - - my $server = $param->{server}; - my $res = PVE::Storage::scan_nfs($server); - - my $data = []; - foreach my $path (sort keys %$res) { - if ($path =~ m!^/([^\s/]+)$!) { - push @$data, { volname => $1 }; - } - } - return $data; - }}); - -__PACKAGE__->register_method ({ - name => 'iscsiscan', - path => 'iscsi', - method => 'GET', - description => "Scan remote iSCSI server.", - protected => 1, - proxyto => "node", - permissions => { - check => ['perm', '/storage', ['Datastore.Allocate']], - }, - parameters => { - additionalProperties => 0, - properties => { - node => get_standard_option('pve-node'), - portal => { - description => "The iSCSI portal (IP or DNS name with optional port).", - type => 'string', format => 'pve-storage-portal-dns', - }, - }, - }, - returns => { - type => 'array', - items => { - type => "object", - properties => { - target => { - description => "The iSCSI target name.", - type => 'string', - }, - portal => { - description => "The iSCSI portal name.", - type => 'string', - }, - }, - }, - }, - code => sub { - my ($param) = @_; - - my $res = PVE::Storage::scan_iscsi($param->{portal}); - - my $data = []; - foreach my $k (sort keys %$res) { - push @$data, { target => $k, portal => join(',', @{$res->{$k}}) }; - } - - return $data; - }}); - -__PACKAGE__->register_method ({ - name => 'lvmscan', - path => 'lvm', - method => 'GET', - description => "List local LVM volume groups.", - protected => 1, - proxyto => "node", - permissions => { - check => ['perm', '/storage', ['Datastore.Allocate']], - }, - parameters => { - additionalProperties => 0, - properties => { - node => get_standard_option('pve-node'), - }, - }, - returns => { - type => 'array', - items => { - type => "object", - properties => { - vg => { - description => "The LVM logical volume group name.", - type => 'string', - }, - }, - }, - }, - code => sub { - my ($param) = @_; - - my $res = PVE::Storage::LVMPlugin::lvm_vgs(); - return PVE::RESTHandler::hash_to_array($res, 'vg'); - }}); - -__PACKAGE__->register_method ({ - name => 'lvmthinscan', - path => 'lvmthin', - method => 'GET', - description => "List local LVM Thin Pools.", - protected => 1, - proxyto => "node", - permissions => { - check => ['perm', '/storage', ['Datastore.Allocate']], - }, - parameters => { - additionalProperties => 0, - properties => { - node => get_standard_option('pve-node'), - vg => { - type => 'string', - pattern => '[a-zA-Z0-9\.\+\_][a-zA-Z0-9\.\+\_\-]+', # see lvm(8) manpage - maxLength => 100, - }, - }, - }, - returns => { - type => 'array', - items => { - type => "object", - properties => { - lv => { - description => "The LVM Thin Pool name (LVM logical volume).", - type => 'string', - }, - }, - }, - }, - code => sub { - my ($param) = @_; - - return PVE::Storage::LvmThinPlugin::list_thinpools($param->{vg}); - }}); - -__PACKAGE__->register_method ({ - name => 'zfsscan', - path => 'zfs', - method => 'GET', - description => "Scan zfs pool list on local node.", - protected => 1, - proxyto => "node", - permissions => { - check => ['perm', '/storage', ['Datastore.Allocate']], - }, - parameters => { - additionalProperties => 0, - properties => { - node => get_standard_option('pve-node'), - }, - }, - returns => { - type => 'array', - items => { - type => "object", - properties => { - pool => { - description => "ZFS pool name.", - type => 'string', - }, - }, - }, - }, - code => sub { - my ($param) = @_; - - return PVE::Storage::scan_zfs(); - }}); - __PACKAGE__->register_method ({ name => 'prunebackups', path => 'prunebackups', @@ -847,7 +521,7 @@ our $cmddef = { free => [ "PVE::API2::Storage::Content", 'delete', ['volume'], { node => $nodename } ], scan => { - nfs => [ __PACKAGE__, 'nfsscan', ['server'], { node => $nodename }, sub { + nfs => [ "PVE::API2::Storage::Scan", 'nfsscan', ['server'], { node => $nodename }, sub { my $res = shift; my $maxlen = 0; @@ -859,7 +533,7 @@ our $cmddef = { printf "%-${maxlen}s %s\n", $rec->{path}, $rec->{options}; } }], - cifs => [ __PACKAGE__, 'cifsscan', ['server'], { node => $nodename }, sub { + cifs => [ "PVE::API2::Storage::Scan", 'cifsscan', ['server'], { node => $nodename }, sub { my $res = shift; my $maxlen = 0; @@ -871,14 +545,14 @@ our $cmddef = { printf "%-${maxlen}s %s\n", $rec->{share}, $rec->{description}; } }], - glusterfs => [ __PACKAGE__, 'glusterfsscan', ['server'], { node => $nodename }, sub { + glusterfs => [ "PVE::API2::Storage::Scan", 'glusterfsscan', ['server'], { node => $nodename }, sub { my $res = shift; foreach my $rec (@$res) { printf "%s\n", $rec->{volname}; } }], - iscsi => [ __PACKAGE__, 'iscsiscan', ['portal'], { node => $nodename }, sub { + iscsi => [ "PVE::API2::Storage::Scan", 'iscsiscan', ['portal'], { node => $nodename }, sub { my $res = shift; my $maxlen = 0; @@ -890,19 +564,19 @@ our $cmddef = { printf "%-${maxlen}s %s\n", $rec->{target}, $rec->{portal}; } }], - lvm => [ __PACKAGE__, 'lvmscan', [], { node => $nodename }, sub { + lvm => [ "PVE::API2::Storage::Scan", 'lvmscan', [], { node => $nodename }, sub { my $res = shift; foreach my $rec (@$res) { printf "$rec->{vg}\n"; } }], - lvmthin => [ __PACKAGE__, 'lvmthinscan', ['vg'], { node => $nodename }, sub { + lvmthin => [ "PVE::API2::Storage::Scan", 'lvmthinscan', ['vg'], { node => $nodename }, sub { my $res = shift; foreach my $rec (@$res) { printf "$rec->{lv}\n"; } }], - zfs => [ __PACKAGE__, 'zfsscan', [], { node => $nodename }, sub { + zfs => [ "PVE::API2::Storage::Scan", 'zfsscan', [], { node => $nodename }, sub { my $res = shift; foreach my $rec (@$res) { -- 2.20.1