From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 7AFC599E84 for ; Thu, 4 May 2023 10:25:07 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 5A1761B53C for ; Thu, 4 May 2023 10:24:37 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Thu, 4 May 2023 10:24:36 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 157E547285 for ; Thu, 4 May 2023 10:24:36 +0200 (CEST) Date: Thu, 04 May 2023 10:24:29 +0200 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= To: Proxmox VE development discussion References: <20230425102136.85334-1-m.frank@proxmox.com> <20230425102136.85334-4-m.frank@proxmox.com> In-Reply-To: <20230425102136.85334-4-m.frank@proxmox.com> MIME-Version: 1.0 User-Agent: astroid/0.16.0 (https://github.com/astroidmail/astroid) Message-Id: <1683187729.g76xr51icf.astroid@yuna.none> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL 0.075 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [nodes.pm, haconfig.pm, certificates.pm, hardware.pm, network.pm, cluster.pm, ceph.pm, dirconfig.pm] Subject: Re: [pve-devel] [PATCH manager v4 3/6] added Config for Shared Filesystem Directories X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 May 2023 08:25:07 -0000 see cover letter for high level aspects! On April 25, 2023 12:21 pm, Markus Frank wrote: > and made an API Endpoint for getting, adding and removing > directories to the config. >=20 > Signed-off-by: Markus Frank > --- > PVE/API2/DirConfig.pm | 129 +++++++++++++++++++++++++++++++++++ > PVE/API2/Makefile | 1 + > PVE/API2/Nodes.pm | 6 ++ > PVE/DirConfig.pm | 155 ++++++++++++++++++++++++++++++++++++++++++ > PVE/Makefile | 1 + > 5 files changed, 292 insertions(+) > create mode 100644 PVE/API2/DirConfig.pm > create mode 100644 PVE/DirConfig.pm >=20 > diff --git a/PVE/API2/DirConfig.pm b/PVE/API2/DirConfig.pm > new file mode 100644 > index 00000000..0cbc6f96 > --- /dev/null > +++ b/PVE/API2/DirConfig.pm > @@ -0,0 +1,129 @@ > +package PVE::API2::DirConfig; > + > +use strict; > +use warnings; > + > +use PVE::JSONSchema qw(get_standard_option); > +use PVE::DirConfig; > +use PVE::Tools qw(extract_param); > + > +use base qw(PVE::RESTHandler); > + > +__PACKAGE__->register_method({ > + name =3D> 'get_config', > + path =3D> '', > + method =3D> 'GET', > + description =3D> "Get Directories for Host Directory Sharing.", > + permissions =3D> { > + check =3D> ['perm', '/map/dirs', [ 'Map.Audit' ]], > + }, > + proxyto =3D> 'node', > + parameters =3D> { > + additionalProperties =3D> 0, > + properties =3D> { > + node =3D> get_standard_option('pve-node'), > + }, > + }, > + returns =3D> { > + type =3D> 'array', > + items =3D> { > + type =3D> 'object', > + properties =3D> { > + dirid =3D> { > + type =3D> 'string', > + description =3D> 'Directory ID', > + }, > + path =3D> { > + type =3D> 'string', > + description =3D> 'Host Directory Path', > + }, > + }, > + }, > + }, > + code =3D> sub { > + my ($param) =3D @_; > + > + my $config =3D PVE::DirConfig::load_config($param->{node}); > + delete $config->{description}; > + my $result =3D []; > + foreach my $key (keys %{$config}) { > + push @$result, { > + dirid =3D> $key, > + path =3D> $config->{$key}, > + }; > + } > + > + return $result; > + } > +}); > + > +__PACKAGE__->register_method({ > + name =3D> 'add_dir', > + path =3D> '', > + method =3D> 'POST', > + description =3D> "Add Directories for Host Directory Sharing.", > + permissions =3D> { > + check =3D> ['perm', '/map/dirs', [ 'Map.Modify' ]], > + }, > + protected =3D> 1, > + proxyto =3D> 'node', > + parameters =3D> { > + additionalProperties =3D> 0, > + properties =3D> { > + node =3D> get_standard_option('pve-node'), > + dirid =3D> { > + type =3D> 'string', > + pattern =3D> '[a-zA-Z0-9\-]+', > + }, > + path =3D> { > + type =3D> 'string', > + maxLength =3D> 4096, > + format =3D> 'pve-storage-path', there's probably a few more options in the future ;) e.g. at least read only would be expected, I am not sure which of the other things should go here and which should go to the usage side in qemu-server: - xattr (support) - acls (support) - recursion (i.e., cross mountpoint boundaries) - caching - direct IO the first three might be configurable here (to signify that the underlying storage supports this) and at the usage side (to control whether we want to pass an enabled feature on to the guest). caching and direct IO might be candidates for only being configurable at usage time. I also haven't checked whether 9P and (either C or rust) virtiofsd support the same things. > + }, > + }, > + }, > + returns =3D> { type =3D> "null" }, > + code =3D> sub { > + my ($param) =3D @_; > + my $node =3D extract_param($param, 'node'); > + my $dirid =3D extract_param($param, 'dirid'); > + my $path =3D extract_param($param, 'path'); > + PVE::DirConfig::add_dir_config($node, $dirid, $path); > + return undef; > + }, > +}); > + > + > +__PACKAGE__->register_method({ > + name =3D> 'del_dir', > + path =3D> '', > + method =3D> 'DELETE', > + description =3D> "Remove Directory from Host Directory Sharing.", > + permissions =3D> { > + check =3D> ['perm', '/map/dirs', [ 'Map.Modify' ]], > + }, > + protected =3D> 1, > + proxyto =3D> 'node', > + parameters =3D> { > + additionalProperties =3D> 0, > + properties =3D> { > + node =3D> get_standard_option('pve-node'), > + dirid =3D> { > + type =3D> 'string', > + pattern =3D> '[a-zA-Z0-9\-]+', > + }, > + }, > + }, > + returns =3D> { type =3D> "null" }, > + code =3D> sub { > + my ($param) =3D @_; > + > + my $node =3D extract_param($param, 'node'); > + my $dirid =3D extract_param($param, 'dirid'); > + PVE::DirConfig::del_dir_config($node, $dirid); > + return undef; > + }, > +}); > + > + > +1; > diff --git a/PVE/API2/Makefile b/PVE/API2/Makefile > index 5c08ebe0..1b96223c 100644 > --- a/PVE/API2/Makefile > +++ b/PVE/API2/Makefile > @@ -12,6 +12,7 @@ PERLSOURCE =3D \ > Ceph.pm \ > Certificates.pm \ > Cluster.pm \ > + DirConfig.pm \ > HAConfig.pm \ > Hardware.pm \ > Network.pm \ > diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm > index bfe5c40a..b4e2992f 100644 > --- a/PVE/API2/Nodes.pm > +++ b/PVE/API2/Nodes.pm > @@ -48,6 +48,7 @@ use PVE::API2::LXC::Status; > use PVE::API2::LXC; > use PVE::API2::Network; > use PVE::API2::NodeConfig; > +use PVE::API2::DirConfig; > use PVE::API2::Qemu::CPU; > use PVE::API2::Qemu; > use PVE::API2::Replication; > @@ -199,6 +200,11 @@ __PACKAGE__->register_method ({ > path =3D> 'config', > }); > =20 > +__PACKAGE__->register_method ({ > + subclass =3D> "PVE::API2::DirConfig", > + path =3D> 'dirs', > +}); > + > if ($have_sdn) { > __PACKAGE__->register_method ({ > subclass =3D> "PVE::API2::Network::SDN::Zones::Status", > diff --git a/PVE/DirConfig.pm b/PVE/DirConfig.pm > new file mode 100644 > index 00000000..56796029 > --- /dev/null > +++ b/PVE/DirConfig.pm > @@ -0,0 +1,155 @@ > +package PVE::DirConfig; > + > +use strict; > +use warnings; > + > +use PVE::JSONSchema qw(get_standard_option); > +use PVE::Tools qw(file_get_contents file_set_contents lock_file); > + > +my $dir_config_lock =3D '/var/lock/dirs.lock'; if this becomes cluster wide it needs to change. it should also probably be a cfs_registered file. > + > +sub config_file { > + my ($node) =3D @_; > + > + return "/etc/pve/nodes/${node}/dirs"; > +} > + > +sub load_config { > + my ($node) =3D @_; > + > + my $filename =3D config_file($node); > + my $raw =3D eval { PVE::Tools::file_get_contents($filename); }; > + return {} if !$raw; > + > + return parse_file_config($raw, $filename); > +} > + > +sub write_config { > + my ($node, $conf) =3D @_; > + > + my $filename =3D config_file($node); > + > + my $raw =3D write_file_config($conf); > + > + PVE::Tools::file_set_contents($filename, $raw); > +} > + > +sub lock_config { > + my ($node, $realcode, @param) =3D @_; > + > + # make sure configuration file is up-to-date > + my $code =3D sub { > + PVE::Cluster::cfs_update(); > + $realcode->(@_); > + }; > + > + my $res =3D lock_file($dir_config_lock, 10, $code, @param); > + > + die $@ if $@; > + > + return $res; > +} > + > +my $descr =3D " Description for Shared Files Directory Config.\n" > + ." Add Directories with:\n dirid: /path/to/share"; > + > +my $dir_desc =3D { > + path =3D> { > + type =3D> 'string', > + format_description =3D> 'path', > + description =3D> 'path of Directory ID', > + default_key =3D> 1, > + }, > +}; this is too restrictive with regards to future extensions IMHO. see above for some ideas. also see the discussions surrounding the hardware map format. > [..] left out the rest since I expect the format to change ;)