From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <f.gruenbichler@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id EC76D7542C
for <pve-devel@lists.proxmox.com>; Wed, 21 Apr 2021 15:26:41 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id E938FFCA2
for <pve-devel@lists.proxmox.com>; Wed, 21 Apr 2021 15:26:41 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[94.136.29.106])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS id A46C4FC98
for <pve-devel@lists.proxmox.com>; Wed, 21 Apr 2021 15:26:40 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 7AC0E45B8F
for <pve-devel@lists.proxmox.com>; Wed, 21 Apr 2021 15:26:40 +0200 (CEST)
Date: Wed, 21 Apr 2021 15:26:32 +0200
From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= <f.gruenbichler@proxmox.com>
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
References: <20210421111539.29261-1-s.reiter@proxmox.com>
<20210421111539.29261-10-s.reiter@proxmox.com>
In-Reply-To: <<20210421111539.29261-10-s.reiter@proxmox.com>
MIME-Version: 1.0
User-Agent: astroid/0.15.0 (https://github.com/astroidmail/astroid)
Message-Id: <1619010196.oa6mrn2qjn.astroid@nora.none>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-SPAM-LEVEL: Spam detection results: 0
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
information. [content.pm, prunebackups.pm, filerestore.pm, proxmox.com,
config.pm, status.pm, scan.pm]
Subject: Re: [pve-devel] [PATCH storage 09/10] add FileRestore API for PBS
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Wed, 21 Apr 2021 13:26:42 -0000
On April 21, 2021 1:15 pm, Stefan Reiter wrote:
> Includes list and restore calls.
>=20
> Requires VM.Backup and Datastore.Audit permissions, for the accessed
> VM/CT and containing datastore respectively.
we require Datastore.AllocateSpace + VM.Backup for the owning guest,=20
or Datastore.Allocate for the storage altogether for accessing backup=20
archives otherwise, maybe this should have the same logic?
>=20
> Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
> ---
>=20
> Requires updated pve-common, pve-http-server.
>=20
> PVE/API2/Storage/FileRestore.pm | 163 ++++++++++++++++++++++++++++++++
> PVE/API2/Storage/Makefile | 2 +-
> PVE/API2/Storage/Status.pm | 6 ++
> 3 files changed, 170 insertions(+), 1 deletion(-)
> create mode 100644 PVE/API2/Storage/FileRestore.pm
>=20
> diff --git a/PVE/API2/Storage/FileRestore.pm b/PVE/API2/Storage/FileResto=
re.pm
> new file mode 100644
> index 0000000..a0b5e88
> --- /dev/null
> +++ b/PVE/API2/Storage/FileRestore.pm
> @@ -0,0 +1,163 @@
> +package PVE::API2::Storage::FileRestore;
> +
> +use strict;
> +use warnings;
> +
> +use PVE::JSONSchema qw(get_standard_option);
> +use PVE::PBSClient;
> +use PVE::Storage;
> +use PVE::Tools qw(extract_param);
> +
> +use PVE::RESTHandler;
> +use base qw(PVE::RESTHandler);
> +
> +__PACKAGE__->register_method ({
> + name =3D> 'list',
> + path =3D> 'list',
> + method =3D> 'GET',
> + proxyto =3D> 'node',
> + permissions =3D> {
> + description =3D> "Requires 'VM.Backup' permission on the VM being acces=
sed, and " .
> + "'Datastore.Audit' on the datastore being restored from.",
> + user =3D> 'all', # checked explicitly
> + },
> + description =3D> "List files and directories for single file restore=
under the given path.",
> + protected =3D> 1,
> + parameters =3D> {
> + additionalProperties =3D> 0,
> + properties =3D> {
> + node =3D> get_standard_option('pve-node'),
> + storage =3D> get_standard_option('pve-storage-id'),
> + snapshot =3D> {
> + description =3D> "Backup snapshot identifier.",
> + type =3D> 'string',
> + },
why not use a volume id here (instead of storage + snapshot ID), and=20
then check inside whether it's a pbs backup? would allow easily=20
extending this to VMA backups as well later on, completion by our usual=20
volume ID helpers/selectors, ..
> + filepath =3D> {
> + description =3D> 'base64-path to the directory or file being listed, o=
r "/".',
> + type =3D> 'string',
> + },
> + },
> + },
> + returns =3D> {
> + type =3D> 'array',
> + items =3D> {
> + type =3D> "object",
> + properties =3D> {
> + filepath =3D> {
> + description =3D> "base64 path of the current entry",
> + type =3D> 'string',
> + },
> + type =3D> {
> + description =3D> "Entry type.",
> + type =3D> 'string',
> + },
> + text =3D> {
> + description =3D> "Entry display text.",
> + type =3D> 'string',
> + },
> + leaf =3D> {
> + description =3D> "If this entry is a leaf in the directory graph."=
,
> + type =3D> 'any', # JSON::PP::Boolean gets passed through
> + },
> + size =3D> {
> + description =3D> "Entry file size.",
> + type =3D> 'integer',
> + optional =3D> 1,
> + },
> + mtime =3D> {
> + description =3D> "Entry last-modified time (unix timestamp).",
> + type =3D> 'integer',
> + optional =3D> 1,
> + },
> + },
> + },
> + },
> + code =3D> sub {
> + my ($param) =3D @_;
> +
> + my $rpcenv =3D PVE::RPCEnvironment::get();
> + my $user =3D $rpcenv->get_user();
> +
> + my $path =3D extract_param($param, 'filepath') || "/";
> + my $base64 =3D $path ne "/";
> + my $snap =3D extract_param($param, 'snapshot');
> + my $storeid =3D extract_param($param, 'storage');
> + my $cfg =3D PVE::Storage::config();
> + my $scfg =3D PVE::Storage::storage_config($cfg, $storeid);
> +
> + my $volid =3D "$storeid:backup/$snap";
> + my (undef, undef, $ownervm) =3D PVE::Storage::parse_volname($cfg, $voli=
d);
> + $rpcenv->check($user, "/storage/$storeid", ['Datastore.Audit']);
> + $rpcenv->check($user, "/vms/$ownervm", ['VM.Backup']);
see comment above, this could then become=20
'PVE::Storage::check_volume_access(..)
> +
> + my $client =3D PVE::PBSClient->new($scfg, $storeid);
> + my $ret =3D $client->file_restore_list($snap, $path, $base64);
> +
> + return $ret;
> + }});
> +
> +__PACKAGE__->register_method ({
> + name =3D> 'download',
> + path =3D> 'download',
> + method =3D> 'GET',
> + proxyto =3D> 'node',
> + permissions =3D> {
> + description =3D> "Requires 'VM.Backup' permission on the VM being acces=
sed, and " .
> + "'Datastore.Audit' on the datastore being restored from.",
> + user =3D> 'all', # checked explicitly
> + },
> + description =3D> "Extract a file or directory (as zip archive) from =
a PBS backup.",
> + parameters =3D> {
> + additionalProperties =3D> 0,
> + properties =3D> {
> + node =3D> get_standard_option('pve-node'),
> + storage =3D> get_standard_option('pve-storage-id'),
> + snapshot =3D> {
> + description =3D> "Backup snapshot identifier.",
> + type =3D> 'string',
> + },
same here as above
> + filepath =3D> {
> + description =3D> 'base64-path to the directory or file being listed.',
> + type =3D> 'string',
> + },
> + },
> + },
> + returns =3D> {
> + type =3D> 'any', # download
> + },
> + protected =3D> 1,
> + code =3D> sub {
> + my ($param) =3D @_;
> +
> + my $rpcenv =3D PVE::RPCEnvironment::get();
> + my $user =3D $rpcenv->get_user();
> +
> + my $path =3D extract_param($param, 'filepath');
> + my $snap =3D extract_param($param, 'snapshot');
> + my $storeid =3D extract_param($param, 'storage');
> + my $cfg =3D PVE::Storage::config();
> + my $scfg =3D PVE::Storage::storage_config($cfg, $storeid);
> +
> + my $volid =3D "$storeid:backup/$snap";
> + my (undef, undef, $ownervm) =3D PVE::Storage::parse_volname($cfg, $voli=
d);
> + $rpcenv->check($user, "/storage/$storeid", ['Datastore.Audit']);
> + $rpcenv->check($user, "/vms/$ownervm", ['VM.Backup']);
and here as well
> +
> + my $client =3D PVE::PBSClient->new($scfg, $storeid);
> + my $fifo =3D $client->file_restore_extract_prepare();
> +
> + $rpcenv->fork_worker('pbs-download', undef, $user, sub {
> + $client->file_restore_extract($fifo, $snap, $path, 1);
> + });
> +
> + my $ret =3D {
> + download =3D> {
> + path =3D> $fifo,
> + stream =3D> 1,
> + 'content-type' =3D> 'application/octet-stream',
> + },
> + };
> + return $ret;
> + }});
> +
> +1;
> diff --git a/PVE/API2/Storage/Makefile b/PVE/API2/Storage/Makefile
> index 690b437..1705080 100644
> --- a/PVE/API2/Storage/Makefile
> +++ b/PVE/API2/Storage/Makefile
> @@ -1,5 +1,5 @@
> =20
> -SOURCES=3D Content.pm Status.pm Config.pm PruneBackups.pm Scan.pm
> +SOURCES=3D Content.pm Status.pm Config.pm PruneBackups.pm Scan.pm FileRe=
store.pm
> =20
> .PHONY: install
> install:
> diff --git a/PVE/API2/Storage/Status.pm b/PVE/API2/Storage/Status.pm
> index d12643f..897b4a7 100644
> --- a/PVE/API2/Storage/Status.pm
> +++ b/PVE/API2/Storage/Status.pm
> @@ -12,6 +12,7 @@ use PVE::RRD;
> use PVE::Storage;
> use PVE::API2::Storage::Content;
> use PVE::API2::Storage::PruneBackups;
> +use PVE::API2::Storage::FileRestore;
> use PVE::RESTHandler;
> use PVE::RPCEnvironment;
> use PVE::JSONSchema qw(get_standard_option);
> @@ -32,6 +33,11 @@ __PACKAGE__->register_method ({
> path =3D> '{storage}/content',
> });
> =20
> +__PACKAGE__->register_method ({
> + subclass =3D> "PVE::API2::Storage::FileRestore",
> + path =3D> '{storage}/file-restore',
> +});
> +
> __PACKAGE__->register_method ({
> name =3D> 'index',
> path =3D> '',
> --=20
> 2.20.1
>=20
>=20
>=20
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>=20
>=20
>=20
=