all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Stefan Reiter <s.reiter@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH common 05/10] PBSClient: add file_restore_extract function
Date: Wed, 21 Apr 2021 13:15:34 +0200	[thread overview]
Message-ID: <20210421111539.29261-6-s.reiter@proxmox.com> (raw)
In-Reply-To: <20210421111539.29261-1-s.reiter@proxmox.com>

*_prepare creates a fifo for streaming data back to clients directly,
filefile_restore_extract blocks and should be called from a background
worker - while it is running outcoming data can be read from the FIFO.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
 src/PVE/PBSClient.pm | 63 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 62 insertions(+), 1 deletion(-)

diff --git a/src/PVE/PBSClient.pm b/src/PVE/PBSClient.pm
index f6b46b2..3d30155 100644
--- a/src/PVE/PBSClient.pm
+++ b/src/PVE/PBSClient.pm
@@ -5,9 +5,10 @@ use strict;
 use warnings;
 
 use Fcntl qw(F_GETFD F_SETFD FD_CLOEXEC);
+use File::Temp qw(tempdir);
 use IO::File;
 use JSON;
-use POSIX qw(strftime ENOENT);
+use POSIX qw(mkfifo strftime ENOENT);
 
 use PVE::JSONSchema qw(get_standard_option);
 use PVE::Tools qw(run_command file_set_contents file_get_contents file_read_firstline $IPV6RE);
@@ -343,4 +344,64 @@ sub file_restore_list {
     );
 }
 
+# call sync from API, returns a fifo path for streaming data to clients,
+# pass it to file_restore_extract to start transfering data
+sub file_restore_extract_prepare {
+    my ($self) = @_;
+
+    my $tmpdir = tempdir();
+    mkfifo("$tmpdir/fifo", 0600)
+	or die "creating file download fifo '$tmpdir/fifo' failed: $!\n";
+
+    # allow reading data for proxy user
+    my $wwwid = getpwnam('www-data') ||
+	die "getpwnam failed";
+    chown $wwwid, -1, "$tmpdir"
+	or die "changing permission on fifo dir '$tmpdir' failed: $!\n";
+    chown $wwwid, -1, "$tmpdir/fifo"
+	or die "changing permission on fifo '$tmpdir/fifo' failed: $!\n";
+
+    return "$tmpdir/fifo";
+}
+
+# this blocks while data is transfered, call this from a background worker
+sub file_restore_extract {
+    my ($self, $output_file, $snapshot, $filepath, $base64) = @_;
+
+    my $ret = eval {
+	local $SIG{ALRM} = sub { die "got timeout\n" };
+	alarm(30);
+	sysopen(my $fh, "$output_file", O_WRONLY)
+	    or die "open target '$output_file' for writing failed: $!\n";
+	alarm(0);
+
+	my $startcmd = sub {
+	    my ($cmd, %opts) = @_;
+
+	    delete $opts{outfunc};
+	    delete $opts{logfunc};
+	    $opts{errfunc} = sub { print $_[0], "\n"; };
+
+	    my $fn = fileno($fh);
+	    $opts{output} = ">&$fn";
+
+	    run_command($cmd, %opts);
+	};
+
+	return run_raw_client_cmd(
+	    $self, "extract",
+	    [ $snapshot, $filepath, "-", "--base64", $base64 ? 1 : 0 ],
+	    binary => "proxmox-file-restore",
+	    startcmd => $startcmd,
+	);
+    };
+
+    unlink($output_file);
+    $output_file =~ s/fifo$//;
+    rmdir($output_file) if -d $output_file;
+
+    die "file restore task failed: $@" if $@;
+    return $ret;
+}
+
 1;
-- 
2.20.1





  parent reply	other threads:[~2021-04-21 11:24 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-21 11:15 [pve-devel] [PATCH 00/10] Single-file-restore GUI for PBS snapshots Stefan Reiter
2021-04-21 11:15 ` [pve-devel] [PATCH RESEND common 01/10] JSONSchema: don't cycle-check 'download' responses Stefan Reiter
2021-04-21 15:37   ` [pve-devel] applied: " Thomas Lamprecht
2021-04-21 11:15 ` [pve-devel] [PATCH common 02/10] PBSClient: allow running other binaries Stefan Reiter
2021-04-21 14:29   ` Thomas Lamprecht
2021-04-21 14:38     ` Stefan Reiter
2021-04-21 14:50       ` Thomas Lamprecht
2021-04-21 15:37   ` [pve-devel] applied: " Thomas Lamprecht
2021-04-21 11:15 ` [pve-devel] [PATCH common 03/10] PBSClient: add file_restore_list command Stefan Reiter
     [not found]   ` <<20210421111539.29261-4-s.reiter@proxmox.com>
2021-04-21 13:19     ` Fabian Grünbichler
2021-04-21 11:15 ` [pve-devel] [PATCH common 04/10] PBSClient: allow different command execution callback Stefan Reiter
     [not found]   ` <<20210421111539.29261-5-s.reiter@proxmox.com>
2021-04-21 13:19     ` Fabian Grünbichler
2021-04-21 13:39       ` Stefan Reiter
2021-04-21 11:15 ` Stefan Reiter [this message]
2021-04-21 11:15 ` [pve-devel] [PATCH RESEND http-server 06/10] allow 'download' to be passed from API handler Stefan Reiter
2021-04-21 15:43   ` [pve-devel] applied: " Thomas Lamprecht
2021-04-21 11:15 ` [pve-devel] [PATCH http-server 07/10] support streaming data form fh to client Stefan Reiter
     [not found]   ` <<20210421111539.29261-8-s.reiter@proxmox.com>
2021-04-21 13:25     ` Fabian Grünbichler
2021-04-21 11:15 ` [pve-devel] [PATCH http-server 08/10] allow stream download from path and over pvedaemon-proxy Stefan Reiter
2021-04-21 11:15 ` [pve-devel] [PATCH storage 09/10] add FileRestore API for PBS Stefan Reiter
     [not found]   ` <<20210421111539.29261-10-s.reiter@proxmox.com>
2021-04-21 13:26     ` Fabian Grünbichler
2021-04-21 13:38       ` Stefan Reiter
2021-04-22  6:19         ` Fabian Grünbichler
2021-04-21 11:15 ` [pve-devel] [PATCH manager 10/10] backupview: add file restore button Stefan Reiter
2021-04-22 10:33 ` [pve-devel] [PATCH 00/10] Single-file-restore GUI for PBS snapshots Dominic Jäger
2021-04-22 12:12   ` Stefan Reiter

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210421111539.29261-6-s.reiter@proxmox.com \
    --to=s.reiter@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal