[pve-devel] [PATCH manager/widget-toolkit v6] fix: #3971 Tasklog download button

[pve-devel] [PATCH manager/widget-toolkit v6] fix: #3971 Tasklog download button

[Daniel Tschlatscher]

This patch series' aim is to add a download button in the tasklog-
viewer GUI so that users may access the tasklog more easily.
(The tasklog-viewer only displays 50 lines at a time)
Instead of suddenly returning a file stream when the 'limit' parameter
is set to 0, now, a new parameter 'download' needs to be passed.
This parameter is mutually exclusive with the other parameters.

With the backend patches for pmg and pbs already being applied, these
are the final patches to make the download work.
Therefore, this revision only contains the missing backend patch for
PVE and the GUI patches in the widget-toolkit for the button in the
TaskViewer.

Changes from v5:
* The last version used parameter 'limit=0' instead of 'download=1'
  for the download URL.
* Cleaned up one NIT in manager.

Thanks to sterzy for the review!

pve-manager:

Daniel Tschlatscher (1):
  make task log downloadable in the PVE manager backend

 PVE/API2/Tasks.pm | 49 ++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 42 insertions(+), 7 deletions(-)


proxmox-widget-toolkit:

Daniel Tschlatscher (2):
  Source file download call in central function
  add task log download button in TaskViewer

 src/Utils.js              | 13 +++++++++++++
 src/window/FileBrowser.js | 11 +++++------
 src/window/TaskViewer.js  | 17 +++++++++++++++--
 3 files changed, 33 insertions(+), 8 deletions(-)

-- 
2.30.2

[pve-devel] [PATCH manager v6 1/3] make task log downloadable in the PVE manager backend

[Daniel Tschlatscher]

The read_tasklog API call now stream the whole log file if the query
parameter 'download' is set to true.
This is done in preparation for the task log download button to be
added in the TaskViewer.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
---
Changes from v5:
* Made the if check for whether the download parameter is used with
  other parameters more readable.

 PVE/API2/Tasks.pm | 49 ++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 42 insertions(+), 7 deletions(-)

diff --git a/PVE/API2/Tasks.pm b/PVE/API2/Tasks.pm
index 9cd1e56b..e541dd55 100644
--- a/PVE/API2/Tasks.pm
+++ b/PVE/API2/Tasks.pm
@@ -342,15 +342,20 @@ __PACKAGE__->register_method({
 		minimum => 0,
 		default => 0,
 		optional => 1,
-		description => "The line number to start printing at.",
+		description => "Start at this line when reading the tasklog",
 	    },
 	    limit => {
 		type => 'integer',
 		minimum => 0,
 		default => 50,
 		optional => 1,
-		description => "The maximum amount of lines that should be printed.",
+		description => "The amount of lines to read from the tasklog.",
 	    },
+	    download => {
+		type => 'boolean',
+		optional => 1,
+		description => "Whether the tasklog file should be downloaded. This parameter can't be used in conjunction with other parameters",
+	    }
 	},
     },
     returns => {
@@ -378,8 +383,6 @@ __PACKAGE__->register_method({
 	my $rpcenv = PVE::RPCEnvironment::get();
 	my $user = $rpcenv->get_user();
 	my $node = $param->{node};
-	my $start = $param->{start} // 0;
-	my $limit = $param->{limit} // 50;
 
 	$convert_token_task->($task);
 
@@ -387,11 +390,43 @@ __PACKAGE__->register_method({
 	    $rpcenv->check($user, "/nodes/$node", [ 'Sys.Audit' ]);
 	}
 
-	my ($count, $lines) = PVE::Tools::dump_logfile($filename, $start, $limit);
+	my $download = $param->{download} // 0;
 
-	$rpcenv->set_result_attrib('total', $count);
+	if ($download) {
+	    if (defined($param->{start}) || defined($param->{limit})) {
+		die "Parameter 'download' can't be used with other parameters\n";
+	    }
+
+	    my $fh;
+	    my $use_compression = ( -s $filename ) > 1024;
 
-	return $lines;
+	    # 1024 is a practical cutoff for the size distribution of our log files.
+	    if ($use_compression) {
+		open($fh, "-|", "/usr/bin/gzip", "-c", "$filename")
+		    or die "Could not create compressed file stream for file '$filename' - $!\n";
+	    } else {
+		open($fh, '<', $filename) or die "Could not open file '$filename' - $!\n";
+	    }
+
+	    return {
+		download => {
+		    fh => $fh,
+		    stream => 1,
+		    'content-encoding' => $use_compression ? 'gzip' : undef,
+		    'content-type' => "text/plain",
+		    'content-disposition' => "attachment; filename=\"".$param->{upid}."\"",
+		},
+	    },
+	} else {
+	    my $start = $param->{start} // 0;
+	    my $limit = $param->{limit} // 50;
+
+	    my ($count, $lines) = PVE::Tools::dump_logfile($filename, $start, $limit);
+
+	    $rpcenv->set_result_attrib('total', $count);
+
+	    return $lines;
+	}
     }});
 
 
-- 
2.30.2

[pve-devel] [PATCH widget-toolkit v6 2/3] Source file download in new Utils function

[Daniel Tschlatscher]

Adds a function for downloading a file from a remote URL in the Utils
class and uses it to revise one similar usage in FileBrowser.js

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
---
No changes from v5 (was last included in v2)

 src/Utils.js              | 13 +++++++++++++
 src/window/FileBrowser.js | 11 +++++------
 2 files changed, 18 insertions(+), 6 deletions(-)

diff --git a/src/Utils.js b/src/Utils.js
index 7255e3f..ef0c2b8 100644
--- a/src/Utils.js
+++ b/src/Utils.js
@@ -1360,6 +1360,19 @@ utilities: {
 	}
 	return `<span class="${cls}" style="${style}">${string}</span>`;
     },
+
+    // Setting filename here when downloading from a remote url sometimes fails in chromium browsers
+    // because of a bug when using attribute download in conjunction with a self signed certificate.
+    // For more info see https://bugs.chromium.org/p/chromium/issues/detail?id=993362
+    downloadAsFile: function(source, fileName) {
+	let hiddenElement = document.createElement('a');
+	hiddenElement.href = source;
+	hiddenElement.target = '_blank';
+	if (fileName) {
+	    hiddenElement.download = fileName;
+	}
+	hiddenElement.click();
+    },
 },
 
     singleton: true,
diff --git a/src/window/FileBrowser.js b/src/window/FileBrowser.js
index a519d6b..4e4c639 100644
--- a/src/window/FileBrowser.js
+++ b/src/window/FileBrowser.js
@@ -101,18 +101,17 @@ Ext.define("Proxmox.window.FileBrowser", {
 	    let params = { ...view.extraParams };
 	    params.filepath = data.filepath;
 
-	    let atag = document.createElement('a');
-	    atag.download = view.downloadPrefix + data.text;
+	    let filename = view.downloadPrefix + data.text;
 	    if (data.type === 'd') {
 		if (tar) {
 		    params.tar = 1;
-		    atag.download += ".tar.zst";
+		    filename += ".tar.zst";
 		} else {
-		    atag.download += ".zip";
+		    filename += ".zip";
 		}
 	    }
-	    atag.href = me.buildUrl(view.downloadURL, params);
-	    atag.click();
+
+	    Proxmox.Utils.downloadAsFile(me.buildUrl(view.downloadURL, params), filename);
 	},
 
 	fileChanged: function() {
-- 
2.30.2

[pve-devel] [PATCH widget-toolkit v6 3/3] add task log download button in TaskViewer

[Daniel Tschlatscher]

Adds a download button in the TaskViewer. Uses the newly created
downloadAsFile() method in the Utils class.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
---
Changes from v5
* Last version erroneously used "limit=0" instead of the download
  parameter in the download URL

 src/window/TaskViewer.js | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/src/window/TaskViewer.js b/src/window/TaskViewer.js
index 5d8bb84..3d8ea00 100644
--- a/src/window/TaskViewer.js
+++ b/src/window/TaskViewer.js
@@ -230,11 +230,23 @@ Ext.define('Proxmox.window.TaskViewer', {
 	    border: false,
 	});
 
+	let downloadBtn = new Ext.Button({
+	    text: gettext('Download'),
+	    iconCls: 'fa fa-download',
+	    handler: function() {
+		let url = '/api2/extjs/nodes/' +
+		    `${task.node}/tasks/${encodeURIComponent(me.upid)}/log?download=1`;
+
+		Proxmox.Utils.downloadAsFile(url);
+	    },
+	});
+
+
 	let logView = Ext.create('Proxmox.panel.LogView', {
 	    title: gettext('Output'),
-	    tbar: [stop_btn2],
+	    tbar: [stop_btn2, '->', downloadBtn],
 	    border: false,
-	    url: "/api2/extjs/nodes/" + task.node + "/tasks/" + encodeURIComponent(me.upid) + "/log",
+	    url: `/api2/extjs/nodes/${task.node}/tasks/${encodeURIComponent(me.upid)}/log`,
 	});
 
 	me.mon(statstore, 'load', function() {
@@ -249,6 +261,7 @@ Ext.define('Proxmox.window.TaskViewer', {
 
 	    stop_btn1.setDisabled(status !== 'running');
 	    stop_btn2.setDisabled(status !== 'running');
+	    downloadBtn.setDisabled(status === 'running');
 	});
 
 	statstore.startUpdate();
-- 
2.30.2

Re: [pve-devel] [PATCH manager/widget-toolkit v6] fix: #3971 Tasklog download button

[Stefan Sterz]

series lgtm now and seems to work as intended across pmg/pve/pbs
so consider this:

Tested-by: Stefan Sterz <s.sterz@proxmox.com>
Reviewed-by: Stefan Sterz <s.sterz@proxmox.com>


On 1/4/23 13:56, Daniel Tschlatscher wrote:
> This patch series' aim is to add a download button in the tasklog-
> viewer GUI so that users may access the tasklog more easily.
> (The tasklog-viewer only displays 50 lines at a time)
> Instead of suddenly returning a file stream when the 'limit' parameter
> is set to 0, now, a new parameter 'download' needs to be passed.
> This parameter is mutually exclusive with the other parameters.
> 
> With the backend patches for pmg and pbs already being applied, these
> are the final patches to make the download work.
> Therefore, this revision only contains the missing backend patch for
> PVE and the GUI patches in the widget-toolkit for the button in the
> TaskViewer.
> 
> Changes from v5:
> * The last version used parameter 'limit=0' instead of 'download=1'
>   for the download URL.
> * Cleaned up one NIT in manager.
> 
> Thanks to sterzy for the review!
> 
> pve-manager:
> 
> Daniel Tschlatscher (1):
>   make task log downloadable in the PVE manager backend
> 
>  PVE/API2/Tasks.pm | 49 ++++++++++++++++++++++++++++++++++++++++-------
>  1 file changed, 42 insertions(+), 7 deletions(-)
> 
> 
> proxmox-widget-toolkit:
> 
> Daniel Tschlatscher (2):
>   Source file download call in central function
>   add task log download button in TaskViewer
> 
>  src/Utils.js              | 13 +++++++++++++
>  src/window/FileBrowser.js | 11 +++++------
>  src/window/TaskViewer.js  | 17 +++++++++++++++--
>  3 files changed, 33 insertions(+), 8 deletions(-)
> 

[pve-devel] applied: [PATCH manager/widget-toolkit v6] fix: #3971 Tasklog download button

[Thomas Lamprecht]

Am 04/01/2023 um 13:56 schrieb Daniel Tschlatscher:
> pve-manager:
> 
> Daniel Tschlatscher (1):
>   make task log downloadable in the PVE manager backend

> proxmox-widget-toolkit:
> 
> Daniel Tschlatscher (2):
>   Source file download call in central function
>   add task log download button in TaskViewer
> 

applied series with Stefan Sterz's R-b and T-b tags, thanks!