all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
To: pve-devel@lists.proxmox.com, pbs-devel@lists.proxmox.com,
	pmg-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH pmg-api v2 5/7] revised task log download function in the PMG backend
Date: Wed,  7 Sep 2022 10:56:31 +0200	[thread overview]
Message-ID: <20220907085633.89113-6-d.tschlatscher@proxmox.com> (raw)
In-Reply-To: <20220907085633.89113-1-d.tschlatscher@proxmox.com>

With the newly added button in the tasklog the implementation for the
PMG server needs to be adapted. I saw an opportunity here to clear
some redundant code for displaying the tasklog and replace it with a
call to dump_logfile(), akin to how this is handled in pve-manager.

The tasklog download functionality now streams the file by invoking
the newly created function in pve-common.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
---
 src/PMG/API2/Tasks.pm | 34 +++++++++++-----------------------
 1 file changed, 11 insertions(+), 23 deletions(-)

diff --git a/src/PMG/API2/Tasks.pm b/src/PMG/API2/Tasks.pm
index 598fb4d..0fd3780 100644
--- a/src/PMG/API2/Tasks.pm
+++ b/src/PMG/API2/Tasks.pm
@@ -5,6 +5,7 @@ use warnings;
 use POSIX;
 use IO::File;
 use File::ReadBackwards;
+use File::stat;
 use PVE::Tools;
 use PVE::SafeSyslog;
 use PVE::RESTHandler;
@@ -272,33 +273,20 @@ __PACKAGE__->register_method({
 
 	my $restenv = PMG::RESTEnvironment->get();
 
-	my $fh = IO::File->new($filename, "r");
-	raise_param_exc({ upid => "no such task - unable to open file - $!" }) if !$fh;
+	my $start = $param->{start} // 0;
+	my $limit = $param->{limit} // 50;
 
-	my $start = $param->{start} || 0;
-	my $limit = $param->{limit} || 50;
-	my $count = 0;
-	my $line;
-	while (defined ($line = <$fh>)) {
-	    next if $count++ < $start;
-	    next if $limit <= 0;
-	    chomp $line;
-	    push @$lines, { n => $count, t => $line};
-	    $limit--;
-	}
+	if ($limit == 0) {
+	    # TCP Max Transfer Unit size is 1500, compression for lower numbers has no effect
+	    my $use_compression = stat($filename)->size > 1500;
+	    return PVE::Tools::stream_logfile($filename, $param->{upid}, $use_compression);
+	} else {
+	    my ($count, $lines) = PVE::Tools::dump_logfile($filename, $start, $limit);
 
-	close($fh);
+	    $restenv->set_result_attrib('total', $count);
 
-	# HACK: ExtJS store.guaranteeRange() does not like empty array
-	# so we add a line
-	if (!$count) {
-	    $count++;
-	    push @$lines, { n => $count, t => "no content"};
+	    return $lines;
 	}
-
-	$restenv->set_result_attrib('total', $count);
-
-	return $lines;
     }});
 
 
-- 
2.30.2





WARNING: multiple messages have this Message-ID
From: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
To: pve-devel@lists.proxmox.com, pbs-devel@lists.proxmox.com,
	pmg-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH pmg-api v2 5/7] revised task log download function in the PMG backend
Date: Wed,  7 Sep 2022 10:56:31 +0200	[thread overview]
Message-ID: <20220907085633.89113-6-d.tschlatscher@proxmox.com> (raw)
In-Reply-To: <20220907085633.89113-1-d.tschlatscher@proxmox.com>

With the newly added button in the tasklog the implementation for the
PMG server needs to be adapted. I saw an opportunity here to clear
some redundant code for displaying the tasklog and replace it with a
call to dump_logfile(), akin to how this is handled in pve-manager.

The tasklog download functionality now streams the file by invoking
the newly created function in pve-common.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
---
 src/PMG/API2/Tasks.pm | 34 +++++++++++-----------------------
 1 file changed, 11 insertions(+), 23 deletions(-)

diff --git a/src/PMG/API2/Tasks.pm b/src/PMG/API2/Tasks.pm
index 598fb4d..0fd3780 100644
--- a/src/PMG/API2/Tasks.pm
+++ b/src/PMG/API2/Tasks.pm
@@ -5,6 +5,7 @@ use warnings;
 use POSIX;
 use IO::File;
 use File::ReadBackwards;
+use File::stat;
 use PVE::Tools;
 use PVE::SafeSyslog;
 use PVE::RESTHandler;
@@ -272,33 +273,20 @@ __PACKAGE__->register_method({
 
 	my $restenv = PMG::RESTEnvironment->get();
 
-	my $fh = IO::File->new($filename, "r");
-	raise_param_exc({ upid => "no such task - unable to open file - $!" }) if !$fh;
+	my $start = $param->{start} // 0;
+	my $limit = $param->{limit} // 50;
 
-	my $start = $param->{start} || 0;
-	my $limit = $param->{limit} || 50;
-	my $count = 0;
-	my $line;
-	while (defined ($line = <$fh>)) {
-	    next if $count++ < $start;
-	    next if $limit <= 0;
-	    chomp $line;
-	    push @$lines, { n => $count, t => $line};
-	    $limit--;
-	}
+	if ($limit == 0) {
+	    # TCP Max Transfer Unit size is 1500, compression for lower numbers has no effect
+	    my $use_compression = stat($filename)->size > 1500;
+	    return PVE::Tools::stream_logfile($filename, $param->{upid}, $use_compression);
+	} else {
+	    my ($count, $lines) = PVE::Tools::dump_logfile($filename, $start, $limit);
 
-	close($fh);
+	    $restenv->set_result_attrib('total', $count);
 
-	# HACK: ExtJS store.guaranteeRange() does not like empty array
-	# so we add a line
-	if (!$count) {
-	    $count++;
-	    push @$lines, { n => $count, t => "no content"};
+	    return $lines;
 	}
-
-	$restenv->set_result_attrib('total', $count);
-
-	return $lines;
     }});
 
 
-- 
2.30.2





WARNING: multiple messages have this Message-ID
From: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
To: pve-devel@lists.proxmox.com, pbs-devel@lists.proxmox.com,
	pmg-devel@lists.proxmox.com
Subject: [pmg-devel] [PATCH pmg-api v2 5/7] revised task log download function in the PMG backend
Date: Wed,  7 Sep 2022 10:56:31 +0200	[thread overview]
Message-ID: <20220907085633.89113-6-d.tschlatscher@proxmox.com> (raw)
In-Reply-To: <20220907085633.89113-1-d.tschlatscher@proxmox.com>

With the newly added button in the tasklog the implementation for the
PMG server needs to be adapted. I saw an opportunity here to clear
some redundant code for displaying the tasklog and replace it with a
call to dump_logfile(), akin to how this is handled in pve-manager.

The tasklog download functionality now streams the file by invoking
the newly created function in pve-common.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
---
 src/PMG/API2/Tasks.pm | 34 +++++++++++-----------------------
 1 file changed, 11 insertions(+), 23 deletions(-)

diff --git a/src/PMG/API2/Tasks.pm b/src/PMG/API2/Tasks.pm
index 598fb4d..0fd3780 100644
--- a/src/PMG/API2/Tasks.pm
+++ b/src/PMG/API2/Tasks.pm
@@ -5,6 +5,7 @@ use warnings;
 use POSIX;
 use IO::File;
 use File::ReadBackwards;
+use File::stat;
 use PVE::Tools;
 use PVE::SafeSyslog;
 use PVE::RESTHandler;
@@ -272,33 +273,20 @@ __PACKAGE__->register_method({
 
 	my $restenv = PMG::RESTEnvironment->get();
 
-	my $fh = IO::File->new($filename, "r");
-	raise_param_exc({ upid => "no such task - unable to open file - $!" }) if !$fh;
+	my $start = $param->{start} // 0;
+	my $limit = $param->{limit} // 50;
 
-	my $start = $param->{start} || 0;
-	my $limit = $param->{limit} || 50;
-	my $count = 0;
-	my $line;
-	while (defined ($line = <$fh>)) {
-	    next if $count++ < $start;
-	    next if $limit <= 0;
-	    chomp $line;
-	    push @$lines, { n => $count, t => $line};
-	    $limit--;
-	}
+	if ($limit == 0) {
+	    # TCP Max Transfer Unit size is 1500, compression for lower numbers has no effect
+	    my $use_compression = stat($filename)->size > 1500;
+	    return PVE::Tools::stream_logfile($filename, $param->{upid}, $use_compression);
+	} else {
+	    my ($count, $lines) = PVE::Tools::dump_logfile($filename, $start, $limit);
 
-	close($fh);
+	    $restenv->set_result_attrib('total', $count);
 
-	# HACK: ExtJS store.guaranteeRange() does not like empty array
-	# so we add a line
-	if (!$count) {
-	    $count++;
-	    push @$lines, { n => $count, t => "no content"};
+	    return $lines;
 	}
-
-	$restenv->set_result_attrib('total', $count);
-
-	return $lines;
     }});
 
 
-- 
2.30.2





  parent reply	other threads:[~2022-09-07  8:58 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-09-07  8:56 [pve-devel] [PATCH http/common/manager/wt/backup/pmg v2] fix: #3971 Tasklog download button Daniel Tschlatscher
2022-09-07  8:56 ` [pmg-devel] " Daniel Tschlatscher
2022-09-07  8:56 ` [pbs-devel] " Daniel Tschlatscher
2022-09-07  8:56 ` [pve-devel] [PATCH proxmox-backup v2 1/7] make tasklog downloadable in the backup server backend Daniel Tschlatscher
2022-09-07  8:56   ` [pmg-devel] " Daniel Tschlatscher
2022-09-07  8:56   ` [pbs-devel] " Daniel Tschlatscher
2022-09-07  8:56 ` [pve-devel] [PATCH http-server v2 2/7] acknowledge content-disposition header Daniel Tschlatscher
2022-09-07  8:56   ` [pmg-devel] " Daniel Tschlatscher
2022-09-07  8:56   ` [pbs-devel] " Daniel Tschlatscher
2022-09-29 12:36   ` [pve-devel] applied: " Thomas Lamprecht
2022-09-29 12:36     ` [pmg-devel] applied: [pve-devel] " Thomas Lamprecht
2022-09-29 12:36     ` [pbs-devel] " Thomas Lamprecht
2022-09-07  8:56 ` [pve-devel] [PATCH common v2 3/7] stream file method for PMG and PVE Daniel Tschlatscher
2022-09-07  8:56   ` [pmg-devel] " Daniel Tschlatscher
2022-09-07  8:56   ` [pbs-devel] " Daniel Tschlatscher
2022-09-07  8:56 ` [pve-devel] [PATCH manager v2 4/7] revised task log API call for PVE Daniel Tschlatscher
2022-09-07  8:56   ` [pmg-devel] " Daniel Tschlatscher
2022-09-07  8:56   ` [pbs-devel] " Daniel Tschlatscher
2022-10-05 12:30   ` [pve-devel] " Thomas Lamprecht
2022-10-05 12:30     ` [pmg-devel] " Thomas Lamprecht
2022-10-05 12:30     ` [pbs-devel] " Thomas Lamprecht
2022-10-10 11:40     ` Daniel Tschlatscher
2022-10-10 11:40       ` [pmg-devel] " Daniel Tschlatscher
2022-10-10 11:40       ` [pbs-devel] " Daniel Tschlatscher
2022-10-10 13:10       ` Thomas Lamprecht
2022-10-10 13:10         ` [pmg-devel] " Thomas Lamprecht
2022-10-10 13:10         ` [pbs-devel] " Thomas Lamprecht
2022-09-07  8:56 ` Daniel Tschlatscher [this message]
2022-09-07  8:56   ` [pmg-devel] [PATCH pmg-api v2 5/7] revised task log download function in the PMG backend Daniel Tschlatscher
2022-09-07  8:56   ` [pbs-devel] " Daniel Tschlatscher
2022-09-08  9:36   ` [pve-devel] [pmg-devel] " Stefan Sterz
2022-09-08  9:36     ` Stefan Sterz
2022-09-08  9:36     ` [pbs-devel] " Stefan Sterz
2022-09-08 11:19     ` [pve-devel] " Daniel Tschlatscher
2022-09-08 11:19       ` Daniel Tschlatscher
2022-09-08 11:19       ` [pbs-devel] " Daniel Tschlatscher
2022-09-07  8:56 ` [pve-devel] [PATCH widget-toolkit v2 6/7] Source file download call in central function Daniel Tschlatscher
2022-09-07  8:56   ` [pmg-devel] " Daniel Tschlatscher
2022-09-07  8:56   ` [pbs-devel] " Daniel Tschlatscher
2022-09-07  8:56 ` [pve-devel] [PATCH widget-toolkit v2 7/7] add task log download button in TaskViewer Daniel Tschlatscher
2022-09-07  8:56   ` [pmg-devel] " Daniel Tschlatscher
2022-09-07  8:56   ` [pbs-devel] " Daniel Tschlatscher
2022-09-08  9:40 ` [pve-devel] [pmg-devel] [PATCH http/common/manager/wt/backup/pmg v2] fix: #3971 Tasklog download button Stefan Sterz
2022-09-08  9:40   ` Stefan Sterz
2022-09-08  9:40   ` [pbs-devel] " Stefan Sterz

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220907085633.89113-6-d.tschlatscher@proxmox.com \
    --to=d.tschlatscher@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    --cc=pmg-devel@lists.proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal