[pve-devel] [PATCH storage] fix #3004: show progress of offline migration in task log

[pve-devel] [PATCH storage] fix #3004: show progress of offline migration in task log

[Leo Nunner]

dd supports a 'status' flag, which enables it to show the copied bytes,
duration, and the transfer rate, which then get printed to stderr.

Signed-off-by: Leo Nunner <l.nunner@proxmox.com>
---
 PVE/Storage/LVMPlugin.pm | 2 +-
 PVE/Storage/Plugin.pm    | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/PVE/Storage/LVMPlugin.pm b/PVE/Storage/LVMPlugin.pm
index a706e0c..4b951e7 100644
--- a/PVE/Storage/LVMPlugin.pm
+++ b/PVE/Storage/LVMPlugin.pm
@@ -645,7 +645,7 @@ sub volume_export {
 	$size = int($1);
     });
     PVE::Storage::Plugin::write_common_header($fh, $size);
-    run_command(['dd', "if=$file", "bs=64k"], output => '>&'.fileno($fh));
+    run_command(['dd', "if=$file", "bs=64k", "status=progress"], output => '>&'.fileno($fh));
 }
 
 sub volume_import_formats {
diff --git a/PVE/Storage/Plugin.pm b/PVE/Storage/Plugin.pm
index 8a41df1..e35fa97 100644
--- a/PVE/Storage/Plugin.pm
+++ b/PVE/Storage/Plugin.pm
@@ -1496,7 +1496,7 @@ sub volume_export {
 	    goto unsupported if $with_snapshots || $file_format eq 'subvol';
 	    write_common_header($fh, $size);
 	    if ($file_format eq 'raw') {
-		run_command(['dd', "if=$file", "bs=4k"], output => '>&'.fileno($fh));
+		run_command(['dd', "if=$file", "bs=4k", "status=progress"], output => '>&'.fileno($fh));
 	    } else {
 		run_command(['qemu-img', 'convert', '-f', $file_format, '-O', 'raw', $file, '/dev/stdout'],
 		            output => '>&'.fileno($fh));
@@ -1506,7 +1506,7 @@ sub volume_export {
 	    my $data_format = $1;
 	    goto unsupported if !$with_snapshots || $file_format ne $data_format;
 	    write_common_header($fh, $size);
-	    run_command(['dd', "if=$file", "bs=4k"], output => '>&'.fileno($fh));
+	    run_command(['dd', "if=$file", "bs=4k", "status=progress"], output => '>&'.fileno($fh));
 	    return;
 	} elsif ($format eq 'tar+size') {
 	    goto unsupported if $file_format ne 'subvol';
-- 
2.30.2

Re: [pve-devel] [PATCH storage] fix #3004: show progress of offline migration in task log

[Thomas Lamprecht]

Am 14/11/2022 um 13:01 schrieb Leo Nunner:
> dd supports a 'status' flag, which enables it to show the copied bytes,
> duration, and the transfer rate, which then get printed to stderr.
> 

can you post an excerpt of what it looks like and how frequent it outputs?

as unlike a TTY it cannot tell the console to override existing lines, so I
guess this will do some periodic printing?

Re: [pve-devel] [PATCH storage] fix #3004: show progress of offline migration in task log

[Leo Nunner]

On 11/14/22 13:12, Thomas Lamprecht wrote:

> Am 14/11/2022 um 13:01 schrieb Leo Nunner:
>> dd supports a 'status' flag, which enables it to show the copied bytes,
>> duration, and the transfer rate, which then get printed to stderr.
>>
> can you post an excerpt of what it looks like and how frequent it outputs?

Currently, the output looks as following:

    2022-11-14 12:01:00 236978176 bytes (237 MB, 226 MiB) copied, 1 s,
    237 MB/s
    2022-11-14 12:01:01 596377600 bytes (596 MB, 569 MiB) copied, 2 s,
    298 MB/s
    2022-11-14 12:01:02 947388416 bytes (947 MB, 904 MiB) copied, 3 s,
    316 MB/s
    2022-11-14 12:01:03 1308295168 bytes (1.3 GB, 1.2 GiB) copied, 4 s,
    327 MB/s
    2022-11-14 12:01:04 1673527296 bytes (1.7 GB, 1.6 GiB) copied, 5 s,
    335 MB/s
    […]

> as unlike a TTY it cannot tell the console to override existing lines, so I
> guess this will do some periodic printing?

Yes, as of right now, it prints a new line every second. Maybe one 
option to make it "prettier"
would be to limit the lines printed (by, say, only printing every nth 
line/second), which should
be rather simple to implement by matching the dd output in Storage.pm, 
where log lines are
already being parsed [1].

[1] https://git.proxmox.com/?p=pve-storage.git;a=blob;f=PVE/Storage.pm#l824

Re: [pve-devel] [PATCH storage] fix #3004: show progress of offline migration in task log

[Thomas Lamprecht]

Am 14/11/2022 um 13:23 schrieb Leo Nunner:
> On 11/14/22 13:12, Thomas Lamprecht wrote:
>> as unlike a TTY it cannot tell the console to override existing lines, so I
>> guess this will do some periodic printing?
> 
> Yes, as of right now, it prints a new line every second. Maybe one option to make it "prettier"
> would be to limit the lines printed (by, say, only printing every nth line/second), which should
> be rather simple to implement by matching the dd output in Storage.pm, where log lines are
> already being parsed [1].


hmm, once every second is really a bit much; we have lots of users with big
disks but relatively slow throughput, so a duration of a few hours is
realistic, a poor sole may even need to wait for over a day (sending 4 TB with
50 MB/s need almost a day), which would then produce about 8.64 MB of progress
information in the task log (assuming 100 bytes per round).

What I always like for such things is to reduce report frequency with time,
that way you provide good value/cost ratio w.r.t. reporting for both ends of
the duration spectrum.

E.g., start out with once every 3s, then after a minute (20 rounds) cool the
frequency off to once every 10s, then after a total of ten minutes reduce
further to once every 30s and keep it at that. Compared to the straight 1 Hz
variant, which required ~8.64 MB, we would only need 293 KB. almost 30 times
less storage.