From: Fiona Ebner <f.ebner@proxmox.com>
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>,
Leo Nunner <l.nunner@proxmox.com>
Subject: Re: [pve-devel] [PATCH v2 storage] fix #3004: show progress of offline migration in task log
Date: Wed, 30 Aug 2023 10:22:08 +0200 [thread overview]
Message-ID: <fb3e343a-f4b3-535f-2f6a-aad92c341407@proxmox.com> (raw)
In-Reply-To: <20221116110233.54160-1-l.nunner@proxmox.com>
Sorry about the late review!
Am 16.11.22 um 12:02 schrieb Leo Nunner:
> --- a/PVE/Storage.pm
> +++ b/PVE/Storage.pm
Needs a rebase, because files got moved to src/
> @@ -821,12 +821,25 @@ sub storage_migrate {
>
> my $new_volid;
> my $pattern = volume_imported_message(undef, 1);
> - my $match_volid_and_log = sub {
> + # Matches new volid and rate-limits dd output
> + my $match_and_log = sub {
Why rename the function?
> my $line = shift;
> + my $show = 1;
> +
> + # rate-limit dd logs
> + if ($line =~ /(?:\d+ bytes)(?:.+?copied, )(\d+) s/) {
> + if ($1 < 60) { # if < 60s, print every 3s
> + $show = !($1 % 3);
> + } elsif($1 < 600) { # if < 10mins, print every 10s
Style nit: missing space after elsif
> + $show = !($1 % 10);
> + } else { # else, print every 30s
> + $show = !($1 % 30);
> + }
> + }
Upon completion something strange happens: There's a duplicate final log
with fractional seconds and duplicate records in/out.
> 2023-08-30 09:57:05 Formatting '/mnt/pve/dir/images/105/vm-105-disk-0.raw', fmt=raw size=1073741824 preallocation=off
> 2023-08-30 09:57:08 1050218496 bytes (1.1 GB, 1002 MiB) copied, 3 s, 350 MB/s
> 2023-08-30 09:57:08 262144+0 records in
> 2023-08-30 09:57:08 262144+0 records out
> 2023-08-30 09:57:08 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 3.06228 s, 351 MB/s
> 2023-08-30 09:57:08 10458+11861 records in
> 2023-08-30 09:57:08 10458+11861 records out
> 2023-08-30 09:57:08 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.75082 s, 390 MB/s
> 2023-08-30 09:57:08 successfully imported 'dir:105/vm-105-disk-0.raw'
> 2023-08-30 10:00:38 Logical volume "vm-105-disk-0" created.
> 2023-08-30 10:00:42 873070592 bytes (873 MB, 833 MiB) copied, 3 s, 291 MB/s
> 2023-08-30 10:00:45 1944322048 bytes (1.9 GB, 1.8 GiB) copied, 6 s, 324 MB/s
> 2023-08-30 10:00:48 2976448512 bytes (3.0 GB, 2.8 GiB) copied, 9 s, 331 MB/s
> 2023-08-30 10:00:51 3998810112 bytes (4.0 GB, 3.7 GiB) copied, 12 s, 333 MB/s
> 2023-08-30 10:00:51 65536+0 records in
> 2023-08-30 10:00:51 65536+0 records out
> 2023-08-30 10:00:51 4294967296 bytes (4.3 GB, 4.0 GiB) copied, 13.1171 s, 327 MB/s
> 2023-08-30 10:00:53 44465+42142 records in
> 2023-08-30 10:00:53 44465+42142 records out
> 2023-08-30 10:00:53 4294967296 bytes (4.3 GB, 4.0 GiB) copied, 14.4084 s, 298 MB/s
> 2023-08-30 10:00:53 successfully imported 'lvmthin:vm-105-disk-0'
I think it's because we get it once from the source and once from the
target as better seen when using insecure migration:
> 2023-08-30 10:10:07 262144+0 records in
> 2023-08-30 10:10:07 262144+0 records out
> 2023-08-30 10:10:07 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.43834 s, 747 MB/s
> 2023-08-30 10:10:07 [pve8a2] Formatting '/mnt/pve/dir/images/105/vm-105-disk-0.raw', fmt=raw size=1073741824 preallocation=off
> 2023-08-30 10:10:07 [pve8a2] 6997+25632 records in
> 2023-08-30 10:10:07 [pve8a2] 6997+25632 records out
> 2023-08-30 10:10:07 [pve8a2] 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.41876 s, 757 MB/s
> 2023-08-30 10:10:07 [pve8a2] successfully imported 'dir:105/vm-105-disk-0.raw'
> 2023-08-30 10:10:07 volume 'dir:105/vm-105-disk-0.raw' is 'dir:105/vm-105-disk-0.raw' on the target
For insecure migration, we know which logs originate from source and
which from the target, so we could avoid the confusing duplicates. Maybe
there is a not too-involved way for secure migration too?
next prev parent reply other threads:[~2023-08-30 8:22 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-16 11:02 Leo Nunner
2023-08-30 8:22 ` Fiona Ebner [this message]
2023-08-31 9:00 ` Leo Nunner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=fb3e343a-f4b3-535f-2f6a-aad92c341407@proxmox.com \
--to=f.ebner@proxmox.com \
--cc=l.nunner@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox