From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 97D891FF0E5 for ; Wed, 15 Jul 2026 14:46:54 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 3F20621429; Wed, 15 Jul 2026 14:46:54 +0200 (CEST) Content-Type: text/plain; charset=UTF-8 Date: Wed, 15 Jul 2026 14:46:49 +0200 Message-Id: From: "Thomas Ellmenreich" To: "Shan Shaji" , Subject: Re: [PATCH proxmox-backup 5/6] fix #7024: cli: add more information inside the backup progress logs Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Mailer: aerc 0.20.0 References: <20260713081529.62859-1-s.shaji@proxmox.com> <20260713081529.62859-6-s.shaji@proxmox.com> In-Reply-To: <20260713081529.62859-6-s.shaji@proxmox.com> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1784119591682 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.317 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_LOW -0.7 Sender listed at https://www.dnswl.org/, low trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: Z6U2FQNYLUQESR3D263NWPZERWYWUZYD X-Message-ID-Hash: Z6U2FQNYLUQESR3D263NWPZERWYWUZYD X-MailFrom: t.ellmenreich@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: I tested the backup client in both `img` as well as `pxar` mode and the out= put works well. The only thing I noticed is that the progress logs during the backup process are now specific to the image archive type. If I have understood correctly this is because the callback is only created for the image archive type. Not sure if that was intentional, or I may have missed something, but if no= t, it could be useful to note this in the commit message. This could potentially also be something that is only active if verbose logging is enabled. As for the tests, I did the following, always using a older client, the new client and the new client with `verbose` active. - Image archive backup of a test node - File archive backup of the node's /etc directory - File archive backup of 25GB of random data - Image archive backup of the same node with the additional 25GB I still have all of the logs in case you want to take a look at them. On Mon Jul 13, 2026 at 10:15 AM CEST, Shan Shaji wrote: > previously, the progress logs only reported the processed stream size > and uploaded size. To improve visibility additional information from > `PxarArchiverProgressStats` and `UploadCounters` has been included in > the logs. > > To access the upload statistics, a new callback field has been added to > the UploadOptions struct that accepts UploadCounters and TimeSpan as > parameters. Additionally, to include pxar metadata in the logs, the > `PxarArchiverProgressStats` instance is now created earlier and shared > between the archiver and the progress logger callback. [snip] > diff --git a/pbs-client/src/backup_writer.rs b/pbs-client/src/backup_writ= er.rs > index 34d8db90b..8ae0cffe8 100644 > --- a/pbs-client/src/backup_writer.rs > +++ b/pbs-client/src/backup_writer.rs > @@ -874,20 +877,13 @@ impl BackupWriter { > let (upload_queue, upload_result) =3D > Self::append_chunk_queue(h2.clone(), wid, append_chunk_path,= counters.clone()); > =20 > - let progress_handle =3D if archive.ends_with(".img.fidx") > - || archive.ends_with(".pxar.didx") > - || archive.ends_with(".ppxar.didx") > - { > + let progress_handle =3D if let Some(callback) =3D progress_loggi= ng_callback { > let counters =3D counters.clone(); > Some(tokio::spawn(async move { > loop { > tokio::time::sleep(tokio::time::Duration::from_secs(= 60)).await; > - > - let size =3D HumanByte::from(counters.total_stream_l= en()); > - let size_uploaded =3D HumanByte::from(counters.uploa= ded_len()); > let elapsed =3D TimeSpan::from(start_time.elapsed())= ; > - > - info!("processed {size} in {elapsed}, uploaded {size= _uploaded}"); The removed logging that logged for all archive types > + callback(&counters, &elapsed); > } > })) > } else { [snip] > diff --git a/proxmox-backup-client/src/main.rs b/proxmox-backup-client/sr= c/main.rs > index 5c52b7917..8ce5540d5 100644 > --- a/proxmox-backup-client/src/main.rs > +++ b/proxmox-backup-client/src/main.rs > @@ -1268,6 +1307,12 @@ async fn create_backup( > (BackupSpecificationType::IMAGE, false) =3D> { > log_file("image", &filename, target.as_ref()); > =20 Current callback (below) is only active for the Image type (as seen above). > + let progress_logger =3D |counters: &UploadCounters, elap= sed: &TimeSpan| { > + let size =3D HumanByte::from(counters.total_stream_l= en()); > + let size_uploaded =3D HumanByte::from(counters.uploa= ded_len()); > + log::info!("processed {size} in {elapsed}, uploaded = {size_uploaded}"); > + }; > + > // 0 means fifo pipe with unknown size > let image_file_size =3D (size !=3D 0).then_some(size); > let upload_options =3D UploadOptions { [snip]