From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 6A8A71FF09F for ; Thu, 03 Sep 2026 14:57:25 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 38705215AF; Thu, 03 Sep 2026 14:57:25 +0200 (CEST) Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 Date: Thu, 03 Sep 2026 14:57:20 +0200 Message-Id: Subject: Re: [PATCH proxmox-backup v2 1/3] server: pull: run blocking file operations on the blocking pool From: "Jakob Klocker" To: "Christian Ebner" , X-Mailer: aerc 0.20.0 References: <20260821111826.299588-1-j.klocker@proxmox.com> <20260821111826.299588-2-j.klocker@proxmox.com> <3d2c267b-e018-4b8e-83c4-665e57ece282@proxmox.com> In-Reply-To: <3d2c267b-e018-4b8e-83c4-665e57ece282@proxmox.com> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1788440237831 X-SPAM-LEVEL: Spam detection results: 0 AWL 1.632 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) KAM_SHORT 0.001 Use of a URL Shortener for very short URL RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: BD5OABZB57U3RJE5NK3H67E4AXVRXBSQ X-Message-ID-Hash: BD5OABZB57U3RJE5NK3H67E4AXVRXBSQ X-MailFrom: j.klocker@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Fri Aug 28, 2026 at 12:30 PM CEST, Christian Ebner wrote: > One comment inline, with that addressed consider: > > Reviewed-by: Christian Ebner > > On 8/21/26 1:18 PM, Jakob Klocker wrote: >> [SNIP] >> + if let Err(err) =3D tokio::fs::rename(&source_path, &path).await { > > comment: there is an additional std::fs::remove_file() in the pull code,= =20 > e.g. removing the temp file just above this (not seen in diff). All of=20 > these should be moved to tokio::fs::remove_file(). > > We need to be careful with performance when using tokio::fs over std::fs= =20 > as stated in [0], but these should be fine to call on the blocking task= =20 > pool. That this might have performance implications might be also noted= =20 > in the commit message. > > [0] https://docs.rs/tokio/latest/tokio/fs/index.html > I'll make sure to replace all existing std::fs::remove_file() with tokio's version and to adapt the commit message in the next revision. Since I'm already at it, I'll do so for all occasions in the pull.rs=20 file -- std::fs::remove_file is also called twice in pull_single_archive(). >> bail!("{archive_prefix}: Atomic rename file {path:?} failed - = {err}"); >> } >> =20 >> @@ -890,7 +890,7 @@ async fn pull_snapshot<'a>( >> nix::unistd::fsync(tmp_manifest_file.as_raw_fd())?; >> } >> =20 >> - if let Err(err) =3D std::fs::rename(&tmp_manifest_name, &manifest_n= ame) { >> + if let Err(err) =3D tokio::fs::rename(&tmp_manifest_name, &manifest= _name).await { >> bail!("{prefix}: Atomic rename file {manifest_name:?} failed -= {err}"); >> } >> if let DatastoreBackend::S3(s3_client) =3D backend { >> @@ -910,8 +910,9 @@ async fn pull_snapshot<'a>( >> =20 >> fetch_log(crypt_config).await?; >> =20 >> - snapshot >> - .cleanup_unreferenced_files(&manifest) >> + let snapshot =3D snapshot.clone(); >> + tokio::task::spawn_blocking(move || snapshot.cleanup_unreferenced_f= iles(&manifest)) >> + .await? >> .map_err(|err| format_err!("{prefix}: failed to cleanup unrefe= renced files - {err}"))?; >> =20 >> Ok(Some(sync_stats))