From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 664B81FF0AB for ; Wed, 23 Sep 2026 13:44:12 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 0605C2147C; Wed, 23 Sep 2026 13:44:12 +0200 (CEST) From: =?UTF-8?q?Fabian=20Gr=C3=BCnbichler?= To: pbs-devel@lists.proxmox.com Subject: [PATCH proxmox] fix #6267: rest-server: fix task log rotation race Date: Wed, 23 Sep 2026 13:43:12 +0200 Message-ID: <20260923114403.856979-1-f.gruenbichler@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1790163844249 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.784 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 6SJANPZ56IRSMMTRTAU7K5KEHEVVXIK6 X-Message-ID-Hash: 6SJANPZ56IRSMMTRTAU7K5KEHEVVXIK6 X-MailFrom: f.gruenbichler@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: if the task log archive files get rotated, the base `archive` file is renamed to `archive.1`. until it is recreated, iterating over the archive files returns no files. on systems with a lot of task activity, task rotation could be effectively broken: - every logrotate task rotated the task archive files - the subsequent handling of the archive files to find out which task logs to clean found no task archive files at all - only rotated copies existed there's a small race window in rotation task: - archives are rotated (and too old archives removed) - old task logs are cleaned up if the base archive file got recreated inbetween (e.g., by an unrelated task finishing), then log rotation would clean up old task logs despite the bug. Signed-off-by: Fabian Grünbichler --- Notes: confirmed the problematic sequence by adding a sleep at the end of the logrotate task in PBS, with an otherwise quiet system the archive file gets rotated but not recreated until the tasks exits, and not ask log cleanup happens. with this patch in place, old task logs get cleaned up as expected. proxmox-rest-server/src/worker_task.rs | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/proxmox-rest-server/src/worker_task.rs b/proxmox-rest-server/src/worker_task.rs index 63f0c14c..a901985b 100644 --- a/proxmox-rest-server/src/worker_task.rs +++ b/proxmox-rest-server/src/worker_task.rs @@ -303,6 +303,21 @@ pub fn rotate_task_log_archive( let mut rotated = logrotate.rotate(size_threshold)?; + // need to recreate base file after rotation + if rotated { + let options = setup + .file_opts + .perm(nix::sys::stat::Mode::from_bits_truncate(0o660)); + + atomic_open_or_create_file( + &setup.task_archive_fn, + OFlag::O_APPEND | OFlag::O_RDWR, + &[], + options, + false, + )?; + } + if let Some(max_days) = max_days { // NOTE: not on exact day-boundary but close enough for what's done here let cutoff_time = proxmox_time::epoch_i64() - (max_days * 24 * 60 * 60) as i64; -- 2.47.3