From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id D6D771FF0A8 for ; Sat, 03 Oct 2026 17:38:30 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id CD9552167D; Sat, 03 Oct 2026 17:38:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dualfroz.com; s=dkim; t=1791041902; h=from:subject:date:message-id:to:mime-version: content-transfer-encoding; bh=+G8X5AePcUcFLC2YJWyPJpbxZxLEM2A7D7XeVGe5nUU=; b=bUWT5PPI6ZEX7qGcOI1WoEOHrTcO5+2dlbck9ZDBLO9KUsT4erhSikNvL4on5bpbqoYgSC GJQx3h2Fgqc1JRPy/CXSO153qymQosREiErm0b5CgVXTrw/eCjntcsddmmi16GY3MjSs6R kB0ZPAxHflzkcxLTOHxd+j3pcrSJ0IpE0ekK5dIVrTu1JEvvOnSRUBzeagAo3HesHUb1lI aJyLFdoWfV+X5qIVUtvYtj4LaUet0cQySH9MdELoJvwMJn3BYYIMUtKgfBZYbHK0657tPG YGZaE43FLG74fiVmeYmpVXuackg+B5WPAEiqlq4XIM3QZPM+ui2DVhK5qp0kSg== From: Michal Fox To: pve-devel@lists.proxmox.com Subject: [PATCH manager] fix #4986: jobs: do not get stuck if the task log is missing Date: Sat, 3 Oct 2026 15:38:20 +0000 Message-ID: <20261003153820.7-1-me@dualfroz.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Last-TLS-Session-Version: TLSv1.3 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.241 Adjusted score from AWL reputation of From: address DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: DPVSYQXOKAHOQQP4OCNI3VM26PF5FO6X X-Message-ID-Hash: DPVSYQXOKAHOQQP4OCNI3VM26PF5FO6X X-MailFrom: me@dualfroz.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: When the scheduler checks whether the last run of a job has finished, it dies if the task log of that run does not exist. The job is then skipped with 'could not update job state, skipping - no such task' on every iteration, so it never runs again until its state file is cleaned up manually. The task log can be gone, for example after a crash. If the task is not running anymore, treat it as stopped with an error status instead, so that the job gets scheduled again, like Proxmox Backup Server does since commit 48fbce07 ("fix #4895: scheduled jobs: ignore task-log not found error"). The check now comes after the one whether the task is still running, so a running task does not get marked as stopped. Signed-off-by: Michal Fox --- Tested with a script that runs run_jobs() with a mocked jobs.cfg on a job state whose task log does not exist: before, the job was skipped with 'no such task' on every run, now it is marked as stopped and started again, while a job whose task is still running stays started. PVE/Jobs.pm | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/PVE/Jobs.pm b/PVE/Jobs.pm index 40caf257..75f87060 100644 --- a/PVE/Jobs.pm +++ b/PVE/Jobs.pm @@ -109,13 +109,15 @@ my $get_job_task_status = sub { my ($task, $filename) = PVE::Tools::upid_decode($state->{upid}, 1); die "unable to parse worker upid - $state->{upid}\n" if !$task; - die "no such task\n" if !-f $filename; my $pstart = PVE::ProcFSTools::read_proc_starttime($task->{pid}); if ($pstart && $pstart == $task->{pstart}) { return; # still running } + # the task log might be gone, e.g. after a crash, do not let that block the job forever + return "task log not found" if !-f $filename; + return PVE::Tools::upid_read_status($state->{upid}); }; -- 2.43.0