superseded: [PATCH proxmox-backup v3 0/3] fix #7400: improve handling of corrupted job statefiles

["Michael Köppl" <m.koeppl@proxmox.com>]

Superseded by https://lore.proxmox.com/pbs-devel/20260403132628.210128-1-m.koeppl@proxmox.com

On Wed Mar 25, 2026 at 5:06 PM CET, Michael Köppl wrote:
> This patch series fixes a problem where an empty or corrupted job state
> file (due to I/O error, abrupt shutdown, ...) would cause API endpoints
> for listing jobs to return an error, breaking the web UI for users
> because they could not view any of their configured jobs of that type.
> It would also cause proxmox-backup-proxy to indefinitely skip the jobs
> until a user manually triggered it to rewrite the statefile.
>
> 1/3 is a preparatory patch that centralizes job statefile loading
> in compute_schedule_status instead of having every handler function
> open the statefile, handle potential errors and then passing the
> JobState to compute_schedule_status.
>
> 2/3 introduces a new JobState `Unknown`, representing cases in which the
> job state could not be determined. In addition, the patch also updates
> the scheduling functions such that errors during reading the statefiles
> will result in the Unknown state.
>
> 3/3 then utilizes this Unknown state and adapts the scheduling functions
> such that the Unknown state will then lead to the statefile being
> overwritten with a new Created state and the job running again at its
> next scheduled run.
>
> changes since v2:
> - introduced the Unknown state in 2/3, adapted 3/3 accordingly (thanks,
>   @Fabian and @Christian)
> - make sure the "could not open statefile" error is also printed in
>   garbage_collection_status if status_in_memory.upid is None (thanks,
>   @Christian)
> - inline jobtype and err variables in error logging
>
> changes since v1:
> - added preparatory patch 1/3, centralizing the statefile loading before
>   adapting the handling of the error case in that centralized place
>   (compute_schedule_status). Thanks, Christian for the suggestion!
> - adapted the error message if job statefile loading fails to make clear
>   that the default status will be returned as a fallback
>
> proxmox-backup:
>
> Michael Köppl (3):
>   api: move statefile loading into compute_schedule_status
>   fix #7400: api: gracefully handle corrupted job statefiles
>   fix #7400: proxy: self-heal corrupted job statefiles
>
>  src/api2/admin/datastore.rs     | 15 ++++------
>  src/api2/admin/prune.rs         |  9 ++----
>  src/api2/admin/sync.rs          |  9 ++----
>  src/api2/admin/verify.rs        |  9 ++----
>  src/api2/tape/backup.rs         |  9 ++----
>  src/bin/proxmox-backup-proxy.rs |  4 ++-
>  src/server/jobstate.rs          | 52 +++++++++++++++++++++++++++++----
>  7 files changed, 67 insertions(+), 40 deletions(-)
>
>
> Summary over all repositories:
>   7 files changed, 67 insertions(+), 40 deletions(-)