all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Jakob Klocker <j.klocker@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH docs/guest-common/manager v3 0/4] fix #6464: backup: add backup retry strategies
Date: Mon, 16 Nov 2026 11:13:40 +0100	[thread overview]
Message-ID: <20261116101344.140523-1-j.klocker@proxmox.com> (raw)

This series introduces automatic backup retries for guest backups in
vzdump to better handle transient failures.

As suggested by @Thomas [0], three strategies were added:
- `retry`: Defers failed guests until all other guests have been
  processed, then retries them in the order they failed up to a
  configurable maximum retry count.
- `retry-immediately`: Immediately retries failed guests up to a
  configurable maximum retry count.
- `ignore`: Retains the existing behavior by ignoring failures.

The number of retries is set with the new `max-retries` option (1-10,
default 5). Between attempts, a backoff delay is applied that starts at
30 seconds, doubles with each retry and is capped at 30 minutes. For
`retry`, the delay is measured from the last failure of that guest. The
interval and cap are intentionally not configurable for now.

If a guest succeeds on a retry, no error is counted, but the task
finishes with a warning and the notification is sent with severity
`warning`. If it still fails after the last retry, the job finishes
with errors as before. The task log of a guest contains all attempts
of the current job.

Testing info:
When testing I'd suggest to fail the backup by locking and
unlocking the guest with following commands:
`qm set <VMID> -lock backup`
`qm unlock <VMID>`

Changes since v2 (thanks @Jonas)
- don't retry guests whose backup was already created (e.g. failing
  backup-end hook)
- with --stdout, reject an explicitly set retry strategy and ignore
  one inherited from vzdump.conf with a warning
- don't append the task log to a stale log of a previous job
- finish the task with warnings and notify with severity 'warning'
  for retried guests
- show the number of retries in the notification subject and guest
  table
- UI: only preselect `retry` for new jobs
- UI: don't store max-retries unless set, show schema defaults in
  description
- UI: use node defaults from vzdump.conf in the manual backup window
- docs: document non-retried failures, --stdout restriction, warnings
  and UI defaults

Changes since v1 (thanks @Shan):
- add documentation for the new retry options to pve-docs
- fix typo in pve-manager commit message
- drop the `advanced` flag from the retry fields in the manual backup
  window, which has no advanced section

[0] https://bugzilla.proxmox.com/show_bug.cgi?id=6464#c5
Link: https://bugzilla.proxmox.com/show_bug.cgi?id=6464


pve-guest-common:

Jakob Klocker (1):
  vzdump: add error-behavior and max-retries options

 src/PVE/VZDump/Common.pm | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)


pve-manager:

Jakob Klocker (2):
  fix #6464: vzdump: add retry strategies for failed guest backups
  ui: backup: add error behavior and retry options

 PVE/VZDump.pm                               | 219 +++++++++++++++++++-
 www/manager6/dc/Backup.js                   |   2 +-
 www/manager6/panel/BackupAdvancedOptions.js |  52 +++++
 www/manager6/window/Backup.js               |  56 ++++-
 4 files changed, 323 insertions(+), 6 deletions(-)


pve-docs:

Jakob Klocker (1):
  vzdump: document backup retry options

 vzdump.adoc | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 65 insertions(+)


Summary over all repositories:
  6 files changed, 406 insertions(+), 6 deletions(-)

-- 
Generated by murpp 0.12.0




             reply	other threads:[~2026-11-16 10:14 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16 10:13 Jakob Klocker [this message]
2026-11-16 10:13 ` [PATCH pve-guest-common v3 1/4] vzdump: add error-behavior and max-retries options Jakob Klocker
2026-11-16 10:13 ` [PATCH pve-manager v3 2/4] fix #6464: vzdump: add retry strategies for failed guest backups Jakob Klocker
2026-11-16 10:13 ` [PATCH pve-manager v3 3/4] ui: backup: add error behavior and retry options Jakob Klocker
2026-11-16 10:13 ` [PATCH pve-docs v3 4/4] vzdump: document backup " Jakob Klocker

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261116101344.140523-1-j.klocker@proxmox.com \
    --to=j.klocker@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal