all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Jakob Klocker <j.klocker@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH pve-docs v3 4/4] vzdump: document backup retry options
Date: Mon, 16 Nov 2026 11:13:44 +0100	[thread overview]
Message-ID: <20261116101344.140523-5-j.klocker@proxmox.com> (raw)
In-Reply-To: <20261116101344.140523-1-j.klocker@proxmox.com>

Describe the new `error-behavior` and `max-retries` options, the
exponential backoff between attempts, and which failures are retried.

Signed-off-by: Jakob Klocker <j.klocker@proxmox.com>
---
 vzdump.adoc | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 65 insertions(+)

diff --git a/vzdump.adoc b/vzdump.adoc
index 44db027..856d3d3 100644
--- a/vzdump.adoc
+++ b/vzdump.adoc
@@ -308,6 +308,71 @@ the `performance` setting, `max-workers` (affects VM backups only) and
 `pbs-entries-max` (affects container backups only). See the
 xref:vzdump_configuration[configuration options] for details.
 
+[[vzdump_retry]]
+Backup Retries
+--------------
+
+The backup of a single guest can fail because of a temporary problem, for
+example when the guest is locked by another operation, or when the connection
+to the backup server is interrupted during the backup. To avoid a failed backup
+in such cases, the `error-behavior` option allows a job to retry failed guest
+backups automatically:
+
+`ignore` ::
+Does not retry failed guests. This is the default.
+
+`retry` ::
+Defers failed guests until all other guests have been processed and then
+retries them in the order in which they failed.
+
+`retry-immediately` ::
+Retries a failed guest right away, before continuing with the next guest.
+
+The `max-retries` option (1 to 10, default 5) limits how often a guest is
+retried. Between attempts, the job uses exponential backoff: the delay starts
+at 30 seconds, doubles with every retry and is capped at 30 minutes. With
+`retry`, the delay is counted from the last failed attempt of that guest, so
+other guests can be retried in the meantime.
+
+If a retry succeeds, the guest does not count as an error, but the task
+finishes with a warning and the notification is sent with severity `warning`
+and shows how many retries were needed. This way, recurring problems stay
+visible even if all backups eventually succeed. If the last retry fails as
+well, the job finishes with errors. The task log of the guest contains all
+attempts.
+
+For example, to back up three guests and retry each failed guest at most three
+times at the end of the job, run:
+
+ # vzdump 101 102 103 --error-behavior retry --max-retries 3
+
+Both options can also be set as node-wide defaults in `/etc/vzdump.conf`, see
+xref:vzdump_configuration[configuration].
+
+In the web interface, the retry strategy for backup jobs can be set in the
+'Advanced' tab of the backup job window. New backup jobs preselect `retry`,
+existing jobs keep their configuration. For a manual backup, the retry strategy
+can be chosen directly in the backup window ('Backup now'), which preselects the
+node's default from `/etc/vzdump.conf`. It only offers `retry` and `ignore`,
+as both retry strategies behave the same for a single guest.
+
+Only failures that happen before the backup archive of a guest has been created
+are retried. If the archive was created, but a later step failed, for example a
+failing `backup-end` hook script, the guest is not retried. This avoids
+creating multiple backups of the same guest and pruning older backups
+repeatedly. Errors that prevent the whole job from starting, for example a
+backup storage that cannot be activated, are not retried either.
+
+Retries are not possible when writing the backup to standard output with
+`--stdout`, as a retry would append a second archive to the data that was
+already written. Setting `error-behavior` to a retry strategy together with
+`--stdout` is rejected, and a retry strategy from `/etc/vzdump.conf` is
+ignored with a warning.
+
+NOTE: Retries can make a backup job take considerably longer. With the default
+of 5 retries, the delays alone add up to more than 15 minutes for a guest that
+keeps failing, and about 2.5 hours with the maximum of 10 retries.
+
 [[vzdump_retention]]
 Backup Retention
 ----------------
-- 
2.47.3





      parent reply	other threads:[~2026-11-16 10:14 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16 10:13 [PATCH docs/guest-common/manager v3 0/4] fix #6464: backup: add backup retry strategies Jakob Klocker
2026-11-16 10:13 ` [PATCH pve-guest-common v3 1/4] vzdump: add error-behavior and max-retries options Jakob Klocker
2026-11-16 10:13 ` [PATCH pve-manager v3 2/4] fix #6464: vzdump: add retry strategies for failed guest backups Jakob Klocker
2026-11-16 10:13 ` [PATCH pve-manager v3 3/4] ui: backup: add error behavior and retry options Jakob Klocker
2026-11-16 10:13 ` Jakob Klocker [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261116101344.140523-5-j.klocker@proxmox.com \
    --to=j.klocker@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal