public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: "Jakob Klocker" <j.klocker@proxmox.com>
To: "Jonas Theisen" <j.theisen@proxmox.com>, <pve-devel@lists.proxmox.com>
Subject: Re: [PATCH docs/guest-common/manager v2 0/4] fix #6464: backup: add backup retry strategies
Date: Thu, 12 Nov 2026 11:11:04 +0100	[thread overview]
Message-ID: <DMT5GE873OOT.WHKLXTSJ7TWJ@proxmox.com> (raw)
In-Reply-To: <8d597954-b22c-41f2-a1ba-5e95a05cea98@proxmox.com>

On Wed Nov 11, 2026 at 4:40 PM CET, Jonas Theisen wrote:
> On 11/11/26 10:35, Jakob Klocker wrote:
>> <snip>
>>
>> If a guest succeeds on a retry, it is reported as successful in the
>> notification and no error is counted. If it still fails after the last
>> retry, the job finishes with errors as before. Task logging has been
>> switched to append mode so the log contains the complete transcript of
>> all attempts.
>
> I personally would prefer if a transient failure would still be
> reported as a warning. Retries are good to not miss the backup
> completely but it still in most cases means something is wrong.

A warning is already logged in the task log, but I'll also include one 
in the notification and the task status, so it's more obvious that
something went wrong.

>> I kept the retry options in the advanced tab of the backup job window,
>> but I'm not sure whether they should be moved to the general tab.
>> Opinions on this are appreciated.
>
> A more general issue is the combination of retries with the --stdout
> parameter, where the tar gets written to a stdout instead of a file.
> The resulting stream gets corrupted by a retry so probably the CLI
> should just bail in that combination.

Thanks for that find, I'll make sure to fix this in the next version.

>>
>> Testing info:
>> When testing I'd suggest to fail the backup by locking and
>> unlocking the guest with following commands:
>> `qm set <VMID> -lock backup`
>> `qm unlock <VMID>`
>
> I also tested this in combination with an unreachable PBS by just
> yanking out the connection on the PBS side.
> This caused the backup task to fail even before the retries since
> it could not activate the storage.
> Since i assume this is one of the more common failure modes
> i think that should be covered by a retry.

As discussed off list, I'd keep the current behavior for this series.
The retry is intended for transient failures that happen while a
backup is running, where a single hiccup would otherwise fail a guest
that was already in progress. The documentation added by this series
also states that errors preventing the job from starting are not
retried.

Storage activation happens once per job, before any guest is backed
up, so retrying there would be a separate mechanism. With
both in place, the delays would stack, making the worstcase runtime
hard to predict. 


Appreciate the feedback!
>> <snip>





  reply	other threads:[~2026-11-12 10:11 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-11  9:34 [PATCH docs/guest-common/manager v2 0/4] fix #6464: backup: add backup retry strategies Jakob Klocker
2026-11-11  9:34 ` [PATCH pve-guest-common v2 1/4] vzdump: add error-behavior and max-retries options Jakob Klocker
2026-11-11  9:34 ` [PATCH pve-manager v2 2/4] fix #6464: vzdump: add retry strategies for failed guest backups Jakob Klocker
2026-11-11 15:40   ` Jonas Theisen
2026-11-11  9:34 ` [PATCH pve-manager v2 3/4] ui: backup: add error behavior and retry options Jakob Klocker
2026-11-11 15:40   ` Jonas Theisen
2026-11-11  9:34 ` [PATCH pve-docs v2 4/4] vzdump: document backup " Jakob Klocker
2026-11-11 15:40 ` [PATCH docs/guest-common/manager v2 0/4] fix #6464: backup: add backup retry strategies Jonas Theisen
2026-11-12 10:11   ` Jakob Klocker [this message]
2026-11-16 10:16 ` superseded: " Jakob Klocker

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DMT5GE873OOT.WHKLXTSJ7TWJ@proxmox.com \
    --to=j.klocker@proxmox.com \
    --cc=j.theisen@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal