From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 132FC1FF0B6 for ; Thu, 12 Nov 2026 11:11:12 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id DC91A21632; Thu, 12 Nov 2026 11:11:08 +0100 (CET) Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 Date: Thu, 12 Nov 2026 11:11:04 +0100 Message-Id: Subject: Re: [PATCH docs/guest-common/manager v2 0/4] fix #6464: backup: add backup retry strategies From: "Jakob Klocker" To: "Jonas Theisen" , X-Mailer: aerc 0.20.0 References: <20261111093407.109872-1-j.klocker@proxmox.com> <8d597954-b22c-41f2-a1ba-5e95a05cea98@proxmox.com> In-Reply-To: <8d597954-b22c-41f2-a1ba-5e95a05cea98@proxmox.com> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1794478264840 X-SPAM-LEVEL: Spam detection results: 0 AWL 1.307 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 65SIBAG3ISYJHNKRXJVQ74SYELAPRSFS X-Message-ID-Hash: 65SIBAG3ISYJHNKRXJVQ74SYELAPRSFS X-MailFrom: j.klocker@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Wed Nov 11, 2026 at 4:40 PM CET, Jonas Theisen wrote: > On 11/11/26 10:35, Jakob Klocker wrote: >> >> >> If a guest succeeds on a retry, it is reported as successful in the >> notification and no error is counted. If it still fails after the last >> retry, the job finishes with errors as before. Task logging has been >> switched to append mode so the log contains the complete transcript of >> all attempts. > > I personally would prefer if a transient failure would still be > reported as a warning. Retries are good to not miss the backup > completely but it still in most cases means something is wrong. A warning is already logged in the task log, but I'll also include one=20 in the notification and the task status, so it's more obvious that something went wrong. >> I kept the retry options in the advanced tab of the backup job window, >> but I'm not sure whether they should be moved to the general tab. >> Opinions on this are appreciated. > > A more general issue is the combination of retries with the --stdout > parameter, where the tar gets written to a stdout instead of a file. > The resulting stream gets corrupted by a retry so probably the CLI > should just bail in that combination. Thanks for that find, I'll make sure to fix this in the next version. >> >> Testing info: >> When testing I'd suggest to fail the backup by locking and >> unlocking the guest with following commands: >> `qm set -lock backup` >> `qm unlock ` > > I also tested this in combination with an unreachable PBS by just > yanking out the connection on the PBS side. > This caused the backup task to fail even before the retries since > it could not activate the storage. > Since i assume this is one of the more common failure modes > i think that should be covered by a retry. As discussed off list, I'd keep the current behavior for this series. The retry is intended for transient failures that happen while a backup is running, where a single hiccup would otherwise fail a guest that was already in progress. The documentation added by this series also states that errors preventing the job from starting are not retried. Storage activation happens once per job, before any guest is backed up, so retrying there would be a separate mechanism. With both in place, the delays would stack, making the worstcase runtime hard to predict.=20 Appreciate the feedback! >>