From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id B6B11BA6AB for ; Wed, 20 Mar 2024 09:38:07 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 87B2BB7AF for ; Wed, 20 Mar 2024 09:37:37 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 20 Mar 2024 09:37:36 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 1C85D48AEF for ; Wed, 20 Mar 2024 09:37:36 +0100 (CET) Message-ID: <469dcf38-c1ae-44f9-ab43-b1e02a2f9c9f@proxmox.com> Date: Wed, 20 Mar 2024 09:37:34 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: =?UTF-8?Q?Fabian_Gr=C3=BCnbichler?= , Proxmox Backup Server development discussion References: <20240305092703.126906-1-c.ebner@proxmox.com> <20240305092703.126906-28-c.ebner@proxmox.com> <1710237573.g7a8c3ms4l.astroid@yuna.none> <582353512.13251.1710852569100@webmail.proxmox.com> Content-Language: en-US, de-DE From: Christian Ebner In-Reply-To: <582353512.13251.1710852569100@webmail.proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.037 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: Re: [pbs-devel] [RFC v2 proxmox-backup 27/36] client: implement prepare reference method X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Mar 2024 08:38:07 -0000 On 3/19/24 13:49, Fabian Grünbichler wrote: > > > you misunderstood what I meant. instead of attempting to download the previous index, we could first check whether it even exists. if it doesn't exist, than we can skip the download, just log an info value, and continue without a reference. if it does (should) exist, and we attempt to download it, but the download fails - then we at least want to print a different message, or potentially fail the backup. > > e.g., this is how the regular incremental backup part of the code handles this: > > Previous manifest does not contain an archive called 'mail2.pxar.meta.didx', skipping download.. > > we used to always try to download and always print an error there if that download failed, which was confusing for host backups when the user added an archive or renamed one (scary message for no reason). there the only downside when a previous index cannot be downloaded is that the chunks have to be re-uploaded, there is no difference in the resulting archives. > > in the new code here we have a different variant - we always try to download (unnecessary!), but always treat a failure as benign (not sure about that part - it actually makes a difference content-wise, but also possibly speed/load wise ;)). > > also we now download the meta index twice I guess (if it exists). > > last, but not least - there is no actual "fallback to regular encoding" (or at least, I think this would confuse users, for whom "regular encoding" might mean v1 archive, not "v2 archive but no re-use via metadata caching"). Okay, thanks for the detailed clarification. I added an additional `manifest.lookup_file_info(..)` call to differentiate the cases as suggested. Also rephrased the info logging with respect to regular encoding.