From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 8F6221FF16C for ; Tue, 17 Sep 2024 13:40:54 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B1EC615228; Tue, 17 Sep 2024 13:41:01 +0200 (CEST) Message-ID: <4611dc8b-ed62-47bd-a4b5-11e76459b299@proxmox.com> Date: Tue, 17 Sep 2024 13:40:57 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird From: Christian Ebner To: Dietmar Maurer , Proxmox Backup Server development discussion References: <20240913135944.317298-1-c.ebner@proxmox.com> <1902796862.144.1726241034647@webmail.proxmox.com> <27cfdbaa-eb6d-4585-a339-5640cf2ac323@proxmox.com> Content-Language: en-US, de-DE In-Reply-To: <27cfdbaa-eb6d-4585-a339-5640cf2ac323@proxmox.com> X-SPAM-LEVEL: Spam detection results: 0 AWL 0.024 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] [RFC proxmox-backup] fix #5710: api: backup: stat known chunks during backup X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" On 9/17/24 09:19, Christian Ebner wrote: > On 9/13/24 17:23, Dietmar Maurer wrote: >> >> >> Is it possible to only stat actually reused chunks at the end? > > Yes, as is all the chunks, even not reused but indexed by the previous > backup snapshot are stat'ed. > > I can add an additional flag to the hashmap's value storing known chunks > to differentiate and only stat these. > > Will send a new version of the patch. Unfortunately, the accounting of known chunks for fixed indices when using the incremental mode turns out to be harder than expected. For this case, the index is initially copied over from the previous snapshot and chunk digests inserted into the known chunks hashmap. On client side, only changed chunks are uploaded via the backup api, and inserted in the fixed index on the corresponding index position. The information which previously indexed chunk it replaced is however lacking. Because of this, I currently tend to rather not modify the current behavior and stat all of the known chunks. Any thoughts on this? Is there a better approach I currently am failing to see? _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel