From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 7DF1E1FF185 for ; Mon, 4 Aug 2025 22:41:03 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 42AE0398E0; Mon, 4 Aug 2025 22:42:33 +0200 (CEST) Message-ID: Date: Mon, 4 Aug 2025 22:42:00 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta To: Proxmox Backup Server development discussion , Christian Ebner References: <20250801141024.626365-1-c.ebner@proxmox.com> <20250801141024.626365-3-c.ebner@proxmox.com> Content-Language: en-US From: Thomas Lamprecht In-Reply-To: <20250801141024.626365-3-c.ebner@proxmox.com> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1754340101496 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.030 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] [PATCH proxmox-backup 2/2] datastore: reinsert unused chunks into cache during instantiation X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" Am 01.08.25 um 16:10 schrieb Christian Ebner: > The local datastore chunk cache stores the currently cached chunk > digests in-memory, the chunk's data is stored however on the > filesystem. The in-memory cache might however be lost when: > - the datastore is removed for the lookup cache when a corresponding > maintenance mode is set. > - the services are restarted. > - the system is rebooted. > > After above actions, the cache is reistantiated again together with > the datastore on the next datastore lookup, calculating a cache > capacity based on the currently available storage space. This however > leaves the previously cached chunks out. > Therefore, reinsert them in an asynchronos task, by iterating over > them an insert the chunk digest again. For these previously used > chunks, increase also the cache size as this is now usable storage > for the cache as well. I really would like some basic numbers for patches doing things with caches, especially if they iterate over all chunks present on disk, IIUC. AFAICIT it at least happens in the background, so doesn't delays the one instantiating the new datastore struct directly, but without some pacing going through all chunks as fast as possible might introduce significant IO pressure I think. Also, what happens if the datastore instance is already dropped again during this cache re-warming? AFAICT that can only realistically happen with maintenance mode, as with restarts/reboots it naturally should not matter. Also, just to be sure, this might also block any shutdown future from resolving, just like any other spawn_blocking, or am I mistaken? _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel