From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 984831FF179 for ; Wed, 26 Nov 2025 16:16:25 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B8023FE9B; Wed, 26 Nov 2025 16:16:38 +0100 (CET) Date: Wed, 26 Nov 2025 16:16:02 +0100 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= To: Proxmox Backup Server development discussion References: <20251124170423.303300-1-s.rufinatscha@proxmox.com> In-Reply-To: <20251124170423.303300-1-s.rufinatscha@proxmox.com> MIME-Version: 1.0 User-Agent: astroid/0.17.0 (https://github.com/astroidmail/astroid) Message-Id: <1764169982.tq606zn1dk.astroid@yuna.none> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1764170129252 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.046 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] [PATCH proxmox-backup v5 0/4] datastore: remove config reload on hot path X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" On November 24, 2025 6:04 pm, Samuel Rufinatscha wrote: > Hi, > > this series reduces CPU time in datastore lookups by avoiding repeated > datastore.cfg reads/parses in both `lookup_datastore()` and > `DataStore::Drop`. It also adds a TTL so manual config edits are > noticed without reintroducing hashing on every request. > > While investigating #6049 [1], cargo-flamegraph [2] showed hotspots > during repeated `/status` calls in `lookup_datastore()` and in `Drop`, > dominated by `pbs_config::datastore::config()` (config parse). > > The parsing cost itself should eventually be investigated in a future > effort. Furthermore, cargo-flamegraph showed that when using a > token-based auth method to access the API, a significant amount of time > is spent in validation on every request request [3]. > > ## Approach > > [PATCH 1/4] Support datastore generation in ConfigVersionCache > > [PATCH 2/4] Fast path for datastore lookups > Cache the parsed datastore.cfg keyed by the shared datastore > generation. lookup_datastore() reuses both the cached config and an > existing DataStoreImpl when the generation matches, and falls back > to the old slow path otherwise. The caching logic is implemented > using the datastore_section_config_cached(update_cache: bool) helper. > > [PATCH 3/4] Fast path for Drop > Make DataStore::Drop use the datastore_section_config_cached() > helper to avoid re-reading/parsing datastore.cfg on every Drop. > Bump generation not only on API config saves, but also on slow-path > lookups (if update_cache is true), to enable Drop handlers see > eventual newer configs. > > [PATCH 4/4] TTL to catch manual edits > Add a TTL to the cached config and bump the datastore generation iff > the digest changed but generation stays the same. This catches manual > edits to datastore.cfg without reintroducing hashing or config > parsing on every request. semantics wise this looks mostly good to me now, sent a few style remarks for the individual patches. let's wait for feedback from the reporter, and then wrap this up hopefully :) _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel