Re: [pbs-devel] [PATCH proxmox-backup v5 0/4] datastore: remove config reload on hot path

["Fabian Grünbichler" <f.gruenbichler@proxmox.com>]

On November 24, 2025 6:04 pm, Samuel Rufinatscha wrote:
> Hi,
> 
> this series reduces CPU time in datastore lookups by avoiding repeated
> datastore.cfg reads/parses in both `lookup_datastore()` and
> `DataStore::Drop`. It also adds a TTL so manual config edits are
> noticed without reintroducing hashing on every request.
> 
> While investigating #6049 [1], cargo-flamegraph [2] showed hotspots
> during repeated `/status` calls in `lookup_datastore()` and in `Drop`,
> dominated by `pbs_config::datastore::config()` (config parse).
> 
> The parsing cost itself should eventually be investigated in a future
> effort. Furthermore, cargo-flamegraph showed that when using a
> token-based auth method to access the API, a significant amount of time
> is spent in validation on every request request [3].
> 
> ## Approach
> 
> [PATCH 1/4] Support datastore generation in ConfigVersionCache
> 
> [PATCH 2/4] Fast path for datastore lookups
>   Cache the parsed datastore.cfg keyed by the shared datastore
>   generation. lookup_datastore() reuses both the cached config and an
>   existing DataStoreImpl when the generation matches, and falls back
>   to the old slow path otherwise. The caching logic is implemented
>   using the datastore_section_config_cached(update_cache: bool) helper.
> 
> [PATCH 3/4] Fast path for Drop
>   Make DataStore::Drop use the datastore_section_config_cached()
>   helper to avoid re-reading/parsing datastore.cfg on every Drop.
>   Bump generation not only on API config saves, but also on slow-path
>   lookups (if update_cache is true), to enable Drop handlers see
>   eventual newer configs.
> 
> [PATCH 4/4] TTL to catch manual edits
>   Add a TTL to the cached config and bump the datastore generation iff
>   the digest changed but generation stays the same. This catches manual
>   edits to datastore.cfg without reintroducing hashing or config
>   parsing on every request.

semantics wise this looks mostly good to me now, sent a few style
remarks for the individual patches. let's wait for feedback from the
reporter, and then wrap this up hopefully :)


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel