Re: [pbs-devel] [PATCH proxmox-backup v5 0/4] datastore: remove config reload on hot path

[Samuel Rufinatscha <s.rufinatscha@proxmox.com>]

On 11/26/25 4:16 PM, Fabian Grünbichler wrote:
> On November 24, 2025 6:04 pm, Samuel Rufinatscha wrote:
>> Hi,
>>
>> this series reduces CPU time in datastore lookups by avoiding repeated
>> datastore.cfg reads/parses in both `lookup_datastore()` and
>> `DataStore::Drop`. It also adds a TTL so manual config edits are
>> noticed without reintroducing hashing on every request.
>>
>> While investigating #6049 [1], cargo-flamegraph [2] showed hotspots
>> during repeated `/status` calls in `lookup_datastore()` and in `Drop`,
>> dominated by `pbs_config::datastore::config()` (config parse).
>>
>> The parsing cost itself should eventually be investigated in a future
>> effort. Furthermore, cargo-flamegraph showed that when using a
>> token-based auth method to access the API, a significant amount of time
>> is spent in validation on every request request [3].
>>
>> ## Approach
>>
>> [PATCH 1/4] Support datastore generation in ConfigVersionCache
>>
>> [PATCH 2/4] Fast path for datastore lookups
>>    Cache the parsed datastore.cfg keyed by the shared datastore
>>    generation. lookup_datastore() reuses both the cached config and an
>>    existing DataStoreImpl when the generation matches, and falls back
>>    to the old slow path otherwise. The caching logic is implemented
>>    using the datastore_section_config_cached(update_cache: bool) helper.
>>
>> [PATCH 3/4] Fast path for Drop
>>    Make DataStore::Drop use the datastore_section_config_cached()
>>    helper to avoid re-reading/parsing datastore.cfg on every Drop.
>>    Bump generation not only on API config saves, but also on slow-path
>>    lookups (if update_cache is true), to enable Drop handlers see
>>    eventual newer configs.
>>
>> [PATCH 4/4] TTL to catch manual edits
>>    Add a TTL to the cached config and bump the datastore generation iff
>>    the digest changed but generation stays the same. This catches manual
>>    edits to datastore.cfg without reintroducing hashing or config
>>    parsing on every request.
> 
> semantics wise this looks mostly good to me now, sent a few style
> remarks for the individual patches. let's wait for feedback from the
> reporter, and then wrap this up hopefully :)

Thanks for the great review Fabian! I agree, will make sure to integrate 
the style remarks and avoid more of the diff noise :)

Thanks!

> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
> 



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel