Re: [PATCH v1 proxmox-backup] api: do not block tokio worker threads during chunk inserts

[Christian Ebner <c.ebner@proxmox.com>]

On 5/27/26 2:37 PM, Robert Obkircher wrote:
> Move synchronous operations off the worker threads to prevent blocking
> the I/O and timer drivers of the entire runtime. This is especially
> important for S3 uploads, which wait up to 3 hours for the chunk lock.
> 
> Also prevent worker starvation, which could happen because S3 uploads
> are wrapped in proxmox_async::runtime::block_on, which prevents other
> futures from running in the current thread.
> 
> In the backtrace from the linked forum post, two workers were waiting
> for chunk locks (presumably due to duplicates) while the remaining 19
> were stuck because block_on called std::thread::park.
> 
> Fixes: https://forum.proxmox.com/threads/183705
> Signed-off-by: Robert Obkircher <r.obkircher@proxmox.com>
> ---

Gave this patch a spin, changes look good to me although a second pair 
of eyes is highly appreciated given the scope.

In order to look for performance regressions, I did also some basic 
benchmarks:

Performed backups of a linux kernel git repo with size 11.488 GiB
(compressed 9.96 GiB).
For each patch state and target datastore, performed 5 runs removing the
backup snapshot again after each run to force re-upload of chunks, but
left chunk in store to avoid overhead of insert/upload. In-memory state
cleared after each run by restarting proxmox-backup-proxy.service.
delta RSS was obtained by taking difference of max and min value of
output as reported by:
`watch -n 1 "ps -p $(pidof proxmox-backup-proxy) -o rss | tail -n 1 | 
tee -a ps-rss.out"`

Regular datastore and S3 store backed by Ceph RGW using a local 
datastore cache on virtualized

regular datastore - unpatched
-----------------------------
runtime (s)  | delta RSS (KiB)
25.24 ± 0.09 | 31772
-----------------------------

S3 datastore - unpatched
-----------------------------
runtime (s)  | delta RSS (KiB)
25.19 ± 0.14 | 28072
-----------------------------

regular datastore - patched
-----------------------------
runtime (s)  | delta RSS (KiB)
25.28 ± 0.08 | 26184
-----------------------------

S3 datastore - patched
-----------------------------
runtime (s)  | delta rss (KiB)
25.42 ± 0.15 | 35024
-----------------------------

Consider:

Reviewed-by: Christian Ebner <c.ebner@proxmox.com>
Tested-by: Christian Ebner <c.ebner@proxmox.com>