Re: [RFC PATCH proxmox-backup 0/4] fix #7903: rebuild local S3 chunk markers

[Christian Ebner <c.ebner@proxmox.com>]

Thanks for the patches, left a few suggestions and major concerns on 
individual patches. It is especially crucial to take the possible 
runtime of this into account!

What is currently complete missing is any form of exposing this optional 
behavior to the user in the UI. It might make sense to have such a flag 
in the datastore create window, which is the main use-case for this. I 
see less use for normal s3 refresh operations.

On 9/15/26 6:00 PM, Shan Shaji wrote:
> Sending this as an RFC to get an initial feedback. I'd really appreciate
> any suggestions or pointers to anything I may have missed.
> 
> Reusing an S3-backed datastore currently restores the backup metadata to
> the local cache without creating local chunk markers. A garbage collection
> run is then needed to recreate markers for chunks referenced by the backup
> indexes.
> 
> This series addresses #7908 by listing the chunk objects in S3 and creating
> zero-byte local markers when a datastore is recreated with
> --reuse-datastore. It also adds an optional rebuild-chunk-markers parameter
> to the s3-refresh API and exposes it through proxmox-backup-manager:
> 
>      proxmox-backup-manager datastore s3-refresh <store> --rebuild-chunk-markers true
> 
> The option is disabled by default for explicit refreshes. When enabled,
> it creates markers for all chunk objects with valid chunk keys, including
> chunks not referenced by the restored backup indexes. Existing cached
> chunk files are replaced with empty markers.
> 
> Shan Shaji (4):
>    datastore: factor out s3 chunk objects pagination into a helper
>    fix #7908: datastore: create zero-byte markers when re-using datastore
>    datastore: api: add option to rebuild S3 chunk markers
>    datastore: backup-manager: expose option to rebuild s3 chunk markers
> 
>   pbs-datastore/src/datastore.rs              | 120 +++++++++++++++-----
>   src/api2/admin/datastore.rs                 |  23 +++-
>   src/api2/config/datastore.rs                |   6 +-
>   src/bin/proxmox_backup_manager/datastore.rs |   7 ++
>   4 files changed, 121 insertions(+), 35 deletions(-)
> 
> --
> 2.47.3
> 
> 
> 
>