Re: [PATCH proxmox-backup v7 2/9] datastore: add move-group

[Christian Ebner <c.ebner@proxmox.com>]

On 4/16/26 7:18 PM, Hannes Laimer wrote:
> Add support for moving a single backup group to a different namespace
> within the same datastore.
> 
> For the filesystem backend each snapshot directory is renamed
> individually. For S3 all objects are copied to the target prefix
> before deleting the source, per snapshot.
> 
> Exclusive locks on the group and all its snapshots are acquired
> before the move to ensure no concurrent operations are active.
> Snapshots are locked and moved in batches to avoid exhausting file
> descriptors on groups with many snapshots.

Unless I overlook it, there currently is still one major issue which can 
lead to data loss with this:

Garbage collection uses the Datastore's list_index_files() method to 
collect all index files at the start of phase 1. This is to know which 
chunks need atime updates to mark them as in use. Snapshots which 
disappear in the mean time can be ignored, as the chunks may then no 
longer be in use. Snapshots created in the mean time are safe, as there 
it is the cutoff time protecting newly written chunks which are not 
referenced by any of the index files which are now not in the list.

But if the move happens after GC started and collected the index files, 
but before reaching that index files. the moved index file still might 
reference chunks which are in-use, but now never get an atime update.

Locking unfortunately does not protect against this.

So if there is an ongoing garbage collection phase 1, there is the need 
for some mechanism to re-inject the index files in the list of indices 
and therefore chunks to process.
This might require to write the moved indices to a file, so they can be 
read and processed at the end of GC phase 1 even if GC is running in a 
different process. And it requires to flock that file and wait for it to 
become available before continuing.

> 
> If the target group already exists and merging is enabled, the
> individual snapshots are moved into the existing target group
> provided both groups have the same owner and the source snapshots
> are strictly newer than the target snapshots.
> 
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---

[..]