Re: [RFC PATCH proxmox-backup 0/4] fix 7333 use multiple tape drives for restoring

[Jakob-Tobias Winter <tobias@stuttgart.de>]

Hi,

On 11/10/26 14:15, Dominik Csapak wrote:
> 
> On 11/6/26 2:39 PM, Jakob-Tobias Winter wrote:
>>
>> A full restore writes the snapshot archives straight into the
>> datastore. That is safe with a single drive, because a tape backup
>> writes the chunks of a snapshot before its snapshot archive, to the
>> same or an earlier media. With several drives, a snapshot archive can
>> be read while an earlier media is still being restored, and an aborted
>> restore would leave snapshots with missing chunks behind. So with more
>> than one drive, a full restore runs the phases of the selective restore
>> for all snapshots, which only add the snapshots once all chunks are
>> restored. The restore window already takes that path whenever it sends
>> a namespace mapping. The trade-offs are that the media set catalog is
>> needed, every media is loaded twice (once for the snapshot archives,
>> once for the chunks), and the snapshots only show up at the end.
>>
> 
> maybe the double loading of tapes could be avoided by restoring
> the snapshots of the subsequent tapes in a temporary location
> and moving those afterwards? the first tape is generally
> exempt here since there the guarantees must hold anyway
> (but special casing this might be overkill)
> 
> so we could simply go through each tape, restore all chunks
> to the chunks dir and snapshots to a tmpdir, then no tape
> has to be loaded twice?
I thought about avoiding the double load as well (it's possible with the 
temp dir approach) but wanted to make as little changes as possible to 
the restore logic overall, so not to do too much at once. I would keep 
the existing restore logic for now and improve it later, if that is fine 
with you?

>> Open questions:
>>
>> - Is 'drive' plus 'additional-drives' fine, or would you prefer
>>    something else, e.g. a 'drives' list, or the changer plus "use all
>>    free drives"?
> 
> i'd probably add a 'drives' parameter that replaces the 'drive' one.
> we could then always use that and deprecate 'drive'
> 
> this is the most flexible and straightforward way i think.
Agreed.>
> 
>> - Is a maximum of four drives in total OK or can we drop the limit?
> 
> I'd probably not restrict it to an arbitrary limit, since it's
> impossible to know what is a good limit beforehand.
> The number of drives directly controls the number of threads and 
> necessary read performance, so the user has to know what his system
> can handle.
Ok.

> 
>> - Standalone drives (without changer) work, but the pool cannot know
>>    which tape is in which drive, so the operator has to follow the
>>    per-drive "insert media" notifications. Is that OK?
> 
> Not sure this feature makes too much sense for standalone drives
> at all, since it's rather unlikely that a single pbs install
> has multiple standalone drives intended for a single restore
> job, but yes, following the notification mails is ok IMO.
> 
>> - Should the "drive 'NAME': " prefix also be used with a single drive,
>>    for consistency?
> 
> no, in that case i would simply write the used drive at the beginning of
> the log somewhere so it's clear (when not done already).
At the moment, it's written into the log when it's loading the media 
into the drive used, so it's clear what's going on without prefixing. So 
let's keep it only when using more drives.

Thanks for your feedback, I will send a new version once you've had a 
chance to go over the complete set.