Re: [pbs-devel] Verify, Prune & GC sequence

[Graeme Seaton <lists@graemes.com>]

Hi Thomas,

Thanks for the reply - didn't really want stay up to midnight to check 
;-)  At the moment, scheduling only allows daily (which is invoked at 
00:00).  If (when?) I can specify custom times then I'll probably use 
the sequence you described.

Regards,

Graeme

On 30/10/2020 11:09, Thomas Lamprecht wrote:
> Hi,
>
> On 30.10.20 10:31, Graeme Seaton wrote:
>> If I have verify, prune & GC all set to run daily do the jobs run concurrently or in sequence?
> Well, you could just try out ;-)
>
> But yes, they can run in parallel.
>
> Since PBS allows multiple potentially interfering operations at the same time (e.g., garbage collect,
> prune, multiple backup creations (only in separate groups), forget, ...), these need to lock against
> each other in certain scenarios. There is no overarching global lock though, instead always the finest
> grained lock possible is used, because running these operations concurrently is treated as a feature
> on its own.
>
> Still, while we handle locking fine grained an admin could try to optimize the order of scheduling
> such things.
>
> 1. Prune, to increase the chance that GC can sweep more data and verify has less to look at.
>     This is rather quick, so one could schedule the next probably half an hour afterwards and
>     be confident it finishes before 2.
>
> 2. GC, as chunks are deleted, which may make verify also faster as it can make the underlying
>     FS faster - but that's rather theory.
>
> 3. Verify
>
> I think the order of 2. and 3. is not to important, but personally, I'd always schedule pruning
> first, can only help.
>
> cheers,
> Thomas
>
>