* [pbs-devel] Verify, Prune & GC sequence @ 2020-10-30 9:31 Graeme Seaton 2020-10-30 11:09 ` Thomas Lamprecht 0 siblings, 1 reply; 7+ messages in thread From: Graeme Seaton @ 2020-10-30 9:31 UTC (permalink / raw) To: Proxmox Backup Server development discussion Hi, If I have verify, prune & GC all set to run daily do the jobs run concurrently or in sequence? Regrds, Graeme ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [pbs-devel] Verify, Prune & GC sequence 2020-10-30 9:31 [pbs-devel] Verify, Prune & GC sequence Graeme Seaton @ 2020-10-30 11:09 ` Thomas Lamprecht 2020-10-30 16:31 ` Graeme Seaton 0 siblings, 1 reply; 7+ messages in thread From: Thomas Lamprecht @ 2020-10-30 11:09 UTC (permalink / raw) To: Proxmox Backup Server development discussion, Graeme Seaton Hi, On 30.10.20 10:31, Graeme Seaton wrote: > If I have verify, prune & GC all set to run daily do the jobs run concurrently or in sequence? Well, you could just try out ;-) But yes, they can run in parallel. Since PBS allows multiple potentially interfering operations at the same time (e.g., garbage collect, prune, multiple backup creations (only in separate groups), forget, ...), these need to lock against each other in certain scenarios. There is no overarching global lock though, instead always the finest grained lock possible is used, because running these operations concurrently is treated as a feature on its own. Still, while we handle locking fine grained an admin could try to optimize the order of scheduling such things. 1. Prune, to increase the chance that GC can sweep more data and verify has less to look at. This is rather quick, so one could schedule the next probably half an hour afterwards and be confident it finishes before 2. 2. GC, as chunks are deleted, which may make verify also faster as it can make the underlying FS faster - but that's rather theory. 3. Verify I think the order of 2. and 3. is not to important, but personally, I'd always schedule pruning first, can only help. cheers, Thomas ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [pbs-devel] Verify, Prune & GC sequence 2020-10-30 11:09 ` Thomas Lamprecht @ 2020-10-30 16:31 ` Graeme Seaton 2020-10-30 17:17 ` Thomas Lamprecht 0 siblings, 1 reply; 7+ messages in thread From: Graeme Seaton @ 2020-10-30 16:31 UTC (permalink / raw) To: Proxmox Backup Server development discussion Hi Thomas, Thanks for the reply - didn't really want stay up to midnight to check ;-) At the moment, scheduling only allows daily (which is invoked at 00:00). If (when?) I can specify custom times then I'll probably use the sequence you described. Regards, Graeme On 30/10/2020 11:09, Thomas Lamprecht wrote: > Hi, > > On 30.10.20 10:31, Graeme Seaton wrote: >> If I have verify, prune & GC all set to run daily do the jobs run concurrently or in sequence? > Well, you could just try out ;-) > > But yes, they can run in parallel. > > Since PBS allows multiple potentially interfering operations at the same time (e.g., garbage collect, > prune, multiple backup creations (only in separate groups), forget, ...), these need to lock against > each other in certain scenarios. There is no overarching global lock though, instead always the finest > grained lock possible is used, because running these operations concurrently is treated as a feature > on its own. > > Still, while we handle locking fine grained an admin could try to optimize the order of scheduling > such things. > > 1. Prune, to increase the chance that GC can sweep more data and verify has less to look at. > This is rather quick, so one could schedule the next probably half an hour afterwards and > be confident it finishes before 2. > > 2. GC, as chunks are deleted, which may make verify also faster as it can make the underlying > FS faster - but that's rather theory. > > 3. Verify > > I think the order of 2. and 3. is not to important, but personally, I'd always schedule pruning > first, can only help. > > cheers, > Thomas > > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [pbs-devel] Verify, Prune & GC sequence 2020-10-30 16:31 ` Graeme Seaton @ 2020-10-30 17:17 ` Thomas Lamprecht 2020-10-31 6:43 ` Dietmar Maurer 0 siblings, 1 reply; 7+ messages in thread From: Thomas Lamprecht @ 2020-10-30 17:17 UTC (permalink / raw) To: Proxmox Backup Server development discussion, Graeme Seaton Hi Graeme, On 30.10.20 17:31, Graeme Seaton wrote: > Hi Thomas, > > Thanks for the reply - didn't really want stay up to midnight to check ;-) At the moment, scheduling only allows daily (which is invoked at 00:00). If (when?) I can specify custom times then I'll probably use the sequence you described. > You can already specify custom times, just edit it, "daily" is just set as default when creating a new job. cheers, Thomas ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [pbs-devel] Verify, Prune & GC sequence 2020-10-30 17:17 ` Thomas Lamprecht @ 2020-10-31 6:43 ` Dietmar Maurer 2020-10-31 6:53 ` Thomas Lamprecht 0 siblings, 1 reply; 7+ messages in thread From: Dietmar Maurer @ 2020-10-31 6:43 UTC (permalink / raw) To: Proxmox Backup Server development discussion, Thomas Lamprecht, Graeme Seaton > > Thanks for the reply - didn't really want stay up to midnight to check ;-) At the moment, scheduling only allows daily (which is invoked at 00:00). If (when?) I can specify custom times then I'll probably use the sequence you described. > > > > You can already specify custom times, just edit it, "daily" is just set as default > when creating a new job. > To make it more clear, we support systemd Calendar events. For format details see: https://www.freedesktop.org/software/systemd/man/systemd.time.html#Calendar%20Events ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [pbs-devel] Verify, Prune & GC sequence 2020-10-31 6:43 ` Dietmar Maurer @ 2020-10-31 6:53 ` Thomas Lamprecht 2020-10-31 8:27 ` Graeme Seaton 0 siblings, 1 reply; 7+ messages in thread From: Thomas Lamprecht @ 2020-10-31 6:53 UTC (permalink / raw) To: Dietmar Maurer, Proxmox Backup Server development discussion, Graeme Seaton On 31.10.20 07:43, Dietmar Maurer wrote: >>> Thanks for the reply - didn't really want stay up to midnight to check ;-) At the moment, scheduling only allows daily (which is invoked at 00:00). If (when?) I can specify custom times then I'll probably use the sequence you described. >>> >> >> You can already specify custom times, just edit it, "daily" is just set as default >> when creating a new job. >> > > To make it more clear, we support systemd Calendar events. For format details see: > > https://www.freedesktop.org/software/systemd/man/systemd.time.html#Calendar%20Events > Here the link to our docs, as we support only the relevant features of Calendar Events: https://pbs.proxmox.com/docs/calendarevents.html ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [pbs-devel] Verify, Prune & GC sequence 2020-10-31 6:53 ` Thomas Lamprecht @ 2020-10-31 8:27 ` Graeme Seaton 0 siblings, 0 replies; 7+ messages in thread From: Graeme Seaton @ 2020-10-31 8:27 UTC (permalink / raw) To: Thomas Lamprecht, Dietmar Maurer, Proxmox Backup Server development discussion Thank you both for the links - tested last night and worked great. My bad on manually editing the schedule - didn't find it immediately obvious that I could edit the value in the dropbox. Would also congratulate you all on the project - it is looking really nice and the UI has come along in leaps and bounds :-D On 31/10/2020 06:53, Thomas Lamprecht wrote: > On 31.10.20 07:43, Dietmar Maurer wrote: >>>> Thanks for the reply - didn't really want stay up to midnight to check ;-) At the moment, scheduling only allows daily (which is invoked at 00:00). If (when?) I can specify custom times then I'll probably use the sequence you described. >>>> >>> You can already specify custom times, just edit it, "daily" is just set as default >>> when creating a new job. >>> >> To make it more clear, we support systemd Calendar events. For format details see: >> >> https://www.freedesktop.org/software/systemd/man/systemd.time.html#Calendar%20Events >> > Here the link to our docs, as we support only the relevant features of Calendar Events: > > https://pbs.proxmox.com/docs/calendarevents.html > > ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2020-10-31 8:27 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-10-30 9:31 [pbs-devel] Verify, Prune & GC sequence Graeme Seaton 2020-10-30 11:09 ` Thomas Lamprecht 2020-10-30 16:31 ` Graeme Seaton 2020-10-30 17:17 ` Thomas Lamprecht 2020-10-31 6:43 ` Dietmar Maurer 2020-10-31 6:53 ` Thomas Lamprecht 2020-10-31 8:27 ` Graeme Seaton
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox