From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 042EB64973 for ; Fri, 30 Oct 2020 17:31:24 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id E7DDF15A78 for ; Fri, 30 Oct 2020 17:31:23 +0100 (CET) Received: from e2i461.smtp2go.com (e2i461.smtp2go.com [103.2.141.205]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 60C9115A6C for ; Fri, 30 Oct 2020 17:31:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=smtpservice.net; s=m6dg70.a1-4.dyn; x=1604076382; h=Feedback-ID: X-Smtpcorp-Track:Date:Message-ID:From:To:Subject:Reply-To:Sender: List-Unsubscribe; bh=qj0ODR1hLjSK6tabUERMdt22xIEOtIFMYQdG20Vc/Dg=; b=NT2IitNF KC3zHc3fhF8NgD3DzLFNsiz1Ln1SYLWaBKOJCKMXyyd573S2wcW99FO1uQIovomD3qA940edoPe38 9TnbTkDBc0AC3Tu9CUlVwi/7OmfIIpEkP7RyEP8RUhVJCQshTE9HOTUxJklXtaoifoW0zM3WkvPAu eL9lbhLwD82+zHcl7Nq4+W/w5sPsuayi57jObw/kQ0cSj6IZub+dnh8JJKRNvnXmG9MHhoZsMqGFS sGb5UIl1kXua4VMkXzyV42YmUEmsaX9qZGsgGD0IpFTe/1xU+E0TKso0I5DixruO9LZm5GN8V0ZjS uZV9zePsKVfJO3LPTrMJqqouhA==; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=graemes.com; i=@graemes.com; q=dns/txt; s=s297367; t=1604075482; h=from : subject : to : message-id : date; bh=qj0ODR1hLjSK6tabUERMdt22xIEOtIFMYQdG20Vc/Dg=; b=PtEgPxfwGsyiLrnBxT8wFRM05JhxGj9a5f6/7mMag+It7zMPDUmi40WZL0y3SisJoMpGR 0AiUKlwHd4LANRFOxERHsRnK0OxOoomdjvmGfWPW1wqN7BNwtz4i/dUWfCjqrvojJq/hBis wpNbD+JOpfj+HXRvoe9fFUvE84a4KQ7fFivaofYoA1baOqySlE960R/+TPB+wQH7D6s6GW1 SkkPMy4rjFmWcxnSh9Lhdqn1lRnV2zbW19Bns8L1HfpagFnqh5xCHkPutYvqGAQn4m1JbOD b6T3xr3kn/9ZAl/fmpBg2WAR0tLtpm5BZHzSjMzKHqypTbvUaugPKpIhr2xw== Received: from [10.176.58.103] (helo=SmtpCorp) by smtpcorp.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92-S2G) (envelope-from ) id 1kYXJB-cp4RjV-Ju for pbs-devel@lists.proxmox.com; Fri, 30 Oct 2020 16:31:21 +0000 Received: from [10.83.151.206] (helo=zappa.graemes.com) by smtpcorp.com with esmtpsa (TLS1.2:DHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92-S2G) (envelope-from ) id 1kYXJB-9EFQNT-BS for pbs-devel@lists.proxmox.com; Fri, 30 Oct 2020 16:31:21 +0000 Received: from [192.168.7.2] (duke.graemes.com [192.168.7.2]) (Authenticated sender: lists) by zappa.graemes.com (Postfix) with ESMTPSA id 763BDC00094D2 for ; Fri, 30 Oct 2020 16:31:20 +0000 (GMT) To: Proxmox Backup Server development discussion References: <215c8b24-84f6-c244-f3d0-347390001707@graemes.com> From: Graeme Seaton Message-ID: Date: Fri, 30 Oct 2020 16:31:20 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 763BDC00094D2 X-Spamd-Result: default: False [-3.10 / 20.00]; GENERIC_REPUTATION(0.00)[-0.62449471079677]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; BAYES_HAM(-3.00)[99.99%] X-Rspamd-Server: zappa.graemes.com X-Smtpcorp-Track: 1kYbJU9EFQNTUS.nmBYXLZOoUyq3 Feedback-ID: 297367m:297367aJZwe00:297367sLw5tQGFU7 X-Report-Abuse: Please forward a copy of this message, including all headers, to X-SPAM-LEVEL: Spam detection results: 0 AWL 0.157 Adjusted score from AWL reputation of From: address DKIMWL_WL_MED -0.001 DKIMwl.org - Medium trust sender DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain NICE_REPLY_A -0.261 Looks like a legit reply (A) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_PASS -0.001 SPF: HELO matches SPF record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [smtpservice.net, graemes.com] Subject: Re: [pbs-devel] Verify, Prune & GC sequence X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Oct 2020 16:31:24 -0000 Hi Thomas, Thanks for the reply - didn't really want stay up to midnight to check ;-)  At the moment, scheduling only allows daily (which is invoked at 00:00).  If (when?) I can specify custom times then I'll probably use the sequence you described. Regards, Graeme On 30/10/2020 11:09, Thomas Lamprecht wrote: > Hi, > > On 30.10.20 10:31, Graeme Seaton wrote: >> If I have verify, prune & GC all set to run daily do the jobs run concurrently or in sequence? > Well, you could just try out ;-) > > But yes, they can run in parallel. > > Since PBS allows multiple potentially interfering operations at the same time (e.g., garbage collect, > prune, multiple backup creations (only in separate groups), forget, ...), these need to lock against > each other in certain scenarios. There is no overarching global lock though, instead always the finest > grained lock possible is used, because running these operations concurrently is treated as a feature > on its own. > > Still, while we handle locking fine grained an admin could try to optimize the order of scheduling > such things. > > 1. Prune, to increase the chance that GC can sweep more data and verify has less to look at. > This is rather quick, so one could schedule the next probably half an hour afterwards and > be confident it finishes before 2. > > 2. GC, as chunks are deleted, which may make verify also faster as it can make the underlying > FS faster - but that's rather theory. > > 3. Verify > > I think the order of 2. and 3. is not to important, but personally, I'd always schedule pruning > first, can only help. > > cheers, > Thomas > >