From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <vrodriguez@soltecsis.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 37FD392219 for <pve-user@lists.proxmox.com>; Mon, 13 Mar 2023 18:18:10 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 10DF7F7E6 for <pve-user@lists.proxmox.com>; Mon, 13 Mar 2023 18:17:40 +0100 (CET) Received: from sts-umail-02.soltecsis.com (smtp.soltecsis.com [51.91.174.135]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for <pve-user@lists.proxmox.com>; Mon, 13 Mar 2023 18:17:39 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=soltecsis.com; s=smtp; t=1678727259; bh=BgSROxHeS2hp9IQAzKqdIqaOogcdSVWO0vrRobLUX0I=; h=Date:To:References:From:Subject:In-Reply-To:From; b=BFuALWU3ZMYieE1SoRyJoJabZyUQ/SfnH/tkDWtcblIB72dQ+gaHxB+4NO3GDvvqs If5qad8oma7kerC2fMx20qNRUqSTlZF9jnk/s57PdO/vAlFYJ2NCuKLGf6BdYKcQhp RPHJyZEgEg5NYfBoQPs+T9iParlb85SZC69M9y7Y= Received: from sts-zimbra.soltecsis.com ([10.99.4.45]) by sts-umail-02.soltecsis.com (8.15.2/8.15.2/Debian-22ubuntu3) with ESMTPS id 32DH7cf2182985 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for <pve-user@lists.proxmox.com>; Mon, 13 Mar 2023 18:07:39 +0100 Received: from sts-zimbra.soltecsis.com (localhost [127.0.0.1]) by sts-zimbra.soltecsis.com (Postfix) with ESMTPS id 4F5F94A04C4 for <pve-user@lists.proxmox.com>; Mon, 13 Mar 2023 17:07:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sts-zimbra.soltecsis.com (Postfix) with ESMTP id 4C3574A04C6 for <pve-user@lists.proxmox.com>; Mon, 13 Mar 2023 17:07:38 +0000 (UTC) Received: from sts-zimbra.soltecsis.com ([127.0.0.1]) by localhost (sts-zimbra.soltecsis.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id wR8spK5zTGy9 for <pve-user@lists.proxmox.com>; Mon, 13 Mar 2023 17:07:38 +0000 (UTC) Received: from [10.99.99.4] (unknown [10.99.99.4]) by sts-zimbra.soltecsis.com (Postfix) with ESMTPSA id 31B464A04C4 for <pve-user@lists.proxmox.com>; Mon, 13 Mar 2023 17:07:38 +0000 (UTC) Message-ID: <83c0b99e-f2ae-009d-db11-51b93575b227@soltecsis.com> Date: Mon, 13 Mar 2023 18:07:37 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Content-Language: en-US To: pve-user@lists.proxmox.com References: <4443ef0f136aed8ca4ef76b9de88677a@shelldog.de> <mailman.74.1678721342.446.pve-user@lists.proxmox.com> From: Victor Rodriguez <vrodriguez@soltecsis.com> In-Reply-To: <mailman.74.1678721342.446.pve-user@lists.proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-SOLTECSIS-MailScanner-ID: 32DH7cf2182985 X-SOLTECSIS-MailScanner: Found to be clean X-SOLTECSIS-MailScanner-SpamCheck: not spam (whitelisted), SpamAssassin (not cached, score=-3.101, required 6, autolearn=not spam, ALL_TRUSTED -1.00, BAYES_00 -1.90, DKIM_SIGNED 0.10, DKIM_VALID -0.10, DKIM_VALID_AU -0.10, DKIM_VALID_EF -0.10, NICE_REPLY_A -0.00) X-SOLTECSIS-MailScanner-From: vrodriguez@soltecsis.com X-Spam-Status: No X-SPAM-LEVEL: Spam detection results: 0 BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain NICE_REPLY_A -0.001 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [PVE-User] Timeouts while creating backups on PBS X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list <pve-user.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-user>, <mailto:pve-user-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-user/> List-Post: <mailto:pve-user@lists.proxmox.com> List-Help: <mailto:pve-user-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user>, <mailto:pve-user-request@lists.proxmox.com?subject=subscribe> X-List-Received-Date: Mon, 13 Mar 2023 17:18:10 -0000 > Because we do want to have verified backups, we tried out scheduled > verification tasks. Then we have the problem, that it takes to long > (>1d) to verify all ~250 new snapshots sequentially. 50 parallel > verify jobs works well. AFAIK, a verification job will not re-verify a chunk that has been verified already by that same job, meaning that a verify job for 2+ snapshots should be more efficient that a verify job for each snapshot or doing an inmediate verify after each backup. On the other hand, verifies are processed sequentially, so even if your host has enough resources the job will take some time to finish. The approach I currently use is to get a few verify jobs for each namespace and start them at the same time if possible. --