From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 99BC0601D8 for ; Fri, 16 Oct 2020 08:26:34 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 8840D17E46 for ; Fri, 16 Oct 2020 08:26:34 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 7B95617E36 for ; Fri, 16 Oct 2020 08:26:33 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 46DFA45AB0 for ; Fri, 16 Oct 2020 08:26:33 +0200 (CEST) Date: Fri, 16 Oct 2020 08:26:15 +0200 (CEST) From: Dietmar Maurer To: Proxmox Backup Server development discussion , Stefan Reiter Message-ID: <1219333710.278.1602829576453@webmail.proxmox.com> In-Reply-To: <20201015104916.21170-2-s.reiter@proxmox.com> References: <20201015104916.21170-1-s.reiter@proxmox.com> <20201015104916.21170-2-s.reiter@proxmox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Priority: 3 Importance: Normal X-Mailer: Open-Xchange Mailer v7.10.4-Rev11 X-Originating-Client: open-xchange-appsuite X-SPAM-LEVEL: Spam detection results: 0 AWL 0.065 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] [PATCH v2 proxmox-backup 1/4] gc: avoid race between phase1 and forget/prune X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Oct 2020 06:26:34 -0000 > ...by saving all forgotten chunks into a HashSet when we detect a GC forgotten chunks? I think you store index file names in that hash... > phase1 currently running. If GC then hits a NotFound, it can check if > the snapshot was simply forgotten behind its back, or if an actual error > occurred and it needs to abort (since it might delete still referenced > chunks, if the error is transient and the index file is still there). Thought more about this, and this "transient" error model makes no sense too me. You can get such "transient" errors during list_images(), so this approach does not really provide any protection for that. So for now, I applied a simpler version which simply ignores vanished files. We can still apply further safety guards if it turns out we need them ... > We have to attach the error message in {fixed,dynamic}_index via the > .context() method, otherwise the original std::io::Error gets lost and > we can't check for NotFound. above comment is obsolete for v2