From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id F26C664FE4 for ; Tue, 21 Jul 2020 17:29:42 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id F0FDE19E46 for ; Tue, 21 Jul 2020 17:29:42 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id B4A5219E3C for ; Tue, 21 Jul 2020 17:29:41 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 81DC6432FB for ; Tue, 21 Jul 2020 17:29:41 +0200 (CEST) From: Aaron Lauterer To: pbs-devel@lists.proxmox.com Date: Tue, 21 Jul 2020 17:29:40 +0200 Message-Id: <20200721152940.10152-1-a.lauterer@proxmox.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.250 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [datastore.rs] Subject: [pbs-devel] [PATCH proxmox-backup] GC: use time pre phase1 to calculate min_atime in phase2 X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jul 2020 15:29:43 -0000 Used chunks are marked in phase1 of the garbage collection process by using the atime property. Each used chunk gets touched so that the atime gets updated (if older than 24h, see relatime). Should there ever be a situation in which the phase1 in the GC run needs a very long time to finish, it could happen that the grace period calculated in phase2 is not long enough and thus the marking of the chunks (atime) becomes invalid. This would result in the removal of needed chunks. Even though the likelyhood of this happening is very low, using the timestamp from right before phase1 is started, to calculate the grace period in phase2 should avoid this situation. Signed-off-by: Aaron Lauterer --- src/backup/chunk_store.rs | 5 ++--- src/backup/datastore.rs | 2 +- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/src/backup/chunk_store.rs b/src/backup/chunk_store.rs index d57befdd..8c35ba0d 100644 --- a/src/backup/chunk_store.rs +++ b/src/backup/chunk_store.rs @@ -291,14 +291,13 @@ impl ChunkStore { pub fn sweep_unused_chunks( &self, oldest_writer: i64, + phase1_start_time: i64, status: &mut GarbageCollectionStatus, worker: &WorkerTask, ) -> Result<(), Error> { use nix::sys::stat::fstatat; - let now = unsafe { libc::time(std::ptr::null_mut()) }; - - let mut min_atime = now - 3600*24; // at least 24h (see mount option relatime) + let mut min_atime = phase1_start_time - 3600*24; // at least 24h (see mount option relatime) if oldest_writer < min_atime { min_atime = oldest_writer; diff --git a/src/backup/datastore.rs b/src/backup/datastore.rs index 5d9cb335..0966d7e9 100644 --- a/src/backup/datastore.rs +++ b/src/backup/datastore.rs @@ -426,7 +426,7 @@ impl DataStore { self.mark_used_chunks(&mut gc_status, &worker)?; worker.log("Start GC phase2 (sweep unused chunks)"); - self.chunk_store.sweep_unused_chunks(oldest_writer, &mut gc_status, &worker)?; + self.chunk_store.sweep_unused_chunks(oldest_writer, now, &mut gc_status, &worker)?; worker.log(&format!("Removed bytes: {}", gc_status.removed_bytes)); worker.log(&format!("Removed chunks: {}", gc_status.removed_chunks)); -- 2.20.1