From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 4E4D91FF2C6 for ; Tue, 9 Jul 2024 12:47:36 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 6AD341D98B; Tue, 9 Jul 2024 12:47:56 +0200 (CEST) From: Stoiko Ivanov To: pve-devel@lists.proxmox.com Date: Tue, 9 Jul 2024 12:47:04 +0200 Message-Id: <20240709104705.1681395-5-s.ivanov@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240709104705.1681395-1-s.ivanov@proxmox.com> References: <20240709104705.1681395-1-s.ivanov@proxmox.com> MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.075 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [pool.rs, docs.rs] Subject: [pve-devel] [PATCH proxmox-offline-mirror v2 4/5] pool: gc: remove empty directories under link_dir X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" garbage collection currently is quite aggressive in removing all files under the link_dir, which are not a hard-link to a checksum file. removing directories that remain empty below the link_dir should thus not too dangerous. without this patch, removing a snapshot on a mirror, running gc there, and syncing everything to a medium, leaves the medium with an hierarchy of empty directories below the removed snapshot (the files get cleaned up the directories remain). using WalkDir::content_first() seems better than to check for emptiness after each file-removal [0] [0] https://docs.rs/walkdir/latest/walkdir/struct.WalkDir.html#method.contents_first Signed-off-by: Stoiko Ivanov --- src/pool.rs | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/src/pool.rs b/src/pool.rs index b4f2a6a..56dcbb4 100644 --- a/src/pool.rs +++ b/src/pool.rs @@ -451,6 +451,7 @@ impl PoolLockGuard<'_> { /// Run a garbage collection, removing /// - any checksum files that have no links outside of `pool_dir` /// - any files in `link_dir` that have no corresponding checksum files + /// - any empty directories below `link_dir` remaining after the file removal pub(crate) fn gc(&self) -> Result<(usize, u64), Error> { let (inode_map, _link_count) = self.get_inode_csum_map()?; @@ -459,7 +460,8 @@ impl PoolLockGuard<'_> { let handle_entry = |entry: Result, count: &mut usize, - size: &mut u64| + size: &mut u64, + remove_empty_dir: bool| -> Result<(), Error> { let path = entry?.into_path(); if path == self.lock_path() { @@ -467,6 +469,10 @@ impl PoolLockGuard<'_> { } let meta = path.metadata()?; + if remove_empty_dir && meta.is_dir() && path.read_dir()?.next().is_none() { + std::fs::remove_dir(path)?; + return Ok(()); + } if !meta.is_file() { return Ok(()); }; @@ -507,11 +513,12 @@ impl PoolLockGuard<'_> { }; WalkDir::new(&self.pool.link_dir) + .contents_first(true) .into_iter() - .try_for_each(|entry| handle_entry(entry, &mut count, &mut size))?; + .try_for_each(|entry| handle_entry(entry, &mut count, &mut size, true))?; WalkDir::new(&self.pool.pool_dir) .into_iter() - .try_for_each(|entry| handle_entry(entry, &mut count, &mut size))?; + .try_for_each(|entry| handle_entry(entry, &mut count, &mut size, false))?; Ok((count, size)) } -- 2.39.2 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel