From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <pbs-devel-bounces@lists.proxmox.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 61A9A1FF176 for <inbox@lore.proxmox.com>; Fri, 21 Feb 2025 15:01:38 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id A0FB13322; Fri, 21 Feb 2025 15:01:35 +0100 (CET) From: Christian Ebner <c.ebner@proxmox.com> To: pbs-devel@lists.proxmox.com Date: Fri, 21 Feb 2025 15:01:07 +0100 Message-Id: <20250221140110.377328-3-c.ebner@proxmox.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250221140110.377328-1-c.ebner@proxmox.com> References: <20250221140110.377328-1-c.ebner@proxmox.com> MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.031 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pbs-devel] [PATCH proxmox-backup 2/5] garbage collection: refactor archive type based chunk marking logic X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion <pbs-devel.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>, <mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/> List-Post: <mailto:pbs-devel@lists.proxmox.com> List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>, <mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe> Reply-To: Proxmox Backup Server development discussion <pbs-devel@lists.proxmox.com> Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" <pbs-devel-bounces@lists.proxmox.com> Move the logic for marking in-use chunks by image files based on archive type and its error handling into its own dedicated method. This is in preparation for optimizing the iteration order to avoid multiple atime updates of chunks. The method can then be reused for both cases, iteration over expected image file paths and unexpected paths, the latter being iterated separately. No functional changes. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> --- pbs-datastore/src/datastore.rs | 48 ++++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 19 deletions(-) diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs index a6a91ca79..eda78193d 100644 --- a/pbs-datastore/src/datastore.rs +++ b/pbs-datastore/src/datastore.rs @@ -1065,6 +1065,34 @@ impl DataStore { Ok(()) } + fn mark_used_chunks_do( + &self, + img: &Path, + status: &mut GarbageCollectionStatus, + worker: &dyn WorkerTaskContext, + ) -> Result<(), Error> { + match std::fs::File::open(img) { + Ok(file) => { + if let Ok(archive_type) = ArchiveType::from_path(img) { + if archive_type == ArchiveType::FixedIndex { + let index = FixedIndexReader::new(file).map_err(|err| { + format_err!("can't read index '{}' - {err}", img.to_string_lossy()) + })?; + self.index_mark_used_chunks(index, img, status, worker)?; + } else if archive_type == ArchiveType::DynamicIndex { + let index = DynamicIndexReader::new(file).map_err(|err| { + format_err!("can't read index '{}' - {err}", img.to_string_lossy()) + })?; + self.index_mark_used_chunks(index, img, status, worker)?; + } + } + } + Err(err) if err.kind() == io::ErrorKind::NotFound => (), // ignore vanished files + Err(err) => bail!("can't open index {} - {err}", img.to_string_lossy()), + } + Ok(()) + } + fn mark_used_chunks( &self, status: &mut GarbageCollectionStatus, @@ -1090,25 +1118,7 @@ impl DataStore { } } - match std::fs::File::open(&img) { - Ok(file) => { - if let Ok(archive_type) = ArchiveType::from_path(&img) { - if archive_type == ArchiveType::FixedIndex { - let index = FixedIndexReader::new(file).map_err(|e| { - format_err!("can't read index '{}' - {}", img.to_string_lossy(), e) - })?; - self.index_mark_used_chunks(index, &img, status, worker)?; - } else if archive_type == ArchiveType::DynamicIndex { - let index = DynamicIndexReader::new(file).map_err(|e| { - format_err!("can't read index '{}' - {}", img.to_string_lossy(), e) - })?; - self.index_mark_used_chunks(index, &img, status, worker)?; - } - } - } - Err(err) if err.kind() == io::ErrorKind::NotFound => (), // ignore vanished files - Err(err) => bail!("can't open index {} - {}", img.to_string_lossy(), err), - } + self.mark_used_chunks_do(&img, status, worker)?; let percentage = (i + 1) * 100 / image_count; if percentage > last_percentage { -- 2.39.5 _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel