public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pbs-devel] [PATCH proxmox-backup] backup info: avoid additional stat syscall for protected check
@ 2025-05-22  6:21 Christian Ebner
  0 siblings, 0 replies; only message in thread
From: Christian Ebner @ 2025-05-22  6:21 UTC (permalink / raw)
  To: pbs-devel

`BackupDir::is_protected` is the general helper method to check the
protected state for a snapshot. This checks for the presence of the
protected marker file, which is performed by stating the file and
requires traversing the full path.

When generating the backup list for a backup group, the snapshot
directory contents are however scanned nevertheless. Take advantage
of this by extending the regex used to filter contents by scandir to
include also the protected marker filename and set the state based on
the presence/absence, thereby avoiding the additional stat syscall
altogether.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---

This was encountered while investigating possible causes for slow GC
phase1 since PBS version 3.4 on some particular storages as reported
in the community forum [0]. While an improvement, this is very
unlikely the issue at hand.

[0] https://forum.proxmox.com/threads/166214/

 pbs-datastore/Cargo.toml         |  2 ++
 pbs-datastore/src/backup_info.rs | 29 ++++++++++++++++++++++++++---
 2 files changed, 28 insertions(+), 3 deletions(-)

diff --git a/pbs-datastore/Cargo.toml b/pbs-datastore/Cargo.toml
index 7623adc28..d7fe71b34 100644
--- a/pbs-datastore/Cargo.toml
+++ b/pbs-datastore/Cargo.toml
@@ -9,6 +9,7 @@ rust-version.workspace = true
 [dependencies]
 anyhow.workspace = true
 base64.workspace = true
+const_format.workspace = true
 crc32fast.workspace = true
 endian_trait.workspace = true
 futures.workspace = true
@@ -17,6 +18,7 @@ libc.workspace = true
 log.workspace = true
 nix.workspace = true
 openssl.workspace = true
+regex.workspace = true
 serde.workspace = true
 serde_json.workspace = true
 tokio = { workspace = true, features = [] }
diff --git a/pbs-datastore/src/backup_info.rs b/pbs-datastore/src/backup_info.rs
index d4732fdd9..dbd0905c6 100644
--- a/pbs-datastore/src/backup_info.rs
+++ b/pbs-datastore/src/backup_info.rs
@@ -7,6 +7,7 @@ use std::sync::{Arc, LazyLock};
 use std::time::Duration;
 
 use anyhow::{bail, format_err, Context, Error};
+use const_format::concatcp;
 
 use proxmox_sys::fs::{lock_dir_noblock, lock_dir_noblock_shared, replace_file, CreateOptions};
 use proxmox_systemd::escape_unit;
@@ -21,6 +22,11 @@ use crate::manifest::{BackupManifest, MANIFEST_LOCK_NAME};
 use crate::{DataBlob, DataStore};
 
 pub const DATASTORE_LOCKS_DIR: &str = "/run/proxmox-backup/locks";
+const PROTECTED_MARKER_FILENAME: &str = ".protected";
+
+proxmox_schema::const_regex! {
+    pub BACKUP_FILES_AND_PROTECTED_REGEX = concatcp!(r"^(.*\.([fd]idx|blob)|\", PROTECTED_MARKER_FILENAME, ")$");
+}
 
 // TODO: Remove with PBS 5
 // Note: The `expect()` call here will only happen if we can neither confirm nor deny the existence
@@ -113,9 +119,26 @@ impl BackupGroup {
                 }
 
                 let backup_dir = self.backup_dir_with_rfc3339(backup_time)?;
-                let files = list_backup_files(l2_fd, backup_time)?;
+                let mut protected = false;
+                let mut files = Vec::new();
 
-                let protected = backup_dir.is_protected();
+                proxmox_sys::fs::scandir(
+                    l2_fd,
+                    backup_time,
+                    &BACKUP_FILES_AND_PROTECTED_REGEX,
+                    |_, filename, file_type| {
+                        if file_type != nix::dir::Type::File {
+                            return Ok(());
+                        }
+                        // avoids more expensive check via `BackupDir::is_protected`
+                        if filename == ".protected" {
+                            protected = true;
+                        } else {
+                            files.push(filename.to_owned());
+                        }
+                        Ok(())
+                    },
+                )?;
 
                 list.push(BackupInfo {
                     backup_dir,
@@ -457,7 +480,7 @@ impl BackupDir {
 
     pub fn protected_file(&self) -> PathBuf {
         let mut path = self.full_path();
-        path.push(".protected");
+        path.push(PROTECTED_MARKER_FILENAME);
         path
     }
 
-- 
2.39.5



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2025-05-22  6:22 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-05-22  6:21 [pbs-devel] [PATCH proxmox-backup] backup info: avoid additional stat syscall for protected check Christian Ebner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal