public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Christian Ebner <c.ebner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-backup 8/8] client: backup: conditionally write catalog for file level backups
Date: Fri,  7 Jun 2024 11:43:13 +0200	[thread overview]
Message-ID: <20240607094313.178742-9-c.ebner@proxmox.com> (raw)
In-Reply-To: <20240607094313.178742-1-c.ebner@proxmox.com>

Only write the catalog when using the regular backup mode, do not write
it when using the split archive mode.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 pbs-client/src/pxar_backup_stream.rs | 11 +++++++----
 proxmox-backup-client/src/main.rs    | 21 ++++++++++++---------
 2 files changed, 19 insertions(+), 13 deletions(-)

diff --git a/pbs-client/src/pxar_backup_stream.rs b/pbs-client/src/pxar_backup_stream.rs
index f322566f0..cdbcdfec2 100644
--- a/pbs-client/src/pxar_backup_stream.rs
+++ b/pbs-client/src/pxar_backup_stream.rs
@@ -15,7 +15,7 @@ use nix::sys::stat::Mode;
 use proxmox_async::blocking::TokioWriterAdapter;
 use proxmox_io::StdChannelWriter;
 
-use pbs_datastore::catalog::CatalogWriter;
+use pbs_datastore::catalog::{BackupCatalogWriter, CatalogWriter};
 
 use crate::inject_reused_chunks::InjectChunks;
 use crate::pxar::create::PxarWriters;
@@ -42,7 +42,7 @@ impl Drop for PxarBackupStream {
 impl PxarBackupStream {
     pub fn new<W: Write + Send + 'static>(
         dir: Dir,
-        catalog: Arc<Mutex<CatalogWriter<W>>>,
+        catalog: Option<Arc<Mutex<CatalogWriter<W>>>>,
         options: crate::pxar::PxarCreateOptions,
         boundaries: Option<mpsc::Sender<InjectChunks>>,
         separate_payload_stream: bool,
@@ -82,7 +82,10 @@ impl PxarBackupStream {
         let handler = async move {
             if let Err(err) = crate::pxar::create_archive(
                 dir,
-                PxarWriters::new(writer, Some(catalog)),
+                PxarWriters::new(
+                    writer,
+                    catalog.map(|c| c as Arc<Mutex<dyn BackupCatalogWriter + Send>>),
+                ),
                 crate::pxar::Flags::DEFAULT,
                 move |path| {
                     log::debug!("{:?}", path);
@@ -122,7 +125,7 @@ impl PxarBackupStream {
 
     pub fn open<W: Write + Send + 'static>(
         dirname: &Path,
-        catalog: Arc<Mutex<CatalogWriter<W>>>,
+        catalog: Option<Arc<Mutex<CatalogWriter<W>>>>,
         options: crate::pxar::PxarCreateOptions,
         boundaries: Option<mpsc::Sender<InjectChunks>>,
         separate_payload_stream: bool,
diff --git a/proxmox-backup-client/src/main.rs b/proxmox-backup-client/src/main.rs
index 09af912df..e50979d6e 100644
--- a/proxmox-backup-client/src/main.rs
+++ b/proxmox-backup-client/src/main.rs
@@ -192,7 +192,7 @@ async fn backup_directory<P: AsRef<Path>>(
     archive_name: &str,
     payload_target: Option<&str>,
     chunk_size: Option<usize>,
-    catalog: Arc<Mutex<CatalogWriter<TokioWriterAdapter<StdChannelWriter<Error>>>>>,
+    catalog: Option<Arc<Mutex<CatalogWriter<TokioWriterAdapter<StdChannelWriter<Error>>>>>>,
     pxar_create_options: pbs_client::pxar::PxarCreateOptions,
     upload_options: UploadOptions,
 ) -> Result<(BackupStats, Option<BackupStats>), Error> {
@@ -1059,19 +1059,20 @@ async fn create_backup(
                     };
 
                 // start catalog upload on first use
-                if catalog.is_none() {
+                if catalog.is_none() && !detection_mode.is_data() && !detection_mode.is_metadata() {
                     let catalog_upload_res =
                         spawn_catalog_upload(client.clone(), crypto.mode == CryptMode::Encrypt)?;
                     catalog = Some(catalog_upload_res.catalog_writer);
                     catalog_result_rx = Some(catalog_upload_res.result);
                 }
-                let catalog = catalog.as_ref().unwrap();
 
                 log_file("directory", &filename, &target);
-                catalog
-                    .lock()
-                    .unwrap()
-                    .start_directory(std::ffi::CString::new(target.as_str())?.as_c_str())?;
+                if let Some(catalog) = catalog.as_ref() {
+                    catalog
+                        .lock()
+                        .unwrap()
+                        .start_directory(std::ffi::CString::new(target.as_str())?.as_c_str())?;
+                }
 
                 let mut previous_ref = None;
                 let max_cache_size = if detection_mode.is_metadata() {
@@ -1139,7 +1140,7 @@ async fn create_backup(
                     &target,
                     payload_target.as_deref(),
                     chunk_size_opt,
-                    catalog.clone(),
+                    catalog.as_ref().map(|c| c.clone()),
                     pxar_options,
                     upload_options,
                 )
@@ -1155,7 +1156,9 @@ async fn create_backup(
                     )?;
                 }
                 manifest.add_file(target, stats.size, stats.csum, crypto.mode)?;
-                catalog.lock().unwrap().end_directory()?;
+                if let Some(catalog) = catalog.as_ref() {
+                    catalog.lock().unwrap().end_directory()?;
+                }
             }
             (BackupSpecificationType::IMAGE, false) => {
                 log_file("image", &filename, &target);
-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


  parent reply	other threads:[~2024-06-07  9:43 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-06-07  9:43 [pbs-devel] [PATCH proxmox-backup 0/8] drop catalog encoding for split pxar archives Christian Ebner
2024-06-07  9:43 ` [pbs-devel] [PATCH proxmox-backup 1/8] api: datastore: factor out path decoding for catalog Christian Ebner
2024-06-07  9:43 ` [pbs-devel] [PATCH proxmox-backup 2/8] api: datastore: move reusable code out of thread Christian Ebner
2024-06-07  9:43 ` [pbs-devel] [PATCH proxmox-backup 3/8] client: tools: add helper to lookup `ArchiveEntry`s via pxar Christian Ebner
2024-06-07  9:43 ` [pbs-devel] [PATCH proxmox-backup 4/8] api: datastore: conditional lookup for catalog endpoint Christian Ebner
2024-06-07 10:23   ` Fabian Grünbichler
2024-06-07 10:34     ` Christian Ebner
2024-06-07  9:43 ` [pbs-devel] [PATCH proxmox-backup 5/8] api: datastore: add optional archive-name to file-restore Christian Ebner
2024-06-07 10:24   ` Fabian Grünbichler
2024-06-07  9:43 ` [pbs-devel] [PATCH proxmox-backup 6/8] file-restore: fallback to mpxar if catalog not present Christian Ebner
2024-06-07 10:32   ` Fabian Grünbichler
2024-06-07 10:43     ` Christian Ebner
2024-06-07 11:35       ` Fabian Grünbichler
2024-06-07 11:41         ` Christian Ebner
2024-06-07  9:43 ` [pbs-devel] [PATCH proxmox-backup 7/8] www: content: lookup via metadata archive instead of catalog Christian Ebner
2024-06-07  9:43 ` Christian Ebner [this message]
2024-06-07 10:48   ` [pbs-devel] [PATCH proxmox-backup 8/8] client: backup: conditionally write catalog for file level backups Fabian Grünbichler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240607094313.178742-9-c.ebner@proxmox.com \
    --to=c.ebner@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal