all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Christian Ebner <c.ebner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH v2 proxmox-backup 9/9] client: backup: conditionally write catalog for file level backups
Date: Fri,  7 Jun 2024 13:37:52 +0200	[thread overview]
Message-ID: <20240607113752.324017-10-c.ebner@proxmox.com> (raw)
In-Reply-To: <20240607113752.324017-1-c.ebner@proxmox.com>

Only write the catalog when using the regular backup mode, do not write
it when using the split archive mode.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 pbs-client/src/pxar_backup_stream.rs | 11 +++++++----
 proxmox-backup-client/src/main.rs    | 21 ++++++++++++---------
 2 files changed, 19 insertions(+), 13 deletions(-)

diff --git a/pbs-client/src/pxar_backup_stream.rs b/pbs-client/src/pxar_backup_stream.rs
index f322566f0..cdbcdfec2 100644
--- a/pbs-client/src/pxar_backup_stream.rs
+++ b/pbs-client/src/pxar_backup_stream.rs
@@ -15,7 +15,7 @@ use nix::sys::stat::Mode;
 use proxmox_async::blocking::TokioWriterAdapter;
 use proxmox_io::StdChannelWriter;
 
-use pbs_datastore::catalog::CatalogWriter;
+use pbs_datastore::catalog::{BackupCatalogWriter, CatalogWriter};
 
 use crate::inject_reused_chunks::InjectChunks;
 use crate::pxar::create::PxarWriters;
@@ -42,7 +42,7 @@ impl Drop for PxarBackupStream {
 impl PxarBackupStream {
     pub fn new<W: Write + Send + 'static>(
         dir: Dir,
-        catalog: Arc<Mutex<CatalogWriter<W>>>,
+        catalog: Option<Arc<Mutex<CatalogWriter<W>>>>,
         options: crate::pxar::PxarCreateOptions,
         boundaries: Option<mpsc::Sender<InjectChunks>>,
         separate_payload_stream: bool,
@@ -82,7 +82,10 @@ impl PxarBackupStream {
         let handler = async move {
             if let Err(err) = crate::pxar::create_archive(
                 dir,
-                PxarWriters::new(writer, Some(catalog)),
+                PxarWriters::new(
+                    writer,
+                    catalog.map(|c| c as Arc<Mutex<dyn BackupCatalogWriter + Send>>),
+                ),
                 crate::pxar::Flags::DEFAULT,
                 move |path| {
                     log::debug!("{:?}", path);
@@ -122,7 +125,7 @@ impl PxarBackupStream {
 
     pub fn open<W: Write + Send + 'static>(
         dirname: &Path,
-        catalog: Arc<Mutex<CatalogWriter<W>>>,
+        catalog: Option<Arc<Mutex<CatalogWriter<W>>>>,
         options: crate::pxar::PxarCreateOptions,
         boundaries: Option<mpsc::Sender<InjectChunks>>,
         separate_payload_stream: bool,
diff --git a/proxmox-backup-client/src/main.rs b/proxmox-backup-client/src/main.rs
index 09af912df..f1c7fbf93 100644
--- a/proxmox-backup-client/src/main.rs
+++ b/proxmox-backup-client/src/main.rs
@@ -192,7 +192,7 @@ async fn backup_directory<P: AsRef<Path>>(
     archive_name: &str,
     payload_target: Option<&str>,
     chunk_size: Option<usize>,
-    catalog: Arc<Mutex<CatalogWriter<TokioWriterAdapter<StdChannelWriter<Error>>>>>,
+    catalog: Option<Arc<Mutex<CatalogWriter<TokioWriterAdapter<StdChannelWriter<Error>>>>>>,
     pxar_create_options: pbs_client::pxar::PxarCreateOptions,
     upload_options: UploadOptions,
 ) -> Result<(BackupStats, Option<BackupStats>), Error> {
@@ -1059,19 +1059,20 @@ async fn create_backup(
                     };
 
                 // start catalog upload on first use
-                if catalog.is_none() {
+                if catalog.is_none() && !detection_mode.is_data() && !detection_mode.is_metadata() {
                     let catalog_upload_res =
                         spawn_catalog_upload(client.clone(), crypto.mode == CryptMode::Encrypt)?;
                     catalog = Some(catalog_upload_res.catalog_writer);
                     catalog_result_rx = Some(catalog_upload_res.result);
                 }
-                let catalog = catalog.as_ref().unwrap();
 
                 log_file("directory", &filename, &target);
-                catalog
-                    .lock()
-                    .unwrap()
-                    .start_directory(std::ffi::CString::new(target.as_str())?.as_c_str())?;
+                if let Some(catalog) = catalog.as_ref() {
+                    catalog
+                        .lock()
+                        .unwrap()
+                        .start_directory(std::ffi::CString::new(target.as_str())?.as_c_str())?;
+                }
 
                 let mut previous_ref = None;
                 let max_cache_size = if detection_mode.is_metadata() {
@@ -1139,7 +1140,7 @@ async fn create_backup(
                     &target,
                     payload_target.as_deref(),
                     chunk_size_opt,
-                    catalog.clone(),
+                    catalog.as_ref().cloned(),
                     pxar_options,
                     upload_options,
                 )
@@ -1155,7 +1156,9 @@ async fn create_backup(
                     )?;
                 }
                 manifest.add_file(target, stats.size, stats.csum, crypto.mode)?;
-                catalog.lock().unwrap().end_directory()?;
+                if let Some(catalog) = catalog.as_ref() {
+                    catalog.lock().unwrap().end_directory()?;
+                }
             }
             (BackupSpecificationType::IMAGE, false) => {
                 log_file("image", &filename, &target);
-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


  parent reply	other threads:[~2024-06-07 11:37 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-06-07 11:37 [pbs-devel] [PATCH v2 proxmox-backup 0/9] drop catalog encoding for split pxar archives Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 1/9] api: datastore: factor out path decoding for catalog Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 2/9] api: datastore: move reusable code out of thread Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 3/9] client: tools: add helper to lookup `ArchiveEntry`s via pxar Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 4/9] api: datastore: conditional lookup for catalog endpoint Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 5/9] api: datastore: add optional archive-name to file-restore Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 6/9] file-restore: never list ppxar as archive Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 7/9] file-restore: fallback to mpxar if catalog not present Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 8/9] www: content: lookup via metadata archive instead of catalog Christian Ebner
2024-06-07 11:37 ` Christian Ebner [this message]
2024-06-07 12:13 ` [pbs-devel] applied-series: [PATCH v2 proxmox-backup 0/9] drop catalog encoding for split pxar archives Fabian Grünbichler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240607113752.324017-10-c.ebner@proxmox.com \
    --to=c.ebner@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal