public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Christian Ebner <c.ebner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-backup 3/3] client: catalog: fallback to metadata archives for catalog dump
Date: Tue, 16 Jul 2024 17:33:13 +0200	[thread overview]
Message-ID: <20240716153313.533807-4-c.ebner@proxmox.com> (raw)
In-Reply-To: <20240716153313.533807-1-c.ebner@proxmox.com>

Commit c0302805c "client: backup: conditionally write catalog for
file level backups" drops encoding of the dedicated catalog when
archives are encoded as split metadata/data archives with the
`change-detection-mode` set to `data` or `metadata`.

Since the catalog is not present anymore, fallback to use the pxar
metadata archives in the manifest (if present) for generating the
listing of contents in a compatible manner.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 proxmox-backup-client/src/catalog.rs | 39 +++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)

diff --git a/proxmox-backup-client/src/catalog.rs b/proxmox-backup-client/src/catalog.rs
index 276457c1d..983da3f06 100644
--- a/proxmox-backup-client/src/catalog.rs
+++ b/proxmox-backup-client/src/catalog.rs
@@ -9,11 +9,12 @@ use proxmox_router::cli::*;
 use proxmox_schema::api;
 
 use pbs_api_types::BackupNamespace;
-use pbs_client::tools::has_pxar_filename_extension;
 use pbs_client::tools::key_source::get_encryption_key_password;
+use pbs_client::tools::{get_remote_pxar_reader, has_pxar_filename_extension};
 use pbs_client::{BackupReader, RemoteChunkReader};
 use pbs_tools::crypt_config::CryptConfig;
 use pbs_tools::json::required_string_param;
+use pxar::accessor::aio::Accessor;
 
 use crate::helper;
 use crate::{
@@ -89,14 +90,46 @@ async fn dump_catalog(param: Value) -> Result<Value, Error> {
     let (manifest, _) = client.download_manifest().await?;
     manifest.check_fingerprint(crypt_config.as_ref().map(Arc::as_ref))?;
 
+    let file_info = match manifest.lookup_file_info(CATALOG_NAME) {
+        Ok(file_info) => file_info,
+        Err(err) => {
+            let mut metadata_archives = Vec::new();
+            // No catalog, fallback to metadata archives if present
+            for archive in manifest.files() {
+                if archive.filename.ends_with(".mpxar.didx") {
+                    metadata_archives.push(archive.filename.clone());
+                }
+            }
+            metadata_archives.sort_unstable_by(|a, b| a.cmp(b));
+
+            for archive in &metadata_archives {
+                let (reader, archive_size) = get_remote_pxar_reader(
+                    &archive,
+                    client.clone(),
+                    &manifest,
+                    crypt_config.clone(),
+                )
+                .await?;
+                // only care about the metadata, don't attach a payload reader
+                let reader = pxar::PxarVariant::Unified(reader);
+                let accessor = Accessor::new(reader, archive_size).await?;
+                pbs_client::tools::pxar_metadata_catalog_dump(accessor, &archive).await?;
+            }
+
+            if !metadata_archives.is_empty() {
+                return Ok(Value::Null);
+            }
+
+            bail!(err);
+        }
+    };
+
     let index = client
         .download_dynamic_index(&manifest, CATALOG_NAME)
         .await?;
 
     let most_used = index.find_most_used_chunks(8);
 
-    let file_info = manifest.lookup_file_info(CATALOG_NAME)?;
-
     let chunk_reader = RemoteChunkReader::new(
         client.clone(),
         crypt_config,
-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


  parent reply	other threads:[~2024-07-16 15:33 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-07-16 15:33 [pbs-devel] [PATCH proxmox-backup 0/3] fix catalog dump for split pxar archives Christian Ebner
2024-07-16 15:33 ` [pbs-devel] [PATCH proxmox-backup 1/3] client: make helper to get remote pxar reader reusable Christian Ebner
2024-07-16 15:33 ` [pbs-devel] [PATCH proxmox-backup 2/3] client: add helper to dump catalog from metadata archive Christian Ebner
2024-07-16 15:33 ` Christian Ebner [this message]
2024-07-22 10:34 ` [pbs-devel] [PATCH proxmox-backup 0/3] fix catalog dump for split pxar archives Christian Ebner
2024-07-22 10:39   ` Christian Ebner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240716153313.533807-4-c.ebner@proxmox.com \
    --to=c.ebner@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal