From: Christian Ebner <c.ebner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-backup 3/3] client: catalog: fallback to metadata archives for catalog dump
Date: Tue, 16 Jul 2024 17:33:13 +0200 [thread overview]
Message-ID: <20240716153313.533807-4-c.ebner@proxmox.com> (raw)
In-Reply-To: <20240716153313.533807-1-c.ebner@proxmox.com>
Commit c0302805c "client: backup: conditionally write catalog for
file level backups" drops encoding of the dedicated catalog when
archives are encoded as split metadata/data archives with the
`change-detection-mode` set to `data` or `metadata`.
Since the catalog is not present anymore, fallback to use the pxar
metadata archives in the manifest (if present) for generating the
listing of contents in a compatible manner.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
proxmox-backup-client/src/catalog.rs | 39 +++++++++++++++++++++++++---
1 file changed, 36 insertions(+), 3 deletions(-)
diff --git a/proxmox-backup-client/src/catalog.rs b/proxmox-backup-client/src/catalog.rs
index 276457c1d..983da3f06 100644
--- a/proxmox-backup-client/src/catalog.rs
+++ b/proxmox-backup-client/src/catalog.rs
@@ -9,11 +9,12 @@ use proxmox_router::cli::*;
use proxmox_schema::api;
use pbs_api_types::BackupNamespace;
-use pbs_client::tools::has_pxar_filename_extension;
use pbs_client::tools::key_source::get_encryption_key_password;
+use pbs_client::tools::{get_remote_pxar_reader, has_pxar_filename_extension};
use pbs_client::{BackupReader, RemoteChunkReader};
use pbs_tools::crypt_config::CryptConfig;
use pbs_tools::json::required_string_param;
+use pxar::accessor::aio::Accessor;
use crate::helper;
use crate::{
@@ -89,14 +90,46 @@ async fn dump_catalog(param: Value) -> Result<Value, Error> {
let (manifest, _) = client.download_manifest().await?;
manifest.check_fingerprint(crypt_config.as_ref().map(Arc::as_ref))?;
+ let file_info = match manifest.lookup_file_info(CATALOG_NAME) {
+ Ok(file_info) => file_info,
+ Err(err) => {
+ let mut metadata_archives = Vec::new();
+ // No catalog, fallback to metadata archives if present
+ for archive in manifest.files() {
+ if archive.filename.ends_with(".mpxar.didx") {
+ metadata_archives.push(archive.filename.clone());
+ }
+ }
+ metadata_archives.sort_unstable_by(|a, b| a.cmp(b));
+
+ for archive in &metadata_archives {
+ let (reader, archive_size) = get_remote_pxar_reader(
+ &archive,
+ client.clone(),
+ &manifest,
+ crypt_config.clone(),
+ )
+ .await?;
+ // only care about the metadata, don't attach a payload reader
+ let reader = pxar::PxarVariant::Unified(reader);
+ let accessor = Accessor::new(reader, archive_size).await?;
+ pbs_client::tools::pxar_metadata_catalog_dump(accessor, &archive).await?;
+ }
+
+ if !metadata_archives.is_empty() {
+ return Ok(Value::Null);
+ }
+
+ bail!(err);
+ }
+ };
+
let index = client
.download_dynamic_index(&manifest, CATALOG_NAME)
.await?;
let most_used = index.find_most_used_chunks(8);
- let file_info = manifest.lookup_file_info(CATALOG_NAME)?;
-
let chunk_reader = RemoteChunkReader::new(
client.clone(),
crypt_config,
--
2.39.2
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
next prev parent reply other threads:[~2024-07-16 15:33 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-07-16 15:33 [pbs-devel] [PATCH proxmox-backup 0/3] fix catalog dump for split pxar archives Christian Ebner
2024-07-16 15:33 ` [pbs-devel] [PATCH proxmox-backup 1/3] client: make helper to get remote pxar reader reusable Christian Ebner
2024-07-16 15:33 ` [pbs-devel] [PATCH proxmox-backup 2/3] client: add helper to dump catalog from metadata archive Christian Ebner
2024-07-16 15:33 ` Christian Ebner [this message]
2024-07-22 10:34 ` [pbs-devel] [PATCH proxmox-backup 0/3] fix catalog dump for split pxar archives Christian Ebner
2024-07-22 10:39 ` Christian Ebner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240716153313.533807-4-c.ebner@proxmox.com \
--to=c.ebner@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal