public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Christian Ebner <c.ebner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH v2 proxmox-backup 7/9] file-restore: fallback to mpxar if catalog not present
Date: Fri,  7 Jun 2024 13:37:50 +0200	[thread overview]
Message-ID: <20240607113752.324017-8-c.ebner@proxmox.com> (raw)
In-Reply-To: <20240607113752.324017-1-c.ebner@proxmox.com>

The `proxmox-file-restore list` command will uses the provided path to
lookup and list directory entries via the catalog. Fallback to using
the metadata archive if the catalog is not present for fast lookups in
a backup snapshot.

This is in preparation for dropping encoding of the catalog for
snapshots using split archive encoding. Proxmox VE's storage plugin
uses this to allow single file restore for LXCs.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 proxmox-file-restore/src/main.rs | 69 +++++++++++++++++++++++++-------
 1 file changed, 54 insertions(+), 15 deletions(-)

diff --git a/proxmox-file-restore/src/main.rs b/proxmox-file-restore/src/main.rs
index 0f16f3d42..a09873467 100644
--- a/proxmox-file-restore/src/main.rs
+++ b/proxmox-file-restore/src/main.rs
@@ -147,24 +147,63 @@ async fn list_files(
             Ok(entries)
         }
         ExtractPath::Pxar(file, mut path) => {
-            let index = client
-                .download_dynamic_index(&manifest, CATALOG_NAME)
+            if let Ok(file_info) = manifest.lookup_file_info(CATALOG_NAME) {
+                let index = client
+                    .download_dynamic_index(&manifest, CATALOG_NAME)
+                    .await?;
+                let most_used = index.find_most_used_chunks(8);
+                let chunk_reader = RemoteChunkReader::new(
+                    client.clone(),
+                    crypt_config,
+                    file_info.chunk_crypt_mode(),
+                    most_used,
+                );
+                let reader = BufferedDynamicReader::new(index, chunk_reader);
+                let mut catalog_reader = CatalogReader::new(reader);
+
+                let mut fullpath = file.into_bytes();
+                fullpath.append(&mut path);
+
+                catalog_reader.list_dir_contents(&fullpath)
+            } else {
+                if path.is_empty() {
+                    path = vec![b'/'];
+                }
+
+                let (archive_name, payload_archive_name) =
+                    pbs_client::tools::get_pxar_archive_names(&file, &manifest)?;
+
+                let (reader, archive_size) = get_remote_pxar_reader(
+                    &archive_name,
+                    client.clone(),
+                    &manifest,
+                    crypt_config.clone(),
+                )
                 .await?;
-            let most_used = index.find_most_used_chunks(8);
-            let file_info = manifest.lookup_file_info(CATALOG_NAME)?;
-            let chunk_reader = RemoteChunkReader::new(
-                client.clone(),
-                crypt_config,
-                file_info.chunk_crypt_mode(),
-                most_used,
-            );
-            let reader = BufferedDynamicReader::new(index, chunk_reader);
-            let mut catalog_reader = CatalogReader::new(reader);
 
-            let mut fullpath = file.into_bytes();
-            fullpath.append(&mut path);
+                let reader = if let Some(payload_archive_name) = payload_archive_name {
+                    let (payload_reader, payload_size) = get_remote_pxar_reader(
+                        &payload_archive_name,
+                        client,
+                        &manifest,
+                        crypt_config,
+                    )
+                    .await?;
+                    pxar::PxarVariant::Split(reader, (payload_reader, payload_size))
+                } else {
+                    pxar::PxarVariant::Unified(reader)
+                };
+
+                let accessor = Accessor::new(reader, archive_size).await?;
+                let path = OsStr::from_bytes(&path);
 
-            catalog_reader.list_dir_contents(&fullpath)
+                pbs_client::tools::pxar_metadata_catalog_lookup(
+                    accessor,
+                    &path,
+                    Some(&archive_name),
+                )
+                .await
+            }
         }
         ExtractPath::VM(file, path) => {
             let details = SnapRestoreDetails {
-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


  parent reply	other threads:[~2024-06-07 11:38 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-06-07 11:37 [pbs-devel] [PATCH v2 proxmox-backup 0/9] drop catalog encoding for split pxar archives Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 1/9] api: datastore: factor out path decoding for catalog Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 2/9] api: datastore: move reusable code out of thread Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 3/9] client: tools: add helper to lookup `ArchiveEntry`s via pxar Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 4/9] api: datastore: conditional lookup for catalog endpoint Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 5/9] api: datastore: add optional archive-name to file-restore Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 6/9] file-restore: never list ppxar as archive Christian Ebner
2024-06-07 11:37 ` Christian Ebner [this message]
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 8/9] www: content: lookup via metadata archive instead of catalog Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 9/9] client: backup: conditionally write catalog for file level backups Christian Ebner
2024-06-07 12:13 ` [pbs-devel] applied-series: [PATCH v2 proxmox-backup 0/9] drop catalog encoding for split pxar archives Fabian Grünbichler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240607113752.324017-8-c.ebner@proxmox.com \
    --to=c.ebner@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal