public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Christian Ebner <c.ebner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH v8 proxmox-backup 35/69] pxar: cover listing for split archives
Date: Tue, 28 May 2024 11:42:29 +0200	[thread overview]
Message-ID: <20240528094303.309806-36-c.ebner@proxmox.com> (raw)
In-Reply-To: <20240528094303.309806-1-c.ebner@proxmox.com>

Allows to list entries of split pxar archives. As the decoder skips
over the file payloads, the corresponding payload file has to be
provided. Otherwise the decoder would skip inside the metadata
archive, leading to incorrect decoding.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
changes since version 7:
- no changes

changes since version 6:
- not present in previous version

 pxar-bin/src/main.rs | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/pxar-bin/src/main.rs b/pxar-bin/src/main.rs
index 903467c98..b64ae1d19 100644
--- a/pxar-bin/src/main.rs
+++ b/pxar-bin/src/main.rs
@@ -454,12 +454,26 @@ async fn mount_archive(archive: String, mountpoint: String, verbose: bool) -> Re
             archive: {
                 description: "Archive name.",
             },
+            "payload-input": {
+                description: "'ppxar' payload input data file for split archive.",
+                optional: true,
+            },
         },
     },
 )]
 /// List the contents of an archive.
-fn dump_archive(archive: String) -> Result<(), Error> {
-    for entry in pxar::decoder::Decoder::open(archive)? {
+fn dump_archive(archive: String, payload_input: Option<String>) -> Result<(), Error> {
+    if archive.ends_with(".mpxar") && payload_input.is_none() {
+        bail!("Payload input required for split pxar archives");
+    }
+
+    let input = if let Some(payload_input) = payload_input {
+        pxar::PxarVariant::Split(archive, payload_input)
+    } else {
+        pxar::PxarVariant::Unified(archive)
+    };
+
+    for entry in pxar::decoder::Decoder::open(input)? {
         let entry = entry?;
 
         if log::log_enabled!(log::Level::Debug) {
-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


  parent reply	other threads:[~2024-05-28  9:43 UTC|newest]

Thread overview: 101+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-05-28  9:41 [pbs-devel] [PATCH v8 pxar proxmox-backup 00/69] fix #3174: improve file-level backup Christian Ebner
2024-05-28  9:41 ` [pbs-devel] [PATCH v8 pxar 01/69] decoder: factor out skip part from skip_entry Christian Ebner
2024-05-28  9:41 ` [pbs-devel] [PATCH v8 pxar 02/69] lib: add type for input/output variant differentiation Christian Ebner
2024-05-28  9:41 ` [pbs-devel] [PATCH v8 pxar 03/69] encoder: move to stack based state tracking Christian Ebner
2024-05-28  9:41 ` [pbs-devel] [PATCH v8 pxar 04/69] format/examples: add header type `PXAR_PAYLOAD_REF` Christian Ebner
2024-05-28  9:41 ` [pbs-devel] [PATCH v8 pxar 05/69] decoder: add method to read payload references Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 pxar 06/69] encoder: allow split output writer for archive creation Christian Ebner
2024-05-29 11:54   ` Dominik Csapak
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 pxar 07/69] decoder/accessor: allow for split input stream variant Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 pxar 08/69] decoder: set payload input range when decoding via accessor Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 pxar 09/69] encoder: add payload reference capability Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 pxar 10/69] encoder: add payload position capability Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 pxar 11/69] encoder: add payload advance capability Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 pxar 12/69] encoder/format: finish payload stream with marker Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 pxar 13/69] format: add payload stream start marker Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 pxar 14/69] format/encoder/decoder: new pxar entry type `Version` Christian Ebner
2024-06-03 11:25   ` Fabian Grünbichler
2024-06-03 11:54     ` Christian Ebner
2024-06-03 12:10       ` Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 pxar 15/69] format/encoder/decoder: new pxar entry type `Prelude` Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 16/69] client: backup: factor out extension from backup target Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 17/69] api: datastore: refactor getting local chunk reader Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 18/69] client: pxar: switch to stack based encoder state Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 19/69] client: pxar: combine writers into struct Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 20/69] client: pxar: optionally split metadata and payload streams Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 21/69] client: helper: add helpers for creating reader instances Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 22/69] client: helper: add method for split archive name mapping Christian Ebner
2024-06-04  8:17   ` Fabian Grünbichler
2024-06-04  8:30     ` Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 23/69] client: tools: helper to check pxar filename extensions Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 24/69] client: restore: read payload from dedicated index Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 25/69] tools: cover extension for split pxar archives Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 26/69] restore: " Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 27/69] client: mount: make split pxar archives mountable Christian Ebner
2024-06-04  8:24   ` Fabian Grünbichler
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 28/69] api: datastore: attach split archive payload chunk reader Christian Ebner
2024-06-04  8:26   ` Fabian Grünbichler
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 29/69] catalog: shell: make split pxar archives accessible Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 30/69] www: cover metadata extension for pxar archives Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 31/69] file restore: factor out getting pxar reader Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 32/69] file restore: cover split metadata and payload archives Christian Ebner
2024-06-04  8:28   ` Fabian Grünbichler
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 33/69] file restore: show more error context when extraction fails Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 34/69] pxar: add optional payload input for archive restore Christian Ebner
2024-06-03 13:23   ` Fabian Grünbichler
2024-05-28  9:42 ` Christian Ebner [this message]
2024-06-03 13:27   ` [pbs-devel] [PATCH v8 proxmox-backup 35/69] pxar: cover listing for split archives Fabian Grünbichler
2024-06-03 13:36     ` Christian Ebner
2024-06-03 14:54       ` Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 36/69] pxar: add more context to extraction error Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 37/69] client: pxar: include payload offset in entry listing Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 38/69] pxar: show padding in debug output on archive list Christian Ebner
2024-06-04  8:34   ` Fabian Grünbichler
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 39/69] datastore: dynamic index: add method to get digest Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 40/69] client: pxar: helper for lookup of reusable dynamic entries Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 41/69] upload stream: implement reused chunk injector Christian Ebner
2024-06-04  8:50   ` Fabian Grünbichler
2024-06-04  8:58     ` Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 42/69] client: chunk stream: add struct to hold injection state Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 43/69] chunker: add method to reset chunker state Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 44/69] client: streams: add channels for dynamic entry injection Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 45/69] specs: add backup detection mode specification Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 46/69] client: implement prepare reference method Christian Ebner
2024-06-04  9:24   ` Fabian Grünbichler
2024-06-04 12:45     ` Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 47/69] client: pxar: add method for metadata comparison Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 48/69] pxar: caching: add look-ahead cache Christian Ebner
2024-06-04  9:35   ` Fabian Grünbichler
2024-06-04 13:58     ` Christian Ebner
2024-06-05 10:56   ` Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 49/69] client: pxar: refactor catalog encoding for directories Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 50/69] fix #3174: client: pxar: enable caching and meta comparison Christian Ebner
2024-06-04 11:50   ` Fabian Grünbichler
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 51/69] client: backup writer: add injected chunk count to stats Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 52/69] pxar: create: keep track of reused chunks and files Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 53/69] pxar: create: show chunk injection stats debug output Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 54/69] client: pxar: add helper to handle optional preludes Christian Ebner
2024-06-04 11:55   ` Fabian Grünbichler
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 55/69] client: pxar: opt encode cli exclude patterns as Prelude Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 56/69] pxar: ignore version and prelude entries in listing Christian Ebner
2024-06-04  8:39   ` Fabian Grünbichler
2024-06-04  8:48     ` Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 57/69] docs: file formats: describe split pxar archive file layout Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 58/69] docs: add section describing change detection mode Christian Ebner
2024-06-04 12:07   ` Fabian Grünbichler
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 59/69] test-suite: add detection mode change benchmark Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 60/69] test-suite: Makefile: add debian package and related files Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 61/69] datastore: chunker: add Chunker trait Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 62/69] datastore: chunker: implement chunker for payload stream Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 63/69] client: chunk stream: switch payload stream chunker Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 64/69] client: pxar: allow to restore prelude to optional path Christian Ebner
2024-06-03 13:57   ` Fabian Grünbichler
2024-06-03 15:02     ` Christian Ebner
2024-05-28  9:42 ` [pbs-devel] [PATCH v8 proxmox-backup 65/69] client: pxar: add archive creation with reference test Christian Ebner
2024-05-28  9:43 ` [pbs-devel] [PATCH v8 proxmox-backup 66/69] client: tools: add helper to raise nofile rlimit Christian Ebner
2024-05-28  9:43 ` [pbs-devel] [PATCH v8 proxmox-backup 67/69] client: pxar: set cache limit based on " Christian Ebner
2024-05-28  9:43 ` [pbs-devel] [PATCH v8 proxmox-backup 68/69] chunker: tests: add regression tests for payload chunker Christian Ebner
2024-05-28  9:43 ` [pbs-devel] [PATCH v8 proxmox-backup 69/69] chunk stream: " Christian Ebner
2024-05-31 10:40 ` [pbs-devel] [PATCH v8 pxar proxmox-backup 00/69] fix #3174: improve file-level backup Dominik Csapak
2024-05-31 11:19   ` Christian Ebner
2024-06-05  8:51 ` [pbs-devel] partially-applied: " Fabian Grünbichler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240528094303.309806-36-c.ebner@proxmox.com \
    --to=c.ebner@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal