From: "Fabian Grünbichler" <f.gruenbichler@proxmox.com>
To: Proxmox Backup Server development discussion
<pbs-devel@lists.proxmox.com>
Subject: [pbs-devel] applied-series: [PATCH v2 proxmox-backup 0/9] drop catalog encoding for split pxar archives
Date: Fri, 07 Jun 2024 14:13:51 +0200 [thread overview]
Message-ID: <1717762413.67ou0ani0a.astroid@yuna.none> (raw)
In-Reply-To: <20240607113752.324017-1-c.ebner@proxmox.com>
with two minor follow-ups (using schema for achive-name, and cargo fmt)
On June 7, 2024 1:37 pm, Christian Ebner wrote:
> The catalog stores metadata needed for fast lookups via e.g. the file
> browser for single file restore.
> Since it is now possible to create snapshots with split pxar archives,
> the same metadata can be accessed in a reasonable performat way also via
> the metadata archive, effectively making the catalog unneeded duplicate
> data for this case.
>
> This patch series therefore allows to lookup directory entries where
> needed via the metadata archive instead of the catalog and conditionally
> drops the catalog encoding for snapshots using split archive encoding.
>
> The patches adapt not only the `catalog` api endpoint, used by the
> Proxmox Backup WebUI to access the snapshots contents but also the
> proxomox-file-restore, which is used by the Proxmox Virtual Environments
> PBS storage plugin to access and restore contents.
>
> Tested that regular backups can still be inspected and restored and that
> the same functionality is still available for split archive backups,
> without now having a catalog encoded.
>
> Functionally was tested with the following path series applied to PVE:
> https://lists.proxmox.com/pipermail/pve-devel/2024-May/064024.html
>
> Please note: For directories with may entries a noticeable performance
> drop is present, which might be improved upon by optimizing decoding.
>
> Changes since version 1:
> - Do not base64 encode archive name
> - Split off file restore archive name listing
>
> The rest of the comments will be addressed as followups as discussed
> off-list.
>
> Christian Ebner (9):
> api: datastore: factor out path decoding for catalog
> api: datastore: move reusable code out of thread
> client: tools: add helper to lookup `ArchiveEntry`s via pxar
> api: datastore: conditional lookup for catalog endpoint
> api: datastore: add optional archive-name to file-restore
> file-restore: never list ppxar as archive
> file-restore: fallback to mpxar if catalog not present
> www: content: lookup via metadata archive instead of catalog
> client: backup: conditionally write catalog for file level backups
>
> pbs-client/src/pxar_backup_stream.rs | 11 ++-
> pbs-client/src/tools/mod.rs | 84 +++++++++++++++++++
> proxmox-backup-client/src/main.rs | 21 +++--
> proxmox-file-restore/src/main.rs | 72 ++++++++++++----
> src/api2/admin/datastore.rs | 121 ++++++++++++++++++---------
> www/datastore/Content.js | 3 +
> 6 files changed, 243 insertions(+), 69 deletions(-)
>
> --
> 2.39.2
>
> From 8dffaab3ce2c20b0c376d6c0dc8b3d561499fa24 Mon Sep 17 00:00:00 2001
> From: Christian Ebner <c.ebner@proxmox.com>
> Date: Fri, 7 Jun 2024 11:25:09 +0200
>
> Christian Ebner (8):
> api: datastore: factor out path decoding for catalog
> api: datastore: move reusable code out of thread
> client: tools: add helper to lookup `ArchiveEntry`s via pxar
> api: datastore: conditional lookup for catalog endpoint
> api: datastore: add optional archive-name to file-restore
> file-restore: fallback to mpxar if catalog not present
> www: content: lookup via metadata archive instead of catalog
> client: backup: conditionally write catalog for file level backups
>
> pbs-client/src/pxar_backup_stream.rs | 11 ++-
> pbs-client/src/tools/mod.rs | 84 +++++++++++++++++
> proxmox-backup-client/src/main.rs | 21 +++--
> proxmox-file-restore/src/main.rs | 72 +++++++++++----
> src/api2/admin/datastore.rs | 129 ++++++++++++++++++---------
> www/datastore/Content.js | 3 +
> 6 files changed, 251 insertions(+), 69 deletions(-)
>
> --
> 2.39.2
>
>
>
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
>
>
>
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
prev parent reply other threads:[~2024-06-07 12:13 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-06-07 11:37 [pbs-devel] " Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 1/9] api: datastore: factor out path decoding for catalog Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 2/9] api: datastore: move reusable code out of thread Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 3/9] client: tools: add helper to lookup `ArchiveEntry`s via pxar Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 4/9] api: datastore: conditional lookup for catalog endpoint Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 5/9] api: datastore: add optional archive-name to file-restore Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 6/9] file-restore: never list ppxar as archive Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 7/9] file-restore: fallback to mpxar if catalog not present Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 8/9] www: content: lookup via metadata archive instead of catalog Christian Ebner
2024-06-07 11:37 ` [pbs-devel] [PATCH v2 proxmox-backup 9/9] client: backup: conditionally write catalog for file level backups Christian Ebner
2024-06-07 12:13 ` Fabian Grünbichler [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1717762413.67ou0ani0a.astroid@yuna.none \
--to=f.gruenbichler@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox