all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Shan Shaji <s.shaji@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [PATCH proxmox-backup v2] fix #5748: docs: add `catalog.pcat1` format specification
Date: Fri, 17 Jul 2026 14:51:44 +0200	[thread overview]
Message-ID: <20260717125144.286823-1-s.shaji@proxmox.com> (raw)

Earlier proxmox catalog file format specification was not present
inside the docs. fixed it by adding format specification under file
formats.

Fixes: https://bugzilla.proxmox.com/show_bug.cgi?id=5748
Signed-off-by: Shan Shaji <s.shaji@proxmox.com>
---

 changes since v1: Thanks @Fabian
  * move the formatting below  .ppxar formatting.
  * add description explaining the variable length encoding.
  * seperate the catalog data contents into its own table and update the
    root start offset description.
  * add more explanation about `offset_back`.

 changes since RFC: Thanks @Christian Ebner
  * Changed from graph to table layout.
  * Explicity mention about the variable length types. 
  * Denoted the root start offset as u64.
  * Add text description for each field.

 docs/file-formats.rst | 89 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 88 insertions(+), 1 deletion(-)

diff --git a/docs/file-formats.rst b/docs/file-formats.rst
index ed3250d11..04d0406a7 100644
--- a/docs/file-formats.rst
+++ b/docs/file-formats.rst
@@ -55,6 +55,94 @@ All headers are stored as little-endian.
    * - ``PAYLOAD_TAIL_MARKER``
      - header of ``[u8; 16]`` consisting of type hash and size;
        marks end
+
+.. _pcat1-format:
+
+Proxmox Catalog File Format (``.pcat1``)
+----------------------------------------
+
+The asterisk notation indicates variable-length integers:
+
+* **u64\***: Unsigned variable-length integer. Each encoded byte is 8 bits wide. Bit 7, the most significant bit, is the continuation
+  flag. Bits 0 through 6 carry value bits. The first encoded byte stores the lowest order 7 value bits, the second encoded byte
+  stores the next 7 value bits, and so on.
+
+* **i64\***: Signed variable-length integer. Each nonzero encoded byte is 8 bits wide. Bit 7, the most significant bit, is the
+  continuation flag. Bits 0 through 6 carry value bits. The first encoded byte stores the lowest-order 7 value bits, the second
+  encoded byte stores the next 7 value bits, and so on. Non-negative values end with a byte whose continuation flag is clear.
+  Negative values end with an additional 0x00 terminator byte.
+
+.. list-table::
+   :widths: auto
+
+   * - ``MAGIC: [u8; 8]``
+     - ``[145, 253, 96, 249, 196, 103, 88, 213]``.
+   * - [Catalog Data]
+     - Raw catalog data. This includes TABLE_LENGTH, ENTRY_COUNT and ENTRIES. See ``Catalog Data`` :ref:`contents <catalog-data>`.
+   * - ...
+     - Further concatenation of catalog data. 
+   * - ``ROOT_START_OFFSET: u64``
+     - Points at the start of the '/' catalog entry. Stored as **little-endian**.
+
+Catalog Data:
+
+.. _catalog-data:
+
+.. list-table::
+   :widths: auto
+
+   * - ``TABLE_LENGTH: u64*``
+     - Total length of the contents inside the Directory.
+   * - ``ENTRY_COUNT: u64*``
+     - Number of entries present inside the Directory.
+   * - ``ENTRIES``
+     - Concatenated ``ENTRY`` structures (see the ``ENTRY`` :ref:`layout <entry-layout>` .) Repeated ``ENTRY_COUNT`` times.
+
+.. _entry-layout:
+
+Except for files and directories, no additional payload is stored beyond the name and its length inside an ENTRY.
+The ``offset_back`` stores the backward byte distance from the start of the parent directory table to the start of the
+referenced child directory table. The reader can then compute the start offset of the child directory table as
+``parent_table_start - offset_back``.
+  
+.. list-table::
+   :widths: auto
+
+   * - ``ENTRY_TYPE: u8``
+     - The type of entry (see entry :ref:`types <entry-types>`).
+   * - ``NAME_LENGTH: u64*``
+     - Length of file or directory name. Stored as variable length encoded u64.
+   * - ``NAME: [u8]``
+     - The raw UTF-8 string bytes of the name.
+   * - ``PAYLOAD``
+     - Bytes based on ``ENTRY_TYPE``
+
+       * **Directory ('d')**: ``offset_back: u64*`` (Child-to-Parent offset distance.)
+       * **File ('f')**: ``size: u64*`` followed by ``mtime: i64*``
+.. _entry-types:
+
+Entry Types:
+
+.. list-table::
+   :widths: auto
+
+   * - ``d``
+     - Directory
+   * - ``f``
+     - File
+   * - ``l``
+     - Symlink
+   * - ``h``
+     - Hardlink
+   * - ``b``
+     - Block device
+   * - ``c``
+     - Char device
+   * - ``p``
+     - FIFO
+   * - ``s``
+     - Socket
+
 .. _data-blob-format:
 
 Data Blob Format (``.blob``)
@@ -103,7 +191,6 @@ encrypted data:
    * - ``TAG: [u8; 16]``
    * - ``Data: (max 16MiB)``
 
-
 .. _fixed-index-format:
 
 Fixed Index Format  (``.fidx``)
-- 
2.47.3





                 reply	other threads:[~2026-07-17 12:52 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260717125144.286823-1-s.shaji@proxmox.com \
    --to=s.shaji@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal