Re: [PATCH proxmox-backup] fix #5748: docs: add `catalog.pcat1` format specification

["Fabian Grünbichler" <f.gruenbichler@proxmox.com>]

On July 2, 2026 1:51 pm, Shan Shaji wrote:
> Earlier proxmox catalog file format specification was not present
> inside the docs. fixed it by adding format specification under file
> formats.
> 
> Fixes: https://bugzilla.proxmox.com/show_bug.cgi?id=5748
> Signed-off-by: Shan Shaji <s.shaji@proxmox.com>
> ---
>  changes since RFC: Thanks @Christian Ebner
>  - Changed from graph to table layout.
>  - Explicity mention about the variable length types. 
>  - Denoted the root start offset as u64.
>  - Add text description for each field.
>  
>  docs/file-formats.rst | 65 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 65 insertions(+)
> 
> diff --git a/docs/file-formats.rst b/docs/file-formats.rst
> index ed3250d11..52cb20ee0 100644
> --- a/docs/file-formats.rst
> +++ b/docs/file-formats.rst
> @@ -23,6 +23,71 @@ overhead introduced by the file payloads.
>  
>  .. graphviz:: meta-format-overview.dot
>  
> +.. _pcat1-format:
> +
> +Proxmox Catalog File Format (``.pcat1``)
> +----------------------------------------

this should go between pxar and mpxar, or below ppxar. there's arguments
for both, but between mpxar and ppxar which always form a pair and never
use catalog files is the wrong place ;)

> +
> +The asterisk notation indicates variable-length encoding: u64* denotes an unsigned variable-length encoded integer, while i64*
> +denotes a signed variable-length encoded integer.

it would be nice to describe the encoding here as well, after all this
part of the docs is supposed to help understand how to parse such a
file.

> +
> +.. list-table::
> +   :widths: auto
> +
> +   * - ``MAGIC: [u8; 8]``
> +     - ``[145, 253, 96, 249, 196, 103, 88, 213]``.
> +   * - ``TABLE_LENGTH: u64*``
> +     - Total length of the contents inside the Directory.
> +   * - ``ENTRY_COUNT: u64*``
> +     - Count of entries present inside the Directory.
> +   * - ``ENTRIES``
> +     - Concatinated ``ENTRY`` structures (see the ``ENTRY`` :ref:`layout <entry-layout>` .) Repeated ``ENTRY_COUNT`` times.

concatenated

> +   * - ...
> +     - Further concatination of sequential directory blocks (Table Length, Entry Count and Entries).

concatenation

what makes directory blocks "sequential"?


> +   * - ``ROOT_START_OFFSET: u64``
> +     - Root marker (/) start offset. Stored as **little-endian**

the root marker is very important, as it points at the offset of the
directory where somebody parsing this file needs to start doing
recursive lookups..

so maybe this should actually be

MAGIC
[catalog data]
ROOT_START_OFFSET

where ROOT_START_OFFSET points at the start of the '/' catalog entry

and then we have

catalog entry:

> +   * - ``TABLE_LENGTH: u64*``
> +     - Total length of the catalog contents of this Directory.
> +   * - ``ENTRY_COUNT: u64*``
> +     - Count of directory entries
> +   * - ``ENTRIES``
> +     - Concatenated ``ENTRY`` structures (see the ``ENTRY`` :ref:`layout <entry-layout>` .), repeated ``ENTRY_COUNT`` times.

> +
> +.. _entry-layout:
> +
> +Except for files and directories, no additional payload is stored beyond the name and its length inside an ENTRY.
> +
> +.. list-table::
> +   :widths: auto
> +
> +   * - ``ENTRY_TYPE: u8``
> +     - The type of entry (see entry :ref:`types <entry-types>`).
> +   * - ``NAME_LENGTH: u64*``
> +     - Length of file or directory name. Stored as variable length encoded u64.
> +   * - ``NAME: [u8]``
> +     - The raw UTF-8 string bytes of the name.
> +   * - ``PAYLOAD``
> +     - Bytes based on ``ENTRY_TYPE``
> +
> +       * **Directory ('d')**: ``offset_back: u64*`` (Child-to-Parent offset distance)

the offset back probably requires more explanation

> +       * **File ('f')**: ``size: u64*`` followed by ``mtime: i64*``
> +.. _entry-types:
> +
> +Entry Types:
> +
> +.. list-table::
> +   :widths: auto
> +
> +   * - ``d``
> +     - Directory
> +   * - ``f``
> +     - File
> +   * - ``l``
> +     - Symlink
> +   * - ``h``
> +     - Hardlink
> +   * - ``b``
> +     - Block device
> +   * - ``c``
> +     - Char device
> +   * - ``p``
> +     - FIFO
> +   * - ``s``
> +     - Socket
>  .. _ppxar-format:
>  
>  Proxmox File Archive Format - Payload (``.ppxar``)
> -- 
> 2.47.3
> 
> 
> 
> 
> 
>