[RFC PATCH proxmox-backup] fix #5748: docs: add `catalog.pcat1` format specification

[RFC PATCH proxmox-backup] fix #5748: docs: add `catalog.pcat1` format specification

[Shan Shaji]

Earlier proxmox catalog file format specification was not present
inside the docs. fixed it by adding format specification under file
formats.

Signed-off-by: Shan Shaji <s.shaji@proxmox.com>
---
 Sending this as an RFC because I am not sure whether the names I used make sense.

 I also marked `TABLE_LENGTH`, `ENTRY_COUNT`, `NAME_LENGTH`, `offset_back`, `size`, and
 `mtime` as 64-bit values. However, AFAIU, when these values are written to the file,
 they are encoded as variable length byte sequences. For example, if `NAME_LENGTH` is
 greater than 127, it will be encoded a multiple `u8` values. So, Is it okay to specify
 the type in logical size?

 docs/catalog-format-overview.dot | 45 ++++++++++++++++++++++++++++++++
 docs/file-formats.rst            |  6 +++++
 2 files changed, 51 insertions(+)
 create mode 100644 docs/catalog-format-overview.dot

diff --git a/docs/catalog-format-overview.dot b/docs/catalog-format-overview.dot
new file mode 100644
index 000000000..a9ec64ad3
--- /dev/null
+++ b/docs/catalog-format-overview.dot
@@ -0,0 +1,45 @@
+digraph structs {
+    graph [
+        rankdir = "LR"
+        fontname="Helvetica"
+    ];
+
+    node [
+        fontsize = "16"
+        shape = "record"
+    ];
+
+    catalog [
+        label = "catalog.pcat1"
+    ];
+
+    catalog_payload [
+        label = "<magic> CATALOG_MAGIC: [u8; 8]\l|<blocks> \{DIRECTORY_BLOCK\}*\l|ROOT_DIRECTORY_OFFSET: [u8; 8]\l"
+    ];
+
+    directory_block [
+        label = "<table_length> TABLE_LENGTH: u64\l|<entry_count> ENTRY_COUNT: u64\l|<entries> \{DIRECTORY_ENTRY\}*\l"
+    ];
+
+    directory_entry [
+        label = "<entry_type> ENTRY_TYPE: u8\l|NAME_LENGTH: u64\l|NAME: [u8]\l|<attr> ATTRIBUTE_PAYLOAD\l"
+    ];
+
+    attribute_payload [
+        label = "{{DIRECTORY_PAYLOAD|offset_back: u64}|{FILE_PAYLOAD|size: u64\nmtime: i64}|{OTHER_PAYLOAD|no extra payload}}"
+    ];
+
+    entry_types [
+        label = "'d' directory|'f' file|'l' symlink|'h' hardlink|'b' block device|'c' char device|'p' fifo|'s' socket"
+    ];
+
+    catalog -> catalog_payload:magic;
+
+    catalog_payload:blocks -> directory_block:table_length;
+
+    directory_block:entries -> directory_entry:entry_type;
+
+    directory_entry:attr -> attribute_payload;
+
+    directory_entry:entry_type -> entry_types;
+}
diff --git a/docs/file-formats.rst b/docs/file-formats.rst
index ed3250d11..f67b1b23c 100644
--- a/docs/file-formats.rst
+++ b/docs/file-formats.rst
@@ -1,5 +1,11 @@
 File Formats
 ============
+.. _pcat1-format:
+
+Proxmox Catalog File Format (``.pcat1``)
+----------------------------------------
+
+.. graphviz:: catalog-format-overview.dot
 
 .. _pxar-format:
 
-- 
2.47.3

Re: [RFC PATCH proxmox-backup] fix #5748: docs: add `catalog.pcat1` format specification

[Christian Ebner]

On 6/17/26 5:42 PM, Shan Shaji wrote:
> Earlier proxmox catalog file format specification was not present
> inside the docs. fixed it by adding format specification under file
> formats.
> 
> Signed-off-by: Shan Shaji <s.shaji@proxmox.com>
> ---
>   Sending this as an RFC because I am not sure whether the names I used make sense.
> 
>   I also marked `TABLE_LENGTH`, `ENTRY_COUNT`, `NAME_LENGTH`, `offset_back`, `size`, and
>   `mtime` as 64-bit values. However, AFAIU, when these values are written to the file,
>   they are encoded as variable length byte sequences. For example, if `NAME_LENGTH` is
>   greater than 127, it will be encoded a multiple `u8` values. So, Is it okay to specify
>   the type in logical size?

I think this could be explicitly mentioned by including some form of 
legend and referencing/marking these types accordingly.

Further, it would help to include a short clear text description for 
each entry field, similar to the pxar file format diagram. That will 
help with easier understanding.

> 
>   docs/catalog-format-overview.dot | 45 ++++++++++++++++++++++++++++++++
>   docs/file-formats.rst            |  6 +++++
>   2 files changed, 51 insertions(+)
>   create mode 100644 docs/catalog-format-overview.dot
> 
> diff --git a/docs/catalog-format-overview.dot b/docs/catalog-format-overview.dot
> new file mode 100644
> index 000000000..a9ec64ad3
> --- /dev/null
> +++ b/docs/catalog-format-overview.dot
> @@ -0,0 +1,45 @@
> +digraph structs {
> +    graph [
> +        rankdir = "LR"
> +        fontname="Helvetica"
> +    ];
> +
> +    node [
> +        fontsize = "16"
> +        shape = "record"
> +    ];
> +
> +    catalog [
> +        label = "catalog.pcat1"
> +    ];
> +
> +    catalog_payload [
> +        label = "<magic> CATALOG_MAGIC: [u8; 8]\l|<blocks> \{DIRECTORY_BLOCK\}*\l|ROOT_DIRECTORY_OFFSET: [u8; 8]\l"

The start offset for the root directory is better denoted as u64 AFAICT?

> +    ];
> +
> +    directory_block [
> +        label = "<table_length> TABLE_LENGTH: u64\l|<entry_count> ENTRY_COUNT: u64\l|<entries> \{DIRECTORY_ENTRY\}*\l"
> +    ];
> +
> +    directory_entry [
> +        label = "<entry_type> ENTRY_TYPE: u8\l|NAME_LENGTH: u64\l|NAME: [u8]\l|<attr> ATTRIBUTE_PAYLOAD\l"
> +    ];
> +
> +    attribute_payload [
> +        label = "{{DIRECTORY_PAYLOAD|offset_back: u64}|{FILE_PAYLOAD|size: u64\nmtime: i64}|{OTHER_PAYLOAD|no extra payload}}"

nit: instead of specifying the attribute payload separately, this can be 
shown as optional entries for the `directory_entry` above? Similar to 
how this is done in the pxar file format specification.

> +    ];
> +
> +    entry_types [
> +        label = "'d' directory|'f' file|'l' symlink|'h' hardlink|'b' block device|'c' char device|'p' fifo|'s' socket"

nit: these should all be left aligned IMHO.

> +    ];
> +
> +    catalog -> catalog_payload:magic;
> +
> +    catalog_payload:blocks -> directory_block:table_length;
> +
> +    directory_block:entries -> directory_entry:entry_type;
> +
> +    directory_entry:attr -> attribute_payload;
> +
> +    directory_entry:entry_type -> entry_types;
> +}
> diff --git a/docs/file-formats.rst b/docs/file-formats.rst
> index ed3250d11..f67b1b23c 100644
> --- a/docs/file-formats.rst
> +++ b/docs/file-formats.rst
> @@ -1,5 +1,11 @@
>   File Formats
>   ============
> +.. _pcat1-format:
> +
> +Proxmox Catalog File Format (``.pcat1``)
> +----------------------------------------
> +
> +.. graphviz:: catalog-format-overview.dot
>   
>   .. _pxar-format:
>   

Re: [RFC PATCH proxmox-backup] fix #5748: docs: add `catalog.pcat1` format specification

[Shan Shaji]

Superseded by v1: https://lore.proxmox.com/pbs-devel/ee4d00cd-efeb-4b66-ac44-e99593b9e7f7@proxmox.com/T/#t

Re: [RFC PATCH proxmox-backup] fix #5748: docs: add `catalog.pcat1` format specification

[Shan Shaji]

Send the wrong link:

v1: https://lore.proxmox.com/pbs-devel/20260702115115.195035-1-s.shaji@proxmox.com/T/#u

sorry for the noise. 

On Thu Jul 2, 2026 at 1:52 PM CEST, Shan Shaji wrote:
> Superseded by v1: https://lore.proxmox.com/pbs-devel/ee4d00cd-efeb-4b66-ac44-e99593b9e7f7@proxmox.com/T/#t