From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 6240C1FF138 for ; Mon, 29 Jun 2026 16:05:42 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id F377F214C6; Mon, 29 Jun 2026 16:05:17 +0200 (CEST) Message-ID: Date: Mon, 29 Jun 2026 16:05:12 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH proxmox-backup] fix #5748: docs: add `catalog.pcat1` format specification To: Shan Shaji , pbs-devel@lists.proxmox.com References: <20260617154214.509835-1-s.shaji@proxmox.com> Content-Language: en-US, de-DE From: Christian Ebner In-Reply-To: <20260617154214.509835-1-s.shaji@proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1782741902146 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.069 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 6BQCP33NTKZ4C3AVCMUU5MQOQSWDWBIF X-Message-ID-Hash: 6BQCP33NTKZ4C3AVCMUU5MQOQSWDWBIF X-MailFrom: c.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 6/17/26 5:42 PM, Shan Shaji wrote: > Earlier proxmox catalog file format specification was not present > inside the docs. fixed it by adding format specification under file > formats. > > Signed-off-by: Shan Shaji > --- > Sending this as an RFC because I am not sure whether the names I used make sense. > > I also marked `TABLE_LENGTH`, `ENTRY_COUNT`, `NAME_LENGTH`, `offset_back`, `size`, and > `mtime` as 64-bit values. However, AFAIU, when these values are written to the file, > they are encoded as variable length byte sequences. For example, if `NAME_LENGTH` is > greater than 127, it will be encoded a multiple `u8` values. So, Is it okay to specify > the type in logical size? I think this could be explicitly mentioned by including some form of legend and referencing/marking these types accordingly. Further, it would help to include a short clear text description for each entry field, similar to the pxar file format diagram. That will help with easier understanding. > > docs/catalog-format-overview.dot | 45 ++++++++++++++++++++++++++++++++ > docs/file-formats.rst | 6 +++++ > 2 files changed, 51 insertions(+) > create mode 100644 docs/catalog-format-overview.dot > > diff --git a/docs/catalog-format-overview.dot b/docs/catalog-format-overview.dot > new file mode 100644 > index 000000000..a9ec64ad3 > --- /dev/null > +++ b/docs/catalog-format-overview.dot > @@ -0,0 +1,45 @@ > +digraph structs { > + graph [ > + rankdir = "LR" > + fontname="Helvetica" > + ]; > + > + node [ > + fontsize = "16" > + shape = "record" > + ]; > + > + catalog [ > + label = "catalog.pcat1" > + ]; > + > + catalog_payload [ > + label = " CATALOG_MAGIC: [u8; 8]\l| \{DIRECTORY_BLOCK\}*\l|ROOT_DIRECTORY_OFFSET: [u8; 8]\l" The start offset for the root directory is better denoted as u64 AFAICT? > + ]; > + > + directory_block [ > + label = " TABLE_LENGTH: u64\l| ENTRY_COUNT: u64\l| \{DIRECTORY_ENTRY\}*\l" > + ]; > + > + directory_entry [ > + label = " ENTRY_TYPE: u8\l|NAME_LENGTH: u64\l|NAME: [u8]\l| ATTRIBUTE_PAYLOAD\l" > + ]; > + > + attribute_payload [ > + label = "{{DIRECTORY_PAYLOAD|offset_back: u64}|{FILE_PAYLOAD|size: u64\nmtime: i64}|{OTHER_PAYLOAD|no extra payload}}" nit: instead of specifying the attribute payload separately, this can be shown as optional entries for the `directory_entry` above? Similar to how this is done in the pxar file format specification. > + ]; > + > + entry_types [ > + label = "'d' directory|'f' file|'l' symlink|'h' hardlink|'b' block device|'c' char device|'p' fifo|'s' socket" nit: these should all be left aligned IMHO. > + ]; > + > + catalog -> catalog_payload:magic; > + > + catalog_payload:blocks -> directory_block:table_length; > + > + directory_block:entries -> directory_entry:entry_type; > + > + directory_entry:attr -> attribute_payload; > + > + directory_entry:entry_type -> entry_types; > +} > diff --git a/docs/file-formats.rst b/docs/file-formats.rst > index ed3250d11..f67b1b23c 100644 > --- a/docs/file-formats.rst > +++ b/docs/file-formats.rst > @@ -1,5 +1,11 @@ > File Formats > ============ > +.. _pcat1-format: > + > +Proxmox Catalog File Format (``.pcat1``) > +---------------------------------------- > + > +.. graphviz:: catalog-format-overview.dot > > .. _pxar-format: >