From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id B4C82EF44 for ; Thu, 28 Sep 2023 10:08:12 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 8DCD713504 for ; Thu, 28 Sep 2023 10:07:42 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Thu, 28 Sep 2023 10:07:41 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 92DB54874D for ; Thu, 28 Sep 2023 10:07:41 +0200 (CEST) Date: Thu, 28 Sep 2023 10:07:40 +0200 (CEST) From: Christian Ebner To: Wolfgang Bumiller Cc: pbs-devel@lists.proxmox.com Message-ID: <702617592.5309.1695888460618@webmail.proxmox.com> In-Reply-To: <908189970.5090.1695815737265@webmail.proxmox.com> References: <20230922071621.12670-1-c.ebner@proxmox.com> <20230922071621.12670-5-c.ebner@proxmox.com> <908189970.5090.1695815737265@webmail.proxmox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Priority: 3 Importance: Normal X-Mailer: Open-Xchange Mailer v7.10.6-Rev50 X-Originating-Client: open-xchange-appsuite X-SPAM-LEVEL: Spam detection results: 0 AWL 0.094 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] [RFC pxar 4/20] fix #3174: metadata: impl fn to calc byte size X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2023 08:08:12 -0000 I was giving this some more thought and are not really convinced that sending this trough an encoder instance, which digests the encoded byte stream and counts the bytes is the right approach here. The purpose of this function is to calculate the bytes, which I can easily skip over *without* having to call any expensive encoding/decoding functionality. I might get around this by simply calling the decoder on the byte stream, than I do not need this at all (if I'm not missing something). Might that be the better approach? Additionally, and maybe even better, I might get rid of this also by letting the PXAR_APPENDIX_REF offset point to the start of the file payload entry, instead of the file entry as is now, thereby being able to blindly skip over this already to begin with. Although I am not sure if that is the best approach for handling the metadata, which should ideally not be encoded twice, once before the PXAR_APPENDIX_REF and the PXAR_PAYLOAD. > On 27.09.2023 13:55 CEST Christian Ebner wrote: > > > > On 27.09.2023 13:38 CEST Wolfgang Bumiller wrote: > > > > > > On Fri, Sep 22, 2023 at 09:16:05AM +0200, Christian Ebner wrote: > > > Add a helper function to calculate the byte size of pxar Metadata > > > objects, needed to be able to recalculate offsets when creating archives > > > with appendix sections. > > > > > > Signed-off-by: Christian Ebner > > > --- > > > src/lib.rs | 44 ++++++++++++++++++++++++++++++++++++++++++++ > > > 1 file changed, 44 insertions(+) > > > > > > diff --git a/src/lib.rs b/src/lib.rs > > > index 210c4b1..ed7ba40 100644 > > > --- a/src/lib.rs > > > +++ b/src/lib.rs > > > @@ -144,6 +144,50 @@ impl Metadata { > > > pub fn builder_from_stat(stat: &libc::stat) -> MetadataBuilder { > > > MetadataBuilder::new(0).fill_from_stat(stat) > > > } > > > + > > > + /// Calculate the number of bytes when serialized in pxar archive > > > + pub fn calculate_byte_len(&self) -> usize { > > > > This looks like a maintenance nightmare with extra sprinkles. > > ;) > > > > > Can we instead create a specialized `SeqWrite` type that just counts > > bytes instead of actually writing anything and call the encoder's > > `encode_metadata()` with it? > > (Either by factorizing `encode_metadat()` out further or by creating an > > actual `EncoderImpl` instance with it...) > > Yes, I will have a look on how to improve this and include this as well in the next version of the patch series, thx for the hints. > > > > > > + let mut bytes = mem::size_of::(); > > > + bytes += mem::size_of_val(&self.stat); > > > + for xattr in &self.xattrs { > > > + bytes += mem::size_of::(); > > > + bytes += mem::size_of_val(xattr); > > > + } > > > + for acl_user in &self.acl.users { > > > + bytes += mem::size_of::(); > > > + bytes += mem::size_of_val(acl_user); > > > + } > > > + for acl_group in &self.acl.groups { > > > + bytes += mem::size_of::(); > > > + bytes += mem::size_of_val(acl_group); > > > + } > > > + if let Some(group_obj) = &self.acl.group_obj { > > > + bytes += mem::size_of::(); > > > + bytes += mem::size_of_val(group_obj); > > > + } > > > + if let Some(default) = &self.acl.default { > > > + bytes += mem::size_of::(); > > > + bytes += mem::size_of_val(default); > > > + } > > > + for acl_default_user in &self.acl.default_users { > > > + bytes += mem::size_of::(); > > > + bytes += mem::size_of_val(acl_default_user); > > > + } > > > + for acl_default_group in &self.acl.default_groups { > > > + bytes += mem::size_of::(); > > > + bytes += mem::size_of_val(acl_default_group); > > > + } > > > + if let Some(fcaps) = &self.fcaps { > > > + bytes += mem::size_of::(); > > > + bytes += mem::size_of_val(fcaps); > > > + } > > > + if let Some(quota_project_id) = &self.quota_project_id { > > > + bytes += mem::size_of::(); > > > + bytes += mem::size_of_val(quota_project_id); > > > + } > > > + > > > + bytes > > > + } > > > } > > > > > > impl From for Metadata { > > > -- > > > 2.39.2