From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 33DAE6A733 for ; Mon, 15 Mar 2021 12:21:21 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 30AD3202E0 for ; Mon, 15 Mar 2021 12:21:21 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id CD590202CA for ; Mon, 15 Mar 2021 12:21:19 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 97F0C420B8 for ; Mon, 15 Mar 2021 12:21:19 +0100 (CET) From: Dominik Csapak To: pbs-devel@lists.proxmox.com Date: Mon, 15 Mar 2021 12:21:17 +0100 Message-Id: <20210315112118.13641-2-d.csapak@proxmox.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210315112118.13641-1-d.csapak@proxmox.com> References: <20210315112118.13641-1-d.csapak@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.188 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pbs-devel] [RFC PATCH proxmox-backup 2/3] tools/zip: only add zip64 field when necessary X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2021 11:21:21 -0000 if neither offset nor size exceeds 32bit, do not add the zip64 extension field Signed-off-by: Dominik Csapak --- it does not harm normally, but can blow up the size of the zip a bit if one has many small files src/tools/zip.rs | 52 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 35 insertions(+), 17 deletions(-) diff --git a/src/tools/zip.rs b/src/tools/zip.rs index cca2e766..55f2a24a 100644 --- a/src/tools/zip.rs +++ b/src/tools/zip.rs @@ -301,10 +301,26 @@ impl ZipEntry { let filename_len = filename.len(); let header_size = size_of::(); let zip_field_size = size_of::(); - let size: usize = header_size + filename_len + zip_field_size; + let mut size: usize = header_size + filename_len; let (date, time) = epoch_to_dos(self.mtime); + let (compressed_size, uncompressed_size, offset, need_zip64) = if self.compressed_size + >= (u32::MAX as u64) + || self.uncompressed_size >= (u32::MAX as u64) + || self.offset >= (u32::MAX as u64) + { + size += zip_field_size; + (0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, true) + } else { + ( + self.compressed_size as u32, + self.uncompressed_size as u32, + self.offset as u32, + false, + ) + }; + write_struct( &mut buf, CentralDirectoryFileHeader { @@ -316,33 +332,35 @@ impl ZipEntry { time, date, crc32: self.crc32, - compressed_size: 0xFFFFFFFF, - uncompressed_size: 0xFFFFFFFF, + compressed_size, + uncompressed_size, filename_len: filename_len as u16, - extra_field_len: zip_field_size as u16, + extra_field_len: if need_zip64 { zip_field_size as u16 } else { 0 }, comment_len: 0, start_disk: 0, internal_flags: 0, external_flags: (self.mode as u32) << 16 | (!self.is_file as u32) << 4, - offset: 0xFFFFFFFF, + offset, }, ) .await?; buf.write_all(filename).await?; - write_struct( - &mut buf, - Zip64FieldWithOffset { - field_type: 1, - field_size: 3 * 8 + 4, - uncompressed_size: self.uncompressed_size, - compressed_size: self.compressed_size, - offset: self.offset, - start_disk: 0, - }, - ) - .await?; + if need_zip64 { + write_struct( + &mut buf, + Zip64FieldWithOffset { + field_type: 1, + field_size: 3 * 8 + 4, + uncompressed_size: self.uncompressed_size, + compressed_size: self.compressed_size, + offset: self.offset, + start_disk: 0, + }, + ) + .await?; + } Ok(size) } -- 2.20.1