From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 0EF7569346 for ; Mon, 13 Sep 2021 10:07:29 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 04AD426FF6 for ; Mon, 13 Sep 2021 10:06:59 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 4DE1D26FED for ; Mon, 13 Sep 2021 10:06:58 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 034CB447A7 for ; Mon, 13 Sep 2021 10:06:58 +0200 (CEST) From: Dominik Csapak To: pbs-devel@lists.proxmox.com Date: Mon, 13 Sep 2021 10:06:57 +0200 Message-Id: <20210913080657.1348360-1-d.csapak@proxmox.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.407 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pbs-devel] [PATCH proxmox-backup] pbs-tools: zip: add conditional EFS flag to zip files X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2021 08:07:29 -0000 this flag marks the file names as 'UTF-8' encoded if they are valid UTF-8. By default, encoding of file names in zips are defined as code page 437, but we save the filenames as bytes (like in linux fs). For linux systems this neither would be a problem since most tools simply use the filenames as bytes, but for the zip utility under windows it's important since NTFS uses UTF-16 for file names. For filenames that are valid UTF-8, they are decoded as UTF-8 everywhere correctly (Linux as UTF-8 bytes, Windows as correct UTF-16 sequence) and for other filenames with a high bit set, it depends on the OS/Software what exactly happens. Some cases below: * Windows + Built-in/7zip: decoded as CP437 * Debian + zip: Bytes taken as-is * Debian + 7z: interpreted as Windows1252, decoded as UTF-8 Signed-off-by: Dominik Csapak --- changes from RFC: * set EFS flag conditionally when filename is valid UTF-8 * fix typo in const name * proper comments for consts pbs-tools/src/zip.rs | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/pbs-tools/src/zip.rs b/pbs-tools/src/zip.rs index 605480a8..62ebd4cf 100644 --- a/pbs-tools/src/zip.rs +++ b/pbs-tools/src/zip.rs @@ -34,6 +34,9 @@ const VERSION_MADE_BY: u16 = 0x032d; const ZIP64_EOCD_RECORD: u32 = 0x06064B50; const ZIP64_EOCD_LOCATOR: u32 = 0x07064B50; +const LFH_GENERAL_PURPOSE_FLAGS: u16 = 1 << 3; // we place crc32 in the data descriptor +const LFH_GPF_EFS_BIT: u16 = 1 << 11; // EFS, marks filename & comment as UTF-8 + // bits for time: // 0-4: day of the month (1-31) // 5-8: month: (1 = jan, etc.) @@ -200,8 +203,11 @@ pub struct ZipEntry { compressed_size: u64, offset: u64, is_file: bool, + is_utf8_filename: bool, } + + impl ZipEntry { /// Creates a new ZipEntry /// @@ -220,8 +226,11 @@ impl ZipEntry { relpath.push(""); // adds trailing slash } + let filename: OsString = relpath.into(); + let is_utf8_filename = filename.to_str().is_some(); + Self { - filename: relpath.into(), + filename, crc32: 0, mtime, mode, @@ -229,6 +238,15 @@ impl ZipEntry { compressed_size: 0, offset: 0, is_file, + is_utf8_filename, + } + } + + fn get_general_purpose_flags(&self) -> u16 { + if self.is_utf8_filename { + LFH_GENERAL_PURPOSE_FLAGS | LFH_GPF_EFS_BIT + } else { + LFH_GENERAL_PURPOSE_FLAGS } } @@ -249,7 +267,7 @@ impl ZipEntry { LocalFileHeader { signature: LOCAL_FH_SIG, version_needed: 0x2d, - flags: 1 << 3, + flags: self.get_general_purpose_flags(), compression: 0x8, time, date, @@ -332,7 +350,7 @@ impl ZipEntry { signature: CENTRAL_DIRECTORY_FH_SIG, version_made_by: VERSION_MADE_BY, version_needed: VERSION_NEEDED, - flags: 1 << 3, + flags: self.get_general_purpose_flags(), compression: 0x8, time, date, -- 2.30.2