From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 2D17379966 for ; Wed, 5 May 2021 09:53:34 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 16F3311B82 for ; Wed, 5 May 2021 09:53:04 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 98C5A11B79 for ; Wed, 5 May 2021 09:53:03 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 76DC242A20 for ; Wed, 5 May 2021 09:53:03 +0200 (CEST) From: Dominik Csapak To: pbs-devel@lists.proxmox.com Date: Wed, 5 May 2021 09:53:02 +0200 Message-Id: <20210505075302.14831-1-d.csapak@proxmox.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.240 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [mod.rs] Subject: [pbs-devel] [RFC PATCH proxmox-backup] api2/types ArchiveEntry: fall back to iso 8859-1 if not valid utf8 X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 07:53:34 -0000 In case a file name is not valid utf-8 we fall back to iso 8859-1. this works neatly, since the byte sequence of visible (non-control) characters in iso maps 1:1 to unicode codepoints which happens when we do 'u8 as char' in rust. Theoretically the source could also be another iso variant (e.g. iso 8859-15), but this is (at this point) impossible to detect because the bytes simply have a different meaning. If we want, we could somehow make this configurable (e.g. with a parameter), but i'm not sure it is worth the effort Signed-off-by: Dominik Csapak --- src/api2/types/mod.rs | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/src/api2/types/mod.rs b/src/api2/types/mod.rs index e829f207..03660531 100644 --- a/src/api2/types/mod.rs +++ b/src/api2/types/mod.rs @@ -1379,10 +1379,17 @@ impl ArchiveEntry { entry_type: Option<&DirEntryAttribute>, size: Option, ) -> Self { + let name = filepath.split(|x| *x == b'/').last().unwrap(); + let text = match String::from_utf8(name.to_vec()) { + Ok(text) => text, + Err(err) => { // fall back to iso-8859-1 + err.as_bytes().iter().map(|&c| c as char).collect() + } + }; + Self { filepath: base64::encode(filepath), - text: String::from_utf8_lossy(filepath.split(|x| *x == b'/').last().unwrap()) - .to_string(), + text, entry_type: match entry_type { Some(entry_type) => CatalogEntryType::from(entry_type).to_string(), None => "v".to_owned(), -- 2.20.1