From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <d.csapak@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 2D17379966
for <pbs-devel@lists.proxmox.com>; Wed, 5 May 2021 09:53:34 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 16F3311B82
for <pbs-devel@lists.proxmox.com>; Wed, 5 May 2021 09:53:04 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[94.136.29.106])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS id 98C5A11B79
for <pbs-devel@lists.proxmox.com>; Wed, 5 May 2021 09:53:03 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 76DC242A20
for <pbs-devel@lists.proxmox.com>; Wed, 5 May 2021 09:53:03 +0200 (CEST)
From: Dominik Csapak <d.csapak@proxmox.com>
To: pbs-devel@lists.proxmox.com
Date: Wed, 5 May 2021 09:53:02 +0200
Message-Id: <20210505075302.14831-1-d.csapak@proxmox.com>
X-Mailer: git-send-email 2.20.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.240 Adjusted score from AWL reputation of From: address
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
information. [mod.rs]
Subject: [pbs-devel] [RFC PATCH proxmox-backup] api2/types ArchiveEntry:
fall back to iso 8859-1 if not valid utf8
X-BeenThere: pbs-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Backup Server development discussion
<pbs-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>,
<mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/>
List-Post: <mailto:pbs-devel@lists.proxmox.com>
List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>,
<mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Wed, 05 May 2021 07:53:34 -0000
In case a file name is not valid utf-8 we fall back to iso 8859-1. this works
neatly, since the byte sequence of visible (non-control) characters in iso
maps 1:1 to unicode codepoints which happens when we do 'u8 as char' in rust.
Theoretically the source could also be another iso variant (e.g. iso 8859-15),
but this is (at this point) impossible to detect because the bytes simply have
a different meaning.
If we want, we could somehow make this configurable (e.g. with a parameter),
but i'm not sure it is worth the effort
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/api2/types/mod.rs | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/src/api2/types/mod.rs b/src/api2/types/mod.rs
index e829f207..03660531 100644
--- a/src/api2/types/mod.rs
+++ b/src/api2/types/mod.rs
@@ -1379,10 +1379,17 @@ impl ArchiveEntry {
entry_type: Option<&DirEntryAttribute>,
size: Option<u64>,
) -> Self {
+ let name = filepath.split(|x| *x == b'/').last().unwrap();
+ let text = match String::from_utf8(name.to_vec()) {
+ Ok(text) => text,
+ Err(err) => { // fall back to iso-8859-1
+ err.as_bytes().iter().map(|&c| c as char).collect()
+ }
+ };
+
Self {
filepath: base64::encode(filepath),
- text: String::from_utf8_lossy(filepath.split(|x| *x == b'/').last().unwrap())
- .to_string(),
+ text,
entry_type: match entry_type {
Some(entry_type) => CatalogEntryType::from(entry_type).to_string(),
None => "v".to_owned(),
--
2.20.1