From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 5A92CE794 for ; Wed, 19 Jul 2023 10:40:11 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 409B83F91 for ; Wed, 19 Jul 2023 10:40:11 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 19 Jul 2023 10:40:10 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id EA9C640E55 for ; Wed, 19 Jul 2023 10:40:09 +0200 (CEST) Message-ID: Date: Wed, 19 Jul 2023 10:40:09 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: de-AT, en-US To: Dominik Csapak , Proxmox VE development discussion , Wolfgang Bumiller , Maximiliano Sandoval References: <20230717150051.710464-1-l.wagner@proxmox.com> From: Lukas Wagner In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.102 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [test.pl, notify.rs] WEIRD_PORT 0.001 Uses non-standard port number for HTTP Subject: Re: [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jul 2023 08:40:11 -0000 Hi again, On 7/18/23 14:34, Dominik Csapak wrote: > * i found one bug, but not quite sure yet where it comes from exactly, >   putting in emojis into a field (e.g. a comment or author) it's accepted, >   but editing a different entry fails with: > > --->8--- > could not serialize configuration: writing 'notifications.cfg' failed: detected unexpected control character in section 'testgroup' key 'comment' (500) > ---8<--- > > not sure where the utf-8 info gets lost. (or we could limit all fields to ascii?) > such a notification target still works AFAICT (but if set as e.g. the author it's > probably the wrong value) > > (i used 😀 as a test) So I investigated a bit and found a minimal reproducer. Turns out it's an encoding issue in the FFI interface (perl->rust). Let's assume that we have the following exported function in the pve-rs bindings: #[export] fn test_emoji(name: &str) { dbg!(&name); } use PVE::RS::Notify; my $str = "😊"; PVE::RS::Notify::test_emoji($str); root@pve:~# perl test.pl [src/notify.rs:562] &name = "ð\u{9f}\u{98}\u{8a}" To me it looks a bit like a UTF-16/UTF-8 mixup: ð = 0x00F0 in UTF16 😊 = 0xF0 0x9F 0x98 0x8A in UTF-8 The issue can be fixed by doing a `$str = encode('utf-8', $str);` before calling `test_emoji`. However, I think this should be probably handled automagically by the perlmod bindings, if at all possible? @Wolfgang, what are your thoughts about this? Maximiliano said he is going to take a look at the perlmod code, but if you have any idea about where to fix this issue, then this would probably be helpful to him. -- - Lukas