public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: "Laurențiu Leahu-Vlăducu" <l.leahu-vladucu@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [RFC PATCH pve-common 1/2] fix #3256: SectionConfig: ensure UTF-8 encoding for general configs
Date: Fri, 14 Feb 2025 16:40:38 +0100	[thread overview]
Message-ID: <20250214154040.159607-2-l.leahu-vladucu@proxmox.com> (raw)
In-Reply-To: <20250214154040.159607-1-l.leahu-vladucu@proxmox.com>

Previously, no decoding happened, meaning that Perl interpreted the
string as single bytes instead of Unicode code points when reading
the config. Note: while I would have preferred to decode the text
right after reading from the file, there are some Perl functions
like Digest::SHA::sha1_hex that expect bytes instead of UTF-8.

Also, config files are now explicitly encoded as UTF-8 when writing
the config, preventing issues the other way around.

For more information, please read:
https://perldoc.perl.org/perlunifaq#When-should-I-decode-or-encode?

Signed-off-by: Laurențiu Leahu-Vlăducu <l.leahu-vladucu@proxmox.com>
---
 src/PVE/SectionConfig.pm | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/src/PVE/SectionConfig.pm b/src/PVE/SectionConfig.pm
index 6a297d3..4e98c1c 100644
--- a/src/PVE/SectionConfig.pm
+++ b/src/PVE/SectionConfig.pm
@@ -98,6 +98,7 @@ use strict;
 use warnings;
 
 use Carp;
+use Encode qw(decode);
 use Digest::SHA;
 
 use PVE::Exception qw(raise_param_exc);
@@ -1091,7 +1092,7 @@ Only used for error messages and warnings, so it may also be something else.
 
 =item C<$raw>
 
-The raw content of C<$filename>.
+The raw content of C<$filename>. It is assumed to be encoded as UTF-8.
 
 =item C<$allow_unknown> (optional)
 
@@ -1185,11 +1186,12 @@ sub parse_config {
     $raw = '' if !defined($raw);
 
     my $digest = Digest::SHA::sha1_hex($raw);
+    my $utf8_text = Encode::decode('UTF-8', $raw);
 
     my $pri = 1;
 
     my $lineno = 0;
-    my @lines = split(/\n/, $raw);
+    my @lines = split(/\n/, $utf8_text);
     my $nextline = sub {
 	while (defined(my $line = shift @lines)) {
 	    $lineno++;
@@ -1430,6 +1432,8 @@ my sub format_config_line {
     $output = $class->write_config($filename, $cfg, $allow_unknown)
 
 Generates the output that should be written to the C<L<PVE::SectionConfig>> file.
+The output is encoded as bytes (encoded from UTF-8) that can be directly
+written to the config file.
 
 =over
 
@@ -1560,7 +1564,7 @@ sub write_config {
 	$out .= "$data\n";
     }
 
-    return $out;
+    return Encode::encode('UTF-8', $out);
 }
 
 sub assert_if_modified {
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

  reply	other threads:[~2025-02-14 15:41 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-02-14 15:40 [pve-devel] [RFC PATCH pve-storage/common] fix #3256: allow special characters in storage-related config files Laurențiu Leahu-Vlăducu
2025-02-14 15:40 ` Laurențiu Leahu-Vlăducu [this message]
2025-02-14 15:40 ` [pve-devel] [RFC PATCH pve-storage 1/1] fix #3256: Storage: PBS: ensure passwords are saved and loaded as UTF-8 Laurențiu Leahu-Vlăducu
2025-02-14 15:40 ` [pve-devel] [RFC PATCH pve-common 2/2] SectionConfig: add unit test for UTF-8 configs Laurențiu Leahu-Vlăducu
2025-02-17 10:15 ` [pve-devel] [RFC PATCH pve-storage/common] fix #3256: allow special characters in storage-related config files Fiona Ebner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250214154040.159607-2-l.leahu-vladucu@proxmox.com \
    --to=l.leahu-vladucu@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal