From: "Laurențiu Leahu-Vlăducu" <l.leahu-vladucu@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [RFC PATCH pve-common 1/2] fix #3256: SectionConfig: ensure UTF-8 encoding for general configs
Date: Fri, 14 Feb 2025 16:40:38 +0100 [thread overview]
Message-ID: <20250214154040.159607-2-l.leahu-vladucu@proxmox.com> (raw)
In-Reply-To: <20250214154040.159607-1-l.leahu-vladucu@proxmox.com>
Previously, no decoding happened, meaning that Perl interpreted the
string as single bytes instead of Unicode code points when reading
the config. Note: while I would have preferred to decode the text
right after reading from the file, there are some Perl functions
like Digest::SHA::sha1_hex that expect bytes instead of UTF-8.
Also, config files are now explicitly encoded as UTF-8 when writing
the config, preventing issues the other way around.
For more information, please read:
https://perldoc.perl.org/perlunifaq#When-should-I-decode-or-encode?
Signed-off-by: Laurențiu Leahu-Vlăducu <l.leahu-vladucu@proxmox.com>
---
src/PVE/SectionConfig.pm | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/src/PVE/SectionConfig.pm b/src/PVE/SectionConfig.pm
index 6a297d3..4e98c1c 100644
--- a/src/PVE/SectionConfig.pm
+++ b/src/PVE/SectionConfig.pm
@@ -98,6 +98,7 @@ use strict;
use warnings;
use Carp;
+use Encode qw(decode);
use Digest::SHA;
use PVE::Exception qw(raise_param_exc);
@@ -1091,7 +1092,7 @@ Only used for error messages and warnings, so it may also be something else.
=item C<$raw>
-The raw content of C<$filename>.
+The raw content of C<$filename>. It is assumed to be encoded as UTF-8.
=item C<$allow_unknown> (optional)
@@ -1185,11 +1186,12 @@ sub parse_config {
$raw = '' if !defined($raw);
my $digest = Digest::SHA::sha1_hex($raw);
+ my $utf8_text = Encode::decode('UTF-8', $raw);
my $pri = 1;
my $lineno = 0;
- my @lines = split(/\n/, $raw);
+ my @lines = split(/\n/, $utf8_text);
my $nextline = sub {
while (defined(my $line = shift @lines)) {
$lineno++;
@@ -1430,6 +1432,8 @@ my sub format_config_line {
$output = $class->write_config($filename, $cfg, $allow_unknown)
Generates the output that should be written to the C<L<PVE::SectionConfig>> file.
+The output is encoded as bytes (encoded from UTF-8) that can be directly
+written to the config file.
=over
@@ -1560,7 +1564,7 @@ sub write_config {
$out .= "$data\n";
}
- return $out;
+ return Encode::encode('UTF-8', $out);
}
sub assert_if_modified {
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
next prev parent reply other threads:[~2025-02-14 15:41 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-02-14 15:40 [pve-devel] [RFC PATCH pve-storage/common] fix #3256: allow special characters in storage-related config files Laurențiu Leahu-Vlăducu
2025-02-14 15:40 ` Laurențiu Leahu-Vlăducu [this message]
2025-02-14 15:40 ` [pve-devel] [RFC PATCH pve-storage 1/1] fix #3256: Storage: PBS: ensure passwords are saved and loaded as UTF-8 Laurențiu Leahu-Vlăducu
2025-02-14 15:40 ` [pve-devel] [RFC PATCH pve-common 2/2] SectionConfig: add unit test for UTF-8 configs Laurențiu Leahu-Vlăducu
2025-02-17 10:15 ` [pve-devel] [RFC PATCH pve-storage/common] fix #3256: allow special characters in storage-related config files Fiona Ebner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250214154040.159607-2-l.leahu-vladucu@proxmox.com \
--to=l.leahu-vladucu@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal