Re: [PATCH common v3 2/5] section config: prepare for supporting UTF-8 encoded configurations

[Fiona Ebner <f.ebner@proxmox.com>]

Am 10.03.26 um 9:39 AM schrieb Dominik Csapak:
> On 3/9/26 4:43 PM, Fiona Ebner wrote:
>> Configurations registered as UTF-8 will be decoded after reading to
>> Perl's internal string format and then contain wide characters. The
>> Digest::SHA::sha1_hex() function croaks on wide characters, so encode
>> again before calling the function if there are wide characters.
> 
> 
> just to clarify, it will only contain wide characters if it contains
> code points bigger than 0xFF, but e.g. the symbol 'Ä' would be
> codepoint U+00C4 so even on decode it's smaller than 0xFF.
> (in utf-8 bytes it'd be 0xC3 0x84)
> 
> it does not play a role here since we only want to be consistent within
> the parser + api, but in some cases it can make a difference
> e.g. when we calculate the digest on a value that is always utf8 encoded.
> 
> i don't think this distinction warrants a new version, but
> if there is a new version, a better wording can maybe avoid confusion
> for a future reader of that commit message.

A very good point. I wonder if we should rather extend parse_config()
with $options too and pass along whether the file is an UTF-8 config
file when calling the parser. What do you think?

>>
>> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
>> ---
>>
>> Changes in v3:
>> * use strict 'UTF-8' encoding.
>>
>>   src/PVE/SectionConfig.pm | 7 ++++++-
>>   1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/src/PVE/SectionConfig.pm b/src/PVE/SectionConfig.pm
>> index 84ff81a..ed5a632 100644
>> --- a/src/PVE/SectionConfig.pm
>> +++ b/src/PVE/SectionConfig.pm
>> @@ -103,6 +103,7 @@ use warnings;
>>     use Carp;
>>   use Digest::SHA;
>> +use Encode qw(encode);
>>     use PVE::Exception qw(raise_param_exc);
>>   use PVE::JSONSchema qw(get_standard_option);
>> @@ -1214,7 +1215,11 @@ sub parse_config {
>>         $raw = '' if !defined($raw);
>>   -    my $digest = Digest::SHA::sha1_hex($raw);
>> +    my $bytes = $raw;
>> +    # Digest::SHA croaks on wide characters
>> +    $bytes = encode('UTF-8', $raw) if $raw =~ /[^\x00-\xFF]/;
>> +
>> +    my $digest = Digest::SHA::sha1_hex($bytes);
>>         my $pri = 1;
>>   
>