Re: [PATCH common v3 2/5] section config: prepare for supporting UTF-8 encoded configurations

[Dominik Csapak <d.csapak@proxmox.com>]

On 3/10/26 10:18 AM, Fiona Ebner wrote:
> Am 10.03.26 um 9:39 AM schrieb Dominik Csapak:
>> On 3/9/26 4:43 PM, Fiona Ebner wrote:
>>> Configurations registered as UTF-8 will be decoded after reading to
>>> Perl's internal string format and then contain wide characters. The
>>> Digest::SHA::sha1_hex() function croaks on wide characters, so encode
>>> again before calling the function if there are wide characters.
>>
>>
>> just to clarify, it will only contain wide characters if it contains
>> code points bigger than 0xFF, but e.g. the symbol 'Ä' would be
>> codepoint U+00C4 so even on decode it's smaller than 0xFF.
>> (in utf-8 bytes it'd be 0xC3 0x84)
>>
>> it does not play a role here since we only want to be consistent within
>> the parser + api, but in some cases it can make a difference
>> e.g. when we calculate the digest on a value that is always utf8 encoded.
>>
>> i don't think this distinction warrants a new version, but
>> if there is a new version, a better wording can maybe avoid confusion
>> for a future reader of that commit message.
> 
> A very good point. I wonder if we should rather extend parse_config()
> with $options too and pass along whether the file is an UTF-8 config
> file when calling the parser. What do you think?

It would be a better way to define/detect utf8 files than checking
for codepoints > 0xFF IMO. Not sure how much work it is though
to wire that up? In practice it won't make much difference, as long
as we use the same logic/bytes everywhere we check the digest.

> 
>>>
>>> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
>>> ---
>>>
>>> Changes in v3:
>>> * use strict 'UTF-8' encoding.
>>>
>>>    src/PVE/SectionConfig.pm | 7 ++++++-
>>>    1 file changed, 6 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/src/PVE/SectionConfig.pm b/src/PVE/SectionConfig.pm
>>> index 84ff81a..ed5a632 100644
>>> --- a/src/PVE/SectionConfig.pm
>>> +++ b/src/PVE/SectionConfig.pm
>>> @@ -103,6 +103,7 @@ use warnings;
>>>      use Carp;
>>>    use Digest::SHA;
>>> +use Encode qw(encode);
>>>      use PVE::Exception qw(raise_param_exc);
>>>    use PVE::JSONSchema qw(get_standard_option);
>>> @@ -1214,7 +1215,11 @@ sub parse_config {
>>>          $raw = '' if !defined($raw);
>>>    -    my $digest = Digest::SHA::sha1_hex($raw);
>>> +    my $bytes = $raw;
>>> +    # Digest::SHA croaks on wide characters
>>> +    $bytes = encode('UTF-8', $raw) if $raw =~ /[^\x00-\xFF]/;
>>> +
>>> +    my $digest = Digest::SHA::sha1_hex($bytes);
>>>          my $pri = 1;
>>>    
>>
>