[pve-devel] [RFC common/cluster] cluster files: support registering UTF-8 configuration file

[pve-devel] [RFC common/cluster] cluster files: support registering UTF-8 configuration file

[Fiona Ebner]

A configuration file registered as UTF-8 will be automatically decoded
from UTF-8 to Perl's internal string format after reading and encoded
in the other direction before writing.

common:

Fiona Ebner (1):
  section config: prepare for supporting UTF-8 encoded configurations

 src/PVE/SectionConfig.pm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

cluster:

Fiona Ebner (1):
  cluster files: support registering UTF-8 configuration file

 src/PVE/Cluster.pm | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [RFC common 1/2] section config: prepare for supporting UTF-8 encoded configurations

[Fiona Ebner]

Configurations registered as UTF-8 will be decoded after reading to
Perl's internal string format and then contain wide characters. The
Digest::SHA::sha1_hex() function croaks on wide characters, so encode
again before calling the function if there are wide characters.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 src/PVE/SectionConfig.pm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/src/PVE/SectionConfig.pm b/src/PVE/SectionConfig.pm
index 6a297d3..e7dbd4c 100644
--- a/src/PVE/SectionConfig.pm
+++ b/src/PVE/SectionConfig.pm
@@ -99,6 +99,7 @@ use warnings;
 
 use Carp;
 use Digest::SHA;
+use Encode;
 
 use PVE::Exception qw(raise_param_exc);
 use PVE::JSONSchema qw(get_standard_option);
@@ -1184,7 +1185,10 @@ sub parse_config {
 
     $raw = '' if !defined($raw);
 
-    my $digest = Digest::SHA::sha1_hex($raw);
+    my $bytes = $raw;
+    $bytes = encode('utf8', $raw) if $raw =~ /[^\x00-\xFF]/; # Digest::SHA croaks on wide characters
+
+    my $digest = Digest::SHA::sha1_hex($bytes);
 
     my $pri = 1;
 
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [RFC cluster 2/2] cluster files: support registering UTF-8 configuration file

[Fiona Ebner]

A configuration file registered as UTF-8 will be automatically decoded
from UTF-8 to Perl's internal string format after reading and encoded
in the other direction before writing.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

Tested a bit with the virtiofs directory mapping config opting into
this, which could be an initial user of this feature.

 src/PVE/Cluster.pm | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
index b9311c7..fe1601e 100644
--- a/src/PVE/Cluster.pm
+++ b/src/PVE/Cluster.pm
@@ -519,7 +519,7 @@ sub verify_token {
 my $file_info = {};
 
 sub cfs_register_file {
-    my ($filename, $parser, $writer) = @_;
+    my ($filename, $parser, $writer, $utf8) = @_;
 
     $observed->{$filename} || die "unknown file '$filename'";
 
@@ -528,11 +528,12 @@ sub cfs_register_file {
     $file_info->{$filename} = {
 	parser => $parser,
 	writer => $writer,
+	utf8 => $utf8,
     };
 }
 
 my $ccache_read = sub {
-    my ($filename, $parser, $version) = @_;
+    my ($filename, $parser, $version, $utf8) = @_;
 
     $ccache->{$filename} = {} if !$ccache->{$filename};
 
@@ -542,6 +543,7 @@ my $ccache_read = sub {
 	# we always call the parser, even when the file does not exist
 	# (in that case $data is undef)
 	my $data = get_config($filename);
+	$data = decode('utf8', $data) if $utf8;
 	$ci->{data} = &$parser("/etc/pve/$filename", $data);
 	$ci->{version} = $version;
     }
@@ -579,7 +581,7 @@ sub cfs_read_file {
     my ($version, $info) = cfs_file_version($filename);
     my $parser = $info->{parser};
 
-    return &$ccache_read($filename, $parser, $version);
+    return &$ccache_read($filename, $parser, $version, $info->{utf8});
 }
 
 sub cfs_write_file {
@@ -597,6 +599,8 @@ sub cfs_write_file {
 	$ci->{version} = undef;
     }
 
+    $force_utf8 = 1 if $info->{utf8};
+
     PVE::Tools::file_set_contents($fsname, $raw, undef, $force_utf8);
 }
 
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel