[pve-devel] [PATCH cluster] cfs_write_file: fix accidental UTF-8 re-encoding

[pve-devel] [PATCH cluster] cfs_write_file: fix accidental UTF-8 re-encoding

[Fiona Ebner]

by correclty passing the $force_utf8 flag to
PVE::Tools::file_set_contents(). The idea was that only callers that
are ready will opt-in to the behavior.

When reading files with PVE::Tools::file_get_contents() or
ipcc_get_config(), the UTF-8 flag on the Perl string is not set, even
if the data is UTF-8. Such data would then be encoded a second time,
as reported in the community forum [0] and bug tracker [1].

[0]: https://forum.proxmox.com/threads/139282/
[1]: https://bugzilla.proxmox.com/show_bug.cgi?id=1909#c1

Fixes: 90c824b ("cluster fs: allow to force UTF-8 encoding for cfs_write_file")
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 src/PVE/Cluster.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
index 4c828c1..f899dbe 100644
--- a/src/PVE/Cluster.pm
+++ b/src/PVE/Cluster.pm
@@ -594,7 +594,7 @@ sub cfs_write_file {
 	$ci->{version} = undef;
     }
 
-    PVE::Tools::file_set_contents($fsname, $raw, undef, 1);
+    PVE::Tools::file_set_contents($fsname, $raw, undef, $force_utf8);
 }
 
 my $cfs_lock = sub {
-- 
2.39.2

Re: [pve-devel] [PATCH cluster] cfs_write_file: fix accidental UTF-8 re-encoding

[Filip Schauer]

Repeatedly creating users no longer re-encodes non-ASCII characters with
this patch applied.

Tested-by: Filip Schauer <f.schauer@proxmox.com>

On 09/01/2024 12:55, Fiona Ebner wrote:
> by correclty passing the $force_utf8 flag to
> PVE::Tools::file_set_contents(). The idea was that only callers that
> are ready will opt-in to the behavior.
>
> When reading files with PVE::Tools::file_get_contents() or
> ipcc_get_config(), the UTF-8 flag on the Perl string is not set, even
> if the data is UTF-8. Such data would then be encoded a second time,
> as reported in the community forum [0] and bug tracker [1].
>
> [0]: https://forum.proxmox.com/threads/139282/
> [1]: https://bugzilla.proxmox.com/show_bug.cgi?id=1909#c1
>
> Fixes: 90c824b ("cluster fs: allow to force UTF-8 encoding for cfs_write_file")
> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
> ---
>   src/PVE/Cluster.pm | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
> index 4c828c1..f899dbe 100644
> --- a/src/PVE/Cluster.pm
> +++ b/src/PVE/Cluster.pm
> @@ -594,7 +594,7 @@ sub cfs_write_file {
>   	$ci->{version} = undef;
>       }
>   
> -    PVE::Tools::file_set_contents($fsname, $raw, undef, 1);
> +    PVE::Tools::file_set_contents($fsname, $raw, undef, $force_utf8);
>   }
>   
>   my $cfs_lock = sub {

[pve-devel] applied: [PATCH cluster] cfs_write_file: fix accidental UTF-8 re-encoding

[Fabian Grünbichler]

with Filip's T-B, thanks!

On January 9, 2024 12:55 pm, Fiona Ebner wrote:
> by correclty passing the $force_utf8 flag to
> PVE::Tools::file_set_contents(). The idea was that only callers that
> are ready will opt-in to the behavior.
> 
> When reading files with PVE::Tools::file_get_contents() or
> ipcc_get_config(), the UTF-8 flag on the Perl string is not set, even
> if the data is UTF-8. Such data would then be encoded a second time,
> as reported in the community forum [0] and bug tracker [1].
> 
> [0]: https://forum.proxmox.com/threads/139282/
> [1]: https://bugzilla.proxmox.com/show_bug.cgi?id=1909#c1
> 
> Fixes: 90c824b ("cluster fs: allow to force UTF-8 encoding for cfs_write_file")
> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
> ---
>  src/PVE/Cluster.pm | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
> index 4c828c1..f899dbe 100644
> --- a/src/PVE/Cluster.pm
> +++ b/src/PVE/Cluster.pm
> @@ -594,7 +594,7 @@ sub cfs_write_file {
>  	$ci->{version} = undef;
>      }
>  
> -    PVE::Tools::file_set_contents($fsname, $raw, undef, 1);
> +    PVE::Tools::file_set_contents($fsname, $raw, undef, $force_utf8);
>  }
>  
>  my $cfs_lock = sub {
> -- 
> 2.39.2
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
>