From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 070CC1FF0AD for ; Sun, 04 Oct 2026 16:32:13 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id E0FDC215E5; Sun, 04 Oct 2026 16:32:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dualfroz.com; s=dkim; t=1791124325; h=from:subject:date:message-id:to:mime-version: content-transfer-encoding:in-reply-to:references; bh=gjDsnadL82VCbEBzgUxfFY8Ou51Gc4INlZORB348OP8=; b=cmlu7KAi/3o3Q1/1xP4jGNzlxLKS/Semk121BFTorj2uewtHvYX5rdXFCq6n90YQWyW4sR Q52VateJ4Q0imT5BUh6S5rNjp8XFbJQIKZ0NJH9TF1lFWJgvFcw2lXU+75WagV7GyztAHs 2dqrjgR5tZ5DnxY6GpR7ukRD1RJLgsY2j11t6CLdv+u79rH8nDDPg+gpPiBS2Voj5Lsslw qvHPAdWhJ8QAkQhGzcpuI9mXhBeGit6XIQIaNtu8O8mu+BxO0LfVZOaRVI3AaNu7Wunclv H/MMIeFQawqyPbCRbELIBHTe43V2wumACR1N6fRrTyIteSFz0qU3ro+i7yLVOw== From: Michal Fox To: pve-devel@lists.proxmox.com Subject: [PATCH common 1/2] fix #6305: cli formatter: avoid encoding YAML output twice Date: Sun, 4 Oct 2026 14:32:01 +0000 Message-ID: <20261004143202.7-2-me@dualfroz.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261004143202.7-1-me@dualfroz.com> References: <20261004143202.7-1-me@dualfroz.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Last-TLS-Session-Version: TLSv1.3 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.034 Adjusted score from AWL reputation of From: address DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: JT42SBDMDDSDADSI6CS5MQVZJHXW3J4E X-Message-ID-Hash: JT42SBDMDDSDADSI6CS5MQVZJHXW3J4E X-MailFrom: me@dualfroz.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: YAML::XS::Dump already returns UTF-8 encoded octets, but its output still got encoded once more. This is a leftover from when the YAML output was generated by CPAN::Meta::YAML, which returns a character string. As a result, all non-ASCII characters got mangled with the 'yaml' output format, for example a cluster description with a German umlaut when running: pvesh get /cluster/options --output-format yaml Print the output of YAML::XS::Dump as is, like it is already done for the 'json' output format, and add a test for the encoding of both. Fixes: 99d0252 ("properly encode YAML via YAML::XS") Signed-off-by: Michal Fox --- src/PVE/CLIFormatter.pm | 3 ++- test/format_test.pl | 15 +++++++++++++++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/src/PVE/CLIFormatter.pm b/src/PVE/CLIFormatter.pm index ca4ee08..d93bed6 100644 --- a/src/PVE/CLIFormatter.pm +++ b/src/PVE/CLIFormatter.pm @@ -380,7 +380,8 @@ sub print_api_result { } if ($format eq 'yaml') { - print encode('UTF-8', YAML::XS::Dump($data)); + # Note: YAML::XS already returns UTF-8 encoded output + print YAML::XS::Dump($data); } elsif ($format eq 'json') { # Note: we always use utf8 encoding for json format print to_json($data, { utf8 => 1, allow_nonref => 1, canonical => 1 }) . "\n"; diff --git a/test/format_test.pl b/test/format_test.pl index 7b461ef..8ccfd5a 100755 --- a/test/format_test.pl +++ b/test/format_test.pl @@ -7,6 +7,7 @@ use lib '../src'; use PVE::JSONSchema; use PVE::CLIFormatter; +use Encode qw(decode); use Test::More; use Test::MockModule; @@ -53,4 +54,18 @@ foreach my $data (@$render_data) { is($actual, $expected, "string format '$renderer_name'"); } +# test that non-ASCII characters survive the structured output formats +my $unicode_data = { description => "Erkl\x{e4}rung" }; +for my $format (qw(json yaml)) { + open(my $fh, '>', \my $output) or die "failed to open in-memory file - $!\n"; + my $stdout = select($fh); + PVE::CLIFormatter::print_api_result( + $unicode_data, undef, undef, { 'output-format' => $format }, {}, + ); + select($stdout); + close($fh); + + like(decode('UTF-8', $output), qr/Erkl\x{e4}rung/, "output format '$format' is UTF-8 encoded"); +} + done_testing(); -- 2.43.0