From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 73C921FF136 for ; Mon, 09 Mar 2026 14:07:09 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id E4A49333EB; Mon, 9 Mar 2026 14:07:00 +0100 (CET) From: Fiona Ebner To: pve-devel@lists.proxmox.com Subject: [PATCH-SERIES qemu-server/storage/common v2 0/4] fix UTF-8 handling for PBS_PASSWORD Date: Mon, 9 Mar 2026 14:06:39 +0100 Message-ID: <20260309130653.90674-1-f.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1773061584030 X-SPAM-LEVEL: Spam detection results: 0 AWL -1.071 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_MSPIKE_H2 0.001 Average reputation (+2) RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.408 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.819 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.903 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: T3FVOVF6EHXMLNCZRCB6RRMUMFWSGZHK X-Message-ID-Hash: T3FVOVF6EHXMLNCZRCB6RRMUMFWSGZHK X-MailFrom: f.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Changes in v2: * Improve commit message - the ENV issues only happen when Perl's internal representation is not valid UTF-8 itself. * Rebase on latest master. There are multiple issues with UTF-8 password handling for PBS backup: 1. As reported in the community forum, PBS backup of VMs would not work when the password contained a multi-byte UTF-8 character. The first patch fixes this by properly encoding the JSON sent via QMP. 2. The PBS password is saved as UTF-8 and decoded to Perl's internal string representation upon reading from the password file. When the password is not valid UTF-8 in Perl's internal representation, which for example happens with a password like 'ääääöööö', backing up a diskless VM would fail with: > Error: error building client for repository XXX - > PBS_PASSWORD contains bad characters 3. The same error would occur for uploading the log file after backup, as well as extracting the configuration file from backup. 4. Restoring would fail with: > restore failed: invalid utf-8 sequence of 1 bytes from index 0 Fix issues 2.-4. by properly encoding the value for the PBS_PASSWORD environment variable value again as UTF-8. 5. For PMG, using passwords that would be auto-encoded by Perl as either ASCII or UTF-8 already worked, but other encodings would not, for example ISO-8859 would result in: > proxmox-backup-client failed: Error: error building client for > repository latin@pbs@10.10.100.180:8007:bigone - PBS_PASSWORD > contains bad characters (500) Follow pve-storage commit 5245e04 ("fix #5181: pbs: store and read passwords as unicode") and align the behavior of the storage plugin and pbs client module. qemu-server: Fiona Ebner (2): qmp client: encode JSON as UTF-8 to fix PBS backup when password contains multi-byte UTF-8 pbs: properly encode PBS password as UTF-8 when setting the environment variable src/PVE/QMPClient.pm | 14 ++++++++------ src/PVE/QemuServer.pm | 4 ++++ src/PVE/VZDump/QemuServer.pm | 4 ++++ 3 files changed, 16 insertions(+), 6 deletions(-) storage: Fiona Ebner (1): pbs plugin: raw client command: properly encode PBS password as UTF-8 when setting the environment variable src/PVE/Storage/PBSPlugin.pm | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) common: Fiona Ebner (1): pbs client: allow using password that would be auto-encoded as neither ASCII nor UTF-8 src/PVE/PBSClient.pm | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) Summary over all repositories: 5 files changed, 32 insertions(+), 11 deletions(-) -- Generated by git-murpp 0.5.0