From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id D6FA21FF13E for ; Fri, 06 Feb 2026 06:47:38 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 7B6FE1F0FC; Fri, 6 Feb 2026 06:48:10 +0100 (CET) From: Kefu Chai To: pve-devel@lists.proxmox.com Subject: [PATCH proxmox-i18n v2] add pgettext() and npgettext() support for context-aware translations Date: Fri, 6 Feb 2026 13:47:08 +0800 Message-ID: <20260206054707.1465261-2-k.chai@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260205101613.1067594-2-k.chai@proxmox.com> References: <20260205101613.1067594-2-k.chai@proxmox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1770356777285 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.140 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: VVMX7KFQN3PQZYUWWMKTJHJWWZS5BINN X-Message-ID-Hash: VVMX7KFQN3PQZYUWWMKTJHJWWZS5BINN X-MailFrom: k.chai@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: This commit adds message context (msgctxt) support to the JavaScript i18n tooling, enabling pgettext() and npgettext() functions. This allows different translations for the same English word based on usage context (e.g., "Monitor" for Ceph vs QEMU monitors). Changes: - po2js.pl: Add fnv31a_ctxt() to hash context+msgid combinations - po2js.pl: Change from load_file_ashash to load_file_asarray to properly handle multiple msgid entries with different contexts - po2js.pl: Add pgettext() and npgettext() JavaScript functions - Makefile: Add --keyword flag for xgettext to extract npgettext Implementation details: - Uses composite hash: hash(msgctxt + "\x04" + msgid) - \x04 is the standard gettext EOT separator - Backward compatible: messages without context use hash(msgid) - Same flat catalog structure, no nesting required JavaScript API: pgettext(context, msgid) - Translates message with context - Example: pgettext("ceph", "Monitor") vs pgettext("qemu", "Monitor") - Returns msgid if no translation found npgettext(context, singular, plural, n) - Translates message with context and plural support - Example: npgettext("file", "1 file", "{n} files", count) - Returns appropriate singular/plural form based on n PO file format: msgctxt "ceph" msgid "Monitor" msgstr "Monitor de Ceph" msgctxt "qemu" msgid "Monitor" msgstr "Monitor de QEMU" # Without context (traditional) msgid "Monitor" msgstr "Monitor" Extraction: xgettext recognizes pgettext() by default (no keyword needed). For npgettext(), we use --keyword=npgettext:1c,2,3 where: 1c = context (argument 1) 2 = singular msgid (argument 2) 3 = plural msgid (argument 3) Note: We cannot use '1c,2' like meson's i18n module does for NC_(). The NC_() macro from glib (used in meson's example) is context-only and does NOT support plural forms. From GTK documentation: "NC_(Context, String) - Only marks a string for translation, with context. Similar to N_(), but allows you to add a context." Our npgettext() function signature is npgettext(context, singular, plural, n), which requires both singular (arg 2) AND plural (arg 3) forms. Using '1c,2' would only extract the singular form and lose the plural, breaking npgettext functionality. Backward compatibility: - Existing PO files without msgctxt work unchanged - Existing gettext()/ngettext() calls work unchanged - Hash values for non-context messages are identical - Same catalog file format Tested with following steps: Modified pve-manager submodule: File: pve-manager/www/manager6/button/ConsoleButton.js - Changed gettext('Console') → pgettext('button', 'Console') - Added pgettext('console menu', 'noVNC/SPICE/xterm.js') - Added npgettext('console', '{0} console', '{0} consoles', count) Ran 'make update_pot' to verify xgettext extraction: Successfully extracted msgctxt entries to pve-manager.pot: msgctxt "button" msgid "Console" msgctxt "console menu" msgid "noVNC" msgctxt "console" msgid "{0} console" msgid_plural "{0} consoles" Created test PO file with context translations and verified po2js.pl generates correct context-aware hashes: - hash("button" + "\x04" + "Console") = 914449940 - hash("console menu" + "\x04" + "noVNC") = 418124897 - Different contexts produce unique hashes for same msgid All tests passed: - xgettext extracts msgctxt from JavaScript - po2js.pl processes msgctxt from PO files - Context-aware hashing produces unique digests - Generated JS includes pgettext/npgettext functions - Complete workflow: JS → POT → PO → JS catalog This matches the existing Rust pgettext/npgettext implementation in proxmox-yew-widget-toolkit, enabling consistent context-aware translations across the Proxmox ecosystem. Signed-off-by: Kefu Chai --- Makefile | 1 + po2js.pl | 69 ++++++++++++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 61 insertions(+), 9 deletions(-) diff --git a/Makefile b/Makefile index 86bd723..3feaee7 100644 --- a/Makefile +++ b/Makefile @@ -155,6 +155,7 @@ define potupdate --package-version="$(shell cd $(2);git rev-parse HEAD)" \ --msgid-bugs-address="" \ --copyright-holder="Copyright (C) Proxmox Server Solutions GmbH & the translation contributors." \ + --keyword=npgettext:1c,2,3 \ --output="$(1)".pot endef diff --git a/po2js.pl b/po2js.pl index 316c0bd..4b7b044 100755 --- a/po2js.pl +++ b/po2js.pl @@ -8,10 +8,6 @@ use Getopt::Long; use JSON; use Locale::PO; -# current limits: -# - we do not support plural. forms -# - no message content support - my $options = {}; GetOptions($options, 't=s', 'o=s', 'v=s') or die "unable to parse options\n"; @@ -34,6 +30,15 @@ sub fnv31a { return $hval & 0x7fffffff; } +# Hash function for messages with context +sub fnv31a_ctxt { + my ($msgctxt, $msgid) = @_; + return fnv31a($msgid) if !length($msgctxt); # Empty context = no context + # Use EOT character (0x04) as separator, standard in gettext + my $combined = $msgctxt . "\x04" . $msgid; + return fnv31a($combined); +} + my $catalog = {}; my $plurals_catalog = {}; @@ -41,11 +46,20 @@ my $nplurals = 2; my $plural_forms = "n!=1"; foreach my $filename (@ARGV) { - my $href = Locale::PO->load_file_ashash($filename) + my $aref = Locale::PO->load_file_asarray($filename) || die "unable to load '$filename'\n"; my $charset; - my $hpo = $href->{'""'} || die "no header"; + # Find header entry (msgid "") + my $hpo; + foreach my $po (@$aref) { + if ($po->msgid eq '""') { + $hpo = $po; + last; + } + } + die "no header" if !$hpo; + my $header = $hpo->dequote($hpo->msgstr); if ($header =~ m|^Content-Type:\s+text/plain;\s+charset=(\S+)$|im) { $charset = $1; @@ -58,8 +72,7 @@ foreach my $filename (@ARGV) { $plural_forms = $2; } - foreach my $k (keys %$href) { - my $po = $href->{$k}; + foreach my $po (@$aref) { next if $po->fuzzy(); # skip fuzzy entries my $ref = $po->reference(); @@ -76,9 +89,18 @@ foreach my $filename (@ARGV) { my $qmsgid_plural = decode($charset, $po->msgid_plural); my $msgid_plural = $po->dequote($qmsgid_plural); + # Extract message context if present + my $msgctxt = ''; + if (defined($po->msgctxt)) { + my $qmsgctxt = decode($charset, $po->msgctxt); + $msgctxt = $po->dequote($qmsgctxt); + } + next if !length($msgid) && !length($msgid_plural); # skip header - my $digest = fnv31a($msgid); + my $digest = length($msgctxt) > 0 + ? fnv31a_ctxt($msgctxt, $msgid) + : fnv31a($msgid); die "duplicate digest" if $catalog->{$digest}; @@ -150,6 +172,35 @@ function ngettext(singular, plural, n) { } return translation[msg_idx]; } + +function fnv31a_ctxt(context, text) { + // Use EOT character (0x04) as separator + var combined = context + "\\x04" + text; + return fnv31a(combined); +} + +function pgettext(context, msgid) { + var digest = fnv31a_ctxt(context, msgid); + var data = __proxmox_i18n_msgcat__[digest]; + if (!data) { + return msgid; // Return msgid (not context) as fallback + } + return data[0] || msgid; +} + +function npgettext(context, singular, plural, n) { + const msg_idx = Number($plural_forms); + const digest = fnv31a_ctxt(context, singular); + const translation = __proxmox_i18n_plurals_msgcat__[digest]; + if (!translation || msg_idx >= translation.length) { + if (n === 1) { + return singular; + } else { + return plural; + } + } + return translation[msg_idx]; +} __EOD if ($outfile) { -- 2.47.3