From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id ECEBC1FF13F for ; Thu, 12 Mar 2026 06:33:14 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B570C573B; Thu, 12 Mar 2026 06:33:08 +0100 (CET) Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 Date: Thu, 12 Mar 2026 13:32:25 +0800 Message-Id: Subject: Re: [PATCH proxmox-i18n v2] add pgettext() and npgettext() support for context-aware translations From: "Kefu Chai" To: "Kefu Chai" , X-Mailer: aerc 0.20.0 References: <20260205101613.1067594-2-k.chai@proxmox.com> <20260206054707.1465261-2-k.chai@proxmox.com> In-Reply-To: <20260206054707.1465261-2-k.chai@proxmox.com> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1773293516839 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.727 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_MSPIKE_H2 0.001 Average reputation (+2) RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.408 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.819 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.903 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: PBBD7ZEJQKSKV7YRN5U6B3ZO4CYW6I3D X-Message-ID-Hash: PBBD7ZEJQKSKV7YRN5U6B3ZO4CYW6I3D X-MailFrom: k.chai@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Hi folks, For all related patchs in the projects including this one: - proxmox-backup: https://lore.proxmox.com/pve-devel/20260228060103.3267076= -2-k.chai@proxmox.com/ - pve-manager: https://lore.proxmox.com/pve-devel/20260228060333.3268576= -2-k.chai@proxmox.com/ - pmg-gui: https://lore.proxmox.com/pve-devel/20260228060757.3271160= -1-k.chai@proxmox.com/ - proxmox-biome: https://lore.proxmox.com/pve-devel/20260228061320.3279182= -1-k.chai@proxmox.com/ (applied) - proxmox-i18n: https://lore.proxmox.com/pve-devel/20260206054707.1465261= -2-k.chai@proxmox.com/ (will update once all the above changes land) cheers, On Fri Feb 6, 2026 at 1:47 PM CST, Kefu Chai wrote: > This commit adds message context (msgctxt) support to the JavaScript > i18n tooling, enabling pgettext() and npgettext() functions. This > allows different translations for the same English word based on usage > context (e.g., "Monitor" for Ceph vs QEMU monitors). > > Changes: > - po2js.pl: Add fnv31a_ctxt() to hash context+msgid combinations > - po2js.pl: Change from load_file_ashash to load_file_asarray to > properly handle multiple msgid entries with different contexts > - po2js.pl: Add pgettext() and npgettext() JavaScript functions > - Makefile: Add --keyword flag for xgettext to extract npgettext > > Implementation details: > - Uses composite hash: hash(msgctxt + "\x04" + msgid) > - \x04 is the standard gettext EOT separator > - Backward compatible: messages without context use hash(msgid) > - Same flat catalog structure, no nesting required > > JavaScript API: > pgettext(context, msgid) > - Translates message with context > - Example: pgettext("ceph", "Monitor") vs pgettext("qemu", "Monitor") > - Returns msgid if no translation found > > npgettext(context, singular, plural, n) > - Translates message with context and plural support > - Example: npgettext("file", "1 file", "{n} files", count) > - Returns appropriate singular/plural form based on n > > PO file format: > msgctxt "ceph" > msgid "Monitor" > msgstr "Monitor de Ceph" > > msgctxt "qemu" > msgid "Monitor" > msgstr "Monitor de QEMU" > > # Without context (traditional) > msgid "Monitor" > msgstr "Monitor" > > Extraction: > xgettext recognizes pgettext() by default (no keyword needed). > > For npgettext(), we use --keyword=3Dnpgettext:1c,2,3 where: > 1c =3D context (argument 1) > 2 =3D singular msgid (argument 2) > 3 =3D plural msgid (argument 3) > > Note: We cannot use '1c,2' like meson's i18n module does for NC_(). > The NC_() macro from glib (used in meson's example) is context-only > and does NOT support plural forms. From GTK documentation: > "NC_(Context, String) - Only marks a string for translation, with > context. Similar to N_(), but allows you to add a context." > > Our npgettext() function signature is npgettext(context, singular, > plural, n), which requires both singular (arg 2) AND plural (arg 3) > forms. Using '1c,2' would only extract the singular form and lose > the plural, breaking npgettext functionality. > > Backward compatibility: > - Existing PO files without msgctxt work unchanged > - Existing gettext()/ngettext() calls work unchanged > - Hash values for non-context messages are identical > - Same catalog file format > > Tested with following steps: > Modified pve-manager submodule: > File: pve-manager/www/manager6/button/ConsoleButton.js > - Changed gettext('Console') =E2=86=92 pgettext('button', 'Console') > - Added pgettext('console menu', 'noVNC/SPICE/xterm.js') > - Added npgettext('console', '{0} console', '{0} consoles', count) > > Ran 'make update_pot' to verify xgettext extraction: > Successfully extracted msgctxt entries to pve-manager.pot: > msgctxt "button" > msgid "Console" > > msgctxt "console menu" > msgid "noVNC" > > msgctxt "console" > msgid "{0} console" > msgid_plural "{0} consoles" > > Created test PO file with context translations and verified po2js.pl > generates correct context-aware hashes: > - hash("button" + "\x04" + "Console") =3D 914449940 > - hash("console menu" + "\x04" + "noVNC") =3D 418124897 > - Different contexts produce unique hashes for same msgid > > All tests passed: > - xgettext extracts msgctxt from JavaScript > - po2js.pl processes msgctxt from PO files > - Context-aware hashing produces unique digests > - Generated JS includes pgettext/npgettext functions > - Complete workflow: JS =E2=86=92 POT =E2=86=92 PO =E2=86=92 JS catalog > > This matches the existing Rust pgettext/npgettext implementation > in proxmox-yew-widget-toolkit, enabling consistent context-aware > translations across the Proxmox ecosystem. > > Signed-off-by: Kefu Chai > --- > Makefile | 1 + > po2js.pl | 69 ++++++++++++++++++++++++++++++++++++++++++++++++-------- > 2 files changed, 61 insertions(+), 9 deletions(-) > > diff --git a/Makefile b/Makefile > index 86bd723..3feaee7 100644 > --- a/Makefile > +++ b/Makefile > @@ -155,6 +155,7 @@ define potupdate > --package-version=3D"$(shell cd $(2);git rev-parse HEAD)" \ > --msgid-bugs-address=3D"" \ > --copyright-holder=3D"Copyright (C) Proxmox Server Solutions GmbH = & the translation contributors." \ > + --keyword=3Dnpgettext:1c,2,3 \ > --output=3D"$(1)".pot > endef > =20 > diff --git a/po2js.pl b/po2js.pl > index 316c0bd..4b7b044 100755 > --- a/po2js.pl > +++ b/po2js.pl > @@ -8,10 +8,6 @@ use Getopt::Long; > use JSON; > use Locale::PO; > =20 > -# current limits: > -# - we do not support plural. forms > -# - no message content support > - > my $options =3D {}; > GetOptions($options, 't=3Ds', 'o=3Ds', 'v=3Ds') or die "unable to parse = options\n"; > =20 > @@ -34,6 +30,15 @@ sub fnv31a { > return $hval & 0x7fffffff; > } > =20 > +# Hash function for messages with context > +sub fnv31a_ctxt { > + my ($msgctxt, $msgid) =3D @_; > + return fnv31a($msgid) if !length($msgctxt); # Empty context =3D no = context > + # Use EOT character (0x04) as separator, standard in gettext > + my $combined =3D $msgctxt . "\x04" . $msgid; > + return fnv31a($combined); > +} > + > my $catalog =3D {}; > my $plurals_catalog =3D {}; > =20 > @@ -41,11 +46,20 @@ my $nplurals =3D 2; > my $plural_forms =3D "n!=3D1"; > =20 > foreach my $filename (@ARGV) { > - my $href =3D Locale::PO->load_file_ashash($filename) > + my $aref =3D Locale::PO->load_file_asarray($filename) > || die "unable to load '$filename'\n"; > =20 > my $charset; > - my $hpo =3D $href->{'""'} || die "no header"; > + # Find header entry (msgid "") > + my $hpo; > + foreach my $po (@$aref) { > + if ($po->msgid eq '""') { > + $hpo =3D $po; > + last; > + } > + } > + die "no header" if !$hpo; > + > my $header =3D $hpo->dequote($hpo->msgstr); > if ($header =3D~ m|^Content-Type:\s+text/plain;\s+charset=3D(\S+)$|i= m) { > $charset =3D $1; > @@ -58,8 +72,7 @@ foreach my $filename (@ARGV) { > $plural_forms =3D $2; > } > =20 > - foreach my $k (keys %$href) { > - my $po =3D $href->{$k}; > + foreach my $po (@$aref) { > next if $po->fuzzy(); # skip fuzzy entries > my $ref =3D $po->reference(); > =20 > @@ -76,9 +89,18 @@ foreach my $filename (@ARGV) { > my $qmsgid_plural =3D decode($charset, $po->msgid_plural); > my $msgid_plural =3D $po->dequote($qmsgid_plural); > =20 > + # Extract message context if present > + my $msgctxt =3D ''; > + if (defined($po->msgctxt)) { > + my $qmsgctxt =3D decode($charset, $po->msgctxt); > + $msgctxt =3D $po->dequote($qmsgctxt); > + } > + > next if !length($msgid) && !length($msgid_plural); # skip header > =20 > - my $digest =3D fnv31a($msgid); > + my $digest =3D length($msgctxt) > 0 > + ? fnv31a_ctxt($msgctxt, $msgid) > + : fnv31a($msgid); > =20 > die "duplicate digest" if $catalog->{$digest}; > =20 > @@ -150,6 +172,35 @@ function ngettext(singular, plural, n) { > } > return translation[msg_idx]; > } > + > +function fnv31a_ctxt(context, text) { > + // Use EOT character (0x04) as separator > + var combined =3D context + "\\x04" + text; > + return fnv31a(combined); > +} > + > +function pgettext(context, msgid) { > + var digest =3D fnv31a_ctxt(context, msgid); > + var data =3D __proxmox_i18n_msgcat__[digest]; > + if (!data) { > + return msgid; // Return msgid (not context) as fallback > + } > + return data[0] || msgid; > +} > + > +function npgettext(context, singular, plural, n) { > + const msg_idx =3D Number($plural_forms); > + const digest =3D fnv31a_ctxt(context, singular); > + const translation =3D __proxmox_i18n_plurals_msgcat__[digest]; > + if (!translation || msg_idx >=3D translation.length) { > + if (n =3D=3D=3D 1) { > + return singular; > + } else { > + return plural; > + } > + } > + return translation[msg_idx]; > +} > __EOD > =20 > if ($outfile) {