[PATCH proxmox-i18n v1] add pgettext() and npgettext() support for context-aware translations

[PATCH proxmox-i18n v1] add pgettext() and npgettext() support for context-aware translations

[Kefu Chai]

This commit adds message context (msgctxt) support to the JavaScript
i18n tooling, enabling pgettext() and npgettext() functions. This
allows different translations for the same English word based on usage
context (e.g., "Save" in a menu vs "Save" for disk operations).

Changes:
- po2js.pl: Add fnv31a_ctxt() to hash context+msgid combinations
- po2js.pl: Change from load_file_ashash to load_file_asarray to
  properly handle multiple msgid entries with different contexts
- po2js.pl: Add pgettext() and npgettext() JavaScript functions
- Makefile: Add --keyword flags for xgettext to extract context

Implementation details:
- Uses composite hash: hash(msgctxt + "\x04" + msgid)
- \x04 is the standard gettext EOT separator
- Backward compatible: messages without context use hash(msgid)
- Same flat catalog structure, no nesting required

JavaScript API:
  pgettext(context, msgid)
    - Translates message with context
    - Example: pgettext("menu", "Save") vs pgettext("disk", "Save")
    - Returns msgid if no translation found

  npgettext(context, singular, plural, n)
    - Translates message with context and plural support
    - Example: npgettext("file", "1 file", "{n} files", count)
    - Returns appropriate singular/plural form based on n

PO file format:
  msgctxt "menu"
  msgid "Save"
  msgstr "Guardar"

  msgctxt "disk operation"
  msgid "Save"
  msgstr "Almacenar"

  # Without context (traditional)
  msgid "Save"
  msgstr "Salvar"

Extraction:
  xgettext now recognizes:
  --keyword=pgettext:1c,2        # 1c = context, 2 = msgid
  --keyword=npgettext:1c,2,3     # 1c = context, 2 = singular, 3 = plural

Backward compatibility:
- Existing PO files without msgctxt work unchanged
- Existing gettext()/ngettext() calls work unchanged
- Hash values for non-context messages are identical
- Same catalog file format

Tested with following steps:
Modified pve-manager submodule:
  File: pve-manager/www/manager6/button/ConsoleButton.js
  - Changed gettext('Console') → pgettext('button', 'Console')
  - Added pgettext('console menu', 'noVNC/SPICE/xterm.js')
  - Added npgettext('console', '{0} console', '{0} consoles', count)

Ran 'make update_pot' to verify xgettext extraction:
  Successfully extracted msgctxt entries to pve-manager.pot:
    msgctxt "button"
    msgid "Console"

    msgctxt "console menu"
    msgid "noVNC"

    msgctxt "console"
    msgid "{0} console"
    msgid_plural "{0} consoles"

Created test PO file with context translations and verified po2js.pl
generates correct context-aware hashes:
  - hash("button" + "\x04" + "Console") = 914449940
  - hash("console menu" + "\x04" + "noVNC") = 418124897
  - Different contexts produce unique hashes for same msgid

All tests passed:
  - xgettext extracts msgctxt from JavaScript
  - po2js.pl processes msgctxt from PO files
  - Context-aware hashing produces unique digests
  - Generated JS includes pgettext/npgettext functions
  - Complete workflow: JS → POT → PO → JS catalog

This matches the existing Rust pgettext/npgettext implementation
in proxmox-yew-widget-toolkit, enabling consistent context-aware
translations across the Proxmox ecosystem.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
---
 Makefile |  2 ++
 po2js.pl | 69 +++++++++++++++++++++++++++++++++++++++++++++++++-------
 2 files changed, 63 insertions(+), 8 deletions(-)

diff --git a/Makefile b/Makefile
index 86bd723..424918e 100644
--- a/Makefile
+++ b/Makefile
@@ -155,6 +155,8 @@ define potupdate
       --package-version="$(shell cd $(2);git rev-parse HEAD)" \
       --msgid-bugs-address="<support@proxmox.com>" \
       --copyright-holder="Copyright (C) Proxmox Server Solutions GmbH <support@proxmox.com> & the translation contributors." \
+      --keyword=pgettext:1c,2 \
+      --keyword=npgettext:1c,2,3 \
       --output="$(1)".pot
 endef
 
diff --git a/po2js.pl b/po2js.pl
index 316c0bd..11939f8 100755
--- a/po2js.pl
+++ b/po2js.pl
@@ -8,9 +8,7 @@ use Getopt::Long;
 use JSON;
 use Locale::PO;
 
-# current limits:
-# - we do not support plural. forms
-# - no message content support
+# Note: This script now supports plural forms and message context (msgctxt)
 
 my $options = {};
 GetOptions($options, 't=s', 'o=s', 'v=s') or die "unable to parse options\n";
@@ -34,6 +32,15 @@ sub fnv31a {
     return $hval & 0x7fffffff;
 }
 
+# Hash function for messages with context
+sub fnv31a_ctxt {
+    my ($msgctxt, $msgid) = @_;
+    return fnv31a($msgid) if !length($msgctxt);  # Empty context = no context
+    # Use EOT character (0x04) as separator, standard in gettext
+    my $combined = $msgctxt . "\x04" . $msgid;
+    return fnv31a($combined);
+}
+
 my $catalog = {};
 my $plurals_catalog = {};
 
@@ -41,11 +48,20 @@ my $nplurals = 2;
 my $plural_forms = "n!=1";
 
 foreach my $filename (@ARGV) {
-    my $href = Locale::PO->load_file_ashash($filename)
+    my $aref = Locale::PO->load_file_asarray($filename)
         || die "unable to load '$filename'\n";
 
     my $charset;
-    my $hpo = $href->{'""'} || die "no header";
+    # Find header entry (msgid "")
+    my $hpo;
+    foreach my $po (@$aref) {
+        if ($po->msgid eq '""') {
+            $hpo = $po;
+            last;
+        }
+    }
+    die "no header" if !$hpo;
+
     my $header = $hpo->dequote($hpo->msgstr);
     if ($header =~ m|^Content-Type:\s+text/plain;\s+charset=(\S+)$|im) {
         $charset = $1;
@@ -58,8 +74,7 @@ foreach my $filename (@ARGV) {
 	$plural_forms = $2;
     }
 
-    foreach my $k (keys %$href) {
-        my $po = $href->{$k};
+    foreach my $po (@$aref) {
         next if $po->fuzzy(); # skip fuzzy entries
         my $ref = $po->reference();
 
@@ -76,9 +91,18 @@ foreach my $filename (@ARGV) {
         my $qmsgid_plural = decode($charset, $po->msgid_plural);
         my $msgid_plural = $po->dequote($qmsgid_plural);
 
+        # Extract message context if present
+        my $msgctxt = '';
+        if (defined($po->msgctxt)) {
+            my $qmsgctxt = decode($charset, $po->msgctxt);
+            $msgctxt = $po->dequote($qmsgctxt);
+        }
+
         next if !length($msgid) && !length($msgid_plural); # skip header
 
-        my $digest = fnv31a($msgid);
+        my $digest = length($msgctxt) > 0
+            ? fnv31a_ctxt($msgctxt, $msgid)
+            : fnv31a($msgid);
 
         die "duplicate digest" if $catalog->{$digest};
 
@@ -150,6 +174,35 @@ function ngettext(singular, plural, n) {
     }
     return translation[msg_idx];
 }
+
+function fnv31a_ctxt(context, text) {
+    // Use EOT character (0x04) as separator
+    var combined = context + "\\x04" + text;
+    return fnv31a(combined);
+}
+
+function pgettext(context, msgid) {
+    var digest = fnv31a_ctxt(context, msgid);
+    var data = __proxmox_i18n_msgcat__[digest];
+    if (!data) {
+        return msgid;  // Return msgid (not context) as fallback
+    }
+    return data[0] || msgid;
+}
+
+function npgettext(context, singular, plural, n) {
+    const msg_idx = Number($plural_forms);
+    const digest = fnv31a_ctxt(context, singular);
+    const translation = __proxmox_i18n_plurals_msgcat__[digest];
+    if (!translation || msg_idx >= translation.length) {
+        if (n === 1) {
+            return singular;
+        } else {
+            return plural;
+        }
+    }
+    return translation[msg_idx];
+}
 __EOD
 
 if ($outfile) {
-- 
2.47.3

Re: [PATCH proxmox-i18n v1] add pgettext() and npgettext() support for context-aware translations

[Maximiliano Sandoval]

Kefu Chai <k.chai@proxmox.com> writes:

Thanks for this! I will need more time to test and review this, but some
small comments bellow.

> This commit adds message context (msgctxt) support to the JavaScript
> i18n tooling, enabling pgettext() and npgettext() functions. This
> allows different translations for the same English word based on usage
> context (e.g., "Save" in a menu vs "Save" for disk operations).

While this is an interesting possibility, it is more interesting to use
it when, for example, one use is a verb and another is for a noun, or
when there are two different nouns altogether, as for example "Monitor"
which is used indiscriminately for "Ceph Monitors" or "QEMU Monitors" in
the web UI.

> Changes:
> - po2js.pl: Add fnv31a_ctxt() to hash context+msgid combinations
> - po2js.pl: Change from load_file_ashash to load_file_asarray to
>   properly handle multiple msgid entries with different contexts
> - po2js.pl: Add pgettext() and npgettext() JavaScript functions
> - Makefile: Add --keyword flags for xgettext to extract context
>
> Implementation details:
> - Uses composite hash: hash(msgctxt + "\x04" + msgid)
> - \x04 is the standard gettext EOT separator
> - Backward compatible: messages without context use hash(msgid)
> - Same flat catalog structure, no nesting required
>
> JavaScript API:
>   pgettext(context, msgid)
>     - Translates message with context
>     - Example: pgettext("menu", "Save") vs pgettext("disk", "Save")
>     - Returns msgid if no translation found
>
>   npgettext(context, singular, plural, n)
>     - Translates message with context and plural support
>     - Example: npgettext("file", "1 file", "{n} files", count)
>     - Returns appropriate singular/plural form based on n
>
> PO file format:
>   msgctxt "menu"
>   msgid "Save"
>   msgstr "Guardar"
>
>   msgctxt "disk operation"
>   msgid "Save"
>   msgstr "Almacenar"
>
>   # Without context (traditional)
>   msgid "Save"
>   msgstr "Salvar"
>
> Extraction:
>   xgettext now recognizes:
>   --keyword=pgettext:1c,2        # 1c = context, 2 = msgid
>   --keyword=npgettext:1c,2,3     # 1c = context, 2 = singular, 3 = plural
>
> Backward compatibility:
> - Existing PO files without msgctxt work unchanged
> - Existing gettext()/ngettext() calls work unchanged
> - Hash values for non-context messages are identical
> - Same catalog file format
>
> Tested with following steps:
> Modified pve-manager submodule:
>   File: pve-manager/www/manager6/button/ConsoleButton.js
>   - Changed gettext('Console') → pgettext('button', 'Console')
>   - Added pgettext('console menu', 'noVNC/SPICE/xterm.js')
>   - Added npgettext('console', '{0} console', '{0} consoles', count)
>
> Ran 'make update_pot' to verify xgettext extraction:
>   Successfully extracted msgctxt entries to pve-manager.pot:
>     msgctxt "button"
>     msgid "Console"
>
>     msgctxt "console menu"
>     msgid "noVNC"
>
>     msgctxt "console"
>     msgid "{0} console"
>     msgid_plural "{0} consoles"
>
> Created test PO file with context translations and verified po2js.pl
> generates correct context-aware hashes:
>   - hash("button" + "\x04" + "Console") = 914449940
>   - hash("console menu" + "\x04" + "noVNC") = 418124897
>   - Different contexts produce unique hashes for same msgid
>
> All tests passed:
>   - xgettext extracts msgctxt from JavaScript
>   - po2js.pl processes msgctxt from PO files
>   - Context-aware hashing produces unique digests
>   - Generated JS includes pgettext/npgettext functions
>   - Complete workflow: JS → POT → PO → JS catalog
>
> This matches the existing Rust pgettext/npgettext implementation
> in proxmox-yew-widget-toolkit, enabling consistent context-aware
> translations across the Proxmox ecosystem.
>
> Signed-off-by: Kefu Chai <k.chai@proxmox.com>
> ---
>  Makefile |  2 ++
>  po2js.pl | 69 +++++++++++++++++++++++++++++++++++++++++++++++++-------
>  2 files changed, 63 insertions(+), 8 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 86bd723..424918e 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -155,6 +155,8 @@ define potupdate
>        --package-version="$(shell cd $(2);git rev-parse HEAD)" \
>        --msgid-bugs-address="<support@proxmox.com>" \
>        --copyright-holder="Copyright (C) Proxmox Server Solutions GmbH <support@proxmox.com> & the translation contributors." \
> +      --keyword=pgettext:1c,2 \

This one is correct, however I think that xgettext by default recognizes
all {p,np,n,}gettext as keywords. This would be useful, if you want to
define custom fns for translations, e.g. tr(), i18n(), _() or C_() the
later two used as an example by glib, e.g.
https://docs.gtk.org/glib/i18n.html.

> +      --keyword=npgettext:1c,2,3 \

>From meson's glib integration in its i18n module
(https://github.com/mesonbuild/meson/blob/61c5859b771eb938f20935cc3da54a9075d24c64/mesonbuild/modules/i18n.py#L105C1-L105C30)
I think this should be "1c,2" instead, however as I said it would be
preferable to not redefine the default keywords.


>        --output="$(1)".pot
>  endef
>  
> diff --git a/po2js.pl b/po2js.pl
> index 316c0bd..11939f8 100755
> --- a/po2js.pl
> +++ b/po2js.pl
> @@ -8,9 +8,7 @@ use Getopt::Long;
>  use JSON;
>  use Locale::PO;
>  
> -# current limits:
> -# - we do not support plural. forms
> -# - no message content support
> +# Note: This script now supports plural forms and message context (msgctxt)

I would just remove this note (or fill it with other limitations, if any
is known).

>  
>  my $options = {};
>  GetOptions($options, 't=s', 'o=s', 'v=s') or die "unable to parse options\n";
> @@ -34,6 +32,15 @@ sub fnv31a {
>      return $hval & 0x7fffffff;
>  }
> ...
> ...

-- 
Maximiliano