Re: [PATCH proxmox-i18n v2] add pgettext() and npgettext() support for context-aware translations

["Kefu Chai" <k.chai@proxmox.com>]

Hi folks,

For all related patchs in the projects including this one:

- proxmox-backup: https://lore.proxmox.com/pve-devel/20260228060103.3267076-2-k.chai@proxmox.com/
- pve-manager:    https://lore.proxmox.com/pve-devel/20260228060333.3268576-2-k.chai@proxmox.com/
- pmg-gui:        https://lore.proxmox.com/pve-devel/20260228060757.3271160-1-k.chai@proxmox.com/
- proxmox-biome:  https://lore.proxmox.com/pve-devel/20260228061320.3279182-1-k.chai@proxmox.com/ (applied)
- proxmox-i18n:   https://lore.proxmox.com/pve-devel/20260206054707.1465261-2-k.chai@proxmox.com/ (will update once all the above changes land)

cheers,


On Fri Feb 6, 2026 at 1:47 PM CST, Kefu Chai wrote:
> This commit adds message context (msgctxt) support to the JavaScript
> i18n tooling, enabling pgettext() and npgettext() functions. This
> allows different translations for the same English word based on usage
> context (e.g., "Monitor" for Ceph vs QEMU monitors).
>
> Changes:
> - po2js.pl: Add fnv31a_ctxt() to hash context+msgid combinations
> - po2js.pl: Change from load_file_ashash to load_file_asarray to
>   properly handle multiple msgid entries with different contexts
> - po2js.pl: Add pgettext() and npgettext() JavaScript functions
> - Makefile: Add --keyword flag for xgettext to extract npgettext
>
> Implementation details:
> - Uses composite hash: hash(msgctxt + "\x04" + msgid)
> - \x04 is the standard gettext EOT separator
> - Backward compatible: messages without context use hash(msgid)
> - Same flat catalog structure, no nesting required
>
> JavaScript API:
>   pgettext(context, msgid)
>     - Translates message with context
>     - Example: pgettext("ceph", "Monitor") vs pgettext("qemu", "Monitor")
>     - Returns msgid if no translation found
>
>   npgettext(context, singular, plural, n)
>     - Translates message with context and plural support
>     - Example: npgettext("file", "1 file", "{n} files", count)
>     - Returns appropriate singular/plural form based on n
>
> PO file format:
>   msgctxt "ceph"
>   msgid "Monitor"
>   msgstr "Monitor de Ceph"
>
>   msgctxt "qemu"
>   msgid "Monitor"
>   msgstr "Monitor de QEMU"
>
>   # Without context (traditional)
>   msgid "Monitor"
>   msgstr "Monitor"
>
> Extraction:
>   xgettext recognizes pgettext() by default (no keyword needed).
>
>   For npgettext(), we use --keyword=npgettext:1c,2,3 where:
>     1c = context (argument 1)
>     2  = singular msgid (argument 2)
>     3  = plural msgid (argument 3)
>
>   Note: We cannot use '1c,2' like meson's i18n module does for NC_().
>   The NC_() macro from glib (used in meson's example) is context-only
>   and does NOT support plural forms. From GTK documentation:
>   "NC_(Context, String) - Only marks a string for translation, with
>   context. Similar to N_(), but allows you to add a context."
>
>   Our npgettext() function signature is npgettext(context, singular,
>   plural, n), which requires both singular (arg 2) AND plural (arg 3)
>   forms. Using '1c,2' would only extract the singular form and lose
>   the plural, breaking npgettext functionality.
>
> Backward compatibility:
> - Existing PO files without msgctxt work unchanged
> - Existing gettext()/ngettext() calls work unchanged
> - Hash values for non-context messages are identical
> - Same catalog file format
>
> Tested with following steps:
> Modified pve-manager submodule:
>   File: pve-manager/www/manager6/button/ConsoleButton.js
>   - Changed gettext('Console') → pgettext('button', 'Console')
>   - Added pgettext('console menu', 'noVNC/SPICE/xterm.js')
>   - Added npgettext('console', '{0} console', '{0} consoles', count)
>
> Ran 'make update_pot' to verify xgettext extraction:
>   Successfully extracted msgctxt entries to pve-manager.pot:
>     msgctxt "button"
>     msgid "Console"
>
>     msgctxt "console menu"
>     msgid "noVNC"
>
>     msgctxt "console"
>     msgid "{0} console"
>     msgid_plural "{0} consoles"
>
> Created test PO file with context translations and verified po2js.pl
> generates correct context-aware hashes:
>   - hash("button" + "\x04" + "Console") = 914449940
>   - hash("console menu" + "\x04" + "noVNC") = 418124897
>   - Different contexts produce unique hashes for same msgid
>
> All tests passed:
>   - xgettext extracts msgctxt from JavaScript
>   - po2js.pl processes msgctxt from PO files
>   - Context-aware hashing produces unique digests
>   - Generated JS includes pgettext/npgettext functions
>   - Complete workflow: JS → POT → PO → JS catalog
>
> This matches the existing Rust pgettext/npgettext implementation
> in proxmox-yew-widget-toolkit, enabling consistent context-aware
> translations across the Proxmox ecosystem.
>
> Signed-off-by: Kefu Chai <k.chai@proxmox.com>
> ---
>  Makefile |  1 +
>  po2js.pl | 69 ++++++++++++++++++++++++++++++++++++++++++++++++--------
>  2 files changed, 61 insertions(+), 9 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 86bd723..3feaee7 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -155,6 +155,7 @@ define potupdate
>        --package-version="$(shell cd $(2);git rev-parse HEAD)" \
>        --msgid-bugs-address="<support@proxmox.com>" \
>        --copyright-holder="Copyright (C) Proxmox Server Solutions GmbH <support@proxmox.com> & the translation contributors." \
> +      --keyword=npgettext:1c,2,3 \
>        --output="$(1)".pot
>  endef
>  
> diff --git a/po2js.pl b/po2js.pl
> index 316c0bd..4b7b044 100755
> --- a/po2js.pl
> +++ b/po2js.pl
> @@ -8,10 +8,6 @@ use Getopt::Long;
>  use JSON;
>  use Locale::PO;
>  
> -# current limits:
> -# - we do not support plural. forms
> -# - no message content support
> -
>  my $options = {};
>  GetOptions($options, 't=s', 'o=s', 'v=s') or die "unable to parse options\n";
>  
> @@ -34,6 +30,15 @@ sub fnv31a {
>      return $hval & 0x7fffffff;
>  }
>  
> +# Hash function for messages with context
> +sub fnv31a_ctxt {
> +    my ($msgctxt, $msgid) = @_;
> +    return fnv31a($msgid) if !length($msgctxt);  # Empty context = no context
> +    # Use EOT character (0x04) as separator, standard in gettext
> +    my $combined = $msgctxt . "\x04" . $msgid;
> +    return fnv31a($combined);
> +}
> +
>  my $catalog = {};
>  my $plurals_catalog = {};
>  
> @@ -41,11 +46,20 @@ my $nplurals = 2;
>  my $plural_forms = "n!=1";
>  
>  foreach my $filename (@ARGV) {
> -    my $href = Locale::PO->load_file_ashash($filename)
> +    my $aref = Locale::PO->load_file_asarray($filename)
>          || die "unable to load '$filename'\n";
>  
>      my $charset;
> -    my $hpo = $href->{'""'} || die "no header";
> +    # Find header entry (msgid "")
> +    my $hpo;
> +    foreach my $po (@$aref) {
> +        if ($po->msgid eq '""') {
> +            $hpo = $po;
> +            last;
> +        }
> +    }
> +    die "no header" if !$hpo;
> +
>      my $header = $hpo->dequote($hpo->msgstr);
>      if ($header =~ m|^Content-Type:\s+text/plain;\s+charset=(\S+)$|im) {
>          $charset = $1;
> @@ -58,8 +72,7 @@ foreach my $filename (@ARGV) {
>  	$plural_forms = $2;
>      }
>  
> -    foreach my $k (keys %$href) {
> -        my $po = $href->{$k};
> +    foreach my $po (@$aref) {
>          next if $po->fuzzy(); # skip fuzzy entries
>          my $ref = $po->reference();
>  
> @@ -76,9 +89,18 @@ foreach my $filename (@ARGV) {
>          my $qmsgid_plural = decode($charset, $po->msgid_plural);
>          my $msgid_plural = $po->dequote($qmsgid_plural);
>  
> +        # Extract message context if present
> +        my $msgctxt = '';
> +        if (defined($po->msgctxt)) {
> +            my $qmsgctxt = decode($charset, $po->msgctxt);
> +            $msgctxt = $po->dequote($qmsgctxt);
> +        }
> +
>          next if !length($msgid) && !length($msgid_plural); # skip header
>  
> -        my $digest = fnv31a($msgid);
> +        my $digest = length($msgctxt) > 0
> +            ? fnv31a_ctxt($msgctxt, $msgid)
> +            : fnv31a($msgid);
>  
>          die "duplicate digest" if $catalog->{$digest};
>  
> @@ -150,6 +172,35 @@ function ngettext(singular, plural, n) {
>      }
>      return translation[msg_idx];
>  }
> +
> +function fnv31a_ctxt(context, text) {
> +    // Use EOT character (0x04) as separator
> +    var combined = context + "\\x04" + text;
> +    return fnv31a(combined);
> +}
> +
> +function pgettext(context, msgid) {
> +    var digest = fnv31a_ctxt(context, msgid);
> +    var data = __proxmox_i18n_msgcat__[digest];
> +    if (!data) {
> +        return msgid;  // Return msgid (not context) as fallback
> +    }
> +    return data[0] || msgid;
> +}
> +
> +function npgettext(context, singular, plural, n) {
> +    const msg_idx = Number($plural_forms);
> +    const digest = fnv31a_ctxt(context, singular);
> +    const translation = __proxmox_i18n_plurals_msgcat__[digest];
> +    if (!translation || msg_idx >= translation.length) {
> +        if (n === 1) {
> +            return singular;
> +        } else {
> +            return plural;
> +        }
> +    }
> +    return translation[msg_idx];
> +}
>  __EOD
>  
>  if ($outfile) {