Re: [pve-devel] [PATCH proxmox-i18n] js generator: try to extract comments from source

[Maximiliano Sandoval <m.sandoval@proxmox.com>]

I did a quick test with

```js
     // Comment 1
     a: gettext('TEST 1'),
     // Comment 2
     b: gettext('TEST 1'),

     c: gettext('TEST 2'),
     // Comment 3
     d: gettext('TEST 2'),

     // Comment 4
     e: gettext('TEST 3'),
     f: gettext('TEST 3'),
```

and the resulting file contains a single instance of each msgid:

```

#. Comment 1
#: pve-manager/www/manager6/ceph/OSD.js:239
#: pve-manager/www/manager6/ceph/OSD.js:241
msgid "TEST 1"
msgstr ""

#: pve-manager/www/manager6/ceph/OSD.js:243
#: pve-manager/www/manager6/ceph/OSD.js:245
msgid "TEST 2"
msgstr ""

#. Comment 4
#: pve-manager/www/manager6/ceph/OSD.js:248
#: pve-manager/www/manager6/ceph/OSD.js:249
msgid "TEST 3"
msgstr ""
```

Compare with the output from xgettext (xgettext --language=JavaScript --add-comments OSD.js)
```
#. Comment 1
#. Comment 2
#: OSD.js:239 OSD.js:241
msgid "TEST 1"
msgstr ""

#. Comment 3
#: OSD.js:243 OSD.js:245
msgid "TEST 2"
msgstr ""

#. Comment 4
#: OSD.js:248 OSD.js:249
msgid "TEST 3"
msgstr ""
```

which means that at worst we will lose a comment depending on which msgid was
found first. Given that this is not using the pgettext method that takes into
account the to generate different translations, this seems fine to me, at least
as a first step into getting feature parity with (x)gettext.

On 8/30/23 15:23, Thomas Lamprecht wrote:
> Am 30/08/2023 um 14:48 schrieb Maximiliano Sandoval:
>> Adds a way to extract comments to the resulting .po files matching
>> xgettext's behaviour. Useful for instances like
>>
>> ```js
>>     // Translators: This is the verb, not the noun
> hmm, but is then grouped with the reference comment in the resulting
> PO file?
>
> And even if, what use is the comment if there are then a usage as noun
> and one as verb? For that we really would need the msgcontext stuff and
> a pgettext implementation so that one could then actually do different
> translations for either variant (e.g., in German then Noun -> "Profil"
> and verb -> "Profilieren")
>
> IMO the comments mostly make sense for describing string formats, but
> support for them can be still fine as it doesn't cost much.
>
> Oh, and while at it, I pondered over adding a wrapper for the common
> Ext.String.format(gettext(MSG), PARAMS)) combi, something like
> gettextf(MSG, PARAMS) or even shorter i18n(MSG, PARAMS) (with params
> optional). This way we might even parse the passed params and add those
> as comment automatically.
>
>>     gettext("Profile");
>>
>>     // Translators: This would read 'Manage OSD'
>>     Ext.String.format(gettext('Manage {0}'), 'OSD');
>> ```
>>
>> where the string is not enough to guarantee is a satisfactory
>> translation.
>>
>> Do note that two identical messages with different comments will count
>> as the same message (same msgid) from the point of view of gettext. To
>> truly differentiate them one would need to support Context, see [1].
>>
>> Caveats:
>>    - Cannot extract multiline comments
>>    - Does not understand comments in the /* comment */ form
> those two are fine, but comments at the end of line won't be recognized either,
> e.g.:
>
>      fieldLabel: gettext('Foo'), // Translators: bar baz
>
> No biggie, but IMO not to hard to parse, especially as we don't have
> to bother that much for performance as this is only triggered manually
> when updating the POT catalogue.
>
>> [1] https://www.gnu.org/software/gettext/manual/html_node/Contexts.html
>>
>> Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
>> ---
>>   jsgettext.pl | 9 ++++++---
>>   1 file changed, 6 insertions(+), 3 deletions(-)
>>
>> diff --git a/jsgettext.pl b/jsgettext.pl
>> index 7f758fd..d0bf7a9 100755
>> --- a/jsgettext.pl
>> +++ b/jsgettext.pl
>> @@ -93,7 +93,7 @@ my $href = {
>>   };
>>   
>>   sub extract_msg {
>> -    my ($filename, $linenr, $line) = @_;
>> +    my ($filename, $linenr, $line, $comment) = @_;
>>   
>>       my $count = 0;
>>   
>> @@ -111,7 +111,7 @@ sub extract_msg {
>>   	if (my $po = $href->{$text}) {
>>   	    $po->reference($po->reference() . " $ref");
>>   	} else {
>> -	    $href->{$text} = Locale::PO->new(-msgid=> $text, -reference=> $ref, -msgstr=> '');
>> +	    $href->{$text} = Locale::PO->new(-msgid=> $text, -reference=> $ref, -msgstr=> '', -automatic=> $comment);
>>   	}
>>       }
>>       die "can't extract gettext message in '$filename' line $linenr\n" if !$count;
>> @@ -122,10 +122,13 @@ my $sources = find_js_sources($dirs);
>>   
>>   foreach my $s (@$sources) {
>>       open(my $SRC_FH, '<', $s) || die "unable to open file '$s' - $!\n";
>> +    my $prev_line;
>>       while(defined(my $line = <$SRC_FH>)) {
>>   	if ($line =~ m/gettext\s*\(/ && $line !~ m/^\s*function gettext/) {
>> -	    extract_msg($s, $., $line);
>> +	    my ($comment) = $prev_line =~ /\/\/\s+(.*)/;
>> +	    extract_msg($s, $., $line, $comment);
>>   	}
>> +	$prev_line = $line;
>>       }
>>       close($SRC_FH);
>>   }