Re: [pmg-devel] [PATCH pmg-api v2 1/3] pmgqm: add plain subject/from fields for reports

[Stoiko Ivanov <s.ivanov@proxmox.com>]

On Fri, 19 Sep 2025 09:00:24 +0200
Hannes Laimer <h.laimer@proxmox.com> wrote:

> On 19.09.25 01:21, Stoiko Ivanov wrote:
> > On Thu, 18 Sep 2025 16:54:01 +0200
> > Thomas Lamprecht <t.lamprecht@proxmox.com> wrote:
> >   
> >> Am 18.09.25 um 16:32 schrieb Hannes Laimer:  
> >>> We don't want plain-text reports to contain HTML-escaped chars,
> >>> this skips html encoding for the plain-text part.
> >>>
> >>> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> >>> ---
> >>> v2:
> >>> - explicitly utf8 encode after `decode_rfc1522`
> >>>
> >>>   src/PMG/CLI/pmgqm.pm | 16 ++++++++++++----
> >>>   1 file changed, 12 insertions(+), 4 deletions(-)
> >>>
> >>> diff --git a/src/PMG/CLI/pmgqm.pm b/src/PMG/CLI/pmgqm.pm
> >>> index 41f9f1a..c26b545 100755
> >>> --- a/src/PMG/CLI/pmgqm.pm
> >>> +++ b/src/PMG/CLI/pmgqm.pm
> >>> @@ -45,17 +45,25 @@ sub get_item_data {
> >>>   
> >>>       $item->{id} = sprintf("C%dR%dT%d", $ref->{cid}, $ref->{rid}, $ref->{ticketid});
> >>>   
> >>> -    $item->{subject} =
> >>> -        PMG::Utils::rfc1522_to_html(PVE::Tools::trim($head->get('subject')) || 'No Subject');
> >>> +    my $raw_subject = PVE::Tools::trim($head->get('subject')) || 'No Subject';
> >>> +    $item->{subject} = PMG::Utils::rfc1522_to_html($raw_subject);
> >>> +    $item->{subject_plain} = Encode::encode('UTF-8', PMG::Utils::decode_rfc1522($raw_subject));  
> >>
> >> it's a bit borderline, but might be worth adding a rfc1522_to_utf8 (or
> >> rfc1522_to_plain_utf8?) helper for this. For one there would be a tiny
> >> bit better code re-use, but it would make it slightly clearer for what
> >> happens here.  
> > FWIW: quickly looked through the code and currently think the encoding
> > here should do the sensible thing - and would agree that the helper sub
> > could help future reviewers see faster what's happening here.
> > 
> > one thing that is still passed through encode_entities is pmail (tried by
> > sending testmails to `discardįšų@test.domain`).
> > 
> > additionally it seems we do a double-encoding somewhere (probably in SMTP.pm)
> > but this should be unrelated to your patch.
> >   
> 
> We could also encode right before giving it to the template, so probably
> in finalize. Since the template is actually the only thing that requires
> this to be utf8 encoded.
hm - I think iterating over `$data` again does not gain too much - and as
it is right now - we do the encode as soon as is conveniently possible (we
fetch data from the database a few lines above.

I looked into the double-encoding - and the issue was my way of testing:
* `swaks(1)` is a great tool for testing smtp-servers - but from a quick
  look at the man-page does not consider smtputf8 at all (and postfix
  sends the things as it sees them[0], while pmg-smtp-filter relies on
  the extension keyword to decide what to do with the data)
Sending the testmails via postfix (+mutt) makes them appear correctly in
quarantine and logs.

[0] https://www.postfix.org/SMTPUTF8_README.html#detecting

> 
> >>  
> >>>   
> >>> -    my $from = PMG::Utils::rfc1522_to_html(PVE::Tools::trim($head->get('from') // $ref->{sender}));
> >>> -    my $sender = PMG::Utils::rfc1522_to_html(PVE::Tools::trim($head->get('sender')));
> >>> +    my $raw_from = PVE::Tools::trim($head->get('from') // $ref->{sender});
> >>> +    my $from = PMG::Utils::rfc1522_to_html($raw_from);
> >>> +    my $from_plain = Encode::encode('UTF-8', PMG::Utils::decode_rfc1522($raw_from));
> >>> +
> >>> +    my $raw_sender = PVE::Tools::trim($head->get('sender'));
> >>> +    my $sender = PMG::Utils::rfc1522_to_html($raw_sender);
> >>> +    my $sender_plain = Encode::encode('UTF-8', PMG::Utils::decode_rfc1522($raw_sender));
> >>>   
> >>>       if ($sender) {
> >>>           $item->{sender} = $sender;
> >>>           $item->{from} = sprintf("%s on behalf of %s", $sender, $from);
> >>> +        $item->{from_plain} = sprintf("%s on behalf of %s", $sender_plain, $from_plain);
> >>>       } else {
> >>>           $item->{from} = $from;
> >>> +        $item->{from_plain} = $from_plain;
> >>>       }
> >>>   
> >>>       $item->{envelope_sender} = $ref->{sender};  
> >>
> >>
> >>
> >> _______________________________________________
> >> pmg-devel mailing list
> >> pmg-devel@lists.proxmox.com
> >> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pmg-devel
> >>
> >>  
> >   
> 



_______________________________________________
pmg-devel mailing list
pmg-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pmg-devel