Re: [pmg-devel] [PATCH pmg-api 2/2] smtputf8: keep smtputf8 from incoming postfix, detect for local mail

[Stoiko Ivanov <s.ivanov@proxmox.com>]

Thanks for the review!

On Wed, 25 Jan 2023 10:30:09 +0100
Thomas Lamprecht <t.lamprecht@proxmox.com> wrote:

> Am 23/01/2023 um 16:55 schrieb Stoiko Ivanov:
> > This patch changes the detection if smtputf8 is needed as option to
> > the 'MAIL' command:
> > * for mail passing arriving through postfix it is only added if the
> >   mail originally was received with it (Accept and BCC actions)
> > * for locally generated mail (Notify, reports, quarantine-link and
> >   ndrs) it is decided based on utf8 characters in the mail-addresses or
> >   headers
> > 
> > This should approximate postfix own behavior in those cases quite
> > closely:
> > https://www.postfix.org/SMTPUTF8_README.html#using
> > 
> > Notable difference is that we check the complete e-mail address and
> > not only the domain part, but I assume non-ascii local-parts to be a
> > very fringe edge-case in environments where smtputf8 is not supported.
> > If this occurs in the wild we would also need to adapt the
> > unconditional encoding of the envelope addresses in reinject_mail
> > 
> > Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> > ---
> >  src/PMG/API2/Quarantine.pm |  7 ++++++-
> >  src/PMG/RuleDB/Notify.pm   |  6 +++++-
> >  src/PMG/SMTP.pm            |  7 ++++++-
> >  src/PMG/Utils.pm           | 16 +++++++++++++---
> >  4 files changed, 30 insertions(+), 6 deletions(-)
> > 
> > diff --git a/src/PMG/API2/Quarantine.pm b/src/PMG/API2/Quarantine.pm
> > index fbb302a..352f6b6 100644
> > --- a/src/PMG/API2/Quarantine.pm
> > +++ b/src/PMG/API2/Quarantine.pm
> > @@ -1239,7 +1239,12 @@ my sub send_link_mail {
> >      );
> >  
> >      # we use an empty envelope sender (we don't want to receive NDRs)
> 
> forgot to keep comment near method call?
was actually by choice - since the empty envelope sender is used in the
mail_needs_smtputf8 call as well - but can gladly move it to before the
reinject_mail call as well

> 
> > -    PMG::Utils::reinject_mail ($mail, '', [$receiver], undef, $fqdn);
> > +
> > +    my $params;
> > +    if (PMG::Utils::mail_needs_smtputf8($mail, '', [$receiver])) {
> > +	$params->{mail}->{smtputf8} = 1;
> > +    }
> 
> I'd rather move this into reinject mail instead of copyi-pastaing the same
> code hunk five times around, after all it has all the info required to
> call mail_needs_smtputf8 there. FWICT, its done on all call sites, so you
> wouldn't even require to add an opt-out param.
The call-sites it's not added are the ones in the rulesystem -
(PMG::RuleDB::Accept/BCC) - where the mail is received from the outside
and where we don't want to autodetect the need, but simply reuse what
postfix sends us.
(maybe I could have written that a bit more explicitly in the
commit-message)

> 
> > +    PMG::Utils::reinject_mail ($mail, '', [$receiver], undef, $fqdn, $params);
> >  }
> >  
> >  __PACKAGE__->register_method ({
> 
> 
> > diff --git a/src/PMG/Utils.pm b/src/PMG/Utils.pm
> > index 9c6f841..1ccd7d2 100644
> > --- a/src/PMG/Utils.pm
> > +++ b/src/PMG/Utils.pm
> > @@ -232,6 +232,10 @@ sub mail_needs_smtputf8 {
> >  	}
> >      }
> >  
> > +    if ($entity->head()->as_string() =~ /([^\p{PosixPrint}\n\r\t])/) {
> > +	return 1;
> > +    }
> 
> 
> you're reintroducing the hunk you removed in patch 1/2 without really adding any
> explicit reasoning, or is 1/2 just intended as uncontroversial stop gap to apply
> while 2/2 is still being checked more closely, or what's the deal here?
The idea was to apply 1/2 (as stop-gap measure) quite soon and get it out -
so that most users with disabled smtputf8 and non-ascii characters in
received mail get their systems working again, while 2/2 was something
that might benefit from a more through review.
I'll try to rewrite the commit message to reference 1/2 (or it's commit
hash once applied) explicitly