From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id B711A913F3 for ; Fri, 17 Mar 2023 19:45:19 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 99A9E1D19A for ; Fri, 17 Mar 2023 19:45:19 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Fri, 17 Mar 2023 19:45:18 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 4CA684521D for ; Fri, 17 Mar 2023 19:45:17 +0100 (CET) From: Stoiko Ivanov To: pmg-devel@lists.proxmox.com Date: Fri, 17 Mar 2023 19:44:50 +0100 Message-Id: <20230317184456.26465-2-s.ivanov@proxmox.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230317184456.26465-1-s.ivanov@proxmox.com> References: <20230317184456.26465-1-s.ivanov@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.142 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [notify.pm, smtp.pm, utils.pm, postfix.org, quarantine.pm] Subject: [pmg-devel] [PATCH pmg-api v3 1/5] smtputf8: keep smtputf8 from incoming postfix, detect for local mail X-BeenThere: pmg-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Mail Gateway development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Mar 2023 18:45:19 -0000 This patch changes the detection if smtputf8 is needed as option to the 'MAIL' command: * for mail arriving through postfix it is only added if the mail originally was received with it (Accept and BCC actions) * for locally generated mail (Notify, reports, quarantine-link and ndrs) it is decided based on utf8 characters in the mail-addresses or headers - this is done by `reinject_local_mail`, as a new helper This should match postfix own behavior in those cases quite closely: https://www.postfix.org/SMTPUTF8_README.html#using Notable difference is that we check the complete e-mail address and not only the domain part, but I assume non-ascii local-parts to be a very fringe edge-case in environments where smtputf8 is not supported. Signed-off-by: Stoiko Ivanov --- src/PMG/API2/Quarantine.pm | 2 +- src/PMG/RuleDB/Notify.pm | 2 +- src/PMG/SMTP.pm | 3 ++- src/PMG/Utils.pm | 28 ++++++++++++++++++++-------- 4 files changed, 24 insertions(+), 11 deletions(-) diff --git a/src/PMG/API2/Quarantine.pm b/src/PMG/API2/Quarantine.pm index fbb302a..6318082 100644 --- a/src/PMG/API2/Quarantine.pm +++ b/src/PMG/API2/Quarantine.pm @@ -1239,7 +1239,7 @@ my sub send_link_mail { ); # we use an empty envelope sender (we don't want to receive NDRs) - PMG::Utils::reinject_mail ($mail, '', [$receiver], undef, $fqdn); + PMG::Utils::reinject_local_mail ($mail, '', [$receiver], undef, $fqdn); } __PACKAGE__->register_method ({ diff --git a/src/PMG/RuleDB/Notify.pm b/src/PMG/RuleDB/Notify.pm index 68f9b4e..c024840 100644 --- a/src/PMG/RuleDB/Notify.pm +++ b/src/PMG/RuleDB/Notify.pm @@ -256,7 +256,7 @@ sub execute { print $fh "notify end\n"; } else { my @targets = split(/\s*,\s*/, $to); - my $qid = PMG::Utils::reinject_mail( + my $qid = PMG::Utils::reinject_local_mail( $top, $from, \@targets, undef, $msginfo->{fqdn}); foreach (@targets) { my $target = encode('UTF-8', $_); diff --git a/src/PMG/SMTP.pm b/src/PMG/SMTP.pm index fbf5c95..b7bc5d3 100644 --- a/src/PMG/SMTP.pm +++ b/src/PMG/SMTP.pm @@ -111,6 +111,7 @@ sub loop { $self->{param}->{mail}->{$1} = $2; } elsif ($opt =~ m/smtputf8/i) { $self->{smtputf8} = 1; + $self->{param}->{mail}->{smtputf8} = 1; $from = decode('UTF-8', $from); } else { #ignore everything else @@ -314,7 +315,7 @@ EOF Encoding => '7bit', Description => 'Delivery report'); - my $qid = PMG::Utils::reinject_mail($ndr, '', [$sender], undef, $hostname); + my $qid = PMG::Utils::reinject_local_mail($ndr, '', [$sender], undef, $hostname); if ($qid) { syslog('info', "sent NDR for rejecting recipients - $qid"); } else { diff --git a/src/PMG/Utils.pm b/src/PMG/Utils.pm index 9c6f841..49939c4 100644 --- a/src/PMG/Utils.pm +++ b/src/PMG/Utils.pm @@ -221,18 +221,28 @@ sub subst_values_for_header { return $res; } -sub mail_needs_smtputf8 { - my ($entity, $sender, $targets) = @_; +# detects the need for setting smtputf8 based on addresses and headers +sub reinject_local_mail { + my ($entity, $sender, $targets, $xforward, $me, $params) = @_; + + my $needs_smtputf8 = 0; - return 1 if ($sender =~ /[^\p{PosixPrint}]/); + $needs_smtputf8 = 1 if ($sender =~ /[^\p{PosixPrint}]/); foreach my $target (@$targets) { if ($target =~ /[^\p{PosixPrint}]/) { - return 1; + $needs_smtputf8 = 1; + last; } } - return 0; + if (!$needs_smtputf8 && $entity->head()->as_string() =~ /([^\p{PosixPrint}\n\r\t])/) { + $needs_smtputf8 = 1; + } + + $params->{mail}->{smtputf8} = $needs_smtputf8; + + return reinject_mail($entity, $sender, $targets, $xforward, $me, $params); } sub reinject_mail { @@ -260,10 +270,12 @@ sub reinject_mail { } my $mail_opts = " BODY=8BITMIME"; - $mail_opts .= " SMTPUTF8" if mail_needs_smtputf8($entity, $sender, $targets); my $sender_addr = encode('UTF-8', $smtp->_addr($sender)); - if (defined($params->{mail})) { + if (delete $params->{mail}->{smtputf8}) { + $mail_opts .= " SMTPUTF8"; + } + my $mailparams = $params->{mail}; for my $p (keys %$mailparams) { $mail_opts .= " $p=$mailparams->{$p}"; @@ -1258,7 +1270,7 @@ sub finalize_report { return; } # we use an empty envelope sender (we don't want to receive NDRs) - PMG::Utils::reinject_mail ($top, '', [$receiver], undef, $data->{fqdn}); + PMG::Utils::reinject_local_mail ($top, '', [$receiver], undef, $data->{fqdn}); } sub lookup_timespan { -- 2.30.2