From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 08908961EE for ; Mon, 23 Jan 2023 16:55:53 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id DBB4829E2E for ; Mon, 23 Jan 2023 16:55:52 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Mon, 23 Jan 2023 16:55:51 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 7261745C4D for ; Mon, 23 Jan 2023 16:55:51 +0100 (CET) From: Stoiko Ivanov To: pmg-devel@lists.proxmox.com Date: Mon, 23 Jan 2023 16:55:21 +0100 Message-Id: <20230123155521.28307-3-s.ivanov@proxmox.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230123155521.28307-1-s.ivanov@proxmox.com> References: <20230123155521.28307-1-s.ivanov@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.157 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to DNSWL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [notify.pm, utils.pm, smtp.pm, quarantine.pm, postfix.org] Subject: [pmg-devel] [PATCH pmg-api 2/2] smtputf8: keep smtputf8 from incoming postfix, detect for local mail X-BeenThere: pmg-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Mail Gateway development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2023 15:55:53 -0000 This patch changes the detection if smtputf8 is needed as option to the 'MAIL' command: * for mail passing arriving through postfix it is only added if the mail originally was received with it (Accept and BCC actions) * for locally generated mail (Notify, reports, quarantine-link and ndrs) it is decided based on utf8 characters in the mail-addresses or headers This should approximate postfix own behavior in those cases quite closely: https://www.postfix.org/SMTPUTF8_README.html#using Notable difference is that we check the complete e-mail address and not only the domain part, but I assume non-ascii local-parts to be a very fringe edge-case in environments where smtputf8 is not supported. If this occurs in the wild we would also need to adapt the unconditional encoding of the envelope addresses in reinject_mail Signed-off-by: Stoiko Ivanov --- src/PMG/API2/Quarantine.pm | 7 ++++++- src/PMG/RuleDB/Notify.pm | 6 +++++- src/PMG/SMTP.pm | 7 ++++++- src/PMG/Utils.pm | 16 +++++++++++++--- 4 files changed, 30 insertions(+), 6 deletions(-) diff --git a/src/PMG/API2/Quarantine.pm b/src/PMG/API2/Quarantine.pm index fbb302a..352f6b6 100644 --- a/src/PMG/API2/Quarantine.pm +++ b/src/PMG/API2/Quarantine.pm @@ -1239,7 +1239,12 @@ my sub send_link_mail { ); # we use an empty envelope sender (we don't want to receive NDRs) - PMG::Utils::reinject_mail ($mail, '', [$receiver], undef, $fqdn); + + my $params; + if (PMG::Utils::mail_needs_smtputf8($mail, '', [$receiver])) { + $params->{mail}->{smtputf8} = 1; + } + PMG::Utils::reinject_mail ($mail, '', [$receiver], undef, $fqdn, $params); } __PACKAGE__->register_method ({ diff --git a/src/PMG/RuleDB/Notify.pm b/src/PMG/RuleDB/Notify.pm index 68f9b4e..7887195 100644 --- a/src/PMG/RuleDB/Notify.pm +++ b/src/PMG/RuleDB/Notify.pm @@ -256,8 +256,12 @@ sub execute { print $fh "notify end\n"; } else { my @targets = split(/\s*,\s*/, $to); + my $params; + if (PMG::Utils::mail_needs_smtputf8($top, $from, \@targets)) { + $params->{mail}->{smtputf8} = 1; + } my $qid = PMG::Utils::reinject_mail( - $top, $from, \@targets, undef, $msginfo->{fqdn}); + $top, $from, \@targets, undef, $msginfo->{fqdn}, $params); foreach (@targets) { my $target = encode('UTF-8', $_); if ($qid) { diff --git a/src/PMG/SMTP.pm b/src/PMG/SMTP.pm index fbf5c95..35ed9c4 100644 --- a/src/PMG/SMTP.pm +++ b/src/PMG/SMTP.pm @@ -111,6 +111,7 @@ sub loop { $self->{param}->{mail}->{$1} = $2; } elsif ($opt =~ m/smtputf8/i) { $self->{smtputf8} = 1; + $self->{param}->{mail}->{smtputf8} = 1; $from = decode('UTF-8', $from); } else { #ignore everything else @@ -314,7 +315,11 @@ EOF Encoding => '7bit', Description => 'Delivery report'); - my $qid = PMG::Utils::reinject_mail($ndr, '', [$sender], undef, $hostname); + my $params; + if (PMG::Utils::mail_needs_smtputf8($ndr, '', [$sender])) { + $params->{mail}->{smtputf8} = 1; + } + my $qid = PMG::Utils::reinject_mail($ndr, '', [$sender], undef, $hostname, $params); if ($qid) { syslog('info', "sent NDR for rejecting recipients - $qid"); } else { diff --git a/src/PMG/Utils.pm b/src/PMG/Utils.pm index 9c6f841..1ccd7d2 100644 --- a/src/PMG/Utils.pm +++ b/src/PMG/Utils.pm @@ -232,6 +232,10 @@ sub mail_needs_smtputf8 { } } + if ($entity->head()->as_string() =~ /([^\p{PosixPrint}\n\r\t])/) { + return 1; + } + return 0; } @@ -260,10 +264,12 @@ sub reinject_mail { } my $mail_opts = " BODY=8BITMIME"; - $mail_opts .= " SMTPUTF8" if mail_needs_smtputf8($entity, $sender, $targets); my $sender_addr = encode('UTF-8', $smtp->_addr($sender)); - if (defined($params->{mail})) { + if (delete $params->{mail}->{smtputf8}) { + $mail_opts .= " SMTPUTF8"; + } + my $mailparams = $params->{mail}; for my $p (keys %$mailparams) { $mail_opts .= " $p=$mailparams->{$p}"; @@ -1258,7 +1264,11 @@ sub finalize_report { return; } # we use an empty envelope sender (we don't want to receive NDRs) - PMG::Utils::reinject_mail ($top, '', [$receiver], undef, $data->{fqdn}); + my $params; + if (PMG::Utils::mail_needs_smtputf8($top, '', [$receiver])) { + $params->{mail}->{smtputf8} = 1; + } + PMG::Utils::reinject_mail ($top, '', [$receiver], undef, $data->{fqdn}, $params); } sub lookup_timespan { -- 2.30.2