From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id C5D56917B2 for ; Wed, 21 Dec 2022 15:54:28 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id A644C9C0F for ; Wed, 21 Dec 2022 15:53:58 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 21 Dec 2022 15:53:57 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 6642C44766 for ; Wed, 21 Dec 2022 15:53:57 +0100 (CET) From: Stoiko Ivanov To: pmg-devel@lists.proxmox.com Date: Wed, 21 Dec 2022 15:53:43 +0100 Message-Id: <20221221145343.80373-1-s.ivanov@proxmox.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.159 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [perl.org, main.cf, utils.pm, rfc-editor.org, proxmox.com] Subject: [pmg-devel] [PATCH pmg-api v2] utils: fix mailflow if smtputf8 is disabled X-BeenThere: pmg-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Mail Gateway development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2022 14:54:28 -0000 with the recent addition of smtputf8 support for the rulesystem setups explicitly disabling smtputf8 in postfix got broken. This is mostly noticeable for the spamreports (the receivers are taken from the database and potentially decoded from utf-8, which sets the 'is_utf8' flag, and then tries to use the smtputf8 extension when reinjecting the mail, which fails (since smtputf8 is disabled) Instead of checking for the internal flag, we check for occurence of characters which are not ascii printable (everything excluding controlcharacters - '[\x20-\x7E]') in the envelope-addresses and headers (there also for [\r\n\t], due to searching all headers and folding). - see https://perldoc.perl.org/perlunifaq#What-is-%22the-UTF8-flag%22? and https://perldoc.perl.org/perlrecharclass#POSIX-Character-Classes The only diversion from the requirements in the smptutf8 rfc https://www.rfc-editor.org/rfc/rfc6531 is that we do not check the headers of all parts of a multipart message (think suggested filename for an attachment), but I assume that this should not be an issue in mail-transit the addresses now always get encoded as UTF-8, as this is robust for aascii-only addresses. reported in our community forum: https://forum.proxmox.com/threads/.119387/ issue is reproducible by setting `smtputf8_enable = no` in postfix main.cf and sending a spamreport using `pmgqm` regular mailflow should not be affected in those setups (as no utf-8 addresses would come into the system) Signed-off-by: Stoiko Ivanov --- v1->v2: * as suggested by Dominik (huge thanks for the thorough review and the suggestions!) the (top-level) mail headers are also scanned for non-ascii printable characters (and \n\r\t, since those occur in headers as strings) * put the test in a sub of its own * addresses are now always encoded as utf-8 (since for ascii only addresses this should be identity src/PMG/Utils.pm | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/src/PMG/Utils.pm b/src/PMG/Utils.pm index 10193f6..825b8d9 100644 --- a/src/PMG/Utils.pm +++ b/src/PMG/Utils.pm @@ -221,6 +221,24 @@ sub subst_values_for_header { return $res; } +sub mail_needs_smtputf8 { + my ($entity, $sender, $targets) = @_; + + return 1 if ($sender =~ /[^\p{PosixPrint}]/); + + foreach my $target (@$targets) { + if ($target =~ /[^\p{PosixPrint}]/) { + return 1; + } + } + + if ($entity->head()->as_string() =~ /([^\p{PosixPrint}\n\r\t])/) { + return 1; + } + + return 0; +} + sub reinject_mail { my ($entity, $sender, $targets, $xforward, $me, $params) = @_; @@ -245,23 +263,9 @@ sub reinject_mail { } } - my $has_utf8_targets = 0; - foreach my $target (@$targets) { - if (utf8::is_utf8($target)) { - $has_utf8_targets = 1; - last; - } - } - my $mail_opts = " BODY=8BITMIME"; - my $sender_addr; - if (utf8::is_utf8($sender)) { - $sender_addr = encode('UTF-8', $smtp->_addr($sender)); - $mail_opts .= " SMTPUTF8"; - } else { - $sender_addr = $smtp->_addr($sender); - $mail_opts .= " SMTPUTF8" if $has_utf8_targets; - } + $mail_opts .= " SMTPUTF8" if mail_needs_smtputf8($entity, $sender, $targets); + my $sender_addr = encode('UTF-8', $smtp->_addr($sender)); if (defined($params->{mail})) { my $mailparams = $params->{mail}; @@ -284,12 +288,8 @@ sub reinject_mail { $rcpt_opts .= " $p=$rcptparams->{$p}"; } } + $rcpt_addr = encode('UTF-8', $smtp->_addr($target)); - if (utf8::is_utf8($target)) { - $rcpt_addr = encode('UTF-8', $smtp->_addr($target)); - } else { - $rcpt_addr = $smtp->_addr($target); - } if (!$smtp->_RCPT("TO:" . $rcpt_addr . $rcpt_opts)) { syslog ('err', "smtp error - got: %s %s", $smtp->code, scalar($smtp->message)); die "smtp to: ERROR"; -- 2.30.2