From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id D6404B1D4 for ; Wed, 23 Nov 2022 15:20:28 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B8F4122BFD for ; Wed, 23 Nov 2022 15:20:28 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 23 Nov 2022 15:20:26 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 2014B44D6C; Wed, 23 Nov 2022 15:20:25 +0100 (CET) Message-ID: <7e1cd423-ec3a-6978-aa4e-f75505a06b18@proxmox.com> Date: Wed, 23 Nov 2022 15:20:24 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:108.0) Gecko/20100101 Thunderbird/108.0 Content-Language: en-US To: Stoiko Ivanov , pmg-devel@lists.proxmox.com References: <20221123092336.11423-1-s.ivanov@proxmox.com> <20221123092336.11423-8-s.ivanov@proxmox.com> From: Dominik Csapak In-Reply-To: <20221123092336.11423-8-s.ivanov@proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.065 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.001 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pmg-devel] [PATCH pmg-api v3 7/8] pmgqm: handle smtputf8 data X-BeenThere: pmg-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Mail Gateway development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Nov 2022 14:20:28 -0000 comments inline On 11/23/22 10:23, Stoiko Ivanov wrote: > $data->{pmail} is both used in the template rendering ('Spam Report for > $pmail'), and as content for the To header, which need different > treatment. Thus introduce 'pmail_raw' additionally. > > Signed-off-by: Stoiko Ivanov > --- > src/PMG/CLI/pmgqm.pm | 24 +++++++++++++----------- > src/PMG/Utils.pm | 7 ++++--- > 2 files changed, 17 insertions(+), 14 deletions(-) > > diff --git a/src/PMG/CLI/pmgqm.pm b/src/PMG/CLI/pmgqm.pm > index dbec8ef..7293579 100755 > --- a/src/PMG/CLI/pmgqm.pm > +++ b/src/PMG/CLI/pmgqm.pm > @@ -2,6 +2,7 @@ package PMG::CLI::pmgqm; > > use strict; > use Data::Dumper; > +use Encode qw(encode); > use Template; > use MIME::Entity; > use HTML::Entities; > @@ -17,6 +18,7 @@ use PVE::SafeSyslog; > use PVE::Tools; > use PVE::INotify; > use PVE::CLIHandler; > +use PVE::JSONSchema qw(get_standard_option); > > use PMG::RESTEnvironment; > use PMG::Utils; > @@ -57,7 +59,7 @@ sub get_item_data { > } > > $item->{envelope_sender} = $ref->{sender}; > - $item->{pmail} = $ref->{pmail}; > + $item->{pmail} = encode_entities(PMG::Utils::try_decode_utf8($ref->{pmail})); > $item->{receiver} = $ref->{receiver} || $ref->{pmail}; > > $item->{date} = strftime("%F", localtime($ref->{time})); > @@ -157,11 +159,10 @@ __PACKAGE__->register_method ({ > parameters => { > additionalProperties => 0, > properties => { > - receiver => { > + receiver => get_standard_option('pmg-email-address', { > description => "Generate report for a single email address. If not specified, generate reports for all users.", > - type => 'string', format => 'email', > optional => 1, > - }, > + }), > timespan => { > description => "Select time span.", > type => 'string', > @@ -175,11 +176,10 @@ __PACKAGE__->register_method ({ > enum => ['short', 'verbose', 'custom'], > optional => 1, > }, > - redirect => { > + redirect => get_standard_option('pmg-email-address', { > description => "Redirect spam report email to this address.", > - type => 'string', format => 'email', > optional => 1, > - }, > + }), > debug => { > description => "Debug mode. Print raw email to stdout instead of sending them.", > type => 'boolean', > @@ -280,7 +280,7 @@ __PACKAGE__->register_method ({ > "ORDER BY pmail, time, receiver"); > > if ($target) { > - $sth->execute($target); > + $sth->execute(encode('UTF-8', $target)); > } else { > $sth->execute(); > } > @@ -302,16 +302,18 @@ __PACKAGE__->register_method ({ > }; > > while (my $ref = $sth->fetchrow_hashref()) { > - if ($creceiver ne $ref->{pmail}) { > + my $decoded_pmail = PMG::Utils::try_decode_utf8($ref->{pmail}); > + if ($creceiver ne $decoded_pmail) { > > $finalize->() if $data; > > $data = clone($global_data); > > - $creceiver = $ref->{pmail}; > + $creceiver = $decoded_pmail; > $mailcount = 0; > > - $data->{pmail} = $creceiver; > + $data->{pmail} = encode_entities($decoded_pmail); > + $data->{pmail_raw} = $ref->{pmail}; > $data->{managehref} = "$protocol_fqdn_port/quarantine"; > if ($data->{authmode} ne 'ldap') { > $data->{ticket} = PMG::Ticket::assemble_quarantine_ticket($data->{pmail}); > diff --git a/src/PMG/Utils.pm b/src/PMG/Utils.pm > index cc30e67..5c9e873 100644 > --- a/src/PMG/Utils.pm > +++ b/src/PMG/Utils.pm > @@ -1143,12 +1143,13 @@ sub rfc1522_to_html { > my ($d, $cs) = @$r; > if ($d) { > if ($cs) { > - $res .= encode_entities(decode($cs, $d)); > + $res .= encode('UTF-8', decode($cs, $d)); > } else { > - $res .= encode_entities($d); > + $res .= $d; > } > } > } > + $res = encode_entities(decode('UTF-8', $res)); this change is not really explained in the commit message and is a bit confusing couldn't we simply do: encode_entities(decode_rfc1522($enc)) ? afaics is rfc1522_to_html mostly the same as decode_rfc1522 but with an 'encode_entities' after decoding > }; > > $res = $enc if $@; > @@ -1257,7 +1258,7 @@ sub finalize_report { > > my $top = MIME::Entity->build( > Type => "multipart/related", > - To => $data->{pmail}, > + To => $data->{pmail_raw}, > From => $mailfrom, > Subject => bencode_header(decode_entities($title))); >