public inbox for pmg-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Dominik Csapak <d.csapak@proxmox.com>
To: Stoiko Ivanov <s.ivanov@proxmox.com>, pmg-devel@lists.proxmox.com
Subject: Re: [pmg-devel] [PATCH pmg-api v3 7/8] pmgqm: handle smtputf8 data
Date: Wed, 23 Nov 2022 15:20:24 +0100	[thread overview]
Message-ID: <7e1cd423-ec3a-6978-aa4e-f75505a06b18@proxmox.com> (raw)
In-Reply-To: <20221123092336.11423-8-s.ivanov@proxmox.com>

comments inline

On 11/23/22 10:23, Stoiko Ivanov wrote:
> $data->{pmail} is both used in the template rendering ('Spam Report for
> $pmail'), and as content for the To header, which need different
> treatment. Thus introduce 'pmail_raw' additionally.
> 
> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> ---
>   src/PMG/CLI/pmgqm.pm | 24 +++++++++++++-----------
>   src/PMG/Utils.pm     |  7 ++++---
>   2 files changed, 17 insertions(+), 14 deletions(-)
> 
> diff --git a/src/PMG/CLI/pmgqm.pm b/src/PMG/CLI/pmgqm.pm
> index dbec8ef..7293579 100755
> --- a/src/PMG/CLI/pmgqm.pm
> +++ b/src/PMG/CLI/pmgqm.pm
> @@ -2,6 +2,7 @@ package PMG::CLI::pmgqm;
>   
>   use strict;
>   use Data::Dumper;
> +use Encode qw(encode);
>   use Template;
>   use MIME::Entity;
>   use HTML::Entities;
> @@ -17,6 +18,7 @@ use PVE::SafeSyslog;
>   use PVE::Tools;
>   use PVE::INotify;
>   use PVE::CLIHandler;
> +use PVE::JSONSchema qw(get_standard_option);
>   
>   use PMG::RESTEnvironment;
>   use PMG::Utils;
> @@ -57,7 +59,7 @@ sub get_item_data {
>       }
>   
>       $item->{envelope_sender} = $ref->{sender};
> -    $item->{pmail} = $ref->{pmail};
> +    $item->{pmail} = encode_entities(PMG::Utils::try_decode_utf8($ref->{pmail}));
>       $item->{receiver} = $ref->{receiver} || $ref->{pmail};
>   
>       $item->{date} = strftime("%F", localtime($ref->{time}));
> @@ -157,11 +159,10 @@ __PACKAGE__->register_method ({
>       parameters => {
>   	additionalProperties => 0,
>   	properties => {
> -	    receiver => {
> +	    receiver => get_standard_option('pmg-email-address', {
>   		description => "Generate report for a single email address. If not specified, generate reports for all users.",
> -		type => 'string', format => 'email',
>   		optional => 1,
> -	    },
> +	    }),
>   	    timespan => {
>   		description => "Select time span.",
>   		type => 'string',
> @@ -175,11 +176,10 @@ __PACKAGE__->register_method ({
>   		enum => ['short', 'verbose', 'custom'],
>   		optional => 1,
>   	    },
> -	    redirect => {
> +	    redirect => get_standard_option('pmg-email-address', {
>   		description => "Redirect spam report email to this address.",
> -		type => 'string', format => 'email',
>   		optional => 1,
> -	    },
> +	    }),
>   	    debug => {
>   		description => "Debug mode. Print raw email to stdout instead of sending them.",
>   		type => 'boolean',
> @@ -280,7 +280,7 @@ __PACKAGE__->register_method ({
>   	    "ORDER BY pmail, time, receiver");
>   
>   	if ($target) {
> -	    $sth->execute($target);
> +	    $sth->execute(encode('UTF-8', $target));
>   	} else {
>   	    $sth->execute();
>   	}
> @@ -302,16 +302,18 @@ __PACKAGE__->register_method ({
>   	};
>   
>   	while (my $ref = $sth->fetchrow_hashref()) {
> -	    if ($creceiver ne $ref->{pmail}) {
> +	    my $decoded_pmail = PMG::Utils::try_decode_utf8($ref->{pmail});
> +	    if ($creceiver ne $decoded_pmail) {
>   
>   		$finalize->() if $data;
>   
>   		$data = clone($global_data);
>   
> -		$creceiver = $ref->{pmail};
> +		$creceiver = $decoded_pmail;
>   		$mailcount = 0;
>   
> -		$data->{pmail} = $creceiver;
> +		$data->{pmail} = encode_entities($decoded_pmail);
> +		$data->{pmail_raw} = $ref->{pmail};
>   		$data->{managehref} = "$protocol_fqdn_port/quarantine";
>   		if ($data->{authmode} ne 'ldap') {
>   		    $data->{ticket} = PMG::Ticket::assemble_quarantine_ticket($data->{pmail});
> diff --git a/src/PMG/Utils.pm b/src/PMG/Utils.pm
> index cc30e67..5c9e873 100644
> --- a/src/PMG/Utils.pm
> +++ b/src/PMG/Utils.pm
> @@ -1143,12 +1143,13 @@ sub rfc1522_to_html {
>   	    my ($d, $cs) = @$r;
>   	    if ($d) {
>   		if ($cs) {
> -		    $res .= encode_entities(decode($cs, $d));
> +		    $res .= encode('UTF-8', decode($cs, $d));
>   		} else {
> -		    $res .= encode_entities($d);
> +		    $res .= $d;
>   		}
>   	    }
>   	}
> +	$res = encode_entities(decode('UTF-8', $res));

this change is not really explained in the commit message
and is a bit confusing

couldn't we simply do:

encode_entities(decode_rfc1522($enc))

?

afaics is rfc1522_to_html mostly the same as decode_rfc1522
but with an 'encode_entities' after decoding


>       };
>   
>       $res = $enc if $@;
> @@ -1257,7 +1258,7 @@ sub finalize_report {
>   
>       my $top = MIME::Entity->build(
>   	Type    => "multipart/related",
> -	To      => $data->{pmail},
> +	To      => $data->{pmail_raw},
>   	From    => $mailfrom,
>   	Subject => bencode_header(decode_entities($title)));
>   





  reply	other threads:[~2022-11-23 14:20 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-11-23  9:23 [pmg-devel] [PATCH pmg-api/pmg-gui v3] ruledb - improve experience for non-ascii tests and mails Stoiko Ivanov
2022-11-23  9:23 ` [pmg-devel] [PATCH pmg-api v3 1/8] utils: return perl string from decode_rfc1522 Stoiko Ivanov
2022-11-23  9:23 ` [pmg-devel] [PATCH pmg-api v3 2/8] ruledb: properly substitute prox_vars in headers Stoiko Ivanov
2022-11-23  9:23 ` [pmg-devel] [PATCH pmg-api v3 3/8] fix #2541 ruledb: encode relevant values as utf-8 in database Stoiko Ivanov
2022-11-23  9:23 ` [pmg-devel] [PATCH pmg-api v3 4/8] ruledb: encode e-mail addresses for syslog Stoiko Ivanov
2022-11-23  9:23 ` [pmg-devel] [PATCH pmg-api v3 5/8] partially fix #2465: handle smtputf8 addresses in the rule-system Stoiko Ivanov
2022-11-23  9:23 ` [pmg-devel] [PATCH pmg-api v3 6/8] quarantine: handle utf8 data Stoiko Ivanov
2022-11-23 14:15   ` Dominik Csapak
2022-11-23  9:23 ` [pmg-devel] [PATCH pmg-api v3 7/8] pmgqm: handle smtputf8 data Stoiko Ivanov
2022-11-23 14:20   ` Dominik Csapak [this message]
2022-11-23  9:23 ` [pmg-devel] [PATCH pmg-api v3 8/8] statistics: handle utf8 data Stoiko Ivanov
2022-11-23 14:26   ` Dominik Csapak
2022-11-23  9:23 ` [pmg-devel] [PATCH pmg-gui v3 1/2] utils: add custom validator for pmg-email-address Stoiko Ivanov
2022-11-23  9:23 ` [pmg-devel] [PATCH pmg-gui v3 2/2] userblocklists: use PMGMail as validator for pmail Stoiko Ivanov
2022-11-23 14:09 ` [pmg-devel] [PATCH pmg-api/pmg-gui v3] ruledb - improve experience for non-ascii tests and mails Dominik Csapak
2022-11-26  7:00 ` [pmg-devel] applied-gui: " Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7e1cd423-ec3a-6978-aa4e-f75505a06b18@proxmox.com \
    --to=d.csapak@proxmox.com \
    --cc=pmg-devel@lists.proxmox.com \
    --cc=s.ivanov@proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal