From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pmg-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id C00021FF187
for <inbox@lore.proxmox.com>; Wed, 23 Apr 2025 10:47:58 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 6845BCE26;
Wed, 23 Apr 2025 10:47:56 +0200 (CEST)
Message-ID: <327503b9-5086-4dd8-ad81-c87960cbb98f@proxmox.com>
Date: Wed, 23 Apr 2025 10:47:23 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
To: Stoiko Ivanov <s.ivanov@proxmox.com>
References: <20250414153656.98610-1-h.duerr@proxmox.com>
<20250417163103.3dda6a64@rosa.proxmox.com>
Content-Language: en-US
From: Hannes Duerr <h.duerr@proxmox.com>
In-Reply-To: <20250417163103.3dda6a64@rosa.proxmox.com>
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.043 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
HTML_MESSAGE 0.001 HTML included in message
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: Re: [pmg-devel] [RFC pmg-api] trim Message-ID when parsing E-mail
X-BeenThere: pmg-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Mail Gateway development discussion
<pmg-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pmg-devel>,
<mailto:pmg-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pmg-devel/>
List-Post: <mailto:pmg-devel@lists.proxmox.com>
List-Help: <mailto:pmg-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pmg-devel>,
<mailto:pmg-devel-request@lists.proxmox.com?subject=subscribe>
Cc: pmg-devel@lists.proxmox.com
Content-Type: multipart/mixed; boundary="===============7255825341040380772=="
Errors-To: pmg-devel-bounces@lists.proxmox.com
Sender: "pmg-devel" <pmg-devel-bounces@lists.proxmox.com>
This is a multi-part message in MIME format.
--===============7255825341040380772==
Content-Type: multipart/alternative;
boundary="------------bJqWup7dIQW0dtk80iKEqYYf"
Content-Language: en-US
This is a multi-part message in MIME format.
--------------bJqWup7dIQW0dtk80iKEqYYf
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
On 4/17/25 16:31, Stoiko Ivanov wrote:
> Thanks for the patch!
>
> generally looks ok - and could be merged - some nits/questions inline (can
> fixup most upon applying):
>
> On Mon, 14 Apr 2025 17:36:56 +0200
> Hannes Duerr<h.duerr@proxmox.com> wrote:
>
>> when we currently parse emails we do not remove trailing newlines from
>> the message-id. The consequence of this is that if you use the rule
>> system macro __MSGID__, there is also a newline at the end of the
>> string. This in turn leads to problems if you create a rule and want to
>> add something after the message ID.
> As we talked off-list about this I know the context - but mentioning the
> case of - modifying the message-id, after stripping <> and the domain to
> not leak information would help people in the future remembering why this
> was added.
yes good point, i will add it in a v2
>> Signed-off-by: Hannes Duerr<h.duerr@proxmox.com>
>> ---
>>
>> Notes:
>> I'm not 100% sure whether we can simply trim the newline without any
>> problems. I have searched through the code a bit, but I have not come
>> across any problematic areas. We use the msgid for the log messages, but
> I was surprised that the newline might be there at all - but the docs of
> MIME::Head (https://metacpan.org/pod/MIME::Head) say so:
> NOTE: The header(s) returned may end with a newline. If you don't want this, then chomp the return value.
>
> Else I looked through our code and also think that this is read once, and
> then used in a log-message and for adding it as template variable
> https://git.proxmox.com/?p=pmg-api.git;a=blob;f=src/bin/pmg-smtp-filter;h=32bad7b881733147e5defe64a590dc709669dd1a;hb=HEAD#l184
>
>
>> it is in the last position there and a newline is set. This means that
>> the log message does not change
>> If anyone has any objections or other ideas on how to tackle the
>> problem, I'd be interested to hear them
> How did you test this - I gave it a quick spin with a modify field object
> (adding a X-my-msgid header) and it looked ok - but out of curiosity?
I constructed the setup described by the customer:
I sent messages from a domain x, stripped the domain from the Message-ID
with postfix header-checks and added another domain y with an header
attribute (modify field) and the __MSGID__ macro
I also checked the log messages when sending some other arbitrary mails.
>> src/PMG/MailQueue.pm | 3 ++-
>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/src/PMG/MailQueue.pm b/src/PMG/MailQueue.pm
>> index 4e37cb9..2af95bd 100644
>> --- a/src/PMG/MailQueue.pm
>> +++ b/src/PMG/MailQueue.pm
>> @@ -4,6 +4,7 @@ use strict;
>> use warnings;
>>
>> use PVE::SafeSyslog;
>> +use PVE::Tools qw (trim);
>> use MIME::Parser;
>> use IO::File;
>> use Encode;
>> @@ -394,7 +395,7 @@ sub parse_mail {
>> PMG::MIMEUtils::fixup_multipart($entity);
>>
>> if ((my $idcount = $entity->head->count ('Message-Id')) > 0) {
>> - $self->msgid ($entity->head->get ('Message-Id', $idcount - 1));
>> + $self->msgid (trim($entity->head->get ('Message-Id', $idcount - 1)));
> nit: when touching this you could remove the space `msgid(...)` vs
> `msgid (...)`
good point, will incorporate it in the v2
The v2 can be found here:
https://lore.proxmox.com/pmg-devel/20250423084549.20411-1-h.duerr@proxmox.com/T/#u
--------------bJqWup7dIQW0dtk80iKEqYYf
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 4/17/25 16:31, Stoiko Ivanov wrote:<br>
</div>
<blockquote type="cite"
cite="mid:20250417163103.3dda6a64@rosa.proxmox.com">
<pre wrap="" class="moz-quote-pre">Thanks for the patch!
generally looks ok - and could be merged - some nits/questions inline (can
fixup most upon applying):
On Mon, 14 Apr 2025 17:36:56 +0200
Hannes Duerr <a class="moz-txt-link-rfc2396E" href="mailto:h.duerr@proxmox.com"><h.duerr@proxmox.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="" class="moz-quote-pre">when we currently parse emails we do not remove trailing newlines from
the message-id. The consequence of this is that if you use the rule
system macro __MSGID__, there is also a newline at the end of the
string. This in turn leads to problems if you create a rule and want to
add something after the message ID.
</pre>
</blockquote>
<pre wrap="" class="moz-quote-pre">As we talked off-list about this I know the context - but mentioning the
case of - modifying the message-id, after stripping <> and the domain to
not leak information would help people in the future remembering why this
was added.
</pre>
</blockquote>
yes good point, i will add it in a v2<br>
<blockquote type="cite"
cite="mid:20250417163103.3dda6a64@rosa.proxmox.com">
<pre wrap="" class="moz-quote-pre">
</pre>
<blockquote type="cite">
<pre wrap="" class="moz-quote-pre">
Signed-off-by: Hannes Duerr <a class="moz-txt-link-rfc2396E" href="mailto:h.duerr@proxmox.com"><h.duerr@proxmox.com></a>
---
Notes:
I'm not 100% sure whether we can simply trim the newline without any
problems. I have searched through the code a bit, but I have not come
across any problematic areas. We use the msgid for the log messages, but
</pre>
</blockquote>
<pre wrap="" class="moz-quote-pre">I was surprised that the newline might be there at all - but the docs of
MIME::Head (<a class="moz-txt-link-freetext" href="https://metacpan.org/pod/MIME::Head">https://metacpan.org/pod/MIME::Head</a>) say so:
NOTE: The header(s) returned may end with a newline. If you don't want this, then chomp the return value.
Else I looked through our code and also think that this is read once, and
then used in a log-message and for adding it as template variable
<a class="moz-txt-link-freetext" href="https://git.proxmox.com/?p=pmg-api.git;a=blob;f=src/bin/pmg-smtp-filter;h=32bad7b881733147e5defe64a590dc709669dd1a;hb=HEAD#l184">https://git.proxmox.com/?p=pmg-api.git;a=blob;f=src/bin/pmg-smtp-filter;h=32bad7b881733147e5defe64a590dc709669dd1a;hb=HEAD#l184</a>
</pre>
<blockquote type="cite">
<pre wrap="" class="moz-quote-pre"> it is in the last position there and a newline is set. This means that
the log message does not change
If anyone has any objections or other ideas on how to tackle the
problem, I'd be interested to hear them
</pre>
</blockquote>
<pre wrap="" class="moz-quote-pre">How did you test this - I gave it a quick spin with a modify field object
(adding a X-my-msgid header) and it looked ok - but out of curiosity?
</pre>
</blockquote>
I constructed the setup described by the customer:<br>
I sent messages from a domain x, stripped the domain from the
Message-ID with postfix header-checks and added another domain y
with an header attribute (modify field) and the __MSGID__ macro<br>
I also checked the log messages when sending some other arbitrary
mails.<br>
<blockquote type="cite"
cite="mid:20250417163103.3dda6a64@rosa.proxmox.com">
<pre wrap="" class="moz-quote-pre">
</pre>
<blockquote type="cite">
<pre wrap="" class="moz-quote-pre">
src/PMG/MailQueue.pm | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/PMG/MailQueue.pm b/src/PMG/MailQueue.pm
index 4e37cb9..2af95bd 100644
--- a/src/PMG/MailQueue.pm
+++ b/src/PMG/MailQueue.pm
@@ -4,6 +4,7 @@ use strict;
use warnings;
use PVE::SafeSyslog;
+use PVE::Tools qw (trim);
use MIME::Parser;
use IO::File;
use Encode;
@@ -394,7 +395,7 @@ sub parse_mail {
PMG::MIMEUtils::fixup_multipart($entity);
if ((my $idcount = $entity->head->count ('Message-Id')) > 0) {
- $self->msgid ($entity->head->get ('Message-Id', $idcount - 1));
+ $self->msgid (trim($entity->head->get ('Message-Id', $idcount - 1)));
</pre>
</blockquote>
<pre wrap="" class="moz-quote-pre">nit: when touching this you could remove the space `msgid(...)` vs
`msgid (...)`
</pre>
</blockquote>
<p>good point, will incorporate it in the v2</p>
<p><span style="white-space: pre-wrap">
</span></p>
<p><span style="white-space: pre-wrap">The v2 can be found here:
<a class="moz-txt-link-freetext" href="https://lore.proxmox.com/pmg-devel/20250423084549.20411-1-h.duerr@proxmox.com/T/#u">https://lore.proxmox.com/pmg-devel/20250423084549.20411-1-h.duerr@proxmox.com/T/#u</a>
</span></p>
</body>
</html>
--------------bJqWup7dIQW0dtk80iKEqYYf--
--===============7255825341040380772==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
pmg-devel mailing list
pmg-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pmg-devel
--===============7255825341040380772==--