public inbox for pmg-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Stoiko Ivanov <s.ivanov@proxmox.com>
To: pmg-devel@lists.proxmox.com
Subject: [pmg-devel] [PATCH pmg-api 1/4] templates: postfix: set same timeouts for before and after-queue
Date: Mon, 11 Sep 2023 16:23:13 +0200	[thread overview]
Message-ID: <20230911142317.19746-2-s.ivanov@proxmox.com> (raw)
In-Reply-To: <20230911142317.19746-1-s.ivanov@proxmox.com>

When a mail takes longer to get processed (observed with some scanned
pages as pdf - where clamav+avast took 2.2 minutes for scanning) the
behavior of the filtering is different between before-queue and
after-queue filtering:
In the before-queue case the timeout is `$smtpd_proxy_timout` (120s)
[0], so the mail does not get processed in time and the listening
smtpd responds with `451 4.3.0 Error: queue file write error`. However
pmg-smtp-filter sends the mail to the smtpd on 10025 once it's done.
the original sender resends the mail due to the 451 error - which
results in the mail getting delivered multiple times (until it gets
removed from the queue on the original sender)

In the after-queue case the timeout is `$lmtp_data_done_timeout`
(pmg-smtp-filter acts as lmtp server) (600s) - so the mail gets
send successfully only once. In case the processing time reaches
600s the behavior is equivalent - but for lmtp postfix logs:
```
...timed out while sending end of data -- message may be sent more
than once
```

The value needs to be set as literal - referring to the builtin
default as `$lmtp_data_done_timeout` does not work.

While the underlying issue of mails getting send multiple times in
case the timeout is reached is not fixed by this - having the same
timeout in both cases is a good idea and 600s increases the chances
of clamav+avast+custom_check_script+spamassassin to get their job done.

tested by adding a `sleep 200` in PMG::Utils::analyze_virus_clam.
(before the eval block running with the 5 minute timeout)

[0] https://www.postfix.org/SMTPD_PROXY_README.html#parameters

Reported-by: Martin Maurer <martin@proxmox.com>
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/templates/main.cf.in | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/src/templates/main.cf.in b/src/templates/main.cf.in
index bce0353..516bc2f 100644
--- a/src/templates/main.cf.in
+++ b/src/templates/main.cf.in
@@ -49,7 +49,9 @@ relay_transport = [% pmg.mail.relayprotocol %]:[% pmg.mail.relay %]:[% pmg.mail.
 default_transport = smtp:[% pmg.mail.smarthost %]:[% pmg.mail.smarthostport %]
 [% END %]
 
-[% IF ! pmg.mail.before_queue_filtering -%]
+[% IF pmg.mail.before_queue_filtering -%]
+smtpd_proxy_timeout = 600s
+[% ELSE %]
 content_filter=scan:127.0.0.1:10024
 [%- END %]
 
-- 
2.39.2





  reply	other threads:[~2023-09-11 14:23 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-11 14:23 [pmg-devel] [PATCH pmg-api/docs] make filter timeout configurable Stoiko Ivanov
2023-09-11 14:23 ` Stoiko Ivanov [this message]
2023-09-11 14:23 ` [pmg-devel] [PATCH pmg-api 2/4] pmg-smtp-filter: refactor use of gettimeofday Stoiko Ivanov
2023-09-11 14:23 ` [pmg-devel] [PATCH pmg-api 3/4] config: postfix: make smtp-filter-timeout configurable Stoiko Ivanov
2024-01-12  9:15   ` Thomas Lamprecht
2023-09-11 14:23 ` [pmg-devel] [PATCH pmg-api 4/4] pmg-smtp-filter: die if processing took longer than the timeout Stoiko Ivanov
2024-01-12  9:19   ` Thomas Lamprecht
2023-09-11 14:23 ` [pmg-devel] [PATCH pmg-docs 1/1] doc-generator: add new option filter_timeout Stoiko Ivanov
2024-01-12  8:35 ` [pmg-devel] [PATCH pmg-api/docs] make filter timeout configurable Dominik Csapak
2024-01-12 19:59   ` Stoiko Ivanov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230911142317.19746-2-s.ivanov@proxmox.com \
    --to=s.ivanov@proxmox.com \
    --cc=pmg-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal