From: Fabian Ebner <f.ebner@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH manager] api: replication: don't send mails about failed replication only once
Date: Fri, 22 Apr 2022 14:15:48 +0200 [thread overview]
Message-ID: <20220422121548.69146-1-f.ebner@proxmox.com> (raw)
but rather multiple times becoming exponentially less frequent.
Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---
PVE/API2/Replication.pm | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/PVE/API2/Replication.pm b/PVE/API2/Replication.pm
index a1bcd89b..a7e54497 100644
--- a/PVE/API2/Replication.pm
+++ b/PVE/API2/Replication.pm
@@ -74,6 +74,19 @@ sub run_single_job {
sub run_jobs {
my ($now, $logfunc, $verbose, $mail) = @_;
+ my $mail_at_fail_count = sub {
+ my ($fail_count) = @_;
+
+ return 1 if $fail_count == 1;
+
+ # failing job is re-tried every half hour, try to send one mail after 1, 2, 4, 8, etc. days
+ my $i = 1;
+ while ($i * 48 < $fail_count) {
+ $i = $i * 2;
+ }
+ return $i * 48 == $fail_count;
+ };
+
my $iteration = $now // time();
my $code = sub {
@@ -93,7 +106,7 @@ sub run_jobs {
my $jobstate = PVE::ReplicationState::extract_job_state($state, $jobcfg);
eval {
PVE::Tools::sendmail('root', "Replication Job: $jobcfg->{id} failed", $err)
- if $jobstate->{fail_count} == 1 && $mail;
+ if $mail && $mail_at_fail_count->($jobstate->{fail_count});
};
warn ": $@" if $@;
}
--
2.30.2
next reply other threads:[~2022-04-22 12:16 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-22 12:15 Fabian Ebner [this message]
2022-04-27 8:30 ` [pve-devel] applied: " Thomas Lamprecht
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220422121548.69146-1-f.ebner@proxmox.com \
--to=f.ebner@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox