From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 9E0746052E for ; Tue, 17 Nov 2020 09:55:24 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 99CB6D920 for ; Tue, 17 Nov 2020 09:54:54 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id A9772D916 for ; Tue, 17 Nov 2020 09:54:53 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 74B6643731 for ; Tue, 17 Nov 2020 09:54:53 +0100 (CET) From: Stoiko Ivanov To: pmg-devel@lists.proxmox.com Date: Tue, 17 Nov 2020 09:54:01 +0100 Message-Id: <20201117085401.23407-3-s.ivanov@proxmox.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201117085401.23407-1-s.ivanov@proxmox.com> References: <20201117085401.23407-1-s.ivanov@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.088 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [mcgrail.com, cf.in, kam.cf] Subject: [pmg-devel] [PATCH pmg-api 2/2] update KAM.cf in pmg-daily X-BeenThere: pmg-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Mail Gateway development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2020 08:55:24 -0000 KAM.cf provides additional rules for SpamAssassin and is shipped with proxmox-spamassassin. Since the rules get updated regularly, they should get updated on a running installation along with the rules provided by SpamAssassin directly. The patch adds the functionalilty for downloading KAM.cf to pmg-daily, which gets run (daily) by `pmg-daily.timer` LWP::UserAgent's mirror method is used since it adds a 'If-Modified-Since' header to the request, based on the provided localfile, sets the mtime to the Last-Modified date and replaces the file only if the download was successful. I chose '/var/lib/pmg/spamassassin-extra' for keeping the updated version to mirror '/usr/share/spamassassin-extra' Tested the updated template: * pmg-smtp-filter/SpamAssassin runs if either file is not present * the file in /var/lib/pmg/spamassassin-extra takes precedence over the one shipped in '/usr/share/spamassassin-extra' (tested by locally modifying a rule-score) Signed-off-by: Stoiko Ivanov --- debian/dirs | 1 + src/bin/pmg-daily | 29 ++++++++++++++++++++++++++++- src/templates/local.cf.in | 2 ++ 3 files changed, 31 insertions(+), 1 deletion(-) diff --git a/debian/dirs b/debian/dirs index f7ac2e7..55393a8 100644 --- a/debian/dirs +++ b/debian/dirs @@ -2,3 +2,4 @@ /etc/pmg/dkim /var/lib/pmg /var/lib/pmg/backup +/var/lib/pmg/spamassassin-extra diff --git a/src/bin/pmg-daily b/src/bin/pmg-daily index 32ccb95..011ec25 100755 --- a/src/bin/pmg-daily +++ b/src/bin/pmg-daily @@ -19,6 +19,26 @@ use PMG::DBTools; use PMG::API2::Subscription; use PMG::API2::APT; +# returns 1 if there was a newer version, 0 if not +sub update_KAM { + + my $kam_url = 'https://mcgrail.com/downloads/KAM.cf'; + my $kam_local = '/var/lib/pmg/spamassassin-extra/KAM.cf'; + my $ua = PMG::Utils::lwp_user_agent(); + $ua->timeout(10); + $ua->max_size(1024*1024); + + my $response; + eval { $response = $ua->mirror($kam_url, $kam_local); }; + die "updating KAM.cf failed: $@\n" if $@; + + return 1 if $response->is_success; + return 0 if $response->code == 304; + + my $err = sprintf("unexpected response: %s - %s", $response->code(), $response->message()); + die "fetching KAM.cf - $err\n"; +} + $SIG{'__WARN__'} = sub { my $err = $@; my $t = $_[0]; @@ -72,13 +92,20 @@ if (my $http_proxy = $cfg->get('admin', 'http_proxy')) { $ENV{http_proxy} = $http_proxy; } +my $restart_filter = 0; # update spamassassin rules if (system('sa-update') == 0) { # if the exit code is 0, new updates were downloaded # then restart the pmg-smtp-filter to load the new rules - PMG::Utils::service_cmd('pmg-smtp-filter', 'restart'); + $restart_filter = 1; +} + +if (update_KAM()) { + $restart_filter = 1; } +PMG::Utils::service_cmd('pmg-smtp-filter', 'restart') if $restart_filter; + # run bayes database maintainance system('sa-learn --force-expire >/dev/null 2>&1'); diff --git a/src/templates/local.cf.in b/src/templates/local.cf.in index 899f970..1a64c78 100644 --- a/src/templates/local.cf.in +++ b/src/templates/local.cf.in @@ -27,3 +27,5 @@ score ANY_BOUNCE_MESSAGE [% pmg.spam.bounce_score %] include /usr/share/spamassassin-extra/KAM.cf +include /var/lib/pmg/spamassassin-extra/KAM.cf + -- 2.20.1