[pmg-devel] [PATCH pmg-api 0/2] fix deprecation warning for user blocklist hits

[pmg-devel] [PATCH pmg-api 0/2] fix deprecation warning for user blocklist hits

[Stoiko Ivanov]

the 2 patches address quite different issues, both becoming visible when a
mail is blocked/quarantined due to being on the receivers blocklist:
* currently we use the description from SA for 'USER_IN_BLACKLIST', which
  includes since a while a deprecation warning (it's been replaced by
  USER_IN_BLOCKLIST, and afaict the old version will be dropped with
  SpamAssassin 4.0 soon). However PMG's SA rule only shares the name with
  the one from upstream SA (in the sense that our source of addresses is
  the database and not the SA-config), so we should make sure that the
  description is consistent
* We should also rethink/adapt our use of racially charged terms (and try
  to be consistent with SpamAssassin upstream in our choices) - I plan on
  addressing this once SpamAssassin 4.0 is released (patch 2/2 is only a
  small part and only touches the user/admin visible parts for the user
  blocklists)

For now the issue is a cosmetic one, and both approaches would address the
spurious DEPRECATED in the spaminfo widget in the GUI (although patch 1/2
moves code around that 2/2 touches)

Stoiko Ivanov (2):
  user-bl: use custom description of USER_IN_BLACKLIST consistently
  user accesslists: reword logging and hits for SA 4.0

 src/PMG/RuleDB/Spam.pm | 10 +++++-----
 src/PMG/Utils.pm       |  6 ++++++
 2 files changed, 11 insertions(+), 5 deletions(-)

-- 
2.30.2

[pmg-devel] [PATCH pmg-api 1/2] user-bl: use custom description of USER_IN_BLACKLIST consistently

[Stoiko Ivanov]

The USER_IN_BLACKLIST spamassassin hit is created by the Spam What
object, if the sending e-mail is in the receivers blacklist.
This 'hit' is kept on the PMG only - it is not written to the SPAMINFO
macro - and only visible in the quarantine interface afaict.

The description shown in the quarantine interface, however is read
from SpamAssassin sources.

They have recently changed to include a 'DEPRECATED' prefix, since the
rules containing 'blacklist' and 'whitelist' have been renamed to
'blocklist' and 'welcomelist' for the upcoming 4.0 series of
spamassassin.

In any case we should keep our description consistent, thus the move
to a sub of its own for reusing in both locations.

The mechanism for welcomlisted/whitelisted mails does not create an
'internal' sa-rule (but simply drops the SA hits for analysis) - so no
symmetric change is needed.

Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/PMG/RuleDB/Spam.pm | 2 +-
 src/PMG/Utils.pm       | 5 +++++
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/src/PMG/RuleDB/Spam.pm b/src/PMG/RuleDB/Spam.pm
index bc1d422..d7bfb63 100644
--- a/src/PMG/RuleDB/Spam.pm
+++ b/src/PMG/RuleDB/Spam.pm
@@ -221,7 +221,7 @@ sub what_match_targets {
 		sa_data => [{
 		    rule => 'USER_IN_BLACKLIST',
 		    score => 100,
-		    desc => 'From: address is in the user black-list',
+		    desc => PMG::Utils::user_bl_description(),
 		}],
 		sa_hits => 'USER_IN_BLACKLIST',
 	    };
diff --git a/src/PMG/Utils.pm b/src/PMG/Utils.pm
index e20fc91..b9b1478 100644
--- a/src/PMG/Utils.pm
+++ b/src/PMG/Utils.pm
@@ -1166,6 +1166,10 @@ sub bencode_header {
     return $res;
 }
 
+sub user_bl_description {
+    return 'From: address is in the user black-list';
+}
+
 sub load_sa_descriptions {
     my ($additional_dirs) = @_;
 
@@ -1202,6 +1206,7 @@ sub load_sa_descriptions {
     }
 
     $res->{'ClamAVHeuristics'}->{desc} = "ClamAV heuristic tests";
+    $res->{'USER_IN_BLACKLIST'}->{desc} = user_bl_description();;
 
     return $res;
 }
-- 
2.30.2

[pmg-devel] [PATCH pmg-api 2/2] user accesslists: reword logging and hits for SA 4.0

[Stoiko Ivanov]

This commit adapts the sa-hits and the logging for the user
block/welcomelist to be consistent with the terms used in the
SpamAssassin 4.0 release, avoiding racially charged terms.

This patch is a (small part) of the fix for #3755, which will be
addressed along with the upgrade to SpamAssassin 4.0 (to be consistent
with the (quite well thought-through) namings used by SA)

keeping the USER_IN_BLACKLIST hit when loading the descriptions
catches mails put in quarantine before the patched version was
installed.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/PMG/RuleDB/Spam.pm | 8 ++++----
 src/PMG/Utils.pm       | 3 ++-
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/src/PMG/RuleDB/Spam.pm b/src/PMG/RuleDB/Spam.pm
index d7bfb63..14d7bea 100644
--- a/src/PMG/RuleDB/Spam.pm
+++ b/src/PMG/RuleDB/Spam.pm
@@ -204,7 +204,7 @@ sub what_match_targets {
 	    if ($queue->{blackwhite}->{$pmail} && 
 		($list = $queue->{blackwhite}->{$pmail}->{whitelist}) &&
 		check_addrlist($list, $queue->{all_from_addrs})) {
-		syslog('info', "%s: sender in user (%s) whitelist", 
+		syslog('info', "%s: sender in user (%s) welcomelist",
 		       $queue->{logid}, encode('UTF-8', $pmail));
 	    } else {
 		$target_info->{$t}->{marks} = []; # never add additional marks here
@@ -219,11 +219,11 @@ sub what_match_targets {
 		sa_score => 100,
 		sa_max => $self->{level},
 		sa_data => [{
-		    rule => 'USER_IN_BLACKLIST',
+		    rule => 'USER_IN_BLOCKLIST',
 		    score => 100,
 		    desc => PMG::Utils::user_bl_description(),
 		}],
-		sa_hits => 'USER_IN_BLACKLIST',
+		sa_hits => 'USER_IN_BLOCKLIST',
 	    };
 
 	    my $list;
@@ -233,7 +233,7 @@ sub what_match_targets {
 		check_addrlist($list, $queue->{all_from_addrs})) {
 		$target_info->{$t}->{marks} = [];
 		$target_info->{$t}->{spaminfo} = $info;
-		syslog ('info', "%s: sender in user (%s) blacklist", 
+		syslog ('info', "%s: sender in user (%s) blocklist",
 			$queue->{logid}, encode('UTF-8',$pmail));
 	    }
 	}
diff --git a/src/PMG/Utils.pm b/src/PMG/Utils.pm
index b9b1478..10193f6 100644
--- a/src/PMG/Utils.pm
+++ b/src/PMG/Utils.pm
@@ -1167,7 +1167,7 @@ sub bencode_header {
 }
 
 sub user_bl_description {
-    return 'From: address is in the user black-list';
+    return 'From: address is in the user block-list';
 }
 
 sub load_sa_descriptions {
@@ -1207,6 +1207,7 @@ sub load_sa_descriptions {
 
     $res->{'ClamAVHeuristics'}->{desc} = "ClamAV heuristic tests";
     $res->{'USER_IN_BLACKLIST'}->{desc} = user_bl_description();;
+    $res->{'USER_IN_BLOCKLIST'}->{desc} = user_bl_description();;
 
     return $res;
 }
-- 
2.30.2

[pmg-devel] applied-series: [PATCH pmg-api 0/2] fix deprecation warning for user blocklist hits

[Thomas Lamprecht]

Am 28/11/2022 um 19:17 schrieb Stoiko Ivanov:
> Stoiko Ivanov (2):
>   user-bl: use custom description of USER_IN_BLACKLIST consistently
>   user accesslists: reword logging and hits for SA 4.0
> 
>  src/PMG/RuleDB/Spam.pm | 10 +++++-----
>  src/PMG/Utils.pm       |  6 ++++++
>  2 files changed, 11 insertions(+), 5 deletions(-)
> 


applied both patches, thanks!