From: Stoiko Ivanov <s.ivanov@proxmox.com>
To: Dominik Csapak <d.csapak@proxmox.com>
Cc: pmg-devel@lists.proxmox.com
Subject: [pmg-devel] applied: [PATCH pmg-api 02/12] RuleCache: reorganize to keep group structure
Date: Tue, 20 Feb 2024 15:45:58 +0100 [thread overview]
Message-ID: <20240220154558.44788a7f@rosa.proxmox.com> (raw)
In-Reply-To: <20240209125440.2572239-3-d.csapak@proxmox.com>
applied, with 2 tiny typos in the commit-message fixed up
Thanks!
On Fri, 9 Feb 2024 13:54:26 +0100
Dominik Csapak <d.csapak@proxmox.com> wrote:
> Currently we 'or' combine all objects of a type (from/to/what/when)
> regardless of group, so we only keep a single list of all objects.
>
> Since we want to introduce different logic (and/invert) we want to keep
> the configured group structure. This patch does this, wihtout chaning
> the current matching logic (still all 'or'-ed).
>
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> src/PMG/RuleCache.pm | 115 ++++++++++++++++++++++++-------------------
> 1 file changed, 64 insertions(+), 51 deletions(-)
>
> diff --git a/src/PMG/RuleCache.pm b/src/PMG/RuleCache.pm
> index 51d8a07..fd22a16 100644
> --- a/src/PMG/RuleCache.pm
> +++ b/src/PMG/RuleCache.pm
> @@ -28,6 +28,14 @@ sub new {
>
> my $sha1 = Digest::SHA->new;
>
> + my $type_map = {
> + 0 => "from",
> + 1 => "to",
> + 2 => "when",
> + 3 => "what",
> + 4 => "action",
> + };
> +
> eval {
> $dbh->begin_work;
>
> @@ -53,7 +61,11 @@ sub new {
> $sha1->add(join(',', $ref->{id}, $ref->{name}, $ref->{priority}, $ref->{active},
> $ref->{direction}) . "|");
>
> - my ($from, $to, $when, $what, $action);
> + $self->{"$ruleid:from"} = { groups => [] };
> + $self->{"$ruleid:to"} = { groups => [] };
> + $self->{"$ruleid:when"} = { groups => [] };
> + $self->{"$ruleid:what"} = { groups => [] };
> + $self->{"$ruleid:action"} = { groups => [] };
>
> my $sth1 = $dbh->prepare(
> "SELECT Objectgroup_ID, Grouptype FROM RuleGroup " .
> @@ -64,20 +76,7 @@ sub new {
> while (my $ref1 = $sth1->fetchrow_hashref()) {
> my $gtype = $ref1->{grouptype};
> my $groupid = $ref1->{objectgroup_id};
> -
> - # emtyp groups differ from non-existent groups!
> -
> - if ($gtype == 0) { #from
> - $from = [] if !defined ($from);
> - } elsif ($gtype == 1) { # to
> - $to = [] if !defined ($to);
> - } elsif ($gtype == 2) { # when
> - $when = [] if !defined ($when);
> - } elsif ($gtype == 3) { # what
> - $what = [] if !defined ($what);
> - } elsif ($gtype == 4) { # action
> - $action = [] if !defined ($action);
> - }
> + my $objects = [];
>
> my $sth2 = $dbh->prepare(
> "SELECT ID FROM Object where Objectgroup_ID = '$groupid' " .
> @@ -90,14 +89,9 @@ sub new {
> $sha1->add (join (',', $objid, $gtype, $groupid) . "|");
> $sha1->add ($obj->{digest}, "|");
>
> - if ($gtype == 0) { #from
> - push @$from, $obj;
> - } elsif ($gtype == 1) { # to
> - push @$to, $obj;
> - } elsif ($gtype == 2) { # when
> - push @$when, $obj;
> - } elsif ($gtype == 3) { # what
> - push @$what, $obj;
> + push @$objects, $obj;
> +
> + if ($gtype == 3) { # what
> if ($obj->otype == PMG::RuleDB::ArchiveFilter->otype ||
> $obj->otype == PMG::RuleDB::MatchArchiveFilename->otype)
> {
> @@ -111,20 +105,20 @@ sub new {
> }
> }
> } elsif ($gtype == 4) { # action
> - push @$action, $obj;
> $self->{"$ruleid:final"} = 1 if $obj->final();
> }
> }
> $sth2->finish();
> +
> + my $group = {
> + objects => $objects,
> + };
> +
> + my $type = $type_map->{$gtype};
> + push $self->{"$ruleid:$type"}->{groups}->@*, $group;
> }
>
> $sth1->finish();
> -
> - $self->{"$ruleid:from"} = $from;
> - $self->{"$ruleid:to"} = $to;
> - $self->{"$ruleid:when"} = $when;
> - $self->{"$ruleid:what"} = $what;
> - $self->{"$ruleid:action"} = $action;
> }
>
> # Cache Greylist Exclusion
> @@ -203,7 +197,15 @@ sub get_actions {
>
> defined($ruleid) || die "undefined rule id: ERROR";
>
> - return $self->{"$ruleid:action"};
> + my $actions = $self->{"$ruleid:action"};
> +
> + return undef if scalar($actions->{groups}->@*) == 0;
> +
> + my $res = [];
> + for my $action ($actions->{groups}->@*) {
> + push $res->@*, $action->{objects}->@*;
> + }
> + return $res;
> }
>
> sub greylist_match {
> @@ -239,15 +241,17 @@ sub from_match {
>
> my $from = $self->{"$ruleid:from"};
>
> - return 1 if !defined ($from);
> + return 1 if scalar($from->{groups}->@*) == 0;
>
> # postfix prefixes ipv6 addresses with IPv6:
> if (defined($ip) && $ip =~ /^IPv6:(.*)/) {
> $ip = $1;
> }
>
> - foreach my $obj (@$from) {
> - return 1 if $obj->who_match($addr, $ip, $ldap);
> + for my $group ($from->{groups}->@*) {
> + for my $obj ($group->{objects}->@*) {
> + return 1 if $obj->who_match($addr, $ip, $ldap);
> + }
> }
>
> return 0;
> @@ -258,12 +262,15 @@ sub to_match {
>
> my $to = $self->{"$ruleid:to"};
>
> - return 1 if !defined ($to);
> + return 1 if scalar($to->{groups}->@*) == 0;
>
> - foreach my $obj (@$to) {
> - return 1 if $obj->who_match($addr, undef, $ldap);
> + for my $group ($to->{groups}->@*) {
> + for my $obj ($group->{objects}->@*) {
> + return 1 if $obj->who_match($addr, undef, $ldap);
> + }
> }
>
> +
> return 0;
> }
>
> @@ -272,10 +279,12 @@ sub when_match {
>
> my $when = $self->{"$ruleid:when"};
>
> - return 1 if !defined ($when);
> + return 1 if scalar($when->{groups}->@*) == 0;
>
> - foreach my $obj (@$when) {
> - return 1 if $obj->when_match($time);
> + for my $group ($when->{groups}->@*) {
> + for my $obj ($group->{objects}->@*) {
> + return 1 if $obj->when_match($time);
> + }
> }
>
> return 0;
> @@ -292,7 +301,7 @@ sub what_match {
> # $res->{$target}->{marks} is only used in apply_rules() to exclude some
> # targets (spam blacklist and whitelist)
>
> - if (!defined ($what)) {
> + if (scalar($what->{groups}->@*) == 0) {
> # match all targets
> foreach my $target (@{$msginfo->{targets}}) {
> $res->{$target}->{marks} = [];
> @@ -304,10 +313,12 @@ sub what_match {
>
> my $marks;
>
> - foreach my $obj (@$what) {
> - if (!$obj->can('what_match_targets')) {
> - if (my $match = $obj->what_match($queue, $element, $msginfo, $dbh)) {
> - push @$marks, @$match;
> + for my $group ($what->{groups}->@*) {
> + for my $obj ($group->{objects}->@*) {
> + if (!$obj->can('what_match_targets')) {
> + if (my $match = $obj->what_match($queue, $element, $msginfo, $dbh)) {
> + push @$marks, @$match;
> + }
> }
> }
> }
> @@ -317,12 +328,14 @@ sub what_match {
> $res->{marks} = $marks;
> }
>
> - foreach my $obj (@$what) {
> - if ($obj->can ("what_match_targets")) {
> - my $target_info;
> - if ($target_info = $obj->what_match_targets($queue, $element, $msginfo, $dbh)) {
> - foreach my $k (keys %$target_info) {
> - $res->{$k} = $target_info->{$k};
> + for my $group ($what->{groups}->@*) {
> + for my $obj ($group->{objects}->@*) {
> + if ($obj->can ("what_match_targets")) {
> + my $target_info;
> + if ($target_info = $obj->what_match_targets($queue, $element, $msginfo, $dbh)) {
> + foreach my $k (keys %$target_info) {
> + $res->{$k} = $target_info->{$k};
> + }
> }
> }
> }
next prev parent reply other threads:[~2024-02-20 14:46 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-09 12:54 [pmg-devel] [PATCH pmg-api/docs/gui] implement and combination and inversion of groups and objects Dominik Csapak
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-api 01/12] RuleCache: remove unnecessary copying of marks Dominik Csapak
2024-02-20 14:42 ` [pmg-devel] applied: " Stoiko Ivanov
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-api 02/12] RuleCache: reorganize to keep group structure Dominik Csapak
2024-02-20 14:45 ` Stoiko Ivanov [this message]
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-api 03/12] RuleCache: reorganize how we gather marks and spaminfo Dominik Csapak
2024-02-20 11:10 ` Stoiko Ivanov
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-api 04/12] api: refactor rule parameters Dominik Csapak
2024-02-20 11:49 ` Stoiko Ivanov
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-api 05/12] add objectgroup attributes and/invert Dominik Csapak
2024-02-20 12:35 ` Stoiko Ivanov
2024-02-20 12:47 ` Stoiko Ivanov
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-api 06/12] add rule attributes and/invert (for each relevant type) Dominik Csapak
2024-02-20 13:03 ` Stoiko Ivanov
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-api 07/12] RuleCache: load rule/objectgroup attributes from database Dominik Csapak
2024-02-20 13:18 ` Stoiko Ivanov
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-api 08/12] RuleCache: implement and/invert for when/from/to Dominik Csapak
2024-02-20 13:09 ` Stoiko Ivanov
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-api 09/12] MailQueue: return maximum AID Dominik Csapak
2024-02-20 13:20 ` Stoiko Ivanov
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-api 10/12] WIP: ModGroup: add possibility to explode to all targets Dominik Csapak
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-api 11/12] RuleCache: implement and/invert for what matches Dominik Csapak
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-api 12/12] pmgdb: extend dump output to include add/invert Dominik Csapak
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-docs 1/2] rule system: add a small section about matching rules Dominik Csapak
2024-02-20 14:47 ` [pmg-devel] applied: " Stoiko Ivanov
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-docs 2/2] rule system: explain new and mode and invert flag Dominik Csapak
2024-02-20 14:40 ` Stoiko Ivanov
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-gui 1/2] rules: use tree panel instead of grouping feature of the grid Dominik Csapak
2024-02-09 12:54 ` [pmg-devel] [PATCH pmg-gui 2/2] rules/objects: add mode selector dropdown Dominik Csapak
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240220154558.44788a7f@rosa.proxmox.com \
--to=s.ivanov@proxmox.com \
--cc=d.csapak@proxmox.com \
--cc=pmg-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox