public inbox for pmg-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pmg-devel] [PATCH pmg-api/pmg-gui] allow / in local part of pmg-email-address
@ 2021-01-15  9:46 Stoiko Ivanov
  2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 1/5] api: statistics: remove unneeded RESTEnvironment Stoiko Ivanov
                   ` (5 more replies)
  0 siblings, 6 replies; 7+ messages in thread
From: Stoiko Ivanov @ 2021-01-15  9:46 UTC (permalink / raw)
  To: pmg-devel

The following patchset is a result of investigating a thread in our
community forum [0].

While I'd consider e-mail addresses with '/' in the local-part quite odd,
as so often smtp RFCs are quite liberal with what smtp servers (have to)
accept. Postfix happily accepts mails with '/' in the local-part, and
should they end up in the quarantine, it is currently not possible to
remove them (short of waiting for quarantine retention period).

the 4 places where we use the pmg-email-address format are:
* quarantine (pmail parameter)
* statistics (contact, receiver, sender detail views)
* fetchmail
* pbsconfig (username)

the first two are problematic in the sense that external sources might
cause a mail-address with '/' to get stored. pbsconfig stores a username
with '/' (which then fails since the repository verification does not
expect a '/' in the username/token), fetchmail accepts it (and stores its
config with all special characters escaped)
only the statistics api calls are problematic, since the detail views pass
the mail-address as path component (and the decoding inside pve-http-server
breaks the api call resolution) - addressing this is the main part of the
patchset.

It follows a similar series by Dominik for the user blocklists [1].

patch 1/5 for the api is a cleanup that caught my eye
patches 2+3 for the api could probably be squashed (happy to send a v2 for
this - but feel free to squash them if this gets applied as is)

tested it a bit on my setup (with a limited set of addresses in the
statistics database).

[0]
https://forum.proxmox.com/threads/pmg-error-parameter-verification-failed-400.82353/
[1] https://lists.proxmox.com/pipermail/pmg-devel/2020-March/000952.html

pmg-api:
Stoiko Ivanov (5):
  api: statistics: remove unneeded RESTEnvironment
  api: statistics: refactor return for detail calls
  api: statistics: refactor detail calls
  api: statistics: make email a parameter
  utils: allow '/' inside email address localpart

 src/PMG/API2/Statistics.pm | 333 +++++++++++++++++++++----------------
 src/PMG/Utils.pm           |   2 +-
 2 files changed, 191 insertions(+), 144 deletions(-)

pmg-gui:
Stoiko Ivanov (1):
  statistics: use new api call for detailed stats

 js/ContactStatistics.js  |  9 ++++-----
 js/ReceiverStatistics.js |  9 ++++-----
 js/SenderStatistics.js   |  9 ++++-----
 js/StatStore.js          | 11 +++++++++--
 4 files changed, 21 insertions(+), 17 deletions(-)

-- 
2.20.1





^ permalink raw reply	[flat|nested] 7+ messages in thread

* [pmg-devel] [PATCH pmg-api 1/5] api: statistics: remove unneeded RESTEnvironment
  2021-01-15  9:46 [pmg-devel] [PATCH pmg-api/pmg-gui] allow / in local part of pmg-email-address Stoiko Ivanov
@ 2021-01-15  9:46 ` Stoiko Ivanov
  2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 2/5] api: statistics: refactor return for detail calls Stoiko Ivanov
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Stoiko Ivanov @ 2021-01-15  9:46 UTC (permalink / raw)
  To: pmg-devel

none of the API calls in PMG::API2::Statistics use the
RESTEnvironment - so remove the unused code.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/PMG/API2/Statistics.pm | 34 ----------------------------------
 1 file changed, 34 deletions(-)

diff --git a/src/PMG/API2/Statistics.pm b/src/PMG/API2/Statistics.pm
index 7eb8d18..adc7650 100644
--- a/src/PMG/API2/Statistics.pm
+++ b/src/PMG/API2/Statistics.pm
@@ -238,9 +238,6 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-	my $cinfo = $restenv->{cinfo};
-
 	my ($start, $end) = $extract_start_end->($param);
 
 	my $cfg = PMG::Config->new();
@@ -383,9 +380,6 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-	my $cinfo = $restenv->{cinfo};
-
 	my ($start, $end) = $extract_start_end->($param);
 
 	my $stat = PMG::Statistic->new($start, $end);
@@ -530,9 +524,6 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-	my $cinfo = $restenv->{cinfo};
-
 	my ($start, $end) = $extract_start_end->($param);
 
 	my $cfg = PMG::Config->new();
@@ -684,9 +675,6 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-	my $cinfo = $restenv->{cinfo};
-
 	my ($start, $end) = $extract_start_end->($param);
 
 	my $stat = PMG::Statistic->new($start, $end);
@@ -791,9 +779,6 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-	my $cinfo = $restenv->{cinfo};
-
 	my ($start, $end) = $extract_start_end->($param);
 
 	my $stat = PMG::Statistic->new($start, $end);
@@ -910,8 +895,6 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-
 	my $hours = $param->{hours} // 12;
 	my $span = $param->{timespan} // 1800;
 
@@ -972,8 +955,6 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-
 	my $hours = $param->{hours} // 12;
 
 	my $limit = $param->{limit} // 5;
@@ -1071,9 +1052,6 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-	my $cinfo = $restenv->{cinfo};
-
 	my ($start, $end) = $extract_start_end->($param);
 
 	my $span = $param->{timespan} // 3600;
@@ -1138,9 +1116,6 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-	my $cinfo = $restenv->{cinfo};
-
 	my ($start, $end) = $extract_start_end->($param);
 
 	my $stat = PMG::Statistic->new($start, $end);
@@ -1185,9 +1160,6 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-	my $cinfo = $restenv->{cinfo};
-
 	my ($start, $end) = $extract_start_end->($param);
 
 	my $stat = PMG::Statistic->new($start, $end);
@@ -1284,9 +1256,6 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-	my $cinfo = $restenv->{cinfo};
-
 	my ($start, $end) = $extract_start_end->($param);
 
 	my $stat = PMG::Statistic->new($start, $end);
@@ -1345,9 +1314,6 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-	my $cinfo = $restenv->{cinfo};
-
 	my ($start, $end) = $extract_start_end->($param);
 
 	my $span = $param->{timespan} // 3600;
-- 
2.20.1





^ permalink raw reply	[flat|nested] 7+ messages in thread

* [pmg-devel] [PATCH pmg-api 2/5] api: statistics: refactor return for detail calls
  2021-01-15  9:46 [pmg-devel] [PATCH pmg-api/pmg-gui] allow / in local part of pmg-email-address Stoiko Ivanov
  2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 1/5] api: statistics: remove unneeded RESTEnvironment Stoiko Ivanov
@ 2021-01-15  9:46 ` Stoiko Ivanov
  2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 3/5] api: statistics: refactor " Stoiko Ivanov
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Stoiko Ivanov @ 2021-01-15  9:46 UTC (permalink / raw)
  To: pmg-devel

all api methods returning information for a particular sender,
receiver or contact have similar returns.

This commit pulls the common ones out into a hash for reusing them.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/PMG/API2/Statistics.pm | 90 ++++++++++++--------------------------
 1 file changed, 27 insertions(+), 63 deletions(-)

diff --git a/src/PMG/API2/Statistics.pm b/src/PMG/API2/Statistics.pm
index adc7650..0d12dd3 100644
--- a/src/PMG/API2/Statistics.pm
+++ b/src/PMG/API2/Statistics.pm
@@ -257,6 +257,30 @@ __PACKAGE__->register_method ({
 	return $res;
     }});
 
+my %detail_return_properties = (
+    time => {
+	description => "Receive time stamp",
+	type => 'integer',
+    },
+    bytes => {
+	description => "Mail traffic (Bytes).",
+	type => 'number',
+    },
+    blocked => {
+	description => "Mail was blocked.",
+	type => 'boolean',
+    },
+    spamlevel => {
+	description => "Spam score.",
+	type => 'number',
+    },
+    virusinfo => {
+	description => "Virus name.",
+	type => 'string',
+	optional => 1,
+    },
+);
+
 __PACKAGE__->register_method ({
     name => 'contactdetails',
     path => 'contact/{contact}',
@@ -283,31 +307,11 @@ __PACKAGE__->register_method ({
 	items => {
 	    type => "object",
 	    properties => {
-		time => {
-		    description => "Receive time stamp",
-		    type => 'integer',
-		},
+		%detail_return_properties,
 		sender => {
 		    description => "Sender email.",
 		    type => 'string',
 		},
-		bytes => {
-		    description => "Mail traffic (Bytes).",
-		    type => 'number',
-		},
-		blocked => {
-		    description => "Mail was blocked.",
-		    type => 'boolean',
-		},
-		spamlevel => {
-		    description => "Spam score.",
-		    type => 'number',
-		},
-		virusinfo => {
-		    description => "Virus name.",
-		    type => 'string',
-		    optional => 1,
-		},
 	    },
 	},
     },
@@ -422,31 +426,11 @@ __PACKAGE__->register_method ({
 	items => {
 	    type => "object",
 	    properties => {
-		time => {
-		    description => "Receive time stamp",
-		    type => 'integer',
-		},
+		%detail_return_properties,
 		receiver => {
 		    description => "Receiver email.",
 		    type => 'string',
 		},
-		bytes => {
-		    description => "Mail traffic (Bytes).",
-		    type => 'number',
-		},
-		blocked => {
-		    description => "Mail was blocked.",
-		    type => 'boolean',
-		},
-		spamlevel => {
-		    description => "Spam score.",
-		    type => 'number',
-		},
-		virusinfo => {
-		    description => "Virus name.",
-		    type => 'string',
-		    optional => 1,
-		},
 	    },
 	},
     },
@@ -569,31 +553,11 @@ __PACKAGE__->register_method ({
 	items => {
 	    type => "object",
 	    properties => {
-		time => {
-		    description => "Receive time stamp",
-		    type => 'integer',
-		},
+		%detail_return_properties,
 		sender => {
 		    description => "Sender email.",
 		    type => 'string',
 		},
-		bytes => {
-		    description => "Mail traffic (Bytes).",
-		    type => 'number',
-		},
-		blocked => {
-		    description => "Mail was blocked.",
-		    type => 'boolean',
-		},
-		spamlevel => {
-		    description => "Spam score.",
-		    type => 'number',
-		},
-		virusinfo => {
-		    description => "Virus name.",
-		    type => 'string',
-		    optional => 1,
-		},
 	    },
 	},
     },
-- 
2.20.1





^ permalink raw reply	[flat|nested] 7+ messages in thread

* [pmg-devel] [PATCH pmg-api 3/5] api: statistics: refactor detail calls
  2021-01-15  9:46 [pmg-devel] [PATCH pmg-api/pmg-gui] allow / in local part of pmg-email-address Stoiko Ivanov
  2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 1/5] api: statistics: remove unneeded RESTEnvironment Stoiko Ivanov
  2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 2/5] api: statistics: refactor return for detail calls Stoiko Ivanov
@ 2021-01-15  9:46 ` Stoiko Ivanov
  2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 4/5] api: statistics: make email a parameter Stoiko Ivanov
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Stoiko Ivanov @ 2021-01-15  9:46 UTC (permalink / raw)
  To: pmg-devel

the API calls returning the detailed statistics for a particular
email use much common code.
This patch introduces a sub to be used in all detail api calls

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/PMG/API2/Statistics.pm | 82 ++++++++++++++++----------------------
 1 file changed, 34 insertions(+), 48 deletions(-)

diff --git a/src/PMG/API2/Statistics.pm b/src/PMG/API2/Statistics.pm
index 0d12dd3..ffddbbe 100644
--- a/src/PMG/API2/Statistics.pm
+++ b/src/PMG/API2/Statistics.pm
@@ -281,6 +281,37 @@ my %detail_return_properties = (
     },
 );
 
+sub get_detail_statistics {
+    my ($type, $param) =@_;
+
+    my ($start, $end) = $extract_start_end->($param);
+
+    my $stat = PMG::Statistic->new($start, $end);
+    my $rdb = PMG::RuleDB->new();
+
+    my $sorters = [];
+    if ($param->{orderby}) {
+	my $props = ['time', 'sender', 'bytes', 'blocked', 'spamlevel', 'virusinfo'];
+	$props->[1] = 'receiver' if $type eq 'sender';
+	$sorters = $decode_orderby->($param->{orderby}, $props);
+    }
+
+    my $res = [];
+    if ($type eq 'contact') {
+	$res = $stat->user_stat_contact_details(
+	    $rdb, $param->{contact}, $userstat_limit, $sorters, $param->{filter});
+    } elsif ($type eq 'sender') {
+	$res = $stat->user_stat_sender_details(
+	    $rdb, $param->{sender}, $userstat_limit, $sorters, $param->{filter});
+    } elsif ($type eq 'receiver') {
+	$res = $stat->user_stat_receiver_details(
+	    $rdb, $param->{receiver}, $userstat_limit, $sorters, $param->{filter});
+    } else {
+	die "invalid type provided (not 'contact', 'sender', 'receiver')\n";
+    }
+    return $res;
+}
+
 __PACKAGE__->register_method ({
     name => 'contactdetails',
     path => 'contact/{contact}',
@@ -318,22 +349,7 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-	my $cinfo = $restenv->{cinfo};
-
-	my ($start, $end) = $extract_start_end->($param);
-
-	my $stat = PMG::Statistic->new($start, $end);
-	my $rdb = PMG::RuleDB->new();
-
-	my $sorters = [];
-	if ($param->{orderby}) {
-	    my $props = ['time', 'sender', 'bytes', 'blocked', 'spamlevel', 'virusinfo'];
-	    $sorters = $decode_orderby->($param->{orderby}, $props);
-	}
-
-	return $stat->user_stat_contact_details(
-	    $rdb, $param->{contact}, $userstat_limit, $sorters, $param->{filter});
+	return get_detail_statistics('contact', $param);
     }});
 
 __PACKAGE__->register_method ({
@@ -437,22 +453,7 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-	my $cinfo = $restenv->{cinfo};
-
-	my ($start, $end) = $extract_start_end->($param);
-
-	my $stat = PMG::Statistic->new($start, $end);
-	my $rdb = PMG::RuleDB->new();
-
-	my $sorters = [];
-	if ($param->{orderby}) {
-	    my $props = ['time', 'receiver', 'bytes', 'blocked', 'spamlevel', 'virusinfo'];
-	    $sorters = $decode_orderby->($param->{orderby}, $props);
-	}
-
-	return $stat->user_stat_sender_details(
-	    $rdb, $param->{sender}, $userstat_limit, $sorters, $param->{filter});
+	return get_detail_statistics('sender', $param);
     }});
 
 __PACKAGE__->register_method ({
@@ -564,22 +565,7 @@ __PACKAGE__->register_method ({
     code => sub {
 	my ($param) = @_;
 
-	my $restenv = PMG::RESTEnvironment->get();
-	my $cinfo = $restenv->{cinfo};
-
-	my ($start, $end) = $extract_start_end->($param);
-
-	my $stat = PMG::Statistic->new($start, $end);
-	my $rdb = PMG::RuleDB->new();
-
-	my $sorters = [];
-	if ($param->{orderby}) {
-	    my $props = ['time', 'sender', 'bytes', 'blocked', 'spamlevel', 'virusinfo'];
-	    $sorters = $decode_orderby->($param->{orderby}, $props);
-	}
-
-	return $stat->user_stat_receiver_details(
-	    $rdb, $param->{receiver}, $userstat_limit, $sorters, $param->{filter});
+	return get_detail_statistics('receiver', $param);
     }});
 
 __PACKAGE__->register_method ({
-- 
2.20.1





^ permalink raw reply	[flat|nested] 7+ messages in thread

* [pmg-devel] [PATCH pmg-api 4/5] api: statistics: make email a parameter
  2021-01-15  9:46 [pmg-devel] [PATCH pmg-api/pmg-gui] allow / in local part of pmg-email-address Stoiko Ivanov
                   ` (2 preceding siblings ...)
  2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 3/5] api: statistics: refactor " Stoiko Ivanov
@ 2021-01-15  9:46 ` Stoiko Ivanov
  2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 5/5] utils: allow '/' inside email address localpart Stoiko Ivanov
  2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-gui 1/1] statistics: use new api call for detailed stats Stoiko Ivanov
  5 siblings, 0 replies; 7+ messages in thread
From: Stoiko Ivanov @ 2021-01-15  9:46 UTC (permalink / raw)
  To: pmg-devel

This patch adds additional endpoints for the detailed contact, sender
and receiver statistics.

instead of passing the e-mail address as path component it is taken
as explicit parameter.

the idea follows a similar change for the user blocklists in
e8d909c11faeb5a4f84f39ef50e0eaf8ea65046d

sadly the path-prefix without pattern (/contact, /sender, /receiver)
is already used (for the list of e-mail addresses), thus the
alternative paths /contacts, /senders/, /receivers are introduced
to maintain backward compatibility.

The new api calls allow the statistics to be displayed for addresses
containing '/' in their localpart, once this is permitted in our api
schema.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/PMG/API2/Statistics.pm | 137 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 134 insertions(+), 3 deletions(-)

diff --git a/src/PMG/API2/Statistics.pm b/src/PMG/API2/Statistics.pm
index ffddbbe..e09d746 100644
--- a/src/PMG/API2/Statistics.pm
+++ b/src/PMG/API2/Statistics.pm
@@ -44,6 +44,7 @@ __PACKAGE__->register_method ({
 
 	return [
 	    { name => "contact" },
+	    { name => "contacts" },
 	    { name => "domains" },
 	    { name => "mail" },
 	    { name => "mailcount" },
@@ -52,8 +53,10 @@ __PACKAGE__->register_method ({
 	    { name => "maildistribution" },
 	    { name => "spamscores" },
 	    { name => "sender" },
+	    { name => "senders" },
 	    { name => "rblcount" },
 	    { name => "receiver" },
+	    { name => "receivers" },
 	    { name => "virus" },
 	];
     }});
@@ -312,11 +315,54 @@ sub get_detail_statistics {
     return $res;
 }
 
+__PACKAGE__->register_method ({
+    name => 'contactdetails_base',
+    path => 'contacts',
+    method => 'GET',
+    description => "Detailed Contact Statistics.",
+    permissions => { check => [ 'admin', 'qmanager', 'audit'] },
+    parameters => {
+	additionalProperties => 0,
+	properties => $default_properties->({
+	    contact => get_standard_option('pmg-email-address', {
+		description => "Contact email address.",
+	    }),
+	    filter => {
+		description => "Sender address filter.",
+		type => 'string',
+		maxLength => 512,
+		optional => 1,
+	    },
+	    orderby => $api_properties->{orderby},
+	}),
+    },
+    returns => {
+	type => 'array',
+	items => {
+	    type => "object",
+	    properties => {
+		%detail_return_properties,
+		sender => {
+		    description => "Sender email.",
+		    type => 'string',
+		},
+	    },
+	},
+    },
+    code => sub {
+	my ($param) = @_;
+
+	return get_detail_statistics('contact', $param);
+    }});
+
+
+# FIXME: remove for PMG 7.0 - addresses can contain stuff like '/' which breaks
+# API path resolution, thus we replaced it by above "un-templated" call
 __PACKAGE__->register_method ({
     name => 'contactdetails',
     path => 'contact/{contact}',
     method => 'GET',
-    description => "Detailed Contact Statistics.",
+    description => "Detailed Contact Statistics. DEPRECATED: use /statistics/contacts instead.",
     permissions => { check => [ 'admin', 'qmanager', 'audit'] },
     parameters => {
 	additionalProperties => 0,
@@ -416,11 +462,54 @@ __PACKAGE__->register_method ({
 	return $res;
     }});
 
+__PACKAGE__->register_method ({
+    name => 'senderdetails_base',
+    path => 'senders',
+    method => 'GET',
+    description => "Detailed Sender Statistics.",
+    permissions => { check => [ 'admin', 'qmanager', 'audit'] },
+    parameters => {
+	additionalProperties => 0,
+	properties => $default_properties->({
+	    sender => get_standard_option('pmg-email-address', {
+		description => "Sender email address.",
+	    }),
+	    filter => {
+		description => "Receiver address filter.",
+		type => 'string',
+		maxLength => 512,
+		optional => 1,
+	    },
+	    orderby => $api_properties->{orderby},
+	}),
+    },
+    returns => {
+	type => 'array',
+	items => {
+	    type => "object",
+	    properties => {
+		%detail_return_properties,
+		receiver => {
+		    description => "Receiver email.",
+		    type => 'string',
+		},
+	    },
+	},
+    },
+    code => sub {
+	my ($param) = @_;
+
+	return get_detail_statistics('sender', $param);
+    }});
+
+
+# FIXME: remove for PMG 7.0 - addresses can contain stuff like '/' which breaks
+# API path resolution, thus we replaced it by above "un-templated" call
 __PACKAGE__->register_method ({
     name => 'senderdetails',
     path => 'sender/{sender}',
     method => 'GET',
-    description => "Detailed Sender Statistics.",
+    description => "Detailed Sender Statistics. DEPRECATED: use /statistics/senders instead.",
     permissions => { check => [ 'admin', 'qmanager', 'audit'] },
     parameters => {
 	additionalProperties => 0,
@@ -528,11 +617,53 @@ __PACKAGE__->register_method ({
 	return $res;
     }});
 
+__PACKAGE__->register_method ({
+    name => 'receiverdetails_base',
+    path => 'receivers',
+    method => 'GET',
+    description => "Detailed Receiver Statistics.",
+    permissions => { check => [ 'admin', 'qmanager', 'audit'] },
+    parameters => {
+	additionalProperties => 0,
+	properties => $default_properties->({
+	    receiver => get_standard_option('pmg-email-address', {
+		description => "Receiver email address.",
+	    }),
+	    filter => {
+		description => "Sender address filter.",
+		type => 'string',
+		maxLength => 512,
+		optional => 1,
+	    },
+	    orderby => $api_properties->{orderby},
+	}),
+    },
+    returns => {
+	type => 'array',
+	items => {
+	    type => "object",
+	    properties => {
+		%detail_return_properties,
+		sender => {
+		    description => "Sender email.",
+		    type => 'string',
+		},
+	    },
+	},
+    },
+    code => sub {
+	my ($param) = @_;
+
+	return get_detail_statistics('receiver', $param);
+    }});
+
+# FIXME: remove for PMG 7.0 - addresses can contain stuff like '/' which breaks
+# API path resolution, thus we replaced it by above "un-templated" call
 __PACKAGE__->register_method ({
     name => 'receiverdetails',
     path => 'receiver/{receiver}',
     method => 'GET',
-    description => "Detailed Receiver Statistics.",
+    description => "Detailed Receiver Statistics. DEPRECATED: use /statistics/receivers instead.",
     permissions => { check => [ 'admin', 'qmanager', 'audit'] },
     parameters => {
 	additionalProperties => 0,
-- 
2.20.1





^ permalink raw reply	[flat|nested] 7+ messages in thread

* [pmg-devel] [PATCH pmg-api 5/5] utils: allow '/' inside email address localpart
  2021-01-15  9:46 [pmg-devel] [PATCH pmg-api/pmg-gui] allow / in local part of pmg-email-address Stoiko Ivanov
                   ` (3 preceding siblings ...)
  2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 4/5] api: statistics: make email a parameter Stoiko Ivanov
@ 2021-01-15  9:46 ` Stoiko Ivanov
  2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-gui 1/1] statistics: use new api call for detailed stats Stoiko Ivanov
  5 siblings, 0 replies; 7+ messages in thread
From: Stoiko Ivanov @ 2021-01-15  9:46 UTC (permalink / raw)
  To: pmg-devel

The change is motivated by a report in our community forum [0], where
a mail addressed to an address containing '/' in its local-part ended
up in the quarantine.
This is permitted by RFC5322 ([1]), and, probably more relevant,
happily accepted and processed by postfix.

Once inside the quarantine (or the statistic database) the records cannot
be displayed (due to the parameter verification failure).

This leaves the user unable to delete the quarantined mail.

Apart from the quarantine and statistics the 'pmg-email-address'
format is only used in the PBSConfig and the fetchmail configuration
(both of which are available only to the admin and can be still be
edited irrespective of the set localpart).

[0]
https://forum.proxmox.com/threads/pmg-error-parameter-verification-failed-400.82353/
[1] https://tools.ietf.org/html/rfc5322

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/PMG/Utils.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/PMG/Utils.pm b/src/PMG/Utils.pm
index d3fae9e..96aaecc 100644
--- a/src/PMG/Utils.pm
+++ b/src/PMG/Utils.pm
@@ -117,7 +117,7 @@ PVE::JSONSchema::register_standard_option('username', {
 PVE::JSONSchema::register_standard_option('pmg-email-address', {
     description => "Email Address (allow most characters).",
     type => 'string',
-    pattern => '(?:[^\s\/\\\@]+\@[^\s\/\\\@]+)',
+    pattern => '(?:[^\s\\\@]+\@[^\s\/\\\@]+)',
     maxLength => 512,
     minLength => 3,
 });
-- 
2.20.1





^ permalink raw reply	[flat|nested] 7+ messages in thread

* [pmg-devel] [PATCH pmg-gui 1/1] statistics: use new api call for detailed stats
  2021-01-15  9:46 [pmg-devel] [PATCH pmg-api/pmg-gui] allow / in local part of pmg-email-address Stoiko Ivanov
                   ` (4 preceding siblings ...)
  2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 5/5] utils: allow '/' inside email address localpart Stoiko Ivanov
@ 2021-01-15  9:46 ` Stoiko Ivanov
  5 siblings, 0 replies; 7+ messages in thread
From: Stoiko Ivanov @ 2021-01-15  9:46 UTC (permalink / raw)
  To: pmg-devel

the new api calls take the address for which to display the statistics
as explicit parameter instead of component of the path.

This makes it possible to accept '/' as part of an e-mail address
which is allowed (in the local-part by RFC5322 [0], and accepted by
postfix.

[0] https://tools.ietf.org/html/rfc5322

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 js/ContactStatistics.js  |  9 ++++-----
 js/ReceiverStatistics.js |  9 ++++-----
 js/SenderStatistics.js   |  9 ++++-----
 js/StatStore.js          | 11 +++++++++--
 4 files changed, 21 insertions(+), 17 deletions(-)

diff --git a/js/ContactStatistics.js b/js/ContactStatistics.js
index 6c9d2d4..dcadf4e 100644
--- a/js/ContactStatistics.js
+++ b/js/ContactStatistics.js
@@ -16,10 +16,11 @@ Ext.define('PMG.ContactDetails', {
 
     plugins: 'gridfilters',
 
-    setUrl: function(url, title) {
+    setUrl: function(contact, title) {
 	var me = this;
 
-	me.store.setUrl(url);
+	var url = "/api2/json/statistics/contacts";
+	me.store.setUrl(url, { contact: contact });
 	me.store.setRemoteFilter(url !== undefined);
 	Proxmox.Utils.setErrorMask(me, false);
 	me.store.reload();
@@ -201,9 +202,7 @@ Ext.define('PMG.ContactStatistics', {
 	    var details = this.lookupReference('details');
 	    if (selected.length > 0) {
 		var contact = selected[0].data.contact;
-		var url = "/api2/json/statistics/contact/" +
-		    encodeURIComponent(contact);
-		details.setUrl(url, '<b>' + gettext('Contact') + ':</b> ' + Ext.htmlEncode(contact));
+		details.setUrl(contact, '<b>' + gettext('Contact') + ':</b> ' + Ext.htmlEncode(contact));
 	    } else {
 		details.setUrl();
 	    }
diff --git a/js/ReceiverStatistics.js b/js/ReceiverStatistics.js
index 6378e06..6d66ab2 100644
--- a/js/ReceiverStatistics.js
+++ b/js/ReceiverStatistics.js
@@ -16,10 +16,11 @@ Ext.define('PMG.ReceiverDetails', {
 
     plugins: 'gridfilters',
 
-    setUrl: function(url, title) {
+    setUrl: function(receiver, title) {
 	var me = this;
 
-	me.store.setUrl(url);
+	var url = "/api2/json/statistics/receivers";
+	me.store.setUrl(url, { receiver: receiver });
 	me.store.setRemoteFilter(url !== undefined);
 	Proxmox.Utils.setErrorMask(me, false);
 	me.store.reload();
@@ -213,9 +214,7 @@ Ext.define('PMG.ReceiverStatistics', {
 	    var details = this.lookupReference('details');
 	    if (selected.length > 0) {
 		var receiver = selected[0].data.receiver;
-		var url = "/api2/json/statistics/receiver/" +
-		    encodeURIComponent(receiver);
-		details.setUrl(url, '<b>' + gettext('Receiver') + ':</b> ' + Ext.htmlEncode(receiver));
+		details.setUrl(receiver, '<b>' + gettext('Receiver') + ':</b> ' + Ext.htmlEncode(receiver));
 	    } else {
 		details.setUrl();
 	    }
diff --git a/js/SenderStatistics.js b/js/SenderStatistics.js
index 43c5438..6617026 100644
--- a/js/SenderStatistics.js
+++ b/js/SenderStatistics.js
@@ -16,10 +16,11 @@ Ext.define('PMG.SenderDetails', {
 
     plugins: 'gridfilters',
 
-    setUrl: function(url, title) {
+    setUrl: function(sender, title) {
 	var me = this;
 
-	me.store.setUrl(url);
+	var url = "/api2/json/statistics/senders";
+	me.store.setUrl(url, { sender: sender });
 	me.store.setRemoteFilter(url !== undefined);
 	Proxmox.Utils.setErrorMask(me, false);
 	me.store.reload();
@@ -201,9 +202,7 @@ Ext.define('PMG.SenderStatistics', {
 	    var details = this.lookupReference('details');
 	    if (selected.length > 0) {
 		var sender = selected[0].data.sender;
-		var url = "/api2/json/statistics/sender/" +
-		    encodeURIComponent(sender);
-		details.setUrl(url, '<b>' + gettext('Sender') + ':</b> ' + Ext.htmlEncode(sender));
+		details.setUrl(sender, '<b>' + gettext('Sender') + ':</b> ' + Ext.htmlEncode(sender));
 	    } else {
 		details.setUrl();
 	    }
diff --git a/js/StatStore.js b/js/StatStore.js
index ec11777..0fc1d31 100644
--- a/js/StatStore.js
+++ b/js/StatStore.js
@@ -8,13 +8,17 @@ Ext.define('PMG.data.StatStore', {
 
     includeTimeSpan: false,
 
-    setUrl: function(url) {
+    setUrl: function(url, extraparam) {
 	var me = this;
 
 	me.proxy.abort(); // abort pending requests
 
 	me.staturl = url;
 	me.proxy.extraParams = {};
+	if (extraparam !== undefined) {
+	    me.proxy.extraParams = extraparam;
+	}
+
 	me.setData([]);
     },
 
@@ -38,7 +42,10 @@ Ext.define('PMG.data.StatStore', {
 	}
 
 	me.proxy.url = me.staturl;
-	me.proxy.extraParams = { starttime: ts.starttime, endtime: ts.endtime };
+	Ext.apply(me.proxy.extraParams, {
+	    starttime: ts.starttime,
+	    endtime: ts.endtime,
+	});
 
 	var timespan = 3600;
 	if (me.includeTimeSpan) {
-- 
2.20.1





^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2021-01-15  9:47 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-15  9:46 [pmg-devel] [PATCH pmg-api/pmg-gui] allow / in local part of pmg-email-address Stoiko Ivanov
2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 1/5] api: statistics: remove unneeded RESTEnvironment Stoiko Ivanov
2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 2/5] api: statistics: refactor return for detail calls Stoiko Ivanov
2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 3/5] api: statistics: refactor " Stoiko Ivanov
2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 4/5] api: statistics: make email a parameter Stoiko Ivanov
2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-api 5/5] utils: allow '/' inside email address localpart Stoiko Ivanov
2021-01-15  9:46 ` [pmg-devel] [PATCH pmg-gui 1/1] statistics: use new api call for detailed stats Stoiko Ivanov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal