[pmg-devel] [PATCH pmg-api/gui v3] add quarantine self service button

[pmg-devel] [PATCH pmg-api/gui v3] add quarantine self service button

[Dominik Csapak]

adds an option/api call to request an quarantine link for an
email whose domain is in the relay domains

for now, we do not expose that option to the ui, but this can easily be
added if wanted

NOTES on security:

this adds a world reachable api call, that can potentially send e-mails
to users that belong to a relay domain

we ratelimit 1 request/5sec and 1 request/user/hour so that a dos is infeasible

for now all text is hardcoded, templates could be used later on
(if users want that)

changes from v2:
* introduce ratelimit
* factor out the sending sub (for readability)
* change the gui window to only have an 'OK' button (without reset)

changes from v1:
* move config to 'spamquar' section
* show button also on admin interface

pmg-api:

Dominik Csapak (3):
  refactor domain_regex to Utils
  add 'quarantinelink' to spamquar config
  api2/quarantine: add global sendlink api call

 src/PMG/API2/Quarantine.pm  | 126 ++++++++++++++++++++++++++++++++++++
 src/PMG/CLI/pmgqm.pm        |  29 +--------
 src/PMG/Config.pm           |   6 ++
 src/PMG/HTTPServer.pm       |   1 +
 src/PMG/Service/pmgproxy.pm |   4 ++
 src/PMG/Utils.pm            |  26 ++++++++
 6 files changed, 165 insertions(+), 27 deletions(-)

pmg-gui:

Dominik Csapak (1):
  add 'Request Quarantine Link' Button to LoginView

 js/LoginView.js   | 33 +++++++++++++++++++++++++++++++++
 pmg-index.html.tt |  3 ++-
 2 files changed, 35 insertions(+), 1 deletion(-)

-- 
2.20.1

[pmg-devel] [PATCH pmg-api v3 1/3] refactor domain_regex to Utils

[Dominik Csapak]

we will need this somewhere else later

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/PMG/CLI/pmgqm.pm | 29 ++---------------------------
 src/PMG/Utils.pm     | 26 ++++++++++++++++++++++++++
 2 files changed, 28 insertions(+), 27 deletions(-)

diff --git a/src/PMG/CLI/pmgqm.pm b/src/PMG/CLI/pmgqm.pm
index 937269f..39253db 100755
--- a/src/PMG/CLI/pmgqm.pm
+++ b/src/PMG/CLI/pmgqm.pm
@@ -33,31 +33,6 @@ sub setup_environment {
     PMG::RESTEnvironment->setup_default_cli_env();
 }
 
-sub domain_regex {
-    my ($domains) = @_;
-
-    my @ra;
-    foreach my $d (@$domains) {
-	# skip domains with non-DNS name characters
-	next if $d =~ m/[^A-Za-z0-9\-\.]/;
-	if ($d =~ m/^\.(.*)$/) {
-	    my $dom = $1;
-	    $dom =~ s/\./\\\./g;
-	    push @ra, $dom;
-	    push @ra, "\.\*\\.$dom";
-	} else {
-	    $d =~ s/\./\\\./g;
-	    push @ra, $d;
-	}
-    }
-
-    my $re = join ('|', @ra);
-
-    my $regex = qr/\@($re)$/i;
-
-    return $regex;
-}
-
 sub get_item_data {
     my ($data, $ref) = @_;
 
@@ -145,7 +120,7 @@ __PACKAGE__->register_method ({
 	my $dbh = PMG::DBTools::open_ruledb();
 
 	my $domains = PVE::INotify::read_file('domains');
-	my $domainregex = domain_regex([keys %$domains]);
+	my $domainregex = PMG::Utils::domain_regex([keys %$domains]);
 
 	my $sth = $dbh->prepare(
 	    "SELECT pmail, AVG(spamlevel) as spamlevel, count(*)  FROM CMailStore, CMSReceivers " .
@@ -278,7 +253,7 @@ __PACKAGE__->register_method ({
 	}
 
 	my $domains = PVE::INotify::read_file('domains');
-	my $domainregex = domain_regex([keys %$domains]);
+	my $domainregex = PMG::Utils::domain_regex([keys %$domains]);
 
 	my $template = "spamreport-${reportstyle}.tt";
 	my $found = 0;
diff --git a/src/PMG/Utils.pm b/src/PMG/Utils.pm
index d0654e1..a04d41b 100644
--- a/src/PMG/Utils.pm
+++ b/src/PMG/Utils.pm
@@ -1417,4 +1417,30 @@ sub reload_smtp_filter {
     return kill (10, $pid); # send SIGUSR1
 }
 
+sub domain_regex {
+    my ($domains) = @_;
+
+    my @ra;
+    foreach my $d (@$domains) {
+	# skip domains with non-DNS name characters
+	next if $d =~ m/[^A-Za-z0-9\-\.]/;
+	if ($d =~ m/^\.(.*)$/) {
+	    my $dom = $1;
+	    $dom =~ s/\./\\\./g;
+	    push @ra, $dom;
+	    push @ra, "\.\*\\.$dom";
+	} else {
+	    $d =~ s/\./\\\./g;
+	    push @ra, $d;
+	}
+    }
+
+    my $re = join ('|', @ra);
+
+    my $regex = qr/\@($re)$/i;
+
+    return $regex;
+}
+
+
 1;
-- 
2.20.1

[pmg-devel] [PATCH pmg-api v3 2/3] add 'quarantinelink' to spamquar config

[Dominik Csapak]

to enable the 'Request Quarantine Link' button and api call

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/PMG/Config.pm           | 6 ++++++
 src/PMG/Service/pmgproxy.pm | 4 ++++
 2 files changed, 10 insertions(+)

diff --git a/src/PMG/Config.pm b/src/PMG/Config.pm
index cd69c9c..155990b 100755
--- a/src/PMG/Config.pm
+++ b/src/PMG/Config.pm
@@ -289,6 +289,11 @@ sub properties {
 	    description => "Text for 'From' header in daily spam report mails.",
 	    type => 'string',
 	},
+	quarantinelink => {
+	    description => "Enables user self-service for Quarantine Links. Caution: this is accessible without authentication",
+	    type => 'boolean',
+	    default => 0,
+	},
     };
 }
 
@@ -303,6 +308,7 @@ sub options {
 	allowhrefs => { optional => 1 },
 	port => { optional => 1 },
 	protocol => { optional => 1 },
+	quarantinelink => { optional => 1 },
     };
 }
 
diff --git a/src/PMG/Service/pmgproxy.pm b/src/PMG/Service/pmgproxy.pm
index ea58b50..cec2754 100755
--- a/src/PMG/Service/pmgproxy.pm
+++ b/src/PMG/Service/pmgproxy.pm
@@ -21,6 +21,7 @@ use PVE::APIServer::Utils;
 
 use PMG::HTTPServer;
 use PMG::API2;
+use PMG::Config;
 
 use Template;
 
@@ -227,6 +228,8 @@ sub get_index {
 	$version = $1;
     };
 
+    my $cfg = PMG::Config->new();
+    my $quarantinelink = $cfg->get('spamquar', 'quarantinelink');
 
     $username = '' if !$username;
 
@@ -242,6 +245,7 @@ sub get_index {
 	debug => $args->{debug} || $server->{debug},
 	version => $version,
 	wtversion => $wtversion,
+	quarantinelink => $quarantinelink,
     };
 
     my $template_name;
-- 
2.20.1

[pmg-devel] [PATCH pmg-api v3 3/3] api2/quarantine: add global sendlink api call

[Dominik Csapak]

this api call takes an email, checks it against the relay domains,
and prepares a custom quarantinelink for that email  and sends it there

this has to happen unauthenticated, since the idea is that the user
want to access the quarantine but has no current ticket (and no
old spam report with a ticket)

we rate limit the requests by allowing only a request per 5 seconds
(to prevent dos'ing the internal mail server) and only
one request per user/hour

this api call is disabled by default

if admins want even more ratelimiting, they can setup something
like fail2ban to block hosts hitting this api call often

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/PMG/API2/Quarantine.pm | 126 +++++++++++++++++++++++++++++++++++++
 src/PMG/HTTPServer.pm      |   1 +
 2 files changed, 127 insertions(+)

diff --git a/src/PMG/API2/Quarantine.pm b/src/PMG/API2/Quarantine.pm
index 73fb0ec..2085eec 100644
--- a/src/PMG/API2/Quarantine.pm
+++ b/src/PMG/API2/Quarantine.pm
@@ -8,6 +8,10 @@ use Data::Dumper;
 use Encode;
 use File::Path;
 use IO::File;
+use MIME::Entity;
+use URI::Escape;
+use Time::HiRes qw(usleep gettimeofday tv_interval);
+use File::stat ();
 
 use Mail::Header;
 use Mail::SpamAssassin;
@@ -195,6 +199,7 @@ __PACKAGE__->register_method ({
 	    { name => 'attachment' },
 	    { name => 'listattachments' },
 	    { name => 'download' },
+	    { name => 'sendlink' },
 	];
 
 	return $result;
@@ -1239,4 +1244,125 @@ __PACKAGE__->register_method ({
 	return undef;
     }});
 
+my $link_map_fn = "/run/pmgproxy/quarantinelink.map";
+my $per_user_limit = 60*60; # 1 hour
+
+my sub send_link_mail {
+    my ($cfg, $receiver) = @_;
+
+    my $hostname = PVE::INotify::nodename();
+    my $fqdn = $cfg->get('spamquar', 'hostname') //
+    PVE::Tools::get_fqdn($hostname);
+
+    my $port = $cfg->get('spamquar', 'port') // 8006;
+
+    my $protocol = $cfg->get('spamquar', 'protocol') // 'https';
+
+    my $protocol_fqdn_port = "$protocol://$fqdn";
+    if (($protocol eq 'https' && $port != 443) ||
+	($protocol eq 'http' && $port != 80)) {
+	$protocol_fqdn_port .= ":$port";
+    }
+
+    my $mailfrom = $cfg->get ('spamquar', 'mailfrom') //
+    "Proxmox Mail Gateway <postmaster>";
+
+    my $ticket = PMG::Ticket::assemble_quarantine_ticket($receiver);
+    my $esc_ticket = uri_escape($ticket);
+    my $link = "$protocol_fqdn_port/quarantine?ticket=${esc_ticket}";
+
+    my $text = "Here is your Link for the Spam Quarantine on $fqdn:\n\n$link\n";
+
+    my $mail = MIME::Entity->build(
+	Type    => "text/plain",
+	To      => $receiver,
+	From    => $mailfrom,
+	Subject => "Proxmox Mail Gateway - Quarantine Link",
+	Data    => $text,
+    );
+
+    # we use an empty envelope sender (we dont want to receive NDRs)
+    PMG::Utils::reinject_mail ($mail, '', [$receiver], undef, $fqdn);
+}
+
+__PACKAGE__->register_method ({
+    name =>'sendlink',
+    path => 'sendlink',
+    method => 'POST',
+    description => "Send Quarantine link to given e-mail.",
+    permissions => { user => 'world' },
+    protected => 1,
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    mail => get_standard_option('pmg-email-address'),
+	},
+    },
+    returns => { type => "null" },
+    code => sub {
+	my ($param) = @_;
+
+	my $starttime = [gettimeofday];
+
+	my $cfg = PMG::Config->new();
+	my $is_enabled = $cfg->get('spamquar', 'quarantinelink');
+	if (!$is_enabled) {
+	    die "This feature is not enabled\n";
+	}
+
+	my $stat = File::stat::stat($link_map_fn);
+
+	if (defined($stat) && ($stat->mtime) + 5 > $starttime->[0]) {
+	    die "Too many requests. Please try again later\n";
+	}
+
+	my $domains = PVE::INotify::read_file('domains');
+	my $domainregex = PMG::Utils::domain_regex([keys %$domains]);
+
+	my $receiver = $param->{mail};
+
+	if ($receiver !~ $domainregex) {
+	    return undef; # silently ignore invalid mails
+	}
+
+	PVE::Tools::lock_file_full("${link_map_fn}.lck", 10, 1, sub {
+	    if (-f $link_map_fn) {
+		# check if user is allowed to request mail
+		my $lines = [split("\n", PVE::Tools::file_get_contents($link_map_fn))];
+		for my $line (@$lines) {
+		    next if $line !~ m/^\Q$receiver\E (\d+)$/;
+		    if (($1 + $per_user_limit) > $starttime->[0]) {
+			die "Too many requests for '$receiver', only one request per hour is permitted. ".
+			"Please try again later\n";
+		    } else {
+			last;
+		    }
+		}
+	    }
+	});
+	die $@ if $@;
+
+	# we are allowed to send mail, lock and update file and send
+	PVE::Tools::lock_file("${link_map_fn}.lck", 10, sub {
+	    my $newdata = "";
+	    if (-f $link_map_fn) {
+		my $data = PVE::Tools::file_get_contents($link_map_fn);
+		for my $line (split("\n", $data)) {
+		    if ($line =~ m/^(.*) (\d+)$/) {
+			if (($2 + $per_user_limit) > $starttime->[0]) {
+			    $newdata .= $line . "\n";
+			}
+		    }
+		}
+	    }
+	    $newdata .= "$receiver $starttime->[0]\n";
+	    PVE::Tools::file_set_contents($link_map_fn, $newdata);
+	});
+	die $@ if $@;
+
+	send_link_mail($cfg, $receiver);
+
+	return undef;
+    }});
+
 1;
diff --git a/src/PMG/HTTPServer.pm b/src/PMG/HTTPServer.pm
index eb48b5f..3dc9655 100755
--- a/src/PMG/HTTPServer.pm
+++ b/src/PMG/HTTPServer.pm
@@ -58,6 +58,7 @@ sub auth_handler {
 
     # explicitly allow some calls without auth
     if (($rel_uri eq '/access/domains' && $method eq 'GET') ||
+	($rel_uri eq '/quarantine/sendlink' && ($method eq 'GET' || $method eq 'POST')) ||
 	($rel_uri eq '/access/ticket' && ($method eq 'GET' || $method eq 'POST'))) {
 	$require_auth = 0;
     }
-- 
2.20.1

[pmg-devel] [PATCH pmg-gui v3 1/1] add 'Request Quarantine Link' Button to LoginView

[Dominik Csapak]

if the template has 'quarantinelink' enabled, we
show a button 'Request Quarantine Link' on the quarantine login ui

there a user can enter their e-mail and request a link to the quarantine

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 js/LoginView.js   | 33 +++++++++++++++++++++++++++++++++
 pmg-index.html.tt |  3 ++-
 2 files changed, 35 insertions(+), 1 deletion(-)

diff --git a/js/LoginView.js b/js/LoginView.js
index 8e610aa..4770494 100644
--- a/js/LoginView.js
+++ b/js/LoginView.js
@@ -10,6 +10,8 @@ Ext.define('PMG.LoginView', {
 
 	    let realmfield = me.lookup('realmfield');
 
+	    me.lookup('quarantineButton').setVisible(!!Proxmox.QuarantineLink);
+
 	    if (view.targetview !== 'quarantineview') {
 		return;
 	    }
@@ -65,6 +67,30 @@ Ext.define('PMG.LoginView', {
 	    }
 	},
 
+	openQuarantineLinkWindow: function() {
+	    let me = this;
+	    me.lookup('loginwindow').setVisible(false);
+	    Ext.create('Proxmox.window.Edit', {
+		title: gettext('Request Quarantine Link'),
+		url: '/quarantine/sendlink',
+		isCreate: true,
+		submitText: gettext('OK'),
+		method: 'POST',
+		items: [
+		    {
+			xtype: 'proxmoxtextfield',
+			name: 'mail',
+			fieldLabel: gettext('Your E-Mail'),
+		    },
+		],
+		listeners: {
+		    destroy: function() {
+			me.lookup('loginwindow').show(true);
+		    },
+		},
+	    }).show();
+	},
+
 	control: {
 	    'field[name=lang]': {
 		change: function(f, value) {
@@ -76,6 +102,9 @@ Ext.define('PMG.LoginView', {
 		    window.location.reload();
 		},
 	    },
+	    'button[reference=quarantineButton]': {
+		click: 'openQuarantineLinkWindow',
+	    },
 	    'button[reference=loginButton]': {
 		click: 'submitForm',
 	    },
@@ -172,6 +201,10 @@ Ext.define('PMG.LoginView', {
                         },
 		    ],
 		    buttons: [
+			{
+			    text: gettext('Request Quarantine Link'),
+			    reference: 'quarantineButton',
+			},
 			{
 			    text: gettext('Login'),
 			    reference: 'loginButton',
diff --git a/pmg-index.html.tt b/pmg-index.html.tt
index 4faf0cf..4a29ba2 100644
--- a/pmg-index.html.tt
+++ b/pmg-index.html.tt
@@ -30,7 +30,8 @@
         Setup: { auth_cookie_name: 'PMGAuthCookie' },
         NodeName: '[% nodename %]',
         UserName: '[% username %]',
-        CSRFPreventionToken: '[% token %]'
+        CSRFPreventionToken: '[% token %]',
+        QuarantineLink: [% IF quarantinelink %] true [% ELSE %] false [% END %],
       };
     </script>
     <script type="text/javascript" src="/proxmoxlib.js?ver=[% wtversion %]"></script>
-- 
2.20.1

[pmg-devel] applied: [PATCH pmg-api/gui v3] add quarantine self service button

[Thomas Lamprecht]

On 18.11.20 11:59, Dominik Csapak wrote:
> adds an option/api call to request an quarantine link for an
> email whose domain is in the relay domains
> 
> for now, we do not expose that option to the ui, but this can easily be
> added if wanted
> 
> NOTES on security:
> 
> this adds a world reachable api call, that can potentially send e-mails
> to users that belong to a relay domain
> 
> we ratelimit 1 request/5sec and 1 request/user/hour so that a dos is infeasible
> 
> for now all text is hardcoded, templates could be used later on
> (if users want that)
> 
> changes from v2:
> * introduce ratelimit
> * factor out the sending sub (for readability)
> * change the gui window to only have an 'OK' button (without reset)
> 
> changes from v1:
> * move config to 'spamquar' section
> * show button also on admin interface
> 
> pmg-api:
> 
> Dominik Csapak (3):
>   refactor domain_regex to Utils
>   add 'quarantinelink' to spamquar config
>   api2/quarantine: add global sendlink api call
> 
>  src/PMG/API2/Quarantine.pm  | 126 ++++++++++++++++++++++++++++++++++++
>  src/PMG/CLI/pmgqm.pm        |  29 +--------
>  src/PMG/Config.pm           |   6 ++
>  src/PMG/HTTPServer.pm       |   1 +
>  src/PMG/Service/pmgproxy.pm |   4 ++
>  src/PMG/Utils.pm            |  26 ++++++++
>  6 files changed, 165 insertions(+), 27 deletions(-)
> 
> pmg-gui:
> 
> Dominik Csapak (1):
>   add 'Request Quarantine Link' Button to LoginView
> 
>  js/LoginView.js   | 33 +++++++++++++++++++++++++++++++++
>  pmg-index.html.tt |  3 ++-
>  2 files changed, 35 insertions(+), 1 deletion(-)
> 



applied, with some followups:
* use built-in time() to get seconds since epoch
* sleep a bit, especially more on the rate limit cases
* code consitency

one thing which feels a bit "flawed" is the fact that if one knows or guesses
correctly a valid domain, the can DOS this requester for all other valid ones by
simply looping and sending a request for "$name++@$domain" where $name gets changed
slightly each round - this way others race with them to get in between the small
timeframe where the mtime is old enough again and the "attacker" gets through a
request again.

But, this can be somewhat solved by fail2ban and it's opt-in anyway.