* [pmg-devel] [PATCH pmg-api v3 1/3] refactor domain_regex to Utils
2020-11-18 10:59 [pmg-devel] [PATCH pmg-api/gui v3] add quarantine self service button Dominik Csapak
@ 2020-11-18 10:59 ` Dominik Csapak
2020-11-18 10:59 ` [pmg-devel] [PATCH pmg-api v3 2/3] add 'quarantinelink' to spamquar config Dominik Csapak
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Dominik Csapak @ 2020-11-18 10:59 UTC (permalink / raw)
To: pmg-devel
we will need this somewhere else later
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/PMG/CLI/pmgqm.pm | 29 ++---------------------------
src/PMG/Utils.pm | 26 ++++++++++++++++++++++++++
2 files changed, 28 insertions(+), 27 deletions(-)
diff --git a/src/PMG/CLI/pmgqm.pm b/src/PMG/CLI/pmgqm.pm
index 937269f..39253db 100755
--- a/src/PMG/CLI/pmgqm.pm
+++ b/src/PMG/CLI/pmgqm.pm
@@ -33,31 +33,6 @@ sub setup_environment {
PMG::RESTEnvironment->setup_default_cli_env();
}
-sub domain_regex {
- my ($domains) = @_;
-
- my @ra;
- foreach my $d (@$domains) {
- # skip domains with non-DNS name characters
- next if $d =~ m/[^A-Za-z0-9\-\.]/;
- if ($d =~ m/^\.(.*)$/) {
- my $dom = $1;
- $dom =~ s/\./\\\./g;
- push @ra, $dom;
- push @ra, "\.\*\\.$dom";
- } else {
- $d =~ s/\./\\\./g;
- push @ra, $d;
- }
- }
-
- my $re = join ('|', @ra);
-
- my $regex = qr/\@($re)$/i;
-
- return $regex;
-}
-
sub get_item_data {
my ($data, $ref) = @_;
@@ -145,7 +120,7 @@ __PACKAGE__->register_method ({
my $dbh = PMG::DBTools::open_ruledb();
my $domains = PVE::INotify::read_file('domains');
- my $domainregex = domain_regex([keys %$domains]);
+ my $domainregex = PMG::Utils::domain_regex([keys %$domains]);
my $sth = $dbh->prepare(
"SELECT pmail, AVG(spamlevel) as spamlevel, count(*) FROM CMailStore, CMSReceivers " .
@@ -278,7 +253,7 @@ __PACKAGE__->register_method ({
}
my $domains = PVE::INotify::read_file('domains');
- my $domainregex = domain_regex([keys %$domains]);
+ my $domainregex = PMG::Utils::domain_regex([keys %$domains]);
my $template = "spamreport-${reportstyle}.tt";
my $found = 0;
diff --git a/src/PMG/Utils.pm b/src/PMG/Utils.pm
index d0654e1..a04d41b 100644
--- a/src/PMG/Utils.pm
+++ b/src/PMG/Utils.pm
@@ -1417,4 +1417,30 @@ sub reload_smtp_filter {
return kill (10, $pid); # send SIGUSR1
}
+sub domain_regex {
+ my ($domains) = @_;
+
+ my @ra;
+ foreach my $d (@$domains) {
+ # skip domains with non-DNS name characters
+ next if $d =~ m/[^A-Za-z0-9\-\.]/;
+ if ($d =~ m/^\.(.*)$/) {
+ my $dom = $1;
+ $dom =~ s/\./\\\./g;
+ push @ra, $dom;
+ push @ra, "\.\*\\.$dom";
+ } else {
+ $d =~ s/\./\\\./g;
+ push @ra, $d;
+ }
+ }
+
+ my $re = join ('|', @ra);
+
+ my $regex = qr/\@($re)$/i;
+
+ return $regex;
+}
+
+
1;
--
2.20.1
^ permalink raw reply [flat|nested] 6+ messages in thread* [pmg-devel] [PATCH pmg-api v3 2/3] add 'quarantinelink' to spamquar config
2020-11-18 10:59 [pmg-devel] [PATCH pmg-api/gui v3] add quarantine self service button Dominik Csapak
2020-11-18 10:59 ` [pmg-devel] [PATCH pmg-api v3 1/3] refactor domain_regex to Utils Dominik Csapak
@ 2020-11-18 10:59 ` Dominik Csapak
2020-11-18 10:59 ` [pmg-devel] [PATCH pmg-api v3 3/3] api2/quarantine: add global sendlink api call Dominik Csapak
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Dominik Csapak @ 2020-11-18 10:59 UTC (permalink / raw)
To: pmg-devel
to enable the 'Request Quarantine Link' button and api call
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/PMG/Config.pm | 6 ++++++
src/PMG/Service/pmgproxy.pm | 4 ++++
2 files changed, 10 insertions(+)
diff --git a/src/PMG/Config.pm b/src/PMG/Config.pm
index cd69c9c..155990b 100755
--- a/src/PMG/Config.pm
+++ b/src/PMG/Config.pm
@@ -289,6 +289,11 @@ sub properties {
description => "Text for 'From' header in daily spam report mails.",
type => 'string',
},
+ quarantinelink => {
+ description => "Enables user self-service for Quarantine Links. Caution: this is accessible without authentication",
+ type => 'boolean',
+ default => 0,
+ },
};
}
@@ -303,6 +308,7 @@ sub options {
allowhrefs => { optional => 1 },
port => { optional => 1 },
protocol => { optional => 1 },
+ quarantinelink => { optional => 1 },
};
}
diff --git a/src/PMG/Service/pmgproxy.pm b/src/PMG/Service/pmgproxy.pm
index ea58b50..cec2754 100755
--- a/src/PMG/Service/pmgproxy.pm
+++ b/src/PMG/Service/pmgproxy.pm
@@ -21,6 +21,7 @@ use PVE::APIServer::Utils;
use PMG::HTTPServer;
use PMG::API2;
+use PMG::Config;
use Template;
@@ -227,6 +228,8 @@ sub get_index {
$version = $1;
};
+ my $cfg = PMG::Config->new();
+ my $quarantinelink = $cfg->get('spamquar', 'quarantinelink');
$username = '' if !$username;
@@ -242,6 +245,7 @@ sub get_index {
debug => $args->{debug} || $server->{debug},
version => $version,
wtversion => $wtversion,
+ quarantinelink => $quarantinelink,
};
my $template_name;
--
2.20.1
^ permalink raw reply [flat|nested] 6+ messages in thread* [pmg-devel] [PATCH pmg-api v3 3/3] api2/quarantine: add global sendlink api call
2020-11-18 10:59 [pmg-devel] [PATCH pmg-api/gui v3] add quarantine self service button Dominik Csapak
2020-11-18 10:59 ` [pmg-devel] [PATCH pmg-api v3 1/3] refactor domain_regex to Utils Dominik Csapak
2020-11-18 10:59 ` [pmg-devel] [PATCH pmg-api v3 2/3] add 'quarantinelink' to spamquar config Dominik Csapak
@ 2020-11-18 10:59 ` Dominik Csapak
2020-11-18 10:59 ` [pmg-devel] [PATCH pmg-gui v3 1/1] add 'Request Quarantine Link' Button to LoginView Dominik Csapak
2020-11-18 16:56 ` [pmg-devel] applied: [PATCH pmg-api/gui v3] add quarantine self service button Thomas Lamprecht
4 siblings, 0 replies; 6+ messages in thread
From: Dominik Csapak @ 2020-11-18 10:59 UTC (permalink / raw)
To: pmg-devel
this api call takes an email, checks it against the relay domains,
and prepares a custom quarantinelink for that email and sends it there
this has to happen unauthenticated, since the idea is that the user
want to access the quarantine but has no current ticket (and no
old spam report with a ticket)
we rate limit the requests by allowing only a request per 5 seconds
(to prevent dos'ing the internal mail server) and only
one request per user/hour
this api call is disabled by default
if admins want even more ratelimiting, they can setup something
like fail2ban to block hosts hitting this api call often
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/PMG/API2/Quarantine.pm | 126 +++++++++++++++++++++++++++++++++++++
src/PMG/HTTPServer.pm | 1 +
2 files changed, 127 insertions(+)
diff --git a/src/PMG/API2/Quarantine.pm b/src/PMG/API2/Quarantine.pm
index 73fb0ec..2085eec 100644
--- a/src/PMG/API2/Quarantine.pm
+++ b/src/PMG/API2/Quarantine.pm
@@ -8,6 +8,10 @@ use Data::Dumper;
use Encode;
use File::Path;
use IO::File;
+use MIME::Entity;
+use URI::Escape;
+use Time::HiRes qw(usleep gettimeofday tv_interval);
+use File::stat ();
use Mail::Header;
use Mail::SpamAssassin;
@@ -195,6 +199,7 @@ __PACKAGE__->register_method ({
{ name => 'attachment' },
{ name => 'listattachments' },
{ name => 'download' },
+ { name => 'sendlink' },
];
return $result;
@@ -1239,4 +1244,125 @@ __PACKAGE__->register_method ({
return undef;
}});
+my $link_map_fn = "/run/pmgproxy/quarantinelink.map";
+my $per_user_limit = 60*60; # 1 hour
+
+my sub send_link_mail {
+ my ($cfg, $receiver) = @_;
+
+ my $hostname = PVE::INotify::nodename();
+ my $fqdn = $cfg->get('spamquar', 'hostname') //
+ PVE::Tools::get_fqdn($hostname);
+
+ my $port = $cfg->get('spamquar', 'port') // 8006;
+
+ my $protocol = $cfg->get('spamquar', 'protocol') // 'https';
+
+ my $protocol_fqdn_port = "$protocol://$fqdn";
+ if (($protocol eq 'https' && $port != 443) ||
+ ($protocol eq 'http' && $port != 80)) {
+ $protocol_fqdn_port .= ":$port";
+ }
+
+ my $mailfrom = $cfg->get ('spamquar', 'mailfrom') //
+ "Proxmox Mail Gateway <postmaster>";
+
+ my $ticket = PMG::Ticket::assemble_quarantine_ticket($receiver);
+ my $esc_ticket = uri_escape($ticket);
+ my $link = "$protocol_fqdn_port/quarantine?ticket=${esc_ticket}";
+
+ my $text = "Here is your Link for the Spam Quarantine on $fqdn:\n\n$link\n";
+
+ my $mail = MIME::Entity->build(
+ Type => "text/plain",
+ To => $receiver,
+ From => $mailfrom,
+ Subject => "Proxmox Mail Gateway - Quarantine Link",
+ Data => $text,
+ );
+
+ # we use an empty envelope sender (we dont want to receive NDRs)
+ PMG::Utils::reinject_mail ($mail, '', [$receiver], undef, $fqdn);
+}
+
+__PACKAGE__->register_method ({
+ name =>'sendlink',
+ path => 'sendlink',
+ method => 'POST',
+ description => "Send Quarantine link to given e-mail.",
+ permissions => { user => 'world' },
+ protected => 1,
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ mail => get_standard_option('pmg-email-address'),
+ },
+ },
+ returns => { type => "null" },
+ code => sub {
+ my ($param) = @_;
+
+ my $starttime = [gettimeofday];
+
+ my $cfg = PMG::Config->new();
+ my $is_enabled = $cfg->get('spamquar', 'quarantinelink');
+ if (!$is_enabled) {
+ die "This feature is not enabled\n";
+ }
+
+ my $stat = File::stat::stat($link_map_fn);
+
+ if (defined($stat) && ($stat->mtime) + 5 > $starttime->[0]) {
+ die "Too many requests. Please try again later\n";
+ }
+
+ my $domains = PVE::INotify::read_file('domains');
+ my $domainregex = PMG::Utils::domain_regex([keys %$domains]);
+
+ my $receiver = $param->{mail};
+
+ if ($receiver !~ $domainregex) {
+ return undef; # silently ignore invalid mails
+ }
+
+ PVE::Tools::lock_file_full("${link_map_fn}.lck", 10, 1, sub {
+ if (-f $link_map_fn) {
+ # check if user is allowed to request mail
+ my $lines = [split("\n", PVE::Tools::file_get_contents($link_map_fn))];
+ for my $line (@$lines) {
+ next if $line !~ m/^\Q$receiver\E (\d+)$/;
+ if (($1 + $per_user_limit) > $starttime->[0]) {
+ die "Too many requests for '$receiver', only one request per hour is permitted. ".
+ "Please try again later\n";
+ } else {
+ last;
+ }
+ }
+ }
+ });
+ die $@ if $@;
+
+ # we are allowed to send mail, lock and update file and send
+ PVE::Tools::lock_file("${link_map_fn}.lck", 10, sub {
+ my $newdata = "";
+ if (-f $link_map_fn) {
+ my $data = PVE::Tools::file_get_contents($link_map_fn);
+ for my $line (split("\n", $data)) {
+ if ($line =~ m/^(.*) (\d+)$/) {
+ if (($2 + $per_user_limit) > $starttime->[0]) {
+ $newdata .= $line . "\n";
+ }
+ }
+ }
+ }
+ $newdata .= "$receiver $starttime->[0]\n";
+ PVE::Tools::file_set_contents($link_map_fn, $newdata);
+ });
+ die $@ if $@;
+
+ send_link_mail($cfg, $receiver);
+
+ return undef;
+ }});
+
1;
diff --git a/src/PMG/HTTPServer.pm b/src/PMG/HTTPServer.pm
index eb48b5f..3dc9655 100755
--- a/src/PMG/HTTPServer.pm
+++ b/src/PMG/HTTPServer.pm
@@ -58,6 +58,7 @@ sub auth_handler {
# explicitly allow some calls without auth
if (($rel_uri eq '/access/domains' && $method eq 'GET') ||
+ ($rel_uri eq '/quarantine/sendlink' && ($method eq 'GET' || $method eq 'POST')) ||
($rel_uri eq '/access/ticket' && ($method eq 'GET' || $method eq 'POST'))) {
$require_auth = 0;
}
--
2.20.1
^ permalink raw reply [flat|nested] 6+ messages in thread* [pmg-devel] [PATCH pmg-gui v3 1/1] add 'Request Quarantine Link' Button to LoginView
2020-11-18 10:59 [pmg-devel] [PATCH pmg-api/gui v3] add quarantine self service button Dominik Csapak
` (2 preceding siblings ...)
2020-11-18 10:59 ` [pmg-devel] [PATCH pmg-api v3 3/3] api2/quarantine: add global sendlink api call Dominik Csapak
@ 2020-11-18 10:59 ` Dominik Csapak
2020-11-18 16:56 ` [pmg-devel] applied: [PATCH pmg-api/gui v3] add quarantine self service button Thomas Lamprecht
4 siblings, 0 replies; 6+ messages in thread
From: Dominik Csapak @ 2020-11-18 10:59 UTC (permalink / raw)
To: pmg-devel
if the template has 'quarantinelink' enabled, we
show a button 'Request Quarantine Link' on the quarantine login ui
there a user can enter their e-mail and request a link to the quarantine
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
js/LoginView.js | 33 +++++++++++++++++++++++++++++++++
pmg-index.html.tt | 3 ++-
2 files changed, 35 insertions(+), 1 deletion(-)
diff --git a/js/LoginView.js b/js/LoginView.js
index 8e610aa..4770494 100644
--- a/js/LoginView.js
+++ b/js/LoginView.js
@@ -10,6 +10,8 @@ Ext.define('PMG.LoginView', {
let realmfield = me.lookup('realmfield');
+ me.lookup('quarantineButton').setVisible(!!Proxmox.QuarantineLink);
+
if (view.targetview !== 'quarantineview') {
return;
}
@@ -65,6 +67,30 @@ Ext.define('PMG.LoginView', {
}
},
+ openQuarantineLinkWindow: function() {
+ let me = this;
+ me.lookup('loginwindow').setVisible(false);
+ Ext.create('Proxmox.window.Edit', {
+ title: gettext('Request Quarantine Link'),
+ url: '/quarantine/sendlink',
+ isCreate: true,
+ submitText: gettext('OK'),
+ method: 'POST',
+ items: [
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'mail',
+ fieldLabel: gettext('Your E-Mail'),
+ },
+ ],
+ listeners: {
+ destroy: function() {
+ me.lookup('loginwindow').show(true);
+ },
+ },
+ }).show();
+ },
+
control: {
'field[name=lang]': {
change: function(f, value) {
@@ -76,6 +102,9 @@ Ext.define('PMG.LoginView', {
window.location.reload();
},
},
+ 'button[reference=quarantineButton]': {
+ click: 'openQuarantineLinkWindow',
+ },
'button[reference=loginButton]': {
click: 'submitForm',
},
@@ -172,6 +201,10 @@ Ext.define('PMG.LoginView', {
},
],
buttons: [
+ {
+ text: gettext('Request Quarantine Link'),
+ reference: 'quarantineButton',
+ },
{
text: gettext('Login'),
reference: 'loginButton',
diff --git a/pmg-index.html.tt b/pmg-index.html.tt
index 4faf0cf..4a29ba2 100644
--- a/pmg-index.html.tt
+++ b/pmg-index.html.tt
@@ -30,7 +30,8 @@
Setup: { auth_cookie_name: 'PMGAuthCookie' },
NodeName: '[% nodename %]',
UserName: '[% username %]',
- CSRFPreventionToken: '[% token %]'
+ CSRFPreventionToken: '[% token %]',
+ QuarantineLink: [% IF quarantinelink %] true [% ELSE %] false [% END %],
};
</script>
<script type="text/javascript" src="/proxmoxlib.js?ver=[% wtversion %]"></script>
--
2.20.1
^ permalink raw reply [flat|nested] 6+ messages in thread* [pmg-devel] applied: [PATCH pmg-api/gui v3] add quarantine self service button
2020-11-18 10:59 [pmg-devel] [PATCH pmg-api/gui v3] add quarantine self service button Dominik Csapak
` (3 preceding siblings ...)
2020-11-18 10:59 ` [pmg-devel] [PATCH pmg-gui v3 1/1] add 'Request Quarantine Link' Button to LoginView Dominik Csapak
@ 2020-11-18 16:56 ` Thomas Lamprecht
4 siblings, 0 replies; 6+ messages in thread
From: Thomas Lamprecht @ 2020-11-18 16:56 UTC (permalink / raw)
To: Dominik Csapak, pmg-devel
On 18.11.20 11:59, Dominik Csapak wrote:
> adds an option/api call to request an quarantine link for an
> email whose domain is in the relay domains
>
> for now, we do not expose that option to the ui, but this can easily be
> added if wanted
>
> NOTES on security:
>
> this adds a world reachable api call, that can potentially send e-mails
> to users that belong to a relay domain
>
> we ratelimit 1 request/5sec and 1 request/user/hour so that a dos is infeasible
>
> for now all text is hardcoded, templates could be used later on
> (if users want that)
>
> changes from v2:
> * introduce ratelimit
> * factor out the sending sub (for readability)
> * change the gui window to only have an 'OK' button (without reset)
>
> changes from v1:
> * move config to 'spamquar' section
> * show button also on admin interface
>
> pmg-api:
>
> Dominik Csapak (3):
> refactor domain_regex to Utils
> add 'quarantinelink' to spamquar config
> api2/quarantine: add global sendlink api call
>
> src/PMG/API2/Quarantine.pm | 126 ++++++++++++++++++++++++++++++++++++
> src/PMG/CLI/pmgqm.pm | 29 +--------
> src/PMG/Config.pm | 6 ++
> src/PMG/HTTPServer.pm | 1 +
> src/PMG/Service/pmgproxy.pm | 4 ++
> src/PMG/Utils.pm | 26 ++++++++
> 6 files changed, 165 insertions(+), 27 deletions(-)
>
> pmg-gui:
>
> Dominik Csapak (1):
> add 'Request Quarantine Link' Button to LoginView
>
> js/LoginView.js | 33 +++++++++++++++++++++++++++++++++
> pmg-index.html.tt | 3 ++-
> 2 files changed, 35 insertions(+), 1 deletion(-)
>
applied, with some followups:
* use built-in time() to get seconds since epoch
* sleep a bit, especially more on the rate limit cases
* code consitency
one thing which feels a bit "flawed" is the fact that if one knows or guesses
correctly a valid domain, the can DOS this requester for all other valid ones by
simply looping and sending a request for "$name++@$domain" where $name gets changed
slightly each round - this way others race with them to get in between the small
timeframe where the mtime is old enough again and the "attacker" gets through a
request again.
But, this can be somewhat solved by fail2ban and it's opt-in anyway.
^ permalink raw reply [flat|nested] 6+ messages in thread