[pmg-devel] [PATCH pmg-api/proxmox-acme/various] acme: add support for http_proxy

[pmg-devel] [PATCH pmg-api/proxmox-acme/various] acme: add support for http_proxy

[Stoiko Ivanov]

this series resulted from a fix for #3536 (for PVE), which I then extended
to cover all products (and their respective acme implementations)

sending as 3 separate series (with common patches in each series) to
hopefully make reviewing this easier.

the individual patches are mostly short and hopefully self-explaining

Tested on my setup with a publicly exposed powerdns-plugin and let's encrypt
(mostly staging)

proxmox-acme:
Stoiko Ivanov (2):
  add support for proxies
  dns-challenge: add 'use-proxy' property

 src/PVE/ACME.pm              |  6 ++++++
 src/PVE/ACME/DNSChallenge.pm | 10 +++++++++-
 2 files changed, 15 insertions(+), 1 deletion(-)

proxmox-acme-rs:
Stoiko Ivanov (1):
  client: add support for proxies

 src/client.rs | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

pmg-rs:
Stoiko Ivanov (2):
  add support for set_proxy
  update to perlmod 0.8

 Cargo.toml  |  2 +-
 src/acme.rs | 11 +++++++++++
 2 files changed, 12 insertions(+), 1 deletion(-)

proxmox-widget-toolkit
Stoiko Ivanov (1):
  acmeplugin: add use-proxy checkbox

 src/window/ACMEPluginEdit.js | 8 ++++++++
 1 file changed, 8 insertions(+)

pmg-api:
Stoiko Ivanov (2):
  api: acme: set http_proxy if configured in pmg.conf
  api: acme: dns-plugin: conditionally pass proxy to acme.sh wrapper

 src/PMG/API2/ACME.pm         | 21 +++++++++++++++++++++
 src/PMG/API2/Certificates.pm | 20 ++++++++++++++++++++
 2 files changed, 41 insertions(+)

-- 
2.30.2

[pmg-devel] [PATCH proxmox-acme 1/2] add support for proxies

[Stoiko Ivanov]

by setting the proxy for the LWP::UserAgent

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/PVE/ACME.pm | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/src/PVE/ACME.pm b/src/PVE/ACME.pm
index 57578d7..3f66182 100644
--- a/src/PVE/ACME.pm
+++ b/src/PVE/ACME.pm
@@ -113,6 +113,12 @@ sub new($$$) {
     return bless $self, $class;
 }
 
+sub set_proxy($$) {
+    my ($self, $proxy) = @_;
+
+    $self->{ua}->proxy('https', $proxy);
+}
+
 # RS256: PKCS#1 padding, no OAEP, SHA256
 my $configure_key = sub {
     my ($key) = @_;
-- 
2.30.2

[pmg-devel] [PATCH proxmox-acme 2/2] dns-challenge: add 'use-proxy' property

[Stoiko Ivanov]

this patch adds an optional 'use-proxy' property to the dns
challenges.

If set to 1 the caller is expected to add the proxy url in the plugin
config, which is then set as 'http_proxy' and 'https_proxy'
environment variable by the plugin caller (and then used by curl)

Tested with the pdns plugin, direct traffic to the pdns server being
dropped, and a configured squid proxy

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/PVE/ACME/DNSChallenge.pm | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/src/PVE/ACME/DNSChallenge.pm b/src/PVE/ACME/DNSChallenge.pm
index 9b28970..c03b3db 100644
--- a/src/PVE/ACME/DNSChallenge.pm
+++ b/src/PVE/ACME/DNSChallenge.pm
@@ -50,7 +50,12 @@ sub properties {
 	    optional => 1,
 	    minimum => 0,
 	    maximum => 2 * 24 * 60 * 60,
-	}
+	},
+	'use-proxy' => {
+	    description => "Flag indicating whether a http proxy should be used.",
+	    type => 'boolean',
+	    optional => 1,
+	},
     };
 }
 
@@ -61,6 +66,7 @@ sub options {
 	nodes => { optional => 1 },
 	disable => { optional => 1 },
 	'validation-delay' => { optional => 1 },
+	'use-proxy' => { optional => 1 },
     };
 }
 
@@ -78,6 +84,7 @@ my $proxmox_acme_command = sub {
     my $txtvalue = PVE::ACME::encode(sha256($key_auth));
     my $dnsplugin = $data->{plugin}->{api};
     my $plugin_conf_string = $data->{plugin}->{data};
+    my $proxy = $data->{plugin}->{proxy};
 
     # for security reasons, we execute the command as nobody
     # we can't verify that the code of the DNSPlugins are harmless.
@@ -93,6 +100,7 @@ my $proxmox_acme_command = sub {
     }
     my $input = "$txtvalue\n";
     $input .= "$plugin_conf_string\n" if $plugin_conf_string;
+    $input .= "https_proxy=$proxy\nhttp_proxy=$proxy\n" if $proxy;
 
     PVE::Tools::run_command($cmd, input => $input);
 
-- 
2.30.2

[pmg-devel] [PATCH proxmox-acme-rs 1/1] client: add support for proxies

[Stoiko Ivanov]

by storing the proxy url as string in the struct and setting it on
each invocation of `execute`, since execute calls reset on the
curl::easy::Easy object.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/client.rs | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/src/client.rs b/src/client.rs
index 8aeff97..9853fb6 100644
--- a/src/client.rs
+++ b/src/client.rs
@@ -91,6 +91,7 @@ impl Headers {
 struct Inner {
     easy: easy::Easy,
     nonce: Option<String>,
+    proxy: Option<String>,
 }
 
 impl Inner {
@@ -98,6 +99,7 @@ impl Inner {
         Self {
             easy: easy::Easy::new(),
             nonce: None,
+            proxy: None,
         }
     }
 
@@ -120,6 +122,10 @@ impl Inner {
 
         self.easy.url(url)?;
 
+        if let Some(p) = &self.proxy {
+            self.easy.proxy(&p)?;
+        }
+
         {
             let mut transfer = self.easy.transfer();
 
@@ -156,6 +162,10 @@ impl Inner {
         })
     }
 
+    pub fn set_proxy(&mut self, proxy: String) {
+            self.proxy = Some(proxy);
+    }
+
     /// Low-level API to run an n API request. This automatically updates the current nonce!
     fn run_request(&mut self, request: Request) -> Result<HttpResponse, Error> {
         self.easy.reset();
@@ -586,6 +596,11 @@ impl Client {
             }
         }
     }
+
+    /// Set a proxy
+    pub fn set_proxy(&mut self, proxy: String) {
+        self.inner.set_proxy(proxy)
+    }
 }
 
 fn parse_header(data: &[u8]) -> Option<(&str, &str)> {
-- 
2.30.2

[pmg-devel] [PATCH pmg-rs 1/2] add support for set_proxy

[Stoiko Ivanov]

this commit needs a versioned dependency bump on proxmox-acme-rs

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/acme.rs | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/src/acme.rs b/src/acme.rs
index a1b5278..0429a0d 100644
--- a/src/acme.rs
+++ b/src/acme.rs
@@ -173,6 +173,10 @@ impl Inner {
     pub fn revoke_certificate(&mut self, data: &[u8], reason: Option<u32>) -> Result<(), Error> {
         Ok(self.client.revoke_certificate(data, reason)?)
     }
+
+    pub fn set_proxy(&mut self, proxy: String) {
+        self.client.set_proxy(proxy)
+    }
 }
 
 #[perlmod::package(name = "PMG::RS::Acme", lib = "pmg_rs")]
@@ -416,4 +420,11 @@ pub mod export {
             .revoke_certificate(&data, reason)?;
         Ok(())
     }
+
+    /// Set a proxy
+    #[export]
+    pub fn set_proxy(#[try_from_ref] this: &Acme, proxy: String) {
+        this.inner.lock().unwrap().set_proxy(proxy)
+    }
+
 }
-- 
2.30.2

[pmg-devel] [PATCH pmg-rs 2/2] update to perlmod 0.8

[Stoiko Ivanov]

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
not directly related to the series

 Carg.toml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Cargo.toml b/Cargo.toml
index 217f0e2..dd1fb18 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -26,7 +26,7 @@ serde = "1.0"
 serde_bytes = "0.11.3"
 serde_json = "1.0"
 
-perlmod = { version = "0.7", features = [ "exporter" ] }
+perlmod = { version = "0.8", features = [ "exporter" ] }
 #perlmod = { path = "../perlmod/perlmod", features = [ "exporter" ] }
 
 proxmox-acme-rs = { version = "0.3", features = ["client"] }
-- 
2.30.2

[pmg-devel] [PATCH widget-toolkit 1/1] acmeplugin: add use-proxy checkbox

[Stoiko Ivanov]

if set the plugin will use the http_proxy configured on the
node/datacenter.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/window/ACMEPluginEdit.js | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/src/window/ACMEPluginEdit.js b/src/window/ACMEPluginEdit.js
index 237b362..406f14e 100644
--- a/src/window/ACMEPluginEdit.js
+++ b/src/window/ACMEPluginEdit.js
@@ -216,6 +216,14 @@ Ext.define('Proxmox.window.ACMEPluginEdit', {
 		    name: 'hint',
 		    hidden: true,
 		},
+		{
+		    xtype: 'proxmoxcheckbox',
+		    fieldLabel: gettext('Use http proxy'),
+		    defaultValue: false,
+		    deleteDefaultValue: true,
+		    labelWidth: 150,
+		    name: 'use-proxy',
+		},
 	    ],
 	},
     ],
-- 
2.30.2

[pmg-devel] [PATCH pmg-api 1/2] api: acme: set http_proxy if configured in pmg.conf

[Stoiko Ivanov]

If a http_proxy is set in pmg.conf, use it for communicating with the
(usually public) Acme provider.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/PMG/API2/ACME.pm         | 21 +++++++++++++++++++++
 src/PMG/API2/Certificates.pm | 13 +++++++++++++
 2 files changed, 34 insertions(+)

diff --git a/src/PMG/API2/ACME.pm b/src/PMG/API2/ACME.pm
index 42c9f4e..bb12177 100644
--- a/src/PMG/API2/ACME.pm
+++ b/src/PMG/API2/ACME.pm
@@ -14,6 +14,7 @@ use PVE::ACME::Challenge;
 use PMG::RESTEnvironment;
 use PMG::RS::Acme;
 use PMG::CertHelpers;
+use PMG::Config;
 
 use PMG::API2::ACMEPlugin;
 
@@ -159,6 +160,11 @@ __PACKAGE__->register_method ({
 
 		print "Registering new ACME account..\n";
 		my $acme = PMG::RS::Acme->new($directory);
+		my $pmg_cfg = PMG::Config->new();
+		if (my $http_proxy = $pmg_cfg->get('admin', 'http_proxy')) {
+		    $acme->set_proxy($http_proxy);
+		}
+
 		eval {
 		    $acme->new_account($account_file, defined($param->{tos_url}), $contact, undef);
 		};
@@ -192,6 +198,11 @@ my $update_account = sub {
 		if ! -e $account_file;
 
 	    my $acme = PMG::RS::Acme->load($account_file);
+	    my $pmg_cfg = PMG::Config->new();
+	    if (my $http_proxy = $pmg_cfg->get('admin', 'http_proxy')) {
+		$acme->set_proxy($http_proxy);
+	    }
+
 	    eval {
 		$acme->update_account(\%info);
 	    };
@@ -297,6 +308,11 @@ __PACKAGE__->register_method ({
 	    if ! -e $account_file;
 
 	my $acme = PMG::RS::Acme->load($account_file);
+	my $pmg_cfg = PMG::Config->new();
+	if (my $http_proxy = $pmg_cfg->get('admin', 'http_proxy')) {
+	    $acme->set_proxy($http_proxy);
+	}
+
 	my $data = $acme->account();
 
 	return {
@@ -365,6 +381,11 @@ __PACKAGE__->register_method ({
 	my $directory = extract_param($param, 'directory') // $acme_default_directory_url;
 
 	my $acme = PMG::RS::Acme->new($directory);
+	my $pmg_cfg = PMG::Config->new();
+	if (my $http_proxy = $pmg_cfg->get('admin', 'http_proxy')) {
+	    $acme->set_proxy($http_proxy);
+	}
+
 	my $meta = $acme->get_meta();
 
 	return $meta ? $meta->{termsOfService} : undef;
diff --git a/src/PMG/API2/Certificates.pm b/src/PMG/API2/Certificates.pm
index b52f010..6627328 100644
--- a/src/PMG/API2/Certificates.pm
+++ b/src/PMG/API2/Certificates.pm
@@ -9,6 +9,7 @@ use PVE::JSONSchema qw(get_standard_option);
 use PVE::Tools qw(extract_param file_get_contents file_set_contents);
 
 use PMG::CertHelpers;
+use PMG::Config;
 use PMG::NodeConfig;
 use PMG::RS::Acme;
 use PMG::RS::CSR;
@@ -535,6 +536,10 @@ __PACKAGE__->register_method ({
 
 	    print "Loading ACME account details\n";
 	    my $acme = PMG::RS::Acme->load($account_file);
+	    my $pmg_cfg = PMG::Config->new();
+	    if (my $http_proxy = $pmg_cfg->get('admin', 'http_proxy')) {
+		$acme->set_proxy($http_proxy);
+	    }
 
 	    my ($cert, $key) = $order_certificate->($acme, $acme_config);
 	    my $certificate = "$key\n$cert";
@@ -609,6 +614,10 @@ __PACKAGE__->register_method ({
 
 	    print "Loading ACME account details\n";
 	    my $acme = PMG::RS::Acme->load($account_file);
+	    my $pmg_cfg = PMG::Config->new();
+	    if (my $http_proxy = $pmg_cfg->get('admin', 'http_proxy')) {
+		$acme->set_proxy($http_proxy);
+	    }
 
 	    my ($cert, $key) = $order_certificate->($acme, $acme_config);
 	    my $certificate = "$key\n$cert";
@@ -676,6 +685,10 @@ __PACKAGE__->register_method ({
 
 	    print "Loading ACME account details\n";
 	    my $acme = PMG::RS::Acme->load($account_file);
+	    my $pmg_cfg = PMG::Config->new();
+	    if (my $http_proxy = $pmg_cfg->get('admin', 'http_proxy')) {
+		$acme->set_proxy($http_proxy);
+	    }
 
 	    print "Revoking old certificate\n";
 	    eval { $acme->revoke_certificate($cert, undef) };
-- 
2.30.2

[pmg-devel] [PATCH pmg-api 2/2] api: acme: dns-plugin: conditionally pass proxy to acme.sh wrapper

[Stoiko Ivanov]

If an acme (dns) plugin has set the 'use-proxy' flag, pass
the http_proxy configured in pmg.conf to be used by the dns-plugin

The setting is configurable for each dns-plugin, based on the
assumption that some dns-apis might be 'local' (e.g. a pdns-server in
the internal network) and communication with them should not pass
through the proxy.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 src/PMG/API2/Certificates.pm | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/src/PMG/API2/Certificates.pm b/src/PMG/API2/Certificates.pm
index 6627328..668134f 100644
--- a/src/PMG/API2/Certificates.pm
+++ b/src/PMG/API2/Certificates.pm
@@ -375,6 +375,13 @@ my $order_certificate = sub {
 	    die "plugin '$plugin_id' for domain '$domain' not found!\n"
 		if !$plugin_cfg;
 
+	    if ($plugin_cfg->{'use-proxy'}) {
+		my $pmg_cfg = PMG::Config->new();
+		if (my $http_proxy = $pmg_cfg->get('admin', 'http_proxy')) {
+		    $plugin_cfg->{proxy} = $http_proxy;
+		}
+	    }
+
 	    my $data = {
 		plugin => $plugin_cfg,
 		alias => $domain_config->{alias},
-- 
2.30.2

[pmg-devel] applied: [PATCH pmg-rs 1/2] add support for set_proxy

[Wolfgang Bumiller]

On Tue, Nov 09, 2021 at 05:20:05PM +0100, Stoiko Ivanov wrote:
> this commit needs a versioned dependency bump on proxmox-acme-rs
> 
> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> ---
>  src/acme.rs | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/src/acme.rs b/src/acme.rs
> index a1b5278..0429a0d 100644
> --- a/src/acme.rs
> +++ b/src/acme.rs
> @@ -173,6 +173,10 @@ impl Inner {
>      pub fn revoke_certificate(&mut self, data: &[u8], reason: Option<u32>) -> Result<(), Error> {
>          Ok(self.client.revoke_certificate(data, reason)?)
>      }
> +
> +    pub fn set_proxy(&mut self, proxy: String) {
> +        self.client.set_proxy(proxy)
> +    }
>  }
>  
>  #[perlmod::package(name = "PMG::RS::Acme", lib = "pmg_rs")]
> @@ -416,4 +420,11 @@ pub mod export {
>              .revoke_certificate(&data, reason)?;
>          Ok(())
>      }
> +
> +    /// Set a proxy
> +    #[export]
> +    pub fn set_proxy(#[try_from_ref] this: &Acme, proxy: String) {
> +        this.inner.lock().unwrap().set_proxy(proxy)
> +    }
> +
>  }
> -- 
> 2.30.2

[pmg-devel] applied: [PATCH pmg-rs 2/2] update to perlmod 0.8

[Wolfgang Bumiller]

applied, thanks

On Tue, Nov 09, 2021 at 05:20:06PM +0100, Stoiko Ivanov wrote:
> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> ---
> not directly related to the series
> 
>  Carg.toml | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/Cargo.toml b/Cargo.toml
> index 217f0e2..dd1fb18 100644
> --- a/Cargo.toml
> +++ b/Cargo.toml
> @@ -26,7 +26,7 @@ serde = "1.0"
>  serde_bytes = "0.11.3"
>  serde_json = "1.0"
>  
> -perlmod = { version = "0.7", features = [ "exporter" ] }
> +perlmod = { version = "0.8", features = [ "exporter" ] }
>  #perlmod = { path = "../perlmod/perlmod", features = [ "exporter" ] }
>  
>  proxmox-acme-rs = { version = "0.3", features = ["client"] }
> -- 
> 2.30.2