[pmg-devel] [RFC api/gui/wtk/acme 0/many] Certificates & ACME

[pmg-devel] [RFC api/gui/wtk/acme 0/many] Certificates & ACME

[Wolfgang Bumiller]

These are the pmg-api, pmg-gui and proxmox-widget-toolkit and
proxmox-acme parts of the ACME series for PMG.

This requires `pmg-rs` package, which replaces the ACME client from
`proxmox-acme` and provides the CSR generation and is written in rust.
Note that the DNS challenge handling still uses proxmox-acme for now.

proxmox-acme:
  * Just a `use` statement fixup
  * Still used for the DNS challenge

pmg-gui:
  Just adds the "certificate view", but the real dirt lives in the
  widget-toolkit.

proxmox-widget-toolkits:
  Gets the Certificate, ACME Account, ACME Plugin and ACME Domain view
  from PVE adapted to be usable for PMG.
  Changes to PVE are mainly:
    * API URLs need to be provided since they differ a bit between PVE
      and PMG.
    * some additional buttons/fields specific to pmg generated if the
      parameters for them are present

pmg-api:
  Simply gets API entry points for the above. These too are mostly
  copied from PVE and adapted (also the ACME client API from pmg-rs is slightly
  different/cleaned up, so that's a minor incompatiblity in some
  otherwise common code, but a `pve-rs` may fix that). But some things
  could definitely already go to pve-common (especially schema stuff).

Note that while I did add the corresponding files to the cluster sync,
this still needs testing *and* issuing an API certificate may break
cluster functionality currently. (Stoiko is working on that)

[pmg-devel] [PATCH api 1/8] depend on libpmg-rs-perl and proxmox-acme

[Wolfgang Bumiller]

This contains `PMG::RS::Acme` and `PMG::RS::CSR` which are
used for letsencrypt certificates.

Note that for the DNS plugins this still uses the perl code
from proxmox-acme for now.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 debian/control | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/debian/control b/debian/control
index 2605e82..bab2596 100644
--- a/debian/control
+++ b/debian/control
@@ -16,9 +16,11 @@ Build-Depends: debhelper (>= 10~),
                libmime-tools-perl,
                libnet-ldap-perl,
                libnet-server-perl,
+               libpmg-rs-perl (>= 0.1-1),
                libpve-apiclient-perl,
                libpve-common-perl (>= 6.0-13),
                libpve-http-server-perl (>= 2.0-12),
+               libproxmox-acme-perl,
                librrds-perl,
                libtemplate-perl,
                libxdgmime-perl,
@@ -60,9 +62,11 @@ Depends: apt,
          libnet-ip-perl,
          libnet-ldap-perl,
          libnet-server-perl,
+         libpmg-rs-perl (>= 0.1-1),
          libpve-apiclient-perl,
          libpve-common-perl (>= 6.2-6),
          libpve-http-server-perl (>= 3.0-4),
+         libproxmox-acme-perl,
          librrds-perl,
          libtemplate-perl,
          libterm-readline-gnu-perl,
-- 
2.20.1

[pmg-devel] [PATCH api 2/8] add PMG::CertHelpers module

[Wolfgang Bumiller]

Contains helpers to update certificates and provide locking
for certificates and when accessing acme accounts.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/Makefile           |   1 +
 src/PMG/CertHelpers.pm | 180 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 181 insertions(+)
 create mode 100644 src/PMG/CertHelpers.pm

diff --git a/src/Makefile b/src/Makefile
index 8891a3c..c1d4812 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -55,6 +55,7 @@ LIBSOURCES =				\
 	PMG/HTMLMail.pm			\
 	PMG/ModGroup.pm			\
 	PMG/SMTPPrinter.pm		\
+	PMG/CertHelpers.pm		\
 	PMG/Config.pm			\
 	PMG/Cluster.pm			\
 	PMG/ClusterConfig.pm		\
diff --git a/src/PMG/CertHelpers.pm b/src/PMG/CertHelpers.pm
new file mode 100644
index 0000000..2cf8a4e
--- /dev/null
+++ b/src/PMG/CertHelpers.pm
@@ -0,0 +1,180 @@
+package PMG::CertHelpers;
+
+use strict;
+use warnings;
+
+use PVE::Certificate;
+use PVE::JSONSchema;
+use PVE::Tools;
+
+use constant {
+    API_CERT => '/etc/pmg/pmg-api.pem',
+    SMTP_CERT => '/etc/pmg/pmg-tls.pem',
+};
+
+my $account_prefix = '/etc/pmg/acme';
+
+# TODO: Move `pve-acme-account-name` to common and reuse instead of this.
+PVE::JSONSchema::register_standard_option('pmg-acme-account-name', {
+    description => 'ACME account config file name.',
+    type => 'string',
+    format => 'pve-configid',
+    format_description => 'name',
+    optional => 1,
+    default => 'default',
+});
+
+PVE::JSONSchema::register_standard_option('pmg-acme-account-contact', {
+    type => 'string',
+    format => 'email-list',
+    description => 'Contact email addresses.',
+});
+
+PVE::JSONSchema::register_standard_option('pmg-acme-directory-url', {
+    type => 'string',
+    description => 'URL of ACME CA directory endpoint.',
+    pattern => '^https?://.*',
+});
+
+PVE::JSONSchema::register_format('pmg-certificate-type', sub {
+    my ($type, $noerr) = @_;
+
+    if ($type =~ /^(?: api | smtp )$/x) {
+	return $type;
+    }
+    return undef if $noerr;
+    die "value '$type' does not look like a valid certificate type\n";
+});
+
+PVE::JSONSchema::register_standard_option('pmg-certificate-type', {
+    type => 'string',
+    description => 'The TLS certificate type (API or SMTP certificate).',
+    enum => ['api', 'smtp'],
+});
+
+PVE::JSONSchema::register_format('pmg-acme-domain', sub {
+    my ($domain, $noerr) = @_;
+
+    my $label = qr/[a-z0-9][a-z0-9_-]*/i;
+
+    return $domain if $domain =~ /^$label(?:\.$label)+$/;
+    return undef if $noerr;
+    die "value '$domain' does not look like a valid domain name!\n";
+});
+
+PVE::JSONSchema::register_format('pmg-acme-alias', sub {
+    my ($alias, $noerr) = @_;
+
+    my $label = qr/[a-z0-9_][a-z0-9_-]*/i;
+
+    return $alias if $alias =~ /^$label(?:\.$label)+$/;
+    return undef if $noerr;
+    die "value '$alias' does not look like a valid alias name!\n";
+});
+
+my $local_cert_lock = '/var/lock/pmg-certs.lock';
+my $local_acme_lock = '/var/lock/pmg-acme.lock';
+
+sub cert_path : prototype($) {
+    my ($type) = @_;
+    if ($type eq 'api') {
+	return API_CERT;
+    } elsif ($type eq 'smtp') {
+	return SMTP_CERT;
+    } else {
+	die "unknown certificate type '$type'\n";
+    }
+}
+
+sub cert_lock {
+    my ($timeout, $code, @param) = @_;
+
+    my $res = PVE::Tools::lock_file($local_cert_lock, $timeout, $code, @param);
+    die $@ if $@;
+    return $res;
+}
+
+sub set_cert_file {
+    my ($cert, $cert_path, $force) = @_;
+
+    my ($old_cert, $info);
+
+    my $cert_path_old = "${cert_path}.old";
+
+    die "Custom certificate file exists but force flag is not set.\n"
+	if !$force && -e $cert_path;
+
+    PVE::Tools::file_copy($cert_path, $cert_path_old) if -e $cert_path;
+
+    eval {
+	my $gid = undef;
+	if ($cert_path eq &API_CERT) {
+	    $gid = getgrnam('www-data') ||
+		die "user www-data not in group file\n";
+	}
+
+	if (defined($gid)) {
+	    my $cert_path_tmp = "${cert_path}.tmp";
+	    PVE::Tools::file_set_contents($cert_path_tmp, $cert, 0640);
+	    if (!chown(-1, $gid, $cert_path_tmp)) {
+		my $msg =
+		    "failed to change group ownership of '$cert_path_tmp' to www-data ($gid): $!\n";
+		unlink($cert_path_tmp);
+		die $msg;
+	    }
+	    if (!rename($cert_path_tmp, $cert_path)) {
+		my $msg =
+		    "failed to rename '$cert_path_tmp' to '$cert_path': $!\n";
+		unlink($cert_path_tmp);
+		die $msg;
+	    }
+	} else {
+	    PVE::Tools::file_set_contents($cert_path, $cert, 0600);
+	}
+
+	$info = PVE::Certificate::get_certificate_info($cert_path);
+    };
+    my $err = $@;
+
+    if ($err) {
+	if (-e $cert_path_old) {
+	    eval {
+		warn "Attempting to restore old certificate file..\n";
+		PVE::Tools::file_copy($cert_path_old, $cert_path);
+	    };
+	    warn "$@\n" if $@;
+	}
+	die "Setting certificate files failed - $err\n"
+    }
+
+    unlink $cert_path_old;
+
+    return $info;
+}
+
+sub lock_acme {
+    my ($account_name, $timeout, $code, @param) = @_;
+
+    my $file = "$local_acme_lock.$account_name";
+
+    return PVE::Tools::lock_file($file, $timeout, $code, @param);
+}
+
+sub acme_account_dir {
+    return $account_prefix;
+}
+
+sub list_acme_accounts {
+    my $accounts = [];
+
+    return $accounts if ! -d $account_prefix;
+
+    PVE::Tools::dir_glob_foreach($account_prefix, qr/[^.]+.*/, sub {
+	my ($name) = @_;
+
+	push @$accounts, $name
+	    if PVE::JSONSchema::pve_verify_configid($name, 1);
+    });
+
+    return $accounts;
+}
-- 
2.20.1

[pmg-devel] [PATCH api 3/8] add PMG::NodeConfig module

[Wolfgang Bumiller]

for node-local configuration, currently only containing acme
domains/account choices

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/Makefile          |   1 +
 src/PMG/NodeConfig.pm | 225 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 226 insertions(+)
 create mode 100644 src/PMG/NodeConfig.pm

diff --git a/src/Makefile b/src/Makefile
index c1d4812..ce76f9f 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -117,6 +117,7 @@ LIBSOURCES =				\
 	PMG/RuleDB.pm			\
 	${CLI_CLASSES} 			\
 	${SERVICE_CLASSES}		\
+	PMG/NodeConfig.pm		\
 	PMG/API2/Subscription.pm	\
 	PMG/API2/APT.pm			\
 	PMG/API2/Network.pm		\
diff --git a/src/PMG/NodeConfig.pm b/src/PMG/NodeConfig.pm
new file mode 100644
index 0000000..84c2141
--- /dev/null
+++ b/src/PMG/NodeConfig.pm
@@ -0,0 +1,225 @@
+package PMG::NodeConfig;
+
+use strict;
+use warnings;
+
+use Digest::SHA;
+
+use PVE::INotify;
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::Tools;
+
+use PMG::API2::ACMEPlugin;
+use PMG::CertHelpers;
+
+# register up to 5 domain names per node for now
+my $MAXDOMAINS = 5;
+
+my $inotify_file_id = 'pmg-node-config.conf';
+my $config_filename = '/etc/pmg/node.conf';
+my $lockfile = "/var/lock/pmg-node-config.lck";
+
+my $acme_domain_desc = {
+    domain => {
+	type => 'string',
+	format => 'pmg-acme-domain',
+	format_description => 'domain',
+	description => 'domain for this node\'s ACME certificate',
+	default_key => 1,
+    },
+    plugin => {
+	type => 'string',
+	format => 'pve-configid',
+	description => 'The ACME plugin ID',
+	format_description => 'name of the plugin configuration',
+	optional => 1,
+	default => 'standalone',
+    },
+    alias => {
+	type => 'string',
+	format => 'pmg-acme-alias',
+	format_description => 'domain',
+	description => 'Alias for the Domain to verify ACME Challenge over DNS',
+	optional => 1,
+    },
+    usage => {
+	type => 'string',
+	format => 'pmg-certificate-type-list',
+	description => 'Whether this domain is used for the API, SMTP or both',
+    },
+};
+
+my $acmedesc = {
+    account => get_standard_option('pmg-acme-account-name'),
+};
+
+my $confdesc = {
+    acme => {
+	type => 'string',
+	description => 'Node specific ACME settings.',
+	format => $acmedesc,
+	optional => 1,
+    },
+    map {(
+	"acmedomain$_" => {
+	    type => 'string',
+	    description => 'ACME domain and validation plugin',
+	    format => $acme_domain_desc,
+	    optional => 1,
+	},
+    )} (0..$MAXDOMAINS),
+};
+
+sub acme_config_schema : prototype(;$) {
+    my ($overrides) = @_;
+
+    $overrides //= {};
+
+    return {
+	type => 'object',
+	additionalProperties => 0,
+	properties => {
+	    %$confdesc,
+	    %$overrides,
+	},
+    }
+}
+
+my $config_schema = acme_config_schema();
+
+# Parse the config's acme property string if it exists.
+#
+# Returns nothing if the entry is not set.
+sub parse_acme : prototype($) {
+    my ($cfg) = @_;
+    my $data = $cfg->{acme};
+    if (defined($data)) {
+	return PVE::JSONSchema::parse_property_string($acmedesc, $data);
+    }
+    return; # empty list otherwise
+}
+
+# Turn the acme object into a property string.
+sub print_acme : prototype($) {
+    my ($acme) = @_;
+    return PVE::JSONSchema::print_property_string($acmedesc, $acme);
+}
+
+# Parse a domain entry from the config.
+sub parse_domain : prototype($) {
+    my ($data) = @_;
+    return PVE::JSONSchema::parse_property_string($acme_domain_desc, $data);
+}
+
+# Turn a domain object into a property string.
+sub print_domain : prototype($) {
+    my ($domain) = @_;
+    return PVE::JSONSchema::print_property_string($acme_domain_desc, $domain);
+}
+
+sub read_pmg_node_config {
+    my ($filename, $fh) = @_;
+    local $/ = undef; # slurp mode
+    my $raw = defined($fh) ? <$fh> : '';
+    my $digest = Digest::SHA::sha1_hex($raw);
+    my $conf = PVE::JSONSchema::parse_config($config_schema, $filename, $raw);
+    $conf->{digest} = $digest;
+    return $conf;
+}
+
+sub write_pmg_node_config {
+    my ($filename, $fh, $cfg) = @_;
+    my $raw = PVE::JSONSchema::dump_config($config_schema, $filename, $cfg);
+    PVE::Tools::safe_print($filename, $fh, $raw);
+}
+
+PVE::INotify::register_file($inotify_file_id, $config_filename,
+			    \&read_pmg_node_config,
+			    \&write_pmg_node_config,
+			    undef,
+			    always_call_parser => 1);
+
+sub lock_config {
+    my ($code) = @_;
+    my $p = PVE::Tools::lock_file($lockfile, undef, $code);
+    die $@ if $@;
+    return $p;
+}
+
+sub load_config {
+    # auto-adds the standalone plugin if no config is there for backwards
+    # compatibility, so ALWAYS call the cfs registered parser
+    return PVE::INotify::read_file($inotify_file_id);
+}
+
+sub write_config {
+    my ($self) = @_;
+    return PVE::INotify::write_file($inotify_file_id, $self);
+}
+
+# we always convert domain values to lower case, since DNS entries are not case
+# sensitive and ACME implementations might convert the ordered identifiers
+# to lower case
+# FIXME: Could also be shared between PVE and PMG
+sub get_acme_conf {
+    my ($conf, $noerr) = @_;
+
+    $conf //= {};
+
+    my $res = {};
+    if (defined($conf->{acme})) {
+	$res = eval {
+	    PVE::JSONSchema::parse_property_string($acmedesc, $conf->{acme})
+	};
+	if (my $err = $@) {
+	    return undef if $noerr;
+	    die $err;
+	}
+	my $standalone_domains = delete($res->{domains}) // '';
+	$res->{domains} = {};
+	for my $domain (split(";", $standalone_domains)) {
+	    $domain = lc($domain);
+	    die "duplicate domain '$domain' in ACME config properties\n"
+		if defined($res->{domains}->{$domain});
+
+	    $res->{domains}->{$domain}->{plugin} = 'standalone';
+	    $res->{domains}->{$domain}->{_configkey} = 'acme';
+	}
+    }
+
+    $res->{account} //= 'default';
+
+    for my $index (0..$MAXDOMAINS) {
+	my $domain_rec = $conf->{"acmedomain$index"};
+	next if !defined($domain_rec);
+
+	my $parsed = eval {
+	    PVE::JSONSchema::parse_property_string($acme_domain_desc, $domain_rec)
+	};
+	if (my $err = $@) {
+	    return undef if $noerr;
+	    die $err;
+	}
+	my $domain = lc(delete $parsed->{domain});
+	if (my $exists = $res->{domains}->{$domain}) {
+	    return undef if $noerr;
+	    die "duplicate domain '$domain' in ACME config properties"
+	        ." 'acmedomain$index' and '$exists->{_configkey}'\n";
+	}
+	$parsed->{plugin} //= 'standalone';
+
+	my $plugin_id = $parsed->{plugin};
+	if ($plugin_id ne 'standalone') {
+	    my $plugins = PMG::API2::ACMEPlugin::load_config();
+	    die "plugin '$plugin_id' for domain '$domain' not found!\n"
+		if !$plugins->{ids}->{$plugin_id};
+	}
+
+	$parsed->{_configkey} = "acmedomain$index";
+	$res->{domains}->{$domain} = $parsed;
+    }
+
+    return $res;
+}
+
+1;
-- 
2.20.1

[pmg-devel] [PATCH api 4/8] cluster: sync acme/ and acme-plugins.conf

[Wolfgang Bumiller]

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/PMG/Cluster.pm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/PMG/Cluster.pm b/src/PMG/Cluster.pm
index ce4f257..6bb940a 100644
--- a/src/PMG/Cluster.pm
+++ b/src/PMG/Cluster.pm
@@ -400,6 +400,7 @@ sub sync_config_from_master {
 	'transport',
 	'tls_policy',
 	'fetchmailrc',
+	'acme-plugins.conf',
 	];
 
     foreach my $filename (@$files) {
@@ -410,6 +411,7 @@ sub sync_config_from_master {
 	'templates',
 	'dkim',
 	'pbs',
+	'acme',
     ];
 
     foreach my $dir (@$dirs) {
-- 
2.20.1

[pmg-devel] [PATCH api 5/8] api: add ACME and ACMEPlugin module

[Wolfgang Bumiller]

This adds the cluster-wide acme account and plugin
configuration:

   * /config/acme
   |`+ account/
   | '- {name}
   |`- tos
   |`- directories
   |`- challenge-schema
    `+ plugins/
     '- {name}

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/Makefile               |   2 +
 src/PMG/API2/ACME.pm       | 436 +++++++++++++++++++++++++++++++++++++
 src/PMG/API2/ACMEPlugin.pm | 270 +++++++++++++++++++++++
 src/PMG/API2/Config.pm     |   7 +
 4 files changed, 715 insertions(+)
 create mode 100644 src/PMG/API2/ACME.pm
 create mode 100644 src/PMG/API2/ACMEPlugin.pm

diff --git a/src/Makefile b/src/Makefile
index ce76f9f..ebc6bd8 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -155,6 +155,8 @@ LIBSOURCES =				\
 	PMG/API2/When.pm		\
 	PMG/API2/What.pm		\
 	PMG/API2/Action.pm		\
+	PMG/API2/ACME.pm		\
+	PMG/API2/ACMEPlugin.pm		\
 	PMG/API2.pm			\
 
 SOURCES = ${LIBSOURCES} ${CLI_BINARIES} ${TEMPLATES_FILES} ${CONF_MANS} ${CLI_MANS} ${SERVICE_MANS} ${SERVICE_UNITS} ${TIMER_UNITS} pmg-sources.list pmg-apt.conf pmg-initramfs.conf
diff --git a/src/PMG/API2/ACME.pm b/src/PMG/API2/ACME.pm
new file mode 100644
index 0000000..3b031fb
--- /dev/null
+++ b/src/PMG/API2/ACME.pm
@@ -0,0 +1,436 @@
+package PMG::API2::ACME;
+
+use strict;
+use warnings;
+
+use PVE::Exception qw(raise_param_exc);
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::Tools qw(extract_param);
+
+use PVE::ACME::Challenge;
+
+use PMG::RESTEnvironment;
+use PMG::RS::Acme;
+use PMG::CertHelpers;
+
+use PMG::API2::ACMEPlugin;
+
+use base qw(PVE::RESTHandler);
+
+__PACKAGE__->register_method ({
+    subclass => "PMG::API2::ACMEPlugin",
+    path => 'plugins',
+});
+
+# FIXME: Put this list in pve-common or proxmox-acme{,-rs}?
+my $acme_directories = [
+    {
+	name => 'Let\'s Encrypt V2',
+	url => 'https://acme-v02.api.letsencrypt.org/directory',
+    },
+    {
+	name => 'Let\'s Encrypt V2 Staging',
+	url => 'https://acme-staging-v02.api.letsencrypt.org/directory',
+    },
+];
+my $acme_default_directory_url = $acme_directories->[0]->{url};
+my $account_contact_from_param = sub {
+    my @addresses = PVE::Tools::split_list(extract_param($_[0], 'contact'));
+    return [ map { "mailto:$_" } @addresses ];
+};
+my $acme_account_dir = PMG::CertHelpers::acme_account_dir();
+
+__PACKAGE__->register_method ({
+    name => 'index',
+    path => '',
+    method => 'GET',
+    permissions => { user => 'all' },
+    description => "ACME index.",
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	},
+    },
+    returns => {
+	type => 'array',
+	items => {
+	    type => "object",
+	    properties => {},
+	},
+	links => [ { rel => 'child', href => "{name}" } ],
+    },
+    code => sub {
+	my ($param) = @_;
+
+	return [
+	    { name => 'account' },
+	    { name => 'tos' },
+	    { name => 'directories' },
+	    { name => 'plugins' },
+	    { name => 'challengeschema' },
+	];
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'account_index',
+    path => 'account',
+    method => 'GET',
+    permissions => { user => 'all' },
+    description => "ACME account index.",
+    protected => 1,
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	},
+    },
+    returns => {
+	type => 'array',
+	items => {
+	    type => "object",
+	    properties => {},
+	},
+	links => [ { rel => 'child', href => "{name}" } ],
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $accounts = PMG::CertHelpers::list_acme_accounts();
+	return [ map { { name => $_ }  } @$accounts ];
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'register_account',
+    path => 'account',
+    method => 'POST',
+    description => "Register a new ACME account with CA.",
+    proxyto => 'master',
+    protected => 1,
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    name => get_standard_option('pmg-acme-account-name'),
+	    contact => get_standard_option('pmg-acme-account-contact'),
+	    tos_url => {
+		type => 'string',
+		description => 'URL of CA TermsOfService - setting this indicates agreement.',
+		optional => 1,
+	    },
+	    directory => get_standard_option('pmg-acme-directory-url', {
+		default => $acme_default_directory_url,
+		optional => 1,
+	    }),
+	},
+    },
+    returns => {
+	type => 'string',
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $rpcenv = PMG::RESTEnvironment->get();
+	my $authuser = $rpcenv->get_user();
+
+	my $account_name = extract_param($param, 'name') // 'default';
+	my $account_file = "${acme_account_dir}/${account_name}";
+	mkdir $acme_account_dir if ! -e $acme_account_dir;
+
+	raise_param_exc({'name' => "ACME account config file '${account_name}' already exists."})
+	    if -e $account_file;
+
+	my $directory = extract_param($param, 'directory') // $acme_default_directory_url;
+	my $contact = $account_contact_from_param->($param);
+
+	my $realcmd = sub {
+	    PMG::CertHelpers::lock_acme($account_name, 10, sub {
+		die "ACME account config file '${account_name}' already exists.\n"
+		    if -e $account_file;
+
+		print "Registering new ACME account..\n";
+		my $acme = PMG::RS::Acme->new($directory);
+		eval {
+		    $acme->new_account($account_file, defined($param->{tos_url}), $contact, undef);
+		};
+		if (my $err = $@) {
+		    unlink $account_file;
+		    die "Registration failed: $err\n";
+		}
+		my $location = $acme->location();
+		print "Registration successful, account URL: '$location'\n";
+	    });
+	    die $@ if $@;
+	};
+
+	return $rpcenv->fork_worker('acmeregister', undef, $authuser, $realcmd);
+    }});
+
+my $update_account = sub {
+    my ($param, $msg, %info) = @_;
+
+    my $account_name = extract_param($param, 'name') // 'default';
+    my $account_file = "${acme_account_dir}/${account_name}";
+
+    raise_param_exc({'name' => "ACME account config file '${account_name}' does not exist."})
+	if ! -e $account_file;
+
+
+    my $rpcenv = PMG::RESTEnvironment->get();
+    my $authuser = $rpcenv->get_user();
+
+    my $realcmd = sub {
+	PMG::CertHelpers::lock_acme($account_name, 10, sub {
+	    die "ACME account config file '${account_name}' does not exist.\n"
+		if ! -e $account_file;
+
+	    my $acme = PMG::RS::Acme->load($account_file);
+	    $acme->update_account(\%info);
+	    if ($info{status} && $info{status} eq 'deactivated') {
+		my $deactivated_name;
+		for my $i (0..100) {
+		    my $candidate = "${acme_account_dir}/_deactivated_${account_name}_${i}";
+		    if (! -e $candidate) {
+			$deactivated_name = $candidate;
+			last;
+		    }
+		}
+		if ($deactivated_name) {
+		    print "Renaming account file from '$account_file' to '$deactivated_name'\n";
+		    rename($account_file, $deactivated_name) or
+			warn ".. failed - $!\n";
+		} else {
+		    warn "No free slot to rename deactivated account file '$account_file', leaving in place\n";
+		}
+	    }
+	});
+	die $@ if $@;
+    };
+
+    return $rpcenv->fork_worker("acme${msg}", undef, $authuser, $realcmd);
+};
+
+__PACKAGE__->register_method ({
+    name => 'update_account',
+    path => 'account/{name}',
+    method => 'PUT',
+    description => "Update existing ACME account information with CA. Note: not specifying any new account information triggers a refresh.",
+    proxyto => 'master',
+    protected => 1,
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    name => get_standard_option('pmg-acme-account-name'),
+	    contact => get_standard_option('pmg-acme-account-contact', {
+		optional => 1,
+	    }),
+	},
+    },
+    returns => {
+	type => 'string',
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $contact = $account_contact_from_param->($param);
+	if (scalar @$contact) {
+	    return $update_account->($param, 'update', contact => $contact);
+	} else {
+	    return $update_account->($param, 'refresh');
+	}
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'get_account',
+    path => 'account/{name}',
+    method => 'GET',
+    description => "Return existing ACME account information.",
+    protected => 1,
+    proxyto => 'master',
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    name => get_standard_option('pmg-acme-account-name'),
+	},
+    },
+    returns => {
+	type => 'object',
+	additionalProperties => 0,
+	properties => {
+	    account => {
+		type => 'object',
+		optional => 1,
+		renderer => 'yaml',
+	    },
+	    directory => get_standard_option('pmg-acme-directory-url', {
+		optional => 1,
+	    }),
+	    location => {
+		type => 'string',
+		optional => 1,
+	    },
+	    tos => {
+		type => 'string',
+		optional => 1,
+	    },
+	},
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $account_name = extract_param($param, 'name') // 'default';
+	my $account_file = "${acme_account_dir}/${account_name}";
+
+	raise_param_exc({'name' => "ACME account config file '${account_name}' does not exist."})
+	    if ! -e $account_file;
+
+	my $acme = PMG::RS::Acme->load($account_file);
+	my $data = $acme->account();
+
+	return {
+	    account => $data->{account},
+	    tos => $data->{tos},
+	    location => $data->{location},
+	    directory => $data->{directoryUrl},
+	};
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'deactivate_account',
+    path => 'account/{name}',
+    method => 'DELETE',
+    description => "Deactivate existing ACME account at CA.",
+    protected => 1,
+    proxyto => 'master',
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    name => get_standard_option('pmg-acme-account-name'),
+	},
+    },
+    returns => {
+	type => 'string',
+    },
+    code => sub {
+	my ($param) = @_;
+
+	return $update_account->($param, 'deactivate', status => 'deactivated');
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'get_tos',
+    path => 'tos',
+    method => 'GET',
+    description => "Retrieve ACME TermsOfService URL from CA.",
+    permissions => { user => 'all' },
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    directory => get_standard_option('pmg-acme-directory-url', {
+		default => $acme_default_directory_url,
+		optional => 1,
+	    }),
+	},
+    },
+    returns => {
+	type => 'string',
+	optional => 1,
+	description => 'ACME TermsOfService URL.',
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $directory = extract_param($param, 'directory') // $acme_default_directory_url;
+
+	my $acme = PMG::RS::Acme->new($directory);
+	my $meta = $acme->get_meta();
+
+	return $meta ? $meta->{termsOfService} : undef;
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'get_directories',
+    path => 'directories',
+    method => 'GET',
+    description => "Get named known ACME directory endpoints.",
+    permissions => { user => 'all' },
+    parameters => {
+	additionalProperties => 0,
+	properties => {},
+    },
+    returns => {
+	type => 'array',
+	items => {
+	    type => 'object',
+	    additionalProperties => 0,
+	    properties => {
+		name => {
+		    type => 'string',
+		},
+		url => get_standard_option('pmg-acme-directory-url'),
+	    },
+	},
+    },
+    code => sub {
+	my ($param) = @_;
+
+	return $acme_directories;
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'challengeschema',
+    path => 'challenge-schema',
+    method => 'GET',
+    description => "Get schema of ACME challenge types.",
+    permissions => { user => 'all' },
+    parameters => {
+	additionalProperties => 0,
+	properties => {},
+    },
+    returns => {
+	type => 'array',
+	items => {
+	    type => 'object',
+	    additionalProperties => 0,
+	    properties => {
+		id => {
+		    type => 'string',
+		},
+		name => {
+		    description => 'Human readable name, falls back to id',
+		    type => 'string',
+		},
+		type => {
+		    type => 'string',
+		},
+		schema => {
+		    type => 'object',
+		},
+	    },
+	},
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $plugin_type_enum = PVE::ACME::Challenge->lookup_types();
+
+	my $res = [];
+
+	for my $type (@$plugin_type_enum) {
+	    my $plugin = PVE::ACME::Challenge->lookup($type);
+	    next if !$plugin->can('get_supported_plugins');
+
+	    my $plugin_type = $plugin->type();
+	    my $plugins = $plugin->get_supported_plugins();
+	    for my $id (sort keys %$plugins) {
+		my $schema = $plugins->{$id};
+		push @$res, {
+		    id => $id,
+		    name => $schema->{name} // $id,
+		    type => $plugin_type,
+		    schema => $schema,
+		};
+	    }
+	}
+
+	return $res;
+    }});
+
+1;
diff --git a/src/PMG/API2/ACMEPlugin.pm b/src/PMG/API2/ACMEPlugin.pm
new file mode 100644
index 0000000..38540b1
--- /dev/null
+++ b/src/PMG/API2/ACMEPlugin.pm
@@ -0,0 +1,270 @@
+package PMG::API2::ACMEPlugin;
+
+use strict;
+use warnings;
+
+use Storable qw(dclone);
+
+use PVE::ACME::Challenge;
+use PVE::ACME::DNSChallenge;
+use PVE::ACME::StandAlone;
+use PVE::INotify;
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::Tools qw(extract_param);
+
+use base qw(PVE::RESTHandler);
+
+my $inotify_file_id = 'pmg-acme-plugins-config.conf';
+my $config_filename = '/etc/pmg/acme-plugins.conf';
+my $lockfile = "/var/lock/pmg-acme-plugins-config.lck";
+
+PVE::ACME::DNSChallenge->register();
+PVE::ACME::StandAlone->register();
+PVE::ACME::Challenge->init();
+
+PVE::JSONSchema::register_standard_option('pmg-acme-pluginid', {
+    type => 'string',
+    format => 'pve-configid',
+    description => 'Unique identifier for ACME plugin instance.',
+});
+
+sub read_pmg_acme_challenge_config {
+    my ($filename, $fh) = @_;
+    local $/ = undef; # slurp mode
+    my $raw = defined($fh) ? <$fh> : '';
+    return PVE::ACME::Challenge->parse_config($filename, $raw);
+}
+
+sub write_pmg_acme_challenge_config {
+    my ($filename, $fh, $cfg) = @_;
+    my $raw = PVE::ACME::Challenge->write_config($filename, $cfg);
+    PVE::Tools::safe_print($filename, $fh, $raw);
+}
+
+PVE::INotify::register_file($inotify_file_id, $config_filename,
+			    \&read_pmg_acme_challenge_config,
+			    \&write_pmg_acme_challenge_config,
+			    undef,
+			    always_call_parser => 1);
+
+sub lock_config {
+    my ($code) = @_;
+    my $p = PVE::Tools::lock_file($lockfile, undef, $code);
+    die $@ if $@;
+    return $p;
+}
+
+sub load_config {
+    # auto-adds the standalone plugin if no config is there for backwards
+    # compatibility, so ALWAYS call the cfs registered parser
+    return PVE::INotify::read_file($inotify_file_id);
+}
+
+sub write_config {
+    my ($self) = @_;
+    return PVE::INotify::write_file($inotify_file_id, $self);
+}
+
+my $plugin_type_enum = PVE::ACME::Challenge->lookup_types();
+
+my $modify_cfg_for_api = sub {
+    my ($cfg, $pluginid) = @_;
+
+    die "ACME plugin '$pluginid' not defined\n" if !defined($cfg->{ids}->{$pluginid});
+
+    my $plugin_cfg = dclone($cfg->{ids}->{$pluginid});
+    $plugin_cfg->{plugin} = $pluginid;
+    $plugin_cfg->{digest} = $cfg->{digest};
+
+    return $plugin_cfg;
+};
+
+__PACKAGE__->register_method ({
+    name => 'index',
+    path => '',
+    method => 'GET',
+    permissions => { check => [ 'admin', 'audit' ] },
+    description => "ACME plugin index.",
+    protected => 1,
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    type => {
+		description => "Only list ACME plugins of a specific type",
+		type => 'string',
+		enum => $plugin_type_enum,
+		optional => 1,
+	    },
+	},
+    },
+    returns => {
+	type => 'array',
+	items => {
+	    type => "object",
+	    properties => {
+		plugin => get_standard_option('pmg-acme-pluginid'),
+	    },
+	},
+	links => [ { rel => 'child', href => "{plugin}" } ],
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $cfg = load_config();
+
+	my $res = [];
+	foreach my $pluginid (keys %{$cfg->{ids}}) {
+	    my $plugin_cfg = $modify_cfg_for_api->($cfg, $pluginid);
+	    next if $param->{type} && $param->{type} ne $plugin_cfg->{type};
+	    push @$res, $plugin_cfg;
+	}
+
+	return $res;
+    }
+});
+
+__PACKAGE__->register_method({
+    name => 'get_plugin_config',
+    path => '{id}',
+    method => 'GET',
+    description => "Get ACME plugin configuration.",
+    permissions => { check => [ 'admin', 'audit' ] },
+    protected => 1,
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    id => get_standard_option('pmg-acme-pluginid'),
+	},
+    },
+    returns => {
+	type => 'object',
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $cfg = load_config();
+	return $modify_cfg_for_api->($cfg, $param->{id});
+    }
+});
+
+__PACKAGE__->register_method({
+    name => 'add_plugin',
+    path => '',
+    method => 'POST',
+    description => "Add ACME plugin configuration.",
+    permissions => { check => [ 'admin' ] },
+    protected => 1,
+    parameters => PVE::ACME::Challenge->createSchema(),
+    returns => {
+	type => "null"
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $id = extract_param($param, 'id');
+	my $type = extract_param($param, 'type');
+
+	lock_config(sub {
+	    my $cfg = load_config();
+	    die "ACME plugin ID '$id' already exists\n" if defined($cfg->{ids}->{$id});
+
+	    my $plugin = PVE::ACME::Challenge->lookup($type);
+	    my $opts = $plugin->check_config($id, $param, 1, 1);
+
+	    $cfg->{ids}->{$id} = $opts;
+	    $cfg->{ids}->{$id}->{type} = $type;
+
+	    write_config($cfg);
+	});
+	die "$@" if $@;
+
+	return undef;
+    }
+});
+
+__PACKAGE__->register_method({
+    name => 'update_plugin',
+    path => '{id}',
+    method => 'PUT',
+    description => "Update ACME plugin configuration.",
+    permissions => { check => [ 'admin' ] },
+    protected => 1,
+    parameters => PVE::ACME::Challenge->updateSchema(),
+    returns => {
+	type => "null"
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $id = extract_param($param, 'id');
+	my $delete = extract_param($param, 'delete');
+	my $digest = extract_param($param, 'digest');
+
+	lock_config(sub {
+	    my $cfg = load_config();
+	    PVE::Tools::assert_if_modified($cfg->{digest}, $digest);
+	    my $plugin_cfg = $cfg->{ids}->{$id};
+	    die "ACME plugin ID '$id' does not exist\n" if !$plugin_cfg;
+
+	    my $type = $plugin_cfg->{type};
+	    my $plugin = PVE::ACME::Challenge->lookup($type);
+
+	    if (defined($delete)) {
+		my $schema = $plugin->private();
+		my $options = $schema->{options}->{$type};
+		for my $k (PVE::Tools::split_list($delete)) {
+		    my $d = $options->{$k} || die "no such option '$k'\n";
+		    die "unable to delete required option '$k'\n" if !$d->{optional};
+
+		    delete $cfg->{ids}->{$id}->{$k};
+		}
+	    }
+
+	    my $opts = $plugin->check_config($id, $param, 0, 1);
+	    for my $k (sort keys %$opts) {
+		$plugin_cfg->{$k} = $opts->{$k};
+	    }
+
+	    write_config($cfg);
+	});
+	die "$@" if $@;
+
+	return undef;
+    }
+});
+
+__PACKAGE__->register_method({
+    name => 'delete_plugin',
+    path => '{id}',
+    method => 'DELETE',
+    description => "Delete ACME plugin configuration.",
+    permissions => { check => [ 'admin' ] },
+    protected => 1,
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    id => get_standard_option('pmg-acme-pluginid'),
+	},
+    },
+    returns => {
+	type => "null"
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $id = extract_param($param, 'id');
+
+	lock_config(sub {
+	    my $cfg = load_config();
+
+	    delete $cfg->{ids}->{$id};
+
+	    write_config($cfg);
+	});
+	die "$@" if $@;
+
+	return undef;
+    }
+});
+
+1;
diff --git a/src/PMG/API2/Config.pm b/src/PMG/API2/Config.pm
index e11eb3f..c5697e1 100644
--- a/src/PMG/API2/Config.pm
+++ b/src/PMG/API2/Config.pm
@@ -26,6 +26,7 @@ use PMG::API2::DestinationTLSPolicy;
 use PMG::API2::DKIMSign;
 use PMG::API2::SACustom;
 use PMG::API2::PBS::Remote;
+use PMG::API2::ACME;
 
 use base qw(PVE::RESTHandler);
 
@@ -99,6 +100,11 @@ __PACKAGE__->register_method ({
     path => 'pbs',
 });
 
+__PACKAGE__->register_method ({
+    subclass => "PMG::API2::ACME",
+    path => 'acme',
+});
+
 __PACKAGE__->register_method ({
     name => 'index', 
     path => '',
@@ -138,6 +144,7 @@ __PACKAGE__->register_method ({
 	push @$res, { section => 'tlspolicy' };
 	push @$res, { section => 'dkim' };
 	push @$res, { section => 'pbs' };
+	push @$res, { section => 'acme' };
 
 	return $res;
     }});
-- 
2.20.1

[pmg-devel] [PATCH api 6/8] add certificates api endpoint

[Wolfgang Bumiller]

This adds /nodes/{nodename}/certificates endpoint
containing:

  /custom/{type} - update smtp or api certificates manually
  /acme/{type} - update via acme

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/Makefile                 |   1 +
 src/PMG/API2/Certificates.pm | 690 +++++++++++++++++++++++++++++++++++
 src/PMG/API2/Nodes.pm        |   7 +
 3 files changed, 698 insertions(+)
 create mode 100644 src/PMG/API2/Certificates.pm

diff --git a/src/Makefile b/src/Makefile
index ebc6bd8..e0629b2 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -155,6 +155,7 @@ LIBSOURCES =				\
 	PMG/API2/When.pm		\
 	PMG/API2/What.pm		\
 	PMG/API2/Action.pm		\
+	PMG/API2/Certificates.pm	\
 	PMG/API2/ACME.pm		\
 	PMG/API2/ACMEPlugin.pm		\
 	PMG/API2.pm			\
diff --git a/src/PMG/API2/Certificates.pm b/src/PMG/API2/Certificates.pm
new file mode 100644
index 0000000..d196af6
--- /dev/null
+++ b/src/PMG/API2/Certificates.pm
@@ -0,0 +1,690 @@
+package PMG::API2::Certificates;
+
+use strict;
+use warnings;
+
+use PVE::Certificate;
+use PVE::Exception qw(raise raise_param_exc);
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::Tools qw(extract_param file_get_contents file_set_contents);
+
+use PMG::CertHelpers;
+use PMG::NodeConfig;
+use PMG::RS::CSR;
+
+use PMG::API2::ACMEPlugin;
+
+use base qw(PVE::RESTHandler);
+
+my $acme_account_dir = PMG::CertHelpers::acme_account_dir();
+
+sub first_typed_pem_entry : prototype($$) {
+    my ($label, $data) = @_;
+
+    if ($data =~ /^(-----BEGIN (?<label>\Q$label\E)-----\n.*?\n-----END \g{label}-----)$/ms) {
+	chomp(my $content = $1);
+	return $content;
+    }
+    return undef;
+}
+
+sub pem_private_key : prototype($) {
+    my ($data) = @_;
+    return first_typed_pem_entry('PRIVATE KEY', $data);
+}
+
+sub pem_certificate : prototype($) {
+    my ($data) = @_;
+    return first_typed_pem_entry('CERTIFICATE', $data);
+}
+
+my sub restart_after_cert_update : prototype($) {
+    my ($type) = @_;
+
+    if ($type eq 'api') {
+	print "Restarting pmgproxy\n";
+	PVE::Tools::run_command(['systemctl', 'reload-or-restart', 'pmgproxy']);
+    }
+};
+
+my sub update_cert : prototype($$$$) {
+    my ($type, $cert_path, $certificate, $force) = @_;
+    my $code = sub {
+	print "Setting custom certificate file $cert_path\n";
+	PMG::CertHelpers::set_cert_file($certificate, $cert_path, $force);
+
+	restart_after_cert_update($type);
+    };
+    PMG::CertHelpers::cert_lock(10, $code);
+};
+
+__PACKAGE__->register_method ({
+    name => 'index',
+    path => '',
+    method => 'GET',
+    permissions => { user => 'all' },
+    description => "Node index.",
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    node => get_standard_option('pve-node'),
+	},
+    },
+    returns => {
+	type => 'array',
+	items => {
+	    type => "object",
+	    properties => {},
+	},
+	links => [ { rel => 'child', href => "{name}" } ],
+    },
+    code => sub {
+	my ($param) = @_;
+
+	return [
+	    { name => 'acme' },
+	    { name => 'custom' },
+	    { name => 'info' },
+	    { name => 'config' },
+	];
+    },
+});
+
+__PACKAGE__->register_method ({
+    name => 'info',
+    path => 'info',
+    method => 'GET',
+    permissions => { user => 'all' },
+    proxyto => 'node',
+    protected => 1,
+    description => "Get information about the node's certificates.",
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    node => get_standard_option('pve-node'),
+	},
+    },
+    returns => {
+	type => 'array',
+	items => get_standard_option('pve-certificate-info'),
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $res = [];
+	for my $path (&PMG::CertHelpers::API_CERT, &PMG::CertHelpers::SMTP_CERT) {
+	    eval {
+		my $info = PVE::Certificate::get_certificate_info($path);
+		push @$res, $info if $info;
+	    };
+	}
+	return $res;
+    },
+});
+
+__PACKAGE__->register_method ({
+    name => 'custom_cert_index',
+    path => 'custom',
+    method => 'GET',
+    permissions => { user => 'all' },
+    description => "Certificate index.",
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    node => get_standard_option('pve-node'),
+	},
+    },
+    returns => {
+	type => 'array',
+	items => {
+	    type => "object",
+	    properties => {},
+	},
+	links => [ { rel => 'child', href => "{type}" } ],
+    },
+    code => sub {
+	my ($param) = @_;
+
+	return [
+	    { type => 'api' },
+	    { type => 'smtp' },
+	];
+    },
+});
+
+__PACKAGE__->register_method ({
+    name => 'upload_custom_cert',
+    path => 'custom/{type}',
+    method => 'POST',
+    permissions => { check => [ 'admin' ] },
+    description => 'Upload or update custom certificate chain and key.',
+    protected => 1,
+    proxyto => 'node',
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    node => get_standard_option('pve-node'),
+	    certificates => {
+		type => 'string',
+		format => 'pem-certificate-chain',
+		description => 'PEM encoded certificate (chain).',
+	    },
+	    key => {
+		type => 'string',
+		description => 'PEM encoded private key.',
+		format => 'pem-string',
+		optional => 0,
+	    },
+	    type => get_standard_option('pmg-certificate-type'),
+	    force => {
+		type => 'boolean',
+		description => 'Overwrite existing custom or ACME certificate files.',
+		optional => 1,
+		default => 0,
+	    },
+	    restart => {
+		type => 'boolean',
+		description => 'Restart services.',
+		optional => 1,
+		default => 0,
+	    },
+	},
+    },
+    returns => get_standard_option('pve-certificate-info'),
+    code => sub {
+	my ($param) = @_;
+
+	my $type = extract_param($param, 'type'); # also used to know which service to restart
+	my $cert_path = PMG::CertHelpers::cert_path($type);
+
+	my $certs = extract_param($param, 'certificates');
+	$certs = PVE::Certificate::strip_leading_text($certs);
+
+	my $key = extract_param($param, 'key');
+	if ($key) {
+	    $key = PVE::Certificate::strip_leading_text($key);
+	    $certs = "$key\n$certs";
+	} else {
+	    my $private_key = pem_private_key($certs);
+	    if (!defined($private_key)) {
+		my $old = file_get_contents($cert_path);
+		$private_key = pem_private_key($old);
+		if (!defined($private_key)) {
+		    raise_param_exc({
+			'key' => "Attempted to upload custom certificate without (existing) key."
+		    })
+		}
+
+		# copy the old certificate's key:
+		$certs = "$key\n$certs";
+	    }
+	}
+
+	my $info;
+
+	my $code = sub {
+	    print "Setting custom certificate file $cert_path\n";
+	    $info = PMG::CertHelpers::set_cert_file($certs, $cert_path, $param->{force});
+
+	    if ($type eq 'api' && $param->{restart}) {
+		print "Restarting pmgproxy\n";
+		PVE::Tools::run_command(['systemctl', 'reload-or-restart', 'pmgproxy']);
+	    }
+	};
+
+	PMG::CertHelpers::cert_lock(10, $code);
+	die "$@\n" if $@;
+
+	if ($type eq 'smtp') {
+	    $code = sub {
+		my $cfg = PMG::Config->new();
+
+		print "Rewriting postfix config\n";
+		$cfg->set('mail', 'tls', 1);
+		$cfg->rewrite_config_postfix();
+
+		if ($param->{restart}) {
+		    print "Reloading postfix\n";
+		    PMG::Utils::service_cmd('postfix', 'reload');
+		}
+	    };
+	    PMG::Config::lock_config($code, "failed to reload postfix");
+	}
+
+	return $info;
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'remove_custom_cert',
+    path => 'custom/{type}',
+    method => 'DELETE',
+    permissions => { check => [ 'admin' ] },
+    description => 'DELETE custom certificate chain and key.',
+    protected => 1,
+    proxyto => 'node',
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    node => get_standard_option('pve-node'),
+	    type => get_standard_option('pmg-certificate-type'),
+	    restart => {
+		type => 'boolean',
+		description => 'Restart pmgproxy.',
+		optional => 1,
+		default => 0,
+	    },
+	},
+    },
+    returns => {
+	type => 'null',
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $type = extract_param($param, 'type');
+	my $cert_path = PMG::CertHelpers::cert_path($type);
+
+	my $code = sub {
+	    print "Deleting custom certificate files\n";
+	    unlink $cert_path;
+
+	    if ($param->{restart}) {
+		restart_after_cert_update($type);
+	    }
+	};
+
+	PMG::CertHelpers::cert_lock(10, $code);
+	die "$@\n" if $@;
+
+	return undef;
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'acme_cert_index',
+    path => 'acme',
+    method => 'GET',
+    permissions => { user => 'all' },
+    description => "ACME Certificate index.",
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    node => get_standard_option('pve-node'),
+	},
+    },
+    returns => {
+	type => 'array',
+	items => {
+	    type => "object",
+	    properties => {},
+	},
+	links => [ { rel => 'child', href => "{type}" } ],
+    },
+    code => sub {
+	my ($param) = @_;
+
+	return [
+	    { type => 'api' },
+	    { type => 'smtp' },
+	];
+    },
+});
+
+my $order_certificate = sub {
+    my ($acme, $acme_node_config) = @_;
+
+    my $plugins = PMG::API2::ACMEPlugin::load_config();
+
+    print "Placing ACME order\n";
+    my ($order_url, $order) = $acme->new_order([ keys %{$acme_node_config->{domains}} ]);
+    print "Order URL: $order_url\n";
+    for my $auth_url (@{$order->{authorizations}}) {
+	print "\nGetting authorization details from '$auth_url'\n";
+	my $auth = $acme->get_authorization($auth_url);
+
+	# force lower case, like get_acme_conf does
+	my $domain = lc($auth->{identifier}->{value});
+	if ($auth->{status} eq 'valid') {
+	    print "$domain is already validated!\n";
+	} else {
+	    print "The validation for $domain is pending!\n";
+
+	    my $domain_config = $acme_node_config->{domains}->{$domain};
+	    die "no config for domain '$domain'\n" if !$domain_config;
+
+	    my $plugin_id = $domain_config->{plugin};
+
+	    my $plugin_cfg = $plugins->{ids}->{$plugin_id};
+	    die "plugin '$plugin_id' for domain '$domain' not found!\n"
+		if !$plugin_cfg;
+
+	    my $data = {
+		plugin => $plugin_cfg,
+		alias => $domain_config->{alias},
+	    };
+
+	    my $plugin = PVE::ACME::Challenge->lookup($plugin_cfg->{type});
+	    $plugin->setup($acme, $auth, $data);
+
+	    print "Triggering validation\n";
+	    eval {
+		die "no validation URL returned by plugin '$plugin_id' for domain '$domain'\n"
+		    if !defined($data->{url});
+
+		$acme->request_challenge_validation($data->{url});
+		print "Sleeping for 5 seconds\n";
+		sleep 5;
+		while (1) {
+		    $auth = $acme->get_authorization($auth_url);
+		    if ($auth->{status} eq 'pending') {
+			print "Status is still 'pending', trying again in 10 seconds\n";
+			sleep 10;
+			next;
+		    } elsif ($auth->{status} eq 'valid') {
+			print "Status is 'valid', domain '$domain' OK!\n";
+			last;
+		    }
+		    die "validating challenge '$auth_url' failed - status: $auth->{status}\n";
+		}
+	    };
+	    my $err = $@;
+	    eval { $plugin->teardown($acme, $auth, $data) };
+	    warn "$@\n" if $@;
+	    die $err if $err;
+	}
+    }
+    print "\nAll domains validated!\n";
+    print "\nCreating CSR\n";
+    # Currently we only support dns entries, so extract those from the order:
+    my $san = [
+	map {
+	    $_->{value}
+	} grep {
+	    $_->{type} eq 'dns'
+	} $order->{identifiers}->@*
+    ];
+    die "DNS identifiers are required to generate a CSR.\n" if !scalar @$san;
+    my ($csr_der, $key) = PMG::RS::CSR::generate_csr($san, {});
+
+    my $finalize_error_cnt = 0;
+    print "Checking order status\n";
+    while (1) {
+	$order = $acme->get_order($order_url);
+	if ($order->{status} eq 'pending') {
+	    print "still pending, trying to finalize order\n";
+	    # FIXME
+	    # to be compatible with and without the order ready state we try to
+	    # finalize even at the 'pending' state and give up after 5
+	    # unsuccessful tries this can be removed when the letsencrypt api
+	    # definitely has implemented the 'ready' state
+	    eval {
+		$acme->finalize_order($order->{finalize}, $csr_der);
+	    };
+	    if (my $err = $@) {
+		die $err if $finalize_error_cnt >= 5;
+
+		$finalize_error_cnt++;
+		warn $err;
+	    }
+	    sleep 5;
+	    next;
+	} elsif ($order->{status} eq 'ready') {
+	    print "Order is ready, finalizing order\n";
+	    $acme->finalize_order($order->{finalize}, $csr_der);
+	    sleep 5;
+	    next;
+	} elsif ($order->{status} eq 'processing') {
+	    print "still processing, trying again in 30 seconds\n";
+	    sleep 30;
+	    next;
+	} elsif ($order->{status} eq 'valid') {
+	    print "valid!\n";
+	    last;
+	}
+	die "order status: $order->{status}\n";
+    }
+
+    print "\nDownloading certificate\n";
+    my $cert = $acme->get_certificate($order->{certificate});
+
+    return ($cert, $key);
+};
+
+# Filter domains and raise an error if the list becomes empty.
+my $filter_domains = sub {
+    my ($acme_config, $type) = @_;
+
+    my $domains = $acme_config->{domains};
+    foreach my $domain (keys %$domains) {
+	my $entry = $domains->{$domain};
+	if (!(grep { $_ eq $type } PVE::Tools::split_list($entry->{usage}))) {
+	    delete $domains->{$domain};
+	}
+    }
+
+    if (!%$domains) {
+	raise("No domains configured for type '$type'\n", 400);
+    }
+};
+
+__PACKAGE__->register_method ({
+    name => 'new_acme_cert',
+    path => 'acme/{type}',
+    method => 'POST',
+    permissions => { check => [ 'admin' ] },
+    description => 'Order a new certificate from ACME-compatible CA.',
+    protected => 1,
+    proxyto => 'node',
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    node => get_standard_option('pve-node'),
+	    type => get_standard_option('pmg-certificate-type'),
+	    force => {
+		type => 'boolean',
+		description => 'Overwrite existing custom certificate.',
+		optional => 1,
+		default => 0,
+	    },
+	},
+    },
+    returns => {
+	type => 'string',
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $type = extract_param($param, 'type'); # also used to know which service to restart
+	my $cert_path = PMG::CertHelpers::cert_path($type);
+	raise_param_exc({'force' => "Custom certificate exists but 'force' is not set."})
+	    if !$param->{force} && -e $cert_path;
+
+	my $node_config = PMG::NodeConfig::load_config();
+	my $acme_config = PMG::NodeConfig::get_acme_conf($node_config);
+	raise("ACME domain list in configuration is missing!", 400)
+	    if !$acme_config || !$acme_config->{domains}->%*;
+
+	$filter_domains->($acme_config, $type);
+
+	my $rpcenv = PMG::RESTEnvironment->get();
+	my $authuser = $rpcenv->get_user();
+
+	my $realcmd = sub {
+	    STDOUT->autoflush(1);
+	    my $account = $acme_config->{account};
+	    my $account_file = "${acme_account_dir}/${account}";
+	    die "ACME account config file '$account' does not exist.\n"
+		if ! -e $account_file;
+
+	    print "Loading ACME account details\n";
+	    my $acme = PMG::RS::Acme->load($account_file);
+
+	    my ($cert, $key) = $order_certificate->($acme, $acme_config);
+	    my $certificate = "$key\n$cert";
+
+	    update_cert($type, $cert_path, $certificate, $param->{force});
+
+	    if ($type eq 'smtp') {
+		my $code = sub {
+		    my $cfg = PMG::Config->new();
+
+		    print "Rewriting postfix config\n";
+		    $cfg->set('mail', 'tls', 1);
+		    if ($cfg->rewrite_config_postfix()) {
+			print "Reloading postfix\n";
+			PMG::Utils::service_cmd('postfix', 'reload');
+		    }
+		};
+		PMG::Config::lock_config($code, "failed to reload postfix");
+	    }
+
+	    die "$@\n" if $@;
+	};
+
+	return $rpcenv->fork_worker("acmenewcert", undef, $authuser, $realcmd);
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'renew_acme_cert',
+    path => 'acme/{type}',
+    method => 'PUT',
+    permissions => { check => [ 'admin' ] },
+    description => "Renew existing certificate from CA.",
+    protected => 1,
+    proxyto => 'node',
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    node => get_standard_option('pve-node'),
+	    type => get_standard_option('pmg-certificate-type'),
+	    force => {
+		type => 'boolean',
+		description => 'Force renewal even if expiry is more than 30 days away.',
+		optional => 1,
+		default => 0,
+	    },
+	},
+    },
+    returns => {
+	type => 'string',
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $type = extract_param($param, 'type'); # also used to know which service to restart
+	my $cert_path = PMG::CertHelpers::cert_path($type);
+
+	raise("No current (custom) certificate found, please order a new certificate!\n")
+	    if ! -e $cert_path;
+
+	my $expires_soon = PVE::Certificate::check_expiry($cert_path, time() + 30*24*60*60);
+	raise_param_exc({'force' => "Certificate does not expire within the next 30 days, and 'force' is not set."})
+	    if !$expires_soon && !$param->{force};
+
+	my $node_config = PMG::NodeConfig::load_config();
+	my $acme_config = PMG::NodeConfig::get_acme_conf($node_config);
+	raise("ACME domain list in configuration is missing!", 400)
+	    if !$acme_config || !$acme_config->{domains}->%*;
+
+	$filter_domains->($acme_config, $type);
+
+	my $rpcenv = PMG::RESTEnvironment->get();
+	my $authuser = $rpcenv->get_user();
+
+	my $old_cert = PVE::Tools::file_get_contents($cert_path);
+
+	my $realcmd = sub {
+	    STDOUT->autoflush(1);
+	    my $account = $acme_config->{account};
+	    my $account_file = "${acme_account_dir}/${account}";
+	    die "ACME account config file '$account' does not exist.\n"
+		if ! -e $account_file;
+
+	    print "Loading ACME account details\n";
+	    my $acme = PMG::RS::Acme->load($account_file);
+
+	    my ($cert, $key) = $order_certificate->($acme, $acme_config);
+	    my $certificate = "$key\n$cert";
+
+	    update_cert($type, $cert_path, $certificate, 1);
+
+	    if (defined($old_cert)) {
+		print "Revoking old certificate\n";
+		eval { $acme->revoke_certificate($old_cert, undef) };
+		warn "Revoke request to CA failed: $@" if $@;
+	    }
+	};
+
+	return $rpcenv->fork_worker("acmerenew", undef, $authuser, $realcmd);
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'revoke_acme_cert',
+    path => 'acme/{type}',
+    method => 'DELETE',
+    permissions => { check => [ 'admin' ] },
+    description => "Revoke existing certificate from CA.",
+    protected => 1,
+    proxyto => 'node',
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    node => get_standard_option('pve-node'),
+	    type => get_standard_option('pmg-certificate-type'),
+	},
+    },
+    returns => {
+	type => 'string',
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $type = extract_param($param, 'type'); # also used to know which service to restart
+	my $cert_path = PMG::CertHelpers::cert_path($type);
+
+	my $node_config = PMG::NodeConfig::load_config();
+	my $acme_config = PMG::NodeConfig::get_acme_conf($node_config);
+	raise("ACME domain list in configuration is missing!", 400)
+	    if !$acme_config || !$acme_config->{domains}->%*;
+
+	$filter_domains->($acme_config, $type);
+
+	my $rpcenv = PMG::RESTEnvironment->get();
+	my $authuser = $rpcenv->get_user();
+
+	my $cert = PVE::Tools::file_get_contents($cert_path);
+	$cert = pem_certificate($cert)
+	    or die "no certificate section found in '$cert_path'\n";
+
+	my $realcmd = sub {
+	    STDOUT->autoflush(1);
+	    my $account = $acme_config->{account};
+	    my $account_file = "${acme_account_dir}/${account}";
+	    die "ACME account config file '$account' does not exist.\n"
+		if ! -e $account_file;
+
+	    print "Loading ACME account details\n";
+	    my $acme = PMG::RS::Acme->load($account_file);
+
+	    print "Revoking old certificate\n";
+	    eval { $acme->revoke_certificate($cert, undef) };
+	    if (my $err = $@) {
+		# is there a better check?
+		die "Revoke request to CA failed: $err" if $err !~ /"Certificate is expired"/;
+	    }
+
+	    my $code = sub {
+		print "Deleting certificate files\n";
+		unlink $cert_path;
+
+		# FIXME: Regenerate self-signed `api` certificate.
+		restart_after_cert_update($type);
+	    };
+
+	    PMG::CertHelpers::cert_lock(10, $code);
+	    die "$@\n" if $@;
+	};
+
+	return $rpcenv->fork_worker("acmerevoke", undef, $authuser, $realcmd);
+    }});
+
+1;
diff --git a/src/PMG/API2/Nodes.pm b/src/PMG/API2/Nodes.pm
index c0f5963..b6f0cd5 100644
--- a/src/PMG/API2/Nodes.pm
+++ b/src/PMG/API2/Nodes.pm
@@ -27,6 +27,7 @@ use PMG::API2::Postfix;
 use PMG::API2::MailTracker;
 use PMG::API2::Backup;
 use PMG::API2::PBS::Job;
+use PMG::API2::Certificates;
 
 use base qw(PVE::RESTHandler);
 
@@ -85,6 +86,11 @@ __PACKAGE__->register_method ({
     path => 'pbs',
 });
 
+__PACKAGE__->register_method ({
+    subclass => "PMG::API2::Certificates",
+    path => 'certificates',
+});
+
 __PACKAGE__->register_method ({
     name => 'index',
     path => '',
@@ -126,6 +132,7 @@ __PACKAGE__->register_method ({
 	    { name => 'subscription' },
 	    { name => 'termproxy' },
 	    { name => 'rrddata' },
+	    { name => 'certificates' },
 	];
 
 	return $result;
-- 
2.20.1

[pmg-devel] [PATCH api 7/8] add node-config api entry points

[Wolfgang Bumiller]

adds /nodes/{nodename}/config to access node config

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/Makefile               |  1 +
 src/PMG/API2/NodeConfig.pm | 90 ++++++++++++++++++++++++++++++++++++++
 src/PMG/API2/Nodes.pm      |  7 +++
 3 files changed, 98 insertions(+)
 create mode 100644 src/PMG/API2/NodeConfig.pm

diff --git a/src/Makefile b/src/Makefile
index e0629b2..eac682b 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -158,6 +158,7 @@ LIBSOURCES =				\
 	PMG/API2/Certificates.pm	\
 	PMG/API2/ACME.pm		\
 	PMG/API2/ACMEPlugin.pm		\
+	PMG/API2/NodeConfig.pm		\
 	PMG/API2.pm			\
 
 SOURCES = ${LIBSOURCES} ${CLI_BINARIES} ${TEMPLATES_FILES} ${CONF_MANS} ${CLI_MANS} ${SERVICE_MANS} ${SERVICE_UNITS} ${TIMER_UNITS} pmg-sources.list pmg-apt.conf pmg-initramfs.conf
diff --git a/src/PMG/API2/NodeConfig.pm b/src/PMG/API2/NodeConfig.pm
new file mode 100644
index 0000000..284f663
--- /dev/null
+++ b/src/PMG/API2/NodeConfig.pm
@@ -0,0 +1,90 @@
+package PMG::API2::NodeConfig;
+
+use strict;
+use warnings;
+
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::Tools qw(extract_param);
+
+use PMG::NodeConfig;
+
+use base qw(PVE::RESTHandler);
+
+__PACKAGE__->register_method ({
+    name => 'get_config',
+    path => '',
+    method => 'GET',
+    description => "Get node configuration options.",
+    protected => 1,
+    proxyto => 'node',
+    permissions => { check => [ 'admin', 'audit' ] },
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    node => get_standard_option('pve-node'),
+	},
+    },
+    returns =>  PMG::NodeConfig::acme_config_schema({
+	digest => {
+	    type => 'string',
+	    description => 'Prevent changes if current configuration file has different SHA1 digest. This can be used to prevent concurrent modifications.',
+	    maxLength => 40,
+	    optional => 1,
+	},
+    }),
+    code => sub {
+	my ($param) = @_;
+
+	return PMG::NodeConfig::load_config();
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'set_config',
+    path => '',
+    method => 'PUT',
+    description => "Set node configuration options.",
+    protected => 1,
+    proxyto => 'node',
+    permissions => { check => [ 'admin', 'audit' ] },
+    parameters => PMG::NodeConfig::acme_config_schema({
+	delete => {
+	    type => 'string', format => 'pve-configid-list',
+	    description => "A list of settings you want to delete.",
+	    optional => 1,
+	},
+	digest => {
+	    type => 'string',
+	    description => 'Prevent changes if current configuration file has different SHA1 digest. This can be used to prevent concurrent modifications.',
+	    maxLength => 40,
+	    optional => 1,
+	},
+	node => get_standard_option('pve-node'),
+    }),
+    returns => { type => "null" },
+    code => sub {
+	my ($param) = @_;
+
+	my $node = extract_param($param, 'node');
+	my $delete = extract_param($param, 'delete');
+	my $digest = extract_param($param, 'digest');
+
+	PMG::NodeConfig::lock_config(sub {
+	    my $conf = PMG::NodeConfig::load_config();
+
+	    PVE::Tools::assert_if_modified($digest, delete $conf->{digest});
+
+	    foreach my $opt (PVE::Tools::split_list($delete)) {
+		delete $conf->{$opt};
+	    };
+
+	    foreach my $opt (keys %$param) {
+		$conf->{$opt} = $param->{$opt};
+	    }
+
+	    PMG::NodeConfig::write_config($conf);
+	});
+
+	return undef;
+    }});
+
+1;
diff --git a/src/PMG/API2/Nodes.pm b/src/PMG/API2/Nodes.pm
index b6f0cd5..8cdf935 100644
--- a/src/PMG/API2/Nodes.pm
+++ b/src/PMG/API2/Nodes.pm
@@ -28,6 +28,7 @@ use PMG::API2::MailTracker;
 use PMG::API2::Backup;
 use PMG::API2::PBS::Job;
 use PMG::API2::Certificates;
+use PMG::API2::NodeConfig;
 
 use base qw(PVE::RESTHandler);
 
@@ -91,6 +92,11 @@ __PACKAGE__->register_method ({
     path => 'certificates',
 });
 
+__PACKAGE__->register_method ({
+    subclass => "PMG::API2::NodeConfig",
+    path => 'config',
+});
+
 __PACKAGE__->register_method ({
     name => 'index',
     path => '',
@@ -133,6 +139,7 @@ __PACKAGE__->register_method ({
 	    { name => 'termproxy' },
 	    { name => 'rrddata' },
 	    { name => 'certificates' },
+	    { name => 'config' },
 	];
 
 	return $result;
-- 
2.20.1

[pmg-devel] [PATCH api 8/8] add acme and cert subcommands to pmgconfig

[Wolfgang Bumiller]

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/PMG/CLI/pmgconfig.pm | 178 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 178 insertions(+)

diff --git a/src/PMG/CLI/pmgconfig.pm b/src/PMG/CLI/pmgconfig.pm
index 85edfa5..4f948cf 100644
--- a/src/PMG/CLI/pmgconfig.pm
+++ b/src/PMG/CLI/pmgconfig.pm
@@ -5,10 +5,13 @@ use warnings;
 use IO::File;
 use Data::Dumper;
 
+use Term::ReadLine;
+
 use PVE::SafeSyslog;
 use PVE::Tools qw(extract_param);
 use PVE::INotify;
 use PVE::CLIHandler;
+use PVE::JSONSchema qw(get_standard_option);
 
 use PMG::RESTEnvironment;
 use PMG::RuleDB;
@@ -18,14 +21,52 @@ use PMG::LDAPConfig;
 use PMG::LDAPSet;
 use PMG::Config;
 use PMG::Ticket;
+
+use PMG::API2::ACME;
+use PMG::API2::ACMEPlugin;
+use PMG::API2::Certificates;
 use PMG::API2::DKIMSign;
 
 use base qw(PVE::CLIHandler);
 
+my $nodename = PVE::INotify::nodename();
+
 sub setup_environment {
     PMG::RESTEnvironment->setup_default_cli_env();
 }
 
+my $upid_exit = sub {
+    my $upid = shift;
+    my $status = PVE::Tools::upid_read_status($upid);
+    print "Task $status\n";
+    exit($status eq 'OK' ? 0 : -1);
+};
+
+sub param_mapping {
+    my ($name) = @_;
+
+    my $load_file_and_encode = sub {
+	my ($filename) = @_;
+
+	return PVE::ACME::Challenge->encode_value('string', 'data', PVE::Tools::file_get_contents($filename));
+    };
+
+    my $mapping = {
+	'upload_custom_cert' => [
+	    'certificates',
+	    'key',
+	],
+	'add_plugin' => [
+	    ['data', $load_file_and_encode, "File with one key-value pair per line, will be base64url encode for storage in plugin config.", 0],
+	],
+	'update_plugin' => [
+	    ['data', $load_file_and_encode, "File with one key-value pair per line, will be base64url encode for storage in plugin config.", 0],
+	],
+    };
+
+    return $mapping->{$name};
+}
+
 __PACKAGE__->register_method ({
     name => 'dump',
     path => 'dump',
@@ -184,6 +225,84 @@ __PACKAGE__->register_method ({
 	return undef;
     }});
 
+__PACKAGE__->register_method({
+    name => 'acme_register',
+    path => 'acme_register',
+    method => 'POST',
+    description => "Register a new ACME account with a compatible CA.",
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    name => get_standard_option('pmg-acme-account-name'),
+	    contact => get_standard_option('pmg-acme-account-contact'),
+	    directory => get_standard_option('pmg-acme-directory-url', {
+		optional => 1,
+	    }),
+	},
+    },
+    returns => { type => 'null' },
+    code => sub {
+	my ($param) = @_;
+
+	if (!$param->{directory}) {
+	    my $directories = PMG::API2::ACME->get_directories({});
+	    print "Directory endpoints:\n";
+	    my $i = 0;
+	    while ($i < @$directories) {
+		print $i, ") ", $directories->[$i]->{name}, " (", $directories->[$i]->{url}, ")\n";
+		$i++;
+	    }
+	    print $i, ") Custom\n";
+
+	    my $term = Term::ReadLine->new('pmgconfig');
+	    my $get_dir_selection = sub {
+		my $selection = $term->readline("Enter selection: ");
+		if ($selection =~ /^(\d+)$/) {
+		    $selection = $1;
+		    if ($selection == $i) {
+			$param->{directory} = $term->readline("Enter custom URL: ");
+			return;
+		    } elsif ($selection < $i && $selection >= 0) {
+			$param->{directory} = $directories->[$selection]->{url};
+			return;
+		    }
+		}
+		print "Invalid selection.\n";
+	    };
+
+	    my $attempts = 0;
+	    while (!$param->{directory}) {
+		die "Aborting.\n" if $attempts > 3;
+		$get_dir_selection->();
+		$attempts++;
+	    }
+	}
+	print "\nAttempting to fetch Terms of Service from '$param->{directory}'..\n";
+	my $tos = PMG::API2::ACME->get_tos({ directory => $param->{directory} });
+	if ($tos) {
+	    print "Terms of Service: $tos\n";
+	    my $term = Term::ReadLine->new('pvenode');
+	    my $agreed = $term->readline('Do you agree to the above terms? [y|N]: ');
+	    die "Cannot continue without agreeing to ToS, aborting.\n"
+		if ($agreed !~ /^y$/i);
+
+	    $param->{tos_url} = $tos;
+	} else {
+	    print "No Terms of Service found, proceeding.\n";
+	}
+	print "\nAttempting to register account with '$param->{directory}'..\n";
+
+	$upid_exit->(PMG::API2::ACME->register_account($param));
+    }});
+
+my $print_cert_info = sub {
+    my ($schema, $cert, $options) = @_;
+
+    my $order = [qw(filename fingerprint subject issuer notbefore notafter public-key-type public-key-bits san)];
+    PVE::CLIFormatter::print_api_result(
+	$cert, $schema, $order, { %$options, noheader => 1, sort_key => 0 });
+};
+
 our $cmddef = {
     'dump' => [ __PACKAGE__, 'dump', []],
     sync => [ __PACKAGE__, 'sync', []],
@@ -198,6 +317,65 @@ our $cmddef = {
 	    die "no dkim_selector configured\n" if !defined($res->{record});
 	    print "$res->{record}\n";
 	}],
+
+    cert => {
+	info => [ 'PMG::API2::Certificates', 'info', [], { node => $nodename }, sub {
+	    my ($res, $schema, $options) = @_;
+
+	    if (!$options->{'output-format'} || $options->{'output-format'} eq 'text') {
+		for my $cert (sort { $a->{filename} cmp $b->{filename} } @$res) {
+		    $print_cert_info->($schema->{items}, $cert, $options);
+		}
+	    } else {
+		PVE::CLIFormatter::print_api_result($res, $schema, undef, $options);
+	    }
+
+	}, $PVE::RESTHandler::standard_output_options],
+	set => [ 'PMG::API2::Certificates', 'upload_custom_cert', ['type', 'certificates', 'key'], { node => $nodename }, sub {
+	    my ($res, $schema, $options) = @_;
+	    $print_cert_info->($schema, $res, $options);
+	}, $PVE::RESTHandler::standard_output_options],
+	delete => [ 'PMG::API2::Certificates', 'remove_custom_cert', ['type', 'restart'], { node => $nodename } ],
+    },
+
+    acme => {
+	account => {
+	    list => [ 'PMG::API2::ACME', 'account_index', [], {}, sub {
+		my ($res) = @_;
+		for my $acc (@$res) {
+		    print "$acc->{name}\n";
+		}
+	    }],
+	    register => [ __PACKAGE__, 'acme_register', ['name', 'contact'], {}, $upid_exit ],
+	    deactivate => [ 'PMG::API2::ACME', 'deactivate_account', ['name'], {}, $upid_exit ],
+	    info => [ 'PMG::API2::ACME', 'get_account', ['name'], {}, sub {
+		my ($data, $schema, $options) = @_;
+		PVE::CLIFormatter::print_api_result($data, $schema, undef, $options);
+	    }, $PVE::RESTHandler::standard_output_options],
+	    update => [ 'PMG::API2::ACME', 'update_account', ['name'], {}, $upid_exit ],
+	},
+	cert => {
+	    order => [ 'PMG::API2::Certificates', 'new_acme_cert', ['type'], { node => $nodename }, $upid_exit ],
+
+
+	    renew => [ 'PMG::API2::Certificates', 'renew_acme_cert', ['type'], { node => $nodename }, $upid_exit ],
+	    revoke => [ 'PMG::API2::Certificates', 'revoke_acme_cert', ['type'], { node => $nodename }, $upid_exit ],
+	},
+	plugin => {
+	    list => [ 'PMG::API2::ACMEPlugin', 'index', [], {}, sub {
+		my ($data, $schema, $options) = @_;
+		PVE::CLIFormatter::print_api_result($data, $schema, undef, $options);
+	    }, $PVE::RESTHandler::standard_output_options ],
+	    config => [ 'PMG::API2::ACMEPlugin', 'get_plugin_config', ['id'], {}, sub {
+		my ($data, $schema, $options) = @_;
+		PVE::CLIFormatter::print_api_result($data, $schema, undef, $options);
+	    }, $PVE::RESTHandler::standard_output_options ],
+	    add => [ 'PMG::API2::ACMEPlugin', 'add_plugin', ['type', 'id'] ],
+	    set => [ 'PMG::API2::ACMEPlugin', 'update_plugin', ['id'] ],
+	    remove => [ 'PMG::API2::ACMEPlugin', 'delete_plugin', ['id'] ],
+	},
+
+    },
 };
 
 
-- 
2.20.1

[pmg-devel] [PATCH gui] add certificates and acme view

[Wolfgang Bumiller]

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 js/Certificates.js   | 108 +++++++++++++++++++++++++++++++++++++++++++
 js/Makefile          |   1 +
 js/NavigationTree.js |   6 +++
 3 files changed, 115 insertions(+)
 create mode 100644 js/Certificates.js

diff --git a/js/Certificates.js b/js/Certificates.js
new file mode 100644
index 0000000..33b1bde
--- /dev/null
+++ b/js/Certificates.js
@@ -0,0 +1,108 @@
+Ext.define('PMG.CertificateConfiguration', {
+    extend: 'Ext.tab.Panel',
+    alias: 'widget.pmgCertificateConfiguration',
+
+    title: gettext('Certificates'),
+
+    border: false,
+    defaults: { border: false },
+
+    items: [
+	{
+	    itemId: 'certificates',
+	    xtype: 'pmgCertificatesView',
+	    border: 0,
+	},
+	{
+	    itemId: 'acme',
+	    xtype: 'pmgACMEConfigView',
+	    border: 0,
+	},
+    ],
+});
+
+Ext.define('PMG.CertificateView', {
+    extend: 'Ext.container.Container',
+    alias: 'widget.pmgCertificatesView',
+
+    title: gettext('Certificates'),
+
+    //onlineHelp: 'sysadmin_certificate_management',
+
+    initComponent: function() {
+	let me = this;
+
+	Ext.apply(me, {
+	    items: [
+		{
+		    xtype: 'pmxCertificates',
+		    border: 0,
+		    infoUrl: '/nodes/' + Proxmox.NodeName + '/certificates/info',
+		    uploadButtons: [
+			{
+			    name: 'API',
+			    id: 'pmg-api.pem',
+			    url: `/nodes/${Proxmox.NodeName}/certificates/custom/api`,
+			    deletable: false,
+			    reloadUi: true,
+			},
+			{
+			    name: 'SMTP',
+			    id: 'pmg-tls.pem',
+			    url: `/nodes/${Proxmox.NodeName}/certificates/custom/smtp`,
+			    deletable: true,
+			},
+		    ],
+		},
+		{
+		    xtype: 'pmxACMEDomains',
+		    border: 0,
+		    url: `/nodes/${Proxmox.NodeName}/config`,
+		    nodename: Proxmox.NodeName,
+		    acmeUrl: '/config/acme',
+		    domainUsages: [
+			{
+			    usage: 'api',
+			    name: 'API',
+			    url: `/nodes/${Proxmox.NodeName}/certificates/acme/api`,
+			    reloadUi: true,
+			},
+			{
+			    usage: 'smtp',
+			    name: 'SMTP',
+			    url: `/nodes/${Proxmox.NodeName}/certificates/acme/smtp`,
+			},
+		    ],
+		},
+	    ],
+	});
+
+	me.callParent();
+    },
+});
+
+Ext.define('PMG.ACMEConfigView', {
+    extend: 'Ext.panel.Panel',
+    alias: 'widget.pmgACMEConfigView',
+
+    title: gettext('ACME Accounts'),
+
+    //onlineHelp: 'sysadmin_certificate_management',
+
+    items: [
+	{
+	    region: 'north',
+	    border: false,
+	    xtype: 'pmxACMEAccounts',
+	    acmeUrl: '/config/acme',
+	},
+	{
+	    region: 'center',
+	    border: false,
+	    xtype: 'pmxACMEPluginView',
+	    acmeUrl: '/config/acme',
+	},
+    ],
+});
+
+
diff --git a/js/Makefile b/js/Makefile
index a5266fc..43d3ad8 100644
--- a/js/Makefile
+++ b/js/Makefile
@@ -91,6 +91,7 @@ JSSRC=							\
 	ContactStatistics.js				\
 	HourlyMailDistribution.js			\
 	SpamContextMenu.js				\
+	Certificates.js					\
 	Application.js
 
 OnlineHelpInfo.js: /usr/bin/asciidoc-pmg
diff --git a/js/NavigationTree.js b/js/NavigationTree.js
index ac01fd6..63f8e94 100644
--- a/js/NavigationTree.js
+++ b/js/NavigationTree.js
@@ -92,6 +92,12 @@ Ext.define('PMG.store.NavigationStore', {
 			path: 'pmgBackupConfiguration',
 			leaf: true,
 		    },
+		    {
+			text: gettext('Certificates'),
+			iconCls: 'fa fa-certificate',
+			path: 'pmgCertificateConfiguration',
+			leaf: true,
+		    },
 		],
 	    },
 	    {
-- 
2.20.1

[pmg-devel] [PATCH acme] add missing 'use PVE::Acme' statement

[Wolfgang Bumiller]

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/PVE/ACME/DNSChallenge.pm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/PVE/ACME/DNSChallenge.pm b/src/PVE/ACME/DNSChallenge.pm
index 0c6314b..632210e 100644
--- a/src/PVE/ACME/DNSChallenge.pm
+++ b/src/PVE/ACME/DNSChallenge.pm
@@ -6,6 +6,8 @@ use warnings;
 use Digest::SHA qw(sha256);
 use PVE::Tools;
 
+use PVE::ACME;
+
 use base qw(PVE::ACME::Challenge);
 
 my $ACME_PATH = '/usr/share/proxmox-acme/proxmox-acme';
-- 
2.20.1

[pmg-devel] [PATCH widget-toolkit 1/7] Utils: add ACME related utilities

[Wolfgang Bumiller]

copied from PVE with linter fixups

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/Utils.js | 179 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 179 insertions(+)

diff --git a/src/Utils.js b/src/Utils.js
index af5f1db..60c96e0 100644
--- a/src/Utils.js
+++ b/src/Utils.js
@@ -809,6 +809,185 @@ utilities: {
 	}
     },
 
+    render_optional_url: function(value) {
+	if (value && value.match(/^https?:\/\//) !== null) {
+	    return '<a target="_blank" href="' + value + '">' + value + '</a>';
+	}
+	return value;
+    },
+
+    render_san: function(value) {
+	var names = [];
+	if (Ext.isArray(value)) {
+	    value.forEach(function(val) {
+		if (!Ext.isNumber(val)) {
+		    names.push(val);
+		}
+	    });
+	    return names.join('<br>');
+	}
+	return value;
+    },
+
+    loadTextFromFile: function(file, callback, maxBytes) {
+	let maxSize = maxBytes || 8192;
+	if (file.size > maxSize) {
+	    Ext.Msg.alert(gettext('Error'), gettext("Invalid file size: ") + file.size);
+	    return;
+	}
+	let reader = new FileReader();
+	reader.onload = evt => callback(evt.target.result);
+	reader.readAsText(file);
+    },
+
+    parsePropertyString: function(value, defaultKey) {
+	var res = {},
+	    error;
+
+	if (typeof value !== 'string' || value === '') {
+	    return res;
+	}
+
+	Ext.Array.each(value.split(','), function(p) {
+	    var kv = p.split('=', 2);
+	    if (Ext.isDefined(kv[1])) {
+		res[kv[0]] = kv[1];
+	    } else if (Ext.isDefined(defaultKey)) {
+		if (Ext.isDefined(res[defaultKey])) {
+		    error = 'defaultKey may be only defined once in propertyString';
+		    return false; // break
+		}
+		res[defaultKey] = kv[0];
+	    } else {
+		error = 'invalid propertyString, not a key=value pair and no defaultKey defined';
+		return false; // break
+	    }
+	    return true;
+	});
+
+	if (error !== undefined) {
+	    console.error(error);
+	    return undefined;
+	}
+
+	return res;
+    },
+
+    printPropertyString: function(data, defaultKey) {
+	var stringparts = [],
+	    gotDefaultKeyVal = false,
+	    defaultKeyVal;
+
+	Ext.Object.each(data, function(key, value) {
+	    if (defaultKey !== undefined && key === defaultKey) {
+		gotDefaultKeyVal = true;
+		defaultKeyVal = value;
+	    } else if (Ext.isArray(value)) {
+		stringparts.push(key + '=' + value.join(';'));
+	    } else if (value !== '') {
+		stringparts.push(key + '=' + value);
+	    }
+	});
+
+	stringparts = stringparts.sort();
+	if (gotDefaultKeyVal) {
+	    stringparts.unshift(defaultKeyVal);
+	}
+
+	return stringparts.join(',');
+    },
+
+    acmedomain_count: 5,
+
+    parseACMEPluginData: function(data) {
+	let res = {};
+	let extradata = [];
+	data.split('\n').forEach((line) => {
+	    // capture everything after the first = as value
+	    let [key, value] = line.split('=');
+	    if (value !== undefined) {
+		res[key] = value;
+	    } else {
+		extradata.push(line);
+	    }
+	});
+	return [res, extradata];
+    },
+
+    delete_if_default: function(values, fieldname, default_val, create) {
+	if (values[fieldname] === '' || values[fieldname] === default_val) {
+	    if (!create) {
+		if (values.delete) {
+		    if (Ext.isArray(values.delete)) {
+			values.delete.push(fieldname);
+		    } else {
+			values.delete += ',' + fieldname;
+		    }
+		} else {
+		    values.delete = fieldname;
+		}
+	    }
+
+	    delete values[fieldname];
+	}
+    },
+
+    printACME: function(value) {
+	if (Ext.isArray(value.domains)) {
+	    value.domains = value.domains.join(';');
+	}
+	return Proxmox.Utils.printPropertyString(value);
+    },
+
+    parseACME: function(value) {
+	if (!value) {
+	    return {};
+	}
+
+	var res = {};
+	var error;
+
+	Ext.Array.each(value.split(','), function(p) {
+	    var kv = p.split('=', 2);
+	    if (Ext.isDefined(kv[1])) {
+		res[kv[0]] = kv[1];
+	    } else {
+		error = 'Failed to parse key-value pair: '+p;
+		return false;
+	    }
+	    return true;
+	});
+
+	if (error !== undefined) {
+	    console.error(error);
+	    return undefined;
+	}
+
+	if (res.domains !== undefined) {
+	    res.domains = res.domains.split(/;/);
+	}
+
+	return res;
+    },
+
+    add_domain_to_acme: function(acme, domain) {
+	if (acme.domains === undefined) {
+	    acme.domains = [domain];
+	} else {
+	    acme.domains.push(domain);
+	    acme.domains = acme.domains.filter((value, index, self) => self.indexOf(value) === index);
+	}
+	return acme;
+    },
+
+    remove_domain_from_acme: function(acme, domain) {
+	if (acme.domains !== undefined) {
+	    acme.domains = acme.domains.filter(
+		(value, index, self) => self.indexOf(value) === index && value !== domain,
+	    );
+	}
+	return acme;
+    },
 },
 
     singleton: true,
-- 
2.20.1

[pmg-devel] [PATCH widget-toolkit 2/7] add ACME related data models

[Wolfgang Bumiller]

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/Makefile                   |  2 ++
 src/data/model/ACME.js         | 30 ++++++++++++++++++++++++++++++
 src/data/model/Certificates.js |  6 ++++++
 3 files changed, 38 insertions(+)
 create mode 100644 src/data/model/ACME.js
 create mode 100644 src/data/model/Certificates.js

diff --git a/src/Makefile b/src/Makefile
index 46b90ae..3861bfc 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -15,6 +15,8 @@ JSSRC=					\
 	data/RRDStore.js		\
 	data/TimezoneStore.js		\
 	data/model/Realm.js		\
+	data/model/Certificates.js	\
+	data/model/ACME.js		\
 	form/DisplayEdit.js		\
 	form/ExpireDate.js		\
 	form/IntegerField.js		\
diff --git a/src/data/model/ACME.js b/src/data/model/ACME.js
new file mode 100644
index 0000000..c05572e
--- /dev/null
+++ b/src/data/model/ACME.js
@@ -0,0 +1,30 @@
+Ext.define('proxmox-acme-accounts', {
+    extend: 'Ext.data.Model',
+    fields: ['name'],
+    proxy: {
+	type: 'proxmox',
+	//url: "/api2/json/cluster/acme/account",
+    },
+    idProperty: 'name',
+});
+
+Ext.define('proxmox-acme-challenges', {
+    extend: 'Ext.data.Model',
+    fields: ['id', 'type', 'schema'],
+    proxy: {
+	type: 'proxmox',
+        //url: "/api2/json/cluster/acme/challenge-schema",
+    },
+    idProperty: 'id',
+});
+
+
+Ext.define('proxmox-acme-plugins', {
+    extend: 'Ext.data.Model',
+    fields: ['type', 'plugin', 'api'],
+    proxy: {
+	type: 'proxmox',
+	//url: "/api2/json/cluster/acme/plugins",
+    },
+    idProperty: 'plugin',
+});
diff --git a/src/data/model/Certificates.js b/src/data/model/Certificates.js
new file mode 100644
index 0000000..f3e2a7f
--- /dev/null
+++ b/src/data/model/Certificates.js
@@ -0,0 +1,6 @@
+Ext.define('proxmox-certificate', {
+    extend: 'Ext.data.Model',
+
+    fields: ['filename', 'fingerprint', 'issuer', 'notafter', 'notbefore', 'subject', 'san', 'public-key-bits', 'public-key-type'],
+    idProperty: 'filename',
+});
-- 
2.20.1

[pmg-devel] [PATCH widget-toolkit 3/7] add ACME forms:

[Wolfgang Bumiller]

Mostly copied from PVE, but the user needs to set the URL
property so their stores can load the data, whereas in PVE
this was hardcoded.

API selector:
  needs its url to point to the challenge-schema url

Acme Account selector:
  needs its url to point to the acme account index

Acme Plugin selector:
  needs its url to point to the plugin index

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/Makefile     |   1 +
 src/form/ACME.js | 109 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 110 insertions(+)
 create mode 100644 src/form/ACME.js

diff --git a/src/Makefile b/src/Makefile
index 3861bfc..d0435b8 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -35,6 +35,7 @@ JSSRC=					\
 	form/DiskSelector.js		\
 	form/MultiDiskSelector.js	\
 	form/TaskTypeSelector.js	\
+	form/ACME.js			\
 	button/Button.js		\
 	button/HelpButton.js		\
 	grid/ObjectGrid.js		\
diff --git a/src/form/ACME.js b/src/form/ACME.js
new file mode 100644
index 0000000..8b93e30
--- /dev/null
+++ b/src/form/ACME.js
@@ -0,0 +1,109 @@
+Ext.define('Proxmox.form.ACMEApiSelector', {
+    extend: 'Ext.form.field.ComboBox',
+    alias: 'widget.pmxACMEApiSelector',
+
+    fieldLabel: gettext('DNS API'),
+    displayField: 'name',
+    valueField: 'id',
+
+    store: {
+	model: 'proxmox-acme-challenges',
+	autoLoad: true,
+    },
+
+    triggerAction: 'all',
+    queryMode: 'local',
+    allowBlank: false,
+    editable: true,
+    forceSelection: true,
+    anyMatch: true,
+    selectOnFocus: true,
+
+    getSchema: function() {
+	let me = this;
+	let val = me.getValue();
+	if (val) {
+	    let record = me.getStore().findRecord('id', val, 0, false, true, true);
+	    if (record) {
+		return record.data.schema;
+	    }
+	}
+	return {};
+    },
+
+    initComponent: function() {
+        let me = this;
+
+        if (!me.url) {
+            throw "no url given";
+        }
+
+        me.callParent();
+        me.getStore().getProxy().setUrl(me.url);
+    },
+});
+
+Ext.define('Proxmox.form.ACMEAccountSelector', {
+    extend: 'Ext.form.field.ComboBox',
+    alias: 'widget.pmxACMEAccountSelector',
+
+    displayField: 'name',
+    valueField: 'name',
+
+    store: {
+	model: 'proxmox-acme-accounts',
+	autoLoad: true,
+    },
+
+    triggerAction: 'all',
+    queryMode: 'local',
+    allowBlank: false,
+    editable: false,
+    forceSelection: true,
+
+    isEmpty: function() {
+	return this.getStore().getData().length === 0;
+    },
+
+    initComponent: function() {
+        let me = this;
+
+        if (!me.url) {
+            throw "no url given";
+        }
+
+        me.callParent();
+        me.getStore().getProxy().setUrl(me.url);
+    },
+});
+
+Ext.define('Proxmox.form.ACMEPluginSelector', {
+    extend: 'Ext.form.field.ComboBox',
+    alias: 'widget.pmxACMEPluginSelector',
+
+    fieldLabel: gettext('Plugin'),
+    displayField: 'plugin',
+    valueField: 'plugin',
+
+    store: {
+	model: 'proxmox-acme-plugins',
+	autoLoad: true,
+	filters: item => item.data.type === 'dns',
+    },
+
+    triggerAction: 'all',
+    queryMode: 'local',
+    allowBlank: false,
+    editable: false,
+
+    initComponent: function() {
+        let me = this;
+
+        if (!me.url) {
+            throw "no url given";
+        }
+
+        me.callParent();
+        me.getStore().getProxy().setUrl(me.url);
+    },
+});
-- 
2.20.1

[pmg-devel] [PATCH widget-toolkit 4/7] add certificate panel

[Wolfgang Bumiller]

Again, initially copied from PVE but adapted so it can be
used by both. (PVE side still needs to be tested though.)

The 'nodename' property is optional (since on PMG we
currently don't expose them via the UI directly). Instead,
the certificate info URL is required and the 'uploadButtons'
need to be passed, which just contains the certificate
"name", id (filename), url, and whether it is deletable and
whether a GUI reload is required after changing it. If only
1 entry is passed, the button stays a regular button (that
way PVE should still look the same), whereas in PMG we have
a menu to select between API and SMTP certificates.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/Makefile               |   2 +
 src/panel/Certificates.js  | 267 +++++++++++++++++++++++++++++++++++++
 src/window/Certificates.js | 205 ++++++++++++++++++++++++++++
 3 files changed, 474 insertions(+)
 create mode 100644 src/panel/Certificates.js
 create mode 100644 src/window/Certificates.js

diff --git a/src/Makefile b/src/Makefile
index d0435b8..d782e92 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -49,6 +49,7 @@ JSSRC=					\
 	panel/PruneKeepPanel.js		\
 	panel/RRDChart.js		\
 	panel/GaugeWidget.js		\
+	panel/Certificates.js		\
 	window/Edit.js			\
 	window/PasswordEdit.js		\
 	window/SafeDestroy.js		\
@@ -56,6 +57,7 @@ JSSRC=					\
 	window/LanguageEdit.js		\
 	window/DiskSmart.js		\
 	window/ZFSDetail.js		\
+	window/Certificates.js		\
 	node/APT.js			\
 	node/NetworkEdit.js		\
 	node/NetworkView.js		\
diff --git a/src/panel/Certificates.js b/src/panel/Certificates.js
new file mode 100644
index 0000000..332a189
--- /dev/null
+++ b/src/panel/Certificates.js
@@ -0,0 +1,267 @@
+Ext.define('Proxmox.panel.Certificates', {
+    extend: 'Ext.grid.Panel',
+    xtype: 'pmxCertificates',
+
+    // array of { name, id (=filename), url, deletable, reloadUi }
+    uploadButtons: undefined,
+
+    // The /info path for the current node.
+    infoUrl: undefined,
+
+    columns: [
+	{
+	    header: gettext('File'),
+	    width: 150,
+	    dataIndex: 'filename',
+	},
+	{
+	    header: gettext('Issuer'),
+	    flex: 1,
+	    dataIndex: 'issuer',
+	},
+	{
+	    header: gettext('Subject'),
+	    flex: 1,
+	    dataIndex: 'subject',
+	},
+	{
+	    header: gettext('Public Key Alogrithm'),
+	    flex: 1,
+	    dataIndex: 'public-key-type',
+	    hidden: true,
+	},
+	{
+	    header: gettext('Public Key Size'),
+	    flex: 1,
+	    dataIndex: 'public-key-bits',
+	    hidden: true,
+	},
+	{
+	    header: gettext('Valid Since'),
+	    width: 150,
+	    dataIndex: 'notbefore',
+	    renderer: Proxmox.Utils.render_timestamp,
+	},
+	{
+	    header: gettext('Expires'),
+	    width: 150,
+	    dataIndex: 'notafter',
+	    renderer: Proxmox.Utils.render_timestamp,
+	},
+	{
+	    header: gettext('Subject Alternative Names'),
+	    flex: 1,
+	    dataIndex: 'san',
+	    renderer: Proxmox.Utils.render_san,
+	},
+	{
+	    header: gettext('Fingerprint'),
+	    dataIndex: 'fingerprint',
+	    hidden: true,
+	},
+	{
+	    header: gettext('PEM'),
+	    dataIndex: 'pem',
+	    hidden: true,
+	},
+    ],
+
+    reload: function() {
+	let me = this;
+	me.rstore.load();
+    },
+
+    delete_certificate: function() {
+	let me = this;
+
+	let rec = me.selModel.getSelection()[0];
+	if (!rec) {
+	    return;
+	}
+
+	let cert = me.certById[rec.id];
+	let url = cert.url;
+	Proxmox.Utils.API2Request({
+	    url: `/api2/extjs/${url}?restart=1`,
+	    method: 'DELETE',
+	    success: function(response, opt) {
+		if (cert.reloadUid) {
+		    let txt =
+			gettext('GUI will be restarted with new certificates, please reload!');
+		    Ext.getBody().mask(txt, ['x-mask-loading']);
+		    // reload after 10 seconds automatically
+		    Ext.defer(function() {
+			window.location.reload(true);
+		    }, 10000);
+		}
+	    },
+	    failure: function(response, opt) {
+		Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+	    },
+	});
+    },
+
+    controller: {
+	xclass: 'Ext.app.ViewController',
+	view_certificate: function() {
+	    let me = this;
+	    let view = me.getView();
+
+	    let selection = view.getSelection();
+	    if (!selection || selection.length < 1) {
+		return;
+	    }
+	    let win = Ext.create('Proxmox.window.CertificateViewer', {
+		cert: selection[0].data.filename,
+		url: `/api2/extjs/${view.infoUrl}`,
+	    });
+	    win.show();
+	},
+    },
+
+    listeners: {
+	itemdblclick: 'view_certificate',
+    },
+
+    initComponent: function() {
+	let me = this;
+
+	if (!me.nodename) {
+	    // only used for the store name
+	    me.nodename = "_all";
+	}
+
+	if (!me.uploadButtons) {
+	    throw "no upload buttons defined";
+	}
+
+	if (!me.infoUrl) {
+	    throw "no certificate store url given";
+	}
+
+	me.rstore = Ext.create('Proxmox.data.UpdateStore', {
+	    storeid: 'certs-' + me.nodename,
+	    model: 'proxmox-certificate',
+	    proxy: {
+		type: 'proxmox',
+		url: `/api2/extjs/${me.infoUrl}`,
+	    },
+	});
+
+	me.store = {
+	    type: 'diff',
+	    rstore: me.rstore,
+	};
+
+	let tbar = [];
+
+	me.deletableCertIds = {};
+	me.certById = {};
+	if (me.uploadButtons.length === 1) {
+	    let cert = me.uploadButtons[0];
+
+	    if (!cert.url) {
+		throw "missing certificate url";
+	    }
+
+	    me.certById[cert.id] = cert;
+
+	    if (cert.deletable) {
+		me.deletableCertIds[cert.id] = true;
+	    }
+
+	    tbar.push(
+		{
+		    xtype: 'button',
+		    text: gettext('Upload Custom Certificate'),
+		    handler: function() {
+			let grid = this.up('grid');
+			let win = Ext.create('Proxmox.window.CertificateUpload', {
+			    url: `/api2/extjs/${cert.url}`,
+			    reloadUi: cert.reloadUi,
+			});
+			win.show();
+			win.on('destroy', grid.reload, grid);
+		    },
+		},
+	    );
+	} else {
+	    let items = [];
+
+	    me.selModel = Ext.create('Ext.selection.RowModel', {});
+
+	    for (const cert of me.uploadButtons) {
+		if (!cert.id) {
+		    throw "missing id in certificate entry";
+		}
+
+		if (!cert.url) {
+		    throw "missing url in certificate entry";
+		}
+
+		if (!cert.name) {
+		    throw "missing name in certificate entry";
+		}
+
+		me.certById[cert.id] = cert;
+
+		if (cert.deletable) {
+		    me.deletableCertIds[cert.id] = true;
+		}
+
+		items.push({
+		    text: Ext.String.format('Upload {0} Certificate', cert.name),
+		    handler: function() {
+			let grid = this.up('grid');
+			let win = Ext.create('Proxmox.window.CertificateUpload', {
+			    url: `/api2/extjs/${cert.url}`,
+			    reloadUi: cert.reloadUi,
+			});
+			win.show();
+			win.on('destroy', grid.reload, grid);
+		    },
+		});
+	    }
+
+	    tbar.push(
+		{
+		    text: gettext('Upload Custom Certificate'),
+		    menu: {
+			xtype: 'menu',
+			items,
+		    },
+		},
+	    );
+	}
+
+	tbar.push(
+	    {
+		xtype: 'proxmoxButton',
+		text: gettext('Delete Custom Certificate'),
+		confirmMsg: rec => Ext.String.format(
+		    gettext('Are you sure you want to remove the certificate used for {0}'),
+		    me.certById[rec.id].name,
+		),
+		callback: () => me.reload(),
+		selModel: me.selModel,
+		disabled: true,
+		enableFn: rec => !!me.deletableCertIds[rec.id],
+		handler: function() { me.delete_certificate(); },
+	    },
+	    '-',
+	    {
+		xtype: 'proxmoxButton',
+		itemId: 'viewbtn',
+		disabled: true,
+		text: gettext('View Certificate'),
+		handler: 'view_certificate',
+	    },
+	);
+	Ext.apply(me, { tbar });
+
+	me.callParent();
+
+	me.rstore.startUpdate();
+	me.on('destroy', me.rstore.stopUpdate, me.rstore);
+    },
+});
diff --git a/src/window/Certificates.js b/src/window/Certificates.js
new file mode 100644
index 0000000..1bdf394
--- /dev/null
+++ b/src/window/Certificates.js
@@ -0,0 +1,205 @@
+Ext.define('Proxmox.window.CertificateViewer', {
+    extend: 'Proxmox.window.Edit',
+    xtype: 'pmxCertViewer',
+
+    title: gettext('Certificate'),
+
+    fieldDefaults: {
+	labelWidth: 120,
+    },
+    width: 800,
+    resizable: true,
+
+    items: [
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('Name'),
+	    name: 'filename',
+	},
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('Fingerprint'),
+	    name: 'fingerprint',
+	},
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('Issuer'),
+	    name: 'issuer',
+	},
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('Subject'),
+	    name: 'subject',
+	},
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('Public Key Type'),
+	    name: 'public-key-type',
+	},
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('Public Key Size'),
+	    name: 'public-key-bits',
+	},
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('Valid Since'),
+	    renderer: Proxmox.Utils.render_timestamp,
+	    name: 'notbefore',
+	},
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('Expires'),
+	    renderer: Proxmox.Utils.render_timestamp,
+	    name: 'notafter',
+	},
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('Subject Alternative Names'),
+	    name: 'san',
+	    renderer: Proxmox.Utils.render_san,
+	},
+	{
+	    xtype: 'textarea',
+	    editable: false,
+	    grow: true,
+	    growMax: 200,
+	    fieldLabel: gettext('Certificate'),
+	    name: 'pem',
+	},
+    ],
+
+    initComponent: function() {
+	var me = this;
+
+	if (!me.cert) {
+	    throw "no cert given";
+	}
+
+	if (!me.url) {
+	    throw "no url given";
+	}
+
+	me.callParent();
+
+	// hide OK/Reset button, because we just want to show data
+	me.down('toolbar[dock=bottom]').setVisible(false);
+
+	me.load({
+	    success: function(response) {
+		if (Ext.isArray(response.result.data)) {
+		    Ext.Array.each(response.result.data, function(item) {
+			if (item.filename === me.cert) {
+			    me.setValues(item);
+			    return false;
+			}
+			return true;
+		    });
+		}
+	    },
+	});
+    },
+});
+
+Ext.define('Proxmox.window.CertificateUpload', {
+    extend: 'Proxmox.window.Edit',
+    xtype: 'pmxCertUpload',
+
+    title: gettext('Upload Custom Certificate'),
+    resizable: false,
+    isCreate: true,
+    submitText: gettext('Upload'),
+    method: 'POST',
+    width: 600,
+
+    // whether the UI needs a reload after this
+    reloadUi: undefined,
+
+    apiCallDone: function(success, response, options) {
+	let me = this;
+
+	if (!success || !me.reloadUi) {
+	    return;
+	}
+
+	var txt = gettext('GUI server will be restarted with new certificates, please reload!');
+	Ext.getBody().mask(txt, ['pve-static-mask']);
+	// reload after 10 seconds automatically
+	Ext.defer(function() {
+	    window.location.reload(true);
+	}, 10000);
+    },
+
+    items: [
+	{
+	    fieldLabel: gettext('Private Key (Optional)'),
+	    labelAlign: 'top',
+	    emptyText: gettext('No change'),
+	    name: 'key',
+	    xtype: 'textarea',
+	},
+	{
+	    xtype: 'filebutton',
+	    text: gettext('From File'),
+	    listeners: {
+		change: function(btn, e, value) {
+		    let form = this.up('form');
+		    e = e.event;
+		    Ext.Array.each(e.target.files, function(file) {
+			Proxmox.Utils.loadSSHKeyFromFile(file, function(res) {
+			    form.down('field[name=key]').setValue(res);
+			});
+		    });
+		    btn.reset();
+		},
+	    },
+	},
+	{
+	    xtype: 'box',
+	    autoEl: 'hr',
+	},
+	{
+	    fieldLabel: gettext('Certificate Chain'),
+	    labelAlign: 'top',
+	    allowBlank: false,
+	    name: 'certificates',
+	    xtype: 'textarea',
+	},
+	{
+	    xtype: 'filebutton',
+	    text: gettext('From File'),
+	    listeners: {
+		change: function(btn, e, value) {
+		    let form = this.up('form');
+		    e = e.event;
+		    Ext.Array.each(e.target.files, function(file) {
+			Proxmox.Utils.loadSSHKeyFromFile(file, function(res) {
+			    form.down('field[name=certificates]').setValue(res);
+			});
+		    });
+		    btn.reset();
+		},
+	    },
+	},
+	{
+	    xtype: 'hidden',
+	    name: 'restart',
+	    value: '1',
+	},
+	{
+	    xtype: 'hidden',
+	    name: 'force',
+	    value: '1',
+	},
+    ],
+
+    initComponent: function() {
+	var me = this;
+
+	if (!me.url) {
+	    throw "neither url given";
+	}
+
+	me.callParent();
+    },
+});
-- 
2.20.1

[pmg-devel] [PATCH widget-toolkit 5/7] add ACME account panel

[Wolfgang Bumiller]

Copied from PVE with URLs now being based on the 'acmeUrl'
property which should point to the acme/ root containing
/tos, /directories, etc.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/Makefile              |   2 +
 src/panel/ACMEAccount.js  | 116 ++++++++++++++++++++++
 src/window/ACMEAccount.js | 204 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 322 insertions(+)
 create mode 100644 src/panel/ACMEAccount.js
 create mode 100644 src/window/ACMEAccount.js

diff --git a/src/Makefile b/src/Makefile
index d782e92..00a25c7 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -50,6 +50,7 @@ JSSRC=					\
 	panel/RRDChart.js		\
 	panel/GaugeWidget.js		\
 	panel/Certificates.js		\
+	panel/ACMEAccount.js		\
 	window/Edit.js			\
 	window/PasswordEdit.js		\
 	window/SafeDestroy.js		\
@@ -58,6 +59,7 @@ JSSRC=					\
 	window/DiskSmart.js		\
 	window/ZFSDetail.js		\
 	window/Certificates.js		\
+	window/ACMEAccount.js		\
 	node/APT.js			\
 	node/NetworkEdit.js		\
 	node/NetworkView.js		\
diff --git a/src/panel/ACMEAccount.js b/src/panel/ACMEAccount.js
new file mode 100644
index 0000000..c7d329e
--- /dev/null
+++ b/src/panel/ACMEAccount.js
@@ -0,0 +1,116 @@
+Ext.define('Proxmox.panel.ACMEAccounts', {
+    extend: 'Ext.grid.Panel',
+    xtype: 'pmxACMEAccounts',
+
+    title: gettext('Accounts'),
+
+    acmeUrl: undefined,
+
+    controller: {
+	xclass: 'Ext.app.ViewController',
+
+	addAccount: function() {
+	    let me = this;
+	    let view = me.getView();
+	    let defaultExists = view.getStore().findExact('name', 'default') !== -1;
+	    Ext.create('Proxmox.window.ACMEAccountCreate', {
+		defaultExists,
+		acmeUrl: view.acmeUrl,
+		taskDone: function() {
+		    me.reload();
+		},
+	    }).show();
+	},
+
+	viewAccount: function() {
+	    let me = this;
+	    let view = me.getView();
+	    let selection = view.getSelection();
+	    if (selection.length < 1) return;
+	    Ext.create('Proxmox.window.ACMEAccountView', {
+	        url: `${view.acmeUrl}/account/${selection[0].data.name}`,
+	    }).show();
+	},
+
+	reload: function() {
+	    let me = this;
+	    let view = me.getView();
+	    view.getStore().rstore.load();
+	},
+
+	showTaskAndReload: function(options, success, response) {
+	    let me = this;
+	    if (!success) return;
+
+	    let upid = response.result.data;
+	    Ext.create('Proxmox.window.TaskProgress', {
+		upid,
+		taskDone: function() {
+		    me.reload();
+		},
+	    }).show();
+	},
+    },
+
+    minHeight: 150,
+    emptyText: gettext('No Accounts configured'),
+
+    columns: [
+	{
+	    dataIndex: 'name',
+	    text: gettext('Name'),
+	    renderer: Ext.String.htmlEncode,
+	    flex: 1,
+	},
+    ],
+
+    listeners: {
+	itemdblclick: 'viewAccount',
+    },
+
+    store: {
+	type: 'diff',
+	autoDestroy: true,
+	autoDestroyRstore: true,
+	rstore: {
+	    type: 'update',
+	    storeid: 'proxmox-acme-accounts',
+	    model: 'proxmox-acme-accounts',
+	    autoStart: true,
+	},
+	sorters: 'name',
+    },
+
+    initComponent: function() {
+	let me = this;
+
+	if (!me.acmeUrl) {
+	    throw "no acmeUrl given";
+	}
+
+	Ext.apply(me, {
+	    tbar: [
+		{
+		    xtype: 'proxmoxButton',
+		    text: gettext('Add'),
+		    selModel: false,
+		    handler: 'addAccount',
+		},
+		{
+		    xtype: 'proxmoxButton',
+		    text: gettext('View'),
+		    handler: 'viewAccount',
+		    disabled: true,
+		},
+		{
+		    xtype: 'proxmoxStdRemoveButton',
+		    baseurl: `${me.acmeUrl}/account`,
+		    callback: 'showTaskAndReload',
+		},
+	    ],
+	});
+
+	me.callParent();
+	me.store.rstore.proxy.setUrl(`/api2/json/${me.acmeUrl}/account`);
+    },
+});
diff --git a/src/window/ACMEAccount.js b/src/window/ACMEAccount.js
new file mode 100644
index 0000000..05278a8
--- /dev/null
+++ b/src/window/ACMEAccount.js
@@ -0,0 +1,204 @@
+Ext.define('Proxmox.window.ACMEAccountCreate', {
+    extend: 'Proxmox.window.Edit',
+    mixins: ['Proxmox.Mixin.CBind'],
+    xtype: 'pmxACMEAccountCreate',
+
+    acmeUrl: undefined,
+
+    width: 450,
+    title: gettext('Register Account'),
+    isCreate: true,
+    method: 'POST',
+    submitText: gettext('Register'),
+    showTaskViewer: true,
+    defaultExists: false,
+
+    items: [
+	{
+	    xtype: 'proxmoxtextfield',
+	    fieldLabel: gettext('Account Name'),
+	    name: 'name',
+	    cbind: {
+		emptyText: (get) => get('defaultExists') ? '' : 'default',
+		allowBlank: (get) => !get('defaultExists'),
+	    },
+	},
+	{
+	    xtype: 'textfield',
+	    name: 'contact',
+	    vtype: 'email',
+	    allowBlank: false,
+	    fieldLabel: gettext('E-Mail'),
+	},
+	{
+	    xtype: 'proxmoxComboGrid',
+	    name: 'directory',
+	    reference: 'directory',
+	    allowBlank: false,
+	    valueField: 'url',
+	    displayField: 'name',
+	    fieldLabel: gettext('ACME Directory'),
+	    store: {
+		autoLoad: true,
+		fields: ['name', 'url'],
+		idProperty: ['name'],
+		proxy: { type: 'proxmox' },
+		sorters: {
+		    property: 'name',
+		    order: 'ASC',
+		},
+	    },
+	    listConfig: {
+		columns: [
+		    {
+			header: gettext('Name'),
+			dataIndex: 'name',
+			flex: 1,
+		    },
+		    {
+			header: gettext('URL'),
+			dataIndex: 'url',
+			flex: 1,
+		    },
+		],
+	    },
+	    listeners: {
+		change: function(combogrid, value) {
+		    let me = this;
+
+		    if (!value) {
+			return;
+		    }
+
+		    let acmeUrl = me.up('window').acmeUrl;
+
+		    let disp = me.up('window').down('#tos_url_display');
+		    let field = me.up('window').down('#tos_url');
+		    let checkbox = me.up('window').down('#tos_checkbox');
+
+		    disp.setValue(gettext('Loading'));
+		    field.setValue(undefined);
+		    checkbox.setValue(undefined);
+		    checkbox.setHidden(true);
+
+		    Proxmox.Utils.API2Request({
+			url: `${acmeUrl}/tos`,
+			method: 'GET',
+			params: {
+			    directory: value,
+			},
+			success: function(response, opt) {
+			    field.setValue(response.result.data);
+			    disp.setValue(response.result.data);
+			    checkbox.setHidden(false);
+			},
+			failure: function(response, opt) {
+			    Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+			},
+		    });
+		},
+	    },
+	},
+	{
+	    xtype: 'displayfield',
+	    itemId: 'tos_url_display',
+	    renderer: Proxmox.Utils.render_optional_url,
+	    name: 'tos_url_display',
+	},
+	{
+	    xtype: 'hidden',
+	    itemId: 'tos_url',
+	    name: 'tos_url',
+	},
+	{
+	    xtype: 'proxmoxcheckbox',
+	    itemId: 'tos_checkbox',
+	    boxLabel: gettext('Accept TOS'),
+	    submitValue: false,
+	    validateValue: function(value) {
+		if (value && this.checked) {
+		    return true;
+		}
+		return false;
+	    },
+	},
+    ],
+
+    initComponent: function() {
+	let me = this;
+
+	if (!me.acmeUrl) {
+	    throw "no acmeUrl given";
+	}
+
+	me.url = `${me.acmeUrl}/account`;
+
+	me.callParent();
+
+	me.lookup('directory')
+	    .store
+	    .proxy
+	    .setUrl(`/api2/json/${me.acmeUrl}/directories`);
+    },
+});
+
+Ext.define('Proxmox.window.ACMEAccountView', {
+    extend: 'Proxmox.window.Edit',
+    xtype: 'pmxACMEAccountView',
+
+    width: 600,
+    fieldDefaults: {
+	labelWidth: 140,
+    },
+
+    title: gettext('Account'),
+
+    items: [
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('E-Mail'),
+	    name: 'email',
+	},
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('Created'),
+	    name: 'createdAt',
+	},
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('Status'),
+	    name: 'status',
+	},
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('Directory'),
+	    renderer: Proxmox.Utils.render_optional_url,
+	    name: 'directory',
+	},
+	{
+	    xtype: 'displayfield',
+	    fieldLabel: gettext('Terms of Services'),
+	    renderer: Proxmox.Utils.render_optional_url,
+	    name: 'tos',
+	},
+    ],
+
+    initComponent: function() {
+	var me = this;
+
+	me.callParent();
+
+	// hide OK/Reset button, because we just want to show data
+	me.down('toolbar[dock=bottom]').setVisible(false);
+
+	me.load({
+	    success: function(response) {
+		var data = response.result.data;
+		data.email = data.account.contact[0];
+		data.createdAt = data.account.createdAt;
+		data.status = data.account.status;
+		me.setValues(data);
+	    },
+	});
+    },
+});
-- 
2.20.1

[pmg-devel] [PATCH widget-toolkit 6/7] add ACME plugin editing

[Wolfgang Bumiller]

Like with the account panel, the 'acmeUrl' base needs to be
specified, otherwise this is copied from PVE

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/Makefile                 |   2 +
 src/panel/ACMEPlugin.js      | 116 +++++++++++++++++
 src/window/ACMEPluginEdit.js | 242 +++++++++++++++++++++++++++++++++++
 3 files changed, 360 insertions(+)
 create mode 100644 src/panel/ACMEPlugin.js
 create mode 100644 src/window/ACMEPluginEdit.js

diff --git a/src/Makefile b/src/Makefile
index 00a25c7..0e1fb45 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -51,6 +51,7 @@ JSSRC=					\
 	panel/GaugeWidget.js		\
 	panel/Certificates.js		\
 	panel/ACMEAccount.js		\
+	panel/ACMEPlugin.js		\
 	window/Edit.js			\
 	window/PasswordEdit.js		\
 	window/SafeDestroy.js		\
@@ -60,6 +61,7 @@ JSSRC=					\
 	window/ZFSDetail.js		\
 	window/Certificates.js		\
 	window/ACMEAccount.js		\
+	window/ACMEPluginEdit.js	\
 	node/APT.js			\
 	node/NetworkEdit.js		\
 	node/NetworkView.js		\
diff --git a/src/panel/ACMEPlugin.js b/src/panel/ACMEPlugin.js
new file mode 100644
index 0000000..ca58106
--- /dev/null
+++ b/src/panel/ACMEPlugin.js
@@ -0,0 +1,116 @@
+Ext.define('Proxmox.panel.ACMEPluginView', {
+    extend: 'Ext.grid.Panel',
+    alias: 'widget.pmxACMEPluginView',
+
+    title: gettext('Challenge Plugins'),
+    acmeUrl: undefined,
+
+    controller: {
+	xclass: 'Ext.app.ViewController',
+
+	addPlugin: function() {
+	    let me = this;
+	    let view = me.getView();
+	    Ext.create('Proxmox.window.ACMEPluginEdit', {
+		acmeUrl: view.acmeUrl,
+		url: `${view.acmeUrl}/plugins`,
+		isCreate: true,
+		apiCallDone: function() {
+		    me.reload();
+		},
+	    }).show();
+	},
+
+	editPlugin: function() {
+	    let me = this;
+	    let view = me.getView();
+	    let selection = view.getSelection();
+	    if (selection.length < 1) return;
+	    let plugin = selection[0].data.plugin;
+	    Ext.create('Proxmox.window.ACMEPluginEdit', {
+		acmeUrl: view.acmeUrl,
+		url: `${view.acmeUrl}/plugins/${plugin}`,
+		apiCallDone: function() {
+		    me.reload();
+		},
+	    }).show();
+	},
+
+	reload: function() {
+	    let me = this;
+	    let view = me.getView();
+	    view.getStore().rstore.load();
+	},
+    },
+
+    minHeight: 150,
+    emptyText: gettext('No Plugins configured'),
+
+    columns: [
+	{
+	    dataIndex: 'plugin',
+	    text: gettext('Plugin'),
+	    renderer: Ext.String.htmlEncode,
+	    flex: 1,
+	},
+	{
+	    dataIndex: 'api',
+	    text: 'API',
+	    renderer: Ext.String.htmlEncode,
+	    flex: 1,
+	},
+    ],
+
+    listeners: {
+	itemdblclick: 'editPlugin',
+    },
+
+    store: {
+	type: 'diff',
+	autoDestroy: true,
+	autoDestroyRstore: true,
+	rstore: {
+	    type: 'update',
+	    storeid: 'proxmox-acme-plugins',
+	    model: 'proxmox-acme-plugins',
+	    autoStart: true,
+	    filters: item => !!item.data.api,
+	},
+	sorters: 'plugin',
+    },
+
+    initComponent: function() {
+	let me = this;
+
+	if (!me.acmeUrl) {
+	    throw "no acmeUrl given";
+	}
+	me.url = `${me.acmeUrl}/plugins`;
+
+	Ext.apply(me, {
+	    tbar: [
+		{
+		    xtype: 'proxmoxButton',
+		    text: gettext('Add'),
+		    handler: 'addPlugin',
+		    selModel: false,
+		},
+		{
+		    xtype: 'proxmoxButton',
+		    text: gettext('Edit'),
+		    handler: 'editPlugin',
+		    disabled: true,
+		},
+		{
+		    xtype: 'proxmoxStdRemoveButton',
+		    callback: 'reload',
+		    baseurl: `${me.acmeUrl}/plugins`,
+		},
+	    ],
+	});
+
+	me.callParent();
+
+	me.store.rstore.proxy.setUrl(`/api2/json/${me.acmeUrl}/plugins`);
+    },
+});
diff --git a/src/window/ACMEPluginEdit.js b/src/window/ACMEPluginEdit.js
new file mode 100644
index 0000000..237b362
--- /dev/null
+++ b/src/window/ACMEPluginEdit.js
@@ -0,0 +1,242 @@
+Ext.define('Proxmox.window.ACMEPluginEdit', {
+    extend: 'Proxmox.window.Edit',
+    xtype: 'pmxACMEPluginEdit',
+    mixins: ['Proxmox.Mixin.CBind'],
+
+    //onlineHelp: 'sysadmin_certs_acme_plugins',
+
+    isAdd: true,
+    isCreate: false,
+
+    width: 550,
+
+    acmeUrl: undefined,
+
+    subject: 'ACME DNS Plugin',
+
+    cbindData: function(config) {
+	let me = this;
+	return {
+	    challengeSchemaUrl: `/api2/json/${me.acmeUrl}/challenge-schema`,
+	};
+    },
+
+    items: [
+	{
+	    xtype: 'inputpanel',
+	    // we dynamically create fields from the given schema
+	    // things we have to do here:
+	    // * save which fields we created to remove them again
+	    // * split the data from the generic 'data' field into the boxes
+	    // * on deletion collect those values again
+	    // * save the original values of the data field
+	    createdFields: {},
+	    createdInitially: false,
+	    originalValues: {},
+	    createSchemaFields: function(schema) {
+		let me = this;
+		// we know where to add because we define it right below
+		let container = me.down('container');
+		let datafield = me.down('field[name=data]');
+		let hintfield = me.down('field[name=hint]');
+		if (!me.createdInitially) {
+		    [me.originalValues] = Proxmox.Utils.parseACMEPluginData(datafield.getValue());
+		}
+
+		// collect values from custom fields and add it to 'data'',
+		// then remove the custom fields
+		let data = [];
+		for (const [name, field] of Object.entries(me.createdFields)) {
+		    let value = field.getValue();
+		    if (value !== undefined && value !== null && value !== '') {
+			data.push(`${name}=${value}`);
+		    }
+		    container.remove(field);
+		}
+		let datavalue = datafield.getValue();
+		if (datavalue !== undefined && datavalue !== null && datavalue !== '') {
+		    data.push(datavalue);
+		}
+		datafield.setValue(data.join('\n'));
+
+		me.createdFields = {};
+
+		if (typeof schema.fields !== 'object') {
+		    schema.fields = {};
+		}
+		// create custom fields according to schema
+		let gotSchemaField = false;
+		for (const [name, definition] of Object
+		    .entries(schema.fields)
+		    .sort((a, b) => a[0].localeCompare(b[0]))
+		) {
+		    let xtype;
+		    switch (definition.type) {
+			case 'string':
+			    xtype = 'proxmoxtextfield';
+			    break;
+			case 'integer':
+			    xtype = 'proxmoxintegerfield';
+			    break;
+			case 'number':
+			    xtype = 'numberfield';
+			    break;
+			default:
+			    console.warn(`unknown type '${definition.type}'`);
+			    xtype = 'proxmoxtextfield';
+			    break;
+		    }
+
+		    let label = name;
+		    if (typeof definition.name === "string") {
+			label = definition.name;
+		    }
+
+		    let field = Ext.create({
+			xtype,
+			name: `custom_${name}`,
+			fieldLabel: label,
+			width: '100%',
+			labelWidth: 150,
+			labelSeparator: '=',
+			emptyText: definition.default || '',
+			autoEl: definition.description ? {
+			    tag: 'div',
+			    'data-qtip': definition.description,
+			} : undefined,
+		    });
+
+		    me.createdFields[name] = field;
+		    container.add(field);
+		    gotSchemaField = true;
+		}
+		datafield.setHidden(gotSchemaField); // prefer schema-fields
+
+		if (schema.description) {
+		    hintfield.setValue(schema.description);
+		    hintfield.setHidden(false);
+		} else {
+		    hintfield.setValue('');
+		    hintfield.setHidden(true);
+		}
+
+		// parse data from field and set it to the custom ones
+		let extradata = [];
+		[data, extradata] = Proxmox.Utils.parseACMEPluginData(datafield.getValue());
+		for (const [key, value] of Object.entries(data)) {
+		    if (me.createdFields[key]) {
+			me.createdFields[key].setValue(value);
+			me.createdFields[key].originalValue = me.originalValues[key];
+		    } else {
+			extradata.push(`${key}=${value}`);
+		    }
+		}
+		datafield.setValue(extradata.join('\n'));
+		if (!me.createdInitially) {
+		    datafield.resetOriginalValue();
+		    me.createdInitially = true; // save that we initally set that
+		}
+	    },
+
+	    onGetValues: function(values) {
+		let me = this;
+		let win = me.up('pmxACMEPluginEdit');
+		if (win.isCreate) {
+		    values.id = values.plugin;
+		    values.type = 'dns'; // the only one for now
+		}
+		delete values.plugin;
+
+		Proxmox.Utils.delete_if_default(values, 'validation-delay', '30', win.isCreate);
+
+		let data = '';
+		for (const [name, field] of Object.entries(me.createdFields)) {
+		    let value = field.getValue();
+		    if (value !== null && value !== undefined && value !== '') {
+			data += `${name}=${value}\n`;
+		    }
+		    delete values[`custom_${name}`];
+		}
+		values.data = Ext.util.Base64.encode(data + values.data);
+		return values;
+	    },
+
+	    items: [
+		{
+		    xtype: 'pmxDisplayEditField',
+		    cbind: {
+			editable: (get) => get('isCreate'),
+			submitValue: (get) => get('isCreate'),
+		    },
+		    editConfig: {
+			flex: 1,
+			xtype: 'proxmoxtextfield',
+			allowBlank: false,
+		    },
+		    name: 'plugin',
+		    labelWidth: 150,
+		    fieldLabel: gettext('Plugin ID'),
+		},
+		{
+		    xtype: 'proxmoxintegerfield',
+		    name: 'validation-delay',
+		    labelWidth: 150,
+		    fieldLabel: gettext('Validation Delay'),
+		    emptyText: 30,
+		    cbind: {
+			deleteEmpty: '{!isCreate}',
+		    },
+		    minValue: 0,
+		    maxValue: 48*60*60,
+		},
+		{
+		    xtype: 'pmxACMEApiSelector',
+		    name: 'api',
+		    labelWidth: 150,
+		    cbind: {
+			url: '{challengeSchemaUrl}',
+		    },
+		    listeners: {
+			change: function(selector) {
+			    let schema = selector.getSchema();
+			    selector.up('inputpanel').createSchemaFields(schema);
+			},
+		    },
+		},
+		{
+		    xtype: 'textarea',
+		    fieldLabel: gettext('API Data'),
+		    labelWidth: 150,
+		    name: 'data',
+		},
+		{
+		    xtype: 'displayfield',
+		    fieldLabel: gettext('Hint'),
+		    labelWidth: 150,
+		    name: 'hint',
+		    hidden: true,
+		},
+	    ],
+	},
+    ],
+
+    initComponent: function() {
+	var me = this;
+
+	if (!me.acmeUrl) {
+	    throw "no acmeUrl given";
+	}
+
+	me.callParent();
+
+	if (!me.isCreate) {
+	    me.load({
+		success: function(response, opts) {
+		    me.setValues(response.result.data);
+		},
+	    });
+	} else {
+	    me.method = 'POST';
+	}
+    },
+});
-- 
2.20.1

[pmg-devel] [PATCH widget-toolkit 7/7] add ACME domain editing

[Wolfgang Bumiller]

Same deal, however, here the PVE code is has a little bug
where changing the plugin type of a domain makes it
disappear, so this also contains some fixups.

Additionally, this now also adds the ability to change a
domain's "usage" (smtp, api or both), so similar to the
uploadButtons info in the Certificates panel, we now have a
domainUsages info. If it is set, the edit window will show a
multiselect combobox, and the panel will show a usage
column.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
 src/Makefile              |   2 +
 src/panel/ACMEDomains.js  | 482 ++++++++++++++++++++++++++++++++++++++
 src/window/ACMEDomains.js | 213 +++++++++++++++++
 3 files changed, 697 insertions(+)
 create mode 100644 src/panel/ACMEDomains.js
 create mode 100644 src/window/ACMEDomains.js

diff --git a/src/Makefile b/src/Makefile
index 0e1fb45..44c11ea 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -52,6 +52,7 @@ JSSRC=					\
 	panel/Certificates.js		\
 	panel/ACMEAccount.js		\
 	panel/ACMEPlugin.js		\
+	panel/ACMEDomains.js		\
 	window/Edit.js			\
 	window/PasswordEdit.js		\
 	window/SafeDestroy.js		\
@@ -62,6 +63,7 @@ JSSRC=					\
 	window/Certificates.js		\
 	window/ACMEAccount.js		\
 	window/ACMEPluginEdit.js	\
+	window/ACMEDomains.js		\
 	node/APT.js			\
 	node/NetworkEdit.js		\
 	node/NetworkView.js		\
diff --git a/src/panel/ACMEDomains.js b/src/panel/ACMEDomains.js
new file mode 100644
index 0000000..dd01e36
--- /dev/null
+++ b/src/panel/ACMEDomains.js
@@ -0,0 +1,482 @@
+Ext.define('proxmox-acme-domains', {
+    extend: 'Ext.data.Model',
+    fields: ['domain', 'type', 'alias', 'plugin', 'configkey'],
+    idProperty: 'domain',
+});
+
+Ext.define('Proxmox.panel.ACMEDomains', {
+    extend: 'Ext.grid.Panel',
+    xtype: 'pmxACMEDomains',
+    mixins: ['Proxmox.Mixin.CBind'],
+
+    margin: '10 0 0 0',
+    title: 'ACME',
+
+    emptyText: gettext('No Domains configured'),
+
+    // URL to the config containing 'acme' and 'acmedomainX' properties
+    url: undefined,
+
+    // array of { name, url, usageLabel }
+    domainUsages: [],
+
+    acmeUrl: undefined,
+
+    cbindData: function(config) {
+	let me = this;
+	return {
+	    acmeUrl: me.acmeUrl,
+	    accountsUrl: `/api2/json/${me.acmeUrl}/accounts`,
+	};
+    },
+
+    viewModel: {
+	data: {
+	    domaincount: 0,
+	    account: undefined, // the account we display
+	    configaccount: undefined, // the account set in the config
+	    accountEditable: false,
+	    accountsAvailable: false,
+	    hasUsage: false,
+	},
+
+	formulas: {
+	    canOrder: (get) => !!get('account') && get('domaincount') > 0,
+	    editBtnIcon: (get) => 'fa black fa-' + (get('accountEditable') ? 'check' : 'pencil'),
+	    accountTextHidden: (get) => get('accountEditable') || !get('accountsAvailable'),
+	    accountValueHidden: (get) => !get('accountEditable') || !get('accountsAvailable'),
+	    hasUsage: (get) => get('hasUsage'),
+	},
+    },
+
+    controller: {
+	xclass: 'Ext.app.ViewController',
+
+	init: function(view) {
+	    let accountSelector = this.lookup('accountselector');
+	    accountSelector.store.on('load', this.onAccountsLoad, this);
+	},
+
+	onAccountsLoad: function(store, records, success) {
+	    let me = this;
+	    let vm = me.getViewModel();
+	    let configaccount = vm.get('configaccount');
+	    vm.set('accountsAvailable', records.length > 0);
+	    if (me.autoChangeAccount && records.length > 0) {
+		me.changeAccount(records[0].data.name, () => {
+		    vm.set('accountEditable', false);
+		    me.reload();
+		});
+		me.autoChangeAccount = false;
+	    } else if (configaccount) {
+		if (store.findExact('name', configaccount) !== -1) {
+		    vm.set('account', configaccount);
+		} else {
+		    vm.set('account', null);
+		}
+	    }
+	},
+
+	addDomain: function() {
+	    let me = this;
+	    let view = me.getView();
+
+	    Ext.create('Proxmox.window.ACMEDomainEdit', {
+		url: view.url,
+		acmeUrl: view.acmeUrl,
+		nodeconfig: view.nodeconfig,
+		domainUsages: view.domainUsages,
+		apiCallDone: function() {
+		    me.reload();
+		},
+	    }).show();
+	},
+
+	editDomain: function() {
+	    let me = this;
+	    let view = me.getView();
+
+	    let selection = view.getSelection();
+	    if (selection.length < 1) return;
+
+	    Ext.create('Proxmox.window.ACMEDomainEdit', {
+		url: view.url,
+		acmeUrl: view.acmeUrl,
+		nodeconfig: view.nodeconfig,
+		domainUsages: view.domainUsages,
+		domain: selection[0].data,
+		apiCallDone: function() {
+		    me.reload();
+		},
+	    }).show();
+	},
+
+	removeDomain: function() {
+	    let me = this;
+	    let view = me.getView();
+	    let selection = view.getSelection();
+	    if (selection.length < 1) return;
+
+	    let rec = selection[0].data;
+	    let params = {};
+	    if (rec.configkey !== 'acme') {
+		params.delete = rec.configkey;
+	    } else {
+		let acme = Proxmox.Utils.parseACME(view.nodeconfig.acme);
+		Proxmox.Utils.remove_domain_from_acme(acme, rec.domain);
+		params.acme = Proxmox.Utils.printACME(acme);
+	    }
+
+	    Proxmox.Utils.API2Request({
+		method: 'PUT',
+		url: view.url,
+		params,
+		success: function(response, opt) {
+		    me.reload();
+		},
+		failure: function(response, opt) {
+		    Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+		},
+	    });
+	},
+
+	toggleEditAccount: function() {
+	    let me = this;
+	    let vm = me.getViewModel();
+	    let editable = vm.get('accountEditable');
+	    if (editable) {
+		me.changeAccount(vm.get('account'), function() {
+		    vm.set('accountEditable', false);
+		    me.reload();
+		});
+	    } else {
+		vm.set('accountEditable', true);
+	    }
+	},
+
+	changeAccount: function(account, callback) {
+	    let me = this;
+	    let view = me.getView();
+	    let params = {};
+
+	    let acme = Proxmox.Utils.parseACME(view.nodeconfig.acme);
+	    acme.account = account;
+	    params.acme = Proxmox.Utils.printACME(acme);
+
+	    Proxmox.Utils.API2Request({
+		method: 'PUT',
+		waitMsgTarget: view,
+		url: view.url,
+		params,
+		success: function(response, opt) {
+		    if (Ext.isFunction(callback)) {
+			callback();
+		    }
+		},
+		failure: function(response, opt) {
+		    Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+		},
+	    });
+	},
+
+	order: function(cert) {
+	    let me = this;
+	    let view = me.getView();
+
+	    Proxmox.Utils.API2Request({
+		method: 'POST',
+		params: {
+		    force: 1,
+		},
+		url: cert ? cert.url : view.url,
+		success: function(response, opt) {
+		    Ext.create('Proxmox.window.TaskViewer', {
+		        upid: response.result.data,
+		        taskDone: function(success) {
+			    me.orderFinished(success, cert);
+		        },
+		    }).show();
+		},
+		failure: function(response, opt) {
+		    Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+		},
+	    });
+	},
+
+	orderFinished: function(success, cert) {
+	    if (!success || !cert.reloadUi) return;
+	    var txt = gettext('gui will be restarted with new certificates, please reload!');
+	    Ext.getBody().mask(txt, ['x-mask-loading']);
+	    // reload after 10 seconds automatically
+	    Ext.defer(function() {
+		window.location.reload(true);
+	    }, 10000);
+	},
+
+	reload: function() {
+	    let me = this;
+	    let view = me.getView();
+	    view.rstore.load();
+	},
+
+	addAccount: function() {
+	    let me = this;
+	    Ext.create('Proxmox.window.ACMEAccountCreate', {
+		autoShow: true,
+		taskDone: function() {
+		    me.reload();
+		    let accountSelector = me.lookup('accountselector');
+		    me.autoChangeAccount = true;
+		    accountSelector.store.load();
+		},
+	    });
+	},
+    },
+
+    tbar: [
+	{
+	    xtype: 'proxmoxButton',
+	    text: gettext('Add'),
+	    handler: 'addDomain',
+	    selModel: false,
+	},
+	{
+	    xtype: 'proxmoxButton',
+	    text: gettext('Edit'),
+	    disabled: true,
+	    handler: 'editDomain',
+	},
+	{
+	    xtype: 'proxmoxStdRemoveButton',
+	    handler: 'removeDomain',
+	},
+	'-',
+	'order-menu', // placeholder, filled in initComponent
+	'-',
+	{
+	    xtype: 'displayfield',
+	    value: gettext('Using Account') + ':',
+	    bind: {
+		hidden: '{!accountsAvailable}',
+	    },
+	},
+	{
+	    xtype: 'displayfield',
+	    reference: 'accounttext',
+	    renderer: (val) => val || Proxmox.Utils.NoneText,
+	    bind: {
+		value: '{account}',
+		hidden: '{accountTextHidden}',
+	    },
+	},
+	{
+	    xtype: 'pmxACMEAccountSelector',
+	    hidden: true,
+	    reference: 'accountselector',
+	    cbind: {
+		url: '{accountsUrl}',
+	    },
+	    bind: {
+		value: '{account}',
+		hidden: '{accountValueHidden}',
+	    },
+	},
+	{
+	    xtype: 'button',
+	    iconCls: 'fa black fa-pencil',
+	    baseCls: 'x-plain',
+	    userCls: 'pointer',
+	    bind: {
+		iconCls: '{editBtnIcon}',
+		hidden: '{!accountsAvailable}',
+	    },
+	    handler: 'toggleEditAccount',
+	},
+	{
+	    xtype: 'displayfield',
+	    value: gettext('No Account available.'),
+	    bind: {
+		hidden: '{accountsAvailable}',
+	    },
+	},
+	{
+	    xtype: 'button',
+	    hidden: true,
+	    reference: 'accountlink',
+	    text: gettext('Add ACME Account'),
+	    bind: {
+		hidden: '{accountsAvailable}',
+	    },
+	    handler: 'addAccount',
+	},
+    ],
+
+    updateStore: function(store, records, success) {
+	let me = this;
+	let data = [];
+	let rec;
+	if (success && records.length > 0) {
+	    rec = records[0];
+	} else {
+	    rec = {
+		data: {},
+	    };
+	}
+
+	me.nodeconfig = rec.data; // save nodeconfig for updates
+
+	let account = 'default';
+
+	if (rec.data.acme) {
+	    let obj = Proxmox.Utils.parseACME(rec.data.acme);
+	    (obj.domains || []).forEach(domain => {
+		if (domain === '') return;
+		let record = {
+		    domain,
+		    type: 'standalone',
+		    configkey: 'acme',
+		};
+		data.push(record);
+	    });
+
+	    if (obj.account) {
+		account = obj.account;
+	    }
+	}
+
+	let vm = me.getViewModel();
+	let oldaccount = vm.get('account');
+
+	// account changed, and we do not edit currently, load again to verify
+	if (oldaccount !== account && !vm.get('accountEditable')) {
+	    vm.set('configaccount', account);
+	    me.lookup('accountselector').store.load();
+	}
+
+	for (let i = 0; i < Proxmox.Utils.acmedomain_count; i++) {
+	    let acmedomain = rec.data[`acmedomain${i}`];
+	    if (!acmedomain) continue;
+
+	    let record = Proxmox.Utils.parsePropertyString(acmedomain, 'domain');
+	    record.type = record.plugin ? 'dns' : 'standalone';
+	    record.configkey = `acmedomain${i}`;
+	    data.push(record);
+	}
+
+	vm.set('domaincount', data.length);
+	me.store.loadData(data, false);
+    },
+
+    listeners: {
+	itemdblclick: 'editDomain',
+    },
+
+    columns: [
+	{
+	    dataIndex: 'domain',
+	    flex: 5,
+	    text: gettext('Domain'),
+	},
+	{
+	    dataIndex: 'usage',
+	    flex: 1,
+	    text: gettext('Usage'),
+	    bind: {
+		hidden: '{!hasUsage}',
+	    },
+	},
+	{
+	    dataIndex: 'type',
+	    flex: 1,
+	    text: gettext('Type'),
+	},
+	{
+	    dataIndex: 'plugin',
+	    flex: 1,
+	    text: gettext('Plugin'),
+	},
+    ],
+
+    initComponent: function() {
+	let me = this;
+
+	if (!me.acmeUrl) {
+	    throw "no acmeUrl given";
+	}
+
+	if (!me.url) {
+	    throw "no url given";
+	}
+
+	if (!me.nodename) {
+	    throw "no nodename given";
+	}
+
+	me.rstore = Ext.create('Proxmox.data.UpdateStore', {
+	    interval: 10 * 1000,
+	    autoStart: true,
+	    storeid: `proxmox-node-domains-${me.nodename}`,
+	    proxy: {
+		type: 'proxmox',
+		url: `/api2/json/${me.url}`,
+	    },
+	});
+
+	me.store = Ext.create('Ext.data.Store', {
+	    model: 'proxmox-acme-domains',
+	    sorters: 'domain',
+	});
+
+	if (me.domainUsages.length > 0) {
+	    let items = [];
+
+	    for (const cert of me.domainUsages) {
+		if (!cert.name) {
+		    throw "missing certificate url";
+		}
+
+		if (!cert.url) {
+		    throw "missing certificate url";
+		}
+
+		items.push({
+		    text: Ext.String.format('Order {0} Certificate Now', cert.name),
+		    handler: function() {
+			return me.getController().order(cert);
+		    },
+		});
+	    }
+	    me.tbar.splice(
+		me.tbar.indexOf("order-menu"),
+		1,
+		{
+		    text: gettext('Order Certificates Now'),
+		    menu: {
+			xtype: 'menu',
+			items,
+		    },
+		},
+	    );
+	} else {
+	    me.tbar.splice(
+		me.tbar.indexOf("order-menu"),
+		1,
+		{
+		    xtype: 'button',
+		    reference: 'order',
+		    text: gettext('Order Certificates Now'),
+		    bind: {
+			disabled: '{!canOrder}',
+		    },
+		    handler: 'order',
+		},
+	    );
+	}
+
+	me.callParent();
+	me.getViewModel().set('hasUsage', me.domainUsages.length > 0);
+	me.mon(me.rstore, 'load', 'updateStore', me);
+	Proxmox.Utils.monStoreErrors(me, me.rstore);
+	me.on('destroy', me.rstore.stopUpdate, me.rstore);
+    },
+});
diff --git a/src/window/ACMEDomains.js b/src/window/ACMEDomains.js
new file mode 100644
index 0000000..0b4b3f6
--- /dev/null
+++ b/src/window/ACMEDomains.js
@@ -0,0 +1,213 @@
+Ext.define('Proxmox.window.ACMEDomainEdit', {
+    extend: 'Proxmox.window.Edit',
+    xtype: 'pmxACMEDomainEdit',
+    mixins: ['Proxmox.Mixin.CBind'],
+
+    subject: gettext('Domain'),
+    isCreate: false,
+    width: 450,
+    //onlineHelp: 'sysadmin_certificate_management',
+
+    acmeUrl: undefined,
+
+    // config url
+    url: undefined,
+
+    // For PMG the we have multiple certificates, so we have a "usage" attribute & column.
+    domainUsages: [],
+
+    cbindData: function(config) {
+	let me = this;
+	return {
+	    pluginsUrl: `/api2/json/${me.acmeUrl}/plugins`,
+	    hasUsage: me.domainUsages.length > 0,
+	};
+    },
+
+    items: [
+	{
+	    xtype: 'inputpanel',
+	    onGetValues: function(values) {
+		let me = this;
+		let win = me.up('pmxACMEDomainEdit');
+		let nodeconfig = win.nodeconfig;
+		let olddomain = win.domain || {};
+
+		let params = {
+		    digest: nodeconfig.digest,
+		};
+
+		let configkey = olddomain.configkey;
+		let acmeObj = Proxmox.Utils.parseACME(nodeconfig.acme);
+
+		let find_free_slot = () => {
+		    for (let i = 0; i < Proxmox.Utils.acmedomain_count; i++) {
+			if (nodeconfig[`acmedomain${i}`] === undefined) {
+			    return `acmedomain${i}`;
+			}
+		    }
+		    throw "too many domains configured";
+		};
+
+		// If we have a 'usage' property (pmg), we only use the `acmedomainX` config keys.
+		if (win.domainUsages.length > 0) {
+		    if (!configkey || configkey === 'acme') {
+			configkey = find_free_slot();
+		    }
+		    delete values.type;
+		    params[configkey] = Proxmox.Utils.printPropertyString(values, 'domain');
+		    return params;
+		}
+
+		// Otherwise we put the standalone entries into the `domains` list of the `acme`
+		// property string.
+
+		// Then insert the domain depending on its type:
+		if (values.type === 'dns') {
+		    if (!olddomain.configkey || olddomain.configkey === 'acme') {
+			configkey = find_free_slot();
+			if (olddomain.domain) {
+			    // we have to remove the domain from the acme domainlist
+			    Proxmox.Utils.remove_domain_from_acme(acmeObj, olddomain.domain);
+			    params.acme = Proxmox.Utils.printACME(acmeObj);
+			}
+		    }
+
+		    delete values.type;
+		    params[configkey] = Proxmox.Utils.printPropertyString(values, 'domain');
+		} else {
+		    if (olddomain.configkey && olddomain.configkey !== 'acme') {
+			// delete the old dns entry, unless we need to declare its usage:
+			params.delete = [olddomain.configkey];
+		    }
+
+		    // add new, remove old and make entries unique
+		    Proxmox.Utils.add_domain_to_acme(acmeObj, values.domain);
+		    if (olddomain.domain !== values.domain) {
+			Proxmox.Utils.remove_domain_from_acme(acmeObj, olddomain.domain);
+		    }
+		    params.acme = Proxmox.Utils.printACME(acmeObj);
+		}
+
+		return params;
+	    },
+	    items: [
+		{
+		    xtype: 'proxmoxKVComboBox',
+		    name: 'type',
+		    fieldLabel: gettext('Challenge Type'),
+		    allowBlank: false,
+		    value: 'standalone',
+		    comboItems: [
+			['standalone', 'HTTP'],
+			['dns', 'DNS'],
+		    ],
+		    validator: function(value) {
+			let me = this;
+			let win = me.up('pmxACMEDomainEdit');
+			let oldconfigkey = win.domain ? win.domain.configkey : undefined;
+			let val = me.getValue();
+			if (val === 'dns' && (!oldconfigkey || oldconfigkey === 'acme')) {
+			    // we have to check if there is a 'acmedomain' slot left
+			    let found = false;
+			    for (let i = 0; i < Proxmox.Utils.acmedomain_count; i++) {
+				if (!win.nodeconfig[`acmedomain${i}`]) {
+				    found = true;
+				}
+			    }
+			    if (!found) {
+				return gettext('Only 5 Domains with type DNS can be configured');
+			    }
+			}
+
+			return true;
+		    },
+		    listeners: {
+			change: function(cb, value) {
+			    let me = this;
+			    let view = me.up('pmxACMEDomainEdit');
+			    let pluginField = view.down('field[name=plugin]');
+			    pluginField.setDisabled(value !== 'dns');
+			    pluginField.setHidden(value !== 'dns');
+			},
+		    },
+		},
+		{
+		    xtype: 'hidden',
+		    name: 'alias',
+		},
+		{
+		    xtype: 'pmxACMEPluginSelector',
+		    name: 'plugin',
+		    disabled: true,
+		    hidden: true,
+		    allowBlank: false,
+		    cbind: {
+			url: '{pluginsUrl}',
+		    },
+		},
+		{
+		    xtype: 'proxmoxtextfield',
+		    name: 'domain',
+		    allowBlank: false,
+		    vtype: 'DnsName',
+		    value: '',
+		    fieldLabel: gettext('Domain'),
+		},
+		{
+		    xtype: 'combobox',
+		    name: 'usage',
+		    multiSelect: true,
+		    editable: false,
+		    fieldLabel: gettext('Usage'),
+		    cbind: {
+			hidden: '{!hasUsage}',
+			allowBlank: '{!hasUsage}',
+		    },
+		    fields: ['usage', 'name'],
+		    displayField: 'name',
+		    valueField: 'usage',
+		    store: {
+			data: [
+			    { usage: 'api', name: 'API' },
+			    { usage: 'smtp', name: 'SMTP' },
+			],
+		    },
+		},
+	    ],
+	},
+    ],
+
+    initComponent: function() {
+	let me = this;
+
+	if (!me.url) {
+	    throw 'no url given';
+	}
+
+	if (!me.acmeUrl) {
+	    throw 'no acmeUrl given';
+	}
+
+	if (!me.nodeconfig) {
+	    throw 'no nodeconfig given';
+	}
+
+	me.isCreate = !me.domain;
+	if (me.isCreate) {
+	    me.domain = `${Proxmox.NodeName}.`; // TODO: FQDN of node
+	}
+
+	me.callParent();
+
+	if (!me.isCreate) {
+	    let values = { ...me.domain };
+	    if (Ext.isDefined(values.usage)) {
+		values.usage = values.usage.split(';');
+	    }
+	    me.setValues(values);
+	} else {
+	    me.setValues({ domain: me.domain });
+	}
+    },
+});
-- 
2.20.1

Re: [pmg-devel] [RFC api/gui/wtk/acme 0/many] Certificates & ACME

[Dominik Csapak]

so i tested this a bit, and mostly works as advertised
(will look a bit deeper into the code today/tomorrow)

i tested with a local 'pebble' instance and for dns i
used a local powerdns instance

a few small problems (some we already discussed off-list):
* proxmox-acme-rs/client could use a 'Content-Length' header
   (necessary for pebble acme server, seems other acme endpoints are not
   as strict)
* the 'Add ACME Account' Button in the cert panel does not work
* cannot delete accounts that do not exist on the ACME server anymore
   (triggered this by restarting the pebble instance, it does not
   save anything persistent)
* for the selection of API/SMTP i would have rather expected
   checkboxes, it was not immediately clear that it is a multi-
   select combobox (though i think thomas nudged you in that
   direction?), but it's not too bad
* the dns plugin window behaves strangely:
   when editing a field on a plugin where we have the schema,
   the form does not get dirty, only when changing another field
   though it is entirely possible that this behaviour
   was already there

otherwise LGTM

On 3/9/21 3:13 PM, Wolfgang Bumiller wrote:
> These are the pmg-api, pmg-gui and proxmox-widget-toolkit and
> proxmox-acme parts of the ACME series for PMG.
> 
> This requires `pmg-rs` package, which replaces the ACME client from
> `proxmox-acme` and provides the CSR generation and is written in rust.
> Note that the DNS challenge handling still uses proxmox-acme for now.
> 
> proxmox-acme:
>    * Just a `use` statement fixup
>    * Still used for the DNS challenge
> 
> pmg-gui:
>    Just adds the "certificate view", but the real dirt lives in the
>    widget-toolkit.
> 
> proxmox-widget-toolkits:
>    Gets the Certificate, ACME Account, ACME Plugin and ACME Domain view
>    from PVE adapted to be usable for PMG.
>    Changes to PVE are mainly:
>      * API URLs need to be provided since they differ a bit between PVE
>        and PMG.
>      * some additional buttons/fields specific to pmg generated if the
>        parameters for them are present
> 
> pmg-api:
>    Simply gets API entry points for the above. These too are mostly
>    copied from PVE and adapted (also the ACME client API from pmg-rs is slightly
>    different/cleaned up, so that's a minor incompatiblity in some
>    otherwise common code, but a `pve-rs` may fix that). But some things
>    could definitely already go to pve-common (especially schema stuff).
> 
> Note that while I did add the corresponding files to the cluster sync,
> this still needs testing *and* issuing an API certificate may break
> cluster functionality currently. (Stoiko is working on that)
> 
> 
> _______________________________________________
> pmg-devel mailing list
> pmg-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pmg-devel
> 
> 

Re: [pmg-devel] [PATCH api 2/8] add PMG::CertHelpers module

[Dominik Csapak]

comments inline

On 3/9/21 3:13 PM, Wolfgang Bumiller wrote:
> Contains helpers to update certificates and provide locking
> for certificates and when accessing acme accounts.
> 
> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
> ---
>   src/Makefile           |   1 +
>   src/PMG/CertHelpers.pm | 180 +++++++++++++++++++++++++++++++++++++++++
>   2 files changed, 181 insertions(+)
>   create mode 100644 src/PMG/CertHelpers.pm
> 
> diff --git a/src/Makefile b/src/Makefile
> index 8891a3c..c1d4812 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -55,6 +55,7 @@ LIBSOURCES =				\
>   	PMG/HTMLMail.pm			\
>   	PMG/ModGroup.pm			\
>   	PMG/SMTPPrinter.pm		\
> +	PMG/CertHelpers.pm		\
>   	PMG/Config.pm			\
>   	PMG/Cluster.pm			\
>   	PMG/ClusterConfig.pm		\
> diff --git a/src/PMG/CertHelpers.pm b/src/PMG/CertHelpers.pm
> new file mode 100644
> index 0000000..2cf8a4e
> --- /dev/null
> +++ b/src/PMG/CertHelpers.pm
> @@ -0,0 +1,180 @@
> +package PMG::CertHelpers;
> +
> +use strict;
> +use warnings;
> +
> +use PVE::Certificate;
> +use PVE::JSONSchema;
> +use PVE::Tools;
> +
> +use constant {
> +    API_CERT => '/etc/pmg/pmg-api.pem',
> +    SMTP_CERT => '/etc/pmg/pmg-tls.pem',
> +};
> +
> +my $account_prefix = '/etc/pmg/acme';
> +
> +# TODO: Move `pve-acme-account-name` to common and reuse instead of this.
> +PVE::JSONSchema::register_standard_option('pmg-acme-account-name', {
> +    description => 'ACME account config file name.',
> +    type => 'string',
> +    format => 'pve-configid',
> +    format_description => 'name',
> +    optional => 1,
> +    default => 'default',
> +});
> +
> +PVE::JSONSchema::register_standard_option('pmg-acme-account-contact', {
> +    type => 'string',
> +    format => 'email-list',
> +    description => 'Contact email addresses.',
> +});
> +
> +PVE::JSONSchema::register_standard_option('pmg-acme-directory-url', {
> +    type => 'string',
> +    description => 'URL of ACME CA directory endpoint.',
> +    pattern => '^https?://.*',
> +});
> +
> +PVE::JSONSchema::register_format('pmg-certificate-type', sub {
> +    my ($type, $noerr) = @_;
> +
> +    if ($type =~ /^(?: api | smtp )$/x) {
> +	return $type;
> +    }
> +    return undef if $noerr;
> +    die "value '$type' does not look like a valid certificate type\n";
> +});
> +
> +PVE::JSONSchema::register_standard_option('pmg-certificate-type', {
> +    type => 'string',
> +    description => 'The TLS certificate type (API or SMTP certificate).',
> +    enum => ['api', 'smtp'],
> +});

i get why you did the format and the option (you need it once as a 
'-list') but would it not have been possible to reuse the format instead
of redefining the enum?

or only using the enum as variable defined somewhere?

feels weird to have a format + option that do basically
the same thing

> +
> +PVE::JSONSchema::register_format('pmg-acme-domain', sub {
> +    my ($domain, $noerr) = @_;
> +
> +    my $label = qr/[a-z0-9][a-z0-9_-]*/i;
> +
> +    return $domain if $domain =~ /^$label(?:\.$label)+$/;
> +    return undef if $noerr;
> +    die "value '$domain' does not look like a valid domain name!\n";
> +});
> +
> +PVE::JSONSchema::register_format('pmg-acme-alias', sub {
> +    my ($alias, $noerr) = @_;
> +
> +    my $label = qr/[a-z0-9_][a-z0-9_-]*/i;
> +
> +    return $alias if $alias =~ /^$label(?:\.$label)+$/;
> +    return undef if $noerr;
> +    die "value '$alias' does not look like a valid alias name!\n";
> +});

could we not reuse the '-domain' format here ?
i know the error message would be different then, but it is still a domain?

if not, we could refactor the regexes though

> +
> +my $local_cert_lock = '/var/lock/pmg-certs.lock';
> +my $local_acme_lock = '/var/lock/pmg-acme.lock';
> +
> +sub cert_path : prototype($) {
> +    my ($type) = @_;
> +    if ($type eq 'api') {
> +	return API_CERT;
> +    } elsif ($type eq 'smtp') {
> +	return SMTP_CERT;
> +    } else {
> +	die "unknown certificate type '$type'\n";
> +    }
> +}
> +
> +sub cert_lock {
> +    my ($timeout, $code, @param) = @_;
> +
> +    my $res = PVE::Tools::lock_file($local_cert_lock, $timeout, $code, @param);
> +    die $@ if $@;
> +    return $res;
> +}
> +
> +sub set_cert_file {
> +    my ($cert, $cert_path, $force) = @_;
> +
> +    my ($old_cert, $info);
> +
> +    my $cert_path_old = "${cert_path}.old";
> +
> +    die "Custom certificate file exists but force flag is not set.\n"
> +	if !$force && -e $cert_path;
> +
> +    PVE::Tools::file_copy($cert_path, $cert_path_old) if -e $cert_path;
> +
> +    eval {
> +	my $gid = undef;
> +	if ($cert_path eq &API_CERT) {
> +	    $gid = getgrnam('www-data') ||
> +		die "user www-data not in group file\n";
> +	}
> +
> +	if (defined($gid)) {
> +	    my $cert_path_tmp = "${cert_path}.tmp";
> +	    PVE::Tools::file_set_contents($cert_path_tmp, $cert, 0640);
> +	    if (!chown(-1, $gid, $cert_path_tmp)) {
> +		my $msg =
> +		    "failed to change group ownership of '$cert_path_tmp' to www-data ($gid): $!\n";
> +		unlink($cert_path_tmp);
> +		die $msg;
> +	    }
> +	    if (!rename($cert_path_tmp, $cert_path)) {
> +		my $msg =
> +		    "failed to rename '$cert_path_tmp' to '$cert_path': $!\n";
> +		unlink($cert_path_tmp);
> +		die $msg;
> +	    }
> +	} else {
> +	    PVE::Tools::file_set_contents($cert_path, $cert, 0600);
> +	}
> +
> +	$info = PVE::Certificate::get_certificate_info($cert_path);
> +    };
> +    my $err = $@;
> +
> +    if ($err) {
> +	if (-e $cert_path_old) {
> +	    eval {
> +		warn "Attempting to restore old certificate file..\n";
> +		PVE::Tools::file_copy($cert_path_old, $cert_path);
> +	    };
> +	    warn "$@\n" if $@;
> +	}
> +	die "Setting certificate files failed - $err\n"
> +    }
> +
> +    unlink $cert_path_old;
> +
> +    return $info;
> +}
> +
> +sub lock_acme {
> +    my ($account_name, $timeout, $code, @param) = @_;
> +
> +    my $file = "$local_acme_lock.$account_name";
> +
> +    return PVE::Tools::lock_file($file, $timeout, $code, @param);
> +}
> +

is there a special reason why you die $@ if $@ above in cert_lock
but not here?

afaics, you do it manually in the later patches always anyway

> +sub acme_account_dir {
> +    return $account_prefix;
> +}
> +
> +sub list_acme_accounts {
> +    my $accounts = [];
> +
> +    return $accounts if ! -d $account_prefix;
> +
> +    PVE::Tools::dir_glob_foreach($account_prefix, qr/[^.]+.*/, sub {
> +	my ($name) = @_;
> +
> +	push @$accounts, $name
> +	    if PVE::JSONSchema::pve_verify_configid($name, 1);
> +    });
> +
> +    return $accounts;
> +}
> 

Re: [pmg-devel] [PATCH api 5/8] api: add ACME and ACMEPlugin module

[Dominik Csapak]

comments inline

On 3/9/21 3:13 PM, Wolfgang Bumiller wrote:
> This adds the cluster-wide acme account and plugin
> configuration:
> 
>     * /config/acme
>     |`+ account/
>     | '- {name}
>     |`- tos
>     |`- directories
>     |`- challenge-schema
>      `+ plugins/
>       '- {name}
> 
> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
> ---
>   src/Makefile               |   2 +
>   src/PMG/API2/ACME.pm       | 436 +++++++++++++++++++++++++++++++++++++
>   src/PMG/API2/ACMEPlugin.pm | 270 +++++++++++++++++++++++
>   src/PMG/API2/Config.pm     |   7 +
>   4 files changed, 715 insertions(+)
>   create mode 100644 src/PMG/API2/ACME.pm
>   create mode 100644 src/PMG/API2/ACMEPlugin.pm
> 
> diff --git a/src/Makefile b/src/Makefile
> index ce76f9f..ebc6bd8 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -155,6 +155,8 @@ LIBSOURCES =				\
>   	PMG/API2/When.pm		\
>   	PMG/API2/What.pm		\
>   	PMG/API2/Action.pm		\
> +	PMG/API2/ACME.pm		\
> +	PMG/API2/ACMEPlugin.pm		\
>   	PMG/API2.pm			\
>   
>   SOURCES = ${LIBSOURCES} ${CLI_BINARIES} ${TEMPLATES_FILES} ${CONF_MANS} ${CLI_MANS} ${SERVICE_MANS} ${SERVICE_UNITS} ${TIMER_UNITS} pmg-sources.list pmg-apt.conf pmg-initramfs.conf
> diff --git a/src/PMG/API2/ACME.pm b/src/PMG/API2/ACME.pm
> new file mode 100644
> index 0000000..3b031fb
> --- /dev/null
> +++ b/src/PMG/API2/ACME.pm
> @@ -0,0 +1,436 @@
> +package PMG::API2::ACME;
> +
> +use strict;
> +use warnings;
> +
> +use PVE::Exception qw(raise_param_exc);
> +use PVE::JSONSchema qw(get_standard_option);
> +use PVE::Tools qw(extract_param);
> +
> +use PVE::ACME::Challenge;
> +
> +use PMG::RESTEnvironment;
> +use PMG::RS::Acme;
> +use PMG::CertHelpers;
> +
> +use PMG::API2::ACMEPlugin;
> +
> +use base qw(PVE::RESTHandler);
> +
> +__PACKAGE__->register_method ({
> +    subclass => "PMG::API2::ACMEPlugin",
> +    path => 'plugins',
> +});
> +
> +# FIXME: Put this list in pve-common or proxmox-acme{,-rs}?
> +my $acme_directories = [
> +    {
> +	name => 'Let\'s Encrypt V2',
> +	url => 'https://acme-v02.api.letsencrypt.org/directory',
> +    },
> +    {
> +	name => 'Let\'s Encrypt V2 Staging',
> +	url => 'https://acme-staging-v02.api.letsencrypt.org/directory',
> +    },
> +];
> +my $acme_default_directory_url = $acme_directories->[0]->{url};
> +my $account_contact_from_param = sub {
> +    my @addresses = PVE::Tools::split_list(extract_param($_[0], 'contact'));
> +    return [ map { "mailto:$_" } @addresses ];
> +};
> +my $acme_account_dir = PMG::CertHelpers::acme_account_dir();
> +
> +__PACKAGE__->register_method ({
> +    name => 'index',
> +    path => '',
> +    method => 'GET',
> +    permissions => { user => 'all' },
> +    description => "ACME index.",
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	},
> +    },
> +    returns => {
> +	type => 'array',
> +	items => {
> +	    type => "object",
> +	    properties => {},
> +	},
> +	links => [ { rel => 'child', href => "{name}" } ],
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	return [
> +	    { name => 'account' },
> +	    { name => 'tos' },
> +	    { name => 'directories' },
> +	    { name => 'plugins' },
> +	    { name => 'challengeschema' },
> +	];
> +    }});
> +
> +__PACKAGE__->register_method ({
> +    name => 'account_index',
> +    path => 'account',
> +    method => 'GET',
> +    permissions => { user => 'all' },

i'd argue that the qmanager should not list the
available acme accounts

> +    description => "ACME account index.",
> +    protected => 1,
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	},
> +    },
> +    returns => {
> +	type => 'array',
> +	items => {
> +	    type => "object",
> +	    properties => {},
> +	},
> +	links => [ { rel => 'child', href => "{name}" } ],
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $accounts = PMG::CertHelpers::list_acme_accounts();
> +	return [ map { { name => $_ }  } @$accounts ];
> +    }});

for the following create/update
the permissions are missing but should be 'admin'
(they are ok for the plugins)

> +
> +__PACKAGE__->register_method ({
> +    name => 'register_account',
> +    path => 'account',
> +    method => 'POST',
> +    description => "Register a new ACME account with CA.",
> +    proxyto => 'master',
> +    protected => 1,
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    name => get_standard_option('pmg-acme-account-name'),
> +	    contact => get_standard_option('pmg-acme-account-contact'),
> +	    tos_url => {
> +		type => 'string',
> +		description => 'URL of CA TermsOfService - setting this indicates agreement.',
> +		optional => 1,
> +	    },
> +	    directory => get_standard_option('pmg-acme-directory-url', {
> +		default => $acme_default_directory_url,
> +		optional => 1,
> +	    }),
> +	},
> +    },
> +    returns => {
> +	type => 'string',
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $rpcenv = PMG::RESTEnvironment->get();
> +	my $authuser = $rpcenv->get_user();
> +
> +	my $account_name = extract_param($param, 'name') // 'default';
> +	my $account_file = "${acme_account_dir}/${account_name}";
> +	mkdir $acme_account_dir if ! -e $acme_account_dir;
> +
> +	raise_param_exc({'name' => "ACME account config file '${account_name}' already exists."})
> +	    if -e $account_file;
> +
> +	my $directory = extract_param($param, 'directory') // $acme_default_directory_url;
> +	my $contact = $account_contact_from_param->($param);
> +
> +	my $realcmd = sub {
> +	    PMG::CertHelpers::lock_acme($account_name, 10, sub {
> +		die "ACME account config file '${account_name}' already exists.\n"
> +		    if -e $account_file;
> +
> +		print "Registering new ACME account..\n";
> +		my $acme = PMG::RS::Acme->new($directory);
> +		eval {
> +		    $acme->new_account($account_file, defined($param->{tos_url}), $contact, undef);
> +		};
> +		if (my $err = $@) {
> +		    unlink $account_file;
> +		    die "Registration failed: $err\n";
> +		}
> +		my $location = $acme->location();
> +		print "Registration successful, account URL: '$location'\n";
> +	    });
> +	    die $@ if $@;
> +	};
> +
> +	return $rpcenv->fork_worker('acmeregister', undef, $authuser, $realcmd);
> +    }});
> +
> +my $update_account = sub {
> +    my ($param, $msg, %info) = @_;
> +
> +    my $account_name = extract_param($param, 'name') // 'default';
> +    my $account_file = "${acme_account_dir}/${account_name}";
> +
> +    raise_param_exc({'name' => "ACME account config file '${account_name}' does not exist."})
> +	if ! -e $account_file;
> +
> +
> +    my $rpcenv = PMG::RESTEnvironment->get();
> +    my $authuser = $rpcenv->get_user();
> +
> +    my $realcmd = sub {
> +	PMG::CertHelpers::lock_acme($account_name, 10, sub {
> +	    die "ACME account config file '${account_name}' does not exist.\n"
> +		if ! -e $account_file;
> +
> +	    my $acme = PMG::RS::Acme->load($account_file);
> +	    $acme->update_account(\%info);
> +	    if ($info{status} && $info{status} eq 'deactivated') {
> +		my $deactivated_name;
> +		for my $i (0..100) {
> +		    my $candidate = "${acme_account_dir}/_deactivated_${account_name}_${i}";
> +		    if (! -e $candidate) {
> +			$deactivated_name = $candidate;
> +			last;
> +		    }
> +		}
> +		if ($deactivated_name) {
> +		    print "Renaming account file from '$account_file' to '$deactivated_name'\n";
> +		    rename($account_file, $deactivated_name) or
> +			warn ".. failed - $!\n";
> +		} else {
> +		    warn "No free slot to rename deactivated account file '$account_file', leaving in place\n";
> +		}
> +	    }
> +	});
> +	die $@ if $@;
> +    };
> +
> +    return $rpcenv->fork_worker("acme${msg}", undef, $authuser, $realcmd);
> +};
> +
> +__PACKAGE__->register_method ({
> +    name => 'update_account',
> +    path => 'account/{name}',
> +    method => 'PUT',
> +    description => "Update existing ACME account information with CA. Note: not specifying any new account information triggers a refresh.",
> +    proxyto => 'master',
> +    protected => 1,
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    name => get_standard_option('pmg-acme-account-name'),
> +	    contact => get_standard_option('pmg-acme-account-contact', {
> +		optional => 1,
> +	    }),
> +	},
> +    },
> +    returns => {
> +	type => 'string',
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $contact = $account_contact_from_param->($param);
> +	if (scalar @$contact) {
> +	    return $update_account->($param, 'update', contact => $contact);
> +	} else {
> +	    return $update_account->($param, 'refresh');
> +	}
> +    }});
> +
> +__PACKAGE__->register_method ({
> +    name => 'get_account',
> +    path => 'account/{name}',
> +    method => 'GET',
> +    description => "Return existing ACME account information.",
> +    protected => 1,
> +    proxyto => 'master',
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    name => get_standard_option('pmg-acme-account-name'),
> +	},
> +    },
> +    returns => {
> +	type => 'object',
> +	additionalProperties => 0,
> +	properties => {
> +	    account => {
> +		type => 'object',
> +		optional => 1,
> +		renderer => 'yaml',
> +	    },
> +	    directory => get_standard_option('pmg-acme-directory-url', {
> +		optional => 1,
> +	    }),
> +	    location => {
> +		type => 'string',
> +		optional => 1,
> +	    },
> +	    tos => {
> +		type => 'string',
> +		optional => 1,
> +	    },
> +	},
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $account_name = extract_param($param, 'name') // 'default';
> +	my $account_file = "${acme_account_dir}/${account_name}";
> +
> +	raise_param_exc({'name' => "ACME account config file '${account_name}' does not exist."})
> +	    if ! -e $account_file;
> +
> +	my $acme = PMG::RS::Acme->load($account_file);
> +	my $data = $acme->account();
> +
> +	return {
> +	    account => $data->{account},
> +	    tos => $data->{tos},
> +	    location => $data->{location},
> +	    directory => $data->{directoryUrl},
> +	};
> +    }});
> +
> +__PACKAGE__->register_method ({
> +    name => 'deactivate_account',
> +    path => 'account/{name}',
> +    method => 'DELETE',
> +    description => "Deactivate existing ACME account at CA.",
> +    protected => 1,
> +    proxyto => 'master',
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    name => get_standard_option('pmg-acme-account-name'),
> +	},
> +    },
> +    returns => {
> +	type => 'string',
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	return $update_account->($param, 'deactivate', status => 'deactivated');
> +    }});
> +
> +__PACKAGE__->register_method ({
> +    name => 'get_tos',
> +    path => 'tos',
> +    method => 'GET',
> +    description => "Retrieve ACME TermsOfService URL from CA.",
> +    permissions => { user => 'all' },
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    directory => get_standard_option('pmg-acme-directory-url', {
> +		default => $acme_default_directory_url,
> +		optional => 1,
> +	    }),
> +	},
> +    },
> +    returns => {
> +	type => 'string',
> +	optional => 1,
> +	description => 'ACME TermsOfService URL.',
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $directory = extract_param($param, 'directory') // $acme_default_directory_url;
> +
> +	my $acme = PMG::RS::Acme->new($directory);
> +	my $meta = $acme->get_meta();
> +
> +	return $meta ? $meta->{termsOfService} : undef;
> +    }});

just for my understanding: what happens here if there is no TOS?
is that valid ACME behaviour? or should we somehow error out?

> +
> +__PACKAGE__->register_method ({
> +    name => 'get_directories',
> +    path => 'directories',
> +    method => 'GET',
> +    description => "Get named known ACME directory endpoints.",
> +    permissions => { user => 'all' },
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {},
> +    },
> +    returns => {
> +	type => 'array',
> +	items => {
> +	    type => 'object',
> +	    additionalProperties => 0,
> +	    properties => {
> +		name => {
> +		    type => 'string',
> +		},
> +		url => get_standard_option('pmg-acme-directory-url'),
> +	    },
> +	},
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	return $acme_directories;
> +    }});
> +
> +__PACKAGE__->register_method ({
> +    name => 'challengeschema',
> +    path => 'challenge-schema',
> +    method => 'GET',
> +    description => "Get schema of ACME challenge types.",
> +    permissions => { user => 'all' },
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {},
> +    },
> +    returns => {
> +	type => 'array',
> +	items => {
> +	    type => 'object',
> +	    additionalProperties => 0,
> +	    properties => {
> +		id => {
> +		    type => 'string',
> +		},
> +		name => {
> +		    description => 'Human readable name, falls back to id',
> +		    type => 'string',
> +		},
> +		type => {
> +		    type => 'string',
> +		},
> +		schema => {
> +		    type => 'object',
> +		},
> +	    },
> +	},
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $plugin_type_enum = PVE::ACME::Challenge->lookup_types();
> +
> +	my $res = [];
> +
> +	for my $type (@$plugin_type_enum) {
> +	    my $plugin = PVE::ACME::Challenge->lookup($type);
> +	    next if !$plugin->can('get_supported_plugins');
> +
> +	    my $plugin_type = $plugin->type();
> +	    my $plugins = $plugin->get_supported_plugins();
> +	    for my $id (sort keys %$plugins) {
> +		my $schema = $plugins->{$id};
> +		push @$res, {
> +		    id => $id,
> +		    name => $schema->{name} // $id,
> +		    type => $plugin_type,
> +		    schema => $schema,
> +		};
> +	    }
> +	}
> +
> +	return $res;
> +    }});
> +
> +1;
> diff --git a/src/PMG/API2/ACMEPlugin.pm b/src/PMG/API2/ACMEPlugin.pm
> new file mode 100644
> index 0000000..38540b1
> --- /dev/null
> +++ b/src/PMG/API2/ACMEPlugin.pm
> @@ -0,0 +1,270 @@
> +package PMG::API2::ACMEPlugin;
> +
> +use strict;
> +use warnings;
> +
> +use Storable qw(dclone);
> +
> +use PVE::ACME::Challenge;
> +use PVE::ACME::DNSChallenge;
> +use PVE::ACME::StandAlone;
> +use PVE::INotify;
> +use PVE::JSONSchema qw(get_standard_option);
> +use PVE::Tools qw(extract_param);
> +
> +use base qw(PVE::RESTHandler);
> +
> +my $inotify_file_id = 'pmg-acme-plugins-config.conf';
> +my $config_filename = '/etc/pmg/acme-plugins.conf';
> +my $lockfile = "/var/lock/pmg-acme-plugins-config.lck";
> +
> +PVE::ACME::DNSChallenge->register();
> +PVE::ACME::StandAlone->register();
> +PVE::ACME::Challenge->init();
> +
> +PVE::JSONSchema::register_standard_option('pmg-acme-pluginid', {
> +    type => 'string',
> +    format => 'pve-configid',
> +    description => 'Unique identifier for ACME plugin instance.',
> +});
> +
> +sub read_pmg_acme_challenge_config {
> +    my ($filename, $fh) = @_;
> +    local $/ = undef; # slurp mode
> +    my $raw = defined($fh) ? <$fh> : '';
> +    return PVE::ACME::Challenge->parse_config($filename, $raw);
> +}
> +
> +sub write_pmg_acme_challenge_config {
> +    my ($filename, $fh, $cfg) = @_;
> +    my $raw = PVE::ACME::Challenge->write_config($filename, $cfg);
> +    PVE::Tools::safe_print($filename, $fh, $raw);
> +}
> +
> +PVE::INotify::register_file($inotify_file_id, $config_filename,
> +			    \&read_pmg_acme_challenge_config,
> +			    \&write_pmg_acme_challenge_config,
> +			    undef,
> +			    always_call_parser => 1);
> +
> +sub lock_config {
> +    my ($code) = @_;
> +    my $p = PVE::Tools::lock_file($lockfile, undef, $code);
> +    die $@ if $@;
> +    return $p;
> +}
> +
> +sub load_config {
> +    # auto-adds the standalone plugin if no config is there for backwards
> +    # compatibility, so ALWAYS call the cfs registered parser
> +    return PVE::INotify::read_file($inotify_file_id);
> +}
> +
> +sub write_config {
> +    my ($self) = @_;
> +    return PVE::INotify::write_file($inotify_file_id, $self);
> +}
> +
> +my $plugin_type_enum = PVE::ACME::Challenge->lookup_types();
> +
> +my $modify_cfg_for_api = sub {
> +    my ($cfg, $pluginid) = @_;
> +
> +    die "ACME plugin '$pluginid' not defined\n" if !defined($cfg->{ids}->{$pluginid});
> +
> +    my $plugin_cfg = dclone($cfg->{ids}->{$pluginid});
> +    $plugin_cfg->{plugin} = $pluginid;
> +    $plugin_cfg->{digest} = $cfg->{digest};
> +
> +    return $plugin_cfg;
> +};
> +
> +__PACKAGE__->register_method ({
> +    name => 'index',
> +    path => '',
> +    method => 'GET',
> +    permissions => { check => [ 'admin', 'audit' ] },
> +    description => "ACME plugin index.",
> +    protected => 1,
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    type => {
> +		description => "Only list ACME plugins of a specific type",
> +		type => 'string',
> +		enum => $plugin_type_enum,
> +		optional => 1,
> +	    },
> +	},
> +    },
> +    returns => {
> +	type => 'array',
> +	items => {
> +	    type => "object",
> +	    properties => {
> +		plugin => get_standard_option('pmg-acme-pluginid'),
> +	    },
> +	},
> +	links => [ { rel => 'child', href => "{plugin}" } ],
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $cfg = load_config();
> +
> +	my $res = [];
> +	foreach my $pluginid (keys %{$cfg->{ids}}) {
> +	    my $plugin_cfg = $modify_cfg_for_api->($cfg, $pluginid);
> +	    next if $param->{type} && $param->{type} ne $plugin_cfg->{type};
> +	    push @$res, $plugin_cfg;
> +	}
> +
> +	return $res;
> +    }
> +});
> +
> +__PACKAGE__->register_method({
> +    name => 'get_plugin_config',
> +    path => '{id}',
> +    method => 'GET',
> +    description => "Get ACME plugin configuration.",
> +    permissions => { check => [ 'admin', 'audit' ] },
> +    protected => 1,
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    id => get_standard_option('pmg-acme-pluginid'),
> +	},
> +    },
> +    returns => {
> +	type => 'object',
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $cfg = load_config();
> +	return $modify_cfg_for_api->($cfg, $param->{id});
> +    }
> +});
> +
> +__PACKAGE__->register_method({
> +    name => 'add_plugin',
> +    path => '',
> +    method => 'POST',
> +    description => "Add ACME plugin configuration.",
> +    permissions => { check => [ 'admin' ] },
> +    protected => 1,
> +    parameters => PVE::ACME::Challenge->createSchema(),
> +    returns => {
> +	type => "null"
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $id = extract_param($param, 'id');
> +	my $type = extract_param($param, 'type');
> +
> +	lock_config(sub {
> +	    my $cfg = load_config();
> +	    die "ACME plugin ID '$id' already exists\n" if defined($cfg->{ids}->{$id});
> +
> +	    my $plugin = PVE::ACME::Challenge->lookup($type);
> +	    my $opts = $plugin->check_config($id, $param, 1, 1);
> +
> +	    $cfg->{ids}->{$id} = $opts;
> +	    $cfg->{ids}->{$id}->{type} = $type;
> +
> +	    write_config($cfg);
> +	});
> +	die "$@" if $@;

you already die in lock_config if $@ is set.

> +
> +	return undef;
> +    }
> +});
> +
> +__PACKAGE__->register_method({
> +    name => 'update_plugin',
> +    path => '{id}',
> +    method => 'PUT',
> +    description => "Update ACME plugin configuration.",
> +    permissions => { check => [ 'admin' ] },
> +    protected => 1,
> +    parameters => PVE::ACME::Challenge->updateSchema(),
> +    returns => {
> +	type => "null"
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $id = extract_param($param, 'id');
> +	my $delete = extract_param($param, 'delete');
> +	my $digest = extract_param($param, 'digest');
> +
> +	lock_config(sub {
> +	    my $cfg = load_config();
> +	    PVE::Tools::assert_if_modified($cfg->{digest}, $digest);
> +	    my $plugin_cfg = $cfg->{ids}->{$id};
> +	    die "ACME plugin ID '$id' does not exist\n" if !$plugin_cfg;
> +
> +	    my $type = $plugin_cfg->{type};
> +	    my $plugin = PVE::ACME::Challenge->lookup($type);
> +
> +	    if (defined($delete)) {
> +		my $schema = $plugin->private();
> +		my $options = $schema->{options}->{$type};
> +		for my $k (PVE::Tools::split_list($delete)) {
> +		    my $d = $options->{$k} || die "no such option '$k'\n";
> +		    die "unable to delete required option '$k'\n" if !$d->{optional};
> +
> +		    delete $cfg->{ids}->{$id}->{$k};
> +		}
> +	    }
> +
> +	    my $opts = $plugin->check_config($id, $param, 0, 1);
> +	    for my $k (sort keys %$opts) {

not that it should make a difference, but why sort?

> +		$plugin_cfg->{$k} = $opts->{$k};
> +	    }
> +
> +	    write_config($cfg);
> +	});
> +	die "$@" if $@;

again

> +
> +	return undef;
> +    }
> +});
> +
> +__PACKAGE__->register_method({
> +    name => 'delete_plugin',
> +    path => '{id}',
> +    method => 'DELETE',
> +    description => "Delete ACME plugin configuration.",
> +    permissions => { check => [ 'admin' ] },
> +    protected => 1,
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    id => get_standard_option('pmg-acme-pluginid'),
> +	},
> +    },
> +    returns => {
> +	type => "null"
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $id = extract_param($param, 'id');
> +
> +	lock_config(sub {
> +	    my $cfg = load_config();
> +
> +	    delete $cfg->{ids}->{$id};
> +
> +	    write_config($cfg);
> +	});
> +	die "$@" if $@;

again

> +
> +	return undef;
> +    }
> +});
> +
> +1;
> diff --git a/src/PMG/API2/Config.pm b/src/PMG/API2/Config.pm
> index e11eb3f..c5697e1 100644
> --- a/src/PMG/API2/Config.pm
> +++ b/src/PMG/API2/Config.pm
> @@ -26,6 +26,7 @@ use PMG::API2::DestinationTLSPolicy;
>   use PMG::API2::DKIMSign;
>   use PMG::API2::SACustom;
>   use PMG::API2::PBS::Remote;
> +use PMG::API2::ACME;
>   
>   use base qw(PVE::RESTHandler);
>   
> @@ -99,6 +100,11 @@ __PACKAGE__->register_method ({
>       path => 'pbs',
>   });
>   
> +__PACKAGE__->register_method ({
> +    subclass => "PMG::API2::ACME",
> +    path => 'acme',
> +});
> +
>   __PACKAGE__->register_method ({
>       name => 'index',
>       path => '',
> @@ -138,6 +144,7 @@ __PACKAGE__->register_method ({
>   	push @$res, { section => 'tlspolicy' };
>   	push @$res, { section => 'dkim' };
>   	push @$res, { section => 'pbs' };
> +	push @$res, { section => 'acme' };
>   
>   	return $res;
>       }});
> 

Re: [pmg-devel] [PATCH api 6/8] add certificates api endpoint

[Dominik Csapak]

On 3/9/21 3:13 PM, Wolfgang Bumiller wrote:
> This adds /nodes/{nodename}/certificates endpoint
> containing:
> 
>    /custom/{type} - update smtp or api certificates manually
>    /acme/{type} - update via acme
> 
> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
> ---
>   src/Makefile                 |   1 +
>   src/PMG/API2/Certificates.pm | 690 +++++++++++++++++++++++++++++++++++
>   src/PMG/API2/Nodes.pm        |   7 +
>   3 files changed, 698 insertions(+)
>   create mode 100644 src/PMG/API2/Certificates.pm
> 
> diff --git a/src/Makefile b/src/Makefile
> index ebc6bd8..e0629b2 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -155,6 +155,7 @@ LIBSOURCES =				\
>   	PMG/API2/When.pm		\
>   	PMG/API2/What.pm		\
>   	PMG/API2/Action.pm		\
> +	PMG/API2/Certificates.pm	\
>   	PMG/API2/ACME.pm		\
>   	PMG/API2/ACMEPlugin.pm		\
>   	PMG/API2.pm			\
> diff --git a/src/PMG/API2/Certificates.pm b/src/PMG/API2/Certificates.pm
> new file mode 100644
> index 0000000..d196af6
> --- /dev/null
> +++ b/src/PMG/API2/Certificates.pm
> @@ -0,0 +1,690 @@
> +package PMG::API2::Certificates;
> +
> +use strict;
> +use warnings;
> +
> +use PVE::Certificate;
> +use PVE::Exception qw(raise raise_param_exc);
> +use PVE::JSONSchema qw(get_standard_option);
> +use PVE::Tools qw(extract_param file_get_contents file_set_contents);
> +
> +use PMG::CertHelpers;
> +use PMG::NodeConfig;
> +use PMG::RS::CSR;
> +
> +use PMG::API2::ACMEPlugin;
> +
> +use base qw(PVE::RESTHandler);
> +
> +my $acme_account_dir = PMG::CertHelpers::acme_account_dir();
> +
> +sub first_typed_pem_entry : prototype($$) {
> +    my ($label, $data) = @_;
> +
> +    if ($data =~ /^(-----BEGIN (?<label>\Q$label\E)-----\n.*?\n-----END \g{label}-----)$/ms) {

nit: isn't
$data =~ /^(-----BEGIN \Q$label\E-----\n.*?\n-----END \Q$label\E-----)$/ms

shorter and does the same?


> +	chomp(my $content = $1);

nit: why chomp? the regex does not allow trailing/whitespace newlines in 
$1 ?

> +	return $content;
> +    }
> +    return undef;
> +}
> +
> +sub pem_private_key : prototype($) {
> +    my ($data) = @_;
> +    return first_typed_pem_entry('PRIVATE KEY', $data);
> +}
> +
> +sub pem_certificate : prototype($) {
> +    my ($data) = @_;
> +    return first_typed_pem_entry('CERTIFICATE', $data);
> +}
> +
> +my sub restart_after_cert_update : prototype($) {
> +    my ($type) = @_;
> +
> +    if ($type eq 'api') {
> +	print "Restarting pmgproxy\n";
> +	PVE::Tools::run_command(['systemctl', 'reload-or-restart', 'pmgproxy']);
> +    }
> +};
> +
> +my sub update_cert : prototype($$$$) {
> +    my ($type, $cert_path, $certificate, $force) = @_;
> +    my $code = sub {
> +	print "Setting custom certificate file $cert_path\n";
> +	PMG::CertHelpers::set_cert_file($certificate, $cert_path, $force);
> +
> +	restart_after_cert_update($type);
> +    };
> +    PMG::CertHelpers::cert_lock(10, $code);
> +};
> +
> +__PACKAGE__->register_method ({
> +    name => 'index',
> +    path => '',
> +    method => 'GET',
> +    permissions => { user => 'all' },
> +    description => "Node index.",
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    node => get_standard_option('pve-node'),
> +	},
> +    },
> +    returns => {
> +	type => 'array',
> +	items => {
> +	    type => "object",
> +	    properties => {},
> +	},
> +	links => [ { rel => 'child', href => "{name}" } ],
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	return [
> +	    { name => 'acme' },
> +	    { name => 'custom' },
> +	    { name => 'info' },
> +	    { name => 'config' },
> +	];
> +    },
> +});
> +
> +__PACKAGE__->register_method ({
> +    name => 'info',
> +    path => 'info',
> +    method => 'GET',
> +    permissions => { user => 'all' },
> +    proxyto => 'node',
> +    protected => 1,
> +    description => "Get information about the node's certificates.",
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    node => get_standard_option('pve-node'),
> +	},
> +    },
> +    returns => {
> +	type => 'array',
> +	items => get_standard_option('pve-certificate-info'),
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $res = [];
> +	for my $path (&PMG::CertHelpers::API_CERT, &PMG::CertHelpers::SMTP_CERT) {
> +	    eval {
> +		my $info = PVE::Certificate::get_certificate_info($path);
> +		push @$res, $info if $info;
> +	    };
> +	}
> +	return $res;
> +    },
> +});
> +
> +__PACKAGE__->register_method ({
> +    name => 'custom_cert_index',
> +    path => 'custom',
> +    method => 'GET',
> +    permissions => { user => 'all' },
> +    description => "Certificate index.",
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    node => get_standard_option('pve-node'),
> +	},
> +    },
> +    returns => {
> +	type => 'array',
> +	items => {
> +	    type => "object",
> +	    properties => {},
> +	},
> +	links => [ { rel => 'child', href => "{type}" } ],
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	return [
> +	    { type => 'api' },
> +	    { type => 'smtp' },
> +	];
> +    },
> +});
> +
> +__PACKAGE__->register_method ({
> +    name => 'upload_custom_cert',
> +    path => 'custom/{type}',
> +    method => 'POST',
> +    permissions => { check => [ 'admin' ] },
> +    description => 'Upload or update custom certificate chain and key.',
> +    protected => 1,
> +    proxyto => 'node',
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    node => get_standard_option('pve-node'),
> +	    certificates => {
> +		type => 'string',
> +		format => 'pem-certificate-chain',
> +		description => 'PEM encoded certificate (chain).',
> +	    },
> +	    key => {
> +		type => 'string',
> +		description => 'PEM encoded private key.',
> +		format => 'pem-string',
> +		optional => 0,
> +	    },
> +	    type => get_standard_option('pmg-certificate-type'),
> +	    force => {
> +		type => 'boolean',
> +		description => 'Overwrite existing custom or ACME certificate files.',
> +		optional => 1,
> +		default => 0,
> +	    },
> +	    restart => {
> +		type => 'boolean',
> +		description => 'Restart services.',
> +		optional => 1,
> +		default => 0,
> +	    },
> +	},
> +    },
> +    returns => get_standard_option('pve-certificate-info'),
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $type = extract_param($param, 'type'); # also used to know which service to restart
> +	my $cert_path = PMG::CertHelpers::cert_path($type);
> +
> +	my $certs = extract_param($param, 'certificates');
> +	$certs = PVE::Certificate::strip_leading_text($certs);
> +
> +	my $key = extract_param($param, 'key');
> +	if ($key) {
> +	    $key = PVE::Certificate::strip_leading_text($key);
> +	    $certs = "$key\n$certs";
> +	} else {
> +	    my $private_key = pem_private_key($certs);
> +	    if (!defined($private_key)) {
> +		my $old = file_get_contents($cert_path);
> +		$private_key = pem_private_key($old);
> +		if (!defined($private_key)) {
> +		    raise_param_exc({
> +			'key' => "Attempted to upload custom certificate without (existing) key."
> +		    })
> +		}
> +
> +		# copy the old certificate's key:
> +		$certs = "$key\n$certs";
> +	    }
> +	}
> +
> +	my $info;
> +
> +	my $code = sub {
> +	    print "Setting custom certificate file $cert_path\n";
> +	    $info = PMG::CertHelpers::set_cert_file($certs, $cert_path, $param->{force});
> +
> +	    if ($type eq 'api' && $param->{restart}) {
> +		print "Restarting pmgproxy\n";
> +		PVE::Tools::run_command(['systemctl', 'reload-or-restart', 'pmgproxy']);

you could reuse the 'restart_after_cert_update' here, no?

> +	    }
> +	};
> +
> +	PMG::CertHelpers::cert_lock(10, $code);
> +	die "$@\n" if $@;
> +
> +	if ($type eq 'smtp') {
> +	    $code = sub {
> +		my $cfg = PMG::Config->new();
> +
> +		print "Rewriting postfix config\n";
> +		$cfg->set('mail', 'tls', 1);
> +		$cfg->rewrite_config_postfix();
> +
> +		if ($param->{restart}) {
> +		    print "Reloading postfix\n";
> +		    PMG::Utils::service_cmd('postfix', 'reload');

also, why couldn't that be handled there too? then we could
combine the two restart/reload calls?

> +		}
> +	    };
> +	    PMG::Config::lock_config($code, "failed to reload postfix");
> +	}
> +
> +	return $info;
> +    }});
> +
> +__PACKAGE__->register_method ({
> +    name => 'remove_custom_cert',
> +    path => 'custom/{type}',
> +    method => 'DELETE',
> +    permissions => { check => [ 'admin' ] },
> +    description => 'DELETE custom certificate chain and key.',
> +    protected => 1,
> +    proxyto => 'node',
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    node => get_standard_option('pve-node'),
> +	    type => get_standard_option('pmg-certificate-type'),
> +	    restart => {
> +		type => 'boolean',
> +		description => 'Restart pmgproxy.',
> +		optional => 1,
> +		default => 0,
> +	    },
> +	},
> +    },
> +    returns => {
> +	type => 'null',
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $type = extract_param($param, 'type');
> +	my $cert_path = PMG::CertHelpers::cert_path($type);
> +
> +	my $code = sub {
> +	    print "Deleting custom certificate files\n";
> +	    unlink $cert_path;
> +
> +	    if ($param->{restart}) {
> +		restart_after_cert_update($type);
> +	    }
> +	};
> +
> +	PMG::CertHelpers::cert_lock(10, $code);
> +	die "$@\n" if $@;
> +
> +	return undef;

don't we need to update the postfix config and reload if the type is 
smtp? or at least error out?

> +    }});
> +
> +__PACKAGE__->register_method ({
> +    name => 'acme_cert_index',
> +    path => 'acme',
> +    method => 'GET',
> +    permissions => { user => 'all' },
> +    description => "ACME Certificate index.",
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    node => get_standard_option('pve-node'),
> +	},
> +    },
> +    returns => {
> +	type => 'array',
> +	items => {
> +	    type => "object",
> +	    properties => {},
> +	},
> +	links => [ { rel => 'child', href => "{type}" } ],
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	return [
> +	    { type => 'api' },
> +	    { type => 'smtp' },
> +	];
> +    },
> +});
> +
> +my $order_certificate = sub {
> +    my ($acme, $acme_node_config) = @_;
> +
> +    my $plugins = PMG::API2::ACMEPlugin::load_config();
> +
> +    print "Placing ACME order\n";
> +    my ($order_url, $order) = $acme->new_order([ keys %{$acme_node_config->{domains}} ]);
> +    print "Order URL: $order_url\n";
> +    for my $auth_url (@{$order->{authorizations}}) {
> +	print "\nGetting authorization details from '$auth_url'\n";
> +	my $auth = $acme->get_authorization($auth_url);
> +
> +	# force lower case, like get_acme_conf does
> +	my $domain = lc($auth->{identifier}->{value});
> +	if ($auth->{status} eq 'valid') {
> +	    print "$domain is already validated!\n";
> +	} else {
> +	    print "The validation for $domain is pending!\n";
> +
> +	    my $domain_config = $acme_node_config->{domains}->{$domain};
> +	    die "no config for domain '$domain'\n" if !$domain_config;
> +
> +	    my $plugin_id = $domain_config->{plugin};
> +
> +	    my $plugin_cfg = $plugins->{ids}->{$plugin_id};
> +	    die "plugin '$plugin_id' for domain '$domain' not found!\n"
> +		if !$plugin_cfg;
> +
> +	    my $data = {
> +		plugin => $plugin_cfg,
> +		alias => $domain_config->{alias},
> +	    };
> +
> +	    my $plugin = PVE::ACME::Challenge->lookup($plugin_cfg->{type});
> +	    $plugin->setup($acme, $auth, $data);
> +
> +	    print "Triggering validation\n";
> +	    eval {
> +		die "no validation URL returned by plugin '$plugin_id' for domain '$domain'\n"
> +		    if !defined($data->{url});
> +
> +		$acme->request_challenge_validation($data->{url});
> +		print "Sleeping for 5 seconds\n";
> +		sleep 5;
> +		while (1) {
> +		    $auth = $acme->get_authorization($auth_url);
> +		    if ($auth->{status} eq 'pending') {
> +			print "Status is still 'pending', trying again in 10 seconds\n";
> +			sleep 10;
> +			next;
> +		    } elsif ($auth->{status} eq 'valid') {
> +			print "Status is 'valid', domain '$domain' OK!\n";
> +			last;
> +		    }
> +		    die "validating challenge '$auth_url' failed - status: $auth->{status}\n";
> +		}
> +	    };
> +	    my $err = $@;
> +	    eval { $plugin->teardown($acme, $auth, $data) };
> +	    warn "$@\n" if $@;
> +	    die $err if $err;
> +	}
> +    }
> +    print "\nAll domains validated!\n";
> +    print "\nCreating CSR\n";
> +    # Currently we only support dns entries, so extract those from the order:
> +    my $san = [
> +	map {
> +	    $_->{value}
> +	} grep {
> +	    $_->{type} eq 'dns'
> +	} $order->{identifiers}->@*
> +    ];
> +    die "DNS identifiers are required to generate a CSR.\n" if !scalar @$san;
> +    my ($csr_der, $key) = PMG::RS::CSR::generate_csr($san, {});
> +
> +    my $finalize_error_cnt = 0;
> +    print "Checking order status\n";
> +    while (1) {
> +	$order = $acme->get_order($order_url);
> +	if ($order->{status} eq 'pending') {
> +	    print "still pending, trying to finalize order\n";
> +	    # FIXME
> +	    # to be compatible with and without the order ready state we try to
> +	    # finalize even at the 'pending' state and give up after 5
> +	    # unsuccessful tries this can be removed when the letsencrypt api
> +	    # definitely has implemented the 'ready' state
> +	    eval {
> +		$acme->finalize_order($order->{finalize}, $csr_der);
> +	    };
> +	    if (my $err = $@) {
> +		die $err if $finalize_error_cnt >= 5;
> +
> +		$finalize_error_cnt++;
> +		warn $err;
> +	    }
> +	    sleep 5;
> +	    next;
> +	} elsif ($order->{status} eq 'ready') {
> +	    print "Order is ready, finalizing order\n";
> +	    $acme->finalize_order($order->{finalize}, $csr_der);
> +	    sleep 5;
> +	    next;
> +	} elsif ($order->{status} eq 'processing') {
> +	    print "still processing, trying again in 30 seconds\n";
> +	    sleep 30;
> +	    next;
> +	} elsif ($order->{status} eq 'valid') {
> +	    print "valid!\n";
> +	    last;
> +	}
> +	die "order status: $order->{status}\n";
> +    }
> +
> +    print "\nDownloading certificate\n";
> +    my $cert = $acme->get_certificate($order->{certificate});
> +
> +    return ($cert, $key);
> +};
> +
> +# Filter domains and raise an error if the list becomes empty.
> +my $filter_domains = sub {
> +    my ($acme_config, $type) = @_;
> +
> +    my $domains = $acme_config->{domains};
> +    foreach my $domain (keys %$domains) {
> +	my $entry = $domains->{$domain};
> +	if (!(grep { $_ eq $type } PVE::Tools::split_list($entry->{usage}))) {
> +	    delete $domains->{$domain};
> +	}
> +    }
> +
> +    if (!%$domains) {
> +	raise("No domains configured for type '$type'\n", 400);
> +    }
> +};
> +
> +__PACKAGE__->register_method ({
> +    name => 'new_acme_cert',
> +    path => 'acme/{type}',
> +    method => 'POST',
> +    permissions => { check => [ 'admin' ] },
> +    description => 'Order a new certificate from ACME-compatible CA.',
> +    protected => 1,
> +    proxyto => 'node',
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    node => get_standard_option('pve-node'),
> +	    type => get_standard_option('pmg-certificate-type'),
> +	    force => {
> +		type => 'boolean',
> +		description => 'Overwrite existing custom certificate.',
> +		optional => 1,
> +		default => 0,
> +	    },
> +	},
> +    },
> +    returns => {
> +	type => 'string',
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $type = extract_param($param, 'type'); # also used to know which service to restart
> +	my $cert_path = PMG::CertHelpers::cert_path($type);
> +	raise_param_exc({'force' => "Custom certificate exists but 'force' is not set."})
> +	    if !$param->{force} && -e $cert_path;
> +
> +	my $node_config = PMG::NodeConfig::load_config();
> +	my $acme_config = PMG::NodeConfig::get_acme_conf($node_config);
> +	raise("ACME domain list in configuration is missing!", 400)
> +	    if !$acme_config || !$acme_config->{domains}->%*;
> +
> +	$filter_domains->($acme_config, $type);
> +
> +	my $rpcenv = PMG::RESTEnvironment->get();
> +	my $authuser = $rpcenv->get_user();
> +
> +	my $realcmd = sub {
> +	    STDOUT->autoflush(1);
> +	    my $account = $acme_config->{account};
> +	    my $account_file = "${acme_account_dir}/${account}";
> +	    die "ACME account config file '$account' does not exist.\n"
> +		if ! -e $account_file;
> +
> +	    print "Loading ACME account details\n";
> +	    my $acme = PMG::RS::Acme->load($account_file);
> +
> +	    my ($cert, $key) = $order_certificate->($acme, $acme_config);
> +	    my $certificate = "$key\n$cert";
> +
> +	    update_cert($type, $cert_path, $certificate, $param->{force});
> +
> +	    if ($type eq 'smtp') {
> +		my $code = sub {
> +		    my $cfg = PMG::Config->new();
> +
> +		    print "Rewriting postfix config\n";
> +		    $cfg->set('mail', 'tls', 1);
> +		    if ($cfg->rewrite_config_postfix()) {
> +			print "Reloading postfix\n";
> +			PMG::Utils::service_cmd('postfix', 'reload');
> +		    }
> +		};
> +		PMG::Config::lock_config($code, "failed to reload postfix");
> +	    }
> +
> +	    die "$@\n" if $@;
> +	};
> +
> +	return $rpcenv->fork_worker("acmenewcert", undef, $authuser, $realcmd);
> +    }});
> +
> +__PACKAGE__->register_method ({
> +    name => 'renew_acme_cert',
> +    path => 'acme/{type}',
> +    method => 'PUT',
> +    permissions => { check => [ 'admin' ] },
> +    description => "Renew existing certificate from CA.",
> +    protected => 1,
> +    proxyto => 'node',
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    node => get_standard_option('pve-node'),
> +	    type => get_standard_option('pmg-certificate-type'),
> +	    force => {
> +		type => 'boolean',
> +		description => 'Force renewal even if expiry is more than 30 days away.',
> +		optional => 1,
> +		default => 0,
> +	    },
> +	},
> +    },
> +    returns => {
> +	type => 'string',
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $type = extract_param($param, 'type'); # also used to know which service to restart
> +	my $cert_path = PMG::CertHelpers::cert_path($type);
> +
> +	raise("No current (custom) certificate found, please order a new certificate!\n")
> +	    if ! -e $cert_path;
> +
> +	my $expires_soon = PVE::Certificate::check_expiry($cert_path, time() + 30*24*60*60);
> +	raise_param_exc({'force' => "Certificate does not expire within the next 30 days, and 'force' is not set."})
> +	    if !$expires_soon && !$param->{force};
> +
> +	my $node_config = PMG::NodeConfig::load_config();
> +	my $acme_config = PMG::NodeConfig::get_acme_conf($node_config);
> +	raise("ACME domain list in configuration is missing!", 400)
> +	    if !$acme_config || !$acme_config->{domains}->%*;
> +
> +	$filter_domains->($acme_config, $type);
> +
> +	my $rpcenv = PMG::RESTEnvironment->get();
> +	my $authuser = $rpcenv->get_user();
> +
> +	my $old_cert = PVE::Tools::file_get_contents($cert_path);
> +
> +	my $realcmd = sub {
> +	    STDOUT->autoflush(1);
> +	    my $account = $acme_config->{account};
> +	    my $account_file = "${acme_account_dir}/${account}";
> +	    die "ACME account config file '$account' does not exist.\n"
> +		if ! -e $account_file;
> +
> +	    print "Loading ACME account details\n";
> +	    my $acme = PMG::RS::Acme->load($account_file);
> +
> +	    my ($cert, $key) = $order_certificate->($acme, $acme_config);
> +	    my $certificate = "$key\n$cert";
> +
> +	    update_cert($type, $cert_path, $certificate, 1);
> +
> +	    if (defined($old_cert)) {
> +		print "Revoking old certificate\n";
> +		eval { $acme->revoke_certificate($old_cert, undef) };
> +		warn "Revoke request to CA failed: $@" if $@;
> +	    }
> +	};
> +
> +	return $rpcenv->fork_worker("acmerenew", undef, $authuser, $realcmd);
> +    }});
> +
> +__PACKAGE__->register_method ({
> +    name => 'revoke_acme_cert',
> +    path => 'acme/{type}',
> +    method => 'DELETE',
> +    permissions => { check => [ 'admin' ] },
> +    description => "Revoke existing certificate from CA.",
> +    protected => 1,
> +    proxyto => 'node',
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    node => get_standard_option('pve-node'),
> +	    type => get_standard_option('pmg-certificate-type'),
> +	},
> +    },
> +    returns => {
> +	type => 'string',
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $type = extract_param($param, 'type'); # also used to know which service to restart
> +	my $cert_path = PMG::CertHelpers::cert_path($type);
> +
> +	my $node_config = PMG::NodeConfig::load_config();
> +	my $acme_config = PMG::NodeConfig::get_acme_conf($node_config);
> +	raise("ACME domain list in configuration is missing!", 400)
> +	    if !$acme_config || !$acme_config->{domains}->%*;
> +
> +	$filter_domains->($acme_config, $type);
> +
> +	my $rpcenv = PMG::RESTEnvironment->get();
> +	my $authuser = $rpcenv->get_user();
> +
> +	my $cert = PVE::Tools::file_get_contents($cert_path);
> +	$cert = pem_certificate($cert)
> +	    or die "no certificate section found in '$cert_path'\n";
> +
> +	my $realcmd = sub {
> +	    STDOUT->autoflush(1);
> +	    my $account = $acme_config->{account};
> +	    my $account_file = "${acme_account_dir}/${account}";
> +	    die "ACME account config file '$account' does not exist.\n"
> +		if ! -e $account_file;
> +
> +	    print "Loading ACME account details\n";
> +	    my $acme = PMG::RS::Acme->load($account_file);
> +
> +	    print "Revoking old certificate\n";
> +	    eval { $acme->revoke_certificate($cert, undef) };
> +	    if (my $err = $@) {
> +		# is there a better check?
> +		die "Revoke request to CA failed: $err" if $err !~ /"Certificate is expired"/;
> +	    }
> +
> +	    my $code = sub {
> +		print "Deleting certificate files\n";
> +		unlink $cert_path;
> +
> +		# FIXME: Regenerate self-signed `api` certificate.
> +		restart_after_cert_update($type);
> +	    };
> +
> +	    PMG::CertHelpers::cert_lock(10, $code);
> +	    die "$@\n" if $@;
> +	};
> +
> +	return $rpcenv->fork_worker("acmerevoke", undef, $authuser, $realcmd);
> +    }});

what happens here with postfix?

> +
> +1;
> diff --git a/src/PMG/API2/Nodes.pm b/src/PMG/API2/Nodes.pm
> index c0f5963..b6f0cd5 100644
> --- a/src/PMG/API2/Nodes.pm
> +++ b/src/PMG/API2/Nodes.pm
> @@ -27,6 +27,7 @@ use PMG::API2::Postfix;
>   use PMG::API2::MailTracker;
>   use PMG::API2::Backup;
>   use PMG::API2::PBS::Job;
> +use PMG::API2::Certificates;
>   
>   use base qw(PVE::RESTHandler);
>   
> @@ -85,6 +86,11 @@ __PACKAGE__->register_method ({
>       path => 'pbs',
>   });
>   
> +__PACKAGE__->register_method ({
> +    subclass => "PMG::API2::Certificates",
> +    path => 'certificates',
> +});
> +
>   __PACKAGE__->register_method ({
>       name => 'index',
>       path => '',
> @@ -126,6 +132,7 @@ __PACKAGE__->register_method ({
>   	    { name => 'subscription' },
>   	    { name => 'termproxy' },
>   	    { name => 'rrddata' },
> +	    { name => 'certificates' },
>   	];
>   
>   	return $result;
> 

Re: [pmg-devel] [PATCH gui] add certificates and acme view

[Dominik Csapak]

comments inline

On 3/9/21 3:13 PM, Wolfgang Bumiller wrote:
> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
> ---
>   js/Certificates.js   | 108 +++++++++++++++++++++++++++++++++++++++++++
>   js/Makefile          |   1 +
>   js/NavigationTree.js |   6 +++
>   3 files changed, 115 insertions(+)
>   create mode 100644 js/Certificates.js
> 
> diff --git a/js/Certificates.js b/js/Certificates.js
> new file mode 100644
> index 0000000..33b1bde
> --- /dev/null
> +++ b/js/Certificates.js
> @@ -0,0 +1,108 @@
> +Ext.define('PMG.CertificateConfiguration', {
> +    extend: 'Ext.tab.Panel',
> +    alias: 'widget.pmgCertificateConfiguration',
> +
> +    title: gettext('Certificates'),
> +
> +    border: false,
> +    defaults: { border: false },
> +
> +    items: [
> +	{
> +	    itemId: 'certificates',
> +	    xtype: 'pmgCertificatesView',
> +	    border: 0,
> +	},
> +	{
> +	    itemId: 'acme',
> +	    xtype: 'pmgACMEConfigView',
> +	    border: 0,
> +	},
> +    ],
> +});

why the defaults: border: false and the border: 0?
one should be enough

> +
> +Ext.define('PMG.CertificateView', {
> +    extend: 'Ext.container.Container',
> +    alias: 'widget.pmgCertificatesView',
> +
> +    title: gettext('Certificates'),
> +
> +    //onlineHelp: 'sysadmin_certificate_management',

seems to be leftover (we should probably have the docs here too?)

> +
> +    initComponent: function() {
> +	let me = this;
> +
> +	Ext.apply(me, {
> +	    items: [
> +		{
> +		    xtype: 'pmxCertificates',
> +		    border: 0,
> +		    infoUrl: '/nodes/' + Proxmox.NodeName + '/certificates/info',
> +		    uploadButtons: [
> +			{
> +			    name: 'API',
> +			    id: 'pmg-api.pem',
> +			    url: `/nodes/${Proxmox.NodeName}/certificates/custom/api`,
> +			    deletable: false,
> +			    reloadUi: true,
> +			},
> +			{
> +			    name: 'SMTP',
> +			    id: 'pmg-tls.pem',
> +			    url: `/nodes/${Proxmox.NodeName}/certificates/custom/smtp`,
> +			    deletable: true,
> +			},
> +		    ],
> +		},
> +		{
> +		    xtype: 'pmxACMEDomains',
> +		    border: 0,
> +		    url: `/nodes/${Proxmox.NodeName}/config`,
> +		    nodename: Proxmox.NodeName,
> +		    acmeUrl: '/config/acme',
> +		    domainUsages: [
> +			{
> +			    usage: 'api',
> +			    name: 'API',
> +			    url: `/nodes/${Proxmox.NodeName}/certificates/acme/api`,
> +			    reloadUi: true,
> +			},
> +			{
> +			    usage: 'smtp',
> +			    name: 'SMTP',
> +			    url: `/nodes/${Proxmox.NodeName}/certificates/acme/smtp`,
> +			},
> +		    ],
> +		},
> +	    ],
> +	});
> +
> +	me.callParent();
> +    },
> +});

this could be written without initComponent, nothing there depends on a
local variable (Proxmox.NodeName is set by the page and always available)

> +
> +Ext.define('PMG.ACMEConfigView', {
> +    extend: 'Ext.panel.Panel',
> +    alias: 'widget.pmgACMEConfigView',
> +
> +    title: gettext('ACME Accounts'),
> +
> +    //onlineHelp: 'sysadmin_certificate_management',
> +
> +    items: [
> +	{
> +	    region: 'north',
> +	    border: false,
> +	    xtype: 'pmxACMEAccounts',
> +	    acmeUrl: '/config/acme',
> +	},
> +	{
> +	    region: 'center',
> +	    border: false,
> +	    xtype: 'pmxACMEPluginView',
> +	    acmeUrl: '/config/acme',
> +	},
> +    ],
> +});
> +
> +
> diff --git a/js/Makefile b/js/Makefile
> index a5266fc..43d3ad8 100644
> --- a/js/Makefile
> +++ b/js/Makefile
> @@ -91,6 +91,7 @@ JSSRC=							\
>   	ContactStatistics.js				\
>   	HourlyMailDistribution.js			\
>   	SpamContextMenu.js				\
> +	Certificates.js					\
>   	Application.js
>   
>   OnlineHelpInfo.js: /usr/bin/asciidoc-pmg
> diff --git a/js/NavigationTree.js b/js/NavigationTree.js
> index ac01fd6..63f8e94 100644
> --- a/js/NavigationTree.js
> +++ b/js/NavigationTree.js
> @@ -92,6 +92,12 @@ Ext.define('PMG.store.NavigationStore', {
>   			path: 'pmgBackupConfiguration',
>   			leaf: true,
>   		    },
> +		    {
> +			text: gettext('Certificates'),
> +			iconCls: 'fa fa-certificate',
> +			path: 'pmgCertificateConfiguration',
> +			leaf: true,
> +		    },
>   		],
>   	    },
>   	    {
> 

Re: [pmg-devel] [PATCH widget-toolkit 2/7] add ACME related data models

[Dominik Csapak]

comments inline

On 3/9/21 3:13 PM, Wolfgang Bumiller wrote:
> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
> ---
>   src/Makefile                   |  2 ++
>   src/data/model/ACME.js         | 30 ++++++++++++++++++++++++++++++
>   src/data/model/Certificates.js |  6 ++++++
>   3 files changed, 38 insertions(+)
>   create mode 100644 src/data/model/ACME.js
>   create mode 100644 src/data/model/Certificates.js
> 
> diff --git a/src/Makefile b/src/Makefile
> index 46b90ae..3861bfc 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -15,6 +15,8 @@ JSSRC=					\
>   	data/RRDStore.js		\
>   	data/TimezoneStore.js		\
>   	data/model/Realm.js		\
> +	data/model/Certificates.js	\
> +	data/model/ACME.js		\
>   	form/DisplayEdit.js		\
>   	form/ExpireDate.js		\
>   	form/IntegerField.js		\
> diff --git a/src/data/model/ACME.js b/src/data/model/ACME.js
> new file mode 100644
> index 0000000..c05572e
> --- /dev/null
> +++ b/src/data/model/ACME.js
> @@ -0,0 +1,30 @@
> +Ext.define('proxmox-acme-accounts', {
> +    extend: 'Ext.data.Model',
> +    fields: ['name'],
> +    proxy: {
> +	type: 'proxmox',
> +	//url: "/api2/json/cluster/acme/account",

i'd prefer not to have the commented out api paths here

> +    },
> +    idProperty: 'name',
> +});
> +
> +Ext.define('proxmox-acme-challenges', {
> +    extend: 'Ext.data.Model',
> +    fields: ['id', 'type', 'schema'],
> +    proxy: {
> +	type: 'proxmox',
> +        //url: "/api2/json/cluster/acme/challenge-schema",

same

> +    },
> +    idProperty: 'id',
> +});
> +
> +
> +Ext.define('proxmox-acme-plugins', {
> +    extend: 'Ext.data.Model',
> +    fields: ['type', 'plugin', 'api'],
> +    proxy: {
> +	type: 'proxmox',
> +	//url: "/api2/json/cluster/acme/plugins",

same


maybe we could have /config/acme also for pbs, then we could
add that here and overwrite it in pve?

> +    },
> +    idProperty: 'plugin',
> +});
> diff --git a/src/data/model/Certificates.js b/src/data/model/Certificates.js
> new file mode 100644
> index 0000000..f3e2a7f
> --- /dev/null
> +++ b/src/data/model/Certificates.js
> @@ -0,0 +1,6 @@
> +Ext.define('proxmox-certificate', {
> +    extend: 'Ext.data.Model',
> +
> +    fields: ['filename', 'fingerprint', 'issuer', 'notafter', 'notbefore', 'subject', 'san', 'public-key-bits', 'public-key-type'],
> +    idProperty: 'filename',
> +});
> 

Re: [pmg-devel] [PATCH widget-toolkit 5/7] add ACME account panel

[Dominik Csapak]

high level question/remark:

would it not be nicer if we set the acmeUrl
in Proxmox.Utils to e.g., /config/acme
and overwrite that in pve ? (like
we do with task descriptions?)

this way the caller does not have to concern
itself with the url and we only set it one time per product

On 3/9/21 3:13 PM, Wolfgang Bumiller wrote:
> Copied from PVE with URLs now being based on the 'acmeUrl'
> property which should point to the acme/ root containing
> /tos, /directories, etc.
> 
> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
> ---
>   src/Makefile              |   2 +
>   src/panel/ACMEAccount.js  | 116 ++++++++++++++++++++++
>   src/window/ACMEAccount.js | 204 ++++++++++++++++++++++++++++++++++++++
>   3 files changed, 322 insertions(+)
>   create mode 100644 src/panel/ACMEAccount.js
>   create mode 100644 src/window/ACMEAccount.js
> 
> diff --git a/src/Makefile b/src/Makefile
> index d782e92..00a25c7 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -50,6 +50,7 @@ JSSRC=					\
>   	panel/RRDChart.js		\
>   	panel/GaugeWidget.js		\
>   	panel/Certificates.js		\
> +	panel/ACMEAccount.js		\
>   	window/Edit.js			\
>   	window/PasswordEdit.js		\
>   	window/SafeDestroy.js		\
> @@ -58,6 +59,7 @@ JSSRC=					\
>   	window/DiskSmart.js		\
>   	window/ZFSDetail.js		\
>   	window/Certificates.js		\
> +	window/ACMEAccount.js		\
>   	node/APT.js			\
>   	node/NetworkEdit.js		\
>   	node/NetworkView.js		\
> diff --git a/src/panel/ACMEAccount.js b/src/panel/ACMEAccount.js
> new file mode 100644
> index 0000000..c7d329e
> --- /dev/null
> +++ b/src/panel/ACMEAccount.js
> @@ -0,0 +1,116 @@
> +Ext.define('Proxmox.panel.ACMEAccounts', {
> +    extend: 'Ext.grid.Panel',
> +    xtype: 'pmxACMEAccounts',
> +
> +    title: gettext('Accounts'),
> +
> +    acmeUrl: undefined,
> +
> +    controller: {
> +	xclass: 'Ext.app.ViewController',
> +
> +	addAccount: function() {
> +	    let me = this;
> +	    let view = me.getView();
> +	    let defaultExists = view.getStore().findExact('name', 'default') !== -1;
> +	    Ext.create('Proxmox.window.ACMEAccountCreate', {
> +		defaultExists,
> +		acmeUrl: view.acmeUrl,
> +		taskDone: function() {
> +		    me.reload();
> +		},
> +	    }).show();
> +	},
> +
> +	viewAccount: function() {
> +	    let me = this;
> +	    let view = me.getView();
> +	    let selection = view.getSelection();
> +	    if (selection.length < 1) return;
> +	    Ext.create('Proxmox.window.ACMEAccountView', {
> +	        url: `${view.acmeUrl}/account/${selection[0].data.name}`,
> +	    }).show();
> +	},
> +
> +	reload: function() {
> +	    let me = this;
> +	    let view = me.getView();
> +	    view.getStore().rstore.load();
> +	},
> +
> +	showTaskAndReload: function(options, success, response) {
> +	    let me = this;
> +	    if (!success) return;
> +
> +	    let upid = response.result.data;
> +	    Ext.create('Proxmox.window.TaskProgress', {
> +		upid,
> +		taskDone: function() {
> +		    me.reload();
> +		},
> +	    }).show();
> +	},
> +    },
> +
> +    minHeight: 150,
> +    emptyText: gettext('No Accounts configured'),
> +
> +    columns: [
> +	{
> +	    dataIndex: 'name',
> +	    text: gettext('Name'),
> +	    renderer: Ext.String.htmlEncode,
> +	    flex: 1,
> +	},
> +    ],
> +
> +    listeners: {
> +	itemdblclick: 'viewAccount',
> +    },
> +
> +    store: {
> +	type: 'diff',
> +	autoDestroy: true,
> +	autoDestroyRstore: true,
> +	rstore: {
> +	    type: 'update',
> +	    storeid: 'proxmox-acme-accounts',
> +	    model: 'proxmox-acme-accounts',
> +	    autoStart: true,
> +	},
> +	sorters: 'name',
> +    },
> +
> +    initComponent: function() {
> +	let me = this;
> +
> +	if (!me.acmeUrl) {
> +	    throw "no acmeUrl given";
> +	}
> +
> +	Ext.apply(me, {
> +	    tbar: [
> +		{
> +		    xtype: 'proxmoxButton',
> +		    text: gettext('Add'),
> +		    selModel: false,
> +		    handler: 'addAccount',
> +		},
> +		{
> +		    xtype: 'proxmoxButton',
> +		    text: gettext('View'),
> +		    handler: 'viewAccount',
> +		    disabled: true,
> +		},
> +		{
> +		    xtype: 'proxmoxStdRemoveButton',
> +		    baseurl: `${me.acmeUrl}/account`,
> +		    callback: 'showTaskAndReload',
> +		},
> +	    ],
> +	});
> +
> +	me.callParent();
> +	me.store.rstore.proxy.setUrl(`/api2/json/${me.acmeUrl}/account`);
> +    },
> +});
> diff --git a/src/window/ACMEAccount.js b/src/window/ACMEAccount.js
> new file mode 100644
> index 0000000..05278a8
> --- /dev/null
> +++ b/src/window/ACMEAccount.js
> @@ -0,0 +1,204 @@
> +Ext.define('Proxmox.window.ACMEAccountCreate', {
> +    extend: 'Proxmox.window.Edit',
> +    mixins: ['Proxmox.Mixin.CBind'],
> +    xtype: 'pmxACMEAccountCreate',
> +
> +    acmeUrl: undefined,
> +
> +    width: 450,
> +    title: gettext('Register Account'),
> +    isCreate: true,
> +    method: 'POST',
> +    submitText: gettext('Register'),
> +    showTaskViewer: true,
> +    defaultExists: false,
> +
> +    items: [
> +	{
> +	    xtype: 'proxmoxtextfield',
> +	    fieldLabel: gettext('Account Name'),
> +	    name: 'name',
> +	    cbind: {
> +		emptyText: (get) => get('defaultExists') ? '' : 'default',
> +		allowBlank: (get) => !get('defaultExists'),
> +	    },
> +	},
> +	{
> +	    xtype: 'textfield',
> +	    name: 'contact',
> +	    vtype: 'email',
> +	    allowBlank: false,
> +	    fieldLabel: gettext('E-Mail'),
> +	},
> +	{
> +	    xtype: 'proxmoxComboGrid',
> +	    name: 'directory',
> +	    reference: 'directory',
> +	    allowBlank: false,
> +	    valueField: 'url',
> +	    displayField: 'name',
> +	    fieldLabel: gettext('ACME Directory'),
> +	    store: {
> +		autoLoad: true,
> +		fields: ['name', 'url'],
> +		idProperty: ['name'],
> +		proxy: { type: 'proxmox' },
> +		sorters: {
> +		    property: 'name',
> +		    order: 'ASC',
> +		},
> +	    },
> +	    listConfig: {
> +		columns: [
> +		    {
> +			header: gettext('Name'),
> +			dataIndex: 'name',
> +			flex: 1,
> +		    },
> +		    {
> +			header: gettext('URL'),
> +			dataIndex: 'url',
> +			flex: 1,
> +		    },
> +		],
> +	    },
> +	    listeners: {
> +		change: function(combogrid, value) {
> +		    let me = this;
> +
> +		    if (!value) {
> +			return;
> +		    }
> +
> +		    let acmeUrl = me.up('window').acmeUrl;
> +
> +		    let disp = me.up('window').down('#tos_url_display');
> +		    let field = me.up('window').down('#tos_url');
> +		    let checkbox = me.up('window').down('#tos_checkbox');
> +
> +		    disp.setValue(gettext('Loading'));
> +		    field.setValue(undefined);
> +		    checkbox.setValue(undefined);
> +		    checkbox.setHidden(true);
> +
> +		    Proxmox.Utils.API2Request({
> +			url: `${acmeUrl}/tos`,
> +			method: 'GET',
> +			params: {
> +			    directory: value,
> +			},
> +			success: function(response, opt) {
> +			    field.setValue(response.result.data);
> +			    disp.setValue(response.result.data);
> +			    checkbox.setHidden(false);
> +			},
> +			failure: function(response, opt) {
> +			    Ext.Msg.alert(gettext('Error'), response.htmlStatus);
> +			},
> +		    });
> +		},
> +	    },
> +	},
> +	{
> +	    xtype: 'displayfield',
> +	    itemId: 'tos_url_display',
> +	    renderer: Proxmox.Utils.render_optional_url,
> +	    name: 'tos_url_display',
> +	},
> +	{
> +	    xtype: 'hidden',
> +	    itemId: 'tos_url',
> +	    name: 'tos_url',
> +	},
> +	{
> +	    xtype: 'proxmoxcheckbox',
> +	    itemId: 'tos_checkbox',
> +	    boxLabel: gettext('Accept TOS'),
> +	    submitValue: false,
> +	    validateValue: function(value) {
> +		if (value && this.checked) {
> +		    return true;
> +		}
> +		return false;
> +	    },
> +	},
> +    ],
> +
> +    initComponent: function() {
> +	let me = this;
> +
> +	if (!me.acmeUrl) {
> +	    throw "no acmeUrl given";
> +	}
> +
> +	me.url = `${me.acmeUrl}/account`;
> +
> +	me.callParent();
> +
> +	me.lookup('directory')
> +	    .store
> +	    .proxy
> +	    .setUrl(`/api2/json/${me.acmeUrl}/directories`);
> +    },
> +});
> +
> +Ext.define('Proxmox.window.ACMEAccountView', {
> +    extend: 'Proxmox.window.Edit',
> +    xtype: 'pmxACMEAccountView',
> +
> +    width: 600,
> +    fieldDefaults: {
> +	labelWidth: 140,
> +    },
> +
> +    title: gettext('Account'),
> +
> +    items: [
> +	{
> +	    xtype: 'displayfield',
> +	    fieldLabel: gettext('E-Mail'),
> +	    name: 'email',
> +	},
> +	{
> +	    xtype: 'displayfield',
> +	    fieldLabel: gettext('Created'),
> +	    name: 'createdAt',
> +	},
> +	{
> +	    xtype: 'displayfield',
> +	    fieldLabel: gettext('Status'),
> +	    name: 'status',
> +	},
> +	{
> +	    xtype: 'displayfield',
> +	    fieldLabel: gettext('Directory'),
> +	    renderer: Proxmox.Utils.render_optional_url,
> +	    name: 'directory',
> +	},
> +	{
> +	    xtype: 'displayfield',
> +	    fieldLabel: gettext('Terms of Services'),
> +	    renderer: Proxmox.Utils.render_optional_url,
> +	    name: 'tos',
> +	},
> +    ],
> +
> +    initComponent: function() {
> +	var me = this;
> +
> +	me.callParent();
> +
> +	// hide OK/Reset button, because we just want to show data
> +	me.down('toolbar[dock=bottom]').setVisible(false);
> +
> +	me.load({
> +	    success: function(response) {
> +		var data = response.result.data;
> +		data.email = data.account.contact[0];
> +		data.createdAt = data.account.createdAt;
> +		data.status = data.account.status;
> +		me.setValues(data);
> +	    },
> +	});
> +    },
> +});
> 

Re: [pmg-devel] [PATCH widget-toolkit 5/7] add ACME account panel

[Thomas Lamprecht]

On 11.03.21 14:51, Dominik Csapak wrote:
> high level question/remark:
> 
> would it not be nicer if we set the acmeUrl
> in Proxmox.Utils to e.g., /config/acme
> and overwrite that in pve ? (like
> we do with task descriptions?)
> 
> this way the caller does not have to concern
> itself with the url and we only set it one time per product
> 
I like the idea in general, but the Utils class are quite overused/crowded
already, and this is rather not a Util but a config, maybe a Proxmox.Config
class could be added and used?

Re: [pmg-devel] [PATCH widget-toolkit 5/7] add ACME account panel

[Dominik Csapak]

On 3/11/21 4:14 PM, Thomas Lamprecht wrote:
> On 11.03.21 14:51, Dominik Csapak wrote:
>> high level question/remark:
>>
>> would it not be nicer if we set the acmeUrl
>> in Proxmox.Utils to e.g., /config/acme
>> and overwrite that in pve ? (like
>> we do with task descriptions?)
>>
>> this way the caller does not have to concern
>> itself with the url and we only set it one time per product
>>
> I like the idea in general, but the Utils class are quite overused/crowded
> already, and this is rather not a Util but a config, maybe a Proxmox.Config
> class could be added and used?
> 
yeah, or maybe we could have a general 'Proxmox.APIPaths' class that 
contains the paths for wt relevant stuff? so that we do not have
to design the api of new products in such a way that all products
are the same (e.g. /nodes/YYY/ stuff)

Re: [pmg-devel] [PATCH widget-toolkit 5/7] add ACME account panel

[Thomas Lamprecht]

On 11.03.21 16:16, Dominik Csapak wrote:
> 
> 
> On 3/11/21 4:14 PM, Thomas Lamprecht wrote:
>> On 11.03.21 14:51, Dominik Csapak wrote:
>>> high level question/remark:
>>>
>>> would it not be nicer if we set the acmeUrl
>>> in Proxmox.Utils to e.g., /config/acme
>>> and overwrite that in pve ? (like
>>> we do with task descriptions?)
>>>
>>> this way the caller does not have to concern
>>> itself with the url and we only set it one time per product
>>>
>> I like the idea in general, but the Utils class are quite overused/crowded
>> already, and this is rather not a Util but a config, maybe a Proxmox.Config
>> class could be added and used?
>>
> yeah, or maybe we could have a general 'Proxmox.APIPaths' class that contains the paths for wt relevant stuff? so that we do not have
> to design the api of new products in such a way that all products
> are the same (e.g. /nodes/YYY/ stuff)

IMO that would be a object of Proxmox.Config, avoiding to many top level classes
which are all would fit in the "product specific properties/configs/api-paths"
category.

Just as example, something like

Ext.define('Proxmox.Cfg', {

    APIPaths: {
        host_disk: {
            get: ...,
            put: ....,
        }
    },

    // possible other stuff

});


But yeah, I'd like to avoid repeating what we did in PBS with starting out with
/api2/ as prefix, that feels a bit weird.

Re: [pmg-devel] [PATCH api 2/8] add PMG::CertHelpers module

[Wolfgang Bumiller]

On Thu, Mar 11, 2021 at 11:05:21AM +0100, Dominik Csapak wrote:
> comments inline
> 
> On 3/9/21 3:13 PM, Wolfgang Bumiller wrote:
> > Contains helpers to update certificates and provide locking
> > for certificates and when accessing acme accounts.
> > 
> > Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
> > ---
> >   src/Makefile           |   1 +
> >   src/PMG/CertHelpers.pm | 180 +++++++++++++++++++++++++++++++++++++++++
> >   2 files changed, 181 insertions(+)
> >   create mode 100644 src/PMG/CertHelpers.pm
> > 
> > diff --git a/src/Makefile b/src/Makefile
> > index 8891a3c..c1d4812 100644
> > --- a/src/Makefile
> > +++ b/src/Makefile
> > @@ -55,6 +55,7 @@ LIBSOURCES =				\
> >   	PMG/HTMLMail.pm			\
> >   	PMG/ModGroup.pm			\
> >   	PMG/SMTPPrinter.pm		\
> > +	PMG/CertHelpers.pm		\
> >   	PMG/Config.pm			\
> >   	PMG/Cluster.pm			\
> >   	PMG/ClusterConfig.pm		\
> > diff --git a/src/PMG/CertHelpers.pm b/src/PMG/CertHelpers.pm
> > new file mode 100644
> > index 0000000..2cf8a4e
> > --- /dev/null
> > +++ b/src/PMG/CertHelpers.pm
> > @@ -0,0 +1,180 @@
> > +package PMG::CertHelpers;
> > +
> > +use strict;
> > +use warnings;
> > +
> > +use PVE::Certificate;
> > +use PVE::JSONSchema;
> > +use PVE::Tools;
> > +
> > +use constant {
> > +    API_CERT => '/etc/pmg/pmg-api.pem',
> > +    SMTP_CERT => '/etc/pmg/pmg-tls.pem',
> > +};
> > +
> > +my $account_prefix = '/etc/pmg/acme';
> > +
> > +# TODO: Move `pve-acme-account-name` to common and reuse instead of this.
> > +PVE::JSONSchema::register_standard_option('pmg-acme-account-name', {
> > +    description => 'ACME account config file name.',
> > +    type => 'string',
> > +    format => 'pve-configid',
> > +    format_description => 'name',
> > +    optional => 1,
> > +    default => 'default',
> > +});
> > +
> > +PVE::JSONSchema::register_standard_option('pmg-acme-account-contact', {
> > +    type => 'string',
> > +    format => 'email-list',
> > +    description => 'Contact email addresses.',
> > +});
> > +
> > +PVE::JSONSchema::register_standard_option('pmg-acme-directory-url', {
> > +    type => 'string',
> > +    description => 'URL of ACME CA directory endpoint.',
> > +    pattern => '^https?://.*',
> > +});
> > +
> > +PVE::JSONSchema::register_format('pmg-certificate-type', sub {
> > +    my ($type, $noerr) = @_;
> > +
> > +    if ($type =~ /^(?: api | smtp )$/x) {
> > +	return $type;
> > +    }
> > +    return undef if $noerr;
> > +    die "value '$type' does not look like a valid certificate type\n";
> > +});
> > +
> > +PVE::JSONSchema::register_standard_option('pmg-certificate-type', {
> > +    type => 'string',
> > +    description => 'The TLS certificate type (API or SMTP certificate).',
> > +    enum => ['api', 'smtp'],
> > +});
> 
> i get why you did the format and the option (you need it once as a '-list')
> but would it not have been possible to reuse the format instead
> of redefining the enum?

Yeah I suppose that works

> or only using the enum as variable defined somewhere?

(or at least that will)

> feels weird to have a format + option that do basically
> the same thing

I know

> 
> > +
> > +PVE::JSONSchema::register_format('pmg-acme-domain', sub {
> > +    my ($domain, $noerr) = @_;
> > +
> > +    my $label = qr/[a-z0-9][a-z0-9_-]*/i;
> > +
> > +    return $domain if $domain =~ /^$label(?:\.$label)+$/;
> > +    return undef if $noerr;
> > +    die "value '$domain' does not look like a valid domain name!\n";
> > +});
> > +
> > +PVE::JSONSchema::register_format('pmg-acme-alias', sub {
> > +    my ($alias, $noerr) = @_;
> > +
> > +    my $label = qr/[a-z0-9_][a-z0-9_-]*/i;
> > +
> > +    return $alias if $alias =~ /^$label(?:\.$label)+$/;
> > +    return undef if $noerr;
> > +    die "value '$alias' does not look like a valid alias name!\n";
> > +});
> 
> could we not reuse the '-domain' format here ?
> i know the error message would be different then, but it is still a domain?
> 
> if not, we could refactor the regexes though

-alias was specifically introduced to allow using underscore prefixes.
(See pve-manager b3d421707)

> 
> > +
> > +my $local_cert_lock = '/var/lock/pmg-certs.lock';
> > +my $local_acme_lock = '/var/lock/pmg-acme.lock';
> > +
> > +sub cert_path : prototype($) {
> > +    my ($type) = @_;
> > +    if ($type eq 'api') {
> > +	return API_CERT;
> > +    } elsif ($type eq 'smtp') {
> > +	return SMTP_CERT;
> > +    } else {
> > +	die "unknown certificate type '$type'\n";
> > +    }
> > +}
> > +
> > +sub cert_lock {
> > +    my ($timeout, $code, @param) = @_;
> > +
> > +    my $res = PVE::Tools::lock_file($local_cert_lock, $timeout, $code, @param);
> > +    die $@ if $@;
> > +    return $res;
> > +}
> > +
> > +sub set_cert_file {
> > +    my ($cert, $cert_path, $force) = @_;
> > +
> > +    my ($old_cert, $info);
> > +
> > +    my $cert_path_old = "${cert_path}.old";
> > +
> > +    die "Custom certificate file exists but force flag is not set.\n"
> > +	if !$force && -e $cert_path;
> > +
> > +    PVE::Tools::file_copy($cert_path, $cert_path_old) if -e $cert_path;
> > +
> > +    eval {
> > +	my $gid = undef;
> > +	if ($cert_path eq &API_CERT) {
> > +	    $gid = getgrnam('www-data') ||
> > +		die "user www-data not in group file\n";
> > +	}
> > +
> > +	if (defined($gid)) {
> > +	    my $cert_path_tmp = "${cert_path}.tmp";
> > +	    PVE::Tools::file_set_contents($cert_path_tmp, $cert, 0640);
> > +	    if (!chown(-1, $gid, $cert_path_tmp)) {
> > +		my $msg =
> > +		    "failed to change group ownership of '$cert_path_tmp' to www-data ($gid): $!\n";
> > +		unlink($cert_path_tmp);
> > +		die $msg;
> > +	    }
> > +	    if (!rename($cert_path_tmp, $cert_path)) {
> > +		my $msg =
> > +		    "failed to rename '$cert_path_tmp' to '$cert_path': $!\n";
> > +		unlink($cert_path_tmp);
> > +		die $msg;
> > +	    }
> > +	} else {
> > +	    PVE::Tools::file_set_contents($cert_path, $cert, 0600);
> > +	}
> > +
> > +	$info = PVE::Certificate::get_certificate_info($cert_path);
> > +    };
> > +    my $err = $@;
> > +
> > +    if ($err) {
> > +	if (-e $cert_path_old) {
> > +	    eval {
> > +		warn "Attempting to restore old certificate file..\n";
> > +		PVE::Tools::file_copy($cert_path_old, $cert_path);
> > +	    };
> > +	    warn "$@\n" if $@;
> > +	}
> > +	die "Setting certificate files failed - $err\n"
> > +    }
> > +
> > +    unlink $cert_path_old;
> > +
> > +    return $info;
> > +}
> > +
> > +sub lock_acme {
> > +    my ($account_name, $timeout, $code, @param) = @_;
> > +
> > +    my $file = "$local_acme_lock.$account_name";
> > +
> > +    return PVE::Tools::lock_file($file, $timeout, $code, @param);
> > +}
> > +
> 
> is there a special reason why you die $@ if $@ above in cert_lock
> but not here?
> 
> afaics, you do it manually in the later patches always anyway

That's just lock api madness... will fix.

Re: [pmg-devel] [PATCH api 5/8] api: add ACME and ACMEPlugin module

[Wolfgang Bumiller]

On Thu, Mar 11, 2021 at 11:41:22AM +0100, Dominik Csapak wrote:
> comments inline
> 
(...)
> > @@ -0,0 +1,436 @@
> > +
> > +__PACKAGE__->register_method ({
> > +    name => 'account_index',
> > +    path => 'account',
> > +    method => 'GET',
> > +    permissions => { user => 'all' },
> 
> i'd argue that the qmanager should not list the
> available acme accounts

right

> 
> > +    description => "ACME account index.",
> > +    protected => 1,
> > +    parameters => {
> > +	additionalProperties => 0,
> > +	properties => {
> > +	},
> > +    },
> > +    returns => {
> > +	type => 'array',
> > +	items => {
> > +	    type => "object",
> > +	    properties => {},
> > +	},
> > +	links => [ { rel => 'child', href => "{name}" } ],
> > +    },
> > +    code => sub {
> > +	my ($param) = @_;
> > +
> > +	my $accounts = PMG::CertHelpers::list_acme_accounts();
> > +	return [ map { { name => $_ }  } @$accounts ];
> > +    }});
> 
> for the following create/update
> the permissions are missing but should be 'admin'
> (they are ok for the plugins)

yeah, fixing

> 
> > +
> > +__PACKAGE__->register_method ({
> > +    name => 'register_account',
> > +    path => 'account',
> > +    method => 'POST',
> > +    description => "Register a new ACME account with CA.",
> > +    proxyto => 'master',
> > +    protected => 1,
> > +    parameters => {
> > +	additionalProperties => 0,
> > +	properties => {
> > +	    name => get_standard_option('pmg-acme-account-name'),
> > +	    contact => get_standard_option('pmg-acme-account-contact'),
> > +	    tos_url => {
> > +		type => 'string',
> > +		description => 'URL of CA TermsOfService - setting this indicates agreement.',
> > +		optional => 1,
> > +	    },
> > +	    directory => get_standard_option('pmg-acme-directory-url', {
> > +		default => $acme_default_directory_url,
> > +		optional => 1,
> > +	    }),
> > +	},
> > +    },
> > +    returns => {
> > +	type => 'string',
> > +    },
> > +    code => sub {
> > +	my ($param) = @_;
> > +
> > +	my $rpcenv = PMG::RESTEnvironment->get();
> > +	my $authuser = $rpcenv->get_user();
> > +
> > +	my $account_name = extract_param($param, 'name') // 'default';
> > +	my $account_file = "${acme_account_dir}/${account_name}";
> > +	mkdir $acme_account_dir if ! -e $acme_account_dir;
> > +
> > +	raise_param_exc({'name' => "ACME account config file '${account_name}' already exists."})
> > +	    if -e $account_file;
> > +
> > +	my $directory = extract_param($param, 'directory') // $acme_default_directory_url;
> > +	my $contact = $account_contact_from_param->($param);
> > +
> > +	my $realcmd = sub {
> > +	    PMG::CertHelpers::lock_acme($account_name, 10, sub {
> > +		die "ACME account config file '${account_name}' already exists.\n"
> > +		    if -e $account_file;
> > +
> > +		print "Registering new ACME account..\n";
> > +		my $acme = PMG::RS::Acme->new($directory);
> > +		eval {
> > +		    $acme->new_account($account_file, defined($param->{tos_url}), $contact, undef);
> > +		};
> > +		if (my $err = $@) {
> > +		    unlink $account_file;
> > +		    die "Registration failed: $err\n";
> > +		}
> > +		my $location = $acme->location();
> > +		print "Registration successful, account URL: '$location'\n";
> > +	    });
> > +	    die $@ if $@;
> > +	};
> > +
> > +	return $rpcenv->fork_worker('acmeregister', undef, $authuser, $realcmd);
> > +    }});
> > +
> > +
> > +__PACKAGE__->register_method ({
> > +    name => 'deactivate_account',
> > +    path => 'account/{name}',
> > +    method => 'DELETE',
> > +    description => "Deactivate existing ACME account at CA.",
> > +    protected => 1,
> > +    proxyto => 'master',
> > +    parameters => {
> > +	additionalProperties => 0,
> > +	properties => {
> > +	    name => get_standard_option('pmg-acme-account-name'),
> > +	},
> > +    },
> > +    returns => {
> > +	type => 'string',
> > +    },
> > +    code => sub {
> > +	my ($param) = @_;
> > +
> > +	return $update_account->($param, 'deactivate', status => 'deactivated');
> > +    }});
> > +
> > +__PACKAGE__->register_method ({
> > +    name => 'get_tos',
> > +    path => 'tos',
> > +    method => 'GET',
> > +    description => "Retrieve ACME TermsOfService URL from CA.",
> > +    permissions => { user => 'all' },
> > +    parameters => {
> > +	additionalProperties => 0,
> > +	properties => {
> > +	    directory => get_standard_option('pmg-acme-directory-url', {
> > +		default => $acme_default_directory_url,
> > +		optional => 1,
> > +	    }),
> > +	},
> > +    },
> > +    returns => {
> > +	type => 'string',
> > +	optional => 1,
> > +	description => 'ACME TermsOfService URL.',
> > +    },
> > +    code => sub {
> > +	my ($param) = @_;
> > +
> > +	my $directory = extract_param($param, 'directory') // $acme_default_directory_url;
> > +
> > +	my $acme = PMG::RS::Acme->new($directory);
> > +	my $meta = $acme->get_meta();
> > +
> > +	return $meta ? $meta->{termsOfService} : undef;
> > +    }});
> 
> just for my understanding: what happens here if there is no TOS?
> is that valid ACME behaviour? or should we somehow error out?

According to the RFC the value is optional and so we should not error
out.

> > +__PACKAGE__->register_method({
> > +    name => 'add_plugin',
> > +    path => '',
> > +    method => 'POST',
> > +    description => "Add ACME plugin configuration.",
> > +    permissions => { check => [ 'admin' ] },
> > +    protected => 1,
> > +    parameters => PVE::ACME::Challenge->createSchema(),
> > +    returns => {
> > +	type => "null"
> > +    },
> > +    code => sub {
> > +	my ($param) = @_;
> > +
> > +	my $id = extract_param($param, 'id');
> > +	my $type = extract_param($param, 'type');
> > +
> > +	lock_config(sub {
> > +	    my $cfg = load_config();
> > +	    die "ACME plugin ID '$id' already exists\n" if defined($cfg->{ids}->{$id});
> > +
> > +	    my $plugin = PVE::ACME::Challenge->lookup($type);
> > +	    my $opts = $plugin->check_config($id, $param, 1, 1);
> > +
> > +	    $cfg->{ids}->{$id} = $opts;
> > +	    $cfg->{ids}->{$id}->{type} = $type;
> > +
> > +	    write_config($cfg);
> > +	});
> > +	die "$@" if $@;
> 
> you already die in lock_config if $@ is set.

fixing all those up

> > +
> > +	return undef;
> > +    }
> > +});
> > +
> > +__PACKAGE__->register_method({
> > +    name => 'update_plugin',
> > +    path => '{id}',
> > +    method => 'PUT',
> > +    description => "Update ACME plugin configuration.",
> > +    permissions => { check => [ 'admin' ] },
> > +    protected => 1,
> > +    parameters => PVE::ACME::Challenge->updateSchema(),
> > +    returns => {
> > +	type => "null"
> > +    },
> > +    code => sub {
> > +	my ($param) = @_;
> > +
> > +	my $id = extract_param($param, 'id');
> > +	my $delete = extract_param($param, 'delete');
> > +	my $digest = extract_param($param, 'digest');
> > +
> > +	lock_config(sub {
> > +	    my $cfg = load_config();
> > +	    PVE::Tools::assert_if_modified($cfg->{digest}, $digest);
> > +	    my $plugin_cfg = $cfg->{ids}->{$id};
> > +	    die "ACME plugin ID '$id' does not exist\n" if !$plugin_cfg;
> > +
> > +	    my $type = $plugin_cfg->{type};
> > +	    my $plugin = PVE::ACME::Challenge->lookup($type);
> > +
> > +	    if (defined($delete)) {
> > +		my $schema = $plugin->private();
> > +		my $options = $schema->{options}->{$type};
> > +		for my $k (PVE::Tools::split_list($delete)) {
> > +		    my $d = $options->{$k} || die "no such option '$k'\n";
> > +		    die "unable to delete required option '$k'\n" if !$d->{optional};
> > +
> > +		    delete $cfg->{ids}->{$id}->{$k};
> > +		}
> > +	    }
> > +
> > +	    my $opts = $plugin->check_config($id, $param, 0, 1);
> > +	    for my $k (sort keys %$opts) {
> 
> not that it should make a difference, but why sort?

PVE copy-pasta ;-) will fix

Re: [pmg-devel] [PATCH api 6/8] add certificates api endpoint

[Wolfgang Bumiller]

On Thu, Mar 11, 2021 at 12:06:23PM +0100, Dominik Csapak wrote:
> 
> 
> On 3/9/21 3:13 PM, Wolfgang Bumiller wrote:
> > This adds /nodes/{nodename}/certificates endpoint
> > containing:
> > 
> >    /custom/{type} - update smtp or api certificates manually
> >    /acme/{type} - update via acme
> > 
> > Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
> > ---
> >   src/Makefile                 |   1 +
> >   src/PMG/API2/Certificates.pm | 690 +++++++++++++++++++++++++++++++++++
> >   src/PMG/API2/Nodes.pm        |   7 +
> >   3 files changed, 698 insertions(+)
> >   create mode 100644 src/PMG/API2/Certificates.pm
> > 
> > diff --git a/src/Makefile b/src/Makefile
> > index ebc6bd8..e0629b2 100644
> > --- a/src/Makefile
> > +++ b/src/Makefile
> > @@ -155,6 +155,7 @@ LIBSOURCES =				\
> >   	PMG/API2/When.pm		\
> >   	PMG/API2/What.pm		\
> >   	PMG/API2/Action.pm		\
> > +	PMG/API2/Certificates.pm	\
> >   	PMG/API2/ACME.pm		\
> >   	PMG/API2/ACMEPlugin.pm		\
> >   	PMG/API2.pm			\
> > diff --git a/src/PMG/API2/Certificates.pm b/src/PMG/API2/Certificates.pm
> > new file mode 100644
> > index 0000000..d196af6
> > --- /dev/null
> > +++ b/src/PMG/API2/Certificates.pm
> > @@ -0,0 +1,690 @@
> > +package PMG::API2::Certificates;
> > +
> > +use strict;
> > +use warnings;
> > +
> > +use PVE::Certificate;
> > +use PVE::Exception qw(raise raise_param_exc);
> > +use PVE::JSONSchema qw(get_standard_option);
> > +use PVE::Tools qw(extract_param file_get_contents file_set_contents);
> > +
> > +use PMG::CertHelpers;
> > +use PMG::NodeConfig;
> > +use PMG::RS::CSR;
> > +
> > +use PMG::API2::ACMEPlugin;
> > +
> > +use base qw(PVE::RESTHandler);
> > +
> > +my $acme_account_dir = PMG::CertHelpers::acme_account_dir();
> > +
> > +sub first_typed_pem_entry : prototype($$) {
> > +    my ($label, $data) = @_;
> > +
> > +    if ($data =~ /^(-----BEGIN (?<label>\Q$label\E)-----\n.*?\n-----END \g{label}-----)$/ms) {
> 
> nit: isn't
> $data =~ /^(-----BEGIN \Q$label\E-----\n.*?\n-----END \Q$label\E-----)$/ms
> 
> shorter and does the same?
> 
> 
> > +	chomp(my $content = $1);
> 
> nit: why chomp? the regex does not allow trailing/whitespace newlines in $1
> ?
> 

Yeah those are leftovers from previous versions.

> > +__PACKAGE__->register_method ({
> > +    name => 'upload_custom_cert',
> > +    path => 'custom/{type}',
> > +    method => 'POST',
> > +    permissions => { check => [ 'admin' ] },
> > +    description => 'Upload or update custom certificate chain and key.',
> > +    protected => 1,
> > +    proxyto => 'node',
> > +    parameters => {
> > +	additionalProperties => 0,
> > +	properties => {
> > +	    node => get_standard_option('pve-node'),
> > +	    certificates => {
> > +		type => 'string',
> > +		format => 'pem-certificate-chain',
> > +		description => 'PEM encoded certificate (chain).',
> > +	    },
> > +	    key => {
> > +		type => 'string',
> > +		description => 'PEM encoded private key.',
> > +		format => 'pem-string',
> > +		optional => 0,
> > +	    },
> > +	    type => get_standard_option('pmg-certificate-type'),
> > +	    force => {
> > +		type => 'boolean',
> > +		description => 'Overwrite existing custom or ACME certificate files.',
> > +		optional => 1,
> > +		default => 0,
> > +	    },
> > +	    restart => {
> > +		type => 'boolean',
> > +		description => 'Restart services.',
> > +		optional => 1,
> > +		default => 0,
> > +	    },
> > +	},
> > +    },
> > +    returns => get_standard_option('pve-certificate-info'),
> > +    code => sub {
> > +	my ($param) = @_;
> > +
> > +	my $type = extract_param($param, 'type'); # also used to know which service to restart
> > +	my $cert_path = PMG::CertHelpers::cert_path($type);
> > +
> > +	my $certs = extract_param($param, 'certificates');
> > +	$certs = PVE::Certificate::strip_leading_text($certs);
> > +
> > +	my $key = extract_param($param, 'key');
> > +	if ($key) {
> > +	    $key = PVE::Certificate::strip_leading_text($key);
> > +	    $certs = "$key\n$certs";
> > +	} else {
> > +	    my $private_key = pem_private_key($certs);
> > +	    if (!defined($private_key)) {
> > +		my $old = file_get_contents($cert_path);
> > +		$private_key = pem_private_key($old);
> > +		if (!defined($private_key)) {
> > +		    raise_param_exc({
> > +			'key' => "Attempted to upload custom certificate without (existing) key."
> > +		    })
> > +		}
> > +
> > +		# copy the old certificate's key:
> > +		$certs = "$key\n$certs";
> > +	    }
> > +	}
> > +
> > +	my $info;
> > +
> > +	my $code = sub {
> > +	    print "Setting custom certificate file $cert_path\n";
> > +	    $info = PMG::CertHelpers::set_cert_file($certs, $cert_path, $param->{force});
> > +
> > +	    if ($type eq 'api' && $param->{restart}) {
> > +		print "Restarting pmgproxy\n";
> > +		PVE::Tools::run_command(['systemctl', 'reload-or-restart', 'pmgproxy']);
> 
> you could reuse the 'restart_after_cert_update' here, no?

Yeah, actually I'll pass $param->{restart} along to update_cert()

> 
> > +	    }
> > +	};
> > +
> > +	PMG::CertHelpers::cert_lock(10, $code);
> > +	die "$@\n" if $@;
> > +
> > +	if ($type eq 'smtp') {
> > +	    $code = sub {
> > +		my $cfg = PMG::Config->new();
> > +
> > +		print "Rewriting postfix config\n";
> > +		$cfg->set('mail', 'tls', 1);
> > +		$cfg->rewrite_config_postfix();
> > +
> > +		if ($param->{restart}) {
> > +		    print "Reloading postfix\n";
> > +		    PMG::Utils::service_cmd('postfix', 'reload');
> 
> also, why couldn't that be handled there too? then we could
> combine the two restart/reload calls?

These run with 2 different locks.

(And failure to reload update the config or reloading postfix should not
undo the certificate update.)

(Technically neither should reloading pmgproxy though that's just the
same as in PVE and can really only fail if the `systemctl` command
fails, so that's rather rare.)

> 
> > +		}
> > +	    };
> > +	    PMG::Config::lock_config($code, "failed to reload postfix");
> > +	}
> > +
> > +	return $info;
> > +    }});
> > +
> > +__PACKAGE__->register_method ({
> > +    name => 'remove_custom_cert',
> > +    path => 'custom/{type}',
> > +    method => 'DELETE',
> > +    permissions => { check => [ 'admin' ] },
> > +    description => 'DELETE custom certificate chain and key.',
> > +    protected => 1,
> > +    proxyto => 'node',
> > +    parameters => {
> > +	additionalProperties => 0,
> > +	properties => {
> > +	    node => get_standard_option('pve-node'),
> > +	    type => get_standard_option('pmg-certificate-type'),
> > +	    restart => {
> > +		type => 'boolean',
> > +		description => 'Restart pmgproxy.',
> > +		optional => 1,
> > +		default => 0,
> > +	    },
> > +	},
> > +    },
> > +    returns => {
> > +	type => 'null',
> > +    },
> > +    code => sub {
> > +	my ($param) = @_;
> > +
> > +	my $type = extract_param($param, 'type');
> > +	my $cert_path = PMG::CertHelpers::cert_path($type);
> > +
> > +	my $code = sub {
> > +	    print "Deleting custom certificate files\n";
> > +	    unlink $cert_path;
> > +
> > +	    if ($param->{restart}) {
> > +		restart_after_cert_update($type);
> > +	    }
> > +	};
> > +
> > +	PMG::CertHelpers::cert_lock(10, $code);
> > +	die "$@\n" if $@;
> > +
> > +	return undef;
> 
> don't we need to update the postfix config and reload if the type is smtp?
> or at least error out?

Yeah, adding that...

> > +    }});
> > +

> > +# Filter domains and raise an error if the list becomes empty.
> > +my $filter_domains = sub {
> > +    my ($acme_config, $type) = @_;
> > +
> > +    my $domains = $acme_config->{domains};
> > +    foreach my $domain (keys %$domains) {
> > +	my $entry = $domains->{$domain};
> > +	if (!(grep { $_ eq $type } PVE::Tools::split_list($entry->{usage}))) {
> > +	    delete $domains->{$domain};
> > +	}
> > +    }
> > +
> > +    if (!%$domains) {
> > +	raise("No domains configured for type '$type'\n", 400);
> > +    }
> > +};
> > +
> > +__PACKAGE__->register_method ({
> > +    name => 'new_acme_cert',
> > +    path => 'acme/{type}',
> > +    method => 'POST',
> > +    permissions => { check => [ 'admin' ] },
> > +    description => 'Order a new certificate from ACME-compatible CA.',
> > +    protected => 1,
> > +    proxyto => 'node',
> > +    parameters => {
> > +	additionalProperties => 0,
> > +	properties => {
> > +	    node => get_standard_option('pve-node'),
> > +	    type => get_standard_option('pmg-certificate-type'),
> > +	    force => {
> > +		type => 'boolean',
> > +		description => 'Overwrite existing custom certificate.',
> > +		optional => 1,
> > +		default => 0,
> > +	    },
> > +	},
> > +    },
> > +    returns => {
> > +	type => 'string',
> > +    },
> > +    code => sub {
> > +	my ($param) = @_;
> > +
> > +	my $type = extract_param($param, 'type'); # also used to know which service to restart
> > +	my $cert_path = PMG::CertHelpers::cert_path($type);
> > +	raise_param_exc({'force' => "Custom certificate exists but 'force' is not set."})
> > +	    if !$param->{force} && -e $cert_path;
> > +
> > +	my $node_config = PMG::NodeConfig::load_config();
> > +	my $acme_config = PMG::NodeConfig::get_acme_conf($node_config);
> > +	raise("ACME domain list in configuration is missing!", 400)
> > +	    if !$acme_config || !$acme_config->{domains}->%*;
> > +
> > +	$filter_domains->($acme_config, $type);
> > +
> > +	my $rpcenv = PMG::RESTEnvironment->get();
> > +	my $authuser = $rpcenv->get_user();
> > +
> > +	my $realcmd = sub {
> > +	    STDOUT->autoflush(1);
> > +	    my $account = $acme_config->{account};
> > +	    my $account_file = "${acme_account_dir}/${account}";
> > +	    die "ACME account config file '$account' does not exist.\n"
> > +		if ! -e $account_file;
> > +
> > +	    print "Loading ACME account details\n";
> > +	    my $acme = PMG::RS::Acme->load($account_file);
> > +
> > +	    my ($cert, $key) = $order_certificate->($acme, $acme_config);
> > +	    my $certificate = "$key\n$cert";
> > +
> > +	    update_cert($type, $cert_path, $certificate, $param->{force});
> > +
> > +	    if ($type eq 'smtp') {
> > +		my $code = sub {
> > +		    my $cfg = PMG::Config->new();
> > +
> > +		    print "Rewriting postfix config\n";
> > +		    $cfg->set('mail', 'tls', 1);
> > +		    if ($cfg->rewrite_config_postfix()) {
> > +			print "Reloading postfix\n";
> > +			PMG::Utils::service_cmd('postfix', 'reload');
> > +		    }
> > +		};
> > +		PMG::Config::lock_config($code, "failed to reload postfix");
> > +	    }
> > +
> > +	    die "$@\n" if $@;
> > +	};
> > +
> > +	return $rpcenv->fork_worker("acmenewcert", undef, $authuser, $realcmd);
> > +    }});
> > +
> > +__PACKAGE__->register_method ({
> > +    name => 'renew_acme_cert',
> > +    path => 'acme/{type}',
> > +    method => 'PUT',
> > +    permissions => { check => [ 'admin' ] },
> > +    description => "Renew existing certificate from CA.",
> > +    protected => 1,
> > +    proxyto => 'node',
> > +    parameters => {
> > +	additionalProperties => 0,
> > +	properties => {
> > +	    node => get_standard_option('pve-node'),
> > +	    type => get_standard_option('pmg-certificate-type'),
> > +	    force => {
> > +		type => 'boolean',
> > +		description => 'Force renewal even if expiry is more than 30 days away.',
> > +		optional => 1,
> > +		default => 0,
> > +	    },
> > +	},
> > +    },
> > +    returns => {
> > +	type => 'string',
> > +    },
> > +    code => sub {
> > +	my ($param) = @_;
> > +
> > +	my $type = extract_param($param, 'type'); # also used to know which service to restart
> > +	my $cert_path = PMG::CertHelpers::cert_path($type);
> > +
> > +	raise("No current (custom) certificate found, please order a new certificate!\n")
> > +	    if ! -e $cert_path;
> > +
> > +	my $expires_soon = PVE::Certificate::check_expiry($cert_path, time() + 30*24*60*60);
> > +	raise_param_exc({'force' => "Certificate does not expire within the next 30 days, and 'force' is not set."})
> > +	    if !$expires_soon && !$param->{force};
> > +
> > +	my $node_config = PMG::NodeConfig::load_config();
> > +	my $acme_config = PMG::NodeConfig::get_acme_conf($node_config);
> > +	raise("ACME domain list in configuration is missing!", 400)
> > +	    if !$acme_config || !$acme_config->{domains}->%*;
> > +
> > +	$filter_domains->($acme_config, $type);
> > +
> > +	my $rpcenv = PMG::RESTEnvironment->get();
> > +	my $authuser = $rpcenv->get_user();
> > +
> > +	my $old_cert = PVE::Tools::file_get_contents($cert_path);
> > +
> > +	my $realcmd = sub {
> > +	    STDOUT->autoflush(1);
> > +	    my $account = $acme_config->{account};
> > +	    my $account_file = "${acme_account_dir}/${account}";
> > +	    die "ACME account config file '$account' does not exist.\n"
> > +		if ! -e $account_file;
> > +
> > +	    print "Loading ACME account details\n";
> > +	    my $acme = PMG::RS::Acme->load($account_file);
> > +
> > +	    my ($cert, $key) = $order_certificate->($acme, $acme_config);
> > +	    my $certificate = "$key\n$cert";
> > +
> > +	    update_cert($type, $cert_path, $certificate, 1);
> > +
> > +	    if (defined($old_cert)) {
> > +		print "Revoking old certificate\n";
> > +		eval { $acme->revoke_certificate($old_cert, undef) };
> > +		warn "Revoke request to CA failed: $@" if $@;
> > +	    }
> > +	};
> > +
> > +	return $rpcenv->fork_worker("acmerenew", undef, $authuser, $realcmd);
> > +    }});
> > +
> > +__PACKAGE__->register_method ({
> > +    name => 'revoke_acme_cert',
> > +    path => 'acme/{type}',
> > +    method => 'DELETE',
> > +    permissions => { check => [ 'admin' ] },
> > +    description => "Revoke existing certificate from CA.",
> > +    protected => 1,
> > +    proxyto => 'node',
> > +    parameters => {
> > +	additionalProperties => 0,
> > +	properties => {
> > +	    node => get_standard_option('pve-node'),
> > +	    type => get_standard_option('pmg-certificate-type'),
> > +	},
> > +    },
> > +    returns => {
> > +	type => 'string',
> > +    },
> > +    code => sub {
> > +	my ($param) = @_;
> > +
> > +	my $type = extract_param($param, 'type'); # also used to know which service to restart
> > +	my $cert_path = PMG::CertHelpers::cert_path($type);
> > +
> > +	my $node_config = PMG::NodeConfig::load_config();
> > +	my $acme_config = PMG::NodeConfig::get_acme_conf($node_config);
> > +	raise("ACME domain list in configuration is missing!", 400)
> > +	    if !$acme_config || !$acme_config->{domains}->%*;
> > +
> > +	$filter_domains->($acme_config, $type);
> > +
> > +	my $rpcenv = PMG::RESTEnvironment->get();
> > +	my $authuser = $rpcenv->get_user();
> > +
> > +	my $cert = PVE::Tools::file_get_contents($cert_path);
> > +	$cert = pem_certificate($cert)
> > +	    or die "no certificate section found in '$cert_path'\n";
> > +
> > +	my $realcmd = sub {
> > +	    STDOUT->autoflush(1);
> > +	    my $account = $acme_config->{account};
> > +	    my $account_file = "${acme_account_dir}/${account}";
> > +	    die "ACME account config file '$account' does not exist.\n"
> > +		if ! -e $account_file;
> > +
> > +	    print "Loading ACME account details\n";
> > +	    my $acme = PMG::RS::Acme->load($account_file);
> > +
> > +	    print "Revoking old certificate\n";
> > +	    eval { $acme->revoke_certificate($cert, undef) };
> > +	    if (my $err = $@) {
> > +		# is there a better check?
> > +		die "Revoke request to CA failed: $err" if $err !~ /"Certificate is expired"/;
> > +	    }
> > +
> > +	    my $code = sub {
> > +		print "Deleting certificate files\n";
> > +		unlink $cert_path;
> > +
> > +		# FIXME: Regenerate self-signed `api` certificate.
> > +		restart_after_cert_update($type);
> > +	    };
> > +
> > +	    PMG::CertHelpers::cert_lock(10, $code);
> > +	    die "$@\n" if $@;
> > +	};
> > +
> > +	return $rpcenv->fork_worker("acmerevoke", undef, $authuser, $realcmd);
> > +    }});
> 
> what happens here with postfix?

will update & reload here too

[pmg-devel] applied: [PATCH acme] add missing 'use PVE::Acme' statement

[Thomas Lamprecht]

On 09.03.21 15:13, Wolfgang Bumiller wrote:
> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
> ---
>  src/PVE/ACME/DNSChallenge.pm | 2 ++
>  1 file changed, 2 insertions(+)
> 
>

applied, thanks!