* [pbs-devel] [RFC backup 01/23] systemd: add reload_unit
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
@ 2021-04-16 13:34 ` Wolfgang Bumiller
2021-04-16 13:34 ` [pbs-devel] [RFC backup 02/23] add dns alias schema Wolfgang Bumiller
` (23 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:34 UTC (permalink / raw)
To: pbs-devel
via try-reload-or-restart
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/tools/systemd.rs | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/src/tools/systemd.rs b/src/tools/systemd.rs
index f8ae789c..c1760781 100644
--- a/src/tools/systemd.rs
+++ b/src/tools/systemd.rs
@@ -131,6 +131,17 @@ pub fn stop_unit(unit: &str) -> Result<(), Error> {
Ok(())
}
+pub fn reload_unit(unit: &str) -> Result<(), Error> {
+
+ let mut command = std::process::Command::new("systemctl");
+ command.arg("try-reload-or-restart");
+ command.arg(unit);
+
+ crate::tools::run_command(command, None)?;
+
+ Ok(())
+}
+
#[test]
fn test_escape_unit() -> Result<(), Error> {
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 02/23] add dns alias schema
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
2021-04-16 13:34 ` [pbs-devel] [RFC backup 01/23] systemd: add reload_unit Wolfgang Bumiller
@ 2021-04-16 13:34 ` Wolfgang Bumiller
2021-04-16 13:34 ` [pbs-devel] [RFC backup 03/23] tools::fs::scan_subdir: use nix::Error instead of anyhow Wolfgang Bumiller
` (22 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:34 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/api2/types/mod.rs | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/src/api2/types/mod.rs b/src/api2/types/mod.rs
index 19186ea2..c32c4ca3 100644
--- a/src/api2/types/mod.rs
+++ b/src/api2/types/mod.rs
@@ -51,6 +51,11 @@ pub const FILENAME_FORMAT: ApiStringFormat = ApiStringFormat::VerifyFn(|name| {
macro_rules! DNS_LABEL { () => (r"(?:[a-zA-Z0-9](?:[a-zA-Z0-9\-]*[a-zA-Z0-9])?)") }
macro_rules! DNS_NAME { () => (concat!(r"(?:(?:", DNS_LABEL!() , r"\.)*", DNS_LABEL!(), ")")) }
+macro_rules! DNS_ALIAS_LABEL { () => (r"(?:[a-zA-Z0-9_](?:[a-zA-Z0-9\-]*[a-zA-Z0-9])?)") }
+macro_rules! DNS_ALIAS_NAME {
+ () => (concat!(r"(?:(?:", DNS_ALIAS_LABEL!() , r"\.)*", DNS_ALIAS_LABEL!(), ")"))
+}
+
macro_rules! CIDR_V4_REGEX_STR { () => (concat!(r"(?:", IPV4RE!(), r"/\d{1,2})$")) }
macro_rules! CIDR_V6_REGEX_STR { () => (concat!(r"(?:", IPV6RE!(), r"/\d{1,3})$")) }
@@ -87,6 +92,8 @@ const_regex!{
pub DNS_NAME_REGEX = concat!(r"^", DNS_NAME!(), r"$");
+ pub DNS_ALIAS_REGEX = concat!(r"^", DNS_ALIAS_NAME!(), r"$");
+
pub DNS_NAME_OR_IP_REGEX = concat!(r"^(?:", DNS_NAME!(), "|", IPRE!(), r")$");
pub BACKUP_REPO_URL_REGEX = concat!(r"^^(?:(?:(", USER_ID_REGEX_STR!(), "|", APITOKEN_ID_REGEX_STR!(), ")@)?(", DNS_NAME!(), "|", IPRE_BRACKET!() ,"):)?(?:([0-9]{1,5}):)?(", PROXMOX_SAFE_ID_REGEX_STR!(), r")$");
@@ -142,6 +149,9 @@ pub const HOSTNAME_FORMAT: ApiStringFormat =
pub const DNS_NAME_FORMAT: ApiStringFormat =
ApiStringFormat::Pattern(&DNS_NAME_REGEX);
+pub const DNS_ALIAS_FORMAT: ApiStringFormat =
+ ApiStringFormat::Pattern(&DNS_ALIAS_REGEX);
+
pub const DNS_NAME_OR_IP_FORMAT: ApiStringFormat =
ApiStringFormat::Pattern(&DNS_NAME_OR_IP_REGEX);
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 03/23] tools::fs::scan_subdir: use nix::Error instead of anyhow
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
2021-04-16 13:34 ` [pbs-devel] [RFC backup 01/23] systemd: add reload_unit Wolfgang Bumiller
2021-04-16 13:34 ` [pbs-devel] [RFC backup 02/23] add dns alias schema Wolfgang Bumiller
@ 2021-04-16 13:34 ` Wolfgang Bumiller
2021-04-16 13:34 ` [pbs-devel] [RFC backup 04/23] tools::http: generic 'fn request' and dedup agent string Wolfgang Bumiller
` (21 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:34 UTC (permalink / raw)
To: pbs-devel
allows using SysError trait on it
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/tools/fs.rs | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/tools/fs.rs b/src/tools/fs.rs
index 72a530d8..6e0b1271 100644
--- a/src/tools/fs.rs
+++ b/src/tools/fs.rs
@@ -117,7 +117,7 @@ pub fn scan_subdir<'a, P: ?Sized + nix::NixPath>(
dirfd: RawFd,
path: &P,
regex: &'a regex::Regex,
-) -> Result<impl Iterator<Item = Result<ReadDirEntry, Error>> + 'a, Error> {
+) -> Result<impl Iterator<Item = Result<ReadDirEntry, Error>> + 'a, nix::Error> {
Ok(read_subdir(dirfd, path)?.filter_file_name_regex(regex))
}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 04/23] tools::http: generic 'fn request' and dedup agent string
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (2 preceding siblings ...)
2021-04-16 13:34 ` [pbs-devel] [RFC backup 03/23] tools::fs::scan_subdir: use nix::Error instead of anyhow Wolfgang Bumiller
@ 2021-04-16 13:34 ` Wolfgang Bumiller
2021-04-16 13:34 ` [pbs-devel] [RFC backup 05/23] config: factor out certificate writing Wolfgang Bumiller
` (20 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:34 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/tools/http.rs | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/src/tools/http.rs b/src/tools/http.rs
index d08ce451..136a678f 100644
--- a/src/tools/http.rs
+++ b/src/tools/http.rs
@@ -28,11 +28,17 @@ lazy_static! {
};
}
+const USER_AGENT_STRING: &str = "proxmox-backup-client/1.0";
+
+pub async fn request(request: http::request::Builder, body: Body) -> Result<Response<Body>, Error> {
+ Ok(HTTP_CLIENT.request(request.header("User-Agent", USER_AGENT_STRING).body(body)?).await?)
+}
+
pub async fn get_string(uri: &str, extra_headers: Option<&HashMap<String, String>>) -> Result<String, Error> {
let mut request = Request::builder()
.method("GET")
.uri(uri)
- .header("User-Agent", "proxmox-backup-client/1.0");
+ .header("User-Agent", USER_AGENT_STRING);
if let Some(hs) = extra_headers {
for (h, v) in hs.iter() {
@@ -73,7 +79,7 @@ pub async fn post(
let request = Request::builder()
.method("POST")
.uri(uri)
- .header("User-Agent", "proxmox-backup-client/1.0")
+ .header("User-Agent", USER_AGENT_STRING)
.header(hyper::header::CONTENT_TYPE, content_type)
.body(body)?;
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 05/23] config: factor out certificate writing
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (3 preceding siblings ...)
2021-04-16 13:34 ` [pbs-devel] [RFC backup 04/23] tools::http: generic 'fn request' and dedup agent string Wolfgang Bumiller
@ 2021-04-16 13:34 ` Wolfgang Bumiller
2021-04-16 13:34 ` [pbs-devel] [RFC backup 06/23] CertInfo: add not_{after, before}_unix Wolfgang Bumiller
` (19 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:34 UTC (permalink / raw)
To: pbs-devel
for reuse in the certificate api
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/config.rs | 39 ++++++++++++++++++---------------------
1 file changed, 18 insertions(+), 21 deletions(-)
diff --git a/src/config.rs b/src/config.rs
index 1557e20a..37df2fd2 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -98,10 +98,6 @@ pub fn create_configdir() -> Result<(), Error> {
/// Update self signed node certificate.
pub fn update_self_signed_cert(force: bool) -> Result<(), Error> {
- let backup_user = crate::backup::backup_user()?;
-
- create_configdir()?;
-
let key_path = PathBuf::from(configdir!("/proxy.key"));
let cert_path = PathBuf::from(configdir!("/proxy.pem"));
@@ -111,15 +107,6 @@ pub fn update_self_signed_cert(force: bool) -> Result<(), Error> {
let priv_pem = rsa.private_key_to_pem()?;
- replace_file(
- &key_path,
- &priv_pem,
- CreateOptions::new()
- .perm(Mode::from_bits_truncate(0o0640))
- .owner(nix::unistd::ROOT)
- .group(backup_user.gid),
- )?;
-
let mut x509 = X509Builder::new()?;
x509.set_version(2)?;
@@ -198,14 +185,24 @@ pub fn update_self_signed_cert(force: bool) -> Result<(), Error> {
let x509 = x509.build();
let cert_pem = x509.to_pem()?;
- replace_file(
- &cert_path,
- &cert_pem,
- CreateOptions::new()
- .perm(Mode::from_bits_truncate(0o0640))
- .owner(nix::unistd::ROOT)
- .group(backup_user.gid),
- )?;
+ set_proxy_certificate(&cert_pem, &priv_pem)?;
Ok(())
}
+
+pub(crate) fn set_proxy_certificate(cert_pem: &[u8], key_pem: &[u8]) -> Result<(), Error> {
+ let backup_user = crate::backup::backup_user()?;
+ let options = CreateOptions::new()
+ .perm(Mode::from_bits_truncate(0o0640))
+ .owner(nix::unistd::ROOT)
+ .group(backup_user.gid);
+ let key_path = PathBuf::from(configdir!("/proxy.key"));
+ let cert_path = PathBuf::from(configdir!("/proxy.pem"));
+
+ create_configdir()?;
+ replace_file(&key_path, &key_pem, options.clone())
+ .map_err(|err| format_err!("error writing certificate private key - {}", err))?;
+ replace_file(&cert_path, &cert_pem, options)
+ .map_err(|err| format_err!("error writing certificate file - {}", err))?;
+ Ok(())
+}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 06/23] CertInfo: add not_{after, before}_unix
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (4 preceding siblings ...)
2021-04-16 13:34 ` [pbs-devel] [RFC backup 05/23] config: factor out certificate writing Wolfgang Bumiller
@ 2021-04-16 13:34 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 07/23] CertInfo: add is_expired_after_epoch Wolfgang Bumiller
` (18 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:34 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
Cargo.toml | 1 +
src/tools/cert.rs | 36 ++++++++++++++++++++++++++++++++++--
2 files changed, 35 insertions(+), 2 deletions(-)
diff --git a/Cargo.toml b/Cargo.toml
index 3739d3af..74ccf361 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -32,6 +32,7 @@ endian_trait = { version = "0.6", features = ["arrays"] }
env_logger = "0.7"
flate2 = "1.0"
anyhow = "1.0"
+foreign-types = "0.3"
thiserror = "1.0"
futures = "0.3"
h2 = { version = "0.3", features = [ "stream" ] }
diff --git a/src/tools/cert.rs b/src/tools/cert.rs
index 0c7e9e5d..d3c221a4 100644
--- a/src/tools/cert.rs
+++ b/src/tools/cert.rs
@@ -1,12 +1,32 @@
use std::path::PathBuf;
+use std::mem::MaybeUninit;
-use anyhow::Error;
+use anyhow::{bail, format_err, Error};
+use foreign_types::ForeignTypeRef;
use openssl::x509::{X509, GeneralName};
use openssl::stack::Stack;
use openssl::pkey::{Public, PKey};
use crate::configdir;
+// C type:
+#[allow(non_camel_case_types)]
+type ASN1_TIME = <openssl::asn1::Asn1TimeRef as ForeignTypeRef>::CType;
+
+extern "C" {
+ fn ASN1_TIME_to_tm(s: *const ASN1_TIME, tm: *mut libc::tm) -> libc::c_int;
+}
+
+fn asn1_time_to_unix(time: &openssl::asn1::Asn1TimeRef) -> Result<i64, Error> {
+ let mut c_tm = MaybeUninit::<libc::tm>::uninit();
+ let rc = unsafe { ASN1_TIME_to_tm(time.as_ptr(), c_tm.as_mut_ptr()) };
+ if rc != 1 {
+ bail!("failed to parse ASN1 time");
+ }
+ let mut c_tm = unsafe { c_tm.assume_init() };
+ proxmox::tools::time::timegm(&mut c_tm)
+}
+
pub struct CertInfo {
x509: X509,
}
@@ -25,7 +45,11 @@ impl CertInfo {
}
pub fn from_path(path: PathBuf) -> Result<Self, Error> {
- let cert_pem = proxmox::tools::fs::file_get_contents(&path)?;
+ Self::from_pem(&proxmox::tools::fs::file_get_contents(&path)?)
+ .map_err(|err| format_err!("failed to load certificate from {:?} - {}", path, err))
+ }
+
+ pub fn from_pem(cert_pem: &[u8]) -> Result<Self, Error> {
let x509 = openssl::x509::X509::from_pem(&cert_pem)?;
Ok(Self{
x509
@@ -64,4 +88,12 @@ impl CertInfo {
pub fn not_after(&self) -> &openssl::asn1::Asn1TimeRef {
self.x509.not_after()
}
+
+ pub fn not_before_unix(&self) -> Result<i64, Error> {
+ asn1_time_to_unix(&self.not_before())
+ }
+
+ pub fn not_after_unix(&self) -> Result<i64, Error> {
+ asn1_time_to_unix(&self.not_after())
+ }
}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 07/23] CertInfo: add is_expired_after_epoch
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (5 preceding siblings ...)
2021-04-16 13:34 ` [pbs-devel] [RFC backup 06/23] CertInfo: add not_{after, before}_unix Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 08/23] tools: add ControlFlow type Wolfgang Bumiller
` (17 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/tools/cert.rs | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/src/tools/cert.rs b/src/tools/cert.rs
index d3c221a4..bcc3e69b 100644
--- a/src/tools/cert.rs
+++ b/src/tools/cert.rs
@@ -96,4 +96,9 @@ impl CertInfo {
pub fn not_after_unix(&self) -> Result<i64, Error> {
asn1_time_to_unix(&self.not_after())
}
+
+ /// Check if the certificate is expired at or after a specific unix epoch.
+ pub fn is_expired_after_epoch(&self, epoch: i64) -> Result<bool, Error> {
+ Ok(self.not_after_unix()? < epoch)
+ }
}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 08/23] tools: add ControlFlow type
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (6 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 07/23] CertInfo: add is_expired_after_epoch Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 09/23] catalog shell: replace LoopState with ControlFlow Wolfgang Bumiller
` (16 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
modeled after std::ops::ControlFlow
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/tools.rs | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/src/tools.rs b/src/tools.rs
index 08af55e5..890db826 100644
--- a/src/tools.rs
+++ b/src/tools.rs
@@ -571,3 +571,14 @@ pub fn create_run_dir() -> Result<(), Error> {
let _: bool = proxmox::tools::fs::create_path(PROXMOX_BACKUP_RUN_DIR_M!(), None, None)?;
Ok(())
}
+
+/// Modeled after the nightly `std::ops::ControlFlow`.
+#[derive(Clone, Copy, Debug, PartialEq)]
+pub enum ControlFlow<B, C = ()> {
+ Continue(C),
+ Break(B),
+}
+
+impl<B> ControlFlow<B> {
+ pub const CONTINUE: ControlFlow<B, ()> = ControlFlow::Continue(());
+}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 09/23] catalog shell: replace LoopState with ControlFlow
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (7 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 08/23] tools: add ControlFlow type Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 10/23] Cargo.toml: depend on proxmox-acme-rs Wolfgang Bumiller
` (15 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/backup/catalog_shell.rs | 18 +++++++-----------
1 file changed, 7 insertions(+), 11 deletions(-)
diff --git a/src/backup/catalog_shell.rs b/src/backup/catalog_shell.rs
index 00b5b580..11866c30 100644
--- a/src/backup/catalog_shell.rs
+++ b/src/backup/catalog_shell.rs
@@ -19,9 +19,10 @@ use proxmox::tools::fs::{create_path, CreateOptions};
use pxar::{EntryKind, Metadata};
use crate::backup::catalog::{self, DirEntryAttribute};
-use crate::pxar::Flags;
use crate::pxar::fuse::{Accessor, FileEntry};
+use crate::pxar::Flags;
use crate::tools::runtime::block_in_place;
+use crate::tools::ControlFlow;
type CatalogReader = crate::backup::CatalogReader<std::fs::File>;
@@ -998,11 +999,6 @@ impl Shell {
}
}
-enum LoopState {
- Break,
- Continue,
-}
-
struct ExtractorState<'a> {
path: Vec<u8>,
path_len: usize,
@@ -1060,8 +1056,8 @@ impl<'a> ExtractorState<'a> {
let entry = match self.read_dir.next() {
Some(entry) => entry,
None => match self.handle_end_of_directory()? {
- LoopState::Break => break, // done with root directory
- LoopState::Continue => continue,
+ ControlFlow::Break(()) => break, // done with root directory
+ ControlFlow::Continue(()) => continue,
},
};
@@ -1079,11 +1075,11 @@ impl<'a> ExtractorState<'a> {
Ok(())
}
- fn handle_end_of_directory(&mut self) -> Result<LoopState, Error> {
+ fn handle_end_of_directory(&mut self) -> Result<ControlFlow<()>, Error> {
// go up a directory:
self.read_dir = match self.read_dir_stack.pop() {
Some(r) => r,
- None => return Ok(LoopState::Break), // out of root directory
+ None => return Ok(ControlFlow::Break(())), // out of root directory
};
self.matches = self
@@ -1102,7 +1098,7 @@ impl<'a> ExtractorState<'a> {
self.extractor.leave_directory()?;
- Ok(LoopState::Continue)
+ Ok(ControlFlow::CONTINUE)
}
async fn handle_new_directory(
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 10/23] Cargo.toml: depend on proxmox-acme-rs
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (8 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 09/23] catalog shell: replace LoopState with ControlFlow Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 11/23] bump d/control Wolfgang Bumiller
` (14 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
Cargo.toml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Cargo.toml b/Cargo.toml
index 74ccf361..322e6ab9 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -79,6 +79,8 @@ zstd = { version = "0.4", features = [ "bindgen" ] }
nom = "5.1"
crossbeam-channel = "0.5"
+proxmox-acme-rs = "0.2.1"
+
[features]
default = []
#valgrind = ["valgrind_request"]
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 11/23] bump d/control
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (9 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 10/23] Cargo.toml: depend on proxmox-acme-rs Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 12/23] config::acl: make /system/certificates a valid path Wolfgang Bumiller
` (13 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
debian/control | 2 ++
1 file changed, 2 insertions(+)
diff --git a/debian/control b/debian/control
index d50ac970..05a3ccd3 100644
--- a/debian/control
+++ b/debian/control
@@ -17,6 +17,7 @@ Build-Depends: debhelper (>= 11),
librust-endian-trait-0.6+default-dev,
librust-env-logger-0.7+default-dev,
librust-flate2-1+default-dev,
+ librust-foreign-types-0.3+default-dev,
librust-futures-0.3+default-dev,
librust-h2-0.3+default-dev,
librust-h2-0.3+stream-dev,
@@ -42,6 +43,7 @@ Build-Depends: debhelper (>= 11),
librust-proxmox-0.11+default-dev (>= 0.11.1-~~),
librust-proxmox-0.11+sortable-macro-dev (>= 0.11.1-~~),
librust-proxmox-0.11+websocket-dev (>= 0.11.1-~~),
+ librust-proxmox-acme-rs-0.2+default-dev (>= 0.2.1-~~),
librust-proxmox-fuse-0.1+default-dev (>= 0.1.1-~~),
librust-pxar-0.10+default-dev (>= 0.10.1-~~),
librust-pxar-0.10+tokio-io-dev (>= 0.10.1-~~),
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 12/23] config::acl: make /system/certificates a valid path
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (10 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 11/23] bump d/control Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 13/23] add 'config file format' to tools::config Wolfgang Bumiller
` (12 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/config/acl.rs | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/config/acl.rs b/src/config/acl.rs
index 61e507ec..04d42854 100644
--- a/src/config/acl.rs
+++ b/src/config/acl.rs
@@ -308,7 +308,7 @@ pub fn check_acl_path(path: &str) -> Result<(), Error> {
return Ok(());
}
match components[1] {
- "disks" | "log" | "status" | "tasks" | "time" => {
+ "certificates" | "disks" | "log" | "status" | "tasks" | "time" => {
if components_len == 2 {
return Ok(());
}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 13/23] add 'config file format' to tools::config
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (11 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 12/23] config::acl: make /system/certificates a valid path Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 14/23] add node config Wolfgang Bumiller
` (11 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
This is a serde-based parser for the file format our perl
code read via `PVE::JSONSchema::parse_config`.
This will be used for the node config.
Some support for indexed arrays at the top level is
available but currently commented out and unused as this
is not really compatible with how we write our schemas,
since we store property strings as actual strings using an
object schema as *format* property in the `StringSchema`.
Ideally this could be changed in the future and we can
integrate the serde parsing model more easily without having
to convert between strings manually in the code.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/tools.rs | 1 +
src/tools/config/de.rs | 656 ++++++++++++++++++++++++++++++++++++++++
src/tools/config/mod.rs | 89 ++++++
src/tools/config/ser.rs | 642 +++++++++++++++++++++++++++++++++++++++
4 files changed, 1388 insertions(+)
create mode 100644 src/tools/config/de.rs
create mode 100644 src/tools/config/mod.rs
create mode 100644 src/tools/config/ser.rs
diff --git a/src/tools.rs b/src/tools.rs
index 890db826..25323881 100644
--- a/src/tools.rs
+++ b/src/tools.rs
@@ -23,6 +23,7 @@ pub mod async_io;
pub mod borrow;
pub mod cert;
pub mod compression;
+pub mod config;
pub mod cpio;
pub mod daemon;
pub mod disks;
diff --git a/src/tools/config/de.rs b/src/tools/config/de.rs
new file mode 100644
index 00000000..f7d9b79b
--- /dev/null
+++ b/src/tools/config/de.rs
@@ -0,0 +1,656 @@
+use std::collections::hash_map::{self, HashMap};
+
+use nom::{
+ branch::alt,
+ bytes::complete::tag,
+ character::complete::{alpha1, alphanumeric1, char, multispace0},
+ combinator::recognize,
+ multi::many0,
+ sequence::{delimited, pair},
+};
+use serde::de::{self, DeserializeSeed, IntoDeserializer, MapAccess, SeqAccess, Visitor};
+use serde::{forward_to_deserialize_any, Deserialize};
+
+use proxmox::api::schema::{parse_simple_value, ArraySchema, ObjectSchemaType, Schema};
+
+use super::Error;
+
+type IResult<I, O, E = nom::error::VerboseError<I>> = Result<(I, O), nom::Err<E>>;
+
+type ObjSchemaType = &'static (dyn ObjectSchemaType + Send + Sync + 'static);
+
+impl de::Error for Error {
+ fn custom<T>(msg: T) -> Self
+ where
+ T: std::fmt::Display,
+ {
+ Error::Custom(msg.to_string())
+ }
+}
+
+/// Top level (line-by-line) parser for our old pve-style config files.
+///
+/// The top level parser is responsible for fetching lines and splitting the `key: value` parts.
+/// It has 2 jobs:
+/// 1) Act as a `MapAccess` implementation in serde and forward the `value` to the 2nd level parser.
+/// 2) Collect values belonging to arrays separately and insert them into the object schema at
+/// the very end.
+///
+/// This of course means that the top level parser only ever handles object schemas.
+struct TopLevelDeserializer<'de> {
+ input: TopLevelInput<'de>,
+ schema: ObjSchemaType,
+
+ // 'current' and 'current_array' could be turned into one 3-state enum
+ current: Option<CurrentProperty<'de>>,
+ arrays: ArrayState<'de>,
+ current_array: Option<TopLevelArrayDeserializer<'de>>,
+}
+
+/// Filled by `MapAccess::next_key_seed` with the current line's info.
+struct CurrentProperty<'de> {
+ key: &'de str,
+ value: &'de str,
+ schema: Option<&'static Schema>,
+}
+
+/// The top level parser's input state is split out for borrowing purposes.
+struct TopLevelInput<'de> {
+ input: std::str::Lines<'de>,
+ line: usize,
+ comments: Vec<&'de str>,
+}
+
+impl<'de> TopLevelInput<'de> {
+ fn next_line(&mut self) -> Option<&'de str> {
+ loop {
+ let line = self.input.next()?.trim_start();
+ self.line += 1;
+ if !line.is_empty() && !line.starts_with('#') {
+ return Some(line.trim());
+ }
+ self.comments.push(line);
+ }
+ }
+}
+
+/// This is used for top-level arrays for which the elements have been collected by the
+/// `TopLevelDeserializer`.
+///
+/// When going through the accumulated arrays in the `TopLevelDeserializer`, it produces an
+/// instance of this struct and hands off deserialization to it.
+struct TopLevelArrayDeserializer<'de> {
+ /// We keep this for error messages.
+ key: &'de str,
+
+ schema: &'static ArraySchema,
+
+ values: std::vec::IntoIter<(usize, &'de str)>,
+}
+
+enum ArrayState<'de> {
+ /// For each array key we accumulate the values to process at the end in order to fit serde's
+ /// parsing model. We store the index, value and, for convenience, the schema.
+ Accumulating(HashMap<&'de str, (Vec<(usize, &'de str)>, &'static ArraySchema)>),
+
+ /// At the end of the file we iterate through the hashmap:
+ Handling(hash_map::IntoIter<&'de str, (Vec<(usize, &'de str)>, &'static ArraySchema)>),
+
+ Done,
+}
+
+impl<'de> TopLevelDeserializer<'de> {
+ pub fn from_str(input: &'de str, schema: &'static Schema) -> Result<Self, Error> {
+ match schema {
+ Schema::Object(schema) => Ok(Self {
+ input: TopLevelInput {
+ input: input.lines(),
+ line: 0,
+ comments: Vec::new(),
+ },
+ schema,
+ current: None,
+ arrays: ArrayState::Accumulating(HashMap::default()),
+ current_array: None,
+ }),
+ _ => Err(Error::BadSchema("toplevel schema must be an ObjectSchema")),
+ }
+ }
+
+ /*
+ * Should we generally parse into a wrapper struct which keeps comments around?
+ pub fn take_comments(&mut self) -> Vec<&'de str> {
+ std::mem::take(&mut self.input.comments)
+ }
+ */
+}
+
+impl<'de, 'a> de::Deserializer<'de> for &'a mut TopLevelDeserializer<'de> {
+ type Error = Error;
+
+ fn deserialize_any<V>(self, visitor: V) -> Result<V::Value, Error>
+ where
+ V: Visitor<'de>,
+ {
+ // At the top level this is always an object schema, so we forward everything to `map`:
+ self.deserialize_map(visitor)
+ }
+
+ fn deserialize_map<V>(self, visitor: V) -> Result<V::Value, Error>
+ where
+ V: Visitor<'de>,
+ {
+ visitor.visit_map(self)
+ }
+
+ // forward the rest as well:
+ forward_to_deserialize_any! {
+ bool i8 i16 i32 i64 i128 u8 u16 u32 u64 u128 f32 f64 char str string
+ bytes byte_buf option unit unit_struct newtype_struct seq tuple
+ tuple_struct struct enum identifier ignored_any
+ }
+}
+
+impl<'de, 'a> MapAccess<'de> for &'a mut TopLevelDeserializer<'de> {
+ type Error = Error;
+
+ fn next_key_seed<K>(&mut self, seed: K) -> Result<Option<K::Value>, Error>
+ where
+ K: DeserializeSeed<'de>,
+ {
+ loop {
+ let array_map = match &mut self.arrays {
+ ArrayState::Accumulating(map) => map,
+ ArrayState::Handling(iter) => match iter.next() {
+ Some((key, (mut values, schema))) => {
+ values.sort_by(|a, b| a.0.cmp(&b.0));
+ self.current_array = Some(TopLevelArrayDeserializer {
+ key,
+ schema,
+ values: values.into_iter(),
+ });
+ return seed.deserialize(key.into_deserializer()).map(Some);
+ }
+ None => {
+ self.arrays = ArrayState::Done;
+ return Ok(None);
+ }
+ },
+ ArrayState::Done => return Ok(None),
+ };
+
+ let input = match self.input.next_line() {
+ Some(line) => line,
+ None => {
+ self.arrays = ArrayState::Handling(std::mem::take(array_map).into_iter());
+ continue;
+ }
+ };
+
+ // Split the line into key and value. `value` is the "rest" of the input, the "pair" is the
+ // key and the colon.
+ let (value, (key, _)) =
+ pair(identifier, delimited(multispace0, char(':'), multispace0))(input)
+ .map_err(|err| nom_err(input, "key", err))?;
+
+ // Array handling:
+ /*
+ * Enabling this without special schema options *will* break hardcoded index-suffixed
+ * "manual" arrays.
+ *
+ if let Some((key, index)) = array_identifier(key)? {
+ match self.schema.lookup(key) {
+ Some((_optional, Schema::Array(schema))) => {
+ array_map
+ .entry(key)
+ .or_insert_with(|| (Vec::new(), schema))
+ .0
+ .push((index, value));
+ continue;
+ }
+ Some(_) => return Err(Error::NotAnArray(key.to_owned())),
+ None => {
+ if self.schema.additional_properties() {
+ return Err(Error::AdditionalArray(key.to_owned()));
+ }
+ return Err(Error::UnexpectedKey(key.to_owned()));
+ }
+ }
+ }
+ */
+
+ match self.schema.lookup(key) {
+ Some((_optional, schema)) => {
+ self.current = Some(CurrentProperty {
+ key,
+ value,
+ schema: Some(schema),
+ });
+ }
+ None => {
+ if self.schema.additional_properties() {
+ self.current = Some(CurrentProperty {
+ key,
+ value,
+ schema: None,
+ });
+ }
+ }
+ }
+
+ return seed.deserialize(key.into_deserializer()).map(Some);
+ }
+ }
+
+ fn next_value_seed<V>(&mut self, seed: V) -> Result<V::Value, Error>
+ where
+ V: DeserializeSeed<'de>,
+ {
+ if let Some(current) = self.current.take() {
+ return de_seed(current.value, Some(current.key), current.schema, seed);
+ }
+
+ if let Some(mut current) = self.current_array.take() {
+ return seed.deserialize(&mut current);
+ }
+
+ Err(Error::BadState("missing current property"))
+ }
+}
+
+impl<'de, 'a> de::Deserializer<'de> for &'a mut TopLevelArrayDeserializer<'de> {
+ type Error = Error;
+
+ fn deserialize_any<V>(self, visitor: V) -> Result<V::Value, Error>
+ where
+ V: Visitor<'de>,
+ {
+ self.deserialize_seq(visitor)
+ }
+
+ fn deserialize_seq<V>(self, visitor: V) -> Result<V::Value, Error>
+ where
+ V: Visitor<'de>,
+ {
+ visitor.visit_seq(self)
+ }
+
+ forward_to_deserialize_any! {
+ bool i8 i16 i32 i64 i128 u8 u16 u32 u64 u128 f32 f64 char str string
+ bytes byte_buf option unit unit_struct newtype_struct tuple
+ tuple_struct map struct enum identifier ignored_any
+ }
+}
+
+impl<'de, 'a> SeqAccess<'de> for &'a mut TopLevelArrayDeserializer<'de> {
+ type Error = Error;
+
+ fn next_element_seed<V>(&mut self, seed: V) -> Result<Option<V::Value>, Error>
+ where
+ V: DeserializeSeed<'de>,
+ {
+ match self.values.next() {
+ Some((_index, input)) => {
+ de_seed(input, Some(self.key), Some(self.schema.items), seed).map(Some)
+ }
+ None => Ok(None),
+ }
+ }
+}
+
+/// Deserialize values of a fixed type while allowing option types. Do not use this genericly
+/// because it drops all the specific info and only goes over `deserialize_any`.
+struct SomeDeserializer<T>(T);
+
+impl<'de, T: de::Deserializer<'de>> de::Deserializer<'de> for SomeDeserializer<T> {
+ type Error = T::Error;
+
+ forward_to_deserialize_any! {
+ bool i8 i16 i32 i64 i128 u8 u16 u32 u64 u128 f32 f64 char str string
+ bytes byte_buf unit unit_struct newtype_struct seq tuple
+ tuple_struct map struct enum identifier ignored_any
+ }
+
+ fn deserialize_any<V>(self, visitor: V) -> Result<V::Value, Self::Error>
+ where
+ V: Visitor<'de>,
+ {
+ self.0.deserialize_any(visitor)
+ }
+
+ fn deserialize_option<V>(self, visitor: V) -> Result<V::Value, Self::Error>
+ where
+ V: Visitor<'de>,
+ {
+ visitor.visit_some(self.0)
+ }
+}
+
+/// The most "common" deserialization code path for object types.
+///
+/// From this point on, objects become property strings and arrays become semicolon, zero, space or
+/// comma separated values. Though commas can only appear if the previous "layer" was not already a
+/// property string.
+fn de_seed<'de, V>(
+ input: &'de str,
+ key: Option<&str>,
+ schema: Option<&'static Schema>,
+ seed: V,
+) -> Result<V::Value, Error>
+where
+ V: DeserializeSeed<'de>,
+{
+ let key = key.unwrap_or("value");
+ match schema {
+ Some(Schema::Null) => Err(Error::NullSchema),
+ Some(Schema::Boolean(_schema)) => {
+ let value = match input.to_lowercase().as_str() {
+ "1" | "on" | "yes" | "true" => true,
+ "0" | "off" | "no" | "false" => false,
+ _ => return Err(Error::Custom(format!("invalid boolean value: {}", input))),
+ };
+ seed.deserialize(SomeDeserializer(value.into_deserializer()))
+ }
+ Some(schema @ Schema::Integer(_)) => seed
+ .deserialize(
+ parse_simple_value(input, schema)
+ .map_err(|err| Error::Custom(format!("bad {} - {}", key, err)))?,
+ )
+ .map_err(|err| Error::Custom(format!("bad {} - {}", key, err))),
+ Some(schema @ Schema::Number(_)) => seed
+ .deserialize(
+ parse_simple_value(input, schema)
+ .map_err(|err| Error::Custom(format!("bad {} - {}", key, err)))?,
+ )
+ .map_err(|err| Error::Custom(format!("bad {} - {}", key, err))),
+ Some(Schema::String(schema)) => {
+ schema
+ .check_constraints(input)
+ .map_err(|err| Error::Custom(format!("bad {} - {}", key, err)))?;
+ seed.deserialize(SomeDeserializer(input.into_deserializer()))
+ }
+ Some(Schema::Object(schema)) => {
+ let mut de = PropertyStringDeserializer {
+ input,
+ schema: &*schema,
+ default_key: schema.default_key,
+ current: None,
+ };
+ seed.deserialize(&mut de)
+ }
+ Some(Schema::AllOf(schema)) => {
+ let mut de = PropertyStringDeserializer {
+ input,
+ schema: &*schema,
+ default_key: None,
+ current: None,
+ };
+ seed.deserialize(&mut de)
+ }
+ Some(Schema::Array(schema)) => {
+ let mut de = ArrayDeserializer { input, schema };
+ seed.deserialize(&mut de)
+ }
+ None => seed.deserialize(SomeDeserializer(input.into_deserializer())),
+ }
+}
+
+pub fn from_str<'de, T>(input: &'de str, schema: &'static Schema) -> Result<T, Error>
+where
+ T: Deserialize<'de>,
+{
+ let mut deserializer = TopLevelDeserializer::from_str(input, schema)?;
+ let t: T = T::deserialize(&mut deserializer)?;
+ if deserializer.input.next_line().is_none() {
+ Ok(t)
+ } else {
+ Err(Error::TrailingCharacters)
+ }
+}
+
+pub fn from_slice<'de, T>(input: &'de [u8], schema: &'static Schema) -> Result<T, Error>
+where
+ T: Deserialize<'de>,
+{
+ from_str(
+ std::str::from_utf8(input).map_err(|_| Error::NonUtf8)?,
+ schema,
+ )
+}
+
+pub fn from_property_string<'de, T>(input: &'de str, schema: &'static Schema) -> Result<T, Error>
+where
+ T: Deserialize<'de>,
+{
+ let (schema, default_key): (ObjSchemaType, Option<&'static str>) = match schema {
+ Schema::Object(obj) => (obj as _, obj.default_key),
+ Schema::AllOf(obj) => (obj as _, None),
+ _ => {
+ return Err(Error::BadSchema(
+ "cannot deserialize non-object from a property string",
+ ));
+ }
+ };
+
+ T::deserialize(&mut PropertyStringDeserializer {
+ input,
+ schema,
+ default_key,
+ current: None,
+ })
+}
+
+fn identifier(i: &str) -> IResult<&str, &str> {
+ recognize(pair(
+ alt((alpha1, tag("_"))),
+ many0(alt((alphanumeric1, tag("_")))),
+ ))(i)
+}
+
+/*
+fn array_identifier(i: &str) -> Result<Option<(&str, usize)>, Error> {
+ if let Some(last_nondigit) = i.rfind(|c: char| !c.is_ascii_digit()) {
+ if last_nondigit != (i.len() - 1) {
+ return Ok(Some((
+ &i[..=last_nondigit],
+ i[(last_nondigit + 1)..]
+ .parse::<usize>()
+ .map_err(|e| Error::Other(e.into()))?,
+ )));
+ }
+ }
+ Ok(None)
+}
+*/
+
+fn nom_err(input: &str, what: &str, res: nom::Err<nom::error::VerboseError<&str>>) -> Error {
+ match res {
+ nom::Err::Error(err) | nom::Err::Failure(err) => Error::Custom(format!(
+ "failed to parse {} - {}",
+ what,
+ nom::error::convert_error(input, err),
+ )),
+ err => Error::Custom(format!("failed to parse {} - {}", what, err)),
+ }
+}
+
+/// This is basically the "2nd tier" parser for our format, the comma separated `key=value` format.
+///
+/// Contrary to the `TopLevelParser` this is only used for the value part of a line and never
+/// contains multiple lines.
+///
+/// At this level, commas *always* separate values, and arrays need to use a different separator
+/// (space, semicolon or zero-byte).
+struct PropertyStringDeserializer<'de> {
+ input: &'de str,
+ schema: ObjSchemaType,
+ default_key: Option<&'static str>,
+ current: Option<CurrentProperty<'de>>,
+}
+
+impl<'de> PropertyStringDeserializer<'de> {
+ /// Returns the next value. The key may be optional due to "default keys".
+ fn next_property(&mut self) -> Option<(Option<&'de str>, &'de str)> {
+ if self.input.is_empty() {
+ return None;
+ }
+
+ let input = self.input;
+ let property = match self.input.find(',') {
+ Some(comma) => {
+ self.input = &input[(comma + 1)..];
+ &input[..comma]
+ }
+ None => {
+ self.input = "";
+ input
+ }
+ };
+
+ Some(match property.find('=') {
+ Some(eq) => (Some(&property[..eq]), &property[(eq + 1)..]),
+ None => (None, property),
+ })
+ }
+
+ /// Assert that we have a current property entry.
+ fn current_property(&mut self) -> Result<CurrentProperty<'de>, Error> {
+ self.current
+ .take()
+ .ok_or_else(|| Error::BadState("missing current property in property string"))
+ }
+}
+
+impl<'de, 'a> de::Deserializer<'de> for &'a mut PropertyStringDeserializer<'de> {
+ type Error = Error;
+
+ fn deserialize_any<V>(self, visitor: V) -> Result<V::Value, Error>
+ where
+ V: Visitor<'de>,
+ {
+ self.deserialize_map(visitor)
+ }
+
+ fn deserialize_map<V>(self, visitor: V) -> Result<V::Value, Error>
+ where
+ V: Visitor<'de>,
+ {
+ visitor.visit_map(self)
+ }
+
+ fn deserialize_option<V>(self, visitor: V) -> Result<V::Value, Error>
+ where
+ V: Visitor<'de>,
+ {
+ visitor.visit_some(self)
+ }
+
+ forward_to_deserialize_any! {
+ bool i8 i16 i32 i64 i128 u8 u16 u32 u64 u128 f32 f64 char str string
+ bytes byte_buf unit unit_struct newtype_struct seq tuple
+ tuple_struct struct enum identifier ignored_any
+ }
+}
+
+impl<'de, 'a> MapAccess<'de> for &'a mut PropertyStringDeserializer<'de> {
+ type Error = Error;
+
+ fn next_key_seed<K>(&mut self, seed: K) -> Result<Option<K::Value>, Error>
+ where
+ K: DeserializeSeed<'de>,
+ {
+ let (key, value) = match self.next_property() {
+ None => return Ok(None),
+ Some((None, value)) => match self.default_key {
+ Some(key) => (key, value),
+ None => return Err(Error::MissingKey),
+ },
+ Some((Some(key), value)) => (key, value),
+ };
+
+ let schema = match self.schema.lookup(key) {
+ Some((_optional, schema)) => Some(schema),
+ None => {
+ if self.schema.additional_properties() {
+ None
+ } else {
+ return Err(Error::UnexpectedKey(key.to_owned()));
+ }
+ }
+ };
+
+ self.current = Some(CurrentProperty { key, value, schema });
+
+ seed.deserialize(key.into_deserializer()).map(Some)
+ }
+
+ fn next_value_seed<V>(&mut self, seed: V) -> Result<V::Value, Error>
+ where
+ V: DeserializeSeed<'de>,
+ {
+ let current = self.current_property()?;
+ de_seed(current.value, Some(current.key), current.schema, seed)
+ }
+}
+
+/// This is the *2nd level* array deserializer handling a single line of elements separated by any
+/// of our standard separators: comma, semicolon, space or null-byte.
+struct ArrayDeserializer<'de> {
+ input: &'de str,
+ schema: &'static ArraySchema,
+}
+
+impl<'de, 'a> de::Deserializer<'de> for &'a mut ArrayDeserializer<'de> {
+ type Error = Error;
+
+ fn deserialize_any<V>(self, visitor: V) -> Result<V::Value, Error>
+ where
+ V: Visitor<'de>,
+ {
+ self.deserialize_seq(visitor)
+ }
+
+ fn deserialize_seq<V>(self, visitor: V) -> Result<V::Value, Error>
+ where
+ V: Visitor<'de>,
+ {
+ visitor.visit_seq(self)
+ }
+
+ forward_to_deserialize_any! {
+ bool i8 i16 i32 i64 i128 u8 u16 u32 u64 u128 f32 f64 char str string
+ bytes byte_buf option unit unit_struct newtype_struct tuple
+ tuple_struct map struct enum identifier ignored_any
+ }
+}
+
+impl<'de, 'a> SeqAccess<'de> for &'a mut ArrayDeserializer<'de> {
+ type Error = Error;
+
+ fn next_element_seed<V>(&mut self, seed: V) -> Result<Option<V::Value>, Error>
+ where
+ V: DeserializeSeed<'de>,
+ {
+ if self.input.is_empty() {
+ return Ok(None);
+ }
+
+ let input = match self
+ .input
+ .find(|c: char| c == ',' || c == ';' || c.is_ascii_whitespace())
+ {
+ Some(pos) => {
+ let value = &self.input[..pos];
+ self.input = self.input[(pos + 1)..].trim_start();
+ value
+ }
+ None => {
+ let value = self.input.trim();
+ self.input = "";
+ value
+ }
+ };
+
+ de_seed(input, None, Some(&self.schema.items), seed).map(Some)
+ }
+}
diff --git a/src/tools/config/mod.rs b/src/tools/config/mod.rs
new file mode 100644
index 00000000..46a90674
--- /dev/null
+++ b/src/tools/config/mod.rs
@@ -0,0 +1,89 @@
+//! Our 'key: value' config format.
+
+pub mod de;
+pub mod ser;
+
+#[doc(inline)]
+pub use de::{from_property_string, from_slice, from_str};
+
+#[doc(inline)]
+pub use ser::{to_bytes, to_property_string, to_writer};
+
+// Note: we need an error type since we need to implement serde traits for it.
+
+/// Config file format or property string parsing error.
+#[derive(thiserror::Error, Debug)]
+pub enum Error {
+ // Common:
+ #[error("invalid schema: Null")]
+ NullSchema,
+
+ #[error("bad schema: {0}")]
+ BadSchema(&'static str),
+
+ #[error("config contains illegal characters (non-utf8)")]
+ NonUtf8,
+
+ // Deserialization:
+ #[error("invalid trailing characters after configuration")]
+ TrailingCharacters,
+
+ #[error("unexpected EOF, expected {0}")]
+ Eof(&'static str),
+
+ #[error("not an array: {0}")]
+ NotAnArray(String),
+
+ // Because it's stupid for the parser.
+ #[error("array keys not allowed as additional property")]
+ AdditionalArray(String),
+
+ #[error("unexpected value, expected {0}")]
+ Type(&'static str),
+
+ #[error("unexpected key '{0}' and schema does not allow additional properties")]
+ UnexpectedKey(String),
+
+ #[error("missing key and schema does not define a default key")]
+ MissingKey,
+
+ #[error("integer literal out of range for '{0}'")]
+ IntegerOutOfRange(String),
+
+ #[error("ObjectSchema found within a property-string value for '{0}'")]
+ ObjectInPropertyString(String),
+
+ #[error("parse error: {0}")]
+ Custom(String),
+
+ #[error("deserialization error: {0}")]
+ BadState(&'static str),
+
+ #[error(transparent)]
+ Other(#[from] anyhow::Error),
+
+ // Serialization:
+ #[error("json serialization failed: {0}")]
+ Json(#[from] serde_json::Error),
+
+ #[error("cannot serialize non-object types as config file")]
+ NotAnObject,
+
+ #[error("schema expected type {0} but data type was {1}")]
+ SchemaError(&'static str, &'static str),
+
+ #[error("value of type type {0} did not fit into a type {1}")]
+ NumericRange(&'static str, &'static str),
+
+ #[error("failed to write serialized output: {0}")]
+ Io(#[from] std::io::Error),
+
+ #[error("bad key type trying to serialize a 'map': {0}")]
+ BadKeyType(String),
+
+ #[error("bad value type serializing to config format: {0}")]
+ BadValueType(&'static str),
+
+ #[error("type is nested too much and cannot be represented as a config file")]
+ TooComplex,
+}
diff --git a/src/tools/config/ser.rs b/src/tools/config/ser.rs
new file mode 100644
index 00000000..f684d1a6
--- /dev/null
+++ b/src/tools/config/ser.rs
@@ -0,0 +1,642 @@
+use std::borrow::Cow;
+use std::mem::replace;
+
+use serde::{ser, Serialize};
+use serde_json::Value;
+
+use proxmox::api::schema::{ArraySchema, ObjectSchemaType, Schema};
+
+use super::Error;
+
+impl ser::Error for Error {
+ fn custom<T>(msg: T) -> Self
+ where
+ T: std::fmt::Display,
+ {
+ Error::Custom(msg.to_string())
+ }
+}
+
+pub fn to_bytes<T: Serialize>(value: &T, schema: &'static Schema) -> Result<Vec<u8>, Error> {
+ let mut out = Vec::<u8>::new();
+ to_writer(value, schema, &mut out)?;
+ Ok(out)
+}
+
+pub fn to_writer<T: Serialize>(
+ value: &T,
+ schema: &'static Schema,
+ output: &mut dyn std::io::Write,
+) -> Result<(), Error> {
+ let obj: &'static dyn ObjectSchemaType = match schema {
+ Schema::Object(obj) => obj as _,
+ Schema::AllOf(obj) => obj as _,
+ _ => {
+ return Err(Error::BadSchema(
+ "config file format only accepts object schemas at the top level",
+ ))
+ }
+ };
+
+ value.serialize(&mut TopLevelSerializer::new(output, obj))
+}
+
+pub fn to_property_string<T: Serialize>(
+ value: &T,
+ schema: &'static Schema,
+) -> Result<String, Error> {
+ if !matches!(schema, Schema::Object(_) | Schema::AllOf(_)) {
+ return Err(Error::BadSchema(
+ "cannot serialize non-object as property string",
+ ));
+ }
+
+ let mut bytes = Vec::<u8>::new();
+ value.serialize(&mut LineSerializer::new_toplevel(
+ &mut bytes,
+ false,
+ Some(schema),
+ ))?;
+
+ String::from_utf8(bytes).map_err(|_| Error::NonUtf8)
+}
+
+/// This is the top level of our config file serializing each 'key' of an object into one line,
+/// handling arrays by suffixing them with indices.
+struct TopLevelSerializer<'out> {
+ output: &'out mut dyn std::io::Write,
+ schema: &'static dyn ObjectSchemaType,
+ current_key: Option<Cow<'static, str>>,
+}
+
+impl<'out> TopLevelSerializer<'out> {
+ fn new(output: &'out mut dyn std::io::Write, schema: &'static dyn ObjectSchemaType) -> Self {
+ Self {
+ output,
+ schema,
+ current_key: None,
+ }
+ }
+}
+
+macro_rules! not_an_object {
+ () => {};
+ ( $name:ident<$($generic:ident)+>($($args:tt)*) ($ret:ty) $($rest:tt)* ) => {
+ fn $name<$($generic: ?Sized + Serialize)+>(self, $($args)*) -> Result<$ret, Error> {
+ Err(Error::NotAnObject)
+ }
+
+ not_an_object!{ $($rest)* }
+ };
+ ( $name:ident($($args:tt)*) ($ret:ty) $($rest:tt)* ) => {
+ fn $name(self, $($args)*) -> Result<$ret, Error> {
+ Err(Error::NotAnObject)
+ }
+
+ not_an_object!{ $($rest)* }
+ };
+}
+
+impl<'a, 'out> ser::Serializer for &'a mut TopLevelSerializer<'out> {
+ type Ok = ();
+ type Error = Error;
+
+ type SerializeSeq = ser::Impossible<Self::Ok, Self::Error>;
+ type SerializeTuple = ser::Impossible<Self::Ok, Self::Error>;
+ type SerializeTupleStruct = ser::Impossible<Self::Ok, Self::Error>;
+ type SerializeTupleVariant = ser::Impossible<Self::Ok, Self::Error>;
+ type SerializeMap = Self;
+ type SerializeStruct = Self;
+ type SerializeStructVariant = ser::Impossible<Self::Ok, Self::Error>;
+
+ not_an_object! {
+ serialize_bool(_: bool)(())
+ serialize_char(_: char)(())
+ serialize_i8(_: i8)(())
+ serialize_i16(_: i16)(())
+ serialize_i32(_: i32)(())
+ serialize_i64(_: i64)(())
+ serialize_u8(_: u8)(())
+ serialize_u16(_: u16)(())
+ serialize_u32(_: u32)(())
+ serialize_u64(_: u64)(())
+ serialize_f32(_: f32)(())
+ serialize_f64(_: f64)(())
+ serialize_str(_: &str)(())
+ serialize_bytes(_: &[u8])(())
+ serialize_unit_struct(_: &'static str)(())
+ serialize_none()(())
+ serialize_some<T>(_: &T)(())
+ serialize_unit()(())
+ serialize_unit_variant(
+ _name: &'static str,
+ _variant_index: u32,
+ _variant: &'static str,
+ )(())
+ serialize_newtype_struct<T>(_name: &'static str, _value: &T)(())
+ serialize_newtype_variant<T>(
+ _name: &'static str,
+ _variant_index: u32,
+ _variant: &'static str,
+ _value: &T,
+ )(())
+ serialize_seq(_len: Option<usize>)(ser::Impossible<Self::Ok, Self::Error>)
+ serialize_tuple(_len: usize)(ser::Impossible<Self::Ok, Self::Error>)
+ serialize_tuple_struct(
+ _name: &'static str,
+ _len: usize,
+ )(ser::Impossible<Self::Ok, Self::Error>)
+ serialize_tuple_variant(
+ _name: &'static str,
+ _variant_index: u32,
+ _variant: &'static str,
+ _len: usize,
+ )(ser::Impossible<Self::Ok, Self::Error>)
+ serialize_struct_variant(
+ _name: &'static str,
+ _variant_index: u32,
+ _variant: &'static str,
+ _len: usize,
+ )(ser::Impossible<Self::Ok, Self::Error>)
+ }
+
+ fn serialize_map(self, _len: Option<usize>) -> Result<Self, Error> {
+ Ok(self)
+ }
+
+ fn serialize_struct(self, _name: &'static str, _len: usize) -> Result<Self, Error> {
+ Ok(self)
+ }
+}
+
+impl<'a, 'out> ser::SerializeMap for &'a mut TopLevelSerializer<'out> {
+ type Ok = ();
+ type Error = Error;
+
+ fn serialize_key<T: ?Sized>(&mut self, key: &T) -> Result<(), Self::Error>
+ where
+ T: Serialize,
+ {
+ match serde_json::to_value(key)? {
+ Value::String(key) => {
+ self.current_key = Some(Cow::Owned(key));
+ Ok(())
+ }
+ other => Err(Error::BadKeyType(
+ serde_json::to_string(&other).unwrap_or_else(|_| "???".to_string()),
+ )),
+ }
+ }
+
+ fn serialize_value<T: ?Sized>(&mut self, value: &T) -> Result<(), Self::Error>
+ where
+ T: Serialize,
+ {
+ let key = self
+ .current_key
+ .take()
+ .ok_or_else(|| Error::BadState("serialize_value called without serialize_key"))?;
+ self.write_value_ln(key, value)?;
+ Ok(())
+ }
+
+ fn end(self) -> Result<(), Error> {
+ Ok(())
+ }
+}
+
+impl<'a, 'out> ser::SerializeStruct for &'a mut TopLevelSerializer<'out> {
+ type Ok = ();
+ type Error = Error;
+
+ fn serialize_field<T: ?Sized>(
+ &mut self,
+ key: &'static str,
+ value: &T,
+ ) -> Result<(), Self::Error>
+ where
+ T: Serialize,
+ {
+ self.write_value_ln(Cow::Borrowed(key), value)?;
+ Ok(())
+ }
+
+ fn end(self) -> Result<(), Error> {
+ Ok(())
+ }
+}
+
+impl<'out> TopLevelSerializer<'out> {
+ fn write_value_ln<T: ?Sized>(&mut self, key: Cow<'static, str>, value: &T) -> Result<(), Error>
+ where
+ T: Serialize,
+ {
+ let (optional, schema) = match self.schema.lookup(&key) {
+ Some((o, s)) => (o, Some(s)),
+ None => (true, None),
+ };
+
+ if let Some(Schema::Array(ArraySchema { items, .. })) = schema {
+ // When serializing arrays at the top level we need to decide whether to serialize them
+ // as a single line or multiple.
+ //
+ // Since we don't attach this sort of information to the schema (yet), our best bet is
+ // to just check the contained type. If it is a more complex type (array or object),
+ // we'll split it up:
+ match items {
+ Schema::Array(_) | Schema::Object(_) | Schema::AllOf(_) => {
+ return value.serialize(&mut LineSerializer::new_multiline_array(
+ self.output,
+ optional,
+ schema,
+ key,
+ ))
+ }
+ _ => (), // use regular deserialization otherwise
+ }
+ }
+
+ self.output.write_all(key.as_bytes())?;
+ self.output.write_all(b": ")?;
+ value.serialize(&mut LineSerializer::new_toplevel(
+ self.output,
+ optional,
+ schema,
+ ))?;
+ self.output.write_all(b"\n")?;
+ Ok(())
+ }
+}
+
+/// This is the second level serializer.
+///
+/// At this point arrays are semicolon separated values, structs/maps are property strings, and
+/// anything else is just "printed".
+struct LineSerializer<'out> {
+ output: &'out mut dyn std::io::Write,
+ optional: bool,
+ schema: Option<&'static Schema>,
+
+ /// When serializing an array containing property strings, this key is used to produce multiple
+ /// lines suffixed with an index for each array element.
+ array_key: Option<(Cow<'static, str>, usize)>,
+
+ /// Used while serializing objects.
+ current_key: Option<String>,
+
+ // This is to prevent invalid nesting of arrays and property strings:
+ is_array: bool,
+ is_object: bool,
+
+ // This is the state for whether we need to already place commas or semicolons during object or
+ // array serialization
+ in_array: bool,
+ in_object: bool,
+}
+
+impl<'out> LineSerializer<'out> {
+ /// The first line level allows serializing more complex structures such as arrays or maps.
+ fn new_toplevel(
+ output: &'out mut dyn std::io::Write,
+ optional: bool,
+ schema: Option<&'static Schema>,
+ ) -> Self {
+ Self {
+ output,
+ array_key: None,
+ optional,
+ schema,
+ current_key: None,
+ is_array: false,
+ is_object: false,
+ in_array: false,
+ in_object: false,
+ }
+ }
+
+ /// Multi-line arrays are also handled by this serializer because the `Serializer`
+ /// implementation is so tedious...
+ fn new_multiline_array(
+ output: &'out mut dyn std::io::Write,
+ optional: bool,
+ schema: Option<&'static Schema>,
+ array_key: Cow<'static, str>,
+ ) -> Self {
+ let mut this = Self::new_toplevel(output, optional, schema);
+ this.array_key = Some((array_key, 0));
+ this
+ }
+
+ fn serialize_object_value<T: ?Sized + Serialize>(
+ &mut self,
+ key: Cow<'static, str>,
+ value: &T,
+ ) -> Result<(), Error> {
+ let next_schema = match self.schema {
+ Some(Schema::Object(schema)) => schema.lookup(&key),
+ Some(Schema::AllOf(schema)) => schema.lookup(&key),
+ Some(_) => {
+ return Err(Error::BadSchema(
+ "struct or map with non-object schema type",
+ ))
+ }
+ None => None,
+ };
+
+ let (optional, schema) = match next_schema {
+ Some((optional, schema)) => (optional, Some(schema)),
+ None => (true, None),
+ };
+
+ if replace(&mut self.in_object, true) {
+ self.output.write_all(b",")?;
+ }
+ self.output.write_all(key.as_bytes())?;
+ self.output.write_all(b"=")?;
+
+ {
+ let mut next = LineSerializer {
+ output: self.output,
+ optional,
+ schema,
+ array_key: None,
+ current_key: None,
+ is_array: self.is_array,
+ is_object: self.is_object,
+ in_array: self.in_array,
+ in_object: self.in_object,
+ };
+ value.serialize(&mut next)?;
+ }
+ Ok(())
+ }
+}
+
+macro_rules! forward_simple {
+ ( $( $name:ident($ty:ty) -> $handler:ident )* ) => {
+ $(
+ fn $name(self, v: $ty) -> Result<(), Error> {
+ self.$handler(v.into())
+ }
+ )*
+ };
+}
+
+macro_rules! bad_value_type {
+ () => {};
+ ( $name:ident<$($generic:ident)+>($($args:tt)*) ($ret:ty) ($error:tt) $($rest:tt)* ) => {
+ fn $name<$($generic: ?Sized + Serialize)+>(self, $($args)*) -> Result<$ret, Error> {
+ Err(Error::BadValueType($error))
+ }
+
+ bad_value_type!{ $($rest)* }
+ };
+ ( $name:ident($($args:tt)*) ($ret:ty) ($error:tt) $($rest:tt)* ) => {
+ fn $name(self, $($args)*) -> Result<$ret, Error> {
+ Err(Error::BadValueType($error))
+ }
+
+ bad_value_type!{ $($rest)* }
+ };
+}
+
+impl<'a, 'out> ser::Serializer for &'a mut LineSerializer<'out> {
+ type Ok = ();
+ type Error = Error;
+
+ type SerializeSeq = Self;
+ type SerializeTuple = ser::Impossible<Self::Ok, Self::Error>;
+ type SerializeTupleStruct = ser::Impossible<Self::Ok, Self::Error>;
+ type SerializeTupleVariant = ser::Impossible<Self::Ok, Self::Error>;
+ type SerializeMap = Self;
+ type SerializeStruct = Self;
+ type SerializeStructVariant = ser::Impossible<Self::Ok, Self::Error>;
+
+ forward_simple! {
+ serialize_i8(i8) -> serialize_i64
+ serialize_i16(i16) -> serialize_i64
+ serialize_i32(i32) -> serialize_i64
+ serialize_u8(u8) -> serialize_u64
+ serialize_u16(u16) -> serialize_u64
+ serialize_u32(u32) -> serialize_u64
+ serialize_f32(f32) -> serialize_f64
+ }
+
+ bad_value_type! {
+ serialize_bytes(_: &[u8])(()) ("byte slice")
+ serialize_unit_struct(_: &'static str)(()) ("unit struct")
+ serialize_unit()(()) ("unit")
+ serialize_unit_variant(
+ _name: &'static str,
+ _variant_index: u32,
+ _variant: &'static str,
+ )(()) ("unit variant")
+ serialize_tuple(_len: usize)(ser::Impossible<Self::Ok, Self::Error>) ("tuple")
+ serialize_tuple_struct(
+ _name: &'static str,
+ _len: usize,
+ )(ser::Impossible<Self::Ok, Self::Error>) ("tuple struct")
+ serialize_tuple_variant(
+ _name: &'static str,
+ _variant_index: u32,
+ _variant: &'static str,
+ _len: usize,
+ )(ser::Impossible<Self::Ok, Self::Error>) ("tuple variant")
+ serialize_struct_variant(
+ _name: &'static str,
+ _variant_index: u32,
+ _variant: &'static str,
+ _len: usize,
+ )(ser::Impossible<Self::Ok, Self::Error>) ("struct variant")
+ }
+
+ fn serialize_i64(self, v: i64) -> Result<(), Error> {
+ match self.schema {
+ None | Some(Schema::Integer(_)) | Some(Schema::Number(_)) => {
+ Ok(write!(self.output, "{}", v)?)
+ }
+ Some(_) => Err(Error::BadSchema("integer schema with non-integer value")),
+ }
+ }
+
+ fn serialize_u64(self, v: u64) -> Result<(), Error> {
+ match self.schema {
+ None | Some(Schema::Integer(_)) | Some(Schema::Number(_)) => {
+ Ok(write!(self.output, "{}", v)?)
+ }
+ Some(_) => Err(Error::BadSchema("integer schema with non-integer value")),
+ }
+ }
+
+ fn serialize_f64(self, v: f64) -> Result<(), Error> {
+ match self.schema {
+ None | Some(Schema::Number(_)) => Ok(write!(self.output, "{}", v)?),
+ Some(_) => Err(Error::BadSchema(
+ "non-number schema with floating poing value",
+ )),
+ }
+ }
+
+ fn serialize_bool(self, v: bool) -> Result<(), Error> {
+ match self.schema {
+ None | Some(Schema::Boolean(_)) => {
+ Ok(self.output.write_all(if v { b"true" } else { b"false" })?)
+ }
+ Some(_) => Err(Error::BadSchema("non-boolean schema with boolean value")),
+ }
+ }
+
+ fn serialize_char(self, v: char) -> Result<(), Error> {
+ match self.schema {
+ None | Some(Schema::String(_)) => Ok(write!(self.output, "{}", v)?),
+ Some(_) => Err(Error::BadSchema("non-string schema with character value")),
+ }
+ }
+
+ fn serialize_str(self, v: &str) -> Result<(), Error> {
+ match self.schema {
+ None | Some(Schema::String(_)) => Ok(self.output.write_all(v.as_bytes())?),
+ Some(_) => Err(Error::BadSchema("non-string schema with string value")),
+ }
+ }
+
+ fn serialize_none(self) -> Result<(), Error> {
+ if self.optional {
+ return Ok(());
+ }
+
+ match self.schema {
+ None | Some(Schema::Null) => Ok(()),
+ Some(_) => Err(Error::BadSchema("encountered None at a non optional value")),
+ }
+ }
+
+ fn serialize_some<T: ?Sized + Serialize>(self, value: &T) -> Result<(), Error> {
+ value.serialize(self)
+ }
+
+ fn serialize_newtype_struct<T: ?Sized + Serialize>(
+ self,
+ _name: &'static str,
+ value: &T,
+ ) -> Result<(), Error> {
+ value.serialize(self)
+ }
+
+ fn serialize_newtype_variant<T: ?Sized + Serialize>(
+ self,
+ _name: &'static str,
+ _variant_index: u32,
+ _variant: &'static str,
+ value: &T,
+ ) -> Result<(), Error> {
+ value.serialize(self)
+ }
+
+ fn serialize_seq(mut self, _len: Option<usize>) -> Result<Self, Error> {
+ if self.is_array {
+ return Err(Error::TooComplex);
+ }
+
+ self.is_array = true;
+ Ok(self)
+ }
+
+ fn serialize_map(mut self, _len: Option<usize>) -> Result<Self, Error> {
+ if self.is_object {
+ return Err(Error::TooComplex);
+ }
+
+ self.is_object = true;
+ Ok(self)
+ }
+
+ fn serialize_struct(mut self, _name: &'static str, _len: usize) -> Result<Self, Error> {
+ if self.is_object {
+ return Err(Error::TooComplex);
+ }
+
+ self.is_object = true;
+ Ok(self)
+ }
+}
+
+impl<'a, 'out> ser::SerializeMap for &'a mut LineSerializer<'out> {
+ type Ok = ();
+ type Error = Error;
+
+ fn serialize_key<T: ?Sized>(&mut self, key: &T) -> Result<(), Self::Error>
+ where
+ T: Serialize,
+ {
+ match serde_json::to_value(key)? {
+ Value::String(s) => {
+ self.current_key = Some(s);
+ Ok(())
+ }
+ other => Err(Error::BadKeyType(
+ serde_json::to_string(&other).unwrap_or_else(|_| "???".to_string()),
+ )),
+ }
+ }
+
+ fn serialize_value<T: ?Sized>(&mut self, value: &T) -> Result<(), Self::Error>
+ where
+ T: Serialize,
+ {
+ let key = self
+ .current_key
+ .take()
+ .ok_or_else(|| Error::BadState("serialize_value called without serialize_key"))?;
+ self.serialize_object_value(Cow::Owned(key), value)
+ }
+
+ fn end(self) -> Result<(), Error> {
+ Ok(())
+ }
+}
+
+impl<'a, 'out> ser::SerializeStruct for &'a mut LineSerializer<'out> {
+ type Ok = ();
+ type Error = Error;
+
+ fn serialize_field<T: ?Sized>(
+ &mut self,
+ key: &'static str,
+ value: &T,
+ ) -> Result<(), Self::Error>
+ where
+ T: Serialize,
+ {
+ self.serialize_object_value(Cow::Borrowed(key), value)
+ }
+
+ fn end(self) -> Result<(), Error> {
+ Ok(())
+ }
+}
+
+impl<'a, 'out> ser::SerializeSeq for &'a mut LineSerializer<'out> {
+ type Ok = ();
+ type Error = Error;
+
+ fn serialize_element<T: ?Sized>(&mut self, value: &T) -> Result<(), Self::Error>
+ where
+ T: Serialize,
+ {
+ if let Some((ref key, ref mut index)) = self.array_key {
+ if replace(&mut self.in_array, true) {
+ write!(self.output, "\n{}{}: ", key, index)?;
+ *index += 1;
+ }
+ } else if replace(&mut self.in_array, true) {
+ self.output.write_all(b";")?;
+ }
+
+ value.serialize(&mut **self)
+ }
+
+ fn end(self) -> Result<(), Error> {
+ Ok(())
+ }
+}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 14/23] add node config
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (12 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 13/23] add 'config file format' to tools::config Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 15/23] add acme config Wolfgang Bumiller
` (10 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/config.rs | 1 +
src/config/node.rs | 225 +++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 226 insertions(+)
create mode 100644 src/config/node.rs
diff --git a/src/config.rs b/src/config.rs
index 37df2fd2..717829e2 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -19,6 +19,7 @@ pub mod acl;
pub mod cached_user_info;
pub mod datastore;
pub mod network;
+pub mod node;
pub mod remote;
pub mod sync;
pub mod tfa;
diff --git a/src/config/node.rs b/src/config/node.rs
new file mode 100644
index 00000000..c85b5128
--- /dev/null
+++ b/src/config/node.rs
@@ -0,0 +1,225 @@
+use std::fs::File;
+use std::time::Duration;
+
+use anyhow::{format_err, Error};
+use nix::sys::stat::Mode;
+use serde::{Deserialize, Serialize};
+
+use proxmox::api::api;
+use proxmox::api::schema::{self, Updater};
+use proxmox::tools::fs::{replace_file, CreateOptions};
+
+use crate::acme::AcmeClient;
+use crate::api2::types::{DNS_ALIAS_FORMAT, DNS_NAME_FORMAT, PROXMOX_SAFE_ID_FORMAT};
+use crate::config::acme::AccountName;
+
+const CONF_FILE: &str = configdir!("/node.cfg");
+const LOCK_FILE: &str = configdir!("/.node.cfg.lock");
+const LOCK_TIMEOUT: Duration = Duration::from_secs(5);
+
+pub fn read_lock() -> Result<File, Error> {
+ proxmox::tools::fs::open_file_locked(LOCK_FILE, LOCK_TIMEOUT, false)
+}
+
+pub fn write_lock() -> Result<File, Error> {
+ proxmox::tools::fs::open_file_locked(LOCK_FILE, LOCK_TIMEOUT, true)
+}
+
+/// Read the Node Config.
+pub fn config() -> Result<(NodeConfig, [u8; 32]), Error> {
+ let content =
+ proxmox::tools::fs::file_read_optional_string(CONF_FILE)?.unwrap_or_else(|| "".to_string());
+
+ let digest = openssl::sha::sha256(content.as_bytes());
+ let data: NodeConfig = crate::tools::config::from_str(&content, &NodeConfig::API_SCHEMA)?;
+
+ Ok((data, digest))
+}
+
+/// Write the Node Config, requires the write lock to be held.
+pub fn save_config(config: &NodeConfig) -> Result<(), Error> {
+ let raw = crate::tools::config::to_bytes(config, &NodeConfig::API_SCHEMA)?;
+
+ let backup_user = crate::backup::backup_user()?;
+ let options = CreateOptions::new()
+ .perm(Mode::from_bits_truncate(0o0640))
+ .owner(nix::unistd::ROOT)
+ .group(backup_user.gid);
+
+ replace_file(CONF_FILE, &raw, options)
+}
+
+#[api(
+ properties: {
+ "domain": { format: &DNS_NAME_FORMAT },
+ "alias": {
+ optional: true,
+ format: &DNS_ALIAS_FORMAT,
+ },
+ "plugin": {
+ optional: true,
+ format: &PROXMOX_SAFE_ID_FORMAT,
+ },
+ },
+ default_key: "domain",
+)]
+#[derive(Deserialize, Serialize)]
+/// A domain entry for an ACME certificate.
+pub struct AcmeDomain {
+ /// The domain to certify for.
+ pub domain: String,
+
+ /// The domain to use for challenges instead of the default acme challenge domain.
+ ///
+ /// This is useful if you use CNAME entries to redirect `_acme-challenge.*` domains to a
+ /// different DNS server.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub alias: Option<String>,
+
+ /// The plugin to use to validate this domain.
+ ///
+ /// Empty means standalone HTTP validation is used.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub plugin: Option<String>,
+}
+
+#[api(
+ properties: {
+ account: { type: AccountName },
+ }
+)]
+#[derive(Deserialize, Serialize)]
+/// The ACME configuration.
+///
+/// Currently only contains the name of the account use.
+pub struct AcmeConfig {
+ /// Account to use to acquire ACME certificates.
+ account: AccountName,
+}
+
+#[api(
+ properties: {
+ acme: {
+ optional: true,
+ type: String,
+ format: &schema::ApiStringFormat::PropertyString(&AcmeConfig::API_SCHEMA),
+ },
+ acmedomain0: {
+ type: String,
+ optional: true,
+ format: &schema::ApiStringFormat::PropertyString(&AcmeDomain::API_SCHEMA),
+ },
+ acmedomain1: {
+ type: String,
+ optional: true,
+ format: &schema::ApiStringFormat::PropertyString(&AcmeDomain::API_SCHEMA),
+ },
+ acmedomain2: {
+ type: String,
+ optional: true,
+ format: &schema::ApiStringFormat::PropertyString(&AcmeDomain::API_SCHEMA),
+ },
+ acmedomain3: {
+ type: String,
+ optional: true,
+ format: &schema::ApiStringFormat::PropertyString(&AcmeDomain::API_SCHEMA),
+ },
+ acmedomain4: {
+ type: String,
+ optional: true,
+ format: &schema::ApiStringFormat::PropertyString(&AcmeDomain::API_SCHEMA),
+ },
+ },
+)]
+#[derive(Deserialize, Serialize, Updater)]
+/// Node specific configuration.
+pub struct NodeConfig {
+ /// The acme account to use on this node.
+ #[serde(skip_serializing_if = "Updater::is_empty")]
+ acme: Option<String>,
+
+ /// ACME domain to get a certificate for for this node.
+ #[serde(skip_serializing_if = "Updater::is_empty")]
+ acmedomain0: Option<String>,
+
+ /// ACME domain to get a certificate for for this node.
+ #[serde(skip_serializing_if = "Updater::is_empty")]
+ acmedomain1: Option<String>,
+
+ /// ACME domain to get a certificate for for this node.
+ #[serde(skip_serializing_if = "Updater::is_empty")]
+ acmedomain2: Option<String>,
+
+ /// ACME domain to get a certificate for for this node.
+ #[serde(skip_serializing_if = "Updater::is_empty")]
+ acmedomain3: Option<String>,
+
+ /// ACME domain to get a certificate for for this node.
+ #[serde(skip_serializing_if = "Updater::is_empty")]
+ acmedomain4: Option<String>,
+}
+
+impl NodeConfig {
+ pub fn acme_config(&self) -> Option<Result<AcmeConfig, Error>> {
+ self.acme.as_deref().map(|config| -> Result<_, Error> {
+ Ok(crate::tools::config::from_property_string(
+ config,
+ &AcmeConfig::API_SCHEMA,
+ )?)
+ })
+ }
+
+ pub async fn acme_client(&self) -> Result<AcmeClient, Error> {
+ AcmeClient::load(
+ &self
+ .acme_config()
+ .ok_or_else(|| format_err!("no acme client configured"))??
+ .account,
+ )
+ .await
+ }
+
+ pub fn acme_domains(&self) -> AcmeDomainIter {
+ AcmeDomainIter::new(self)
+ }
+}
+
+pub struct AcmeDomainIter<'a> {
+ config: &'a NodeConfig,
+ index: usize,
+}
+
+impl<'a> AcmeDomainIter<'a> {
+ fn new(config: &'a NodeConfig) -> Self {
+ Self { config, index: 0 }
+ }
+}
+
+impl<'a> Iterator for AcmeDomainIter<'a> {
+ type Item = Result<AcmeDomain, crate::tools::config::Error>;
+
+ fn next(&mut self) -> Option<Self::Item> {
+ let domain = loop {
+ let index = self.index;
+ self.index += 1;
+
+ let domain = match index {
+ 0 => self.config.acmedomain0.as_deref(),
+ 1 => self.config.acmedomain1.as_deref(),
+ 2 => self.config.acmedomain2.as_deref(),
+ 3 => self.config.acmedomain3.as_deref(),
+ 4 => self.config.acmedomain4.as_deref(),
+ _ => return None,
+ };
+
+ if let Some(domain) = domain {
+ break domain;
+ }
+ };
+
+ Some(crate::tools::config::from_property_string(
+ domain,
+ &AcmeDomain::API_SCHEMA,
+ ))
+ }
+}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 15/23] add acme config
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (13 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 14/23] add node config Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 16/23] add async acme client implementation Wolfgang Bumiller
` (9 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/config.rs | 1 +
src/config/acme/mod.rs | 198 ++++++++++++++++++++
src/config/acme/plugin.rs | 380 ++++++++++++++++++++++++++++++++++++++
3 files changed, 579 insertions(+)
create mode 100644 src/config/acme/mod.rs
create mode 100644 src/config/acme/plugin.rs
diff --git a/src/config.rs b/src/config.rs
index 717829e2..94b7fb6c 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -16,6 +16,7 @@ use proxmox::try_block;
use crate::buildcfg;
pub mod acl;
+pub mod acme;
pub mod cached_user_info;
pub mod datastore;
pub mod network;
diff --git a/src/config/acme/mod.rs b/src/config/acme/mod.rs
new file mode 100644
index 00000000..ac409c20
--- /dev/null
+++ b/src/config/acme/mod.rs
@@ -0,0 +1,198 @@
+use std::collections::HashMap;
+use std::fmt;
+use std::path::Path;
+
+use anyhow::{bail, format_err, Error};
+use serde::{Deserialize, Serialize};
+
+use proxmox::api::api;
+use proxmox::sys::error::SysError;
+
+use crate::api2::types::{PROXMOX_SAFE_ID_FORMAT, PROXMOX_SAFE_ID_REGEX};
+use crate::tools::ControlFlow;
+
+pub(crate) const ACME_ACCOUNT_DIR: &str = configdir!("/acme/accounts");
+
+pub mod plugin;
+
+#[api(
+ properties: {
+ name: { type: String },
+ url: { type: String },
+ },
+)]
+/// An ACME directory endpoint with a name and URL.
+#[derive(Serialize)]
+pub struct KnownAcmeDirectory {
+ /// The ACME directory's name.
+ pub name: &'static str,
+
+ /// The ACME directory's endpoint URL.
+ pub url: &'static str,
+}
+
+pub const KNOWN_ACME_DIRECTORIES: &[KnownAcmeDirectory] = &[
+ KnownAcmeDirectory {
+ name: "Let's Encrypt V2",
+ url: "https://acme-v02.api.letsencrypt.org/directory",
+ },
+ KnownAcmeDirectory {
+ name: "Let's Encrypt V2 Staging",
+ url: "https://acme-staging-v02.api.letsencrypt.org/directory",
+ },
+];
+
+pub const DEFAULT_ACME_DIRECTORY_ENTRY: &KnownAcmeDirectory = &KNOWN_ACME_DIRECTORIES[0];
+
+pub fn account_path(name: &str) -> String {
+ format!("{}/{}", ACME_ACCOUNT_DIR, name)
+}
+
+#[api(format: &PROXMOX_SAFE_ID_FORMAT)]
+/// ACME account name.
+#[derive(Clone, Eq, PartialEq, Hash, Deserialize, Serialize)]
+#[serde(transparent)]
+pub struct AccountName(String);
+
+impl AccountName {
+ pub fn into_string(self) -> String {
+ self.0
+ }
+}
+
+impl std::ops::Deref for AccountName {
+ type Target = str;
+
+ #[inline]
+ fn deref(&self) -> &str {
+ &self.0
+ }
+}
+
+impl std::ops::DerefMut for AccountName {
+ #[inline]
+ fn deref_mut(&mut self) -> &mut str {
+ &mut self.0
+ }
+}
+
+impl AsRef<str> for AccountName {
+ #[inline]
+ fn as_ref(&self) -> &str {
+ self.0.as_ref()
+ }
+}
+
+impl fmt::Debug for AccountName {
+ #[inline]
+ fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
+ fmt::Debug::fmt(&self.0, f)
+ }
+}
+
+impl fmt::Display for AccountName {
+ #[inline]
+ fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
+ fmt::Display::fmt(&self.0, f)
+ }
+}
+
+pub fn foreach_acme_account<F>(mut func: F) -> Result<(), Error>
+where
+ F: FnMut(AccountName) -> ControlFlow<Result<(), Error>>,
+{
+ match crate::tools::fs::scan_subdir(-1, ACME_ACCOUNT_DIR, &PROXMOX_SAFE_ID_REGEX) {
+ Ok(files) => {
+ for file in files {
+ let file = file?;
+ let file_name = unsafe { file.file_name_utf8_unchecked() };
+
+ if file_name.starts_with('_') {
+ continue;
+ }
+
+ let account_name = AccountName(file_name.to_owned());
+
+ if let ControlFlow::Break(result) = func(account_name) {
+ return result;
+ }
+ }
+ Ok(())
+ }
+ Err(err) if err.not_found() => Ok(()),
+ Err(err) => Err(err.into()),
+ }
+}
+
+/// Run a function for each DNS plugin ID.
+pub fn foreach_dns_plugin<F>(mut func: F) -> Result<(), Error>
+where
+ F: FnMut(&str) -> ControlFlow<Result<(), Error>>,
+{
+ match crate::tools::fs::read_subdir(-1, "/usr/share/proxmox-acme/dnsapi") {
+ Ok(files) => {
+ for file in files.filter_map(Result::ok) {
+ if let Some(id) = file
+ .file_name()
+ .to_str()
+ .ok()
+ .and_then(|name| name.strip_prefix("dns_"))
+ .and_then(|name| name.strip_suffix(".sh"))
+ {
+ if let ControlFlow::Break(result) = func(id) {
+ return result;
+ }
+ }
+ }
+
+ Ok(())
+ }
+ Err(err) if err.not_found() => Ok(()),
+ Err(err) => Err(err.into()),
+ }
+}
+
+pub fn mark_account_deactivated(name: &str) -> Result<(), Error> {
+ let from = account_path(name);
+ for i in 0..100 {
+ let to = account_path(&format!("_deactivated_{}_{}", name, i));
+ if !Path::new(&to).exists() {
+ return std::fs::rename(&from, &to).map_err(|err| {
+ format_err!(
+ "failed to move account path {:?} to {:?} - {}",
+ from,
+ to,
+ err
+ )
+ });
+ }
+ }
+ bail!(
+ "No free slot to rename deactivated account {:?}, please cleanup {:?}",
+ from,
+ ACME_ACCOUNT_DIR
+ );
+}
+
+pub fn complete_acme_account(_arg: &str, _param: &HashMap<String, String>) -> Vec<String> {
+ let mut out = Vec::new();
+ let _ = foreach_acme_account(|name| {
+ out.push(name.into_string());
+ ControlFlow::CONTINUE
+ });
+ out
+}
+
+pub fn complete_acme_plugin(_arg: &str, _param: &HashMap<String, String>) -> Vec<String> {
+ match plugin::config() {
+ Ok((config, _digest)) => config
+ .iter()
+ .map(|(id, (_type, _cfg))| id.clone())
+ .collect(),
+ Err(_) => Vec::new(),
+ }
+}
+
+pub fn complete_acme_plugin_type(_arg: &str, _param: &HashMap<String, String>) -> Vec<String> {
+ vec!["dns".to_string(), "http".to_string()]
+}
diff --git a/src/config/acme/plugin.rs b/src/config/acme/plugin.rs
new file mode 100644
index 00000000..b0c655c0
--- /dev/null
+++ b/src/config/acme/plugin.rs
@@ -0,0 +1,380 @@
+use std::future::Future;
+use std::pin::Pin;
+use std::process::Stdio;
+
+use anyhow::{bail, format_err, Error};
+use lazy_static::lazy_static;
+use serde::{Deserialize, Serialize};
+use serde_json::Value;
+use tokio::io::AsyncWriteExt;
+use tokio::process::Command;
+
+use proxmox::api::{
+ api,
+ schema::*,
+ section_config::{SectionConfig, SectionConfigData, SectionConfigPlugin},
+};
+
+use proxmox::tools::{fs::replace_file, fs::CreateOptions};
+
+use proxmox_acme_rs::{Authorization, Challenge};
+
+use crate::acme::AcmeClient;
+use crate::api2::types::PROXMOX_SAFE_ID_FORMAT;
+use crate::config::node::AcmeDomain;
+
+const ACME_PATH: &str = "/usr/share/proxmox-acme/proxmox-acme";
+
+pub const PLUGIN_ID_SCHEMA: Schema = StringSchema::new("ACME Challenge Plugin ID.")
+ .format(&PROXMOX_SAFE_ID_FORMAT)
+ .schema();
+
+lazy_static! {
+ pub static ref CONFIG: SectionConfig = init();
+}
+
+#[api(
+ properties: {
+ id: { schema: PLUGIN_ID_SCHEMA },
+ },
+)]
+#[derive(Deserialize, Serialize)]
+/// Standalone ACME Plugin for the http-1 challenge.
+pub struct StandalonePlugin {
+ /// Plugin ID.
+ id: String,
+}
+
+impl Default for StandalonePlugin {
+ fn default() -> Self {
+ Self {
+ id: "standalone".to_string(),
+ }
+ }
+}
+
+/// In PVE/PMG we store the plugin's "data" member as base64url encoded string. The UI sends
+/// regular base64 encoded data. We need to "fix" this up.
+
+#[api(
+ properties: {
+ id: { schema: PLUGIN_ID_SCHEMA },
+ disable: {
+ optional: true,
+ default: false,
+ },
+ "validation-delay": {
+ default: 30,
+ optional: true,
+ minimum: 0,
+ maximum: 2 * 24 * 60 * 60,
+ },
+ },
+)]
+/// DNS ACME Challenge Plugin core data.
+#[derive(Deserialize, Serialize, Updater)]
+#[serde(rename_all = "kebab-case")]
+pub struct DnsPluginCore {
+ /// Plugin ID.
+ pub(crate) id: String,
+
+ /// DNS API Plugin Id.
+ api: String,
+
+ /// Extra delay in seconds to wait before requesting validation.
+ ///
+ /// Allows to cope with long TTL of DNS records.
+ #[serde(skip_serializing_if = "Option::is_none", default)]
+ validation_delay: Option<u32>,
+
+ /// Flag to disable the config.
+ #[serde(skip_serializing_if = "Option::is_none", default)]
+ disable: Option<bool>,
+}
+
+#[api(
+ properties: {
+ core: { type: DnsPluginCore },
+ },
+)]
+/// DNS ACME Challenge Plugin.
+#[derive(Deserialize, Serialize)]
+#[serde(rename_all = "kebab-case")]
+pub struct DnsPlugin {
+ #[serde(flatten)]
+ pub(crate) core: DnsPluginCore,
+
+ // FIXME: The `Updater` should allow:
+ // * having different descriptions for this and the Updater version
+ // * having different `#[serde]` attributes for the Updater
+ // * or, well, leaving fields out completely in teh Updater but this means we may need to
+ // separate Updater and Builder deriving.
+ // We handle this property separately in the API calls.
+ /// DNS plugin data (base64url encoded without padding).
+ #[serde(with = "proxmox::tools::serde::string_as_base64url_nopad")]
+ pub(crate) data: String,
+}
+
+impl DnsPlugin {
+ pub fn decode_data(&self, output: &mut Vec<u8>) -> Result<(), Error> {
+ Ok(base64::decode_config_buf(&self.data, base64::URL_SAFE_NO_PAD, output)?)
+ }
+}
+
+//impl DnsPluginUpdater {
+// // The UI passes regular base64 data, we need base64url data. In PVE/PMG this happens magically
+// // since perl parses both on decode...
+// pub fn api_fixup(&mut self) -> Result<(), Error> {
+// if let Some(data) = self.data.as_mut() {
+// let new = base64::encode_config(&base64::decode(&data)?, base64::URL_SAFE_NO_PAD);
+// *data = new;
+// }
+// Ok(())
+// }
+//}
+
+fn init() -> SectionConfig {
+ let mut config = SectionConfig::new(&PLUGIN_ID_SCHEMA);
+
+ let standalone_schema = match &StandalonePlugin::API_SCHEMA {
+ Schema::Object(schema) => schema,
+ _ => unreachable!(),
+ };
+ let standalone_plugin = SectionConfigPlugin::new(
+ "standalone".to_string(),
+ Some("id".to_string()),
+ standalone_schema,
+ );
+ config.register_plugin(standalone_plugin);
+
+ let dns_challenge_schema = match DnsPlugin::API_SCHEMA {
+ Schema::AllOf(ref schema) => schema,
+ _ => unreachable!(),
+ };
+ let dns_challenge_plugin = SectionConfigPlugin::new(
+ "dns".to_string(),
+ Some("id".to_string()),
+ dns_challenge_schema,
+ );
+ config.register_plugin(dns_challenge_plugin);
+
+ config
+}
+
+pub const ACME_PLUGIN_CFG_FILENAME: &str = "/etc/proxmox-backup/acme/plugins.cfg";
+pub const ACME_PLUGIN_CFG_LOCKFILE: &str = "/etc/proxmox-backup/acme/.plugins.lck";
+const LOCK_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(5);
+
+pub fn read_lock() -> Result<std::fs::File, Error> {
+ proxmox::tools::fs::open_file_locked(ACME_PLUGIN_CFG_LOCKFILE, LOCK_TIMEOUT, false)
+}
+
+pub fn write_lock() -> Result<std::fs::File, Error> {
+ proxmox::tools::fs::open_file_locked(ACME_PLUGIN_CFG_LOCKFILE, LOCK_TIMEOUT, true)
+}
+
+pub fn config() -> Result<(PluginData, [u8; 32]), Error> {
+ let content = proxmox::tools::fs::file_read_optional_string(ACME_PLUGIN_CFG_FILENAME)?
+ .unwrap_or_else(|| "".to_string());
+
+ let digest = openssl::sha::sha256(content.as_bytes());
+ let mut data = CONFIG.parse(ACME_PLUGIN_CFG_FILENAME, &content)?;
+
+ if data.sections.get("standalone").is_none() {
+ let standalone = StandalonePlugin::default();
+ data.set_data("standalone", "standalone", &standalone)
+ .unwrap();
+ }
+
+ Ok((PluginData { data }, digest))
+}
+
+pub fn save_config(config: &PluginData) -> Result<(), Error> {
+ let raw = CONFIG.write(ACME_PLUGIN_CFG_FILENAME, &config.data)?;
+
+ let backup_user = crate::backup::backup_user()?;
+ let mode = nix::sys::stat::Mode::from_bits_truncate(0o0640);
+ // set the correct owner/group/permissions while saving file
+ // owner(rw) = root, group(r)= backup
+ let options = CreateOptions::new()
+ .perm(mode)
+ .owner(nix::unistd::ROOT)
+ .group(backup_user.gid);
+
+ replace_file(ACME_PLUGIN_CFG_FILENAME, raw.as_bytes(), options)?;
+
+ Ok(())
+}
+
+pub struct PluginData {
+ data: SectionConfigData,
+}
+
+impl PluginData {
+ #[inline]
+ pub fn remove(&mut self, name: &str) -> Option<(String, Value)> {
+ self.data.sections.remove(name)
+ }
+
+ #[inline]
+ pub fn contains_key(&mut self, name: &str) -> bool {
+ self.data.sections.contains_key(name)
+ }
+
+ #[inline]
+ pub fn get(&self, name: &str) -> Option<&(String, Value)> {
+ self.data.sections.get(name)
+ }
+
+ #[inline]
+ pub fn get_mut(&mut self, name: &str) -> Option<&mut (String, Value)> {
+ self.data.sections.get_mut(name)
+ }
+
+ // FIXME: Verify the plugin type *exists* and check its config schema...
+ pub fn insert(&mut self, id: String, ty: String, plugin: Value) {
+ self.data.sections.insert(id, (ty, plugin));
+ }
+
+ pub fn get_plugin(
+ &self,
+ name: &str,
+ ) -> Result<Option<Box<dyn AcmePlugin + Send + Sync + 'static>>, Error> {
+ let (ty, data) = match self.get(name) {
+ Some(plugin) => plugin,
+ None => return Ok(None),
+ };
+
+ Ok(Some(match ty.as_str() {
+ "dns" => {
+ let plugin: DnsPlugin = serde_json::from_value(data.clone())?;
+ Box::new(plugin)
+ }
+ // "standalone" => todo!("standalone plugin"),
+ other => bail!("missing implementation for plugin type '{}'", other),
+ }))
+ }
+
+ pub fn iter(&self) -> impl Iterator<Item = (&String, &(String, Value))> + Send {
+ self.data.sections.iter()
+ }
+}
+
+pub trait AcmePlugin {
+ /// Setup everything required to trigger the validation and return the corresponding validation
+ /// URL.
+ fn setup<'fut, 'a: 'fut, 'b: 'fut, 'c: 'fut, 'd: 'fut>(
+ &'a self,
+ client: &'b mut AcmeClient,
+ authorization: &'c Authorization,
+ domain: &'d AcmeDomain,
+ ) -> Pin<Box<dyn Future<Output = Result<&'c str, Error>> + Send + 'fut>>;
+
+ fn teardown<'fut, 'a: 'fut, 'b: 'fut, 'c: 'fut, 'd: 'fut>(
+ &'a self,
+ client: &'b mut AcmeClient,
+ authorization: &'c Authorization,
+ domain: &'d AcmeDomain,
+ ) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'fut>>;
+}
+
+impl DnsPlugin {
+ fn extract_challenge(authorization: &Authorization) -> Result<&Challenge, Error> {
+ authorization
+ .challenges
+ .iter()
+ .find(|ch| ch.ty == "dns-01")
+ .ok_or_else(|| format_err!("no supported challenge type (dns-01) found"))
+ }
+
+ async fn action<'a>(
+ &self,
+ client: &mut AcmeClient,
+ authorization: &'a Authorization,
+ domain: &AcmeDomain,
+ action: &str,
+ ) -> Result<&'a str, Error> {
+ let challenge = Self::extract_challenge(authorization)?;
+ let mut stdin_data = client
+ .dns_01_txt_value(
+ challenge
+ .token()
+ .ok_or_else(|| format_err!("missing token in challenge"))?,
+ )?
+ .into_bytes();
+ stdin_data.push(b'\n');
+ stdin_data.extend(self.data.as_bytes());
+ if stdin_data.last() != Some(&b'\n') {
+ stdin_data.push(b'\n');
+ }
+
+ let mut command = Command::new("/usr/bin/setpriv");
+
+ #[rustfmt::skip]
+ command.args(&[
+ "--reuid", "nobody",
+ "--regid", "nogroup",
+ "--clear-groups",
+ "--reset-env",
+ "--",
+ "/bin/bash",
+ ACME_PATH,
+ action,
+ &self.core.api,
+ domain.alias.as_deref().unwrap_or(&domain.domain),
+ ]);
+
+ let mut child = command.stdin(Stdio::piped()).spawn()?;
+
+ let mut stdin = child.stdin.take().expect("Stdio::piped()");
+ match async move {
+ stdin.write_all(&stdin_data).await?;
+ stdin.flush().await?;
+ Ok::<_, std::io::Error>(())
+ }.await {
+ Ok(()) => (),
+ Err(err) => {
+ if let Err(err) = child.kill().await {
+ eprintln!("failed to kill '{} {}' command: {}", ACME_PATH, action, err);
+ }
+ bail!("'{}' failed: {}", ACME_PATH, err);
+ }
+ }
+
+ let status = child.wait().await?;
+ if !status.success() {
+ bail!(
+ "'{} {}' exited with error ({})",
+ ACME_PATH,
+ action,
+ status.code().unwrap_or(-1)
+ );
+ }
+
+ Ok(&challenge.url)
+ }
+}
+
+impl AcmePlugin for DnsPlugin {
+ fn setup<'fut, 'a: 'fut, 'b: 'fut, 'c: 'fut, 'd: 'fut>(
+ &'a self,
+ client: &'b mut AcmeClient,
+ authorization: &'c Authorization,
+ domain: &'d AcmeDomain,
+ ) -> Pin<Box<dyn Future<Output = Result<&'c str, Error>> + Send + 'fut>> {
+ Box::pin(self.action(client, authorization, domain, "setup"))
+ }
+
+ fn teardown<'fut, 'a: 'fut, 'b: 'fut, 'c: 'fut, 'd: 'fut>(
+ &'a self,
+ client: &'b mut AcmeClient,
+ authorization: &'c Authorization,
+ domain: &'d AcmeDomain,
+ ) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'fut>> {
+ Box::pin(async move {
+ self.action(client, authorization, domain, "teardown")
+ .await
+ .map(drop)
+ })
+ }
+}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 16/23] add async acme client implementation
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (14 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 15/23] add acme config Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 17/23] add config/acme api path Wolfgang Bumiller
` (8 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
This is the highlevel part using proxmox-acme-rs to create
requests and our hyper code to issue them to the acme
server.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/acme/client.rs | 627 +++++++++++++++++++++++++++++++++++++++++++++
src/acme/mod.rs | 2 +
src/lib.rs | 2 +
3 files changed, 631 insertions(+)
create mode 100644 src/acme/client.rs
create mode 100644 src/acme/mod.rs
diff --git a/src/acme/client.rs b/src/acme/client.rs
new file mode 100644
index 00000000..cb3ff551
--- /dev/null
+++ b/src/acme/client.rs
@@ -0,0 +1,627 @@
+//! HTTP Client for the ACME protocol.
+
+use std::fs::OpenOptions;
+use std::io;
+use std::os::unix::fs::OpenOptionsExt;
+
+use anyhow::format_err;
+use bytes::Bytes;
+use hyper::{Body, Request};
+use nix::sys::stat::Mode;
+use serde::{Deserialize, Serialize};
+
+use proxmox::tools::fs::{replace_file, CreateOptions};
+use proxmox_acme_rs::account::AccountCreator;
+use proxmox_acme_rs::account::AccountData as AcmeAccountData;
+use proxmox_acme_rs::order::{Order, OrderData};
+use proxmox_acme_rs::Request as AcmeRequest;
+use proxmox_acme_rs::{Account, Authorization, Challenge, Directory, Error, ErrorResponse};
+
+use crate::config::acme::{account_path, AccountName};
+use crate::tools::http as http_client;
+
+/// Our on-disk format inherited from PVE's proxmox-acme code.
+#[derive(Deserialize, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct AccountData {
+ /// The account's location URL.
+ location: String,
+
+ /// The account data.
+ account: AcmeAccountData,
+
+ /// The private key as PEM formatted string.
+ key: String,
+
+ /// ToS URL the user agreed to.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ tos: Option<String>,
+
+ #[serde(skip_serializing_if = "is_false", default)]
+ debug: bool,
+
+ /// The directory's URL.
+ directory_url: String,
+}
+
+#[inline]
+fn is_false(b: &bool) -> bool {
+ !*b
+}
+
+pub struct AcmeClient {
+ directory_url: String,
+ debug: bool,
+ account_path: Option<String>,
+ tos: Option<String>,
+ account: Option<Account>,
+ directory: Option<Directory>,
+ nonce: Option<String>,
+}
+
+impl AcmeClient {
+ /// Create a new ACME client for a given ACME directory URL.
+ pub fn new(directory_url: String) -> Self {
+ Self {
+ directory_url,
+ debug: false,
+ account_path: None,
+ tos: None,
+ account: None,
+ directory: None,
+ nonce: None,
+ }
+ }
+
+ /// Load an existing ACME account by name.
+ pub async fn load(account_name: &AccountName) -> Result<Self, anyhow::Error> {
+ Self::load_path(account_path(account_name.as_ref())).await
+ }
+
+ /// Load an existing ACME account by path.
+ async fn load_path(account_path: String) -> Result<Self, anyhow::Error> {
+ let data = tokio::fs::read(&account_path).await?;
+ let data: AccountData = serde_json::from_slice(&data)?;
+
+ let account = Account::from_parts(data.location, data.key, data.account);
+
+ Ok(Self {
+ directory_url: data.directory_url,
+ debug: data.debug,
+ account_path: Some(account_path),
+ tos: data.tos,
+ account: Some(account),
+ directory: None,
+ nonce: None,
+ })
+ }
+
+ pub async fn new_account<'a>(
+ &'a mut self,
+ account_name: &AccountName,
+ tos_agreed: bool,
+ contact: Vec<String>,
+ rsa_bits: Option<u32>,
+ ) -> Result<&'a Account, anyhow::Error> {
+ self.tos = if tos_agreed {
+ self.terms_of_service_url().await?.map(str::to_owned)
+ } else {
+ None
+ };
+
+ let account = Account::creator()
+ .set_contacts(contact)
+ .agree_to_tos(tos_agreed);
+
+ let account = if let Some(bits) = rsa_bits {
+ account.generate_rsa_key(bits)?
+ } else {
+ account.generate_ec_key()?
+ };
+
+ let _ = self.register_account(account).await?;
+
+ let account_path = account_path(account_name.as_ref());
+ let file = OpenOptions::new()
+ .write(true)
+ .create(true)
+ .mode(0o600)
+ .open(&account_path)
+ .map_err(|err| format_err!("failed to open {:?} for writing: {}", account_path, err))?;
+ self.write_to(file).map_err(|err| {
+ format_err!(
+ "failed to write acme account to {:?}: {}",
+ account_path,
+ err
+ )
+ })?;
+ self.account_path = Some(account_path);
+
+ // unwrap: Setting `self.account` is literally this function's job, we just can't keep
+ // the borrow from from `self.register_account()` active due to clashes.
+ Ok(self.account.as_ref().unwrap())
+ }
+
+ fn save(&self) -> Result<(), anyhow::Error> {
+ let mut data = Vec::<u8>::new();
+ self.write_to(&mut data)?;
+ let account_path = self.account_path.as_ref().ok_or_else(|| {
+ format_err!("no account path set, cannot save upated account information")
+ })?;
+ replace_file(
+ account_path,
+ &data,
+ CreateOptions::new()
+ .perm(Mode::from_bits_truncate(0o600))
+ .owner(nix::unistd::ROOT)
+ .group(nix::unistd::Gid::from_raw(0)),
+ )
+ }
+
+ /// Shortcut to `account().ok_or_else(...).key_authorization()`.
+ pub fn key_authorization(&self, token: &str) -> Result<String, anyhow::Error> {
+ Ok(Self::need_account(&self.account)?.key_authorization(token)?)
+ }
+
+ /// Shortcut to `account().ok_or_else(...).dns_01_txt_value()`.
+ /// the key authorization value.
+ pub fn dns_01_txt_value(&self, token: &str) -> Result<String, anyhow::Error> {
+ Ok(Self::need_account(&self.account)?.dns_01_txt_value(token)?)
+ }
+
+ async fn register_account(
+ &mut self,
+ account: AccountCreator,
+ ) -> Result<&Account, anyhow::Error> {
+ let mut retry = retry();
+ let mut response = loop {
+ retry.tick()?;
+
+ let (directory, nonce) =
+ Self::get_dir_nonce(&self.directory_url, &mut self.directory, &mut self.nonce)
+ .await?;
+ let request = account.request(directory, nonce)?;
+ match self.run_request(request).await {
+ Ok(response) => break response,
+ Err(err) if err.is_bad_nonce() => continue,
+ Err(err) => return Err(err.into()),
+ }
+ };
+
+ let account = account.response(response.location_required()?, &response.body)?;
+
+ self.account = Some(account);
+ Ok(self.account.as_ref().unwrap())
+ }
+
+ pub async fn update_account<T: Serialize>(
+ &mut self,
+ data: &T,
+ ) -> Result<&Account, anyhow::Error> {
+ let account = Self::need_account(&self.account)?;
+
+ let mut retry = retry();
+ let response = loop {
+ retry.tick()?;
+
+ let (_directory, nonce) =
+ Self::get_dir_nonce(&self.directory_url, &mut self.directory, &mut self.nonce)
+ .await?;
+
+ let request = account.post_request(&account.location, &nonce, data)?;
+ match Self::execute(request, &mut self.nonce).await {
+ Ok(response) => break response,
+ Err(err) if err.is_bad_nonce() => continue,
+ Err(err) => return Err(err.into()),
+ }
+ };
+
+ // unwrap: we've been keeping an immutable reference to it from the top of the method
+ let _ = account;
+ self.account.as_mut().unwrap().data = response.json()?;
+ self.save()?;
+ Ok(self.account.as_ref().unwrap())
+ }
+
+ pub async fn new_order<I>(&mut self, domains: I) -> Result<Order, anyhow::Error>
+ where
+ I: IntoIterator<Item = String>,
+ {
+ let account = Self::need_account(&self.account)?;
+
+ let order = domains
+ .into_iter()
+ .fold(OrderData::new(), |order, domain| order.domain(domain));
+
+ let mut retry = retry();
+ loop {
+ retry.tick()?;
+
+ let (directory, nonce) =
+ Self::get_dir_nonce(&self.directory_url, &mut self.directory, &mut self.nonce)
+ .await?;
+
+ let mut new_order = account.new_order(&order, directory, nonce)?;
+ let mut response =
+ match Self::execute(new_order.request.take().unwrap(), &mut self.nonce).await {
+ Ok(response) => response,
+ Err(err) if err.is_bad_nonce() => continue,
+ Err(err) => return Err(err.into()),
+ };
+
+ return Ok(
+ new_order.response(response.location_required()?, response.bytes().as_ref())?
+ );
+ }
+ }
+
+ /// Low level "POST-as-GET" request.
+ async fn post_as_get(&mut self, url: &str) -> Result<AcmeResponse, anyhow::Error> {
+ let account = Self::need_account(&self.account)?;
+
+ let mut retry = retry();
+ loop {
+ retry.tick()?;
+
+ let (_directory, nonce) =
+ Self::get_dir_nonce(&self.directory_url, &mut self.directory, &mut self.nonce)
+ .await?;
+
+ let request = account.get_request(url, nonce)?;
+ match Self::execute(request, &mut self.nonce).await {
+ Ok(response) => return Ok(response),
+ Err(err) if err.is_bad_nonce() => continue,
+ Err(err) => return Err(err.into()),
+ }
+ }
+ }
+
+ /// Low level POST request.
+ async fn post<T: Serialize>(
+ &mut self,
+ url: &str,
+ data: &T,
+ ) -> Result<AcmeResponse, anyhow::Error> {
+ let account = Self::need_account(&self.account)?;
+
+ let mut retry = retry();
+ loop {
+ retry.tick()?;
+
+ let (_directory, nonce) =
+ Self::get_dir_nonce(&self.directory_url, &mut self.directory, &mut self.nonce)
+ .await?;
+
+ let request = account.post_request(url, nonce, data)?;
+ match Self::execute(request, &mut self.nonce).await {
+ Ok(response) => return Ok(response),
+ Err(err) if err.is_bad_nonce() => continue,
+ Err(err) => return Err(err.into()),
+ }
+ }
+ }
+
+ /// Request challenge validation. Afterwards, the challenge should be polled.
+ pub async fn request_challenge_validation(
+ &mut self,
+ url: &str,
+ ) -> Result<Challenge, anyhow::Error> {
+ Ok(self
+ .post(url, &serde_json::Value::Object(Default::default()))
+ .await?
+ .json()?)
+ }
+
+ /// Assuming the provided URL is an 'Authorization' URL, get and deserialize it.
+ pub async fn get_authorization(&mut self, url: &str) -> Result<Authorization, anyhow::Error> {
+ Ok(self.post_as_get(url).await?.json()?)
+ }
+
+ /// Assuming the provided URL is an 'Order' URL, get and deserialize it.
+ pub async fn get_order(&mut self, url: &str) -> Result<OrderData, anyhow::Error> {
+ Ok(self.post_as_get(url).await?.json()?)
+ }
+
+ /// Finalize an Order via its `finalize` URL property and the DER encoded CSR.
+ pub async fn finalize(&mut self, url: &str, csr: &[u8]) -> Result<(), anyhow::Error> {
+ let csr = base64::encode_config(csr, base64::URL_SAFE_NO_PAD);
+ let data = serde_json::json!({ "csr": csr });
+ self.post(url, &data).await?;
+ Ok(())
+ }
+
+ /// Download a certificate via its 'certificate' URL property.
+ ///
+ /// The certificate will be a PEM certificate chain.
+ pub async fn get_certificate(&mut self, url: &str) -> Result<Bytes, anyhow::Error> {
+ Ok(self.post_as_get(url).await?.body)
+ }
+
+ /// Revoke an existing certificate (PEM or DER formatted).
+ pub async fn revoke_certificate(
+ &mut self,
+ certificate: &[u8],
+ reason: Option<u32>,
+ ) -> Result<(), anyhow::Error> {
+ // TODO: This can also work without an account.
+ let account = Self::need_account(&self.account)?;
+
+ let revocation = account.revoke_certificate(certificate, reason)?;
+
+ let mut retry = retry();
+ loop {
+ retry.tick()?;
+
+ let (directory, nonce) =
+ Self::get_dir_nonce(&self.directory_url, &mut self.directory, &mut self.nonce)
+ .await?;
+
+ let request = revocation.request(&directory, nonce)?;
+ match Self::execute(request, &mut self.nonce).await {
+ Ok(_response) => return Ok(()),
+ Err(err) if err.is_bad_nonce() => continue,
+ Err(err) => return Err(err.into()),
+ }
+ }
+ }
+
+ fn need_account(account: &Option<Account>) -> Result<&Account, anyhow::Error> {
+ account
+ .as_ref()
+ .ok_or_else(|| format_err!("cannot use client without an account"))
+ }
+
+ pub(crate) fn account(&self) -> Result<&Account, anyhow::Error> {
+ Self::need_account(&self.account)
+ }
+
+ pub fn tos(&self) -> Option<&str> {
+ self.tos.as_deref()
+ }
+
+ pub fn directory_url(&self) -> &str {
+ &self.directory_url
+ }
+
+ fn to_account_data(&self) -> Result<AccountData, anyhow::Error> {
+ let account = self.account()?;
+
+ Ok(AccountData {
+ location: account.location.clone(),
+ key: account.private_key.clone(),
+ account: AcmeAccountData {
+ only_return_existing: false, // don't actually write this out in case it's set
+ ..account.data.clone()
+ },
+ tos: self.tos.clone(),
+ debug: self.debug,
+ directory_url: self.directory_url.clone(),
+ })
+ }
+
+ fn write_to<T: io::Write>(&self, out: T) -> Result<(), anyhow::Error> {
+ let data = self.to_account_data()?;
+
+ Ok(serde_json::to_writer_pretty(out, &data)?)
+ }
+}
+
+struct AcmeResponse {
+ body: Bytes,
+ location: Option<String>,
+ got_nonce: bool,
+}
+
+impl AcmeResponse {
+ /// Convenience helper to assert that a location header was part of the response.
+ fn location_required(&mut self) -> Result<String, anyhow::Error> {
+ self.location
+ .take()
+ .ok_or_else(|| format_err!("missing Location header"))
+ }
+
+ /// Convenience shortcut to perform json deserialization of the returned body.
+ fn json<T: for<'a> Deserialize<'a>>(&self) -> Result<T, Error> {
+ Ok(serde_json::from_slice(&self.body)?)
+ }
+
+ /// Convenience shortcut to get the body as bytes.
+ fn bytes(&self) -> &[u8] {
+ &self.body
+ }
+}
+
+impl AcmeClient {
+ /// Non-self-borrowing run_request version for borrow workarounds.
+ async fn execute(
+ request: AcmeRequest,
+ nonce: &mut Option<String>,
+ ) -> Result<AcmeResponse, Error> {
+ let mut req_builder = Request::builder().method(request.method).uri(&request.url);
+
+ let body: Body = if !request.content_type.is_empty() {
+ req_builder = req_builder
+ .header("Content-Type", request.content_type)
+ .header("Content-Length", request.body.len());
+ request.body.into()
+ } else {
+ Body::empty()
+ };
+
+ let response = http_client::request(req_builder, body)
+ .await
+ .map_err(|err| Error::Custom(err.to_string()))?;
+ let (parts, body) = response.into_parts();
+
+ let status = parts.status.as_u16();
+ let body = hyper::body::to_bytes(body)
+ .await
+ .map_err(|err| Error::Custom(format!("failed to retrieve response body: {}", err)))?;
+
+ let got_nonce = if let Some(new_nonce) = parts.headers.get(proxmox_acme_rs::REPLAY_NONCE) {
+ let new_nonce = new_nonce.to_str().map_err(|err| {
+ Error::Client(format!(
+ "received invalid replay-nonce header from ACME server: {}",
+ err
+ ))
+ })?;
+ *nonce = Some(new_nonce.to_owned());
+ true
+ } else {
+ false
+ };
+
+ if parts.status.is_success() {
+ if status != request.expected {
+ return Err(Error::InvalidApi(format!(
+ "ACME server responded with unexpected status code: {:?}",
+ parts.status
+ )));
+ }
+
+ let location = parts
+ .headers
+ .get("Location")
+ .map(|header| {
+ header.to_str().map(str::to_owned).map_err(|err| {
+ Error::Client(format!(
+ "received invalid location header from ACME server: {}",
+ err
+ ))
+ })
+ })
+ .transpose()?;
+
+ return Ok(AcmeResponse {
+ body,
+ location,
+ got_nonce,
+ });
+ }
+
+ let error: ErrorResponse = serde_json::from_slice(&body).map_err(|err| {
+ Error::Client(format!(
+ "error status with improper error ACME response: {}",
+ err
+ ))
+ })?;
+
+ if error.ty == proxmox_acme_rs::error::BAD_NONCE {
+ if !got_nonce {
+ return Err(Error::InvalidApi(
+ "badNonce without a new Replay-Nonce header".to_string(),
+ ));
+ }
+ return Err(Error::BadNonce);
+ }
+
+ Err(Error::Api(error))
+ }
+
+ /// Low-level API to run an n API request. This automatically updates the current nonce!
+ async fn run_request(&mut self, request: AcmeRequest) -> Result<AcmeResponse, Error> {
+ Self::execute(request, &mut self.nonce).await
+ }
+
+ async fn directory(&mut self) -> Result<&Directory, Error> {
+ Ok(
+ Self::get_directory(&self.directory_url, &mut self.directory, &mut self.nonce)
+ .await?
+ .0,
+ )
+ }
+
+ async fn get_directory<'a, 'b>(
+ directory_url: &str,
+ directory: &'a mut Option<Directory>,
+ nonce: &'b mut Option<String>,
+ ) -> Result<(&'a Directory, Option<&'b str>), Error> {
+ if let Some(d) = directory {
+ return Ok((d, nonce.as_deref()));
+ }
+
+ let response = Self::execute(
+ AcmeRequest {
+ url: directory_url.to_string(),
+ method: "GET",
+ content_type: "",
+ body: String::new(),
+ expected: 200,
+ },
+ nonce,
+ )
+ .await?;
+
+ *directory = Some(Directory::from_parts(
+ directory_url.to_string(),
+ response.json()?,
+ ));
+
+ Ok((directory.as_ref().unwrap(), nonce.as_deref()))
+ }
+
+ /// Like `get_directory`, but if the directory provides no nonce, also performs a `HEAD`
+ /// request on the new nonce URL.
+ async fn get_dir_nonce<'a, 'b>(
+ directory_url: &str,
+ directory: &'a mut Option<Directory>,
+ nonce: &'b mut Option<String>,
+ ) -> Result<(&'a Directory, &'b str), Error> {
+ // this let construct is a lifetime workaround:
+ let _ = Self::get_directory(directory_url, directory, nonce).await?;
+ let dir = directory.as_ref().unwrap(); // the above fails if it couldn't fill this option
+ if nonce.is_none() {
+ // this is also a lifetime issue...
+ let _ = Self::get_nonce(nonce, dir.new_nonce_url()).await?;
+ };
+ Ok((dir, nonce.as_deref().unwrap()))
+ }
+
+ pub async fn terms_of_service_url(&mut self) -> Result<Option<&str>, Error> {
+ Ok(self.directory().await?.terms_of_service_url())
+ }
+
+ async fn get_nonce<'a>(
+ nonce: &'a mut Option<String>,
+ new_nonce_url: &str,
+ ) -> Result<&'a str, Error> {
+ let response = Self::execute(
+ AcmeRequest {
+ url: new_nonce_url.to_owned(),
+ method: "HEAD",
+ content_type: "",
+ body: String::new(),
+ expected: 200,
+ },
+ nonce,
+ )
+ .await?;
+
+ if !response.got_nonce {
+ return Err(Error::InvalidApi(
+ "no new nonce received from new nonce URL".to_string(),
+ ));
+ }
+
+ nonce
+ .as_deref()
+ .ok_or_else(|| Error::Client("failed to update nonce".to_string()))
+ }
+}
+
+/// bad nonce retry count helper
+struct Retry(usize);
+
+const fn retry() -> Retry {
+ Retry(0)
+}
+
+impl Retry {
+ fn tick(&mut self) -> Result<(), Error> {
+ if self.0 >= 3 {
+ Error::Client(format!("kept getting a badNonce error!"));
+ }
+ self.0 += 1;
+ Ok(())
+ }
+}
diff --git a/src/acme/mod.rs b/src/acme/mod.rs
new file mode 100644
index 00000000..5923f8da
--- /dev/null
+++ b/src/acme/mod.rs
@@ -0,0 +1,2 @@
+pub mod client;
+pub use client::AcmeClient;
diff --git a/src/lib.rs b/src/lib.rs
index 200cf496..1b1de527 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -32,3 +32,5 @@ pub mod auth;
pub mod rrd;
pub mod tape;
+
+pub mod acme;
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 17/23] add config/acme api path
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (15 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 16/23] add async acme client implementation Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 18/23] add node/{node}/certificates api call Wolfgang Bumiller
` (7 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/api2/config.rs | 2 +
src/api2/config/acme.rs | 719 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 721 insertions(+)
create mode 100644 src/api2/config/acme.rs
diff --git a/src/api2/config.rs b/src/api2/config.rs
index 996ec268..9befa0e5 100644
--- a/src/api2/config.rs
+++ b/src/api2/config.rs
@@ -4,6 +4,7 @@ use proxmox::api::router::{Router, SubdirMap};
use proxmox::list_subdirs_api_method;
pub mod access;
+pub mod acme;
pub mod datastore;
pub mod remote;
pub mod sync;
@@ -16,6 +17,7 @@ pub mod tape_backup_job;
const SUBDIRS: SubdirMap = &[
("access", &access::ROUTER),
+ ("acme", &acme::ROUTER),
("changer", &changer::ROUTER),
("datastore", &datastore::ROUTER),
("drive", &drive::ROUTER),
diff --git a/src/api2/config/acme.rs b/src/api2/config/acme.rs
new file mode 100644
index 00000000..4f72a94e
--- /dev/null
+++ b/src/api2/config/acme.rs
@@ -0,0 +1,719 @@
+use std::path::Path;
+
+use anyhow::{bail, format_err, Error};
+use serde::{Deserialize, Serialize};
+use serde_json::{json, Value};
+
+use proxmox::api::router::SubdirMap;
+use proxmox::api::schema::Updatable;
+use proxmox::api::{api, Permission, Router, RpcEnvironment};
+use proxmox::http_bail;
+use proxmox::list_subdirs_api_method;
+
+use proxmox_acme_rs::account::AccountData as AcmeAccountData;
+use proxmox_acme_rs::Account;
+
+use crate::acme::AcmeClient;
+use crate::api2::types::Authid;
+use crate::config::acl::PRIV_SYS_MODIFY;
+use crate::config::acme::plugin::{
+ DnsPlugin, DnsPluginCore, DnsPluginCoreUpdater, PLUGIN_ID_SCHEMA,
+};
+use crate::config::acme::{AccountName, KnownAcmeDirectory};
+use crate::server::WorkerTask;
+use crate::tools::ControlFlow;
+
+pub(crate) const ROUTER: Router = Router::new()
+ .get(&list_subdirs_api_method!(SUBDIRS))
+ .subdirs(SUBDIRS);
+
+const SUBDIRS: SubdirMap = &[
+ (
+ "account",
+ &Router::new()
+ .get(&API_METHOD_LIST_ACCOUNTS)
+ .post(&API_METHOD_REGISTER_ACCOUNT)
+ .match_all("name", &ACCOUNT_ITEM_ROUTER),
+ ),
+ (
+ "challenge-schema",
+ &Router::new().get(&API_METHOD_GET_CHALLENGE_SCHEMA),
+ ),
+ (
+ "directories",
+ &Router::new().get(&API_METHOD_GET_DIRECTORIES),
+ ),
+ (
+ "plugins",
+ &Router::new()
+ .get(&API_METHOD_LIST_PLUGINS)
+ .post(&API_METHOD_ADD_PLUGIN)
+ .match_all("id", &PLUGIN_ITEM_ROUTER),
+ ),
+ ("tos", &Router::new().get(&API_METHOD_GET_TOS)),
+];
+
+const ACCOUNT_ITEM_ROUTER: Router = Router::new()
+ .get(&API_METHOD_GET_ACCOUNT)
+ .put(&API_METHOD_UPDATE_ACCOUNT)
+ .delete(&API_METHOD_DEACTIVATE_ACCOUNT);
+
+const PLUGIN_ITEM_ROUTER: Router = Router::new()
+ .get(&API_METHOD_GET_PLUGIN)
+ .put(&API_METHOD_UPDATE_PLUGIN)
+ .delete(&API_METHOD_DELETE_PLUGIN);
+
+#[api(
+ properties: {
+ name: { type: AccountName },
+ },
+)]
+/// An ACME Account entry.
+///
+/// Currently only contains a 'name' property.
+#[derive(Serialize)]
+pub struct AccountEntry {
+ name: AccountName,
+}
+
+#[api(
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ returns: {
+ type: Array,
+ items: { type: AccountEntry },
+ description: "List of ACME accounts.",
+ },
+ protected: true,
+)]
+/// List ACME accounts.
+pub fn list_accounts() -> Result<Vec<AccountEntry>, Error> {
+ let mut entries = Vec::new();
+ crate::config::acme::foreach_acme_account(|name| {
+ entries.push(AccountEntry { name });
+ ControlFlow::Continue(())
+ })?;
+ Ok(entries)
+}
+
+#[api(
+ properties: {
+ account: { type: Object, properties: {}, additional_properties: true },
+ tos: {
+ type: String,
+ optional: true,
+ },
+ },
+)]
+/// ACME Account information.
+///
+/// This is what we return via the API.
+#[derive(Serialize)]
+pub struct AccountInfo {
+ /// Raw account data.
+ account: AcmeAccountData,
+
+ /// The ACME directory URL the account was created at.
+ directory: String,
+
+ /// The account's own URL within the ACME directory.
+ location: String,
+
+ /// The ToS URL, if the user agreed to one.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ tos: Option<String>,
+}
+
+#[api(
+ input: {
+ properties: {
+ name: { type: AccountName },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ returns: { type: AccountInfo },
+ protected: true,
+)]
+/// Return existing ACME account information.
+pub async fn get_account(name: AccountName) -> Result<AccountInfo, Error> {
+ let client = AcmeClient::load(&name).await?;
+ let account = client.account()?;
+ Ok(AccountInfo {
+ location: account.location.clone(),
+ tos: client.tos().map(str::to_owned),
+ directory: client.directory_url().to_owned(),
+ account: AcmeAccountData {
+ only_return_existing: false, // don't actually write this out in case it's set
+ ..account.data.clone()
+ },
+ })
+}
+
+fn account_contact_from_string(s: &str) -> Vec<String> {
+ s.split(&[' ', ';', ',', '\0'][..])
+ .map(|s| format!("mailto:{}", s))
+ .collect()
+}
+
+#[api(
+ input: {
+ properties: {
+ name: { type: AccountName },
+ contact: {
+ description: "List of email addresses.",
+ },
+ tos_url: {
+ description: "URL of CA TermsOfService - setting this indicates agreement.",
+ optional: true,
+ },
+ directory: {
+ type: String,
+ description: "The ACME Directory.",
+ optional: true,
+ },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ protected: true,
+)]
+/// Register an ACME account.
+fn register_account(
+ name: AccountName,
+ // Todo: email & email-list schema
+ contact: String,
+ tos_url: Option<String>,
+ directory: Option<String>,
+ rpcenv: &mut dyn RpcEnvironment,
+) -> Result<String, Error> {
+ let auth_id: Authid = rpcenv.get_auth_id().unwrap().parse()?;
+
+ if Path::new(&crate::config::acme::account_path(&name)).exists() {
+ http_bail!(BAD_REQUEST, "account {:?} already exists", name);
+ }
+
+ let directory = directory.unwrap_or_else(|| {
+ crate::config::acme::DEFAULT_ACME_DIRECTORY_ENTRY
+ .url
+ .to_owned()
+ });
+
+ WorkerTask::spawn(
+ "acme-register",
+ None,
+ auth_id,
+ true,
+ move |worker| async move {
+ let mut client = AcmeClient::new(directory);
+
+ worker.log("Registering ACME account...");
+
+ let account =
+ do_register_account(&mut client, &name, tos_url.is_some(), contact, None).await?;
+
+ worker.log(format!(
+ "Registration successful, account URL: {}",
+ account.location
+ ));
+
+ Ok(())
+ },
+ )
+}
+
+pub async fn do_register_account<'a>(
+ client: &'a mut AcmeClient,
+ name: &AccountName,
+ agree_to_tos: bool,
+ contact: String,
+ rsa_bits: Option<u32>,
+) -> Result<&'a Account, Error> {
+ let contact = account_contact_from_string(&contact);
+ Ok(client
+ .new_account(name, agree_to_tos, contact, rsa_bits)
+ .await?)
+}
+
+#[api(
+ input: {
+ properties: {
+ name: { type: AccountName },
+ contact: {
+ description: "List of email addresses.",
+ optional: true,
+ },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ protected: true,
+)]
+/// Update an ACME account.
+pub fn update_account(
+ name: AccountName,
+ // Todo: email & email-list schema
+ contact: Option<String>,
+ rpcenv: &mut dyn RpcEnvironment,
+) -> Result<String, Error> {
+ let auth_id: Authid = rpcenv.get_auth_id().unwrap().parse()?;
+
+ WorkerTask::spawn(
+ "acme-update",
+ None,
+ auth_id,
+ true,
+ move |_worker| async move {
+ let data = match contact {
+ Some(data) => json!({
+ "contact": account_contact_from_string(&data),
+ }),
+ None => json!({}),
+ };
+
+ AcmeClient::load(&name).await?.update_account(&data).await?;
+
+ Ok(())
+ },
+ )
+}
+
+#[api(
+ input: {
+ properties: {
+ name: { type: AccountName },
+ force: {
+ description:
+ "Delete account data even if the server refuses to deactivate the account.",
+ optional: true,
+ default: false,
+ },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ protected: true,
+)]
+/// Deactivate an ACME account.
+pub fn deactivate_account(
+ name: AccountName,
+ force: bool,
+ rpcenv: &mut dyn RpcEnvironment,
+) -> Result<String, Error> {
+ let auth_id: Authid = rpcenv.get_auth_id().unwrap().parse()?;
+
+ WorkerTask::spawn(
+ "acme-deactivate",
+ None,
+ auth_id,
+ true,
+ move |worker| async move {
+ match AcmeClient::load(&name)
+ .await?
+ .update_account(&json!({"status": "deactivated"}))
+ .await
+ {
+ Ok(_account) => (),
+ Err(err) if !force => return Err(err),
+ Err(err) => {
+ worker.warn(format!(
+ "error deactivating account {:?}, proceedeing anyway - {}",
+ name, err,
+ ));
+ }
+ }
+ crate::config::acme::mark_account_deactivated(&name)?;
+ Ok(())
+ },
+ )
+}
+
+#[api(
+ input: {
+ properties: {
+ directory: {
+ type: String,
+ description: "The ACME Directory.",
+ optional: true,
+ },
+ },
+ },
+ access: {
+ permission: &Permission::Anybody,
+ },
+ returns: {
+ type: String,
+ optional: true,
+ description: "The ACME Directory's ToS URL, if any.",
+ },
+)]
+/// Get the Terms of Service URL for an ACME directory.
+async fn get_tos(directory: Option<String>) -> Result<Option<String>, Error> {
+ let directory = directory.unwrap_or_else(|| {
+ crate::config::acme::DEFAULT_ACME_DIRECTORY_ENTRY
+ .url
+ .to_owned()
+ });
+ Ok(AcmeClient::new(directory)
+ .terms_of_service_url()
+ .await?
+ .map(str::to_owned))
+}
+
+#[api(
+ access: {
+ permission: &Permission::Anybody,
+ },
+ returns: {
+ description: "List of known ACME directories.",
+ type: Array,
+ items: { type: KnownAcmeDirectory },
+ },
+)]
+/// Get named known ACME directory endpoints.
+fn get_directories() -> Result<&'static [KnownAcmeDirectory], Error> {
+ Ok(crate::config::acme::KNOWN_ACME_DIRECTORIES)
+}
+
+#[api(
+ properties: {
+ schema: {
+ type: Object,
+ additional_properties: true,
+ properties: {},
+ },
+ type: {
+ type: String,
+ },
+ },
+)]
+#[derive(Serialize)]
+/// Schema for an ACME challenge plugin.
+pub struct ChallengeSchema {
+ /// Plugin ID.
+ id: String,
+
+ /// Human readable name, falls back to id.
+ name: String,
+
+ /// Plugin Type.
+ #[serde(rename = "type")]
+ ty: &'static str,
+
+ /// The plugin's parameter schema.
+ schema: Value,
+}
+
+#[api(
+ access: {
+ permission: &Permission::Anybody,
+ },
+ returns: {
+ description: "ACME Challenge Plugin Shema.",
+ type: Array,
+ items: { type: ChallengeSchema },
+ },
+)]
+/// Get named known ACME directory endpoints.
+fn get_challenge_schema() -> Result<Vec<ChallengeSchema>, Error> {
+ let mut out = Vec::new();
+ crate::config::acme::foreach_dns_plugin(|id| {
+ out.push(ChallengeSchema {
+ id: id.to_owned(),
+ name: id.to_owned(),
+ ty: "dns",
+ schema: Value::Object(Default::default()),
+ });
+ ControlFlow::Continue(())
+ })?;
+ Ok(out)
+}
+
+#[api]
+#[derive(Default, Deserialize, Serialize)]
+#[serde(rename_all = "kebab-case")]
+/// The API's format is inherited from PVE/PMG:
+pub struct PluginConfig {
+ /// Plugin ID.
+ plugin: String,
+
+ /// Plugin type.
+ #[serde(rename = "type")]
+ ty: String,
+
+ /// DNS Api name.
+ api: Option<String>,
+
+ /// Plugin configuration data.
+ data: Option<String>,
+
+ /// Extra delay in seconds to wait before requesting validation.
+ ///
+ /// Allows to cope with long TTL of DNS records.
+ #[serde(skip_serializing_if = "Option::is_none", default)]
+ validation_delay: Option<u32>,
+
+ /// Flag to disable the config.
+ #[serde(skip_serializing_if = "Option::is_none", default)]
+ disable: Option<bool>,
+}
+
+// See PMG/PVE's $modify_cfg_for_api sub
+fn modify_cfg_for_api(id: &str, ty: &str, data: &Value) -> PluginConfig {
+ let mut entry = data.clone();
+
+ let obj = entry.as_object_mut().unwrap();
+ obj.remove("id");
+ obj.insert("plugin".to_string(), Value::String(id.to_owned()));
+ obj.insert("type".to_string(), Value::String(ty.to_owned()));
+
+ // FIXME: This needs to go once the `Updater` is fixed.
+ // None of these should be able to fail unless the user changed the files by hand, in which
+ // case we leave the unmodified string in the Value for now. This will be handled with an error
+ // later.
+ if let Some(Value::String(ref mut data)) = obj.get_mut("data") {
+ if let Ok(new) = base64::decode_config(&data, base64::URL_SAFE_NO_PAD) {
+ if let Ok(utf8) = String::from_utf8(new) {
+ *data = utf8;
+ }
+ }
+ }
+
+ // PVE/PMG do this explicitly for ACME plugins...
+ // obj.insert("digest".to_string(), Value::String(digest.clone()));
+
+ serde_json::from_value(entry).unwrap_or_else(|_| PluginConfig {
+ plugin: "*Error*".to_string(),
+ ty: "*Error*".to_string(),
+ ..Default::default()
+ })
+}
+
+#[api(
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ protected: true,
+ returns: {
+ type: Array,
+ description: "List of ACME plugin configurations.",
+ items: { type: PluginConfig },
+ },
+)]
+/// List ACME challenge plugins.
+pub fn list_plugins(mut rpcenv: &mut dyn RpcEnvironment) -> Result<Vec<PluginConfig>, Error> {
+ use crate::config::acme::plugin;
+
+ let (plugins, digest) = plugin::config()?;
+ rpcenv["digest"] = proxmox::tools::digest_to_hex(&digest).into();
+ Ok(plugins
+ .iter()
+ .map(|(id, (ty, data))| modify_cfg_for_api(&id, &ty, data))
+ .collect())
+}
+
+#[api(
+ input: {
+ properties: {
+ id: { schema: PLUGIN_ID_SCHEMA },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ protected: true,
+ returns: { type: PluginConfig },
+)]
+/// List ACME challenge plugins.
+pub fn get_plugin(id: String, mut rpcenv: &mut dyn RpcEnvironment) -> Result<PluginConfig, Error> {
+ use crate::config::acme::plugin;
+
+ let (plugins, digest) = plugin::config()?;
+ rpcenv["digest"] = proxmox::tools::digest_to_hex(&digest).into();
+
+ match plugins.get(&id) {
+ Some((ty, data)) => Ok(modify_cfg_for_api(&id, &ty, &data)),
+ None => http_bail!(NOT_FOUND, "no such plugin"),
+ }
+}
+
+// Currently we only have "the" standalone plugin and DNS plugins so we can just flatten a
+// DnsPluginUpdater:
+//
+// FIXME: The 'id' parameter should not be "optional" in the schema.
+#[api(
+ input: {
+ properties: {
+ type: {
+ type: String,
+ description: "The ACME challenge plugin type.",
+ },
+ core: {
+ type: DnsPluginCoreUpdater,
+ flatten: true,
+ },
+ data: {
+ type: String,
+ // This is different in the API!
+ description: "DNS plugin data (base64 encoded with padding).",
+ },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ protected: true,
+)]
+/// Add ACME plugin configuration.
+pub fn add_plugin(r#type: String, core: DnsPluginCoreUpdater, data: String) -> Result<(), Error> {
+ use crate::config::acme::plugin;
+
+ // Currently we only support DNS plugins and the standalone plugin is "fixed":
+ if r#type != "dns" {
+ bail!("invalid ACME plugin type: {:?}", r#type);
+ }
+
+ let data = String::from_utf8(base64::decode(&data)?)
+ .map_err(|_| format_err!("data must be valid UTF-8"))?;
+ //core.api_fixup()?;
+
+ // FIXME: Solve the Updater with non-optional fields thing...
+ let id = core
+ .id
+ .clone()
+ .ok_or_else(|| format_err!("missing required 'id' parameter"))?;
+
+ let _lock = plugin::write_lock()?;
+
+ let (mut plugins, _digest) = plugin::config()?;
+ if plugins.contains_key(&id) {
+ bail!("ACME plugin ID {:?} already exists", id);
+ }
+
+ let plugin = serde_json::to_value(DnsPlugin {
+ core: DnsPluginCore::try_build_from(core)?,
+ data,
+ })?;
+
+ plugins.insert(id, r#type, plugin);
+
+ plugin::save_config(&plugins)?;
+
+ Ok(())
+}
+
+#[api(
+ input: {
+ properties: {
+ id: { schema: PLUGIN_ID_SCHEMA },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ protected: true,
+)]
+/// Delete an ACME plugin configuration.
+pub fn delete_plugin(id: String) -> Result<(), Error> {
+ use crate::config::acme::plugin;
+
+ let _lock = plugin::write_lock()?;
+
+ let (mut plugins, _digest) = plugin::config()?;
+ if plugins.remove(&id).is_none() {
+ http_bail!(NOT_FOUND, "no such plugin");
+ }
+ plugin::save_config(&plugins)?;
+
+ Ok(())
+}
+
+#[api(
+ input: {
+ properties: {
+ core_update: {
+ type: DnsPluginCoreUpdater,
+ flatten: true,
+ },
+ data: {
+ type: String,
+ optional: true,
+ // This is different in the API!
+ description: "DNS plugin data (base64 encoded with padding).",
+ },
+ digest: {
+ description: "Digest to protect against concurrent updates",
+ optional: true,
+ },
+ delete: {
+ description: "Options to remove from the configuration",
+ optional: true,
+ },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ protected: true,
+)]
+/// Update an ACME plugin configuration.
+pub fn update_plugin(
+ core_update: DnsPluginCoreUpdater,
+ data: Option<String>,
+ delete: Option<String>,
+ digest: Option<String>,
+) -> Result<(), Error> {
+ use crate::config::acme::plugin;
+
+ let data = data
+ .as_deref()
+ .map(base64::decode)
+ .transpose()?
+ .map(String::from_utf8)
+ .transpose()
+ .map_err(|_| format_err!("data must be valid UTF-8"))?;
+ //core_update.api_fixup()?;
+
+ // unwrap: the id is matched by this method's API path
+ let id = core_update.id.clone().unwrap();
+
+ let delete: Vec<&str> = delete
+ .as_deref()
+ .unwrap_or("")
+ .split(&[' ', ',', ';', '\0'][..])
+ .collect();
+
+ let _lock = plugin::write_lock()?;
+
+ let (mut plugins, expected_digest) = plugin::config()?;
+
+ if let Some(digest) = digest {
+ let digest = proxmox::tools::hex_to_digest(&digest)?;
+ crate::tools::detect_modified_configuration_file(&digest, &expected_digest)?;
+ }
+
+ match plugins.get_mut(&id) {
+ Some((ty, ref mut entry)) => {
+ if ty != "dns" {
+ bail!("cannot update plugin of type {:?}", ty);
+ }
+
+ let mut plugin: DnsPlugin = serde_json::from_value(entry.clone())?;
+ plugin.core.update_from(core_update, &delete)?;
+ if let Some(data) = data {
+ plugin.data = data;
+ }
+ *entry = serde_json::to_value(plugin)?;
+ }
+ None => http_bail!(NOT_FOUND, "no such plugin"),
+ }
+
+ plugin::save_config(&plugins)?;
+
+ Ok(())
+}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 18/23] add node/{node}/certificates api call
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (16 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 17/23] add config/acme api path Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 19/23] add node/{node}/config api path Wolfgang Bumiller
` (6 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
API like in PVE:
GET .../info => current cert information
POST .../custom => upload custom certificate
DELETE .../custom => delete custom certificate
POST .../acme/certificate => order acme certificate
PUT .../acme/certificate => renew expiring acme cert
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/api2/node.rs | 2 +
src/api2/node/certificates.rs | 572 ++++++++++++++++++++++++++++++++++
src/config.rs | 18 +-
3 files changed, 590 insertions(+), 2 deletions(-)
create mode 100644 src/api2/node/certificates.rs
diff --git a/src/api2/node.rs b/src/api2/node.rs
index 1f3e46a9..ebb51aaf 100644
--- a/src/api2/node.rs
+++ b/src/api2/node.rs
@@ -27,6 +27,7 @@ use crate::tools;
use crate::tools::ticket::{self, Empty, Ticket};
pub mod apt;
+pub mod certificates;
pub mod disks;
pub mod dns;
pub mod network;
@@ -314,6 +315,7 @@ fn upgrade_to_websocket(
pub const SUBDIRS: SubdirMap = &[
("apt", &apt::ROUTER),
+ ("certificates", &certificates::ROUTER),
("disks", &disks::ROUTER),
("dns", &dns::ROUTER),
("journal", &journal::ROUTER),
diff --git a/src/api2/node/certificates.rs b/src/api2/node/certificates.rs
new file mode 100644
index 00000000..4dd75027
--- /dev/null
+++ b/src/api2/node/certificates.rs
@@ -0,0 +1,572 @@
+use std::convert::TryFrom;
+use std::sync::Arc;
+use std::time::Duration;
+
+use anyhow::{bail, format_err, Error};
+use openssl::pkey::PKey;
+use openssl::x509::X509;
+use serde::{Deserialize, Serialize};
+
+use proxmox::api::router::SubdirMap;
+use proxmox::api::{api, Permission, Router, RpcEnvironment};
+use proxmox::list_subdirs_api_method;
+
+use crate::acme::AcmeClient;
+use crate::api2::types::Authid;
+use crate::api2::types::NODE_SCHEMA;
+use crate::config::acl::PRIV_SYS_MODIFY;
+use crate::config::node::{AcmeDomain, NodeConfig};
+use crate::server::WorkerTask;
+use crate::tools::cert;
+
+pub const ROUTER: Router = Router::new()
+ .get(&list_subdirs_api_method!(SUBDIRS))
+ .subdirs(SUBDIRS);
+
+const SUBDIRS: SubdirMap = &[
+ ("acme", &ACME_ROUTER),
+ (
+ "custom",
+ &Router::new()
+ .post(&API_METHOD_UPLOAD_CUSTOM_CERTIFICATE)
+ .delete(&API_METHOD_DELETE_CUSTOM_CERTIFICATE),
+ ),
+ ("info", &Router::new().get(&API_METHOD_GET_INFO)),
+];
+
+const ACME_ROUTER: Router = Router::new()
+ .get(&list_subdirs_api_method!(ACME_SUBDIRS))
+ .subdirs(ACME_SUBDIRS);
+
+const ACME_SUBDIRS: SubdirMap = &[(
+ "certificate",
+ &Router::new()
+ .post(&API_METHOD_NEW_ACME_CERT)
+ .put(&API_METHOD_RENEW_ACME_CERT),
+)];
+
+#[api(
+ properties: {
+ san: {
+ type: Array,
+ items: {
+ description: "A SubjectAlternateName entry.",
+ type: String,
+ },
+ },
+ },
+)]
+/// Certificate information.
+#[derive(Deserialize, Serialize)]
+#[serde(rename_all = "kebab-case")]
+pub struct CertificateInfo {
+ /// Certificate file name.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ filename: Option<String>,
+
+ /// Certificate subject name.
+ subject: String,
+
+ /// List of certificate's SubjectAlternativeName entries.
+ san: Vec<String>,
+
+ /// Certificate issuer name.
+ issuer: String,
+
+ /// Certificate's notBefore timestamp (UNIX epoch).
+ #[serde(skip_serializing_if = "Option::is_none")]
+ notbefore: Option<i64>,
+
+ /// Certificate's notAfter timestamp (UNIX epoch).
+ #[serde(skip_serializing_if = "Option::is_none")]
+ notafter: Option<i64>,
+
+ /// Certificate in PEM format.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pem: Option<String>,
+
+ /// Certificate's public key algorithm.
+ public_key_type: String,
+
+ /// Certificate's public key size if available.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ public_key_bits: Option<u32>,
+
+ /// The SSL Fingerprint.
+ fingerprint: Option<String>,
+}
+
+impl TryFrom<&cert::CertInfo> for CertificateInfo {
+ type Error = Error;
+
+ fn try_from(info: &cert::CertInfo) -> Result<Self, Self::Error> {
+ let pubkey = info.public_key()?;
+
+ Ok(Self {
+ filename: None,
+ subject: info.subject_name()?,
+ san: info
+ .subject_alt_names()
+ .map(|san| {
+ san.into_iter()
+ // FIXME: Support `.ipaddress()`?
+ .filter_map(|name| name.dnsname().map(str::to_owned))
+ .collect()
+ })
+ .unwrap_or_default(),
+ issuer: info.issuer_name()?,
+ notbefore: info.not_before_unix().ok(),
+ notafter: info.not_after_unix().ok(),
+ pem: None,
+ public_key_type: openssl::nid::Nid::from_raw(pubkey.id().as_raw())
+ .long_name()
+ .unwrap_or("<unsupported key type>")
+ .to_owned(),
+ public_key_bits: Some(pubkey.bits()),
+ fingerprint: Some(info.fingerprint()?),
+ })
+ }
+}
+
+fn get_certificate_pem() -> Result<String, Error> {
+ let cert_path = configdir!("/proxy.pem");
+ let cert_pem = proxmox::tools::fs::file_get_contents(&cert_path)?;
+ String::from_utf8(cert_pem)
+ .map_err(|_| format_err!("certificate in {:?} is not a valid PEM file", cert_path))
+}
+
+// to deduplicate error messages
+fn pem_to_cert_info(pem: &[u8]) -> Result<cert::CertInfo, Error> {
+ cert::CertInfo::from_pem(pem)
+ .map_err(|err| format_err!("error loading proxy certificate: {}", err))
+}
+
+#[api(
+ input: {
+ properties: {
+ node: { schema: NODE_SCHEMA },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ returns: {
+ type: Array,
+ items: { type: CertificateInfo },
+ description: "List of certificate infos.",
+ },
+)]
+/// Get certificate info.
+pub fn get_info() -> Result<Vec<CertificateInfo>, Error> {
+ let cert_pem = get_certificate_pem()?;
+ let cert = pem_to_cert_info(cert_pem.as_bytes())?;
+
+ Ok(vec![CertificateInfo {
+ filename: Some("proxy.pem".to_string()), // we only have the one
+ pem: Some(cert_pem),
+ ..CertificateInfo::try_from(&cert)?
+ }])
+}
+
+#[api(
+ input: {
+ properties: {
+ node: { schema: NODE_SCHEMA },
+ certificates: { description: "PEM encoded certificate (chain)." },
+ key: { description: "PEM encoded private key." },
+ restart: {
+ description: "Restart proxmox-backup-proxy",
+ optional: true,
+ default: false,
+ },
+ // FIXME: widget-toolkit should have an option to disable using this parameter...
+ force: {
+ description: "Force replacement of existing files.",
+ type: Boolean,
+ optional: true,
+ default: false,
+ },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ returns: {
+ type: Array,
+ items: { type: CertificateInfo },
+ description: "List of certificate infos.",
+ },
+ protected: true,
+)]
+/// Upload a custom certificate.
+pub fn upload_custom_certificate(
+ certificates: String,
+ key: String,
+ restart: bool,
+) -> Result<Vec<CertificateInfo>, Error> {
+ let certificates = X509::stack_from_pem(certificates.as_bytes())
+ .map_err(|err| format_err!("failed to decode certificate chain: {}", err))?;
+ let key = PKey::private_key_from_pem(key.as_bytes())
+ .map_err(|err| format_err!("failed to parse private key: {}", err))?;
+
+ let certificates = certificates
+ .into_iter()
+ .try_fold(Vec::<u8>::new(), |mut stack, cert| -> Result<_, Error> {
+ if !stack.is_empty() {
+ stack.push(b'\n');
+ }
+ stack.extend(cert.to_pem()?);
+ Ok(stack)
+ })
+ .map_err(|err| format_err!("error formatting certificate chain as PEM: {}", err))?;
+
+ let key = key.private_key_to_pem_pkcs8()?;
+
+ crate::config::set_proxy_certificate(&certificates, &key, restart)?;
+
+ get_info()
+}
+
+#[api(
+ input: {
+ properties: {
+ node: { schema: NODE_SCHEMA },
+ restart: {
+ description: "Restart proxmox-backup-proxy",
+ optional: true,
+ default: false,
+ },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ protected: true,
+)]
+/// Delete the current certificate and regenerate a self signed one.
+pub fn delete_custom_certificate(restart: bool) -> Result<(), Error> {
+ let cert_path = configdir!("/proxy.pem");
+ // Here we fail since if this fails nothing else breaks anyway
+ std::fs::remove_file(&cert_path)
+ .map_err(|err| format_err!("failed to unlink {:?} - {}", cert_path, err))?;
+
+ let key_path = configdir!("/proxy.key");
+ if let Err(err) = std::fs::remove_file(&key_path) {
+ // Here we just log since the certificate is already gone and we'd rather try to generate
+ // the self-signed certificate even if this fails:
+ log::error!(
+ "failed to remove certificate private key {:?} - {}",
+ key_path,
+ err
+ );
+ }
+
+ crate::config::update_self_signed_cert(true)?;
+
+ if restart {
+ crate::config::reload_proxy()?;
+ }
+
+ Ok(())
+}
+
+struct OrderedCertificate {
+ certificate: hyper::body::Bytes,
+ private_key_pem: Vec<u8>,
+}
+
+async fn order_certificate(
+ worker: Arc<WorkerTask>,
+ node_config: &NodeConfig,
+) -> Result<Option<OrderedCertificate>, Error> {
+ use proxmox_acme_rs::authorization::Status;
+ use proxmox_acme_rs::order::Identifier;
+
+ let domains = node_config.acme_domains().try_fold(
+ Vec::<AcmeDomain>::new(),
+ |mut acc, domain| -> Result<_, Error> {
+ let mut domain = domain?;
+ domain.domain.make_ascii_lowercase();
+ if let Some(alias) = &mut domain.alias {
+ alias.make_ascii_lowercase();
+ }
+ acc.push(domain);
+ Ok(acc)
+ },
+ )?;
+
+ let get_domain_config = |domain: &str| {
+ domains
+ .iter()
+ .find(|d| d.domain == domain)
+ .ok_or_else(|| format_err!("no config for domain '{}'", domain))
+ };
+
+ if domains.is_empty() {
+ worker.log("No domains configured to be ordered from an ACME server.");
+ return Ok(None);
+ }
+
+ let (plugins, _) = crate::config::acme::plugin::config()?;
+
+ let mut acme = node_config.acme_client().await?;
+
+ worker.log("Placing ACME order");
+ let order = acme
+ .new_order(domains.iter().map(|d| d.domain.to_ascii_lowercase()))
+ .await?;
+ worker.log(format!("Order URL: {}", order.location));
+
+ let identifiers: Vec<String> = order
+ .data
+ .identifiers
+ .iter()
+ .map(|identifier| match identifier {
+ Identifier::Dns(domain) => domain.clone(),
+ })
+ .collect();
+
+ for auth_url in &order.data.authorizations {
+ worker.log(format!("Getting authorization details from '{}'", auth_url));
+ let mut auth = acme.get_authorization(&auth_url).await?;
+
+ let domain = match &mut auth.identifier {
+ Identifier::Dns(domain) => domain.to_ascii_lowercase(),
+ };
+
+ if auth.status == Status::Valid {
+ worker.log(format!("{} is already validated!", domain));
+ continue;
+ }
+
+ worker.log(format!("The validation for {} is pending", domain));
+ let domain_config: &AcmeDomain = get_domain_config(&domain)?;
+ let plugin_id = domain_config.plugin.as_deref().unwrap_or("standalone");
+ let plugin_cfg = plugins.get_plugin(plugin_id)?.ok_or_else(|| {
+ format_err!("plugin '{}' for domain '{}' not found!", plugin_id, domain)
+ })?;
+
+ worker.log("Setting up validation plugin");
+ let validation_url = plugin_cfg.setup(&mut acme, &auth, domain_config).await?;
+
+ let result = request_validation(&worker, &mut acme, auth_url, validation_url).await;
+
+ if let Err(err) = plugin_cfg.teardown(&mut acme, &auth, domain_config).await {
+ worker.warn(format!(
+ "Failed to teardown plugin '{}' for domain '{}' - {}",
+ plugin_id, domain, err
+ ));
+ }
+
+ let _: () = result?;
+ }
+
+ worker.log("All domains validated");
+ worker.log("Creating CSR");
+
+ let csr = proxmox_acme_rs::util::Csr::generate(&identifiers, &Default::default())?;
+ let mut finalize_error_cnt = 0u8;
+ let order_url = &order.location;
+ let mut order;
+ loop {
+ use proxmox_acme_rs::order::Status;
+
+ order = acme.get_order(order_url).await?;
+
+ match order.status {
+ Status::Pending => {
+ worker.log("still pending, trying to finalize anyway");
+ let finalize = order
+ .finalize
+ .as_deref()
+ .ok_or_else(|| format_err!("missing 'finalize' URL in order"))?;
+ if let Err(err) = acme.finalize(finalize, &csr.data).await {
+ if finalize_error_cnt >= 5 {
+ return Err(err.into());
+ }
+
+ finalize_error_cnt += 1;
+ }
+ tokio::time::sleep(Duration::from_secs(5)).await;
+ }
+ Status::Ready => {
+ worker.log("order is ready, finalizing");
+ let finalize = order
+ .finalize
+ .as_deref()
+ .ok_or_else(|| format_err!("missing 'finalize' URL in order"))?;
+ acme.finalize(finalize, &csr.data).await?;
+ tokio::time::sleep(Duration::from_secs(5)).await;
+ }
+ Status::Processing => {
+ worker.log("still processing, trying again in 30 seconds");
+ tokio::time::sleep(Duration::from_secs(30)).await;
+ }
+ Status::Valid => {
+ worker.log("valid");
+ break;
+ }
+ other => bail!("order status: {:?}", other),
+ }
+ }
+
+ worker.log("Downloading certificate");
+ let certificate = acme
+ .get_certificate(
+ order
+ .certificate
+ .as_deref()
+ .ok_or_else(|| format_err!("missing certificate url in finalized order"))?,
+ )
+ .await?;
+
+ Ok(Some(OrderedCertificate {
+ certificate,
+ private_key_pem: csr.private_key_pem,
+ }))
+}
+
+async fn request_validation(
+ worker: &WorkerTask,
+ acme: &mut AcmeClient,
+ auth_url: &str,
+ validation_url: &str,
+) -> Result<(), Error> {
+ worker.log("Triggering validation");
+ acme.request_challenge_validation(&validation_url).await?;
+
+ worker.log("Sleeping for 5 seconds");
+ tokio::time::sleep(Duration::from_secs(5)).await;
+
+ loop {
+ use proxmox_acme_rs::authorization::Status;
+
+ let auth = acme.get_authorization(&auth_url).await?;
+ match auth.status {
+ Status::Pending => {
+ worker.log("Status is still 'pending', trying again in 10 seconds");
+ tokio::time::sleep(Duration::from_secs(10)).await;
+ }
+ Status::Valid => return Ok(()),
+ other => bail!(
+ "validating challenge '{}' failed - status: {:?}",
+ validation_url,
+ other
+ ),
+ }
+ }
+}
+
+#[api(
+ input: {
+ properties: {
+ node: { schema: NODE_SCHEMA },
+ force: {
+ description: "Force replacement of existing files.",
+ type: Boolean,
+ optional: true,
+ default: false,
+ },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ protected: true,
+)]
+/// Order a new ACME certificate.
+pub fn new_acme_cert(force: bool, rpcenv: &mut dyn RpcEnvironment) -> Result<String, Error> {
+ spawn_certificate_worker("acme-new-cert", force, rpcenv)
+}
+
+#[api(
+ input: {
+ properties: {
+ node: { schema: NODE_SCHEMA },
+ force: {
+ description: "Force replacement of existing files.",
+ type: Boolean,
+ optional: true,
+ default: false,
+ },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ protected: true,
+)]
+/// Renew the current ACME certificate if it expires within 30 days (or always if the `force`
+/// parameter is set).
+pub fn renew_acme_cert(force: bool, rpcenv: &mut dyn RpcEnvironment) -> Result<String, Error> {
+ if !cert_expires_soon()? && !force {
+ bail!("Certificate does not expire within the next 30 days and 'force' is not set.")
+ }
+
+ spawn_certificate_worker("acme-renew-cert", force, rpcenv)
+}
+
+/// Check whether the current certificate expires within the next 30 days.
+pub fn cert_expires_soon() -> Result<bool, Error> {
+ let cert = pem_to_cert_info(get_certificate_pem()?.as_bytes())?;
+ cert.is_expired_after_epoch(proxmox::tools::time::epoch_i64() + 30 * 24 * 60 * 60)
+ .map_err(|err| format_err!("Failed to check certificate expiration date: {}", err))
+}
+
+fn spawn_certificate_worker(
+ name: &'static str,
+ force: bool,
+ rpcenv: &mut dyn RpcEnvironment,
+) -> Result<String, Error> {
+ // We only have 1 certificate path in PBS which makes figuring out whether or not it is a
+ // custom one too hard... We keep the parameter because the widget-toolkit may be using it...
+ let _ = force;
+
+ let (node_config, _digest) = crate::config::node::config()?;
+
+ let auth_id: Authid = rpcenv.get_auth_id().unwrap().parse()?;
+
+ WorkerTask::spawn(name, None, auth_id, true, move |worker| async move {
+ if let Some(cert) = order_certificate(worker, &node_config).await? {
+ crate::config::set_proxy_certificate(&cert.certificate, &cert.private_key_pem, true)?;
+ }
+ Ok(())
+ })
+}
+
+#[api(
+ input: {
+ properties: {
+ node: { schema: NODE_SCHEMA },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "certificates"], PRIV_SYS_MODIFY, false),
+ },
+ protected: true,
+)]
+/// Renew the current ACME certificate if it expires within 30 days (or always if the `force`
+/// parameter is set).
+pub fn revoke_acme_cert(rpcenv: &mut dyn RpcEnvironment) -> Result<String, Error> {
+ let (node_config, _digest) = crate::config::node::config()?;
+
+ let cert_pem = get_certificate_pem()?;
+
+ let auth_id: Authid = rpcenv.get_auth_id().unwrap().parse()?;
+
+ WorkerTask::spawn(
+ "acme-revoke-cert",
+ None,
+ auth_id,
+ true,
+ move |worker| async move {
+ worker.log("Loading ACME account");
+ let mut acme = node_config.acme_client().await?;
+ worker.log("Revoking old certificate");
+ acme.revoke_certificate(cert_pem.as_bytes(), None).await?;
+ worker.log("Deleting certificate and regenerating a self-signed one");
+ delete_custom_certificate(true)?;
+ Ok(())
+ },
+ )
+}
diff --git a/src/config.rs b/src/config.rs
index 94b7fb6c..22c293c9 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -187,12 +187,16 @@ pub fn update_self_signed_cert(force: bool) -> Result<(), Error> {
let x509 = x509.build();
let cert_pem = x509.to_pem()?;
- set_proxy_certificate(&cert_pem, &priv_pem)?;
+ set_proxy_certificate(&cert_pem, &priv_pem, false)?;
Ok(())
}
-pub(crate) fn set_proxy_certificate(cert_pem: &[u8], key_pem: &[u8]) -> Result<(), Error> {
+pub(crate) fn set_proxy_certificate(
+ cert_pem: &[u8],
+ key_pem: &[u8],
+ reload: bool,
+) -> Result<(), Error> {
let backup_user = crate::backup::backup_user()?;
let options = CreateOptions::new()
.perm(Mode::from_bits_truncate(0o0640))
@@ -206,5 +210,15 @@ pub(crate) fn set_proxy_certificate(cert_pem: &[u8], key_pem: &[u8]) -> Result<(
.map_err(|err| format_err!("error writing certificate private key - {}", err))?;
replace_file(&cert_path, &cert_pem, options)
.map_err(|err| format_err!("error writing certificate file - {}", err))?;
+
+ if reload {
+ reload_proxy()?;
+ }
+
Ok(())
}
+
+pub(crate) fn reload_proxy() -> Result<(), Error> {
+ crate::tools::systemd::reload_unit("proxmox-backup-proxy")
+ .map_err(|err| format_err!("error signaling reload to pbs proxy: {}", err))
+}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 19/23] add node/{node}/config api path
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (17 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 18/23] add node/{node}/certificates api call Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 20/23] add acme commands to proxmox-backup-manager Wolfgang Bumiller
` (5 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/api2/node.rs | 2 +
src/api2/node/config.rs | 81 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 83 insertions(+)
create mode 100644 src/api2/node/config.rs
diff --git a/src/api2/node.rs b/src/api2/node.rs
index ebb51aaf..75271cd5 100644
--- a/src/api2/node.rs
+++ b/src/api2/node.rs
@@ -28,6 +28,7 @@ use crate::tools::ticket::{self, Empty, Ticket};
pub mod apt;
pub mod certificates;
+pub mod config;
pub mod disks;
pub mod dns;
pub mod network;
@@ -316,6 +317,7 @@ fn upgrade_to_websocket(
pub const SUBDIRS: SubdirMap = &[
("apt", &apt::ROUTER),
("certificates", &certificates::ROUTER),
+ ("config", &config::ROUTER),
("disks", &disks::ROUTER),
("dns", &dns::ROUTER),
("journal", &journal::ROUTER),
diff --git a/src/api2/node/config.rs b/src/api2/node/config.rs
new file mode 100644
index 00000000..2e7fd670
--- /dev/null
+++ b/src/api2/node/config.rs
@@ -0,0 +1,81 @@
+use anyhow::Error;
+use serde_json::Value;
+
+use proxmox::api::schema::Updatable;
+use proxmox::api::{api, Permission, Router, RpcEnvironment};
+
+use crate::api2::types::NODE_SCHEMA;
+use crate::config::acl::PRIV_SYS_MODIFY;
+use crate::config::node::NodeConfigUpdater;
+
+pub const ROUTER: Router = Router::new()
+ .get(&API_METHOD_GET_NODE_CONFIG)
+ .put(&API_METHOD_UPDATE_NODE_CONFIG);
+
+#[api(
+ input: {
+ properties: {
+ node: { schema: NODE_SCHEMA },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system"], PRIV_SYS_MODIFY, false),
+ },
+)]
+/// Create a new changer device.
+pub fn get_node_config(mut rpcenv: &mut dyn RpcEnvironment) -> Result<Value, Error> {
+ let _lock = crate::config::node::read_lock()?;
+ let (config, digest) = crate::config::node::config()?;
+ rpcenv["digest"] = proxmox::tools::digest_to_hex(&digest).into();
+ Ok(serde_json::to_value(config)?)
+}
+
+#[api(
+ input: {
+ properties: {
+ node: { schema: NODE_SCHEMA },
+ digest: {
+ description: "Digest to protect against concurrent updates",
+ optional: true,
+ },
+ updater: {
+ type: NodeConfigUpdater,
+ flatten: true,
+ },
+ delete: {
+ description: "Options to remove from the configuration",
+ optional: true,
+ },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system"], PRIV_SYS_MODIFY, false),
+ },
+ protected: true,
+)]
+/// Create a new changer device.
+pub fn update_node_config(
+ updater: NodeConfigUpdater,
+ delete: Option<String>,
+ digest: Option<String>,
+) -> Result<(), Error> {
+ let _lock = crate::config::node::write_lock()?;
+ let (mut config, expected_digest) = crate::config::node::config()?;
+ if let Some(digest) = digest {
+ // FIXME: GUI doesn't handle our non-inlined digest part here properly...
+ if !digest.is_empty() {
+ let digest = proxmox::tools::hex_to_digest(&digest)?;
+ crate::tools::detect_modified_configuration_file(&digest, &expected_digest)?;
+ }
+ }
+
+ let delete: Vec<&str> = delete
+ .as_deref()
+ .unwrap_or("")
+ .split(&[' ', ',', ';', '\0'][..])
+ .collect();
+
+ config.update_from(updater, &delete)?;
+
+ crate::config::node::save_config(&config)
+}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 20/23] add acme commands to proxmox-backup-manager
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (18 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 19/23] add node/{node}/config api path Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 21/23] implement standalone acme validation Wolfgang Bumiller
` (4 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/bin/proxmox-backup-manager.rs | 1 +
src/bin/proxmox_backup_manager/acme.rs | 414 +++++++++++++++++++++++++
src/bin/proxmox_backup_manager/mod.rs | 2 +
3 files changed, 417 insertions(+)
create mode 100644 src/bin/proxmox_backup_manager/acme.rs
diff --git a/src/bin/proxmox-backup-manager.rs b/src/bin/proxmox-backup-manager.rs
index 105a11f8..522c800e 100644
--- a/src/bin/proxmox-backup-manager.rs
+++ b/src/bin/proxmox-backup-manager.rs
@@ -355,6 +355,7 @@ fn main() {
.insert("user", user_commands())
.insert("remote", remote_commands())
.insert("garbage-collection", garbage_collection_commands())
+ .insert("acme", acme_mgmt_cli())
.insert("cert", cert_mgmt_cli())
.insert("subscription", subscription_commands())
.insert("sync-job", sync_job_commands())
diff --git a/src/bin/proxmox_backup_manager/acme.rs b/src/bin/proxmox_backup_manager/acme.rs
new file mode 100644
index 00000000..bb8fb9b3
--- /dev/null
+++ b/src/bin/proxmox_backup_manager/acme.rs
@@ -0,0 +1,414 @@
+use std::io::Write;
+
+use anyhow::{bail, Error};
+use serde_json::Value;
+
+use proxmox::api::{api, cli::*, ApiHandler, RpcEnvironment};
+use proxmox::tools::fs::file_get_contents;
+
+use proxmox_backup::acme::AcmeClient;
+use proxmox_backup::api2;
+use proxmox_backup::config::acme::plugin::DnsPluginCoreUpdater;
+use proxmox_backup::config::acme::{AccountName, KNOWN_ACME_DIRECTORIES};
+
+pub fn acme_mgmt_cli() -> CommandLineInterface {
+ let cmd_def = CliCommandMap::new()
+ .insert("account", account_cli())
+ .insert("cert", cert_cli())
+ .insert("plugin", plugin_cli());
+
+ cmd_def.into()
+}
+
+#[api(
+ input: {
+ properties: {
+ "output-format": {
+ schema: OUTPUT_FORMAT,
+ optional: true,
+ },
+ }
+ }
+)]
+/// List acme accounts.
+fn list_accounts(param: Value, rpcenv: &mut dyn RpcEnvironment) -> Result<(), Error> {
+ let output_format = get_output_format(¶m);
+
+ let info = &api2::config::acme::API_METHOD_LIST_ACCOUNTS;
+ let mut data = match info.handler {
+ ApiHandler::Sync(handler) => (handler)(param, info, rpcenv)?,
+ _ => unreachable!(),
+ };
+
+ let options = default_table_format_options();
+ format_and_print_result_full(&mut data, &info.returns, &output_format, &options);
+
+ Ok(())
+}
+
+#[api(
+ input: {
+ properties: {
+ name: { type: AccountName },
+ "output-format": {
+ schema: OUTPUT_FORMAT,
+ optional: true,
+ },
+ }
+ }
+)]
+/// Show acme account information.
+async fn get_account(param: Value, rpcenv: &mut dyn RpcEnvironment) -> Result<(), Error> {
+ let output_format = get_output_format(¶m);
+
+ let info = &api2::config::acme::API_METHOD_GET_ACCOUNT;
+ let mut data = match info.handler {
+ ApiHandler::Async(handler) => (handler)(param, info, rpcenv).await?,
+ _ => unreachable!(),
+ };
+
+ let options = default_table_format_options()
+ .column(
+ ColumnConfig::new("account")
+ .renderer(|value, _record| Ok(serde_json::to_string_pretty(value)?)),
+ )
+ .column(ColumnConfig::new("directory"))
+ .column(ColumnConfig::new("location"))
+ .column(ColumnConfig::new("tos"));
+ format_and_print_result_full(&mut data, &info.returns, &output_format, &options);
+
+ Ok(())
+}
+
+#[api(
+ input: {
+ properties: {
+ name: { type: AccountName },
+ contact: {
+ description: "List of email addresses.",
+ },
+ directory: {
+ type: String,
+ description: "The ACME Directory.",
+ optional: true,
+ },
+ }
+ }
+)]
+/// Register an ACME account.
+async fn register_account(
+ name: AccountName,
+ contact: String,
+ directory: Option<String>,
+) -> Result<(), Error> {
+ let directory = match directory {
+ Some(directory) => directory,
+ None => {
+ println!("Directory endpoints:");
+ for (i, dir) in KNOWN_ACME_DIRECTORIES.iter().enumerate() {
+ println!("{}) {}", i, dir.url);
+ }
+
+ println!("{}) Custom", KNOWN_ACME_DIRECTORIES.len());
+ let mut attempt = 0;
+ loop {
+ print!("Enter selection: ");
+ std::io::stdout().flush()?;
+
+ let mut input = String::new();
+ std::io::stdin().read_line(&mut input)?;
+
+ match input.trim().parse::<usize>() {
+ Ok(n) if n < KNOWN_ACME_DIRECTORIES.len() => {
+ break KNOWN_ACME_DIRECTORIES[n].url.to_owned();
+ }
+ Ok(n) if n == KNOWN_ACME_DIRECTORIES.len() => {
+ input.clear();
+ std::io::stdin().read_line(&mut input)?;
+ break input.trim().to_owned();
+ }
+ _ => eprintln!("Invalid selection."),
+ }
+
+ attempt += 1;
+ if attempt >= 3 {
+ bail!("Aborting.");
+ }
+ }
+ }
+ };
+
+ println!("Attempting to fetch Terms of Service from {:?}", directory);
+ let mut client = AcmeClient::new(directory.clone());
+ let tos_agreed = if let Some(tos_url) = client.terms_of_service_url().await? {
+ println!("Terms of Service: {}", tos_url);
+ print!("Do you agree to the above terms? [y|N]: ");
+ std::io::stdout().flush()?;
+ let mut input = String::new();
+ std::io::stdin().read_line(&mut input)?;
+ if input.trim().eq_ignore_ascii_case("y") {
+ true
+ } else {
+ false
+ }
+ } else {
+ false
+ };
+
+ println!("Attempting to register account with {:?}...", directory);
+
+ let account =
+ api2::config::acme::do_register_account(&mut client, &name, tos_agreed, contact, None)
+ .await?;
+
+ println!("Registration successful, account URL: {}", account.location);
+
+ Ok(())
+}
+
+#[api(
+ input: {
+ properties: {
+ name: { type: AccountName },
+ contact: {
+ description: "List of email addresses.",
+ type: String,
+ optional: true,
+ },
+ }
+ }
+)]
+/// Update an ACME account.
+async fn update_account(param: Value, rpcenv: &mut dyn RpcEnvironment) -> Result<(), Error> {
+ let info = &api2::config::acme::API_METHOD_UPDATE_ACCOUNT;
+ let result = match info.handler {
+ ApiHandler::Sync(handler) => (handler)(param, info, rpcenv)?,
+ _ => unreachable!(),
+ };
+
+ crate::wait_for_local_worker(result.as_str().unwrap()).await?;
+
+ Ok(())
+}
+
+#[api(
+ input: {
+ properties: {
+ name: { type: AccountName },
+ force: {
+ description:
+ "Delete account data even if the server refuses to deactivate the account.",
+ type: Boolean,
+ optional: true,
+ default: false,
+ },
+ }
+ }
+)]
+/// Deactivate an ACME account.
+async fn deactivate_account(param: Value, rpcenv: &mut dyn RpcEnvironment) -> Result<(), Error> {
+ let info = &api2::config::acme::API_METHOD_DEACTIVATE_ACCOUNT;
+ let result = match info.handler {
+ ApiHandler::Sync(handler) => (handler)(param, info, rpcenv)?,
+ _ => unreachable!(),
+ };
+
+ crate::wait_for_local_worker(result.as_str().unwrap()).await?;
+
+ Ok(())
+}
+
+pub fn account_cli() -> CommandLineInterface {
+ let cmd_def = CliCommandMap::new()
+ .insert("list", CliCommand::new(&API_METHOD_LIST_ACCOUNTS))
+ .insert(
+ "register",
+ CliCommand::new(&API_METHOD_REGISTER_ACCOUNT).arg_param(&["name", "contact"]),
+ )
+ .insert(
+ "deactivate",
+ CliCommand::new(&API_METHOD_DEACTIVATE_ACCOUNT)
+ .arg_param(&["name"])
+ .completion_cb("name", crate::config::acme::complete_acme_account),
+ )
+ .insert(
+ "info",
+ CliCommand::new(&API_METHOD_GET_ACCOUNT)
+ .arg_param(&["name"])
+ .completion_cb("name", crate::config::acme::complete_acme_account),
+ )
+ .insert(
+ "update",
+ CliCommand::new(&API_METHOD_UPDATE_ACCOUNT)
+ .arg_param(&["name"])
+ .completion_cb("name", crate::config::acme::complete_acme_account),
+ );
+
+ cmd_def.into()
+}
+
+#[api(
+ input: {
+ properties: {
+ "output-format": {
+ schema: OUTPUT_FORMAT,
+ optional: true,
+ },
+ }
+ }
+)]
+/// List acme plugins.
+fn list_plugins(param: Value, rpcenv: &mut dyn RpcEnvironment) -> Result<(), Error> {
+ let output_format = get_output_format(¶m);
+
+ let info = &api2::config::acme::API_METHOD_LIST_PLUGINS;
+ let mut data = match info.handler {
+ ApiHandler::Sync(handler) => (handler)(param, info, rpcenv)?,
+ _ => unreachable!(),
+ };
+
+ let options = default_table_format_options();
+ format_and_print_result_full(&mut data, &info.returns, &output_format, &options);
+
+ Ok(())
+}
+
+#[api(
+ input: {
+ properties: {
+ id: {
+ type: String,
+ description: "Plugin ID",
+ },
+ "output-format": {
+ schema: OUTPUT_FORMAT,
+ optional: true,
+ },
+ }
+ }
+)]
+/// Show acme account information.
+fn get_plugin(param: Value, rpcenv: &mut dyn RpcEnvironment) -> Result<(), Error> {
+ let output_format = get_output_format(¶m);
+
+ let info = &api2::config::acme::API_METHOD_GET_PLUGIN;
+ let mut data = match info.handler {
+ ApiHandler::Sync(handler) => (handler)(param, info, rpcenv)?,
+ _ => unreachable!(),
+ };
+
+ let options = default_table_format_options();
+ format_and_print_result_full(&mut data, &info.returns, &output_format, &options);
+
+ Ok(())
+}
+
+#[api(
+ input: {
+ properties: {
+ type: {
+ type: String,
+ description: "The ACME challenge plugin type.",
+ },
+ core: {
+ type: DnsPluginCoreUpdater,
+ flatten: true,
+ },
+ data: {
+ type: String,
+ description: "File containing the plugin data.",
+ },
+ }
+ }
+)]
+/// Show acme account information.
+fn add_plugin(r#type: String, core: DnsPluginCoreUpdater, data: String) -> Result<(), Error> {
+ let data = base64::encode(&file_get_contents(&data)?);
+ api2::config::acme::add_plugin(r#type, core, data)?;
+ Ok(())
+}
+
+pub fn plugin_cli() -> CommandLineInterface {
+ use proxmox_backup::api2::config::acme;
+ let cmd_def = CliCommandMap::new()
+ .insert("list", CliCommand::new(&API_METHOD_LIST_PLUGINS))
+ .insert(
+ "config", // name comes from pve/pmg
+ CliCommand::new(&API_METHOD_GET_PLUGIN)
+ .arg_param(&["id"])
+ .completion_cb("id", crate::config::acme::complete_acme_plugin),
+ )
+ .insert(
+ "add",
+ CliCommand::new(&API_METHOD_ADD_PLUGIN)
+ .arg_param(&["type", "id"])
+ .completion_cb("id", crate::config::acme::complete_acme_plugin)
+ .completion_cb("type", crate::config::acme::complete_acme_plugin_type),
+ )
+ .insert(
+ "remove",
+ CliCommand::new(&acme::API_METHOD_DELETE_PLUGIN)
+ .arg_param(&["id"])
+ .completion_cb("id", crate::config::acme::complete_acme_plugin),
+ )
+ .insert(
+ "set",
+ CliCommand::new(&acme::API_METHOD_UPDATE_PLUGIN)
+ .arg_param(&["id"])
+ .completion_cb("id", crate::config::acme::complete_acme_plugin),
+ );
+
+ cmd_def.into()
+}
+
+#[api(
+ input: {
+ properties: {
+ force: {
+ description: "Force renewal even if the certificate does not expire soon.",
+ type: Boolean,
+ optional: true,
+ default: false,
+ },
+ },
+ },
+)]
+/// Order a new ACME certificate.
+async fn order_acme_cert(param: Value, rpcenv: &mut dyn RpcEnvironment) -> Result<(), Error> {
+ if !api2::node::certificates::cert_expires_soon()? {
+ println!("Certificate does not expire within the next 30 days, not renewing.");
+ return Ok(());
+ }
+
+ let info = &api2::node::certificates::API_METHOD_RENEW_ACME_CERT;
+ let result = match info.handler {
+ ApiHandler::Sync(handler) => (handler)(param, info, rpcenv)?,
+ _ => unreachable!(),
+ };
+
+ crate::wait_for_local_worker(result.as_str().unwrap()).await?;
+
+ Ok(())
+}
+
+#[api]
+/// Order a new ACME certificate.
+async fn revoke_acme_cert(param: Value, rpcenv: &mut dyn RpcEnvironment) -> Result<(), Error> {
+ let info = &api2::node::certificates::API_METHOD_REVOKE_ACME_CERT;
+ let result = match info.handler {
+ ApiHandler::Sync(handler) => (handler)(param, info, rpcenv)?,
+ _ => unreachable!(),
+ };
+
+ crate::wait_for_local_worker(result.as_str().unwrap()).await?;
+
+ Ok(())
+}
+
+pub fn cert_cli() -> CommandLineInterface {
+ let cmd_def = CliCommandMap::new()
+ .insert("order", CliCommand::new(&API_METHOD_ORDER_ACME_CERT))
+ .insert("revoke", CliCommand::new(&API_METHOD_REVOKE_ACME_CERT));
+
+ cmd_def.into()
+}
diff --git a/src/bin/proxmox_backup_manager/mod.rs b/src/bin/proxmox_backup_manager/mod.rs
index 900144aa..e574e4d4 100644
--- a/src/bin/proxmox_backup_manager/mod.rs
+++ b/src/bin/proxmox_backup_manager/mod.rs
@@ -1,5 +1,7 @@
mod acl;
pub use acl::*;
+mod acme;
+pub use acme::*;
mod cert;
pub use cert::*;
mod datastore;
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 21/23] implement standalone acme validation
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (19 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 20/23] add acme commands to proxmox-backup-manager Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 22/23] ui: add certificate & acme view Wolfgang Bumiller
` (3 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/api2/node/certificates.rs | 2 +-
src/config/acme/plugin.rs | 144 ++++++++++++++++++++++++++++++----
2 files changed, 129 insertions(+), 17 deletions(-)
diff --git a/src/api2/node/certificates.rs b/src/api2/node/certificates.rs
index 4dd75027..f6da31ec 100644
--- a/src/api2/node/certificates.rs
+++ b/src/api2/node/certificates.rs
@@ -342,7 +342,7 @@ async fn order_certificate(
worker.log(format!("The validation for {} is pending", domain));
let domain_config: &AcmeDomain = get_domain_config(&domain)?;
let plugin_id = domain_config.plugin.as_deref().unwrap_or("standalone");
- let plugin_cfg = plugins.get_plugin(plugin_id)?.ok_or_else(|| {
+ let mut plugin_cfg = plugins.get_plugin(plugin_id)?.ok_or_else(|| {
format_err!("plugin '{}' for domain '{}' not found!", plugin_id, domain)
})?;
diff --git a/src/config/acme/plugin.rs b/src/config/acme/plugin.rs
index b0c655c0..a2fa0490 100644
--- a/src/config/acme/plugin.rs
+++ b/src/config/acme/plugin.rs
@@ -1,8 +1,10 @@
use std::future::Future;
use std::pin::Pin;
use std::process::Stdio;
+use std::sync::Arc;
use anyhow::{bail, format_err, Error};
+use hyper::{Body, Request, Response};
use lazy_static::lazy_static;
use serde::{Deserialize, Serialize};
use serde_json::Value;
@@ -117,7 +119,11 @@ pub struct DnsPlugin {
impl DnsPlugin {
pub fn decode_data(&self, output: &mut Vec<u8>) -> Result<(), Error> {
- Ok(base64::decode_config_buf(&self.data, base64::URL_SAFE_NO_PAD, output)?)
+ Ok(base64::decode_config_buf(
+ &self.data,
+ base64::URL_SAFE_NO_PAD,
+ output,
+ )?)
}
}
@@ -250,7 +256,10 @@ impl PluginData {
let plugin: DnsPlugin = serde_json::from_value(data.clone())?;
Box::new(plugin)
}
- // "standalone" => todo!("standalone plugin"),
+ "standalone" => {
+ // this one has no config
+ Box::new(StandaloneServer::default())
+ }
other => bail!("missing implementation for plugin type '{}'", other),
}))
}
@@ -264,29 +273,32 @@ pub trait AcmePlugin {
/// Setup everything required to trigger the validation and return the corresponding validation
/// URL.
fn setup<'fut, 'a: 'fut, 'b: 'fut, 'c: 'fut, 'd: 'fut>(
- &'a self,
+ &'a mut self,
client: &'b mut AcmeClient,
authorization: &'c Authorization,
domain: &'d AcmeDomain,
) -> Pin<Box<dyn Future<Output = Result<&'c str, Error>> + Send + 'fut>>;
fn teardown<'fut, 'a: 'fut, 'b: 'fut, 'c: 'fut, 'd: 'fut>(
- &'a self,
+ &'a mut self,
client: &'b mut AcmeClient,
authorization: &'c Authorization,
domain: &'d AcmeDomain,
) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'fut>>;
}
-impl DnsPlugin {
- fn extract_challenge(authorization: &Authorization) -> Result<&Challenge, Error> {
- authorization
- .challenges
- .iter()
- .find(|ch| ch.ty == "dns-01")
- .ok_or_else(|| format_err!("no supported challenge type (dns-01) found"))
- }
+fn extract_challenge<'a>(
+ authorization: &'a Authorization,
+ ty: &str,
+) -> Result<&'a Challenge, Error> {
+ authorization
+ .challenges
+ .iter()
+ .find(|ch| ch.ty == ty)
+ .ok_or_else(|| format_err!("no supported challenge type (dns-01) found"))
+}
+impl DnsPlugin {
async fn action<'a>(
&self,
client: &mut AcmeClient,
@@ -294,7 +306,7 @@ impl DnsPlugin {
domain: &AcmeDomain,
action: &str,
) -> Result<&'a str, Error> {
- let challenge = Self::extract_challenge(authorization)?;
+ let challenge = extract_challenge(authorization, "dns-01")?;
let mut stdin_data = client
.dns_01_txt_value(
challenge
@@ -331,7 +343,9 @@ impl DnsPlugin {
stdin.write_all(&stdin_data).await?;
stdin.flush().await?;
Ok::<_, std::io::Error>(())
- }.await {
+ }
+ .await
+ {
Ok(()) => (),
Err(err) => {
if let Err(err) = child.kill().await {
@@ -357,7 +371,7 @@ impl DnsPlugin {
impl AcmePlugin for DnsPlugin {
fn setup<'fut, 'a: 'fut, 'b: 'fut, 'c: 'fut, 'd: 'fut>(
- &'a self,
+ &'a mut self,
client: &'b mut AcmeClient,
authorization: &'c Authorization,
domain: &'d AcmeDomain,
@@ -366,7 +380,7 @@ impl AcmePlugin for DnsPlugin {
}
fn teardown<'fut, 'a: 'fut, 'b: 'fut, 'c: 'fut, 'd: 'fut>(
- &'a self,
+ &'a mut self,
client: &'b mut AcmeClient,
authorization: &'c Authorization,
domain: &'d AcmeDomain,
@@ -378,3 +392,101 @@ impl AcmePlugin for DnsPlugin {
})
}
}
+
+#[derive(Default)]
+struct StandaloneServer {
+ abort_handle: Option<futures::future::AbortHandle>,
+}
+
+// In case the "order_certificates" future gets dropped between setup & teardown, let's also cancel
+// the HTTP listener on Drop:
+impl Drop for StandaloneServer {
+ fn drop(&mut self) {
+ self.stop();
+ }
+}
+
+impl StandaloneServer {
+ fn stop(&mut self) {
+ if let Some(abort) = self.abort_handle.take() {
+ abort.abort();
+ }
+ }
+}
+
+async fn standalone_respond(
+ req: Request<Body>,
+ path: Arc<String>,
+ key_auth: Arc<String>,
+) -> Result<Response<Body>, hyper::Error> {
+ if req.method() == hyper::Method::GET && req.uri().path() == path.as_str() {
+ Ok(Response::builder()
+ .status(http::StatusCode::OK)
+ .body(key_auth.as_bytes().to_vec().into())
+ .unwrap())
+ } else {
+ Ok(Response::builder()
+ .status(http::StatusCode::NOT_FOUND)
+ .body("Not found.".into())
+ .unwrap())
+ }
+}
+
+impl AcmePlugin for StandaloneServer {
+ fn setup<'fut, 'a: 'fut, 'b: 'fut, 'c: 'fut, 'd: 'fut>(
+ &'a mut self,
+ client: &'b mut AcmeClient,
+ authorization: &'c Authorization,
+ _domain: &'d AcmeDomain,
+ ) -> Pin<Box<dyn Future<Output = Result<&'c str, Error>> + Send + 'fut>> {
+ use hyper::server::conn::AddrIncoming;
+ use hyper::service::{make_service_fn, service_fn};
+
+ Box::pin(async move {
+ self.stop();
+
+ let challenge = extract_challenge(authorization, "http-01")?;
+ let token = challenge
+ .token()
+ .ok_or_else(|| format_err!("missing token in challenge"))?;
+ let key_auth = Arc::new(client.key_authorization(&token)?);
+ let path = Arc::new(format!("/.well-known/acme-challenge/{}", token));
+
+ let service = make_service_fn(move |_| {
+ let path = Arc::clone(&path);
+ let key_auth = Arc::clone(&key_auth);
+ async move {
+ Ok::<_, hyper::Error>(service_fn(move |request| {
+ standalone_respond(request, Arc::clone(&path), Arc::clone(&key_auth))
+ }))
+ }
+ });
+
+ // `[::]:80` first, then `*:80`
+ let incoming = AddrIncoming::bind(&(([0u16; 8], 80).into()))
+ .or_else(|_| AddrIncoming::bind(&(([0u8; 4], 80).into())))?;
+
+ let server = hyper::Server::builder(incoming).serve(service);
+
+ let (future, abort) = futures::future::abortable(server);
+ self.abort_handle = Some(abort);
+ tokio::spawn(future);
+
+ Ok(challenge.url.as_str())
+ })
+ }
+
+ fn teardown<'fut, 'a: 'fut, 'b: 'fut, 'c: 'fut, 'd: 'fut>(
+ &'a mut self,
+ _client: &'b mut AcmeClient,
+ _authorization: &'c Authorization,
+ _domain: &'d AcmeDomain,
+ ) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'fut>> {
+ Box::pin(async move {
+ if let Some(abort) = self.abort_handle.take() {
+ abort.abort();
+ }
+ Ok(())
+ })
+ }
+}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 22/23] ui: add certificate & acme view
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (20 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 21/23] implement standalone acme validation Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC backup 23/23] daily-update: check acme certificates Wolfgang Bumiller
` (2 subsequent siblings)
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
www/Makefile | 1 +
www/NavigationTree.js | 6 +++
www/config/CertificateView.js | 80 +++++++++++++++++++++++++++++++++++
3 files changed, 87 insertions(+)
create mode 100644 www/config/CertificateView.js
diff --git a/www/Makefile b/www/Makefile
index 2b847e74..f0b795ca 100644
--- a/www/Makefile
+++ b/www/Makefile
@@ -53,6 +53,7 @@ JSSRC= \
config/SyncView.js \
config/VerifyView.js \
config/WebauthnView.js \
+ config/CertificateView.js \
window/ACLEdit.js \
window/AddTfaRecovery.js \
window/AddTotp.js \
diff --git a/www/NavigationTree.js b/www/NavigationTree.js
index 8b1b96d9..6035526c 100644
--- a/www/NavigationTree.js
+++ b/www/NavigationTree.js
@@ -50,6 +50,12 @@ Ext.define('PBS.store.NavigationStore', {
path: 'pbsRemoteView',
leaf: true,
},
+ {
+ text: gettext('Certificates'),
+ iconCls: 'fa fa-certificate',
+ path: 'pbsCertificateConfiguration',
+ leaf: true,
+ },
{
text: gettext('Subscription'),
iconCls: 'fa fa-support',
diff --git a/www/config/CertificateView.js b/www/config/CertificateView.js
new file mode 100644
index 00000000..8472ad64
--- /dev/null
+++ b/www/config/CertificateView.js
@@ -0,0 +1,80 @@
+Ext.define('PBS.config.CertificateConfiguration', {
+ extend: 'Ext.tab.Panel',
+ alias: 'widget.pbsCertificateConfiguration',
+
+ title: gettext('Certificates'),
+
+ border: false,
+ defaults: { border: false },
+
+ items: [
+ {
+ itemId: 'certificates',
+ xtype: 'pbsCertificatesView',
+ },
+ {
+ itemId: 'acme',
+ xtype: 'pbsACMEConfigView',
+ },
+ ],
+});
+
+Ext.define('PBS.config.CertificatesView', {
+ extend: 'Ext.panel.Panel',
+ alias: 'widget.pbsCertificatesView',
+
+ title: gettext('Certificates'),
+ border: false,
+ defaults: {
+ border: false,
+ },
+
+ items: [
+ {
+ xtype: 'pmxCertificates',
+ nodename: 'localhost',
+ infoUrl: '/nodes/localhost/certificates/info',
+ uploadButtons: [
+ {
+ id: 'proxy.pem',
+ url: '/nodes/localhost/certificates/custom',
+ deletable: true,
+ reloadUi: true,
+ },
+ ],
+ },
+ {
+ xtype: 'pmxACMEDomains',
+ border: 0,
+ url: `/nodes/localhost/config`,
+ nodename: 'localhost',
+ acmeUrl: '/config/acme',
+ orderUrl: `/nodes/localhost/certificates/acme/certificate`,
+ separateDomainEntries: true,
+ },
+ ],
+})
+
+Ext.define('PBS.ACMEConfigView', {
+ extend: 'Ext.panel.Panel',
+ alias: 'widget.pbsACMEConfigView',
+
+ title: gettext('ACME Accounts'),
+
+ //onlineHelp: 'sysadmin_certificate_management',
+
+ items: [
+ {
+ region: 'north',
+ border: false,
+ xtype: 'pmxACMEAccounts',
+ acmeUrl: '/config/acme',
+ },
+ {
+ region: 'center',
+ border: false,
+ xtype: 'pmxACMEPluginView',
+ acmeUrl: '/config/acme',
+ },
+ ],
+});
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC backup 23/23] daily-update: check acme certificates
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (21 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 22/23] ui: add certificate & acme view Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-16 13:35 ` [pbs-devel] [RFC widget-toolkit] acme: separate flag to disable the 'domains=' array Wolfgang Bumiller
2021-04-20 10:27 ` [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Dominic Jäger
24 siblings, 0 replies; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/bin/proxmox-daily-update.rs | 30 +++++++++++++++++++++++++++++-
1 file changed, 29 insertions(+), 1 deletion(-)
diff --git a/src/bin/proxmox-daily-update.rs b/src/bin/proxmox-daily-update.rs
index 83c6b80c..be3bfe44 100644
--- a/src/bin/proxmox-daily-update.rs
+++ b/src/bin/proxmox-daily-update.rs
@@ -50,13 +50,41 @@ async fn do_update(
};
wait_for_local_worker(upid.as_str().unwrap()).await?;
- // TODO: certificate checks/renewal/... ?
+ match check_acme_certificates(rpcenv).await {
+ Ok(()) => (),
+ Err(err) => {
+ eprintln!("error checking certificates: {}", err);
+ }
+ }
// TODO: cleanup tasks like in PVE?
Ok(Value::Null)
}
+async fn check_acme_certificates(rpcenv: &mut dyn RpcEnvironment) -> Result<(), Error> {
+ let (config, _) = proxmox_backup::config::node::config()?;
+
+ // do we even have any acme domains configures?
+ if config.acme_domains().next().is_none() {
+ return Ok(());
+ }
+
+ if !api2::node::certificates::cert_expires_soon()? {
+ println!("Certificate does not expire within the next 30 days, not renewing.");
+ return Ok(());
+ }
+
+ let info = &api2::node::certificates::API_METHOD_RENEW_ACME_CERT;
+ let result = match info.handler {
+ ApiHandler::Sync(handler) => (handler)(json!({}), info, rpcenv)?,
+ _ => unreachable!(),
+ };
+ wait_for_local_worker(result.as_str().unwrap()).await?;
+
+ Ok(())
+}
+
fn main() {
proxmox_backup::tools::setup_safe_path_env();
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] [RFC widget-toolkit] acme: separate flag to disable the 'domains=' array
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (22 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC backup 23/23] daily-update: check acme certificates Wolfgang Bumiller
@ 2021-04-16 13:35 ` Wolfgang Bumiller
2021-04-22 17:57 ` [pbs-devel] applied: " Thomas Lamprecht
2021-04-20 10:27 ` [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Dominic Jäger
24 siblings, 1 reply; 30+ messages in thread
From: Wolfgang Bumiller @ 2021-04-16 13:35 UTC (permalink / raw)
To: pbs-devel
PVE has 2 domain lists, PMG only 1 since it requires the
additional type.
In PBS I do not want to have 2 lists either, since it seems
rather inconvenient to have 2 different ways to access the
same list.
Currently we decide this based on whether we have multiple
certificate types, which in PBS we don't, so we need a
separate option for this.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
src/panel/ACMEDomains.js | 4 ++++
src/window/ACMEDomains.js | 5 ++++-
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/src/panel/ACMEDomains.js b/src/panel/ACMEDomains.js
index 6cfc501..a7fb088 100644
--- a/src/panel/ACMEDomains.js
+++ b/src/panel/ACMEDomains.js
@@ -21,6 +21,8 @@ Ext.define('Proxmox.panel.ACMEDomains', {
domainUsages: undefined,
// if no domainUsages parameter is supllied, the orderUrl is required instead:
orderUrl: undefined,
+ // Force the use of 'acmedomainX' properties.
+ separateDomainEntries: undefined,
acmeUrl: undefined,
@@ -87,6 +89,7 @@ Ext.define('Proxmox.panel.ACMEDomains', {
acmeUrl: view.acmeUrl,
nodeconfig: view.nodeconfig,
domainUsages: view.domainUsages,
+ separateDomainEntries: view.separateDomainEntries,
apiCallDone: function() {
me.reload();
},
@@ -105,6 +108,7 @@ Ext.define('Proxmox.panel.ACMEDomains', {
acmeUrl: view.acmeUrl,
nodeconfig: view.nodeconfig,
domainUsages: view.domainUsages,
+ separateDomainEntries: view.separateDomainEntries,
domain: selection[0].data,
apiCallDone: function() {
me.reload();
diff --git a/src/window/ACMEDomains.js b/src/window/ACMEDomains.js
index 930a4c3..b040e33 100644
--- a/src/window/ACMEDomains.js
+++ b/src/window/ACMEDomains.js
@@ -16,6 +16,9 @@ Ext.define('Proxmox.window.ACMEDomainEdit', {
// For PMG the we have multiple certificates, so we have a "usage" attribute & column.
domainUsages: undefined,
+ // Force the use of 'acmedomainX' properties.
+ separateDomainEntries: undefined,
+
cbindData: function(config) {
let me = this;
return {
@@ -50,7 +53,7 @@ Ext.define('Proxmox.window.ACMEDomainEdit', {
};
// If we have a 'usage' property (pmg), we only use the `acmedomainX` config keys.
- if (win.domainUsages) {
+ if (win.separateDomainEntries || win.domainUsages) {
if (!configkey || configkey === 'acme') {
configkey = find_free_slot();
}
--
2.20.1
^ permalink raw reply [flat|nested] 30+ messages in thread* [pbs-devel] applied: [RFC widget-toolkit] acme: separate flag to disable the 'domains=' array
2021-04-16 13:35 ` [pbs-devel] [RFC widget-toolkit] acme: separate flag to disable the 'domains=' array Wolfgang Bumiller
@ 2021-04-22 17:57 ` Thomas Lamprecht
0 siblings, 0 replies; 30+ messages in thread
From: Thomas Lamprecht @ 2021-04-22 17:57 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Wolfgang Bumiller
On 16.04.21 15:35, Wolfgang Bumiller wrote:
> PVE has 2 domain lists, PMG only 1 since it requires the
> additional type.
>
> In PBS I do not want to have 2 lists either, since it seems
> rather inconvenient to have 2 different ways to access the
> same list.
>
> Currently we decide this based on whether we have multiple
> certificate types, which in PBS we don't, so we need a
> separate option for this.
>
> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
> ---
> src/panel/ACMEDomains.js | 4 ++++
> src/window/ACMEDomains.js | 5 ++++-
> 2 files changed, 8 insertions(+), 1 deletion(-)
>
>
applied, thanks!
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS
2021-04-16 13:34 [pbs-devel] [RFC backup 00/23] Implements ACME suport for PBS Wolfgang Bumiller
` (23 preceding siblings ...)
2021-04-16 13:35 ` [pbs-devel] [RFC widget-toolkit] acme: separate flag to disable the 'domains=' array Wolfgang Bumiller
@ 2021-04-20 10:27 ` Dominic Jäger
24 siblings, 0 replies; 30+ messages in thread
From: Dominic Jäger @ 2021-04-20 10:27 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Wolfgang Bumiller
Creating the first account gives missing directory
> TASK ERROR: failed to open "/etc/proxmox-backup/acme/accounts/test" for
> writing: No such file or directory (os error 2)
After manually adding it, the HTTP Challenged worked for me.
In the Window "Add: ACME DNS Plugin" choosing (or writing) something in the
dropdown menu DNS API is not possible with only the PBS repositories
configured. It is necessary to install libproxmox-acme-perl from PVE
repositories in addition.
Deleting a certificate shows a confirmation dialog with a truncated message:
"Are you sure you want to remove the certificate used for"
In the window "Register Account" the textfield "Account Name" has the empty
text "default". As far as I know, we use empty texts for real default values.
So this should be removed and get a validator (already in the GUI) instead.
But the API rejects correctly: "parameter verification errors parameter 'name':
parameter is missing and it is not optional."
Registering accounts for both staging and production works. Ordering
certificates with HTTP challenge generally works for both, too. A few times
the HTTP challenge required a manual retry. Maybe we could do something like
increasing timeouts?
I couldn't set up PowerDNS yet & my domains were not fast enough, so finishing
the DNS challenge testing remains todo.
Tested-by: Dominic Jäger <d.jaeger@proxmox.com>
^ permalink raw reply [flat|nested] 30+ messages in thread