[pve-devel] [PATCH v3 many 0/7] notifications: add SMTP endpoint

[pve-devel] [PATCH v3 many 0/7] notifications: add SMTP endpoint

[Lukas Wagner]

This patch series adds support for a new notification endpoint type,
smtp. As the name suggests, this new endpoint allows PVE to talk
to SMTP server directly, without using the system's MTA (postfix).

On the Rust side, these patches add a new dependency to the `lettre`
crate for SMTP communication. This crate was chosen as it is:
  - by far the most popular mailing crate for Rust
  - well maintained
  - has reasonable dependencies
  - has async support, enabling us to asyncify the proxmox-notify
    crate at some point, if needed

These patches are based on top of the `system mail` patch series [1].
From [1], we need the patches for `proxmox` and `proxmox-perl-rs`.
If `pve-docs` from [1] is not applied first, there might be a small conflict,
however that one should be trivial to resolve.

Tested against
  - the gmail SMTP server
  - the posteo SMTP server
  - our own webmail SMTP server

Changes since v2:
  - Rebased proxmox-widget-toolkit onto the latest master to avoid
    any conflicts.

Changes since v1:
  - Rebased on top of [1]
  - Added a mechanism for mails forwarded by `proxmox-mail-forward`
    These are forwarded inline as "message/rfc822" to avoid having 
    to rewrite mail headers (otherwise, some SMTP relays might reject the 
    mail, because the `From` header of the forwarded mail does not match the
    mail account)

[1] https://lists.proxmox.com/pipermail/pve-devel/2023-August/058956.html



proxmox:

Lukas Wagner (3):
  notify: fix typo in doc comments
  notify: add 'smtp' endpoint
  notify: add api for smtp endpoints

 Cargo.toml                                  |   1 +
 proxmox-notify/Cargo.toml                   |   4 +-
 proxmox-notify/src/api/mod.rs               |  48 +++
 proxmox-notify/src/api/smtp.rs              | 373 ++++++++++++++++++++
 proxmox-notify/src/config.rs                |  23 ++
 proxmox-notify/src/endpoints/common/mail.rs |  24 ++
 proxmox-notify/src/endpoints/common/mod.rs  |   2 +
 proxmox-notify/src/endpoints/mod.rs         |   4 +
 proxmox-notify/src/endpoints/sendmail.rs    |  22 +-
 proxmox-notify/src/endpoints/smtp.rs        | 260 ++++++++++++++
 proxmox-notify/src/lib.rs                   |  19 +-
 11 files changed, 761 insertions(+), 19 deletions(-)
 create mode 100644 proxmox-notify/src/api/smtp.rs
 create mode 100644 proxmox-notify/src/endpoints/common/mail.rs
 create mode 100644 proxmox-notify/src/endpoints/common/mod.rs
 create mode 100644 proxmox-notify/src/endpoints/smtp.rs


proxmox-perl-rs:

Lukas Wagner (1):
  notify: add bindings for smtp API calls

 common/src/notify.rs | 110 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 110 insertions(+)


pve-manager:

Lukas Wagner (1):
  notify: add API routes for smtp endpoints

 PVE/API2/Cluster/Notifications.pm | 337 ++++++++++++++++++++++++++++++
 1 file changed, 337 insertions(+)


proxmox-widget-toolkit:

Lukas Wagner (1):
  panel: notification: add gui for SMTP endpoints

 src/Makefile                     |   2 +
 src/Schema.js                    |   5 +
 src/panel/EmailRecipientPanel.js |  89 ++++++++++++++
 src/panel/SendmailEditPanel.js   |  67 ++---------
 src/panel/SmtpEditPanel.js       | 192 +++++++++++++++++++++++++++++++
 5 files changed, 296 insertions(+), 59 deletions(-)
 create mode 100644 src/panel/EmailRecipientPanel.js
 create mode 100644 src/panel/SmtpEditPanel.js


pve-docs:

Lukas Wagner (1):
  notifications: document SMTP endpoints

 notifications.adoc | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)


Summary over all repositories:
  19 files changed, 1532 insertions(+), 78 deletions(-)

-- 
murpp v0.4.0

[pve-devel] [PATCH v3 proxmox 1/7] notify: fix typo in doc comments

[Lukas Wagner]

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
 proxmox-notify/src/lib.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
index eebc57a..ca68388 100644
--- a/proxmox-notify/src/lib.rs
+++ b/proxmox-notify/src/lib.rs
@@ -106,7 +106,7 @@ pub enum Severity {
 
 /// Notification endpoint trait, implemented by all endpoint plugins
 pub trait Endpoint {
-    /// Send a documentation
+    /// Send a notification
     fn send(&self, notification: &Notification) -> Result<(), Error>;
 
     /// The name/identifier for this endpoint
-- 
2.39.2

[pve-devel] [PATCH v3 proxmox 2/7] notify: add 'smtp' endpoint

[Lukas Wagner]

This commit adds a new endpoint type, namely 'smtp'. This endpoint
uses the `lettre` crate to directly send emails to SMTP relays.

The `lettre` crate was chosen since it is by far the most popular SMTP
implementation for Rust that looks like it is well maintained.
Also, it includes async support (for when we want to extend
proxmox-notify to be async).

For this new endpoint type, a new section-config type was introduced
(smtp). It has the same fields as the type for `sendmail`, with the
addition of some new options (smtp server, authentication, tls mode,
etc.).

Some of the behavior that is shared between sendmail and smtp
endpoints has been moved to a new `endpoints::common::mail` module.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
 Cargo.toml                                  |   1 +
 proxmox-notify/Cargo.toml                   |   4 +-
 proxmox-notify/src/config.rs                |  23 ++
 proxmox-notify/src/endpoints/common/mail.rs |  24 ++
 proxmox-notify/src/endpoints/common/mod.rs  |   2 +
 proxmox-notify/src/endpoints/mod.rs         |   4 +
 proxmox-notify/src/endpoints/sendmail.rs    |  22 +-
 proxmox-notify/src/endpoints/smtp.rs        | 260 ++++++++++++++++++++
 proxmox-notify/src/lib.rs                   |  17 ++
 9 files changed, 339 insertions(+), 18 deletions(-)
 create mode 100644 proxmox-notify/src/endpoints/common/mail.rs
 create mode 100644 proxmox-notify/src/endpoints/common/mod.rs
 create mode 100644 proxmox-notify/src/endpoints/smtp.rs

diff --git a/Cargo.toml b/Cargo.toml
index 9adfe59..1ffcf20 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -59,6 +59,7 @@ http = "0.2"
 hyper = "0.14.5"
 lazy_static = "1.4"
 ldap3 = { version = "0.11", default-features = false }
+lettre = "0.10.4"
 libc = "0.2.107"
 log = "0.4.17"
 mail-parser = "0.8.2"
diff --git a/proxmox-notify/Cargo.toml b/proxmox-notify/Cargo.toml
index 8d8caaf..a4f73eb 100644
--- a/proxmox-notify/Cargo.toml
+++ b/proxmox-notify/Cargo.toml
@@ -10,6 +10,7 @@ exclude.workspace = true
 [dependencies]
 handlebars = { workspace = true }
 lazy_static.workspace = true
+lettre = { workspace = true, optional = true }
 log.workspace = true
 mail-parser = { workspace = true, optional = true }
 openssl.workspace = true
@@ -25,9 +26,10 @@ serde = { workspace = true, features = ["derive"]}
 serde_json.workspace = true
 
 [features]
-default = ["sendmail", "gotify"]
+default = ["sendmail", "gotify", "smtp"]
 mail-forwarder = ["dep:mail-parser"]
 sendmail = ["dep:proxmox-sys"]
 gotify = ["dep:proxmox-http"]
+smtp = ["dep:lettre"]
 pve-context = ["dep:proxmox-sys"]
 pbs-context = ["dep:proxmox-sys"]
diff --git a/proxmox-notify/src/config.rs b/proxmox-notify/src/config.rs
index cdbf42a..138a3e0 100644
--- a/proxmox-notify/src/config.rs
+++ b/proxmox-notify/src/config.rs
@@ -27,6 +27,17 @@ fn config_init() -> SectionConfig {
             SENDMAIL_SCHEMA,
         ));
     }
+    #[cfg(feature = "smtp")]
+    {
+        use crate::endpoints::smtp::{SmtpConfig, SMTP_TYPENAME};
+
+        const SMTP_SCHEMA: &ObjectSchema = SmtpConfig::API_SCHEMA.unwrap_object_schema();
+        config.register_plugin(SectionConfigPlugin::new(
+            SMTP_TYPENAME.to_string(),
+            Some(String::from("name")),
+            SMTP_SCHEMA,
+        ));
+    }
     #[cfg(feature = "gotify")]
     {
         use crate::endpoints::gotify::{GotifyConfig, GOTIFY_TYPENAME};
@@ -73,6 +84,18 @@ fn private_config_init() -> SectionConfig {
         ));
     }
 
+    #[cfg(feature = "smtp")]
+    {
+        use crate::endpoints::smtp::{SmtpPrivateConfig, SMTP_TYPENAME};
+
+        const SMTP_SCHEMA: &ObjectSchema = SmtpPrivateConfig::API_SCHEMA.unwrap_object_schema();
+        config.register_plugin(SectionConfigPlugin::new(
+            SMTP_TYPENAME.to_string(),
+            Some(String::from("name")),
+            SMTP_SCHEMA,
+        ));
+    }
+
     config
 }
 
diff --git a/proxmox-notify/src/endpoints/common/mail.rs b/proxmox-notify/src/endpoints/common/mail.rs
new file mode 100644
index 0000000..0929d7c
--- /dev/null
+++ b/proxmox-notify/src/endpoints/common/mail.rs
@@ -0,0 +1,24 @@
+use std::collections::HashSet;
+
+use crate::context;
+
+pub(crate) fn get_recipients(
+    email_addrs: Option<&[String]>,
+    users: Option<&[String]>,
+) -> HashSet<String> {
+    let mut recipients = HashSet::new();
+
+    if let Some(mailto_addrs) = email_addrs {
+        for addr in mailto_addrs {
+            recipients.insert(addr.clone());
+        }
+    }
+    if let Some(users) = users {
+        for user in users {
+            if let Some(addr) = context::context().lookup_email_for_user(user) {
+                recipients.insert(addr);
+            }
+        }
+    }
+    recipients
+}
diff --git a/proxmox-notify/src/endpoints/common/mod.rs b/proxmox-notify/src/endpoints/common/mod.rs
new file mode 100644
index 0000000..60e0761
--- /dev/null
+++ b/proxmox-notify/src/endpoints/common/mod.rs
@@ -0,0 +1,2 @@
+#[cfg(any(feature = "sendmail", feature = "smtp"))]
+pub(crate) mod mail;
diff --git a/proxmox-notify/src/endpoints/mod.rs b/proxmox-notify/src/endpoints/mod.rs
index d1cec65..97f79fc 100644
--- a/proxmox-notify/src/endpoints/mod.rs
+++ b/proxmox-notify/src/endpoints/mod.rs
@@ -2,3 +2,7 @@
 pub mod gotify;
 #[cfg(feature = "sendmail")]
 pub mod sendmail;
+#[cfg(feature = "smtp")]
+pub mod smtp;
+
+mod common;
diff --git a/proxmox-notify/src/endpoints/sendmail.rs b/proxmox-notify/src/endpoints/sendmail.rs
index 9cc3f31..ce4f3d7 100644
--- a/proxmox-notify/src/endpoints/sendmail.rs
+++ b/proxmox-notify/src/endpoints/sendmail.rs
@@ -1,11 +1,10 @@
-use std::collections::HashSet;
-
 use serde::{Deserialize, Serialize};
 
 use proxmox_schema::api_types::COMMENT_SCHEMA;
 use proxmox_schema::{api, Updater};
 
 use crate::context::context;
+use crate::endpoints::common::mail;
 use crate::renderer::TemplateRenderer;
 use crate::schema::{EMAIL_SCHEMA, ENTITY_NAME_SCHEMA, USER_SCHEMA};
 use crate::{renderer, Content, Endpoint, Error, Notification};
@@ -86,21 +85,10 @@ pub struct SendmailEndpoint {
 
 impl Endpoint for SendmailEndpoint {
     fn send(&self, notification: &Notification) -> Result<(), Error> {
-        let mut recipients = HashSet::new();
-
-        if let Some(mailto_addrs) = self.config.mailto.as_ref() {
-            for addr in mailto_addrs {
-                recipients.insert(addr.clone());
-            }
-        }
-
-        if let Some(users) = self.config.mailto_user.as_ref() {
-            for user in users {
-                if let Some(addr) = context().lookup_email_for_user(user) {
-                    recipients.insert(addr);
-                }
-            }
-        }
+        let recipients = mail::get_recipients(
+            self.config.mailto.as_deref(),
+            self.config.mailto_user.as_deref(),
+        );
 
         let properties = notification.properties.as_ref();
         let recipients_str: Vec<&str> = recipients.iter().map(String::as_str).collect();
diff --git a/proxmox-notify/src/endpoints/smtp.rs b/proxmox-notify/src/endpoints/smtp.rs
new file mode 100644
index 0000000..4f2e13b
--- /dev/null
+++ b/proxmox-notify/src/endpoints/smtp.rs
@@ -0,0 +1,260 @@
+use lettre::message::{Mailbox, MultiPart, SinglePart};
+use lettre::transport::smtp::client::{Tls, TlsParameters};
+use lettre::{message::header::ContentType, Message, SmtpTransport, Transport};
+use serde::{Deserialize, Serialize};
+use std::time::Duration;
+
+use proxmox_schema::api_types::COMMENT_SCHEMA;
+use proxmox_schema::{api, Updater};
+
+use crate::context::context;
+use crate::endpoints::common::mail;
+use crate::renderer::TemplateRenderer;
+use crate::schema::{EMAIL_SCHEMA, ENTITY_NAME_SCHEMA, USER_SCHEMA};
+use crate::{renderer, Content, Endpoint, Error, Notification};
+
+pub(crate) const SMTP_TYPENAME: &str = "smtp";
+
+const SMTP_PORT: u16 = 25;
+const SMTP_SUBMISSION_STARTTLS_PORT: u16 = 587;
+const SMTP_SUBMISSION_TLS_PORT: u16 = 465;
+const SMTP_TIMEOUT: u16 = 5;
+
+#[api]
+#[derive(Debug, Serialize, Deserialize, Default, Clone, Copy)]
+#[serde(rename_all = "kebab-case")]
+/// Connection security
+pub enum SmtpMode {
+    /// No encryption (insecure), plain SMTP
+    Insecure,
+    /// Upgrade to TLS after connecting
+    #[serde(rename = "starttls")]
+    StartTls,
+    /// Use TLS-secured connection
+    #[default]
+    Tls,
+}
+
+#[api(
+    properties: {
+        name: {
+            schema: ENTITY_NAME_SCHEMA,
+        },
+        mailto: {
+            type: Array,
+                items: {
+                    schema: EMAIL_SCHEMA,
+            },
+            optional: true,
+        },
+        "mailto-user": {
+            type: Array,
+            items: {
+                schema: USER_SCHEMA,
+            },
+            optional: true,
+        },
+        comment: {
+            optional: true,
+            schema: COMMENT_SCHEMA,
+        },
+        filter: {
+            optional: true,
+            schema: ENTITY_NAME_SCHEMA,
+        },
+    },
+)]
+#[derive(Debug, Serialize, Deserialize, Updater, Default)]
+#[serde(rename_all = "kebab-case")]
+/// Config for Sendmail notification endpoints
+pub struct SmtpConfig {
+    /// Name of the endpoint
+    #[updater(skip)]
+    pub name: String,
+    /// Host name or IP of the SMTP relay
+    pub server: String,
+    /// Port to use when connecting to the SMTP relay
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub port: Option<u16>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub mode: Option<SmtpMode>,
+    /// Username for authentication
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub username: Option<String>,
+    /// Mail recipients
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub mailto: Option<Vec<String>>,
+    /// Mail recipients
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub mailto_user: Option<Vec<String>>,
+    /// `From` address for the mail
+    pub from_address: String,
+    /// Author of the mail
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub author: Option<String>,
+    /// Comment
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub comment: Option<String>,
+    /// Filter to apply
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub filter: Option<String>,
+}
+
+#[derive(Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case")]
+pub enum DeleteableSmtpProperty {
+    Author,
+    Comment,
+    Filter,
+    Mailto,
+    MailtoUser,
+    Password,
+    Port,
+    Username,
+}
+
+#[api]
+#[derive(Serialize, Deserialize, Clone, Updater, Debug)]
+#[serde(rename_all = "kebab-case")]
+/// Private configuration for SMTP notification endpoints.
+/// This config will be saved to a separate configuration file with stricter
+/// permissions (root:root 0600)
+pub struct SmtpPrivateConfig {
+    /// Name of the endpoint
+    #[updater(skip)]
+    pub name: String,
+    /// Authentication token
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub password: Option<String>,
+}
+
+/// A sendmail notification endpoint.
+pub struct SmtpEndpoint {
+    pub config: SmtpConfig,
+    pub private_config: SmtpPrivateConfig,
+}
+
+impl Endpoint for SmtpEndpoint {
+    fn send(&self, notification: &Notification) -> Result<(), Error> {
+        let tls_parameters = TlsParameters::new(self.config.server.clone())
+            .map_err(|err| Error::NotifyFailed(self.name().into(), Box::new(err)))?;
+
+        let (port, tls) = match self.config.mode.unwrap_or_default() {
+            SmtpMode::Insecure => {
+                let port = self.config.port.unwrap_or(SMTP_PORT);
+                (port, Tls::None)
+            }
+            SmtpMode::StartTls => {
+                let port = self.config.port.unwrap_or(SMTP_SUBMISSION_STARTTLS_PORT);
+                (port, Tls::Required(tls_parameters))
+            }
+            SmtpMode::Tls => {
+                let port = self.config.port.unwrap_or(SMTP_SUBMISSION_TLS_PORT);
+                (port, Tls::Wrapper(tls_parameters))
+            }
+        };
+
+        let mut transport_builder = SmtpTransport::builder_dangerous(&self.config.server)
+            .tls(tls)
+            .port(port)
+            .timeout(Some(Duration::from_secs(SMTP_TIMEOUT.into())));
+
+        if let Some(username) = self.config.username.as_deref() {
+            if let Some(password) = self.private_config.password.as_deref() {
+                transport_builder = transport_builder.credentials((username, password).into());
+            } else {
+                return Err(Error::NotifyFailed(
+                    self.name().into(),
+                    Box::new(Error::Generic(
+                        "username is set but no password was provided".to_owned(),
+                    )),
+                ));
+            }
+        }
+
+        let transport = transport_builder.build();
+
+        let recipients = mail::get_recipients(
+            self.config.mailto.as_deref(),
+            self.config.mailto_user.as_deref(),
+        );
+        let mail_from = self.config.from_address.clone();
+
+        let parse_address = |addr: &str| -> Result<Mailbox, Error> {
+            addr.parse()
+                .map_err(|err| Error::NotifyFailed(self.name().into(), Box::new(err)))
+        };
+
+        let author = self
+            .config
+            .author
+            .clone()
+            .unwrap_or_else(|| context().default_sendmail_author());
+
+        let mut email_builder =
+            Message::builder().from(parse_address(&format!("{author} <{mail_from}>"))?);
+
+        for recipient in recipients {
+            email_builder = email_builder.to(parse_address(&recipient)?);
+        }
+
+        let email = match &notification.content {
+            Content::Template { title, body } => {
+                let properties = notification.properties.as_ref();
+
+                let subject =
+                    renderer::render_template(TemplateRenderer::Plaintext, title, properties)?;
+                let html_part =
+                    renderer::render_template(TemplateRenderer::Html, body, properties)?;
+                let text_part =
+                    renderer::render_template(TemplateRenderer::Plaintext, body, properties)?;
+
+                email_builder = email_builder.subject(subject);
+
+                email_builder
+                    .multipart(
+                        MultiPart::alternative()
+                            .singlepart(
+                                SinglePart::builder()
+                                    .header(ContentType::TEXT_PLAIN)
+                                    .body(text_part),
+                            )
+                            .singlepart(
+                                SinglePart::builder()
+                                    .header(ContentType::TEXT_HTML)
+                                    .body(html_part),
+                            ),
+                    )
+                    .map_err(|err| Error::NotifyFailed(self.name().into(), Box::new(err)))?
+            }
+            #[cfg(feature = "mail-forwarder")]
+            Content::ForwardedMail { ref raw, title, .. } => {
+                email_builder = email_builder.subject(title);
+
+                // Forwarded messages are embedded inline as 'message/rfc822'
+                // this let's us avoid rewriting any headers (e.g. From)
+                email_builder
+                    .singlepart(
+                        SinglePart::builder()
+                            .header(ContentType::parse("message/rfc822").unwrap())
+                            .body(raw.to_owned()),
+                    )
+                    .map_err(|err| Error::NotifyFailed(self.name().into(), Box::new(err)))?
+            }
+        };
+
+        transport
+            .send(&email)
+            .map_err(|err| Error::NotifyFailed(self.name().into(), err.into()))?;
+
+        Ok(())
+    }
+
+    fn name(&self) -> &str {
+        &self.config.name
+    }
+
+    fn filter(&self) -> Option<&str> {
+        self.config.filter.as_deref()
+    }
+}
diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
index ca68388..609337f 100644
--- a/proxmox-notify/src/lib.rs
+++ b/proxmox-notify/src/lib.rs
@@ -339,6 +339,23 @@ impl Bus {
             );
         }
 
+        #[cfg(feature = "smtp")]
+        {
+            use endpoints::smtp::SMTP_TYPENAME;
+            use endpoints::smtp::{SmtpConfig, SmtpEndpoint, SmtpPrivateConfig};
+            endpoints.extend(
+                parse_endpoints_with_private_config!(
+                    config,
+                    SmtpConfig,
+                    SmtpPrivateConfig,
+                    SmtpEndpoint,
+                    SMTP_TYPENAME
+                )?
+                .into_iter()
+                .map(|e| (e.name().into(), e)),
+            );
+        }
+
         let groups: HashMap<String, GroupConfig> = config
             .config
             .convert_to_typed_array(GROUP_TYPENAME)
-- 
2.39.2

[pve-devel] [PATCH v3 proxmox 3/7] notify: add api for smtp endpoints

[Lukas Wagner]

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
 proxmox-notify/src/api/mod.rs  |  48 +++++
 proxmox-notify/src/api/smtp.rs | 373 +++++++++++++++++++++++++++++++++
 2 files changed, 421 insertions(+)
 create mode 100644 proxmox-notify/src/api/smtp.rs

diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
index 8dc9b4e..097d816 100644
--- a/proxmox-notify/src/api/mod.rs
+++ b/proxmox-notify/src/api/mod.rs
@@ -1,3 +1,4 @@
+use serde::Serialize;
 use std::collections::HashSet;
 
 use proxmox_http_error::HttpError;
@@ -11,6 +12,8 @@ pub mod gotify;
 pub mod group;
 #[cfg(feature = "sendmail")]
 pub mod sendmail;
+#[cfg(feature = "smtp")]
+pub mod smtp;
 
 // We have our own, local versions of http_err and http_bail, because
 // we don't want to wrap the error in anyhow::Error. If we were to do that,
@@ -61,6 +64,10 @@ fn ensure_endpoint_exists(#[allow(unused)] config: &Config, name: &str) -> Resul
     {
         exists = exists || gotify::get_endpoint(config, name).is_ok();
     }
+    #[cfg(feature = "smtp")]
+    {
+        exists = exists || smtp::get_endpoint(config, name).is_ok();
+    }
 
     if !exists {
         http_bail!(NOT_FOUND, "endpoint '{name}' does not exist")
@@ -124,6 +131,15 @@ fn get_referrers(config: &Config, entity: &str) -> Result<HashSet<String>, HttpE
         }
     }
 
+    #[cfg(feature = "smtp")]
+    for endpoint in smtp::get_endpoints(config)? {
+        if let Some(filter) = endpoint.filter {
+            if filter == entity {
+                referrers.insert(endpoint.name);
+            }
+        }
+    }
+
     Ok(referrers)
 }
 
@@ -170,6 +186,13 @@ fn get_referenced_entities(config: &Config, entity: &str) -> HashSet<String> {
                     new.insert(filter.clone());
                 }
             }
+
+            #[cfg(feature = "smtp")]
+            if let Ok(target) = smtp::get_endpoint(config, entity) {
+                if let Some(filter) = target.filter {
+                    new.insert(filter.clone());
+                }
+            }
         }
 
         new
@@ -184,6 +207,31 @@ fn get_referenced_entities(config: &Config, entity: &str) -> HashSet<String> {
     expanded
 }
 
+#[allow(unused)]
+fn set_private_config_entry<T: Serialize>(
+    config: &mut Config,
+    private_config: &T,
+    typename: &str,
+    name: &str,
+) -> Result<(), HttpError> {
+    config
+        .private_config
+        .set_data(name, typename, private_config)
+        .map_err(|e| {
+            http_err!(
+                INTERNAL_SERVER_ERROR,
+                "could not save private config for endpoint '{}': {e}",
+                name
+            )
+        })
+}
+
+#[allow(unused)]
+fn remove_private_config_entry(config: &mut Config, name: &str) -> Result<(), HttpError> {
+    config.private_config.sections.remove(name);
+    Ok(())
+}
+
 #[cfg(test)]
 mod test_helpers {
     use crate::Config;
diff --git a/proxmox-notify/src/api/smtp.rs b/proxmox-notify/src/api/smtp.rs
new file mode 100644
index 0000000..f3aca77
--- /dev/null
+++ b/proxmox-notify/src/api/smtp.rs
@@ -0,0 +1,373 @@
+use proxmox_http_error::HttpError;
+
+use crate::api::{http_bail, http_err};
+use crate::endpoints::smtp::{
+    DeleteableSmtpProperty, SmtpConfig, SmtpConfigUpdater, SmtpPrivateConfig,
+    SmtpPrivateConfigUpdater, SMTP_TYPENAME,
+};
+use crate::Config;
+
+/// Get a list of all smtp endpoints.
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns a list of all smtp endpoints or a `HttpError` if the config is
+/// erroneous (`500 Internal server error`).
+pub fn get_endpoints(config: &Config) -> Result<Vec<SmtpConfig>, HttpError> {
+    config
+        .config
+        .convert_to_typed_array(SMTP_TYPENAME)
+        .map_err(|e| http_err!(NOT_FOUND, "Could not fetch endpoints: {e}"))
+}
+
+/// Get smtp endpoint with given `name`.
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns the endpoint or a `HttpError` if the endpoint was not found (`404 Not found`).
+pub fn get_endpoint(config: &Config, name: &str) -> Result<SmtpConfig, HttpError> {
+    config
+        .config
+        .lookup(SMTP_TYPENAME, name)
+        .map_err(|_| http_err!(NOT_FOUND, "endpoint '{name}' not found"))
+}
+
+/// Add a new smtp endpoint.
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns a `HttpError` if:
+///   - an entity with the same name already exists (`400 Bad request`)
+///   - a referenced filter does not exist (`400 Bad request`)
+///   - the configuration could not be saved (`500 Internal server error`)
+///   - mailto *and* mailto_user are both set to `None`
+pub fn add_endpoint(
+    config: &mut Config,
+    endpoint_config: &SmtpConfig,
+    private_endpoint_config: &SmtpPrivateConfig,
+) -> Result<(), HttpError> {
+    if endpoint_config.name != private_endpoint_config.name {
+        // Programming error by the user of the crate, thus we panic
+        panic!("name for endpoint config and private config must be identical");
+    }
+
+    super::ensure_unique(config, &endpoint_config.name)?;
+
+    if let Some(filter) = &endpoint_config.filter {
+        // Check if filter exists
+        super::filter::get_filter(config, filter)?;
+    }
+
+    if endpoint_config.mailto.is_none() && endpoint_config.mailto_user.is_none() {
+        http_bail!(
+            BAD_REQUEST,
+            "must at least provide one recipient, either in mailto or in mailto-user"
+        );
+    }
+
+    super::set_private_config_entry(
+        config,
+        private_endpoint_config,
+        SMTP_TYPENAME,
+        &endpoint_config.name,
+    )?;
+
+    config
+        .config
+        .set_data(&endpoint_config.name, SMTP_TYPENAME, endpoint_config)
+        .map_err(|e| {
+            http_err!(
+                INTERNAL_SERVER_ERROR,
+                "could not save endpoint '{}': {e}",
+                endpoint_config.name
+            )
+        })
+}
+
+/// Update existing smtp endpoint
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns a `HttpError` if:
+///   - a referenced filter does not exist (`400 Bad request`)
+///   - the configuration could not be saved (`500 Internal server error`)
+///   - mailto *and* mailto_user are both set to `None`
+pub fn update_endpoint(
+    config: &mut Config,
+    name: &str,
+    updater: &SmtpConfigUpdater,
+    private_endpoint_config_updater: &SmtpPrivateConfigUpdater,
+    delete: Option<&[DeleteableSmtpProperty]>,
+    digest: Option<&[u8]>,
+) -> Result<(), HttpError> {
+    super::verify_digest(config, digest)?;
+
+    let mut endpoint = get_endpoint(config, name)?;
+
+    if let Some(delete) = delete {
+        for deleteable_property in delete {
+            match deleteable_property {
+                DeleteableSmtpProperty::Author => endpoint.author = None,
+                DeleteableSmtpProperty::Comment => endpoint.comment = None,
+                DeleteableSmtpProperty::Filter => endpoint.filter = None,
+                DeleteableSmtpProperty::Mailto => endpoint.mailto = None,
+                DeleteableSmtpProperty::MailtoUser => endpoint.mailto_user = None,
+                DeleteableSmtpProperty::Password => super::set_private_config_entry(
+                    config,
+                    &SmtpPrivateConfig {
+                        name: name.to_string(),
+                        password: None,
+                    },
+                    SMTP_TYPENAME,
+                    name,
+                )?,
+                DeleteableSmtpProperty::Port => endpoint.port = None,
+                DeleteableSmtpProperty::Username => endpoint.username = None,
+            }
+        }
+    }
+
+    if let Some(mailto) = &updater.mailto {
+        endpoint.mailto = Some(mailto.iter().map(String::from).collect());
+    }
+    if let Some(mailto_user) = &updater.mailto_user {
+        endpoint.mailto_user = Some(mailto_user.iter().map(String::from).collect());
+    }
+    if let Some(from_address) = &updater.from_address {
+        endpoint.from_address = from_address.into();
+    }
+    if let Some(server) = &updater.server {
+        endpoint.server = server.into();
+    }
+    if let Some(port) = &updater.port {
+        endpoint.port = Some(*port);
+    }
+    if let Some(username) = &updater.username {
+        endpoint.username = Some(username.into());
+    }
+    if let Some(mode) = &updater.mode {
+        endpoint.mode = Some(*mode);
+    }
+    if let Some(password) = &private_endpoint_config_updater.password {
+        super::set_private_config_entry(
+            config,
+            &SmtpPrivateConfig {
+                name: name.into(),
+                password: Some(password.into()),
+            },
+            SMTP_TYPENAME,
+            name,
+        )?;
+    }
+
+    if let Some(author) = &updater.author {
+        endpoint.author = Some(author.into());
+    }
+
+    if let Some(comment) = &updater.comment {
+        endpoint.comment = Some(comment.into());
+    }
+
+    if let Some(filter) = &updater.filter {
+        let _ = super::filter::get_filter(config, filter)?;
+        endpoint.filter = Some(filter.into());
+    }
+
+    if endpoint.mailto.is_none() && endpoint.mailto_user.is_none() {
+        http_bail!(
+            BAD_REQUEST,
+            "must at least provide one recipient, either in mailto or in mailto-user"
+        );
+    }
+
+    config
+        .config
+        .set_data(name, SMTP_TYPENAME, &endpoint)
+        .map_err(|e| {
+            http_err!(
+                INTERNAL_SERVER_ERROR,
+                "could not save endpoint '{}': {e}",
+                endpoint.name
+            )
+        })
+}
+
+/// Delete existing smtp endpoint
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns a `HttpError` if:
+///   - an entity with the same name already exists (`400 Bad request`)
+///   - a referenced filter does not exist (`400 Bad request`)
+///   - the configuration could not be saved (`500 Internal server error`)
+pub fn delete_endpoint(config: &mut Config, name: &str) -> Result<(), HttpError> {
+    // Check if the endpoint exists
+    let _ = get_endpoint(config, name)?;
+    super::ensure_unused(config, name)?;
+
+    super::remove_private_config_entry(config, name)?;
+    config.config.sections.remove(name);
+
+    Ok(())
+}
+
+#[cfg(test)]
+pub mod tests {
+    use super::*;
+    use crate::api::test_helpers::*;
+    use crate::endpoints::smtp::SmtpMode;
+
+    pub fn add_smtp_endpoint_for_test(config: &mut Config, name: &str) -> Result<(), HttpError> {
+        add_endpoint(
+            config,
+            &SmtpConfig {
+                name: name.into(),
+                mailto: Some(vec!["user1@example.com".into()]),
+                mailto_user: None,
+                from_address: "from@example.com".into(),
+                author: Some("root".into()),
+                comment: Some("Comment".into()),
+                filter: None,
+                mode: Some(SmtpMode::StartTls),
+                server: "localhost".into(),
+                port: Some(555),
+                username: Some("username".into()),
+            },
+            &SmtpPrivateConfig {
+                name: name.into(),
+                password: Some("password".into()),
+            },
+        )?;
+
+        assert!(get_endpoint(config, name).is_ok());
+        Ok(())
+    }
+
+    #[test]
+    fn test_smtp_create() -> Result<(), HttpError> {
+        let mut config = empty_config();
+
+        assert_eq!(get_endpoints(&config)?.len(), 0);
+        add_smtp_endpoint_for_test(&mut config, "smtp-endpoint")?;
+
+        // Endpoints must have a unique name
+        assert!(add_smtp_endpoint_for_test(&mut config, "smtp-endpoint").is_err());
+        assert_eq!(get_endpoints(&config)?.len(), 1);
+        Ok(())
+    }
+
+    #[test]
+    fn test_update_not_existing_returns_error() -> Result<(), HttpError> {
+        let mut config = empty_config();
+
+        assert!(update_endpoint(
+            &mut config,
+            "test",
+            &Default::default(),
+            &Default::default(),
+            None,
+            None,
+        )
+        .is_err());
+
+        Ok(())
+    }
+
+    #[test]
+    fn test_update_invalid_digest_returns_error() -> Result<(), HttpError> {
+        let mut config = empty_config();
+        add_smtp_endpoint_for_test(&mut config, "sendmail-endpoint")?;
+
+        assert!(update_endpoint(
+            &mut config,
+            "sendmail-endpoint",
+            &Default::default(),
+            &Default::default(),
+            None,
+            Some(&[0; 32]),
+        )
+        .is_err());
+
+        Ok(())
+    }
+
+    #[test]
+    fn test_update() -> Result<(), HttpError> {
+        let mut config = empty_config();
+        add_smtp_endpoint_for_test(&mut config, "smtp-endpoint")?;
+
+        let digest = config.digest;
+
+        update_endpoint(
+            &mut config,
+            "smtp-endpoint",
+            &SmtpConfigUpdater {
+                mailto: Some(vec!["user2@example.com".into(), "user3@example.com".into()]),
+                mailto_user: Some(vec!["root@pam".into()]),
+                from_address: Some("root@example.com".into()),
+                author: Some("newauthor".into()),
+                comment: Some("new comment".into()),
+                mode: Some(SmtpMode::Insecure),
+                server: Some("pali".into()),
+                port: Some(444),
+                username: Some("newusername".into()),
+                ..Default::default()
+            },
+            &Default::default(),
+            None,
+            Some(&digest),
+        )?;
+
+        let endpoint = get_endpoint(&config, "smtp-endpoint")?;
+
+        assert_eq!(
+            endpoint.mailto,
+            Some(vec![
+                "user2@example.com".to_string(),
+                "user3@example.com".to_string()
+            ])
+        );
+        assert_eq!(endpoint.mailto_user, Some(vec!["root@pam".to_string(),]));
+        assert_eq!(endpoint.from_address, "root@example.com".to_string());
+        assert_eq!(endpoint.author, Some("newauthor".to_string()));
+        assert_eq!(endpoint.comment, Some("new comment".to_string()));
+
+        // Test property deletion
+        update_endpoint(
+            &mut config,
+            "smtp-endpoint",
+            &Default::default(),
+            &Default::default(),
+            Some(&[
+                DeleteableSmtpProperty::Author,
+                DeleteableSmtpProperty::MailtoUser,
+                DeleteableSmtpProperty::Port,
+                DeleteableSmtpProperty::Username,
+                DeleteableSmtpProperty::Filter,
+                DeleteableSmtpProperty::Comment,
+            ]),
+            None,
+        )?;
+
+        let endpoint = get_endpoint(&config, "smtp-endpoint")?;
+
+        assert_eq!(endpoint.author, None);
+        assert_eq!(endpoint.comment, None);
+        assert_eq!(endpoint.port, None);
+        assert_eq!(endpoint.username, None);
+        assert_eq!(endpoint.filter, None);
+        assert_eq!(endpoint.mailto_user, None);
+
+        Ok(())
+    }
+
+    #[test]
+    fn test_delete() -> Result<(), HttpError> {
+        let mut config = empty_config();
+        add_smtp_endpoint_for_test(&mut config, "smtp-endpoint")?;
+
+        delete_endpoint(&mut config, "smtp-endpoint")?;
+        assert!(delete_endpoint(&mut config, "smtp-endpoint").is_err());
+        assert_eq!(get_endpoints(&config)?.len(), 0);
+
+        Ok(())
+    }
+}
-- 
2.39.2

[pve-devel] [PATCH v3 proxmox-perl-rs 4/7] notify: add bindings for smtp API calls

[Lukas Wagner]

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
 common/src/notify.rs | 110 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 110 insertions(+)

diff --git a/common/src/notify.rs b/common/src/notify.rs
index 203acca..b0a7d44 100644
--- a/common/src/notify.rs
+++ b/common/src/notify.rs
@@ -13,6 +13,10 @@ mod export {
     use proxmox_notify::endpoints::sendmail::{
         DeleteableSendmailProperty, SendmailConfig, SendmailConfigUpdater,
     };
+    use proxmox_notify::endpoints::smtp::{
+        DeleteableSmtpProperty, SmtpConfig, SmtpConfigUpdater, SmtpMode, SmtpPrivateConfig,
+        SmtpPrivateConfigUpdater,
+    };
     use proxmox_notify::filter::{
         DeleteableFilterProperty, FilterConfig, FilterConfigUpdater, FilterModeOperator,
     };
@@ -344,6 +348,112 @@ mod export {
         api::gotify::delete_gotify_endpoint(&mut config, name)
     }
 
+    #[export(serialize_error)]
+    fn get_smtp_endpoints(
+        #[try_from_ref] this: &NotificationConfig,
+    ) -> Result<Vec<SmtpConfig>, HttpError> {
+        let config = this.config.lock().unwrap();
+        api::smtp::get_endpoints(&config)
+    }
+
+    #[export(serialize_error)]
+    fn get_smtp_endpoint(
+        #[try_from_ref] this: &NotificationConfig,
+        id: &str,
+    ) -> Result<SmtpConfig, HttpError> {
+        let config = this.config.lock().unwrap();
+        api::smtp::get_endpoint(&config, id)
+    }
+
+    #[export(serialize_error)]
+    #[allow(clippy::too_many_arguments)]
+    fn add_smtp_endpoint(
+        #[try_from_ref] this: &NotificationConfig,
+        name: String,
+        server: String,
+        port: Option<u16>,
+        mode: Option<SmtpMode>,
+        username: Option<String>,
+        password: Option<String>,
+        mailto: Option<Vec<String>>,
+        mailto_user: Option<Vec<String>>,
+        from_address: String,
+        author: Option<String>,
+        comment: Option<String>,
+        filter: Option<String>,
+    ) -> Result<(), HttpError> {
+        let mut config = this.config.lock().unwrap();
+        api::smtp::add_endpoint(
+            &mut config,
+            &SmtpConfig {
+                name: name.clone(),
+                server,
+                port,
+                mode,
+                username,
+                mailto,
+                mailto_user,
+                from_address,
+                author,
+                comment,
+                filter,
+            },
+            &SmtpPrivateConfig { name, password },
+        )
+    }
+
+    #[export(serialize_error)]
+    #[allow(clippy::too_many_arguments)]
+    fn update_smtp_endpoint(
+        #[try_from_ref] this: &NotificationConfig,
+        name: &str,
+        server: Option<String>,
+        port: Option<u16>,
+        mode: Option<SmtpMode>,
+        username: Option<String>,
+        password: Option<String>,
+        mailto: Option<Vec<String>>,
+        mailto_user: Option<Vec<String>>,
+        from_address: Option<String>,
+        author: Option<String>,
+        comment: Option<String>,
+        filter: Option<String>,
+        delete: Option<Vec<DeleteableSmtpProperty>>,
+        digest: Option<&str>,
+    ) -> Result<(), HttpError> {
+        let mut config = this.config.lock().unwrap();
+        let digest = decode_digest(digest)?;
+
+        api::smtp::update_endpoint(
+            &mut config,
+            name,
+            &SmtpConfigUpdater {
+                server,
+                port,
+                mode,
+                username,
+                mailto,
+                mailto_user,
+                from_address,
+                author,
+                comment,
+                filter,
+            },
+            &SmtpPrivateConfigUpdater { password },
+            delete.as_deref(),
+            digest.as_deref(),
+        )
+    }
+
+    #[export(serialize_error)]
+    fn delete_smtp_endpoint(
+        #[try_from_ref] this: &NotificationConfig,
+        name: &str,
+    ) -> Result<(), HttpError> {
+        let mut config = this.config.lock().unwrap();
+        api::smtp::delete_endpoint(&mut config, name)
+    }
+
     #[export(serialize_error)]
     fn get_filters(
         #[try_from_ref] this: &NotificationConfig,
-- 
2.39.2

[pve-devel] [PATCH v3 pve-manager 5/7] notify: add API routes for smtp endpoints

[Lukas Wagner]

The Perl part of the API methods primarily defines the API schema,
checks for any needed privileges and then calls the actual Rust
implementation exposed via perlmod. Any errors returned by the Rust
code are translated into PVE::Exception, so that the API call fails
with the correct HTTP error code.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
 PVE/API2/Cluster/Notifications.pm | 337 ++++++++++++++++++++++++++++++
 1 file changed, 337 insertions(+)

diff --git a/PVE/API2/Cluster/Notifications.pm b/PVE/API2/Cluster/Notifications.pm
index ec666903..0f9d6432 100644
--- a/PVE/API2/Cluster/Notifications.pm
+++ b/PVE/API2/Cluster/Notifications.pm
@@ -221,6 +221,14 @@ __PACKAGE__->register_method ({
 		};
 	    }
 
+	    for my $target (@{$config->get_smtp_endpoints()}) {
+		push @$result, {
+		    name => $target->{name},
+		    comment => $target->{comment},
+		    type => 'smtp',
+		};
+	    }
+
 	    for my $target (@{$config->get_groups()}) {
 		push @$result, {
 		    name => $target->{name},
@@ -1076,6 +1084,335 @@ __PACKAGE__->register_method ({
     }
 });
 
+my $smtp_properties= {
+    name => {
+	description => 'The name of the endpoint.',
+	type => 'string',
+	format => 'pve-configid',
+    },
+    server => {
+	description => 'The address of the SMTP server.',
+	type => 'string',
+    },
+    port => {
+	description => 'The port to be used. Defaults to 465 for TLS based connections,'
+	    . ' 587 for STARTTLS based connections and port 25 for insecure plain-text'
+	    . ' connections.',
+	type => 'integer',
+	optional => 1,
+    },
+    mode => {
+	description => 'Determine which encryption method shall be used for the connection.',
+	type => 'string',
+	enum => [ qw(insecure starttls tls) ],
+	default => 'tls',
+	optional => 1,
+    },
+    username => {
+	description => 'Username for SMTP authentication',
+	type => 'string',
+	optional => 1,
+    },
+    password => {
+	description => 'Password for SMTP authentication',
+	type => 'string',
+	optional => 1,
+    },
+    mailto => {
+	type => 'array',
+	items => {
+	    type => 'string',
+	    format => 'email-or-username',
+	},
+	description => 'List of email recipients',
+	optional => 1,
+    },
+    'mailto-user' => {
+	type => 'array',
+	items => {
+	    type => 'string',
+	    format => 'pve-userid',
+	},
+	description => 'List of users',
+	optional => 1,
+    },
+    'from-address' => {
+	description => '`From` address for the mail',
+	type => 'string',
+    },
+    author => {
+	description => 'Author of the mail. Defaults to \'Proxmox VE\'.',
+	type => 'string',
+	optional => 1,
+    },
+    'comment' => {
+	description => 'Comment',
+	type        => 'string',
+	optional    => 1,
+    },
+    filter => {
+	description => 'Name of the filter that should be applied.',
+	type => 'string',
+	format => 'pve-configid',
+	optional => 1,
+    },
+};
+
+__PACKAGE__->register_method ({
+    name => 'get_smtp_endpoints',
+    path => 'endpoints/smtp',
+    method => 'GET',
+    description => 'Returns a list of all smtp endpoints',
+    permissions => {
+	description => "Only lists entries where you have 'Mapping.Modify', 'Mapping.Use' or"
+	    . " 'Mapping.Audit' permissions on '/mapping/notification/<name>'.",
+	user => 'all',
+    },
+    protected => 1,
+    parameters => {
+	additionalProperties => 0,
+	properties => {},
+    },
+    returns => {
+	type => 'array',
+	items => {
+	    type => 'object',
+	    properties => $smtp_properties,
+	},
+	links => [ { rel => 'child', href => '{name}' } ],
+    },
+    code => sub {
+	my $config = PVE::Notify::read_config();
+	my $rpcenv = PVE::RPCEnvironment::get();
+
+	my $entities = eval {
+	    $config->get_smtp_endpoints();
+	};
+	raise_api_error($@) if $@;
+
+	return filter_entities_by_privs($rpcenv, $entities);
+    }
+});
+
+__PACKAGE__->register_method ({
+    name => 'get_smtp_endpoint',
+    path => 'endpoints/smtp/{name}',
+    method => 'GET',
+    description => 'Return a specific smtp endpoint',
+    permissions => {
+	check => ['or',
+	    ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+	    ['perm', '/mapping/notification/{name}', ['Mapping.Audit']],
+	],
+    },
+    protected => 1,
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    name => {
+		type => 'string',
+		format => 'pve-configid',
+	    },
+	}
+    },
+    returns => {
+	type => 'object',
+	properties => {
+	    %{ remove_protected_properties($smtp_properties, ['password']) },
+	    digest => get_standard_option('pve-config-digest'),
+	}
+
+    },
+    code => sub {
+	my ($param) = @_;
+	my $name = extract_param($param, 'name');
+
+	my $config = PVE::Notify::read_config();
+	my $endpoint = eval {
+	    $config->get_smtp_endpoint($name)
+	};
+
+	raise_api_error($@) if $@;
+	$endpoint->{digest} = $config->digest();
+
+	return $endpoint;
+    }
+});
+
+__PACKAGE__->register_method ({
+    name => 'create_smtp_endpoint',
+    path => 'endpoints/smtp',
+    protected => 1,
+    method => 'POST',
+    description => 'Create a new smtp endpoint',
+    permissions => {
+	check => ['perm', '/mapping/notification', ['Mapping.Modify']],
+    },
+    parameters => {
+	additionalProperties => 0,
+	properties => $smtp_properties,
+    },
+    returns => { type => 'null' },
+    code => sub {
+	my ($param) = @_;
+
+	my $name = extract_param($param, 'name');
+	my $server = extract_param($param, 'server');
+	my $port = extract_param($param, 'port');
+	my $mode = extract_param($param, 'mode');
+	my $username = extract_param($param, 'username');
+	my $password = extract_param($param, 'password');
+	my $mailto = extract_param($param, 'mailto');
+	my $mailto_user = extract_param($param, 'mailto-user');
+	my $from_address = extract_param($param, 'from-address');
+	my $author = extract_param($param, 'author');
+	my $comment = extract_param($param, 'comment');
+	my $filter = extract_param($param, 'filter');
+
+	eval {
+	    PVE::Notify::lock_config(sub {
+		my $config = PVE::Notify::read_config();
+
+		$config->add_smtp_endpoint(
+		    $name,
+		    $server,
+		    $port,
+		    $mode,
+		    $username,
+		    $password,
+		    $mailto,
+		    $mailto_user,
+		    $from_address,
+		    $author,
+		    $comment,
+		    $filter
+		);
+
+		PVE::Notify::write_config($config);
+	    });
+	};
+
+	raise_api_error($@) if $@;
+	return;
+    }
+});
+
+__PACKAGE__->register_method ({
+    name => 'update_smtp_endpoint',
+    path => 'endpoints/smtp/{name}',
+    protected => 1,
+    method => 'PUT',
+    description => 'Update existing smtp endpoint',
+    permissions => {
+	check => ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+    },
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    %{ make_properties_optional($smtp_properties) },
+	    delete => {
+		type => 'array',
+		items => {
+		    type => 'string',
+		    format => 'pve-configid',
+		},
+		optional => 1,
+		description => 'A list of settings you want to delete.',
+	    },
+	    digest => get_standard_option('pve-config-digest'),
+
+	}
+    },
+    returns => { type => 'null' },
+    code => sub {
+	my ($param) = @_;
+
+	my $name = extract_param($param, 'name');
+	my $server = extract_param($param, 'server');
+	my $port = extract_param($param, 'port');
+	my $mode = extract_param($param, 'mode');
+	my $username = extract_param($param, 'username');
+	my $password = extract_param($param, 'password');
+	my $mailto = extract_param($param, 'mailto');
+	my $mailto_user = extract_param($param, 'mailto-user');
+	my $from_address = extract_param($param, 'from-address');
+	my $author = extract_param($param, 'author');
+	my $comment = extract_param($param, 'comment');
+	my $filter = extract_param($param, 'filter');
+
+	my $delete = extract_param($param, 'delete');
+	my $digest = extract_param($param, 'digest');
+
+	eval {
+	    PVE::Notify::lock_config(sub {
+		my $config = PVE::Notify::read_config();
+
+		$config->update_smtp_endpoint(
+		    $name,
+		    $server,
+		    $port,
+		    $mode,
+		    $username,
+		    $password,
+		    $mailto,
+		    $mailto_user,
+		    $from_address,
+		    $author,
+		    $comment,
+		    $filter,
+		    $delete,
+		    $digest,
+		);
+
+		PVE::Notify::write_config($config);
+	    });
+	};
+
+	raise_api_error($@) if $@;
+	return;
+    }
+});
+
+__PACKAGE__->register_method ({
+    name => 'delete_smtp_endpoint',
+    protected => 1,
+    path => 'endpoints/smtp/{name}',
+    method => 'DELETE',
+    description => 'Remove smtp endpoint',
+    permissions => {
+	check => ['perm', '/mapping/notification', ['Mapping.Modify']],
+    },
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    name => {
+		type => 'string',
+		format => 'pve-configid',
+	    },
+	}
+    },
+    returns => { type => 'null' },
+    code => sub {
+	my ($param) = @_;
+	my $name = extract_param($param, 'name');
+
+	my $used_by = target_used_by($name);
+	if ($used_by) {
+	    raise_param_exc({'name' => "Cannot remove $name, used by: $used_by"});
+	}
+
+	eval {
+	    PVE::Notify::lock_config(sub {
+		my $config = PVE::Notify::read_config();
+		$config->delete_smtp_endpoint($name);
+		PVE::Notify::write_config($config);
+	    });
+	};
+
+	raise_api_error($@) if ($@);
+	return;
+    }
+});
 my $filter_properties = {
     name => {
 	description => 'Name of the endpoint.',
-- 
2.39.2

[pve-devel] [PATCH v3 proxmox-widget-toolkit 6/7] panel: notification: add gui for SMTP endpoints

[Lukas Wagner]

This new endpoint configuration panel is embedded in the existing
EndpointEditBase dialog window. This commit also factors out some of
the non-trivial common form elements that are shared between the new
panel and the already existing SendmailEditPanel into a separate panel
EmailRecipientPanel.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
 src/Makefile                     |   2 +
 src/Schema.js                    |   5 +
 src/panel/EmailRecipientPanel.js |  89 ++++++++++++++
 src/panel/SendmailEditPanel.js   |  67 ++---------
 src/panel/SmtpEditPanel.js       | 192 +++++++++++++++++++++++++++++++
 5 files changed, 296 insertions(+), 59 deletions(-)
 create mode 100644 src/panel/EmailRecipientPanel.js
 create mode 100644 src/panel/SmtpEditPanel.js

diff --git a/src/Makefile b/src/Makefile
index 21fbe76..113064d 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -73,7 +73,9 @@ JSSRC=					\
 	panel/ACMEAccount.js		\
 	panel/ACMEPlugin.js		\
 	panel/ACMEDomains.js		\
+	panel/EmailRecipientPanel.js	\
 	panel/SendmailEditPanel.js	\
+	panel/SmtpEditPanel.js	\
 	panel/StatusView.js		\
 	panel/TfaView.js		\
 	panel/NotesView.js		\
diff --git a/src/Schema.js b/src/Schema.js
index a7ffdf8..2653b99 100644
--- a/src/Schema.js
+++ b/src/Schema.js
@@ -43,6 +43,11 @@ Ext.define('Proxmox.Schema', { // a singleton
 	    ipanel: 'pmxSendmailEditPanel',
 	    iconCls: 'fa-envelope-o',
 	},
+	smtp: {
+	    name: gettext('SMTP'),
+	    ipanel: 'pmxSmtpEditPanel',
+	    iconCls: 'fa-envelope-o',
+	},
 	gotify: {
 	    name: gettext('Gotify'),
 	    ipanel: 'pmxGotifyEditPanel',
diff --git a/src/panel/EmailRecipientPanel.js b/src/panel/EmailRecipientPanel.js
new file mode 100644
index 0000000..26c93c4
--- /dev/null
+++ b/src/panel/EmailRecipientPanel.js
@@ -0,0 +1,89 @@
+Ext.define('Proxmox.panel.EmailRecipientPanel', {
+    extend: 'Proxmox.panel.InputPanel',
+    xtype: 'pmxEmailRecipientPanel',
+    mixins: ['Proxmox.Mixin.CBind'],
+
+    mailValidator: function() {
+	let mailto_user = this.down(`[name=mailto-user]`);
+	let mailto = this.down(`[name=mailto]`);
+
+	if (!mailto_user.getValue()?.length && !mailto.getValue()) {
+	    return gettext('Either mailto or mailto-user must be set');
+	}
+
+	return true;
+    },
+
+    items: [
+	{
+	    xtype: 'pmxUserSelector',
+	    name: 'mailto-user',
+	    multiSelect: true,
+	    allowBlank: true,
+	    editable: false,
+	    skipEmptyText: true,
+	    fieldLabel: gettext('Recipient(s)'),
+	    cbind: {
+		deleteEmpty: '{!isCreate}',
+	    },
+	    validator: function() {
+		return this.up('pmxEmailRecipientPanel').mailValidator();
+	    },
+	    autoEl: {
+		tag: 'div',
+		'data-qtip': gettext('The notification will be sent to the user\'s configured mail address'),
+	    },
+	    listConfig: {
+		width: 600,
+		columns: [
+		    {
+			header: gettext('User'),
+			sortable: true,
+			dataIndex: 'userid',
+			renderer: Ext.String.htmlEncode,
+			flex: 1,
+		    },
+		    {
+			header: gettext('E-Mail'),
+			sortable: true,
+			dataIndex: 'email',
+			renderer: Ext.String.htmlEncode,
+			flex: 1,
+		    },
+		    {
+			header: gettext('Comment'),
+			sortable: false,
+			dataIndex: 'comment',
+			renderer: Ext.String.htmlEncode,
+			flex: 1,
+		    },
+		],
+	    },
+	},
+	{
+	    xtype: 'proxmoxtextfield',
+	    fieldLabel: gettext('Additional Recipient(s)'),
+	    name: 'mailto',
+	    allowBlank: true,
+	    emptyText: 'user@example.com, ...',
+	    cbind: {
+		deleteEmpty: '{!isCreate}',
+	    },
+	    autoEl: {
+		tag: 'div',
+		'data-qtip': gettext('Multiple recipients must be separated by spaces, commas or semicolons'),
+	    },
+	    validator: function() {
+		return this.up('pmxEmailRecipientPanel').mailValidator();
+	    },
+	},
+    ],
+
+    onGetValues: function(values) {
+	if (values.mailto) {
+	    values.mailto = values.mailto.split(/[\s,;]+/);
+	}
+
+	return values;
+    },
+});
diff --git a/src/panel/SendmailEditPanel.js b/src/panel/SendmailEditPanel.js
index ace6129..5773529 100644
--- a/src/panel/SendmailEditPanel.js
+++ b/src/panel/SendmailEditPanel.js
@@ -28,62 +28,9 @@ Ext.define('Proxmox.panel.SendmailEditPanel', {
 	    allowBlank: false,
 	},
 	{
-	    xtype: 'pmxUserSelector',
-	    name: 'mailto-user',
-	    reference: 'mailto-user',
-	    multiSelect: true,
-	    allowBlank: true,
-	    editable: false,
-	    skipEmptyText: true,
-	    fieldLabel: gettext('User(s)'),
+	    xtype: 'pmxEmailRecipientPanel',
 	    cbind: {
-		deleteEmpty: '{!isCreate}',
-	    },
-	    validator: function() {
-		return this.up('pmxSendmailEditPanel').mailValidator();
-	    },
-	    listConfig: {
-		width: 600,
-		columns: [
-		    {
-			header: gettext('User'),
-			sortable: true,
-			dataIndex: 'userid',
-			renderer: Ext.String.htmlEncode,
-			flex: 1,
-		    },
-		    {
-			header: gettext('E-Mail'),
-			sortable: true,
-			dataIndex: 'email',
-			renderer: Ext.String.htmlEncode,
-			flex: 1,
-		    },
-		    {
-			header: gettext('Comment'),
-			sortable: false,
-			dataIndex: 'comment',
-			renderer: Ext.String.htmlEncode,
-			flex: 1,
-		    },
-		],
-	    },
-	},
-	{
-	    xtype: 'proxmoxtextfield',
-	    fieldLabel: gettext('Additional Recipient(s)'),
-	    name: 'mailto',
-	    reference: 'mailto',
-	    allowBlank: true,
-	    cbind: {
-		deleteEmpty: '{!isCreate}',
-	    },
-	    autoEl: {
-		tag: 'div',
-		'data-qtip': gettext('Multiple recipients must be separated by spaces, commas or semicolons'),
-	    },
-	    validator: function() {
-		return this.up('pmxSendmailEditPanel').mailValidator();
+		isCreate: '{isCreate}',
 	    },
 	},
 	{
@@ -128,10 +75,12 @@ Ext.define('Proxmox.panel.SendmailEditPanel', {
 	},
     ],
 
-    onGetValues: (values) => {
-	if (values.mailto) {
-	    values.mailto = values.mailto.split(/[\s,;]+/);
-	}
+    onGetValues: function(values) {
+	// Since mailto and mailto-user are in a separate InputPanel, we have
+	// to delete them here. Otherwise, their values will be collected twice.
+	delete values.mailto;
+	delete values['mailto-user'];
+
 	return values;
     },
 });
diff --git a/src/panel/SmtpEditPanel.js b/src/panel/SmtpEditPanel.js
new file mode 100644
index 0000000..6f9012a
--- /dev/null
+++ b/src/panel/SmtpEditPanel.js
@@ -0,0 +1,192 @@
+Ext.define('Proxmox.panel.SmtpEditPanel', {
+    extend: 'Proxmox.panel.InputPanel',
+    xtype: 'pmxSmtpEditPanel',
+    mixins: ['Proxmox.Mixin.CBind'],
+
+    type: 'smtp',
+
+    viewModel: {
+	xtype: 'viewmodel',
+	cbind: {
+	    isCreate: "{isCreate}",
+	},
+	data: {
+	    mode: 'tls',
+	    authentication: true,
+	},
+	formulas: {
+	    portEmptyText: function(get) {
+		let port;
+
+		switch (get('mode')) {
+		    case 'insecure':
+			port = 25;
+			break;
+		    case 'starttls':
+			port = 587;
+			break;
+		    case 'tls':
+			port = 465;
+			break;
+		}
+		return `${Proxmox.Utils.defaultText} (${port})`;
+	    },
+	    passwordEmptyText: function(get) {
+		let isCreate = this.isCreate;
+		return get('authentication') && !isCreate ? gettext('Unchanged') : '';
+	    },
+	},
+    },
+
+    columnT: [
+	{
+	    xtype: 'pmxDisplayEditField',
+	    name: 'name',
+	    cbind: {
+		value: '{name}',
+		editable: '{isCreate}',
+	    },
+	    fieldLabel: gettext('Endpoint Name'),
+	    allowBlank: false,
+	},
+    ],
+
+    column1: [
+	{
+	    xtype: 'proxmoxtextfield',
+	    fieldLabel: gettext('Server'),
+	    name: 'server',
+	    allowBlank: false,
+	    emptyText: gettext('mail.example.com'),
+	},
+	{
+	    xtype: 'proxmoxKVComboBox',
+	    name: 'mode',
+	    fieldLabel: gettext('Encryption'),
+	    editable: false,
+	    comboItems: [
+		['insecure', Proxmox.Utils.noneText + ' (' + gettext('insecure') + ')'],
+		['starttls', 'STARTTLS'],
+		['tls', 'TLS'],
+	    ],
+	    bind: "{mode}",
+	    cbind: {
+		deleteEmpty: '{!isCreate}',
+	    },
+	},
+	{
+	    xtype: 'proxmoxintegerfield',
+	    name: 'port',
+	    fieldLabel: gettext('Port'),
+	    minValue: 1,
+	    maxValue: 65535,
+	    bind: {
+		emptyText: "{portEmptyText}",
+	    },
+	    submitEmptyText: false,
+	    cbind: {
+		deleteEmpty: '{!isCreate}',
+	    },
+	},
+    ],
+    column2: [
+	{
+	    xtype: 'proxmoxcheckbox',
+	    fieldLabel: gettext('Authenticate'),
+	    name: 'authentication',
+	    bind: {
+		value: '{authentication}',
+	    },
+	},
+	{
+	    xtype: 'proxmoxtextfield',
+	    fieldLabel: gettext('Username'),
+	    name: 'username',
+	    allowBlank: false,
+	    cbind: {
+		deleteEmpty: '{!isCreate}',
+	    },
+	    bind: {
+		disabled: '{!authentication}',
+	    },
+	},
+	{
+	    xtype: 'proxmoxtextfield',
+	    inputType: 'password',
+	    fieldLabel: gettext('Password'),
+	    name: 'password',
+	    allowBlank: true,
+	    bind: {
+		disabled: '{!authentication}',
+		emptyText: '{passwordEmptyText}',
+	    },
+	},
+    ],
+    columnB: [
+	{
+	    xtype: 'proxmoxtextfield',
+	    fieldLabel: gettext('From Address'),
+	    name: 'from-address',
+	    allowBlank: false,
+	    emptyText: gettext('user@example.com'),
+	},
+	{
+	    xtype: 'pmxEmailRecipientPanel',
+	    cbind: {
+		isCreate: '{isCreate}',
+	    },
+	},
+	{
+	    xtype: 'pmxNotificationFilterSelector',
+	    name: 'filter',
+	    fieldLabel: gettext('Filter'),
+	    cbind: {
+		deleteEmpty: '{!isCreate}',
+		baseUrl: '{baseUrl}',
+	    },
+	},
+	{
+	    xtype: 'proxmoxtextfield',
+	    name: 'comment',
+	    fieldLabel: gettext('Comment'),
+	    cbind: {
+		deleteEmpty: '{!isCreate}',
+	    },
+	},
+    ],
+
+    advancedColumnB: [
+	{
+	    xtype: 'proxmoxtextfield',
+	    fieldLabel: gettext('Author'),
+	    name: 'author',
+	    allowBlank: true,
+	    emptyText: gettext('Proxmox VE'),
+	    cbind: {
+		deleteEmpty: '{!isCreate}',
+	    },
+	},
+    ],
+
+    onGetValues: function(values) {
+	// Since mailto and mailto-user are in a separate InputPanel, we have
+	// to delete them here. Otherwise, their values will be collected twice.
+	delete values.mailto;
+	delete values['mailto-user'];
+
+	if (!values.authentication && !this.isCreate) {
+	    Proxmox.Utils.assemble_field_data(values, { 'delete': 'username' });
+	    Proxmox.Utils.assemble_field_data(values, { 'delete': 'password' });
+	}
+
+	delete values.authentication;
+
+	return values;
+    },
+
+    onSetValues: function(values) {
+	values.authentication = !!values.username;
+
+	return values;
+    },
+});
-- 
2.39.2

[pve-devel] [PATCH v3 pve-docs 7/7] notifications: document SMTP endpoints

[Lukas Wagner]

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
 notifications.adoc | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/notifications.adoc b/notifications.adoc
index 0b00b1e..c2fe393 100644
--- a/notifications.adoc
+++ b/notifications.adoc
@@ -73,9 +73,37 @@ accomodate multiple recipients.
 set, the plugin will fall back to the `email_from` setting from
 `datacenter.cfg`. If that is also not set, the plugin will default to
 `root@$hostname`, where `$hostname` is the hostname of the node.
+The `From` header in the email will be set to `$author <$from-address>`.
 
 * `filter`: The name of the filter to use for this target.
 
+SMTP
+~~~~
+
+SMTP notification targets can send emails directly to an SMTP mail relay.
+
+The configuration for SMTP target plugins has the following options:
+
+* `mailto`: E-Mail address to which the notification shall be sent to. Can be
+set multiple times to accomodate multiple recipients.
+* `mailto-user`: Users to which emails shall be sent to. The user's email
+address will be looked up in `users.cfg`. Can be set multiple times to
+accomodate multiple recipients.
+* `author`: Sets the author of the E-Mail. Defaults to `Proxmox VE`.
+* `from-address`: Sets the From-addresss of the email. SMTP relays might require
+that this address is owned by the user in order to avoid spoofing.
+The `From` header in the email will be set to `$author <$from-address>`.
+* `username`: Username to use during authentication. If no username is set,
+no authentication will be performed. The PLAIN and LOGIN authentication methods
+are supported.
+* `password`: Password to use when authenticating.
+* `mode`: Sets the encryption mode (`insecure`, `starttls` or `tls`). Defaults
+to `tls`.
+* `port`: The SMTP to use. If not set, the used port
+defaults to 25 (`insecure`), 465 (`tls`) or 587 (`starttls`), deping on the
+value of `mode`.
+* `filter`: The name of the filter to use for this target.
+
 Gotify
 ~~~~~~
 
-- 
2.39.2

Re: [pve-devel] [PATCH v3 many 0/7] notifications: add SMTP endpoint

[Lukas Wagner]

The notification system overhaul is available in the pvetest repository,
however that does not yet include the SMTP endpoint type. Only gotify 
and sendmail (uses the system's sendmail command to send mails) are 
available so far.

-- 
- Lukas

Re: [pve-devel] [PATCH v3 many 0/7] notifications: add SMTP endpoint

[Lukas Wagner]

Ping - would be great to get some reviews on this to get this merged for
the next release.

On 9/18/23 13:14, Lukas Wagner wrote:
> This patch series adds support for a new notification endpoint type,
> smtp. As the name suggests, this new endpoint allows PVE to talk
> to SMTP server directly, without using the system's MTA (postfix).
> 


-- 
- Lukas

Re: [pve-devel] [PATCH v3 many 0/7] notifications: add SMTP endpoint

[Lukas Wagner]

On 10/17/23 09:27, Lukas Wagner wrote:
> Ping - would be great to get some reviews on this to get this merged for
> the next release.
> 
> On 9/18/23 13:14, Lukas Wagner wrote:
>> This patch series adds support for a new notification endpoint type,
>> smtp. As the name suggests, this new endpoint allows PVE to talk
>> to SMTP server directly, without using the system's MTA (postfix).
>>

There will be some conceptual changes to the notification system, 
requiring some changes to this series as well. A new version will
be posted at some point.

-- 
- Lukas