* [pbs-devel] [PATCH proxmox v3 01/14] notify: renderer: adapt to changes in proxmox-time
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox v3 02/14] notify: implement webhook targets Lukas Wagner
` (13 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
A recent commit [1] changed the `Display` implementation of `TimeSpan` such
that minutes are now displayed as `20m` instead of `20min`.
This commit adapts the tests for the notification template renderer
accordingly.
[1] 19129960 ("time: display minute/month such that it can be parsed again")
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/renderer/mod.rs | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/proxmox-notify/src/renderer/mod.rs b/proxmox-notify/src/renderer/mod.rs
index 8574a3fb..82473d03 100644
--- a/proxmox-notify/src/renderer/mod.rs
+++ b/proxmox-notify/src/renderer/mod.rs
@@ -329,8 +329,8 @@ mod tests {
Some("1 KiB".to_string())
);
- assert_eq!(value_to_duration(&json!(60)), Some("1min ".to_string()));
- assert_eq!(value_to_duration(&json!("60")), Some("1min ".to_string()));
+ assert_eq!(value_to_duration(&json!(60)), Some("1m".to_string()));
+ assert_eq!(value_to_duration(&json!("60")), Some("1m".to_string()));
// The rendered value is in localtime, so we only check if the result is `Some`...
// ... otherwise the test will break in another timezone :S
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] [PATCH proxmox v3 02/14] notify: implement webhook targets
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox v3 01/14] notify: renderer: adapt to changes in proxmox-time Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox v3 03/14] notify: add api for " Lukas Wagner
` (12 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
This target type allows users to perform HTTP requests to arbitrary
third party (notification) services, for instance
ntfy.sh/Discord/Slack.
The configuration for these endpoints allows one to freely configure
the URL, HTTP Method, headers and body. The URL, header values and
body support handlebars templating to inject notification text,
metadata and secrets. Secrets are stored in the protected
configuration file (e.g. /etc/pve/priv/notification.cfg) as key value
pairs, allowing users to protect sensitive tokens/passwords.
Secrets are accessible in handlebar templating via the secrets.*
namespace, e.g. if there is a secret named 'token', a body
could contain '{{ secrets.token }}' to inject the token into the
payload.
A couple of handlebars helpers are also provided:
- url-encoding (useful for templating in URLs)
- escape (escape any control characters in strings)
- json (print a property as json)
In the configuration, the body, header values and secret values
are stored in base64 encoding so that we can store any string we want.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
---
proxmox-notify/Cargo.toml | 9 +-
proxmox-notify/src/config.rs | 23 +
proxmox-notify/src/endpoints/mod.rs | 2 +
proxmox-notify/src/endpoints/webhook.rs | 550 ++++++++++++++++++++++++
proxmox-notify/src/lib.rs | 17 +
5 files changed, 598 insertions(+), 3 deletions(-)
create mode 100644 proxmox-notify/src/endpoints/webhook.rs
diff --git a/proxmox-notify/Cargo.toml b/proxmox-notify/Cargo.toml
index d57a36cd..5a631bfc 100644
--- a/proxmox-notify/Cargo.toml
+++ b/proxmox-notify/Cargo.toml
@@ -13,13 +13,15 @@ rust-version.workspace = true
[dependencies]
anyhow.workspace = true
-base64.workspace = true
+base64 = { workspace = true, optional = true }
const_format.workspace = true
handlebars = { workspace = true }
+http = { workspace = true, optional = true }
lettre = { workspace = true, optional = true }
log.workspace = true
mail-parser = { workspace = true, optional = true }
openssl.workspace = true
+percent-encoding = { workspace = true, optional = true }
regex.workspace = true
serde = { workspace = true, features = ["derive"] }
serde_json.workspace = true
@@ -35,10 +37,11 @@ proxmox-time.workspace = true
proxmox-uuid = { workspace = true, features = ["serde"] }
[features]
-default = ["sendmail", "gotify", "smtp"]
+default = ["sendmail", "gotify", "smtp", "webhook"]
mail-forwarder = ["dep:mail-parser", "dep:proxmox-sys"]
-sendmail = ["dep:proxmox-sys"]
+sendmail = ["dep:proxmox-sys", "dep:base64"]
gotify = ["dep:proxmox-http"]
pve-context = ["dep:proxmox-sys"]
pbs-context = ["dep:proxmox-sys"]
smtp = ["dep:lettre"]
+webhook = ["dep:base64", "dep:http", "dep:percent-encoding", "dep:proxmox-http"]
diff --git a/proxmox-notify/src/config.rs b/proxmox-notify/src/config.rs
index 789c4a7d..4d0b53f7 100644
--- a/proxmox-notify/src/config.rs
+++ b/proxmox-notify/src/config.rs
@@ -57,6 +57,17 @@ fn config_init() -> SectionConfig {
GOTIFY_SCHEMA,
));
}
+ #[cfg(feature = "webhook")]
+ {
+ use crate::endpoints::webhook::{WebhookConfig, WEBHOOK_TYPENAME};
+
+ const WEBHOOK_SCHEMA: &ObjectSchema = WebhookConfig::API_SCHEMA.unwrap_object_schema();
+ config.register_plugin(SectionConfigPlugin::new(
+ WEBHOOK_TYPENAME.to_string(),
+ Some(String::from("name")),
+ WEBHOOK_SCHEMA,
+ ));
+ }
const MATCHER_SCHEMA: &ObjectSchema = MatcherConfig::API_SCHEMA.unwrap_object_schema();
config.register_plugin(SectionConfigPlugin::new(
@@ -110,6 +121,18 @@ fn private_config_init() -> SectionConfig {
));
}
+ #[cfg(feature = "webhook")]
+ {
+ use crate::endpoints::webhook::{WebhookPrivateConfig, WEBHOOK_TYPENAME};
+
+ const WEBHOOK_SCHEMA: &ObjectSchema =
+ WebhookPrivateConfig::API_SCHEMA.unwrap_object_schema();
+ config.register_plugin(SectionConfigPlugin::new(
+ WEBHOOK_TYPENAME.to_string(),
+ Some(String::from("name")),
+ WEBHOOK_SCHEMA,
+ ));
+ }
config
}
diff --git a/proxmox-notify/src/endpoints/mod.rs b/proxmox-notify/src/endpoints/mod.rs
index 97f79fcc..f20bee21 100644
--- a/proxmox-notify/src/endpoints/mod.rs
+++ b/proxmox-notify/src/endpoints/mod.rs
@@ -4,5 +4,7 @@ pub mod gotify;
pub mod sendmail;
#[cfg(feature = "smtp")]
pub mod smtp;
+#[cfg(feature = "webhook")]
+pub mod webhook;
mod common;
diff --git a/proxmox-notify/src/endpoints/webhook.rs b/proxmox-notify/src/endpoints/webhook.rs
new file mode 100644
index 00000000..4ad9cb2f
--- /dev/null
+++ b/proxmox-notify/src/endpoints/webhook.rs
@@ -0,0 +1,550 @@
+//! This endpoint implements a generic webhook target, allowing users to send notifications through
+//! a highly customizable HTTP request.
+//!
+//! The configuration options include specifying the HTTP method, URL, headers, and body.
+//! URLs, headers, and the body support template expansion using the [`handlebars`] templating engine.
+//! For secure handling of passwords or tokens, these values can be stored as secrets.
+//! Secrets are kept in a private configuration file, accessible only by root, and are not retrievable via the API.
+//! Within templates, secrets can be referenced using `{{ secrets.<name> }}`.
+//! Additionally, we take measures to prevent secrets from appearing in logs or error messages.
+use handlebars::{
+ Context as HandlebarsContext, Handlebars, Helper, HelperResult, Output, RenderContext,
+ RenderError as HandlebarsRenderError,
+};
+use http::Request;
+use percent_encoding::AsciiSet;
+use proxmox_schema::property_string::PropertyString;
+use serde::{Deserialize, Serialize};
+use serde_json::{json, Map, Value};
+
+use proxmox_http::client::sync::Client;
+use proxmox_http::{HttpClient, HttpOptions, ProxyConfig};
+use proxmox_schema::api_types::{COMMENT_SCHEMA, HTTP_URL_SCHEMA};
+use proxmox_schema::{api, ApiStringFormat, ApiType, Schema, StringSchema, Updater};
+
+use crate::context::context;
+use crate::renderer::TemplateType;
+use crate::schema::ENTITY_NAME_SCHEMA;
+use crate::{renderer, Content, Endpoint, Error, Notification, Origin};
+
+/// This will be used as a section type in the public/private configuration file.
+pub(crate) const WEBHOOK_TYPENAME: &str = "webhook";
+
+#[api]
+#[derive(Serialize, Deserialize, Clone, Copy, Default)]
+#[serde(rename_all = "kebab-case")]
+/// HTTP Method to use.
+pub enum HttpMethod {
+ /// HTTP POST
+ #[default]
+ Post,
+ /// HTTP PUT
+ Put,
+ /// HTTP GET
+ Get,
+}
+
+// We only ever need a &str, so we rather implement this
+// instead of Display.
+impl From<HttpMethod> for &str {
+ fn from(value: HttpMethod) -> Self {
+ match value {
+ HttpMethod::Post => "POST",
+ HttpMethod::Put => "PUT",
+ HttpMethod::Get => "GET",
+ }
+ }
+}
+
+#[api(
+ properties: {
+ name: {
+ schema: ENTITY_NAME_SCHEMA,
+ },
+ url: {
+ schema: HTTP_URL_SCHEMA,
+ },
+ comment: {
+ optional: true,
+ schema: COMMENT_SCHEMA,
+ },
+ header: {
+ type: Array,
+ items: {
+ schema: KEY_AND_BASE64_VALUE_SCHEMA,
+ },
+ optional: true,
+ },
+ secret: {
+ type: Array,
+ items: {
+ schema: KEY_AND_BASE64_VALUE_SCHEMA,
+ },
+ optional: true,
+ },
+ }
+)]
+#[derive(Serialize, Deserialize, Updater, Default, Clone)]
+#[serde(rename_all = "kebab-case")]
+/// Config for Webhook notification endpoints
+pub struct WebhookConfig {
+ /// Name of the endpoint.
+ #[updater(skip)]
+ pub name: String,
+
+ pub method: HttpMethod,
+
+ /// Webhook URL. Supports templating.
+ pub url: String,
+ /// Array of HTTP headers. Each entry is a property string with a name and a value.
+ /// The value property contains the header in base64 encoding. Supports templating.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[updater(serde(skip_serializing_if = "Option::is_none"))]
+ pub header: Vec<PropertyString<KeyAndBase64Val>>,
+ /// The HTTP body to send. Supports templating.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub body: Option<String>,
+
+ /// Comment.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub comment: Option<String>,
+ /// Disable this target.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub disable: Option<bool>,
+ /// Origin of this config entry.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ #[updater(skip)]
+ pub origin: Option<Origin>,
+ /// Array of secrets. Each entry is a property string with a name and an optional value.
+ /// The value property contains the secret in base64 encoding.
+ /// For any API endpoints returning the endpoint config,
+ /// only the secret name but not the value will be returned.
+ /// When updating the config, also send all secrets that you want
+ /// to keep, setting only the name but not the value. Can be accessed from templates.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[updater(serde(skip_serializing_if = "Option::is_none"))]
+ pub secret: Vec<PropertyString<KeyAndBase64Val>>,
+}
+
+#[api(
+ properties: {
+ name: {
+ schema: ENTITY_NAME_SCHEMA,
+ },
+ secret: {
+ type: Array,
+ items: {
+ schema: KEY_AND_BASE64_VALUE_SCHEMA,
+ },
+ optional: true,
+ },
+ }
+)]
+#[derive(Serialize, Deserialize, Clone, Updater, Default)]
+#[serde(rename_all = "kebab-case")]
+/// Private configuration for Webhook notification endpoints.
+/// This config will be saved to a separate configuration file with stricter
+/// permissions (root:root 0600).
+pub struct WebhookPrivateConfig {
+ /// Name of the endpoint
+ #[updater(skip)]
+ pub name: String,
+
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[updater(serde(skip_serializing_if = "Option::is_none"))]
+ /// Array of secrets. Each entry is a property string with a name,
+ /// and a value property. The value property contains the secret
+ /// in base64 encoding. Can be accessed from templates.
+ pub secret: Vec<PropertyString<KeyAndBase64Val>>,
+}
+
+/// A Webhook notification endpoint.
+pub struct WebhookEndpoint {
+ pub config: WebhookConfig,
+ pub private_config: WebhookPrivateConfig,
+}
+
+#[api]
+#[derive(Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case")]
+/// Webhook configuration properties that can be deleted.
+pub enum DeleteableWebhookProperty {
+ /// Delete `comment`.
+ Comment,
+ /// Delete `disable`.
+ Disable,
+ /// Delete `header`.
+ Header,
+ /// Delete `body`.
+ Body,
+ /// Delete `secret`.
+ Secret,
+}
+
+#[api]
+#[derive(Serialize, Deserialize, Debug, Default, Clone)]
+/// Datatype used to represent key-value pairs, the value
+/// being encoded in base64.
+pub struct KeyAndBase64Val {
+ /// Name
+ pub name: String,
+ /// Base64 encoded value
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub value: Option<String>,
+}
+
+impl KeyAndBase64Val {
+ #[cfg(test)]
+ pub fn new_with_plain_value(name: &str, value: &str) -> Self {
+ let value = base64::encode(value);
+
+ Self {
+ name: name.into(),
+ value: Some(value),
+ }
+ }
+
+ /// Decode the contained value, returning the plaintext value
+ ///
+ /// Returns an error if the contained value is not valid base64-encoded
+ /// text.
+ pub fn decode_value(&self) -> Result<String, Error> {
+ let value = self.value.as_deref().unwrap_or_default();
+ let bytes = base64::decode(value).map_err(|_| {
+ Error::Generic(format!(
+ "could not decode base64 value with key '{}'",
+ self.name
+ ))
+ })?;
+ let value = String::from_utf8(bytes).map_err(|_| {
+ Error::Generic(format!(
+ "could not decode UTF8 string from base64, key '{}'",
+ self.name
+ ))
+ })?;
+
+ Ok(value)
+ }
+}
+
+pub const KEY_AND_BASE64_VALUE_SCHEMA: Schema =
+ StringSchema::new("String schema for pairs of keys and base64 encoded values")
+ .format(&ApiStringFormat::PropertyString(
+ &KeyAndBase64Val::API_SCHEMA,
+ ))
+ .schema();
+
+impl Endpoint for WebhookEndpoint {
+ /// Send a notification to a webhook endpoint.
+ fn send(&self, notification: &Notification) -> Result<(), Error> {
+ let request = self.build_request(notification)?;
+
+ self.create_client()?
+ .request(request)
+ .map_err(|err| self.mask_secret_in_error(err))?;
+
+ Ok(())
+ }
+
+ /// Return the name of the endpoint.
+ fn name(&self) -> &str {
+ &self.config.name
+ }
+
+ /// Check if the endpoint is disabled
+ fn disabled(&self) -> bool {
+ self.config.disable.unwrap_or_default()
+ }
+}
+
+impl WebhookEndpoint {
+ fn create_client(&self) -> Result<Client, Error> {
+ let proxy_config = context()
+ .http_proxy_config()
+ .map(|url| ProxyConfig::parse_proxy_url(&url))
+ .transpose()
+ .map_err(|err| Error::NotifyFailed(self.name().to_string(), err.into()))?;
+
+ let options = HttpOptions {
+ proxy_config,
+ ..Default::default()
+ };
+
+ Ok(Client::new(options))
+ }
+
+ fn build_request(&self, notification: &Notification) -> Result<Request<String>, Error> {
+ let (title, message) = match ¬ification.content {
+ Content::Template {
+ template_name,
+ data,
+ } => {
+ let rendered_title =
+ renderer::render_template(TemplateType::Subject, template_name, data)?;
+ let rendered_message =
+ renderer::render_template(TemplateType::PlaintextBody, template_name, data)?;
+
+ (rendered_title, rendered_message)
+ }
+ #[cfg(feature = "mail-forwarder")]
+ Content::ForwardedMail { title, body, .. } => (title.clone(), body.clone()),
+ };
+
+ let mut fields = Map::new();
+
+ for (field_name, field_value) in ¬ification.metadata.additional_fields {
+ fields.insert(field_name.clone(), Value::String(field_value.to_string()));
+ }
+
+ let mut secrets = Map::new();
+
+ for secret in &self.private_config.secret {
+ let value = secret.decode_value()?;
+ secrets.insert(secret.name.clone(), Value::String(value));
+ }
+
+ let data = json!({
+ "title": &title,
+ "message": &message,
+ "severity": notification.metadata.severity,
+ "timestamp": notification.metadata.timestamp,
+ "fields": fields,
+ "secrets": secrets,
+ });
+
+ let handlebars = setup_handlebars();
+ let body_template = self.base64_decode(self.config.body.as_deref().unwrap_or_default())?;
+
+ let body = handlebars
+ .render_template(&body_template, &data)
+ .map_err(|err| self.mask_secret_in_error(err))
+ .map_err(|err| Error::Generic(format!("failed to render webhook body: {err}")))?;
+
+ let url = handlebars
+ .render_template(&self.config.url, &data)
+ .map_err(|err| self.mask_secret_in_error(err))
+ .map_err(|err| Error::Generic(format!("failed to render webhook url: {err}")))?;
+
+ let method: &str = self.config.method.into();
+ let mut builder = http::Request::builder().uri(url).method(method);
+
+ for header in &self.config.header {
+ let value = header.decode_value()?;
+
+ let value = handlebars
+ .render_template(&value, &data)
+ .map_err(|err| self.mask_secret_in_error(err))
+ .map_err(|err| {
+ Error::Generic(format!(
+ "failed to render header value template: {value}: {err}"
+ ))
+ })?;
+
+ builder = builder.header(header.name.clone(), value);
+ }
+
+ let request = builder
+ .body(body)
+ .map_err(|err| self.mask_secret_in_error(err))
+ .map_err(|err| Error::Generic(format!("failed to build http request: {err}")))?;
+
+ Ok(request)
+ }
+
+ fn base64_decode(&self, s: &str) -> Result<String, Error> {
+ // Also here, TODO: revisit Error variants for the *whole* crate.
+ let s = base64::decode(s)
+ .map_err(|err| Error::Generic(format!("could not decode base64 value: {err}")))?;
+
+ String::from_utf8(s).map_err(|err| {
+ Error::Generic(format!(
+ "base64 encoded value did not contain valid utf8: {err}"
+ ))
+ })
+ }
+
+ /// Mask secrets in errors to avoid them showing up in error messages and log files
+ ///
+ /// Use this for any error from third-party code where you are not 100%
+ /// sure whether it could leak the content of secrets in the error.
+ /// For instance, the http client will contain the URL, including
+ /// any URL parameters that could contain tokens.
+ ///
+ /// This function will only mask exact matches, but this should suffice
+ /// for the majority of cases.
+ fn mask_secret_in_error(&self, error: impl std::fmt::Display) -> Error {
+ let mut s = error.to_string();
+
+ for secret_value in &self.private_config.secret {
+ match secret_value.decode_value() {
+ Ok(value) => s = s.replace(&value, "<masked>"),
+ Err(e) => return e,
+ }
+ }
+
+ Error::Generic(s)
+ }
+}
+
+fn setup_handlebars() -> Handlebars<'static> {
+ let mut handlebars = Handlebars::new();
+
+ handlebars.register_helper("url-encode", Box::new(handlebars_percent_encode));
+ handlebars.register_helper("json", Box::new(handlebars_json));
+ handlebars.register_helper("escape", Box::new(handlebars_escape));
+
+ // There is no escape.
+ handlebars.register_escape_fn(handlebars::no_escape);
+
+ handlebars
+}
+
+fn handlebars_percent_encode(
+ h: &Helper,
+ _: &Handlebars,
+ _: &HandlebarsContext,
+ _rc: &mut RenderContext,
+ out: &mut dyn Output,
+) -> HelperResult {
+ let param0 = h
+ .param(0)
+ .and_then(|v| v.value().as_str())
+ .ok_or_else(|| HandlebarsRenderError::new("url-encode: missing parameter"))?;
+
+ // See https://developer.mozilla.org/en-US/docs/Glossary/Percent-encoding
+ const FRAGMENT: &AsciiSet = &percent_encoding::CONTROLS
+ .add(b':')
+ .add(b'/')
+ .add(b'?')
+ .add(b'#')
+ .add(b'[')
+ .add(b']')
+ .add(b'@')
+ .add(b'!')
+ .add(b'$')
+ .add(b'&')
+ .add(b'\'')
+ .add(b'(')
+ .add(b')')
+ .add(b'*')
+ .add(b'+')
+ .add(b',')
+ .add(b';')
+ .add(b'=')
+ .add(b'%')
+ .add(b' ');
+ let a = percent_encoding::utf8_percent_encode(param0, FRAGMENT);
+
+ out.write(&a.to_string())?;
+
+ Ok(())
+}
+
+fn handlebars_json(
+ h: &Helper,
+ _: &Handlebars,
+ _: &HandlebarsContext,
+ _rc: &mut RenderContext,
+ out: &mut dyn Output,
+) -> HelperResult {
+ let param0 = h
+ .param(0)
+ .map(|v| v.value())
+ .ok_or_else(|| HandlebarsRenderError::new("json: missing parameter"))?;
+
+ let json = serde_json::to_string(param0)?;
+ out.write(&json)?;
+
+ Ok(())
+}
+
+fn handlebars_escape(
+ h: &Helper,
+ _: &Handlebars,
+ _: &HandlebarsContext,
+ _rc: &mut RenderContext,
+ out: &mut dyn Output,
+) -> HelperResult {
+ let text = h
+ .param(0)
+ .and_then(|v| v.value().as_str())
+ .ok_or_else(|| HandlebarsRenderError::new("escape: missing text parameter"))?;
+
+ let val = Value::String(text.to_string());
+ let json = serde_json::to_string(&val)?;
+ out.write(&json[1..json.len() - 1])?;
+
+ Ok(())
+}
+
+#[cfg(test)]
+mod tests {
+ use std::collections::HashMap;
+
+ use super::*;
+ use crate::Severity;
+
+ #[test]
+ fn test_build_request() -> Result<(), Error> {
+ let data = HashMap::from_iter([
+ ("hello".into(), "hello world".into()),
+ ("test".into(), "escaped\nstring".into()),
+ ]);
+
+ let body_template = r#"
+{{ fields.test }}
+{{ escape fields.test }}
+
+{{ json fields }}
+{{ json fields.hello }}
+
+{{ url-encode fields.hello }}
+
+{{ json severity }}
+
+"#;
+
+ let expected_body = r#"
+escaped
+string
+escaped\nstring
+
+{"hello":"hello world","test":"escaped\nstring"}
+"hello world"
+
+hello%20world
+
+"info"
+
+"#;
+
+ let endpoint = WebhookEndpoint {
+ config: WebhookConfig {
+ name: "test".into(),
+ method: HttpMethod::Post,
+ url: "http://localhost/{{ url-encode fields.hello }}".into(),
+ header: vec![
+ KeyAndBase64Val::new_with_plain_value("X-Severity", "{{ severity }}").into(),
+ ],
+ body: Some(base64::encode(body_template)),
+ ..Default::default()
+ },
+ private_config: WebhookPrivateConfig {
+ name: "test".into(),
+ ..Default::default()
+ },
+ };
+
+ let notification = Notification::from_template(Severity::Info, "foo", json!({}), data);
+
+ let request = endpoint.build_request(¬ification)?;
+
+ assert_eq!(request.uri(), "http://localhost/hello%20world");
+ assert_eq!(request.body(), expected_body);
+ assert_eq!(request.method(), "POST");
+
+ assert_eq!(request.headers().get("X-Severity").unwrap(), "info");
+
+ Ok(())
+ }
+}
diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
index 015d9b9c..12f3866b 100644
--- a/proxmox-notify/src/lib.rs
+++ b/proxmox-notify/src/lib.rs
@@ -500,6 +500,23 @@ impl Bus {
);
}
+ #[cfg(feature = "webhook")]
+ {
+ use endpoints::webhook::WEBHOOK_TYPENAME;
+ use endpoints::webhook::{WebhookConfig, WebhookEndpoint, WebhookPrivateConfig};
+ endpoints.extend(
+ parse_endpoints_with_private_config!(
+ config,
+ WebhookConfig,
+ WebhookPrivateConfig,
+ WebhookEndpoint,
+ WEBHOOK_TYPENAME
+ )?
+ .into_iter()
+ .map(|e| (e.name().into(), e)),
+ );
+ }
+
let matchers = config
.config
.convert_to_typed_array(MATCHER_TYPENAME)
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] [PATCH proxmox v3 03/14] notify: add api for webhook targets
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox v3 01/14] notify: renderer: adapt to changes in proxmox-time Lukas Wagner
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox v3 02/14] notify: implement webhook targets Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox-perl-rs v3 04/14] common: notify: add bindings for webhook API routes Lukas Wagner
` (11 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
All in all pretty similar to other endpoint APIs.
One thing worth noting is how secrets are handled. We never ever
return the values of previously stored secrets in get_endpoint(s)
calls, but only a list of the names of all secrets. This is needed
to build the UI, where we display all secrets that were set before in
a table.
For update calls, one is supposed to send all secrets that should be
kept and updated. If the value should be updated, the name and value
is expected, and if the current value should preseved, only the name
is sent. If a secret's name is not present in the updater, it will be
dropped. If 'secret' is present in the 'delete' array, all secrets
will be dropped, apart from those which are also set/preserved in the
same update call.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
---
proxmox-notify/src/api/mod.rs | 20 ++
proxmox-notify/src/api/webhook.rs | 432 ++++++++++++++++++++++++++++++
2 files changed, 452 insertions(+)
create mode 100644 proxmox-notify/src/api/webhook.rs
diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
index a7f6261c..7f823bc7 100644
--- a/proxmox-notify/src/api/mod.rs
+++ b/proxmox-notify/src/api/mod.rs
@@ -15,6 +15,8 @@ pub mod matcher;
pub mod sendmail;
#[cfg(feature = "smtp")]
pub mod smtp;
+#[cfg(feature = "webhook")]
+pub mod webhook;
// We have our own, local versions of http_err and http_bail, because
// we don't want to wrap the error in anyhow::Error. If we were to do that,
@@ -54,6 +56,9 @@ pub enum EndpointType {
/// Gotify endpoint
#[cfg(feature = "gotify")]
Gotify,
+ /// Webhook endpoint
+ #[cfg(feature = "webhook")]
+ Webhook,
}
#[api]
@@ -113,6 +118,17 @@ pub fn get_targets(config: &Config) -> Result<Vec<Target>, HttpError> {
})
}
+ #[cfg(feature = "webhook")]
+ for endpoint in webhook::get_endpoints(config)? {
+ targets.push(Target {
+ name: endpoint.name,
+ origin: endpoint.origin.unwrap_or(Origin::UserCreated),
+ endpoint_type: EndpointType::Webhook,
+ disable: endpoint.disable,
+ comment: endpoint.comment,
+ })
+ }
+
Ok(targets)
}
@@ -145,6 +161,10 @@ fn ensure_endpoint_exists(#[allow(unused)] config: &Config, name: &str) -> Resul
{
exists = exists || smtp::get_endpoint(config, name).is_ok();
}
+ #[cfg(feature = "webhook")]
+ {
+ exists = exists || webhook::get_endpoint(config, name).is_ok();
+ }
if !exists {
http_bail!(NOT_FOUND, "endpoint '{name}' does not exist")
diff --git a/proxmox-notify/src/api/webhook.rs b/proxmox-notify/src/api/webhook.rs
new file mode 100644
index 00000000..f786c36b
--- /dev/null
+++ b/proxmox-notify/src/api/webhook.rs
@@ -0,0 +1,432 @@
+//! CRUD API for webhook targets.
+//!
+//! All methods assume that the caller has already done any required permission checks.
+
+use proxmox_http_error::HttpError;
+use proxmox_schema::property_string::PropertyString;
+
+use crate::api::http_err;
+use crate::endpoints::webhook::{
+ DeleteableWebhookProperty, KeyAndBase64Val, WebhookConfig, WebhookConfigUpdater,
+ WebhookPrivateConfig, WEBHOOK_TYPENAME,
+};
+use crate::{http_bail, Config};
+
+use super::remove_private_config_entry;
+use super::set_private_config_entry;
+
+/// Get a list of all webhook endpoints.
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns a list of all webhook endpoints or a [`HttpError`] if the config is
+/// erroneous (`500 Internal server error`).
+pub fn get_endpoints(config: &Config) -> Result<Vec<WebhookConfig>, HttpError> {
+ let mut endpoints: Vec<WebhookConfig> = config
+ .config
+ .convert_to_typed_array(WEBHOOK_TYPENAME)
+ .map_err(|e| http_err!(NOT_FOUND, "Could not fetch endpoints: {e}"))?;
+
+ for endpoint in &mut endpoints {
+ let priv_config: WebhookPrivateConfig = config
+ .private_config
+ .lookup(WEBHOOK_TYPENAME, &endpoint.name)
+ .unwrap_or_default();
+
+ let mut secret_names = Vec::new();
+ // We only return *which* secrets we have stored, but not their values.
+ for secret in priv_config.secret {
+ secret_names.push(
+ KeyAndBase64Val {
+ name: secret.name.clone(),
+ value: None,
+ }
+ .into(),
+ )
+ }
+
+ endpoint.secret = secret_names;
+ }
+
+ Ok(endpoints)
+}
+
+/// Get webhook endpoint with given `name`
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns the endpoint or a [`HttpError`] if the endpoint was not found (`404 Not found`).
+pub fn get_endpoint(config: &Config, name: &str) -> Result<WebhookConfig, HttpError> {
+ let mut endpoint: WebhookConfig = config
+ .config
+ .lookup(WEBHOOK_TYPENAME, name)
+ .map_err(|_| http_err!(NOT_FOUND, "endpoint '{name}' not found"))?;
+
+ let priv_config: Option<WebhookPrivateConfig> = config
+ .private_config
+ .lookup(WEBHOOK_TYPENAME, &endpoint.name)
+ .ok();
+
+ let mut secret_names = Vec::new();
+ if let Some(priv_config) = priv_config {
+ for secret in &priv_config.secret {
+ secret_names.push(
+ KeyAndBase64Val {
+ name: secret.name.clone(),
+ value: None,
+ }
+ .into(),
+ );
+ }
+ }
+
+ endpoint.secret = secret_names;
+
+ Ok(endpoint)
+}
+
+/// Add a new webhook endpoint.
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns a [`HttpError`] if:
+/// - the target name is already used (`400 Bad request`)
+/// - an entity with the same name already exists (`400 Bad request`)
+/// - the configuration could not be saved (`500 Internal server error`)
+pub fn add_endpoint(
+ config: &mut Config,
+ mut endpoint_config: WebhookConfig,
+) -> Result<(), HttpError> {
+ super::ensure_unique(config, &endpoint_config.name)?;
+
+ let secrets = std::mem::take(&mut endpoint_config.secret);
+
+ set_private_config_entry(
+ config,
+ &WebhookPrivateConfig {
+ name: endpoint_config.name.clone(),
+ secret: secrets,
+ },
+ WEBHOOK_TYPENAME,
+ &endpoint_config.name,
+ )?;
+
+ config
+ .config
+ .set_data(&endpoint_config.name, WEBHOOK_TYPENAME, &endpoint_config)
+ .map_err(|e| {
+ http_err!(
+ INTERNAL_SERVER_ERROR,
+ "could not save endpoint '{}': {e}",
+ endpoint_config.name
+ )
+ })
+}
+
+/// Update existing webhook endpoint.
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns a `HttpError` if:
+/// - the passed `digest` does not match (`400 Bad request`)
+/// - parameters are ill-formed (empty header value, invalid base64, unknown header/secret)
+/// (`400 Bad request`)
+/// - an entity with the same name already exists (`400 Bad request`)
+/// - the configuration could not be saved (`500 Internal server error`)
+pub fn update_endpoint(
+ config: &mut Config,
+ name: &str,
+ config_updater: WebhookConfigUpdater,
+ delete: Option<&[DeleteableWebhookProperty]>,
+ digest: Option<&[u8]>,
+) -> Result<(), HttpError> {
+ super::verify_digest(config, digest)?;
+
+ let mut endpoint = get_endpoint(config, name)?;
+ endpoint.secret.clear();
+
+ let old_secrets = config
+ .private_config
+ .lookup::<WebhookPrivateConfig>(WEBHOOK_TYPENAME, name)
+ .map_err(|err| http_err!(INTERNAL_SERVER_ERROR, "could not read secret config: {err}"))?
+ .secret;
+
+ if let Some(delete) = delete {
+ for deleteable_property in delete {
+ match deleteable_property {
+ DeleteableWebhookProperty::Comment => endpoint.comment = None,
+ DeleteableWebhookProperty::Disable => endpoint.disable = None,
+ DeleteableWebhookProperty::Header => endpoint.header = Vec::new(),
+ DeleteableWebhookProperty::Body => endpoint.body = None,
+ DeleteableWebhookProperty::Secret => {
+ set_private_config_entry(
+ config,
+ &WebhookPrivateConfig {
+ name: name.into(),
+ secret: Vec::new(),
+ },
+ WEBHOOK_TYPENAME,
+ name,
+ )?;
+ }
+ }
+ }
+ }
+
+ // Destructuring makes sure we don't forget any members
+ let WebhookConfigUpdater {
+ url,
+ body,
+ header,
+ method,
+ disable,
+ comment,
+ secret,
+ } = config_updater;
+
+ if let Some(url) = url {
+ endpoint.url = url;
+ }
+
+ if let Some(body) = body {
+ endpoint.body = Some(body);
+ }
+
+ if let Some(header) = header {
+ for h in &header {
+ if h.value.is_none() {
+ http_bail!(BAD_REQUEST, "header '{}' has empty value", h.name);
+ }
+ if h.decode_value().is_err() {
+ http_bail!(
+ BAD_REQUEST,
+ "header '{}' does not have valid base64 encoded data",
+ h.name
+ )
+ }
+ }
+ endpoint.header = header;
+ }
+
+ if let Some(method) = method {
+ endpoint.method = method;
+ }
+
+ if let Some(disable) = disable {
+ endpoint.disable = Some(disable);
+ }
+
+ if let Some(comment) = comment {
+ endpoint.comment = Some(comment);
+ }
+
+ if let Some(secret) = secret {
+ let mut new_secrets: Vec<PropertyString<KeyAndBase64Val>> = Vec::new();
+
+ for new_secret in &secret {
+ let sec = if new_secret.value.is_some() {
+ // Updating or creating a secret
+
+ // Make sure it is valid base64 encoded data
+ if new_secret.decode_value().is_err() {
+ http_bail!(
+ BAD_REQUEST,
+ "secret '{}' does not have valid base64 encoded data",
+ new_secret.name
+ )
+ }
+ new_secret.clone()
+ } else if let Some(old_secret) = old_secrets.iter().find(|v| v.name == new_secret.name)
+ {
+ // Keeping an already existing secret
+ old_secret.clone()
+ } else {
+ http_bail!(BAD_REQUEST, "secret '{}' not known", new_secret.name);
+ };
+
+ if new_secrets.iter().any(|s| sec.name == s.name) {
+ http_bail!(BAD_REQUEST, "secret '{}' defined multiple times", sec.name)
+ }
+
+ new_secrets.push(sec);
+ }
+
+ set_private_config_entry(
+ config,
+ &WebhookPrivateConfig {
+ name: name.into(),
+ secret: new_secrets,
+ },
+ WEBHOOK_TYPENAME,
+ name,
+ )?;
+ }
+
+ config
+ .config
+ .set_data(name, WEBHOOK_TYPENAME, &endpoint)
+ .map_err(|e| {
+ http_err!(
+ INTERNAL_SERVER_ERROR,
+ "could not save endpoint '{name}': {e}"
+ )
+ })
+}
+
+/// Delete existing webhook endpoint.
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns a `HttpError` if:
+/// - the entity does not exist (`404 Not found`)
+/// - the endpoint is still referenced by another entity (`400 Bad request`)
+pub fn delete_endpoint(config: &mut Config, name: &str) -> Result<(), HttpError> {
+ // Check if the endpoint exists
+ let _ = get_endpoint(config, name)?;
+ super::ensure_safe_to_delete(config, name)?;
+
+ remove_private_config_entry(config, name)?;
+ config.config.sections.remove(name);
+
+ Ok(())
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use crate::{api::test_helpers::empty_config, endpoints::webhook::HttpMethod};
+
+ use base64::encode;
+
+ pub fn add_default_webhook_endpoint(config: &mut Config) -> Result<(), HttpError> {
+ add_endpoint(
+ config,
+ WebhookConfig {
+ name: "webhook-endpoint".into(),
+ method: HttpMethod::Post,
+ url: "http://example.com/webhook".into(),
+ header: vec![KeyAndBase64Val::new_with_plain_value(
+ "Content-Type",
+ "application/json",
+ )
+ .into()],
+ body: Some(encode("this is the body")),
+ comment: Some("comment".into()),
+ disable: Some(false),
+ secret: vec![KeyAndBase64Val::new_with_plain_value("token", "secret").into()],
+ ..Default::default()
+ },
+ )?;
+
+ assert!(get_endpoint(config, "webhook-endpoint").is_ok());
+ Ok(())
+ }
+
+ #[test]
+ fn test_update_not_existing_returns_error() -> Result<(), HttpError> {
+ let mut config = empty_config();
+
+ assert!(update_endpoint(&mut config, "test", Default::default(), None, None).is_err());
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_update_invalid_digest_returns_error() -> Result<(), HttpError> {
+ let mut config = empty_config();
+ add_default_webhook_endpoint(&mut config)?;
+
+ assert!(update_endpoint(
+ &mut config,
+ "webhook-endpoint",
+ Default::default(),
+ None,
+ Some(&[0; 32])
+ )
+ .is_err());
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_update() -> Result<(), HttpError> {
+ let mut config = empty_config();
+ add_default_webhook_endpoint(&mut config)?;
+
+ let digest = config.digest;
+
+ update_endpoint(
+ &mut config,
+ "webhook-endpoint",
+ WebhookConfigUpdater {
+ url: Some("http://new.example.com/webhook".into()),
+ comment: Some("newcomment".into()),
+ method: Some(HttpMethod::Put),
+ // Keep the old token and set a new one
+ secret: Some(vec![
+ KeyAndBase64Val::new_with_plain_value("token2", "newsecret").into(),
+ KeyAndBase64Val {
+ name: "token".into(),
+ value: None,
+ }
+ .into(),
+ ]),
+ ..Default::default()
+ },
+ None,
+ Some(&digest),
+ )?;
+
+ let endpoint = get_endpoint(&config, "webhook-endpoint")?;
+
+ assert_eq!(endpoint.url, "http://new.example.com/webhook".to_string());
+ assert_eq!(endpoint.comment, Some("newcomment".to_string()));
+ assert!(matches!(endpoint.method, HttpMethod::Put));
+
+ let secrets = config
+ .private_config
+ .lookup::<WebhookPrivateConfig>(WEBHOOK_TYPENAME, "webhook-endpoint")
+ .unwrap()
+ .secret;
+
+ assert_eq!(secrets[1].name, "token".to_string());
+ assert_eq!(secrets[1].value, Some(encode("secret")));
+ assert_eq!(secrets[0].name, "token2".to_string());
+ assert_eq!(secrets[0].value, Some(encode("newsecret")));
+
+ // Test property deletion
+ update_endpoint(
+ &mut config,
+ "webhook-endpoint",
+ Default::default(),
+ Some(&[
+ DeleteableWebhookProperty::Comment,
+ DeleteableWebhookProperty::Secret,
+ ]),
+ None,
+ )?;
+
+ let endpoint = get_endpoint(&config, "webhook-endpoint")?;
+ assert_eq!(endpoint.comment, None);
+
+ let secrets = config
+ .private_config
+ .lookup::<WebhookPrivateConfig>(WEBHOOK_TYPENAME, "webhook-endpoint")
+ .unwrap()
+ .secret;
+
+ assert!(secrets.is_empty());
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_delete() -> Result<(), HttpError> {
+ let mut config = empty_config();
+ add_default_webhook_endpoint(&mut config)?;
+
+ delete_endpoint(&mut config, "webhook-endpoint")?;
+ assert!(delete_endpoint(&mut config, "webhook-endpoint").is_err());
+ assert_eq!(get_endpoints(&config)?.len(), 0);
+
+ Ok(())
+ }
+}
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] [PATCH proxmox-perl-rs v3 04/14] common: notify: add bindings for webhook API routes
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
` (2 preceding siblings ...)
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox v3 03/14] notify: add api for " Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox-perl-rs v3 05/14] common: notify: add bindings for get_targets Lukas Wagner
` (10 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
---
common/src/notify.rs | 63 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 63 insertions(+)
diff --git a/common/src/notify.rs b/common/src/notify.rs
index e1b006b..fe192d5 100644
--- a/common/src/notify.rs
+++ b/common/src/notify.rs
@@ -19,6 +19,9 @@ mod export {
DeleteableSmtpProperty, SmtpConfig, SmtpConfigUpdater, SmtpMode, SmtpPrivateConfig,
SmtpPrivateConfigUpdater,
};
+ use proxmox_notify::endpoints::webhook::{
+ DeleteableWebhookProperty, WebhookConfig, WebhookConfigUpdater,
+ };
use proxmox_notify::matcher::{
CalendarMatcher, DeleteableMatcherProperty, FieldMatcher, MatchModeOperator, MatcherConfig,
MatcherConfigUpdater, SeverityMatcher,
@@ -393,6 +396,66 @@ mod export {
api::smtp::delete_endpoint(&mut config, name)
}
+ #[export(serialize_error)]
+ fn get_webhook_endpoints(
+ #[try_from_ref] this: &NotificationConfig,
+ ) -> Result<Vec<WebhookConfig>, HttpError> {
+ let config = this.config.lock().unwrap();
+ api::webhook::get_endpoints(&config)
+ }
+
+ #[export(serialize_error)]
+ fn get_webhook_endpoint(
+ #[try_from_ref] this: &NotificationConfig,
+ id: &str,
+ ) -> Result<WebhookConfig, HttpError> {
+ let config = this.config.lock().unwrap();
+ api::webhook::get_endpoint(&config, id)
+ }
+
+ #[export(serialize_error)]
+ #[allow(clippy::too_many_arguments)]
+ fn add_webhook_endpoint(
+ #[try_from_ref] this: &NotificationConfig,
+ endpoint_config: WebhookConfig,
+ ) -> Result<(), HttpError> {
+ let mut config = this.config.lock().unwrap();
+ api::webhook::add_endpoint(
+ &mut config,
+ endpoint_config,
+ )
+ }
+
+ #[export(serialize_error)]
+ #[allow(clippy::too_many_arguments)]
+ fn update_webhook_endpoint(
+ #[try_from_ref] this: &NotificationConfig,
+ name: &str,
+ config_updater: WebhookConfigUpdater,
+ delete: Option<Vec<DeleteableWebhookProperty>>,
+ digest: Option<&str>,
+ ) -> Result<(), HttpError> {
+ let mut config = this.config.lock().unwrap();
+ let digest = decode_digest(digest)?;
+
+ api::webhook::update_endpoint(
+ &mut config,
+ name,
+ config_updater,
+ delete.as_deref(),
+ digest.as_deref(),
+ )
+ }
+
+ #[export(serialize_error)]
+ fn delete_webhook_endpoint(
+ #[try_from_ref] this: &NotificationConfig,
+ name: &str,
+ ) -> Result<(), HttpError> {
+ let mut config = this.config.lock().unwrap();
+ api::webhook::delete_endpoint(&mut config, name)
+ }
+
#[export(serialize_error)]
fn get_matchers(
#[try_from_ref] this: &NotificationConfig,
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] [PATCH proxmox-perl-rs v3 05/14] common: notify: add bindings for get_targets
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
` (3 preceding siblings ...)
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox-perl-rs v3 04/14] common: notify: add bindings for webhook API routes Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-08 14:41 ` [pbs-devel] [PATCH widget-toolkit v3 06/14] utils: add base64 conversion helper Lukas Wagner
` (9 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
This allows us to drop the impl of that function on the perl side.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
---
common/src/notify.rs | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/common/src/notify.rs b/common/src/notify.rs
index fe192d5..0f8a35d 100644
--- a/common/src/notify.rs
+++ b/common/src/notify.rs
@@ -27,6 +27,7 @@ mod export {
MatcherConfigUpdater, SeverityMatcher,
};
use proxmox_notify::{api, Config, Notification, Severity};
+ use proxmox_notify::api::Target;
pub struct NotificationConfig {
config: Mutex<Config>,
@@ -112,6 +113,14 @@ mod export {
api::common::send(&config, ¬ification)
}
+ #[export(serialize_error)]
+ fn get_targets(
+ #[try_from_ref] this: &NotificationConfig,
+ ) -> Result<Vec<Target>, HttpError> {
+ let config = this.config.lock().unwrap();
+ api::get_targets(&config)
+ }
+
#[export(serialize_error)]
fn test_target(
#[try_from_ref] this: &NotificationConfig,
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] [PATCH widget-toolkit v3 06/14] utils: add base64 conversion helper
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
` (4 preceding siblings ...)
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox-perl-rs v3 05/14] common: notify: add bindings for get_targets Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-10 17:27 ` [pbs-devel] applied: [pve-devel] " Thomas Lamprecht
2024-11-08 14:41 ` [pbs-devel] [PATCH widget-toolkit v3 07/14] notification: add UI for adding/updating webhook targets Lukas Wagner
` (8 subsequent siblings)
14 siblings, 1 reply; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
From: Gabriel Goller <g.goller@proxmox.com>
Add helper functions to convert from a utf8 string to a base64 string
and vice-versa. Using the TextEncoder/TextDecoder we can support unicode
such as emojis as well [0].
[0]: https://developer.mozilla.org/en-US/docs/Glossary/Base64#the_unicode_problem
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/Utils.js | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/src/Utils.js b/src/Utils.js
index b68c0f4..4ff95af 100644
--- a/src/Utils.js
+++ b/src/Utils.js
@@ -1356,6 +1356,24 @@ utilities: {
);
},
+ // Convert utf-8 string to base64.
+ // This also escapes unicode characters such as emojis.
+ utf8ToBase64: function(string) {
+ let bytes = new TextEncoder().encode(string);
+ const escapedString = Array.from(bytes, (byte) =>
+ String.fromCodePoint(byte),
+ ).join("");
+ return btoa(escapedString);
+ },
+
+ // Converts a base64 string into a utf8 string.
+ // Decodes escaped unicode characters correctly.
+ base64ToUtf8: function(b64_string) {
+ let string = atob(b64_string);
+ let bytes = Uint8Array.from(string, (m) => m.codePointAt(0));
+ return new TextDecoder().decode(bytes);
+ },
+
stringToRGB: function(string) {
let hash = 0;
if (!string) {
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] [PATCH widget-toolkit v3 07/14] notification: add UI for adding/updating webhook targets
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
` (5 preceding siblings ...)
2024-11-08 14:41 ` [pbs-devel] [PATCH widget-toolkit v3 06/14] utils: add base64 conversion helper Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-10 17:27 ` [pbs-devel] applied: [pve-devel] " Thomas Lamprecht
2024-11-11 22:09 ` [pbs-devel] " Thomas Lamprecht
2024-11-08 14:41 ` [pbs-devel] [PATCH manager v3 08/14] api: notifications: use get_targets impl from proxmox-notify Lukas Wagner
` (7 subsequent siblings)
14 siblings, 2 replies; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
The widgets for editing the headers/secrets were adapted from
the 'Tag Edit' dialog from PVE's datacenter options.
Apart from that, the new dialog is rather standard. I've decided
to put the http method and url in a single row, mostly to
save space and also to make it analogous to how an actual http request
is structured (VERB URL, followed by headers, followed by the body).
The secrets are a mechanism to store tokens/passwords in the
protected notification config. Secrets are accessible via
templating in the URL, headers and body via {{ secrets.NAME }}.
Secrets can only be set/updated, but not retrieved/displayed.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
---
src/Makefile | 1 +
src/Schema.js | 5 +
src/Utils.js | 20 ++
src/panel/WebhookEditPanel.js | 424 ++++++++++++++++++++++++++++++++++
4 files changed, 450 insertions(+)
create mode 100644 src/panel/WebhookEditPanel.js
diff --git a/src/Makefile b/src/Makefile
index 0478251..cfaffd7 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -78,6 +78,7 @@ JSSRC= \
panel/StatusView.js \
panel/TfaView.js \
panel/NotesView.js \
+ panel/WebhookEditPanel.js \
window/Edit.js \
window/PasswordEdit.js \
window/SafeDestroy.js \
diff --git a/src/Schema.js b/src/Schema.js
index 42541e0..cd1c306 100644
--- a/src/Schema.js
+++ b/src/Schema.js
@@ -65,6 +65,11 @@ Ext.define('Proxmox.Schema', { // a singleton
ipanel: 'pmxGotifyEditPanel',
iconCls: 'fa-bell-o',
},
+ webhook: {
+ name: 'Webhook',
+ ipanel: 'pmxWebhookEditPanel',
+ iconCls: 'fa-bell-o',
+ },
},
// to add or change existing for product specific ones
diff --git a/src/Utils.js b/src/Utils.js
index 4ff95af..52375d2 100644
--- a/src/Utils.js
+++ b/src/Utils.js
@@ -1524,6 +1524,26 @@ utilities: {
me.IP6_dotnotation_match = new RegExp("^(" + IPV6_REGEXP + ")(?:\\.(\\d+))?$");
me.Vlan_match = /^vlan(\d+)/;
me.VlanInterface_match = /(\w+)\.(\d+)/;
+
+
+ // Taken from proxmox-schema and ported to JS
+ let PORT_REGEX_STR = "(?:[0-9]{1,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])";
+ let IPRE_BRACKET_STR = "(?:" + IPV4_REGEXP + "|\\[(?:" + IPV6_REGEXP + ")\\])";
+ let DNS_NAME_STR = "(?:(?:" + DnsName_REGEXP + "\\.)*" + DnsName_REGEXP + ")";
+ let HTTP_URL_REGEX = "^https?://(?:(?:(?:"
+ + DNS_NAME_STR
+ + "|"
+ + IPRE_BRACKET_STR
+ + ")(?::"
+ + PORT_REGEX_STR
+ + ")?)|"
+ + IPV6_REGEXP
+ + ")(?:/[^\x00-\x1F\x7F]*)?$";
+
+ me.httpUrlRegex = new RegExp(HTTP_URL_REGEX);
+
+ // Same as SAFE_ID_REGEX in proxmox-schema
+ me.safeIdRegex = /^(?:[A-Za-z0-9_][A-Za-z0-9._\\-]*)$/;
},
});
diff --git a/src/panel/WebhookEditPanel.js b/src/panel/WebhookEditPanel.js
new file mode 100644
index 0000000..0a39f3c
--- /dev/null
+++ b/src/panel/WebhookEditPanel.js
@@ -0,0 +1,424 @@
+Ext.define('Proxmox.panel.WebhookEditPanel', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pmxWebhookEditPanel',
+ mixins: ['Proxmox.Mixin.CBind'],
+ onlineHelp: 'notification_targets_webhook',
+
+ type: 'webhook',
+
+ columnT: [
+
+ ],
+
+ column1: [
+ {
+ xtype: 'pmxDisplayEditField',
+ name: 'name',
+ cbind: {
+ value: '{name}',
+ editable: '{isCreate}',
+ },
+ fieldLabel: gettext('Endpoint Name'),
+ regex: Proxmox.Utils.safeIdRegex,
+ allowBlank: false,
+ },
+ ],
+
+ column2: [
+ {
+ xtype: 'proxmoxcheckbox',
+ name: 'enable',
+ fieldLabel: gettext('Enable'),
+ allowBlank: false,
+ checked: true,
+ },
+ ],
+
+ columnB: [
+ {
+ layout: 'hbox',
+ border: false,
+ margin: '0 0 5 0',
+ items: [
+ {
+ xtype: 'displayfield',
+ value: gettext('Method/URL:'),
+ width: 125,
+ },
+ {
+ xtype: 'proxmoxKVComboBox',
+ name: 'method',
+ editable: false,
+ value: 'post',
+ comboItems: [
+ ['post', 'POST'],
+ ['put', 'PUT'],
+ ['get', 'GET'],
+ ],
+ width: 80,
+ margin: '0 5 0 0',
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'url',
+ allowBlank: false,
+ emptyText: "https://example.com/hook",
+ regex: Proxmox.Utils.httpUrlRegex,
+ regexText: gettext('Must be a valid URL'),
+ flex: 4,
+ },
+ ],
+ },
+ {
+ xtype: 'pmxWebhookKeyValueList',
+ name: 'header',
+ fieldLabel: gettext('Headers'),
+ maskValues: false,
+ cbind: {
+ isCreate: '{isCreate}',
+ },
+ },
+ {
+ xtype: 'textarea',
+ fieldLabel: gettext('Body'),
+ name: 'body',
+ allowBlank: true,
+ minHeight: '150',
+ fieldStyle: {
+ 'font-family': 'monospace',
+ },
+ margin: '15 0 0 0',
+ },
+ {
+ xtype: 'pmxWebhookKeyValueList',
+ name: 'secret',
+ fieldLabel: gettext('Secrets'),
+ maskValues: true,
+ cbind: {
+ isCreate: '{isCreate}',
+ },
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'comment',
+ fieldLabel: gettext('Comment'),
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
+ ],
+
+ onSetValues: (values) => {
+ values.enable = !values.disable;
+
+ if (values.body) {
+ values.body = Proxmox.Utils.base64ToUtf8(values.body);
+ }
+
+ delete values.disable;
+ return values;
+ },
+
+ onGetValues: function(values) {
+ let me = this;
+
+ if (values.enable) {
+ if (!me.isCreate) {
+ Proxmox.Utils.assemble_field_data(values, { 'delete': 'disable' });
+ }
+ } else {
+ values.disable = 1;
+ }
+
+ if (values.body) {
+ values.body = Proxmox.Utils.utf8ToBase64(values.body);
+ } else {
+ delete values.body;
+ if (!me.isCreate) {
+ Proxmox.Utils.assemble_field_data(values, { 'delete': 'body' });
+ }
+ }
+
+ if (Ext.isArray(values.header) && !values.header.length) {
+ delete values.header;
+ if (!me.isCreate) {
+ Proxmox.Utils.assemble_field_data(values, { 'delete': 'header' });
+ }
+ }
+
+ if (Ext.isArray(values.secret) && !values.secret.length) {
+ delete values.secret;
+ if (!me.isCreate) {
+ Proxmox.Utils.assemble_field_data(values, { 'delete': 'secret' });
+ }
+ }
+ delete values.enable;
+
+ return values;
+ },
+});
+
+Ext.define('Proxmox.form.WebhookKeyValueList', {
+ extend: 'Ext.container.Container',
+ alias: 'widget.pmxWebhookKeyValueList',
+
+ mixins: [
+ 'Ext.form.field.Field',
+ ],
+
+ // override for column header
+ fieldTitle: gettext('Item'),
+
+ // will be applied to the textfields
+ maskRe: undefined,
+
+ allowBlank: true,
+ selectAll: false,
+ isFormField: true,
+ deleteEmpty: false,
+ config: {
+ deleteEmpty: false,
+ maskValues: false,
+ },
+
+ setValue: function(list) {
+ let me = this;
+
+ list = Ext.isArray(list) ? list : (list ?? '').split(';').filter(t => t !== '');
+
+ let store = me.lookup('grid').getStore();
+ if (list.length > 0) {
+ store.setData(list.map(item => {
+ let properties = Proxmox.Utils.parsePropertyString(item);
+
+ // decode base64
+ let value = me.maskValues ? '' : Proxmox.Utils.base64ToUtf8(properties.value);
+
+ let obj = {
+ headerName: properties.name,
+ headerValue: value,
+ };
+
+ if (!me.isCreate && me.maskValues) {
+ obj.emptyText = gettext('Unchanged');
+ }
+
+ return obj;
+ }));
+ } else {
+ store.removeAll();
+ }
+ me.checkChange();
+ return me;
+ },
+
+ getValue: function() {
+ let me = this;
+ let values = [];
+ me.lookup('grid').getStore().each((rec) => {
+ if (rec.data.headerName) {
+ let obj = {
+ name: rec.data.headerName,
+ value: Proxmox.Utils.utf8ToBase64(rec.data.headerValue),
+ };
+
+ values.push(Proxmox.Utils.printPropertyString(obj));
+ }
+ });
+
+ return values;
+ },
+
+ getErrors: function(value) {
+ let me = this;
+ let empty = false;
+
+ me.lookup('grid').getStore().each((rec) => {
+ if (!rec.data.headerName) {
+ empty = true;
+ }
+
+ if (!rec.data.headerValue && rec.data.newValue) {
+ empty = true;
+ }
+
+ if (!rec.data.headerValue && !me.maskValues) {
+ empty = true;
+ }
+ });
+ if (empty) {
+ return [gettext('Name/value must not be empty.')];
+ }
+ return [];
+ },
+
+ // override framework function to implement deleteEmpty behaviour
+ getSubmitData: function() {
+ let me = this,
+ data = null,
+ val;
+ if (!me.disabled && me.submitValue) {
+ val = me.getValue();
+ if (val !== null && val !== '') {
+ data = {};
+ data[me.getName()] = val;
+ } else if (me.getDeleteEmpty()) {
+ data = {};
+ data.delete = me.getName();
+ }
+ }
+ return data;
+ },
+
+ controller: {
+ xclass: 'Ext.app.ViewController',
+
+ addLine: function() {
+ let me = this;
+ me.lookup('grid').getStore().add({
+ headerName: '',
+ headerValue: '',
+ emptyText: '',
+ newValue: true,
+ });
+ },
+
+ removeSelection: function(field) {
+ let me = this;
+ let view = me.getView();
+ let grid = me.lookup('grid');
+
+ let record = field.getWidgetRecord();
+ if (record === undefined) {
+ // this is sometimes called before a record/column is initialized
+ return;
+ }
+
+ grid.getStore().remove(record);
+ view.checkChange();
+ view.validate();
+ },
+
+ itemChange: function(field, newValue) {
+ let rec = field.getWidgetRecord();
+ if (!rec) {
+ return;
+ }
+
+ let column = field.getWidgetColumn();
+ rec.set(column.dataIndex, newValue);
+ let list = field.up('pmxWebhookKeyValueList');
+ list.checkChange();
+ list.validate();
+ },
+
+ control: {
+ 'grid button': {
+ click: 'removeSelection',
+ },
+ },
+ },
+
+ margin: '10 0 5 0',
+
+ items: [
+ {
+ layout: 'hbox',
+ border: false,
+ items: [
+ {
+ xtype: 'displayfield',
+ width: 125,
+ },
+ {
+ xtype: 'button',
+ text: gettext('Add'),
+ iconCls: 'fa fa-plus-circle',
+ handler: 'addLine',
+ margin: '0 5 5 0',
+ },
+ ],
+ },
+ {
+ xtype: 'grid',
+ reference: 'grid',
+ minHeight: 100,
+ maxHeight: 100,
+ scrollable: 'vertical',
+ margin: '0 0 0 125',
+
+ viewConfig: {
+ deferEmptyText: false,
+ },
+
+ store: {
+ listeners: {
+ update: function() {
+ this.commitChanges();
+ },
+ },
+ },
+ },
+ ],
+
+ initComponent: function() {
+ let me = this;
+
+ for (const [key, value] of Object.entries(me.gridConfig ?? {})) {
+ me.items[1][key] = value;
+ }
+
+ me.items[0].items[0].value = me.fieldLabel + ':';
+
+ me.items[1].columns = [
+ {
+ header: me.fieldTtitle,
+ dataIndex: 'headerName',
+ xtype: 'widgetcolumn',
+ widget: {
+ xtype: 'textfield',
+ isFormField: false,
+ maskRe: me.maskRe,
+ allowBlank: false,
+ queryMode: 'local',
+ listeners: {
+ change: 'itemChange',
+ },
+ },
+ flex: 1,
+ },
+ {
+ header: me.fieldTtitle,
+ dataIndex: 'headerValue',
+ xtype: 'widgetcolumn',
+ widget: {
+ xtype: 'proxmoxtextfield',
+ inputType: me.maskValues ? 'password' : 'text',
+ isFormField: false,
+ maskRe: me.maskRe,
+ queryMode: 'local',
+ listeners: {
+ change: 'itemChange',
+ },
+ allowBlank: !me.isCreate && me.maskValues,
+
+ bind: {
+ emptyText: '{record.emptyText}',
+ },
+ },
+ flex: 1,
+ },
+ {
+ xtype: 'widgetcolumn',
+ width: 40,
+ widget: {
+ xtype: 'button',
+ iconCls: 'fa fa-trash-o',
+ },
+ },
+ ];
+
+ me.callParent();
+ me.initField();
+ },
+});
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] applied: [pve-devel] [PATCH widget-toolkit v3 07/14] notification: add UI for adding/updating webhook targets
2024-11-08 14:41 ` [pbs-devel] [PATCH widget-toolkit v3 07/14] notification: add UI for adding/updating webhook targets Lukas Wagner
@ 2024-11-10 17:27 ` Thomas Lamprecht
2024-11-11 22:09 ` [pbs-devel] " Thomas Lamprecht
1 sibling, 0 replies; 21+ messages in thread
From: Thomas Lamprecht @ 2024-11-10 17:27 UTC (permalink / raw)
To: Proxmox VE development discussion, Lukas Wagner, pbs-devel
Am 08.11.24 um 15:41 schrieb Lukas Wagner:
> The widgets for editing the headers/secrets were adapted from
> the 'Tag Edit' dialog from PVE's datacenter options.
>
> Apart from that, the new dialog is rather standard. I've decided
> to put the http method and url in a single row, mostly to
> save space and also to make it analogous to how an actual http request
> is structured (VERB URL, followed by headers, followed by the body).
>
> The secrets are a mechanism to store tokens/passwords in the
> protected notification config. Secrets are accessible via
> templating in the URL, headers and body via {{ secrets.NAME }}.
> Secrets can only be set/updated, but not retrieved/displayed.
>
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
> ---
> src/Makefile | 1 +
> src/Schema.js | 5 +
> src/Utils.js | 20 ++
> src/panel/WebhookEditPanel.js | 424 ++++++++++++++++++++++++++++++++++
> 4 files changed, 450 insertions(+)
> create mode 100644 src/panel/WebhookEditPanel.js
>
>
applied, thanks!
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [pbs-devel] [pve-devel] [PATCH widget-toolkit v3 07/14] notification: add UI for adding/updating webhook targets
2024-11-08 14:41 ` [pbs-devel] [PATCH widget-toolkit v3 07/14] notification: add UI for adding/updating webhook targets Lukas Wagner
2024-11-10 17:27 ` [pbs-devel] applied: [pve-devel] " Thomas Lamprecht
@ 2024-11-11 22:09 ` Thomas Lamprecht
2024-11-19 8:05 ` Lukas Wagner
1 sibling, 1 reply; 21+ messages in thread
From: Thomas Lamprecht @ 2024-11-11 22:09 UTC (permalink / raw)
To: Proxmox VE development discussion, Lukas Wagner, pbs-devel
Am 08.11.24 um 15:41 schrieb Lukas Wagner:
> The widgets for editing the headers/secrets were adapted from
> the 'Tag Edit' dialog from PVE's datacenter options.
>
> Apart from that, the new dialog is rather standard. I've decided
> to put the http method and url in a single row, mostly to
> save space and also to make it analogous to how an actual http request
> is structured (VERB URL, followed by headers, followed by the body).
>
> The secrets are a mechanism to store tokens/passwords in the
> protected notification config. Secrets are accessible via
> templating in the URL, headers and body via {{ secrets.NAME }}.
> Secrets can only be set/updated, but not retrieved/displayed.
>
I re-checked this now with a bit more time and while it works fine I
think there can be still some UX enhancements:
- Move the "Add" buttons below the grids, like we do for most other
such UIs (like e.g. in the "Match Rules" tab of the Matcher Add/Edit
window). You probably need to change the layout a bit to keep the
label on top; might want to ask Dominik for how to best achieve that.
Maybe include what is added in the button text, like "Add Header" or
"Add Secret"
- always show an emptyText for the key/value fields to better convey
what is what, alternatively show colum headers, but that takes more
space.
- improve validity checking for header/secret fields, without clicking
in them no invalid/required border is shown, but the overall form
state is invalid, thus disabling the window's Add button. This can
be confusing if no field is marked as invalid.
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [pbs-devel] [pve-devel] [PATCH widget-toolkit v3 07/14] notification: add UI for adding/updating webhook targets
2024-11-11 22:09 ` [pbs-devel] " Thomas Lamprecht
@ 2024-11-19 8:05 ` Lukas Wagner
0 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2024-11-19 8:05 UTC (permalink / raw)
To: Proxmox VE development discussion, pbs-devel, t.lamprecht, d.csapak
On Mon Nov 11, 2024 at 11:09 PM CET, Thomas Lamprecht wrote:
> Am 08.11.24 um 15:41 schrieb Lukas Wagner:
> > The widgets for editing the headers/secrets were adapted from
> > the 'Tag Edit' dialog from PVE's datacenter options.
> >
> > Apart from that, the new dialog is rather standard. I've decided
> > to put the http method and url in a single row, mostly to
> > save space and also to make it analogous to how an actual http request
> > is structured (VERB URL, followed by headers, followed by the body).
> >
> > The secrets are a mechanism to store tokens/passwords in the
> > protected notification config. Secrets are accessible via
> > templating in the URL, headers and body via {{ secrets.NAME }}.
> > Secrets can only be set/updated, but not retrieved/displayed.
> >
>
> I re-checked this now with a bit more time and while it works fine I
> think there can be still some UX enhancements:
>
> - Move the "Add" buttons below the grids, like we do for most other
> such UIs (like e.g. in the "Match Rules" tab of the Matcher Add/Edit
> window). You probably need to change the layout a bit to keep the
> label on top; might want to ask Dominik for how to best achieve that.
> Maybe include what is added in the button text, like "Add Header" or
> "Add Secret"
>
> - always show an emptyText for the key/value fields to better convey
> what is what, alternatively show colum headers, but that takes more
> space.
>
> - improve validity checking for header/secret fields, without clicking
> in them no invalid/required border is shown, but the overall form
> state is invalid, thus disabling the window's Add button. This can
> be confusing if no field is marked as invalid.
>
Thanks for the feedback, much appreciated.
Seems like these issues were already taken care of while I was on sick
leave, thanks Dominik!
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pbs-devel] [PATCH manager v3 08/14] api: notifications: use get_targets impl from proxmox-notify
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
` (6 preceding siblings ...)
2024-11-08 14:41 ` [pbs-devel] [PATCH widget-toolkit v3 07/14] notification: add UI for adding/updating webhook targets Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-08 14:41 ` [pbs-devel] [PATCH manager v3 09/14] api: add routes for webhook notification endpoints Lukas Wagner
` (6 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
The get_targets API endpoint is now implemented in Rust.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
---
PVE/API2/Cluster/Notifications.pm | 34 +------------------------------
1 file changed, 1 insertion(+), 33 deletions(-)
diff --git a/PVE/API2/Cluster/Notifications.pm b/PVE/API2/Cluster/Notifications.pm
index 2b202c28..8c9be1ed 100644
--- a/PVE/API2/Cluster/Notifications.pm
+++ b/PVE/API2/Cluster/Notifications.pm
@@ -309,39 +309,7 @@ __PACKAGE__->register_method ({
my $config = PVE::Notify::read_config();
my $targets = eval {
- my $result = [];
-
- for my $target (@{$config->get_sendmail_endpoints()}) {
- push @$result, {
- name => $target->{name},
- comment => $target->{comment},
- type => 'sendmail',
- disable => $target->{disable},
- origin => $target->{origin},
- };
- }
-
- for my $target (@{$config->get_gotify_endpoints()}) {
- push @$result, {
- name => $target->{name},
- comment => $target->{comment},
- type => 'gotify',
- disable => $target->{disable},
- origin => $target->{origin},
- };
- }
-
- for my $target (@{$config->get_smtp_endpoints()}) {
- push @$result, {
- name => $target->{name},
- comment => $target->{comment},
- type => 'smtp',
- disable => $target->{disable},
- origin => $target->{origin},
- };
- }
-
- $result
+ $config->get_targets();
};
raise_api_error($@) if $@;
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] [PATCH manager v3 09/14] api: add routes for webhook notification endpoints
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
` (7 preceding siblings ...)
2024-11-08 14:41 ` [pbs-devel] [PATCH manager v3 08/14] api: notifications: use get_targets impl from proxmox-notify Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-08 14:41 ` [pbs-devel] [PATCH docs v3 10/14] notification: add documentation for webhook target endpoints Lukas Wagner
` (5 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
These just call the API implementation via the perl-rs bindings.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
---
PVE/API2/Cluster/Notifications.pm | 263 +++++++++++++++++++++++++++++-
1 file changed, 262 insertions(+), 1 deletion(-)
diff --git a/PVE/API2/Cluster/Notifications.pm b/PVE/API2/Cluster/Notifications.pm
index 8c9be1ed..7a89f4e9 100644
--- a/PVE/API2/Cluster/Notifications.pm
+++ b/PVE/API2/Cluster/Notifications.pm
@@ -247,6 +247,7 @@ __PACKAGE__->register_method ({
{ name => 'gotify' },
{ name => 'sendmail' },
{ name => 'smtp' },
+ { name => 'webhook' },
];
return $result;
@@ -283,7 +284,7 @@ __PACKAGE__->register_method ({
'type' => {
description => 'Type of the target.',
type => 'string',
- enum => [qw(sendmail gotify smtp)],
+ enum => [qw(sendmail gotify smtp webhook)],
},
'comment' => {
description => 'Comment',
@@ -1233,6 +1234,266 @@ __PACKAGE__->register_method ({
}
});
+my $webhook_properties = {
+ name => {
+ description => 'The name of the endpoint.',
+ type => 'string',
+ format => 'pve-configid',
+ },
+ url => {
+ description => 'Server URL',
+ type => 'string',
+ },
+ method => {
+ description => 'HTTP method',
+ type => 'string',
+ enum => [qw(post put get)],
+ },
+ header => {
+ description => 'HTTP headers to set. These have to be formatted as'
+ . ' a property string in the format name=<name>,value=<base64 of value>',
+ type => 'array',
+ items => {
+ type => 'string',
+ },
+ optional => 1,
+ },
+ body => {
+ description => 'HTTP body, base64 encoded',
+ type => 'string',
+ optional => 1,
+ },
+ secret => {
+ description => 'Secrets to set. These have to be formatted as'
+ . ' a property string in the format name=<name>,value=<base64 of value>',
+ type => 'array',
+ items => {
+ type => 'string',
+ },
+ optional => 1,
+ },
+ comment => {
+ description => 'Comment',
+ type => 'string',
+ optional => 1,
+ },
+ disable => {
+ description => 'Disable this target',
+ type => 'boolean',
+ optional => 1,
+ default => 0,
+ },
+};
+
+__PACKAGE__->register_method ({
+ name => 'get_webhook_endpoints',
+ path => 'endpoints/webhook',
+ method => 'GET',
+ description => 'Returns a list of all webhook endpoints',
+ protected => 1,
+ permissions => {
+ check => ['perm', '/mapping/notifications', ['Mapping.Modify']],
+ check => ['perm', '/mapping/notifications', ['Mapping.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => 'object',
+ properties => {
+ %$webhook_properties,
+ 'origin' => {
+ description => 'Show if this entry was created by a user or was built-in',
+ type => 'string',
+ enum => [qw(user-created builtin modified-builtin)],
+ },
+ },
+ },
+ links => [ { rel => 'child', href => '{name}' } ],
+ },
+ code => sub {
+ my $config = PVE::Notify::read_config();
+ my $rpcenv = PVE::RPCEnvironment::get();
+
+ my $entities = eval {
+ $config->get_webhook_endpoints();
+ };
+ raise_api_error($@) if $@;
+
+ return $entities;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'get_webhook_endpoint',
+ path => 'endpoints/webhook/{name}',
+ method => 'GET',
+ description => 'Return a specific webhook endpoint',
+ protected => 1,
+ permissions => {
+ check => ['or',
+ ['perm', '/mapping/notifications', ['Mapping.Modify']],
+ ['perm', '/mapping/notifications', ['Mapping.Audit']],
+ ],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ name => {
+ type => 'string',
+ format => 'pve-configid',
+ description => 'Name of the endpoint.'
+ },
+ }
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ %$webhook_properties,
+ digest => get_standard_option('pve-config-digest'),
+ }
+ },
+ code => sub {
+ my ($param) = @_;
+ my $name = extract_param($param, 'name');
+
+ my $config = PVE::Notify::read_config();
+ my $endpoint = eval {
+ $config->get_webhook_endpoint($name)
+ };
+
+ raise_api_error($@) if $@;
+ $endpoint->{digest} = $config->digest();
+
+ return $endpoint;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'create_webhook_endpoint',
+ path => 'endpoints/webhook',
+ protected => 1,
+ method => 'POST',
+ description => 'Create a new webhook endpoint',
+ permissions => {
+ check => ['perm', '/mapping/notifications', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => $webhook_properties,
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+
+ $config->add_webhook_endpoint(
+ $param,
+ );
+
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if $@;
+ return;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'update_webhook_endpoint',
+ path => 'endpoints/webhook/{name}',
+ protected => 1,
+ method => 'PUT',
+ description => 'Update existing webhook endpoint',
+ permissions => {
+ check => ['perm', '/mapping/notifications', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ %{ make_properties_optional($webhook_properties) },
+ delete => {
+ type => 'array',
+ items => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ optional => 1,
+ description => 'A list of settings you want to delete.',
+ },
+ digest => get_standard_option('pve-config-digest'),
+ }
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $name = extract_param($param, 'name');
+ my $delete = extract_param($param, 'delete');
+ my $digest = extract_param($param, 'digest');
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+
+ $config->update_webhook_endpoint(
+ $name,
+ $param, # Config updater
+ $delete,
+ $digest,
+ );
+
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if $@;
+ return;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'delete_webhook_endpoint',
+ protected => 1,
+ path => 'endpoints/webhook/{name}',
+ method => 'DELETE',
+ description => 'Remove webhook endpoint',
+ permissions => {
+ check => ['perm', '/mapping/notifications', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ name => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ }
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+ my $name = extract_param($param, 'name');
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+ $config->delete_webhook_endpoint($name);
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if $@;
+ return;
+ }
+});
+
my $matcher_properties = {
name => {
description => 'Name of the matcher.',
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] [PATCH docs v3 10/14] notification: add documentation for webhook target endpoints.
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
` (8 preceding siblings ...)
2024-11-08 14:41 ` [pbs-devel] [PATCH manager v3 09/14] api: add routes for webhook notification endpoints Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox-backup v3 11/14] api: notification: add API routes for webhook targets Lukas Wagner
` (4 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
notifications.adoc | 93 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 93 insertions(+)
diff --git a/notifications.adoc b/notifications.adoc
index 2459095..b7470fe 100644
--- a/notifications.adoc
+++ b/notifications.adoc
@@ -178,6 +178,99 @@ gotify: example
token somesecrettoken
----
+[[notification_targets_webhook]]
+Webhook
+~~~~~~~
+
+Webhook notification targets perform HTTP requests to a configurable URL.
+
+The following configuration options are available:
+
+* `url`: The URL to which to perform the HTTP requests.
+Supports templating to inject message contents, metadata and secrets.
+* `method`: HTTP Method to use (POST/PUT/GET)
+* `header`: Array of HTTP headers that should be set for the request.
+Supports templating to inject message contents, metadata and secrets.
+* `body`: HTTP body that should be sent.
+Supports templating to inject message contents, metadata and secrets.
+* `secret`: Array of secret key-value pairs. These will be stored in
+a protected configuration file only readable by root. Secrets can be
+accessed in body/header/URL templates via the `secrets` namespace.
+* `comment`: Comment for this target.
+
+For configuration options that support templating, the
+https://handlebarsjs.com/[Handlebars] syntax can be used to
+access the following properties:
+
+* `{{ title }}`: The rendered notification title
+* `{{ message }}`: The rendered notification body
+* `{{ severity }}`: The severity of the notification (`info`, `notice`,
+`warning`, `error`, `unknown`)
+* `{{ timestamp }}`: The notification's timestamp as a UNIX epoch (in seconds).
+* `{{ fields.<name> }}`: Sub-namespace for any metadata fields of the notification.
+For instance, `fields.type` contains the notification type - for all available fields refer
+to xref:notification_events[Notification Events].
+* `{{ secrets.<name> }}`: Sub-namespace for secrets. For instance, a secret named `token`
+is accessible via `secrets.token`.
+
+For convenience, the following helpers are available:
+
+* `{{ url-encode <value/property> }}`: URL-encode a property/literal.
+* `{{ escape <value/property> }}`: Escape any control characters that cannot be
+safely represented as a JSON string.
+* `{{ json <value/property> }}`: Render a value as JSON. This can be useful to
+pass a whole sub-namespace (e.g. `fields`) as a part of a JSON payload
+(e.g. `{{ json fields }}`).
+
+==== Examples
+
+===== `ntfy.sh`
+
+* Method: `POST`
+* URL: `https://ntfy.sh/{{ secrets.channel }}`
+* Headers:
+** `Markdown`: `Yes`
+* Body:
+----
+```
+{{ message }}
+```
+----
+* Secrets:
+** `channel`: `<your ntfy.sh channel>`
+
+===== Discord
+
+* Method: `POST`
+* URL: `https://discord.com/api/webhooks/{{ secrets.token }}`
+* Headers:
+** `Content-Type`: `application/json`
+* Body:
+----
+{
+ "content": "``` {{ escape message }}```"
+}
+----
+* Secrets:
+** `token`: `<token>`
+
+===== Slack
+
+* Method: `POST`
+* URL: `https://hooks.slack.com/services/{{ secrets.token }}`
+* Headers:
+** `Content-Type`: `application/json`
+* Body:
+----
+{
+ "text": "``` {{escape message}}```",
+ "type": "mrkdwn"
+}
+----
+* Secrets:
+** `token`: `<token>`
+
+
[[notification_matchers]]
Notification Matchers
---------------------
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] [PATCH proxmox-backup v3 11/14] api: notification: add API routes for webhook targets
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
` (9 preceding siblings ...)
2024-11-08 14:41 ` [pbs-devel] [PATCH docs v3 10/14] notification: add documentation for webhook target endpoints Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-26 10:59 ` [pbs-devel] applied-series: " Thomas Lamprecht
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox-backup v3 12/14] management cli: add CLI " Lukas Wagner
` (3 subsequent siblings)
14 siblings, 1 reply; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
Copied and adapted from the Gotify ones.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
---
src/api2/config/notifications/mod.rs | 2 +
src/api2/config/notifications/webhook.rs | 175 +++++++++++++++++++++++
2 files changed, 177 insertions(+)
create mode 100644 src/api2/config/notifications/webhook.rs
diff --git a/src/api2/config/notifications/mod.rs b/src/api2/config/notifications/mod.rs
index dfe82ed0..81ca9800 100644
--- a/src/api2/config/notifications/mod.rs
+++ b/src/api2/config/notifications/mod.rs
@@ -22,6 +22,7 @@ pub mod matchers;
pub mod sendmail;
pub mod smtp;
pub mod targets;
+pub mod webhook;
#[sortable]
const SUBDIRS: SubdirMap = &sorted!([
@@ -41,6 +42,7 @@ const ENDPOINT_SUBDIRS: SubdirMap = &sorted!([
("gotify", &gotify::ROUTER),
("sendmail", &sendmail::ROUTER),
("smtp", &smtp::ROUTER),
+ ("webhook", &webhook::ROUTER),
]);
const ENDPOINT_ROUTER: Router = Router::new()
diff --git a/src/api2/config/notifications/webhook.rs b/src/api2/config/notifications/webhook.rs
new file mode 100644
index 00000000..4a040024
--- /dev/null
+++ b/src/api2/config/notifications/webhook.rs
@@ -0,0 +1,175 @@
+use anyhow::Error;
+use serde_json::Value;
+
+use proxmox_notify::endpoints::webhook::{
+ DeleteableWebhookProperty, WebhookConfig, WebhookConfigUpdater,
+};
+use proxmox_notify::schema::ENTITY_NAME_SCHEMA;
+use proxmox_router::{Permission, Router, RpcEnvironment};
+use proxmox_schema::api;
+
+use pbs_api_types::{PRIV_SYS_AUDIT, PRIV_SYS_MODIFY, PROXMOX_CONFIG_DIGEST_SCHEMA};
+
+#[api(
+ protected: true,
+ input: {
+ properties: {},
+ },
+ returns: {
+ description: "List of webhook endpoints.",
+ type: Array,
+ items: { type: WebhookConfig },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "notifications"], PRIV_SYS_AUDIT, false),
+ },
+)]
+/// List all webhook endpoints.
+pub fn list_endpoints(
+ _param: Value,
+ _rpcenv: &mut dyn RpcEnvironment,
+) -> Result<Vec<WebhookConfig>, Error> {
+ let config = pbs_config::notifications::config()?;
+
+ let endpoints = proxmox_notify::api::webhook::get_endpoints(&config)?;
+
+ Ok(endpoints)
+}
+
+#[api(
+ protected: true,
+ input: {
+ properties: {
+ name: {
+ schema: ENTITY_NAME_SCHEMA,
+ }
+ },
+ },
+ returns: { type: WebhookConfig },
+ access: {
+ permission: &Permission::Privilege(&["system", "notifications"], PRIV_SYS_AUDIT, false),
+ },
+)]
+/// Get a webhook endpoint.
+pub fn get_endpoint(name: String, rpcenv: &mut dyn RpcEnvironment) -> Result<WebhookConfig, Error> {
+ let config = pbs_config::notifications::config()?;
+ let endpoint = proxmox_notify::api::webhook::get_endpoint(&config, &name)?;
+
+ rpcenv["digest"] = hex::encode(config.digest()).into();
+
+ Ok(endpoint)
+}
+
+#[api(
+ protected: true,
+ input: {
+ properties: {
+ endpoint: {
+ type: WebhookConfig,
+ flatten: true,
+ },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "notifications"], PRIV_SYS_MODIFY, false),
+ },
+)]
+/// Add a new webhook endpoint.
+pub fn add_endpoint(
+ endpoint: WebhookConfig,
+ _rpcenv: &mut dyn RpcEnvironment,
+) -> Result<(), Error> {
+ let _lock = pbs_config::notifications::lock_config()?;
+ let mut config = pbs_config::notifications::config()?;
+
+ proxmox_notify::api::webhook::add_endpoint(&mut config, endpoint)?;
+
+ pbs_config::notifications::save_config(config)?;
+ Ok(())
+}
+
+#[api(
+ protected: true,
+ input: {
+ properties: {
+ name: {
+ schema: ENTITY_NAME_SCHEMA,
+ },
+ updater: {
+ type: WebhookConfigUpdater,
+ flatten: true,
+ },
+ delete: {
+ description: "List of properties to delete.",
+ type: Array,
+ optional: true,
+ items: {
+ type: DeleteableWebhookProperty,
+ }
+ },
+ digest: {
+ optional: true,
+ schema: PROXMOX_CONFIG_DIGEST_SCHEMA,
+ },
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "notifications"], PRIV_SYS_MODIFY, false),
+ },
+)]
+/// Update webhook endpoint.
+pub fn update_endpoint(
+ name: String,
+ updater: WebhookConfigUpdater,
+ delete: Option<Vec<DeleteableWebhookProperty>>,
+ digest: Option<String>,
+ _rpcenv: &mut dyn RpcEnvironment,
+) -> Result<(), Error> {
+ let _lock = pbs_config::notifications::lock_config()?;
+ let mut config = pbs_config::notifications::config()?;
+ let digest = digest.map(hex::decode).transpose()?;
+
+ proxmox_notify::api::webhook::update_endpoint(
+ &mut config,
+ &name,
+ updater,
+ delete.as_deref(),
+ digest.as_deref(),
+ )?;
+
+ pbs_config::notifications::save_config(config)?;
+ Ok(())
+}
+
+#[api(
+ protected: true,
+ input: {
+ properties: {
+ name: {
+ schema: ENTITY_NAME_SCHEMA,
+ }
+ },
+ },
+ access: {
+ permission: &Permission::Privilege(&["system", "notifications"], PRIV_SYS_MODIFY, false),
+ },
+)]
+/// Delete webhook endpoint.
+pub fn delete_endpoint(name: String, _rpcenv: &mut dyn RpcEnvironment) -> Result<(), Error> {
+ let _lock = pbs_config::notifications::lock_config()?;
+ let mut config = pbs_config::notifications::config()?;
+ proxmox_notify::api::webhook::delete_endpoint(&mut config, &name)?;
+
+ pbs_config::notifications::save_config(config)?;
+ Ok(())
+}
+
+const ITEM_ROUTER: Router = Router::new()
+ .get(&API_METHOD_GET_ENDPOINT)
+ .put(&API_METHOD_UPDATE_ENDPOINT)
+ .delete(&API_METHOD_DELETE_ENDPOINT);
+
+pub const ROUTER: Router = Router::new()
+ .get(&API_METHOD_LIST_ENDPOINTS)
+ .post(&API_METHOD_ADD_ENDPOINT)
+ .match_all("name", &ITEM_ROUTER);
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] applied-series: [PATCH proxmox-backup v3 11/14] api: notification: add API routes for webhook targets
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox-backup v3 11/14] api: notification: add API routes for webhook targets Lukas Wagner
@ 2024-11-26 10:59 ` Thomas Lamprecht
0 siblings, 0 replies; 21+ messages in thread
From: Thomas Lamprecht @ 2024-11-26 10:59 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Lukas Wagner, pve-devel
Am 08.11.24 um 15:41 schrieb Lukas Wagner:
> Copied and adapted from the Gotify ones.
>
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
> ---
> src/api2/config/notifications/mod.rs | 2 +
> src/api2/config/notifications/webhook.rs | 175 +++++++++++++++++++++++
> 2 files changed, 177 insertions(+)
> create mode 100644 src/api2/config/notifications/webhook.rs
>
>
applied the four remaining PBS patches, thanks!
FYI I used the `--whitespace=fix` git apply/am option to drop a few trailing
whitespaces in the docs patches, just noting to avoid any surprise why this
cannot be directly rebased on.
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v3 12/14] management cli: add CLI for webhook targets
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
` (10 preceding siblings ...)
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox-backup v3 11/14] api: notification: add API routes for webhook targets Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox-backup v3 13/14] ui: utils: enable webhook edit window Lukas Wagner
` (2 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
The code was copied and adapted from the gotify target CLI.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
.../notifications/mod.rs | 4 +-
.../notifications/webhook.rs | 94 +++++++++++++++++++
2 files changed, 97 insertions(+), 1 deletion(-)
create mode 100644 src/bin/proxmox_backup_manager/notifications/webhook.rs
diff --git a/src/bin/proxmox_backup_manager/notifications/mod.rs b/src/bin/proxmox_backup_manager/notifications/mod.rs
index 678f9c54..9180a273 100644
--- a/src/bin/proxmox_backup_manager/notifications/mod.rs
+++ b/src/bin/proxmox_backup_manager/notifications/mod.rs
@@ -5,12 +5,14 @@ mod matchers;
mod sendmail;
mod smtp;
mod targets;
+mod webhook;
pub fn notification_commands() -> CommandLineInterface {
let endpoint_def = CliCommandMap::new()
.insert("gotify", gotify::commands())
.insert("sendmail", sendmail::commands())
- .insert("smtp", smtp::commands());
+ .insert("smtp", smtp::commands())
+ .insert("webhook", webhook::commands());
let cmd_def = CliCommandMap::new()
.insert("endpoint", endpoint_def)
diff --git a/src/bin/proxmox_backup_manager/notifications/webhook.rs b/src/bin/proxmox_backup_manager/notifications/webhook.rs
new file mode 100644
index 00000000..bd0ac41b
--- /dev/null
+++ b/src/bin/proxmox_backup_manager/notifications/webhook.rs
@@ -0,0 +1,94 @@
+use anyhow::Error;
+use serde_json::Value;
+
+use proxmox_notify::schema::ENTITY_NAME_SCHEMA;
+use proxmox_router::{cli::*, ApiHandler, RpcEnvironment};
+use proxmox_schema::api;
+
+use proxmox_backup::api2;
+
+#[api(
+ input: {
+ properties: {
+ "output-format": {
+ schema: OUTPUT_FORMAT,
+ optional: true,
+ },
+ }
+ }
+)]
+/// List all endpoints.
+fn list_endpoints(param: Value, rpcenv: &mut dyn RpcEnvironment) -> Result<Value, Error> {
+ let output_format = get_output_format(¶m);
+
+ let info = &api2::config::notifications::webhook::API_METHOD_LIST_ENDPOINTS;
+ let mut data = match info.handler {
+ ApiHandler::Sync(handler) => (handler)(param, info, rpcenv)?,
+ _ => unreachable!(),
+ };
+
+ let options = default_table_format_options()
+ .column(ColumnConfig::new("disable"))
+ .column(ColumnConfig::new("name"))
+ .column(ColumnConfig::new("method"))
+ .column(ColumnConfig::new("url"))
+ .column(ColumnConfig::new("comment"));
+
+ format_and_print_result_full(&mut data, &info.returns, &output_format, &options);
+
+ Ok(Value::Null)
+}
+
+#[api(
+ input: {
+ properties: {
+ name: {
+ schema: ENTITY_NAME_SCHEMA,
+ },
+ "output-format": {
+ schema: OUTPUT_FORMAT,
+ optional: true,
+ },
+ }
+ }
+)]
+/// Show a single endpoint.
+fn show_endpoint(param: Value, rpcenv: &mut dyn RpcEnvironment) -> Result<Value, Error> {
+ let output_format = get_output_format(¶m);
+
+ let info = &api2::config::notifications::webhook::API_METHOD_GET_ENDPOINT;
+ let mut data = match info.handler {
+ ApiHandler::Sync(handler) => (handler)(param, info, rpcenv)?,
+ _ => unreachable!(),
+ };
+
+ let options = default_table_format_options();
+ format_and_print_result_full(&mut data, &info.returns, &output_format, &options);
+
+ Ok(Value::Null)
+}
+
+pub fn commands() -> CommandLineInterface {
+ let cmd_def = CliCommandMap::new()
+ .insert("list", CliCommand::new(&API_METHOD_LIST_ENDPOINTS))
+ .insert(
+ "show",
+ CliCommand::new(&API_METHOD_SHOW_ENDPOINT).arg_param(&["name"]),
+ )
+ .insert(
+ "create",
+ CliCommand::new(&api2::config::notifications::webhook::API_METHOD_ADD_ENDPOINT)
+ .arg_param(&["name"]),
+ )
+ .insert(
+ "update",
+ CliCommand::new(&api2::config::notifications::webhook::API_METHOD_UPDATE_ENDPOINT)
+ .arg_param(&["name"]),
+ )
+ .insert(
+ "delete",
+ CliCommand::new(&api2::config::notifications::webhook::API_METHOD_DELETE_ENDPOINT)
+ .arg_param(&["name"]),
+ );
+ cmd_def.into()
+}
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] [PATCH proxmox-backup v3 13/14] ui: utils: enable webhook edit window
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
` (11 preceding siblings ...)
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox-backup v3 12/14] management cli: add CLI " Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox-backup v3 14/14] docs: notification: add webhook endpoint documentation Lukas Wagner
2024-11-11 22:02 ` [pbs-devel] partially-applied: [pve-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Thomas Lamprecht
14 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
This allows users to add/edit new webhook targets.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
---
www/Utils.js | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/www/Utils.js b/www/Utils.js
index 4853be36..b715972f 100644
--- a/www/Utils.js
+++ b/www/Utils.js
@@ -482,6 +482,11 @@ Ext.define('PBS.Utils', {
ipanel: 'pmxGotifyEditPanel',
iconCls: 'fa-bell-o',
},
+ webhook: {
+ name: 'Webhook',
+ ipanel: 'pmxWebhookEditPanel',
+ iconCls: 'fa-bell-o',
+ },
};
},
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] [PATCH proxmox-backup v3 14/14] docs: notification: add webhook endpoint documentation
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
` (12 preceding siblings ...)
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox-backup v3 13/14] ui: utils: enable webhook edit window Lukas Wagner
@ 2024-11-08 14:41 ` Lukas Wagner
2024-11-11 22:02 ` [pbs-devel] partially-applied: [pve-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Thomas Lamprecht
14 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2024-11-08 14:41 UTC (permalink / raw)
To: pve-devel, pbs-devel
Same information as in pve-docs but translated to restructured text.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
docs/notifications.rst | 100 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 100 insertions(+)
diff --git a/docs/notifications.rst b/docs/notifications.rst
index 4ba8db86..d059fa76 100644
--- a/docs/notifications.rst
+++ b/docs/notifications.rst
@@ -85,6 +85,106 @@ integrate with different platforms and services.
See :ref:`notifications.cfg` for all configuration options.
+.. _notification_targets_webhook:
+Webhook
+^^^^^^^
+Webhook notification targets perform HTTP requests to a configurable URL.
+
+The following configuration options are available:
+
+* ``url``: The URL to which to perform the HTTP requests.
+ Supports templating to inject message contents, metadata and secrets.
+* ``method``: HTTP Method to use (POST/PUT/GET)
+* ``header``: Array of HTTP headers that should be set for the request.
+ Supports templating to inject message contents, metadata and secrets.
+* ``body``: HTTP body that should be sent.
+ Supports templating to inject message contents, metadata and secrets.
+* ``secret``: Array of secret key-value pairs. These will be stored in
+ a protected configuration file only readable by root. Secrets can be
+ accessed in body/header/URL templates via the ``secrets`` namespace.
+* ``comment``: Comment for this target.
+
+For configuration options that support templating, the
+`Handlebars <https://handlebarsjs.com>`_ syntax can be used to
+access the following properties:
+
+* ``{{ title }}``: The rendered notification title
+* ``{{ message }}``: The rendered notification body
+* ``{{ severity }}``: The severity of the notification (``info``, ``notice``,
+ ``warning``, ``error``, ``unknown``)
+* ``{{ timestamp }}``: The notification's timestamp as a UNIX epoch (in seconds).
+* ``{{ fields.<name> }}``: Sub-namespace for any metadata fields of the
+ notification. For instance, ``fields.type`` contains the notification
+ type - for all available fields refer to :ref:`notification_events`.
+* ``{{ secrets.<name> }}``: Sub-namespace for secrets. For instance, a secret
+ named ``token`` is accessible via ``secrets.token``.
+
+For convenience, the following helpers are available:
+
+* ``{{ url-encode <value/property> }}``: URL-encode a property/literal.
+* ``{{ escape <value/property> }}``: Escape any control characters that cannot
+ be safely represented as a JSON string.
+* ``{{ json <value/property> }}``: Render a value as JSON. This can be useful
+ to pass a whole sub-namespace (e.g. ``fields``) as a part of a JSON payload
+ (e.g. ``{{ json fields }}``).
+
+Example - ntfy.sh
+"""""""""""""""""
+
+* Method: ``POST``
+* URL: ``https://ntfy.sh/{{ secrets.channel }}``
+* Headers:
+
+ * ``Markdown``: ``Yes``
+* Body::
+
+ ```
+ {{ message }}
+ ```
+
+* Secrets:
+
+ * ``channel``: ``<your ntfy.sh channel>``
+
+Example - Discord
+"""""""""""""""""
+
+* Method: ``POST``
+* URL: ``https://discord.com/api/webhooks/{{ secrets.token }}``
+* Headers:
+
+ * ``Content-Type``: ``application/json``
+
+* Body::
+
+ {
+ "content": "``` {{ escape message }}```"
+ }
+
+* Secrets:
+
+ * ``token``: ``<token>``
+
+Example - Slack
+"""""""""""""""
+
+* Method: ``POST``
+* URL: ``https://hooks.slack.com/services/{{ secrets.token }}``
+* Headers:
+
+ * ``Content-Type``: ``application/json``
+
+* Body::
+
+ {
+ "text": "``` {{escape message}}```",
+ "type": "mrkdwn"
+ }
+
+* Secrets:
+
+ * ``token``: ``<token>``
+
.. _notification_matchers:
Notification Matchers
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread* [pbs-devel] partially-applied: [pve-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints
2024-11-08 14:41 [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints Lukas Wagner
` (13 preceding siblings ...)
2024-11-08 14:41 ` [pbs-devel] [PATCH proxmox-backup v3 14/14] docs: notification: add webhook endpoint documentation Lukas Wagner
@ 2024-11-11 22:02 ` Thomas Lamprecht
14 siblings, 0 replies; 21+ messages in thread
From: Thomas Lamprecht @ 2024-11-11 22:02 UTC (permalink / raw)
To: Proxmox VE development discussion, Lukas Wagner, pbs-devel
Am 08.11.24 um 15:41 schrieb Lukas Wagner:
> This series adds support for webhook notification targets to PVE
> and PBS.
>
> A webhook is a HTTP API route provided by a third-party service that
> can be used to inform the third-party about an event. In our case,
> we can easily interact with various third-party notification/messaging
> systems and send PVE/PBS notifications via this service.
> The changes were tested against ntfy.sh, Discord and Slack.
>
> The configuration of webhook targets allows one to configure:
> - The URL
> - The HTTP method (GET/POST/PUT)
> - HTTP Headers
> - Body
>
> One can use handlebar templating to inject notification text and metadata
> in the url, headers and body.
>
> One challenge is the handling of sensitve tokens and other secrets.
> Since the endpoint is completely generic, we cannot know in advance
> whether the body/header/url contains sensitive values.
> Thus we add 'secrets' which are stored in the protected config only
> accessible by root (e.g. /etc/pve/priv/notifications.cfg). These
> secrets are accessible in URLs/headers/body via templating:
>
> Url: https://example.com/{{ secrets.token }}
>
> Secrets can only be set and updated, but never retrieved via the API.
> In the UI, secrets are handled like other secret tokens/passwords.
>
> Bumps for PVE:
> - libpve-rs-perl needs proxmox-notify bumped
> - pve-manager needs proxmox-widget-toolkit and libpve-rs-perl bumped
> - proxmox-mail-forward needs proxmox-notify bumped
>
> Bumps for PBS:
> - proxmox-backup needs proxmox-notify bumped
> - proxmox-mail-forward needs proxmox-notify bumped
>
>
> Changes v1 -> v2:
> - Rebase proxmox-notify changes
>
> Changes v2 -> v3:
> - Fix utf8 -> base64 encoding bug (thx @ Stefan)
> - Fix bug that allowed one to save a target with an empty header
> value when updating the target
> - Additional UI-side input validation (e.g. target name, URL)
> - Code documentation improvments
> - Mask secrets in errors returned from the proxmox-notify crate, hopefully
> preventing them to be shown in logs or error messages
> - Rebased on the latest master branches
>
> proxmox:
>
> Lukas Wagner (3):
> notify: renderer: adapt to changes in proxmox-time
> notify: implement webhook targets
> notify: add api for webhook targets
>
> proxmox-notify/Cargo.toml | 9 +-
> proxmox-notify/src/api/mod.rs | 20 +
> proxmox-notify/src/api/webhook.rs | 432 +++++++++++++++++++
> proxmox-notify/src/config.rs | 23 +
> proxmox-notify/src/endpoints/mod.rs | 2 +
> proxmox-notify/src/endpoints/webhook.rs | 550 ++++++++++++++++++++++++
> proxmox-notify/src/lib.rs | 17 +
> proxmox-notify/src/renderer/mod.rs | 4 +-
> 8 files changed, 1052 insertions(+), 5 deletions(-)
> create mode 100644 proxmox-notify/src/api/webhook.rs
> create mode 100644 proxmox-notify/src/endpoints/webhook.rs
>
>
> proxmox-perl-rs:
>
> Lukas Wagner (2):
> common: notify: add bindings for webhook API routes
> common: notify: add bindings for get_targets
>
> common/src/notify.rs | 72 ++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 72 insertions(+)
>
>
> proxmox-widget-toolkit:
>
> Gabriel Goller (1):
> utils: add base64 conversion helper
>
> Lukas Wagner (1):
> notification: add UI for adding/updating webhook targets
>
> src/Makefile | 1 +
> src/Schema.js | 5 +
> src/Utils.js | 38 +++
> src/panel/WebhookEditPanel.js | 424 ++++++++++++++++++++++++++++++++++
> 4 files changed, 468 insertions(+)
> create mode 100644 src/panel/WebhookEditPanel.js
>
>
> pve-manager:
>
> Lukas Wagner (2):
> api: notifications: use get_targets impl from proxmox-notify
> api: add routes for webhook notification endpoints
>
> PVE/API2/Cluster/Notifications.pm | 297 ++++++++++++++++++++++++++----
> 1 file changed, 263 insertions(+), 34 deletions(-)
>
>
> pve-docs:
>
> Lukas Wagner (1):
> notification: add documentation for webhook target endpoints.
>
> notifications.adoc | 93 ++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 93 insertions(+)
>
>
applied the common and PVE part of this series, thanks!
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 21+ messages in thread