public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Lukas Wagner <l.wagner@proxmox.com>
To: pve-devel@lists.proxmox.com, pbs-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints
Date: Fri,  8 Nov 2024 15:41:10 +0100	[thread overview]
Message-ID: <20241108144124.273550-1-l.wagner@proxmox.com> (raw)

This series adds support for webhook notification targets to PVE
and PBS.

A webhook is a HTTP API route provided by a third-party service that
can be used to inform the third-party about an event. In our case,
we can easily interact with various third-party notification/messaging
systems and send PVE/PBS notifications via this service.
The changes were tested against ntfy.sh, Discord and Slack.

The configuration of webhook targets allows one to configure:
  - The URL
  - The HTTP method (GET/POST/PUT)
  - HTTP Headers
  - Body

One can use handlebar templating to inject notification text and metadata
in the url, headers and body.

One challenge is the handling of sensitve tokens and other secrets.
Since the endpoint is completely generic, we cannot know in advance
whether the body/header/url contains sensitive values.
Thus we add 'secrets' which are stored in the protected config only
accessible by root (e.g. /etc/pve/priv/notifications.cfg). These
secrets are accessible in URLs/headers/body via templating:

  Url: https://example.com/{{ secrets.token }}

Secrets can only be set and updated, but never retrieved via the API.
In the UI, secrets are handled like other secret tokens/passwords.

Bumps for PVE:
  - libpve-rs-perl needs proxmox-notify bumped
  - pve-manager needs proxmox-widget-toolkit and libpve-rs-perl bumped
  - proxmox-mail-forward needs proxmox-notify bumped

Bumps for PBS:
  - proxmox-backup needs proxmox-notify bumped
  - proxmox-mail-forward needs proxmox-notify bumped


Changes v1 -> v2:
  - Rebase proxmox-notify changes

Changes v2 -> v3:
  - Fix utf8 -> base64 encoding bug (thx @ Stefan)
  - Fix bug that allowed one to save a target with an empty header
    value when updating the target
  - Additional UI-side input validation (e.g. target name, URL)
  - Code documentation improvments
  - Mask secrets in errors returned from the proxmox-notify crate, hopefully
    preventing them to be shown in logs or error messages
  - Rebased on the latest master branches

proxmox:

Lukas Wagner (3):
  notify: renderer: adapt to changes in proxmox-time
  notify: implement webhook targets
  notify: add api for webhook targets

 proxmox-notify/Cargo.toml               |   9 +-
 proxmox-notify/src/api/mod.rs           |  20 +
 proxmox-notify/src/api/webhook.rs       | 432 +++++++++++++++++++
 proxmox-notify/src/config.rs            |  23 +
 proxmox-notify/src/endpoints/mod.rs     |   2 +
 proxmox-notify/src/endpoints/webhook.rs | 550 ++++++++++++++++++++++++
 proxmox-notify/src/lib.rs               |  17 +
 proxmox-notify/src/renderer/mod.rs      |   4 +-
 8 files changed, 1052 insertions(+), 5 deletions(-)
 create mode 100644 proxmox-notify/src/api/webhook.rs
 create mode 100644 proxmox-notify/src/endpoints/webhook.rs


proxmox-perl-rs:

Lukas Wagner (2):
  common: notify: add bindings for webhook API routes
  common: notify: add bindings for get_targets

 common/src/notify.rs | 72 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 72 insertions(+)


proxmox-widget-toolkit:

Gabriel Goller (1):
  utils: add base64 conversion helper

Lukas Wagner (1):
  notification: add UI for adding/updating webhook targets

 src/Makefile                  |   1 +
 src/Schema.js                 |   5 +
 src/Utils.js                  |  38 +++
 src/panel/WebhookEditPanel.js | 424 ++++++++++++++++++++++++++++++++++
 4 files changed, 468 insertions(+)
 create mode 100644 src/panel/WebhookEditPanel.js


pve-manager:

Lukas Wagner (2):
  api: notifications: use get_targets impl from proxmox-notify
  api: add routes for webhook notification endpoints

 PVE/API2/Cluster/Notifications.pm | 297 ++++++++++++++++++++++++++----
 1 file changed, 263 insertions(+), 34 deletions(-)


pve-docs:

Lukas Wagner (1):
  notification: add documentation for webhook target endpoints.

 notifications.adoc | 93 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 93 insertions(+)


proxmox-backup:

Lukas Wagner (4):
  api: notification: add API routes for webhook targets
  management cli: add CLI for webhook targets
  ui: utils: enable webhook edit window
  docs: notification: add webhook endpoint documentation

 docs/notifications.rst                        | 100 ++++++++++
 src/api2/config/notifications/mod.rs          |   2 +
 src/api2/config/notifications/webhook.rs      | 175 ++++++++++++++++++
 .../notifications/mod.rs                      |   4 +-
 .../notifications/webhook.rs                  |  94 ++++++++++
 www/Utils.js                                  |   5 +
 6 files changed, 379 insertions(+), 1 deletion(-)
 create mode 100644 src/api2/config/notifications/webhook.rs
 create mode 100644 src/bin/proxmox_backup_manager/notifications/webhook.rs


Summary over all repositories:
  21 files changed, 2327 insertions(+), 40 deletions(-)

-- 
Generated by git-murpp 0.7.3


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


             reply	other threads:[~2024-11-08 14:41 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-11-08 14:41 Lukas Wagner [this message]
2024-11-08 14:41 ` [pve-devel] [PATCH proxmox v3 01/14] notify: renderer: adapt to changes in proxmox-time Lukas Wagner
2024-11-08 14:41 ` [pve-devel] [PATCH proxmox v3 02/14] notify: implement webhook targets Lukas Wagner
2024-11-08 14:41 ` [pve-devel] [PATCH proxmox v3 03/14] notify: add api for " Lukas Wagner
2024-11-08 14:41 ` [pve-devel] [PATCH proxmox-perl-rs v3 04/14] common: notify: add bindings for webhook API routes Lukas Wagner
2024-11-08 14:41 ` [pve-devel] [PATCH proxmox-perl-rs v3 05/14] common: notify: add bindings for get_targets Lukas Wagner
2024-11-08 14:41 ` [pve-devel] [PATCH widget-toolkit v3 06/14] utils: add base64 conversion helper Lukas Wagner
2024-11-10 17:27   ` [pve-devel] applied: " Thomas Lamprecht
2024-11-08 14:41 ` [pve-devel] [PATCH widget-toolkit v3 07/14] notification: add UI for adding/updating webhook targets Lukas Wagner
2024-11-10 17:27   ` [pve-devel] applied: " Thomas Lamprecht
2024-11-11 22:09   ` [pve-devel] " Thomas Lamprecht
2024-11-08 14:41 ` [pve-devel] [PATCH manager v3 08/14] api: notifications: use get_targets impl from proxmox-notify Lukas Wagner
2024-11-08 14:41 ` [pve-devel] [PATCH manager v3 09/14] api: add routes for webhook notification endpoints Lukas Wagner
2024-11-08 14:41 ` [pve-devel] [PATCH docs v3 10/14] notification: add documentation for webhook target endpoints Lukas Wagner
2024-11-08 14:41 ` [pve-devel] [PATCH proxmox-backup v3 11/14] api: notification: add API routes for webhook targets Lukas Wagner
2024-11-08 14:41 ` [pve-devel] [PATCH proxmox-backup v3 12/14] management cli: add CLI " Lukas Wagner
2024-11-08 14:41 ` [pve-devel] [PATCH proxmox-backup v3 13/14] ui: utils: enable webhook edit window Lukas Wagner
2024-11-08 14:41 ` [pve-devel] [PATCH proxmox-backup v3 14/14] docs: notification: add webhook endpoint documentation Lukas Wagner
2024-11-11 22:02 ` [pve-devel] partially-applied: [PATCH many v3 00/14] notifications: add support for webhook endpoints Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20241108144124.273550-1-l.wagner@proxmox.com \
    --to=l.wagner@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal