From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 0F1F41FF164 for ; Fri, 8 Nov 2024 15:41:39 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 9BBB01008B; Fri, 8 Nov 2024 15:41:36 +0100 (CET) From: Lukas Wagner To: pve-devel@lists.proxmox.com, pbs-devel@lists.proxmox.com Date: Fri, 8 Nov 2024 15:41:10 +0100 Message-Id: <20241108144124.273550-1-l.wagner@proxmox.com> X-Mailer: git-send-email 2.39.5 MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.007 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [config.rs, mod.rs, notifications.pm, webhook.rs, notify.rs, lib.rs] Subject: [pbs-devel] [PATCH many v3 00/14] notifications: add support for webhook endpoints X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" This series adds support for webhook notification targets to PVE and PBS. A webhook is a HTTP API route provided by a third-party service that can be used to inform the third-party about an event. In our case, we can easily interact with various third-party notification/messaging systems and send PVE/PBS notifications via this service. The changes were tested against ntfy.sh, Discord and Slack. The configuration of webhook targets allows one to configure: - The URL - The HTTP method (GET/POST/PUT) - HTTP Headers - Body One can use handlebar templating to inject notification text and metadata in the url, headers and body. One challenge is the handling of sensitve tokens and other secrets. Since the endpoint is completely generic, we cannot know in advance whether the body/header/url contains sensitive values. Thus we add 'secrets' which are stored in the protected config only accessible by root (e.g. /etc/pve/priv/notifications.cfg). These secrets are accessible in URLs/headers/body via templating: Url: https://example.com/{{ secrets.token }} Secrets can only be set and updated, but never retrieved via the API. In the UI, secrets are handled like other secret tokens/passwords. Bumps for PVE: - libpve-rs-perl needs proxmox-notify bumped - pve-manager needs proxmox-widget-toolkit and libpve-rs-perl bumped - proxmox-mail-forward needs proxmox-notify bumped Bumps for PBS: - proxmox-backup needs proxmox-notify bumped - proxmox-mail-forward needs proxmox-notify bumped Changes v1 -> v2: - Rebase proxmox-notify changes Changes v2 -> v3: - Fix utf8 -> base64 encoding bug (thx @ Stefan) - Fix bug that allowed one to save a target with an empty header value when updating the target - Additional UI-side input validation (e.g. target name, URL) - Code documentation improvments - Mask secrets in errors returned from the proxmox-notify crate, hopefully preventing them to be shown in logs or error messages - Rebased on the latest master branches proxmox: Lukas Wagner (3): notify: renderer: adapt to changes in proxmox-time notify: implement webhook targets notify: add api for webhook targets proxmox-notify/Cargo.toml | 9 +- proxmox-notify/src/api/mod.rs | 20 + proxmox-notify/src/api/webhook.rs | 432 +++++++++++++++++++ proxmox-notify/src/config.rs | 23 + proxmox-notify/src/endpoints/mod.rs | 2 + proxmox-notify/src/endpoints/webhook.rs | 550 ++++++++++++++++++++++++ proxmox-notify/src/lib.rs | 17 + proxmox-notify/src/renderer/mod.rs | 4 +- 8 files changed, 1052 insertions(+), 5 deletions(-) create mode 100644 proxmox-notify/src/api/webhook.rs create mode 100644 proxmox-notify/src/endpoints/webhook.rs proxmox-perl-rs: Lukas Wagner (2): common: notify: add bindings for webhook API routes common: notify: add bindings for get_targets common/src/notify.rs | 72 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) proxmox-widget-toolkit: Gabriel Goller (1): utils: add base64 conversion helper Lukas Wagner (1): notification: add UI for adding/updating webhook targets src/Makefile | 1 + src/Schema.js | 5 + src/Utils.js | 38 +++ src/panel/WebhookEditPanel.js | 424 ++++++++++++++++++++++++++++++++++ 4 files changed, 468 insertions(+) create mode 100644 src/panel/WebhookEditPanel.js pve-manager: Lukas Wagner (2): api: notifications: use get_targets impl from proxmox-notify api: add routes for webhook notification endpoints PVE/API2/Cluster/Notifications.pm | 297 ++++++++++++++++++++++++++---- 1 file changed, 263 insertions(+), 34 deletions(-) pve-docs: Lukas Wagner (1): notification: add documentation for webhook target endpoints. notifications.adoc | 93 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) proxmox-backup: Lukas Wagner (4): api: notification: add API routes for webhook targets management cli: add CLI for webhook targets ui: utils: enable webhook edit window docs: notification: add webhook endpoint documentation docs/notifications.rst | 100 ++++++++++ src/api2/config/notifications/mod.rs | 2 + src/api2/config/notifications/webhook.rs | 175 ++++++++++++++++++ .../notifications/mod.rs | 4 +- .../notifications/webhook.rs | 94 ++++++++++ www/Utils.js | 5 + 6 files changed, 379 insertions(+), 1 deletion(-) create mode 100644 src/api2/config/notifications/webhook.rs create mode 100644 src/bin/proxmox_backup_manager/notifications/webhook.rs Summary over all repositories: 21 files changed, 2327 insertions(+), 40 deletions(-) -- Generated by git-murpp 0.7.3 _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel