From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pbs-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id 7E8761FF348
for <inbox@lore.proxmox.com>; Wed, 17 Apr 2024 16:38:08 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 18AE8BCC7;
Wed, 17 Apr 2024 16:38:09 +0200 (CEST)
Message-ID: <b97754c0-bdcc-4a6e-ba40-8541288dd8ad@proxmox.com>
Date: Wed, 17 Apr 2024 16:38:03 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Lukas Wagner <l.wagner@proxmox.com>
To: pbs-devel@lists.proxmox.com
References: <20240412100631.94218-1-l.wagner@proxmox.com>
Content-Language: de-AT, en-US
In-Reply-To: <20240412100631.94218-1-l.wagner@proxmox.com>
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.152 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
POISEN_SPAM_PILL 0.1 Meta: its spam
POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes
POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: Re: [pbs-devel] [PATCH proxmox-backup 00/33] integrate notification
system
X-BeenThere: pbs-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Backup Server development discussion
<pbs-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>,
<mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/>
List-Post: <mailto:pbs-devel@lists.proxmox.com>
List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>,
<mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox Backup Server development discussion
<pbs-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pbs-devel-bounces@lists.proxmox.com
Sender: "pbs-devel" <pbs-devel-bounces@lists.proxmox.com>
Superseded-by: https://lists.proxmox.com/pipermail/pbs-devel/2024-April/008639.html
On 2024-04-12 12:05, Lukas Wagner wrote:
> These patches integrate the notification system which was introduced in Proxmox VE
> in 8.1 into Proxmox Backup Server.
>
> Some highlights/noteworthy details from the series:
> - notification template files are installed in /usr/share/proxmox-backup/templates/default
> and are rendered when a notification is sent
> - since sending notifications needs to be done from a privileged context
> (to read protected passwords/tokens from the priv config file),
> we queue notifications to be sent in /var/lib/proxmox-backup/notifications
> and periodically send any queued notifications via a worker in the
> privileged process
> - The API endpoint paths are prefixed with /config/notifications
> - API endpoints which read/modify notification system config require
> Sys.Audit (read) or Sys.Modify (modify) permissions on /system/notifications
>
> - tape-{backup,restore} settings and datastore options now have a
> 'notification-mode' parameter, which allows to choose between the
> 'legacy' behavior (sendmail to a selected user's email address) and
> the event-based notification system. If the parameter is not set, we
> default to the legacy behavior in order to keep existing behavior
> as is for now. For new datastores/tape backup jobs created
> interactively from teh UI, we set the parameter to 'notification-system'
> and opt in into the new system by default.
>
> Still missing/rough edges:
> - documentation (will follow soon)
> - Datastore option view in UI could be improved. When 'notification-mode' is
> set to 'notification-system', we should indicate that 'mailto-user' and the
> other notification settings have no effect. Already did that in the edit
> window, but in the grid panel I did not find a quick way to do that.
>
> - still needs a bit more testing (e.g. ACME notifications) - would greatly
> appreciate people helping here!
>
> Open questions:
> - Right now the 'default-matcher' matches every notification, same as in PVE.
> Up until now, datastore prune notifications were only sent on errors, so
> maybe we should also have the default-matcher work the same?
> The default matcher would then probably be:
> - !All
> - severity: error
> - match-field: exact:type=prune
>
> This patch series requires the patches for 'proxmox' from
> https://lists.proxmox.com/pipermail/pve-devel/2024-April/062708.html
> to be applied.
>
> The dependency for proxmox-widget-toolkit should be bumped to at least
> 4.1.4 (we need: "utils: add mechanism to add and override translatable notification event
> descriptions in the product specific UIs")
> Also, while not strictly needed, the patches for widget-toolkit from
> https://lists.proxmox.com/pipermail/pve-devel/2024-February/061992.html
> make creating matchers much easier - the required API endpoints were
> already implemented by this series.
>
> proxmox-backup:
>
> Lukas Wagner (33):
> pbs-config: add module for loading notification config
> server: rename email_notifications module to notifications
> notifications: allow sending notifications via proxmox_notify
> buildsys: install templates for test notifications
> pbs-config: acl: add /system/notifications as known ACL path
> api: add endpoints for querying/testing notification targets
> api: add endpoints for notification matchers
> api: add endpoints for sendmail targets
> api: add endpoints for smtp targets
> api: add endpoints for gotify targets
> api: add endpoints for querying known notification values/fields
> api-types: api: datatore: add notification-mode parameter
> api-types: api: tape: add notification-mode parameter
> server: notifications: send GC notifications via notification system
> server: notifications: send prune notifications via notification
> system
> server: notifications: send verify notifications via notification
> system
> server: notifications: send sync notifications via notification system
> server: notifications: send update notifications via notification
> system
> server: notifications: send acme notifications via notification system
> server: notifications: send tape notifications via notification system
> ui: add notification config panel
> ui: tape backup job: add selector for notification-mode
> ui: tape backup: add selector for 'notification-mode'
> ui: tape restore: add 'notification-mode' parameter
> ui: datastore options: add 'notification-mode' parameter
> ui: utils: add overrides for known notification metadata fields/values
> ui: datastore edit: make new stores use notification system by default
> ui: permissions paths: add /system/notifications to combobox
> proxmox-backup-manager: add CLI for notification targets
> proxmox-backup-manager: add CLI for notification matchers
> proxmox-backup-manager: add CLI for gotify endpoints
> proxmox-backup-manager: add CLI for sendmail endpoints
> proxmox-backup-manager: add CLI for SMTP endpoints
>
> Cargo.toml | 3 +
> Makefile | 5 +-
> debian/proxmox-backup-server.install | 29 +
> pbs-api-types/src/datastore.rs | 22 +
> pbs-api-types/src/jobs.rs | 8 +-
> pbs-config/Cargo.toml | 1 +
> pbs-config/src/acl.rs | 3 +-
> pbs-config/src/lib.rs | 1 +
> pbs-config/src/notifications.rs | 41 +
> src/api2/config/datastore.rs | 9 +
> src/api2/config/mod.rs | 2 +
> src/api2/config/notifications/gotify.rs | 190 +++++
> src/api2/config/notifications/matchers.rs | 170 ++++
> src/api2/config/notifications/mod.rs | 175 ++++
> src/api2/config/notifications/sendmail.rs | 178 ++++
> src/api2/config/notifications/smtp.rs | 191 +++++
> src/api2/config/notifications/targets.rs | 63 ++
> src/api2/config/tape_backup_job.rs | 8 +
> src/api2/pull.rs | 10 +-
> src/api2/tape/backup.rs | 62 +-
> src/api2/tape/restore.rs | 46 +-
> src/bin/proxmox-backup-api.rs | 11 +
> src/bin/proxmox-backup-manager.rs | 2 +
> src/bin/proxmox-backup-proxy.rs | 1 +
> src/bin/proxmox_backup_manager/mod.rs | 2 +
> .../notifications/gotify.rs | 93 +++
> .../notifications/matchers.rs | 93 +++
> .../notifications/mod.rs | 21 +
> .../notifications/sendmail.rs | 94 +++
> .../notifications/smtp.rs | 96 +++
> .../notifications/targets.rs | 51 ++
> src/server/email_notifications.rs | 763 ------------------
> src/server/gc_job.rs | 10 +-
> src/server/mod.rs | 4 +-
> src/server/notifications.rs | 544 +++++++++++++
> src/server/verify_job.rs | 10 +-
> src/tape/drive/mod.rs | 22 +-
> src/tape/mod.rs | 27 +
> src/tape/pool_writer/mod.rs | 11 +-
> templates/Makefile | 41 +
> templates/default/acme-err-body.txt.hbs | 7 +
> templates/default/acme-err-subject.txt.hbs | 1 +
> templates/default/gc-err-body.txt.hbs | 8 +
> templates/default/gc-err-subject.txt.hbs | 1 +
> templates/default/gc-ok-body.txt.hbs | 23 +
> templates/default/gc-ok-subject.txt.hbs | 1 +
> .../default/package-updates-body.txt.hbs | 8 +
> .../default/package-updates-subject.txt.hbs | 1 +
> templates/default/prune-err-body.txt.hbs | 10 +
> templates/default/prune-err-subject.txt.hbs | 1 +
> templates/default/prune-ok-body.txt.hbs | 10 +
> templates/default/prune-ok-subject.txt.hbs | 1 +
> templates/default/sync-err-body.txt.hbs | 14 +
> templates/default/sync-err-subject.txt.hbs | 5 +
> templates/default/sync-ok-body.txt.hbs | 14 +
> templates/default/sync-ok-subject.txt.hbs | 5 +
> .../default/tape-backup-err-body.txt.hbs | 26 +
> .../default/tape-backup-err-subject.txt.hbs | 5 +
> templates/default/tape-backup-ok-body.txt.hbs | 27 +
> .../default/tape-backup-ok-subject.txt.hbs | 5 +
> templates/default/tape-load-body.txt.hbs | 15 +
> templates/default/tape-load-subject.txt.hbs | 1 +
> templates/default/test-body.html.hbs | 1 +
> templates/default/test-body.txt.hbs | 1 +
> templates/default/test-subject.txt.hbs | 1 +
> templates/default/verify-err-body.txt.hbs | 14 +
> templates/default/verify-err-subject.txt.hbs | 1 +
> templates/default/verify-ok-body.txt.hbs | 10 +
> templates/default/verify-ok-subject.txt.hbs | 1 +
> www/Makefile | 1 +
> www/NavigationTree.js | 6 +
> www/Utils.js | 16 +
> www/config/NotificationConfigView.js | 11 +
> www/datastore/OptionView.js | 15 +
> www/form/PermissionPathSelector.js | 1 +
> www/tape/window/TapeBackup.js | 25 +
> www/tape/window/TapeBackupJob.js | 24 +
> www/tape/window/TapeRestore.js | 17 +
> www/window/DataStoreEdit.js | 13 +
> www/window/NotifyOptions.js | 38 +
> 80 files changed, 2630 insertions(+), 868 deletions(-)
> create mode 100644 pbs-config/src/notifications.rs
> create mode 100644 src/api2/config/notifications/gotify.rs
> create mode 100644 src/api2/config/notifications/matchers.rs
> create mode 100644 src/api2/config/notifications/mod.rs
> create mode 100644 src/api2/config/notifications/sendmail.rs
> create mode 100644 src/api2/config/notifications/smtp.rs
> create mode 100644 src/api2/config/notifications/targets.rs
> create mode 100644 src/bin/proxmox_backup_manager/notifications/gotify.rs
> create mode 100644 src/bin/proxmox_backup_manager/notifications/matchers.rs
> create mode 100644 src/bin/proxmox_backup_manager/notifications/mod.rs
> create mode 100644 src/bin/proxmox_backup_manager/notifications/sendmail.rs
> create mode 100644 src/bin/proxmox_backup_manager/notifications/smtp.rs
> create mode 100644 src/bin/proxmox_backup_manager/notifications/targets.rs
> delete mode 100644 src/server/email_notifications.rs
> create mode 100644 src/server/notifications.rs
> create mode 100644 templates/Makefile
> create mode 100644 templates/default/acme-err-body.txt.hbs
> create mode 100644 templates/default/acme-err-subject.txt.hbs
> create mode 100644 templates/default/gc-err-body.txt.hbs
> create mode 100644 templates/default/gc-err-subject.txt.hbs
> create mode 100644 templates/default/gc-ok-body.txt.hbs
> create mode 100644 templates/default/gc-ok-subject.txt.hbs
> create mode 100644 templates/default/package-updates-body.txt.hbs
> create mode 100644 templates/default/package-updates-subject.txt.hbs
> create mode 100644 templates/default/prune-err-body.txt.hbs
> create mode 100644 templates/default/prune-err-subject.txt.hbs
> create mode 100644 templates/default/prune-ok-body.txt.hbs
> create mode 100644 templates/default/prune-ok-subject.txt.hbs
> create mode 100644 templates/default/sync-err-body.txt.hbs
> create mode 100644 templates/default/sync-err-subject.txt.hbs
> create mode 100644 templates/default/sync-ok-body.txt.hbs
> create mode 100644 templates/default/sync-ok-subject.txt.hbs
> create mode 100644 templates/default/tape-backup-err-body.txt.hbs
> create mode 100644 templates/default/tape-backup-err-subject.txt.hbs
> create mode 100644 templates/default/tape-backup-ok-body.txt.hbs
> create mode 100644 templates/default/tape-backup-ok-subject.txt.hbs
> create mode 100644 templates/default/tape-load-body.txt.hbs
> create mode 100644 templates/default/tape-load-subject.txt.hbs
> create mode 100644 templates/default/test-body.html.hbs
> create mode 100644 templates/default/test-body.txt.hbs
> create mode 100644 templates/default/test-subject.txt.hbs
> create mode 100644 templates/default/verify-err-body.txt.hbs
> create mode 100644 templates/default/verify-err-subject.txt.hbs
> create mode 100644 templates/default/verify-ok-body.txt.hbs
> create mode 100644 templates/default/verify-ok-subject.txt.hbs
> create mode 100644 www/config/NotificationConfigView.js
>
>
> Summary over all repositories:
> 80 files changed, 2630 insertions(+), 868 deletions(-)
>
--
- Lukas
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel