Re: [pbs-devel] [PATCH proxmox-backup 00/33] integrate notification system

[Lukas Wagner <l.wagner@proxmox.com>]

Superseded-by: https://lists.proxmox.com/pipermail/pbs-devel/2024-April/008639.html

On  2024-04-12 12:05, Lukas Wagner wrote:
> These patches integrate the notification system which was introduced in Proxmox VE
> in 8.1 into Proxmox Backup Server.
> 
> Some highlights/noteworthy details from the series:
>   - notification template files are installed in /usr/share/proxmox-backup/templates/default
>     and are rendered when a notification is sent
>   - since sending notifications needs to be done from a privileged context
>     (to read protected passwords/tokens from the priv config file),
>     we queue notifications to be sent in /var/lib/proxmox-backup/notifications
>     and periodically send any queued notifications via a worker in the
>     privileged process
>   - The API endpoint paths are prefixed with /config/notifications
>   - API endpoints which read/modify notification system config require
>     Sys.Audit (read) or Sys.Modify (modify) permissions on /system/notifications
> 
>   - tape-{backup,restore} settings and datastore options now have a
>     'notification-mode' parameter, which allows to choose between the
>     'legacy' behavior (sendmail to a selected user's email address) and
>     the event-based notification system. If the parameter is not set, we
>     default to the legacy behavior in order to keep existing behavior
>     as is for now. For new datastores/tape backup jobs created
>     interactively from teh UI, we set the parameter to 'notification-system'
>     and opt in into the new system by default.
> 
> Still missing/rough edges:
>   - documentation (will follow soon)
>   - Datastore option view in UI could be improved. When 'notification-mode' is
>     set to 'notification-system', we should indicate that 'mailto-user' and the
>     other notification settings have no effect. Already did that in the edit
>     window, but in the grid panel I did not find a quick way to do that.
> 
>   - still needs a bit more testing (e.g. ACME notifications) - would greatly
>     appreciate people helping here!
> 
> Open questions:
>   - Right now the 'default-matcher' matches every notification, same as in PVE.
>     Up until now, datastore prune notifications were only sent on errors, so
>     maybe we should also have the default-matcher work the same?
>     The default matcher would then probably be:
>       - !All
>           - severity: error
>           - match-field: exact:type=prune
> 
> This patch series requires the patches for 'proxmox' from
> https://lists.proxmox.com/pipermail/pve-devel/2024-April/062708.html
> to be applied.
> 
> The dependency for proxmox-widget-toolkit should be bumped to at least
> 4.1.4 (we need: "utils: add mechanism to add and override translatable notification event
> descriptions in the product specific UIs")
> Also, while not strictly needed, the patches for widget-toolkit from
> https://lists.proxmox.com/pipermail/pve-devel/2024-February/061992.html
> make creating matchers much easier - the required API endpoints were
> already implemented by this series.
> 
> proxmox-backup:
> 
> Lukas Wagner (33):
>   pbs-config: add module for loading notification config
>   server: rename email_notifications module to notifications
>   notifications: allow sending notifications via proxmox_notify
>   buildsys: install templates for test notifications
>   pbs-config: acl: add /system/notifications as known ACL path
>   api: add endpoints for querying/testing notification targets
>   api: add endpoints for notification matchers
>   api: add endpoints for sendmail targets
>   api: add endpoints for smtp targets
>   api: add endpoints for gotify targets
>   api: add endpoints for querying known notification values/fields
>   api-types: api: datatore: add notification-mode parameter
>   api-types: api: tape: add notification-mode parameter
>   server: notifications: send GC notifications via notification system
>   server: notifications: send prune notifications via notification
>     system
>   server: notifications: send verify notifications via notification
>     system
>   server: notifications: send sync notifications via notification system
>   server: notifications: send update notifications via notification
>     system
>   server: notifications: send acme notifications via notification system
>   server: notifications: send tape notifications via notification system
>   ui: add notification config panel
>   ui: tape backup job: add selector for notification-mode
>   ui: tape backup: add selector for 'notification-mode'
>   ui: tape restore: add 'notification-mode' parameter
>   ui: datastore options: add 'notification-mode' parameter
>   ui: utils: add overrides for known notification metadata fields/values
>   ui: datastore edit: make new stores use notification system by default
>   ui: permissions paths: add /system/notifications to combobox
>   proxmox-backup-manager: add CLI for notification targets
>   proxmox-backup-manager: add CLI for notification matchers
>   proxmox-backup-manager: add CLI for gotify endpoints
>   proxmox-backup-manager: add CLI for sendmail endpoints
>   proxmox-backup-manager: add CLI for SMTP endpoints
> 
>  Cargo.toml                                    |   3 +
>  Makefile                                      |   5 +-
>  debian/proxmox-backup-server.install          |  29 +
>  pbs-api-types/src/datastore.rs                |  22 +
>  pbs-api-types/src/jobs.rs                     |   8 +-
>  pbs-config/Cargo.toml                         |   1 +
>  pbs-config/src/acl.rs                         |   3 +-
>  pbs-config/src/lib.rs                         |   1 +
>  pbs-config/src/notifications.rs               |  41 +
>  src/api2/config/datastore.rs                  |   9 +
>  src/api2/config/mod.rs                        |   2 +
>  src/api2/config/notifications/gotify.rs       | 190 +++++
>  src/api2/config/notifications/matchers.rs     | 170 ++++
>  src/api2/config/notifications/mod.rs          | 175 ++++
>  src/api2/config/notifications/sendmail.rs     | 178 ++++
>  src/api2/config/notifications/smtp.rs         | 191 +++++
>  src/api2/config/notifications/targets.rs      |  63 ++
>  src/api2/config/tape_backup_job.rs            |   8 +
>  src/api2/pull.rs                              |  10 +-
>  src/api2/tape/backup.rs                       |  62 +-
>  src/api2/tape/restore.rs                      |  46 +-
>  src/bin/proxmox-backup-api.rs                 |  11 +
>  src/bin/proxmox-backup-manager.rs             |   2 +
>  src/bin/proxmox-backup-proxy.rs               |   1 +
>  src/bin/proxmox_backup_manager/mod.rs         |   2 +
>  .../notifications/gotify.rs                   |  93 +++
>  .../notifications/matchers.rs                 |  93 +++
>  .../notifications/mod.rs                      |  21 +
>  .../notifications/sendmail.rs                 |  94 +++
>  .../notifications/smtp.rs                     |  96 +++
>  .../notifications/targets.rs                  |  51 ++
>  src/server/email_notifications.rs             | 763 ------------------
>  src/server/gc_job.rs                          |  10 +-
>  src/server/mod.rs                             |   4 +-
>  src/server/notifications.rs                   | 544 +++++++++++++
>  src/server/verify_job.rs                      |  10 +-
>  src/tape/drive/mod.rs                         |  22 +-
>  src/tape/mod.rs                               |  27 +
>  src/tape/pool_writer/mod.rs                   |  11 +-
>  templates/Makefile                            |  41 +
>  templates/default/acme-err-body.txt.hbs       |   7 +
>  templates/default/acme-err-subject.txt.hbs    |   1 +
>  templates/default/gc-err-body.txt.hbs         |   8 +
>  templates/default/gc-err-subject.txt.hbs      |   1 +
>  templates/default/gc-ok-body.txt.hbs          |  23 +
>  templates/default/gc-ok-subject.txt.hbs       |   1 +
>  .../default/package-updates-body.txt.hbs      |   8 +
>  .../default/package-updates-subject.txt.hbs   |   1 +
>  templates/default/prune-err-body.txt.hbs      |  10 +
>  templates/default/prune-err-subject.txt.hbs   |   1 +
>  templates/default/prune-ok-body.txt.hbs       |  10 +
>  templates/default/prune-ok-subject.txt.hbs    |   1 +
>  templates/default/sync-err-body.txt.hbs       |  14 +
>  templates/default/sync-err-subject.txt.hbs    |   5 +
>  templates/default/sync-ok-body.txt.hbs        |  14 +
>  templates/default/sync-ok-subject.txt.hbs     |   5 +
>  .../default/tape-backup-err-body.txt.hbs      |  26 +
>  .../default/tape-backup-err-subject.txt.hbs   |   5 +
>  templates/default/tape-backup-ok-body.txt.hbs |  27 +
>  .../default/tape-backup-ok-subject.txt.hbs    |   5 +
>  templates/default/tape-load-body.txt.hbs      |  15 +
>  templates/default/tape-load-subject.txt.hbs   |   1 +
>  templates/default/test-body.html.hbs          |   1 +
>  templates/default/test-body.txt.hbs           |   1 +
>  templates/default/test-subject.txt.hbs        |   1 +
>  templates/default/verify-err-body.txt.hbs     |  14 +
>  templates/default/verify-err-subject.txt.hbs  |   1 +
>  templates/default/verify-ok-body.txt.hbs      |  10 +
>  templates/default/verify-ok-subject.txt.hbs   |   1 +
>  www/Makefile                                  |   1 +
>  www/NavigationTree.js                         |   6 +
>  www/Utils.js                                  |  16 +
>  www/config/NotificationConfigView.js          |  11 +
>  www/datastore/OptionView.js                   |  15 +
>  www/form/PermissionPathSelector.js            |   1 +
>  www/tape/window/TapeBackup.js                 |  25 +
>  www/tape/window/TapeBackupJob.js              |  24 +
>  www/tape/window/TapeRestore.js                |  17 +
>  www/window/DataStoreEdit.js                   |  13 +
>  www/window/NotifyOptions.js                   |  38 +
>  80 files changed, 2630 insertions(+), 868 deletions(-)
>  create mode 100644 pbs-config/src/notifications.rs
>  create mode 100644 src/api2/config/notifications/gotify.rs
>  create mode 100644 src/api2/config/notifications/matchers.rs
>  create mode 100644 src/api2/config/notifications/mod.rs
>  create mode 100644 src/api2/config/notifications/sendmail.rs
>  create mode 100644 src/api2/config/notifications/smtp.rs
>  create mode 100644 src/api2/config/notifications/targets.rs
>  create mode 100644 src/bin/proxmox_backup_manager/notifications/gotify.rs
>  create mode 100644 src/bin/proxmox_backup_manager/notifications/matchers.rs
>  create mode 100644 src/bin/proxmox_backup_manager/notifications/mod.rs
>  create mode 100644 src/bin/proxmox_backup_manager/notifications/sendmail.rs
>  create mode 100644 src/bin/proxmox_backup_manager/notifications/smtp.rs
>  create mode 100644 src/bin/proxmox_backup_manager/notifications/targets.rs
>  delete mode 100644 src/server/email_notifications.rs
>  create mode 100644 src/server/notifications.rs
>  create mode 100644 templates/Makefile
>  create mode 100644 templates/default/acme-err-body.txt.hbs
>  create mode 100644 templates/default/acme-err-subject.txt.hbs
>  create mode 100644 templates/default/gc-err-body.txt.hbs
>  create mode 100644 templates/default/gc-err-subject.txt.hbs
>  create mode 100644 templates/default/gc-ok-body.txt.hbs
>  create mode 100644 templates/default/gc-ok-subject.txt.hbs
>  create mode 100644 templates/default/package-updates-body.txt.hbs
>  create mode 100644 templates/default/package-updates-subject.txt.hbs
>  create mode 100644 templates/default/prune-err-body.txt.hbs
>  create mode 100644 templates/default/prune-err-subject.txt.hbs
>  create mode 100644 templates/default/prune-ok-body.txt.hbs
>  create mode 100644 templates/default/prune-ok-subject.txt.hbs
>  create mode 100644 templates/default/sync-err-body.txt.hbs
>  create mode 100644 templates/default/sync-err-subject.txt.hbs
>  create mode 100644 templates/default/sync-ok-body.txt.hbs
>  create mode 100644 templates/default/sync-ok-subject.txt.hbs
>  create mode 100644 templates/default/tape-backup-err-body.txt.hbs
>  create mode 100644 templates/default/tape-backup-err-subject.txt.hbs
>  create mode 100644 templates/default/tape-backup-ok-body.txt.hbs
>  create mode 100644 templates/default/tape-backup-ok-subject.txt.hbs
>  create mode 100644 templates/default/tape-load-body.txt.hbs
>  create mode 100644 templates/default/tape-load-subject.txt.hbs
>  create mode 100644 templates/default/test-body.html.hbs
>  create mode 100644 templates/default/test-body.txt.hbs
>  create mode 100644 templates/default/test-subject.txt.hbs
>  create mode 100644 templates/default/verify-err-body.txt.hbs
>  create mode 100644 templates/default/verify-err-subject.txt.hbs
>  create mode 100644 templates/default/verify-ok-body.txt.hbs
>  create mode 100644 templates/default/verify-ok-subject.txt.hbs
>  create mode 100644 www/config/NotificationConfigView.js
> 
> 
> Summary over all repositories:
>   80 files changed, 2630 insertions(+), 868 deletions(-)
> 

-- 
- Lukas


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel