[pve-devel] [PATCH many 00/27] overhaul notification system, use matchers instead of filters

[Lukas Wagner <l.wagner@proxmox.com>]

This series replaces notification filters and groups with notification
matchers. Instead of having a per-notification event target/policy 
setting (at the moment stored in datacenter.cfg and jobs.cfg), this 
shifts the routing part into the matcher completely.

Config example, I think this demonstrates the principle quite nicely:

sendmail: default-target
  mailto-user root@pam

matcher: fencing-for-node
  mode all       # all match-directives have to match, default
  match-field exact:hostname=pve.example.com
  match-field exact:type=fencing
  target default-target


--> Send all fencing notifications for a certain host to a certain 
target.

Right now, there are three different match-directives:
  - match-field: exact/regex match for notification metadata fields
  - match-severity: match notification severities (info,notice,warning,error)
  - match-calender: match notification timestamp
    example: match-calendar mon..fri 8-12

The old target/policy based notification was already in the pvetest repository.
Thus we take special care that there is no breakage when the notification
system encounters old settings/configuration keys. It will clean them
out/migrate them if possible.

These changes also modify pve-manager's postinst hook. It will now 
create a default config if /etc/pve/notifications.cfg does not exist yet.

What I tested:
  - Made sure existing notifications continue to work
    (replication/fencing in a cluster setup, backups, system updates)
  - Made sure that the 'legacy' mailto parameter for backups also works
  - Tested the new UI for notification matchers
  - Tested postinst hook for pve-manager
  - Tested whether old config keys for filters and groups break anything

@TESTERS:
I have built packages for these changes to ease testing, they are 
on nasi: packages/notifications

Followup work in the near future:
  - Adapt documentation, this will follow asap
  - UI code for notification matcher config is a bit messy, I will
    send a cleanup-patch - main focus right now was to get it working
  - Mark 'mailto' in backup jobs as deprecated in UI - while also
    migrating automatically to the new system (create an endpoint/matcher
    when creating/updating a backup job)



proxmox:

Lukas Wagner (6):
  notify: introduce Error::Generic
  notify: factor out notification content into its own type
  notify: replace filters and groups with matcher-based system
  notify: add calendar matcher
  notify: matcher: introduce common trait for match directives
  notify: let a matcher always match if it has no matching directives

 proxmox-notify/Cargo.toml                |   2 +
 proxmox-notify/examples/render.rs        |   4 +-
 proxmox-notify/src/api/common.rs         |   6 +-
 proxmox-notify/src/api/filter.rs         | 231 -----------
 proxmox-notify/src/api/gotify.rs         |  16 -
 proxmox-notify/src/api/group.rs          | 259 ------------
 proxmox-notify/src/api/matcher.rs        | 260 ++++++++++++
 proxmox-notify/src/api/mod.rs            | 115 +-----
 proxmox-notify/src/api/sendmail.rs       |  15 -
 proxmox-notify/src/config.rs             |  34 +-
 proxmox-notify/src/endpoints/gotify.rs   |  41 +-
 proxmox-notify/src/endpoints/sendmail.rs |  76 ++--
 proxmox-notify/src/filter.rs             | 193 +--------
 proxmox-notify/src/group.rs              |  40 +-
 proxmox-notify/src/lib.rs                | 387 ++++++++----------
 proxmox-notify/src/matcher.rs            | 484 +++++++++++++++++++++++
 proxmox-notify/src/renderer/mod.rs       |  15 +-
 proxmox-notify/src/schema.rs             |  11 +-
 18 files changed, 1046 insertions(+), 1143 deletions(-)
 delete mode 100644 proxmox-notify/src/api/filter.rs
 delete mode 100644 proxmox-notify/src/api/group.rs
 create mode 100644 proxmox-notify/src/api/matcher.rs
 create mode 100644 proxmox-notify/src/matcher.rs


proxmox-perl-rs:

Lukas Wagner (1):
  notify: adapt to new matcher-based notification routing

 common/src/notify.rs | 167 +++++++++++++------------------------------
 1 file changed, 50 insertions(+), 117 deletions(-)


pve-cluster:

Lukas Wagner (1):
  notify: adapt to matcher based notification system

 src/PVE/Notify.pm | 101 +++++++++++++++++++++-------------------------
 1 file changed, 47 insertions(+), 54 deletions(-)


pve-guest-common:

Lukas Wagner (1):
  vzdump: deprecate mailto/mailnotification/notification-{target,policy}

 src/PVE/VZDump/Common.pm | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)


pve-ha-manager:

Lukas Wagner (1):
  env: switch to matcher-based notification system

 src/PVE/HA/Env/PVE2.pm   | 10 ++--------
 src/PVE/HA/NodeStatus.pm | 11 +++++++++--
 2 files changed, 11 insertions(+), 10 deletions(-)


pve-manager:

Lukas Wagner (10):
  api: notification: remove notification groups
  api: notification: add new matcher-based notification API
  ui: dc: remove unneeded notification events panel
  vzdump: adapt to new matcher based notification system
  api: apt: adapt to matcher-based notifications
  api: replication: adapt to matcher-based notification system
  debian: postinst: create notifications.cfg if it does not exist
  test: fix vzdump notification test
  ui: vzdump: remove left-overs from target/policy based notifications
  ui: dc: config: show notification panel again

 PVE/API2/APT.pm                               |  27 +-
 PVE/API2/Cluster/Notifications.pm             | 462 ++++--------------
 PVE/API2/Replication.pm                       |  25 +-
 PVE/API2/VZDump.pm                            |   8 +-
 PVE/VZDump.pm                                 |  40 +-
 debian/postinst                               |  28 ++
 test/vzdump_notification_test.pl              |   6 +-
 www/manager6/Makefile                         |   4 -
 www/manager6/dc/Backup.js                     |  81 +--
 www/manager6/dc/Config.js                     |  28 +-
 www/manager6/dc/NotificationEvents.js         | 276 -----------
 .../form/NotificationPolicySelector.js        |   1 -
 www/manager6/window/Backup.js                 |  35 +-
 13 files changed, 186 insertions(+), 835 deletions(-)
 delete mode 100644 www/manager6/dc/NotificationEvents.js


proxmox-widget-toolkit:

Lukas Wagner (7):
  notification ui: add target selector for matcher
  notification ui: remove filter setting for targets
  notification ui: remove notification groups
  notification ui: rename filter to matcher
  notification: matcher: add UI for matcher editing
  notification ui: unprotected mailto-root target
  noficiation: matcher edit: make 'field' an editable combobox

 src/Makefile                            |    4 +-
 src/Schema.js                           |    5 -
 src/data/model/NotificationConfig.js    |    2 +-
 src/form/NotificationFilterSelector.js  |   58 --
 src/panel/GotifyEditPanel.js            |    9 -
 src/panel/NotificationConfigView.js     |   36 +-
 src/panel/NotificationGroupEditPanel.js |  183 ----
 src/panel/SendmailEditPanel.js          |    9 -
 src/window/NotificationFilterEdit.js    |  109 ---
 src/window/NotificationMatcherEdit.js   | 1036 +++++++++++++++++++++++
 10 files changed, 1051 insertions(+), 400 deletions(-)
 delete mode 100644 src/form/NotificationFilterSelector.js
 delete mode 100644 src/panel/NotificationGroupEditPanel.js
 delete mode 100644 src/window/NotificationFilterEdit.js
 create mode 100644 src/window/NotificationMatcherEdit.js


Summary over all repositories:
  46 files changed, 2398 insertions(+), 2568 deletions(-)

-- 
murpp v0.4.0