[PATCH many 00/29] notifications: add nested match expressions

[Lukas Wagner <l.wagner@proxmox.com>]

NOTE: I have a couple of other patches planned for the notification stack (**);
so I guess it would make sense wait a bit before applying this -- just to avoid
too many version bumps and too much maintainer churn.

This resolves a long-standing limitation of notification matchers. Up until
now, notification match rules could only be one level deep, so one top-level
combinator (all/any) and an arbitrary number of match rules as direct children.
This severely limits which kind of matching behaviors can be represented.

The notification stack was always designed with the possibility of arbitrarily
nested match rules, however, due to limitations in our API and config formats
(lacking support for nested data structures), this was not implemented yet.

A prior attempt to resolve the limitations imposed by the lack of nesting was
to add 'sub-matchers', where one matcher could evaluate the result of another
match. After some consideration, I decided not to pursue this approach further,
due to concerns about the UX of that approach [1].

This new series chooses a different approach. Nested match rules can now be
represented by a single matcher in the config. The lack of support for nested
data structures in the API was circumvented by storing a serialized form of the
'rule tree' as a JSON blob in a single new configuration parameter
'expression'. If this new key is set, the old 'match-*', 'invert-match' and
'mode' keys are ignored; 'expression' is intended as a replacement for these
keys. I have not yet marked the old keys as deprecated. Theoretically these can
stay supported for now, however when editing a matcher via the UI, the old keys
will always be translated into an equivalent 'expression'. For any new products
receiving the notification stack, we can drop the support for the old keys by
compiling proxmox-notify without the 'legacy-matchers' flag.

The expression evaluation logic was implemented in a new generic crate
'proxmox-match-expression'. 'proxmox-notify' is the first user of this crate,
however it is generic enough to be potentially useful in other contexts as
well.

Bumps:
  - pve-rs needs proxmox-notify bumped
  - pve-manager needs pve-rs and widget-toolkit bumped
  - proxmox-backup needs widget-toolkit and proxmox-notify bumped
  - proxmox-notify needs a first release of proxmox-match-expression
  
  - proxmox-mail-forward needs to be rebuilt against bumped proxmox-notify (unless
    it has already been changed to not call into proxmox-notify anymore, see below)

[1] https://lore.proxmox.com/pbs-devel/20250521142309.264719-1-l.wagner@proxmox.com/


(**): Some of the things I want to improve/implement:
   - Migrate PVE to use a worker-based approach, same as in PBS
     -> once that is done, we can fully migrate proxmox-mail-forward
        to use the worker/spool dir approach, instead of using proxmox-notify
        directly
   - Introduce a notification history
   - Refactor error types in proxmox-notify a bit

Also, Arthur's XOAUTH2 patch series should probably be applied in one go with
my efforts. He probably needs to rebase on top of my changes anyway.


proxmox:

Lukas Wagner (15):
  add new proxmox-match-expression crate
  notify: promote matcher to dir-style module
  notify: fix doc comment
  notify: matcher: break out severity matcher into submodule
  notify: matcher: break out field matcher into submodule
  notify: matcher: break out calendar matcher into submodule
  notify: matcher: calendar: add basic unit test
  notify: matcher: add InlineSeverityMatcher
  notify: matcher: add InlineFieldMatcher
  notify: matcher: add InlineCalendarMatcher
  notify: matcher: add expression support
  notify: api: support new expression parameter
  notify: api: add `get_matcher_as_expression`
  notify: migrate PBS's and PVE's default matcher to expression syntax
  notify: move legacy matcher keys behind feature flag

 Cargo.toml                                    |   3 +
 proxmox-match-expression/Cargo.toml           |  18 +
 proxmox-match-expression/debian/changelog     |   6 +
 proxmox-match-expression/debian/control       |  34 ++
 proxmox-match-expression/debian/copyright     |  18 +
 proxmox-match-expression/debian/debcargo.toml |   7 +
 proxmox-match-expression/src/lib.rs           | 511 +++++++++++++++++
 proxmox-notify/Cargo.toml                     |   3 +
 proxmox-notify/src/api/matcher.rs             | 248 +++++++-
 proxmox-notify/src/context/mod.rs             |   4 +-
 proxmox-notify/src/context/pbs.rs             |  87 ++-
 proxmox-notify/src/context/pve.rs             |  67 ++-
 proxmox-notify/src/context/test.rs            |   9 +-
 proxmox-notify/src/lib.rs                     |  29 +-
 proxmox-notify/src/matcher.rs                 | 532 ------------------
 proxmox-notify/src/matcher/calendar.rs        | 150 +++++
 proxmox-notify/src/matcher/expression.rs      |  84 +++
 proxmox-notify/src/matcher/field.rs           | 233 ++++++++
 proxmox-notify/src/matcher/mod.rs             | 401 +++++++++++++
 proxmox-notify/src/matcher/severity.rs        | 105 ++++
 20 files changed, 1953 insertions(+), 596 deletions(-)
 create mode 100644 proxmox-match-expression/Cargo.toml
 create mode 100644 proxmox-match-expression/debian/changelog
 create mode 100644 proxmox-match-expression/debian/control
 create mode 100644 proxmox-match-expression/debian/copyright
 create mode 100644 proxmox-match-expression/debian/debcargo.toml
 create mode 100644 proxmox-match-expression/src/lib.rs
 delete mode 100644 proxmox-notify/src/matcher.rs
 create mode 100644 proxmox-notify/src/matcher/calendar.rs
 create mode 100644 proxmox-notify/src/matcher/expression.rs
 create mode 100644 proxmox-notify/src/matcher/field.rs
 create mode 100644 proxmox-notify/src/matcher/mod.rs
 create mode 100644 proxmox-notify/src/matcher/severity.rs


proxmox-widget-toolkit:

Lukas Wagner (4):
  notification: increase matcher window width
  notifications: matcher: add support for match expressions
  notification: matcher: add better calendar editor
  notifications: matcher: consistently use title case for UI elements

 src/Makefile                                  |   1 +
 src/Schema.js                                 |  10 +
 src/css/ext6-pmx.css                          |  27 +
 .../NotificationMatchExpressionEditPanel.js   | 786 ++++++++++++++++++
 src/proxmox-dark/scss/extjs/_treepanel.scss   |   5 +
 src/proxmox-dark/scss/proxmox/_general.scss   |   4 +
 src/window/NotificationMatcherEdit.js         | 319 ++++++-
 7 files changed, 1133 insertions(+), 19 deletions(-)
 create mode 100644 src/panel/NotificationMatchExpressionEditPanel.js


proxmox-backup:

Lukas Wagner (3):
  notification: opt into 'legacy-matchers' feature in proxmox-notify
  api: notification: add 'migrate-to-expression' parameter to
    get_matcher
  ui: notification: enable new matcher UI

 Cargo.toml                                |  2 +-
 src/api2/config/notifications/matchers.rs | 18 ++++++++++++++++--
 www/Utils.js                              |  3 +++
 3 files changed, 20 insertions(+), 3 deletions(-)


proxmox-perl-rs:

Lukas Wagner (3):
  notify: matcher: pass matcher config / updater directly
  notify: opt into 'legacy-matchers' feature in proxmox-notify
  notify: add 'migrate_to_expression' parameter for get_matcher

 common/src/bindings/notify.rs | 207 ++++++----------------------------
 pve-rs/Cargo.toml             |   2 +-
 2 files changed, 33 insertions(+), 176 deletions(-)


pve-manager:

Lukas Wagner (4):
  api: notification: pass config/updater directly to rust bindings
  api: notification: get_matcher: add 'migrate-to-expression' parameter
  api: notification: add 'expression' to matcher parameter schema
  ui: notification: enable new matcher UI

 PVE/API2/Cluster/Notifications.pm | 165 ++++++++----------------------
 www/manager6/Utils.js             |   2 +
 2 files changed, 43 insertions(+), 124 deletions(-)


Summary over all repositories:
  34 files changed, 3182 insertions(+), 918 deletions(-)

-- 
Generated by murpp 0.12.0