From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 68F8E1FF0B0 for ; Fri, 09 Oct 2026 14:59:58 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 617622191C; Fri, 09 Oct 2026 14:57:43 +0200 (CEST) From: Lukas Wagner To: pbs-devel@lists.proxmox.com, pve-devel@lists.proxmox.com Subject: [PATCH many v3 00/39] notifications: add nested match expressions Date: Fri, 9 Oct 2026 14:56:07 +0200 Message-ID: <20261009125646.284673-1-l.wagner@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1791550609281 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.336 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: E2JGBMH277OICHCO5VGLVAN5FXOH6HAJ X-Message-ID-Hash: E2JGBMH277OICHCO5VGLVAN5FXOH6HAJ X-MailFrom: l.wagner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: NOTE: The last patch for pve-manager needs to be updated depending on which version of pve-manager includes these series! (see patch note) NOTE: I have a couple of other patches planned for the notification stack (**); so I guess it would make sense wait a bit before applying this -- just to avoid too many version bumps and too much maintainer churn. I have a patch series already in the works that builds on top of this one, so I would prefer to perform any non-trivial changes to this feature as follow-ups; rebasing the new series I'm working on right now is a huge pain, as it moves a lot of code around in proxmox-notify (preparations for proxmox-notify v2, essentially) This resolves a long-standing limitation of notification matchers. Up until now, notification match rules could only be one level deep, so one top-level combinator (all/any) and an arbitrary number of match rules as direct children. This severely limits which kind of matching behaviors can be represented. The notification stack was always designed with the possibility of arbitrarily nested match rules, however, due to limitations in our API and config formats (lacking support for nested data structures), this was not implemented yet. A prior attempt to resolve the limitations imposed by the lack of nesting was to add 'sub-matchers', where one matcher could evaluate the result of another matcher. After some consideration, I decided not to pursue this approach further, due to concerns about the UX of that approach [1]. This new series chooses a different approach. Nested match rules can now be represented by a single matcher in the config. The lack of support for nested data structures in the API was circumvented by storing a serialized form of the 'rule tree' as a JSON blob in a single new configuration parameter 'expression'. If this new key is set, the old 'match-*', 'invert-match' and 'mode' keys are ignored; 'expression' is intended as a replacement for these keys. I have not yet marked the old keys as deprecated. Theoretically these can stay supported for now, however when editing a matcher via the UI, the old keys will always be translated into an equivalent 'expression'. For any new products receiving the notification stack, we can drop the support for the old keys by compiling proxmox-notify without the 'legacy-matchers' flag. The expression evaluation logic was implemented in a new generic crate 'proxmox-match-expression'. 'proxmox-notify' is the first user of this crate, however it is generic enough to be potentially useful in other contexts as well. Documentation will be added in a later patch series, I will include all relevant changes to the notification stack in one go. Bumps: - pve-rs needs proxmox-notify bumped - pve-cluster (libpve-notify-perl) needs pve-rs and pve-common bumped - pve-manager needs pve-cluster (libpve-notify-perl, libpve-cluster-perl), pve-rs and widget-toolkit bumped - proxmox-backup needs widget-toolkit and proxmox-notify bumped - proxmox-notify needs a first release of proxmox-match-expression - proxmox-mail-forward needs to be rebuilt against bumped proxmox-notify (unless it has already been changed to not call into proxmox-notify anymore, see below) [1] https://lore.proxmox.com/pbs-devel/20250521142309.264719-1-l.wagner@proxmox.com/ (**): Some of the things I want to improve/implement: - Migrate PVE to use a worker-based approach, same as in PBS -> once that is done, we can fully migrate proxmox-mail-forward to use the worker/spool dir approach, instead of using proxmox-notify directly - Introduce a notification history - Refactor error types in proxmox-notify a bit Changes since v1: - Move Proxmox::RS::Notify module to PVE::RS::Notify, since it is only used by PVE - Fix legacy-sendmail mode, the calling code was not adapted to the new API of Proxmox::RS::Notify in v1 - proxmox-match-expression: all-of([]) now evaluates to true, similar to Iterator::all or all([]) in python - proxmox-notify: Don't allow empty all-of/any-of/one-of rules (in both UI and API, proxmox-match-expression still allow them) - Also don't allow empty severity/field matchers (empty arrays of matched values) - Fix slight inconsistency with the default notification matcher for PBS; now prune notifications with severity 'notice' are included once more - Other minor review comments - Overall polish of the new matcher UI - Add hint about nodes being drag&drop'able - Add error field if a node is invalid - Fixed erroreous caldendar string generated by the UI under certain circumenstances Changes since v2: - re-serialize matcher expression on create/update, ensuring that the JSON is minified and stripped of control characters (such as line breaks) - reject creating/updating matchers with expressions if not all nodes in a PVE cluster support it - Properly handle nested 'not' nodes in the UI - In the new calendar edit view, support entering the time in formats such as O:00, 7, 730. The UI will automatically normalize the input to the appropriate 24h format (e.g. 07:30) - Cherry-picked two patches (patch 31, 32) from Fiona [1] for the cluster version check. Left untentionally unfixed: Due to the new representation of the default matcher, a previously serialized but unmodified default matcher might be shown as 'modified' in the UI after the update. This is merely a cosmetic issue that can be rectified by pressing "Reset". Thanks go to Wolfgang and Arthur for their feedback of v1. Thanks also go to Michael Köppl for his feedback on v2. [1] https://lore.proxmox.com/pve-devel/20260918160841.128088-1-f.ebner@proxmox.com/ proxmox: Lukas Wagner (18): add new proxmox-match-expression crate notify: log which matchers matched on a notification notify: promote matcher to dir-style module notify: fix doc comment notify: matcher: break out severity matcher into submodule notify: matcher: break out field matcher into submodule notify: matcher: break out calendar matcher into submodule notify: matcher: calendar: add basic unit test notify: matcher: add InlineSeverityMatcher notify: matcher: add InlineFieldMatcher notify: matcher: add InlineCalendarMatcher notify: matcher: add expression support notify: api: support new expression parameter notify: api: add `get_matcher_as_expression` notify: migrate PBS's and PVE's default matcher to expression syntax notify: move legacy matcher keys behind feature flag notify: matcher: don't allow empty field/severity matchers notify: matcher: re-serialize expression on create/update Cargo.toml | 3 + proxmox-match-expression/Cargo.toml | 18 + proxmox-match-expression/debian/changelog | 6 + proxmox-match-expression/debian/control | 34 ++ proxmox-match-expression/debian/copyright | 18 + proxmox-match-expression/debian/debcargo.toml | 7 + proxmox-match-expression/src/lib.rs | 524 +++++++++++++++++ proxmox-notify/Cargo.toml | 7 +- proxmox-notify/src/api/matcher.rs | 253 ++++++++- proxmox-notify/src/context/mod.rs | 4 +- proxmox-notify/src/context/pbs.rs | 87 ++- proxmox-notify/src/context/pve.rs | 67 ++- proxmox-notify/src/context/test.rs | 9 +- proxmox-notify/src/lib.rs | 29 +- proxmox-notify/src/matcher.rs | 532 ----------------- proxmox-notify/src/matcher/calendar.rs | 158 ++++++ proxmox-notify/src/matcher/expression.rs | 97 ++++ proxmox-notify/src/matcher/field.rs | 249 ++++++++ proxmox-notify/src/matcher/mod.rs | 537 ++++++++++++++++++ proxmox-notify/src/matcher/severity.rs | 119 ++++ 20 files changed, 2159 insertions(+), 599 deletions(-) create mode 100644 proxmox-match-expression/Cargo.toml create mode 100644 proxmox-match-expression/debian/changelog create mode 100644 proxmox-match-expression/debian/control create mode 100644 proxmox-match-expression/debian/copyright create mode 100644 proxmox-match-expression/debian/debcargo.toml create mode 100644 proxmox-match-expression/src/lib.rs delete mode 100644 proxmox-notify/src/matcher.rs create mode 100644 proxmox-notify/src/matcher/calendar.rs create mode 100644 proxmox-notify/src/matcher/expression.rs create mode 100644 proxmox-notify/src/matcher/field.rs create mode 100644 proxmox-notify/src/matcher/mod.rs create mode 100644 proxmox-notify/src/matcher/severity.rs proxmox-widget-toolkit: Lukas Wagner (4): notifications: increase matcher window width notifications: matcher: add support for match expressions notifications: matcher: add better calendar editor notifications: matcher: consistently use title case for UI elements src/Makefile | 1 + src/Schema.js | 10 + src/Utils.js | 2 +- src/css/ext6-pmx.css | 51 + .../NotificationMatchExpressionEditPanel.js | 892 ++++++++++++++++++ src/proxmox-dark/scss/extjs/_treepanel.scss | 5 + src/proxmox-dark/scss/proxmox/_general.scss | 4 + src/window/NotificationMatcherEdit.js | 358 ++++++- 8 files changed, 1303 insertions(+), 20 deletions(-) create mode 100644 src/panel/NotificationMatchExpressionEditPanel.js proxmox-backup: Lukas Wagner (3): notification: opt into 'legacy-matchers' feature in proxmox-notify api: notification: add 'migrate-to-expression' parameter to get_matcher ui: notification: enable new matcher UI Cargo.toml | 2 +- src/api2/config/notifications/matchers.rs | 18 ++++++++++++++++-- www/Utils.js | 3 +++ 3 files changed, 20 insertions(+), 3 deletions(-) proxmox-perl-rs: Lukas Wagner (4): notify: move proxmox-notify bindings from common to pve-rs notify: matcher: pass matcher config / updater directly notify: opt into 'legacy-matchers' feature in proxmox-notify notify: add 'migrate_to_expression' parameter for get_matcher common/pkg/Makefile | 1 - common/src/bindings/mod.rs | 3 - pmg-rs/Cargo.toml | 1 - pve-rs/Cargo.toml | 2 +- pve-rs/Makefile | 1 + pve-rs/src/bindings/mod.rs | 3 + {common => pve-rs}/src/bindings/notify.rs | 212 ++++------------------ 7 files changed, 40 insertions(+), 183 deletions(-) rename {common => pve-rs}/src/bindings/notify.rs (73%) pve-cluster: Fiona Ebner (1): cluster: move pvecfg node version helpers from qemu-server Lukas Wagner (1): notify: adapt to changed module prefix for PVE::RS::Notify src/PVE/Cluster.pm | 30 ++++++++++++++++++++++++++++++ src/PVE/Notify.pm | 4 ++-- 2 files changed, 32 insertions(+), 2 deletions(-) pve-common: Fiona Ebner (1): tools: move version_cmp() helper from qemu-server src/PVE/Tools.pm | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) pve-manager: Lukas Wagner (6): vzdump: adapt to changed module prefix for PVE::RS::Notify api: notification: pass config/updater directly to rust bindings api: notification: get_matcher: add 'migrate-to-expression' parameter api: notification: add 'expression' to matcher parameter schema ui: notification: enable new matcher UI api: notification: reject matchers with expression if not supported by all nodes PVE/API2/Cluster/Notifications.pm | 195 +++++++++++------------------- PVE/VZDump.pm | 21 ++-- www/manager6/Utils.js | 2 + 3 files changed, 84 insertions(+), 134 deletions(-) proxmox-mail-forward: Lukas Wagner (1): cargo: opt into legacy-matchers feature Cargo.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Summary over all repositories: 45 files changed, 3665 insertions(+), 942 deletions(-) -- Generated by murpp 0.12.1