From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id D74A19FA8A for ; Tue, 7 Nov 2023 11:19:33 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B75883017F for ; Tue, 7 Nov 2023 11:19:03 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Tue, 7 Nov 2023 11:19:02 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 78A9A4693B for ; Tue, 7 Nov 2023 11:19:02 +0100 (CET) From: Lukas Wagner To: pve-devel@lists.proxmox.com Date: Tue, 7 Nov 2023 11:18:00 +0100 Message-Id: <20231107101827.340100-1-l.wagner@proxmox.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.017 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: [pve-devel] [PATCH many 00/27] overhaul notification system, use matchers instead of filters X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Nov 2023 10:19:33 -0000 This series replaces notification filters and groups with notification matchers. Instead of having a per-notification event target/policy setting (at the moment stored in datacenter.cfg and jobs.cfg), this shifts the routing part into the matcher completely. Config example, I think this demonstrates the principle quite nicely: sendmail: default-target mailto-user root@pam matcher: fencing-for-node mode all # all match-directives have to match, default match-field exact:hostname=pve.example.com match-field exact:type=fencing target default-target --> Send all fencing notifications for a certain host to a certain target. Right now, there are three different match-directives: - match-field: exact/regex match for notification metadata fields - match-severity: match notification severities (info,notice,warning,error) - match-calender: match notification timestamp example: match-calendar mon..fri 8-12 The old target/policy based notification was already in the pvetest repository. Thus we take special care that there is no breakage when the notification system encounters old settings/configuration keys. It will clean them out/migrate them if possible. These changes also modify pve-manager's postinst hook. It will now create a default config if /etc/pve/notifications.cfg does not exist yet. What I tested: - Made sure existing notifications continue to work (replication/fencing in a cluster setup, backups, system updates) - Made sure that the 'legacy' mailto parameter for backups also works - Tested the new UI for notification matchers - Tested postinst hook for pve-manager - Tested whether old config keys for filters and groups break anything @TESTERS: I have built packages for these changes to ease testing, they are on nasi: packages/notifications Followup work in the near future: - Adapt documentation, this will follow asap - UI code for notification matcher config is a bit messy, I will send a cleanup-patch - main focus right now was to get it working - Mark 'mailto' in backup jobs as deprecated in UI - while also migrating automatically to the new system (create an endpoint/matcher when creating/updating a backup job) proxmox: Lukas Wagner (6): notify: introduce Error::Generic notify: factor out notification content into its own type notify: replace filters and groups with matcher-based system notify: add calendar matcher notify: matcher: introduce common trait for match directives notify: let a matcher always match if it has no matching directives proxmox-notify/Cargo.toml | 2 + proxmox-notify/examples/render.rs | 4 +- proxmox-notify/src/api/common.rs | 6 +- proxmox-notify/src/api/filter.rs | 231 ----------- proxmox-notify/src/api/gotify.rs | 16 - proxmox-notify/src/api/group.rs | 259 ------------ proxmox-notify/src/api/matcher.rs | 260 ++++++++++++ proxmox-notify/src/api/mod.rs | 115 +----- proxmox-notify/src/api/sendmail.rs | 15 - proxmox-notify/src/config.rs | 34 +- proxmox-notify/src/endpoints/gotify.rs | 41 +- proxmox-notify/src/endpoints/sendmail.rs | 76 ++-- proxmox-notify/src/filter.rs | 193 +-------- proxmox-notify/src/group.rs | 40 +- proxmox-notify/src/lib.rs | 387 ++++++++---------- proxmox-notify/src/matcher.rs | 484 +++++++++++++++++++++++ proxmox-notify/src/renderer/mod.rs | 15 +- proxmox-notify/src/schema.rs | 11 +- 18 files changed, 1046 insertions(+), 1143 deletions(-) delete mode 100644 proxmox-notify/src/api/filter.rs delete mode 100644 proxmox-notify/src/api/group.rs create mode 100644 proxmox-notify/src/api/matcher.rs create mode 100644 proxmox-notify/src/matcher.rs proxmox-perl-rs: Lukas Wagner (1): notify: adapt to new matcher-based notification routing common/src/notify.rs | 167 +++++++++++++------------------------------ 1 file changed, 50 insertions(+), 117 deletions(-) pve-cluster: Lukas Wagner (1): notify: adapt to matcher based notification system src/PVE/Notify.pm | 101 +++++++++++++++++++++------------------------- 1 file changed, 47 insertions(+), 54 deletions(-) pve-guest-common: Lukas Wagner (1): vzdump: deprecate mailto/mailnotification/notification-{target,policy} src/PVE/VZDump/Common.pm | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) pve-ha-manager: Lukas Wagner (1): env: switch to matcher-based notification system src/PVE/HA/Env/PVE2.pm | 10 ++-------- src/PVE/HA/NodeStatus.pm | 11 +++++++++-- 2 files changed, 11 insertions(+), 10 deletions(-) pve-manager: Lukas Wagner (10): api: notification: remove notification groups api: notification: add new matcher-based notification API ui: dc: remove unneeded notification events panel vzdump: adapt to new matcher based notification system api: apt: adapt to matcher-based notifications api: replication: adapt to matcher-based notification system debian: postinst: create notifications.cfg if it does not exist test: fix vzdump notification test ui: vzdump: remove left-overs from target/policy based notifications ui: dc: config: show notification panel again PVE/API2/APT.pm | 27 +- PVE/API2/Cluster/Notifications.pm | 462 ++++-------------- PVE/API2/Replication.pm | 25 +- PVE/API2/VZDump.pm | 8 +- PVE/VZDump.pm | 40 +- debian/postinst | 28 ++ test/vzdump_notification_test.pl | 6 +- www/manager6/Makefile | 4 - www/manager6/dc/Backup.js | 81 +-- www/manager6/dc/Config.js | 28 +- www/manager6/dc/NotificationEvents.js | 276 ----------- .../form/NotificationPolicySelector.js | 1 - www/manager6/window/Backup.js | 35 +- 13 files changed, 186 insertions(+), 835 deletions(-) delete mode 100644 www/manager6/dc/NotificationEvents.js proxmox-widget-toolkit: Lukas Wagner (7): notification ui: add target selector for matcher notification ui: remove filter setting for targets notification ui: remove notification groups notification ui: rename filter to matcher notification: matcher: add UI for matcher editing notification ui: unprotected mailto-root target noficiation: matcher edit: make 'field' an editable combobox src/Makefile | 4 +- src/Schema.js | 5 - src/data/model/NotificationConfig.js | 2 +- src/form/NotificationFilterSelector.js | 58 -- src/panel/GotifyEditPanel.js | 9 - src/panel/NotificationConfigView.js | 36 +- src/panel/NotificationGroupEditPanel.js | 183 ---- src/panel/SendmailEditPanel.js | 9 - src/window/NotificationFilterEdit.js | 109 --- src/window/NotificationMatcherEdit.js | 1036 +++++++++++++++++++++++ 10 files changed, 1051 insertions(+), 400 deletions(-) delete mode 100644 src/form/NotificationFilterSelector.js delete mode 100644 src/panel/NotificationGroupEditPanel.js delete mode 100644 src/window/NotificationFilterEdit.js create mode 100644 src/window/NotificationMatcherEdit.js Summary over all repositories: 46 files changed, 2398 insertions(+), 2568 deletions(-) -- murpp v0.4.0