* [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system
@ 2023-07-17 14:59 Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 01/66] add proxmox-notify crate Lukas Wagner
` (68 more replies)
0 siblings, 69 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
# Overview
The purpose of this patch series is to overhaul the existing mail
notification infrastructure in Proxmox VE.
The series replaces calls to 'sendmail' with calls to a
new, configurable notification module. The module was designed to
support multiple notification endpoints, 'sendmail' using the system's
sendmail command being the first one. As a proof of the extensibility
of the current approach, the 'gotify' [1] plugin was also implemented.
The patch series also includes groups. They allow to send a notification
to multiple endpoints at the same time. Furthermore, there are filters.
Endpoints and groups can configure filters to determine if a notification
should be sent. For now, filters can only be configured based on notification
severity.
A short summary of what is included in this patch series:
- Sendmail endpoint plugin: uses the system's `sendmail` command
to send - well - mail. The sendmail plugin sends multi-part mails
containing HTML as well as plain text.
- Gotify endpoint plugin: sends a notification to a gotify server
- Groups: As for any notification event one is only able to select a single
target, groups can be created to notify multiply endpoints at the same time
- Filters: Endpoints and groups can also have filtering: The filter
can match on the notification's metadata (only severity for now) to
determine if it will be sent or not. Filters can be easily extended in
the future to match on other structured metadata as well.
- REST API for managing endpoints, groups and filters
- Overhauled GUI for backup jobs/one-off backups - here the use can now
select a notification target
- GUI for configuring the other notification events
(APT, replication, fencing) - here the user can configure *when* and
*where* to send a notification
- Notification rendering based on templates:
From a single template, the system can render notifications to either
plain text or HTML.
# Configuration
For every single notification event (backup jobs, replication, APT, fencing),
it is now possible to configure
a.) whether to send a notification at all
b.) where to send the notification (target)
For backup jobs, this is configured in the respective backup job config in
`jobs.cfg`, everything else is configured using an extended `notify`
configuration key in `datacenter.cfg`:
root@pve:~# cat /etc/pve/datacenter.cfg
keyboard: en-us
notify: target-fencing=foo,fencing=always,package-updates=auto,target-package-updates=bar
root@pve:~# cat /etc/pve/jobs.cfg
vzdump: backup-2e021835-88ea
...
notes-template {{guestname}}
notification-policy always
notification-target mail-to-root
...
Targets (endpoint plugins or groups) and filters are configured in two new
configuration files: `notifications.cfg` and `priv/notifications.cfg` -
the latter is only readable by root and is used to store any sensitive
information for notification endpoint plugins (tokens, passwords, etc.).
These two new configuration files are opaque to the Perl code, it only
interacts with it through the bindings to the Rust implementation.
root@pve:~# cat /etc/pve/notifications.cfg
filter: only-errors
min-severity error
sendmail: sm1
mailto-user root@pam
mailto user1@example.com
mailto user2@example.com
filter on-errors
gotify: gt1
filter only-errors
server https://<IP:port>
root@pve:~# cat /etc/pve/priv/notifications.cfg
gotify: gt1
token foobar
# Permissions
The new notification module is fully integrated in the permission system.
For every target or filter, there exists a corresponding ACL path
/mapping/notifications/<name>. A user must have the Mapping.Use permissions
on that path to send the notification/use the filter. Mapping.Modify and
Mapping.Audit are needed for writing/reading a target's/filter's config.
For groups, the user must have the Mapping.Use permission for every
single endpoint included in the group. If a group/endpoint has a filter,
the user must have the Mapping.Use permissions for that as well.
# Backwards Compatibility
Great care was taken so that existing setups work exactly as before.
Also, every new feature of this series is strictly opt-in, meaning that
if no settings are changed, the system *should* behave exactly as before with
regards to sending notifications. The formatting of the notification emails
has changed *slightly*. If any users relied on that, they might require some
changes (e.g. if they scraped the mails to get the results from backup jobs).
# A note to testers
Since this patch series covers a lot of different repos, it's a bit
cumbersome to set it up correctly. Thus, I've built .deb files that you can
just install (staging repos should be configured).
The packages are at `iso/packages/notification` on nasi. I'll try
to rebuild them regularly with my changes rebased on the current master
branches. Holler at me if you think anything is broken or if the packages
do not install properly.
The changes are also available on my staff repos. Development happens on the
`notification` branches, the version posted here corresponds to the
`notification_v3` tag.
The recommend setup to test this is a 3 node cluster (fencing notifications)
with ZFS storage (for replication notifications). For update notifications,
you can use `pvesh create /node/<hostname>/apt/update --notify 1`.
You might need to clear/rm `/var/lib/pve-manager/pkgupdates` first.
# Follow-up work (in no particular order)
- Revisit the template helpers. Once they are considered 'stable', we
could offer an API for users to send notifications themselves.
- Maybe add some fancier styling to the HTML mails?
e.g. a light, but recognizable Proxmox-colored style? Unfortunately
HTML mails have no proper support for CSS and require inline styling
- In the future, the API might be changed/extended so that supports
"registering" notifications.
This also allows us to specify a 'contract' on what properties
are guaranteed to be included with a specific notification.
Consequently, this allows us to:
a.) generate a list of all possible notification sources in the system
b.) add more sophisticated filtering (e.g. match property for
c.) add other endpoints types where a fixed set of metadata fields makes
sense (e.g. webhook)
In my head, using the notification module could look like this
then:
# Global context
my backup_failed_notification = PVE::Notify::register({
'id' => 'backup-failed',
'severity' => 'error',
'properties' => ['host', 'vmlist', 'logs'],
'title' => '{{ host }}: Backup failed'
'body' => <<'EOF'
A backup has failed for the following VMs: {{ vmlist }}
{{ logs }}
EOF
});
# Later, to send the notification:
PVE::Notify::send(target, backup_failed_notification->instantiate({
'host' => 'earth',
'vmlist' => ... ,
'logs' => ... ,
}));
- proxmox-mail-forward could be integrated as well. This would feed
e.g. zfs-zed events into our notification infrastructure. Special
care must be taken to not create recursive notification loops
(e.g. zed sends to root, forwarder uses notification module, a
configured sendmail endpoint sends to root, forwarder uses module
--> loop)
- Maybe add some CLI so that admins can send notifications in
scripts (an API endpoint callable via pvesh might be enough for a
start).
- Add more notification events - the 'chattier' ones should probably be
disabled by default
- Add other endpoints, e.g. webhook, a generic mail endpoint that
sends emails directly via SMTP, etc.
- Integrate the new module into the other products
- for PBS, this should be relatively straight-forward
[1] https://gotify.net/
[2] https://bugzilla.proxmox.com/show_bug.cgi?id=4526
# Changelog
Note: I omitted per-commit changelogs due to already huge complexity managing
this patch series.
Changes since v2:
- Transformed 'channels' into 'groups'. Allow to notify via a single endpoint
or via a group (containing multiple endpoints)
- Dropped some of the features for filters for now, such as sub-filters and
property-matchers. The property matching would require use to stabilize
notification properties, which will be done later. Right now, the
notifications only have the properties required for rendering the
notification template.
- Groups can now also have filters
- Check if a filter/group/endpoint is still used before deleting it
- Ensure that filter/group/endpoint names are unique
- Add new options to datacenter.cfg, allowing to configure the target
and whether to notify at all for all notification events
- Added new GUI panels, one for modifying the new settings in
datacenter.cfg, the other one for adding/modifying/deleting notification
targets and filters
- Integrate notification targets in the backup job UI
- Full integration with the permission system
- Allow users as mail recipients, looking up their email address in the user
configuration file `user.cfg`.
- Look up mail-from and author from datacenter.cfg, if it is not configured
for a sendmail endpoint
- Support proxies for gotify endpoints. Proxy config is read from
datacenter.cfg
- Move `PVE::Notify` to `pve-cluster`, building a new `libpve-notify-perl`
package
- The API now uses the new array type
- Add always-available `mail-to-root` target, which is used as a fallback
if no other target is configured for an event
- Template rendering: Don't HTML-escape if rendering to plain text
- many other minor changes
Changes since v1:
- Some renaming:
- PVE::Notification -> PVE::Notify
- proxmox-notification -> proxmox-notify
- Split configuration for gotify endpoints into a public part in
`notifications.cfg` and a private part for the token in
`priv/notifications.cfg`
- Add template-based notification rendering (`proxmox`), including helpers
for:
- tables
- pretty printed JSON
- duration, timestamps
- byte sizes
- Add notification channels (repo `proxmox`)
- Add API routes for channels, endpoints, filters (implementation in
`proxmox-notify`, glue code in `proxmox-perl-rs` and handler in
`pve-manager`)
- Integrated new notification channels in backup jobs/one-off backups (repo
`pve-manager`)
- Replication/APT/Fencing use an 'anonymous' channel with a temporary
sendmail endpoint, sending mails to `root`
- Added new options for backup jobs
- Reworked git history
Versions of this patch series:
v2: https://lists.proxmox.com/pipermail/pve-devel/2023-May/056927.html
v1: https://lists.proxmox.com/pipermail/pve-devel/2023-March/056445.html
proxmox:
Lukas Wagner (23):
add proxmox-notify crate
notify: preparation for the first endpoint plugin
notify: preparation for the API
notify: api: add API for sending notifications/testing endpoints
notify: add sendmail plugin
notify: api: add API for sendmail endpoints
notify: add gotify endpoint
notify: api: add API for gotify endpoints
notify: add notification groups
notify: api: add API for groups
notify: add notification filter mechanism
notify: api: add API for filters
notify: add template rendering
notify: add example for template rendering
notify: add context
notify: sendmail: allow users as recipients
notify: sendmail: query default author/mailfrom from context
notify: gotify: add proxy support
notify: api: allow to query entities referenced by filter/target
notify: on deletion, check if a filter/endp. is still used by anything
notify: ensure that filter/group/endpoint names are unique
notify: additional logging when sending a notification
notify: add debian packaging
Cargo.toml | 3 +
proxmox-notify/Cargo.toml | 28 ++
proxmox-notify/debian/changelog | 5 +
proxmox-notify/debian/control | 108 ++++
proxmox-notify/debian/copyright | 16 +
proxmox-notify/debian/debcargo.toml | 7 +
proxmox-notify/examples/render.rs | 63 +++
proxmox-notify/src/api/common.rs | 55 +++
proxmox-notify/src/api/filter.rs | 227 +++++++++
proxmox-notify/src/api/gotify.rs | 292 +++++++++++
proxmox-notify/src/api/group.rs | 261 ++++++++++
proxmox-notify/src/api/mod.rs | 372 ++++++++++++++
proxmox-notify/src/api/sendmail.rs | 288 +++++++++++
proxmox-notify/src/config.rs | 103 ++++
proxmox-notify/src/context.rs | 18 +
proxmox-notify/src/endpoints/gotify.rs | 152 ++++++
proxmox-notify/src/endpoints/mod.rs | 4 +
proxmox-notify/src/endpoints/sendmail.rs | 146 ++++++
proxmox-notify/src/filter.rs | 198 ++++++++
proxmox-notify/src/group.rs | 48 ++
proxmox-notify/src/lib.rs | 602 +++++++++++++++++++++++
proxmox-notify/src/renderer/html.rs | 100 ++++
proxmox-notify/src/renderer/mod.rs | 366 ++++++++++++++
proxmox-notify/src/renderer/plaintext.rs | 141 ++++++
proxmox-notify/src/renderer/table.rs | 24 +
proxmox-notify/src/schema.rs | 49 ++
26 files changed, 3676 insertions(+)
create mode 100644 proxmox-notify/Cargo.toml
create mode 100644 proxmox-notify/debian/changelog
create mode 100644 proxmox-notify/debian/control
create mode 100644 proxmox-notify/debian/copyright
create mode 100644 proxmox-notify/debian/debcargo.toml
create mode 100644 proxmox-notify/examples/render.rs
create mode 100644 proxmox-notify/src/api/common.rs
create mode 100644 proxmox-notify/src/api/filter.rs
create mode 100644 proxmox-notify/src/api/gotify.rs
create mode 100644 proxmox-notify/src/api/group.rs
create mode 100644 proxmox-notify/src/api/mod.rs
create mode 100644 proxmox-notify/src/api/sendmail.rs
create mode 100644 proxmox-notify/src/config.rs
create mode 100644 proxmox-notify/src/context.rs
create mode 100644 proxmox-notify/src/endpoints/gotify.rs
create mode 100644 proxmox-notify/src/endpoints/mod.rs
create mode 100644 proxmox-notify/src/endpoints/sendmail.rs
create mode 100644 proxmox-notify/src/filter.rs
create mode 100644 proxmox-notify/src/group.rs
create mode 100644 proxmox-notify/src/lib.rs
create mode 100644 proxmox-notify/src/renderer/html.rs
create mode 100644 proxmox-notify/src/renderer/mod.rs
create mode 100644 proxmox-notify/src/renderer/plaintext.rs
create mode 100644 proxmox-notify/src/renderer/table.rs
create mode 100644 proxmox-notify/src/schema.rs
proxmox-perl-rs:
Lukas Wagner (10):
add PVE::RS::Notify module
notify: add api for sending notifications/testing endpoints
notify: add api for notification groups
notify: add api for sendmail endpoints
notify: add api for gotify endpoints
notify: add api for notification filters
notify: sendmail: support the `mailto-user` parameter
notify: implement context for getting default author/mailfrom
notify: add context for getting http_proxy from datacenter.cfg
notify: add wrapper for `get_referenced_entities`
pve-rs/Cargo.toml | 2 +
pve-rs/Makefile | 1 +
pve-rs/src/lib.rs | 4 +-
pve-rs/src/notify.rs | 555 +++++++++++++++++++++++++++++++++++++++++++
4 files changed, 561 insertions(+), 1 deletion(-)
create mode 100644 pve-rs/src/notify.rs
pve-cluster:
Lukas Wagner (3):
cluster files: add notifications.cfg
datacenter: add APT/fencing/replication notification configuration
add libpve-notify-perl package
debian/control | 9 ++
debian/libpve-notify-perl.docs | 1 +
debian/libpve-notify-perl.install | 1 +
src/PVE/Cluster.pm | 2 +
src/PVE/DataCenterConfig.pm | 63 ++++++++++++-
src/PVE/Makefile | 2 +-
src/PVE/Notify.pm | 145 ++++++++++++++++++++++++++++++
src/pmxcfs/status.c | 2 +
8 files changed, 222 insertions(+), 3 deletions(-)
create mode 100644 debian/libpve-notify-perl.docs
create mode 100644 debian/libpve-notify-perl.install
create mode 100644 src/PVE/Notify.pm
pve-guest-common:
Lukas Wagner (1):
vzdump: add config options for new notification backend
src/PVE/VZDump/Common.pm | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)
pve-common:
Lukas Wagner (1):
JSONSchema: increase maxLength of config-digest to 64
src/PVE/JSONSchema.pm | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
pve-ha-manager:
Lukas Wagner (1):
manager: send notifications via new notification module
debian/control | 2 ++
src/PVE/HA/Env.pm | 6 ++---
src/PVE/HA/Env/PVE2.pm | 21 +++++++++-------
src/PVE/HA/NodeStatus.pm | 52 ++++++++++++++++++++++++----------------
src/PVE/HA/Sim/Env.pm | 10 ++++++--
5 files changed, 57 insertions(+), 34 deletions(-)
pve-manager:
Lukas Wagner (21):
test: fix names of .PHONY targets
d/control: add dependency to `libpve-notify-perl`
vzdump: send notifications via new notification module
test: rename mail_test.pl to vzdump_notification_test.pl
api: apt: send notification via new notification module
api: replication: send notifications via new notification module
api: prepare api handler module for notification config
api: notification: add api routes for groups
api: notification: add api routes for sendmail endpoints
api: notification: add api routes for gotify endpoints
api: notification: add api routes for filters
api: notification: allow fetching notification targets
api: notification: allow to test targets
api: notification: disallow removing targets if they are used
ui: backup: allow to select notification target for jobs
ui: backup: adapt backup job details to new notification params
ui: backup: allow to set notification-target for one-off backups
ui: allow to configure notification event -> target mapping
ui: add notification target configuration panel
ui: perm path: load notification target/filter acl entries
ui: perm path: increase width of the perm path selector combobox
PVE/API2/APT.pm | 99 +-
PVE/API2/Cluster.pm | 7 +
PVE/API2/Cluster/Makefile | 1 +
PVE/API2/Cluster/Notifications.pm | 1356 +++++++++++++++++
PVE/API2/Replication.pm | 63 +-
PVE/API2/VZDump.pm | 10 +-
PVE/VZDump.pm | 335 ++--
debian/control | 2 +
test/Makefile | 16 +-
...il_test.pl => vzdump_notification_test.pl} | 36 +-
www/manager6/Makefile | 5 +-
www/manager6/data/PermPathStore.js | 26 +-
www/manager6/dc/Backup.js | 84 +-
www/manager6/dc/BackupJobDetail.js | 20 +-
www/manager6/dc/Config.js | 28 +
www/manager6/dc/NotificationEvents.js | 238 +++
www/manager6/form/NotificationModeSelector.js | 8 +
...ector.js => NotificationPolicySelector.js} | 1 +
.../form/NotificationTargetSelector.js | 54 +
www/manager6/form/PermPathSelector.js | 1 +
www/manager6/window/Backup.js | 35 +-
21 files changed, 2190 insertions(+), 235 deletions(-)
create mode 100644 PVE/API2/Cluster/Notifications.pm
rename test/{mail_test.pl => vzdump_notification_test.pl} (62%)
create mode 100644 www/manager6/dc/NotificationEvents.js
create mode 100644 www/manager6/form/NotificationModeSelector.js
rename www/manager6/form/{EmailNotificationSelector.js => NotificationPolicySelector.js} (87%)
create mode 100644 www/manager6/form/NotificationTargetSelector.js
proxmox-widget-toolkit:
Lukas Wagner (5):
notification: add gui for sendmail notification endpoints
notification: add gui for gotify notification endpoints
notification: add gui for notification groups
notification: allow to select filter for notification targets
notification: add ui for managing notification filters
src/Makefile | 8 +
src/Schema.js | 18 ++
src/data/model/NotificationConfig.js | 17 ++
src/form/NotificationFilterSelector.js | 58 +++++
src/panel/GotifyEditPanel.js | 61 +++++
src/panel/NotificationConfigView.js | 315 ++++++++++++++++++++++++
src/panel/NotificationGroupEditPanel.js | 186 ++++++++++++++
src/panel/SendmailEditPanel.js | 149 +++++++++++
src/window/EndpointEditBase.js | 55 +++++
src/window/NotificationFilterEdit.js | 115 +++++++++
10 files changed, 982 insertions(+)
create mode 100644 src/data/model/NotificationConfig.js
create mode 100644 src/form/NotificationFilterSelector.js
create mode 100644 src/panel/GotifyEditPanel.js
create mode 100644 src/panel/NotificationConfigView.js
create mode 100644 src/panel/NotificationGroupEditPanel.js
create mode 100644 src/panel/SendmailEditPanel.js
create mode 100644 src/window/EndpointEditBase.js
create mode 100644 src/window/NotificationFilterEdit.js
pve-docs:
Lukas Wagner (1):
add documentation for the new notification system
notifications.adoc | 159 +++++++++++++++++++++++++++++++++++++++++++
pve-admin-guide.adoc | 2 +
pve-gui.adoc | 2 +
vzdump.adoc | 5 ++
4 files changed, 168 insertions(+)
create mode 100644 notifications.adoc
Summary over all repositories:
80 files changed, 7880 insertions(+), 277 deletions(-)
--
murpp v0.4.0
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 01/66] add proxmox-notify crate
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 02/66] notify: preparation for the first endpoint plugin Lukas Wagner
` (67 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
Notes:
Changes v1 -> v2:
- Renamed crate from 'proxmox-notification' to 'proxmox-notify'
Cargo.toml | 1 +
proxmox-notify/Cargo.toml | 10 ++++++++++
proxmox-notify/src/lib.rs | 0
3 files changed, 11 insertions(+)
create mode 100644 proxmox-notify/Cargo.toml
create mode 100644 proxmox-notify/src/lib.rs
diff --git a/Cargo.toml b/Cargo.toml
index 1ecebabc..317593f0 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -13,6 +13,7 @@ members = [
"proxmox-ldap",
"proxmox-login",
"proxmox-metrics",
+ "proxmox-notify",
"proxmox-openid",
"proxmox-rest-server",
"proxmox-router",
diff --git a/proxmox-notify/Cargo.toml b/proxmox-notify/Cargo.toml
new file mode 100644
index 00000000..2e69d5b0
--- /dev/null
+++ b/proxmox-notify/Cargo.toml
@@ -0,0 +1,10 @@
+[package]
+name = "proxmox-notify"
+version = "0.1.0"
+authors.workspace = true
+edition.workspace = true
+license.workspace = true
+repository.workspace = true
+exclude.workspace = true
+
+[dependencies]
diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
new file mode 100644
index 00000000..e69de29b
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 02/66] notify: preparation for the first endpoint plugin
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 01/66] add proxmox-notify crate Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-17 15:48 ` Maximiliano Sandoval
2023-07-18 11:54 ` Wolfgang Bumiller
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 03/66] notify: preparation for the API Lukas Wagner
` (66 subsequent siblings)
68 siblings, 2 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
Cargo.toml | 1 +
proxmox-notify/Cargo.toml | 9 +
proxmox-notify/src/config.rs | 51 +++++
proxmox-notify/src/endpoints/mod.rs | 0
proxmox-notify/src/lib.rs | 311 ++++++++++++++++++++++++++++
proxmox-notify/src/schema.rs | 43 ++++
6 files changed, 415 insertions(+)
create mode 100644 proxmox-notify/src/config.rs
create mode 100644 proxmox-notify/src/endpoints/mod.rs
create mode 100644 proxmox-notify/src/schema.rs
diff --git a/Cargo.toml b/Cargo.toml
index 317593f0..ef8a050a 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -93,6 +93,7 @@ proxmox-lang = { version = "1.1", path = "proxmox-lang" }
proxmox-rest-server = { version = "0.4.0", path = "proxmox-rest-server" }
proxmox-router = { version = "1.3.1", path = "proxmox-router" }
proxmox-schema = { version = "1.3.7", path = "proxmox-schema" }
+proxmox-section-config = { version = "1.0.2", path = "proxmox-section-config" }
proxmox-serde = { version = "0.1.1", path = "proxmox-serde", features = [ "serde_json" ] }
proxmox-sortable-macro = { version = "0.1.2", path = "proxmox-sortable-macro" }
proxmox-sys = { version = "0.5.0", path = "proxmox-sys" }
diff --git a/proxmox-notify/Cargo.toml b/proxmox-notify/Cargo.toml
index 2e69d5b0..37d175f0 100644
--- a/proxmox-notify/Cargo.toml
+++ b/proxmox-notify/Cargo.toml
@@ -8,3 +8,12 @@ repository.workspace = true
exclude.workspace = true
[dependencies]
+lazy_static.workspace = true
+log.workspace = true
+openssl.workspace = true
+proxmox-schema = { workspace = true, features = ["api-macro"]}
+proxmox-section-config = { workspace = true }
+proxmox-sys.workspace = true
+regex.workspace = true
+serde.workspace = true
+serde_json.workspace = true
diff --git a/proxmox-notify/src/config.rs b/proxmox-notify/src/config.rs
new file mode 100644
index 00000000..362ca0fc
--- /dev/null
+++ b/proxmox-notify/src/config.rs
@@ -0,0 +1,51 @@
+use lazy_static::lazy_static;
+use proxmox_schema::{ApiType, ObjectSchema};
+use proxmox_section_config::{SectionConfig, SectionConfigData, SectionConfigPlugin};
+
+use crate::schema::BACKEND_NAME_SCHEMA;
+use crate::Error;
+
+lazy_static! {
+ pub static ref CONFIG: SectionConfig = config_init();
+ pub static ref PRIVATE_CONFIG: SectionConfig = private_config_init();
+}
+
+fn config_init() -> SectionConfig {
+ let mut config = SectionConfig::new(&BACKEND_NAME_SCHEMA);
+
+ config
+}
+
+fn private_config_init() -> SectionConfig {
+ let mut config = SectionConfig::new(&BACKEND_NAME_SCHEMA);
+
+ config
+}
+
+pub fn config(raw_config: &str) -> Result<(SectionConfigData, [u8; 32]), Error> {
+ let digest = openssl::sha::sha256(raw_config.as_bytes());
+ let data = CONFIG
+ .parse("notifications.cfg", raw_config)
+ .map_err(|err| Error::ConfigDeserialization(err.into()))?;
+ Ok((data, digest))
+}
+
+pub fn private_config(raw_config: &str) -> Result<(SectionConfigData, [u8; 32]), Error> {
+ let digest = openssl::sha::sha256(raw_config.as_bytes());
+ let data = PRIVATE_CONFIG
+ .parse("priv/notifications.cfg", raw_config)
+ .map_err(|err| Error::ConfigDeserialization(err.into()))?;
+ Ok((data, digest))
+}
+
+pub fn write(config: &SectionConfigData) -> Result<String, Error> {
+ CONFIG
+ .write("notifications.cfg", config)
+ .map_err(|err| Error::ConfigSerialization(err.into()))
+}
+
+pub fn write_private(config: &SectionConfigData) -> Result<String, Error> {
+ PRIVATE_CONFIG
+ .write("priv/notifications.cfg", config)
+ .map_err(|err| Error::ConfigSerialization(err.into()))
+}
diff --git a/proxmox-notify/src/endpoints/mod.rs b/proxmox-notify/src/endpoints/mod.rs
new file mode 100644
index 00000000..e69de29b
diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
index e69de29b..7b90ee15 100644
--- a/proxmox-notify/src/lib.rs
+++ b/proxmox-notify/src/lib.rs
@@ -0,0 +1,311 @@
+use std::collections::HashMap;
+use std::fmt::Display;
+
+use proxmox_schema::api;
+use proxmox_section_config::SectionConfigData;
+use serde::{Deserialize, Serialize};
+use serde_json::json;
+use serde_json::Value;
+
+use std::error::Error as StdError;
+
+mod config;
+pub mod endpoints;
+pub mod schema;
+
+#[derive(Debug)]
+pub enum Error {
+ ConfigSerialization(Box<dyn StdError + Send + Sync + 'static>),
+ ConfigDeserialization(Box<dyn StdError + Send + Sync + 'static>),
+ NotifyFailed(String, Box<dyn StdError + Send + Sync + 'static>),
+ TargetDoesNotExist(String),
+}
+
+impl Display for Error {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ match self {
+ Error::ConfigSerialization(err) => {
+ write!(f, "could not serialize configuration: {err}")
+ }
+ Error::ConfigDeserialization(err) => {
+ write!(f, "could not deserialize configuration: {err}")
+ }
+ Error::NotifyFailed(endpoint, err) => {
+ write!(f, "could not notify via endpoint(s): {endpoint}: {err}")
+ }
+ Error::TargetDoesNotExist(target) => {
+ write!(f, "notification target '{target}' does not exist")
+ }
+ }
+ }
+}
+
+impl StdError for Error {
+ fn source(&self) -> Option<&(dyn StdError + 'static)> {
+ match self {
+ Error::ConfigSerialization(err) => Some(&**err),
+ Error::ConfigDeserialization(err) => Some(&**err),
+ Error::NotifyFailed(_, err) => Some(&**err),
+ Error::TargetDoesNotExist(_) => None,
+ }
+ }
+}
+
+#[api()]
+#[derive(Clone, Debug, Copy, Serialize, Deserialize, PartialEq, Eq, PartialOrd)]
+#[serde(rename_all = "kebab-case")]
+/// Severity of a notification
+pub enum Severity {
+ /// General information
+ Info,
+ /// A noteworthy event
+ Notice,
+ /// Warning
+ Warning,
+ /// Error
+ Error,
+}
+
+/// Notification endpoint trait, implemented by all endpoint plugins
+pub trait Endpoint {
+ /// Send a documentation
+ fn send(&self, notification: &Notification) -> Result<(), Error>;
+
+ /// The name/identifier for this endpoint
+ fn name(&self) -> &str;
+}
+
+#[derive(Debug, Clone)]
+/// Notification which can be sent
+pub struct Notification {
+ /// Notification severity
+ pub severity: Severity,
+ /// The title of the notification
+ pub title: String,
+ /// Notification text
+ pub body: String,
+ /// Additional metadata for the notification
+ pub properties: Option<Value>,
+}
+
+/// Notification configuration
+pub struct Config {
+ config: SectionConfigData,
+ private_config: SectionConfigData,
+ digest: [u8; 32],
+ private_digest: [u8; 32],
+}
+
+impl Clone for Config {
+ fn clone(&self) -> Self {
+ Self {
+ config: SectionConfigData {
+ sections: self.config.sections.clone(),
+ order: self.config.order.clone(),
+ },
+ private_config: SectionConfigData {
+ sections: self.private_config.sections.clone(),
+ order: self.private_config.order.clone(),
+ },
+ digest: self.digest,
+ private_digest: self.private_digest,
+ }
+ }
+}
+
+impl Config {
+ /// Parse raw config
+ pub fn new(raw_config: &str, raw_private_config: &str) -> Result<Self, Error> {
+ let (config, digest) = config::config(raw_config)?;
+ let (private_config, private_digest) = config::private_config(raw_private_config)?;
+
+ Ok(Self {
+ config,
+ digest,
+ private_config,
+ private_digest,
+ })
+ }
+
+ /// Serialize config
+ pub fn write(&self) -> Result<(String, String), Error> {
+ Ok((
+ config::write(&self.config)?,
+ config::write_private(&self.private_config)?,
+ ))
+ }
+
+ /// Returns the SHA256 digest of the configuration.
+ /// The digest is only computed once when the configuration deserialized.
+ pub fn digest(&self) -> &[u8; 32] {
+ &self.digest
+ }
+}
+
+/// Notification bus - distributes notifications to all registered endpoints
+// The reason for the split between `Config` and this struct is to make testing with mocked
+// endpoints a bit easier.
+#[derive(Default)]
+pub struct Bus {
+ endpoints: HashMap<String, Box<dyn Endpoint>>,
+}
+
+#[allow(unused_macros)]
+macro_rules! parse_endpoints_with_private_config {
+ ($config:ident, $public_config:ty, $private_config:ty, $endpoint_type:ident, $type_name:expr) => {
+ (|| -> Result<Vec<Box<dyn Endpoint>>, Error> {
+ let mut endpoints: Vec<Box<dyn Endpoint>> = Vec::new();
+
+ let configs: Vec<$public_config> = $config
+ .config
+ .convert_to_typed_array($type_name)
+ .map_err(|err| Error::ConfigDeserialization(err.into()))?;
+
+ let private_configs: Vec<$private_config> = $config
+ .private_config
+ .convert_to_typed_array($type_name)
+ .map_err(|err| Error::ConfigDeserialization(err.into()))?;
+
+ for config in configs {
+ if let Some(private_config) = private_configs.iter().find(|p| p.name == config.name)
+ {
+ endpoints.push(Box::new($endpoint_type {
+ config,
+ private_config: private_config.clone(),
+ }));
+ } else {
+ log::error!(
+ "Could not instantiate endpoint '{name}': private config does not exist",
+ name = config.name
+ );
+ }
+ }
+
+ Ok(endpoints)
+ })()
+ };
+}
+
+#[allow(unused_macros)]
+macro_rules! parse_endpoints_without_private_config {
+ ($config:ident, $public_config:ty, $endpoint_type:ident, $type_name:expr) => {
+ (|| -> Result<Vec<Box<dyn Endpoint>>, Error> {
+ let mut endpoints: Vec<Box<dyn Endpoint>> = Vec::new();
+
+ let configs: Vec<$public_config> = $config
+ .config
+ .convert_to_typed_array($type_name)
+ .map_err(|err| Error::ConfigDeserialization(err.into()))?;
+
+ for config in configs {
+ endpoints.push(Box::new($endpoint_type { config }));
+ }
+
+ Ok(endpoints)
+ })()
+ };
+}
+
+impl Bus {
+ /// Instantiate notification bus from a given configuration.
+ pub fn from_config(config: &Config) -> Result<Self, Error> {
+ let mut endpoints = HashMap::new();
+
+ Ok(Bus { endpoints })
+ }
+
+ #[cfg(test)]
+ pub fn add_endpoint(&mut self, endpoint: Box<dyn Endpoint>) {
+ self.endpoints.insert(endpoint.name().to_string(), endpoint);
+ }
+
+ pub fn send(&self, target: &str, notification: &Notification) -> Result<(), Error> {
+ log::info!(
+ "sending notification with title '{title}'",
+ title = notification.title
+ );
+
+ let endpoint = self
+ .endpoints
+ .get(target)
+ .ok_or(Error::TargetDoesNotExist(target.into()))?;
+
+ endpoint.send(notification).unwrap_or_else(|e| {
+ log::error!(
+ "could not notfiy via endpoint `{name}`: {e}",
+ name = endpoint.name()
+ )
+ });
+
+ Ok(())
+ }
+
+ pub fn test_target(&self, target: &str) -> Result<(), Error> {
+ let endpoint = self
+ .endpoints
+ .get(target)
+ .ok_or(Error::TargetDoesNotExist(target.into()))?;
+
+ endpoint.send(&Notification {
+ severity: Severity::Info,
+ title: "Test notification".into(),
+ body: "This is a test of the notification target '{{ target }}'".into(),
+ properties: Some(json!({ "target": target })),
+ })?;
+
+ Ok(())
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use std::{cell::RefCell, rc::Rc};
+
+ use super::*;
+
+ #[derive(Default, Clone)]
+ struct MockEndpoint {
+ messages: Rc<RefCell<Vec<Notification>>>,
+ }
+
+ impl Endpoint for MockEndpoint {
+ fn send(&self, message: &Notification) -> Result<(), Error> {
+ self.messages.borrow_mut().push(message.clone());
+
+ Ok(())
+ }
+
+ fn name(&self) -> &str {
+ "mock-endpoint"
+ }
+ }
+
+ impl MockEndpoint {
+ fn messages(&self) -> Vec<Notification> {
+ self.messages.borrow().clone()
+ }
+ }
+
+ #[test]
+ fn test_add_mock_endpoint() -> Result<(), Error> {
+ let mock = MockEndpoint::default();
+
+ let mut bus = Bus::default();
+
+ bus.add_endpoint(Box::new(mock.clone()));
+
+ bus.send(
+ "mock-endpoint",
+ &Notification {
+ title: "Title".into(),
+ body: "Body".into(),
+ severity: Severity::Info,
+ properties: Default::default(),
+ },
+ )?;
+ let messages = mock.messages();
+ assert_eq!(messages.len(), 1);
+
+ Ok(())
+ }
+}
diff --git a/proxmox-notify/src/schema.rs b/proxmox-notify/src/schema.rs
new file mode 100644
index 00000000..68f11959
--- /dev/null
+++ b/proxmox-notify/src/schema.rs
@@ -0,0 +1,43 @@
+use proxmox_schema::{const_regex, ApiStringFormat, Schema, StringSchema};
+
+// Copied from PBS
+macro_rules! proxmox_safe_id_regex_str {
+ () => {
+ r"(?:[A-Za-z0-9_][A-Za-z0-9._\-]*)"
+ };
+}
+
+const_regex! {
+ pub SINGLE_LINE_COMMENT_REGEX = r"^[[:^cntrl:]]*$";
+ pub PROXMOX_SAFE_ID_REGEX = concat!(r"^", proxmox_safe_id_regex_str!(), r"$");
+}
+
+const SINGLE_LINE_COMMENT_FORMAT: ApiStringFormat =
+ ApiStringFormat::Pattern(&SINGLE_LINE_COMMENT_REGEX);
+
+pub const COMMENT_SCHEMA: Schema = StringSchema::new("Comment.")
+ .format(&SINGLE_LINE_COMMENT_FORMAT)
+ .max_length(128)
+ .schema();
+
+pub const EMAIL_SCHEMA: Schema = StringSchema::new("E-Mail Address.")
+ .format(&SINGLE_LINE_COMMENT_FORMAT)
+ .min_length(2)
+ .max_length(64)
+ .schema();
+
+pub const PROXMOX_SAFE_ID_FORMAT: ApiStringFormat =
+ ApiStringFormat::Pattern(&PROXMOX_SAFE_ID_REGEX);
+
+pub const BACKEND_NAME_SCHEMA: Schema = StringSchema::new("Notification backend name.")
+ .format(&PROXMOX_SAFE_ID_FORMAT)
+ .min_length(3)
+ .max_length(32)
+ .schema();
+
+pub const ENTITY_NAME_SCHEMA: Schema =
+ StringSchema::new("Name schema for endpoints, filters and groups")
+ .format(&PROXMOX_SAFE_ID_FORMAT)
+ .min_length(2)
+ .max_length(32)
+ .schema();
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 03/66] notify: preparation for the API
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 01/66] add proxmox-notify crate Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 02/66] notify: preparation for the first endpoint plugin Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-18 12:02 ` Wolfgang Bumiller
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 04/66] notify: api: add API for sending notifications/testing endpoints Lukas Wagner
` (65 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/api/mod.rs | 94 +++++++++++++++++++++++++++++++++++
proxmox-notify/src/lib.rs | 1 +
2 files changed, 95 insertions(+)
create mode 100644 proxmox-notify/src/api/mod.rs
diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
new file mode 100644
index 00000000..be596b93
--- /dev/null
+++ b/proxmox-notify/src/api/mod.rs
@@ -0,0 +1,94 @@
+use std::error::Error as StdError;
+use std::fmt::Display;
+
+use crate::Config;
+use serde::Serialize;
+
+#[derive(Debug, Serialize)]
+pub struct ApiError {
+ /// HTTP Error code
+ code: u16,
+ /// Error message
+ message: String,
+ #[serde(skip_serializing)]
+ /// The underlying cause of the error
+ source: Option<Box<dyn StdError + Send + Sync + 'static>>,
+}
+
+impl ApiError {
+ fn new<S: AsRef<str>>(
+ message: S,
+ code: u16,
+ source: Option<Box<dyn StdError + Send + Sync + 'static>>,
+ ) -> Self {
+ Self {
+ message: message.as_ref().into(),
+ code,
+ source,
+ }
+ }
+
+ pub fn bad_request<S: AsRef<str>>(
+ message: S,
+ source: Option<Box<dyn StdError + Send + Sync + 'static>>,
+ ) -> Self {
+ Self::new(message, 400, source)
+ }
+
+ pub fn not_found<S: AsRef<str>>(
+ message: S,
+ source: Option<Box<dyn StdError + Send + Sync + 'static>>,
+ ) -> Self {
+ Self::new(message, 404, source)
+ }
+
+ pub fn internal_server_error<S: AsRef<str>>(
+ message: S,
+ source: Option<Box<dyn StdError + Send + Sync + 'static>>,
+ ) -> Self {
+ Self::new(message, 500, source)
+ }
+}
+
+impl Display for ApiError {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ f.write_str(&format!("{} {}", self.code, self.message))
+ }
+}
+
+impl StdError for ApiError {
+ fn source(&self) -> Option<&(dyn StdError + 'static)> {
+ match &self.source {
+ None => None,
+ Some(source) => Some(&**source),
+ }
+ }
+}
+
+fn verify_digest(config: &Config, digest: Option<&[u8]>) -> Result<(), ApiError> {
+ if let Some(digest) = digest {
+ if config.digest != *digest {
+ return Err(ApiError::bad_request(
+ "detected modified configuration - file changed by other user? Try again.",
+ None,
+ ));
+ }
+ }
+
+ Ok(())
+}
+
+fn endpoint_exists(config: &Config, name: &str) -> bool {
+ let mut exists = false;
+
+ exists
+}
+
+#[cfg(test)]
+mod test_helpers {
+ use crate::Config;
+
+ pub fn empty_config() -> Config {
+ Config::new("", "").unwrap()
+ }
+}
diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
index 7b90ee15..43feac25 100644
--- a/proxmox-notify/src/lib.rs
+++ b/proxmox-notify/src/lib.rs
@@ -9,6 +9,7 @@ use serde_json::Value;
use std::error::Error as StdError;
+pub mod api;
mod config;
pub mod endpoints;
pub mod schema;
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 04/66] notify: api: add API for sending notifications/testing endpoints
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (2 preceding siblings ...)
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 03/66] notify: preparation for the API Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 05/66] notify: add sendmail plugin Lukas Wagner
` (64 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/api/common.rs | 44 ++++++++++++++++++++++++++++++++
proxmox-notify/src/api/mod.rs | 2 ++
2 files changed, 46 insertions(+)
create mode 100644 proxmox-notify/src/api/common.rs
diff --git a/proxmox-notify/src/api/common.rs b/proxmox-notify/src/api/common.rs
new file mode 100644
index 00000000..518caa8f
--- /dev/null
+++ b/proxmox-notify/src/api/common.rs
@@ -0,0 +1,44 @@
+use crate::api::ApiError;
+use crate::{Bus, Config, Notification};
+
+/// Send a notification to a given target.
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns an `ApiError` in case of an error.
+pub fn send(config: &Config, channel: &str, notification: &Notification) -> Result<(), ApiError> {
+ let bus = Bus::from_config(config).map_err(|err| {
+ ApiError::internal_server_error(
+ "Could not instantiate notification bus",
+ Some(Box::new(err)),
+ )
+ })?;
+
+ bus.send(channel, notification);
+
+ Ok(())
+}
+
+/// Test target (group or single endpoint) identified by its `name`.
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns an `ApiError` if sending via the endpoint failed.
+pub fn test_target(config: &Config, endpoint: &str) -> Result<(), ApiError> {
+ let bus = Bus::from_config(config).map_err(|err| {
+ ApiError::internal_server_error(
+ "Could not instantiate notification bus",
+ Some(Box::new(err)),
+ )
+ })?;
+
+ bus.test_target(endpoint).map_err(|err| match err {
+ crate::Error::TargetDoesNotExist(endpoint) => {
+ ApiError::not_found(format!("endpoint '{endpoint}' does not exist"), None)
+ }
+ _ => ApiError::internal_server_error(
+ format!("Could not test target: {err}"),
+ Some(Box::new(err)),
+ ),
+ })?;
+
+ Ok(())
+}
diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
index be596b93..db9ad1ca 100644
--- a/proxmox-notify/src/api/mod.rs
+++ b/proxmox-notify/src/api/mod.rs
@@ -4,6 +4,8 @@ use std::fmt::Display;
use crate::Config;
use serde::Serialize;
+pub mod common;
+
#[derive(Debug, Serialize)]
pub struct ApiError {
/// HTTP Error code
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 05/66] notify: add sendmail plugin
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (3 preceding siblings ...)
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 04/66] notify: api: add API for sending notifications/testing endpoints Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 06/66] notify: api: add API for sendmail endpoints Lukas Wagner
` (63 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
This plugin uses the 'sendmail' command to send an email
to one or more recipients.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/Cargo.toml | 9 ++-
proxmox-notify/src/config.rs | 12 ++++
proxmox-notify/src/endpoints/mod.rs | 2 +
proxmox-notify/src/endpoints/sendmail.rs | 88 ++++++++++++++++++++++++
proxmox-notify/src/lib.rs | 18 +++++
5 files changed, 127 insertions(+), 2 deletions(-)
create mode 100644 proxmox-notify/src/endpoints/sendmail.rs
diff --git a/proxmox-notify/Cargo.toml b/proxmox-notify/Cargo.toml
index 37d175f0..f7329295 100644
--- a/proxmox-notify/Cargo.toml
+++ b/proxmox-notify/Cargo.toml
@@ -8,12 +8,17 @@ repository.workspace = true
exclude.workspace = true
[dependencies]
+handlebars = { workspace = true, optional = true }
lazy_static.workspace = true
log.workspace = true
openssl.workspace = true
proxmox-schema = { workspace = true, features = ["api-macro"]}
proxmox-section-config = { workspace = true }
-proxmox-sys.workspace = true
+proxmox-sys = { workspace = true, optional = true }
regex.workspace = true
-serde.workspace = true
+serde = { workspace = true, features = ["derive"]}
serde_json.workspace = true
+
+[features]
+default = ["sendmail"]
+sendmail = ["dep:handlebars", "dep:proxmox-sys"]
\ No newline at end of file
diff --git a/proxmox-notify/src/config.rs b/proxmox-notify/src/config.rs
index 362ca0fc..6269ec3c 100644
--- a/proxmox-notify/src/config.rs
+++ b/proxmox-notify/src/config.rs
@@ -13,6 +13,18 @@ lazy_static! {
fn config_init() -> SectionConfig {
let mut config = SectionConfig::new(&BACKEND_NAME_SCHEMA);
+ #[cfg(feature = "sendmail")]
+ {
+ use crate::endpoints::sendmail::{SendmailConfig, SENDMAIL_TYPENAME};
+
+ const SENDMAIL_SCHEMA: &ObjectSchema = SendmailConfig::API_SCHEMA.unwrap_object_schema();
+ config.register_plugin(SectionConfigPlugin::new(
+ SENDMAIL_TYPENAME.to_string(),
+ Some(String::from("name")),
+ SENDMAIL_SCHEMA,
+ ));
+ }
+
config
}
diff --git a/proxmox-notify/src/endpoints/mod.rs b/proxmox-notify/src/endpoints/mod.rs
index e69de29b..dd80d9bc 100644
--- a/proxmox-notify/src/endpoints/mod.rs
+++ b/proxmox-notify/src/endpoints/mod.rs
@@ -0,0 +1,2 @@
+#[cfg(feature = "sendmail")]
+pub mod sendmail;
diff --git a/proxmox-notify/src/endpoints/sendmail.rs b/proxmox-notify/src/endpoints/sendmail.rs
new file mode 100644
index 00000000..dd971438
--- /dev/null
+++ b/proxmox-notify/src/endpoints/sendmail.rs
@@ -0,0 +1,88 @@
+use crate::schema::{COMMENT_SCHEMA, EMAIL_SCHEMA, ENTITY_NAME_SCHEMA};
+use crate::{Endpoint, Error, Notification};
+
+use proxmox_schema::{api, Updater};
+use serde::{Deserialize, Serialize};
+
+pub(crate) const SENDMAIL_TYPENAME: &str = "sendmail";
+
+#[api(
+ properties: {
+ name: {
+ schema: ENTITY_NAME_SCHEMA,
+ },
+ mailto: {
+ type: Array,
+ items: {
+ schema: EMAIL_SCHEMA,
+ },
+ },
+ comment: {
+ optional: true,
+ schema: COMMENT_SCHEMA,
+ },
+ },
+)]
+#[derive(Debug, Serialize, Deserialize, Updater, Default)]
+#[serde(rename_all = "kebab-case")]
+/// Config for Sendmail notification endpoints
+pub struct SendmailConfig {
+ /// Name of the endpoint
+ #[updater(skip)]
+ pub name: String,
+ /// Mail recipients
+ pub mailto: Vec<String>,
+ /// `From` address for the mail
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub from_address: Option<String>,
+ /// Author of the mail
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub author: Option<String>,
+ /// Comment
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub comment: Option<String>,
+}
+
+#[derive(Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case")]
+pub enum DeleteableSendmailProperty {
+ FromAddress,
+ Author,
+ Comment,
+}
+
+/// A sendmail notification endpoint.
+pub struct SendmailEndpoint {
+ pub config: SendmailConfig,
+}
+
+impl Endpoint for SendmailEndpoint {
+ fn send(&self, notification: &Notification) -> Result<(), Error> {
+ let recipients: Vec<&str> = self.config.mailto.iter().map(String::as_str).collect();
+
+ // Note: OX has serious problems displaying text mails,
+ // so we include html as well
+ let html = format!(
+ "<html><body><pre>\n{}\n<pre>",
+ handlebars::html_escape(¬ification.body)
+ );
+
+ // proxmox_sys::email::sendmail will set the author to
+ // "Proxmox Backup Server" if it is not set.
+ let author = self.config.author.as_deref().or(Some(""));
+
+ proxmox_sys::email::sendmail(
+ &recipients,
+ ¬ification.title,
+ Some(¬ification.body),
+ Some(&html),
+ self.config.from_address.as_deref(),
+ author,
+ )
+ .map_err(|err| Error::NotifyFailed(self.config.name.clone(), err.into()))
+ }
+
+ fn name(&self) -> &str {
+ &self.config.name
+ }
+}
diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
index 43feac25..041a4d7a 100644
--- a/proxmox-notify/src/lib.rs
+++ b/proxmox-notify/src/lib.rs
@@ -212,6 +212,24 @@ impl Bus {
pub fn from_config(config: &Config) -> Result<Self, Error> {
let mut endpoints = HashMap::new();
+ // Instantiate endpoints
+
+ #[cfg(feature = "sendmail")]
+ {
+ use endpoints::sendmail::SENDMAIL_TYPENAME;
+ use endpoints::sendmail::{SendmailConfig, SendmailEndpoint};
+ endpoints.extend(
+ parse_endpoints_without_private_config!(
+ config,
+ SendmailConfig,
+ SendmailEndpoint,
+ SENDMAIL_TYPENAME
+ )?
+ .into_iter()
+ .map(|e| (e.name().into(), e)),
+ );
+ }
+
Ok(Bus { endpoints })
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 06/66] notify: api: add API for sendmail endpoints
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (4 preceding siblings ...)
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 05/66] notify: add sendmail plugin Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-18 12:36 ` Wolfgang Bumiller
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 07/66] notify: add gotify endpoint Lukas Wagner
` (62 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/api/mod.rs | 7 +
proxmox-notify/src/api/sendmail.rs | 254 +++++++++++++++++++++++++++++
2 files changed, 261 insertions(+)
create mode 100644 proxmox-notify/src/api/sendmail.rs
diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
index db9ad1ca..4baae899 100644
--- a/proxmox-notify/src/api/mod.rs
+++ b/proxmox-notify/src/api/mod.rs
@@ -5,6 +5,8 @@ use crate::Config;
use serde::Serialize;
pub mod common;
+#[cfg(feature = "sendmail")]
+pub mod sendmail;
#[derive(Debug, Serialize)]
pub struct ApiError {
@@ -83,6 +85,11 @@ fn verify_digest(config: &Config, digest: Option<&[u8]>) -> Result<(), ApiError>
fn endpoint_exists(config: &Config, name: &str) -> bool {
let mut exists = false;
+ #[cfg(feature = "sendmail")]
+ {
+ exists = exists || sendmail::get_endpoint(config, name).is_ok();
+ }
+
exists
}
diff --git a/proxmox-notify/src/api/sendmail.rs b/proxmox-notify/src/api/sendmail.rs
new file mode 100644
index 00000000..8eafe359
--- /dev/null
+++ b/proxmox-notify/src/api/sendmail.rs
@@ -0,0 +1,254 @@
+use crate::api::ApiError;
+use crate::endpoints::sendmail::{
+ DeleteableSendmailProperty, SendmailConfig, SendmailConfigUpdater, SENDMAIL_TYPENAME,
+};
+use crate::Config;
+
+/// Get a list of all sendmail endpoints.
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns a list of all sendmail endpoints or an `ApiError` if the config is erroneous.
+pub fn get_endpoints(config: &Config) -> Result<Vec<SendmailConfig>, ApiError> {
+ config
+ .config
+ .convert_to_typed_array(SENDMAIL_TYPENAME)
+ .map_err(|e| ApiError::internal_server_error("Could not fetch endpoints", Some(e.into())))
+}
+
+/// Get sendmail endpoint with given `name`.
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns the endpoint or an `ApiError` if the endpoint was not found.
+pub fn get_endpoint(config: &Config, name: &str) -> Result<SendmailConfig, ApiError> {
+ config
+ .config
+ .lookup(SENDMAIL_TYPENAME, name)
+ .map_err(|_| ApiError::not_found(format!("endpoint '{name}' not found"), None))
+}
+
+/// Add a new sendmail endpoint.
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns an `ApiError` if an endpoint with the same name already exists,
+/// or if the endpoint could not be saved.
+pub fn add_endpoint(config: &mut Config, endpoint: &SendmailConfig) -> Result<(), ApiError> {
+ if super::endpoint_exists(config, &endpoint.name) {
+ return Err(ApiError::bad_request(
+ format!("endpoint with name '{}' already exists!", &endpoint.name),
+ None,
+ ));
+ }
+
+ config
+ .config
+ .set_data(&endpoint.name, SENDMAIL_TYPENAME, endpoint)
+ .map_err(|e| {
+ ApiError::internal_server_error(
+ format!("could not save endpoint '{}'", endpoint.name),
+ Some(e.into()),
+ )
+ })?;
+
+ Ok(())
+}
+
+/// Update existing sendmail endpoint
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns an `ApiError` if the config could not be saved.
+pub fn update_endpoint(
+ config: &mut Config,
+ name: &str,
+ updater: &SendmailConfigUpdater,
+ delete: Option<&[DeleteableSendmailProperty]>,
+ digest: Option<&[u8]>,
+) -> Result<(), ApiError> {
+ super::verify_digest(config, digest)?;
+
+ let mut endpoint = get_endpoint(config, name)?;
+
+ if let Some(delete) = delete {
+ for deleteable_property in delete {
+ match deleteable_property {
+ DeleteableSendmailProperty::FromAddress => endpoint.from_address = None,
+ DeleteableSendmailProperty::Author => endpoint.author = None,
+ DeleteableSendmailProperty::Comment => endpoint.comment = None,
+ }
+ }
+ }
+
+ if let Some(mailto) = &updater.mailto {
+ endpoint.mailto = mailto.iter().map(String::from).collect();
+ }
+
+ if let Some(from_address) = &updater.from_address {
+ endpoint.from_address = Some(from_address.into());
+ }
+
+ if let Some(author) = &updater.author {
+ endpoint.author = Some(author.into());
+ }
+
+ if let Some(comment) = &updater.comment {
+ endpoint.comment = Some(comment.into());
+ }
+
+ config
+ .config
+ .set_data(name, SENDMAIL_TYPENAME, &endpoint)
+ .map_err(|e| {
+ ApiError::internal_server_error(
+ format!("could not save endpoint '{name}'"),
+ Some(e.into()),
+ )
+ })?;
+
+ Ok(())
+}
+
+/// Delete existing sendmail endpoint
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns an `ApiError` if the endpoint does not exist.
+pub fn delete_endpoint(config: &mut Config, name: &str) -> Result<(), ApiError> {
+ // Check if the endpoint exists
+ let _ = get_endpoint(config, name)?;
+
+ config.config.sections.remove(name);
+
+ Ok(())
+}
+
+#[cfg(test)]
+pub mod tests {
+ use super::*;
+ use crate::api::test_helpers::*;
+
+ pub fn add_sendmail_endpoint_for_test(config: &mut Config, name: &str) -> Result<(), ApiError> {
+ add_endpoint(
+ config,
+ &SendmailConfig {
+ name: name.into(),
+ mailto: vec!["user1@example.com".into()],
+ from_address: Some("from@example.com".into()),
+ author: Some("root".into()),
+ comment: Some("Comment".into()),
+ },
+ )?;
+
+ assert!(get_endpoint(config, name).is_ok());
+ Ok(())
+ }
+
+ #[test]
+ fn test_sendmail_create() -> Result<(), ApiError> {
+ let mut config = empty_config();
+
+ assert_eq!(get_endpoints(&config)?.len(), 0);
+ add_sendmail_endpoint_for_test(&mut config, "sendmail-endpoint")?;
+
+ // Endpoints must have a unique name
+ assert!(add_sendmail_endpoint_for_test(&mut config, "sendmail-endpoint").is_err());
+ assert_eq!(get_endpoints(&config)?.len(), 1);
+ Ok(())
+ }
+
+ #[test]
+ fn test_update_not_existing_returns_error() -> Result<(), ApiError> {
+ let mut config = empty_config();
+
+ assert!(update_endpoint(&mut config, "test", &Default::default(), None, None,).is_err());
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_update_invalid_digest_returns_error() -> Result<(), ApiError> {
+ let mut config = empty_config();
+ add_sendmail_endpoint_for_test(&mut config, "sendmail-endpoint")?;
+
+ assert!(update_endpoint(
+ &mut config,
+ "sendmail-endpoint",
+ &SendmailConfigUpdater {
+ mailto: Some(vec!["user2@example.com".into(), "user3@example.com".into()]),
+ from_address: Some("root@example.com".into()),
+ author: Some("newauthor".into()),
+ comment: Some("new comment".into()),
+ },
+ None,
+ Some(&[0; 32]),
+ )
+ .is_err());
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_sendmail_update() -> Result<(), ApiError> {
+ let mut config = empty_config();
+ add_sendmail_endpoint_for_test(&mut config, "sendmail-endpoint")?;
+
+ let digest = config.digest;
+
+ update_endpoint(
+ &mut config,
+ "sendmail-endpoint",
+ &SendmailConfigUpdater {
+ mailto: Some(vec!["user2@example.com".into(), "user3@example.com".into()]),
+ from_address: Some("root@example.com".into()),
+ author: Some("newauthor".into()),
+ comment: Some("new comment".into()),
+ },
+ None,
+ Some(&digest),
+ )?;
+
+ let endpoint = get_endpoint(&config, "sendmail-endpoint")?;
+
+ assert_eq!(
+ endpoint.mailto,
+ vec![
+ "user2@example.com".to_string(),
+ "user3@example.com".to_string()
+ ]
+ );
+ assert_eq!(endpoint.from_address, Some("root@example.com".to_string()));
+ assert_eq!(endpoint.author, Some("newauthor".to_string()));
+ assert_eq!(endpoint.comment, Some("new comment".to_string()));
+
+ // Test property deletion
+ update_endpoint(
+ &mut config,
+ "sendmail-endpoint",
+ &Default::default(),
+ Some(&[
+ DeleteableSendmailProperty::FromAddress,
+ DeleteableSendmailProperty::Author,
+ ]),
+ None,
+ )?;
+
+ let endpoint = get_endpoint(&config, "sendmail-endpoint")?;
+
+ assert_eq!(endpoint.from_address, None);
+ assert_eq!(endpoint.author, None);
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_sendmail_delete() -> Result<(), ApiError> {
+ let mut config = empty_config();
+ add_sendmail_endpoint_for_test(&mut config, "sendmail-endpoint")?;
+
+ delete_endpoint(&mut config, "sendmail-endpoint")?;
+ assert!(delete_endpoint(&mut config, "sendmail-endpoint").is_err());
+ assert_eq!(get_endpoints(&config)?.len(), 0);
+
+ Ok(())
+ }
+}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 07/66] notify: add gotify endpoint
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (5 preceding siblings ...)
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 06/66] notify: api: add API for sendmail endpoints Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 08/66] notify: api: add API for gotify endpoints Lukas Wagner
` (61 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
Add an endpoint for Gotify [1], showing the how easy it is to add new
endpoint implementations.
[1] https://gotify.net/
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/Cargo.toml | 6 +-
proxmox-notify/src/config.rs | 23 +++++
proxmox-notify/src/endpoints/gotify.rs | 116 +++++++++++++++++++++++++
proxmox-notify/src/endpoints/mod.rs | 2 +
proxmox-notify/src/lib.rs | 18 +++-
5 files changed, 162 insertions(+), 3 deletions(-)
create mode 100644 proxmox-notify/src/endpoints/gotify.rs
diff --git a/proxmox-notify/Cargo.toml b/proxmox-notify/Cargo.toml
index f7329295..738674ae 100644
--- a/proxmox-notify/Cargo.toml
+++ b/proxmox-notify/Cargo.toml
@@ -12,6 +12,7 @@ handlebars = { workspace = true, optional = true }
lazy_static.workspace = true
log.workspace = true
openssl.workspace = true
+proxmox-http = { workspace = true, features = ["client-sync"], optional = true }
proxmox-schema = { workspace = true, features = ["api-macro"]}
proxmox-section-config = { workspace = true }
proxmox-sys = { workspace = true, optional = true }
@@ -20,5 +21,6 @@ serde = { workspace = true, features = ["derive"]}
serde_json.workspace = true
[features]
-default = ["sendmail"]
-sendmail = ["dep:handlebars", "dep:proxmox-sys"]
\ No newline at end of file
+default = ["sendmail", "gotify"]
+sendmail = ["dep:handlebars", "dep:proxmox-sys"]
+gotify = ["dep:proxmox-http"]
diff --git a/proxmox-notify/src/config.rs b/proxmox-notify/src/config.rs
index 6269ec3c..5508c916 100644
--- a/proxmox-notify/src/config.rs
+++ b/proxmox-notify/src/config.rs
@@ -24,6 +24,17 @@ fn config_init() -> SectionConfig {
SENDMAIL_SCHEMA,
));
}
+ #[cfg(feature = "gotify")]
+ {
+ use crate::endpoints::gotify::{GotifyConfig, GOTIFY_TYPENAME};
+
+ const GOTIFY_SCHEMA: &ObjectSchema = GotifyConfig::API_SCHEMA.unwrap_object_schema();
+ config.register_plugin(SectionConfigPlugin::new(
+ GOTIFY_TYPENAME.to_string(),
+ Some(String::from("name")),
+ GOTIFY_SCHEMA,
+ ));
+ }
config
}
@@ -31,6 +42,18 @@ fn config_init() -> SectionConfig {
fn private_config_init() -> SectionConfig {
let mut config = SectionConfig::new(&BACKEND_NAME_SCHEMA);
+ #[cfg(feature = "gotify")]
+ {
+ use crate::endpoints::gotify::{GotifyPrivateConfig, GOTIFY_TYPENAME};
+
+ const GOTIFY_SCHEMA: &ObjectSchema = GotifyPrivateConfig::API_SCHEMA.unwrap_object_schema();
+ config.register_plugin(SectionConfigPlugin::new(
+ GOTIFY_TYPENAME.to_string(),
+ Some(String::from("name")),
+ GOTIFY_SCHEMA,
+ ));
+ }
+
config
}
diff --git a/proxmox-notify/src/endpoints/gotify.rs b/proxmox-notify/src/endpoints/gotify.rs
new file mode 100644
index 00000000..57b330c2
--- /dev/null
+++ b/proxmox-notify/src/endpoints/gotify.rs
@@ -0,0 +1,116 @@
+use std::collections::HashMap;
+
+use crate::schema::{COMMENT_SCHEMA, ENTITY_NAME_SCHEMA};
+use crate::{Endpoint, Error, Notification, Severity};
+
+use serde::{Deserialize, Serialize};
+
+use proxmox_http::client::sync::Client;
+use proxmox_http::{HttpClient, HttpOptions};
+use proxmox_schema::{api, Updater};
+
+#[derive(Serialize)]
+struct GotifyMessageBody<'a> {
+ title: &'a str,
+ message: &'a str,
+ priority: u32,
+}
+
+fn severity_to_priority(level: Severity) -> u32 {
+ match level {
+ Severity::Info => 1,
+ Severity::Notice => 3,
+ Severity::Warning => 5,
+ Severity::Error => 9,
+ }
+}
+
+pub(crate) const GOTIFY_TYPENAME: &str = "gotify";
+
+#[api(
+ properties: {
+ name: {
+ schema: ENTITY_NAME_SCHEMA,
+ },
+ comment: {
+ optional: true,
+ schema: COMMENT_SCHEMA,
+ },
+ }
+)]
+#[derive(Serialize, Deserialize, Updater, Default)]
+#[serde(rename_all = "kebab-case")]
+/// Config for Gotify notification endpoints
+pub struct GotifyConfig {
+ /// Name of the endpoint
+ #[updater(skip)]
+ pub name: String,
+ /// Gotify Server URL
+ pub server: String,
+ /// Comment
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub comment: Option<String>,
+}
+
+#[api()]
+#[derive(Serialize, Deserialize, Clone, Updater)]
+#[serde(rename_all = "kebab-case")]
+/// Private configuration for Gotify notification endpoints.
+/// This config will be saved to a separate configuration file with stricter
+/// permissions (root:root 0600)
+pub struct GotifyPrivateConfig {
+ /// Name of the endpoint
+ #[updater(skip)]
+ pub name: String,
+ /// Authentication token
+ pub token: String,
+}
+
+/// A Gotify notification endpoint.
+pub struct GotifyEndpoint {
+ pub config: GotifyConfig,
+ pub private_config: GotifyPrivateConfig,
+}
+
+#[derive(Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case")]
+pub enum DeleteableGotifyProperty {
+ Comment,
+}
+
+impl Endpoint for GotifyEndpoint {
+ fn send(&self, notification: &Notification) -> Result<(), Error> {
+ // TODO: What about proxy configuration?
+ let client = Client::new(HttpOptions::default());
+
+ let uri = format!("{}/message", self.config.server);
+
+ let body = GotifyMessageBody {
+ title: ¬ification.title,
+ message: ¬ification.body,
+ priority: severity_to_priority(notification.severity),
+ };
+
+ let body = serde_json::to_vec(&body)
+ .map_err(|err| Error::NotifyFailed(self.name().to_string(), err.into()))?;
+ let extra_headers = HashMap::from([(
+ "Authorization".into(),
+ format!("Bearer {}", self.private_config.token),
+ )]);
+
+ client
+ .post(
+ &uri,
+ Some(body.as_slice()),
+ Some("application/json"),
+ Some(&extra_headers),
+ )
+ .map_err(|err| Error::NotifyFailed(self.name().to_string(), err.into()))?;
+
+ Ok(())
+ }
+
+ fn name(&self) -> &str {
+ &self.config.name
+ }
+}
diff --git a/proxmox-notify/src/endpoints/mod.rs b/proxmox-notify/src/endpoints/mod.rs
index dd80d9bc..d1cec654 100644
--- a/proxmox-notify/src/endpoints/mod.rs
+++ b/proxmox-notify/src/endpoints/mod.rs
@@ -1,2 +1,4 @@
+#[cfg(feature = "gotify")]
+pub mod gotify;
#[cfg(feature = "sendmail")]
pub mod sendmail;
diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
index 041a4d7a..83991add 100644
--- a/proxmox-notify/src/lib.rs
+++ b/proxmox-notify/src/lib.rs
@@ -213,7 +213,6 @@ impl Bus {
let mut endpoints = HashMap::new();
// Instantiate endpoints
-
#[cfg(feature = "sendmail")]
{
use endpoints::sendmail::SENDMAIL_TYPENAME;
@@ -230,6 +229,23 @@ impl Bus {
);
}
+ #[cfg(feature = "gotify")]
+ {
+ use endpoints::gotify::GOTIFY_TYPENAME;
+ use endpoints::gotify::{GotifyConfig, GotifyEndpoint, GotifyPrivateConfig};
+ endpoints.extend(
+ parse_endpoints_with_private_config!(
+ config,
+ GotifyConfig,
+ GotifyPrivateConfig,
+ GotifyEndpoint,
+ GOTIFY_TYPENAME
+ )?
+ .into_iter()
+ .map(|e| (e.name().into(), e)),
+ );
+ }
+
Ok(Bus { endpoints })
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 08/66] notify: api: add API for gotify endpoints
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (6 preceding siblings ...)
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 07/66] notify: add gotify endpoint Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-18 12:44 ` Wolfgang Bumiller
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 09/66] notify: add notification groups Lukas Wagner
` (60 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/api/gotify.rs | 284 +++++++++++++++++++++++++++++++
proxmox-notify/src/api/mod.rs | 6 +
2 files changed, 290 insertions(+)
create mode 100644 proxmox-notify/src/api/gotify.rs
diff --git a/proxmox-notify/src/api/gotify.rs b/proxmox-notify/src/api/gotify.rs
new file mode 100644
index 00000000..fcc1aabf
--- /dev/null
+++ b/proxmox-notify/src/api/gotify.rs
@@ -0,0 +1,284 @@
+use crate::api::ApiError;
+use crate::endpoints::gotify::{
+ DeleteableGotifyProperty, GotifyConfig, GotifyConfigUpdater, GotifyPrivateConfig,
+ GotifyPrivateConfigUpdater, GOTIFY_TYPENAME,
+};
+use crate::Config;
+
+/// Get a list of all gotify endpoints.
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns a list of all gotify endpoints or an `ApiError` if the config is erroneous.
+pub fn get_endpoints(config: &Config) -> Result<Vec<GotifyConfig>, ApiError> {
+ config
+ .config
+ .convert_to_typed_array(GOTIFY_TYPENAME)
+ .map_err(|e| ApiError::internal_server_error("Could not fetch endpoints", Some(e.into())))
+}
+
+/// Get gotify endpoint with given `name`
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns the endpoint or an `ApiError` if the endpoint was not found.
+pub fn get_endpoint(config: &Config, name: &str) -> Result<GotifyConfig, ApiError> {
+ config
+ .config
+ .lookup(GOTIFY_TYPENAME, name)
+ .map_err(|_| ApiError::not_found(format!("endpoint '{name}' not found"), None))
+}
+
+/// Add a new gotify endpoint.
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns an `ApiError` if an endpoint with the same name already exists,
+/// or if the endpoint could not be saved.
+pub fn add_endpoint(
+ config: &mut Config,
+ endpoint_config: &GotifyConfig,
+ private_endpoint_config: &GotifyPrivateConfig,
+) -> Result<(), ApiError> {
+ if endpoint_config.name != private_endpoint_config.name {
+ // Programming error by the user of the crate, thus we panic
+ panic!("name for endpoint config and private config must be identical");
+ }
+
+ if super::endpoint_exists(config, &endpoint_config.name) {
+ return Err(ApiError::bad_request(
+ format!(
+ "endpoint with name '{}' already exists!",
+ endpoint_config.name
+ ),
+ None,
+ ));
+ }
+
+ set_private_config_entry(config, private_endpoint_config)?;
+
+ config
+ .config
+ .set_data(&endpoint_config.name, GOTIFY_TYPENAME, endpoint_config)
+ .map_err(|e| {
+ ApiError::internal_server_error(
+ format!("could not save endpoint '{}'", endpoint_config.name),
+ Some(e.into()),
+ )
+ })?;
+
+ Ok(())
+}
+
+/// Update existing gotify endpoint
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns an `ApiError` if the config could not be saved.
+pub fn update_endpoint(
+ config: &mut Config,
+ name: &str,
+ endpoint_config_updater: &GotifyConfigUpdater,
+ private_endpoint_config_updater: &GotifyPrivateConfigUpdater,
+ delete: Option<&[DeleteableGotifyProperty]>,
+ digest: Option<&[u8]>,
+) -> Result<(), ApiError> {
+ super::verify_digest(config, digest)?;
+
+ let mut endpoint = get_endpoint(config, name)?;
+
+ if let Some(delete) = delete {
+ for deleteable_property in delete {
+ match deleteable_property {
+ DeleteableGotifyProperty::Comment => endpoint.comment = None,
+ }
+ }
+ }
+
+ if let Some(server) = &endpoint_config_updater.server {
+ endpoint.server = server.into();
+ }
+
+ if let Some(token) = &private_endpoint_config_updater.token {
+ set_private_config_entry(
+ config,
+ &GotifyPrivateConfig {
+ name: name.into(),
+ token: token.into(),
+ },
+ )?;
+ }
+
+ if let Some(comment) = &endpoint_config_updater.comment {
+ endpoint.comment = Some(comment.into());
+ }
+
+ config
+ .config
+ .set_data(name, GOTIFY_TYPENAME, &endpoint)
+ .map_err(|e| {
+ ApiError::internal_server_error(
+ format!("could not save endpoint '{name}'"),
+ Some(e.into()),
+ )
+ })?;
+
+ Ok(())
+}
+
+/// Delete existing gotify endpoint
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns an `ApiError` if the endpoint does not exist.
+pub fn delete_gotify_endpoint(config: &mut Config, name: &str) -> Result<(), ApiError> {
+ // Check if the endpoint exists
+ let _ = get_endpoint(config, name)?;
+
+ remove_private_config_entry(config, name)?;
+ config.config.sections.remove(name);
+
+ Ok(())
+}
+
+fn set_private_config_entry(
+ config: &mut Config,
+ private_config: &GotifyPrivateConfig,
+) -> Result<(), ApiError> {
+ config
+ .private_config
+ .set_data(&private_config.name, GOTIFY_TYPENAME, private_config)
+ .map_err(|e| {
+ ApiError::internal_server_error(
+ format!(
+ "could not save private config for endpoint '{}'",
+ private_config.name
+ ),
+ Some(e.into()),
+ )
+ })
+}
+
+fn remove_private_config_entry(config: &mut Config, name: &str) -> Result<(), ApiError> {
+ config.private_config.sections.remove(name);
+ Ok(())
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use crate::api::test_helpers::empty_config;
+
+ pub fn add_default_gotify_endpoint(config: &mut Config) -> Result<(), ApiError> {
+ add_endpoint(
+ config,
+ &GotifyConfig {
+ name: "gotify-endpoint".into(),
+ server: "localhost".into(),
+ comment: Some("comment".into()),
+ },
+ &GotifyPrivateConfig {
+ name: "gotify-endpoint".into(),
+ token: "supersecrettoken".into(),
+ },
+ )?;
+
+ assert!(get_endpoint(config, "gotify-endpoint").is_ok());
+ Ok(())
+ }
+
+ #[test]
+ fn test_update_not_existing_returns_error() -> Result<(), ApiError> {
+ let mut config = empty_config();
+
+ assert!(update_endpoint(
+ &mut config,
+ "test",
+ &Default::default(),
+ &Default::default(),
+ None,
+ None
+ )
+ .is_err());
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_update_invalid_digest_returns_error() -> Result<(), ApiError> {
+ let mut config = empty_config();
+ add_default_gotify_endpoint(&mut config)?;
+
+ assert!(update_endpoint(
+ &mut config,
+ "gotify-endpoint",
+ &Default::default(),
+ &Default::default(),
+ None,
+ Some(&[0; 32])
+ )
+ .is_err());
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_gotify_update() -> Result<(), ApiError> {
+ let mut config = empty_config();
+ add_default_gotify_endpoint(&mut config)?;
+
+ let digest = config.digest;
+
+ update_endpoint(
+ &mut config,
+ "gotify-endpoint",
+ &GotifyConfigUpdater {
+ server: Some("newhost".into()),
+ comment: Some("newcomment".into()),
+ },
+ &GotifyPrivateConfigUpdater {
+ token: Some("changedtoken".into()),
+ },
+ None,
+ Some(&digest),
+ )?;
+
+ let endpoint = get_endpoint(&config, "gotify-endpoint")?;
+
+ assert_eq!(endpoint.server, "newhost".to_string());
+
+ let token = config
+ .private_config
+ .lookup::<GotifyPrivateConfig>(GOTIFY_TYPENAME, "gotify-endpoint")
+ .unwrap()
+ .token;
+
+ assert_eq!(token, "changedtoken".to_string());
+ assert_eq!(endpoint.comment, Some("newcomment".to_string()));
+
+ // Test property deletion
+ update_endpoint(
+ &mut config,
+ "gotify-endpoint",
+ &Default::default(),
+ &Default::default(),
+ Some(&[DeleteableGotifyProperty::Comment]),
+ None,
+ )?;
+
+ let endpoint = get_endpoint(&config, "gotify-endpoint")?;
+ assert_eq!(endpoint.comment, None);
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_gotify_endpoint_delete() -> Result<(), ApiError> {
+ let mut config = empty_config();
+ add_default_gotify_endpoint(&mut config)?;
+
+ delete_gotify_endpoint(&mut config, "gotify-endpoint")?;
+ assert!(delete_gotify_endpoint(&mut config, "gotify-endpoint").is_err());
+ assert_eq!(get_endpoints(&config)?.len(), 0);
+
+ Ok(())
+ }
+}
diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
index 4baae899..1a4cc873 100644
--- a/proxmox-notify/src/api/mod.rs
+++ b/proxmox-notify/src/api/mod.rs
@@ -5,6 +5,8 @@ use crate::Config;
use serde::Serialize;
pub mod common;
+#[cfg(feature = "gotify")]
+pub mod gotify;
#[cfg(feature = "sendmail")]
pub mod sendmail;
@@ -89,6 +91,10 @@ fn endpoint_exists(config: &Config, name: &str) -> bool {
{
exists = exists || sendmail::get_endpoint(config, name).is_ok();
}
+ #[cfg(feature = "gotify")]
+ {
+ exists = exists || gotify::get_endpoint(config, name).is_ok();
+ }
exists
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 09/66] notify: add notification groups
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (7 preceding siblings ...)
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 08/66] notify: api: add API for gotify endpoints Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 10/66] notify: api: add API for groups Lukas Wagner
` (59 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
When notifying via a group, all endpoints contained in that group
will send out the notification.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/config.rs | 9 ++
proxmox-notify/src/group.rs | 40 +++++++++
proxmox-notify/src/lib.rs | 170 ++++++++++++++++++++++++++++-------
3 files changed, 189 insertions(+), 30 deletions(-)
create mode 100644 proxmox-notify/src/group.rs
diff --git a/proxmox-notify/src/config.rs b/proxmox-notify/src/config.rs
index 5508c916..53817254 100644
--- a/proxmox-notify/src/config.rs
+++ b/proxmox-notify/src/config.rs
@@ -2,6 +2,7 @@ use lazy_static::lazy_static;
use proxmox_schema::{ApiType, ObjectSchema};
use proxmox_section_config::{SectionConfig, SectionConfigData, SectionConfigPlugin};
+use crate::group::{GroupConfig, GROUP_TYPENAME};
use crate::schema::BACKEND_NAME_SCHEMA;
use crate::Error;
@@ -36,6 +37,14 @@ fn config_init() -> SectionConfig {
));
}
+ const GROUP_SCHEMA: &ObjectSchema = GroupConfig::API_SCHEMA.unwrap_object_schema();
+
+ config.register_plugin(SectionConfigPlugin::new(
+ GROUP_TYPENAME.to_string(),
+ Some(String::from("name")),
+ GROUP_SCHEMA,
+ ));
+
config
}
diff --git a/proxmox-notify/src/group.rs b/proxmox-notify/src/group.rs
new file mode 100644
index 00000000..d9ded2dd
--- /dev/null
+++ b/proxmox-notify/src/group.rs
@@ -0,0 +1,40 @@
+use crate::schema::COMMENT_SCHEMA;
+use proxmox_schema::{api, Updater};
+use serde::{Deserialize, Serialize};
+
+pub(crate) const GROUP_TYPENAME: &str = "group";
+
+#[api(
+ properties: {
+ "endpoint": {
+ type: Array,
+ items: {
+ description: "Name of the included endpoint(s)",
+ type: String,
+ },
+ },
+ comment: {
+ optional: true,
+ schema: COMMENT_SCHEMA,
+ },
+ },
+)]
+#[derive(Debug, Serialize, Deserialize, Updater, Default)]
+#[serde(rename_all = "kebab-case")]
+/// Config for notification channels
+pub struct GroupConfig {
+ /// Name of the channel
+ #[updater(skip)]
+ pub name: String,
+ /// Endpoints for this channel
+ pub endpoint: Vec<String>,
+ /// Comment
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub comment: Option<String>,
+}
+
+#[derive(Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case")]
+pub enum DeleteableGroupProperty {
+ Comment,
+}
diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
index 83991add..35d5208b 100644
--- a/proxmox-notify/src/lib.rs
+++ b/proxmox-notify/src/lib.rs
@@ -1,6 +1,7 @@
use std::collections::HashMap;
use std::fmt::Display;
+use group::{GroupConfig, GROUP_TYPENAME};
use proxmox_schema::api;
use proxmox_section_config::SectionConfigData;
use serde::{Deserialize, Serialize};
@@ -12,6 +13,7 @@ use std::error::Error as StdError;
pub mod api;
mod config;
pub mod endpoints;
+pub mod group;
pub mod schema;
#[derive(Debug)]
@@ -20,6 +22,7 @@ pub enum Error {
ConfigDeserialization(Box<dyn StdError + Send + Sync + 'static>),
NotifyFailed(String, Box<dyn StdError + Send + Sync + 'static>),
TargetDoesNotExist(String),
+ TargetTestFailed(Vec<Box<dyn StdError + Send + Sync + 'static>>),
}
impl Display for Error {
@@ -37,6 +40,13 @@ impl Display for Error {
Error::TargetDoesNotExist(target) => {
write!(f, "notification target '{target}' does not exist")
}
+ Error::TargetTestFailed(errs) => {
+ for err in errs {
+ writeln!(f, "{err}")?;
+ }
+
+ Ok(())
+ }
}
}
}
@@ -48,6 +58,7 @@ impl StdError for Error {
Error::ConfigDeserialization(err) => Some(&**err),
Error::NotifyFailed(_, err) => Some(&**err),
Error::TargetDoesNotExist(_) => None,
+ Error::TargetTestFailed(errs) => Some(&*errs[0]),
}
}
}
@@ -149,6 +160,7 @@ impl Config {
#[derive(Default)]
pub struct Bus {
endpoints: HashMap<String, Box<dyn Endpoint>>,
+ groups: HashMap<String, GroupConfig>,
}
#[allow(unused_macros)]
@@ -246,7 +258,15 @@ impl Bus {
);
}
- Ok(Bus { endpoints })
+ let groups: HashMap<String, GroupConfig> = config
+ .config
+ .convert_to_typed_array(GROUP_TYPENAME)
+ .map_err(|err| Error::ConfigDeserialization(err.into()))?
+ .into_iter()
+ .map(|group: GroupConfig| (group.name.clone(), group))
+ .collect();
+
+ Ok(Bus { endpoints, groups })
}
#[cfg(test)]
@@ -254,39 +274,76 @@ impl Bus {
self.endpoints.insert(endpoint.name().to_string(), endpoint);
}
- pub fn send(&self, target: &str, notification: &Notification) -> Result<(), Error> {
- log::info!(
- "sending notification with title '{title}'",
- title = notification.title
- );
-
- let endpoint = self
- .endpoints
- .get(target)
- .ok_or(Error::TargetDoesNotExist(target.into()))?;
+ #[cfg(test)]
+ pub fn add_group(&mut self, group: GroupConfig) {
+ self.groups.insert(group.name.clone(), group);
+ }
- endpoint.send(notification).unwrap_or_else(|e| {
- log::error!(
- "could not notfiy via endpoint `{name}`: {e}",
- name = endpoint.name()
- )
- });
+ /// Send a notification to a given target (endpoint or group).
+ ///
+ /// Any errors will not be returned but only logged.
+ pub fn send(&self, endpoint_or_group: &str, notification: &Notification) {
+ if let Some(group) = self.groups.get(endpoint_or_group) {
+ for endpoint in &group.endpoint {
+ self.send_via_single_endpoint(endpoint, notification);
+ }
+ } else {
+ self.send_via_single_endpoint(endpoint_or_group, notification);
+ }
+ }
- Ok(())
+ fn send_via_single_endpoint(&self, endpoint: &str, notification: &Notification) {
+ if let Some(endpoint) = self.endpoints.get(endpoint) {
+ if let Err(e) = endpoint.send(notification) {
+ // Only log on errors, do not propagate fail to the caller.
+ log::error!(
+ "could not notify via target `{name}`: {e}",
+ name = endpoint.name()
+ );
+ } else {
+ log::info!("notified via endpoint `{name}`", name = endpoint.name());
+ }
+ } else {
+ log::error!("could not notify via endpoint '{endpoint}', it does not exist");
+ }
}
+ /// Send a test notification to a target (endpoint or group).
+ ///
+ /// In contrast to the `send` function, this function will return
+ /// any errors to the caller.
pub fn test_target(&self, target: &str) -> Result<(), Error> {
- let endpoint = self
- .endpoints
- .get(target)
- .ok_or(Error::TargetDoesNotExist(target.into()))?;
-
- endpoint.send(&Notification {
+ let notification = Notification {
severity: Severity::Info,
title: "Test notification".into(),
body: "This is a test of the notification target '{{ target }}'".into(),
properties: Some(json!({ "target": target })),
- })?;
+ };
+
+ let mut errors: Vec<Box<dyn StdError + Send + Sync>> = Vec::new();
+
+ let mut my_send = |target: &str| -> Result<(), Error> {
+ if let Some(endpoint) = self.endpoints.get(target) {
+ if let Err(e) = endpoint.send(¬ification) {
+ errors.push(Box::new(e));
+ }
+ } else {
+ return Err(Error::TargetDoesNotExist(target.to_string()));
+ }
+ Ok(())
+ };
+
+ if let Some(group) = self.groups.get(target) {
+ for endpoint_name in &group.endpoint {
+ my_send(endpoint_name)?;
+ }
+ } else {
+ my_send(target)?;
+ }
+
+ if !errors.is_empty() {
+ return Err(Error::TargetTestFailed(errors));
+ }
Ok(())
}
@@ -300,6 +357,7 @@ mod tests {
#[derive(Default, Clone)]
struct MockEndpoint {
+ name: &'static str,
messages: Rc<RefCell<Vec<Notification>>>,
}
@@ -311,11 +369,18 @@ mod tests {
}
fn name(&self) -> &str {
- "mock-endpoint"
+ self.name
}
}
impl MockEndpoint {
+ fn new(name: &'static str, filter: Option<String>) -> Self {
+ Self {
+ name,
+ ..Default::default()
+ }
+ }
+
fn messages(&self) -> Vec<Notification> {
self.messages.borrow().clone()
}
@@ -323,24 +388,69 @@ mod tests {
#[test]
fn test_add_mock_endpoint() -> Result<(), Error> {
- let mock = MockEndpoint::default();
+ let mock = MockEndpoint::new("endpoint", None);
let mut bus = Bus::default();
-
bus.add_endpoint(Box::new(mock.clone()));
+ // Send directly to endpoint
bus.send(
- "mock-endpoint",
+ "endpoint",
&Notification {
title: "Title".into(),
body: "Body".into(),
severity: Severity::Info,
properties: Default::default(),
},
- )?;
+ );
let messages = mock.messages();
assert_eq!(messages.len(), 1);
Ok(())
}
+
+ #[test]
+ fn test_groups() -> Result<(), Error> {
+ let endpoint1 = MockEndpoint::new("mock1", None);
+ let endpoint2 = MockEndpoint::new("mock2", None);
+
+ let mut bus = Bus::default();
+
+ bus.add_group(GroupConfig {
+ name: "group1".to_string(),
+ endpoint: vec!["mock1".into()],
+ comment: None,
+ });
+
+ bus.add_group(GroupConfig {
+ name: "group2".to_string(),
+ endpoint: vec!["mock2".into()],
+ comment: None,
+ });
+
+ bus.add_endpoint(Box::new(endpoint1.clone()));
+ bus.add_endpoint(Box::new(endpoint2.clone()));
+
+ let send_to_group = |channel| {
+ bus.send(
+ channel,
+ &Notification {
+ title: "Title".into(),
+ body: "Body".into(),
+ severity: Severity::Info,
+ properties: Default::default(),
+ },
+ )
+ };
+
+ send_to_group("group1");
+ assert_eq!(endpoint1.messages().len(), 1);
+ assert_eq!(endpoint2.messages().len(), 0);
+
+ send_to_group("group2");
+ assert_eq!(endpoint1.messages().len(), 1);
+ assert_eq!(endpoint2.messages().len(), 1);
+
+ Ok(())
+ }
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 10/66] notify: api: add API for groups
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (8 preceding siblings ...)
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 09/66] notify: add notification groups Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 11/66] notify: add notification filter mechanism Lukas Wagner
` (58 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/api/group.rs | 264 ++++++++++++++++++++++++++++++++
proxmox-notify/src/api/mod.rs | 1 +
2 files changed, 265 insertions(+)
create mode 100644 proxmox-notify/src/api/group.rs
diff --git a/proxmox-notify/src/api/group.rs b/proxmox-notify/src/api/group.rs
new file mode 100644
index 00000000..cc847364
--- /dev/null
+++ b/proxmox-notify/src/api/group.rs
@@ -0,0 +1,264 @@
+use crate::api::ApiError;
+use crate::group::{DeleteableGroupProperty, GroupConfig, GroupConfigUpdater, GROUP_TYPENAME};
+use crate::Config;
+
+/// Get all notification groups
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns a list of all groups or an `ApiError` if the config is erroneous.
+pub fn get_groups(config: &Config) -> Result<Vec<GroupConfig>, ApiError> {
+ config
+ .config
+ .convert_to_typed_array(GROUP_TYPENAME)
+ .map_err(|e| ApiError::internal_server_error("Could not fetch groups", Some(e.into())))
+}
+
+/// Get group with given `name`
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns the endpoint or an `ApiError` if the group was not found.
+pub fn get_group(config: &Config, name: &str) -> Result<GroupConfig, ApiError> {
+ config
+ .config
+ .lookup(GROUP_TYPENAME, name)
+ .map_err(|_| ApiError::not_found(format!("group '{name}' not found"), None))
+}
+
+/// Add a new group.
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns an `ApiError` if a group with the same name already exists, or
+/// if the group could not be saved
+pub fn add_group(config: &mut Config, group_config: &GroupConfig) -> Result<(), ApiError> {
+ if get_group(config, &group_config.name).is_ok() {
+ return Err(ApiError::bad_request(
+ format!("group '{}' already exists", group_config.name),
+ None,
+ ));
+ }
+
+ if group_config.endpoint.is_empty() {
+ return Err(ApiError::bad_request(
+ "group must contain at least one endpoint",
+ None,
+ ));
+ }
+
+ check_if_endpoints_exist(config, &group_config.endpoint)?;
+
+ config
+ .config
+ .set_data(&group_config.name, GROUP_TYPENAME, group_config)
+ .map_err(|e| {
+ ApiError::internal_server_error(
+ format!("could not save group '{}'", group_config.name),
+ Some(e.into()),
+ )
+ })?;
+
+ Ok(())
+}
+
+/// Update existing group
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns an `ApiError` if the config could not be saved.
+pub fn update_group(
+ config: &mut Config,
+ name: &str,
+ updater: &GroupConfigUpdater,
+ delete: Option<&[DeleteableGroupProperty]>,
+ digest: Option<&[u8]>,
+) -> Result<(), ApiError> {
+ super::verify_digest(config, digest)?;
+
+ let mut group = get_group(config, name)?;
+
+ if let Some(delete) = delete {
+ for deleteable_property in delete {
+ match deleteable_property {
+ DeleteableGroupProperty::Comment => group.comment = None,
+ }
+ }
+ }
+
+ if let Some(endpoints) = &updater.endpoint {
+ check_if_endpoints_exist(config, endpoints)?;
+ if endpoints.is_empty() {
+ return Err(ApiError::bad_request(
+ "group must contain at least one endpoint",
+ None,
+ ));
+ }
+ group.endpoint = endpoints.iter().map(Into::into).collect()
+ }
+
+ if let Some(comment) = &updater.comment {
+ group.comment = Some(comment.into());
+ }
+
+ config
+ .config
+ .set_data(name, GROUP_TYPENAME, &group)
+ .map_err(|e| {
+ ApiError::internal_server_error(
+ format!("could not save group '{name}'"),
+ Some(e.into()),
+ )
+ })?;
+
+ Ok(())
+}
+
+/// Delete existing group
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns an `ApiError` if the group does not exist.
+pub fn delete_group(config: &mut Config, name: &str) -> Result<(), ApiError> {
+ // Check if the group exists
+ let _ = get_group(config, name)?;
+
+ config.config.sections.remove(name);
+
+ Ok(())
+}
+
+fn check_if_endpoints_exist(config: &Config, endpoints: &[String]) -> Result<(), ApiError> {
+ for endpoint in endpoints {
+ if !super::endpoint_exists(config, endpoint) {
+ return Err(ApiError::not_found(
+ format!("endoint '{endpoint}' does not exist"),
+ None,
+ ));
+ }
+ }
+
+ Ok(())
+}
+
+// groups cannot be empty, so only build the tests if we have the
+// sendmail endpoint available
+#[cfg(all(test, feature = "sendmail"))]
+mod tests {
+ use super::*;
+ use crate::api::sendmail::tests::add_sendmail_endpoint_for_test;
+ use crate::api::test_helpers::*;
+
+ fn add_default_group(config: &mut Config) -> Result<(), ApiError> {
+ add_sendmail_endpoint_for_test(config, "test")?;
+
+ add_group(
+ config,
+ &GroupConfig {
+ name: "group1".into(),
+ endpoint: vec!["test".to_string()],
+ comment: None,
+ },
+ )?;
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_add_group_fails_if_endpoint_does_not_exist() {
+ let mut config = empty_config();
+ assert!(add_group(
+ &mut config,
+ &GroupConfig {
+ name: "group1".into(),
+ endpoint: vec!["foo".into()],
+ comment: None,
+ },
+ )
+ .is_err());
+ }
+
+ #[test]
+ fn test_add_group() -> Result<(), ApiError> {
+ let mut config = empty_config();
+ assert!(add_default_group(&mut config).is_ok());
+ Ok(())
+ }
+
+ #[test]
+ fn test_update_group_fails_if_endpoint_does_not_exist() -> Result<(), ApiError> {
+ let mut config = empty_config();
+ add_default_group(&mut config)?;
+
+ assert!(update_group(
+ &mut config,
+ "group1",
+ &GroupConfigUpdater {
+ endpoint: Some(vec!["foo".into()]),
+ ..Default::default()
+ },
+ None,
+ None
+ )
+ .is_err());
+ Ok(())
+ }
+
+ #[test]
+ fn test_update_group_fails_if_digest_invalid() -> Result<(), ApiError> {
+ let mut config = empty_config();
+ add_default_group(&mut config)?;
+
+ assert!(update_group(
+ &mut config,
+ "group1",
+ &Default::default(),
+ None,
+ Some(&[0u8; 32])
+ )
+ .is_err());
+ Ok(())
+ }
+
+ #[test]
+ fn test_update_group() -> Result<(), ApiError> {
+ let mut config = empty_config();
+ add_default_group(&mut config)?;
+
+ assert!(update_group(
+ &mut config,
+ "group1",
+ &GroupConfigUpdater {
+ endpoint: None,
+ comment: Some("newcomment".into())
+ },
+ None,
+ None,
+ )
+ .is_ok());
+ let group = get_group(&config, "group1")?;
+ assert_eq!(group.comment, Some("newcomment".into()));
+
+ assert!(update_group(
+ &mut config,
+ "group1",
+ &Default::default(),
+ Some(&[DeleteableGroupProperty::Comment]),
+ None
+ )
+ .is_ok());
+ let group = get_group(&config, "group1")?;
+ assert_eq!(group.comment, None);
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_group_delete() -> Result<(), ApiError> {
+ let mut config = empty_config();
+ add_default_group(&mut config)?;
+
+ assert!(delete_group(&mut config, "group1").is_ok());
+ assert!(delete_group(&mut config, "group1").is_err());
+
+ Ok(())
+ }
+}
diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
index 1a4cc873..ac917910 100644
--- a/proxmox-notify/src/api/mod.rs
+++ b/proxmox-notify/src/api/mod.rs
@@ -7,6 +7,7 @@ use serde::Serialize;
pub mod common;
#[cfg(feature = "gotify")]
pub mod gotify;
+pub mod group;
#[cfg(feature = "sendmail")]
pub mod sendmail;
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 11/66] notify: add notification filter mechanism
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (9 preceding siblings ...)
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 10/66] notify: api: add API for groups Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 12/66] notify: api: add API for filters Lukas Wagner
` (57 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
This commit adds a way to filter notifications based on severity. The
filter module also has the necessary foundation work for more complex
filters, e.g. matching on properties or for creating arbitarily complex
filter structures using nested sub-filters.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/api/gotify.rs | 3 +
proxmox-notify/src/api/group.rs | 10 +-
proxmox-notify/src/api/sendmail.rs | 4 +
proxmox-notify/src/config.rs | 8 +
proxmox-notify/src/endpoints/gotify.rs | 12 ++
proxmox-notify/src/endpoints/sendmail.rs | 12 ++
proxmox-notify/src/filter.rs | 198 +++++++++++++++++++++++
proxmox-notify/src/group.rs | 10 +-
proxmox-notify/src/lib.rs | 146 ++++++++++++++++-
9 files changed, 395 insertions(+), 8 deletions(-)
create mode 100644 proxmox-notify/src/filter.rs
diff --git a/proxmox-notify/src/api/gotify.rs b/proxmox-notify/src/api/gotify.rs
index fcc1aabf..fdb9cf53 100644
--- a/proxmox-notify/src/api/gotify.rs
+++ b/proxmox-notify/src/api/gotify.rs
@@ -89,6 +89,7 @@ pub fn update_endpoint(
for deleteable_property in delete {
match deleteable_property {
DeleteableGotifyProperty::Comment => endpoint.comment = None,
+ DeleteableGotifyProperty::Filter => endpoint.filter = None,
}
}
}
@@ -174,6 +175,7 @@ mod tests {
name: "gotify-endpoint".into(),
server: "localhost".into(),
comment: Some("comment".into()),
+ filter: None,
},
&GotifyPrivateConfig {
name: "gotify-endpoint".into(),
@@ -233,6 +235,7 @@ mod tests {
&GotifyConfigUpdater {
server: Some("newhost".into()),
comment: Some("newcomment".into()),
+ filter: None,
},
&GotifyPrivateConfigUpdater {
token: Some("changedtoken".into()),
diff --git a/proxmox-notify/src/api/group.rs b/proxmox-notify/src/api/group.rs
index cc847364..d62167ab 100644
--- a/proxmox-notify/src/api/group.rs
+++ b/proxmox-notify/src/api/group.rs
@@ -80,6 +80,7 @@ pub fn update_group(
for deleteable_property in delete {
match deleteable_property {
DeleteableGroupProperty::Comment => group.comment = None,
+ DeleteableGroupProperty::Filter => group.filter = None,
}
}
}
@@ -99,6 +100,10 @@ pub fn update_group(
group.comment = Some(comment.into());
}
+ if let Some(filter) = &updater.filter {
+ group.filter = Some(filter.into());
+ }
+
config
.config
.set_data(name, GROUP_TYPENAME, &group)
@@ -156,6 +161,7 @@ mod tests {
name: "group1".into(),
endpoint: vec!["test".to_string()],
comment: None,
+ filter: None,
},
)?;
@@ -171,6 +177,7 @@ mod tests {
name: "group1".into(),
endpoint: vec!["foo".into()],
comment: None,
+ filter: None,
},
)
.is_err());
@@ -228,7 +235,8 @@ mod tests {
"group1",
&GroupConfigUpdater {
endpoint: None,
- comment: Some("newcomment".into())
+ comment: Some("newcomment".into()),
+ filter: None
},
None,
None,
diff --git a/proxmox-notify/src/api/sendmail.rs b/proxmox-notify/src/api/sendmail.rs
index 8eafe359..3917a2e3 100644
--- a/proxmox-notify/src/api/sendmail.rs
+++ b/proxmox-notify/src/api/sendmail.rs
@@ -75,6 +75,7 @@ pub fn update_endpoint(
DeleteableSendmailProperty::FromAddress => endpoint.from_address = None,
DeleteableSendmailProperty::Author => endpoint.author = None,
DeleteableSendmailProperty::Comment => endpoint.comment = None,
+ DeleteableSendmailProperty::Filter => endpoint.filter = None,
}
}
}
@@ -136,6 +137,7 @@ pub mod tests {
from_address: Some("from@example.com".into()),
author: Some("root".into()),
comment: Some("Comment".into()),
+ filter: None,
},
)?;
@@ -178,6 +180,7 @@ pub mod tests {
from_address: Some("root@example.com".into()),
author: Some("newauthor".into()),
comment: Some("new comment".into()),
+ filter: None,
},
None,
Some(&[0; 32]),
@@ -202,6 +205,7 @@ pub mod tests {
from_address: Some("root@example.com".into()),
author: Some("newauthor".into()),
comment: Some("new comment".into()),
+ filter: None,
},
None,
Some(&digest),
diff --git a/proxmox-notify/src/config.rs b/proxmox-notify/src/config.rs
index 53817254..645b7bf6 100644
--- a/proxmox-notify/src/config.rs
+++ b/proxmox-notify/src/config.rs
@@ -2,6 +2,7 @@ use lazy_static::lazy_static;
use proxmox_schema::{ApiType, ObjectSchema};
use proxmox_section_config::{SectionConfig, SectionConfigData, SectionConfigPlugin};
+use crate::filter::{FilterConfig, FILTER_TYPENAME};
use crate::group::{GroupConfig, GROUP_TYPENAME};
use crate::schema::BACKEND_NAME_SCHEMA;
use crate::Error;
@@ -45,6 +46,13 @@ fn config_init() -> SectionConfig {
GROUP_SCHEMA,
));
+ const FILTER_SCHEMA: &ObjectSchema = FilterConfig::API_SCHEMA.unwrap_object_schema();
+ config.register_plugin(SectionConfigPlugin::new(
+ FILTER_TYPENAME.to_string(),
+ Some(String::from("name")),
+ FILTER_SCHEMA,
+ ));
+
config
}
diff --git a/proxmox-notify/src/endpoints/gotify.rs b/proxmox-notify/src/endpoints/gotify.rs
index 57b330c2..6c42100e 100644
--- a/proxmox-notify/src/endpoints/gotify.rs
+++ b/proxmox-notify/src/endpoints/gotify.rs
@@ -36,6 +36,10 @@ pub(crate) const GOTIFY_TYPENAME: &str = "gotify";
optional: true,
schema: COMMENT_SCHEMA,
},
+ filter: {
+ optional: true,
+ schema: ENTITY_NAME_SCHEMA,
+ },
}
)]
#[derive(Serialize, Deserialize, Updater, Default)]
@@ -50,6 +54,9 @@ pub struct GotifyConfig {
/// Comment
#[serde(skip_serializing_if = "Option::is_none")]
pub comment: Option<String>,
+ /// Filter to apply
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub filter: Option<String>,
}
#[api()]
@@ -76,6 +83,7 @@ pub struct GotifyEndpoint {
#[serde(rename_all = "kebab-case")]
pub enum DeleteableGotifyProperty {
Comment,
+ Filter,
}
impl Endpoint for GotifyEndpoint {
@@ -113,4 +121,8 @@ impl Endpoint for GotifyEndpoint {
fn name(&self) -> &str {
&self.config.name
}
+
+ fn filter(&self) -> Option<&str> {
+ self.config.filter.as_deref()
+ }
}
diff --git a/proxmox-notify/src/endpoints/sendmail.rs b/proxmox-notify/src/endpoints/sendmail.rs
index dd971438..fcac6248 100644
--- a/proxmox-notify/src/endpoints/sendmail.rs
+++ b/proxmox-notify/src/endpoints/sendmail.rs
@@ -21,6 +21,10 @@ pub(crate) const SENDMAIL_TYPENAME: &str = "sendmail";
optional: true,
schema: COMMENT_SCHEMA,
},
+ filter: {
+ optional: true,
+ schema: ENTITY_NAME_SCHEMA,
+ },
},
)]
#[derive(Debug, Serialize, Deserialize, Updater, Default)]
@@ -41,6 +45,9 @@ pub struct SendmailConfig {
/// Comment
#[serde(skip_serializing_if = "Option::is_none")]
pub comment: Option<String>,
+ /// Filter to apply
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub filter: Option<String>,
}
#[derive(Serialize, Deserialize)]
@@ -49,6 +56,7 @@ pub enum DeleteableSendmailProperty {
FromAddress,
Author,
Comment,
+ Filter,
}
/// A sendmail notification endpoint.
@@ -85,4 +93,8 @@ impl Endpoint for SendmailEndpoint {
fn name(&self) -> &str {
&self.config.name
}
+
+ fn filter(&self) -> Option<&str> {
+ self.config.filter.as_deref()
+ }
}
diff --git a/proxmox-notify/src/filter.rs b/proxmox-notify/src/filter.rs
new file mode 100644
index 00000000..11ffe065
--- /dev/null
+++ b/proxmox-notify/src/filter.rs
@@ -0,0 +1,198 @@
+use serde::{Deserialize, Serialize};
+use std::collections::{HashMap, HashSet};
+
+use proxmox_schema::{api, Updater};
+
+use crate::schema::{COMMENT_SCHEMA, ENTITY_NAME_SCHEMA};
+use crate::{Error, Notification, Severity};
+
+pub const FILTER_TYPENAME: &str = "filter";
+
+#[api]
+#[derive(Debug, Serialize, Deserialize, Default, Clone, Copy)]
+#[serde(rename_all = "kebab-case")]
+pub enum FilterModeOperator {
+ /// All filter properties have to match (AND)
+ #[default]
+ And,
+ /// At least one filter property has to match (OR)
+ Or,
+}
+
+impl FilterModeOperator {
+ /// Apply the mode operator to two bools, lhs and rhs
+ fn apply(&self, lhs: bool, rhs: bool) -> bool {
+ match self {
+ FilterModeOperator::And => lhs && rhs,
+ FilterModeOperator::Or => lhs || rhs,
+ }
+ }
+
+ fn neutral_element(&self) -> bool {
+ match self {
+ FilterModeOperator::And => true,
+ FilterModeOperator::Or => false,
+ }
+ }
+}
+
+#[api(
+ properties: {
+ name: {
+ schema: ENTITY_NAME_SCHEMA,
+ },
+ comment: {
+ optional: true,
+ schema: COMMENT_SCHEMA,
+ },
+ })]
+#[derive(Debug, Serialize, Deserialize, Updater, Default)]
+#[serde(rename_all = "kebab-case")]
+/// Config for Sendmail notification endpoints
+pub struct FilterConfig {
+ /// Name of the filter
+ #[updater(skip)]
+ pub name: String,
+
+ /// Minimum severity to match
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub min_severity: Option<Severity>,
+
+ /// Choose between 'and' and 'or' for when multiple properties are specified
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub mode: Option<FilterModeOperator>,
+
+ /// Invert match of the whole filter
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub invert_match: Option<bool>,
+
+ /// Comment
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub comment: Option<String>,
+}
+
+#[derive(Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case")]
+pub enum DeleteableFilterProperty {
+ MinSeverity,
+ Mode,
+ InvertMatch,
+ Comment,
+}
+
+/// A caching, lazily-evaluating notification filter. Parameterized with the notification itself,
+/// since there are usually multiple filters to check for a single notification that is to be sent.
+pub(crate) struct FilterMatcher<'a> {
+ filters: HashMap<&'a str, &'a FilterConfig>,
+ cached_results: HashMap<&'a str, bool>,
+ notification: &'a Notification,
+}
+
+impl<'a> FilterMatcher<'a> {
+ pub(crate) fn new(filters: &'a [FilterConfig], notification: &'a Notification) -> Self {
+ let filters = filters.iter().map(|f| (f.name.as_str(), f)).collect();
+
+ Self {
+ filters,
+ cached_results: Default::default(),
+ notification,
+ }
+ }
+
+ /// Check if the notification that was used to instantiate Self matches a given filter
+ pub(crate) fn check_filter_match(&mut self, filter_name: &str) -> Result<bool, Error> {
+ let mut visited = HashSet::new();
+
+ self.do_check_filter(filter_name, &mut visited)
+ }
+
+ fn do_check_filter(
+ &mut self,
+ filter_name: &str,
+ visited: &mut HashSet<String>,
+ ) -> Result<bool, Error> {
+ if visited.contains(filter_name) {
+ return Err(Error::FilterFailed(format!(
+ "recursive filter definition: {filter_name}"
+ )));
+ }
+
+ if let Some(is_match) = self.cached_results.get(filter_name) {
+ return Ok(*is_match);
+ }
+
+ visited.insert(filter_name.into());
+
+ let filter_config =
+ self.filters.get(filter_name).copied().ok_or_else(|| {
+ Error::FilterFailed(format!("filter '{filter_name}' does not exist"))
+ })?;
+
+ let invert_match = filter_config.invert_match.unwrap_or_default();
+
+ let mode_operator = filter_config.mode.unwrap_or_default();
+
+ let mut notification_matches = mode_operator.neutral_element();
+
+ notification_matches = mode_operator.apply(
+ notification_matches,
+ self.check_severity_match(filter_config, mode_operator),
+ );
+
+ Ok(notification_matches != invert_match)
+ }
+
+ fn check_severity_match(
+ &self,
+ filter_config: &FilterConfig,
+ mode_operator: FilterModeOperator,
+ ) -> bool {
+ if let Some(min_severity) = filter_config.min_severity {
+ self.notification.severity >= min_severity
+ } else {
+ mode_operator.neutral_element()
+ }
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use crate::config;
+
+ fn parse_filters(config: &str) -> Result<Vec<FilterConfig>, Error> {
+ let (config, _) = config::config(config)?;
+ Ok(config.convert_to_typed_array("filter").unwrap())
+ }
+
+ fn empty_notification_with_severity(severity: Severity) -> Notification {
+ Notification {
+ title: String::new(),
+ body: String::new(),
+ severity,
+ properties: Default::default(),
+ }
+ }
+
+ #[test]
+ fn test_trivial_severity_filters() -> Result<(), Error> {
+ let config = "
+filter: test
+ min-severity warning
+";
+
+ let filters = parse_filters(config)?;
+
+ let is_match = |severity| {
+ let notification = empty_notification_with_severity(severity);
+ let mut results = FilterMatcher::new(&filters, ¬ification);
+ results.check_filter_match("test")
+ };
+
+ assert!(is_match(Severity::Warning)?);
+ assert!(!is_match(Severity::Notice)?);
+ assert!(is_match(Severity::Error)?);
+
+ Ok(())
+ }
+}
diff --git a/proxmox-notify/src/group.rs b/proxmox-notify/src/group.rs
index d9ded2dd..ae861065 100644
--- a/proxmox-notify/src/group.rs
+++ b/proxmox-notify/src/group.rs
@@ -1,4 +1,4 @@
-use crate::schema::COMMENT_SCHEMA;
+use crate::schema::{COMMENT_SCHEMA, ENTITY_NAME_SCHEMA};
use proxmox_schema::{api, Updater};
use serde::{Deserialize, Serialize};
@@ -17,6 +17,10 @@ pub(crate) const GROUP_TYPENAME: &str = "group";
optional: true,
schema: COMMENT_SCHEMA,
},
+ filter: {
+ optional: true,
+ schema: ENTITY_NAME_SCHEMA,
+ },
},
)]
#[derive(Debug, Serialize, Deserialize, Updater, Default)]
@@ -31,10 +35,14 @@ pub struct GroupConfig {
/// Comment
#[serde(skip_serializing_if = "Option::is_none")]
pub comment: Option<String>,
+ /// Filter to apply
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub filter: Option<String>,
}
#[derive(Serialize, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum DeleteableGroupProperty {
Comment,
+ Filter,
}
diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
index 35d5208b..5d408c85 100644
--- a/proxmox-notify/src/lib.rs
+++ b/proxmox-notify/src/lib.rs
@@ -1,6 +1,7 @@
use std::collections::HashMap;
use std::fmt::Display;
+use filter::{FilterConfig, FilterMatcher, FILTER_TYPENAME};
use group::{GroupConfig, GROUP_TYPENAME};
use proxmox_schema::api;
use proxmox_section_config::SectionConfigData;
@@ -13,6 +14,7 @@ use std::error::Error as StdError;
pub mod api;
mod config;
pub mod endpoints;
+mod filter;
pub mod group;
pub mod schema;
@@ -23,6 +25,7 @@ pub enum Error {
NotifyFailed(String, Box<dyn StdError + Send + Sync + 'static>),
TargetDoesNotExist(String),
TargetTestFailed(Vec<Box<dyn StdError + Send + Sync + 'static>>),
+ FilterFailed(String),
}
impl Display for Error {
@@ -47,6 +50,9 @@ impl Display for Error {
Ok(())
}
+ Error::FilterFailed(message) => {
+ write!(f, "could not apply filter: {message}")
+ }
}
}
}
@@ -59,6 +65,7 @@ impl StdError for Error {
Error::NotifyFailed(_, err) => Some(&**err),
Error::TargetDoesNotExist(_) => None,
Error::TargetTestFailed(errs) => Some(&*errs[0]),
+ Error::FilterFailed(_) => None,
}
}
}
@@ -85,6 +92,9 @@ pub trait Endpoint {
/// The name/identifier for this endpoint
fn name(&self) -> &str;
+
+ /// The name of the filter to use
+ fn filter(&self) -> Option<&str>;
}
#[derive(Debug, Clone)]
@@ -161,6 +171,7 @@ impl Config {
pub struct Bus {
endpoints: HashMap<String, Box<dyn Endpoint>>,
groups: HashMap<String, GroupConfig>,
+ filters: Vec<FilterConfig>,
}
#[allow(unused_macros)]
@@ -266,7 +277,16 @@ impl Bus {
.map(|group: GroupConfig| (group.name.clone(), group))
.collect();
- Ok(Bus { endpoints, groups })
+ let filters = config
+ .config
+ .convert_to_typed_array(FILTER_TYPENAME)
+ .map_err(|err| Error::ConfigDeserialization(err.into()))?;
+
+ Ok(Bus {
+ endpoints,
+ groups,
+ filters,
+ })
}
#[cfg(test)]
@@ -279,29 +299,63 @@ impl Bus {
self.groups.insert(group.name.clone(), group);
}
+ #[cfg(test)]
+ pub fn add_filter(&mut self, filter: FilterConfig) {
+ self.filters.push(filter)
+ }
+
/// Send a notification to a given target (endpoint or group).
///
/// Any errors will not be returned but only logged.
pub fn send(&self, endpoint_or_group: &str, notification: &Notification) {
+ let mut filter_matcher = FilterMatcher::new(&self.filters, notification);
+
if let Some(group) = self.groups.get(endpoint_or_group) {
+ if !Bus::check_filter(&mut filter_matcher, group.filter.as_deref()) {
+ return;
+ }
+
for endpoint in &group.endpoint {
- self.send_via_single_endpoint(endpoint, notification);
+ self.send_via_single_endpoint(endpoint, notification, &mut filter_matcher);
}
} else {
- self.send_via_single_endpoint(endpoint_or_group, notification);
+ self.send_via_single_endpoint(endpoint_or_group, notification, &mut filter_matcher);
}
}
- fn send_via_single_endpoint(&self, endpoint: &str, notification: &Notification) {
+ fn check_filter(filter_matcher: &mut FilterMatcher, filter: Option<&str>) -> bool {
+ if let Some(filter) = filter {
+ match filter_matcher.check_filter_match(filter) {
+ // If the filter does not match, do nothing
+ Ok(r) => r,
+ Err(err) => {
+ // If there is an error, only log it and still send
+ log::error!("could not apply filter '{filter}': {err}");
+ true
+ }
+ }
+ } else {
+ true
+ }
+ }
+
+ fn send_via_single_endpoint(
+ &self,
+ endpoint: &str,
+ notification: &Notification,
+ filter_matcher: &mut FilterMatcher,
+ ) {
if let Some(endpoint) = self.endpoints.get(endpoint) {
+ if !Bus::check_filter(filter_matcher, endpoint.filter()) {
+ return;
+ }
+
if let Err(e) = endpoint.send(notification) {
// Only log on errors, do not propagate fail to the caller.
log::error!(
"could not notify via target `{name}`: {e}",
name = endpoint.name()
);
- } else {
- log::info!("notified via endpoint `{name}`", name = endpoint.name());
}
} else {
log::error!("could not notify via endpoint '{endpoint}', it does not exist");
@@ -359,6 +413,7 @@ mod tests {
struct MockEndpoint {
name: &'static str,
messages: Rc<RefCell<Vec<Notification>>>,
+ filter: Option<String>,
}
impl Endpoint for MockEndpoint {
@@ -371,12 +426,17 @@ mod tests {
fn name(&self) -> &str {
self.name
}
+
+ fn filter(&self) -> Option<&str> {
+ self.filter.as_deref()
+ }
}
impl MockEndpoint {
fn new(name: &'static str, filter: Option<String>) -> Self {
Self {
name,
+ filter,
..Default::default()
}
}
@@ -420,12 +480,14 @@ mod tests {
name: "group1".to_string(),
endpoint: vec!["mock1".into()],
comment: None,
+ filter: None,
});
bus.add_group(GroupConfig {
name: "group2".to_string(),
endpoint: vec!["mock2".into()],
comment: None,
+ filter: None,
});
bus.add_endpoint(Box::new(endpoint1.clone()));
@@ -453,4 +515,76 @@ mod tests {
Ok(())
}
+
+ #[test]
+ fn test_severity_ordering() {
+ // Not intended to be exhaustive, just a quick
+ // sanity check ;)
+
+ assert!(Severity::Info < Severity::Notice);
+ assert!(Severity::Info < Severity::Warning);
+ assert!(Severity::Info < Severity::Error);
+ assert!(Severity::Error > Severity::Warning);
+ assert!(Severity::Warning > Severity::Notice);
+ }
+
+ #[test]
+ fn test_multiple_endpoints_with_different_filters() -> Result<(), Error> {
+ let endpoint1 = MockEndpoint::new("mock1", Some("filter1".into()));
+ let endpoint2 = MockEndpoint::new("mock2", Some("filter2".into()));
+
+ let mut bus = Bus::default();
+
+ bus.add_endpoint(Box::new(endpoint1.clone()));
+ bus.add_endpoint(Box::new(endpoint2.clone()));
+
+ bus.add_group(GroupConfig {
+ name: "channel1".to_string(),
+ endpoint: vec!["mock1".into(), "mock2".into()],
+ comment: None,
+ filter: None,
+ });
+
+ bus.add_filter(FilterConfig {
+ name: "filter1".into(),
+ min_severity: Some(Severity::Warning),
+ mode: None,
+ invert_match: None,
+ comment: None,
+ });
+
+ bus.add_filter(FilterConfig {
+ name: "filter2".into(),
+ min_severity: Some(Severity::Error),
+ mode: None,
+ invert_match: None,
+ comment: None,
+ });
+
+ let send_with_severity = |severity| {
+ bus.send(
+ "channel1",
+ &Notification {
+ title: "Title".into(),
+ body: "Body".into(),
+ severity,
+ properties: Default::default(),
+ },
+ );
+ };
+
+ send_with_severity(Severity::Info);
+ assert_eq!(endpoint1.messages().len(), 0);
+ assert_eq!(endpoint2.messages().len(), 0);
+
+ send_with_severity(Severity::Warning);
+ assert_eq!(endpoint1.messages().len(), 1);
+ assert_eq!(endpoint2.messages().len(), 0);
+
+ send_with_severity(Severity::Error);
+ assert_eq!(endpoint1.messages().len(), 2);
+ assert_eq!(endpoint2.messages().len(), 1);
+
+ Ok(())
+ }
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 12/66] notify: api: add API for filters
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (10 preceding siblings ...)
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 11/66] notify: add notification filter mechanism Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 13/66] notify: add template rendering Lukas Wagner
` (56 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/api/filter.rs | 231 +++++++++++++++++++++++++++++
proxmox-notify/src/api/gotify.rs | 12 ++
proxmox-notify/src/api/group.rs | 7 +
proxmox-notify/src/api/mod.rs | 1 +
proxmox-notify/src/api/sendmail.rs | 10 ++
5 files changed, 261 insertions(+)
create mode 100644 proxmox-notify/src/api/filter.rs
diff --git a/proxmox-notify/src/api/filter.rs b/proxmox-notify/src/api/filter.rs
new file mode 100644
index 00000000..3fcff6b9
--- /dev/null
+++ b/proxmox-notify/src/api/filter.rs
@@ -0,0 +1,231 @@
+use crate::api::ApiError;
+use crate::filter::{DeleteableFilterProperty, FilterConfig, FilterConfigUpdater, FILTER_TYPENAME};
+use crate::Config;
+
+/// Get a list of all filters
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns a list of all filters or an `ApiError` if the config is erroneous.
+pub fn get_filters(config: &Config) -> Result<Vec<FilterConfig>, ApiError> {
+ config
+ .config
+ .convert_to_typed_array(FILTER_TYPENAME)
+ .map_err(|e| ApiError::internal_server_error("Could not fetch filters", Some(e.into())))
+}
+
+/// Get filter with given `name`
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns the endpoint or an `ApiError` if the filter was not found.
+pub fn get_filter(config: &Config, name: &str) -> Result<FilterConfig, ApiError> {
+ config
+ .config
+ .lookup(FILTER_TYPENAME, name)
+ .map_err(|_| ApiError::not_found(format!("filter '{name}' not found"), None))
+}
+
+/// Add new notification filter.
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns an `ApiError` if a filter with the same name already exists or
+/// if the filter could not be saved.
+pub fn add_filter(config: &mut Config, filter_config: &FilterConfig) -> Result<(), ApiError> {
+ if get_filter(config, &filter_config.name).is_ok() {
+ return Err(ApiError::bad_request(
+ format!("filter '{}' already exists", filter_config.name),
+ None,
+ ));
+ }
+
+ config
+ .config
+ .set_data(&filter_config.name, FILTER_TYPENAME, filter_config)
+ .map_err(|e| {
+ ApiError::internal_server_error(
+ format!("could not save filter '{}'", filter_config.name),
+ Some(e.into()),
+ )
+ })?;
+
+ Ok(())
+}
+
+/// Update existing filter
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns an `ApiError` if the config could not be saved.
+pub fn update_filter(
+ config: &mut Config,
+ name: &str,
+ filter_updater: &FilterConfigUpdater,
+ delete: Option<&[DeleteableFilterProperty]>,
+ digest: Option<&[u8]>,
+) -> Result<(), ApiError> {
+ super::verify_digest(config, digest)?;
+
+ let mut filter = get_filter(config, name)?;
+
+ if let Some(delete) = delete {
+ for deleteable_property in delete {
+ match deleteable_property {
+ DeleteableFilterProperty::MinSeverity => filter.min_severity = None,
+ DeleteableFilterProperty::Mode => filter.mode = None,
+ DeleteableFilterProperty::InvertMatch => filter.invert_match = None,
+ DeleteableFilterProperty::Comment => filter.comment = None,
+ }
+ }
+ }
+
+ if let Some(min_severity) = filter_updater.min_severity {
+ filter.min_severity = Some(min_severity);
+ }
+
+ if let Some(mode) = filter_updater.mode {
+ filter.mode = Some(mode);
+ }
+
+ if let Some(invert_match) = filter_updater.invert_match {
+ filter.invert_match = Some(invert_match);
+ }
+
+ if let Some(comment) = &filter_updater.comment {
+ filter.comment = Some(comment.into());
+ }
+
+ config
+ .config
+ .set_data(name, FILTER_TYPENAME, &filter)
+ .map_err(|e| {
+ ApiError::internal_server_error(
+ format!("could not save filter '{name}'"),
+ Some(e.into()),
+ )
+ })?;
+
+ Ok(())
+}
+
+/// Delete existing filter
+///
+/// The caller is responsible for any needed permission checks.
+/// The caller also responsible for locking the configuration files.
+/// Returns an `ApiError` if the filter does not exist.
+pub fn delete_filter(config: &mut Config, name: &str) -> Result<(), ApiError> {
+ // Check if the filter exists
+ let _ = get_filter(config, name)?;
+
+ config.config.sections.remove(name);
+
+ Ok(())
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use crate::filter::FilterModeOperator;
+ use crate::Severity;
+
+ fn empty_config() -> Config {
+ Config::new("", "").unwrap()
+ }
+
+ fn config_with_two_filters() -> Config {
+ Config::new(
+ "
+filter: filter1
+ min-severity info
+
+filter: filter2
+ min-severity warning
+",
+ "",
+ )
+ .unwrap()
+ }
+
+ #[test]
+ fn test_update_not_existing_returns_error() -> Result<(), ApiError> {
+ let mut config = empty_config();
+ assert!(update_filter(&mut config, "test", &Default::default(), None, None).is_err());
+ Ok(())
+ }
+
+ #[test]
+ fn test_update_invalid_digest_returns_error() -> Result<(), ApiError> {
+ let mut config = config_with_two_filters();
+ assert!(update_filter(
+ &mut config,
+ "filter1",
+ &Default::default(),
+ None,
+ Some(&[0u8; 32])
+ )
+ .is_err());
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_filter_update() -> Result<(), ApiError> {
+ let mut config = config_with_two_filters();
+
+ let digest = config.digest;
+
+ update_filter(
+ &mut config,
+ "filter1",
+ &FilterConfigUpdater {
+ min_severity: Some(Severity::Error),
+ mode: Some(FilterModeOperator::Or),
+ invert_match: Some(true),
+ comment: Some("new comment".into()),
+ },
+ None,
+ Some(&digest),
+ )?;
+
+ let filter = get_filter(&config, "filter1")?;
+
+ assert!(matches!(filter.mode, Some(FilterModeOperator::Or)));
+ assert!(matches!(filter.min_severity, Some(Severity::Error)));
+ assert_eq!(filter.invert_match, Some(true));
+ assert_eq!(filter.comment, Some("new comment".into()));
+
+ // Test property deletion
+ update_filter(
+ &mut config,
+ "filter1",
+ &Default::default(),
+ Some(&[
+ DeleteableFilterProperty::InvertMatch,
+ DeleteableFilterProperty::Mode,
+ DeleteableFilterProperty::InvertMatch,
+ DeleteableFilterProperty::MinSeverity,
+ DeleteableFilterProperty::Comment,
+ ]),
+ Some(&digest),
+ )?;
+
+ let filter = get_filter(&config, "filter1")?;
+
+ assert_eq!(filter.invert_match, None);
+ assert_eq!(filter.min_severity, None);
+ assert!(matches!(filter.mode, None));
+ assert_eq!(filter.comment, None);
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_filter_delete() -> Result<(), ApiError> {
+ let mut config = config_with_two_filters();
+
+ delete_filter(&mut config, "filter1")?;
+ assert!(delete_filter(&mut config, "filter1").is_err());
+ assert_eq!(get_filters(&config)?.len(), 1);
+
+ Ok(())
+ }
+}
diff --git a/proxmox-notify/src/api/gotify.rs b/proxmox-notify/src/api/gotify.rs
index fdb9cf53..d6f33064 100644
--- a/proxmox-notify/src/api/gotify.rs
+++ b/proxmox-notify/src/api/gotify.rs
@@ -53,6 +53,11 @@ pub fn add_endpoint(
));
}
+ if let Some(filter) = &endpoint_config.filter {
+ // Check if filter exists
+ super::filter::get_filter(config, filter)?;
+ }
+
set_private_config_entry(config, private_endpoint_config)?;
config
@@ -112,6 +117,13 @@ pub fn update_endpoint(
endpoint.comment = Some(comment.into());
}
+ if let Some(filter) = &endpoint_config_updater.filter {
+ // Check if filter exists
+ let _ = super::filter::get_filter(config, filter)?;
+
+ endpoint.filter = Some(filter.into());
+ }
+
config
.config
.set_data(name, GOTIFY_TYPENAME, &endpoint)
diff --git a/proxmox-notify/src/api/group.rs b/proxmox-notify/src/api/group.rs
index d62167ab..fe3de12f 100644
--- a/proxmox-notify/src/api/group.rs
+++ b/proxmox-notify/src/api/group.rs
@@ -45,6 +45,11 @@ pub fn add_group(config: &mut Config, group_config: &GroupConfig) -> Result<(),
));
}
+ if let Some(filter) = &group_config.filter {
+ // Check if filter exists
+ super::filter::get_filter(config, filter)?;
+ }
+
check_if_endpoints_exist(config, &group_config.endpoint)?;
config
@@ -101,6 +106,8 @@ pub fn update_group(
}
if let Some(filter) = &updater.filter {
+ // Check if filter exists
+ let _ = super::filter::get_filter(config, filter)?;
group.filter = Some(filter.into());
}
diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
index ac917910..12811baf 100644
--- a/proxmox-notify/src/api/mod.rs
+++ b/proxmox-notify/src/api/mod.rs
@@ -5,6 +5,7 @@ use crate::Config;
use serde::Serialize;
pub mod common;
+pub mod filter;
#[cfg(feature = "gotify")]
pub mod gotify;
pub mod group;
diff --git a/proxmox-notify/src/api/sendmail.rs b/proxmox-notify/src/api/sendmail.rs
index 3917a2e3..6b0323e3 100644
--- a/proxmox-notify/src/api/sendmail.rs
+++ b/proxmox-notify/src/api/sendmail.rs
@@ -40,6 +40,11 @@ pub fn add_endpoint(config: &mut Config, endpoint: &SendmailConfig) -> Result<()
));
}
+ if let Some(filter) = &endpoint.filter {
+ // Check if filter exists
+ super::filter::get_filter(config, filter)?;
+ }
+
config
.config
.set_data(&endpoint.name, SENDMAIL_TYPENAME, endpoint)
@@ -96,6 +101,11 @@ pub fn update_endpoint(
endpoint.comment = Some(comment.into());
}
+ if let Some(filter) = &updater.filter {
+ let _ = super::filter::get_filter(config, filter)?;
+ endpoint.filter = Some(filter.into());
+ }
+
config
.config
.set_data(name, SENDMAIL_TYPENAME, &endpoint)
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 13/66] notify: add template rendering
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (11 preceding siblings ...)
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 12/66] notify: api: add API for filters Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 14/66] notify: add example for " Lukas Wagner
` (55 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
This commit adds template rendering to the `proxmox-notify` crate, based
on the `handlebars` crate.
Title and body of a notification are rendered using any `properties`
passed along with the notification. There are also a few helpers,
allowing to render tables from `serde_json::Value`.
'Value' renderers. These can also be used in table cells using the
'renderer' property in a table schema:
- {{human-bytes val}}
Render bytes with human-readable units (base 2)
- {{duration val}}
Render a duration (based on seconds)
- {{timestamp val}}
Render a unix-epoch (based on seconds)
There are also a few 'block-level' helpers.
- {{table val}}
Render a table from given val (containing a schema for the columns,
as well as the table data)
- {{object val}}
Render a value as a pretty-printed json
- {{heading_1 val}}
Render a top-level heading
- {{heading_2 val}}
Render a not-so-top-level heading
- {{verbatim val}} or {{/verbatim}}<content>{{#verbatim}}
Do not reflow text. NOP for plain text, but for HTML output the text
will be contained in a <pre> with a regular font.
- {{verbatim-monospaced val}} or
{{/verbatim-monospaced}}<content>{{#verbatim-monospaced}}
Do not reflow text. NOP for plain text, but for HTML output the text
will be contained in a <pre> with a monospaced font.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
Cargo.toml | 1 +
proxmox-notify/Cargo.toml | 6 +-
proxmox-notify/src/endpoints/gotify.rs | 40 ++-
proxmox-notify/src/endpoints/sendmail.rs | 26 +-
proxmox-notify/src/lib.rs | 6 +-
proxmox-notify/src/renderer/html.rs | 100 +++++++
proxmox-notify/src/renderer/mod.rs | 366 +++++++++++++++++++++++
proxmox-notify/src/renderer/plaintext.rs | 141 +++++++++
proxmox-notify/src/renderer/table.rs | 24 ++
9 files changed, 684 insertions(+), 26 deletions(-)
create mode 100644 proxmox-notify/src/renderer/html.rs
create mode 100644 proxmox-notify/src/renderer/mod.rs
create mode 100644 proxmox-notify/src/renderer/plaintext.rs
create mode 100644 proxmox-notify/src/renderer/table.rs
diff --git a/Cargo.toml b/Cargo.toml
index ef8a050a..c30131fe 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -88,6 +88,7 @@ proxmox-api-macro = { version = "1.0.4", path = "proxmox-api-macro" }
proxmox-async = { version = "0.4.1", path = "proxmox-async" }
proxmox-compression = { version = "0.2.0", path = "proxmox-compression" }
proxmox-http = { version = "0.9.0", path = "proxmox-http" }
+proxmox-human-byte = { version = "0.1.0", path = "proxmox-human-byte" }
proxmox-io = { version = "1.0.0", path = "proxmox-io" }
proxmox-lang = { version = "1.1", path = "proxmox-lang" }
proxmox-rest-server = { version = "0.4.0", path = "proxmox-rest-server" }
diff --git a/proxmox-notify/Cargo.toml b/proxmox-notify/Cargo.toml
index 738674ae..a635798b 100644
--- a/proxmox-notify/Cargo.toml
+++ b/proxmox-notify/Cargo.toml
@@ -8,19 +8,21 @@ repository.workspace = true
exclude.workspace = true
[dependencies]
-handlebars = { workspace = true, optional = true }
+handlebars = { workspace = true }
lazy_static.workspace = true
log.workspace = true
openssl.workspace = true
proxmox-http = { workspace = true, features = ["client-sync"], optional = true }
+proxmox-human-byte.workspace = true
proxmox-schema = { workspace = true, features = ["api-macro"]}
proxmox-section-config = { workspace = true }
proxmox-sys = { workspace = true, optional = true }
+proxmox-time.workspace = true
regex.workspace = true
serde = { workspace = true, features = ["derive"]}
serde_json.workspace = true
[features]
default = ["sendmail", "gotify"]
-sendmail = ["dep:handlebars", "dep:proxmox-sys"]
+sendmail = ["dep:proxmox-sys"]
gotify = ["dep:proxmox-http"]
diff --git a/proxmox-notify/src/endpoints/gotify.rs b/proxmox-notify/src/endpoints/gotify.rs
index 6c42100e..b278b90d 100644
--- a/proxmox-notify/src/endpoints/gotify.rs
+++ b/proxmox-notify/src/endpoints/gotify.rs
@@ -1,21 +1,16 @@
use std::collections::HashMap;
+use crate::renderer::TemplateRenderer;
use crate::schema::{COMMENT_SCHEMA, ENTITY_NAME_SCHEMA};
-use crate::{Endpoint, Error, Notification, Severity};
+use crate::{renderer, Endpoint, Error, Notification, Severity};
use serde::{Deserialize, Serialize};
+use serde_json::json;
use proxmox_http::client::sync::Client;
use proxmox_http::{HttpClient, HttpOptions};
use proxmox_schema::{api, Updater};
-#[derive(Serialize)]
-struct GotifyMessageBody<'a> {
- title: &'a str,
- message: &'a str,
- priority: u32,
-}
-
fn severity_to_priority(level: Severity) -> u32 {
match level {
Severity::Info => 1,
@@ -93,11 +88,30 @@ impl Endpoint for GotifyEndpoint {
let uri = format!("{}/message", self.config.server);
- let body = GotifyMessageBody {
- title: ¬ification.title,
- message: ¬ification.body,
- priority: severity_to_priority(notification.severity),
- };
+ let properties = notification.properties.as_ref();
+
+ let title = renderer::render_template(
+ TemplateRenderer::Plaintext,
+ ¬ification.title,
+ properties,
+ )?;
+ let message =
+ renderer::render_template(TemplateRenderer::Plaintext, ¬ification.body, properties)?;
+
+ // We don't have a TemplateRenderer::Markdown yet, so simply put everything
+ // in code tags. Otherwise tables etc. are not formatted properly
+ let message = format!("```\n{message}\n```");
+
+ let body = json!({
+ "title": &title,
+ "message": &message,
+ "priority": severity_to_priority(notification.severity),
+ "extras": {
+ "client::display": {
+ "contentType": "text/markdown"
+ }
+ }
+ });
let body = serde_json::to_vec(&body)
.map_err(|err| Error::NotifyFailed(self.name().to_string(), err.into()))?;
diff --git a/proxmox-notify/src/endpoints/sendmail.rs b/proxmox-notify/src/endpoints/sendmail.rs
index fcac6248..9d06e7c4 100644
--- a/proxmox-notify/src/endpoints/sendmail.rs
+++ b/proxmox-notify/src/endpoints/sendmail.rs
@@ -1,5 +1,6 @@
+use crate::renderer::TemplateRenderer;
use crate::schema::{COMMENT_SCHEMA, EMAIL_SCHEMA, ENTITY_NAME_SCHEMA};
-use crate::{Endpoint, Error, Notification};
+use crate::{renderer, Endpoint, Error, Notification};
use proxmox_schema::{api, Updater};
use serde::{Deserialize, Serialize};
@@ -68,12 +69,17 @@ impl Endpoint for SendmailEndpoint {
fn send(&self, notification: &Notification) -> Result<(), Error> {
let recipients: Vec<&str> = self.config.mailto.iter().map(String::as_str).collect();
- // Note: OX has serious problems displaying text mails,
- // so we include html as well
- let html = format!(
- "<html><body><pre>\n{}\n<pre>",
- handlebars::html_escape(¬ification.body)
- );
+ let properties = notification.properties.as_ref();
+
+ let subject = renderer::render_template(
+ TemplateRenderer::Plaintext,
+ ¬ification.title,
+ properties,
+ )?;
+ let html_part =
+ renderer::render_template(TemplateRenderer::Html, ¬ification.body, properties)?;
+ let text_part =
+ renderer::render_template(TemplateRenderer::Plaintext, ¬ification.body, properties)?;
// proxmox_sys::email::sendmail will set the author to
// "Proxmox Backup Server" if it is not set.
@@ -81,9 +87,9 @@ impl Endpoint for SendmailEndpoint {
proxmox_sys::email::sendmail(
&recipients,
- ¬ification.title,
- Some(¬ification.body),
- Some(&html),
+ &subject,
+ Some(&text_part),
+ Some(&html_part),
self.config.from_address.as_deref(),
author,
)
diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
index 5d408c85..548cc56f 100644
--- a/proxmox-notify/src/lib.rs
+++ b/proxmox-notify/src/lib.rs
@@ -14,8 +14,9 @@ use std::error::Error as StdError;
pub mod api;
mod config;
pub mod endpoints;
-mod filter;
+pub mod filter;
pub mod group;
+pub mod renderer;
pub mod schema;
#[derive(Debug)]
@@ -26,6 +27,7 @@ pub enum Error {
TargetDoesNotExist(String),
TargetTestFailed(Vec<Box<dyn StdError + Send + Sync + 'static>>),
FilterFailed(String),
+ RenderError(Box<dyn StdError + Send + Sync + 'static>),
}
impl Display for Error {
@@ -53,6 +55,7 @@ impl Display for Error {
Error::FilterFailed(message) => {
write!(f, "could not apply filter: {message}")
}
+ Error::RenderError(err) => write!(f, "could not render notification template: {err}"),
}
}
}
@@ -66,6 +69,7 @@ impl StdError for Error {
Error::TargetDoesNotExist(_) => None,
Error::TargetTestFailed(errs) => Some(&*errs[0]),
Error::FilterFailed(_) => None,
+ Error::RenderError(err) => Some(&**err),
}
}
}
diff --git a/proxmox-notify/src/renderer/html.rs b/proxmox-notify/src/renderer/html.rs
new file mode 100644
index 00000000..7a41e873
--- /dev/null
+++ b/proxmox-notify/src/renderer/html.rs
@@ -0,0 +1,100 @@
+use crate::define_helper_with_prefix_and_postfix;
+use crate::renderer::BlockRenderFunctions;
+use handlebars::{
+ Context, Handlebars, Helper, HelperResult, Output, RenderContext,
+ RenderError as HandlebarsRenderError,
+};
+use serde_json::Value;
+
+use super::{table::Table, value_to_string};
+
+fn render_html_table(
+ h: &Helper,
+ _: &Handlebars,
+ _: &Context,
+ _: &mut RenderContext,
+ out: &mut dyn Output,
+) -> HelperResult {
+ let param = h
+ .param(0)
+ .ok_or_else(|| HandlebarsRenderError::new("parameter not found"))?;
+
+ let value = param.value();
+
+ let table: Table = serde_json::from_value(value.clone())?;
+
+ out.write("<table style=\"border: 1px solid\";border-style=\"collapse\">\n")?;
+
+ // Write header
+ out.write(" <tr>\n")?;
+ for column in &table.schema.columns {
+ out.write(" <th style=\"border: 1px solid\">")?;
+ out.write(&handlebars::html_escape(&column.label))?;
+ out.write("</th>\n")?;
+ }
+ out.write(" </tr>\n")?;
+
+ // Write individual rows
+ for row in &table.data {
+ out.write(" <tr>\n")?;
+
+ for column in &table.schema.columns {
+ let entry = row.get(&column.id).unwrap_or(&Value::Null);
+
+ let text = if let Some(renderer) = &column.renderer {
+ renderer.render(entry)?
+ } else {
+ value_to_string(entry)
+ };
+
+ out.write(" <td style=\"border: 1px solid\">")?;
+ out.write(&handlebars::html_escape(&text))?;
+ out.write("</td>\n")?;
+ }
+ out.write(" </tr>\n")?;
+ }
+
+ out.write("</table>\n")?;
+
+ Ok(())
+}
+
+fn render_object(
+ h: &Helper,
+ _: &Handlebars,
+ _: &Context,
+ _: &mut RenderContext,
+ out: &mut dyn Output,
+) -> HelperResult {
+ let param = h
+ .param(0)
+ .ok_or_else(|| HandlebarsRenderError::new("parameter not found"))?;
+
+ let value = param.value();
+
+ out.write("\n<pre>")?;
+ out.write(&serde_json::to_string_pretty(&value)?)?;
+ out.write("\n</pre>\n")?;
+
+ Ok(())
+}
+
+define_helper_with_prefix_and_postfix!(verbatim_monospaced, "<pre>", "</pre>");
+define_helper_with_prefix_and_postfix!(heading_1, "<h1 style=\"font-size: 1.2em\">", "</h1>");
+define_helper_with_prefix_and_postfix!(heading_2, "<h2 style=\"font-size: 1em\">", "</h2>");
+define_helper_with_prefix_and_postfix!(
+ verbatim,
+ "<pre style=\"font-family: sans-serif\">",
+ "</pre>"
+);
+
+pub(super) fn block_render_functions() -> BlockRenderFunctions {
+ BlockRenderFunctions {
+ table: Box::new(render_html_table),
+ verbatim_monospaced: Box::new(verbatim_monospaced),
+ object: Box::new(render_object),
+ heading_1: Box::new(heading_1),
+ heading_2: Box::new(heading_2),
+ verbatim: Box::new(verbatim),
+ }
+}
diff --git a/proxmox-notify/src/renderer/mod.rs b/proxmox-notify/src/renderer/mod.rs
new file mode 100644
index 00000000..2cf64a9f
--- /dev/null
+++ b/proxmox-notify/src/renderer/mod.rs
@@ -0,0 +1,366 @@
+//! Module for rendering notification templates.
+
+use handlebars::{
+ Context, Handlebars, Helper, HelperDef, HelperResult, Output, RenderContext,
+ RenderError as HandlebarsRenderError,
+};
+use std::time::Duration;
+
+use serde::{Deserialize, Serialize};
+use serde_json::Value;
+
+use crate::Error;
+use proxmox_human_byte::HumanByte;
+use proxmox_time::TimeSpan;
+
+mod html;
+mod plaintext;
+mod table;
+
+/// Convert a serde_json::Value to a String.
+///
+/// The main difference between this and simply calling Value::to_string is that
+/// this will print strings without double quotes
+fn value_to_string(value: &Value) -> String {
+ match value {
+ Value::String(s) => s.clone(),
+ v => v.to_string(),
+ }
+}
+
+/// Render a serde_json::Value as a byte size with proper units (IEC, base 2)
+///
+/// Will return `None` if `val` does not contain a number.
+fn value_to_byte_size(val: &Value) -> Option<String> {
+ let size = val.as_f64()?;
+ Some(format!("{}", HumanByte::new_binary(size)))
+}
+
+/// Render a serde_json::Value as a duration.
+/// The value is expected to contain the duration in seconds.
+///
+/// Will return `None` if `val` does not contain a number.
+fn value_to_duration(val: &Value) -> Option<String> {
+ let duration = val.as_u64()?;
+ let time_span = TimeSpan::from(Duration::from_secs(duration));
+
+ Some(format!("{time_span}"))
+}
+
+/// Render as serde_json::Value as a timestamp.
+/// The value is expected to contain the timestamp as a unix epoch.
+///
+/// Will return `None` if `val` does not contain a number.
+fn value_to_timestamp(val: &Value) -> Option<String> {
+ let timestamp = val.as_i64()?;
+ proxmox_time::strftime_local("%F %H:%M:%S", timestamp).ok()
+}
+
+/// Available render functions for `serde_json::Values``
+///
+/// May be used as a handlebars helper, e.g.
+/// ```text
+/// {{human-bytes 1024}}
+/// ```
+///
+/// Value renderer can also be used for rendering values in table columns:
+/// ```text
+/// let properties = json!({
+/// "table": {
+/// "schema": {
+/// "columns": [
+/// {
+/// "label": "Size",
+/// "id": "size",
+/// "renderer": "human-bytes"
+/// }
+/// ],
+/// },
+/// "data" : [
+/// {
+/// "size": 1024 * 1024,
+/// },
+/// ]
+/// }
+/// });
+/// ```
+///
+#[derive(Debug, Deserialize, Serialize)]
+#[serde(rename_all = "kebab-case")]
+pub enum ValueRenderFunction {
+ HumanBytes,
+ Duration,
+ Timestamp,
+}
+
+impl ValueRenderFunction {
+ fn render(&self, value: &Value) -> Result<String, HandlebarsRenderError> {
+ match self {
+ ValueRenderFunction::HumanBytes => value_to_byte_size(value),
+ ValueRenderFunction::Duration => value_to_duration(value),
+ ValueRenderFunction::Timestamp => value_to_timestamp(value),
+ }
+ .ok_or_else(|| {
+ HandlebarsRenderError::new(format!(
+ "could not render value {value} with renderer {self:?}"
+ ))
+ })
+ }
+
+ fn register_helpers(handlebars: &mut Handlebars) {
+ ValueRenderFunction::HumanBytes.register_handlebars_helper(handlebars);
+ ValueRenderFunction::Duration.register_handlebars_helper(handlebars);
+ ValueRenderFunction::Timestamp.register_handlebars_helper(handlebars);
+ }
+
+ fn register_handlebars_helper(&'static self, handlebars: &mut Handlebars) {
+ // Use serde to get own kebab-case representation that is later used
+ // to register the helper, e.g. HumanBytes -> human-bytes
+ let tag = serde_json::to_string(self)
+ .expect("serde failed to serialize ValueRenderFunction enum");
+
+ // But as it's a string value, the generated string is quoted,
+ // so remove leading/trailing double quotes
+ let tag = tag
+ .strip_prefix('\"')
+ .and_then(|t| t.strip_suffix('\"'))
+ .expect("serde serialized string representation was not contained in double quotes");
+
+ handlebars.register_helper(
+ tag,
+ Box::new(
+ |h: &Helper,
+ _r: &Handlebars,
+ _: &Context,
+ _rc: &mut RenderContext,
+ out: &mut dyn Output|
+ -> HelperResult {
+ let param = h
+ .param(0)
+ .ok_or(HandlebarsRenderError::new("parameter not found"))?;
+
+ let value = param.value();
+ out.write(&self.render(value)?)?;
+
+ Ok(())
+ },
+ ),
+ );
+ }
+}
+
+/// Available renderers for notification templates.
+#[derive(Copy, Clone)]
+pub enum TemplateRenderer {
+ /// Render to HTML code
+ Html,
+ /// Render to plain text
+ Plaintext,
+}
+
+impl TemplateRenderer {
+ fn prefix(&self) -> &str {
+ match self {
+ TemplateRenderer::Html => "<html>\n<body>\n",
+ TemplateRenderer::Plaintext => "",
+ }
+ }
+
+ fn postfix(&self) -> &str {
+ match self {
+ TemplateRenderer::Html => "\n</body>\n</html>",
+ TemplateRenderer::Plaintext => "",
+ }
+ }
+
+ fn block_render_fns(&self) -> BlockRenderFunctions {
+ match self {
+ TemplateRenderer::Html => html::block_render_functions(),
+ TemplateRenderer::Plaintext => plaintext::block_render_functions(),
+ }
+ }
+
+ fn escape_fn(&self) -> fn(&str) -> String {
+ match self {
+ TemplateRenderer::Html => handlebars::html_escape,
+ TemplateRenderer::Plaintext => handlebars::no_escape,
+ }
+ }
+}
+
+type HelperFn = dyn HelperDef + Send + Sync;
+
+struct BlockRenderFunctions {
+ table: Box<HelperFn>,
+ verbatim_monospaced: Box<HelperFn>,
+ object: Box<HelperFn>,
+ heading_1: Box<HelperFn>,
+ heading_2: Box<HelperFn>,
+ verbatim: Box<HelperFn>,
+}
+
+impl BlockRenderFunctions {
+ fn register_helpers(self, handlebars: &mut Handlebars) {
+ handlebars.register_helper("table", self.table);
+ handlebars.register_helper("verbatim", self.verbatim);
+ handlebars.register_helper("verbatim-monospaced", self.verbatim_monospaced);
+ handlebars.register_helper("object", self.object);
+ handlebars.register_helper("heading-1", self.heading_1);
+ handlebars.register_helper("heading-2", self.heading_2);
+ }
+}
+
+fn render_template_impl(
+ template: &str,
+ properties: Option<&Value>,
+ renderer: TemplateRenderer,
+) -> Result<String, Error> {
+ let properties = properties.unwrap_or(&Value::Null);
+
+ let mut handlebars = Handlebars::new();
+ handlebars.register_escape_fn(renderer.escape_fn());
+
+ let block_render_fns = renderer.block_render_fns();
+ block_render_fns.register_helpers(&mut handlebars);
+
+ ValueRenderFunction::register_helpers(&mut handlebars);
+
+ let rendered_template = handlebars
+ .render_template(template, properties)
+ .map_err(|err| Error::RenderError(err.into()))?;
+
+ Ok(rendered_template)
+}
+
+/// Render a template string.
+///
+/// The output format can be chosen via the `renderer` parameter (see [TemplateRenderer]
+/// for available options).
+pub fn render_template(
+ renderer: TemplateRenderer,
+ template: &str,
+ properties: Option<&Value>,
+) -> Result<String, Error> {
+ let mut rendered_template = String::from(renderer.prefix());
+
+ rendered_template.push_str(&render_template_impl(template, properties, renderer)?);
+ rendered_template.push_str(renderer.postfix());
+
+ Ok(rendered_template)
+}
+
+#[macro_export]
+macro_rules! define_helper_with_prefix_and_postfix {
+ ($name:ident, $pre:expr, $post:expr) => {
+ fn $name<'reg, 'rc>(
+ h: &Helper<'reg, 'rc>,
+ handlebars: &'reg Handlebars,
+ context: &'rc Context,
+ render_context: &mut RenderContext<'reg, 'rc>,
+ out: &mut dyn Output,
+ ) -> HelperResult {
+ use handlebars::Renderable;
+
+ let block_text = h.template();
+ let param = h.param(0);
+
+ out.write($pre)?;
+ match (param, block_text) {
+ (None, Some(block_text)) => {
+ block_text.render(handlebars, context, render_context, out)
+ }
+ (Some(param), None) => {
+ let value = param.value();
+ let text = value.as_str().ok_or_else(|| {
+ HandlebarsRenderError::new(format!("value {value} is not a string"))
+ })?;
+
+ out.write(text)?;
+ Ok(())
+ }
+ (Some(_), Some(_)) => Err(HandlebarsRenderError::new(
+ "Cannot use parameter and template at the same time",
+ )),
+ (None, None) => Err(HandlebarsRenderError::new(
+ "Neither parameter nor template was provided",
+ )),
+ }?;
+ out.write($post)?;
+ Ok(())
+ }
+ };
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use serde_json::json;
+
+ #[test]
+ fn test_render_template() -> Result<(), Error> {
+ let properties = json!({
+ "dur": 12345,
+ "size": 1024 * 15,
+
+ "table": {
+ "schema": {
+ "columns": [
+ {
+ "id": "col1",
+ "label": "Column 1"
+ },
+ {
+ "id": "col2",
+ "label": "Column 2"
+ }
+ ]
+ },
+ "data": [
+ {
+ "col1": "val1",
+ "col2": "val2"
+ },
+ {
+ "col1": "val3",
+ "col2": "val4"
+ },
+ ]
+ }
+
+ });
+
+ let template = r#"
+{{heading-1 "Hello World"}}
+
+{{heading-2 "Hello World"}}
+
+{{human-bytes size}}
+{{duration dur}}
+
+{{table table}}"#;
+
+ let expected_plaintext = r#"
+Hello World
+===========
+
+Hello World
+-----------
+
+15 KiB
+3h 25min 45s
+
+Column 1 Column 2
+val1 val2
+val3 val4
+"#;
+
+ let rendered_plaintext =
+ render_template(TemplateRenderer::Plaintext, template, Some(&properties))?;
+
+ // Let's not bother about testing the HTML output, too fragile.
+
+ assert_eq!(rendered_plaintext, expected_plaintext);
+
+ Ok(())
+ }
+}
diff --git a/proxmox-notify/src/renderer/plaintext.rs b/proxmox-notify/src/renderer/plaintext.rs
new file mode 100644
index 00000000..58c51599
--- /dev/null
+++ b/proxmox-notify/src/renderer/plaintext.rs
@@ -0,0 +1,141 @@
+use crate::define_helper_with_prefix_and_postfix;
+use crate::renderer::BlockRenderFunctions;
+use handlebars::{
+ Context, Handlebars, Helper, HelperResult, Output, RenderContext,
+ RenderError as HandlebarsRenderError,
+};
+use serde_json::Value;
+use std::collections::HashMap;
+
+use super::{table::Table, value_to_string};
+
+fn optimal_column_widths(table: &Table) -> HashMap<&str, usize> {
+ let mut widths = HashMap::new();
+
+ for column in &table.schema.columns {
+ let mut min_width = column.label.len();
+
+ for row in &table.data {
+ let entry = row.get(&column.id).unwrap_or(&Value::Null);
+
+ let text = if let Some(renderer) = &column.renderer {
+ renderer.render(entry).unwrap_or_default()
+ } else {
+ value_to_string(entry)
+ };
+
+ min_width = std::cmp::max(text.len(), min_width);
+ }
+
+ widths.insert(column.label.as_str(), min_width + 4);
+ }
+
+ widths
+}
+
+fn render_plaintext_table(
+ h: &Helper,
+ _: &Handlebars,
+ _: &Context,
+ _: &mut RenderContext,
+ out: &mut dyn Output,
+) -> HelperResult {
+ let param = h
+ .param(0)
+ .ok_or_else(|| HandlebarsRenderError::new("parameter not found"))?;
+ let value = param.value();
+ let table: Table = serde_json::from_value(value.clone())?;
+ let widths = optimal_column_widths(&table);
+
+ // Write header
+ for column in &table.schema.columns {
+ let width = widths.get(column.label.as_str()).unwrap_or(&0);
+ out.write(&format!("{label:width$}", label = column.label))?;
+ }
+
+ out.write("\n")?;
+
+ // Write individual rows
+ for row in &table.data {
+ for column in &table.schema.columns {
+ let entry = row.get(&column.id).unwrap_or(&Value::Null);
+ let width = widths.get(column.label.as_str()).unwrap_or(&0);
+
+ let text = if let Some(renderer) = &column.renderer {
+ renderer.render(entry)?
+ } else {
+ value_to_string(entry)
+ };
+
+ out.write(&format!("{text:width$}",))?;
+ }
+ out.write("\n")?;
+ }
+
+ Ok(())
+}
+
+macro_rules! define_underlining_heading_fn {
+ ($name:ident, $underline:expr) => {
+ fn $name<'reg, 'rc>(
+ h: &Helper<'reg, 'rc>,
+ _handlebars: &'reg Handlebars,
+ _context: &'rc Context,
+ _render_context: &mut RenderContext<'reg, 'rc>,
+ out: &mut dyn Output,
+ ) -> HelperResult {
+ let param = h
+ .param(0)
+ .ok_or_else(|| HandlebarsRenderError::new("No parameter provided"))?;
+
+ let value = param.value();
+ let text = value.as_str().ok_or_else(|| {
+ HandlebarsRenderError::new(format!("value {value} is not a string"))
+ })?;
+
+ out.write(text)?;
+ out.write("\n")?;
+
+ for _ in 0..text.len() {
+ out.write($underline)?;
+ }
+ Ok(())
+ }
+ };
+}
+
+define_helper_with_prefix_and_postfix!(verbatim_monospaced, "", "");
+define_underlining_heading_fn!(heading_1, "=");
+define_underlining_heading_fn!(heading_2, "-");
+define_helper_with_prefix_and_postfix!(verbatim, "", "");
+
+fn render_object(
+ h: &Helper,
+ _: &Handlebars,
+ _: &Context,
+ _: &mut RenderContext,
+ out: &mut dyn Output,
+) -> HelperResult {
+ let param = h
+ .param(0)
+ .ok_or_else(|| HandlebarsRenderError::new("parameter not found"))?;
+
+ let value = param.value();
+
+ out.write("\n")?;
+ out.write(&serde_json::to_string_pretty(&value)?)?;
+ out.write("\n")?;
+
+ Ok(())
+}
+
+pub(super) fn block_render_functions() -> BlockRenderFunctions {
+ BlockRenderFunctions {
+ table: Box::new(render_plaintext_table),
+ verbatim_monospaced: Box::new(verbatim_monospaced),
+ verbatim: Box::new(verbatim),
+ object: Box::new(render_object),
+ heading_1: Box::new(heading_1),
+ heading_2: Box::new(heading_2),
+ }
+}
diff --git a/proxmox-notify/src/renderer/table.rs b/proxmox-notify/src/renderer/table.rs
new file mode 100644
index 00000000..74f68482
--- /dev/null
+++ b/proxmox-notify/src/renderer/table.rs
@@ -0,0 +1,24 @@
+use std::collections::HashMap;
+
+use serde::Deserialize;
+use serde_json::Value;
+
+use super::ValueRenderFunction;
+
+#[derive(Debug, Deserialize)]
+pub struct ColumnSchema {
+ pub label: String,
+ pub id: String,
+ pub renderer: Option<ValueRenderFunction>,
+}
+
+#[derive(Debug, Deserialize)]
+pub struct TableSchema {
+ pub columns: Vec<ColumnSchema>,
+}
+
+#[derive(Debug, Deserialize)]
+pub struct Table {
+ pub schema: TableSchema,
+ pub data: Vec<HashMap<String, Value>>,
+}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 14/66] notify: add example for template rendering
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (12 preceding siblings ...)
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 13/66] notify: add template rendering Lukas Wagner
@ 2023-07-17 14:59 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 15/66] notify: add context Lukas Wagner
` (54 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 14:59 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/examples/render.rs | 63 +++++++++++++++++++++++++++++++
1 file changed, 63 insertions(+)
create mode 100644 proxmox-notify/examples/render.rs
diff --git a/proxmox-notify/examples/render.rs b/proxmox-notify/examples/render.rs
new file mode 100644
index 00000000..5c1f2b2d
--- /dev/null
+++ b/proxmox-notify/examples/render.rs
@@ -0,0 +1,63 @@
+use proxmox_notify::renderer::{render_template, TemplateRenderer};
+use proxmox_notify::Error;
+
+use serde_json::json;
+
+const TEMPLATE: &str = r#"
+{{ heading-1 "Backup Report"}}
+A backup job on host {{host}} was run.
+
+{{ heading-2 "Guests"}}
+{{ table table }}
+The total size of all backups is {{human-bytes total-size}}.
+
+The backup job took {{duration total-time}}.
+
+{{ heading-2 "Logs"}}
+{{ verbatim-monospaced logs}}
+
+{{ heading-2 "Objects"}}
+{{ object table }}
+"#;
+
+fn main() -> Result<(), Error> {
+ let properties = json!({
+ "host": "pali",
+ "logs": "100: starting backup\n100: backup failed",
+ "total-size": 1024 * 1024 + 2048 * 1024,
+ "total-time": 100,
+ "table": {
+ "schema": {
+ "columns": [
+ {
+ "label": "VMID",
+ "id": "vmid"
+ },
+ {
+ "label": "Size",
+ "id": "size",
+ "renderer": "human-bytes"
+ }
+ ],
+ },
+ "data" : [
+ {
+ "vmid": 1001,
+ "size": 1024 * 1024,
+ },
+ {
+ "vmid": 1002,
+ "size": 2048 * 1024,
+ }
+ ]
+ }
+ });
+
+ let output = render_template(TemplateRenderer::Html, TEMPLATE, Some(&properties))?;
+ println!("{output}");
+
+ let output = render_template(TemplateRenderer::Plaintext, TEMPLATE, Some(&properties))?;
+ println!("{output}");
+
+ Ok(())
+}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 15/66] notify: add context
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (13 preceding siblings ...)
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 14/66] notify: add example for " Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-18 12:57 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 16/66] notify: sendmail: allow users as recipients Lukas Wagner
` (53 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Since `proxmox-notify` is intended to be used by multiple products,
there needs to be a way to inject product-specific behavior.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/context.rs | 13 +++++++++++++
proxmox-notify/src/lib.rs | 1 +
2 files changed, 14 insertions(+)
create mode 100644 proxmox-notify/src/context.rs
diff --git a/proxmox-notify/src/context.rs b/proxmox-notify/src/context.rs
new file mode 100644
index 00000000..55c0eda1
--- /dev/null
+++ b/proxmox-notify/src/context.rs
@@ -0,0 +1,13 @@
+use std::sync::Mutex;
+
+pub trait Context: Send + Sync {}
+
+static CONTEXT: Mutex<Option<&'static dyn Context>> = Mutex::new(None);
+
+pub fn set_context(context: &'static dyn Context) {
+ *CONTEXT.lock().unwrap() = Some(context);
+}
+
+pub(crate) fn context() -> &'static dyn Context {
+ (*CONTEXT.lock().unwrap()).expect("context for proxmox-notify has not been set yet")
+}
diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
index 548cc56f..3c2b6d55 100644
--- a/proxmox-notify/src/lib.rs
+++ b/proxmox-notify/src/lib.rs
@@ -13,6 +13,7 @@ use std::error::Error as StdError;
pub mod api;
mod config;
+pub mod context;
pub mod endpoints;
pub mod filter;
pub mod group;
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 16/66] notify: sendmail: allow users as recipients
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (14 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 15/66] notify: add context Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 17/66] notify: sendmail: query default author/mailfrom from context Lukas Wagner
` (52 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
This introduces a new configuration parameter `mailto-user`.
A user's email address will be looked up in the product-specific
user database.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/api/sendmail.rs | 32 ++++++++++++++++---
proxmox-notify/src/context.rs | 4 ++-
proxmox-notify/src/endpoints/sendmail.rs | 40 +++++++++++++++++++++---
proxmox-notify/src/schema.rs | 6 ++++
4 files changed, 73 insertions(+), 9 deletions(-)
diff --git a/proxmox-notify/src/api/sendmail.rs b/proxmox-notify/src/api/sendmail.rs
index 6b0323e3..070ed6e7 100644
--- a/proxmox-notify/src/api/sendmail.rs
+++ b/proxmox-notify/src/api/sendmail.rs
@@ -45,6 +45,13 @@ pub fn add_endpoint(config: &mut Config, endpoint: &SendmailConfig) -> Result<()
super::filter::get_filter(config, filter)?;
}
+ if endpoint.mailto.is_none() && endpoint.mailto_user.is_none() {
+ return Err(ApiError::bad_request(
+ "must at least provide one recipient, either in mailto or in mailto-user",
+ None,
+ ));
+ }
+
config
.config
.set_data(&endpoint.name, SENDMAIL_TYPENAME, endpoint)
@@ -81,12 +88,18 @@ pub fn update_endpoint(
DeleteableSendmailProperty::Author => endpoint.author = None,
DeleteableSendmailProperty::Comment => endpoint.comment = None,
DeleteableSendmailProperty::Filter => endpoint.filter = None,
+ DeleteableSendmailProperty::Mailto => endpoint.mailto = None,
+ DeleteableSendmailProperty::MailtoUser => endpoint.mailto_user = None,
}
}
}
if let Some(mailto) = &updater.mailto {
- endpoint.mailto = mailto.iter().map(String::from).collect();
+ endpoint.mailto = Some(mailto.iter().map(String::from).collect());
+ }
+
+ if let Some(mailto_user) = &updater.mailto_user {
+ endpoint.mailto_user = Some(mailto_user.iter().map(String::from).collect());
}
if let Some(from_address) = &updater.from_address {
@@ -106,6 +119,13 @@ pub fn update_endpoint(
endpoint.filter = Some(filter.into());
}
+ if endpoint.mailto.is_none() && endpoint.mailto_user.is_none() {
+ return Err(ApiError::bad_request(
+ "must at least provide one recipient, either in mailto or in mailto-user",
+ None,
+ ));
+ }
+
config
.config
.set_data(name, SENDMAIL_TYPENAME, &endpoint)
@@ -143,7 +163,8 @@ pub mod tests {
config,
&SendmailConfig {
name: name.into(),
- mailto: vec!["user1@example.com".into()],
+ mailto: Some(vec!["user1@example.com".into()]),
+ mailto_user: None,
from_address: Some("from@example.com".into()),
author: Some("root".into()),
comment: Some("Comment".into()),
@@ -187,6 +208,7 @@ pub mod tests {
"sendmail-endpoint",
&SendmailConfigUpdater {
mailto: Some(vec!["user2@example.com".into(), "user3@example.com".into()]),
+ mailto_user: None,
from_address: Some("root@example.com".into()),
author: Some("newauthor".into()),
comment: Some("new comment".into()),
@@ -212,6 +234,7 @@ pub mod tests {
"sendmail-endpoint",
&SendmailConfigUpdater {
mailto: Some(vec!["user2@example.com".into(), "user3@example.com".into()]),
+ mailto_user: Some(vec!["root@pam".into()]),
from_address: Some("root@example.com".into()),
author: Some("newauthor".into()),
comment: Some("new comment".into()),
@@ -225,11 +248,12 @@ pub mod tests {
assert_eq!(
endpoint.mailto,
- vec![
+ Some(vec![
"user2@example.com".to_string(),
"user3@example.com".to_string()
- ]
+ ])
);
+ assert_eq!(endpoint.mailto_user, Some(vec!["root@pam".to_string(),]));
assert_eq!(endpoint.from_address, Some("root@example.com".to_string()));
assert_eq!(endpoint.author, Some("newauthor".to_string()));
assert_eq!(endpoint.comment, Some("new comment".to_string()));
diff --git a/proxmox-notify/src/context.rs b/proxmox-notify/src/context.rs
index 55c0eda1..25be949a 100644
--- a/proxmox-notify/src/context.rs
+++ b/proxmox-notify/src/context.rs
@@ -1,6 +1,8 @@
use std::sync::Mutex;
-pub trait Context: Send + Sync {}
+pub trait Context: Send + Sync {
+ fn lookup_email_for_user(&self, user: &str) -> Option<String>;
+}
static CONTEXT: Mutex<Option<&'static dyn Context>> = Mutex::new(None);
diff --git a/proxmox-notify/src/endpoints/sendmail.rs b/proxmox-notify/src/endpoints/sendmail.rs
index 9d06e7c4..c2b44b2d 100644
--- a/proxmox-notify/src/endpoints/sendmail.rs
+++ b/proxmox-notify/src/endpoints/sendmail.rs
@@ -1,6 +1,8 @@
+use crate::context::context;
use crate::renderer::TemplateRenderer;
-use crate::schema::{COMMENT_SCHEMA, EMAIL_SCHEMA, ENTITY_NAME_SCHEMA};
+use crate::schema::{COMMENT_SCHEMA, EMAIL_SCHEMA, ENTITY_NAME_SCHEMA, USER_SCHEMA};
use crate::{renderer, Endpoint, Error, Notification};
+use std::collections::HashSet;
use proxmox_schema::{api, Updater};
use serde::{Deserialize, Serialize};
@@ -17,6 +19,14 @@ pub(crate) const SENDMAIL_TYPENAME: &str = "sendmail";
items: {
schema: EMAIL_SCHEMA,
},
+ optional: true,
+ },
+ "mailto-user": {
+ type: Array,
+ items: {
+ schema: USER_SCHEMA,
+ },
+ optional: true,
},
comment: {
optional: true,
@@ -36,7 +46,11 @@ pub struct SendmailConfig {
#[updater(skip)]
pub name: String,
/// Mail recipients
- pub mailto: Vec<String>,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub mailto: Option<Vec<String>>,
+ /// Mail recipients
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub mailto_user: Option<Vec<String>>,
/// `From` address for the mail
#[serde(skip_serializing_if = "Option::is_none")]
pub from_address: Option<String>,
@@ -58,6 +72,8 @@ pub enum DeleteableSendmailProperty {
Author,
Comment,
Filter,
+ Mailto,
+ MailtoUser,
}
/// A sendmail notification endpoint.
@@ -67,7 +83,21 @@ pub struct SendmailEndpoint {
impl Endpoint for SendmailEndpoint {
fn send(&self, notification: &Notification) -> Result<(), Error> {
- let recipients: Vec<&str> = self.config.mailto.iter().map(String::as_str).collect();
+ let mut recipients = HashSet::new();
+
+ if let Some(mailto_addrs) = self.config.mailto.as_ref() {
+ for addr in mailto_addrs {
+ recipients.insert(addr.clone());
+ }
+ }
+
+ if let Some(users) = self.config.mailto_user.as_ref() {
+ for user in users {
+ if let Some(addr) = context().lookup_email_for_user(user) {
+ recipients.insert(addr);
+ }
+ }
+ }
let properties = notification.properties.as_ref();
@@ -85,8 +115,10 @@ impl Endpoint for SendmailEndpoint {
// "Proxmox Backup Server" if it is not set.
let author = self.config.author.as_deref().or(Some(""));
+ let recipients_str: Vec<&str> = recipients.iter().map(String::as_str).collect();
+
proxmox_sys::email::sendmail(
- &recipients,
+ &recipients_str,
&subject,
Some(&text_part),
Some(&html_part),
diff --git a/proxmox-notify/src/schema.rs b/proxmox-notify/src/schema.rs
index 68f11959..006e918b 100644
--- a/proxmox-notify/src/schema.rs
+++ b/proxmox-notify/src/schema.rs
@@ -26,6 +26,12 @@ pub const EMAIL_SCHEMA: Schema = StringSchema::new("E-Mail Address.")
.max_length(64)
.schema();
+pub const USER_SCHEMA: Schema = StringSchema::new("User ID including realm, e.g. root@pam.")
+ .format(&SINGLE_LINE_COMMENT_FORMAT)
+ .min_length(2)
+ .max_length(64)
+ .schema();
+
pub const PROXMOX_SAFE_ID_FORMAT: ApiStringFormat =
ApiStringFormat::Pattern(&PROXMOX_SAFE_ID_REGEX);
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 17/66] notify: sendmail: query default author/mailfrom from context
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (15 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 16/66] notify: sendmail: allow users as recipients Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 18/66] notify: gotify: add proxy support Lukas Wagner
` (51 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/context.rs | 2 ++
proxmox-notify/src/endpoints/sendmail.rs | 18 +++++++++++++-----
2 files changed, 15 insertions(+), 5 deletions(-)
diff --git a/proxmox-notify/src/context.rs b/proxmox-notify/src/context.rs
index 25be949a..8b55a2d0 100644
--- a/proxmox-notify/src/context.rs
+++ b/proxmox-notify/src/context.rs
@@ -2,6 +2,8 @@ use std::sync::Mutex;
pub trait Context: Send + Sync {
fn lookup_email_for_user(&self, user: &str) -> Option<String>;
+ fn default_sendmail_author(&self) -> String;
+ fn default_sendmail_from(&self) -> String;
}
static CONTEXT: Mutex<Option<&'static dyn Context>> = Mutex::new(None);
diff --git a/proxmox-notify/src/endpoints/sendmail.rs b/proxmox-notify/src/endpoints/sendmail.rs
index c2b44b2d..850f48f8 100644
--- a/proxmox-notify/src/endpoints/sendmail.rs
+++ b/proxmox-notify/src/endpoints/sendmail.rs
@@ -111,9 +111,17 @@ impl Endpoint for SendmailEndpoint {
let text_part =
renderer::render_template(TemplateRenderer::Plaintext, ¬ification.body, properties)?;
- // proxmox_sys::email::sendmail will set the author to
- // "Proxmox Backup Server" if it is not set.
- let author = self.config.author.as_deref().or(Some(""));
+ let author = self
+ .config
+ .author
+ .clone()
+ .unwrap_or_else(|| context().default_sendmail_author());
+
+ let mailfrom = self
+ .config
+ .from_address
+ .clone()
+ .unwrap_or_else(|| context().default_sendmail_from());
let recipients_str: Vec<&str> = recipients.iter().map(String::as_str).collect();
@@ -122,8 +130,8 @@ impl Endpoint for SendmailEndpoint {
&subject,
Some(&text_part),
Some(&html_part),
- self.config.from_address.as_deref(),
- author,
+ Some(&mailfrom),
+ Some(&author),
)
.map_err(|err| Error::NotifyFailed(self.config.name.clone(), err.into()))
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 18/66] notify: gotify: add proxy support
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (16 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 17/66] notify: sendmail: query default author/mailfrom from context Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 19/66] notify: api: allow to query entities referenced by filter/target Lukas Wagner
` (50 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
The proxy configuration will be read from datacenter.cfg via
a new method of the `Context` trait.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/context.rs | 1 +
proxmox-notify/src/endpoints/gotify.rs | 22 ++++++++++++++++------
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git a/proxmox-notify/src/context.rs b/proxmox-notify/src/context.rs
index 8b55a2d0..418d3d56 100644
--- a/proxmox-notify/src/context.rs
+++ b/proxmox-notify/src/context.rs
@@ -4,6 +4,7 @@ pub trait Context: Send + Sync {
fn lookup_email_for_user(&self, user: &str) -> Option<String>;
fn default_sendmail_author(&self) -> String;
fn default_sendmail_from(&self) -> String;
+ fn http_proxy_config(&self) -> Option<String>;
}
static CONTEXT: Mutex<Option<&'static dyn Context>> = Mutex::new(None);
diff --git a/proxmox-notify/src/endpoints/gotify.rs b/proxmox-notify/src/endpoints/gotify.rs
index b278b90d..eae22f42 100644
--- a/proxmox-notify/src/endpoints/gotify.rs
+++ b/proxmox-notify/src/endpoints/gotify.rs
@@ -7,8 +7,9 @@ use crate::{renderer, Endpoint, Error, Notification, Severity};
use serde::{Deserialize, Serialize};
use serde_json::json;
+use crate::context::context;
use proxmox_http::client::sync::Client;
-use proxmox_http::{HttpClient, HttpOptions};
+use proxmox_http::{HttpClient, HttpOptions, ProxyConfig};
use proxmox_schema::{api, Updater};
fn severity_to_priority(level: Severity) -> u32 {
@@ -83,11 +84,6 @@ pub enum DeleteableGotifyProperty {
impl Endpoint for GotifyEndpoint {
fn send(&self, notification: &Notification) -> Result<(), Error> {
- // TODO: What about proxy configuration?
- let client = Client::new(HttpOptions::default());
-
- let uri = format!("{}/message", self.config.server);
-
let properties = notification.properties.as_ref();
let title = renderer::render_template(
@@ -120,6 +116,20 @@ impl Endpoint for GotifyEndpoint {
format!("Bearer {}", self.private_config.token),
)]);
+ let proxy_config = context()
+ .http_proxy_config()
+ .map(|url| ProxyConfig::parse_proxy_url(&url))
+ .transpose()
+ .map_err(|err| Error::NotifyFailed(self.name().to_string(), err.into()))?;
+
+ let options = HttpOptions {
+ proxy_config,
+ ..Default::default()
+ };
+
+ let client = Client::new(options);
+ let uri = format!("{}/message", self.config.server);
+
client
.post(
&uri,
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 19/66] notify: api: allow to query entities referenced by filter/target
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (17 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 18/66] notify: gotify: add proxy support Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-18 13:02 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 20/66] notify: on deletion, check if a filter/endp. is still used by anything Lukas Wagner
` (49 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/api/common.rs | 11 +++
proxmox-notify/src/api/mod.rs | 125 +++++++++++++++++++++++++++++++
2 files changed, 136 insertions(+)
diff --git a/proxmox-notify/src/api/common.rs b/proxmox-notify/src/api/common.rs
index 518caa8f..48761fbb 100644
--- a/proxmox-notify/src/api/common.rs
+++ b/proxmox-notify/src/api/common.rs
@@ -42,3 +42,14 @@ pub fn test_target(config: &Config, endpoint: &str) -> Result<(), ApiError> {
Ok(())
}
+
+/// Return all entities (targets, groups, filters) that are linked to the entity.
+/// For instance, if a group 'grp1' contains the targets 'a', 'b' and 'c',
+/// where grp1 has 'filter1' and 'a' has 'filter2' as filters, then
+/// the result for 'grp1' would be [grp1, a, b, c, filter1, filter2].
+/// The result will always contain the entity that was passed as a parameter.
+/// If the entity does not exist, the result will only contain the entity.
+pub fn get_referenced_entities(config: &Config, entity: &str) -> Result<Vec<String>, ApiError> {
+ let entities = super::get_referenced_entities(config, entity);
+ Ok(Vec::from_iter(entities.into_iter()))
+}
diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
index 12811baf..d8a44bf2 100644
--- a/proxmox-notify/src/api/mod.rs
+++ b/proxmox-notify/src/api/mod.rs
@@ -1,3 +1,4 @@
+use std::collections::HashSet;
use std::error::Error as StdError;
use std::fmt::Display;
@@ -101,6 +102,48 @@ fn endpoint_exists(config: &Config, name: &str) -> bool {
exists
}
+fn get_referenced_entities(config: &Config, entity: &str) -> HashSet<String> {
+ let mut to_expand = HashSet::new();
+ let mut expanded = HashSet::new();
+ to_expand.insert(entity.to_string());
+
+ let expand = |entities: &HashSet<String>| -> HashSet<String> {
+ let mut new = HashSet::new();
+
+ for entity in entities {
+ if let Ok(group) = group::get_group(config, entity) {
+ for target in group.endpoint {
+ new.insert(target.clone());
+ }
+ }
+
+ #[cfg(feature = "sendmail")]
+ if let Ok(target) = sendmail::get_endpoint(config, entity) {
+ if let Some(filter) = target.filter {
+ new.insert(filter.clone());
+ }
+ }
+
+ #[cfg(feature = "gotify")]
+ if let Ok(target) = gotify::get_endpoint(config, entity) {
+ if let Some(filter) = target.filter {
+ new.insert(filter.clone());
+ }
+ }
+ }
+
+ new
+ };
+
+ while !to_expand.is_empty() {
+ let new = expand(&to_expand);
+ expanded.extend(to_expand.drain());
+ to_expand = new;
+ }
+
+ expanded
+}
+
#[cfg(test)]
mod test_helpers {
use crate::Config;
@@ -109,3 +152,85 @@ mod test_helpers {
Config::new("", "").unwrap()
}
}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use crate::endpoints::gotify::{GotifyConfig, GotifyPrivateConfig};
+ use crate::endpoints::sendmail::SendmailConfig;
+ use crate::filter::FilterConfig;
+ use crate::group::GroupConfig;
+
+ #[test]
+ fn test_get_referenced_entities() {
+ let mut config = super::test_helpers::empty_config();
+
+ filter::add_filter(
+ &mut config,
+ &FilterConfig {
+ name: "filter".to_string(),
+ ..Default::default()
+ },
+ )
+ .unwrap();
+
+ sendmail::add_endpoint(
+ &mut config,
+ &SendmailConfig {
+ name: "sendmail".to_string(),
+ mailto: Some(vec!["foo@example.com".to_string()]),
+ filter: Some("filter".to_string()),
+ ..Default::default()
+ },
+ )
+ .unwrap();
+
+ gotify::add_endpoint(
+ &mut config,
+ &GotifyConfig {
+ name: "gotify".to_string(),
+ server: "localhost".to_string(),
+ filter: Some("filter".to_string()),
+ ..Default::default()
+ },
+ &GotifyPrivateConfig {
+ name: "gotify".to_string(),
+ token: "foo".to_string(),
+ },
+ )
+ .unwrap();
+
+ group::add_group(
+ &mut config,
+ &GroupConfig {
+ name: "group".to_string(),
+ endpoint: vec!["gotify".to_string(), "sendmail".to_string()],
+ filter: Some("filter".to_string()),
+ ..Default::default()
+ },
+ )
+ .unwrap();
+
+ assert_eq!(
+ get_referenced_entities(&config, "filter"),
+ HashSet::from(["filter".to_string()])
+ );
+ assert_eq!(
+ get_referenced_entities(&config, "sendmail"),
+ HashSet::from(["filter".to_string(), "sendmail".to_string()])
+ );
+ assert_eq!(
+ get_referenced_entities(&config, "gotify"),
+ HashSet::from(["filter".to_string(), "gotify".to_string()])
+ );
+ assert_eq!(
+ get_referenced_entities(&config, "group"),
+ HashSet::from([
+ "filter".to_string(),
+ "gotify".to_string(),
+ "sendmail".to_string(),
+ "group".to_string()
+ ])
+ );
+ }
+}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 20/66] notify: on deletion, check if a filter/endp. is still used by anything
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (18 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 19/66] notify: api: allow to query entities referenced by filter/target Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-18 13:20 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 21/66] notify: ensure that filter/group/endpoint names are unique Lukas Wagner
` (48 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/api/filter.rs | 1 +
proxmox-notify/src/api/gotify.rs | 1 +
proxmox-notify/src/api/mod.rs | 113 ++++++++++++++++++++++++++---
proxmox-notify/src/api/sendmail.rs | 1 +
4 files changed, 106 insertions(+), 10 deletions(-)
diff --git a/proxmox-notify/src/api/filter.rs b/proxmox-notify/src/api/filter.rs
index 3fcff6b9..824f802d 100644
--- a/proxmox-notify/src/api/filter.rs
+++ b/proxmox-notify/src/api/filter.rs
@@ -115,6 +115,7 @@ pub fn update_filter(
pub fn delete_filter(config: &mut Config, name: &str) -> Result<(), ApiError> {
// Check if the filter exists
let _ = get_filter(config, name)?;
+ super::ensure_unused(config, name)?;
config.config.sections.remove(name);
diff --git a/proxmox-notify/src/api/gotify.rs b/proxmox-notify/src/api/gotify.rs
index d6f33064..5c4db4be 100644
--- a/proxmox-notify/src/api/gotify.rs
+++ b/proxmox-notify/src/api/gotify.rs
@@ -145,6 +145,7 @@ pub fn update_endpoint(
pub fn delete_gotify_endpoint(config: &mut Config, name: &str) -> Result<(), ApiError> {
// Check if the endpoint exists
let _ = get_endpoint(config, name)?;
+ super::ensure_unused(config, name)?;
remove_private_config_entry(config, name)?;
config.config.sections.remove(name);
diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
index d8a44bf2..81c182c7 100644
--- a/proxmox-notify/src/api/mod.rs
+++ b/proxmox-notify/src/api/mod.rs
@@ -102,6 +102,59 @@ fn endpoint_exists(config: &Config, name: &str) -> bool {
exists
}
+fn get_referrers(config: &Config, entity: &str) -> Result<HashSet<String>, ApiError> {
+ let mut referrers = HashSet::new();
+
+ for group in group::get_groups(config)? {
+ for endpoint in group.endpoint {
+ if endpoint == entity {
+ referrers.insert(group.name.clone());
+ }
+ }
+
+ if let Some(filter) = group.filter {
+ if filter == entity {
+ referrers.insert(group.name);
+ }
+ }
+ }
+
+ #[cfg(feature = "sendmail")]
+ for endpoint in sendmail::get_endpoints(config)? {
+ if let Some(filter) = endpoint.filter {
+ if filter == entity {
+ referrers.insert(endpoint.name);
+ }
+ }
+ }
+
+ #[cfg(feature = "gotify")]
+ for endpoint in gotify::get_endpoints(config)? {
+ if let Some(filter) = endpoint.filter {
+ if filter == entity {
+ referrers.insert(endpoint.name);
+ }
+ }
+ }
+
+ Ok(referrers)
+}
+
+fn ensure_unused(config: &Config, entity: &str) -> Result<(), ApiError> {
+ let referrers = get_referrers(config, entity)?;
+
+ if !referrers.is_empty() {
+ let used_by = referrers.into_iter().collect::<Vec<_>>().join(", ");
+
+ return Err(ApiError::bad_request(
+ format!("cannot delete '{entity}', referenced by: {used_by}"),
+ None,
+ ));
+ }
+
+ Ok(())
+}
+
fn get_referenced_entities(config: &Config, entity: &str) -> HashSet<String> {
let mut to_expand = HashSet::new();
let mut expanded = HashSet::new();
@@ -161,8 +214,7 @@ mod tests {
use crate::filter::FilterConfig;
use crate::group::GroupConfig;
- #[test]
- fn test_get_referenced_entities() {
+ fn prepare_config() -> Result<Config, ApiError> {
let mut config = super::test_helpers::empty_config();
filter::add_filter(
@@ -171,8 +223,7 @@ mod tests {
name: "filter".to_string(),
..Default::default()
},
- )
- .unwrap();
+ )?;
sendmail::add_endpoint(
&mut config,
@@ -182,8 +233,7 @@ mod tests {
filter: Some("filter".to_string()),
..Default::default()
},
- )
- .unwrap();
+ )?;
gotify::add_endpoint(
&mut config,
@@ -197,8 +247,7 @@ mod tests {
name: "gotify".to_string(),
token: "foo".to_string(),
},
- )
- .unwrap();
+ )?;
group::add_group(
&mut config,
@@ -208,8 +257,14 @@ mod tests {
filter: Some("filter".to_string()),
..Default::default()
},
- )
- .unwrap();
+ )?;
+
+ Ok(config)
+ }
+
+ #[test]
+ fn test_get_referenced_entities() {
+ let config = prepare_config().unwrap();
assert_eq!(
get_referenced_entities(&config, "filter"),
@@ -233,4 +288,42 @@ mod tests {
])
);
}
+
+ #[test]
+ fn test_get_referrers_for_entity() -> Result<(), ApiError> {
+ let config = prepare_config().unwrap();
+
+ assert_eq!(
+ get_referrers(&config, "filter")?,
+ HashSet::from([
+ "gotify".to_string(),
+ "sendmail".to_string(),
+ "group".to_string()
+ ])
+ );
+
+ assert_eq!(
+ get_referrers(&config, "sendmail")?,
+ HashSet::from(["group".to_string()])
+ );
+
+ assert_eq!(
+ get_referrers(&config, "gotify")?,
+ HashSet::from(["group".to_string()])
+ );
+
+ assert!(get_referrers(&config, "group")?.is_empty(),);
+
+ Ok(())
+ }
+
+ #[test]
+ fn test_ensure_unused() {
+ let config = prepare_config().unwrap();
+
+ assert!(ensure_unused(&config, "filter").is_err());
+ assert!(ensure_unused(&config, "gotify").is_err());
+ assert!(ensure_unused(&config, "sendmail").is_err());
+ assert!(ensure_unused(&config, "group").is_ok());
+ }
}
diff --git a/proxmox-notify/src/api/sendmail.rs b/proxmox-notify/src/api/sendmail.rs
index 070ed6e7..bf225f29 100644
--- a/proxmox-notify/src/api/sendmail.rs
+++ b/proxmox-notify/src/api/sendmail.rs
@@ -147,6 +147,7 @@ pub fn update_endpoint(
pub fn delete_endpoint(config: &mut Config, name: &str) -> Result<(), ApiError> {
// Check if the endpoint exists
let _ = get_endpoint(config, name)?;
+ super::ensure_unused(config, name)?;
config.config.sections.remove(name);
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 21/66] notify: ensure that filter/group/endpoint names are unique
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (19 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 20/66] notify: on deletion, check if a filter/endp. is still used by anything Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 22/66] notify: additional logging when sending a notification Lukas Wagner
` (47 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Otherwise, a filter with the same name as an already existing
endpoint or group can overwrite it.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/api/filter.rs | 7 +----
proxmox-notify/src/api/gotify.rs | 10 +------
proxmox-notify/src/api/group.rs | 24 ++-------------
proxmox-notify/src/api/mod.rs | 47 ++++++++++++++++++++++++++++--
proxmox-notify/src/api/sendmail.rs | 7 +----
5 files changed, 51 insertions(+), 44 deletions(-)
diff --git a/proxmox-notify/src/api/filter.rs b/proxmox-notify/src/api/filter.rs
index 824f802d..b5b62849 100644
--- a/proxmox-notify/src/api/filter.rs
+++ b/proxmox-notify/src/api/filter.rs
@@ -31,12 +31,7 @@ pub fn get_filter(config: &Config, name: &str) -> Result<FilterConfig, ApiError>
/// Returns an `ApiError` if a filter with the same name already exists or
/// if the filter could not be saved.
pub fn add_filter(config: &mut Config, filter_config: &FilterConfig) -> Result<(), ApiError> {
- if get_filter(config, &filter_config.name).is_ok() {
- return Err(ApiError::bad_request(
- format!("filter '{}' already exists", filter_config.name),
- None,
- ));
- }
+ super::ensure_unique(config, &filter_config.name)?;
config
.config
diff --git a/proxmox-notify/src/api/gotify.rs b/proxmox-notify/src/api/gotify.rs
index 5c4db4be..521dd167 100644
--- a/proxmox-notify/src/api/gotify.rs
+++ b/proxmox-notify/src/api/gotify.rs
@@ -43,15 +43,7 @@ pub fn add_endpoint(
panic!("name for endpoint config and private config must be identical");
}
- if super::endpoint_exists(config, &endpoint_config.name) {
- return Err(ApiError::bad_request(
- format!(
- "endpoint with name '{}' already exists!",
- endpoint_config.name
- ),
- None,
- ));
- }
+ super::ensure_unique(config, &endpoint_config.name)?;
if let Some(filter) = &endpoint_config.filter {
// Check if filter exists
diff --git a/proxmox-notify/src/api/group.rs b/proxmox-notify/src/api/group.rs
index fe3de12f..d6b8f38d 100644
--- a/proxmox-notify/src/api/group.rs
+++ b/proxmox-notify/src/api/group.rs
@@ -31,12 +31,7 @@ pub fn get_group(config: &Config, name: &str) -> Result<GroupConfig, ApiError> {
/// Returns an `ApiError` if a group with the same name already exists, or
/// if the group could not be saved
pub fn add_group(config: &mut Config, group_config: &GroupConfig) -> Result<(), ApiError> {
- if get_group(config, &group_config.name).is_ok() {
- return Err(ApiError::bad_request(
- format!("group '{}' already exists", group_config.name),
- None,
- ));
- }
+ super::ensure_unique(config, &group_config.name)?;
if group_config.endpoint.is_empty() {
return Err(ApiError::bad_request(
@@ -50,7 +45,7 @@ pub fn add_group(config: &mut Config, group_config: &GroupConfig) -> Result<(),
super::filter::get_filter(config, filter)?;
}
- check_if_endpoints_exist(config, &group_config.endpoint)?;
+ super::ensure_endpoints_exist(config, &group_config.endpoint)?;
config
.config
@@ -91,7 +86,7 @@ pub fn update_group(
}
if let Some(endpoints) = &updater.endpoint {
- check_if_endpoints_exist(config, endpoints)?;
+ super::ensure_endpoints_exist(config, endpoints)?;
if endpoints.is_empty() {
return Err(ApiError::bad_request(
"group must contain at least one endpoint",
@@ -138,19 +133,6 @@ pub fn delete_group(config: &mut Config, name: &str) -> Result<(), ApiError> {
Ok(())
}
-fn check_if_endpoints_exist(config: &Config, endpoints: &[String]) -> Result<(), ApiError> {
- for endpoint in endpoints {
- if !super::endpoint_exists(config, endpoint) {
- return Err(ApiError::not_found(
- format!("endoint '{endpoint}' does not exist"),
- None,
- ));
- }
- }
-
- Ok(())
-}
-
// groups cannot be empty, so only build the tests if we have the
// sendmail endpoint available
#[cfg(all(test, feature = "sendmail"))]
diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
index 81c182c7..994af4d3 100644
--- a/proxmox-notify/src/api/mod.rs
+++ b/proxmox-notify/src/api/mod.rs
@@ -87,7 +87,7 @@ fn verify_digest(config: &Config, digest: Option<&[u8]>) -> Result<(), ApiError>
Ok(())
}
-fn endpoint_exists(config: &Config, name: &str) -> bool {
+fn ensure_endpoint_exists(config: &Config, name: &str) -> Result<(), ApiError> {
let mut exists = false;
#[cfg(feature = "sendmail")]
@@ -99,7 +99,33 @@ fn endpoint_exists(config: &Config, name: &str) -> bool {
exists = exists || gotify::get_endpoint(config, name).is_ok();
}
- exists
+ if !exists {
+ Err(ApiError::not_found(
+ format!("endpoint '{name}' does not exist"),
+ None,
+ ))
+ } else {
+ Ok(())
+ }
+}
+
+fn ensure_endpoints_exist<T: AsRef<str>>(config: &Config, endpoints: &[T]) -> Result<(), ApiError> {
+ for endpoint in endpoints {
+ ensure_endpoint_exists(config, endpoint.as_ref())?;
+ }
+
+ Ok(())
+}
+
+fn ensure_unique(config: &Config, entity: &str) -> Result<(), ApiError> {
+ if config.config.sections.contains_key(entity) {
+ return Err(ApiError::bad_request(
+ format!("Cannot create '{entity}', an entity with the same name already exists"),
+ None,
+ ));
+ }
+
+ Ok(())
}
fn get_referrers(config: &Config, entity: &str) -> Result<HashSet<String>, ApiError> {
@@ -326,4 +352,21 @@ mod tests {
assert!(ensure_unused(&config, "sendmail").is_err());
assert!(ensure_unused(&config, "group").is_ok());
}
+
+ #[test]
+ fn test_ensure_unique() {
+ let config = prepare_config().unwrap();
+
+ assert!(ensure_unique(&config, "sendmail").is_err());
+ assert!(ensure_unique(&config, "group").is_err());
+ assert!(ensure_unique(&config, "new").is_ok());
+ }
+
+ #[test]
+ fn test_ensure_endpoints_exist() {
+ let config = prepare_config().unwrap();
+
+ assert!(ensure_endpoints_exist(&config, &vec!["sendmail", "gotify"]).is_ok());
+ assert!(ensure_endpoints_exist(&config, &vec!["group", "filter"]).is_err());
+ }
}
diff --git a/proxmox-notify/src/api/sendmail.rs b/proxmox-notify/src/api/sendmail.rs
index bf225f29..59a57b47 100644
--- a/proxmox-notify/src/api/sendmail.rs
+++ b/proxmox-notify/src/api/sendmail.rs
@@ -33,12 +33,7 @@ pub fn get_endpoint(config: &Config, name: &str) -> Result<SendmailConfig, ApiEr
/// Returns an `ApiError` if an endpoint with the same name already exists,
/// or if the endpoint could not be saved.
pub fn add_endpoint(config: &mut Config, endpoint: &SendmailConfig) -> Result<(), ApiError> {
- if super::endpoint_exists(config, &endpoint.name) {
- return Err(ApiError::bad_request(
- format!("endpoint with name '{}' already exists!", &endpoint.name),
- None,
- ));
- }
+ super::ensure_unique(config, &endpoint.name)?;
if let Some(filter) = &endpoint.filter {
// Check if filter exists
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 22/66] notify: additional logging when sending a notification
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (20 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 21/66] notify: ensure that filter/group/endpoint names are unique Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 23/66] notify: add debian packaging Lukas Wagner
` (46 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/src/lib.rs | 21 ++++++++++++++-------
1 file changed, 14 insertions(+), 7 deletions(-)
diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
index 3c2b6d55..d2f15496 100644
--- a/proxmox-notify/src/lib.rs
+++ b/proxmox-notify/src/lib.rs
@@ -317,9 +317,12 @@ impl Bus {
if let Some(group) = self.groups.get(endpoint_or_group) {
if !Bus::check_filter(&mut filter_matcher, group.filter.as_deref()) {
+ log::info!("skipped target '{endpoint_or_group}', filter did not match");
return;
}
+ log::info!("target '{endpoint_or_group}' is a group, notifying all members...");
+
for endpoint in &group.endpoint {
self.send_via_single_endpoint(endpoint, notification, &mut filter_matcher);
}
@@ -351,19 +354,23 @@ impl Bus {
filter_matcher: &mut FilterMatcher,
) {
if let Some(endpoint) = self.endpoints.get(endpoint) {
+ let name = endpoint.name();
if !Bus::check_filter(filter_matcher, endpoint.filter()) {
+ log::info!("skipped target '{name}', filter did not match");
return;
}
- if let Err(e) = endpoint.send(notification) {
- // Only log on errors, do not propagate fail to the caller.
- log::error!(
- "could not notify via target `{name}`: {e}",
- name = endpoint.name()
- );
+ match endpoint.send(notification) {
+ Ok(_) => {
+ log::info!("notified via target `{name}`");
+ }
+ Err(e) => {
+ // Only log on errors, do not propagate fail to the caller.
+ log::error!("could not notify via target `{name}`: {e}");
+ }
}
} else {
- log::error!("could not notify via endpoint '{endpoint}', it does not exist");
+ log::error!("could not notify via target '{endpoint}', it does not exist");
}
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox 23/66] notify: add debian packaging
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (21 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 22/66] notify: additional logging when sending a notification Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-18 13:25 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 24/66] add PVE::RS::Notify module Lukas Wagner
` (45 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
proxmox-notify/debian/changelog | 5 ++
proxmox-notify/debian/control | 108 ++++++++++++++++++++++++++++
proxmox-notify/debian/copyright | 16 +++++
proxmox-notify/debian/debcargo.toml | 7 ++
4 files changed, 136 insertions(+)
create mode 100644 proxmox-notify/debian/changelog
create mode 100644 proxmox-notify/debian/control
create mode 100644 proxmox-notify/debian/copyright
create mode 100644 proxmox-notify/debian/debcargo.toml
diff --git a/proxmox-notify/debian/changelog b/proxmox-notify/debian/changelog
new file mode 100644
index 00000000..9d0a4d03
--- /dev/null
+++ b/proxmox-notify/debian/changelog
@@ -0,0 +1,5 @@
+rust-proxmox-notify (0.1.0-1) stable; urgency=medium
+
+ * Initial release.
+
+ -- Proxmox Support Team <support@proxmox.com> Thu, 12 Jan 2023 11:42:11 +0200
diff --git a/proxmox-notify/debian/control b/proxmox-notify/debian/control
new file mode 100644
index 00000000..20465c07
--- /dev/null
+++ b/proxmox-notify/debian/control
@@ -0,0 +1,108 @@
+Source: rust-proxmox-notify
+Section: rust
+Priority: optional
+Build-Depends: debhelper (>= 12),
+ dh-cargo (>= 25),
+ cargo:native <!nocheck>,
+ rustc:native <!nocheck>,
+ libstd-rust-dev <!nocheck>,
+ librust-handlebars-3+default-dev <!nocheck>,
+ librust-lazy-static-1+default-dev (>= 1.4-~~) <!nocheck>,
+ librust-log-0.4+default-dev (>= 0.4.17-~~) <!nocheck>,
+ librust-openssl-0.10+default-dev <!nocheck>,
+ librust-proxmox-http-0.9+client-sync-dev <!nocheck>,
+ librust-proxmox-http-0.9+default-dev <!nocheck>,
+ librust-proxmox-human-byte-0.1+default-dev <!nocheck>,
+ librust-proxmox-schema-1+api-macro-dev (>= 1.3.7-~~) <!nocheck>,
+ librust-proxmox-schema-1+default-dev (>= 1.3.7-~~) <!nocheck>,
+ librust-proxmox-section-config-1+default-dev (>= 1.0.2-~~) <!nocheck>,
+ librust-proxmox-sys-0.5+default-dev <!nocheck>,
+ librust-proxmox-time-1+default-dev (>= 1.1.4-~~) <!nocheck>,
+ librust-regex-1+default-dev (>= 1.5-~~) <!nocheck>,
+ librust-serde-1+default-dev <!nocheck>,
+ librust-serde-1+derive-dev <!nocheck>,
+ librust-serde-json-1+default-dev <!nocheck>
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Standards-Version: 4.6.1
+Vcs-Git: git://git.proxmox.com/git/proxmox.git
+Vcs-Browser: https://git.proxmox.com/?p=proxmox.git
+X-Cargo-Crate: proxmox-notify
+Rules-Requires-Root: no
+
+Package: librust-proxmox-notify-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-handlebars-3+default-dev,
+ librust-lazy-static-1+default-dev (>= 1.4-~~),
+ librust-log-0.4+default-dev (>= 0.4.17-~~),
+ librust-openssl-0.10+default-dev,
+ librust-proxmox-human-byte-0.1+default-dev,
+ librust-proxmox-schema-1+api-macro-dev (>= 1.3.7-~~),
+ librust-proxmox-schema-1+default-dev (>= 1.3.7-~~),
+ librust-proxmox-section-config-1+default-dev (>= 1.0.2-~~),
+ librust-proxmox-time-1+default-dev (>= 1.1.4-~~),
+ librust-regex-1+default-dev (>= 1.5-~~),
+ librust-serde-1+default-dev,
+ librust-serde-1+derive-dev,
+ librust-serde-json-1+default-dev
+Recommends:
+ librust-proxmox-notify+default-dev (= ${binary:Version})
+Suggests:
+ librust-proxmox-notify+gotify-dev (= ${binary:Version}),
+ librust-proxmox-notify+sendmail-dev (= ${binary:Version})
+Provides:
+ librust-proxmox-notify-0-dev (= ${binary:Version}),
+ librust-proxmox-notify-0.1-dev (= ${binary:Version}),
+ librust-proxmox-notify-0.1.0-dev (= ${binary:Version})
+Description: Rust crate "proxmox-notify" - Rust source code
+ This package contains the source for the Rust proxmox-notify crate, packaged by
+ debcargo for use with cargo and dh-cargo.
+
+Package: librust-proxmox-notify+default-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-proxmox-notify-dev (= ${binary:Version}),
+ librust-proxmox-notify+sendmail-dev (= ${binary:Version}),
+ librust-proxmox-notify+gotify-dev (= ${binary:Version})
+Provides:
+ librust-proxmox-notify-0+default-dev (= ${binary:Version}),
+ librust-proxmox-notify-0.1+default-dev (= ${binary:Version}),
+ librust-proxmox-notify-0.1.0+default-dev (= ${binary:Version})
+Description: Rust crate "proxmox-notify" - feature "default"
+ This metapackage enables feature "default" for the Rust proxmox-notify crate,
+ by pulling in any additional dependencies needed by that feature.
+
+Package: librust-proxmox-notify+gotify-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-proxmox-notify-dev (= ${binary:Version}),
+ librust-proxmox-http-0.9+client-sync-dev,
+ librust-proxmox-http-0.9+default-dev
+Provides:
+ librust-proxmox-notify-0+gotify-dev (= ${binary:Version}),
+ librust-proxmox-notify-0.1+gotify-dev (= ${binary:Version}),
+ librust-proxmox-notify-0.1.0+gotify-dev (= ${binary:Version})
+Description: Rust crate "proxmox-notify" - feature "gotify"
+ This metapackage enables feature "gotify" for the Rust proxmox-notify crate, by
+ pulling in any additional dependencies needed by that feature.
+
+Package: librust-proxmox-notify+sendmail-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-proxmox-notify-dev (= ${binary:Version}),
+ librust-proxmox-sys-0.5+default-dev
+Provides:
+ librust-proxmox-notify-0+sendmail-dev (= ${binary:Version}),
+ librust-proxmox-notify-0.1+sendmail-dev (= ${binary:Version}),
+ librust-proxmox-notify-0.1.0+sendmail-dev (= ${binary:Version})
+Description: Rust crate "proxmox-notify" - feature "sendmail"
+ This metapackage enables feature "sendmail" for the Rust proxmox-notify crate,
+ by pulling in any additional dependencies needed by that feature.
diff --git a/proxmox-notify/debian/copyright b/proxmox-notify/debian/copyright
new file mode 100644
index 00000000..4fce23a5
--- /dev/null
+++ b/proxmox-notify/debian/copyright
@@ -0,0 +1,16 @@
+Copyright (C) 2023 Proxmox Server Solutions GmbH
+
+This software is written by Proxmox Server Solutions GmbH <support@proxmox.com>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU Affero General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU Affero General Public License for more details.
+
+You should have received a copy of the GNU Affero General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
diff --git a/proxmox-notify/debian/debcargo.toml b/proxmox-notify/debian/debcargo.toml
new file mode 100644
index 00000000..b7864cdb
--- /dev/null
+++ b/proxmox-notify/debian/debcargo.toml
@@ -0,0 +1,7 @@
+overlay = "."
+crate_src_path = ".."
+maintainer = "Proxmox Support Team <support@proxmox.com>"
+
+[source]
+vcs_git = "git://git.proxmox.com/git/proxmox.git"
+vcs_browser = "https://git.proxmox.com/?p=proxmox.git"
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-perl-rs 24/66] add PVE::RS::Notify module
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (22 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 23/66] notify: add debian packaging Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-19 10:10 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 25/66] notify: add api for sending notifications/testing endpoints Lukas Wagner
` (44 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
pve-rs/Cargo.toml | 1 +
pve-rs/Makefile | 1 +
pve-rs/src/lib.rs | 1 +
pve-rs/src/notify.rs | 71 ++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 74 insertions(+)
create mode 100644 pve-rs/src/notify.rs
diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
index 3076a13..b28c118 100644
--- a/pve-rs/Cargo.toml
+++ b/pve-rs/Cargo.toml
@@ -34,6 +34,7 @@ perlmod = { version = "0.13", features = [ "exporter" ] }
proxmox-apt = "0.10"
proxmox-http = { version = "0.9", features = ["client-sync", "client-trait"] }
+proxmox-notify = "0.1"
proxmox-openid = "0.10"
proxmox-resource-scheduling = "0.3.0"
proxmox-subscription = "0.4"
diff --git a/pve-rs/Makefile b/pve-rs/Makefile
index de35c69..9d737c0 100644
--- a/pve-rs/Makefile
+++ b/pve-rs/Makefile
@@ -27,6 +27,7 @@ PERLMOD_GENPACKAGE := /usr/lib/perlmod/genpackage.pl \
PERLMOD_PACKAGES := \
PVE::RS::APT::Repositories \
+ PVE::RS::Notify \
PVE::RS::OpenId \
PVE::RS::ResourceScheduling::Static \
PVE::RS::TFA
diff --git a/pve-rs/src/lib.rs b/pve-rs/src/lib.rs
index eb6ae02..0d63c28 100644
--- a/pve-rs/src/lib.rs
+++ b/pve-rs/src/lib.rs
@@ -4,6 +4,7 @@
pub mod common;
pub mod apt;
+pub mod notify;
pub mod openid;
pub mod resource_scheduling;
pub mod tfa;
diff --git a/pve-rs/src/notify.rs b/pve-rs/src/notify.rs
new file mode 100644
index 0000000..9677d8b
--- /dev/null
+++ b/pve-rs/src/notify.rs
@@ -0,0 +1,71 @@
+#[perlmod::package(name = "PVE::RS::Notify")]
+mod export {
+ use anyhow::{bail, Error};
+ use perlmod::Value;
+
+ use std::sync::Mutex;
+
+ use proxmox_notify::Config;
+
+ pub struct NotificationConfig {
+ config: Mutex<Config>,
+ }
+
+ perlmod::declare_magic!(Box<NotificationConfig> : &NotificationConfig as "PVE::RS::Notify");
+
+ /// Support `dclone` so this can be put into the `ccache` of `PVE::Cluster`.
+ #[export(name = "STORABLE_freeze", raw_return)]
+ fn storable_freeze(
+ #[try_from_ref] this: &NotificationConfig,
+ cloning: bool,
+ ) -> Result<Value, Error> {
+ if !cloning {
+ bail!("freezing Notification config not supported!");
+ }
+
+ let mut cloned = Box::new(NotificationConfig {
+ config: Mutex::new(this.config.lock().unwrap().clone()),
+ });
+ let value = Value::new_pointer::<NotificationConfig>(&mut *cloned);
+ let _perl = Box::leak(cloned);
+ Ok(value)
+ }
+
+ /// Instead of `thaw` we implement `attach` for `dclone`.
+ #[export(name = "STORABLE_attach", raw_return)]
+ fn storable_attach(
+ #[raw] class: Value,
+ cloning: bool,
+ #[raw] serialized: Value,
+ ) -> Result<Value, Error> {
+ if !cloning {
+ bail!("STORABLE_attach called with cloning=false");
+ }
+ let data = unsafe { Box::from_raw(serialized.pv_raw::<NotificationConfig>()?) };
+ Ok(perlmod::instantiate_magic!(&class, MAGIC => data))
+ }
+
+ #[export(raw_return)]
+ fn parse_config(
+ #[raw] class: Value,
+ raw_config: &str,
+ raw_private_config: &str,
+ ) -> Result<Value, Error> {
+ Ok(perlmod::instantiate_magic!(&class, MAGIC => Box::new(
+ NotificationConfig {
+ config: Mutex::new(Config::new(raw_config, raw_private_config)?)
+ }
+ )))
+ }
+
+ #[export]
+ fn write_config(#[try_from_ref] this: &NotificationConfig) -> Result<(String, String), Error> {
+ Ok(this.config.lock().unwrap().write()?)
+ }
+
+ #[export]
+ fn digest(#[try_from_ref] this: &NotificationConfig) -> String {
+ let config = this.config.lock().unwrap();
+ hex::encode(config.digest())
+ }
+}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-perl-rs 25/66] notify: add api for sending notifications/testing endpoints
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (23 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 24/66] add PVE::RS::Notify module Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 26/66] notify: add api for notification groups Lukas Wagner
` (43 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
pve-rs/src/notify.rs | 34 ++++++++++++++++++++++++++++++++--
1 file changed, 32 insertions(+), 2 deletions(-)
diff --git a/pve-rs/src/notify.rs b/pve-rs/src/notify.rs
index 9677d8b..74a872b 100644
--- a/pve-rs/src/notify.rs
+++ b/pve-rs/src/notify.rs
@@ -2,10 +2,10 @@
mod export {
use anyhow::{bail, Error};
use perlmod::Value;
-
+ use serde_json::Value as JSONValue;
use std::sync::Mutex;
- use proxmox_notify::Config;
+ use proxmox_notify::{api, api::ApiError, Config, Notification, Severity};
pub struct NotificationConfig {
config: Mutex<Config>,
@@ -68,4 +68,34 @@ mod export {
let config = this.config.lock().unwrap();
hex::encode(config.digest())
}
+
+ #[export(serialize_error)]
+ fn send(
+ #[try_from_ref] this: &NotificationConfig,
+ channel: &str,
+ severity: Severity,
+ title: String,
+ body: String,
+ properties: Option<JSONValue>,
+ ) -> Result<(), ApiError> {
+ let config = this.config.lock().unwrap();
+
+ let notification = Notification {
+ severity,
+ title,
+ body,
+ properties,
+ };
+
+ api::common::send(&config, channel, ¬ification)
+ }
+
+ #[export(serialize_error)]
+ fn test_target(
+ #[try_from_ref] this: &NotificationConfig,
+ target: &str,
+ ) -> Result<(), ApiError> {
+ let config = this.config.lock().unwrap();
+ api::common::test_target(&config, target)
+ }
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-perl-rs 26/66] notify: add api for notification groups
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (24 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 25/66] notify: add api for sending notifications/testing endpoints Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 27/66] notify: add api for sendmail endpoints Lukas Wagner
` (42 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
pve-rs/src/notify.rs | 70 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 70 insertions(+)
diff --git a/pve-rs/src/notify.rs b/pve-rs/src/notify.rs
index 74a872b..cac233a 100644
--- a/pve-rs/src/notify.rs
+++ b/pve-rs/src/notify.rs
@@ -5,6 +5,7 @@ mod export {
use serde_json::Value as JSONValue;
use std::sync::Mutex;
+ use proxmox_notify::group::{DeleteableGroupProperty, GroupConfig, GroupConfigUpdater};
use proxmox_notify::{api, api::ApiError, Config, Notification, Severity};
pub struct NotificationConfig {
@@ -98,4 +99,73 @@ mod export {
let config = this.config.lock().unwrap();
api::common::test_target(&config, target)
}
+
+ #[export(serialize_error)]
+ fn get_groups(#[try_from_ref] this: &NotificationConfig) -> Result<Vec<GroupConfig>, ApiError> {
+ let config = this.config.lock().unwrap();
+ api::group::get_groups(&config)
+ }
+
+ #[export(serialize_error)]
+ fn get_group(
+ #[try_from_ref] this: &NotificationConfig,
+ id: &str,
+ ) -> Result<GroupConfig, ApiError> {
+ let config = this.config.lock().unwrap();
+ api::group::get_group(&config, id)
+ }
+
+ #[export(serialize_error)]
+ fn add_group(
+ #[try_from_ref] this: &NotificationConfig,
+ name: String,
+ endpoints: Vec<String>,
+ comment: Option<String>,
+ filter: Option<String>,
+ ) -> Result<(), ApiError> {
+ let mut config = this.config.lock().unwrap();
+ api::group::add_group(
+ &mut config,
+ &GroupConfig {
+ name,
+ endpoint: endpoints,
+ comment,
+ filter,
+ },
+ )
+ }
+
+ #[export(serialize_error)]
+ fn update_group(
+ #[try_from_ref] this: &NotificationConfig,
+ name: &str,
+ endpoints: Option<Vec<String>>,
+ comment: Option<String>,
+ filter: Option<String>,
+ delete: Option<Vec<DeleteableGroupProperty>>,
+ digest: Option<&str>,
+ ) -> Result<(), ApiError> {
+ let mut config = this.config.lock().unwrap();
+ let digest = digest.map(hex::decode).transpose().map_err(|e| {
+ ApiError::internal_server_error(format!("invalid digest: {e}"), Some(Box::new(e)))
+ })?;
+
+ api::group::update_group(
+ &mut config,
+ name,
+ &GroupConfigUpdater {
+ endpoint: endpoints,
+ comment,
+ filter,
+ },
+ delete.as_deref(),
+ digest.as_deref(),
+ )
+ }
+
+ #[export(serialize_error)]
+ fn delete_group(#[try_from_ref] this: &NotificationConfig, name: &str) -> Result<(), ApiError> {
+ let mut config = this.config.lock().unwrap();
+ api::group::delete_group(&mut config, name)
+ }
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-perl-rs 27/66] notify: add api for sendmail endpoints
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (25 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 26/66] notify: add api for notification groups Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 28/66] notify: add api for gotify endpoints Lukas Wagner
` (41 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
pve-rs/src/notify.rs | 88 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 88 insertions(+)
diff --git a/pve-rs/src/notify.rs b/pve-rs/src/notify.rs
index cac233a..9490ea8 100644
--- a/pve-rs/src/notify.rs
+++ b/pve-rs/src/notify.rs
@@ -5,6 +5,9 @@ mod export {
use serde_json::Value as JSONValue;
use std::sync::Mutex;
+ use proxmox_notify::endpoints::sendmail::{
+ DeleteableSendmailProperty, SendmailConfig, SendmailConfigUpdater,
+ };
use proxmox_notify::group::{DeleteableGroupProperty, GroupConfig, GroupConfigUpdater};
use proxmox_notify::{api, api::ApiError, Config, Notification, Severity};
@@ -168,4 +171,89 @@ mod export {
let mut config = this.config.lock().unwrap();
api::group::delete_group(&mut config, name)
}
+
+ #[export(serialize_error)]
+ fn get_sendmail_endpoints(
+ #[try_from_ref] this: &NotificationConfig,
+ ) -> Result<Vec<SendmailConfig>, ApiError> {
+ let config = this.config.lock().unwrap();
+ api::sendmail::get_endpoints(&config)
+ }
+
+ #[export(serialize_error)]
+ fn get_sendmail_endpoint(
+ #[try_from_ref] this: &NotificationConfig,
+ id: &str,
+ ) -> Result<SendmailConfig, ApiError> {
+ let config = this.config.lock().unwrap();
+ api::sendmail::get_endpoint(&config, id)
+ }
+
+ #[export(serialize_error)]
+ #[allow(clippy::too_many_arguments)]
+ fn add_sendmail_endpoint(
+ #[try_from_ref] this: &NotificationConfig,
+ name: String,
+ mailto: Vec<String>,
+ from_address: Option<String>,
+ author: Option<String>,
+ comment: Option<String>,
+ filter: Option<String>,
+ ) -> Result<(), ApiError> {
+ let mut config = this.config.lock().unwrap();
+
+ api::sendmail::add_endpoint(
+ &mut config,
+ &SendmailConfig {
+ name,
+ mailto,
+ from_address,
+ author,
+ comment,
+ filter,
+ },
+ )
+ }
+
+ #[export(serialize_error)]
+ #[allow(clippy::too_many_arguments)]
+ fn update_sendmail_endpoint(
+ #[try_from_ref] this: &NotificationConfig,
+ name: &str,
+ mailto: Option<Vec<String>>,
+ from_address: Option<String>,
+ author: Option<String>,
+ comment: Option<String>,
+ filter: Option<String>,
+ delete: Option<Vec<DeleteableSendmailProperty>>,
+ digest: Option<&str>,
+ ) -> Result<(), ApiError> {
+ let mut config = this.config.lock().unwrap();
+ let digest = digest.map(hex::decode).transpose().map_err(|e| {
+ ApiError::internal_server_error(format!("invalid digest: {e}"), Some(Box::new(e)))
+ })?;
+
+ api::sendmail::update_endpoint(
+ &mut config,
+ name,
+ &SendmailConfigUpdater {
+ mailto,
+ from_address,
+ author,
+ comment,
+ filter,
+ },
+ delete.as_deref(),
+ digest.as_deref(),
+ )
+ }
+
+ #[export(serialize_error)]
+ fn delete_sendmail_endpoint(
+ #[try_from_ref] this: &NotificationConfig,
+ name: &str,
+ ) -> Result<(), ApiError> {
+ let mut config = this.config.lock().unwrap();
+ api::sendmail::delete_endpoint(&mut config, name)
+ }
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-perl-rs 28/66] notify: add api for gotify endpoints
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (26 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 27/66] notify: add api for sendmail endpoints Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 29/66] notify: add api for notification filters Lukas Wagner
` (40 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
pve-rs/src/notify.rs | 83 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 83 insertions(+)
diff --git a/pve-rs/src/notify.rs b/pve-rs/src/notify.rs
index 9490ea8..aa2c312 100644
--- a/pve-rs/src/notify.rs
+++ b/pve-rs/src/notify.rs
@@ -5,6 +5,10 @@ mod export {
use serde_json::Value as JSONValue;
use std::sync::Mutex;
+ use proxmox_notify::endpoints::gotify::{
+ DeleteableGotifyProperty, GotifyConfig, GotifyConfigUpdater, GotifyPrivateConfig,
+ GotifyPrivateConfigUpdater,
+ };
use proxmox_notify::endpoints::sendmail::{
DeleteableSendmailProperty, SendmailConfig, SendmailConfigUpdater,
};
@@ -256,4 +260,83 @@ mod export {
let mut config = this.config.lock().unwrap();
api::sendmail::delete_endpoint(&mut config, name)
}
+
+ #[export(serialize_error)]
+ fn get_gotify_endpoints(
+ #[try_from_ref] this: &NotificationConfig,
+ ) -> Result<Vec<GotifyConfig>, ApiError> {
+ let config = this.config.lock().unwrap();
+ api::gotify::get_endpoints(&config)
+ }
+
+ #[export(serialize_error)]
+ fn get_gotify_endpoint(
+ #[try_from_ref] this: &NotificationConfig,
+ id: &str,
+ ) -> Result<GotifyConfig, ApiError> {
+ let config = this.config.lock().unwrap();
+ api::gotify::get_endpoint(&config, id)
+ }
+
+ #[export(serialize_error)]
+ fn add_gotify_endpoint(
+ #[try_from_ref] this: &NotificationConfig,
+ name: String,
+ server: String,
+ token: String,
+ comment: Option<String>,
+ filter: Option<String>,
+ ) -> Result<(), ApiError> {
+ let mut config = this.config.lock().unwrap();
+ api::gotify::add_endpoint(
+ &mut config,
+ &GotifyConfig {
+ name: name.clone(),
+ server,
+ comment,
+ filter,
+ },
+ &GotifyPrivateConfig { name, token },
+ )
+ }
+
+ #[export(serialize_error)]
+ #[allow(clippy::too_many_arguments)]
+ fn update_gotify_endpoint(
+ #[try_from_ref] this: &NotificationConfig,
+ name: &str,
+ server: Option<String>,
+ token: Option<String>,
+ comment: Option<String>,
+ filter: Option<String>,
+ delete: Option<Vec<DeleteableGotifyProperty>>,
+ digest: Option<&str>,
+ ) -> Result<(), ApiError> {
+ let mut config = this.config.lock().unwrap();
+ let digest = digest.map(hex::decode).transpose().map_err(|e| {
+ ApiError::internal_server_error(format!("invalid digest: {e}"), Some(Box::new(e)))
+ })?;
+
+ api::gotify::update_endpoint(
+ &mut config,
+ name,
+ &GotifyConfigUpdater {
+ server,
+ comment,
+ filter,
+ },
+ &GotifyPrivateConfigUpdater { token },
+ delete.as_deref(),
+ digest.as_deref(),
+ )
+ }
+
+ #[export(serialize_error)]
+ fn delete_gotify_endpoint(
+ #[try_from_ref] this: &NotificationConfig,
+ name: &str,
+ ) -> Result<(), ApiError> {
+ let mut config = this.config.lock().unwrap();
+ api::gotify::delete_gotify_endpoint(&mut config, name)
+ }
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-perl-rs 29/66] notify: add api for notification filters
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (27 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 28/66] notify: add api for gotify endpoints Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 30/66] notify: sendmail: support the `mailto-user` parameter Lukas Wagner
` (39 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
pve-rs/src/notify.rs | 83 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 83 insertions(+)
diff --git a/pve-rs/src/notify.rs b/pve-rs/src/notify.rs
index aa2c312..a6143fc 100644
--- a/pve-rs/src/notify.rs
+++ b/pve-rs/src/notify.rs
@@ -12,6 +12,9 @@ mod export {
use proxmox_notify::endpoints::sendmail::{
DeleteableSendmailProperty, SendmailConfig, SendmailConfigUpdater,
};
+ use proxmox_notify::filter::{
+ DeleteableFilterProperty, FilterConfig, FilterConfigUpdater, FilterModeOperator,
+ };
use proxmox_notify::group::{DeleteableGroupProperty, GroupConfig, GroupConfigUpdater};
use proxmox_notify::{api, api::ApiError, Config, Notification, Severity};
@@ -339,4 +342,84 @@ mod export {
let mut config = this.config.lock().unwrap();
api::gotify::delete_gotify_endpoint(&mut config, name)
}
+
+ #[export(serialize_error)]
+ fn get_filters(
+ #[try_from_ref] this: &NotificationConfig,
+ ) -> Result<Vec<FilterConfig>, ApiError> {
+ let config = this.config.lock().unwrap();
+ api::filter::get_filters(&config)
+ }
+
+ #[export(serialize_error)]
+ fn get_filter(
+ #[try_from_ref] this: &NotificationConfig,
+ id: &str,
+ ) -> Result<FilterConfig, ApiError> {
+ let config = this.config.lock().unwrap();
+ api::filter::get_filter(&config, id)
+ }
+
+ #[export(serialize_error)]
+ #[allow(clippy::too_many_arguments)]
+ fn add_filter(
+ #[try_from_ref] this: &NotificationConfig,
+ name: String,
+ min_severity: Option<Severity>,
+ mode: Option<FilterModeOperator>,
+ invert_match: Option<bool>,
+ comment: Option<String>,
+ ) -> Result<(), ApiError> {
+ let mut config = this.config.lock().unwrap();
+ api::filter::add_filter(
+ &mut config,
+ &FilterConfig {
+ name,
+ min_severity,
+ mode,
+ invert_match,
+ comment,
+ },
+ )
+ }
+
+ #[export(serialize_error)]
+ #[allow(clippy::too_many_arguments)]
+ fn update_filter(
+ #[try_from_ref] this: &NotificationConfig,
+ name: &str,
+ min_severity: Option<Severity>,
+ mode: Option<FilterModeOperator>,
+ invert_match: Option<bool>,
+ comment: Option<String>,
+ delete: Option<Vec<DeleteableFilterProperty>>,
+ digest: Option<&str>,
+ ) -> Result<(), ApiError> {
+ let mut config = this.config.lock().unwrap();
+ let digest = digest.map(hex::decode).transpose().map_err(|e| {
+ ApiError::internal_server_error(format!("invalid digest: {e}"), Some(Box::new(e)))
+ })?;
+
+ api::filter::update_filter(
+ &mut config,
+ name,
+ &FilterConfigUpdater {
+ min_severity,
+ mode,
+ invert_match,
+ comment,
+ },
+ delete.as_deref(),
+ digest.as_deref(),
+ )
+ }
+
+ #[export(serialize_error)]
+ fn delete_filter(
+ #[try_from_ref] this: &NotificationConfig,
+ name: &str,
+ ) -> Result<(), ApiError> {
+ let mut config = this.config.lock().unwrap();
+ api::filter::delete_filter(&mut config, name)
+ }
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-perl-rs 30/66] notify: sendmail: support the `mailto-user` parameter
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (28 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 29/66] notify: add api for notification filters Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 31/66] notify: implement context for getting default author/mailfrom Lukas Wagner
` (38 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
This parameter allows to send mails to the email address configured
for users from the product's user database.
`proxmox-notify` now has a `Context` that must be set via
`proxmox_notify::context::set_context` before the crate is used.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
pve-rs/Cargo.toml | 1 +
pve-rs/src/lib.rs | 3 +-
pve-rs/src/notify.rs | 81 +++++++++++++++++++++++++++++++++++++++++++-
3 files changed, 83 insertions(+), 2 deletions(-)
diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
index b28c118..954665f 100644
--- a/pve-rs/Cargo.toml
+++ b/pve-rs/Cargo.toml
@@ -23,6 +23,7 @@ env_logger = "0.9"
hex = "0.4"
http = "0.2.7"
libc = "0.2"
+log = "0.4.17"
nix = "0.26"
openssl = "0.10.40"
serde = "1.0"
diff --git a/pve-rs/src/lib.rs b/pve-rs/src/lib.rs
index 0d63c28..49483d7 100644
--- a/pve-rs/src/lib.rs
+++ b/pve-rs/src/lib.rs
@@ -11,10 +11,11 @@ pub mod tfa;
#[perlmod::package(name = "Proxmox::Lib::PVE", lib = "pve_rs")]
mod export {
- use crate::common;
+ use crate::{common, notify};
#[export]
pub fn init() {
common::logger::init("PVE_LOG", "info");
+ notify::init();
}
}
diff --git a/pve-rs/src/notify.rs b/pve-rs/src/notify.rs
index a6143fc..ea34bfe 100644
--- a/pve-rs/src/notify.rs
+++ b/pve-rs/src/notify.rs
@@ -1,3 +1,78 @@
+use std::path::Path;
+
+use log;
+
+use proxmox_notify::context::Context;
+
+// Some helpers borrowed and slightly adapted from `proxmox-mail-forward`
+
+fn normalize_for_return(s: Option<&str>) -> Option<String> {
+ match s?.trim() {
+ "" => None,
+ s => Some(s.to_string()),
+ }
+}
+
+fn attempt_file_read<P: AsRef<Path>>(path: P) -> Option<String> {
+ match proxmox_sys::fs::file_read_optional_string(path) {
+ Ok(contents) => contents,
+ Err(err) => {
+ log::error!("{err}");
+ None
+ }
+ }
+}
+
+fn lookup_mail_address(content: &str, user: &str) -> Option<String> {
+ normalize_for_return(content.lines().find_map(|line| {
+ let fields: Vec<&str> = line.split(':').collect();
+ #[allow(clippy::get_first)] // to keep expression style consistent
+ match fields.get(0)?.trim() == "user" && fields.get(1)?.trim() == user {
+ true => fields.get(6).copied(),
+ false => None,
+ }
+ }))
+}
+
+struct PVEContext;
+
+impl Context for PVEContext {
+ fn lookup_email_for_user(&self, user: &str) -> Option<String> {
+ let content = attempt_file_read("/etc/pve/user.cfg");
+ content.and_then(|content| lookup_mail_address(&content, user))
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use crate::notify::lookup_mail_address;
+
+ const USER_CONFIG: &str = "
+user:root@pam:1:0:::root@example.com:::
+user:test@pve:1:0:::test@example.com:::
+user:no-mail@pve:1:0::::::
+ ";
+
+ #[test]
+ fn test_parse_mail() {
+ assert_eq!(
+ lookup_mail_address(USER_CONFIG, "root@pam"),
+ Some("root@example.com".to_string())
+ );
+ assert_eq!(
+ lookup_mail_address(USER_CONFIG, "test@pve"),
+ Some("test@example.com".to_string())
+ );
+ assert_eq!(lookup_mail_address(USER_CONFIG, "no-mail@pve"), None);
+ }
+}
+
+static CONTEXT: PVEContext = PVEContext;
+
+pub fn init() {
+ proxmox_notify::context::set_context(&CONTEXT)
+}
+
#[perlmod::package(name = "PVE::RS::Notify")]
mod export {
use anyhow::{bail, Error};
@@ -201,7 +276,8 @@ mod export {
fn add_sendmail_endpoint(
#[try_from_ref] this: &NotificationConfig,
name: String,
- mailto: Vec<String>,
+ mailto: Option<Vec<String>>,
+ mailto_user: Option<Vec<String>>,
from_address: Option<String>,
author: Option<String>,
comment: Option<String>,
@@ -214,6 +290,7 @@ mod export {
&SendmailConfig {
name,
mailto,
+ mailto_user,
from_address,
author,
comment,
@@ -228,6 +305,7 @@ mod export {
#[try_from_ref] this: &NotificationConfig,
name: &str,
mailto: Option<Vec<String>>,
+ mailto_user: Option<Vec<String>>,
from_address: Option<String>,
author: Option<String>,
comment: Option<String>,
@@ -245,6 +323,7 @@ mod export {
name,
&SendmailConfigUpdater {
mailto,
+ mailto_user,
from_address,
author,
comment,
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-perl-rs 31/66] notify: implement context for getting default author/mailfrom
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (29 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 30/66] notify: sendmail: support the `mailto-user` parameter Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-19 11:15 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 32/66] notify: add context for getting http_proxy from datacenter.cfg Lukas Wagner
` (37 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
pve-rs/src/notify.rs | 33 ++++++++++++++++++++++++++++++++-
1 file changed, 32 insertions(+), 1 deletion(-)
diff --git a/pve-rs/src/notify.rs b/pve-rs/src/notify.rs
index ea34bfe..04e902c 100644
--- a/pve-rs/src/notify.rs
+++ b/pve-rs/src/notify.rs
@@ -34,6 +34,14 @@ fn lookup_mail_address(content: &str, user: &str) -> Option<String> {
}))
}
+fn lookup_datacenter_config_key(content: &str, key: &str) -> Option<String> {
+ normalize_for_return(
+ content
+ .lines()
+ .find_map(|line| line.strip_prefix(&format!("{key}:"))),
+ )
+}
+
struct PVEContext;
impl Context for PVEContext {
@@ -41,11 +49,22 @@ impl Context for PVEContext {
let content = attempt_file_read("/etc/pve/user.cfg");
content.and_then(|content| lookup_mail_address(&content, user))
}
+
+ fn default_sendmail_author(&self) -> String {
+ "Proxmox VE".into()
+ }
+
+ fn default_sendmail_from(&self) -> String {
+ let content = attempt_file_read("/etc/pve/datacenter.cfg");
+ content
+ .and_then(|content| lookup_datacenter_config_key(&content, "mail_from"))
+ .unwrap_or_else(|| String::from("root"))
+ }
}
#[cfg(test)]
mod tests {
- use crate::notify::lookup_mail_address;
+ use crate::notify::{lookup_datacenter_config_key, lookup_mail_address};
const USER_CONFIG: &str = "
user:root@pam:1:0:::root@example.com:::
@@ -65,6 +84,18 @@ user:no-mail@pve:1:0::::::
);
assert_eq!(lookup_mail_address(USER_CONFIG, "no-mail@pve"), None);
}
+
+ const DC_CONFIG: &str = "
+email_from: user@example.com
+keyboard: en-us
+";
+ #[test]
+ fn test_parse_dc_config() {
+ assert_eq!(
+ lookup_datacenter_config_key(DC_CONFIG, "email_from"),
+ Some("user@example.com".to_string())
+ );
+ }
}
static CONTEXT: PVEContext = PVEContext;
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-perl-rs 32/66] notify: add context for getting http_proxy from datacenter.cfg
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (30 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 31/66] notify: implement context for getting default author/mailfrom Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 33/66] notify: add wrapper for `get_referenced_entities` Lukas Wagner
` (36 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
pve-rs/src/notify.rs | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/pve-rs/src/notify.rs b/pve-rs/src/notify.rs
index 04e902c..0c00b03 100644
--- a/pve-rs/src/notify.rs
+++ b/pve-rs/src/notify.rs
@@ -60,6 +60,11 @@ impl Context for PVEContext {
.and_then(|content| lookup_datacenter_config_key(&content, "mail_from"))
.unwrap_or_else(|| String::from("root"))
}
+
+ fn http_proxy_config(&self) -> Option<String> {
+ let content = attempt_file_read("/etc/pve/datacenter.cfg");
+ content.and_then(|content| lookup_datacenter_config_key(&content, "http_proxy"))
+ }
}
#[cfg(test)]
@@ -87,6 +92,7 @@ user:no-mail@pve:1:0::::::
const DC_CONFIG: &str = "
email_from: user@example.com
+http_proxy: http://localhost:1234
keyboard: en-us
";
#[test]
@@ -95,6 +101,11 @@ keyboard: en-us
lookup_datacenter_config_key(DC_CONFIG, "email_from"),
Some("user@example.com".to_string())
);
+ assert_eq!(
+ lookup_datacenter_config_key(DC_CONFIG, "http_proxy"),
+ Some("http://localhost:1234".to_string())
+ );
+ assert_eq!(lookup_datacenter_config_key(DC_CONFIG, "foo"), None);
}
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-perl-rs 33/66] notify: add wrapper for `get_referenced_entities`
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (31 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 32/66] notify: add context for getting http_proxy from datacenter.cfg Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-cluster 34/66] cluster files: add notifications.cfg Lukas Wagner
` (35 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
The function returns all other entities referenced by a filter/target.
This is useful for permission checks, where the user must have the
appropriate permissions for all entities.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
pve-rs/src/notify.rs | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/pve-rs/src/notify.rs b/pve-rs/src/notify.rs
index 0c00b03..d34ec85 100644
--- a/pve-rs/src/notify.rs
+++ b/pve-rs/src/notify.rs
@@ -543,4 +543,13 @@ mod export {
let mut config = this.config.lock().unwrap();
api::filter::delete_filter(&mut config, name)
}
+
+ #[export]
+ fn get_referenced_entities(
+ #[try_from_ref] this: &NotificationConfig,
+ name: &str,
+ ) -> Result<Vec<String>, ApiError> {
+ let config = this.config.lock().unwrap();
+ api::common::get_referenced_entities(&config, name)
+ }
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-cluster 34/66] cluster files: add notifications.cfg
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (32 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 33/66] notify: add wrapper for `get_referenced_entities` Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-cluster 35/66] datacenter: add APT/fencing/replication notification configuration Lukas Wagner
` (34 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
src/PVE/Cluster.pm | 2 ++
src/pmxcfs/status.c | 2 ++
2 files changed, 4 insertions(+)
diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
index c310a67..e3705b6 100644
--- a/src/PVE/Cluster.pm
+++ b/src/PVE/Cluster.pm
@@ -55,6 +55,8 @@ my $observed = {
'firewall/cluster.fw' => 1,
'user.cfg' => 1,
'domains.cfg' => 1,
+ 'notifications.cfg' => 1,
+ 'priv/notifications.cfg' => 1,
'priv/shadow.cfg' => 1,
'priv/tfa.cfg' => 1,
'priv/token.cfg' => 1,
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index 1f29b07..c8094ac 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -82,6 +82,8 @@ static memdb_change_t memdb_change_array[] = {
{ .path = "storage.cfg" },
{ .path = "user.cfg" },
{ .path = "domains.cfg" },
+ { .path = "notifications.cfg"},
+ { .path = "priv/notifications.cfg"},
{ .path = "priv/shadow.cfg" },
{ .path = "priv/acme/plugins.cfg" },
{ .path = "priv/tfa.cfg" },
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-cluster 35/66] datacenter: add APT/fencing/replication notification configuration
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (33 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-cluster 34/66] cluster files: add notifications.cfg Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-cluster 36/66] add libpve-notify-perl package Lukas Wagner
` (33 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
These options allow setting the notification target for package update
notifications, node fencing notifications and replication notifications.
Also, fencing and replication has now new options that allow disabling
notifications altogether.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
src/PVE/DataCenterConfig.pm | 63 +++++++++++++++++++++++++++++++++++--
1 file changed, 61 insertions(+), 2 deletions(-)
diff --git a/src/PVE/DataCenterConfig.pm b/src/PVE/DataCenterConfig.pm
index 3d24e78..09be6eb 100644
--- a/src/PVE/DataCenterConfig.pm
+++ b/src/PVE/DataCenterConfig.pm
@@ -49,13 +49,72 @@ my $notification_format = {
'package-updates' => {
type => 'string',
enum => ['auto', 'always', 'never'],
- description => "Control when the daily update job should send out notification mails.",
- verbose_description => "Control how often the daily update job should send out notification mails:\n"
+ description => "Control when the daily update job should send out notifications.",
+ verbose_description => "Control how often the daily update job should send out notifications:\n"
."* 'auto' daily for systems with a valid subscription, as those are assumed to be "
." production-ready and thus should know about pending updates.\n"
."* 'always' every update, if there are new pending updates.\n"
."* 'never' never send a notification for new pending updates.\n",
default => 'auto',
+ optional => 1,
+ },
+ 'target-package-updates' => {
+ type => 'string',
+ format_description => 'TARGET',
+ description => "Control where notifications about available updates should be sent to.",
+ verbose_description => "Control where notifications about available"
+ . " updates should be sent to."
+ . " Has to be the name of a notification target (endpoint or notification group)."
+ . " If the 'target-package-updates' parameter is not set, the system will send mails"
+ . " to root via a 'sendmail' notification endpoint.",
+ optional => 1,
+ },
+ 'fencing' => {
+ type => 'string',
+ enum => ['always', 'never'],
+ description => "Control if notifications about node fencing should be sent.",
+ verbose_description => "Control if notifications about node fencing should be sent.\n"
+ . "* 'always' always send out notifications\n"
+ . "* 'never' never send out notifications.\n"
+ . "For production systems, turning off node fencing notifications is not"
+ . "recommended!\n",
+ default => 'always',
+ optional => 1,
+ },
+ 'target-fencing' => {
+ type => 'string',
+ format_description => 'TARGET',
+ description => "Control where notifications about fenced cluster nodes should be sent to.",
+ verbose_description => "Control where notifications about fenced cluster nodes"
+ . " should be sent to."
+ . " Has to be the name of a notification target (endpoint or notification group)."
+ . " If the 'target-fencing' parameter is not set, the system will send mails"
+ . " to root via a 'sendmail' notification endpoint.",
+ optional => 1,
+ },
+ 'replication' => {
+ type => 'string',
+ enum => ['always', 'never'],
+ description => "Control if notifications for replication failures should be sent.",
+ verbose_description => "Control if notifications for replication failures should be sent.\n"
+ . "* 'always' always send out notifications\n"
+ . "* 'never' never send out notifications.\n"
+ . "For production systems, turning off replication notifications is not"
+ . "recommended!\n",
+ default => 'always',
+ optional => 1,
+ },
+ 'target-replication' => {
+ type => 'string',
+ format_description => 'TARGET',
+ description => "Control where notifications for failed storage replication jobs should"
+ . " be sent to.",
+ verbose_description => "Control where notifications for failed storage replication jobs"
+ . " should be sent to."
+ . " Has to be the name of a notification target (endpoint or notification group)."
+ . " If the 'target-replication' parameter is not set, the system will send mails"
+ . " to root via a 'sendmail' notification endpoint.",
+ optional => 1,
},
};
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-cluster 36/66] add libpve-notify-perl package
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (34 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-cluster 35/66] datacenter: add APT/fencing/replication notification configuration Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-19 12:27 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-guest-common 37/66] vzdump: add config options for new notification backend Lukas Wagner
` (32 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
The package contains the PVE::Notify. It is a very thin wrapper
around the Proxmox::RS::Notify module, feeding the configuration
from the new 'notifications.cfg' and 'priv/notifications.cfg' files
into it.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
debian/control | 9 ++
debian/libpve-notify-perl.docs | 1 +
debian/libpve-notify-perl.install | 1 +
src/PVE/Makefile | 2 +-
src/PVE/Notify.pm | 145 ++++++++++++++++++++++++++++++
5 files changed, 157 insertions(+), 1 deletion(-)
create mode 100644 debian/libpve-notify-perl.docs
create mode 100644 debian/libpve-notify-perl.install
create mode 100644 src/PVE/Notify.pm
diff --git a/debian/control b/debian/control
index f1c13cb..77d7f8b 100644
--- a/debian/control
+++ b/debian/control
@@ -85,3 +85,12 @@ Breaks: pve-cluster (<= 6.0-7),
Replaces: pve-cluster (<= 6.0-7),
Description: Proxmox Virtual Environment cluster Perl API modules.
This package contains the API2 endpoints and CLI binary 'pvecm'.
+
+Package: libpve-notify-perl
+Architecture: all
+Pre-Depends: ${misc:Pre-Depends},
+Depends: libpve-cluster-perl (= ${binary:Version}),
+ libpve-rs-perl (>= 0.7.1),
+ ${misc:Depends},
+ ${perl:Depends},
+Description: Notify helper module
diff --git a/debian/libpve-notify-perl.docs b/debian/libpve-notify-perl.docs
new file mode 100644
index 0000000..8696672
--- /dev/null
+++ b/debian/libpve-notify-perl.docs
@@ -0,0 +1 @@
+debian/SOURCE
diff --git a/debian/libpve-notify-perl.install b/debian/libpve-notify-perl.install
new file mode 100644
index 0000000..b590d07
--- /dev/null
+++ b/debian/libpve-notify-perl.install
@@ -0,0 +1 @@
+usr/share/perl5/PVE/Notify.pm
diff --git a/src/PVE/Makefile b/src/PVE/Makefile
index 10291a6..ac4a9ce 100644
--- a/src/PVE/Makefile
+++ b/src/PVE/Makefile
@@ -11,7 +11,7 @@ PVE_VENDORARCH=$(DESTDIR)/$(PERL_VENDORARCH)/auto/PVE/IPCC
PERL_DOC_INC_DIRS:=..
SUBDIRS=Cluster CLI API2
-SOURCES=IPCC.pm Cluster.pm Corosync.pm RRD.pm DataCenterConfig.pm SSHInfo.pm
+SOURCES=IPCC.pm Cluster.pm Corosync.pm RRD.pm DataCenterConfig.pm Notify.pm SSHInfo.pm
all:
diff --git a/src/PVE/Notify.pm b/src/PVE/Notify.pm
new file mode 100644
index 0000000..a3bcfbb
--- /dev/null
+++ b/src/PVE/Notify.pm
@@ -0,0 +1,145 @@
+package PVE::Notify;
+
+use strict;
+use warnings;
+
+use PVE::Cluster qw(cfs_register_file cfs_read_file cfs_lock_file cfs_write_file);
+use PVE::RS::Notify;
+
+cfs_register_file(
+ 'notifications.cfg',
+ \&parse_notification_config,
+ \&write_notification_config,
+);
+
+cfs_register_file(
+ 'priv/notifications.cfg',
+ \&parse_notification_config,
+ \&write_notification_config,
+);
+
+my $mail_to_root_target = 'mail-to-root';
+
+sub parse_notification_config {
+ my ($filename, $raw) = @_;
+
+ $raw = '' if !defined($raw);
+ return $raw;
+}
+
+sub write_notification_config {
+ my ($filename, $config) = @_;
+ return $config;
+}
+
+sub lock_config {
+ my ($code, $timeout) = @_;
+
+ cfs_lock_file('notifications.cfg', $timeout, sub {
+ cfs_lock_file('priv/notifications.cfg', $timeout, $code);
+ die $@ if $@;
+ });
+ die $@ if $@;
+}
+
+sub read_config {
+ my $config = cfs_read_file('notifications.cfg');
+ my $priv_config = cfs_read_file('priv/notifications.cfg');
+
+ my $notification_config = PVE::RS::Notify->parse_config($config, $priv_config);
+
+ eval {
+ # This target should always be available...
+ $notification_config->add_sendmail_endpoint(
+ $mail_to_root_target,
+ undef,
+ ['root@pam'],
+ undef,
+ undef,
+ 'Send mail to root@pam\'s email address'
+ );
+ };
+
+ return $notification_config;
+}
+
+sub write_config {
+ my ($notification_config) = @_;
+
+ eval {
+ # ... but don't persist it to the config.
+ # Rationale: If it is in the config, the user might think
+ # that it can be changed by editing the configuration there.
+ # However, since we always add it in `read_config`, any changes
+ # will be implicitly overridden by the default.
+
+ # If users want's to change the configuration, they are supposed to
+ # create a new sendmail endpoint.
+ $notification_config->delete_sendmail_endpoint($mail_to_root_target);
+ };
+
+ my ($config, $priv_config) = $notification_config->write_config();
+ cfs_write_file('notifications.cfg', $config);
+ cfs_write_file('priv/notifications.cfg', $priv_config);
+}
+
+sub default_target {
+ return $mail_to_root_target;
+}
+
+sub notify {
+ my ($target, $severity, $title, $message, $properties, $config) = @_;
+ _send_notification($target, $severity, $title, $message, $properties, $config);
+}
+
+sub info {
+ my ($target, $title, $message, $properties, $config) = @_;
+ _send_notification($target, 'info', $title, $message, $properties, $config);
+}
+
+sub notice {
+ my ($target, $title, $message, $properties, $config) = @_;
+ _send_notification($target, 'notice', $title, $message, $properties, $config);
+}
+
+sub warning {
+ my ($target, $title, $message, $properties, $config) = @_;
+ _send_notification($target, 'warning', $title, $message, $properties, $config);
+}
+
+sub error {
+ my ($target, $title, $message, $properties, $config) = @_;
+ _send_notification($target, 'error', $title, $message, $properties, $config);
+}
+
+sub _send_notification {
+ my ($target, $severity, $title, $message, $properties, $config) = @_;
+ $config = read_config() if !defined($config);
+ my ($module, $file, $line) = caller(1);
+
+ # Augment properties with the source code location of the notify call
+ my $props_with_source = {
+ %$properties,
+ source => {
+ module => $module,
+ file => $file,
+ line => $line,
+ }
+ };
+
+ $config->send($target, $severity, $title, $message, $props_with_source);
+}
+
+sub check_may_use_target {
+ my ($target, $rpcenv) = @_;
+ my $user = $rpcenv->get_user();
+
+ my $config = read_config();
+ my $entities = $config->get_referenced_entities($target);
+
+ for my $entity (@$entities) {
+ $rpcenv->check($user, "/mapping/notification/$entity", [ 'Mapping.Use' ]);
+ }
+}
+
+1;
\ No newline at end of file
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-guest-common 37/66] vzdump: add config options for new notification backend
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (35 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-cluster 36/66] add libpve-notify-perl package Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-common 38/66] JSONSchema: increase maxLength of config-digest to 64 Lukas Wagner
` (31 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
- Add new option 'notification-target'
Allows to select to which endpoint/group notifications shall be sent
- Add new option 'notification-policy'
Replacement for the now deprecated 'mailnotification' option. Mostly
just a rename for consistency, but also adds the 'never' option.
- Mark 'mailnotification' as deprecated in favor of 'notification-policy'
- Clarify that 'mailto' is ignored if 'notification-target' is set
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
src/PVE/VZDump/Common.pm | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)
diff --git a/src/PVE/VZDump/Common.pm b/src/PVE/VZDump/Common.pm
index a6fe483..31e9fdb 100644
--- a/src/PVE/VZDump/Common.pm
+++ b/src/PVE/VZDump/Common.pm
@@ -167,16 +167,33 @@ my $confdesc = {
type => 'string',
format => 'email-or-username-list',
description => "Comma-separated list of email addresses or users that should" .
- " receive email notifications.",
+ " receive email notifications. Has no effect if the 'notification-target' option " .
+ " is set at the same time.",
optional => 1,
},
mailnotification => {
type => 'string',
- description => "Specify when to send an email",
+ description => "Deprecated: use 'notification-policy' instead.",
optional => 1,
enum => [ 'always', 'failure' ],
default => 'always',
},
+ 'notification-policy' => {
+ type => 'string',
+ description => "Specify when to send a notification",
+ optional => 1,
+ enum => [ 'always', 'failure', 'never'],
+ default => 'always',
+ },
+ 'notification-target' => {
+ type => 'string',
+ format => 'pve-configid',
+ description => "Determine the target to which notifications should be sent." .
+ " Can either be a notification endpoint or a notification group." .
+ " This option takes precedence over 'mailto', meaning that if both are " .
+ " set, the 'mailto' option will be ignored.",
+ optional => 1,
+ },
tmpdir => {
type => 'string',
description => "Store temporary files to specified directory.",
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-common 38/66] JSONSchema: increase maxLength of config-digest to 64
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (36 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-guest-common 37/66] vzdump: add config options for new notification backend Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-19 12:31 ` Wolfgang Bumiller
2023-07-19 12:41 ` Fiona Ebner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-ha-manager 39/66] manager: send notifications via new notification module Lukas Wagner
` (30 subsequent siblings)
68 siblings, 2 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
The new notification backend is implemented in Rust where we use SHA256
for config digests.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
src/PVE/JSONSchema.pm | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/src/PVE/JSONSchema.pm b/src/PVE/JSONSchema.pm
index 7589bba..49e0d7a 100644
--- a/src/PVE/JSONSchema.pm
+++ b/src/PVE/JSONSchema.pm
@@ -93,10 +93,13 @@ register_standard_option('pve-bridge-id', {
});
register_standard_option('pve-config-digest', {
- description => 'Prevent changes if current configuration file has different SHA1 digest. This can be used to prevent concurrent modifications.',
+ description => 'Prevent changes if current configuration file has a different digest. '
+ . 'This can be used to prevent concurrent modifications.',
type => 'string',
optional => 1,
- maxLength => 40, # sha1 hex digest length is 40
+ # sha1 hex digests are 40 characters long
+ # sha256 hex digests are 64 characters long (sha256 is used in our Rust code)
+ maxLength => 64,
});
register_standard_option('skiplock', {
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-ha-manager 39/66] manager: send notifications via new notification module
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (37 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-common 38/66] JSONSchema: increase maxLength of config-digest to 64 Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 40/66] test: fix names of .PHONY targets Lukas Wagner
` (29 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
... instead of using sendmail directly.
If the new 'notify.target-fencing' parameter from datacenter config
is set, we use it as a target for notifications. If it is not set,
we send the notification to the default target (mail-to-root).
There is also a new 'notify.fencing' paramter which controls if
notifications should be sent at all. If it is not set, we
default to the old behavior, which is to send.
Also add dependency to the `libpve-notify-perl` package to d/control.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
debian/control | 2 ++
src/PVE/HA/Env.pm | 6 ++---
src/PVE/HA/Env/PVE2.pm | 21 +++++++++-------
src/PVE/HA/NodeStatus.pm | 52 ++++++++++++++++++++++++----------------
src/PVE/HA/Sim/Env.pm | 10 ++++++--
5 files changed, 57 insertions(+), 34 deletions(-)
diff --git a/debian/control b/debian/control
index 5bff56a..ffa9c1c 100644
--- a/debian/control
+++ b/debian/control
@@ -8,6 +8,7 @@ Build-Depends: debhelper-compat (= 13),
libpve-access-control,
libpve-cluster-perl,
libpve-common-perl,
+ libpve-notify-perl,
libpve-rs-perl (>= 0.7.3),
lintian,
pve-cluster,
@@ -21,6 +22,7 @@ Architecture: any
Depends: libjson-perl,
libpve-cluster-perl,
libpve-common-perl,
+ libpve-notify-perl,
libpve-rs-perl (>= 0.7.3),
pve-cluster (>= 3.0-17),
pve-container (>= 5.0.1),
diff --git a/src/PVE/HA/Env.pm b/src/PVE/HA/Env.pm
index 16603ec..b7060a4 100644
--- a/src/PVE/HA/Env.pm
+++ b/src/PVE/HA/Env.pm
@@ -144,10 +144,10 @@ sub log {
return $self->{plug}->log($level, @args);
}
-sub sendmail {
- my ($self, $subject, $text) = @_;
+sub send_notification {
+ my ($self, $subject, $text, $properties) = @_;
- return $self->{plug}->sendmail($subject, $text);
+ return $self->{plug}->send_notification($subject, $text, $properties);
}
# acquire a cluster wide manager lock
diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index f6ebfeb..ea9e6e4 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -13,6 +13,7 @@ use PVE::Cluster qw(cfs_register_file cfs_read_file cfs_write_file cfs_lock_file
use PVE::DataCenterConfig;
use PVE::INotify;
use PVE::RPCEnvironment;
+use PVE::Notify;
use PVE::HA::Tools ':exit_codes';
use PVE::HA::Env;
@@ -219,16 +220,20 @@ sub log {
syslog($level, $msg);
}
-sub sendmail {
- my ($self, $subject, $text) = @_;
+sub send_notification {
+ my ($self, $subject, $text, $properties) = @_;
- # Leave it to postfix to append the correct hostname
- my $mailfrom = 'root';
- # /root/.forward makes pvemailforward redirect the
- # mail to the address configured in the datacenter
- my $mailto = 'root';
+ eval {
+ my $dc_config = PVE::Cluster::cfs_read_file('datacenter.cfg');
+ my $target = $dc_config->{notify}->{'target-fencing'} // PVE::Notify::default_target();
+ my $notify = $dc_config->{notify}->{fencing} // 'always';
+
+ if ($notify eq 'always') {
+ PVE::Notify::error($target, $subject, $text, $properties);
+ }
+ };
- PVE::Tools::sendmail($mailto, $subject, $text, undef, $mailfrom);
+ $self->log("warning", "could not notify: $@") if $@;
}
my $last_lock_status_hash = {};
diff --git a/src/PVE/HA/NodeStatus.pm b/src/PVE/HA/NodeStatus.pm
index ee5be8e..b264a36 100644
--- a/src/PVE/HA/NodeStatus.pm
+++ b/src/PVE/HA/NodeStatus.pm
@@ -188,35 +188,45 @@ sub update {
}
}
-# assembles a commont text for fence emails
-my $send_fence_state_email = sub {
- my ($self, $subject_prefix, $subject, $node) = @_;
-
- my $haenv = $self->{haenv};
-
- my $mail_text = <<EOF
-The node '$node' failed and needs manual intervention.
+my $body_template = <<EOT;
+{{#verbatim}}
+The node '{{node}}' failed and needs manual intervention.
-The PVE HA manager tries to fence it and recover the
-configured HA resources to a healthy node if possible.
+The PVE HA manager tries to fence it and recover the configured HA resources to
+a healthy node if possible.
-Current fence status: $subject_prefix
-$subject
+Current fence status: {{subject-prefix}}
+{{subject}}
+{{/verbatim}}
+{{heading-2 "Overall Cluster status:"}}
+{{object status-data}}
+EOT
-Overall Cluster status:
------------------------
+my $subject_template = "{{subject-prefix}}: {{subject}}";
-EOF
-;
- my $mail_subject = $subject_prefix . ': ' . $subject;
+# assembles a commont text for fence emails
+my $send_fence_state_email = sub {
+ my ($self, $subject_prefix, $subject, $node) = @_;
+ my $haenv = $self->{haenv};
my $status = $haenv->read_manager_status();
- my $data = { manager_status => $status, node_status => $self->{status} };
-
- $mail_text .= to_json($data, { pretty => 1, canonical => 1});
- $haenv->sendmail($mail_subject, $mail_text);
+ my $notification_properties = {
+ "status-data" => {
+ manager_status => $status,
+ node_status => $self->{status}
+ },
+ "node" => $node,
+ "subject-prefix" => $subject_prefix,
+ "subject" => $subject,
+ };
+
+ $haenv->send_notification(
+ $subject_template,
+ $body_template,
+ $notification_properties
+ );
};
diff --git a/src/PVE/HA/Sim/Env.pm b/src/PVE/HA/Sim/Env.pm
index c6ea73c..d3aea8d 100644
--- a/src/PVE/HA/Sim/Env.pm
+++ b/src/PVE/HA/Sim/Env.pm
@@ -288,8 +288,14 @@ sub log {
printf("%-5s %5d %12s: $msg\n", $level, $time, "$self->{nodename}/$self->{log_id}");
}
-sub sendmail {
- my ($self, $subject, $text) = @_;
+sub send_notification {
+ my ($self, $subject, $text, $properties) = @_;
+
+ # The template for the subject is "{{subject-prefix}}: {{subject}}"
+ # We have to perform poor-man's template rendering to pass the test cases.
+
+ $subject = $subject =~ s/\{\{subject-prefix}}/$properties->{"subject-prefix"}/r;
+ $subject = $subject =~ s/\{\{subject}}/$properties->{"subject"}/r;
# only log subject, do not spam the logs
$self->log('email', $subject);
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 40/66] test: fix names of .PHONY targets
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (38 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-ha-manager 39/66] manager: send notifications via new notification module Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 41/66] d/control: add dependency to `libpve-notify-perl` Lukas Wagner
` (28 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
They need to have the same name as the target.
Took the opportunity to move the .PHONY right next to the target recipe,
so that mistakes like these are hopefully easier caught.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
test/Makefile | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/test/Makefile b/test/Makefile
index 670a3611..cccdc1c9 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -4,29 +4,35 @@ all:
export PERLLIB=..
-.PHONY: check balloon-test replication-test mail-test vzdump-test
+.PHONY: check
check: test-replication test-balloon test-mail test-vzdump test-osd
+.PHONY: test-balloon
test-balloon:
./balloontest.pl
+.PHONY: test-replication
test-replication: replication1.t replication2.t replication3.t replication4.t replication5.t replication6.t
replication%.t: replication_test%.pl
./$<
+.PHONY: test-mail
test-mail:
./mail_test.pl
+.PHONY: test-vzdump
test-vzdump: test-vzdump-guest-included test-vzdump-new
-.PHONY: test-vzdump-guest-included test-vzdump-new
+.PHONY: test-vzdump-guest-included
test-vzdump-guest-included:
./vzdump_guest_included_test.pl
+.PHONY: test-vzdump-new
test-vzdump-new:
./vzdump_new_test.pl
+.PHONY: test-osd
test-osd:
./OSD_test.pl
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 41/66] d/control: add dependency to `libpve-notify-perl`
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (39 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 40/66] test: fix names of .PHONY targets Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 42/66] vzdump: send notifications via new notification module Lukas Wagner
` (27 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
Notes:
Did not add version number since I do not know which it will be yet.
debian/control | 2 ++
1 file changed, 2 insertions(+)
diff --git a/debian/control b/debian/control
index 3206b514..b807dbfe 100644
--- a/debian/control
+++ b/debian/control
@@ -14,6 +14,7 @@ Build-Depends: debhelper-compat (= 13),
libpve-common-perl (>= 7.2-6),
libpve-guest-common-perl (>= 5.0.2),
libpve-http-server-perl (>= 2.0-12),
+ libpve-notify-perl,
libpve-rs-perl (>= 0.7.1),
libpve-storage-perl (>= 6.3-2),
libtemplate-perl,
@@ -61,6 +62,7 @@ Depends: apt (>= 1.5~),
libpve-common-perl (>= 7.2-7),
libpve-guest-common-perl (>= 5.0.2),
libpve-http-server-perl (>= 4.1-1),
+ libpve-notify-perl,
libpve-rs-perl (>= 0.7.1),
libpve-storage-perl (>= 7.2-12),
librados2-perl (>= 1.3-1),
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 42/66] vzdump: send notifications via new notification module
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (40 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 41/66] d/control: add dependency to `libpve-notify-perl` Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 43/66] test: rename mail_test.pl to vzdump_notification_test.pl Lukas Wagner
` (26 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
... instead of using sendmail directly.
If the new 'notification-target' parameter is set,
we send the notification to this endpoint or group.
If 'mailto' is set, we add a temporary endpoint and a
temporary group containg both targets.
This commit also refactors the old 'sendmail' sub heavily:
- Use template-based notification text instead of endless
string concatenations
- Removing the old plaintext/HTML table rendering in favor of
the new template/property-based approach offered by the
`proxmox-notify` crate.
- Rename `sendmail` sub to `send_notification`
- Breaking out some of the code into helper subs, hopefully
reducing the spaghetti factor a bit
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
PVE/API2/VZDump.pm | 10 +-
PVE/VZDump.pm | 335 +++++++++++++++++++++++++--------------------
test/mail_test.pl | 36 ++---
3 files changed, 214 insertions(+), 167 deletions(-)
diff --git a/PVE/API2/VZDump.pm b/PVE/API2/VZDump.pm
index e3dcd0bd..3886772e 100644
--- a/PVE/API2/VZDump.pm
+++ b/PVE/API2/VZDump.pm
@@ -44,7 +44,9 @@ __PACKAGE__->register_method ({
."'Datastore.AllocateSpace' on the backup storage. The 'tmpdir', 'dumpdir' and "
."'script' parameters are restricted to the 'root\@pam' user. The 'maxfiles' and "
."'prune-backups' settings require 'Datastore.Allocate' on the backup storage. The "
- ."'bwlimit', 'performance' and 'ionice' parameters require 'Sys.Modify' on '/'.",
+ ."'bwlimit', 'performance' and 'ionice' parameters require 'Sys.Modify' on '/'. "
+ ."If 'notification-target' is set, then the 'Mapping.Use' permission is needed on "
+ ."'/mapping/notification/<target>'.",
user => 'all',
},
protected => 1,
@@ -113,6 +115,10 @@ __PACKAGE__->register_method ({
$rpcenv->check($user, "/storage/$storeid", [ 'Datastore.AllocateSpace' ]);
}
+ if (my $target = $param->{'notification-target'}) {
+ PVE::Notify::check_may_use_target($target, $rpcenv);
+ }
+
my $worker = sub {
my $upid = shift;
@@ -127,7 +133,7 @@ __PACKAGE__->register_method ({
$vzdump->getlock($upid); # only one process allowed
};
if (my $err = $@) {
- $vzdump->sendmail([], 0, $err);
+ $vzdump->send_notification([], 0, $err);
exit(-1);
}
diff --git a/PVE/VZDump.pm b/PVE/VZDump.pm
index c58e5f78..7dc9f31e 100644
--- a/PVE/VZDump.pm
+++ b/PVE/VZDump.pm
@@ -19,6 +19,7 @@ use PVE::Exception qw(raise_param_exc);
use PVE::HA::Config;
use PVE::HA::Env::PVE2;
use PVE::JSONSchema qw(get_standard_option);
+use PVE::Notify;
use PVE::RPCEnvironment;
use PVE::Storage;
use PVE::VZDump::Common;
@@ -317,21 +318,90 @@ sub read_vzdump_defaults {
return $res;
}
-use constant MAX_MAIL_SIZE => 1024*1024;
-sub sendmail {
- my ($self, $tasklist, $totaltime, $err, $detail_pre, $detail_post) = @_;
+sub read_backup_task_logs {
+ my ($task_list) = @_;
- my $opts = $self->{opts};
+ my $task_logs = "";
- my $mailto = $opts->{mailto};
+ for my $task (@$task_list) {
+ my $vmid = $task->{vmid};
+ my $log_file = $task->{tmplog};
+ if (!$task->{tmplog}) {
+ $task_logs .= "$vmid: no log available\n\n";
+ next;
+ }
+ if (open (my $TMP, '<', "$log_file")) {
+ while (my $line = <$TMP>) {
+ next if $line =~ /^status: \d+/; # not useful in mails
+ $task_logs .= encode8bit ("$vmid: $line");
+ }
+ close ($TMP);
+ } else {
+ $task_logs .= "$vmid: Could not open log file\n\n";
+ }
+ $task_logs .= "\n";
+ }
- return if !($mailto && scalar(@$mailto));
+ return $task_logs;
+}
- my $cmdline = $self->{cmdline};
+sub build_guest_table {
+ my ($task_list) = @_;
+
+ my $table = {
+ schema => {
+ columns => [
+ {
+ label => "VMID",
+ id => "vmid"
+ },
+ {
+ label => "Name",
+ id => "name"
+ },
+ {
+ label => "Status",
+ id => "status"
+ },
+ {
+ label => "Time",
+ id => "time",
+ renderer => "duration"
+ },
+ {
+ label => "Size",
+ id => "size",
+ renderer => "human-bytes"
+ },
+ {
+ label => "Filename",
+ id => "filename"
+ },
+ ]
+ },
+ data => []
+ };
+
+ for my $task (@$task_list) {
+ my $successful = $task->{state} eq 'ok';
+ my $size = $successful ? $task->{size} : 0;
+ my $filename = $successful ? $task->{target} : undef;
+ push @{$table->{data}}, {
+ "vmid" => $task->{vmid},
+ "name" => $task->{hostname},
+ "status" => $task->{state},
+ "time" => $task->{backuptime},
+ "size" => $size,
+ "filename" => $filename,
+ };
+ }
+
+ return $table;
+}
- my $ecount = 0;
- foreach my $task (@$tasklist) {
- $ecount++ if $task->{state} ne 'ok';
+sub sanitize_task_list {
+ my ($task_list) = @_;
+ for my $task (@$task_list) {
chomp $task->{msg} if $task->{msg};
$task->{backuptime} = 0 if !$task->{backuptime};
$task->{size} = 0 if !$task->{size};
@@ -342,164 +412,133 @@ sub sendmail {
$task->{msg} = 'aborted';
}
}
+}
- my $notify = $opts->{mailnotification} || 'always';
- return if (!$ecount && !$err && ($notify eq 'failure'));
+sub count_failed_tasks {
+ my ($tasklist) = @_;
- my $stat = ($ecount || $err) ? 'backup failed' : 'backup successful';
- if ($err) {
- if ($err =~ /\n/) {
- $stat .= ": multiple problems";
- } else {
- $stat .= ": $err";
- $err = undef;
- }
+ my $error_count = 0;
+ for my $task (@$tasklist) {
+ $error_count++ if $task->{state} ne 'ok';
}
+ return $error_count;
+}
+
+sub get_hostname {
my $hostname = `hostname -f` || PVE::INotify::nodename();
chomp $hostname;
+ return $hostname;
+}
- # text part
- my $text = $err ? "$err\n\n" : '';
- my $namelength = 20;
- $text .= sprintf (
- "%-10s %-${namelength}s %-6s %10s %10s %s\n",
- qw(VMID NAME STATUS TIME SIZE FILENAME)
- );
- foreach my $task (@$tasklist) {
- my $name = substr($task->{hostname}, 0, $namelength);
- my $successful = $task->{state} eq 'ok';
- my $size = $successful ? format_size ($task->{size}) : 0;
- my $filename = $successful ? $task->{target} : '-';
- my $size_fmt = $successful ? "%10s": "%8.2fMB";
- $text .= sprintf(
- "%-10s %-${namelength}s %-6s %10s $size_fmt %s\n",
- $task->{vmid},
- $name,
- $task->{state},
- format_time($task->{backuptime}),
- $size,
- $filename,
- );
- }
+my $subject_template = "vzdump backup status ({{hostname}}): {{status-text}}";
- my $text_log_part;
- $text_log_part .= "\nDetailed backup logs:\n\n";
- $text_log_part .= "$cmdline\n\n";
+my $body_template = <<EOT;
+{{error-message}}
+{{heading-1 "Details"}}
+{{table guest-table}}
- $text_log_part .= $detail_pre . "\n" if defined($detail_pre);
- foreach my $task (@$tasklist) {
- my $vmid = $task->{vmid};
- my $log = $task->{tmplog};
- if (!$log) {
- $text_log_part .= "$vmid: no log available\n\n";
- next;
- }
- if (open (my $TMP, '<', "$log")) {
- while (my $line = <$TMP>) {
- next if $line =~ /^status: \d+/; # not useful in mails
- $text_log_part .= encode8bit ("$vmid: $line");
- }
- close ($TMP);
- } else {
- $text_log_part .= "$vmid: Could not open log file\n\n";
- }
- $text_log_part .= "\n";
- }
- $text_log_part .= $detail_post if defined($detail_post);
+Total running time: {{duration total-time}}
- # html part
- my $html = "<html><body>\n";
- $html .= "<p>" . (escape_html($err) =~ s/\n/<br>/gr) . "</p>\n" if $err;
- $html .= "<table border=1 cellpadding=3>\n";
- $html .= "<tr><td>VMID<td>NAME<td>STATUS<td>TIME<td>SIZE<td>FILENAME</tr>\n";
+{{heading-1 "Logs"}}
+{{verbatim-monospaced logs}}
+EOT
- my $ssize = 0;
- foreach my $task (@$tasklist) {
- my $vmid = $task->{vmid};
- my $name = $task->{hostname};
-
- if ($task->{state} eq 'ok') {
- $ssize += $task->{size};
-
- $html .= sprintf (
- "<tr><td>%s<td>%s<td>OK<td>%s<td align=right>%s<td>%s</tr>\n",
- $vmid,
- $name,
- format_time($task->{backuptime}),
- format_size ($task->{size}),
- escape_html ($task->{target}),
- );
- } else {
- $html .= sprintf (
- "<tr><td>%s<td>%s<td><font color=red>FAILED<td>%s<td colspan=2>%s</tr>\n",
- $vmid,
- $name,
- format_time($task->{backuptime}),
- escape_html ($task->{msg}),
- );
- }
- }
+use constant MAX_LOG_SIZE => 1024*1024;
- $html .= sprintf ("<tr><td align=left colspan=3>TOTAL<td>%s<td>%s<td></tr>",
- format_time ($totaltime), format_size ($ssize));
+sub send_notification {
+ my ($self, $tasklist, $total_time, $err, $detail_pre, $detail_post) = @_;
- $html .= "\n</table><br><br>\n";
- my $html_log_part;
- $html_log_part .= "Detailed backup logs:<br /><br />\n";
- $html_log_part .= "<pre>\n";
- $html_log_part .= escape_html($cmdline) . "\n\n";
+ my $opts = $self->{opts};
+ my $mailto = $opts->{mailto};
+ my $cmdline = $self->{cmdline};
+ my $target = $opts->{"notification-target"};
+ # Fall back to 'mailnotification' if 'notification-policy' is not set.
+ # If both are set, 'notification-policy' takes precedence
+ my $policy = $opts->{"notification-policy"} // $opts->{mailnotification} // 'always';
- $html_log_part .= escape_html($detail_pre) . "\n" if defined($detail_pre);
- foreach my $task (@$tasklist) {
- my $vmid = $task->{vmid};
- my $log = $task->{tmplog};
- if (!$log) {
- $html_log_part .= "$vmid: no log available\n\n";
- next;
- }
- if (open (my $TMP, '<', "$log")) {
- while (my $line = <$TMP>) {
- next if $line =~ /^status: \d+/; # not useful in mails
- if ($line =~ m/^\S+\s\d+\s+\d+:\d+:\d+\s+(ERROR|WARN):/) {
- $html_log_part .= encode8bit ("$vmid: <font color=red>".
- escape_html ($line) . "</font>");
- } else {
- $html_log_part .= encode8bit ("$vmid: " . escape_html ($line));
- }
- }
- close ($TMP);
+ return if ($policy eq 'never');
+
+ sanitize_task_list($tasklist);
+ my $error_count = count_failed_tasks($tasklist);
+
+ my $failed = ($error_count || $err);
+
+ return if (!$failed && ($policy eq 'failure'));
+
+ my $status_text = $failed ? 'backup failed' : 'backup successful';
+
+ if ($err) {
+ if ($err =~ /\n/) {
+ $status_text .= ": multiple problems";
} else {
- $html_log_part .= "$vmid: Could not open log file\n\n";
+ $status_text .= ": $err";
+ $err = undef;
}
- $html_log_part .= "\n";
}
- $html_log_part .= escape_html($detail_post) if defined($detail_post);
- $html_log_part .= "</pre>";
- my $html_end = "\n</body></html>\n";
- # end html part
-
- if (length($text) + length($text_log_part) +
- length($html) + length($html_log_part) +
- length($html_end) < MAX_MAIL_SIZE)
+
+ my $text_log_part = "$cmdline\n\n";
+ $text_log_part .= $detail_pre . "\n" if defined($detail_pre);
+ $text_log_part .= read_backup_task_logs($tasklist);
+ $text_log_part .= $detail_post if defined($detail_post);
+
+ if (length($text_log_part) > MAX_LOG_SIZE)
{
- $html .= $html_log_part;
- $html .= $html_end;
- $text .= $text_log_part;
- } else {
- my $msg = "Log output was too long to be sent by mail. ".
+ # Let's limit the maximum length of included logs
+ $text_log_part = "Log output was too long to be sent. ".
"See Task History for details!\n";
- $text .= $msg;
- $html .= "<p>$msg</p>";
- $html .= $html_end;
+ };
+
+ my $notification_props = {
+ "hostname" => get_hostname(),
+ "error-message" => $err,
+ "guest-table" => build_guest_table($tasklist),
+ "logs" => $text_log_part,
+ "status-text" => $status_text,
+ "total-time" => $total_time,
+ };
+
+ my $notification_config = PVE::Notify::read_config();
+
+ if ($mailto && scalar(@$mailto)) {
+ # <, >, @ is not allowed in endpoint names, but only it is only
+ # verified once the config is serialized. That means that
+ # we can rely on that fact that no other endpoint with this name exists.
+ my $endpoint_name = "mail-to-<" . join(",", @$mailto) . ">";
+ $notification_config->add_sendmail_endpoint(
+ $endpoint_name,
+ $mailto,
+ undef,
+ undef,
+ "vzdump backup tool");
+
+ my $endpoints = [$endpoint_name];
+
+ # Create an anonymous group containing the sendmail endpoint and the
+ # $target endpoint, if specified
+ if ($target) {
+ push @$endpoints, $target;
+ }
+
+ $target = "group-$endpoint_name";
+ $notification_config->add_group(
+ $target,
+ $endpoints,
+ );
}
- my $subject = "vzdump backup status ($hostname) : $stat";
+ return if (!$target);
- my $dcconf = PVE::Cluster::cfs_read_file('datacenter.cfg');
- my $mailfrom = $dcconf->{email_from} || "root";
+ my $severity = $failed ? "error" : "info";
- PVE::Tools::sendmail($mailto, $subject, $text, $html, $mailfrom, "vzdump backup tool");
+ PVE::Notify::notify(
+ $target,
+ $severity,
+ $subject_template,
+ $body_template,
+ $notification_props,
+ $notification_config
+ );
};
sub new {
@@ -632,7 +671,7 @@ sub new {
}
if ($errors) {
- eval { $self->sendmail([], 0, $errors); };
+ eval { $self->send_notification([], 0, $errors); };
debugmsg ('err', $@) if $@;
die "$errors\n";
}
@@ -1322,11 +1361,11 @@ sub exec_backup {
my $totaltime = time() - $starttime;
eval {
- # otherwise $self->sendmail() will interpret it as multiple problems
+ # otherwise $self->send_notification() will interpret it as multiple problems
my $chomped_err = $err;
chomp($chomped_err) if $chomped_err;
- $self->sendmail(
+ $self->send_notification(
$tasklist,
$totaltime,
$chomped_err,
diff --git a/test/mail_test.pl b/test/mail_test.pl
index d0114441..0635ebb7 100755
--- a/test/mail_test.pl
+++ b/test/mail_test.pl
@@ -5,7 +5,7 @@ use warnings;
use lib '..';
-use Test::More tests => 5;
+use Test::More tests => 3;
use Test::MockModule;
use PVE::VZDump;
@@ -29,17 +29,19 @@ sub prepare_mail_with_status {
sub prepare_long_mail {
open(TEST_FILE, '>', $TEST_FILE_PATH); # Removes previous content
# 0.5 MB * 2 parts + the overview tables gives more than 1 MB mail
- print TEST_FILE "a" x (1024*1024/2);
+ print TEST_FILE "a" x (1024*1024);
close(TEST_FILE);
}
-my ($result_text, $result_html);
+my $result_text;
+my $result_properties;
+
+my $mock_notification_module = Test::MockModule->new('PVE::Notify');
+$mock_notification_module->mock('send_notification', sub {
+ my ($channel, $severity, $title, $text, $properties) = @_;
-my $mock_tools_module = Test::MockModule->new('PVE::Tools');
-$mock_tools_module->mock('sendmail', sub {
- my (undef, undef, $text, $html, undef, undef) = @_;
$result_text = $text;
- $result_html = $html;
+ $result_properties = $properties;
});
my $mock_cluster_module = Test::MockModule->new('PVE::Cluster');
@@ -47,7 +49,9 @@ $mock_cluster_module->mock('cfs_read_file', sub {
my $path = shift;
if ($path eq 'datacenter.cfg') {
- return {};
+ return {};
+ } elsif ($path eq 'notifications.cfg' || $path eq 'priv/notifications.cfg') {
+ return '';
} else {
die "unexpected cfs_read_file\n";
}
@@ -62,28 +66,26 @@ my $SELF = {
my $task = { state => 'ok', vmid => '100', };
my $tasklist;
sub prepare_test {
- $result_text = $result_html = undef;
+ $result_text = undef;
$task->{tmplog} = shift;
$tasklist = [ $task ];
}
{
prepare_test($TEST_FILE_WRONG_PATH);
- PVE::VZDump::sendmail($SELF, $tasklist, 0, undef, undef, undef);
- like($result_text, $NO_LOGFILE, "Missing logfile is detected");
+ PVE::VZDump::send_notification($SELF, $tasklist, 0, undef, undef, undef);
+ like($result_properties->{logs}, $NO_LOGFILE, "Missing logfile is detected");
}
{
prepare_test($TEST_FILE_PATH);
prepare_mail_with_status();
- PVE::VZDump::sendmail($SELF, $tasklist, 0, undef, undef, undef);
- unlike($result_text, $STATUS, "Status are not in text part of mails");
- unlike($result_html, $STATUS, "Status are not in HTML part of mails");
+ PVE::VZDump::send_notification($SELF, $tasklist, 0, undef, undef, undef);
+ unlike($result_properties->{"status-text"}, $STATUS, "Status are not in text part of mails");
}
{
prepare_test($TEST_FILE_PATH);
prepare_long_mail();
- PVE::VZDump::sendmail($SELF, $tasklist, 0, undef, undef, undef);
- like($result_text, $LOG_TOO_LONG, "Text part of mails gets shortened");
- like($result_html, $LOG_TOO_LONG, "HTML part of mails gets shortened");
+ PVE::VZDump::send_notification($SELF, $tasklist, 0, undef, undef, undef);
+ like($result_properties->{logs}, $LOG_TOO_LONG, "Text part of mails gets shortened");
}
unlink $TEST_FILE_PATH;
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 43/66] test: rename mail_test.pl to vzdump_notification_test.pl
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (41 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 42/66] vzdump: send notifications via new notification module Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 44/66] api: apt: send notification via new notification module Lukas Wagner
` (25 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
test/Makefile | 8 ++++----
test/{mail_test.pl => vzdump_notification_test.pl} | 0
2 files changed, 4 insertions(+), 4 deletions(-)
rename test/{mail_test.pl => vzdump_notification_test.pl} (100%)
diff --git a/test/Makefile b/test/Makefile
index cccdc1c9..62d75050 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -5,7 +5,7 @@ all:
export PERLLIB=..
.PHONY: check
-check: test-replication test-balloon test-mail test-vzdump test-osd
+check: test-replication test-balloon test-vzdump-notification test-vzdump test-osd
.PHONY: test-balloon
test-balloon:
@@ -17,9 +17,9 @@ test-replication: replication1.t replication2.t replication3.t replication4.t re
replication%.t: replication_test%.pl
./$<
-.PHONY: test-mail
-test-mail:
- ./mail_test.pl
+.PHONY: test-vzdump-notification
+test-vzdump-notification:
+ ./vzdump_notification_test.pl
.PHONY: test-vzdump
test-vzdump: test-vzdump-guest-included test-vzdump-new
diff --git a/test/mail_test.pl b/test/vzdump_notification_test.pl
similarity index 100%
rename from test/mail_test.pl
rename to test/vzdump_notification_test.pl
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 44/66] api: apt: send notification via new notification module
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (42 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 43/66] test: rename mail_test.pl to vzdump_notification_test.pl Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 45/66] api: replication: send notifications " Lukas Wagner
` (24 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
... instead of using sendmail directly
If the new 'target-package-updates' is set, we send a
notification to this target. If not, we continue to send
a mail to root@pam (if the mail address is configured)
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
PVE/API2/APT.pm | 99 ++++++++++++++++++++++++++++++++++---------------
1 file changed, 70 insertions(+), 29 deletions(-)
diff --git a/PVE/API2/APT.pm b/PVE/API2/APT.pm
index 6694dbeb..93a6970e 100644
--- a/PVE/API2/APT.pm
+++ b/PVE/API2/APT.pm
@@ -19,6 +19,7 @@ use PVE::DataCenterConfig;
use PVE::SafeSyslog;
use PVE::INotify;
use PVE::Exception;
+use PVE::Notify;
use PVE::RESTHandler;
use PVE::RPCEnvironment;
use PVE::API2Tools;
@@ -272,6 +273,12 @@ __PACKAGE__->register_method({
return $pkglist;
}});
+my $updates_available_subject_template = "New software packages available ({{hostname}})";
+my $updates_available_body_template = <<EOT;
+The following updates are available:
+{{table updates}}
+EOT
+
__PACKAGE__->register_method({
name => 'update_database',
path => 'update',
@@ -279,6 +286,8 @@ __PACKAGE__->register_method({
description => "This is used to resynchronize the package index files from their sources (apt-get update).",
permissions => {
check => ['perm', '/nodes/{node}', [ 'Sys.Modify' ]],
+ description => "If 'notify: target-package-updates' is set, then the user must have the "
+ . "'Mapping.Use' permission on '/mapping/notification/<target>'",
},
protected => 1,
proxyto => 'node',
@@ -307,6 +316,17 @@ __PACKAGE__->register_method({
my ($param) = @_;
my $rpcenv = PVE::RPCEnvironment::get();
+ my $dcconf = PVE::Cluster::cfs_read_file('datacenter.cfg');
+ my $target = $dcconf->{notify}->{'target-package-updates'} //
+ PVE::Notify::default_target();
+
+ if ($param->{notify} && $target ne PVE::Notify::default_target()) {
+ # If we notify via anything other than the default target (mail to root),
+ # then the user must have the proper permissions for the target.
+ # The mail-to-root target does not require these, as otherwise
+ # we would break compatibility.
+ PVE::Notify::check_may_use_target($target, $rpcenv);
+ }
my $authuser = $rpcenv->get_user();
@@ -314,7 +334,6 @@ __PACKAGE__->register_method({
my $upid = shift;
# setup proxy for apt
- my $dcconf = PVE::Cluster::cfs_read_file('datacenter.cfg');
my $aptconf = "// no proxy configured\n";
if ($dcconf->{http_proxy}) {
@@ -336,39 +355,59 @@ __PACKAGE__->register_method({
my $pkglist = &$update_pve_pkgstatus();
if ($param->{notify} && scalar(@$pkglist)) {
+ my $updates_table = {
+ schema => {
+ columns => [
+ {
+ label => "Package",
+ id => "package",
+ },
+ {
+ label => "Old Version",
+ id => "old-version",
+ },
+ {
+ label => "New Version",
+ id => "new-version",
+ }
+ ]
+ },
+ data => []
+ };
+
+ my $hostname = `hostname -f` || PVE::INotify::nodename();
+ chomp $hostname;
+
+ my $count = 0;
+ foreach my $p (sort {$a->{Package} cmp $b->{Package} } @$pkglist) {
+ next if $p->{NotifyStatus} && $p->{NotifyStatus} eq $p->{Version};
+ $count++;
+
+ push @{$updates_table->{data}}, {
+ "package" => $p->{Package},
+ "old-version" => $p->{OldVersion},
+ "new-version" => $p->{Version}
+ };
+ }
- my $usercfg = PVE::Cluster::cfs_read_file("user.cfg");
- my $rootcfg = $usercfg->{users}->{'root@pam'} || {};
- my $mailto = $rootcfg->{email};
-
- if ($mailto) {
- my $hostname = `hostname -f` || PVE::INotify::nodename();
- chomp $hostname;
- my $mailfrom = $dcconf->{email_from} || "root";
- my $subject = "New software packages available ($hostname)";
-
- my $data = "The following updates are available:\n\n";
-
- my $count = 0;
- foreach my $p (sort {$a->{Package} cmp $b->{Package} } @$pkglist) {
- next if $p->{NotifyStatus} && $p->{NotifyStatus} eq $p->{Version};
- $count++;
- if ($p->{OldVersion}) {
- $data .= "$p->{Package}: $p->{OldVersion} ==> $p->{Version}\n";
- } else {
- $data .= "$p->{Package}: $p->{Version} (new)\n";
- }
- }
+ return if !$count;
- return if !$count;
+ my $properties = {
+ updates => $updates_table,
+ hostname => $hostname,
+ };
- PVE::Tools::sendmail($mailto, $subject, $data, undef, $mailfrom, '');
+ PVE::Notify::info(
+ $target,
+ $updates_available_subject_template,
+ $updates_available_body_template,
+ $properties,
+ );
- foreach my $pi (@$pkglist) {
- $pi->{NotifyStatus} = $pi->{Version};
- }
- PVE::Tools::file_set_contents($pve_pkgstatus_fn, encode_json($pkglist));
+ foreach my $pi (@$pkglist) {
+ $pi->{NotifyStatus} = $pi->{Version};
}
+ PVE::Tools::file_set_contents($pve_pkgstatus_fn, encode_json($pkglist));
}
return;
@@ -378,6 +417,8 @@ __PACKAGE__->register_method({
}});
+
+
__PACKAGE__->register_method({
name => 'changelog',
path => 'changelog',
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 45/66] api: replication: send notifications via new notification module
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (43 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 44/66] api: apt: send notification via new notification module Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 46/66] api: prepare api handler module for notification config Lukas Wagner
` (23 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
If the new 'target-replication' option in datacenter.cfg is set
to a notification target, we send notifications that way.
If it is not set, we continue send a notification to the
default target (mail to root@pam).
There is also a new 'replication' option. It controls whether
to send a notification at all.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
PVE/API2/Replication.pm | 63 ++++++++++++++++++++++++++++-------------
1 file changed, 43 insertions(+), 20 deletions(-)
diff --git a/PVE/API2/Replication.pm b/PVE/API2/Replication.pm
index 89c5a802..d61518ba 100644
--- a/PVE/API2/Replication.pm
+++ b/PVE/API2/Replication.pm
@@ -15,6 +15,7 @@ use PVE::QemuConfig;
use PVE::QemuServer;
use PVE::LXC::Config;
use PVE::LXC;
+use PVE::Notify;
use PVE::RESTHandler;
@@ -91,6 +92,24 @@ my sub _should_mail_at_failcount {
return $i * 48 == $fail_count;
};
+my $replication_error_subject_template = "Replication Job: '{{job-id}}' failed";
+my $replication_error_body_template = <<EOT;
+{{#verbatim}}
+Replication job '{{job-id}}' with target '{{job-target}}' and schedule '{{job-schedule}}' failed!
+
+Last successful sync: {{timestamp last-sync}}
+Next sync try: {{timestamp next-sync}}
+Failure count: {{failure-count}}
+
+{{#if (eq failure-count 3)}}
+Note: The system will now reduce the frequency of error reports, as the job
+appears to be stuck.
+{{/if}}
+Error:
+{{verbatim-monospaced error}}
+{{/verbatim}}
+EOT
+
my sub _handle_job_err {
my ($job, $err, $mail) = @_;
@@ -103,33 +122,37 @@ my sub _handle_job_err {
return if !_should_mail_at_failcount($fail_count);
- my $schedule = $job->{schedule} // '*/15';
-
- my $msg = "Replication job $job->{id} with target '$job->{target}' and schedule";
- $msg .= " '$schedule' failed!\n";
-
- $msg .= " Last successful sync: ";
- if (my $last_sync = $jobstate->{last_sync}) {
- $msg .= render_timestamp($last_sync) ."\n";
- } else {
- $msg .= "None/Unknown\n";
- }
# not yet updated, so $job->next_sync here is actually the current one.
# NOTE: Copied from PVE::ReplicationState::job_status()
my $next_sync = $job->{next_sync} + 60 * ($fail_count <= 3 ? 5 * $fail_count : 30);
- $msg .= " Next sync try: " . render_timestamp($next_sync) ."\n";
- $msg .= " Failure count: $fail_count\n";
-
- if ($fail_count == 3) {
- $msg .= "\nNote: The system will now reduce the frequency of error reports,";
- $msg .= " as the job appears to be stuck.\n";
- }
+ # The replication job is run every 15 mins if no schedule is set.
+ my $schedule = $job->{schedule} // '*/15';
- $msg .= "\nError:\n$err";
+ my $properties = {
+ "failure-count" => $fail_count,
+ "last-sync" => $jobstate->{last_sync},
+ "next-sync" => $next_sync,
+ "job-id" => $job->{id},
+ "job-target" => $job->{target},
+ "job-schedule" => $schedule,
+ "error" => $err,
+ };
eval {
- PVE::Tools::sendmail('root', "Replication Job: $job->{id} failed", $msg)
+ my $dcconf = PVE::Cluster::cfs_read_file('datacenter.cfg');
+ my $target = $dcconf->{notify}->{'target-replication'} // PVE::Notify::default_target();
+ my $notify = $dcconf->{notify}->{'replication'} // 'always';
+
+ if ($notify eq 'always') {
+ PVE::Notify::error(
+ $target,
+ $replication_error_subject_template,
+ $replication_error_body_template,
+ $properties
+ );
+ }
+
};
warn ": $@" if $@;
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 46/66] api: prepare api handler module for notification config
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (44 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 45/66] api: replication: send notifications " Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 47/66] api: notification: add api routes for groups Lukas Wagner
` (22 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
PVE/API2/Cluster.pm | 7 +++
PVE/API2/Cluster/Makefile | 1 +
PVE/API2/Cluster/Notifications.pm | 71 +++++++++++++++++++++++++++++++
3 files changed, 79 insertions(+)
create mode 100644 PVE/API2/Cluster/Notifications.pm
diff --git a/PVE/API2/Cluster.pm b/PVE/API2/Cluster.pm
index 3daf6ae5..2e4cd9cc 100644
--- a/PVE/API2/Cluster.pm
+++ b/PVE/API2/Cluster.pm
@@ -29,6 +29,7 @@ use PVE::API2::Cluster::Ceph;
use PVE::API2::Cluster::Mapping;
use PVE::API2::Cluster::Jobs;
use PVE::API2::Cluster::MetricServer;
+use PVE::API2::Cluster::Notifications;
use PVE::API2::ClusterConfig;
use PVE::API2::Firewall::Cluster;
use PVE::API2::HAConfig;
@@ -52,6 +53,11 @@ __PACKAGE__->register_method ({
path => 'metrics',
});
+__PACKAGE__->register_method ({
+ subclass => "PVE::API2::Cluster::Notifications",
+ path => 'notifications',
+});
+
__PACKAGE__->register_method ({
subclass => "PVE::API2::ClusterConfig",
path => 'config',
@@ -149,6 +155,7 @@ __PACKAGE__->register_method ({
{ name => 'log' },
{ name => 'mapping' },
{ name => 'metrics' },
+ { name => 'notifications' },
{ name => 'nextid' },
{ name => 'options' },
{ name => 'replication' },
diff --git a/PVE/API2/Cluster/Makefile b/PVE/API2/Cluster/Makefile
index 0c52a241..b109e5cb 100644
--- a/PVE/API2/Cluster/Makefile
+++ b/PVE/API2/Cluster/Makefile
@@ -8,6 +8,7 @@ PERLSOURCE= \
BackupInfo.pm \
MetricServer.pm \
Mapping.pm \
+ Notifications.pm \
Jobs.pm \
Ceph.pm
diff --git a/PVE/API2/Cluster/Notifications.pm b/PVE/API2/Cluster/Notifications.pm
new file mode 100644
index 00000000..1efebbc1
--- /dev/null
+++ b/PVE/API2/Cluster/Notifications.pm
@@ -0,0 +1,71 @@
+package PVE::API2::Cluster::Notifications;
+
+use warnings;
+use strict;
+
+use Storable qw(dclone);
+use JSON;
+
+use PVE::Tools qw(extract_param);
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::RESTHandler;
+use PVE::Notify;
+
+use base qw(PVE::RESTHandler);
+
+sub make_properties_optional {
+ my ($properties) = @_;
+ $properties = dclone($properties);
+
+ for my $key (keys %$properties) {
+ $properties->{$key}->{optional} = 1 if $key ne 'name';
+ }
+
+ return $properties;
+}
+
+sub raise_api_error {
+ my ($api_error) = @_;
+
+ if (!(ref($api_error) eq 'HASH' && $api_error->{message} && $api_error->{code})) {
+ die $api_error;
+ }
+
+ my $msg = "$api_error->{message}\n";
+ my $exc = PVE::Exception->new($msg, code => $api_error->{code});
+
+ my (undef, $filename, $line) = caller;
+
+ $exc->{filename} = $filename;
+ $exc->{line} = $line;
+
+ die $exc;
+}
+
+__PACKAGE__->register_method ({
+ name => 'index',
+ path => '',
+ method => 'GET',
+ description => 'Index for notification-related API endpoints.',
+ permissions => { user => 'all' },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => 'object',
+ properties => {},
+ },
+ links => [ { rel => 'child', href => '{name}' } ],
+ },
+ code => sub {
+ my $result = [
+ ];
+
+ return $result;
+ }
+});
+
+1;
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 47/66] api: notification: add api routes for groups
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (45 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 46/66] api: prepare api handler module for notification config Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 48/66] api: notification: add api routes for sendmail endpoints Lukas Wagner
` (21 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
PVE/API2/Cluster/Notifications.pm | 254 ++++++++++++++++++++++++++++++
1 file changed, 254 insertions(+)
diff --git a/PVE/API2/Cluster/Notifications.pm b/PVE/API2/Cluster/Notifications.pm
index 1efebbc1..b1971911 100644
--- a/PVE/API2/Cluster/Notifications.pm
+++ b/PVE/API2/Cluster/Notifications.pm
@@ -62,10 +62,264 @@ __PACKAGE__->register_method ({
},
code => sub {
my $result = [
+ { name => 'groups' },
];
return $result;
}
});
+my $group_properties = {
+ name => {
+ description => 'Name of the group.',
+ type => 'string',
+ format => 'pve-configid',
+ },
+ 'endpoint' => {
+ type => 'array',
+ items => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ description => 'List of included endpoints',
+ },
+ 'comment' => {
+ description => 'Comment',
+ type => 'string',
+ optional => 1,
+ },
+ filter => {
+ description => 'Name of the filter that should be applied.',
+ type => 'string',
+ format => 'pve-configid',
+ optional => 1,
+ },
+};
+
+__PACKAGE__->register_method ({
+ name => 'get_groups',
+ path => 'groups',
+ method => 'GET',
+ description => 'Returns a list of all groups',
+ protected => 1,
+ permissions => {
+ description => "Only lists entries where you have 'Mapping.Modify', 'Mapping.Use' or"
+ . " 'Mapping.Audit' permissions on '/mapping/notification/<name>'.",
+ user => 'all',
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => 'object',
+ properties => $group_properties,
+ },
+ links => [ { rel => 'child', href => '{name}' } ],
+ },
+ code => sub {
+ my $config = PVE::Notify::read_config();
+ my $rpcenv = PVE::RPCEnvironment::get();
+ my $authuser = $rpcenv->get_user();
+ my $can_see_mapping_privs = ['Mapping.Modify', 'Mapping.Use', 'Mapping.Audit'];
+
+ my $groups = [grep {
+ $rpcenv->check_any(
+ $authuser,
+ "/mapping/notification/$_->{name}",
+ $can_see_mapping_privs,
+ 1
+ )
+ } eval {
+ @{$config->get_groups()}
+ }];
+
+ raise_api_error($@) if ($@);
+ return $groups;
+
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'get_group',
+ path => 'groups/{name}',
+ method => 'GET',
+ description => 'Return a specific group',
+ protected => 1,
+ permissions => {
+ check => ['or',
+ ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+ ['perm', '/mapping/notification/{name}', ['Mapping.Audit']],
+ ],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ name => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ }
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ %$group_properties,
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ code => sub {
+ my ($param) = @_;
+ my $name = extract_param($param, 'name');
+
+ my $config = PVE::Notify::read_config();
+
+ my $group = eval {
+ $config->get_group($name)
+ };
+ $group->{digest} = $config->digest();
+
+ raise_api_error($@) if ($@);
+ return $group;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'create_group',
+ path => 'groups',
+ protected => 1,
+ method => 'POST',
+ description => 'Create a new group',
+ permissions => {
+ check => ['perm', '/mapping/notification', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => $group_properties,
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $name = extract_param($param, 'name');
+ my $endpoint = extract_param($param, 'endpoint');
+ my $comment = extract_param($param, 'comment');
+ my $filter = extract_param($param, 'filter');
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+
+ $config->add_group(
+ $name,
+ $endpoint,
+ $comment,
+ $filter,
+ );
+
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if ($@);
+ return;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'update_group',
+ path => 'groups/{name}',
+ protected => 1,
+ method => 'PUT',
+ description => 'Update existing group',
+ permissions => {
+ check => ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ %{ make_properties_optional($group_properties) },
+ delete => {
+ type => 'array',
+ items => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ optional => 1,
+ description => 'A list of settings you want to delete.',
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $name = extract_param($param, 'name');
+ my $endpoint = extract_param($param, 'endpoint');
+ my $comment = extract_param($param, 'comment');
+ my $filter = extract_param($param, 'filter');
+ my $digest = extract_param($param, 'digest');
+ my $delete = extract_param($param, 'delete');
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+
+ $config->update_group(
+ $name,
+ $endpoint,
+ $comment,
+ $filter,
+ $delete,
+ $digest,
+ );
+
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if ($@);
+ return;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'delete_group',
+ protected => 1,
+ path => 'groups/{name}',
+ method => 'DELETE',
+ description => 'Remove group',
+ permissions => {
+ check => ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ name => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ }
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+ my $name = extract_param($param, 'name');
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+ $config->delete_group($name);
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if ($@);
+ return;
+ }
+});
+
1;
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 48/66] api: notification: add api routes for sendmail endpoints
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (46 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 47/66] api: notification: add api routes for groups Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 49/66] api: notification: add api routes for gotify endpoints Lukas Wagner
` (20 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
PVE/API2/Cluster/Notifications.pm | 315 ++++++++++++++++++++++++++++++
1 file changed, 315 insertions(+)
diff --git a/PVE/API2/Cluster/Notifications.pm b/PVE/API2/Cluster/Notifications.pm
index b1971911..aea571f0 100644
--- a/PVE/API2/Cluster/Notifications.pm
+++ b/PVE/API2/Cluster/Notifications.pm
@@ -62,6 +62,7 @@ __PACKAGE__->register_method ({
},
code => sub {
my $result = [
+ { name => 'endpoints' },
{ name => 'groups' },
];
@@ -69,6 +70,33 @@ __PACKAGE__->register_method ({
}
});
+__PACKAGE__->register_method ({
+ name => 'endpoints_index',
+ path => 'endpoints',
+ method => 'GET',
+ description => 'Index for all available endpoint types.',
+ permissions => { user => 'all' },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => 'object',
+ properties => {},
+ },
+ links => [ { rel => 'child', href => '{name}' } ],
+ },
+ code => sub {
+ my $result = [
+ { name => 'sendmail' },
+ ];
+
+ return $result;
+ }
+});
+
my $group_properties = {
name => {
description => 'Name of the group.',
@@ -322,4 +350,291 @@ __PACKAGE__->register_method ({
}
});
+my $sendmail_properties = {
+ name => {
+ description => 'The name of the endpoint.',
+ type => 'string',
+ format => 'pve-configid',
+ },
+ mailto => {
+ type => 'array',
+ items => {
+ type => 'string',
+ format => 'email-or-username',
+ },
+ description => 'List of email recipients',
+ optional => 1,
+ },
+ 'mailto-user' => {
+ type => 'array',
+ items => {
+ type => 'string',
+ format => 'pve-userid',
+ },
+ description => 'List of users',
+ optional => 1,
+ },
+ 'from-address' => {
+ description => '`From` address for the mail',
+ type => 'string',
+ optional => 1,
+ },
+ author => {
+ description => 'Author of the mail',
+ type => 'string',
+ optional => 1,
+ },
+ 'comment' => {
+ description => 'Comment',
+ type => 'string',
+ optional => 1,
+ },
+ filter => {
+ description => 'Name of the filter that should be applied.',
+ type => 'string',
+ format => 'pve-configid',
+ optional => 1,
+ },
+};
+
+__PACKAGE__->register_method ({
+ name => 'get_sendmail_endpoints',
+ path => 'endpoints/sendmail',
+ method => 'GET',
+ description => 'Returns a list of all sendmail endpoints',
+ permissions => {
+ description => "Only lists entries where you have 'Mapping.Modify', 'Mapping.Use' or"
+ . " 'Mapping.Audit' permissions on '/mapping/notification/<name>'.",
+ user => 'all',
+ },
+ protected => 1,
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => 'object',
+ properties => $sendmail_properties,
+ },
+ links => [ { rel => 'child', href => '{name}' } ],
+ },
+ code => sub {
+ my $config = PVE::Notify::read_config();
+ my $rpcenv = PVE::RPCEnvironment::get();
+ my $authuser = $rpcenv->get_user();
+ my $can_see_mapping_privs = ['Mapping.Modify', 'Mapping.Use', 'Mapping.Audit'];
+
+ my $endpoints = [grep {
+ $rpcenv->check_any(
+ $authuser,
+ "/mapping/notification/$_->{name}",
+ $can_see_mapping_privs,
+ 1
+ )
+ } eval {
+ @{$config->get_sendmail_endpoints()}
+ }];
+
+ raise_api_error($@) if ($@);
+ return $endpoints;
+
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'get_sendmail_endpoint',
+ path => 'endpoints/sendmail/{name}',
+ method => 'GET',
+ description => 'Return a specific sendmail endpoint',
+ permissions => {
+ check => ['or',
+ ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+ ['perm', '/mapping/notification/{name}', ['Mapping.Audit']],
+ ],
+ },
+ protected => 1,
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ name => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ }
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ %$sendmail_properties,
+ digest => get_standard_option('pve-config-digest'),
+ }
+
+ },
+ code => sub {
+ my ($param) = @_;
+ my $name = extract_param($param, 'name');
+
+ my $config = PVE::Notify::read_config();
+ my $endpoint = eval {
+ $config->get_sendmail_endpoint($name)
+ };
+
+ $endpoint->{digest} = $config->digest();
+
+ raise_api_error($@) if ($@);
+ return $endpoint;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'create_sendmail_endpoint',
+ path => 'endpoints/sendmail',
+ protected => 1,
+ method => 'POST',
+ description => 'Create a new sendmail endpoint',
+ permissions => {
+ check => ['perm', '/mapping/notification', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => $sendmail_properties,
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $name = extract_param($param, 'name');
+ my $mailto = extract_param($param, 'mailto');
+ my $mailto_user = extract_param($param, 'mailto-user');
+ my $from_address = extract_param($param, 'from-address');
+ my $author = extract_param($param, 'author');
+ my $comment = extract_param($param, 'comment');
+ my $filter = extract_param($param, 'filter');
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+
+ $config->add_sendmail_endpoint(
+ $name,
+ $mailto,
+ $mailto_user,
+ $from_address,
+ $author,
+ $comment,
+ $filter
+ );
+
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if ($@);
+ return;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'update_sendmail_endpoint',
+ path => 'endpoints/sendmail/{name}',
+ protected => 1,
+ method => 'PUT',
+ description => 'Update existing sendmail endpoint',
+ permissions => {
+ check => ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ %{ make_properties_optional($sendmail_properties) },
+ delete => {
+ type => 'array',
+ items => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ optional => 1,
+ description => 'A list of settings you want to delete.',
+ },
+ digest => get_standard_option('pve-config-digest'),
+
+ }
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $name = extract_param($param, 'name');
+ my $mailto = extract_param($param, 'mailto');
+ my $mailto_user = extract_param($param, 'mailto-user');
+ my $from_address = extract_param($param, 'from-address');
+ my $author = extract_param($param, 'author');
+ my $comment = extract_param($param, 'comment');
+ my $filter = extract_param($param, 'filter');
+
+ my $delete = extract_param($param, 'delete');
+ my $digest = extract_param($param, 'digest');
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+
+ $config->update_sendmail_endpoint(
+ $name,
+ $mailto,
+ $mailto_user,
+ $from_address,
+ $author,
+ $comment,
+ $filter,
+ $delete,
+ $digest,
+ );
+
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if ($@);
+ return;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'delete_sendmail_endpoint',
+ protected => 1,
+ path => 'endpoints/sendmail/{name}',
+ method => 'DELETE',
+ description => 'Remove sendmail endpoint',
+ permissions => {
+ check => ['perm', '/mapping/notification', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ name => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ }
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+ $config->delete_sendmail_endpoint($param->{name});
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if ($@);
+ return;
+ }
+});
+
1;
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 49/66] api: notification: add api routes for gotify endpoints
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (47 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 48/66] api: notification: add api routes for sendmail endpoints Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 50/66] api: notification: add api routes for filters Lukas Wagner
` (19 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
PVE/API2/Cluster/Notifications.pm | 271 ++++++++++++++++++++++++++++++
1 file changed, 271 insertions(+)
diff --git a/PVE/API2/Cluster/Notifications.pm b/PVE/API2/Cluster/Notifications.pm
index aea571f0..8f0b6429 100644
--- a/PVE/API2/Cluster/Notifications.pm
+++ b/PVE/API2/Cluster/Notifications.pm
@@ -24,6 +24,19 @@ sub make_properties_optional {
return $properties;
}
+sub remove_protected_properties {
+ my ($properties, $to_remove) = @_;
+ $properties = dclone($properties);
+
+ for my $key (keys %$properties) {
+ if (grep /^$key$/, @$to_remove) {
+ delete $properties->{$key};
+ }
+ }
+
+ return $properties;
+}
+
sub raise_api_error {
my ($api_error) = @_;
@@ -90,6 +103,7 @@ __PACKAGE__->register_method ({
},
code => sub {
my $result = [
+ { name => 'gotify' },
{ name => 'sendmail' },
];
@@ -637,4 +651,261 @@ __PACKAGE__->register_method ({
}
});
+my $gotify_properties = {
+ name => {
+ description => 'The name of the endpoint.',
+ type => 'string',
+ format => 'pve-configid',
+ },
+ 'server' => {
+ description => 'Server URL',
+ type => 'string',
+ },
+ 'token' => {
+ description => 'Secret token',
+ type => 'string',
+ },
+ 'comment' => {
+ description => 'Comment',
+ type => 'string',
+ optional => 1,
+ },
+ 'filter' => {
+ description => 'Name of the filter that should be applied.',
+ type => 'string',
+ format => 'pve-configid',
+ optional => 1,
+ }
+};
+
+__PACKAGE__->register_method ({
+ name => 'get_gotify_endpoints',
+ path => 'endpoints/gotify',
+ method => 'GET',
+ description => 'Returns a list of all gotify endpoints',
+ protected => 1,
+ permissions => {
+ description => "Only lists entries where you have 'Mapping.Modify', 'Mapping.Use' or"
+ . " 'Mapping.Audit' permissions on '/mapping/notification/<name>'.",
+ user => 'all',
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => 'object',
+ properties => remove_protected_properties($gotify_properties, ['token']),
+ },
+ links => [ { rel => 'child', href => '{name}' } ],
+ },
+ code => sub {
+ my $config = PVE::Notify::read_config();
+ my $rpcenv = PVE::RPCEnvironment::get();
+ my $authuser = $rpcenv->get_user();
+ my $can_see_mapping_privs = ['Mapping.Modify', 'Mapping.Use', 'Mapping.Audit'];
+
+ my $endpoints = [grep {
+ $rpcenv->check_any(
+ $authuser,
+ "/mapping/notification/$_->{name}",
+ $can_see_mapping_privs,
+ 1
+ )
+ } eval {
+ @{$config->get_gotify_endpoints()}
+ }];
+
+ raise_api_error($@) if ($@);
+ return $endpoints;
+
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'get_gotify_endpoint',
+ path => 'endpoints/gotify/{name}',
+ method => 'GET',
+ description => 'Return a specific gotify endpoint',
+ protected => 1,
+ permissions => {
+ check => ['or',
+ ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+ ['perm', '/mapping/notification/{name}', ['Mapping.Audit']],
+ ],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ name => {
+ type => 'string',
+ format => 'pve-configid',
+ description => 'Name of the endpoint.'
+ },
+ }
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ %{ remove_protected_properties($gotify_properties, ['token']) },
+ digest => get_standard_option('pve-config-digest'),
+ }
+ },
+ code => sub {
+ my ($param) = @_;
+ my $name = extract_param($param, 'name');
+
+ my $config = PVE::Notify::read_config();
+ my $endpoint = eval {
+ $config->get_gotify_endpoint($name)
+ };
+ $endpoint->{digest} = $config->digest();
+
+ raise_api_error($@) if ($@);
+ return $endpoint;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'create_gotify_endpoint',
+ path => 'endpoints/gotify',
+ protected => 1,
+ method => 'POST',
+ description => 'Create a new gotify endpoint',
+ permissions => {
+ check => ['perm', '/mapping/notification', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => $gotify_properties,
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $name = extract_param($param, 'name');
+ my $server = extract_param($param, 'server');
+ my $token = extract_param($param, 'token');
+ my $comment = extract_param($param, 'comment');
+ my $filter = extract_param($param, 'filter');
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+
+ $config->add_gotify_endpoint(
+ $name,
+ $server,
+ $token,
+ $comment,
+ $filter
+ );
+
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if ($@);
+ return;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'update_gotify_endpoint',
+ path => 'endpoints/gotify/{name}',
+ protected => 1,
+ method => 'PUT',
+ description => 'Update existing gotify endpoint',
+ permissions => {
+ check => ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ %{ make_properties_optional($gotify_properties) },
+ delete => {
+ type => 'array',
+ items => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ optional => 1,
+ description => 'A list of settings you want to delete.',
+ },
+ digest => get_standard_option('pve-config-digest'),
+ }
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $name = extract_param($param, 'name');
+ my $server = extract_param($param, 'server');
+ my $token = extract_param($param, 'token');
+ my $comment = extract_param($param, 'comment');
+ my $filter = extract_param($param, 'filter');
+
+ my $delete = extract_param($param, 'delete');
+ my $digest = extract_param($param, 'digest');
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+
+ $config->update_gotify_endpoint(
+ $name,
+ $server,
+ $token,
+ $comment,
+ $filter,
+ $delete,
+ $digest,
+ );
+
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if ($@);
+ return;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'delete_gotify_endpoint',
+ protected => 1,
+ path => 'endpoints/gotify/{name}',
+ method => 'DELETE',
+ description => 'Remove gotify endpoint',
+ permissions => {
+ check => ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ name => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ }
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+ my $name = extract_param($param, 'name');
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+ $config->delete_gotify_endpoint($name);
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if ($@);
+ return;
+ }
+});
1;
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 50/66] api: notification: add api routes for filters
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (48 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 49/66] api: notification: add api routes for gotify endpoints Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 51/66] api: notification: allow fetching notification targets Lukas Wagner
` (18 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
PVE/API2/Cluster/Notifications.pm | 263 ++++++++++++++++++++++++++++++
1 file changed, 263 insertions(+)
diff --git a/PVE/API2/Cluster/Notifications.pm b/PVE/API2/Cluster/Notifications.pm
index 8f0b6429..e358573c 100644
--- a/PVE/API2/Cluster/Notifications.pm
+++ b/PVE/API2/Cluster/Notifications.pm
@@ -76,6 +76,7 @@ __PACKAGE__->register_method ({
code => sub {
my $result = [
{ name => 'endpoints' },
+ { name => 'filters' },
{ name => 'groups' },
];
@@ -908,4 +909,266 @@ __PACKAGE__->register_method ({
return;
}
});
+
+my $filter_properties = {
+ name => {
+ description => 'Name of the endpoint.',
+ type => 'string',
+ format => 'pve-configid',
+ },
+ 'min-severity' => {
+ type => 'string',
+ description => 'Minimum severity to match',
+ optional => 1,
+ enum => [qw(info notice warning error)],
+ },
+ mode => {
+ type => 'string',
+ description => "Choose between 'and' and 'or' for when multiple properties are specified",
+ optional => 1,
+ enum => [qw(and or)],
+ default => 'and',
+ },
+ 'invert-match' => {
+ type => 'boolean',
+ description => 'Invert match of the whole filter',
+ optional => 1,
+ },
+ 'comment' => {
+ description => 'Comment',
+ type => 'string',
+ optional => 1,
+ },
+};
+
+__PACKAGE__->register_method ({
+ name => 'get_filters',
+ path => 'filters',
+ method => 'GET',
+ description => 'Returns a list of all filters',
+ protected => 1,
+ permissions => {
+ description => "Only lists entries where you have 'Mapping.Modify', 'Mapping.Use' or"
+ . " 'Mapping.Audit' permissions on '/mapping/notification/<name>'.",
+ user => 'all',
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => 'object',
+ properties => $filter_properties,
+ },
+ links => [ { rel => 'child', href => '{name}' } ],
+ },
+ code => sub {
+ my $config = PVE::Notify::read_config();
+ my $rpcenv = PVE::RPCEnvironment::get();
+ my $authuser = $rpcenv->get_user();
+ my $can_see_mapping_privs = ['Mapping.Modify', 'Mapping.Use', 'Mapping.Audit'];
+
+ my $filters = [grep {
+ $rpcenv->check_any(
+ $authuser,
+ "/mapping/notification/$_->{name}",
+ $can_see_mapping_privs,
+ 1
+ )
+ } eval {
+ @{$config->get_filters()}
+ }];
+
+ raise_api_error($@) if ($@);
+ return $filters;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'get_filter',
+ path => 'filters/{name}',
+ method => 'GET',
+ description => 'Return a specific filter',
+ protected => 1,
+ permissions => {
+ check => ['or',
+ ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+ ['perm', '/mapping/notification/{name}', ['Mapping.Audit']],
+ ],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ name => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ }
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ %$filter_properties,
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ code => sub {
+ my ($param) = @_;
+ my $name = extract_param($param, 'name');
+
+ my $config = PVE::Notify::read_config();
+
+ my $filter = eval {
+ $config->get_filter($name)
+ };
+ $filter->{digest} = $config->digest();
+
+ raise_api_error($@) if ($@);
+ return $filter;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'create_filter',
+ path => 'filters',
+ protected => 1,
+ method => 'POST',
+ description => 'Create a new filter',
+ protected => 1,
+ permissions => {
+ check => ['perm', '/mapping/notification', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => $filter_properties,
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $name = extract_param($param, 'name');
+ my $min_severity = extract_param($param, 'min-severity');
+ my $mode = extract_param($param, 'mode');
+ my $invert_match = extract_param($param, 'invert-match');
+ my $comment = extract_param($param, 'comment');
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+
+ $config->add_filter(
+ $name,
+ $min_severity,
+ $mode,
+ $invert_match,
+ $comment,
+ );
+
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if ($@);
+ return;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'update_filter',
+ path => 'filters/{name}',
+ protected => 1,
+ method => 'PUT',
+ description => 'Update existing filter',
+ permissions => {
+ check => ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ %{ make_properties_optional($filter_properties) },
+ delete => {
+ type => 'array',
+ items => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ optional => 1,
+ description => 'A list of settings you want to delete.',
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $name = extract_param($param, 'name');
+ my $min_severity = extract_param($param, 'min-severity');
+ my $mode = extract_param($param, 'mode');
+ my $invert_match = extract_param($param, 'invert-match');
+ my $comment = extract_param($param, 'comment');
+ my $digest = extract_param($param, 'digest');
+ my $delete = extract_param($param, 'delete');
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+
+ $config->update_filter(
+ $name,
+ $min_severity,
+ $mode,
+ $invert_match,
+ $comment,
+ $delete,
+ $digest,
+ );
+
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if ($@);
+ return;
+ }
+});
+
+__PACKAGE__->register_method ({
+ name => 'delete_filter',
+ protected => 1,
+ path => 'filters/{name}',
+ method => 'DELETE',
+ description => 'Remove filter',
+ permissions => {
+ check => ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ name => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ }
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+ my $name = extract_param($param, 'name');
+
+ eval {
+ PVE::Notify::lock_config(sub {
+ my $config = PVE::Notify::read_config();
+ $config->delete_filter($name);
+ PVE::Notify::write_config($config);
+ });
+ };
+
+ raise_api_error($@) if ($@);
+ return;
+ }
+});
+
1;
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 51/66] api: notification: allow fetching notification targets
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (49 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 50/66] api: notification: add api routes for filters Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 52/66] api: notification: allow to test targets Lukas Wagner
` (17 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
PVE/API2/Cluster/Notifications.pm | 100 ++++++++++++++++++++++++++++++
1 file changed, 100 insertions(+)
diff --git a/PVE/API2/Cluster/Notifications.pm b/PVE/API2/Cluster/Notifications.pm
index e358573c..32a873a7 100644
--- a/PVE/API2/Cluster/Notifications.pm
+++ b/PVE/API2/Cluster/Notifications.pm
@@ -78,6 +78,7 @@ __PACKAGE__->register_method ({
{ name => 'endpoints' },
{ name => 'filters' },
{ name => 'groups' },
+ { name => 'targets' },
];
return $result;
@@ -112,6 +113,105 @@ __PACKAGE__->register_method ({
}
});
+__PACKAGE__->register_method ({
+ name => 'get_all_targets',
+ path => 'targets',
+ method => 'GET',
+ description => 'Returns a list of all entities that can be used as notification targets' .
+ ' (endpoints and groups).',
+ permissions => {
+ description => "Only lists entries where you have 'Mapping.Modify', 'Mapping.Use' or"
+ . " 'Mapping.Audit' permissions on '/mapping/notification/<name>'.",
+ user => 'all',
+ },
+ protected => 1,
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => 'object',
+ properties => {
+ name => {
+ description => 'Name of the endpoint/group.',
+ type => 'string',
+ format => 'pve-configid',
+ },
+ 'type' => {
+ description => 'Type of the endpoint or group.',
+ type => 'string',
+ enum => [qw(sendmail gotify group)],
+ },
+ 'comment' => {
+ description => 'Comment',
+ type => 'string',
+ optional => 1,
+ },
+ },
+ },
+ links => [ { rel => 'child', href => '{name}' } ],
+ },
+ code => sub {
+ my $config = PVE::Notify::read_config();
+ my $rpcenv = PVE::RPCEnvironment::get();
+ my $authuser = $rpcenv->get_user();
+ my $can_see_mapping_privs = ['Mapping.Modify', 'Mapping.Use', 'Mapping.Audit'];
+
+ my $endpoints = eval {
+ my $result = [];
+
+ for my $endpoint (@{$config->get_sendmail_endpoints()}) {
+ next if !$rpcenv->check_any(
+ $authuser,
+ "/mapping/notification/$endpoint->{name}",
+ $can_see_mapping_privs,
+ 1
+ );
+ push @$result, {
+ name => $endpoint->{name},
+ comment => $endpoint->{comment},
+ type => 'sendmail',
+ };
+ }
+
+ for my $endpoint (@{$config->get_gotify_endpoints()}) {
+ next if !$rpcenv->check_any(
+ $authuser,
+ "/mapping/notification/$endpoint->{name}",
+ $can_see_mapping_privs,
+ 1
+ );
+ push @$result, {
+ name => $endpoint->{name},
+ comment => $endpoint->{comment},
+ type => 'gotify',
+ };
+ }
+
+ for my $endpoint (@{$config->get_groups()}) {
+ next if !$rpcenv->check_any(
+ $authuser,
+ "/mapping/notification/$endpoint->{name}",
+ $can_see_mapping_privs,
+ 1
+ );
+ push @$result, {
+ name => $endpoint->{name},
+ comment => $endpoint->{comment},
+ type => 'group',
+ };
+ }
+
+ $result
+ };
+
+ raise_api_error($@) if ($@);
+ return $endpoints;
+ }
+});
+
my $group_properties = {
name => {
description => 'Name of the group.',
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 52/66] api: notification: allow to test targets
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (50 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 51/66] api: notification: allow fetching notification targets Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 53/66] api: notification: disallow removing targets if they are used Lukas Wagner
` (16 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
PVE/API2/Cluster/Notifications.pm | 40 +++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
diff --git a/PVE/API2/Cluster/Notifications.pm b/PVE/API2/Cluster/Notifications.pm
index 32a873a7..fa2c1d9d 100644
--- a/PVE/API2/Cluster/Notifications.pm
+++ b/PVE/API2/Cluster/Notifications.pm
@@ -212,6 +212,46 @@ __PACKAGE__->register_method ({
}
});
+__PACKAGE__->register_method ({
+ name => 'test_target',
+ path => 'targets/{name}/test',
+ protected => 1,
+ method => 'POST',
+ description => 'Send a test notification to a provided target.',
+ permissions => {
+ check => ['or',
+ ['perm', '/mapping/notification/{name}', ['Mapping.Use']],
+ ['perm', '/mapping/notification/{name}', ['Mapping.Modify']],
+ ['perm', '/mapping/notification/{name}', ['Mapping.Audit']],
+ ],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ name => {
+ description => 'Name of the target.',
+ type => 'string',
+ format => 'pve-configid'
+ },
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+ my $name = extract_param($param, 'name');
+
+ my $config = PVE::Notify::read_config();
+
+ eval {
+ $config->test_target($name);
+ };
+
+ raise_api_error($@) if ($@);
+
+ return;
+ }
+});
+
my $group_properties = {
name => {
description => 'Name of the group.',
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 53/66] api: notification: disallow removing targets if they are used
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (51 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 52/66] api: notification: allow to test targets Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 54/66] ui: backup: allow to select notification target for jobs Lukas Wagner
` (15 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Check notification targets configured in datacenter.cfg and jobs.cfg,
failing if the group/endpoint to be removed is still in use there.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
PVE/API2/Cluster/Notifications.pm | 44 ++++++++++++++++++++++++++++++-
1 file changed, 43 insertions(+), 1 deletion(-)
diff --git a/PVE/API2/Cluster/Notifications.pm b/PVE/API2/Cluster/Notifications.pm
index fa2c1d9d..cf913452 100644
--- a/PVE/API2/Cluster/Notifications.pm
+++ b/PVE/API2/Cluster/Notifications.pm
@@ -6,6 +6,7 @@ use strict;
use Storable qw(dclone);
use JSON;
+use PVE::Exception qw(raise_param_exc);
use PVE::Tools qw(extract_param);
use PVE::JSONSchema qw(get_standard_option);
use PVE::RESTHandler;
@@ -55,6 +56,31 @@ sub raise_api_error {
die $exc;
}
+sub target_used_by {
+ my ($target) = @_;
+
+ my $used_by = [];
+
+ # Check keys in datacenter.cfg
+ my $dc_conf = PVE::Cluster::cfs_read_file('datacenter.cfg');
+ for my $key (qw(target-package-updates target-replication target-fencing)) {
+ if ($dc_conf->{notify} && $dc_conf->{notify}->{$key} eq $target) {
+ push @$used_by, $key;
+ }
+ }
+
+ # Check backup jobs
+ my $jobs_conf = PVE::Cluster::cfs_read_file('jobs.cfg');
+ for my $key (keys %{$jobs_conf->{ids}}) {
+ my $job = $jobs_conf->{ids}->{$key};
+ if ($job->{'notification-target'} eq $target) {
+ push @$used_by, $key;
+ }
+ }
+
+ return join(', ', @$used_by);
+}
+
__PACKAGE__->register_method ({
name => 'index',
path => '',
@@ -492,6 +518,11 @@ __PACKAGE__->register_method ({
my ($param) = @_;
my $name = extract_param($param, 'name');
+ my $used_by = target_used_by($name);
+ if ($used_by) {
+ raise_param_exc({'name' => "Cannot remove $name, used by: $used_by"});
+ }
+
eval {
PVE::Notify::lock_config(sub {
my $config = PVE::Notify::read_config();
@@ -778,11 +809,17 @@ __PACKAGE__->register_method ({
returns => { type => 'null' },
code => sub {
my ($param) = @_;
+ my $name = extract_param($param, 'name');
+
+ my $used_by = target_used_by($name);
+ if ($used_by) {
+ raise_param_exc({'name' => "Cannot remove $name, used by: $used_by"});
+ }
eval {
PVE::Notify::lock_config(sub {
my $config = PVE::Notify::read_config();
- $config->delete_sendmail_endpoint($param->{name});
+ $config->delete_sendmail_endpoint($name);
PVE::Notify::write_config($config);
});
};
@@ -1037,6 +1074,11 @@ __PACKAGE__->register_method ({
my ($param) = @_;
my $name = extract_param($param, 'name');
+ my $used_by = target_used_by($name);
+ if ($used_by) {
+ raise_param_exc({'name' => "Cannot remove $name, used by: $used_by"});
+ }
+
eval {
PVE::Notify::lock_config(sub {
my $config = PVE::Notify::read_config();
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 54/66] ui: backup: allow to select notification target for jobs
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (52 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 53/66] api: notification: disallow removing targets if they are used Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-19 12:20 ` Dominik Csapak
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 55/66] ui: backup: adapt backup job details to new notification params Lukas Wagner
` (14 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
This commit adds a possibility to choose between different options
for notifications for backup jobs:
- Notify via email, in the same manner as before
- Notify via an endpoint/group
If 'notify via mail' is selected, a text field where an email address
can be entered is displayed:
Notify: | Always notify v |
Notify via: | E-Mail v |
Send Mail to: | foo@example.com |
Compression: | ..... v |
If the other option is selected selected, a combo picker for selecting
a channel is displayed:
Notify: | Always notify v |
Notify via: | Endpoint/Group v |
Target: | endpoint-foo v |
Compression: | ..... v |
The code has also been adapted to use the newly introduced
'notification-policy' parameter, which replaces the 'mailnotification'
paramter for backup jobs. Some logic which automatically migrates from
'mailnotification' has been added.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
www/manager6/Makefile | 4 +-
www/manager6/dc/Backup.js | 84 +++++++++++++++++--
www/manager6/form/NotificationModeSelector.js | 8 ++
...ector.js => NotificationPolicySelector.js} | 1 +
.../form/NotificationTargetSelector.js | 54 ++++++++++++
5 files changed, 143 insertions(+), 8 deletions(-)
create mode 100644 www/manager6/form/NotificationModeSelector.js
rename www/manager6/form/{EmailNotificationSelector.js => NotificationPolicySelector.js} (87%)
create mode 100644 www/manager6/form/NotificationTargetSelector.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 5b455c80..140b20f0 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -36,7 +36,6 @@ JSSRC= \
form/DayOfWeekSelector.js \
form/DiskFormatSelector.js \
form/DiskStorageSelector.js \
- form/EmailNotificationSelector.js \
form/FileSelector.js \
form/FirewallPolicySelector.js \
form/GlobalSearchField.js \
@@ -51,6 +50,9 @@ JSSRC= \
form/MultiPCISelector.js \
form/NetworkCardSelector.js \
form/NodeSelector.js \
+ form/NotificationModeSelector.js \
+ form/NotificationTargetSelector.js \
+ form/NotificationPolicySelector.js \
form/PCISelector.js \
form/PCIMapSelector.js \
form/PermPathSelector.js \
diff --git a/www/manager6/dc/Backup.js b/www/manager6/dc/Backup.js
index 03a02651..625b5430 100644
--- a/www/manager6/dc/Backup.js
+++ b/www/manager6/dc/Backup.js
@@ -36,6 +36,30 @@ Ext.define('PVE.dc.BackupEdit', {
delete values.node;
}
+ if (!isCreate) {
+ // 'mailnotification' is deprecated in favor of 'notification-policy'
+ // -> Migration to the new parameter happens in init, so we are
+ // safe to remove the old parameter here.
+ Proxmox.Utils.assemble_field_data(values, { 'delete': 'mailnotification' });
+
+ // If sending notifications via mail, remove the current value of
+ // 'notification-target'
+ if (values['notification-mode'] === "mailto") {
+ Proxmox.Utils.assemble_field_data(
+ values,
+ { 'delete': 'notification-target' },
+ );
+ } else {
+ // and vice versa...
+ Proxmox.Utils.assemble_field_data(
+ values,
+ { 'delete': 'mailto' },
+ );
+ }
+ }
+
+ delete values['notification-mode'];
+
if (!values.id && isCreate) {
values.id = 'backup-' + Ext.data.identifier.Uuid.Global.generate().slice(0, 13);
}
@@ -146,6 +170,22 @@ Ext.define('PVE.dc.BackupEdit', {
success: function(response, _options) {
let data = response.result.data;
+ // 'mailnotification' is deprecated. Let's automatically
+ // migrate to the compatible 'notification-policy' parameter
+ if (data.mailnotification) {
+ if (!data["notification-policy"]) {
+ data["notification-policy"] = data.mailnotification;
+ }
+
+ delete data.mailnotification;
+ }
+
+ if (data['notification-target']) {
+ data['notification-mode'] = 'notification-target';
+ } else if (data.mailto) {
+ data['notification-mode'] = 'mailto';
+ }
+
if (data.exclude) {
data.vmid = data.exclude;
data.selMode = 'exclude';
@@ -188,11 +228,13 @@ Ext.define('PVE.dc.BackupEdit', {
viewModel: {
data: {
selMode: 'include',
+ notificationMode: 'notification-target',
},
formulas: {
poolMode: (get) => get('selMode') === 'pool',
disableVMSelection: (get) => get('selMode') !== 'include' && get('selMode') !== 'exclude',
+ mailNotificationSelected: (get) => get('notificationMode') === 'mailto',
},
},
@@ -282,20 +324,48 @@ Ext.define('PVE.dc.BackupEdit', {
},
],
column2: [
- {
- xtype: 'textfield',
- fieldLabel: gettext('Send email to'),
- name: 'mailto',
- },
{
xtype: 'pveEmailNotificationSelector',
- fieldLabel: gettext('Email'),
- name: 'mailnotification',
+ fieldLabel: gettext('Notify'),
+ name: 'notification-policy',
cbind: {
value: (get) => get('isCreate') ? 'always' : '',
deleteEmpty: '{!isCreate}',
},
},
+ {
+ xtype: 'pveNotificationModeSelector',
+ fieldLabel: gettext('Notify via'),
+ name: 'notification-mode',
+ bind: {
+ value: '{notificationMode}',
+ },
+ },
+ {
+ xtype: 'pveNotificationTargetSelector',
+ fieldLabel: gettext('Notification Target'),
+ name: 'notification-target',
+ allowBlank: true,
+ editable: true,
+ autoSelect: false,
+ bind: {
+ hidden: '{mailNotificationSelected}',
+ disabled: '{mailNotificationSelected}',
+ },
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
+ {
+ xtype: 'textfield',
+ fieldLabel: gettext('Send email to'),
+ name: 'mailto',
+ hidden: true,
+ bind: {
+ hidden: '{!mailNotificationSelected}',
+ disabled: '{!mailNotificationSelected}',
+ },
+ },
{
xtype: 'pveCompressionSelector',
reference: 'compressionSelector',
diff --git a/www/manager6/form/NotificationModeSelector.js b/www/manager6/form/NotificationModeSelector.js
new file mode 100644
index 00000000..58fddd56
--- /dev/null
+++ b/www/manager6/form/NotificationModeSelector.js
@@ -0,0 +1,8 @@
+Ext.define('PVE.form.NotificationModeSelector', {
+ extend: 'Proxmox.form.KVComboBox',
+ alias: ['widget.pveNotificationModeSelector'],
+ comboItems: [
+ ['notification-target', gettext('Target')],
+ ['mailto', gettext('E-Mail')],
+ ],
+});
diff --git a/www/manager6/form/EmailNotificationSelector.js b/www/manager6/form/NotificationPolicySelector.js
similarity index 87%
rename from www/manager6/form/EmailNotificationSelector.js
rename to www/manager6/form/NotificationPolicySelector.js
index f318ea18..68087275 100644
--- a/www/manager6/form/EmailNotificationSelector.js
+++ b/www/manager6/form/NotificationPolicySelector.js
@@ -4,5 +4,6 @@ Ext.define('PVE.form.EmailNotificationSelector', {
comboItems: [
['always', gettext('Notify always')],
['failure', gettext('On failure only')],
+ ['never', gettext('Notify never')],
],
});
diff --git a/www/manager6/form/NotificationTargetSelector.js b/www/manager6/form/NotificationTargetSelector.js
new file mode 100644
index 00000000..9ead28e7
--- /dev/null
+++ b/www/manager6/form/NotificationTargetSelector.js
@@ -0,0 +1,54 @@
+Ext.define('PVE.form.NotificationTargetSelector', {
+ extend: 'Proxmox.form.ComboGrid',
+ alias: ['widget.pveNotificationTargetSelector'],
+
+ // set default value to empty array, else it inits it with
+ // null and after the store load it is an empty array,
+ // triggering dirtychange
+ value: [],
+ valueField: 'name',
+ displayField: 'name',
+ deleteEmpty: true,
+ skipEmptyText: true,
+
+ store: {
+ fields: ['name', 'type', 'comment'],
+ proxy: {
+ type: 'proxmox',
+ url: '/api2/json/cluster/notifications/targets',
+ },
+ sorters: [
+ {
+ property: 'name',
+ direction: 'ASC',
+ },
+ ],
+ autoLoad: true,
+ },
+
+ listConfig: {
+ columns: [
+ {
+ header: gettext('Target'),
+ dataIndex: 'name',
+ sortable: true,
+ hideable: false,
+ flex: 1,
+ },
+ {
+ header: gettext('Type'),
+ dataIndex: 'type',
+ sortable: true,
+ hideable: false,
+ flex: 1,
+ },
+ {
+ header: gettext('Comment'),
+ dataIndex: 'comment',
+ sortable: true,
+ hideable: false,
+ flex: 2,
+ },
+ ],
+ },
+});
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 55/66] ui: backup: adapt backup job details to new notification params
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (53 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 54/66] ui: backup: allow to select notification target for jobs Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 56/66] ui: backup: allow to set notification-target for one-off backups Lukas Wagner
` (13 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
www/manager6/dc/BackupJobDetail.js | 20 ++++++++++++++++----
1 file changed, 16 insertions(+), 4 deletions(-)
diff --git a/www/manager6/dc/BackupJobDetail.js b/www/manager6/dc/BackupJobDetail.js
index c4683a47..8b9bb749 100644
--- a/www/manager6/dc/BackupJobDetail.js
+++ b/www/manager6/dc/BackupJobDetail.js
@@ -202,15 +202,27 @@ Ext.define('PVE.dc.BackupInfo', {
column2: [
{
xtype: 'displayfield',
- name: 'mailnotification',
+ name: 'notification-policy',
fieldLabel: gettext('Notification'),
renderer: function(value) {
- let mailto = this.up('pveBackupInfo')?.record?.mailto || 'root@localhost';
+ let record = this.up('pveBackupInfo')?.record;
+
+ // Fall back to old value, in case this option is not migrated yet.
+ let policy = value || record?.mailnotification || 'always';
+
let when = gettext('Always');
- if (value === 'failure') {
+ if (policy === 'failure') {
when = gettext('On failure only');
+ } else if (policy === 'never') {
+ when = gettext('Never');
}
- return `${when} (${mailto})`;
+
+ // Notification-target takes precedence
+ let target = record?.['notification-target'] ||
+ record?.mailto ||
+ gettext('No target configured');
+
+ return `${when} (${target})`;
},
},
{
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 56/66] ui: backup: allow to set notification-target for one-off backups
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (54 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 55/66] ui: backup: adapt backup job details to new notification params Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 57/66] ui: allow to configure notification event -> target mapping Lukas Wagner
` (12 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
In essence the same change as for backup jobs.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
www/manager6/window/Backup.js | 35 ++++++++++++++++++++++++++++++++++-
1 file changed, 34 insertions(+), 1 deletion(-)
diff --git a/www/manager6/window/Backup.js b/www/manager6/window/Backup.js
index 4b21c746..17a37e12 100644
--- a/www/manager6/window/Backup.js
+++ b/www/manager6/window/Backup.js
@@ -30,12 +30,32 @@ Ext.define('PVE.window.Backup', {
name: 'mode',
});
+ let notificationTargetSelector = Ext.create('PVE.form.NotificationTargetSelector', {
+ fieldLabel: gettext('Notification target'),
+ name: 'notification-target',
+ emptyText: Proxmox.Utils.noneText,
+ hidden: true,
+ });
+
let mailtoField = Ext.create('Ext.form.field.Text', {
fieldLabel: gettext('Send email to'),
name: 'mailto',
emptyText: Proxmox.Utils.noneText,
});
+ let notificationModeSelector = Ext.create('PVE.form.NotificationModeSelector', {
+ fieldLabel: gettext('Notify via'),
+ value: 'mailto',
+ name: 'notification-mode',
+ listeners: {
+ change: function(f, v) {
+ let mailSelected = v === 'mailto';
+ notificationTargetSelector.setHidden(mailSelected);
+ mailtoField.setHidden(!mailSelected);
+ },
+ },
+ });
+
const keepNames = [
['keep-last', gettext('Keep Last')],
['keep-hourly', gettext('Keep Hourly')],
@@ -107,6 +127,12 @@ Ext.define('PVE.window.Backup', {
success: function(response, opts) {
const data = response.result.data;
+ if (!initialDefaults && data['notification-mode'] !== undefined) {
+ notificationModeSelector.setValue(data['notification-mode']);
+ }
+ if (!initialDefaults && data['notification-channel'] !== undefined) {
+ notificationTargetSelector.setValue(data['notification-channel']);
+ }
if (!initialDefaults && data.mailto !== undefined) {
mailtoField.setValue(data.mailto);
}
@@ -176,6 +202,8 @@ Ext.define('PVE.window.Backup', {
],
column2: [
compressionSelector,
+ notificationModeSelector,
+ notificationTargetSelector,
mailtoField,
removeCheckbox,
],
@@ -252,10 +280,15 @@ Ext.define('PVE.window.Backup', {
remove: values.remove,
};
- if (values.mailto) {
+ if (values.mailto && values['notification-mode'] === 'mailto') {
params.mailto = values.mailto;
}
+ if (values['notification-target'] &&
+ values['notification-mode'] === 'notification-target') {
+ params['notification-target'] = values['notification-target'];
+ }
+
if (values.compress) {
params.compress = values.compress;
}
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 57/66] ui: allow to configure notification event -> target mapping
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (55 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 56/66] ui: backup: allow to set notification-target for one-off backups Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-19 12:45 ` Dominik Csapak
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 58/66] ui: add notification target configuration panel Lukas Wagner
` (11 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
www/manager6/Makefile | 1 +
www/manager6/dc/Config.js | 12 ++
www/manager6/dc/NotificationEvents.js | 238 ++++++++++++++++++++++++++
3 files changed, 251 insertions(+)
create mode 100644 www/manager6/dc/NotificationEvents.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 140b20f0..452abbd4 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -158,6 +158,7 @@ JSSRC= \
dc/Health.js \
dc/Log.js \
dc/NodeView.js \
+ dc/NotificationEvents.js \
dc/OptionView.js \
dc/PermissionView.js \
dc/PoolEdit.js \
diff --git a/www/manager6/dc/Config.js b/www/manager6/dc/Config.js
index 04ed04f0..aa025c8d 100644
--- a/www/manager6/dc/Config.js
+++ b/www/manager6/dc/Config.js
@@ -317,6 +317,18 @@ Ext.define('PVE.dc.Config', {
);
}
+ if (caps.dc['Sys.Audit']) {
+ me.items.push(
+ {
+ xtype: 'pveNotificationEvents',
+ title: gettext('Notifications'),
+ onlineHelp: 'notification_events',
+ iconCls: 'fa fa-bell-o',
+ itemId: 'notifications',
+ },
+ );
+ }
+
if (caps.dc['Sys.Audit']) {
me.items.push({
xtype: 'pveDcSupport',
diff --git a/www/manager6/dc/NotificationEvents.js b/www/manager6/dc/NotificationEvents.js
new file mode 100644
index 00000000..8ba0a844
--- /dev/null
+++ b/www/manager6/dc/NotificationEvents.js
@@ -0,0 +1,238 @@
+Ext.define('PVE.dc.NotificationEventsPolicySelector', {
+ alias: ['widget.pveNotificationEventsPolicySelector'],
+ extend: 'Proxmox.form.KVComboBox',
+ deleteEmpty: false,
+ value: '__default__',
+ comboItems: [
+ ['__default__', `${Proxmox.Utils.defaultText} (always)`],
+ ['always', gettext('Always')],
+ ['never', gettext('Never')],
+ ],
+ defaultValue: '__default__',
+});
+
+Ext.define('PVE.dc.NotificationEventsTargetSelector', {
+ alias: ['widget.pveNotificationEventsTargetSelector'],
+ extend: 'PVE.form.NotificationTargetSelector',
+ fieldLabel: gettext('Notification Target'),
+ allowBlank: true,
+ editable: true,
+ autoSelect: false,
+ deleteEmpty: false,
+ emptyText: `${Proxmox.Utils.defaultText} (${gettext("mail-to-root")})`,
+});
+
+Ext.define('PVE.dc.NotificationEvents', {
+ extend: 'Proxmox.grid.ObjectGrid',
+ alias: ['widget.pveNotificationEvents'],
+
+ // Taken from OptionView.js, but adapted slightly.
+ addInputPanelRow: function(name, propertyName, text, opts) {
+ let me = this;
+
+ opts = opts || {};
+ me.rows = me.rows || {};
+
+ me.rows[name] = {
+ required: true,
+ defaultValue: opts.defaultValue,
+ header: text,
+ renderer: opts.renderer,
+ name: propertyName,
+ editor: {
+ xtype: 'proxmoxWindowEdit',
+ width: opts.width || 400,
+ subject: text,
+ onlineHelp: opts.onlineHelp,
+ fieldDefaults: {
+ labelWidth: opts.labelWidth || 150,
+ },
+ setValues: function(values) {
+ let value = values[propertyName];
+
+ if (opts.parseBeforeSet) {
+ value = PVE.Parser.parsePropertyString(value);
+ }
+
+ Ext.Array.each(this.query('inputpanel'), function(panel) {
+ panel.setValues(value);
+ panel.originalValue = {
+ ...value,
+ };
+ });
+ },
+ url: opts.url,
+ items: [{
+ xtype: 'inputpanel',
+ onGetValues: function(values) {
+ let fields = this.config.items.map(field => field.name).filter(n => n);
+
+ for (const [key, value] of Object.entries(this.originalValue)) {
+ if (!fields.includes(key)) {
+ values[key] = value;
+ }
+ }
+
+ let value = {};
+ if (Object.keys(values).length > 0) {
+ value[propertyName] = PVE.Parser.printPropertyString(values);
+ } else {
+ Proxmox.Utils.assemble_field_data(value, { 'delete': propertyName });
+ }
+
+ return value;
+ },
+ items: opts.items,
+ }],
+ },
+ };
+ },
+
+ initComponent: function() {
+ let me = this;
+
+ // Helper function for rendering the property
+ // Needed since the actual value is always stored in the 'notify' property
+ let render_value = (store, target_key, mode_key) => {
+ let value = store.getById('notify')?.get('value') ?? {};
+ let target = value[target_key] ?? gettext('mail-to-root');
+ let template;
+
+ switch (value[mode_key]) {
+ case 'always':
+ template = gettext('Always, notify via target \'{0}\'');
+ break;
+ case 'never':
+ template = gettext('Never');
+ break;
+ default:
+ template = gettext('{1} (Always), notify via target \'{0}\'');
+ break;
+ }
+
+ return Ext.String.format(template, target, Proxmox.Utils.defaultText);
+ };
+
+ me.addInputPanelRow('fencing', 'notify', gettext('Node Fencing'), {
+ renderer: (value, metaData, record, rowIndex, colIndex, store) =>
+ render_value(store, 'target-fencing', 'fencing'),
+ url: "/api2/extjs/cluster/options",
+ items: [
+ {
+ xtype: 'pveNotificationEventsPolicySelector',
+ name: 'fencing',
+ fieldLabel: gettext('Notify'),
+ },
+ {
+ xtype: 'pveNotificationEventsTargetSelector',
+ name: 'target-fencing',
+ },
+ {
+ xtype: 'displayfield',
+ userCls: 'pmx-hint',
+ value: gettext('Disabling notifications is not ' +
+ 'recommended for production systems!'),
+ },
+ ],
+ });
+
+ me.addInputPanelRow('replication', 'notify', gettext('Replication'), {
+ renderer: (value, metaData, record, rowIndex, colIndex, store) =>
+ render_value(store, 'target-replication', 'replication'),
+ url: "/api2/extjs/cluster/options",
+ items: [
+ {
+ xtype: 'pveNotificationEventsPolicySelector',
+ name: 'replication',
+ fieldLabel: gettext('Notify'),
+ },
+ {
+ xtype: 'pveNotificationEventsTargetSelector',
+ name: 'target-replication',
+ },
+ {
+ xtype: 'displayfield',
+ userCls: 'pmx-hint',
+ value: gettext('Disabling notifications is not ' +
+ 'recommended for production systems!'),
+ },
+ ],
+ });
+
+ me.addInputPanelRow('updates', 'notify', gettext('Package Updates'), {
+ renderer: (value, metaData, record, rowIndex, colIndex, store) => {
+ value = store.getById('notify')?.get('value') ?? {};
+ let target = value['target-package-updates'] ?? gettext('mail-to-root');
+ let template;
+
+ switch (value['package-updates']) {
+ case 'always':
+ template = gettext('Always, notify via \'{0}\'');
+ break;
+ case 'auto':
+ template = gettext('Automatically, notify via target \'{0}\'');
+ break;
+ case 'never':
+ template = gettext('Never');
+ break;
+ default:
+ template = gettext('{1} (Automatically), notify via target \'{0}\'');
+ break;
+ }
+
+ return Ext.String.format(template, target, Proxmox.Utils.defaultText);
+ },
+ url: "/api2/extjs/cluster/options",
+ items: [
+ {
+ xtype: 'pveNotificationEventsPolicySelector',
+ name: 'package-updates',
+ fieldLabel: gettext('Notify'),
+ comboItems: [
+ ['__default__', Proxmox.Utils.defaultText + ' (auto)'],
+ ['auto', gettext('Automatically')],
+ ['always', gettext('Always')],
+ ['never', gettext('Never')],
+ ],
+ },
+ {
+ xtype: 'pveNotificationEventsTargetSelector',
+ name: 'target-package-updates',
+ },
+ ],
+ });
+
+ // Hack: Also load the notify property to make it accessible
+ // for our render functions. initComponents later hides it.
+ me.add_text_row('notify', gettext('Notify'), {});
+
+ me.selModel = Ext.create('Ext.selection.RowModel', {});
+
+ Ext.apply(me, {
+ tbar: [{
+ text: gettext('Edit'),
+ xtype: 'proxmoxButton',
+ disabled: true,
+ handler: () => me.run_editor(),
+ selModel: me.selModel,
+ }],
+ url: "/api2/json/cluster/options",
+ editorConfig: {
+ url: "/api2/extjs/cluster/options",
+ },
+ interval: 5000,
+ cwidth1: 200,
+ listeners: {
+ itemdblclick: me.run_editor,
+ },
+ });
+
+ me.callParent();
+
+ me.rows.notify.visible = false;
+
+ me.on('activate', me.rstore.startUpdate);
+ me.on('destroy', me.rstore.stopUpdate);
+ me.on('deactivate', me.rstore.stopUpdate);
+ },
+});
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 58/66] ui: add notification target configuration panel
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (56 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 57/66] ui: allow to configure notification event -> target mapping Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 59/66] ui: perm path: load notification target/filter acl entries Lukas Wagner
` (10 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
www/manager6/dc/Config.js | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/www/manager6/dc/Config.js b/www/manager6/dc/Config.js
index aa025c8d..9ba7b301 100644
--- a/www/manager6/dc/Config.js
+++ b/www/manager6/dc/Config.js
@@ -329,6 +329,22 @@ Ext.define('PVE.dc.Config', {
);
}
+ if (caps.mapping['Mapping.Audit'] ||
+ caps.mapping['Mapping.Use'] ||
+ caps.mapping['Mapping.Modify']) {
+ me.items.push(
+ {
+ xtype: 'pmxNotificationConfigView',
+ title: gettext('Notification Targets'),
+ onlineHelp: 'notification_targets',
+ itemId: 'notification-targets',
+ iconCls: 'fa fa-dot-circle-o',
+ baseUrl: '/cluster/notifications',
+ groups: ['notifications'],
+ },
+ );
+ }
+
if (caps.dc['Sys.Audit']) {
me.items.push({
xtype: 'pveDcSupport',
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 59/66] ui: perm path: load notification target/filter acl entries
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (57 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 58/66] ui: add notification target configuration panel Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-19 12:53 ` Dominik Csapak
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 60/66] ui: perm path: increase width of the perm path selector combobox Lukas Wagner
` (9 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
Notes:
I'm not sure if I like this solution, but adding notification targets to
the resources API endpoint would not have make sense.
Maybe we could create a new API endpoint that returns all possible ACL
paths and then use a normal store for the perm path combobox?
www/manager6/data/PermPathStore.js | 26 +++++++++++++++++++++++++-
1 file changed, 25 insertions(+), 1 deletion(-)
diff --git a/www/manager6/data/PermPathStore.js b/www/manager6/data/PermPathStore.js
index c3ac7f0e..b7e4fa33 100644
--- a/www/manager6/data/PermPathStore.js
+++ b/www/manager6/data/PermPathStore.js
@@ -9,6 +9,7 @@ Ext.define('PVE.data.PermPathStore', {
{ 'value': '/access/groups' },
{ 'value': '/access/realm' },
{ 'value': '/mapping' },
+ { 'value': '/mapping/notification' },
{ 'value': '/nodes' },
{ 'value': '/pool' },
{ 'value': '/sdn/zones' },
@@ -46,8 +47,31 @@ Ext.define('PVE.data.PermPathStore', {
donePaths[path] = 1;
}
});
- me.resumeEvents();
+ Ext.Ajax.request({
+ url: '/api2/json/cluster/notifications/targets',
+ method: 'GET',
+ success: function(response, opts) {
+ let obj = Ext.decode(response.responseText);
+
+ for (let target of obj.data) {
+ me.add({ value: `/mapping/notification/${target.name}` });
+ }
+ },
+ });
+ Ext.Ajax.request({
+ url: '/api2/json/cluster/notifications/filters',
+ method: 'GET',
+ success: function(response, opts) {
+ let obj = Ext.decode(response.responseText);
+
+ for (let filter of obj.data) {
+ me.add({ value: `/mapping/notification/${filter.name}` });
+ }
+ },
+ });
+
+ me.resumeEvents();
me.fireEvent('refresh', me);
me.fireEvent('datachanged', me);
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-manager 60/66] ui: perm path: increase width of the perm path selector combobox
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (58 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 59/66] ui: perm path: load notification target/filter acl entries Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 61/66] notification: add gui for sendmail notification endpoints Lukas Wagner
` (8 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
ACL paths for notification targets can become quite long, e.g.:
/mappings/notifications/<target name>
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
www/manager6/form/PermPathSelector.js | 1 +
1 file changed, 1 insertion(+)
diff --git a/www/manager6/form/PermPathSelector.js b/www/manager6/form/PermPathSelector.js
index c20d8b65..e8d395fc 100644
--- a/www/manager6/form/PermPathSelector.js
+++ b/www/manager6/form/PermPathSelector.js
@@ -6,6 +6,7 @@ Ext.define('PVE.form.PermPathSelector', {
displayField: 'value',
typeAhead: true,
queryMode: 'local',
+ width: 380,
store: {
type: 'pvePermPath',
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-widget-toolkit 61/66] notification: add gui for sendmail notification endpoints
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (59 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 60/66] ui: perm path: increase width of the perm path selector combobox Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-19 13:25 ` Dominik Csapak
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 62/66] notification: add gui for gotify " Lukas Wagner
` (7 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
src/Makefile | 4 +
src/Schema.js | 8 ++
src/data/model/NotificationConfig.js | 8 ++
src/panel/NotificationConfigView.js | 192 +++++++++++++++++++++++++++
src/panel/SendmailEditPanel.js | 140 +++++++++++++++++++
src/window/EndpointEditBase.js | 50 +++++++
6 files changed, 402 insertions(+)
create mode 100644 src/data/model/NotificationConfig.js
create mode 100644 src/panel/NotificationConfigView.js
create mode 100644 src/panel/SendmailEditPanel.js
create mode 100644 src/window/EndpointEditBase.js
diff --git a/src/Makefile b/src/Makefile
index baa90ec..e83038f 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -22,6 +22,7 @@ JSSRC= \
data/ObjectStore.js \
data/RRDStore.js \
data/TimezoneStore.js \
+ data/model/NotificationConfig.js \
data/model/Realm.js \
data/model/Certificates.js \
data/model/ACME.js \
@@ -59,6 +60,7 @@ JSSRC= \
panel/InfoWidget.js \
panel/LogView.js \
panel/NodeInfoRepoStatus.js \
+ panel/NotificationConfigView.js \
panel/JournalView.js \
panel/PermissionView.js \
panel/PruneKeepPanel.js \
@@ -68,6 +70,7 @@ JSSRC= \
panel/ACMEAccount.js \
panel/ACMEPlugin.js \
panel/ACMEDomains.js \
+ panel/SendmailEditPanel.js \
panel/StatusView.js \
panel/TfaView.js \
panel/NotesView.js \
@@ -83,6 +86,7 @@ JSSRC= \
window/ACMEAccount.js \
window/ACMEPluginEdit.js \
window/ACMEDomains.js \
+ window/EndpointEditBase.js \
window/FileBrowser.js \
window/AuthEditBase.js \
window/AuthEditOpenId.js \
diff --git a/src/Schema.js b/src/Schema.js
index b247b1e..99bb3fa 100644
--- a/src/Schema.js
+++ b/src/Schema.js
@@ -37,6 +37,14 @@ Ext.define('Proxmox.Schema', { // a singleton
}
},
+ notificationEndpointTypes: {
+ sendmail: {
+ name: gettext('Sendmail'),
+ ipanel: 'pmxSendmailEditPanel',
+ iconCls: 'fa-envelope-o',
+ },
+ },
+
pxarFileTypes: {
b: { icon: 'cube', label: gettext('Block Device') },
c: { icon: 'tty', label: gettext('Character Device') },
diff --git a/src/data/model/NotificationConfig.js b/src/data/model/NotificationConfig.js
new file mode 100644
index 0000000..c2ce843
--- /dev/null
+++ b/src/data/model/NotificationConfig.js
@@ -0,0 +1,8 @@
+Ext.define('proxmox-notification-endpoints', {
+ extend: 'Ext.data.Model',
+ fields: ['name', 'type', 'comment'],
+ proxy: {
+ type: 'proxmox',
+ },
+ idProperty: 'name',
+});
diff --git a/src/panel/NotificationConfigView.js b/src/panel/NotificationConfigView.js
new file mode 100644
index 0000000..f6e6a8b
--- /dev/null
+++ b/src/panel/NotificationConfigView.js
@@ -0,0 +1,192 @@
+Ext.define('Proxmox.panel.NotificationConfigView', {
+ extend: 'Ext.panel.Panel',
+ alias: 'widget.pmxNotificationConfigView',
+ mixins: ['Proxmox.Mixin.CBind'],
+ layout: {
+ type: 'border',
+ },
+
+ items: [
+ {
+ region: 'center',
+ border: false,
+ xtype: 'pmxNotificationEndpointView',
+ cbind: {
+ baseUrl: '{baseUrl}',
+ },
+ },
+ ],
+});
+
+Ext.define('Proxmox.panel.NotificationEndpointView', {
+ extend: 'Ext.grid.Panel',
+ alias: 'widget.pmxNotificationEndpointView',
+
+ title: gettext('Notification Targets'),
+
+ controller: {
+ xclass: 'Ext.app.ViewController',
+
+ openEditWindow: function(endpointType, endpoint) {
+ let me = this;
+
+ if (endpoint === 'mail-to-root') {
+ return;
+ }
+
+ Ext.create('Proxmox.window.EndpointEditBase', {
+ baseUrl: me.getView().baseUrl,
+ type: endpointType,
+
+ name: endpoint,
+ listeners: {
+ destroy: () => me.reload(),
+ },
+ }).show();
+ },
+
+ openEditForSelectedItem: function() {
+ let me = this;
+ let view = me.getView();
+
+ let selection = view.getSelection();
+ if (selection.length < 1) return;
+ let endpointName = selection[0].data.name;
+ let type = selection[0].data.type;
+
+ me.openEditWindow(type, endpointName);
+ },
+
+ reload: function() {
+ let me = this;
+ let view = me.getView();
+ view.getStore().rstore.load();
+ },
+
+ testEndpoint: function() {
+ let me = this;
+ let view = me.getView();
+
+ let selection = view.getSelection();
+ if (selection.length < 1) return;
+ let target = selection[0].data.name;
+
+ Ext.Msg.confirm(
+ gettext("Notification Target Test"),
+ gettext(`Do you want to send a test notification to '${target}'?`),
+ function(decision) {
+ if (decision !== "yes") {
+ return;
+ }
+
+ Proxmox.Utils.API2Request({
+ method: 'POST',
+ url: `${view.baseUrl}/targets/${target}/test`,
+
+ success: function(response, opt) {
+ Ext.Msg.show({
+ title: gettext('Notification Target Test'),
+ message: gettext(`Sent test notification to '${target}'.`),
+ buttons: Ext.Msg.OK,
+ icon: Ext.Msg.INFO,
+ });
+ },
+ autoErrorAlert: true,
+ });
+ });
+ },
+ },
+
+ listeners: {
+ itemdblclick: 'openEditForSelectedItem',
+ activate: 'reload',
+ },
+
+ emptyText: gettext('No notification targets configured'),
+
+ columns: [
+ {
+ dataIndex: 'name',
+ text: gettext('Target Name'),
+ renderer: Ext.String.htmlEncode,
+ flex: 1,
+ },
+ {
+ dataIndex: 'type',
+ text: gettext('Type'),
+ renderer: Ext.String.htmlEncode,
+ flex: 1,
+ },
+ {
+ dataIndex: 'comment',
+ text: gettext('Comment'),
+ renderer: Ext.String.htmlEncode,
+ flex: 1,
+ },
+ ],
+
+ store: {
+ type: 'diff',
+ autoDestroy: true,
+ autoDestroyRstore: true,
+ rstore: {
+ type: 'update',
+ storeid: 'proxmox-notification-endpoints',
+ model: 'proxmox-notification-endpoints',
+ autoStart: true,
+ },
+ sorters: 'name',
+ },
+
+ initComponent: function() {
+ let me = this;
+
+ let menuItems = [];
+ for (const [endpointType, config] of Object.entries(
+ Proxmox.Schema.notificationEndpointTypes).sort()) {
+ menuItems.push({
+ text: config.name,
+ iconCls: 'fa fa-fw ' + (config.iconCls || 'fa-bell-o'),
+ handler: () => me.controller.openEditWindow(endpointType),
+ });
+ }
+
+ Ext.apply(me, {
+ tbar: [
+ {
+ text: gettext('Add'),
+ menu: menuItems,
+ },
+ {
+ xtype: 'proxmoxButton',
+ text: gettext('Modify'),
+ handler: 'openEditForSelectedItem',
+ enableFn: rec => rec.data.name !== 'mail-to-root',
+ disabled: true,
+ },
+ {
+ xtype: 'proxmoxStdRemoveButton',
+ callback: 'reload',
+ enableFn: rec => rec.data.name !== 'mail-to-root',
+ getUrl: function(rec) {
+ if (rec.data.type === 'group') {
+ return `${me.baseUrl}/groups/${rec.getId()}`;
+ }
+
+ return `${me.baseUrl}/endpoints/${rec.data.type}/${rec.getId()}`;
+ },
+ },
+ '-',
+ {
+ xtype: 'proxmoxButton',
+ text: gettext('Test'),
+ handler: 'testEndpoint',
+ disabled: true,
+ },
+ ],
+ });
+
+ me.callParent();
+ me.store.rstore.proxy.setUrl(`/api2/json/${me.baseUrl}/targets`);
+ },
+});
diff --git a/src/panel/SendmailEditPanel.js b/src/panel/SendmailEditPanel.js
new file mode 100644
index 0000000..9444a8c
--- /dev/null
+++ b/src/panel/SendmailEditPanel.js
@@ -0,0 +1,140 @@
+Ext.define('Proxmox.panel.SendmailEditPanel', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pmxSendmailEditPanel',
+ mixins: ['Proxmox.Mixin.CBind'],
+
+ type: 'sendmail',
+
+ columnT: [
+ {
+ xtype: 'pmxDisplayEditField',
+ name: 'name',
+ cbind: {
+ value: '{name}',
+ editable: '{isCreate}',
+ },
+ fieldLabel: gettext('Endpoint Name'),
+ allowBlank: false,
+ },
+ {
+ xtype: 'pmxUserSelector',
+ name: 'mailto-user',
+ reference: 'mailto-user',
+ multiSelect: true,
+ allowBlank: true,
+ editable: false,
+ skipEmptyText: true,
+ fieldLabel: gettext('User(s)'),
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ validator: function(value) {
+ let up = this.up('pmxSendmailEditPanel');
+ let other = up.down('[name=mailto]');
+
+ if (!value && !other.getValue()) {
+ return gettext('Either mailto or mailto-user must be set');
+ }
+
+ return true;
+ },
+ listConfig: {
+ width: 600,
+ columns: [
+ {
+ header: gettext('User'),
+ sortable: true,
+ dataIndex: 'userid',
+ renderer: Ext.String.htmlEncode,
+ flex: 1,
+ },
+ {
+ header: gettext('E-Mail'),
+ sortable: true,
+ dataIndex: 'email',
+ renderer: Ext.String.htmlEncode,
+ flex: 1,
+ },
+ {
+ header: gettext('Comment'),
+ sortable: false,
+ dataIndex: 'comment',
+ renderer: Ext.String.htmlEncode,
+ flex: 1,
+ },
+ ],
+ },
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ fieldLabel: gettext('Additional Recipient(s)'),
+ name: 'mailto',
+ reference: 'mailto',
+ allowBlank: true,
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ autoEl: {
+ tag: 'div',
+ 'data-qtip': gettext(
+ 'Multiple recipients must be separated by spaces, commas or semicolons',
+ ),
+ },
+ validator: function(value) {
+ let up = this.up('pmxSendmailEditPanel');
+ let other = up.down('[name=mailto-user]');
+
+ if (!value && !other.getValue().length) {
+ return gettext('Either mailto or mailto-user must be set');
+ }
+
+ return true;
+ },
+ },
+ ],
+
+ column1: [],
+
+ column2: [],
+
+ columnB: [
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'comment',
+ fieldLabel: gettext('Comment'),
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
+ ],
+ advancedColumnB: [
+ {
+ xtype: 'proxmoxtextfield',
+ fieldLabel: gettext('Author'),
+ name: 'author',
+ allowBlank: true,
+ emptyText: gettext('Proxmox VE'),
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ fieldLabel: gettext('From Address'),
+ name: 'from-address',
+ allowBlank: true,
+ emptyText: gettext('Defaults to datacenter configuration, or root@$hostname'),
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
+ ],
+
+ onGetValues: (values) => {
+ if (values.mailto) {
+ values.mailto = values.mailto.split(/[\s,;]+/);
+ }
+ return values;
+ },
+});
+
diff --git a/src/window/EndpointEditBase.js b/src/window/EndpointEditBase.js
new file mode 100644
index 0000000..81e5951
--- /dev/null
+++ b/src/window/EndpointEditBase.js
@@ -0,0 +1,50 @@
+Ext.define('Proxmox.window.EndpointEditBase', {
+ extend: 'Proxmox.window.Edit',
+
+ isAdd: true,
+
+ fieldDefaults: {
+ labelWidth: 120,
+ },
+
+ initComponent: function() {
+ let me = this;
+
+ me.isCreate = !me.name;
+
+ if (!me.baseUrl) {
+ throw "baseUrl not set";
+ }
+
+ me.url = `/api2/extjs${me.baseUrl}/endpoints/${me.type}`;
+
+ if (me.isCreate) {
+ me.method = 'POST';
+ } else {
+ me.url += `/${me.name}`;
+ me.method = 'PUT';
+ }
+
+ let endpointConfig = Proxmox.Schema.notificationEndpointTypes[me.type];
+ if (!endpointConfig) {
+ throw 'unknown endpoint type';
+ }
+
+ me.subject = endpointConfig.name;
+
+ Ext.apply(me, {
+ items: [{
+ name: me.name,
+ xtype: endpointConfig.ipanel,
+ isCreate: me.isCreate,
+ type: me.type,
+ }],
+ });
+
+ me.callParent();
+
+ if (!me.isCreate) {
+ me.load();
+ }
+ },
+});
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-widget-toolkit 62/66] notification: add gui for gotify notification endpoints
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (60 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 61/66] notification: add gui for sendmail notification endpoints Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 63/66] notification: add gui for notification groups Lukas Wagner
` (6 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
src/Makefile | 1 +
src/Schema.js | 5 ++++
src/panel/GotifyEditPanel.js | 52 ++++++++++++++++++++++++++++++++++++
3 files changed, 58 insertions(+)
create mode 100644 src/panel/GotifyEditPanel.js
diff --git a/src/Makefile b/src/Makefile
index e83038f..2e620e3 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -66,6 +66,7 @@ JSSRC= \
panel/PruneKeepPanel.js \
panel/RRDChart.js \
panel/GaugeWidget.js \
+ panel/GotifyEditPanel.js \
panel/Certificates.js \
panel/ACMEAccount.js \
panel/ACMEPlugin.js \
diff --git a/src/Schema.js b/src/Schema.js
index 99bb3fa..37ecd88 100644
--- a/src/Schema.js
+++ b/src/Schema.js
@@ -43,6 +43,11 @@ Ext.define('Proxmox.Schema', { // a singleton
ipanel: 'pmxSendmailEditPanel',
iconCls: 'fa-envelope-o',
},
+ gotify: {
+ name: gettext('Gotify'),
+ ipanel: 'pmxGotifyEditPanel',
+ iconCls: 'fa-bell-o',
+ },
},
pxarFileTypes: {
diff --git a/src/panel/GotifyEditPanel.js b/src/panel/GotifyEditPanel.js
new file mode 100644
index 0000000..a661c1a
--- /dev/null
+++ b/src/panel/GotifyEditPanel.js
@@ -0,0 +1,52 @@
+Ext.define('Proxmox.panel.GotifyEditPanel', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pmxGotifyEditPanel',
+ mixins: ['Proxmox.Mixin.CBind'],
+
+ type: 'gotify',
+
+ columnT: [
+ {
+ xtype: 'pmxDisplayEditField',
+ name: 'name',
+ cbind: {
+ value: '{name}',
+ editable: '{isCreate}',
+ },
+ fieldLabel: gettext('Endpoint Name'),
+ allowBlank: false,
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ fieldLabel: gettext('Server URL'),
+ name: 'server',
+ allowBlank: false,
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ inputType: 'password',
+ fieldLabel: gettext('API Token'),
+ name: 'token',
+ cbind: {
+ emptyText: get => !get('isCreate') ? gettext('Unchanged') : '',
+ allowBlank: '{!isCreate}',
+ },
+ },
+ ],
+
+ column1: [],
+
+ column2: [],
+
+ columnB: [
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'comment',
+ fieldLabel: gettext('Comment'),
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
+ ],
+});
+
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-widget-toolkit 63/66] notification: add gui for notification groups
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (61 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 62/66] notification: add gui for gotify " Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-19 13:32 ` Dominik Csapak
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 64/66] notification: allow to select filter for notification targets Lukas Wagner
` (5 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
src/Makefile | 1 +
src/Schema.js | 5 +
src/panel/NotificationGroupEditPanel.js | 177 ++++++++++++++++++++++++
src/window/EndpointEditBase.js | 6 +-
4 files changed, 188 insertions(+), 1 deletion(-)
create mode 100644 src/panel/NotificationGroupEditPanel.js
diff --git a/src/Makefile b/src/Makefile
index 2e620e3..829081d 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -61,6 +61,7 @@ JSSRC= \
panel/LogView.js \
panel/NodeInfoRepoStatus.js \
panel/NotificationConfigView.js \
+ panel/NotificationGroupEditPanel.js \
panel/JournalView.js \
panel/PermissionView.js \
panel/PruneKeepPanel.js \
diff --git a/src/Schema.js b/src/Schema.js
index 37ecd88..a7ffdf8 100644
--- a/src/Schema.js
+++ b/src/Schema.js
@@ -48,6 +48,11 @@ Ext.define('Proxmox.Schema', { // a singleton
ipanel: 'pmxGotifyEditPanel',
iconCls: 'fa-bell-o',
},
+ group: {
+ name: gettext('Notification Group'),
+ ipanel: 'pmxNotificationGroupEditPanel',
+ iconCls: 'fa-bell-o',
+ },
},
pxarFileTypes: {
diff --git a/src/panel/NotificationGroupEditPanel.js b/src/panel/NotificationGroupEditPanel.js
new file mode 100644
index 0000000..0a7a469
--- /dev/null
+++ b/src/panel/NotificationGroupEditPanel.js
@@ -0,0 +1,177 @@
+Ext.define('Proxmox.panel.NotificationGroupEditPanel', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pmxNotificationGroupEditPanel',
+ mixins: ['Proxmox.Mixin.CBind'],
+
+ type: 'group',
+
+ columnT: [
+ {
+ xtype: 'pmxDisplayEditField',
+ name: 'name',
+ cbind: {
+ value: '{name}',
+ editable: '{isCreate}',
+ },
+ fieldLabel: gettext('Group Name'),
+ allowBlank: false,
+ },
+ {
+ xtype: 'pmxNotificationEndpointSelector',
+ name: 'endpoint',
+ allowBlank: false,
+ },
+ ],
+
+ column1: [],
+
+ column2: [],
+
+ columnB: [
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'comment',
+ fieldLabel: gettext('Comment'),
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
+ ],
+});
+
+Ext.define('Proxmox.form.NotificationEndpointSelector', {
+ extend: 'Ext.grid.Panel',
+ alias: 'widget.pmxNotificationEndpointSelector',
+
+ mixins: {
+ field: 'Ext.form.field.Field',
+ },
+
+ allowBlank: true,
+ selectAll: false,
+ isFormField: true,
+
+ store: {
+ autoLoad: true,
+ model: 'proxmox-notification-endpoints',
+ sorters: 'name',
+ filters: item => item.data.type !== 'group',
+ },
+
+ columns: [
+ {
+ header: gettext('Endpoint Name'),
+ dataIndex: 'name',
+ flex: 1,
+ },
+ {
+ header: gettext('Type'),
+ dataIndex: 'type',
+ flex: 1,
+ },
+ {
+ header: gettext('Comment'),
+ dataIndex: 'comment',
+ flex: 3,
+ },
+ ],
+
+ selModel: {
+ selType: 'checkboxmodel',
+ mode: 'SIMPLE',
+ },
+
+ checkChangeEvents: [
+ 'selectionchange',
+ 'change',
+ ],
+
+ listeners: {
+ selectionchange: function() {
+ // to trigger validity and error checks
+ this.checkChange();
+ },
+ },
+
+ getSubmitData: function() {
+ let me = this;
+ let res = {};
+ res[me.name] = me.getValue();
+ return res;
+ },
+
+ getValue: function() {
+ let me = this;
+ if (me.savedValue !== undefined) {
+ return me.savedValue;
+ }
+ let sm = me.getSelectionModel();
+ let selection = sm.getSelection();
+ let values = [];
+ selection.forEach(function(item) {
+ values.push(item.data.name);
+ });
+ return values;
+ },
+
+ setValueSelection: function(value) {
+ let me = this;
+
+ let store = me.getStore();
+
+ let notFound = [];
+ let selection = value.map(item => {
+ let found = store.findRecord('name', item, 0, false, true, true);
+ if (!found) {
+ notFound.push(item);
+ }
+ return found;
+ }).filter(r => r);
+
+ for (const name of notFound) {
+ let rec = store.add({
+ name,
+ type: '-',
+ comment: gettext('Included endpoint does not exist!'),
+ });
+ selection.push(rec[0]);
+ }
+
+ let sm = me.getSelectionModel();
+ if (selection.length) {
+ sm.select(selection);
+ } else {
+ sm.deselectAll();
+ }
+ // to correctly trigger invalid class
+ me.getErrors();
+ },
+
+ setValue: function(value) {
+ let me = this;
+
+ let store = me.getStore();
+ if (!store.isLoaded()) {
+ me.savedValue = value;
+ store.on('load', function() {
+ me.setValueSelection(value);
+ delete me.savedValue;
+ }, { single: true });
+ } else {
+ me.setValueSelection(value);
+ }
+ return me.mixins.field.setValue.call(me, value);
+ },
+
+ getErrors: function(value) {
+ let me = this;
+ if (!me.isDisabled() && me.allowBlank === false &&
+ me.getSelectionModel().getCount() === 0) {
+ me.addBodyCls(['x-form-trigger-wrap-default', 'x-form-trigger-wrap-invalid']);
+ return [gettext('No endpoint selected')];
+ }
+
+ me.removeBodyCls(['x-form-trigger-wrap-default', 'x-form-trigger-wrap-invalid']);
+ return [];
+ },
+});
diff --git a/src/window/EndpointEditBase.js b/src/window/EndpointEditBase.js
index 81e5951..bcf6879 100644
--- a/src/window/EndpointEditBase.js
+++ b/src/window/EndpointEditBase.js
@@ -16,7 +16,11 @@ Ext.define('Proxmox.window.EndpointEditBase', {
throw "baseUrl not set";
}
- me.url = `/api2/extjs${me.baseUrl}/endpoints/${me.type}`;
+ if (me.type === 'group') {
+ me.url = `/api2/extjs${me.baseUrl}/groups`;
+ } else {
+ me.url = `/api2/extjs${me.baseUrl}/endpoints/${me.type}`;
+ }
if (me.isCreate) {
me.method = 'POST';
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-widget-toolkit 64/66] notification: allow to select filter for notification targets
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (62 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 63/66] notification: add gui for notification groups Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 65/66] notification: add ui for managing notification filters Lukas Wagner
` (4 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
src/Makefile | 1 +
src/form/NotificationFilterSelector.js | 58 +++++++++++++++++++++++++
src/panel/GotifyEditPanel.js | 9 ++++
src/panel/NotificationConfigView.js | 4 ++
src/panel/NotificationGroupEditPanel.js | 9 ++++
src/panel/SendmailEditPanel.js | 9 ++++
src/window/EndpointEditBase.js | 1 +
7 files changed, 91 insertions(+)
create mode 100644 src/form/NotificationFilterSelector.js
diff --git a/src/Makefile b/src/Makefile
index 829081d..f661bb6 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -44,6 +44,7 @@ JSSRC= \
form/RoleSelector.js \
form/DiskSelector.js \
form/MultiDiskSelector.js \
+ form/NotificationFilterSelector.js \
form/TaskTypeSelector.js \
form/ACME.js \
form/UserSelector.js \
diff --git a/src/form/NotificationFilterSelector.js b/src/form/NotificationFilterSelector.js
new file mode 100644
index 0000000..d2ab8be
--- /dev/null
+++ b/src/form/NotificationFilterSelector.js
@@ -0,0 +1,58 @@
+Ext.define('Proxmox.form.NotificationFilterSelector', {
+ extend: 'Proxmox.form.ComboGrid',
+ alias: ['widget.pmxNotificationFilterSelector'],
+
+ // set default value to empty array, else it inits it with
+ // null and after the store load it is an empty array,
+ // triggering dirtychange
+ value: [],
+ valueField: 'name',
+ displayField: 'name',
+ deleteEmpty: true,
+ skipEmptyText: true,
+ allowBlank: true,
+ editable: false,
+ autoSelect: false,
+
+ listConfig: {
+ columns: [
+ {
+ header: gettext('Filter'),
+ dataIndex: 'name',
+ sortable: true,
+ hideable: false,
+ flex: 1,
+ },
+ {
+ header: gettext('Comment'),
+ dataIndex: 'comment',
+ sortable: true,
+ hideable: false,
+ flex: 2,
+ },
+ ],
+ },
+
+ initComponent: function() {
+ let me = this;
+
+ Ext.apply(me, {
+ store: {
+ fields: ['name', 'comment'],
+ proxy: {
+ type: 'proxmox',
+ url: `/api2/json/${me.baseUrl}/filters`,
+ },
+ sorters: [
+ {
+ property: 'name',
+ direction: 'ASC',
+ },
+ ],
+ autoLoad: true,
+ },
+ });
+
+ me.callParent();
+ },
+});
diff --git a/src/panel/GotifyEditPanel.js b/src/panel/GotifyEditPanel.js
index a661c1a..29cea1e 100644
--- a/src/panel/GotifyEditPanel.js
+++ b/src/panel/GotifyEditPanel.js
@@ -32,6 +32,15 @@ Ext.define('Proxmox.panel.GotifyEditPanel', {
allowBlank: '{!isCreate}',
},
},
+ {
+ xtype: 'pmxNotificationFilterSelector',
+ name: 'filter',
+ fieldLabel: gettext('Filter'),
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ baseUrl: '{baseUrl}',
+ },
+ },
],
column1: [],
diff --git a/src/panel/NotificationConfigView.js b/src/panel/NotificationConfigView.js
index f6e6a8b..2aa04da 100644
--- a/src/panel/NotificationConfigView.js
+++ b/src/panel/NotificationConfigView.js
@@ -141,6 +141,10 @@ Ext.define('Proxmox.panel.NotificationEndpointView', {
initComponent: function() {
let me = this;
+ if (!me.baseUrl) {
+ throw "baseUrl is not set!";
+ }
+
let menuItems = [];
for (const [endpointType, config] of Object.entries(
Proxmox.Schema.notificationEndpointTypes).sort()) {
diff --git a/src/panel/NotificationGroupEditPanel.js b/src/panel/NotificationGroupEditPanel.js
index 0a7a469..39fdbc4 100644
--- a/src/panel/NotificationGroupEditPanel.js
+++ b/src/panel/NotificationGroupEditPanel.js
@@ -28,6 +28,15 @@ Ext.define('Proxmox.panel.NotificationGroupEditPanel', {
column2: [],
columnB: [
+ {
+ xtype: 'pmxNotificationFilterSelector',
+ name: 'filter',
+ fieldLabel: gettext('Filter'),
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ baseUrl: '{baseUrl}',
+ },
+ },
{
xtype: 'proxmoxtextfield',
name: 'comment',
diff --git a/src/panel/SendmailEditPanel.js b/src/panel/SendmailEditPanel.js
index 9444a8c..c151b92 100644
--- a/src/panel/SendmailEditPanel.js
+++ b/src/panel/SendmailEditPanel.js
@@ -91,6 +91,15 @@ Ext.define('Proxmox.panel.SendmailEditPanel', {
return true;
},
},
+ {
+ xtype: 'pmxNotificationFilterSelector',
+ name: 'filter',
+ fieldLabel: gettext('Filter'),
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ baseUrl: '{baseUrl}',
+ },
+ },
],
column1: [],
diff --git a/src/window/EndpointEditBase.js b/src/window/EndpointEditBase.js
index bcf6879..f94a8e4 100644
--- a/src/window/EndpointEditBase.js
+++ b/src/window/EndpointEditBase.js
@@ -41,6 +41,7 @@ Ext.define('Proxmox.window.EndpointEditBase', {
name: me.name,
xtype: endpointConfig.ipanel,
isCreate: me.isCreate,
+ baseUrl: me.baseUrl,
type: me.type,
}],
});
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 proxmox-widget-toolkit 65/66] notification: add ui for managing notification filters
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (63 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 64/66] notification: allow to select filter for notification targets Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-19 13:53 ` Dominik Csapak
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-docs 66/66] add documentation for the new notification system Lukas Wagner
` (3 subsequent siblings)
68 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
src/Makefile | 3 +-
src/data/model/NotificationConfig.js | 9 ++
src/panel/NotificationConfigView.js | 119 +++++++++++++++++++++++++++
src/window/NotificationFilterEdit.js | 115 ++++++++++++++++++++++++++
4 files changed, 245 insertions(+), 1 deletion(-)
create mode 100644 src/window/NotificationFilterEdit.js
diff --git a/src/Makefile b/src/Makefile
index f661bb6..21fbe76 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -89,7 +89,8 @@ JSSRC= \
window/ACMEAccount.js \
window/ACMEPluginEdit.js \
window/ACMEDomains.js \
- window/EndpointEditBase.js \
+ window/EndpointEditBase.js \
+ window/NotificationFilterEdit.js \
window/FileBrowser.js \
window/AuthEditBase.js \
window/AuthEditOpenId.js \
diff --git a/src/data/model/NotificationConfig.js b/src/data/model/NotificationConfig.js
index c2ce843..bb4ef85 100644
--- a/src/data/model/NotificationConfig.js
+++ b/src/data/model/NotificationConfig.js
@@ -6,3 +6,12 @@ Ext.define('proxmox-notification-endpoints', {
},
idProperty: 'name',
});
+
+Ext.define('proxmox-notification-filters', {
+ extend: 'Ext.data.Model',
+ fields: ['name', 'comment'],
+ proxy: {
+ type: 'proxmox',
+ },
+ idProperty: 'name',
+});
diff --git a/src/panel/NotificationConfigView.js b/src/panel/NotificationConfigView.js
index 2aa04da..c3f7c40 100644
--- a/src/panel/NotificationConfigView.js
+++ b/src/panel/NotificationConfigView.js
@@ -15,6 +15,17 @@ Ext.define('Proxmox.panel.NotificationConfigView', {
baseUrl: '{baseUrl}',
},
},
+ {
+ region: 'south',
+ height: '50%',
+ border: false,
+ collapsible: true,
+ animCollapse: false,
+ xtype: 'pmxNotificationFilterView',
+ cbind: {
+ baseUrl: '{baseUrl}',
+ },
+ },
],
});
@@ -194,3 +205,111 @@ Ext.define('Proxmox.panel.NotificationEndpointView', {
me.store.rstore.proxy.setUrl(`/api2/json/${me.baseUrl}/targets`);
},
});
+
+Ext.define('Proxmox.panel.NotificationFilterView', {
+ extend: 'Ext.grid.Panel',
+ alias: 'widget.pmxNotificationFilterView',
+
+ title: gettext('Notification Filters'),
+
+ controller: {
+ xclass: 'Ext.app.ViewController',
+
+ openEditWindow: function(filter) {
+ let me = this;
+
+ Ext.create('Proxmox.window.NotificationFilterEdit', {
+ baseUrl: me.getView().baseUrl,
+ name: filter,
+ listeners: {
+ destroy: () => me.reload(),
+ },
+ }).show();
+ },
+
+ openEditForSelectedItem: function() {
+ let me = this;
+ let view = me.getView();
+
+ let selection = view.getSelection();
+ if (selection.length < 1) return;
+ let filterName = selection[0].data.name;
+
+ me.openEditWindow(filterName);
+ },
+
+ reload: function() {
+ let me = this;
+ let view = me.getView();
+ view.getStore().rstore.load();
+ },
+ },
+
+ listeners: {
+ itemdblclick: 'openEditForSelectedItem',
+ activate: 'reload',
+ },
+
+ emptyText: gettext('No notification filters configured'),
+
+ columns: [
+ {
+ dataIndex: 'name',
+ text: gettext('Filter Name'),
+ renderer: Ext.String.htmlEncode,
+ flex: 1,
+ },
+ {
+ dataIndex: 'comment',
+ text: gettext('Comment'),
+ renderer: Ext.String.htmlEncode,
+ flex: 2,
+ },
+ ],
+
+ store: {
+ type: 'diff',
+ autoDestroy: true,
+ autoDestroyRstore: true,
+ rstore: {
+ type: 'update',
+ storeid: 'proxmox-notification-filters',
+ model: 'proxmox-notification-filters',
+ autoStart: true,
+ },
+ sorters: 'name',
+ },
+
+ initComponent: function() {
+ let me = this;
+
+ if (!me.baseUrl) {
+ throw "baseUrl is not set!";
+ }
+
+ Ext.apply(me, {
+ tbar: [
+ {
+ xtype: 'proxmoxButton',
+ text: gettext('Add'),
+ handler: () => me.getController().openEditWindow(),
+ selModel: false,
+ },
+ {
+ xtype: 'proxmoxButton',
+ text: gettext('Modify'),
+ handler: 'openEditForSelectedItem',
+ disabled: true,
+ },
+ {
+ xtype: 'proxmoxStdRemoveButton',
+ callback: 'reload',
+ baseurl: `${me.baseUrl}/filters`,
+ },
+ ],
+ });
+
+ me.callParent();
+ me.store.rstore.proxy.setUrl(`/api2/json/${me.baseUrl}/filters`);
+ },
+});
diff --git a/src/window/NotificationFilterEdit.js b/src/window/NotificationFilterEdit.js
new file mode 100644
index 0000000..6a48446
--- /dev/null
+++ b/src/window/NotificationFilterEdit.js
@@ -0,0 +1,115 @@
+Ext.define('Proxmox.panel.NotificationFilterEditPanel', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pmxNotificationFilterEditPanel',
+ mixins: ['Proxmox.Mixin.CBind'],
+
+ columnT: [
+ {
+ xtype: 'pmxDisplayEditField',
+ name: 'name',
+ cbind: {
+ value: '{name}',
+ editable: '{isCreate}',
+ },
+ fieldLabel: gettext('Filter Name'),
+ allowBlank: false,
+ },
+ {
+ xtype: 'proxmoxKVComboBox',
+ name: 'min-severity',
+ fieldLabel: gettext('Minimum Severity'),
+ value: null,
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ comboItems: [
+ ['info', 'info'],
+ ['notice', 'notice'],
+ ['warning', 'warning'],
+ ['error', 'error'],
+ ],
+ triggers: {
+ clear: {
+ cls: 'pmx-clear-trigger',
+ weight: -1,
+ hidden: false,
+ handler: function() {
+ this.setValue('');
+ },
+ },
+ },
+ },
+ {
+ xtype: 'proxmoxcheckbox',
+ fieldLabel: gettext('Invert match'),
+ name: 'invert-match',
+ uncheckedValue: 0,
+ defaultValue: 0,
+ cbind: {
+ deleteDefaultValue: '{!isCreate}',
+ },
+ },
+ ],
+
+ column1: [],
+
+ column2: [],
+
+ columnB: [
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'comment',
+ fieldLabel: gettext('Comment'),
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ },
+ ],
+});
+
+Ext.define('Proxmox.window.NotificationFilterEdit', {
+ extend: 'Proxmox.window.Edit',
+
+ isAdd: true,
+
+ fieldDefaults: {
+ labelWidth: 120,
+ },
+
+ initComponent: function() {
+ let me = this;
+
+ me.isCreate = !me.name;
+
+ if (!me.baseUrl) {
+ throw "baseUrl not set";
+ }
+
+ me.url = `/api2/extjs${me.baseUrl}/filters`;
+
+ if (me.isCreate) {
+ me.method = 'POST';
+ } else {
+ me.url += `/${me.name}`;
+ me.method = 'PUT';
+ }
+
+ me.subject = gettext('Notification Filter');
+
+ Ext.apply(me, {
+ items: [{
+ name: me.name,
+ xtype: 'pmxNotificationFilterEditPanel',
+ isCreate: me.isCreate,
+ baseUrl: me.baseUrl,
+ }],
+ });
+
+ me.callParent();
+
+ if (!me.isCreate) {
+ me.load();
+ }
+ },
+});
+
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* [pve-devel] [PATCH v3 pve-docs 66/66] add documentation for the new notification system
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (64 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 65/66] notification: add ui for managing notification filters Lukas Wagner
@ 2023-07-17 15:00 ` Lukas Wagner
2023-07-18 12:34 ` [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce " Dominik Csapak
` (2 subsequent siblings)
68 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-17 15:00 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
notifications.adoc | 159 +++++++++++++++++++++++++++++++++++++++++++
pve-admin-guide.adoc | 2 +
pve-gui.adoc | 2 +
vzdump.adoc | 5 ++
4 files changed, 168 insertions(+)
create mode 100644 notifications.adoc
diff --git a/notifications.adoc b/notifications.adoc
new file mode 100644
index 0000000..c4d2931
--- /dev/null
+++ b/notifications.adoc
@@ -0,0 +1,159 @@
+[[chapter_notifications]]
+Notifications
+=============
+ifndef::manvolnum[]
+:pve-toplevel:
+endif::manvolnum[]
+
+[[notification_events]]
+Notification Events
+-------------------
+
+{pve} will attempt to notify system administrators in case of certain events,
+such as:
+
+[width="100%",options="header"]
+|===========================================================================
+| Event name | Description | Severity
+| `package-updates` | System updates are available | `info`
+| `fencing` | The {pve} HA manager has fenced a node | `error`
+| `replication` | A storage replication job has failed | `error`
+| `vzdump` | vzdump backup finished | `info` (`error` on failure)
+|===========================================================================
+
+In the 'Notification' panel of the datacenter view, the system's behavior can be
+configured for all events except backup jobs. For backup jobs,
+the settings can be found in the respective backup job configuration.
+For every notification event there is an option to configure the notification
+behavior (*when* to send a notification) and the notification target (*where* to
+send the notification).
+
+
+See also:
+
+* xref:datacenter_configuration_file[Datacenter Configuration]
+* xref:datacenter_configuration_file[vzdump]
+
+[[notification_targets]]
+Notification Targets
+--------------------
+
+Notification targets can be configured in the 'Notification Targets' panel.
+
+NOTE: The `mail-to-root` target is always available and cannot be modified or
+removed. It sends a mail the `root@pam` user by using the `sendmail` command and
+serves as a fallback target if no other target is configured for an event.
+
+Sendmail
+~~~~~~~~
+The sendmail binary is a program commonly found on Unix-like operating systems
+that handles the sending of email messages.
+It is a command-line utility that allows users and applications to send emails
+directly from the command line or from within scripts.
+
+The sendmail notification target uses the `sendmail` binary to send emails.
+
+
+NOTE: In standard {pve} installations, the `sendmail` binary is provided by
+Postfix. For this type of target to work correctly, it might be necessary to
+change Postfix's configuration so that it can correctly deliver emails.
+For cluster setups it is necessary to have a working Postfix configuration on
+every single cluster node.
+
+The configuration for Sendmail target plugins has the following options:
+
+* `mailto`: E-Mail address to which the notification shall be sent to. Can be
+set multiple times to accomodate multiple recipients.
+* `mailto-user`: Users to which emails shall be sent to. The user's email
+address will be looked up in `users.cfg`. Can be set multiple times to
+accomodate multiple recipients.
+* `author`: Sets the author of the E-Mail. Defaults to `Proxmox VE`.
+* `from-address`: Sets the from address of the E-Mail. If the parameter is not
+set, the plugin will fall back to the `email_from` setting from
+`datacenter.cfg`. If that is also not set, the plugin will default to
+`root@$hostname`, where `$hostname` is the hostname of the node.
+
+* `filter`: The name of the filter to use for this target.
+
+Gotify
+~~~~~~
+
+http://gotify.net[Gotify] is an open-source self-hosted notification server that
+allows you to send and receive push notifications to various devices and
+applications. It provides a simple API and web interface, making it easy to
+integrate with different platforms and services.
+
+The configuration for Gotify target plugins has the following options:
+
+* `server`: The base URL of the Gotify server, e.g. `http://<ip>:8888`
+* `token`: The authentication token. Tokens can be generated within the Gotify
+web interface.
+* `filter`: The name of the filter to use for this target.
+
+NOTE: The Gotify target plugin will respect the HTTP proxy settings from the
+ xref:datacenter_configuration_file[datacenter configuration]
+
+Group
+~~~~~
+
+One can only select a single target for notification events.
+To notify via multiple targets at the same time, a group can be created.
+A group can reference multiple targets. If a group is used as a target,
+the notification will be sent to all referenced targets. Groups can reference
+all targets except other groups.
+
+
+Notification Filters
+--------------------
+A notification target can be configured to use a *notification filter*.
+If a notification is sent to a target with a filter, the
+filter will determine if the notification will be actually sent or not.
+
+The following matchers are available:
+
+* `min-severity`: Matches notifications with equal or higher severity
+
+It is also possible to configure the evaluation of the individual matchers:
+
+* `invert-match`: Inverts the result of the whole filter
+* `mode`: Sets the logical operator used to connect the individual matchers to
+`and` or `or`. Defaults to `and`.
+
+The `mode` option also influences the evaluation of filters without any
+matchers. If set to `or`, an empty filter evaluates to `false` (do not notify).
+If set to `and`, the result is `true` (send a notification).
+----
+filter: always-matches
+ mode and
+
+filter: never-matches
+ mode or
+----
+
+Permissions
+-----------
+
+For every target or filter, there exists a corresponding ACL path
+`/mapping/notification/<name>`.
+If an operation can be triggered by a user (e.g. via the GUI or API) and if
+that operation is configured to notify via a given target, then
+the user must have the `Mapping.Use` permission on the corresponding
+node in the ACL tree.
+`Mapping.Modify` and `Mapping.Audit` are needed for
+writing/reading the configuration of a target or filter.
+
+NOTE: For backwards-compatibility, the special `mail-to-root` target
+does not require `Mapping.Use`.
+
+NOTE: When sending notifications via a group target,
+the user must have the `Mapping.Use` permission for every single endpoint
+included in the group. If a group/endpoint is configured to
+use a filter, the user must have the `Mapping.Use` permission for the filter
+as well.
+
+
+
+
+
+
+
diff --git a/pve-admin-guide.adoc b/pve-admin-guide.adoc
index 5bac3d6..ce21d8f 100644
--- a/pve-admin-guide.adoc
+++ b/pve-admin-guide.adoc
@@ -51,6 +51,8 @@ include::ha-manager.adoc[]
include::vzdump.adoc[]
+include::notifications.adoc[]
+
// Return to normal title levels.
:leveloffset: 0
diff --git a/pve-gui.adoc b/pve-gui.adoc
index e0fc873..9f63a7e 100644
--- a/pve-gui.adoc
+++ b/pve-gui.adoc
@@ -246,6 +246,8 @@ On the datacenter level, you can access cluster-wide settings and information.
* *Metric Server:* define external metric servers for {pve}.
+* *Notifications:* configurate notification behavior and targets for {pve}.
+
* *Support:* display information about your support subscription.
diff --git a/vzdump.adoc b/vzdump.adoc
index f3eadcd..a7c3d1e 100644
--- a/vzdump.adoc
+++ b/vzdump.adoc
@@ -552,6 +552,11 @@ Backup all guest systems and send notification mails to root and admin.
# vzdump --all --mode suspend --mailto root --mailto admin
+Backup guest 777 and notify via the `notify-admins` notification target on
+failure.
+
+ # vzdump 777 --notification-target notify-admins --notification-policy failure
+
Use snapshot mode (no downtime) and non-default dump directory.
# vzdump 777 --dumpdir /mnt/backup --mode snapshot
--
2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 02/66] notify: preparation for the first endpoint plugin
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 02/66] notify: preparation for the first endpoint plugin Lukas Wagner
@ 2023-07-17 15:48 ` Maximiliano Sandoval
2023-07-18 7:19 ` Lukas Wagner
2023-07-18 11:54 ` Wolfgang Bumiller
1 sibling, 1 reply; 114+ messages in thread
From: Maximiliano Sandoval @ 2023-07-17 15:48 UTC (permalink / raw)
To: Proxmox VE development discussion
Lukas Wagner <l.wagner@proxmox.com> writes:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> Cargo.toml | 1 +
> proxmox-notify/Cargo.toml | 9 +
> proxmox-notify/src/config.rs | 51 +++++
> proxmox-notify/src/endpoints/mod.rs | 0
> proxmox-notify/src/lib.rs | 311 ++++++++++++++++++++++++++++
> proxmox-notify/src/schema.rs | 43 ++++
> 6 files changed, 415 insertions(+)
> create mode 100644 proxmox-notify/src/config.rs
> create mode 100644 proxmox-notify/src/endpoints/mod.rs
> create mode 100644 proxmox-notify/src/schema.rs
>
> diff --git a/Cargo.toml b/Cargo.toml
> index 317593f0..ef8a050a 100644
> --- a/Cargo.toml
> +++ b/Cargo.toml
> @@ -93,6 +93,7 @@ proxmox-lang = { version = "1.1", path = "proxmox-lang" }
> proxmox-rest-server = { version = "0.4.0", path = "proxmox-rest-server" }
> proxmox-router = { version = "1.3.1", path = "proxmox-router" }
> proxmox-schema = { version = "1.3.7", path = "proxmox-schema" }
> +proxmox-section-config = { version = "1.0.2", path = "proxmox-section-config" }
> proxmox-serde = { version = "0.1.1", path = "proxmox-serde", features = [ "serde_json" ] }
> proxmox-sortable-macro = { version = "0.1.2", path = "proxmox-sortable-macro" }
> proxmox-sys = { version = "0.5.0", path = "proxmox-sys" }
> diff --git a/proxmox-notify/Cargo.toml b/proxmox-notify/Cargo.toml
> index 2e69d5b0..37d175f0 100644
> --- a/proxmox-notify/Cargo.toml
> +++ b/proxmox-notify/Cargo.toml
> @@ -8,3 +8,12 @@ repository.workspace = true
> exclude.workspace = true
>
> [dependencies]
> +lazy_static.workspace = true
> +log.workspace = true
> +openssl.workspace = true
> +proxmox-schema = { workspace = true, features = ["api-macro"]}
> +proxmox-section-config = { workspace = true }
> +proxmox-sys.workspace = true
> +regex.workspace = true
> +serde.workspace = true
> +serde_json.workspace = true
> diff --git a/proxmox-notify/src/config.rs b/proxmox-notify/src/config.rs
> new file mode 100644
> index 00000000..362ca0fc
> --- /dev/null
> +++ b/proxmox-notify/src/config.rs
> @@ -0,0 +1,51 @@
> +use lazy_static::lazy_static;
> +use proxmox_schema::{ApiType, ObjectSchema};
> +use proxmox_section_config::{SectionConfig, SectionConfigData, SectionConfigPlugin};
> +
> +use crate::schema::BACKEND_NAME_SCHEMA;
> +use crate::Error;
> +
> +lazy_static! {
Ideally this uses once_cell::sync::Lazy.
> + pub static ref CONFIG: SectionConfig = config_init();
> + pub static ref PRIVATE_CONFIG: SectionConfig = private_config_init();
> +}
> +
> +fn config_init() -> SectionConfig {
> + let mut config = SectionConfig::new(&BACKEND_NAME_SCHEMA);
unneeded mut keyword
> +
> + config
> +}
> +
> +fn private_config_init() -> SectionConfig {
> + let mut config = SectionConfig::new(&BACKEND_NAME_SCHEMA);
Ditto
> +
> + config
> +}
> +
> +pub fn config(raw_config: &str) -> Result<(SectionConfigData, [u8; 32]), Error> {
> + let digest = openssl::sha::sha256(raw_config.as_bytes());
> + let data = CONFIG
> + .parse("notifications.cfg", raw_config)
> + .map_err(|err| Error::ConfigDeserialization(err.into()))?;
> + Ok((data, digest))
> +}
> +
> +pub fn private_config(raw_config: &str) -> Result<(SectionConfigData, [u8; 32]), Error> {
> + let digest = openssl::sha::sha256(raw_config.as_bytes());
> + let data = PRIVATE_CONFIG
> + .parse("priv/notifications.cfg", raw_config)
> + .map_err(|err| Error::ConfigDeserialization(err.into()))?;
> + Ok((data, digest))
> +}
> +
> +pub fn write(config: &SectionConfigData) -> Result<String, Error> {
> + CONFIG
> + .write("notifications.cfg", config)
> + .map_err(|err| Error::ConfigSerialization(err.into()))
> +}
> +
> +pub fn write_private(config: &SectionConfigData) -> Result<String, Error> {
> + PRIVATE_CONFIG
> + .write("priv/notifications.cfg", config)
> + .map_err(|err| Error::ConfigSerialization(err.into()))
> +}
> diff --git a/proxmox-notify/src/endpoints/mod.rs b/proxmox-notify/src/endpoints/mod.rs
> new file mode 100644
> index 00000000..e69de29b
> diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
> index e69de29b..7b90ee15 100644
> --- a/proxmox-notify/src/lib.rs
> +++ b/proxmox-notify/src/lib.rs
> @@ -0,0 +1,311 @@
> +use std::collections::HashMap;
> +use std::fmt::Display;
> +
> +use proxmox_schema::api;
> +use proxmox_section_config::SectionConfigData;
> +use serde::{Deserialize, Serialize};
> +use serde_json::json;
> +use serde_json::Value;
> +
> +use std::error::Error as StdError;
> +
> +mod config;
> +pub mod endpoints;
> +pub mod schema;
> +
> +#[derive(Debug)]
> +pub enum Error {
> + ConfigSerialization(Box<dyn StdError + Send + Sync + 'static>),
> + ConfigDeserialization(Box<dyn StdError + Send + Sync + 'static>),
> + NotifyFailed(String, Box<dyn StdError + Send + Sync + 'static>),
> + TargetDoesNotExist(String),
> +}
> +
> +impl Display for Error {
> + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
> + match self {
> + Error::ConfigSerialization(err) => {
> + write!(f, "could not serialize configuration: {err}")
> + }
> + Error::ConfigDeserialization(err) => {
> + write!(f, "could not deserialize configuration: {err}")
> + }
> + Error::NotifyFailed(endpoint, err) => {
> + write!(f, "could not notify via endpoint(s): {endpoint}: {err}")
> + }
> + Error::TargetDoesNotExist(target) => {
> + write!(f, "notification target '{target}' does not exist")
> + }
> + }
> + }
> +}
> +
> +impl StdError for Error {
> + fn source(&self) -> Option<&(dyn StdError + 'static)> {
> + match self {
> + Error::ConfigSerialization(err) => Some(&**err),
Does this really need the double deref?
> + Error::ConfigDeserialization(err) => Some(&**err),
> + Error::NotifyFailed(_, err) => Some(&**err),
> + Error::TargetDoesNotExist(_) => None,
> + }
> + }
> +}
> +
> +#[api()]
> +#[derive(Clone, Debug, Copy, Serialize, Deserialize, PartialEq, Eq, PartialOrd)]
> +#[serde(rename_all = "kebab-case")]
> +/// Severity of a notification
> +pub enum Severity {
> + /// General information
> + Info,
> + /// A noteworthy event
> + Notice,
> + /// Warning
> + Warning,
> + /// Error
> + Error,
> +}
> +
> +/// Notification endpoint trait, implemented by all endpoint plugins
> +pub trait Endpoint {
> + /// Send a documentation
> + fn send(&self, notification: &Notification) -> Result<(), Error>;
> +
> + /// The name/identifier for this endpoint
> + fn name(&self) -> &str;
> +}
> +
> +#[derive(Debug, Clone)]
> +/// Notification which can be sent
> +pub struct Notification {
> + /// Notification severity
> + pub severity: Severity,
> + /// The title of the notification
> + pub title: String,
> + /// Notification text
> + pub body: String,
> + /// Additional metadata for the notification
> + pub properties: Option<Value>,
> +}
> +
> +/// Notification configuration
> +pub struct Config {
Might be good to derive Debug here.
> + config: SectionConfigData,
> + private_config: SectionConfigData,
> + digest: [u8; 32],
> + private_digest: [u8; 32],
> +}
> +
> +impl Clone for Config {
You can just derive Clone on Config, in general if a Copy type does the correct
thing they will be copied when clone() is called.
> + fn clone(&self) -> Self {
> + Self {
> + config: SectionConfigData {
> + sections: self.config.sections.clone(),
> + order: self.config.order.clone(),
> + },
> + private_config: SectionConfigData {
> + sections: self.private_config.sections.clone(),
> + order: self.private_config.order.clone(),
> + },
> + digest: self.digest,
> + private_digest: self.private_digest,
> + }
> + }
> +}
> +
> +impl Config {
> + /// Parse raw config
> + pub fn new(raw_config: &str, raw_private_config: &str) -> Result<Self, Error> {
> + let (config, digest) = config::config(raw_config)?;
> + let (private_config, private_digest) = config::private_config(raw_private_config)?;
> +
> + Ok(Self {
> + config,
> + digest,
> + private_config,
> + private_digest,
> + })
> + }
> +
> + /// Serialize config
> + pub fn write(&self) -> Result<(String, String), Error> {
> + Ok((
> + config::write(&self.config)?,
> + config::write_private(&self.private_config)?,
> + ))
> + }
> +
> + /// Returns the SHA256 digest of the configuration.
> + /// The digest is only computed once when the configuration deserialized.
> + pub fn digest(&self) -> &[u8; 32] {
> + &self.digest
> + }
> +}
> +
> +/// Notification bus - distributes notifications to all registered endpoints
> +// The reason for the split between `Config` and this struct is to make testing with mocked
> +// endpoints a bit easier.
> +#[derive(Default)]
> +pub struct Bus {
> + endpoints: HashMap<String, Box<dyn Endpoint>>,
> +}
> +
> +#[allow(unused_macros)]
> +macro_rules! parse_endpoints_with_private_config {
> + ($config:ident, $public_config:ty, $private_config:ty, $endpoint_type:ident, $type_name:expr) => {
> + (|| -> Result<Vec<Box<dyn Endpoint>>, Error> {
> + let mut endpoints: Vec<Box<dyn Endpoint>> = Vec::new();
> +
> + let configs: Vec<$public_config> = $config
> + .config
> + .convert_to_typed_array($type_name)
> + .map_err(|err| Error::ConfigDeserialization(err.into()))?;
> +
> + let private_configs: Vec<$private_config> = $config
> + .private_config
> + .convert_to_typed_array($type_name)
> + .map_err(|err| Error::ConfigDeserialization(err.into()))?;
> +
> + for config in configs {
> + if let Some(private_config) = private_configs.iter().find(|p| p.name == config.name)
> + {
> + endpoints.push(Box::new($endpoint_type {
> + config,
> + private_config: private_config.clone(),
> + }));
> + } else {
> + log::error!(
> + "Could not instantiate endpoint '{name}': private config does not exist",
> + name = config.name
> + );
> + }
> + }
> +
> + Ok(endpoints)
> + })()
> + };
> +}
> +
> +#[allow(unused_macros)]
> +macro_rules! parse_endpoints_without_private_config {
> + ($config:ident, $public_config:ty, $endpoint_type:ident, $type_name:expr) => {
> + (|| -> Result<Vec<Box<dyn Endpoint>>, Error> {
> + let mut endpoints: Vec<Box<dyn Endpoint>> = Vec::new();
> +
> + let configs: Vec<$public_config> = $config
> + .config
> + .convert_to_typed_array($type_name)
> + .map_err(|err| Error::ConfigDeserialization(err.into()))?;
> +
> + for config in configs {
> + endpoints.push(Box::new($endpoint_type { config }));
> + }
> +
> + Ok(endpoints)
> + })()
> + };
> +}
> +
> +impl Bus {
> + /// Instantiate notification bus from a given configuration.
> + pub fn from_config(config: &Config) -> Result<Self, Error> {
We are not using the config here? underscore config -> `_config`.
> + let mut endpoints = HashMap::new();
Remove this mut.
> +
> + Ok(Bus { endpoints })
> + }
> +
> + #[cfg(test)]
> + pub fn add_endpoint(&mut self, endpoint: Box<dyn Endpoint>) {
> + self.endpoints.insert(endpoint.name().to_string(), endpoint);
> + }
> +
> + pub fn send(&self, target: &str, notification: &Notification) -> Result<(), Error> {
> + log::info!(
> + "sending notification with title '{title}'",
> + title = notification.title
> + );
> +
> + let endpoint = self
> + .endpoints
> + .get(target)
> + .ok_or(Error::TargetDoesNotExist(target.into()))?;
Clippy: Use ok_or_else here 📎
> +
> + endpoint.send(notification).unwrap_or_else(|e| {
> + log::error!(
> + "could not notfiy via endpoint `{name}`: {e}",
typo notfiy.
> + name = endpoint.name()
> + )
> + });
> +
> + Ok(())
> + }
> +
> + pub fn test_target(&self, target: &str) -> Result<(), Error> {
> + let endpoint = self
> + .endpoints
> + .get(target)
> + .ok_or(Error::TargetDoesNotExist(target.into()))?;
ok_or_else 📎
> +
> + endpoint.send(&Notification {
> + severity: Severity::Info,
> + title: "Test notification".into(),
> + body: "This is a test of the notification target '{{ target }}'".into(),
> + properties: Some(json!({ "target": target })),
> + })?;
> +
> + Ok(())
> + }
> +}
> +
> +#[cfg(test)]
> +mod tests {
> + use std::{cell::RefCell, rc::Rc};
> +
> + use super::*;
> +
> + #[derive(Default, Clone)]
> + struct MockEndpoint {
> + messages: Rc<RefCell<Vec<Notification>>>,
> + }
> +
> + impl Endpoint for MockEndpoint {
> + fn send(&self, message: &Notification) -> Result<(), Error> {
> + self.messages.borrow_mut().push(message.clone());
> +
> + Ok(())
> + }
> +
> + fn name(&self) -> &str {
nit, maybe this trait method should return a static str instead.
> + "mock-endpoint"
> + }
> + }
> +
> + impl MockEndpoint {
> + fn messages(&self) -> Vec<Notification> {
> + self.messages.borrow().clone()
> + }
> + }
> +
> + #[test]
> + fn test_add_mock_endpoint() -> Result<(), Error> {
> + let mock = MockEndpoint::default();
> +
> + let mut bus = Bus::default();
> +
> + bus.add_endpoint(Box::new(mock.clone()));
> +
> + bus.send(
> + "mock-endpoint",
> + &Notification {
> + title: "Title".into(),
> + body: "Body".into(),
> + severity: Severity::Info,
> + properties: Default::default(),
> + },
> + )?;
> + let messages = mock.messages();
> + assert_eq!(messages.len(), 1);
> +
> + Ok(())
> + }
> +}
> diff --git a/proxmox-notify/src/schema.rs b/proxmox-notify/src/schema.rs
> new file mode 100644
> index 00000000..68f11959
> --- /dev/null
> +++ b/proxmox-notify/src/schema.rs
> @@ -0,0 +1,43 @@
> +use proxmox_schema::{const_regex, ApiStringFormat, Schema, StringSchema};
> +
> +// Copied from PBS
> +macro_rules! proxmox_safe_id_regex_str {
> + () => {
> + r"(?:[A-Za-z0-9_][A-Za-z0-9._\-]*)"
> + };
> +}
> +
> +const_regex! {
> + pub SINGLE_LINE_COMMENT_REGEX = r"^[[:^cntrl:]]*$";
> + pub PROXMOX_SAFE_ID_REGEX = concat!(r"^", proxmox_safe_id_regex_str!(), r"$");
> +}
> +
> +const SINGLE_LINE_COMMENT_FORMAT: ApiStringFormat =
> + ApiStringFormat::Pattern(&SINGLE_LINE_COMMENT_REGEX);
> +
> +pub const COMMENT_SCHEMA: Schema = StringSchema::new("Comment.")
> + .format(&SINGLE_LINE_COMMENT_FORMAT)
> + .max_length(128)
> + .schema();
> +
> +pub const EMAIL_SCHEMA: Schema = StringSchema::new("E-Mail Address.")
> + .format(&SINGLE_LINE_COMMENT_FORMAT)
> + .min_length(2)
> + .max_length(64)
> + .schema();
> +
> +pub const PROXMOX_SAFE_ID_FORMAT: ApiStringFormat =
> + ApiStringFormat::Pattern(&PROXMOX_SAFE_ID_REGEX);
> +
> +pub const BACKEND_NAME_SCHEMA: Schema = StringSchema::new("Notification backend name.")
> + .format(&PROXMOX_SAFE_ID_FORMAT)
> + .min_length(3)
> + .max_length(32)
> + .schema();
> +
> +pub const ENTITY_NAME_SCHEMA: Schema =
> + StringSchema::new("Name schema for endpoints, filters and groups")
> + .format(&PROXMOX_SAFE_ID_FORMAT)
> + .min_length(2)
> + .max_length(32)
> + .schema();
nit, maybe these could go on the top.
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 02/66] notify: preparation for the first endpoint plugin
2023-07-17 15:48 ` Maximiliano Sandoval
@ 2023-07-18 7:19 ` Lukas Wagner
2023-07-18 10:13 ` Wolfgang Bumiller
0 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-18 7:19 UTC (permalink / raw)
To: Maximiliano Sandoval, Proxmox VE development discussion
Thanks for the review!
On 7/17/23 17:48, Maximiliano Sandoval wrote:
>
> Lukas Wagner <l.wagner@proxmox.com> writes:
>
>> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
>> ---
>> Cargo.toml | 1 +
>> proxmox-notify/Cargo.toml | 9 +
>> proxmox-notify/src/config.rs | 51 +++++
>> proxmox-notify/src/endpoints/mod.rs | 0
>> proxmox-notify/src/lib.rs | 311 ++++++++++++++++++++++++++++
>> proxmox-notify/src/schema.rs | 43 ++++
>> 6 files changed, 415 insertions(+)
>> create mode 100644 proxmox-notify/src/config.rs
>> create mode 100644 proxmox-notify/src/endpoints/mod.rs
>> create mode 100644 proxmox-notify/src/schema.rs
>>
>> diff --git a/Cargo.toml b/Cargo.toml
>> index 317593f0..ef8a050a 100644
>> --- a/Cargo.toml
>> +++ b/Cargo.toml
>> @@ -93,6 +93,7 @@ proxmox-lang = { version = "1.1", path = "proxmox-lang" }
>> proxmox-rest-server = { version = "0.4.0", path = "proxmox-rest-server" }
>> proxmox-router = { version = "1.3.1", path = "proxmox-router" }
>> proxmox-schema = { version = "1.3.7", path = "proxmox-schema" }
>> +proxmox-section-config = { version = "1.0.2", path = "proxmox-section-config" }
>> proxmox-serde = { version = "0.1.1", path = "proxmox-serde", features = [ "serde_json" ] }
>> proxmox-sortable-macro = { version = "0.1.2", path = "proxmox-sortable-macro" }
>> proxmox-sys = { version = "0.5.0", path = "proxmox-sys" }
>> diff --git a/proxmox-notify/Cargo.toml b/proxmox-notify/Cargo.toml
>> index 2e69d5b0..37d175f0 100644
>> --- a/proxmox-notify/Cargo.toml
>> +++ b/proxmox-notify/Cargo.toml
>> @@ -8,3 +8,12 @@ repository.workspace = true
>> exclude.workspace = true
>>
>> [dependencies]
>> +lazy_static.workspace = true
>> +log.workspace = true
>> +openssl.workspace = true
>> +proxmox-schema = { workspace = true, features = ["api-macro"]}
>> +proxmox-section-config = { workspace = true }
>> +proxmox-sys.workspace = true
>> +regex.workspace = true
>> +serde.workspace = true
>> +serde_json.workspace = true
>> diff --git a/proxmox-notify/src/config.rs b/proxmox-notify/src/config.rs
>> new file mode 100644
>> index 00000000..362ca0fc
>> --- /dev/null
>> +++ b/proxmox-notify/src/config.rs
>> @@ -0,0 +1,51 @@
>> +use lazy_static::lazy_static;
>> +use proxmox_schema::{ApiType, ObjectSchema};
>> +use proxmox_section_config::{SectionConfig, SectionConfigData, SectionConfigPlugin};
>> +
>> +use crate::schema::BACKEND_NAME_SCHEMA;
>> +use crate::Error;
>> +
>> +lazy_static! {
>
> Ideally this uses once_cell::sync::Lazy.
Noted, I'll do that if the need for a v4 arises or as a separate follow-up series.
>
>> + pub static ref CONFIG: SectionConfig = config_init();
>> + pub static ref PRIVATE_CONFIG: SectionConfig = private_config_init();
>> +}
>> +
>> +fn config_init() -> SectionConfig {
>> + let mut config = SectionConfig::new(&BACKEND_NAME_SCHEMA);
>
> unneeded mut keyword
This was 'intentional'... the following sendmail/gotify patches
will insert their config init code here and originally I wanted to
have it in a way where gotify could be applied without sendmail and
vice versa. So this commit sets the base for both commits, if that makes
any sense.
>
>> +
>> + config
>> +}
>> +
>> +fn private_config_init() -> SectionConfig {
>> + let mut config = SectionConfig::new(&BACKEND_NAME_SCHEMA);
>
> Ditto
Same here.
>
>> +
>> + config
>> +}
>> +
>> +pub fn config(raw_config: &str) -> Result<(SectionConfigData, [u8; 32]), Error> {
>> + let digest = openssl::sha::sha256(raw_config.as_bytes());
>> + let data = CONFIG
>> + .parse("notifications.cfg", raw_config)
>> + .map_err(|err| Error::ConfigDeserialization(err.into()))?;
>> + Ok((data, digest))
>> +}
>> +
>> +pub fn private_config(raw_config: &str) -> Result<(SectionConfigData, [u8; 32]), Error> {
>> + let digest = openssl::sha::sha256(raw_config.as_bytes());
>> + let data = PRIVATE_CONFIG
>> + .parse("priv/notifications.cfg", raw_config)
>> + .map_err(|err| Error::ConfigDeserialization(err.into()))?;
>> + Ok((data, digest))
>> +}
>> +
>> +pub fn write(config: &SectionConfigData) -> Result<String, Error> {
>> + CONFIG
>> + .write("notifications.cfg", config)
>> + .map_err(|err| Error::ConfigSerialization(err.into()))
>> +}
>> +
>> +pub fn write_private(config: &SectionConfigData) -> Result<String, Error> {
>> + PRIVATE_CONFIG
>> + .write("priv/notifications.cfg", config)
>> + .map_err(|err| Error::ConfigSerialization(err.into()))
>> +}
>> diff --git a/proxmox-notify/src/endpoints/mod.rs b/proxmox-notify/src/endpoints/mod.rs
>> new file mode 100644
>> index 00000000..e69de29b
>> diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
>> index e69de29b..7b90ee15 100644
>> --- a/proxmox-notify/src/lib.rs
>> +++ b/proxmox-notify/src/lib.rs
>> @@ -0,0 +1,311 @@
>> +use std::collections::HashMap;
>> +use std::fmt::Display;
>> +
>> +use proxmox_schema::api;
>> +use proxmox_section_config::SectionConfigData;
>> +use serde::{Deserialize, Serialize};
>> +use serde_json::json;
>> +use serde_json::Value;
>> +
>> +use std::error::Error as StdError;
>> +
>> +mod config;
>> +pub mod endpoints;
>> +pub mod schema;
>> +
>> +#[derive(Debug)]
>> +pub enum Error {
>> + ConfigSerialization(Box<dyn StdError + Send + Sync + 'static>),
>> + ConfigDeserialization(Box<dyn StdError + Send + Sync + 'static>),
>> + NotifyFailed(String, Box<dyn StdError + Send + Sync + 'static>),
>> + TargetDoesNotExist(String),
>> +}
>> +
>> +impl Display for Error {
>> + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
>> + match self {
>> + Error::ConfigSerialization(err) => {
>> + write!(f, "could not serialize configuration: {err}")
>> + }
>> + Error::ConfigDeserialization(err) => {
>> + write!(f, "could not deserialize configuration: {err}")
>> + }
>> + Error::NotifyFailed(endpoint, err) => {
>> + write!(f, "could not notify via endpoint(s): {endpoint}: {err}")
>> + }
>> + Error::TargetDoesNotExist(target) => {
>> + write!(f, "notification target '{target}' does not exist")
>> + }
>> + }
>> + }
>> +}
>> +
>> +impl StdError for Error {
>> + fn source(&self) -> Option<&(dyn StdError + 'static)> {
>> + match self {
>> + Error::ConfigSerialization(err) => Some(&**err),
>
> Does this really need the double deref?
Yeah, does not seem to work any way. Though I'm not sure I can fully explain why.
Copied the approach from (I think) the TFA crate.
>
>> + Error::ConfigDeserialization(err) => Some(&**err),
>> + Error::NotifyFailed(_, err) => Some(&**err),
>> + Error::TargetDoesNotExist(_) => None,
>> + }
>> + }
>> +}
>> +
>> +#[api()]
>> +#[derive(Clone, Debug, Copy, Serialize, Deserialize, PartialEq, Eq, PartialOrd)]
>> +#[serde(rename_all = "kebab-case")]
>> +/// Severity of a notification
>> +pub enum Severity {
>> + /// General information
>> + Info,
>> + /// A noteworthy event
>> + Notice,
>> + /// Warning
>> + Warning,
>> + /// Error
>> + Error,
>> +}
>> +
>> +/// Notification endpoint trait, implemented by all endpoint plugins
>> +pub trait Endpoint {
>> + /// Send a documentation
>> + fn send(&self, notification: &Notification) -> Result<(), Error>;
>> +
>> + /// The name/identifier for this endpoint
>> + fn name(&self) -> &str;
>> +}
>> +
>> +#[derive(Debug, Clone)]
>> +/// Notification which can be sent
>> +pub struct Notification {
>> + /// Notification severity
>> + pub severity: Severity,
>> + /// The title of the notification
>> + pub title: String,
>> + /// Notification text
>> + pub body: String,
>> + /// Additional metadata for the notification
>> + pub properties: Option<Value>,
>> +}
>> +
>> +/// Notification configuration
>> +pub struct Config {
>
> Might be good to derive Debug here.
I mostly only add Debug once I need it. Do you always add it?
>
>> + config: SectionConfigData,
>> + private_config: SectionConfigData,
>> + digest: [u8; 32],
>> + private_digest: [u8; 32],
>> +}
>> +
>> +impl Clone for Config {
>
> You can just derive Clone on Config, in general if a Copy type does the correct
> thing they will be copied when clone() is called.
True, forgot about that. Originally I did not derive Clone since I did not want
to touch another sub-crate from this repo, as this makes deploying quite a bite
more cumbersome. Will be fixed in a follow-up or in a v4.
>
>> + fn clone(&self) -> Self {
>> + Self {
>> + config: SectionConfigData {
>> + sections: self.config.sections.clone(),
>> + order: self.config.order.clone(),
>> + },
>> + private_config: SectionConfigData {
>> + sections: self.private_config.sections.clone(),
>> + order: self.private_config.order.clone(),
>> + },
>> + digest: self.digest,
>> + private_digest: self.private_digest,
>> + }
>> + }
>> +}
>> +
>> +impl Config {
>> + /// Parse raw config
>> + pub fn new(raw_config: &str, raw_private_config: &str) -> Result<Self, Error> {
>> + let (config, digest) = config::config(raw_config)?;
>> + let (private_config, private_digest) = config::private_config(raw_private_config)?;
>> +
>> + Ok(Self {
>> + config,
>> + digest,
>> + private_config,
>> + private_digest,
>> + })
>> + }
>> +
>> + /// Serialize config
>> + pub fn write(&self) -> Result<(String, String), Error> {
>> + Ok((
>> + config::write(&self.config)?,
>> + config::write_private(&self.private_config)?,
>> + ))
>> + }
>> +
>> + /// Returns the SHA256 digest of the configuration.
>> + /// The digest is only computed once when the configuration deserialized.
>> + pub fn digest(&self) -> &[u8; 32] {
>> + &self.digest
>> + }
>> +}
>> +
>> +/// Notification bus - distributes notifications to all registered endpoints
>> +// The reason for the split between `Config` and this struct is to make testing with mocked
>> +// endpoints a bit easier.
>> +#[derive(Default)]
>> +pub struct Bus {
>> + endpoints: HashMap<String, Box<dyn Endpoint>>,
>> +}
>> +
>> +#[allow(unused_macros)]
>> +macro_rules! parse_endpoints_with_private_config {
>> + ($config:ident, $public_config:ty, $private_config:ty, $endpoint_type:ident, $type_name:expr) => {
>> + (|| -> Result<Vec<Box<dyn Endpoint>>, Error> {
>> + let mut endpoints: Vec<Box<dyn Endpoint>> = Vec::new();
>> +
>> + let configs: Vec<$public_config> = $config
>> + .config
>> + .convert_to_typed_array($type_name)
>> + .map_err(|err| Error::ConfigDeserialization(err.into()))?;
>> +
>> + let private_configs: Vec<$private_config> = $config
>> + .private_config
>> + .convert_to_typed_array($type_name)
>> + .map_err(|err| Error::ConfigDeserialization(err.into()))?;
>> +
>> + for config in configs {
>> + if let Some(private_config) = private_configs.iter().find(|p| p.name == config.name)
>> + {
>> + endpoints.push(Box::new($endpoint_type {
>> + config,
>> + private_config: private_config.clone(),
>> + }));
>> + } else {
>> + log::error!(
>> + "Could not instantiate endpoint '{name}': private config does not exist",
>> + name = config.name
>> + );
>> + }
>> + }
>> +
>> + Ok(endpoints)
>> + })()
>> + };
>> +}
>> +
>> +#[allow(unused_macros)]
>> +macro_rules! parse_endpoints_without_private_config {
>> + ($config:ident, $public_config:ty, $endpoint_type:ident, $type_name:expr) => {
>> + (|| -> Result<Vec<Box<dyn Endpoint>>, Error> {
>> + let mut endpoints: Vec<Box<dyn Endpoint>> = Vec::new();
>> +
>> + let configs: Vec<$public_config> = $config
>> + .config
>> + .convert_to_typed_array($type_name)
>> + .map_err(|err| Error::ConfigDeserialization(err.into()))?;
>> +
>> + for config in configs {
>> + endpoints.push(Box::new($endpoint_type { config }));
>> + }
>> +
>> + Ok(endpoints)
>> + })()
>> + };
>> +}
>> +
>> +impl Bus {
>> + /// Instantiate notification bus from a given configuration.
>> + pub fn from_config(config: &Config) -> Result<Self, Error> {
>
> We are not using the config here? underscore config -> `_config`.
Same reason as above, fragment from splitting commits.
>
>> + let mut endpoints = HashMap::new();
>
> Remove this mut.
Same reason as above, fragment from splitting commits.
>
>> +
>> + Ok(Bus { endpoints })
>> + }
>> +
>> + #[cfg(test)]
>> + pub fn add_endpoint(&mut self, endpoint: Box<dyn Endpoint>) {
>> + self.endpoints.insert(endpoint.name().to_string(), endpoint);
>> + }
>> +
>> + pub fn send(&self, target: &str, notification: &Notification) -> Result<(), Error> {
>> + log::info!(
>> + "sending notification with title '{title}'",
>> + title = notification.title
>> + );
>> +
>> + let endpoint = self
>> + .endpoints
>> + .get(target)
>> + .ok_or(Error::TargetDoesNotExist(target.into()))?;
>
> Clippy: Use ok_or_else here 📎
Noted, thanks!
>
>> +
>> + endpoint.send(notification).unwrap_or_else(|e| {
>> + log::error!(
>> + "could not notfiy via endpoint `{name}`: {e}",
>
> typo notfiy.
Noted, thanks!
>
>> + name = endpoint.name()
>> + )
>> + });
>> +
>> + Ok(())
>> + }
>> +
>> + pub fn test_target(&self, target: &str) -> Result<(), Error> {
>> + let endpoint = self
>> + .endpoints
>> + .get(target)
>> + .ok_or(Error::TargetDoesNotExist(target.into()))?;
>
> ok_or_else 📎
Ack
>
>> +
>> + endpoint.send(&Notification {
>> + severity: Severity::Info,
>> + title: "Test notification".into(),
>> + body: "This is a test of the notification target '{{ target }}'".into(),
>> + properties: Some(json!({ "target": target })),
>> + })?;
>> +
>> + Ok(())
>> + }
>> +}
>> +
>> +#[cfg(test)]
>> +mod tests {
>> + use std::{cell::RefCell, rc::Rc};
>> +
>> + use super::*;
>> +
>> + #[derive(Default, Clone)]
>> + struct MockEndpoint {
>> + messages: Rc<RefCell<Vec<Notification>>>,
>> + }
>> +
>> + impl Endpoint for MockEndpoint {
>> + fn send(&self, message: &Notification) -> Result<(), Error> {
>> + self.messages.borrow_mut().push(message.clone());
>> +
>> + Ok(())
>> + }
>> +
>> + fn name(&self) -> &str {
>
> nit, maybe this trait method should return a static str instead.
>
Nah, this will not work once actual endpoint plugins are implemenented. They
store the name as a string inside the struct and thus return a &str
>> + "mock-endpoint"
>> + }
>> + }
>> +
>> + impl MockEndpoint {
>> + fn messages(&self) -> Vec<Notification> {
>> + self.messages.borrow().clone()
>> + }
>> + }
>> +
>> + #[test]
>> + fn test_add_mock_endpoint() -> Result<(), Error> {
>> + let mock = MockEndpoint::default();
>> +
>> + let mut bus = Bus::default();
>> +
>> + bus.add_endpoint(Box::new(mock.clone()));
>> +
>> + bus.send(
>> + "mock-endpoint",
>> + &Notification {
>> + title: "Title".into(),
>> + body: "Body".into(),
>> + severity: Severity::Info,
>> + properties: Default::default(),
>> + },
>> + )?;
>> + let messages = mock.messages();
>> + assert_eq!(messages.len(), 1);
>> +
>> + Ok(())
>> + }
>> +}
>> diff --git a/proxmox-notify/src/schema.rs b/proxmox-notify/src/schema.rs
>> new file mode 100644
>> index 00000000..68f11959
>> --- /dev/null
>> +++ b/proxmox-notify/src/schema.rs
>> @@ -0,0 +1,43 @@
>> +use proxmox_schema::{const_regex, ApiStringFormat, Schema, StringSchema};
>> +
>> +// Copied from PBS
>> +macro_rules! proxmox_safe_id_regex_str {
>> + () => {
>> + r"(?:[A-Za-z0-9_][A-Za-z0-9._\-]*)"
>> + };
>> +}
>> +
>> +const_regex! {
>> + pub SINGLE_LINE_COMMENT_REGEX = r"^[[:^cntrl:]]*$";
>> + pub PROXMOX_SAFE_ID_REGEX = concat!(r"^", proxmox_safe_id_regex_str!(), r"$");
>> +}
>> +
>> +const SINGLE_LINE_COMMENT_FORMAT: ApiStringFormat =
>> + ApiStringFormat::Pattern(&SINGLE_LINE_COMMENT_REGEX);
>> +
>> +pub const COMMENT_SCHEMA: Schema = StringSchema::new("Comment.")
>> + .format(&SINGLE_LINE_COMMENT_FORMAT)
>> + .max_length(128)
>> + .schema();
>> +
>> +pub const EMAIL_SCHEMA: Schema = StringSchema::new("E-Mail Address.")
>> + .format(&SINGLE_LINE_COMMENT_FORMAT)
>> + .min_length(2)
>> + .max_length(64)
>> + .schema();
>> +
>> +pub const PROXMOX_SAFE_ID_FORMAT: ApiStringFormat =
>> + ApiStringFormat::Pattern(&PROXMOX_SAFE_ID_REGEX);
>> +
>> +pub const BACKEND_NAME_SCHEMA: Schema = StringSchema::new("Notification backend name.")
>> + .format(&PROXMOX_SAFE_ID_FORMAT)
>> + .min_length(3)
>> + .max_length(32)
>> + .schema();
>> +
>> +pub const ENTITY_NAME_SCHEMA: Schema =
>> + StringSchema::new("Name schema for endpoints, filters and groups")
>> + .format(&PROXMOX_SAFE_ID_FORMAT)
>> + .min_length(2)
>> + .max_length(32)
>> + .schema();
>
> nit, maybe these could go on the top.
These are in a separate file, maybe you missed that? Or what exactly do you mean?
--
- Lukas
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 02/66] notify: preparation for the first endpoint plugin
2023-07-18 7:19 ` Lukas Wagner
@ 2023-07-18 10:13 ` Wolfgang Bumiller
0 siblings, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-18 10:13 UTC (permalink / raw)
To: Lukas Wagner; +Cc: Maximiliano Sandoval, Proxmox VE development discussion
Not an in-depth review, just an explanation:
On Tue, Jul 18, 2023 at 09:19:54AM +0200, Lukas Wagner wrote:
> Thanks for the review!
>
> On 7/17/23 17:48, Maximiliano Sandoval wrote:
> >
> > Lukas Wagner <l.wagner@proxmox.com> writes:
> >
(...)
> > > +#[derive(Debug)]
> > > +pub enum Error {
> > > + ConfigSerialization(Box<dyn StdError + Send + Sync + 'static>),
FYI you can leave out the `'static` in type definitions like this. It's
the only possible choice and therefor automatic ;-)
^ Here we have `Box<dyn StdError>`
...
> > > + ConfigDeserialization(Box<dyn StdError + Send + Sync + 'static>),
> > > + NotifyFailed(String, Box<dyn StdError + Send + Sync + 'static>),
> > > + TargetDoesNotExist(String),
> > > +}
> > > +
> > > +impl Display for Error {
> > > + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
> > > + match self {
> > > + Error::ConfigSerialization(err) => {
> > > + write!(f, "could not serialize configuration: {err}")
> > > + }
> > > + Error::ConfigDeserialization(err) => {
> > > + write!(f, "could not deserialize configuration: {err}")
> > > + }
> > > + Error::NotifyFailed(endpoint, err) => {
> > > + write!(f, "could not notify via endpoint(s): {endpoint}: {err}")
> > > + }
> > > + Error::TargetDoesNotExist(target) => {
> > > + write!(f, "notification target '{target}' does not exist")
> > > + }
> > > + }
> > > + }
> > > +}
> > > +
> > > +impl StdError for Error {
> > > + fn source(&self) -> Option<&(dyn StdError + 'static)> {
> > > + match self {
^ Here, `self` is `&Self`
> > > + Error::ConfigSerialization(err) => Some(&**err),
> >
> > Does this really need the double deref?
>
> Yeah, does not seem to work any way. Though I'm not sure I can fully explain why.
> Copied the approach from (I think) the TFA crate.
Since, as noted above, `self` is `&Self`, `err` here will be `&T`.
`T` is `Box<dyn StdError + Send + Sync>`.
So we have `&Box<dyn StdError + Send + Sync>`
The first deref goes to `Box<dyn StdError + Send + Sync>` via an immutable ref.
We cannot return this as this would be a moving expression and rust does
not implicitly dereference multiple times automatically.
The second deref calls `<Box as Deref>::deref` and implicitly
dereferences it (since 'Deref' is kind of "transparent" to the '*'
operator), from `&(dyn StdError + Send + Sync)` to `(dyn StdError + Send
+ Sync)`.
Finally we borrow -> `&**` :-)
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 02/66] notify: preparation for the first endpoint plugin
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 02/66] notify: preparation for the first endpoint plugin Lukas Wagner
2023-07-17 15:48 ` Maximiliano Sandoval
@ 2023-07-18 11:54 ` Wolfgang Bumiller
1 sibling, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-18 11:54 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
On Mon, Jul 17, 2023 at 04:59:47PM +0200, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
(...)
> +/// Notification bus - distributes notifications to all registered endpoints
> +// The reason for the split between `Config` and this struct is to make testing with mocked
> +// endpoints a bit easier.
> +#[derive(Default)]
> +pub struct Bus {
> + endpoints: HashMap<String, Box<dyn Endpoint>>,
> +}
> +
> +#[allow(unused_macros)]
> +macro_rules! parse_endpoints_with_private_config {
> + ($config:ident, $public_config:ty, $private_config:ty, $endpoint_type:ident, $type_name:expr) => {
> + (|| -> Result<Vec<Box<dyn Endpoint>>, Error> {
> + let mut endpoints: Vec<Box<dyn Endpoint>> = Vec::new();
nit: Less to type would be
let mut endpoints = Vec::<Box<dyn Endpoint>>::new();
;-)
> +
> + let configs: Vec<$public_config> = $config
> + .config
> + .convert_to_typed_array($type_name)
> + .map_err(|err| Error::ConfigDeserialization(err.into()))?;
> +
> + let private_configs: Vec<$private_config> = $config
> + .private_config
> + .convert_to_typed_array($type_name)
> + .map_err(|err| Error::ConfigDeserialization(err.into()))?;
> +
> + for config in configs {
> + if let Some(private_config) = private_configs.iter().find(|p| p.name == config.name)
If you use `private_config` only for this kind of lookup, maybe this
should use `$config.private_config.sections.get(&config.name)` and match
the type name after the lookup, since you're now linear-searching an ID
for something that was previously a HashMap from id to data ;-)
> + {
> + endpoints.push(Box::new($endpoint_type {
> + config,
> + private_config: private_config.clone(),
> + }));
> + } else {
> + log::error!(
> + "Could not instantiate endpoint '{name}': private config does not exist",
> + name = config.name
> + );
> + }
> + }
> +
> + Ok(endpoints)
> + })()
> + };
> +}
> +
> +#[allow(unused_macros)]
> +macro_rules! parse_endpoints_without_private_config {
> + ($config:ident, $public_config:ty, $endpoint_type:ident, $type_name:expr) => {
> + (|| -> Result<Vec<Box<dyn Endpoint>>, Error> {
> + let mut endpoints: Vec<Box<dyn Endpoint>> = Vec::new();
> +
> + let configs: Vec<$public_config> = $config
> + .config
> + .convert_to_typed_array($type_name)
> + .map_err(|err| Error::ConfigDeserialization(err.into()))?;
> +
> + for config in configs {
> + endpoints.push(Box::new($endpoint_type { config }));
> + }
> +
> + Ok(endpoints)
> + })()
> + };
> +}
> +
(...)
> diff --git a/proxmox-notify/src/schema.rs b/proxmox-notify/src/schema.rs
> new file mode 100644
> index 00000000..68f11959
> --- /dev/null
> +++ b/proxmox-notify/src/schema.rs
> @@ -0,0 +1,43 @@
> +use proxmox_schema::{const_regex, ApiStringFormat, Schema, StringSchema};
> +
> +// Copied from PBS
> +macro_rules! proxmox_safe_id_regex_str {
^ You can drop this, since you depend on proxmox_schema, where by now we
have `SAFE_ID_REGEX` and `SAFE_ID_FORMAT` if you enable the `api-types`
feature :-)
> + () => {
> + r"(?:[A-Za-z0-9_][A-Za-z0-9._\-]*)"
> + };
> +}
> +
> +const_regex! {
> + pub SINGLE_LINE_COMMENT_REGEX = r"^[[:^cntrl:]]*$";
^ Feel free to move this to `proxmox_schema::api_types` as well.
> + pub PROXMOX_SAFE_ID_REGEX = concat!(r"^", proxmox_safe_id_regex_str!(), r"$");
> +}
> +
> +const SINGLE_LINE_COMMENT_FORMAT: ApiStringFormat =
> + ApiStringFormat::Pattern(&SINGLE_LINE_COMMENT_REGEX);
^ And this
> +
> +pub const COMMENT_SCHEMA: Schema = StringSchema::new("Comment.")
^ And this.
> + .format(&SINGLE_LINE_COMMENT_FORMAT)
> + .max_length(128)
> + .schema();
> +
> +pub const EMAIL_SCHEMA: Schema = StringSchema::new("E-Mail Address.")
> + .format(&SINGLE_LINE_COMMENT_FORMAT)
> + .min_length(2)
> + .max_length(64)
> + .schema();
> +
> +pub const PROXMOX_SAFE_ID_FORMAT: ApiStringFormat =
> + ApiStringFormat::Pattern(&PROXMOX_SAFE_ID_REGEX);
> +
> +pub const BACKEND_NAME_SCHEMA: Schema = StringSchema::new("Notification backend name.")
> + .format(&PROXMOX_SAFE_ID_FORMAT)
> + .min_length(3)
> + .max_length(32)
> + .schema();
> +
> +pub const ENTITY_NAME_SCHEMA: Schema =
> + StringSchema::new("Name schema for endpoints, filters and groups")
> + .format(&PROXMOX_SAFE_ID_FORMAT)
> + .min_length(2)
> + .max_length(32)
> + .schema();
> --
> 2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 03/66] notify: preparation for the API
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 03/66] notify: preparation for the API Lukas Wagner
@ 2023-07-18 12:02 ` Wolfgang Bumiller
0 siblings, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-18 12:02 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
FYI: I'm not through the entire series yet, but I think it's fine for
the *API* methods to use `anyhow::Error` instead.
Also:
I'd like to move this here to a separate crate to replace
`proxmox_router::HttpError` (since the router one has no `source`
support atm and we don't want to depend on the router here, since *this*
crate also ends up in perl bindings), along with the `http_bail!` and
`http_err!` macros. To make sure that we don't have any HTTP error types
which might get past our server's automatic "delay on auth issues"
mechanism. Specifically, the rest server code uses `downcast` on an
`anyhow::Error` to check if it's a `HttpError` with status code
`UNAUTHORIZED` and then introduces a delay.
On Mon, Jul 17, 2023 at 04:59:48PM +0200, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> proxmox-notify/src/api/mod.rs | 94 +++++++++++++++++++++++++++++++++++
> proxmox-notify/src/lib.rs | 1 +
> 2 files changed, 95 insertions(+)
> create mode 100644 proxmox-notify/src/api/mod.rs
>
> diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
> new file mode 100644
> index 00000000..be596b93
> --- /dev/null
> +++ b/proxmox-notify/src/api/mod.rs
> @@ -0,0 +1,94 @@
> +use std::error::Error as StdError;
> +use std::fmt::Display;
> +
> +use crate::Config;
> +use serde::Serialize;
> +
> +#[derive(Debug, Serialize)]
> +pub struct ApiError {
> + /// HTTP Error code
> + code: u16,
> + /// Error message
> + message: String,
> + #[serde(skip_serializing)]
> + /// The underlying cause of the error
> + source: Option<Box<dyn StdError + Send + Sync + 'static>>,
> +}
> +
> +impl ApiError {
> + fn new<S: AsRef<str>>(
> + message: S,
> + code: u16,
> + source: Option<Box<dyn StdError + Send + Sync + 'static>>,
> + ) -> Self {
> + Self {
> + message: message.as_ref().into(),
> + code,
> + source,
> + }
> + }
> +
> + pub fn bad_request<S: AsRef<str>>(
> + message: S,
> + source: Option<Box<dyn StdError + Send + Sync + 'static>>,
> + ) -> Self {
> + Self::new(message, 400, source)
> + }
> +
> + pub fn not_found<S: AsRef<str>>(
> + message: S,
> + source: Option<Box<dyn StdError + Send + Sync + 'static>>,
> + ) -> Self {
> + Self::new(message, 404, source)
> + }
> +
> + pub fn internal_server_error<S: AsRef<str>>(
> + message: S,
> + source: Option<Box<dyn StdError + Send + Sync + 'static>>,
> + ) -> Self {
> + Self::new(message, 500, source)
> + }
> +}
> +
> +impl Display for ApiError {
> + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
> + f.write_str(&format!("{} {}", self.code, self.message))
> + }
> +}
> +
> +impl StdError for ApiError {
> + fn source(&self) -> Option<&(dyn StdError + 'static)> {
> + match &self.source {
> + None => None,
> + Some(source) => Some(&**source),
> + }
> + }
> +}
> +
> +fn verify_digest(config: &Config, digest: Option<&[u8]>) -> Result<(), ApiError> {
> + if let Some(digest) = digest {
> + if config.digest != *digest {
> + return Err(ApiError::bad_request(
> + "detected modified configuration - file changed by other user? Try again.",
> + None,
> + ));
> + }
> + }
> +
> + Ok(())
> +}
> +
> +fn endpoint_exists(config: &Config, name: &str) -> bool {
> + let mut exists = false;
> +
> + exists
> +}
> +
> +#[cfg(test)]
> +mod test_helpers {
> + use crate::Config;
> +
> + pub fn empty_config() -> Config {
> + Config::new("", "").unwrap()
> + }
> +}
> diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
> index 7b90ee15..43feac25 100644
> --- a/proxmox-notify/src/lib.rs
> +++ b/proxmox-notify/src/lib.rs
> @@ -9,6 +9,7 @@ use serde_json::Value;
>
> use std::error::Error as StdError;
>
> +pub mod api;
> mod config;
> pub mod endpoints;
> pub mod schema;
> --
> 2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (65 preceding siblings ...)
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-docs 66/66] add documentation for the new notification system Lukas Wagner
@ 2023-07-18 12:34 ` Dominik Csapak
2023-07-18 13:14 ` Lukas Wagner
` (2 more replies)
2023-07-18 13:27 ` Wolfgang Bumiller
2023-07-19 12:11 ` Wolfgang Bumiller
68 siblings, 3 replies; 114+ messages in thread
From: Dominik Csapak @ 2023-07-18 12:34 UTC (permalink / raw)
To: Proxmox VE development discussion, Lukas Wagner
gave the series a quick spin, review of the gui patches comes later ;)
a few high level comments from a user perspective:
* the node fencing/replication edit windows always shows the warning that it shouldn't be
disabled, that should imo only be there if i select 'never' ?
(conversely, the package update window never shows the warning...)
* when we have this, we could remove the pacakge notify line from the datacenter options, no?
* imho having "Notification Targets" directly below "Notifications" is a bit redundant
(and wasting space), but it's just a minor thing
* the filter 'mode' is also not exposed on the gui (intentionally?)
* also the mode is not quite clear since only one filter can be active per target?
(or is it meant when there are multiple filter by means of notification groups?)
* what is a filter without a minimum severity? what does it do?
(does it make sense to allow such filters in the first place?)
* the backup edit window is rather tall at this point, and if we assume a minimum
screen size of 1280x720 there is not quite enough space when you subtract
the (typical) os bar and browser window decoration, maybe a 'notification'
tab would be better (we already have some tabs there anyway)
* i found one bug, but not quite sure yet where it comes from exactly,
putting in emojis into a field (e.g. a comment or author) it's accepted,
but editing a different entry fails with:
--->8---
could not serialize configuration: writing 'notifications.cfg' failed: detected unexpected control
character in section 'testgroup' key 'comment' (500)
---8<---
not sure where the utf-8 info gets lost. (or we could limit all fields to ascii?)
such a notification target still works AFAICT (but if set as e.g. the author it's
probably the wrong value)
(i used 😀 as a test)
otherwise works AFAICT
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 06/66] notify: api: add API for sendmail endpoints
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 06/66] notify: api: add API for sendmail endpoints Lukas Wagner
@ 2023-07-18 12:36 ` Wolfgang Bumiller
2023-07-19 11:51 ` Lukas Wagner
0 siblings, 1 reply; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-18 12:36 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
On Mon, Jul 17, 2023 at 04:59:51PM +0200, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> proxmox-notify/src/api/mod.rs | 7 +
> proxmox-notify/src/api/sendmail.rs | 254 +++++++++++++++++++++++++++++
> 2 files changed, 261 insertions(+)
> create mode 100644 proxmox-notify/src/api/sendmail.rs
>
> diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
> index db9ad1ca..4baae899 100644
> --- a/proxmox-notify/src/api/mod.rs
> +++ b/proxmox-notify/src/api/mod.rs
> @@ -5,6 +5,8 @@ use crate::Config;
> use serde::Serialize;
>
> pub mod common;
> +#[cfg(feature = "sendmail")]
> +pub mod sendmail;
>
> #[derive(Debug, Serialize)]
> pub struct ApiError {
> @@ -83,6 +85,11 @@ fn verify_digest(config: &Config, digest: Option<&[u8]>) -> Result<(), ApiError>
> fn endpoint_exists(config: &Config, name: &str) -> bool {
> let mut exists = false;
>
> + #[cfg(feature = "sendmail")]
> + {
> + exists = exists || sendmail::get_endpoint(config, name).is_ok();
> + }
> +
> exists
> }
>
> diff --git a/proxmox-notify/src/api/sendmail.rs b/proxmox-notify/src/api/sendmail.rs
> new file mode 100644
> index 00000000..8eafe359
> --- /dev/null
> +++ b/proxmox-notify/src/api/sendmail.rs
> @@ -0,0 +1,254 @@
> +use crate::api::ApiError;
> +use crate::endpoints::sendmail::{
> + DeleteableSendmailProperty, SendmailConfig, SendmailConfigUpdater, SENDMAIL_TYPENAME,
> +};
> +use crate::Config;
> +
> +/// Get a list of all sendmail endpoints.
> +///
> +/// The caller is responsible for any needed permission checks.
> +/// Returns a list of all sendmail endpoints or an `ApiError` if the config is erroneous.
> +pub fn get_endpoints(config: &Config) -> Result<Vec<SendmailConfig>, ApiError> {
> + config
> + .config
> + .convert_to_typed_array(SENDMAIL_TYPENAME)
> + .map_err(|e| ApiError::internal_server_error("Could not fetch endpoints", Some(e.into())))
> +}
> +
> +/// Get sendmail endpoint with given `name`.
> +///
> +/// The caller is responsible for any needed permission checks.
> +/// Returns the endpoint or an `ApiError` if the endpoint was not found.
> +pub fn get_endpoint(config: &Config, name: &str) -> Result<SendmailConfig, ApiError> {
> + config
> + .config
> + .lookup(SENDMAIL_TYPENAME, name)
> + .map_err(|_| ApiError::not_found(format!("endpoint '{name}' not found"), None))
^ Technically `.lookup()` could have found the name but as a wrong type.
It might make sense to not use `.lookup()` in this case.
> +}
> +
(...)
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 08/66] notify: api: add API for gotify endpoints
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 08/66] notify: api: add API for gotify endpoints Lukas Wagner
@ 2023-07-18 12:44 ` Wolfgang Bumiller
2023-07-18 13:19 ` Lukas Wagner
0 siblings, 1 reply; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-18 12:44 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
On Mon, Jul 17, 2023 at 04:59:53PM +0200, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> proxmox-notify/src/api/gotify.rs | 284 +++++++++++++++++++++++++++++++
> proxmox-notify/src/api/mod.rs | 6 +
> 2 files changed, 290 insertions(+)
> create mode 100644 proxmox-notify/src/api/gotify.rs
>
> diff --git a/proxmox-notify/src/api/gotify.rs b/proxmox-notify/src/api/gotify.rs
> new file mode 100644
> index 00000000..fcc1aabf
> --- /dev/null
> +++ b/proxmox-notify/src/api/gotify.rs
> @@ -0,0 +1,284 @@
> +use crate::api::ApiError;
> +use crate::endpoints::gotify::{
> + DeleteableGotifyProperty, GotifyConfig, GotifyConfigUpdater, GotifyPrivateConfig,
> + GotifyPrivateConfigUpdater, GOTIFY_TYPENAME,
> +};
> +use crate::Config;
> +
> +/// Get a list of all gotify endpoints.
> +///
> +/// The caller is responsible for any needed permission checks.
> +/// Returns a list of all gotify endpoints or an `ApiError` if the config is erroneous.
> +pub fn get_endpoints(config: &Config) -> Result<Vec<GotifyConfig>, ApiError> {
> + config
> + .config
> + .convert_to_typed_array(GOTIFY_TYPENAME)
> + .map_err(|e| ApiError::internal_server_error("Could not fetch endpoints", Some(e.into())))
> +}
> +
> +/// Get gotify endpoint with given `name`
> +///
> +/// The caller is responsible for any needed permission checks.
> +/// Returns the endpoint or an `ApiError` if the endpoint was not found.
> +pub fn get_endpoint(config: &Config, name: &str) -> Result<GotifyConfig, ApiError> {
> + config
> + .config
> + .lookup(GOTIFY_TYPENAME, name)
> + .map_err(|_| ApiError::not_found(format!("endpoint '{name}' not found"), None))
> +}
> +
> +/// Add a new gotify endpoint.
> +///
> +/// The caller is responsible for any needed permission checks.
> +/// The caller also responsible for locking the configuration files.
> +/// Returns an `ApiError` if an endpoint with the same name already exists,
> +/// or if the endpoint could not be saved.
> +pub fn add_endpoint(
> + config: &mut Config,
> + endpoint_config: &GotifyConfig,
> + private_endpoint_config: &GotifyPrivateConfig,
> +) -> Result<(), ApiError> {
> + if endpoint_config.name != private_endpoint_config.name {
> + // Programming error by the user of the crate, thus we panic
> + panic!("name for endpoint config and private config must be identical");
> + }
> +
> + if super::endpoint_exists(config, &endpoint_config.name) {
(*could* dedup the whole if into a helper in `super` so we don't need to
copy-pasta the message to every new endpoint - we already have this
twice now ;-) )
> + return Err(ApiError::bad_request(
> + format!(
> + "endpoint with name '{}' already exists!",
> + endpoint_config.name
> + ),
> + None,
> + ));
> + }
> +
> + set_private_config_entry(config, private_endpoint_config)?;
> +
> + config
> + .config
> + .set_data(&endpoint_config.name, GOTIFY_TYPENAME, endpoint_config)
> + .map_err(|e| {
> + ApiError::internal_server_error(
> + format!("could not save endpoint '{}'", endpoint_config.name),
> + Some(e.into()),
> + )
> + })?;
> +
> + Ok(())
> +}
> +
(...)
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 15/66] notify: add context
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 15/66] notify: add context Lukas Wagner
@ 2023-07-18 12:57 ` Wolfgang Bumiller
0 siblings, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-18 12:57 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
On Mon, Jul 17, 2023 at 05:00:00PM +0200, Lukas Wagner wrote:
> Since `proxmox-notify` is intended to be used by multiple products,
> there needs to be a way to inject product-specific behavior.
>
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> proxmox-notify/src/context.rs | 13 +++++++++++++
> proxmox-notify/src/lib.rs | 1 +
> 2 files changed, 14 insertions(+)
> create mode 100644 proxmox-notify/src/context.rs
>
> diff --git a/proxmox-notify/src/context.rs b/proxmox-notify/src/context.rs
> new file mode 100644
> index 00000000..55c0eda1
> --- /dev/null
> +++ b/proxmox-notify/src/context.rs
> @@ -0,0 +1,13 @@
> +use std::sync::Mutex;
> +
> +pub trait Context: Send + Sync {}
> +
> +static CONTEXT: Mutex<Option<&'static dyn Context>> = Mutex::new(None);
Hmmm.
Given how this will be accsssed, we could probably just use a Cell
instead of a Mutex to make this even cheaper...
> +
> +pub fn set_context(context: &'static dyn Context) {
> + *CONTEXT.lock().unwrap() = Some(context);
> +}
> +
> +pub(crate) fn context() -> &'static dyn Context {
> + (*CONTEXT.lock().unwrap()).expect("context for proxmox-notify has not been set yet")
> +}
> diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
> index 548cc56f..3c2b6d55 100644
> --- a/proxmox-notify/src/lib.rs
> +++ b/proxmox-notify/src/lib.rs
> @@ -13,6 +13,7 @@ use std::error::Error as StdError;
>
> pub mod api;
> mod config;
> +pub mod context;
> pub mod endpoints;
> pub mod filter;
> pub mod group;
> --
> 2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 19/66] notify: api: allow to query entities referenced by filter/target
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 19/66] notify: api: allow to query entities referenced by filter/target Lukas Wagner
@ 2023-07-18 13:02 ` Wolfgang Bumiller
0 siblings, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-18 13:02 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
On Mon, Jul 17, 2023 at 05:00:04PM +0200, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> proxmox-notify/src/api/common.rs | 11 +++
> proxmox-notify/src/api/mod.rs | 125 +++++++++++++++++++++++++++++++
> 2 files changed, 136 insertions(+)
>
> diff --git a/proxmox-notify/src/api/common.rs b/proxmox-notify/src/api/common.rs
> index 518caa8f..48761fbb 100644
> --- a/proxmox-notify/src/api/common.rs
> +++ b/proxmox-notify/src/api/common.rs
> @@ -42,3 +42,14 @@ pub fn test_target(config: &Config, endpoint: &str) -> Result<(), ApiError> {
>
> Ok(())
> }
> +
> +/// Return all entities (targets, groups, filters) that are linked to the entity.
> +/// For instance, if a group 'grp1' contains the targets 'a', 'b' and 'c',
> +/// where grp1 has 'filter1' and 'a' has 'filter2' as filters, then
> +/// the result for 'grp1' would be [grp1, a, b, c, filter1, filter2].
> +/// The result will always contain the entity that was passed as a parameter.
> +/// If the entity does not exist, the result will only contain the entity.
> +pub fn get_referenced_entities(config: &Config, entity: &str) -> Result<Vec<String>, ApiError> {
> + let entities = super::get_referenced_entities(config, entity);
> + Ok(Vec::from_iter(entities.into_iter()))
> +}
> diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
> index 12811baf..d8a44bf2 100644
> --- a/proxmox-notify/src/api/mod.rs
> +++ b/proxmox-notify/src/api/mod.rs
> @@ -1,3 +1,4 @@
> +use std::collections::HashSet;
> use std::error::Error as StdError;
> use std::fmt::Display;
>
> @@ -101,6 +102,48 @@ fn endpoint_exists(config: &Config, name: &str) -> bool {
> exists
> }
>
> +fn get_referenced_entities(config: &Config, entity: &str) -> HashSet<String> {
> + let mut to_expand = HashSet::new();
> + let mut expanded = HashSet::new();
> + to_expand.insert(entity.to_string());
> +
> + let expand = |entities: &HashSet<String>| -> HashSet<String> {
> + let mut new = HashSet::new();
> +
> + for entity in entities {
> + if let Ok(group) = group::get_group(config, entity) {
> + for target in group.endpoint {
> + new.insert(target.clone());
> + }
> + }
> +
> + #[cfg(feature = "sendmail")]
> + if let Ok(target) = sendmail::get_endpoint(config, entity) {
> + if let Some(filter) = target.filter {
> + new.insert(filter.clone());
> + }
> + }
> +
> + #[cfg(feature = "gotify")]
> + if let Ok(target) = gotify::get_endpoint(config, entity) {
> + if let Some(filter) = target.filter {
> + new.insert(filter.clone());
> + }
> + }
> + }
> +
> + new
> + };
> +
> + while !to_expand.is_empty() {
> + let new = expand(&to_expand);
> + expanded.extend(to_expand.drain());
^ could drop the `drain()` call and move it in since you replace it with
new afterwards anyway
> + to_expand = new;
> + }
> +
> + expanded
> +}
> +
> #[cfg(test)]
> mod test_helpers {
> use crate::Config;
(...)
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system
2023-07-18 12:34 ` [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce " Dominik Csapak
@ 2023-07-18 13:14 ` Lukas Wagner
2023-07-18 13:58 ` Dominik Csapak
2023-07-18 14:37 ` Thomas Lamprecht
2023-07-19 8:40 ` Lukas Wagner
2 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-18 13:14 UTC (permalink / raw)
To: Dominik Csapak, Proxmox VE development discussion
On 7/18/23 14:34, Dominik Csapak wrote:
> gave the series a quick spin, review of the gui patches comes later ;)
>
> a few high level comments from a user perspective:
>
> * the node fencing/replication edit windows always shows the warning that it shouldn't be
> disabled, that should imo only be there if i select 'never' ?
> (conversely, the package update window never shows the warning...)
Good point, I'll try to make it only show up if 'never' is selected.
The warning came actually to be as an input from Aaron, so that user's don't turn
off stuff for critical events without knowing what they are doing.
I didn't add them to the package update notifications because there, the setting
already existed in datacenter config without a warning and it seemed less critical to me.
But yeah, I guess it would make sense to add it there as well.
> * when we have this, we could remove the pacakge notify line from the datacenter options, no?
Yes, you are right, forgot about that.
> * imho having "Notification Targets" directly below "Notifications" is a bit redundant
> (and wasting space), but it's just a minor thing
True, I can probably remove that, since it's already clear from the menu on the left-hand side.
> * the filter 'mode' is also not exposed on the gui (intentionally?)> * also the mode is not quite clear since only one filter can be active per target?
> (or is it meant when there are multiple filter by means of notification groups?)> * what is a filter without a minimum severity? what does it do?
> (does it make sense to allow such filters in the first place?)
Filters will be extended in the future so that they can match on multiple properties.
Every notification has a severity as well as arbitrary metadata attached to it (e.g. could be
hostname, backup-job ID, etc.).
In future, a filter could be like
filter: foo
mode and|or
min-severity error
match-property hostname=pali
So here, the mode determines if `min-severity` AND/OR `match-property` have to match.
That is also the reason why a filter without min-severity can exist.
Also, I'm thinking of supporting 'sub-filters':
filter: foo
mode or
sub-filter a
sub-filter b
filter: a
mode and
min-severity error
match-property hostname=pali
filter: b
mode and
min-severity info
match-property hostname=haya
`mode`, `invert-match` and `sub-filter` together basically would allow users to
create arbitrarily complex filter 'formulas'.
I already have patches laying around that implement the additional filter matchers, but
decided to not include them in the patch series yet. Before the new matchers are merged,
I would need to 'stabilize' the properties associate with every single notification event,
as at that point those become part of our public facing API. At the moment, the properties
are only an implementation detail that is used for rendering notification templates.
This is also the reason why the filter implementation (filter.rs) is somewhat overkill
atm for _just_ severity filtering. Everything needed for these advanced features is already
in place - because I already implemented that stuff and cut it out later for the patch series.
> * the backup edit window is rather tall at this point, and if we assume a minimum
> screen size of 1280x720 there is not quite enough space when you subtract
> the (typical) os bar and browser window decoration, maybe a 'notification'
> tab would be better (we already have some tabs there anyway)
Good point, will look into that.
> * i found one bug, but not quite sure yet where it comes from exactly,
> putting in emojis into a field (e.g. a comment or author) it's accepted,
> but editing a different entry fails with:
>
> --->8---
> could not serialize configuration: writing 'notifications.cfg' failed: detected unexpected control character in section 'testgroup' key 'comment' (500)
> ---8<---
>
> not sure where the utf-8 info gets lost. (or we could limit all fields to ascii?)
> such a notification target still works AFAICT (but if set as e.g. the author it's
> probably the wrong value)
>
> (i used 😀 as a test)
I will investigate, thanks!
>
>
>
> otherwise works AFAICT
>
Thanks so much for giving this a spin!
--
- Lukas
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 08/66] notify: api: add API for gotify endpoints
2023-07-18 12:44 ` Wolfgang Bumiller
@ 2023-07-18 13:19 ` Lukas Wagner
0 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-18 13:19 UTC (permalink / raw)
To: Wolfgang Bumiller; +Cc: pve-devel
On 7/18/23 14:44, Wolfgang Bumiller wrote:
>> + }
>> +
>> + if super::endpoint_exists(config, &endpoint_config.name) {
>
> (*could* dedup the whole if into a helper in `super` so we don't need to
> copy-pasta the message to every new endpoint - we already have this
> twice now ;-) )
>
Yup, a later commit actually moves that to a common helper `super::ensure_unique`.
--
- Lukas
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 20/66] notify: on deletion, check if a filter/endp. is still used by anything
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 20/66] notify: on deletion, check if a filter/endp. is still used by anything Lukas Wagner
@ 2023-07-18 13:20 ` Wolfgang Bumiller
0 siblings, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-18 13:20 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
On Mon, Jul 17, 2023 at 05:00:05PM +0200, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> proxmox-notify/src/api/filter.rs | 1 +
> proxmox-notify/src/api/gotify.rs | 1 +
> proxmox-notify/src/api/mod.rs | 113 ++++++++++++++++++++++++++---
> proxmox-notify/src/api/sendmail.rs | 1 +
> 4 files changed, 106 insertions(+), 10 deletions(-)
>
> diff --git a/proxmox-notify/src/api/filter.rs b/proxmox-notify/src/api/filter.rs
> index 3fcff6b9..824f802d 100644
> --- a/proxmox-notify/src/api/filter.rs
> +++ b/proxmox-notify/src/api/filter.rs
> @@ -115,6 +115,7 @@ pub fn update_filter(
> pub fn delete_filter(config: &mut Config, name: &str) -> Result<(), ApiError> {
> // Check if the filter exists
> let _ = get_filter(config, name)?;
> + super::ensure_unused(config, name)?;
>
> config.config.sections.remove(name);
>
> diff --git a/proxmox-notify/src/api/gotify.rs b/proxmox-notify/src/api/gotify.rs
> index d6f33064..5c4db4be 100644
> --- a/proxmox-notify/src/api/gotify.rs
> +++ b/proxmox-notify/src/api/gotify.rs
> @@ -145,6 +145,7 @@ pub fn update_endpoint(
> pub fn delete_gotify_endpoint(config: &mut Config, name: &str) -> Result<(), ApiError> {
> // Check if the endpoint exists
> let _ = get_endpoint(config, name)?;
> + super::ensure_unused(config, name)?;
>
> remove_private_config_entry(config, name)?;
> config.config.sections.remove(name);
> diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
> index d8a44bf2..81c182c7 100644
> --- a/proxmox-notify/src/api/mod.rs
> +++ b/proxmox-notify/src/api/mod.rs
> @@ -102,6 +102,59 @@ fn endpoint_exists(config: &Config, name: &str) -> bool {
> exists
> }
>
> +fn get_referrers(config: &Config, entity: &str) -> Result<HashSet<String>, ApiError> {
> + let mut referrers = HashSet::new();
> +
> + for group in group::get_groups(config)? {
> + for endpoint in group.endpoint {
> + if endpoint == entity {
> + referrers.insert(group.name.clone());
We could `break` here, since `group` comes from the outer loop.
Or, in fact, replace the inner for loop with:
if group.endpoint.iter().any(|endpoint| endpoint == entity) {
referrers.insert(group.name.clone());
}
> + }
> + }
> +
> + if let Some(filter) = group.filter {
> + if filter == entity {
> + referrers.insert(group.name);
> + }
> + }
> + }
> +
> + #[cfg(feature = "sendmail")]
> + for endpoint in sendmail::get_endpoints(config)? {
> + if let Some(filter) = endpoint.filter {
> + if filter == entity {
> + referrers.insert(endpoint.name);
> + }
> + }
> + }
> +
> + #[cfg(feature = "gotify")]
> + for endpoint in gotify::get_endpoints(config)? {
> + if let Some(filter) = endpoint.filter {
> + if filter == entity {
> + referrers.insert(endpoint.name);
> + }
> + }
> + }
> +
> + Ok(referrers)
> +}
> +
> +fn ensure_unused(config: &Config, entity: &str) -> Result<(), ApiError> {
> + let referrers = get_referrers(config, entity)?;
> +
> + if !referrers.is_empty() {
> + let used_by = referrers.into_iter().collect::<Vec<_>>().join(", ");
*sigh*...
iterators vs join...
Some day this can hopefully be
refererrs
.into_iter()
.map(Cow::Owned)
.intersperse(Cow::Borrowed(", "))
.collect::<String>();
Yeah okay fine, the ergonomics with `String` vs the `&'static str`
aren't great either... exactly as long as the for loop variant, but w/e.
Could also just be
referrers.iter().map(String::as_str).intersperse(", ").collect::<String>();
... Oh well ...
> +
> + return Err(ApiError::bad_request(
> + format!("cannot delete '{entity}', referenced by: {used_by}"),
> + None,
> + ));
> + }
> +
> + Ok(())
> +}
> +
> fn get_referenced_entities(config: &Config, entity: &str) -> HashSet<String> {
> let mut to_expand = HashSet::new();
> let mut expanded = HashSet::new();
(...)
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 23/66] notify: add debian packaging
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 23/66] notify: add debian packaging Lukas Wagner
@ 2023-07-18 13:25 ` Wolfgang Bumiller
0 siblings, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-18 13:25 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
On Mon, Jul 17, 2023 at 05:00:08PM +0200, Lukas Wagner wrote:
(...)
> +++ b/proxmox-notify/debian/copyright
This is still the old template.
> @@ -0,0 +1,16 @@
> +Copyright (C) 2023 Proxmox Server Solutions GmbH
> +
> +This software is written by Proxmox Server Solutions GmbH <support@proxmox.com>
> +
> +This program is free software: you can redistribute it and/or modify
> +it under the terms of the GNU Affero General Public License as published by
> +the Free Software Foundation, either version 3 of the License, or
> +(at your option) any later version.
> +
> +This program is distributed in the hope that it will be useful,
> +but WITHOUT ANY WARRANTY; without even the implied warranty of
> +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +GNU Affero General Public License for more details.
> +
> +You should have received a copy of the GNU Affero General Public License
> +along with this program. If not, see <http://www.gnu.org/licenses/>.
> diff --git a/proxmox-notify/debian/debcargo.toml b/proxmox-notify/debian/debcargo.toml
> new file mode 100644
> index 00000000..b7864cdb
(...)
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (66 preceding siblings ...)
2023-07-18 12:34 ` [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce " Dominik Csapak
@ 2023-07-18 13:27 ` Wolfgang Bumiller
2023-07-19 12:11 ` Wolfgang Bumiller
68 siblings, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-18 13:27 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
The `proxmox.git` part here so far is
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Requested changes can happen as follow-ups.
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system
2023-07-18 13:14 ` Lukas Wagner
@ 2023-07-18 13:58 ` Dominik Csapak
2023-07-18 14:07 ` Lukas Wagner
0 siblings, 1 reply; 114+ messages in thread
From: Dominik Csapak @ 2023-07-18 13:58 UTC (permalink / raw)
To: Lukas Wagner, Proxmox VE development discussion
On 7/18/23 15:14, Lukas Wagner wrote:
> On 7/18/23 14:34, Dominik Csapak wrote:
>> gave the series a quick spin, review of the gui patches comes later ;)
>>
>> a few high level comments from a user perspective:
>>
>> * the node fencing/replication edit windows always shows the warning that it shouldn't be
>> disabled, that should imo only be there if i select 'never' ?
>> (conversely, the package update window never shows the warning...)
> Good point, I'll try to make it only show up if 'never' is selected.
> The warning came actually to be as an input from Aaron, so that user's don't turn
> off stuff for critical events without knowing what they are doing.
> I didn't add them to the package update notifications because there, the setting
> already existed in datacenter config without a warning and it seemed less critical to me.
> But yeah, I guess it would make sense to add it there as well.
>
>> * when we have this, we could remove the pacakge notify line from the datacenter options, no?
>
> Yes, you are right, forgot about that.
>
>> * imho having "Notification Targets" directly below "Notifications" is a bit redundant
>> (and wasting space), but it's just a minor thing
>
> True, I can probably remove that, since it's already clear from the menu on the left-hand side.
>
>> * the filter 'mode' is also not exposed on the gui (intentionally?)> * also the mode is not quite
>> clear since only one filter can be active per target?
>> (or is it meant when there are multiple filter by means of notification groups?)> * what is a
>> filter without a minimum severity? what does it do?
>> (does it make sense to allow such filters in the first place?)
>
> Filters will be extended in the future so that they can match on multiple properties.
> Every notification has a severity as well as arbitrary metadata attached to it (e.g. could be
> hostname, backup-job ID, etc.).
>
> In future, a filter could be like
>
> filter: foo
> mode and|or
> min-severity error
> match-property hostname=pali
>
>
> So here, the mode determines if `min-severity` AND/OR `match-property` have to match.
> That is also the reason why a filter without min-severity can exist.
>
> Also, I'm thinking of supporting 'sub-filters':
>
> filter: foo
> mode or
> sub-filter a
> sub-filter b
>
> filter: a
> mode and
> min-severity error
> match-property hostname=pali
>
> filter: b
> mode and
> min-severity info
> match-property hostname=haya
>
> `mode`, `invert-match` and `sub-filter` together basically would allow users to
> create arbitrarily complex filter 'formulas'.
>
>
> I already have patches laying around that implement the additional filter matchers, but
> decided to not include them in the patch series yet. Before the new matchers are merged,
> I would need to 'stabilize' the properties associate with every single notification event,
> as at that point those become part of our public facing API. At the moment, the properties
> are only an implementation detail that is used for rendering notification templates.
>
> This is also the reason why the filter implementation (filter.rs) is somewhat overkill
> atm for _just_ severity filtering. Everything needed for these advanced features is already
> in place - because I already implemented that stuff and cut it out later for the patch series.
ah ok, so the mode is currently unused.
one of my questions remains though, does it make sense to configure a filter without
any filtering properties? i guess not really
>
>> * the backup edit window is rather tall at this point, and if we assume a minimum
>> screen size of 1280x720 there is not quite enough space when you subtract
>> the (typical) os bar and browser window decoration, maybe a 'notification'
>> tab would be better (we already have some tabs there anyway)
>
> Good point, will look into that.
>
>> * i found one bug, but not quite sure yet where it comes from exactly,
>> putting in emojis into a field (e.g. a comment or author) it's accepted,
>> but editing a different entry fails with:
>>
>> --->8---
>> could not serialize configuration: writing 'notifications.cfg' failed: detected unexpected control
>> character in section 'testgroup' key 'comment' (500)
>> ---8<---
>>
>> not sure where the utf-8 info gets lost. (or we could limit all fields to ascii?)
>> such a notification target still works AFAICT (but if set as e.g. the author it's
>> probably the wrong value)
>>
>> (i used 😀 as a test)
>
> I will investigate, thanks!
>>
>>
>>
>> otherwise works AFAICT
>>
>
> Thanks so much for giving this a spin!
>
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system
2023-07-18 13:58 ` Dominik Csapak
@ 2023-07-18 14:07 ` Lukas Wagner
0 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-18 14:07 UTC (permalink / raw)
To: Dominik Csapak, Proxmox VE development discussion
On 7/18/23 15:58, Dominik Csapak wrote:
>> I already have patches laying around that implement the additional filter matchers, but
>> decided to not include them in the patch series yet. Before the new matchers are merged,
>> I would need to 'stabilize' the properties associate with every single notification event,
>> as at that point those become part of our public facing API. At the moment, the properties
>> are only an implementation detail that is used for rendering notification templates.
>>
>> This is also the reason why the filter implementation (filter.rs) is somewhat overkill
>> atm for _just_ severity filtering. Everything needed for these advanced features is already
>> in place - because I already implemented that stuff and cut it out later for the patch series.
>
> ah ok, so the mode is currently unused.
> one of my questions remains though, does it make sense to configure a filter without
> any filtering properties? i guess not really
>
Yes, I guess a filter without any matchers does not make much sense. I could add a check
that ensures that at least one is configured - making the min-severity matcher required for now, as there
are not any other matchers.
Thanks!
--
- Lukas
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system
2023-07-18 12:34 ` [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce " Dominik Csapak
2023-07-18 13:14 ` Lukas Wagner
@ 2023-07-18 14:37 ` Thomas Lamprecht
2023-07-19 13:13 ` Lukas Wagner
2023-07-19 8:40 ` Lukas Wagner
2 siblings, 1 reply; 114+ messages in thread
From: Thomas Lamprecht @ 2023-07-18 14:37 UTC (permalink / raw)
To: Proxmox VE development discussion, Dominik Csapak, Lukas Wagner
Am 18/07/2023 um 14:34 schrieb Dominik Csapak:
> * the backup edit window is rather tall at this point, and if we assume a minimum
> screen size of 1280x720 there is not quite enough space when you subtract
> the (typical) os bar and browser window decoration, maybe a 'notification'
> tab would be better (we already have some tabs there anyway)
FWIW and not that I'm against tabs here (I didn't even checked the GUI at all), I
came to the conclusion that 720p *and* OS/browser bar deduction is just to small
to be practical.. 768p (1366 x 768) and ~90px deduction for height and 66px for
those desktops that have the bar configured on the side would be more practicable,
leaving 1300 x 680 as minimum.
As alternative we can say 720p but in full-screen mode, i.e., so that the whole
1280 x 720 pixels are available for the web UI, or at least say that overflow is
OK with smaller sizes but we ensure for scroll overflow handler on those dialogues.
Could be also maybe added at a general place in EditWindow, or InputPanel if the
viewport is really small, one would need to experiment with this a bit though..
Whatever the decision then is (I'm open for opinions, but pretty much set that
making all work very nicely for sizes around 1200 x 600 is going to be a PITA
(or just overlooked), especially as we gain more complex features), we should
then update docs and maybe check if we can augment the (Edit)Window such that
it outputs a warning (or even shows an error) if in debug mode and a window is
bigger than the (newly) decided size, with manual resize being excluded from
that check – then we'd have a higher chance in actually enforcing such a limit.
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system
2023-07-18 12:34 ` [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce " Dominik Csapak
2023-07-18 13:14 ` Lukas Wagner
2023-07-18 14:37 ` Thomas Lamprecht
@ 2023-07-19 8:40 ` Lukas Wagner
2023-07-19 9:54 ` Wolfgang Bumiller
2 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-19 8:40 UTC (permalink / raw)
To: Dominik Csapak, Proxmox VE development discussion,
Wolfgang Bumiller, Maximiliano Sandoval
Hi again,
On 7/18/23 14:34, Dominik Csapak wrote:
> * i found one bug, but not quite sure yet where it comes from exactly,
> putting in emojis into a field (e.g. a comment or author) it's accepted,
> but editing a different entry fails with:
>
> --->8---
> could not serialize configuration: writing 'notifications.cfg' failed: detected unexpected control character in section 'testgroup' key 'comment' (500)
> ---8<---
>
> not sure where the utf-8 info gets lost. (or we could limit all fields to ascii?)
> such a notification target still works AFAICT (but if set as e.g. the author it's
> probably the wrong value)
>
> (i used 😀 as a test)
So I investigated a bit and found a minimal reproducer. Turns out it's an encoding issue
in the FFI interface (perl->rust).
Let's assume that we have the following exported function in the pve-rs bindings:
#[export]
fn test_emoji(name: &str) {
dbg!(&name);
}
use PVE::RS::Notify;
my $str = "😊";
PVE::RS::Notify::test_emoji($str);
root@pve:~# perl test.pl
[src/notify.rs:562] &name = "ð\u{9f}\u{98}\u{8a}"
To me it looks a bit like a UTF-16/UTF-8 mixup:
ð = 0x00F0 in UTF16
😊 = 0xF0 0x9F 0x98 0x8A in UTF-8
The issue can be fixed by doing a `$str = encode('utf-8', $str);` before calling
`test_emoji`.
However, I think this should be probably handled automagically by the perlmod bindings, if
at all possible?
@Wolfgang, what are your thoughts about this? Maximiliano said he is going to take a look
at the perlmod code, but if you have any idea about where to fix this issue, then
this would probably be helpful to him.
--
- Lukas
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system
2023-07-19 8:40 ` Lukas Wagner
@ 2023-07-19 9:54 ` Wolfgang Bumiller
0 siblings, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-19 9:54 UTC (permalink / raw)
To: Lukas Wagner
Cc: Dominik Csapak, Proxmox VE development discussion, Maximiliano Sandoval
On Wed, Jul 19, 2023 at 10:40:09AM +0200, Lukas Wagner wrote:
> Hi again,
>
> On 7/18/23 14:34, Dominik Csapak wrote:
> > * i found one bug, but not quite sure yet where it comes from exactly,
> > putting in emojis into a field (e.g. a comment or author) it's accepted,
> > but editing a different entry fails with:
> >
> > --->8---
> > could not serialize configuration: writing 'notifications.cfg' failed: detected unexpected control character in section 'testgroup' key 'comment' (500)
> > ---8<---
> >
> > not sure where the utf-8 info gets lost. (or we could limit all fields to ascii?)
> > such a notification target still works AFAICT (but if set as e.g. the author it's
> > probably the wrong value)
> >
> > (i used 😀 as a test)
>
> So I investigated a bit and found a minimal reproducer. Turns out it's an encoding issue
> in the FFI interface (perl->rust).
>
> Let's assume that we have the following exported function in the pve-rs bindings:
>
> #[export]
> fn test_emoji(name: &str) {
> dbg!(&name);
> }
>
>
>
> use PVE::RS::Notify;
> my $str = "😊";
Without `use utf8;`, this produces a "byte string":
$ perl -MDevel::Peek -e 'my $str = "😊"; Dump($str);'
SV = PV(0x5576f4e0cea0) at 0x5576f4e39370
REFCNT = 1
FLAGS = (POK,IsCOW,pPOK)
PV = 0x5576f4e424d0 "\xF0\x9F\x98\x8A"\0
CUR = 4
LEN = 10
COW_REFCNT = 1
Note that \xF0\x9F\x98\x8A.
> PVE::RS::Notify::test_emoji($str);
>
>
> root@pve:~# perl test.pl
> [src/notify.rs:562] &name = "ð\u{9f}\u{98}\u{8a}"
Note the `\u` portions here. This string contains
the *UTF-8* characters 0xF0, 0x9F, 0x98, 0x8A.
And how is it supposed to know any better.
>
> To me it looks a bit like a UTF-16/UTF-8 mixup:
>
> ð = 0x00F0 in UTF16
> 😊 = 0xF0 0x9F 0x98 0x8A in UTF-8
>
> The issue can be fixed by doing a `$str = encode('utf-8', $str);` before calling
> `test_emoji`.
Perl and most of our perl code never cared (hence we already ran into a
bunch of utf-8 issues and for a long time did the whole "transport
encoding vs actual encoding" in HTTP vs JS vs json vs perl strings
completely *wrong* (and probably still do)), and a lot of *files* aren't
even *defined* to have a specific encoding (eg. interpreting bytes >0x80
from `/etc/network/interfaces` as utf-8 may simply be the *wrong* thing
to do).
Sure, the perlmod layer could be an issue. But I wouldn't jump to
conclusions there.
Also, note what *actually* happens if you `encode('utf-8', $str)`:
$ perl -MEncode -MDevel::Peek -e 'my $a = encode("utf-8", "👍"); Dump($a);'
SV = PV(0x55f62dfe6170) at 0x55f62e012430
REFCNT = 1
FLAGS = (POK,IsCOW,pPOK)
PV = 0x55f62e1cbe60 "\xC3\xB0\xC2\x9F\xC2\x91\xC2\x8D"\0
CUR = 8
LEN = 10
COW_REFCNT = 0
Now you have the UTF-8 encoding of each character in there explicitly.
What you really want would be for perl to acknowledge that you already
have utf-8:
$ perl -MDevel::Peek -e 'use utf8; my $a = "👍"; Dump($a);'
SV = PV(0x56265764cea0) at 0x5626576793e8
REFCNT = 1
FLAGS = (POK,IsCOW,pPOK,UTF8)
PV = 0x5626576a2b60 "\xF0\x9F\x91\x8D"\0 [UTF8 "\x{1f44d}"]
CUR = 4
LEN = 10
COW_REFCNT = 1
But we don't use `use utf8;` in our code base because it has too many
side effects.
To mark an utf-8 encoded not-as-utf-8-marked string as utf-8 in perl,
you can *decode* it:
$ perl -MDevel::Peek -e 'use utf8; no utf8; my $a = "👍"; utf8::decode($a); Dump($a);'
SV = PV(0x55a5c9c45ea0) at 0x55a5c9c723a8
REFCNT = 1
FLAGS = (POK,pPOK,UTF8)
PV = 0x55a5c9c64280 "\xF0\x9F\x91\x8D"\0 [UTF8 "\x{1f44d}"]
CUR = 4
LEN = 10
All that said, I have not yet looked at the perl side (or perlmod side)
and cannot say what's going on.
But If you hand utf-8 *bytes* which aren't marked as utf-8 to perlmod,
it'll do what perl does and just encode each byte as utf-8.
*Guessing* that it's utf-8 would surely work - *this time* - but might
simply be *wrong* other times.
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox-perl-rs 24/66] add PVE::RS::Notify module
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 24/66] add PVE::RS::Notify module Lukas Wagner
@ 2023-07-19 10:10 ` Wolfgang Bumiller
2023-07-19 10:23 ` Wolfgang Bumiller
0 siblings, 1 reply; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-19 10:10 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
On Mon, Jul 17, 2023 at 05:00:09PM +0200, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> pve-rs/Cargo.toml | 1 +
> pve-rs/Makefile | 1 +
> pve-rs/src/lib.rs | 1 +
> pve-rs/src/notify.rs | 71 ++++++++++++++++++++++++++++++++++++++++++++
> 4 files changed, 74 insertions(+)
> create mode 100644 pve-rs/src/notify.rs
>
> diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
> index 3076a13..b28c118 100644
> --- a/pve-rs/Cargo.toml
> +++ b/pve-rs/Cargo.toml
> @@ -34,6 +34,7 @@ perlmod = { version = "0.13", features = [ "exporter" ] }
>
> proxmox-apt = "0.10"
> proxmox-http = { version = "0.9", features = ["client-sync", "client-trait"] }
> +proxmox-notify = "0.1"
> proxmox-openid = "0.10"
> proxmox-resource-scheduling = "0.3.0"
> proxmox-subscription = "0.4"
> diff --git a/pve-rs/Makefile b/pve-rs/Makefile
> index de35c69..9d737c0 100644
> --- a/pve-rs/Makefile
> +++ b/pve-rs/Makefile
> @@ -27,6 +27,7 @@ PERLMOD_GENPACKAGE := /usr/lib/perlmod/genpackage.pl \
>
> PERLMOD_PACKAGES := \
> PVE::RS::APT::Repositories \
> + PVE::RS::Notify \
> PVE::RS::OpenId \
> PVE::RS::ResourceScheduling::Static \
> PVE::RS::TFA
> diff --git a/pve-rs/src/lib.rs b/pve-rs/src/lib.rs
> index eb6ae02..0d63c28 100644
> --- a/pve-rs/src/lib.rs
> +++ b/pve-rs/src/lib.rs
> @@ -4,6 +4,7 @@
> pub mod common;
>
> pub mod apt;
> +pub mod notify;
> pub mod openid;
> pub mod resource_scheduling;
> pub mod tfa;
> diff --git a/pve-rs/src/notify.rs b/pve-rs/src/notify.rs
> new file mode 100644
> index 0000000..9677d8b
> --- /dev/null
> +++ b/pve-rs/src/notify.rs
> @@ -0,0 +1,71 @@
> +#[perlmod::package(name = "PVE::RS::Notify")]
> +mod export {
> + use anyhow::{bail, Error};
> + use perlmod::Value;
> +
> + use std::sync::Mutex;
> +
> + use proxmox_notify::Config;
> +
> + pub struct NotificationConfig {
> + config: Mutex<Config>,
> + }
> +
> + perlmod::declare_magic!(Box<NotificationConfig> : &NotificationConfig as "PVE::RS::Notify");
> +
> + /// Support `dclone` so this can be put into the `ccache` of `PVE::Cluster`.
> + #[export(name = "STORABLE_freeze", raw_return)]
> + fn storable_freeze(
> + #[try_from_ref] this: &NotificationConfig,
> + cloning: bool,
> + ) -> Result<Value, Error> {
> + if !cloning {
> + bail!("freezing Notification config not supported!");
> + }
> +
> + let mut cloned = Box::new(NotificationConfig {
> + config: Mutex::new(this.config.lock().unwrap().clone()),
> + });
> + let value = Value::new_pointer::<NotificationConfig>(&mut *cloned);
> + let _perl = Box::leak(cloned);
> + Ok(value)
> + }
> +
> + /// Instead of `thaw` we implement `attach` for `dclone`.
> + #[export(name = "STORABLE_attach", raw_return)]
> + fn storable_attach(
> + #[raw] class: Value,
> + cloning: bool,
> + #[raw] serialized: Value,
> + ) -> Result<Value, Error> {
> + if !cloning {
> + bail!("STORABLE_attach called with cloning=false");
> + }
> + let data = unsafe { Box::from_raw(serialized.pv_raw::<NotificationConfig>()?) };
> + Ok(perlmod::instantiate_magic!(&class, MAGIC => data))
> + }
> +
> + #[export(raw_return)]
> + fn parse_config(
> + #[raw] class: Value,
> + raw_config: &str,
> + raw_private_config: &str,
I half-suspect that using &[u8] here - since that's what perl *actually*
gives us if we just read the data out of files without setting the mode
to 'utf8' or anything - might just fix the emoji issues.
Alternatively, perl would have to actually declare an encoding ;-)
> + ) -> Result<Value, Error> {
> + Ok(perlmod::instantiate_magic!(&class, MAGIC => Box::new(
> + NotificationConfig {
> + config: Mutex::new(Config::new(raw_config, raw_private_config)?)
> + }
> + )))
> + }
> +
> + #[export]
> + fn write_config(#[try_from_ref] this: &NotificationConfig) -> Result<(String, String), Error> {
> + Ok(this.config.lock().unwrap().write()?)
> + }
> +
> + #[export]
> + fn digest(#[try_from_ref] this: &NotificationConfig) -> String {
> + let config = this.config.lock().unwrap();
> + hex::encode(config.digest())
> + }
> +}
> --
> 2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox-perl-rs 24/66] add PVE::RS::Notify module
2023-07-19 10:10 ` Wolfgang Bumiller
@ 2023-07-19 10:23 ` Wolfgang Bumiller
2023-07-19 10:37 ` Lukas Wagner
0 siblings, 1 reply; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-19 10:23 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
On Wed, Jul 19, 2023 at 12:10:02PM +0200, Wolfgang Bumiller wrote:
> On Mon, Jul 17, 2023 at 05:00:09PM +0200, Lukas Wagner wrote:
> > Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> > ---
> > pve-rs/Cargo.toml | 1 +
> > pve-rs/Makefile | 1 +
> > pve-rs/src/lib.rs | 1 +
> > pve-rs/src/notify.rs | 71 ++++++++++++++++++++++++++++++++++++++++++++
> > 4 files changed, 74 insertions(+)
> > create mode 100644 pve-rs/src/notify.rs
> >
> > diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
> > index 3076a13..b28c118 100644
> > --- a/pve-rs/Cargo.toml
> > +++ b/pve-rs/Cargo.toml
> > @@ -34,6 +34,7 @@ perlmod = { version = "0.13", features = [ "exporter" ] }
> >
> > proxmox-apt = "0.10"
> > proxmox-http = { version = "0.9", features = ["client-sync", "client-trait"] }
> > +proxmox-notify = "0.1"
> > proxmox-openid = "0.10"
> > proxmox-resource-scheduling = "0.3.0"
> > proxmox-subscription = "0.4"
> > diff --git a/pve-rs/Makefile b/pve-rs/Makefile
> > index de35c69..9d737c0 100644
> > --- a/pve-rs/Makefile
> > +++ b/pve-rs/Makefile
> > @@ -27,6 +27,7 @@ PERLMOD_GENPACKAGE := /usr/lib/perlmod/genpackage.pl \
> >
> > PERLMOD_PACKAGES := \
> > PVE::RS::APT::Repositories \
> > + PVE::RS::Notify \
> > PVE::RS::OpenId \
> > PVE::RS::ResourceScheduling::Static \
> > PVE::RS::TFA
> > diff --git a/pve-rs/src/lib.rs b/pve-rs/src/lib.rs
> > index eb6ae02..0d63c28 100644
> > --- a/pve-rs/src/lib.rs
> > +++ b/pve-rs/src/lib.rs
> > @@ -4,6 +4,7 @@
> > pub mod common;
> >
> > pub mod apt;
> > +pub mod notify;
> > pub mod openid;
> > pub mod resource_scheduling;
> > pub mod tfa;
> > diff --git a/pve-rs/src/notify.rs b/pve-rs/src/notify.rs
> > new file mode 100644
> > index 0000000..9677d8b
> > --- /dev/null
> > +++ b/pve-rs/src/notify.rs
> > @@ -0,0 +1,71 @@
> > +#[perlmod::package(name = "PVE::RS::Notify")]
> > +mod export {
> > + use anyhow::{bail, Error};
> > + use perlmod::Value;
> > +
> > + use std::sync::Mutex;
> > +
> > + use proxmox_notify::Config;
> > +
> > + pub struct NotificationConfig {
> > + config: Mutex<Config>,
> > + }
> > +
> > + perlmod::declare_magic!(Box<NotificationConfig> : &NotificationConfig as "PVE::RS::Notify");
> > +
> > + /// Support `dclone` so this can be put into the `ccache` of `PVE::Cluster`.
> > + #[export(name = "STORABLE_freeze", raw_return)]
> > + fn storable_freeze(
> > + #[try_from_ref] this: &NotificationConfig,
> > + cloning: bool,
> > + ) -> Result<Value, Error> {
> > + if !cloning {
> > + bail!("freezing Notification config not supported!");
> > + }
> > +
> > + let mut cloned = Box::new(NotificationConfig {
> > + config: Mutex::new(this.config.lock().unwrap().clone()),
> > + });
> > + let value = Value::new_pointer::<NotificationConfig>(&mut *cloned);
> > + let _perl = Box::leak(cloned);
> > + Ok(value)
> > + }
> > +
> > + /// Instead of `thaw` we implement `attach` for `dclone`.
> > + #[export(name = "STORABLE_attach", raw_return)]
> > + fn storable_attach(
> > + #[raw] class: Value,
> > + cloning: bool,
> > + #[raw] serialized: Value,
> > + ) -> Result<Value, Error> {
> > + if !cloning {
> > + bail!("STORABLE_attach called with cloning=false");
> > + }
> > + let data = unsafe { Box::from_raw(serialized.pv_raw::<NotificationConfig>()?) };
> > + Ok(perlmod::instantiate_magic!(&class, MAGIC => data))
> > + }
> > +
> > + #[export(raw_return)]
> > + fn parse_config(
> > + #[raw] class: Value,
> > + raw_config: &str,
> > + raw_private_config: &str,
>
> I half-suspect that using &[u8] here - since that's what perl *actually*
> gives us if we just read the data out of files without setting the mode
> to 'utf8' or anything - might just fix the emoji issues.
>
> Alternatively, perl would have to actually declare an encoding ;-)
Though ultimately the alter patches will probably need to do the same
for things where utf-8 matters (at least the comments).
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox-perl-rs 24/66] add PVE::RS::Notify module
2023-07-19 10:23 ` Wolfgang Bumiller
@ 2023-07-19 10:37 ` Lukas Wagner
0 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-19 10:37 UTC (permalink / raw)
To: Wolfgang Bumiller; +Cc: pve-devel
On 7/19/23 12:23, Wolfgang Bumiller wrote:
>>> + #[export(raw_return)]
>>> + fn parse_config(
>>> + #[raw] class: Value,
>>> + raw_config: &str,
>>> + raw_private_config: &str,
>>
>> I half-suspect that using &[u8] here - since that's what perl *actually*
>> gives us if we just read the data out of files without setting the mode
>> to 'utf8' or anything - might just fix the emoji issues.
>>
>> Alternatively, perl would have to actually declare an encoding ;-)
>
> Though ultimately the alter patches will probably need to do the same
> for things where utf-8 matters (at least the comments).
#[export(raw_return)]
fn parse_config(
#[raw] class: Value,
raw_config: &[u8],
raw_private_config: &[u8],
) -> Result<Value, Error> {
let raw_config = std::str::from_utf8(raw_config)?;
let raw_private_config = std::str::from_utf8(raw_private_config)?;
...
This seems to have done the trick. The `update` calls do not seem to need it,
it appears to work fine with just this modification.
Thanks a lot!
--
- Lukas
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox-perl-rs 31/66] notify: implement context for getting default author/mailfrom
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 31/66] notify: implement context for getting default author/mailfrom Lukas Wagner
@ 2023-07-19 11:15 ` Wolfgang Bumiller
0 siblings, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-19 11:15 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
On Mon, Jul 17, 2023 at 05:00:16PM +0200, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> pve-rs/src/notify.rs | 33 ++++++++++++++++++++++++++++++++-
> 1 file changed, 32 insertions(+), 1 deletion(-)
>
> diff --git a/pve-rs/src/notify.rs b/pve-rs/src/notify.rs
> index ea34bfe..04e902c 100644
> --- a/pve-rs/src/notify.rs
> +++ b/pve-rs/src/notify.rs
> @@ -34,6 +34,14 @@ fn lookup_mail_address(content: &str, user: &str) -> Option<String> {
> }))
> }
>
> +fn lookup_datacenter_config_key(content: &str, key: &str) -> Option<String> {
> + normalize_for_return(
> + content
> + .lines()
> + .find_map(|line| line.strip_prefix(&format!("{key}:"))),
nit: you could format!() first then `move |...|` it into the closure so
it doesn't get recreated on every closure invocation.
> + )
> +}
> +
> struct PVEContext;
>
> impl Context for PVEContext {
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 06/66] notify: api: add API for sendmail endpoints
2023-07-18 12:36 ` Wolfgang Bumiller
@ 2023-07-19 11:51 ` Lukas Wagner
2023-07-19 12:09 ` Wolfgang Bumiller
0 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-19 11:51 UTC (permalink / raw)
To: Wolfgang Bumiller; +Cc: pve-devel
On 7/18/23 14:36, Wolfgang Bumiller wrote:
>> +
>> +/// Get sendmail endpoint with given `name`.
>> +///
>> +/// The caller is responsible for any needed permission checks.
>> +/// Returns the endpoint or an `ApiError` if the endpoint was not found.
>> +pub fn get_endpoint(config: &Config, name: &str) -> Result<SendmailConfig, ApiError> {
>> + config
>> + .config
>> + .lookup(SENDMAIL_TYPENAME, name)
>> + .map_err(|_| ApiError::not_found(format!("endpoint '{name}' not found"), None))
>
> ^ Technically `.lookup()` could have found the name but as a wrong type.
> It might make sense to not use `.lookup()` in this case.
>
`.lookup()` should be fine here from what I can tell. I actually want a `not found` if there is
another endpoint with that name but a different type, because the endpoints have a distinct
typed path prefix in the REST API:
GET endpoints/sendmail/<name>
GET endpoints/gotify/<name>
--
- Lukas
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox 06/66] notify: api: add API for sendmail endpoints
2023-07-19 11:51 ` Lukas Wagner
@ 2023-07-19 12:09 ` Wolfgang Bumiller
0 siblings, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-19 12:09 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
On Wed, Jul 19, 2023 at 01:51:33PM +0200, Lukas Wagner wrote:
>
>
> On 7/18/23 14:36, Wolfgang Bumiller wrote:
> > > +
> > > +/// Get sendmail endpoint with given `name`.
> > > +///
> > > +/// The caller is responsible for any needed permission checks.
> > > +/// Returns the endpoint or an `ApiError` if the endpoint was not found.
> > > +pub fn get_endpoint(config: &Config, name: &str) -> Result<SendmailConfig, ApiError> {
> > > + config
> > > + .config
> > > + .lookup(SENDMAIL_TYPENAME, name)
> > > + .map_err(|_| ApiError::not_found(format!("endpoint '{name}' not found"), None))
> >
> > ^ Technically `.lookup()` could have found the name but as a wrong type.
> > It might make sense to not use `.lookup()` in this case.
> >
>
> `.lookup()` should be fine here from what I can tell. I actually want a `not found` if there is
> another endpoint with that name but a different type, because the endpoints have a distinct
> typed path prefix in the REST API:
>
> GET endpoints/sendmail/<name>
> GET endpoints/gotify/<name>
Fair enough.
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
` (67 preceding siblings ...)
2023-07-18 13:27 ` Wolfgang Bumiller
@ 2023-07-19 12:11 ` Wolfgang Bumiller
2023-07-19 12:17 ` Lukas Wagner
68 siblings, 1 reply; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-19 12:11 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
The proxmox-perl-rs part can be considered
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
The String -> &[u8] modification in the parse method may be a follow-up
or a v4 - depending on how we proceed with applying this the whole
thing.
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system
2023-07-19 12:11 ` Wolfgang Bumiller
@ 2023-07-19 12:17 ` Lukas Wagner
0 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-19 12:17 UTC (permalink / raw)
To: Wolfgang Bumiller; +Cc: pve-devel
On 7/19/23 14:11, Wolfgang Bumiller wrote:
> The String -> &[u8] modification in the parse method may be a follow-up
> or a v4 - depending on how we proceed with applying this the whole
> thing.
Discussed this with Thomas yesterday - I'll post a v4 once all reviews are done.
Though I'll leave some things for a followup, e.g. the new HttpError crate.
--
- Lukas
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-manager 54/66] ui: backup: allow to select notification target for jobs
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 54/66] ui: backup: allow to select notification target for jobs Lukas Wagner
@ 2023-07-19 12:20 ` Dominik Csapak
0 siblings, 0 replies; 114+ messages in thread
From: Dominik Csapak @ 2023-07-19 12:20 UTC (permalink / raw)
To: Proxmox VE development discussion, Lukas Wagner
looks fine, one comment inline
On 7/17/23 17:00, Lukas Wagner wrote:
> This commit adds a possibility to choose between different options
> for notifications for backup jobs:
> - Notify via email, in the same manner as before
> - Notify via an endpoint/group
>
> If 'notify via mail' is selected, a text field where an email address
> can be entered is displayed:
>
> Notify: | Always notify v |
> Notify via: | E-Mail v |
> Send Mail to: | foo@example.com |
> Compression: | ..... v |
>
> If the other option is selected selected, a combo picker for selecting
> a channel is displayed:
>
> Notify: | Always notify v |
> Notify via: | Endpoint/Group v |
> Target: | endpoint-foo v |
> Compression: | ..... v |
>
> The code has also been adapted to use the newly introduced
> 'notification-policy' parameter, which replaces the 'mailnotification'
> paramter for backup jobs. Some logic which automatically migrates from
> 'mailnotification' has been added.
>
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> www/manager6/Makefile | 4 +-
> www/manager6/dc/Backup.js | 84 +++++++++++++++++--
> www/manager6/form/NotificationModeSelector.js | 8 ++
> ...ector.js => NotificationPolicySelector.js} | 1 +
> .../form/NotificationTargetSelector.js | 54 ++++++++++++
> 5 files changed, 143 insertions(+), 8 deletions(-)
> create mode 100644 www/manager6/form/NotificationModeSelector.js
> rename www/manager6/form/{EmailNotificationSelector.js => NotificationPolicySelector.js} (87%)
> create mode 100644 www/manager6/form/NotificationTargetSelector.js
>
> diff --git a/www/manager6/Makefile b/www/manager6/Makefile
> index 5b455c80..140b20f0 100644
> --- a/www/manager6/Makefile
> +++ b/www/manager6/Makefile
> @@ -36,7 +36,6 @@ JSSRC= \
> form/DayOfWeekSelector.js \
> form/DiskFormatSelector.js \
> form/DiskStorageSelector.js \
> - form/EmailNotificationSelector.js \
> form/FileSelector.js \
> form/FirewallPolicySelector.js \
> form/GlobalSearchField.js \
> @@ -51,6 +50,9 @@ JSSRC= \
> form/MultiPCISelector.js \
> form/NetworkCardSelector.js \
> form/NodeSelector.js \
> + form/NotificationModeSelector.js \
> + form/NotificationTargetSelector.js \
> + form/NotificationPolicySelector.js \
> form/PCISelector.js \
> form/PCIMapSelector.js \
> form/PermPathSelector.js \
> diff --git a/www/manager6/dc/Backup.js b/www/manager6/dc/Backup.js
> index 03a02651..625b5430 100644
> --- a/www/manager6/dc/Backup.js
> +++ b/www/manager6/dc/Backup.js
> @@ -36,6 +36,30 @@ Ext.define('PVE.dc.BackupEdit', {
> delete values.node;
> }
>
> + if (!isCreate) {
> + // 'mailnotification' is deprecated in favor of 'notification-policy'
> + // -> Migration to the new parameter happens in init, so we are
> + // safe to remove the old parameter here.
> + Proxmox.Utils.assemble_field_data(values, { 'delete': 'mailnotification' });
> +
> + // If sending notifications via mail, remove the current value of
> + // 'notification-target'
> + if (values['notification-mode'] === "mailto") {
> + Proxmox.Utils.assemble_field_data(
> + values,
> + { 'delete': 'notification-target' },
> + );
> + } else {
> + // and vice versa...
> + Proxmox.Utils.assemble_field_data(
> + values,
> + { 'delete': 'mailto' },
> + );
> + }
> + }
> +
> + delete values['notification-mode'];
> +
> if (!values.id && isCreate) {
> values.id = 'backup-' + Ext.data.identifier.Uuid.Global.generate().slice(0, 13);
> }
> @@ -146,6 +170,22 @@ Ext.define('PVE.dc.BackupEdit', {
> success: function(response, _options) {
> let data = response.result.data;
>
> + // 'mailnotification' is deprecated. Let's automatically
> + // migrate to the compatible 'notification-policy' parameter
> + if (data.mailnotification) {
> + if (!data["notification-policy"]) {
> + data["notification-policy"] = data.mailnotification;
> + }
> +
> + delete data.mailnotification;
> + }
> +
> + if (data['notification-target']) {
> + data['notification-mode'] = 'notification-target';
> + } else if (data.mailto) {
> + data['notification-mode'] = 'mailto';
> + }
> +
> if (data.exclude) {
> data.vmid = data.exclude;
> data.selMode = 'exclude';
> @@ -188,11 +228,13 @@ Ext.define('PVE.dc.BackupEdit', {
> viewModel: {
> data: {
> selMode: 'include',
> + notificationMode: 'notification-target',
> },
>
> formulas: {
> poolMode: (get) => get('selMode') === 'pool',
> disableVMSelection: (get) => get('selMode') !== 'include' && get('selMode') !== 'exclude',
> + mailNotificationSelected: (get) => get('notificationMode') === 'mailto',
> },
> },
>
> @@ -282,20 +324,48 @@ Ext.define('PVE.dc.BackupEdit', {
> },
> ],
> column2: [
> - {
> - xtype: 'textfield',
> - fieldLabel: gettext('Send email to'),
> - name: 'mailto',
> - },
> {
> xtype: 'pveEmailNotificationSelector',
> - fieldLabel: gettext('Email'),
> - name: 'mailnotification',
> + fieldLabel: gettext('Notify'),
> + name: 'notification-policy',
> cbind: {
> value: (get) => get('isCreate') ? 'always' : '',
> deleteEmpty: '{!isCreate}',
> },
> },
> + {
> + xtype: 'pveNotificationModeSelector',
> + fieldLabel: gettext('Notify via'),
> + name: 'notification-mode',
> + bind: {
> + value: '{notificationMode}',
> + },
> + },
> + {
> + xtype: 'pveNotificationTargetSelector',
> + fieldLabel: gettext('Notification Target'),
> + name: 'notification-target',
> + allowBlank: true,
> + editable: true,
> + autoSelect: false,
> + bind: {
> + hidden: '{mailNotificationSelected}',
> + disabled: '{mailNotificationSelected}',
> + },
> + cbind: {
> + deleteEmpty: '{!isCreate}',
> + },
> + },
> + {
> + xtype: 'textfield',
> + fieldLabel: gettext('Send email to'),
> + name: 'mailto',
> + hidden: true,
> + bind: {
> + hidden: '{!mailNotificationSelected}',
> + disabled: '{!mailNotificationSelected}',
> + },
> + },
> {
> xtype: 'pveCompressionSelector',
> reference: 'compressionSelector',
> diff --git a/www/manager6/form/NotificationModeSelector.js b/www/manager6/form/NotificationModeSelector.js
> new file mode 100644
> index 00000000..58fddd56
> --- /dev/null
> +++ b/www/manager6/form/NotificationModeSelector.js
> @@ -0,0 +1,8 @@
> +Ext.define('PVE.form.NotificationModeSelector', {
> + extend: 'Proxmox.form.KVComboBox',
> + alias: ['widget.pveNotificationModeSelector'],
> + comboItems: [
> + ['notification-target', gettext('Target')],
> + ['mailto', gettext('E-Mail')],
> + ],
> +});
> diff --git a/www/manager6/form/EmailNotificationSelector.js b/www/manager6/form/NotificationPolicySelector.js
> similarity index 87%
> rename from www/manager6/form/EmailNotificationSelector.js
> rename to www/manager6/form/NotificationPolicySelector.js
> index f318ea18..68087275 100644
> --- a/www/manager6/form/EmailNotificationSelector.js
> +++ b/www/manager6/form/NotificationPolicySelector.js
> @@ -4,5 +4,6 @@ Ext.define('PVE.form.EmailNotificationSelector', {
> comboItems: [
> ['always', gettext('Notify always')],
> ['failure', gettext('On failure only')],
> + ['never', gettext('Notify never')],
> ],
> });
> diff --git a/www/manager6/form/NotificationTargetSelector.js b/www/manager6/form/NotificationTargetSelector.js
> new file mode 100644
> index 00000000..9ead28e7
> --- /dev/null
> +++ b/www/manager6/form/NotificationTargetSelector.js
> @@ -0,0 +1,54 @@
> +Ext.define('PVE.form.NotificationTargetSelector', {
> + extend: 'Proxmox.form.ComboGrid',
> + alias: ['widget.pveNotificationTargetSelector'],
> +
> + // set default value to empty array, else it inits it with
> + // null and after the store load it is an empty array,
> + // triggering dirtychange
> + value: [],
seeing this sent me down a small rabbit hole, since i was convinced this would
only be necessary for multiselect combogrids (like the nodeselector, which
i guess is the component you copied this from?)
anyway, i sent a series for wt/manager that should make that unnecessary,
so we might coordinate that (if my patches are not applied when you send the next
version, just leave it as is, we can remove it later)
> + valueField: 'name',
> + displayField: 'name',
> + deleteEmpty: true,
> + skipEmptyText: true,
> +
> + store: {
> + fields: ['name', 'type', 'comment'],
> + proxy: {
> + type: 'proxmox',
> + url: '/api2/json/cluster/notifications/targets',
> + },
> + sorters: [
> + {
> + property: 'name',
> + direction: 'ASC',
> + },
> + ],
> + autoLoad: true,
> + },
> +
> + listConfig: {
> + columns: [
> + {
> + header: gettext('Target'),
> + dataIndex: 'name',
> + sortable: true,
> + hideable: false,
> + flex: 1,
> + },
> + {
> + header: gettext('Type'),
> + dataIndex: 'type',
> + sortable: true,
> + hideable: false,
> + flex: 1,
> + },
> + {
> + header: gettext('Comment'),
> + dataIndex: 'comment',
> + sortable: true,
> + hideable: false,
> + flex: 2,
> + },
> + ],
> + },
> +});
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-cluster 36/66] add libpve-notify-perl package
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-cluster 36/66] add libpve-notify-perl package Lukas Wagner
@ 2023-07-19 12:27 ` Wolfgang Bumiller
0 siblings, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-19 12:27 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
On Mon, Jul 17, 2023 at 05:00:21PM +0200, Lukas Wagner wrote:
> The package contains the PVE::Notify. It is a very thin wrapper
> around the Proxmox::RS::Notify module, feeding the configuration
> from the new 'notifications.cfg' and 'priv/notifications.cfg' files
> into it.
>
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> debian/control | 9 ++
> debian/libpve-notify-perl.docs | 1 +
> debian/libpve-notify-perl.install | 1 +
> src/PVE/Makefile | 2 +-
> src/PVE/Notify.pm | 145 ++++++++++++++++++++++++++++++
> 5 files changed, 157 insertions(+), 1 deletion(-)
> create mode 100644 debian/libpve-notify-perl.docs
> create mode 100644 debian/libpve-notify-perl.install
> create mode 100644 src/PVE/Notify.pm
>
> diff --git a/debian/control b/debian/control
> index f1c13cb..77d7f8b 100644
> --- a/debian/control
> +++ b/debian/control
> @@ -85,3 +85,12 @@ Breaks: pve-cluster (<= 6.0-7),
> Replaces: pve-cluster (<= 6.0-7),
> Description: Proxmox Virtual Environment cluster Perl API modules.
> This package contains the API2 endpoints and CLI binary 'pvecm'.
> +
> +Package: libpve-notify-perl
> +Architecture: all
> +Pre-Depends: ${misc:Pre-Depends},
> +Depends: libpve-cluster-perl (= ${binary:Version}),
> + libpve-rs-perl (>= 0.7.1),
> + ${misc:Depends},
> + ${perl:Depends},
> +Description: Notify helper module
> diff --git a/debian/libpve-notify-perl.docs b/debian/libpve-notify-perl.docs
> new file mode 100644
> index 0000000..8696672
> --- /dev/null
> +++ b/debian/libpve-notify-perl.docs
> @@ -0,0 +1 @@
> +debian/SOURCE
> diff --git a/debian/libpve-notify-perl.install b/debian/libpve-notify-perl.install
> new file mode 100644
> index 0000000..b590d07
> --- /dev/null
> +++ b/debian/libpve-notify-perl.install
> @@ -0,0 +1 @@
> +usr/share/perl5/PVE/Notify.pm
> diff --git a/src/PVE/Makefile b/src/PVE/Makefile
> index 10291a6..ac4a9ce 100644
> --- a/src/PVE/Makefile
> +++ b/src/PVE/Makefile
> @@ -11,7 +11,7 @@ PVE_VENDORARCH=$(DESTDIR)/$(PERL_VENDORARCH)/auto/PVE/IPCC
> PERL_DOC_INC_DIRS:=..
>
> SUBDIRS=Cluster CLI API2
> -SOURCES=IPCC.pm Cluster.pm Corosync.pm RRD.pm DataCenterConfig.pm SSHInfo.pm
> +SOURCES=IPCC.pm Cluster.pm Corosync.pm RRD.pm DataCenterConfig.pm Notify.pm SSHInfo.pm
>
> all:
>
> diff --git a/src/PVE/Notify.pm b/src/PVE/Notify.pm
> new file mode 100644
> index 0000000..a3bcfbb
> --- /dev/null
> +++ b/src/PVE/Notify.pm
> @@ -0,0 +1,145 @@
> +package PVE::Notify;
> +
> +use strict;
> +use warnings;
> +
> +use PVE::Cluster qw(cfs_register_file cfs_read_file cfs_lock_file cfs_write_file);
> +use PVE::RS::Notify;
> +
> +cfs_register_file(
> + 'notifications.cfg',
> + \&parse_notification_config,
> + \&write_notification_config,
> +);
> +
> +cfs_register_file(
> + 'priv/notifications.cfg',
> + \&parse_notification_config,
> + \&write_notification_config,
> +);
> +
> +my $mail_to_root_target = 'mail-to-root';
> +
> +sub parse_notification_config {
> + my ($filename, $raw) = @_;
> +
> + $raw = '' if !defined($raw);
> + return $raw;
> +}
> +
> +sub write_notification_config {
> + my ($filename, $config) = @_;
> + return $config;
> +}
> +
> +sub lock_config {
> + my ($code, $timeout) = @_;
> +
> + cfs_lock_file('notifications.cfg', $timeout, sub {
> + cfs_lock_file('priv/notifications.cfg', $timeout, $code);
> + die $@ if $@;
> + });
> + die $@ if $@;
> +}
> +
> +sub read_config {
> + my $config = cfs_read_file('notifications.cfg');
> + my $priv_config = cfs_read_file('priv/notifications.cfg');
> +
> + my $notification_config = PVE::RS::Notify->parse_config($config, $priv_config);
> +
> + eval {
> + # This target should always be available...
> + $notification_config->add_sendmail_endpoint(
> + $mail_to_root_target,
> + undef,
> + ['root@pam'],
> + undef,
> + undef,
> + 'Send mail to root@pam\'s email address'
> + );
> + };
> +
> + return $notification_config;
> +}
> +
> +sub write_config {
> + my ($notification_config) = @_;
> +
> + eval {
> + # ... but don't persist it to the config.
> + # Rationale: If it is in the config, the user might think
> + # that it can be changed by editing the configuration there.
> + # However, since we always add it in `read_config`, any changes
> + # will be implicitly overridden by the default.
> +
> + # If users want's to change the configuration, they are supposed to
> + # create a new sendmail endpoint.
> + $notification_config->delete_sendmail_endpoint($mail_to_root_target);
> + };
> +
> + my ($config, $priv_config) = $notification_config->write_config();
> + cfs_write_file('notifications.cfg', $config);
> + cfs_write_file('priv/notifications.cfg', $priv_config);
> +}
> +
> +sub default_target {
> + return $mail_to_root_target;
> +}
> +
> +sub notify {
> + my ($target, $severity, $title, $message, $properties, $config) = @_;
> + _send_notification($target, $severity, $title, $message, $properties, $config);
> +}
> +
> +sub info {
> + my ($target, $title, $message, $properties, $config) = @_;
> + _send_notification($target, 'info', $title, $message, $properties, $config);
> +}
> +
> +sub notice {
> + my ($target, $title, $message, $properties, $config) = @_;
> + _send_notification($target, 'notice', $title, $message, $properties, $config);
> +}
> +
> +sub warning {
> + my ($target, $title, $message, $properties, $config) = @_;
> + _send_notification($target, 'warning', $title, $message, $properties, $config);
> +}
> +
> +sub error {
> + my ($target, $title, $message, $properties, $config) = @_;
> + _send_notification($target, 'error', $title, $message, $properties, $config);
> +}
> +
> +sub _send_notification {
^ do we need this accessible to the outside? If not, you could move it
above its users and turn it into a `my sub`.
> + my ($target, $severity, $title, $message, $properties, $config) = @_;
> + $config = read_config() if !defined($config);
> + my ($module, $file, $line) = caller(1);
> +
> + # Augment properties with the source code location of the notify call
> + my $props_with_source = {
> + %$properties,
> + source => {
> + module => $module,
> + file => $file,
> + line => $line,
> + }
> + };
> +
> + $config->send($target, $severity, $title, $message, $props_with_source);
> +}
> +
> +sub check_may_use_target {
> + my ($target, $rpcenv) = @_;
> + my $user = $rpcenv->get_user();
> +
> + my $config = read_config();
> + my $entities = $config->get_referenced_entities($target);
> +
> + for my $entity (@$entities) {
> + $rpcenv->check($user, "/mapping/notification/$entity", [ 'Mapping.Use' ]);
> + }
> +}
> +
> +1;
> \ No newline at end of file
^ your editor has some kind of disease ;-)
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-common 38/66] JSONSchema: increase maxLength of config-digest to 64
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-common 38/66] JSONSchema: increase maxLength of config-digest to 64 Lukas Wagner
@ 2023-07-19 12:31 ` Wolfgang Bumiller
2023-07-19 12:41 ` Fiona Ebner
1 sibling, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-19 12:31 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pve-devel
On Mon, Jul 17, 2023 at 05:00:23PM +0200, Lukas Wagner wrote:
> The new notification backend is implemented in Rust where we use SHA256
> for config digests.
>
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
> ---
> src/PVE/JSONSchema.pm | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/src/PVE/JSONSchema.pm b/src/PVE/JSONSchema.pm
> index 7589bba..49e0d7a 100644
> --- a/src/PVE/JSONSchema.pm
> +++ b/src/PVE/JSONSchema.pm
> @@ -93,10 +93,13 @@ register_standard_option('pve-bridge-id', {
> });
>
> register_standard_option('pve-config-digest', {
> - description => 'Prevent changes if current configuration file has different SHA1 digest. This can be used to prevent concurrent modifications.',
> + description => 'Prevent changes if current configuration file has a different digest. '
> + . 'This can be used to prevent concurrent modifications.',
> type => 'string',
> optional => 1,
> - maxLength => 40, # sha1 hex digest length is 40
> + # sha1 hex digests are 40 characters long
> + # sha256 hex digests are 64 characters long (sha256 is used in our Rust code)
> + maxLength => 64,
> });
>
> register_standard_option('skiplock', {
> --
> 2.39.2
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-common 38/66] JSONSchema: increase maxLength of config-digest to 64
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-common 38/66] JSONSchema: increase maxLength of config-digest to 64 Lukas Wagner
2023-07-19 12:31 ` Wolfgang Bumiller
@ 2023-07-19 12:41 ` Fiona Ebner
2023-07-19 12:49 ` Wolfgang Bumiller
1 sibling, 1 reply; 114+ messages in thread
From: Fiona Ebner @ 2023-07-19 12:41 UTC (permalink / raw)
To: Proxmox VE development discussion, Lukas Wagner
Am 17.07.23 um 17:00 schrieb Lukas Wagner:
> The new notification backend is implemented in Rust where we use SHA256
> for config digests.
>
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> src/PVE/JSONSchema.pm | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/src/PVE/JSONSchema.pm b/src/PVE/JSONSchema.pm
> index 7589bba..49e0d7a 100644
> --- a/src/PVE/JSONSchema.pm
> +++ b/src/PVE/JSONSchema.pm
> @@ -93,10 +93,13 @@ register_standard_option('pve-bridge-id', {
> });
>
> register_standard_option('pve-config-digest', {
> - description => 'Prevent changes if current configuration file has different SHA1 digest. This can be used to prevent concurrent modifications.',
> + description => 'Prevent changes if current configuration file has a different digest. '
> + . 'This can be used to prevent concurrent modifications.',
Should we instead create a separate standard option
"pve-config-digest-sha256"? Then we can still clearly communicate which
digest it is to users of the API (mostly ourselves for this one, but
still). Might prevent some mix-up at some point in the future.
> type => 'string',
> optional => 1,
> - maxLength => 40, # sha1 hex digest length is 40
> + # sha1 hex digests are 40 characters long
> + # sha256 hex digests are 64 characters long (sha256 is used in our Rust code)
> + maxLength => 64,
> });
>
> register_standard_option('skiplock', {
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-manager 57/66] ui: allow to configure notification event -> target mapping
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 57/66] ui: allow to configure notification event -> target mapping Lukas Wagner
@ 2023-07-19 12:45 ` Dominik Csapak
2023-07-19 15:25 ` Lukas Wagner
0 siblings, 1 reply; 114+ messages in thread
From: Dominik Csapak @ 2023-07-19 12:45 UTC (permalink / raw)
To: Proxmox VE development discussion, Lukas Wagner
some comments inline:
On 7/17/23 17:00, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> www/manager6/Makefile | 1 +
> www/manager6/dc/Config.js | 12 ++
> www/manager6/dc/NotificationEvents.js | 238 ++++++++++++++++++++++++++
> 3 files changed, 251 insertions(+)
> create mode 100644 www/manager6/dc/NotificationEvents.js
>
> diff --git a/www/manager6/Makefile b/www/manager6/Makefile
> index 140b20f0..452abbd4 100644
> --- a/www/manager6/Makefile
> +++ b/www/manager6/Makefile
> @@ -158,6 +158,7 @@ JSSRC= \
> dc/Health.js \
> dc/Log.js \
> dc/NodeView.js \
> + dc/NotificationEvents.js \
> dc/OptionView.js \
> dc/PermissionView.js \
> dc/PoolEdit.js \
> diff --git a/www/manager6/dc/Config.js b/www/manager6/dc/Config.js
> index 04ed04f0..aa025c8d 100644
> --- a/www/manager6/dc/Config.js
> +++ b/www/manager6/dc/Config.js
> @@ -317,6 +317,18 @@ Ext.define('PVE.dc.Config', {
> );
> }
>
> + if (caps.dc['Sys.Audit']) {
> + me.items.push(
> + {
> + xtype: 'pveNotificationEvents',
> + title: gettext('Notifications'),
> + onlineHelp: 'notification_events',
> + iconCls: 'fa fa-bell-o',
> + itemId: 'notifications',
> + },
> + );
> + }
> +
> if (caps.dc['Sys.Audit']) {
> me.items.push({
> xtype: 'pveDcSupport',
> diff --git a/www/manager6/dc/NotificationEvents.js b/www/manager6/dc/NotificationEvents.js
> new file mode 100644
> index 00000000..8ba0a844
> --- /dev/null
> +++ b/www/manager6/dc/NotificationEvents.js
> @@ -0,0 +1,238 @@
> +Ext.define('PVE.dc.NotificationEventsPolicySelector', {
> + alias: ['widget.pveNotificationEventsPolicySelector'],
> + extend: 'Proxmox.form.KVComboBox',
> + deleteEmpty: false,
> + value: '__default__',
> + comboItems: [
> + ['__default__', `${Proxmox.Utils.defaultText} (always)`],
> + ['always', gettext('Always')],
> + ['never', gettext('Never')],
> + ],
> + defaultValue: '__default__',
> +});
mhmm.. are we sure all future things that notify have 'always' as default?
maybe we should make that text configurable?
also the third time you use this, you basically change the essence of it,
namely the options to choose from (i'd argue that should be a plain
KVComboBox there, you only save 3 lines, but there is no confusion that
it's a seperate selector anymore)
> +
> +Ext.define('PVE.dc.NotificationEventsTargetSelector', {
> + alias: ['widget.pveNotificationEventsTargetSelector'],
> + extend: 'PVE.form.NotificationTargetSelector',
> + fieldLabel: gettext('Notification Target'),
> + allowBlank: true,
> + editable: true,
> + autoSelect: false,
> + deleteEmpty: false,
> + emptyText: `${Proxmox.Utils.defaultText} (${gettext("mail-to-root")})`,
> +});
> +
> +Ext.define('PVE.dc.NotificationEvents', {
> + extend: 'Proxmox.grid.ObjectGrid',
> + alias: ['widget.pveNotificationEvents'],
> +
> + // Taken from OptionView.js, but adapted slightly.
what exactly was adapted? i can ofc diff it myself, but it would
be nicer to have that info either in a comment or the commit message.
also we should factor this out and reuse it in OptionView and here?
maybe just adding it to the ObjectGrid itself?
(if possible)
the code looks generic enough to be useful there
> + addInputPanelRow: function(name, propertyName, text, opts) {
> + let me = this;
> +
> + opts = opts || {};
> + me.rows = me.rows || {};
> +
> + me.rows[name] = {
> + required: true,
> + defaultValue: opts.defaultValue,
> + header: text,
> + renderer: opts.renderer,
> + name: propertyName,
> + editor: {
> + xtype: 'proxmoxWindowEdit',
> + width: opts.width || 400,
> + subject: text,
> + onlineHelp: opts.onlineHelp,
> + fieldDefaults: {
> + labelWidth: opts.labelWidth || 150,
> + },
> + setValues: function(values) {
> + let value = values[propertyName];
> +
> + if (opts.parseBeforeSet) {
> + value = PVE.Parser.parsePropertyString(value);
> + }
> +
> + Ext.Array.each(this.query('inputpanel'), function(panel) {
> + panel.setValues(value);
> + panel.originalValue = {
> + ...value,
> + };
> + });
> + },
> + url: opts.url,
> + items: [{
> + xtype: 'inputpanel',
> + onGetValues: function(values) {
> + let fields = this.config.items.map(field => field.name).filter(n => n);
> +
> + for (const [key, value] of Object.entries(this.originalValue)) {
> + if (!fields.includes(key)) {
> + values[key] = value;
> + }
> + }
> +
> + let value = {};
> + if (Object.keys(values).length > 0) {
> + value[propertyName] = PVE.Parser.printPropertyString(values);
> + } else {
> + Proxmox.Utils.assemble_field_data(value, { 'delete': propertyName });
> + }
> +
> + return value;
> + },
> + items: opts.items,
> + }],
> + },
> + };
> + },
> +
> + initComponent: function() {
> + let me = this;
> +
> + // Helper function for rendering the property
> + // Needed since the actual value is always stored in the 'notify' property
> + let render_value = (store, target_key, mode_key) => {
> + let value = store.getById('notify')?.get('value') ?? {};
> + let target = value[target_key] ?? gettext('mail-to-root');
> + let template;
> +
> + switch (value[mode_key]) {
> + case 'always':
> + template = gettext('Always, notify via target \'{0}\'');
> + break;
> + case 'never':
> + template = gettext('Never');
> + break;
> + default:
> + template = gettext('{1} (Always), notify via target \'{0}\'');
> + break;
> + }
> +
> + return Ext.String.format(template, target, Proxmox.Utils.defaultText);
> + };
> +
> + me.addInputPanelRow('fencing', 'notify', gettext('Node Fencing'), {
> + renderer: (value, metaData, record, rowIndex, colIndex, store) =>
> + render_value(store, 'target-fencing', 'fencing'),
> + url: "/api2/extjs/cluster/options",
> + items: [
> + {
> + xtype: 'pveNotificationEventsPolicySelector',
> + name: 'fencing',
> + fieldLabel: gettext('Notify'),
> + },
> + {
> + xtype: 'pveNotificationEventsTargetSelector',
> + name: 'target-fencing',
> + },
> + {
> + xtype: 'displayfield',
> + userCls: 'pmx-hint',
> + value: gettext('Disabling notifications is not ' +
> + 'recommended for production systems!'),
> + },
> + ],
> + });
> +
> + me.addInputPanelRow('replication', 'notify', gettext('Replication'), {
> + renderer: (value, metaData, record, rowIndex, colIndex, store) =>
> + render_value(store, 'target-replication', 'replication'),
> + url: "/api2/extjs/cluster/options",
> + items: [
> + {
> + xtype: 'pveNotificationEventsPolicySelector',
> + name: 'replication',
> + fieldLabel: gettext('Notify'),
> + },
> + {
> + xtype: 'pveNotificationEventsTargetSelector',
> + name: 'target-replication',
> + },
> + {
> + xtype: 'displayfield',
> + userCls: 'pmx-hint',
> + value: gettext('Disabling notifications is not ' +
> + 'recommended for production systems!'),
> + },
> + ],
> + });
> +
> + me.addInputPanelRow('updates', 'notify', gettext('Package Updates'), {
> + renderer: (value, metaData, record, rowIndex, colIndex, store) => {
> + value = store.getById('notify')?.get('value') ?? {};
> + let target = value['target-package-updates'] ?? gettext('mail-to-root');
> + let template;
> +
> + switch (value['package-updates']) {
> + case 'always':
> + template = gettext('Always, notify via \'{0}\'');
> + break;
> + case 'auto':
> + template = gettext('Automatically, notify via target \'{0}\'');
you should be able to reuse the render_value if you add this there also
it can't trigger for the others anyway?
> + break;
> + case 'never':
> + template = gettext('Never');
> + break;
> + default:
> + template = gettext('{1} (Automatically), notify via target \'{0}\'');
> + break;
> + }
> +
> + return Ext.String.format(template, target, Proxmox.Utils.defaultText);
> + },
> + url: "/api2/extjs/cluster/options",
> + items: [
> + {
> + xtype: 'pveNotificationEventsPolicySelector',
as said above i'd simply make this a KVComboBox to indicate it's
basically a seperate component
> + name: 'package-updates',
> + fieldLabel: gettext('Notify'),
> + comboItems: [
> + ['__default__', Proxmox.Utils.defaultText + ' (auto)'],
> + ['auto', gettext('Automatically')],
> + ['always', gettext('Always')],
> + ['never', gettext('Never')],
> + ],
> + },
> + {
> + xtype: 'pveNotificationEventsTargetSelector',
> + name: 'target-package-updates',
> + },
> + ],
> + });
> +
> + // Hack: Also load the notify property to make it accessible
> + // for our render functions. initComponents later hides it.
> + me.add_text_row('notify', gettext('Notify'), {});
it should be possible to simply add it directly here with something like:
me.rows.notify = {
visible: false,
};
?
we e.g. do something like this in dc/Optionsview and qemu/HardwareView
(the latter is a PendingObjectGrid, but still inherits from ObjectGrid)
> +
> + me.selModel = Ext.create('Ext.selection.RowModel', {});
> +
> + Ext.apply(me, {
> + tbar: [{
> + text: gettext('Edit'),
> + xtype: 'proxmoxButton',
> + disabled: true,
> + handler: () => me.run_editor(),
> + selModel: me.selModel,
> + }],
> + url: "/api2/json/cluster/options",
> + editorConfig: {
> + url: "/api2/extjs/cluster/options",
> + },
> + interval: 5000,
> + cwidth1: 200,
> + listeners: {
> + itemdblclick: me.run_editor,
> + },
> + });
> +
> + me.callParent();
> +
> + me.rows.notify.visible = false;
> +
> + me.on('activate', me.rstore.startUpdate);
> + me.on('destroy', me.rstore.stopUpdate);
> + me.on('deactivate', me.rstore.stopUpdate);
> + },
> +});
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-common 38/66] JSONSchema: increase maxLength of config-digest to 64
2023-07-19 12:41 ` Fiona Ebner
@ 2023-07-19 12:49 ` Wolfgang Bumiller
0 siblings, 0 replies; 114+ messages in thread
From: Wolfgang Bumiller @ 2023-07-19 12:49 UTC (permalink / raw)
To: Fiona Ebner; +Cc: Proxmox VE development discussion, Lukas Wagner
On Wed, Jul 19, 2023 at 02:41:17PM +0200, Fiona Ebner wrote:
> Am 17.07.23 um 17:00 schrieb Lukas Wagner:
> > The new notification backend is implemented in Rust where we use SHA256
> > for config digests.
> >
> > Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> > ---
> > src/PVE/JSONSchema.pm | 7 +++++--
> > 1 file changed, 5 insertions(+), 2 deletions(-)
> >
> > diff --git a/src/PVE/JSONSchema.pm b/src/PVE/JSONSchema.pm
> > index 7589bba..49e0d7a 100644
> > --- a/src/PVE/JSONSchema.pm
> > +++ b/src/PVE/JSONSchema.pm
> > @@ -93,10 +93,13 @@ register_standard_option('pve-bridge-id', {
> > });
> >
> > register_standard_option('pve-config-digest', {
> > - description => 'Prevent changes if current configuration file has different SHA1 digest. This can be used to prevent concurrent modifications.',
> > + description => 'Prevent changes if current configuration file has a different digest. '
> > + . 'This can be used to prevent concurrent modifications.',
>
> Should we instead create a separate standard option
> "pve-config-digest-sha256"? Then we can still clearly communicate which
> digest it is to users of the API (mostly ourselves for this one, but
> still). Might prevent some mix-up at some point in the future.
We could, but the current one is already only limited to a `maxLength`,
not an exact one, so we don't get proper verification errors here for
shorter strings either.
Also, theoretically we could bring all the digests up to sha256 over
time while supporting both as input in the API without having to update
API the schema of each call that starts supporting the newer one.
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-manager 59/66] ui: perm path: load notification target/filter acl entries
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 59/66] ui: perm path: load notification target/filter acl entries Lukas Wagner
@ 2023-07-19 12:53 ` Dominik Csapak
2023-07-20 7:46 ` Lukas Wagner
0 siblings, 1 reply; 114+ messages in thread
From: Dominik Csapak @ 2023-07-19 12:53 UTC (permalink / raw)
To: Proxmox VE development discussion, Lukas Wagner
On 7/17/23 17:00, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
>
> Notes:
> I'm not sure if I like this solution, but adding notification targets to
> the resources API endpoint would not have make sense.
> Maybe we could create a new API endpoint that returns all possible ACL
> paths and then use a normal store for the perm path combobox?
i'd also prefer that, it would make this much simpler, and more manageable
for the pci/usb mappings i simply omitted them here, so for now
we could simply hardcode the different types here
/mapping/usb
/mapping/pci
/mapping/notification
and be done with it until we have said api call ;)
AFAIR thomas opposed it the last time i wanted to add an api call here
(i can't remember what for though)
>
> www/manager6/data/PermPathStore.js | 26 +++++++++++++++++++++++++-
> 1 file changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/www/manager6/data/PermPathStore.js b/www/manager6/data/PermPathStore.js
> index c3ac7f0e..b7e4fa33 100644
> --- a/www/manager6/data/PermPathStore.js
> +++ b/www/manager6/data/PermPathStore.js
> @@ -9,6 +9,7 @@ Ext.define('PVE.data.PermPathStore', {
> { 'value': '/access/groups' },
> { 'value': '/access/realm' },
> { 'value': '/mapping' },
> + { 'value': '/mapping/notification' },
> { 'value': '/nodes' },
> { 'value': '/pool' },
> { 'value': '/sdn/zones' },
> @@ -46,8 +47,31 @@ Ext.define('PVE.data.PermPathStore', {
> donePaths[path] = 1;
> }
> });
> - me.resumeEvents();
>
> + Ext.Ajax.request({
also why ext.ajax.request and not API2Request ? ;)
> + url: '/api2/json/cluster/notifications/targets',
> + method: 'GET',
> + success: function(response, opts) {
> + let obj = Ext.decode(response.responseText);
> +
> + for (let target of obj.data) {
> + me.add({ value: `/mapping/notification/${target.name}` });
> + }
> + },
> + });
> + Ext.Ajax.request({
> + url: '/api2/json/cluster/notifications/filters',
> + method: 'GET',
> + success: function(response, opts) {
> + let obj = Ext.decode(response.responseText);
> +
> + for (let filter of obj.data) {
> + me.add({ value: `/mapping/notification/${filter.name}` });
> + }
> + },
> + });
> +
> + me.resumeEvents();
> me.fireEvent('refresh', me);
> me.fireEvent('datachanged', me);
>
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system
2023-07-18 14:37 ` Thomas Lamprecht
@ 2023-07-19 13:13 ` Lukas Wagner
0 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-19 13:13 UTC (permalink / raw)
To: Thomas Lamprecht, Proxmox VE development discussion, Dominik Csapak
On 7/18/23 16:37, Thomas Lamprecht wrote:
> Am 18/07/2023 um 14:34 schrieb Dominik Csapak:
>> * the backup edit window is rather tall at this point, and if we assume a minimum
>> screen size of 1280x720 there is not quite enough space when you subtract
>> the (typical) os bar and browser window decoration, maybe a 'notification'
>> tab would be better (we already have some tabs there anyway)
>
> FWIW and not that I'm against tabs here (I didn't even checked the GUI at all), I
> came to the conclusion that 720p *and* OS/browser bar deduction is just to small
> to be practical.. 768p (1366 x 768) and ~90px deduction for height and 66px for
> those desktops that have the bar configured on the side would be more practicable,
> leaving 1300 x 680 as minimum.
>
Right now, the backup windows is a bit taller than 680px, however it still is usable without any problems.
I'll experiment with a seperate tab in a follow-up to this patch series.
--
- Lukas
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox-widget-toolkit 61/66] notification: add gui for sendmail notification endpoints
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 61/66] notification: add gui for sendmail notification endpoints Lukas Wagner
@ 2023-07-19 13:25 ` Dominik Csapak
0 siblings, 0 replies; 114+ messages in thread
From: Dominik Csapak @ 2023-07-19 13:25 UTC (permalink / raw)
To: Proxmox VE development discussion, Lukas Wagner
some comments/nits inline:
On 7/17/23 17:00, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> src/Makefile | 4 +
> src/Schema.js | 8 ++
> src/data/model/NotificationConfig.js | 8 ++
> src/panel/NotificationConfigView.js | 192 +++++++++++++++++++++++++++
> src/panel/SendmailEditPanel.js | 140 +++++++++++++++++++
> src/window/EndpointEditBase.js | 50 +++++++
> 6 files changed, 402 insertions(+)
> create mode 100644 src/data/model/NotificationConfig.js
> create mode 100644 src/panel/NotificationConfigView.js
> create mode 100644 src/panel/SendmailEditPanel.js
> create mode 100644 src/window/EndpointEditBase.js
>
> diff --git a/src/Makefile b/src/Makefile
> index baa90ec..e83038f 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -22,6 +22,7 @@ JSSRC= \
> data/ObjectStore.js \
> data/RRDStore.js \
> data/TimezoneStore.js \
> + data/model/NotificationConfig.js \
> data/model/Realm.js \
> data/model/Certificates.js \
> data/model/ACME.js \
> @@ -59,6 +60,7 @@ JSSRC= \
> panel/InfoWidget.js \
> panel/LogView.js \
> panel/NodeInfoRepoStatus.js \
> + panel/NotificationConfigView.js \
> panel/JournalView.js \
> panel/PermissionView.js \
> panel/PruneKeepPanel.js \
> @@ -68,6 +70,7 @@ JSSRC= \
> panel/ACMEAccount.js \
> panel/ACMEPlugin.js \
> panel/ACMEDomains.js \
> + panel/SendmailEditPanel.js \
> panel/StatusView.js \
> panel/TfaView.js \
> panel/NotesView.js \
> @@ -83,6 +86,7 @@ JSSRC= \
> window/ACMEAccount.js \
> window/ACMEPluginEdit.js \
> window/ACMEDomains.js \
> + window/EndpointEditBase.js \
> window/FileBrowser.js \
> window/AuthEditBase.js \
> window/AuthEditOpenId.js \
> diff --git a/src/Schema.js b/src/Schema.js
> index b247b1e..99bb3fa 100644
> --- a/src/Schema.js
> +++ b/src/Schema.js
> @@ -37,6 +37,14 @@ Ext.define('Proxmox.Schema', { // a singleton
> }
> },
>
> + notificationEndpointTypes: {
> + sendmail: {
> + name: gettext('Sendmail'),
> + ipanel: 'pmxSendmailEditPanel',
> + iconCls: 'fa-envelope-o',
> + },
> + },
> +
> pxarFileTypes: {
> b: { icon: 'cube', label: gettext('Block Device') },
> c: { icon: 'tty', label: gettext('Character Device') },
> diff --git a/src/data/model/NotificationConfig.js b/src/data/model/NotificationConfig.js
> new file mode 100644
> index 0000000..c2ce843
> --- /dev/null
> +++ b/src/data/model/NotificationConfig.js
> @@ -0,0 +1,8 @@
> +Ext.define('proxmox-notification-endpoints', {
> + extend: 'Ext.data.Model',
> + fields: ['name', 'type', 'comment'],
> + proxy: {
> + type: 'proxmox',
> + },
> + idProperty: 'name',
> +});
> diff --git a/src/panel/NotificationConfigView.js b/src/panel/NotificationConfigView.js
> new file mode 100644
> index 0000000..f6e6a8b
> --- /dev/null
> +++ b/src/panel/NotificationConfigView.js
> @@ -0,0 +1,192 @@
> +Ext.define('Proxmox.panel.NotificationConfigView', {
> + extend: 'Ext.panel.Panel',
> + alias: 'widget.pmxNotificationConfigView',
> + mixins: ['Proxmox.Mixin.CBind'],
> + layout: {
> + type: 'border',
> + },
> +
> + items: [
> + {
> + region: 'center',
> + border: false,
> + xtype: 'pmxNotificationEndpointView',
> + cbind: {
> + baseUrl: '{baseUrl}',
> + },
> + },
> + ],
> +});
> +
> +Ext.define('Proxmox.panel.NotificationEndpointView', {
> + extend: 'Ext.grid.Panel',
> + alias: 'widget.pmxNotificationEndpointView',
> +
> + title: gettext('Notification Targets'),
> +
> + controller: {
> + xclass: 'Ext.app.ViewController',
> +
> + openEditWindow: function(endpointType, endpoint) {
> + let me = this;
> +
> + if (endpoint === 'mail-to-root') {
> + return;
> + }
> +
> + Ext.create('Proxmox.window.EndpointEditBase', {
> + baseUrl: me.getView().baseUrl,
> + type: endpointType,
> +
> + name: endpoint,
> + listeners: {
> + destroy: () => me.reload(),
> + },
> + }).show();
you could use 'autoShow: true' instead
> + },
> +
> + openEditForSelectedItem: function() {
> + let me = this;
> + let view = me.getView();
> +
> + let selection = view.getSelection();
> + if (selection.length < 1) return;
this is against your style guide, please don't use single line if statements.
> + let endpointName = selection[0].data.name;
> + let type = selection[0].data.type;
> +
> + me.openEditWindow(type, endpointName);
for this, you don't really need to extract it into seperate variables, simply call it
with
me.openEditWindow(selection[0].data.type, selection[0].data.name);
should still fit in one line ;)
> + },
> +
> + reload: function() {
> + let me = this;
> + let view = me.getView();
> + view.getStore().rstore.load();
> + },
> +
> + testEndpoint: function() {
> + let me = this;
> + let view = me.getView();
> +
> + let selection = view.getSelection();
> + if (selection.length < 1) return;
again style issue
> + let target = selection[0].data.name;
> +
> + Ext.Msg.confirm(
> + gettext("Notification Target Test"),
> + gettext(`Do you want to send a test notification to '${target}'?`),
> + function(decision) {
> + if (decision !== "yes") {
> + return;
> + }
> +
> + Proxmox.Utils.API2Request({
> + method: 'POST',
> + url: `${view.baseUrl}/targets/${target}/test`,
> +
> + success: function(response, opt) {
> + Ext.Msg.show({
> + title: gettext('Notification Target Test'),
> + message: gettext(`Sent test notification to '${target}'.`),
> + buttons: Ext.Msg.OK,
> + icon: Ext.Msg.INFO,
> + });
> + },
> + autoErrorAlert: true,
> + });
> + });
> + },
> + },
> +
> + listeners: {
> + itemdblclick: 'openEditForSelectedItem',
> + activate: 'reload',
> + },
> +
> + emptyText: gettext('No notification targets configured'),
> +
> + columns: [
> + {
> + dataIndex: 'name',
> + text: gettext('Target Name'),
> + renderer: Ext.String.htmlEncode,
> + flex: 1,
> + },
> + {
> + dataIndex: 'type',
> + text: gettext('Type'),
> + renderer: Ext.String.htmlEncode,
> + flex: 1,
> + },
> + {
> + dataIndex: 'comment',
> + text: gettext('Comment'),
> + renderer: Ext.String.htmlEncode,
> + flex: 1,
> + },
> + ],
> +
> + store: {
> + type: 'diff',
> + autoDestroy: true,
> + autoDestroyRstore: true,
> + rstore: {
> + type: 'update',
> + storeid: 'proxmox-notification-endpoints',
> + model: 'proxmox-notification-endpoints',
> + autoStart: true,
> + },
> + sorters: 'name',
> + },
> +
> + initComponent: function() {
> + let me = this;
> +
> + let menuItems = [];
> + for (const [endpointType, config] of Object.entries(
> + Proxmox.Schema.notificationEndpointTypes).sort()) {
> + menuItems.push({
> + text: config.name,
> + iconCls: 'fa fa-fw ' + (config.iconCls || 'fa-bell-o'),
> + handler: () => me.controller.openEditWindow(endpointType),
> + });
> + }
> +
> + Ext.apply(me, {
> + tbar: [
> + {
> + text: gettext('Add'),
> + menu: menuItems,
> + },
> + {
> + xtype: 'proxmoxButton',
> + text: gettext('Modify'),
> + handler: 'openEditForSelectedItem',
> + enableFn: rec => rec.data.name !== 'mail-to-root',
> + disabled: true,
> + },
> + {
> + xtype: 'proxmoxStdRemoveButton',
> + callback: 'reload',
> + enableFn: rec => rec.data.name !== 'mail-to-root',
> + getUrl: function(rec) {
> + if (rec.data.type === 'group') {
> + return `${me.baseUrl}/groups/${rec.getId()}`;
> + }
> +
> + return `${me.baseUrl}/endpoints/${rec.data.type}/${rec.getId()}`;
> + },
> + },
> + '-',
> + {
> + xtype: 'proxmoxButton',
> + text: gettext('Test'),
> + handler: 'testEndpoint',
> + disabled: true,
> + },
> + ],
> + });
> +
> + me.callParent();
> + me.store.rstore.proxy.setUrl(`/api2/json/${me.baseUrl}/targets`);
would it maybe nicer in this case to have the store defined in the initcomponent
before the callParent to be able to inject it there?
no hard feelings though
> + },
> +});
> diff --git a/src/panel/SendmailEditPanel.js b/src/panel/SendmailEditPanel.js
> new file mode 100644
> index 0000000..9444a8c
> --- /dev/null
> +++ b/src/panel/SendmailEditPanel.js
> @@ -0,0 +1,140 @@
> +Ext.define('Proxmox.panel.SendmailEditPanel', {
> + extend: 'Proxmox.panel.InputPanel',
> + xtype: 'pmxSendmailEditPanel',
> + mixins: ['Proxmox.Mixin.CBind'],
> +
> + type: 'sendmail',
> +
> + columnT: [
if you only need need one column, why not simply use items/advancedItems and increase
the width of the window/inputpanel?
> + {
> + xtype: 'pmxDisplayEditField',
> + name: 'name',
> + cbind: {
> + value: '{name}',
> + editable: '{isCreate}',
> + },
> + fieldLabel: gettext('Endpoint Name'),
> + allowBlank: false,
> + },
> + {
> + xtype: 'pmxUserSelector',
> + name: 'mailto-user',
> + reference: 'mailto-user',
> + multiSelect: true,
> + allowBlank: true,
> + editable: false,
> + skipEmptyText: true,
> + fieldLabel: gettext('User(s)'),
> + cbind: {
> + deleteEmpty: '{!isCreate}',
> + },
> + validator: function(value) {
> + let up = this.up('pmxSendmailEditPanel');
> + let other = up.down('[name=mailto]');
> +
> + if (!value && !other.getValue()) {
> + return gettext('Either mailto or mailto-user must be set');
> + }
> +
> + return true;
> + },
the validator/gettext could be factored out since it's basically the same as below,
but it's ok this way if thats too much work
> + listConfig: {
> + width: 600,
> + columns: [
> + {
> + header: gettext('User'),
> + sortable: true,
> + dataIndex: 'userid',
> + renderer: Ext.String.htmlEncode,
> + flex: 1,
> + },
> + {
> + header: gettext('E-Mail'),
> + sortable: true,
> + dataIndex: 'email',
> + renderer: Ext.String.htmlEncode,
> + flex: 1,
> + },
> + {
> + header: gettext('Comment'),
> + sortable: false,
> + dataIndex: 'comment',
> + renderer: Ext.String.htmlEncode,
> + flex: 1,
> + },
> + ],
> + },
> + },
> + {
> + xtype: 'proxmoxtextfield',
> + fieldLabel: gettext('Additional Recipient(s)'),
> + name: 'mailto',
> + reference: 'mailto',
> + allowBlank: true,
> + cbind: {
> + deleteEmpty: '{!isCreate}',
> + },
> + autoEl: {
> + tag: 'div',
> + 'data-qtip': gettext(
> + 'Multiple recipients must be separated by spaces, commas or semicolons',
> + ),
> + },
> + validator: function(value) {
> + let up = this.up('pmxSendmailEditPanel');
> + let other = up.down('[name=mailto-user]');
> +
> + if (!value && !other.getValue().length) {
> + return gettext('Either mailto or mailto-user must be set');
> + }
> +
> + return true;
> + },
> + },
> + ],
> +
> + column1: [],
> +
> + column2: [],
> +
> + columnB: [
why in columnB instead of columnT if it's the only one in use?
> + {
> + xtype: 'proxmoxtextfield',
> + name: 'comment',
> + fieldLabel: gettext('Comment'),
> + cbind: {
> + deleteEmpty: '{!isCreate}',
> + },
> + },
> + ],
> + advancedColumnB: [
> + {
> + xtype: 'proxmoxtextfield',
> + fieldLabel: gettext('Author'),
> + name: 'author',
> + allowBlank: true,
> + emptyText: gettext('Proxmox VE'),
> + cbind: {
> + deleteEmpty: '{!isCreate}',
> + },
> + },
> + {
> + xtype: 'proxmoxtextfield',
> + fieldLabel: gettext('From Address'),
> + name: 'from-address',
> + allowBlank: true,
> + emptyText: gettext('Defaults to datacenter configuration, or root@$hostname'),
> + cbind: {
> + deleteEmpty: '{!isCreate}',
> + },
> + },
> + ],
> +
> + onGetValues: (values) => {
> + if (values.mailto) {
> + values.mailto = values.mailto.split(/[\s,;]+/);
> + }
> + return values;
> + },
> +});
> +
> diff --git a/src/window/EndpointEditBase.js b/src/window/EndpointEditBase.js
> new file mode 100644
> index 0000000..81e5951
> --- /dev/null
> +++ b/src/window/EndpointEditBase.js
> @@ -0,0 +1,50 @@
> +Ext.define('Proxmox.window.EndpointEditBase', {
> + extend: 'Proxmox.window.Edit',
> +
> + isAdd: true,
> +
> + fieldDefaults: {
> + labelWidth: 120,
> + },
> +
> + initComponent: function() {
> + let me = this;
> +
> + me.isCreate = !me.name;
> +
> + if (!me.baseUrl) {
> + throw "baseUrl not set";
> + }
> +
> + me.url = `/api2/extjs${me.baseUrl}/endpoints/${me.type}`;
> +
> + if (me.isCreate) {
> + me.method = 'POST';
> + } else {
> + me.url += `/${me.name}`;
> + me.method = 'PUT';
> + }
> +
> + let endpointConfig = Proxmox.Schema.notificationEndpointTypes[me.type];
> + if (!endpointConfig) {
> + throw 'unknown endpoint type';
> + }
> +
> + me.subject = endpointConfig.name;
> +
> + Ext.apply(me, {
> + items: [{
> + name: me.name,
> + xtype: endpointConfig.ipanel,
> + isCreate: me.isCreate,
> + type: me.type,
> + }],
> + });
> +
> + me.callParent();
> +
> + if (!me.isCreate) {
> + me.load();
> + }
> + },
> +});
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox-widget-toolkit 63/66] notification: add gui for notification groups
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 63/66] notification: add gui for notification groups Lukas Wagner
@ 2023-07-19 13:32 ` Dominik Csapak
2023-07-20 12:31 ` Lukas Wagner
0 siblings, 1 reply; 114+ messages in thread
From: Dominik Csapak @ 2023-07-19 13:32 UTC (permalink / raw)
To: Proxmox VE development discussion, Lukas Wagner
comments inline
On 7/17/23 17:00, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> src/Makefile | 1 +
> src/Schema.js | 5 +
> src/panel/NotificationGroupEditPanel.js | 177 ++++++++++++++++++++++++
> src/window/EndpointEditBase.js | 6 +-
> 4 files changed, 188 insertions(+), 1 deletion(-)
> create mode 100644 src/panel/NotificationGroupEditPanel.js
>
> diff --git a/src/Makefile b/src/Makefile
> index 2e620e3..829081d 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -61,6 +61,7 @@ JSSRC= \
> panel/LogView.js \
> panel/NodeInfoRepoStatus.js \
> panel/NotificationConfigView.js \
> + panel/NotificationGroupEditPanel.js \
> panel/JournalView.js \
> panel/PermissionView.js \
> panel/PruneKeepPanel.js \
> diff --git a/src/Schema.js b/src/Schema.js
> index 37ecd88..a7ffdf8 100644
> --- a/src/Schema.js
> +++ b/src/Schema.js
> @@ -48,6 +48,11 @@ Ext.define('Proxmox.Schema', { // a singleton
> ipanel: 'pmxGotifyEditPanel',
> iconCls: 'fa-bell-o',
> },
> + group: {
> + name: gettext('Notification Group'),
> + ipanel: 'pmxNotificationGroupEditPanel',
> + iconCls: 'fa-bell-o',
> + },
> },
>
> pxarFileTypes: {
> diff --git a/src/panel/NotificationGroupEditPanel.js b/src/panel/NotificationGroupEditPanel.js
> new file mode 100644
> index 0000000..0a7a469
> --- /dev/null
> +++ b/src/panel/NotificationGroupEditPanel.js
> @@ -0,0 +1,177 @@
> +Ext.define('Proxmox.panel.NotificationGroupEditPanel', {
> + extend: 'Proxmox.panel.InputPanel',
> + xtype: 'pmxNotificationGroupEditPanel',
> + mixins: ['Proxmox.Mixin.CBind'],
> +
> + type: 'group',
> +
> + columnT: [
> + {
> + xtype: 'pmxDisplayEditField',
> + name: 'name',
> + cbind: {
> + value: '{name}',
> + editable: '{isCreate}',
> + },
> + fieldLabel: gettext('Group Name'),
> + allowBlank: false,
> + },
> + {
> + xtype: 'pmxNotificationEndpointSelector',
> + name: 'endpoint',
> + allowBlank: false,
> + },
> + ],
> +
> + column1: [],
> +
> + column2: [],
> +
> + columnB: [
> + {
> + xtype: 'proxmoxtextfield',
> + name: 'comment',
> + fieldLabel: gettext('Comment'),
> + cbind: {
> + deleteEmpty: '{!isCreate}',
> + },
> + },
> + ],
> +});
> +
> +Ext.define('Proxmox.form.NotificationEndpointSelector', {
> + extend: 'Ext.grid.Panel',
> + alias: 'widget.pmxNotificationEndpointSelector',
> +
> + mixins: {
> + field: 'Ext.form.field.Field',
> + },
when implementing the field mixin you have to (quote from the extjs docs)
---8<---
You will also need to make sure that initField is called during the component's initialization.
--->8---
so you normally need a (small) initComponent that calls that
(otherwise you can have some strange effects when using the field)
> +
> + allowBlank: true,
> + selectAll: false,
> + isFormField: true,
> +
> + store: {
> + autoLoad: true,
> + model: 'proxmox-notification-endpoints',
> + sorters: 'name',
> + filters: item => item.data.type !== 'group',
> + },
> +
> + columns: [
> + {
> + header: gettext('Endpoint Name'),
> + dataIndex: 'name',
> + flex: 1,
> + },
> + {
> + header: gettext('Type'),
> + dataIndex: 'type',
> + flex: 1,
> + },
> + {
> + header: gettext('Comment'),
> + dataIndex: 'comment',
> + flex: 3,
> + },
> + ],
> +
> + selModel: {
> + selType: 'checkboxmodel',
> + mode: 'SIMPLE',
> + },
> +
> + checkChangeEvents: [
> + 'selectionchange',
> + 'change',
> + ],
> +
> + listeners: {
> + selectionchange: function() {
> + // to trigger validity and error checks
> + this.checkChange();
> + },
> + },
> +
> + getSubmitData: function() {
> + let me = this;
> + let res = {};
> + res[me.name] = me.getValue();
> + return res;
> + },
> +
> + getValue: function() {
> + let me = this;
> + if (me.savedValue !== undefined) {
> + return me.savedValue;
> + }
> + let sm = me.getSelectionModel();
> + let selection = sm.getSelection();
> + let values = [];
> + selection.forEach(function(item) {
> + values.push(item.data.name);
> + });
> + return values;
could also be:
return (sm.getSelection() ?? []).map(item => item.data.name);
> + },
> +
> + setValueSelection: function(value) {
> + let me = this;
> +
> + let store = me.getStore();
> +
> + let notFound = [];
> + let selection = value.map(item => {
> + let found = store.findRecord('name', item, 0, false, true, true);
> + if (!found) {
> + notFound.push(item);
> + }
> + return found;
> + }).filter(r => r);
> +
> + for (const name of notFound) {
> + let rec = store.add({
> + name,
> + type: '-',
> + comment: gettext('Included endpoint does not exist!'),
> + });
> + selection.push(rec[0]);
> + }
> +
> + let sm = me.getSelectionModel();
> + if (selection.length) {
> + sm.select(selection);
> + } else {
> + sm.deselectAll();
> + }
> + // to correctly trigger invalid class
> + me.getErrors();
> + },
> +
> + setValue: function(value) {
> + let me = this;
> +
> + let store = me.getStore();
> + if (!store.isLoaded()) {
> + me.savedValue = value;
> + store.on('load', function() {
> + me.setValueSelection(value);
> + delete me.savedValue;
> + }, { single: true });
> + } else {
> + me.setValueSelection(value);
> + }
> + return me.mixins.field.setValue.call(me, value);
> + },
> +
> + getErrors: function(value) {
> + let me = this;
> + if (!me.isDisabled() && me.allowBlank === false &&
> + me.getSelectionModel().getCount() === 0) {
> + me.addBodyCls(['x-form-trigger-wrap-default', 'x-form-trigger-wrap-invalid']);
> + return [gettext('No endpoint selected')];
> + }
> +
> + me.removeBodyCls(['x-form-trigger-wrap-default', 'x-form-trigger-wrap-invalid']);
> + return [];
> + },
> +});
> diff --git a/src/window/EndpointEditBase.js b/src/window/EndpointEditBase.js
> index 81e5951..bcf6879 100644
> --- a/src/window/EndpointEditBase.js
> +++ b/src/window/EndpointEditBase.js
> @@ -16,7 +16,11 @@ Ext.define('Proxmox.window.EndpointEditBase', {
> throw "baseUrl not set";
> }
>
> - me.url = `/api2/extjs${me.baseUrl}/endpoints/${me.type}`;
> + if (me.type === 'group') {
> + me.url = `/api2/extjs${me.baseUrl}/groups`;
> + } else {
> + me.url = `/api2/extjs${me.baseUrl}/endpoints/${me.type}`;
> + }
>
> if (me.isCreate) {
> me.method = 'POST';
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox-widget-toolkit 65/66] notification: add ui for managing notification filters
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 65/66] notification: add ui for managing notification filters Lukas Wagner
@ 2023-07-19 13:53 ` Dominik Csapak
0 siblings, 0 replies; 114+ messages in thread
From: Dominik Csapak @ 2023-07-19 13:53 UTC (permalink / raw)
To: Proxmox VE development discussion, Lukas Wagner
more or less the same comments as for 61/66
autoShow,
single line if,
column use,
etc.
On 7/17/23 17:00, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> src/Makefile | 3 +-
> src/data/model/NotificationConfig.js | 9 ++
> src/panel/NotificationConfigView.js | 119 +++++++++++++++++++++++++++
> src/window/NotificationFilterEdit.js | 115 ++++++++++++++++++++++++++
> 4 files changed, 245 insertions(+), 1 deletion(-)
> create mode 100644 src/window/NotificationFilterEdit.js
>
> diff --git a/src/Makefile b/src/Makefile
> index f661bb6..21fbe76 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -89,7 +89,8 @@ JSSRC= \
> window/ACMEAccount.js \
> window/ACMEPluginEdit.js \
> window/ACMEDomains.js \
> - window/EndpointEditBase.js \
> + window/EndpointEditBase.js \
> + window/NotificationFilterEdit.js \
> window/FileBrowser.js \
> window/AuthEditBase.js \
> window/AuthEditOpenId.js \
> diff --git a/src/data/model/NotificationConfig.js b/src/data/model/NotificationConfig.js
> index c2ce843..bb4ef85 100644
> --- a/src/data/model/NotificationConfig.js
> +++ b/src/data/model/NotificationConfig.js
> @@ -6,3 +6,12 @@ Ext.define('proxmox-notification-endpoints', {
> },
> idProperty: 'name',
> });
> +
> +Ext.define('proxmox-notification-filters', {
> + extend: 'Ext.data.Model',
> + fields: ['name', 'comment'],
> + proxy: {
> + type: 'proxmox',
> + },
> + idProperty: 'name',
> +});
> diff --git a/src/panel/NotificationConfigView.js b/src/panel/NotificationConfigView.js
> index 2aa04da..c3f7c40 100644
> --- a/src/panel/NotificationConfigView.js
> +++ b/src/panel/NotificationConfigView.js
> @@ -15,6 +15,17 @@ Ext.define('Proxmox.panel.NotificationConfigView', {
> baseUrl: '{baseUrl}',
> },
> },
> + {
> + region: 'south',
> + height: '50%',
> + border: false,
> + collapsible: true,
> + animCollapse: false,
> + xtype: 'pmxNotificationFilterView',
> + cbind: {
> + baseUrl: '{baseUrl}',
> + },
> + },
> ],
> });
>
> @@ -194,3 +205,111 @@ Ext.define('Proxmox.panel.NotificationEndpointView', {
> me.store.rstore.proxy.setUrl(`/api2/json/${me.baseUrl}/targets`);
> },
> });
> +
> +Ext.define('Proxmox.panel.NotificationFilterView', {
> + extend: 'Ext.grid.Panel',
> + alias: 'widget.pmxNotificationFilterView',
> +
> + title: gettext('Notification Filters'),
> +
> + controller: {
> + xclass: 'Ext.app.ViewController',
> +
> + openEditWindow: function(filter) {
> + let me = this;
> +
> + Ext.create('Proxmox.window.NotificationFilterEdit', {
> + baseUrl: me.getView().baseUrl,
> + name: filter,
> + listeners: {
> + destroy: () => me.reload(),
> + },
> + }).show();
> + },
> +
> + openEditForSelectedItem: function() {
> + let me = this;
> + let view = me.getView();
> +
> + let selection = view.getSelection();
> + if (selection.length < 1) return;
> + let filterName = selection[0].data.name;
> +
> + me.openEditWindow(filterName);
> + },
> +
> + reload: function() {
> + let me = this;
> + let view = me.getView();
> + view.getStore().rstore.load();
> + },
> + },
> +
> + listeners: {
> + itemdblclick: 'openEditForSelectedItem',
> + activate: 'reload',
> + },
> +
> + emptyText: gettext('No notification filters configured'),
> +
> + columns: [
> + {
> + dataIndex: 'name',
> + text: gettext('Filter Name'),
> + renderer: Ext.String.htmlEncode,
> + flex: 1,
> + },
> + {
> + dataIndex: 'comment',
> + text: gettext('Comment'),
> + renderer: Ext.String.htmlEncode,
> + flex: 2,
> + },
> + ],
> +
> + store: {
> + type: 'diff',
> + autoDestroy: true,
> + autoDestroyRstore: true,
> + rstore: {
> + type: 'update',
> + storeid: 'proxmox-notification-filters',
> + model: 'proxmox-notification-filters',
> + autoStart: true,
> + },
> + sorters: 'name',
> + },
> +
> + initComponent: function() {
> + let me = this;
> +
> + if (!me.baseUrl) {
> + throw "baseUrl is not set!";
> + }
> +
> + Ext.apply(me, {
> + tbar: [
> + {
> + xtype: 'proxmoxButton',
> + text: gettext('Add'),
> + handler: () => me.getController().openEditWindow(),
> + selModel: false,
> + },
> + {
> + xtype: 'proxmoxButton',
> + text: gettext('Modify'),
> + handler: 'openEditForSelectedItem',
> + disabled: true,
> + },
> + {
> + xtype: 'proxmoxStdRemoveButton',
> + callback: 'reload',
> + baseurl: `${me.baseUrl}/filters`,
> + },
> + ],
> + });
> +
> + me.callParent();
> + me.store.rstore.proxy.setUrl(`/api2/json/${me.baseUrl}/filters`);
> + },
> +});
> diff --git a/src/window/NotificationFilterEdit.js b/src/window/NotificationFilterEdit.js
> new file mode 100644
> index 0000000..6a48446
> --- /dev/null
> +++ b/src/window/NotificationFilterEdit.js
> @@ -0,0 +1,115 @@
> +Ext.define('Proxmox.panel.NotificationFilterEditPanel', {
> + extend: 'Proxmox.panel.InputPanel',
> + xtype: 'pmxNotificationFilterEditPanel',
> + mixins: ['Proxmox.Mixin.CBind'],
> +
> + columnT: [
> + {
> + xtype: 'pmxDisplayEditField',
> + name: 'name',
> + cbind: {
> + value: '{name}',
> + editable: '{isCreate}',
> + },
> + fieldLabel: gettext('Filter Name'),
> + allowBlank: false,
> + },
> + {
> + xtype: 'proxmoxKVComboBox',
> + name: 'min-severity',
> + fieldLabel: gettext('Minimum Severity'),
> + value: null,
> + cbind: {
> + deleteEmpty: '{!isCreate}',
> + },
> + comboItems: [
> + ['info', 'info'],
> + ['notice', 'notice'],
> + ['warning', 'warning'],
> + ['error', 'error'],
> + ],
> + triggers: {
> + clear: {
> + cls: 'pmx-clear-trigger',
> + weight: -1,
> + hidden: false,
> + handler: function() {
> + this.setValue('');
> + },
> + },
> + },
> + },
> + {
> + xtype: 'proxmoxcheckbox',
> + fieldLabel: gettext('Invert match'),
> + name: 'invert-match',
> + uncheckedValue: 0,
> + defaultValue: 0,
> + cbind: {
> + deleteDefaultValue: '{!isCreate}',
> + },
> + },
> + ],
> +
> + column1: [],
> +
> + column2: [],
> +
> + columnB: [
> + {
> + xtype: 'proxmoxtextfield',
> + name: 'comment',
> + fieldLabel: gettext('Comment'),
> + cbind: {
> + deleteEmpty: '{!isCreate}',
> + },
> + },
> + ],
> +});
> +
> +Ext.define('Proxmox.window.NotificationFilterEdit', {
> + extend: 'Proxmox.window.Edit',
> +
> + isAdd: true,
> +
> + fieldDefaults: {
> + labelWidth: 120,
> + },
> +
> + initComponent: function() {
> + let me = this;
> +
> + me.isCreate = !me.name;
> +
> + if (!me.baseUrl) {
> + throw "baseUrl not set";
> + }
> +
> + me.url = `/api2/extjs${me.baseUrl}/filters`;
> +
> + if (me.isCreate) {
> + me.method = 'POST';
> + } else {
> + me.url += `/${me.name}`;
> + me.method = 'PUT';
> + }
> +
> + me.subject = gettext('Notification Filter');
> +
> + Ext.apply(me, {
> + items: [{
> + name: me.name,
> + xtype: 'pmxNotificationFilterEditPanel',
> + isCreate: me.isCreate,
> + baseUrl: me.baseUrl,
> + }],
> + });
> +
> + me.callParent();
> +
> + if (!me.isCreate) {
> + me.load();
> + }
> + },
> +});
> +
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-manager 57/66] ui: allow to configure notification event -> target mapping
2023-07-19 12:45 ` Dominik Csapak
@ 2023-07-19 15:25 ` Lukas Wagner
0 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-19 15:25 UTC (permalink / raw)
To: Dominik Csapak, Proxmox VE development discussion
Hi,
On 7/19/23 14:45, Dominik Csapak wrote:
>> +
>> +Ext.define('PVE.dc.NotificationEventsTargetSelector', {
>> + alias: ['widget.pveNotificationEventsTargetSelector'],
>> + extend: 'PVE.form.NotificationTargetSelector',
>> + fieldLabel: gettext('Notification Target'),
>> + allowBlank: true,
>> + editable: true,
>> + autoSelect: false,
>> + deleteEmpty: false,
>> + emptyText: `${Proxmox.Utils.defaultText} (${gettext("mail-to-root")})`,
>> +});
>> +
>> +Ext.define('PVE.dc.NotificationEvents', {
>> + extend: 'Proxmox.grid.ObjectGrid',
>> + alias: ['widget.pveNotificationEvents'],
>> +
>> + // Taken from OptionView.js, but adapted slightly.
>
> what exactly was adapted? i can ofc diff it myself, but it would
> be nicer to have that info either in a comment or the commit message.
> also we should factor this out and reuse it in OptionView and here?
> maybe just adding it to the ObjectGrid itself?
> (if possible)
I added some more comments to explain what has been changed. However I'm not sure
if there is an easy way to reuse the changes anywhere. Basically the changes were needed
because I'm trying to render multiple rows in the ObjectGrid from a single
key in datacenter.cfg (notify), which contains the target/policy settings.
>
>> + addInputPanelRow: function(name, propertyName, text, opts) {
>> + let me = this;
>> +
>> + opts = opts || {};
(...)
>
> you should be able to reuse the render_value if you add this there also
> it can't trigger for the others anyway?
>
Originally, I did not reuse render_value because `package-updates` has a different default,
`auto` instead of `always`. However, with an additional parameter for `render_value` I was
able to consolidate both variants.
>> + break;
>> + case 'never':
>> + template = gettext('Never');
>> + break;
>> + default:
>> + template = gettext('{1} (Automatically), notify via target \'{0}\'');
>> + break;
>> + }
>> +
>> + return Ext.String.format(template, target, Proxmox.Utils.defaultText);
>> + },
>> + url: "/api2/extjs/cluster/options",
>> + items: [
>> + {
>> + xtype: 'pveNotificationEventsPolicySelector',
>
> as said above i'd simply make this a KVComboBox to indicate it's
> basically a seperate component
I actually decided to keep the separate component due to some other changes,
mainly the addition of a custom listener that shows/hides the warning textfield.
That would have led to a lot of code duplication that is now avoided by the
component.
However, I decided to move the `comboItems` to where the component is actually used.
>
>> + name: 'package-updates',
>> + fieldLabel: gettext('Notify'),
>> + comboItems: [
>> + ['__default__', Proxmox.Utils.defaultText + ' (auto)'],
>> + ['auto', gettext('Automatically')],
>> + ['always', gettext('Always')],
>> + ['never', gettext('Never')],
>> + ],
>> + },
>> + {
>> + xtype: 'pveNotificationEventsTargetSelector',
>> + name: 'target-package-updates',
>> + },
>> + ],
>> + });
>> +
>> + // Hack: Also load the notify property to make it accessible
>> + // for our render functions. initComponents later hides it.
>> + me.add_text_row('notify', gettext('Notify'), {});
>
> it should be possible to simply add it directly here with something like:
>
> me.rows.notify = {
> visible: false,
> };
>
>>
> we e.g. do something like this in dc/Optionsview and qemu/HardwareView
> (the latter is a PendingObjectGrid, but still inherits from ObjectGrid)
Thanks, that did work rather nicely. It also fixed a graphical glitch where the 'notify' row
would be visible for a split second after a page refresh.
--
- Lukas
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-manager 59/66] ui: perm path: load notification target/filter acl entries
2023-07-19 12:53 ` Dominik Csapak
@ 2023-07-20 7:46 ` Lukas Wagner
2023-07-20 7:54 ` Dominik Csapak
0 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-20 7:46 UTC (permalink / raw)
To: Dominik Csapak, Proxmox VE development discussion
On 7/19/23 14:53, Dominik Csapak wrote:
> On 7/17/23 17:00, Lukas Wagner wrote:
>> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
>> ---
>>
>> Notes:
>> I'm not sure if I like this solution, but adding notification targets to
>> the resources API endpoint would not have make sense.
>> Maybe we could create a new API endpoint that returns all possible ACL
>> paths and then use a normal store for the perm path combobox?
>
> i'd also prefer that, it would make this much simpler, and more manageable
> for the pci/usb mappings i simply omitted them here, so for now
> we could simply hardcode the different types here
>
> /mapping/usb
> /mapping/pci
> /mapping/notification
>
> and be done with it until we have said api call ;)
I guess that's a good compromise for now. If any user needs the per-endpoint permissions
in the meanwhile, they could just use the CLI to set it.
I'll also put the 'enumerate ACL paths' API call on my task backlog.
>
> AFAIR thomas opposed it the last time i wanted to add an api call here
> (i can't remember what for though)
>
>>
>> www/manager6/data/PermPathStore.js | 26 +++++++++++++++++++++++++-
>> 1 file changed, 25 insertions(+), 1 deletion(-)
>>
>> diff --git a/www/manager6/data/PermPathStore.js b/www/manager6/data/PermPathStore.js
>> index c3ac7f0e..b7e4fa33 100644
>> --- a/www/manager6/data/PermPathStore.js
>> +++ b/www/manager6/data/PermPathStore.js
>> @@ -9,6 +9,7 @@ Ext.define('PVE.data.PermPathStore', {
>> { 'value': '/access/groups' },
>> { 'value': '/access/realm' },
>> { 'value': '/mapping' },
>> + { 'value': '/mapping/notification' },
>> { 'value': '/nodes' },
>> { 'value': '/pool' },
>> { 'value': '/sdn/zones' },
>> @@ -46,8 +47,31 @@ Ext.define('PVE.data.PermPathStore', {
>> donePaths[path] = 1;
>> }
>> });
>> - me.resumeEvents();
>> + Ext.Ajax.request({
>
> also why ext.ajax.request and not API2Request ? ;)
>
Good question, there is no good reason.
I'm not sure where I copied that from :D. Thanks for the hint!
--
- Lukas
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-manager 59/66] ui: perm path: load notification target/filter acl entries
2023-07-20 7:46 ` Lukas Wagner
@ 2023-07-20 7:54 ` Dominik Csapak
2023-07-20 8:22 ` Lukas Wagner
0 siblings, 1 reply; 114+ messages in thread
From: Dominik Csapak @ 2023-07-20 7:54 UTC (permalink / raw)
To: Lukas Wagner, Proxmox VE development discussion
On 7/20/23 09:46, Lukas Wagner wrote:
>
>
> On 7/19/23 14:53, Dominik Csapak wrote:
>> On 7/17/23 17:00, Lukas Wagner wrote:
>>> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
>>> ---
>>>
>>> Notes:
>>> I'm not sure if I like this solution, but adding notification targets to
>>> the resources API endpoint would not have make sense.
>>> Maybe we could create a new API endpoint that returns all possible ACL
>>> paths and then use a normal store for the perm path combobox?
>>
>> i'd also prefer that, it would make this much simpler, and more manageable
>> for the pci/usb mappings i simply omitted them here, so for now
>> we could simply hardcode the different types here
>>
>> /mapping/usb
>> /mapping/pci
>> /mapping/notification
>>
>> and be done with it until we have said api call ;)
>
> I guess that's a good compromise for now. If any user needs the per-endpoint permissions
> in the meanwhile, they could just use the CLI to set it.
> I'll also put the 'enumerate ACL paths' API call on my task backlog.
>
i'ts even easier, the user can simply edit the field manually ;)
i know this trips up many people, but we often have dropdown fields
that are actually manually editable (like the permpathselector)
but it's only hinted at by the blinking cursor in the field.
we should come up with a more apparent way of making that clear...
(but i currently can't think of one)
>>
>> AFAIR thomas opposed it the last time i wanted to add an api call here
>> (i can't remember what for though)
>>
>>>
>>> www/manager6/data/PermPathStore.js | 26 +++++++++++++++++++++++++-
>>> 1 file changed, 25 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/www/manager6/data/PermPathStore.js b/www/manager6/data/PermPathStore.js
>>> index c3ac7f0e..b7e4fa33 100644
>>> --- a/www/manager6/data/PermPathStore.js
>>> +++ b/www/manager6/data/PermPathStore.js
>>> @@ -9,6 +9,7 @@ Ext.define('PVE.data.PermPathStore', {
>>> { 'value': '/access/groups' },
>>> { 'value': '/access/realm' },
>>> { 'value': '/mapping' },
>>> + { 'value': '/mapping/notification' },
>>> { 'value': '/nodes' },
>>> { 'value': '/pool' },
>>> { 'value': '/sdn/zones' },
>>> @@ -46,8 +47,31 @@ Ext.define('PVE.data.PermPathStore', {
>>> donePaths[path] = 1;
>>> }
>>> });
>>> - me.resumeEvents();
>>> + Ext.Ajax.request({
>>
>> also why ext.ajax.request and not API2Request ? ;)
>>
>
> Good question, there is no good reason.
> I'm not sure where I copied that from :D. Thanks for the hint!
>
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-manager 59/66] ui: perm path: load notification target/filter acl entries
2023-07-20 7:54 ` Dominik Csapak
@ 2023-07-20 8:22 ` Lukas Wagner
2023-07-20 8:29 ` Fiona Ebner
0 siblings, 1 reply; 114+ messages in thread
From: Lukas Wagner @ 2023-07-20 8:22 UTC (permalink / raw)
To: Dominik Csapak, Proxmox VE development discussion
On 7/20/23 09:54, Dominik Csapak wrote:
>
> i'ts even easier, the user can simply edit the field manually ;)
> i know this trips up many people, but we often have dropdown fields
> that are actually manually editable (like the permpathselector)
> but it's only hinted at by the blinking cursor in the field.
>
> we should come up with a more apparent way of making that clear...
> (but i currently can't think of one)
>
Aaaah true, I didn't realize that. These non-obvious editable combo boxes
have bugged me for a while now. Maybe I'll look into it to see if there is a way to make
these a bit more obvious to use. IIRC, our schedule pickers have '(editable)' in the empty text,
but that feels more like a workaround rather than an actual solution to that issue.
--
- Lukas
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-manager 59/66] ui: perm path: load notification target/filter acl entries
2023-07-20 8:22 ` Lukas Wagner
@ 2023-07-20 8:29 ` Fiona Ebner
2023-07-20 9:26 ` Maximiliano Sandoval
0 siblings, 1 reply; 114+ messages in thread
From: Fiona Ebner @ 2023-07-20 8:29 UTC (permalink / raw)
To: Proxmox VE development discussion, Lukas Wagner, Dominik Csapak
Am 20.07.23 um 10:22 schrieb Lukas Wagner:
>
>
> On 7/20/23 09:54, Dominik Csapak wrote:
>>
>> i'ts even easier, the user can simply edit the field manually ;)
>> i know this trips up many people, but we often have dropdown fields
>> that are actually manually editable (like the permpathselector)
>> but it's only hinted at by the blinking cursor in the field.
>>
>> we should come up with a more apparent way of making that clear...
>> (but i currently can't think of one)
>>
>
> Aaaah true, I didn't realize that. These non-obvious editable combo boxes
> have bugged me for a while now. Maybe I'll look into it to see if there
> is a way to make
> these a bit more obvious to use. IIRC, our schedule pickers have
> '(editable)' in the empty text,
> but that feels more like a workaround rather than an actual solution to
> that issue.
>
Maybe some kind of pencil icon on the side within the field? With an
"Edit" tooltip, selecting the text when clicking it? Just an idea that
came to mind, not sure if it's any good.
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-manager 59/66] ui: perm path: load notification target/filter acl entries
2023-07-20 8:29 ` Fiona Ebner
@ 2023-07-20 9:26 ` Maximiliano Sandoval
2023-07-20 15:02 ` Thomas Lamprecht
0 siblings, 1 reply; 114+ messages in thread
From: Maximiliano Sandoval @ 2023-07-20 9:26 UTC (permalink / raw)
To: Proxmox VE development discussion, Fiona Ebner, Lukas Wagner,
Dominik Csapak
Just as a reference, in GNOME editable entries, when in a context
where it is not clear they are editable, have a pencil symbolic icon as you
suggested, but there is no "Edit" tooltip. Another thing is that the icon
disappears if the user has the text cursor in the entry.
I have no strong opinion on the tooltip, I will look into implementing this.
> On 20.07.2023 10:29 CEST Fiona Ebner <f.ebner@proxmox.com> wrote:
>
>
> Am 20.07.23 um 10:22 schrieb Lukas Wagner:
> >
> >
> > On 7/20/23 09:54, Dominik Csapak wrote:
> >>
> >> i'ts even easier, the user can simply edit the field manually ;)
> >> i know this trips up many people, but we often have dropdown fields
> >> that are actually manually editable (like the permpathselector)
> >> but it's only hinted at by the blinking cursor in the field.
> >>
> >> we should come up with a more apparent way of making that clear...
> >> (but i currently can't think of one)
> >>
> >
> > Aaaah true, I didn't realize that. These non-obvious editable combo boxes
> > have bugged me for a while now. Maybe I'll look into it to see if there
> > is a way to make
> > these a bit more obvious to use. IIRC, our schedule pickers have
> > '(editable)' in the empty text,
> > but that feels more like a workaround rather than an actual solution to
> > that issue.
> >
>
> Maybe some kind of pencil icon on the side within the field? With an
> "Edit" tooltip, selecting the text when clicking it? Just an idea that
> came to mind, not sure if it's any good.
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 proxmox-widget-toolkit 63/66] notification: add gui for notification groups
2023-07-19 13:32 ` Dominik Csapak
@ 2023-07-20 12:31 ` Lukas Wagner
0 siblings, 0 replies; 114+ messages in thread
From: Lukas Wagner @ 2023-07-20 12:31 UTC (permalink / raw)
To: Dominik Csapak, Proxmox VE development discussion
On 7/19/23 15:32, Dominik Csapak wrote:
>> + mixins: {
>> + field: 'Ext.form.field.Field',
>> + },
>
> when implementing the field mixin you have to (quote from the extjs docs)
>
> ---8<---
> You will also need to make sure that initField is called during the component's initialization.
> --->8---
>
> so you normally need a (small) initComponent that calls that
> (otherwise you can have some strange effects when using the field)
>
Thanks, added. For the record, `initField` requires `callParent` to be called before.
I noticed that we are missing a call to `initField` in the following components as well:
MultiDiskSelector (widget-toolkit)
VMCPUFlagSelector (pve-manager)
VMSelector (pve-manager)
Haven't looked into pmg/pbs stuff, maybe some components there might be missing it as well.
--
- Lukas
^ permalink raw reply [flat|nested] 114+ messages in thread
* Re: [pve-devel] [PATCH v3 pve-manager 59/66] ui: perm path: load notification target/filter acl entries
2023-07-20 9:26 ` Maximiliano Sandoval
@ 2023-07-20 15:02 ` Thomas Lamprecht
0 siblings, 0 replies; 114+ messages in thread
From: Thomas Lamprecht @ 2023-07-20 15:02 UTC (permalink / raw)
To: Proxmox VE development discussion, Maximiliano Sandoval,
Fiona Ebner, Lukas Wagner, Dominik Csapak
On 20/07/2023 11:26, Maximiliano Sandoval wrote:
>> On 20.07.2023 10:29 CEST Fiona Ebner <f.ebner@proxmox.com> wrote:
>> Am 20.07.23 um 10:22 schrieb Lukas Wagner:
>>> Aaaah true, I didn't realize that. These non-obvious editable combo boxes
>>> have bugged me for a while now. Maybe I'll look into it to see if there
>>> is a way to make
>>> these a bit more obvious to use. IIRC, our schedule pickers have
>>> '(editable)' in the empty text,
Yeah I did that and am not really happy with it, and we had at least some
users that were still confused, we naturally have no numbers of people where
this empty-text alone was enough to understand it clearly, so hard to say what
the net benefit of it was.
>>> but that feels more like a workaround rather than an actual solution to
>>> that issue.
>>>
>>
>> Maybe some kind of pencil icon on the side within the field? With an
>> "Edit" tooltip, selecting the text when clicking it? Just an idea that
>> came to mind, not sure if it's any good.
>>
For editable dropdowns I'd might be better option to have a extra entry in
the selector, like "custom" or "manual",maybe even with the pencil icon to
distinguish it from the actual choices. If selected it can simply set the
text to '' and focus the field (i.e., just give it a value of '').
ps. can we *please* avoid top-posting, rather use interleaved posting (or
at least bottom) also trim the context of replies to what matters (applies
not only to you in this thread)
^ permalink raw reply [flat|nested] 114+ messages in thread
end of thread, other threads:[~2023-07-20 15:02 UTC | newest]
Thread overview: 114+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-07-17 14:59 [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 01/66] add proxmox-notify crate Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 02/66] notify: preparation for the first endpoint plugin Lukas Wagner
2023-07-17 15:48 ` Maximiliano Sandoval
2023-07-18 7:19 ` Lukas Wagner
2023-07-18 10:13 ` Wolfgang Bumiller
2023-07-18 11:54 ` Wolfgang Bumiller
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 03/66] notify: preparation for the API Lukas Wagner
2023-07-18 12:02 ` Wolfgang Bumiller
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 04/66] notify: api: add API for sending notifications/testing endpoints Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 05/66] notify: add sendmail plugin Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 06/66] notify: api: add API for sendmail endpoints Lukas Wagner
2023-07-18 12:36 ` Wolfgang Bumiller
2023-07-19 11:51 ` Lukas Wagner
2023-07-19 12:09 ` Wolfgang Bumiller
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 07/66] notify: add gotify endpoint Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 08/66] notify: api: add API for gotify endpoints Lukas Wagner
2023-07-18 12:44 ` Wolfgang Bumiller
2023-07-18 13:19 ` Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 09/66] notify: add notification groups Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 10/66] notify: api: add API for groups Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 11/66] notify: add notification filter mechanism Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 12/66] notify: api: add API for filters Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 13/66] notify: add template rendering Lukas Wagner
2023-07-17 14:59 ` [pve-devel] [PATCH v3 proxmox 14/66] notify: add example for " Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 15/66] notify: add context Lukas Wagner
2023-07-18 12:57 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 16/66] notify: sendmail: allow users as recipients Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 17/66] notify: sendmail: query default author/mailfrom from context Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 18/66] notify: gotify: add proxy support Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 19/66] notify: api: allow to query entities referenced by filter/target Lukas Wagner
2023-07-18 13:02 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 20/66] notify: on deletion, check if a filter/endp. is still used by anything Lukas Wagner
2023-07-18 13:20 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 21/66] notify: ensure that filter/group/endpoint names are unique Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 22/66] notify: additional logging when sending a notification Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox 23/66] notify: add debian packaging Lukas Wagner
2023-07-18 13:25 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 24/66] add PVE::RS::Notify module Lukas Wagner
2023-07-19 10:10 ` Wolfgang Bumiller
2023-07-19 10:23 ` Wolfgang Bumiller
2023-07-19 10:37 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 25/66] notify: add api for sending notifications/testing endpoints Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 26/66] notify: add api for notification groups Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 27/66] notify: add api for sendmail endpoints Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 28/66] notify: add api for gotify endpoints Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 29/66] notify: add api for notification filters Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 30/66] notify: sendmail: support the `mailto-user` parameter Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 31/66] notify: implement context for getting default author/mailfrom Lukas Wagner
2023-07-19 11:15 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 32/66] notify: add context for getting http_proxy from datacenter.cfg Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-perl-rs 33/66] notify: add wrapper for `get_referenced_entities` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-cluster 34/66] cluster files: add notifications.cfg Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-cluster 35/66] datacenter: add APT/fencing/replication notification configuration Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-cluster 36/66] add libpve-notify-perl package Lukas Wagner
2023-07-19 12:27 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-guest-common 37/66] vzdump: add config options for new notification backend Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-common 38/66] JSONSchema: increase maxLength of config-digest to 64 Lukas Wagner
2023-07-19 12:31 ` Wolfgang Bumiller
2023-07-19 12:41 ` Fiona Ebner
2023-07-19 12:49 ` Wolfgang Bumiller
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-ha-manager 39/66] manager: send notifications via new notification module Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 40/66] test: fix names of .PHONY targets Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 41/66] d/control: add dependency to `libpve-notify-perl` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 42/66] vzdump: send notifications via new notification module Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 43/66] test: rename mail_test.pl to vzdump_notification_test.pl Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 44/66] api: apt: send notification via new notification module Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 45/66] api: replication: send notifications " Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 46/66] api: prepare api handler module for notification config Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 47/66] api: notification: add api routes for groups Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 48/66] api: notification: add api routes for sendmail endpoints Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 49/66] api: notification: add api routes for gotify endpoints Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 50/66] api: notification: add api routes for filters Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 51/66] api: notification: allow fetching notification targets Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 52/66] api: notification: allow to test targets Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 53/66] api: notification: disallow removing targets if they are used Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 54/66] ui: backup: allow to select notification target for jobs Lukas Wagner
2023-07-19 12:20 ` Dominik Csapak
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 55/66] ui: backup: adapt backup job details to new notification params Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 56/66] ui: backup: allow to set notification-target for one-off backups Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 57/66] ui: allow to configure notification event -> target mapping Lukas Wagner
2023-07-19 12:45 ` Dominik Csapak
2023-07-19 15:25 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 58/66] ui: add notification target configuration panel Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 59/66] ui: perm path: load notification target/filter acl entries Lukas Wagner
2023-07-19 12:53 ` Dominik Csapak
2023-07-20 7:46 ` Lukas Wagner
2023-07-20 7:54 ` Dominik Csapak
2023-07-20 8:22 ` Lukas Wagner
2023-07-20 8:29 ` Fiona Ebner
2023-07-20 9:26 ` Maximiliano Sandoval
2023-07-20 15:02 ` Thomas Lamprecht
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-manager 60/66] ui: perm path: increase width of the perm path selector combobox Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 61/66] notification: add gui for sendmail notification endpoints Lukas Wagner
2023-07-19 13:25 ` Dominik Csapak
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 62/66] notification: add gui for gotify " Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 63/66] notification: add gui for notification groups Lukas Wagner
2023-07-19 13:32 ` Dominik Csapak
2023-07-20 12:31 ` Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 64/66] notification: allow to select filter for notification targets Lukas Wagner
2023-07-17 15:00 ` [pve-devel] [PATCH v3 proxmox-widget-toolkit 65/66] notification: add ui for managing notification filters Lukas Wagner
2023-07-19 13:53 ` Dominik Csapak
2023-07-17 15:00 ` [pve-devel] [PATCH v3 pve-docs 66/66] add documentation for the new notification system Lukas Wagner
2023-07-18 12:34 ` [pve-devel] [PATCH v3 many 00/66] fix #4156: introduce " Dominik Csapak
2023-07-18 13:14 ` Lukas Wagner
2023-07-18 13:58 ` Dominik Csapak
2023-07-18 14:07 ` Lukas Wagner
2023-07-18 14:37 ` Thomas Lamprecht
2023-07-19 13:13 ` Lukas Wagner
2023-07-19 8:40 ` Lukas Wagner
2023-07-19 9:54 ` Wolfgang Bumiller
2023-07-18 13:27 ` Wolfgang Bumiller
2023-07-19 12:11 ` Wolfgang Bumiller
2023-07-19 12:17 ` Lukas Wagner
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox