From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id CBB3F1FF13F for ; Thu, 09 Apr 2026 14:38:57 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 61FA545A1; Thu, 9 Apr 2026 14:39:41 +0200 (CEST) Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 Date: Thu, 09 Apr 2026 14:39:05 +0200 Message-Id: To: "Lukas Wagner" , "Arthur Bied-Charreton" , Subject: Re: [PATCH datacenter-manager v2] Add notifications backend X-Mailer: aerc 0.20.0 References: <20260409045819.19858-1-a.bied-charreton@proxmox.com> In-Reply-To: From: "Shannon Sterz" X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1775738276749 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.123 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [lib.rs,context.rs,targets.rs,mod.rs,gotify.rs,sendmail.rs,smtp.rs,notifications.rs,defines.mk,matchers.rs,webhook.rs] Message-ID-Hash: 3AZTQNPLQAB2OLJJ47QTTZYC2JJVPCW2 X-Message-ID-Hash: 3AZTQNPLQAB2OLJJ47QTTZYC2JJVPCW2 X-MailFrom: s.sterz@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Datacenter Manager development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Thu Apr 9, 2026 at 2:07 PM CEST, Lukas Wagner wrote: > On Thu Apr 9, 2026 at 11:20 AM CEST, Shannon Sterz wrote: >> On Thu Apr 9, 2026 at 6:57 AM CEST, Arthur Bied-Charreton wrote: -->8 snip 8<-- >> >> any reason to not go ahead and add at least one notification in this >> series too? being able to test that notifications function, but not >> having anything to be notified about seems a little odd to me. one easy >> notification you could add is the `send_certificate_renewal_mail` in >> `api::nodes::certificates::spawn_certificate_worker()`. that should >> essentially be identical to pbs anyway. >> > > This was coordinated with me, I told him that it would be okay to send > this part early. He can either build on top of this series and include > more patches (e.g. adding the GUI, notification events, docs, etc.) in > future versions, or if it makes sense we can also apply certain parts > early. > ack, was not aware of that. > >>> The endpoints are made unprotected and the configuration files >>> read/writable by `www-data`, like for `remotes.cfg`. >>> >>> Signed-off-by: Arthur Bied-Charreton >>> --- >>> Cargo.toml | 1 + >>> Makefile | 3 +- >>> debian/proxmox-datacenter-manager.install | 2 + >>> defines.mk | 1 + >>> lib/pdm-config/Cargo.toml | 1 + >>> lib/pdm-config/src/lib.rs | 1 + >>> lib/pdm-config/src/notifications.rs | 39 ++++ >>> server/Cargo.toml | 1 + >>> server/src/api/config/mod.rs | 2 + >>> server/src/api/config/notifications/gotify.rs | 185 +++++++++++++++++ >>> .../src/api/config/notifications/matchers.rs | 165 ++++++++++++++++ >>> server/src/api/config/notifications/mod.rs | 102 ++++++++++ >>> .../src/api/config/notifications/sendmail.rs | 173 ++++++++++++++++ >>> server/src/api/config/notifications/smtp.rs | 186 ++++++++++++++++++ >>> .../src/api/config/notifications/targets.rs | 61 ++++++ >>> .../src/api/config/notifications/webhook.rs | 170 ++++++++++++++++ >>> server/src/context.rs | 2 + >>> server/src/lib.rs | 1 + >>> server/src/notifications/mod.rs | 115 +++++++++++ >>> templates/Makefile | 14 ++ >>> templates/default/test-body.txt.hbs | 1 + >>> templates/default/test-subject.txt.hbs | 1 + >>> 22 files changed, 1226 insertions(+), 1 deletion(-) >>> create mode 100644 lib/pdm-config/src/notifications.rs >>> create mode 100644 server/src/api/config/notifications/gotify.rs >>> create mode 100644 server/src/api/config/notifications/matchers.rs >>> create mode 100644 server/src/api/config/notifications/mod.rs >>> create mode 100644 server/src/api/config/notifications/sendmail.rs >>> create mode 100644 server/src/api/config/notifications/smtp.rs >>> create mode 100644 server/src/api/config/notifications/targets.rs >>> create mode 100644 server/src/api/config/notifications/webhook.rs >>> create mode 100644 server/src/notifications/mod.rs >>> create mode 100644 templates/Makefile >>> create mode 100644 templates/default/test-body.txt.hbs >>> create mode 100644 templates/default/test-subject.txt.hbs >>> > > [...] > >>> + >>> +use pdm_buildcfg::configdir; >>> +use proxmox_product_config::{open_api_lockfile, replace_config, ApiLoc= kGuard}; >>> +use proxmox_sys::fs::file_read_optional_string; >>> + >>> +/// Configuration file location for notification targets/matchers. >>> +pub const NOTIFICATION_CONFIG_PATH: &str =3D configdir!("/notification= s.cfg"); >>> + >>> +/// Private configuration file location for secrets - only readable by= `root`. >>> +pub const NOTIFICATION_PRIV_CONFIG_PATH: &str =3D configdir!("/notific= ations-priv.cfg"); >>> + >>> +/// Lockfile to prevent concurrent write access. >>> +pub const NOTIFICATION_LOCK_FILE: &str =3D configdir!("/.notifications= .lck"); >>> + >> >> you might want to move these to their own "notifications" sub-folder or >> similar (maybe just "notify"?). in pdm we tend to put config files in >> topically fitting sub-folders (e.g. access, acme, auth) >> > > Would be okay for me, but I'd prefer `notifications` for consistency > with what we do on other products. > sounds fine to me, i mostly suggested "notify" because it's shorter. >>> +/// Get exclusive lock for `notifications.cfg`. >>> +pub fn lock_config() -> Result { >>> + open_api_lockfile(NOTIFICATION_LOCK_FILE, None, true) >>> +} >>> + >>> +/// Load notification config. >>> +pub fn config() -> Result { >>> + let content =3D file_read_optional_string(NOTIFICATION_CONFIG_PATH= )?.unwrap_or_default(); >>> + >>> + let priv_content =3D >>> + file_read_optional_string(NOTIFICATION_PRIV_CONFIG_PATH)?.unwr= ap_or_default(); >>> + >>> + Ok(Config::new(&content, &priv_content)?) >> >> you might want to consider returning a `ConfigDigest` here too. >> returning that encourages validating the digest as its less easily >> forgotten. > > digest-checking is handled in the API handler implementations in > proxmox_notify::api where it seemed useful (see my later comments), it > cannot really be forgotten since it is a required parameter for these > functions. yes that's true for this crate, but from what i can tell returning a config alongside its digest is a common pattern. but you are righ, in this case we enforce it through `proxmox_notify`, so should be fine. > I guess we could do a > > let config =3D Config::new(&content, &priv_content)?; > > Ok((config, config.digest())) > > here, but I'm not sure it gains us much? > >> >>> +} >>> + >>> +/// Save notification config. >>> +pub fn save_config(config: Config) -> Result<(), Error> { >>> + let (cfg, priv_cfg) =3D config.write()?; >>> + replace_config(NOTIFICATION_CONFIG_PATH, cfg.as_bytes())?; >>> + replace_config(NOTIFICATION_PRIV_CONFIG_PATH, priv_cfg.as_bytes())= ?; >> >> the privileged config should probably be saved with higher permission >> than the general config. consider using `replace_secret_config` here. >> > > Unlike PBS, the notification system will run in the unprivileged process > in PDM, so that webhook/smtp stuff will not run as root. This means that > we also need to store the secrets with less strict permissions. We > already do the same for remotes.cfg and remotes.shadow, so this should > be fine here as well, I think. > Of course, we could also use the same approach as in PBS (notification > code runs as root, secrets are 0600 root:root), but I prefer the > approach used here. ack, was not aware that we want to switch to this approach here. but in hindsight it makes sense why `test_target` wasn't marked as protected. >>> +/// Add a new gotify endpoint. >>> +pub fn add_endpoint( >>> + endpoint: GotifyConfig, >>> + token: String, >>> + _rpcenv: &mut dyn RpcEnvironment, >>> +) -> Result<(), Error> { >>> + let _lock =3D pdm_config::notifications::lock_config()?; >>> + let mut config =3D pdm_config::notifications::config()?; >> >> this should probably verify the digest. if you return a config >> digest as described above this could become: >> >> let mut (config, expected_digest) =3D domains::config()?; >> expected_digest.detect_modification(digest.as_ref())?; >> >> assuming that you take the digest as an input parameter to this api call >> with: >> >> digest: { >> optional: true, >> type: ConfigDigest, >> } >> >> and: >> >> digest: Option >> > > We lock the config anyway, so it does not really matter (I think?) if > somebody else modified some other entity? And if somebody had added an > entity with the same id, then we fail anyways... So I'm not sure if > using the digest here gains us anything? > > I quickly checked a couple add_* handlers in PBS, we don't > really check the digest when adding new entities there as well. > hm yeah i guess you are right, it wouldn't add too much. guess im just a little paranoid about modifying on top of an unknown state. but should probably be fine in these cases. > Generally, with regards to using the ConfigDigest type: +1 > I think this type was introduced well after the notification API was > added to PBS, which might be the reason I did not use them there. yep hence why i think switching to it would make sense now. >>> +pub fn delete_endpoint(name: String, _rpcenv: &mut dyn RpcEnvironment)= -> Result<(), Error> { >>> + let _lock =3D pdm_config::notifications::lock_config()?; >>> + let mut config =3D pdm_config::notifications::config()?; >>> + proxmox_notify::api::gotify::delete_gotify_endpoint(&mut config, &= name)?; >>> + >> >> this would probably also benefit from checking the config digest. >> > > Same here, I'm not really sure what the benefit would be here? > > IMO there are a couple cases to consider for concurrent modifications: > - somebody modified entity A, we delete B -> should be fine > - somebody modified entity A, we delete A -> does not matter, we want > to delete it anyways > - we deleted A, somebody else modifies A -> update fails anyways due > to the wrong config digest or the entity missing already > - both try to delete A -> fails for one of both due to the already > deleted entity (HTTP 404) > > Did I miss something? > see above. -->8 snip 8<--