[pve-devel] [PATCH v4 many 00/11] notifications: add SMTP endpoint

[Lukas Wagner <l.wagner@proxmox.com>]

This patch series adds support for a new notification endpoint type,
smtp. As the name suggests, this new endpoint allows PVE to talk
to SMTP server directly, without using the system's MTA (postfix).

On the Rust side, these patches add a new dependency to the `lettre`
crate for SMTP communication. This crate was chosen as it is:
  - by far the most popular mailing crate for Rust
  - well maintained
  - has reasonable dependencies
  - has async support, enabling us to asyncify the proxmox-notify
    crate at some point, if needed

Tested against:
  - the gmail SMTP server
  - the posteo SMTP server
  - our own webmail SMTP server

These patches require a couple of other patches that have not been applied yet:

  series: "overhaul notification system, use matchers instead of filters" [3], 
  pve-manager: "api: notifications: give targets and matchers their own ACL namespace" [4]
  pve-docs: "notifications: update docs to for matcher-based notifications" [5]

The first two patches were cherry-picked and rebased from the  'system mail 
forwarding' patch series from [2].  I decided to pull them in so that I can
already implement the mail forwarding part for SMTP targets.

This series also required updating the 'lettre' crate since
one of lettre's deps was bumped to a new version by us.

Changes since v3:
  - Rebased on top of the matcher-based notification revamp
  - Removed 'filter' setting from target configuration
  - Pulled in required patches from 'system mail forwarding' patch series

Changes since v2:
  - Rebased proxmox-widget-toolkit onto the latest master to avoid
    any conflicts.

Changes since v1:
  - Rebased on top of [1]
  - Added a mechanism for mails forwarded by `proxmox-mail-forward`
    These are forwarded inline as "message/rfc822" to avoid having 
    to rewrite mail headers (otherwise, some SMTP relays might reject the 
    mail, because the `From` header of the forwarded mail does not match the
    mail account)

[1] https://lists.proxmox.com/pipermail/pve-devel/2023-August/058956.html
[2] https://lists.proxmox.com/pipermail/pve-devel/2023-October/059299.html
[3] https://lists.proxmox.com/pipermail/pve-devel/2023-November/059818.html
[4] https://lists.proxmox.com/pipermail/pve-devel/2023-November/059843.html
[5] https://lists.proxmox.com/pipermail/pve-devel/2023-November/059872.html



debcargo-conf:

Lukas Wagner (2):
  cherry-pick chumsky 0.9.2 from debian unstable
  update lettre to 0.11.1

 src/chumsky/debian/changelog                  |  5 ++
 src/chumsky/debian/copyright                  | 39 +++++++++++
 src/chumsky/debian/copyright.debcargo.hint    | 51 ++++++++++++++
 src/chumsky/debian/debcargo.toml              |  2 +
 src/lettre/debian/changelog                   | 10 +++
 .../debian/patches/downgrade_fastrand.patch   | 13 ++++
 .../debian/patches/downgrade_idna.patch       | 13 ++++
 src/lettre/debian/patches/downgrade_url.patch | 13 ++++
 .../patches/remove_unused_features.patch      | 69 ++++++++++---------
 src/lettre/debian/patches/series              |  4 +-
 .../patches/upgrade_quoted_printable.patch    | 13 ----
 11 files changed, 185 insertions(+), 47 deletions(-)
 create mode 100644 src/chumsky/debian/changelog
 create mode 100644 src/chumsky/debian/copyright
 create mode 100644 src/chumsky/debian/copyright.debcargo.hint
 create mode 100644 src/chumsky/debian/debcargo.toml
 create mode 100644 src/lettre/debian/patches/downgrade_fastrand.patch
 create mode 100644 src/lettre/debian/patches/downgrade_idna.patch
 create mode 100644 src/lettre/debian/patches/downgrade_url.patch
 delete mode 100644 src/lettre/debian/patches/upgrade_quoted_printable.patch


proxmox:

Lukas Wagner (4):
  sys: email: add `forward`
  notify: add mechanisms for email message forwarding
  notify: add 'smtp' endpoint
  notify: add api for smtp endpoints

 Cargo.toml                                  |   2 +
 proxmox-notify/Cargo.toml                   |   6 +-
 proxmox-notify/src/api/mod.rs               |  33 ++
 proxmox-notify/src/api/smtp.rs              | 356 ++++++++++++++++++++
 proxmox-notify/src/config.rs                |  23 ++
 proxmox-notify/src/endpoints/common/mail.rs |  24 ++
 proxmox-notify/src/endpoints/common/mod.rs  |   2 +
 proxmox-notify/src/endpoints/gotify.rs      |   3 +
 proxmox-notify/src/endpoints/mod.rs         |   4 +
 proxmox-notify/src/endpoints/sendmail.rs    |  27 +-
 proxmox-notify/src/endpoints/smtp.rs        | 250 ++++++++++++++
 proxmox-notify/src/lib.rs                   |  57 ++++
 proxmox-sys/src/email.rs                    |  52 ++-
 13 files changed, 820 insertions(+), 19 deletions(-)
 create mode 100644 proxmox-notify/src/api/smtp.rs
 create mode 100644 proxmox-notify/src/endpoints/common/mail.rs
 create mode 100644 proxmox-notify/src/endpoints/common/mod.rs
 create mode 100644 proxmox-notify/src/endpoints/smtp.rs


proxmox-perl-rs:

Lukas Wagner (1):
  notify: add bindings for smtp API calls

 common/src/notify.rs | 106 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 106 insertions(+)


pve-manager:

Lukas Wagner (1):
  notify: add API routes for smtp endpoints

 PVE/API2/Cluster/Notifications.pm | 323 ++++++++++++++++++++++++++++++
 1 file changed, 323 insertions(+)


proxmox-widget-toolkit:

Lukas Wagner (1):
  panel: notification: add gui for SMTP endpoints

 src/Makefile                     |   2 +
 src/Schema.js                    |   5 +
 src/panel/EmailRecipientPanel.js |  88 +++++++++++++++
 src/panel/SendmailEditPanel.js   |  58 +---------
 src/panel/SmtpEditPanel.js       | 183 +++++++++++++++++++++++++++++++
 5 files changed, 281 insertions(+), 55 deletions(-)
 create mode 100644 src/panel/EmailRecipientPanel.js
 create mode 100644 src/panel/SmtpEditPanel.js


pve-docs:

Lukas Wagner (2):
  notifications: document SMTP endpoints
  notifications: document 'comment' option for targets/matchers

 notifications.adoc | 50 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)


Summary over all repositories:
  32 files changed, 1765 insertions(+), 121 deletions(-)

-- 
murpp v0.4.0