From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <l.wagner@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by lists.proxmox.com (Postfix) with ESMTPS id 129C0A0456
 for <pve-devel@lists.proxmox.com>; Wed,  8 Nov 2023 16:40:10 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
 by firstgate.proxmox.com (Proxmox) with ESMTP id D5197A332
 for <pve-devel@lists.proxmox.com>; Wed,  8 Nov 2023 16:40:09 +0100 (CET)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
 [94.136.29.106])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by firstgate.proxmox.com (Proxmox) with ESMTPS
 for <pve-devel@lists.proxmox.com>; Wed,  8 Nov 2023 16:40:08 +0100 (CET)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
 by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 4B0AF47493
 for <pve-devel@lists.proxmox.com>; Wed,  8 Nov 2023 16:40:08 +0100 (CET)
From: Lukas Wagner <l.wagner@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Wed,  8 Nov 2023 16:39:54 +0100
Message-Id: <20231108154005.895814-1-l.wagner@proxmox.com>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results:  0
 AWL -0.016 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DMARC_MISSING             0.1 Missing DMARC policy
 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
 SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
 SPF_PASS               -0.001 SPF: sender matches SPF record
 T_SCC_BODY_TEXT_LINE    -0.01 -
Subject: [pve-devel] [PATCH v4 many 00/11] notifications: add SMTP endpoint
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Wed, 08 Nov 2023 15:40:10 -0000

This patch series adds support for a new notification endpoint type,
smtp. As the name suggests, this new endpoint allows PVE to talk
to SMTP server directly, without using the system's MTA (postfix).

On the Rust side, these patches add a new dependency to the `lettre`
crate for SMTP communication. This crate was chosen as it is:
  - by far the most popular mailing crate for Rust
  - well maintained
  - has reasonable dependencies
  - has async support, enabling us to asyncify the proxmox-notify
    crate at some point, if needed

Tested against:
  - the gmail SMTP server
  - the posteo SMTP server
  - our own webmail SMTP server

These patches require a couple of other patches that have not been applied yet:

  series: "overhaul notification system, use matchers instead of filters" [3], 
  pve-manager: "api: notifications: give targets and matchers their own ACL namespace" [4]
  pve-docs: "notifications: update docs to for matcher-based notifications" [5]

The first two patches were cherry-picked and rebased from the  'system mail 
forwarding' patch series from [2].  I decided to pull them in so that I can
already implement the mail forwarding part for SMTP targets.

This series also required updating the 'lettre' crate since
one of lettre's deps was bumped to a new version by us.

Changes since v3:
  - Rebased on top of the matcher-based notification revamp
  - Removed 'filter' setting from target configuration
  - Pulled in required patches from 'system mail forwarding' patch series

Changes since v2:
  - Rebased proxmox-widget-toolkit onto the latest master to avoid
    any conflicts.

Changes since v1:
  - Rebased on top of [1]
  - Added a mechanism for mails forwarded by `proxmox-mail-forward`
    These are forwarded inline as "message/rfc822" to avoid having 
    to rewrite mail headers (otherwise, some SMTP relays might reject the 
    mail, because the `From` header of the forwarded mail does not match the
    mail account)

[1] https://lists.proxmox.com/pipermail/pve-devel/2023-August/058956.html
[2] https://lists.proxmox.com/pipermail/pve-devel/2023-October/059299.html
[3] https://lists.proxmox.com/pipermail/pve-devel/2023-November/059818.html
[4] https://lists.proxmox.com/pipermail/pve-devel/2023-November/059843.html
[5] https://lists.proxmox.com/pipermail/pve-devel/2023-November/059872.html



debcargo-conf:

Lukas Wagner (2):
  cherry-pick chumsky 0.9.2 from debian unstable
  update lettre to 0.11.1

 src/chumsky/debian/changelog                  |  5 ++
 src/chumsky/debian/copyright                  | 39 +++++++++++
 src/chumsky/debian/copyright.debcargo.hint    | 51 ++++++++++++++
 src/chumsky/debian/debcargo.toml              |  2 +
 src/lettre/debian/changelog                   | 10 +++
 .../debian/patches/downgrade_fastrand.patch   | 13 ++++
 .../debian/patches/downgrade_idna.patch       | 13 ++++
 src/lettre/debian/patches/downgrade_url.patch | 13 ++++
 .../patches/remove_unused_features.patch      | 69 ++++++++++---------
 src/lettre/debian/patches/series              |  4 +-
 .../patches/upgrade_quoted_printable.patch    | 13 ----
 11 files changed, 185 insertions(+), 47 deletions(-)
 create mode 100644 src/chumsky/debian/changelog
 create mode 100644 src/chumsky/debian/copyright
 create mode 100644 src/chumsky/debian/copyright.debcargo.hint
 create mode 100644 src/chumsky/debian/debcargo.toml
 create mode 100644 src/lettre/debian/patches/downgrade_fastrand.patch
 create mode 100644 src/lettre/debian/patches/downgrade_idna.patch
 create mode 100644 src/lettre/debian/patches/downgrade_url.patch
 delete mode 100644 src/lettre/debian/patches/upgrade_quoted_printable.patch


proxmox:

Lukas Wagner (4):
  sys: email: add `forward`
  notify: add mechanisms for email message forwarding
  notify: add 'smtp' endpoint
  notify: add api for smtp endpoints

 Cargo.toml                                  |   2 +
 proxmox-notify/Cargo.toml                   |   6 +-
 proxmox-notify/src/api/mod.rs               |  33 ++
 proxmox-notify/src/api/smtp.rs              | 356 ++++++++++++++++++++
 proxmox-notify/src/config.rs                |  23 ++
 proxmox-notify/src/endpoints/common/mail.rs |  24 ++
 proxmox-notify/src/endpoints/common/mod.rs  |   2 +
 proxmox-notify/src/endpoints/gotify.rs      |   3 +
 proxmox-notify/src/endpoints/mod.rs         |   4 +
 proxmox-notify/src/endpoints/sendmail.rs    |  27 +-
 proxmox-notify/src/endpoints/smtp.rs        | 250 ++++++++++++++
 proxmox-notify/src/lib.rs                   |  57 ++++
 proxmox-sys/src/email.rs                    |  52 ++-
 13 files changed, 820 insertions(+), 19 deletions(-)
 create mode 100644 proxmox-notify/src/api/smtp.rs
 create mode 100644 proxmox-notify/src/endpoints/common/mail.rs
 create mode 100644 proxmox-notify/src/endpoints/common/mod.rs
 create mode 100644 proxmox-notify/src/endpoints/smtp.rs


proxmox-perl-rs:

Lukas Wagner (1):
  notify: add bindings for smtp API calls

 common/src/notify.rs | 106 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 106 insertions(+)


pve-manager:

Lukas Wagner (1):
  notify: add API routes for smtp endpoints

 PVE/API2/Cluster/Notifications.pm | 323 ++++++++++++++++++++++++++++++
 1 file changed, 323 insertions(+)


proxmox-widget-toolkit:

Lukas Wagner (1):
  panel: notification: add gui for SMTP endpoints

 src/Makefile                     |   2 +
 src/Schema.js                    |   5 +
 src/panel/EmailRecipientPanel.js |  88 +++++++++++++++
 src/panel/SendmailEditPanel.js   |  58 +---------
 src/panel/SmtpEditPanel.js       | 183 +++++++++++++++++++++++++++++++
 5 files changed, 281 insertions(+), 55 deletions(-)
 create mode 100644 src/panel/EmailRecipientPanel.js
 create mode 100644 src/panel/SmtpEditPanel.js


pve-docs:

Lukas Wagner (2):
  notifications: document SMTP endpoints
  notifications: document 'comment' option for targets/matchers

 notifications.adoc | 50 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)


Summary over all repositories:
  32 files changed, 1765 insertions(+), 121 deletions(-)

-- 
murpp v0.4.0