From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id ED7BDC0074 for ; Wed, 10 Jan 2024 09:28:56 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id CDE993081F for ; Wed, 10 Jan 2024 09:28:26 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 10 Jan 2024 09:28:25 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 076534902C for ; Wed, 10 Jan 2024 09:28:25 +0100 (CET) Message-ID: Date: Wed, 10 Jan 2024 09:28:23 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: Wolfgang Bumiller Cc: pve-devel@lists.proxmox.com References: <20231213164201.395505-1-l.wagner@proxmox.com> Content-Language: de-AT, en-US From: Lukas Wagner In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.004 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com, smtp.rs] Subject: Re: [pve-devel] [PATCH proxmox 1/2] notify: smtp: forward original message instead nesting X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jan 2024 08:28:57 -0000 Thanks for the review! On 1/9/24 14:26, Wolfgang Bumiller wrote: > On Wed, Dec 13, 2023 at 05:42:00PM +0100, Lukas Wagner wrote: >> For mails forwarded by `proxmox-mail-forward` to an SMTP target, the >> original message was nested as a 'message/rfc822' message part. >> Originally this approach was chosen to avoid having to rewrite >> message headers. >> Good email-clients, such as Thunderbird can display these inline. >> Other, more limited clients will show these messages as an attached >> .eml file, which is not really a good user experience. >> >> This patch changes the approach for message forwarding to be more like >> forwarding mails in a mail client. We create a new message and >> add the original message body as a body. Additionally, we also copy >> over all message headers that are relevant to correctly display the >> original message body (e.g. Content-Type, Content-Transfer-Encoding) >> >> Tested with a couple of different email messages (varying in >> structure, body parts, encoding, etc.) against the following SMTP >> relays: >> - gmail >> - outlook >> - our own webmail service >> >> Originally reported in our community forum: >> https://forum.proxmox.com/threads/proxmox-mail-forward-sends-mails-as-eml.137710/ >> >> Signed-off-by: Lukas Wagner >> --- >> >> Notes: >> proxmox-mail-forward needs a bump once this is applied. >> >> proxmox-notify/src/endpoints/smtp.rs | 99 ++++++++++++++++++++++++---- >> 1 file changed, 88 insertions(+), 11 deletions(-) >> >> diff --git a/proxmox-notify/src/endpoints/smtp.rs b/proxmox-notify/src/endpoints/smtp.rs >> index 064c9f9..4b8ff2d 100644 >> --- a/proxmox-notify/src/endpoints/smtp.rs >> +++ b/proxmox-notify/src/endpoints/smtp.rs >> @@ -1,8 +1,9 @@ >> +use std::time::Duration; >> + >> use lettre::message::{Mailbox, MultiPart, SinglePart}; >> use lettre::transport::smtp::client::{Tls, TlsParameters}; >> use lettre::{message::header::ContentType, Message, SmtpTransport, Transport}; >> use serde::{Deserialize, Serialize}; >> -use std::time::Duration; >> >> use proxmox_schema::api_types::COMMENT_SCHEMA; >> use proxmox_schema::{api, Updater}; >> @@ -231,17 +232,93 @@ impl Endpoint for SmtpEndpoint { >> } >> #[cfg(feature = "mail-forwarder")] >> Content::ForwardedMail { ref raw, title, .. } => { >> - email_builder = email_builder.subject(title); >> + use lettre::message::header::{ContentTransferEncoding, HeaderName, HeaderValue}; >> + use lettre::message::Body; >> >> - // Forwarded messages are embedded inline as 'message/rfc822' >> - // this let's us avoid rewriting any headers (e.g. From) >> - email_builder >> - .singlepart( >> - SinglePart::builder() >> - .header(ContentType::parse("message/rfc822").unwrap()) >> - .body(raw.to_owned()), >> - ) >> - .map_err(|err| Error::NotifyFailed(self.name().into(), Box::new(err)))? >> + let parsed_message = mail_parser::Message::parse(raw) >> + .ok_or_else(|| Error::Generic("could not parse forwarded email".to_string()))?; >> + >> + let root_part = parsed_message >> + .part(0) >> + .ok_or_else(|| Error::Generic("root message part not present".to_string()))?; >> + >> + let raw_body = parsed_message >> + .raw_message() >> + .get(root_part.offset_body..root_part.offset_end) >> + .ok_or_else(|| Error::Generic("could not get raw body content".to_string()))?; >> + >> + // We assume that the original message content is already properly >> + // encoded, thus we add the original message body 'Binary' encoding. >> + // This prohibits lettre from trying to re-encode our raw body data. >> + // lettre will automatically set the `Content-Transfer-Encoding: binary` header, >> + // which we need to remove. The actual transfer encoding is later >> + // copied from the original message headers. >> + let body = >> + Body::new_with_encoding(raw_body.to_vec(), ContentTransferEncoding::Binary) >> + .map_err(|_| Error::Generic("could not create body".into()))?; >> + let mut message = email_builder >> + .subject(title) >> + .body(body) >> + .map_err(|err| Error::NotifyFailed(self.name().into(), Box::new(err)))?; >> + message >> + .headers_mut() >> + .remove_raw("Content-Transfer-Encoding"); >> + >> + // Copy over all headers that are relevant to display the original body correctly. >> + // Unfortunately this is a bit cumbersome, as we use separate crates for mail parsing (mail-parser) >> + // and creating/sending mails (lettre). >> + // Note: Other MIME-Headers, such as Content-{ID,Description,Disposition} are only used >> + // for body-parts in multipart messages, so we can ignore them for the messages headers. >> + // Since we send the original raw body, the part-headers will be included any way. >> + for header in parsed_message.headers() { >> + let header_name = header.name.as_str(); >> + // Email headers are case-insensitive, so convert to lowercase... >> + let value = match header_name.to_lowercase().as_str() { >> + "content-type" => { >> + if let mail_parser::HeaderValue::ContentType(ct) = header.value() { >> + // mail_parser does not give us access to the full decoded and unfolded >> + // header value, so we unfortunately need to reassemble it ourselves. >> + // Meh. > > urgh > urgh indeed... >> + let mut value = ct.ctype().to_string(); >> + if let Some(subtype) = ct.subtype() { >> + value.push('/'); >> + value.push_str(subtype); >> + } >> + if let Some(attributes) = ct.attributes() { >> + for attribute in attributes { >> + value.push_str(&format!( >> + "; {}=\"{}\"", >> + attribute.0, attribute.1 >> + )); > > should be more efficient to use > let _ = write!(value, "..."); > instead of `push_str()` on a temporarily allocated (format!()) string. Right, thanks! Will send a v2. > >> + } >> + } >> + Some(value) >> + } else { >> + None >> + } >> + } >> + "content-transfer-encoding" | "mime-version" => { >> + if let mail_parser::HeaderValue::Text(text) = header.value() { >> + Some(text.to_string()) >> + } else { >> + None >> + } >> + } >> + _ => None, >> + }; >> + >> + if let Some(value) = value { >> + match HeaderName::new_from_ascii(header_name.into()) { >> + Ok(name) => { >> + let header = HeaderValue::new(name, value); >> + message.headers_mut().insert_raw(header); >> + } >> + Err(e) => log::error!("could not set header: {e}"), >> + } >> + } >> + } >> + >> + message >> } >> }; >> >> -- >> 2.39.2 -- - Lukas