public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Lukas Wagner <l.wagner@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH proxmox 33/55] notify: serialize raw, forwarded mail content as base64
Date: Fri, 13 Nov 2026 16:08:06 +0100	[thread overview]
Message-ID: <20261113150828.756836-34-l.wagner@proxmox.com> (raw)
In-Reply-To: <20261113150828.756836-1-l.wagner@proxmox.com>

base64 is much more space-efficient than a JSON number array for
storing binary data. In an array of numbers, each byte consumes 2-4
bytes of space (1-3 digits plus a comma), while base64 has a constant
1.33x overhead.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
 proxmox-notify/Cargo.toml          |  4 +---
 proxmox-notify/src/notification.rs | 37 ++++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+), 3 deletions(-)

diff --git a/proxmox-notify/Cargo.toml b/proxmox-notify/Cargo.toml
index 1971f5b60..e47912d11 100644
--- a/proxmox-notify/Cargo.toml
+++ b/proxmox-notify/Cargo.toml
@@ -26,7 +26,7 @@ serde_regex = { workspace = true, optional = true }
 thiserror.workspace = true
 
 proxmox-api-macro = { workspace = true, optional = true }
-proxmox-base64 = { workspace = true, optional = true }
+proxmox-base64 = { workspace = true, features = ["serde"]}
 proxmox-config-digest = { workspace = true, features = ["openssl"], optional = true }
 proxmox-http = { workspace = true, optional = true }
 proxmox-http-error = { workspace = true, optional = true }
@@ -55,7 +55,6 @@ api-impl = [
 ]
 api-types = [
     "dep:proxmox-api-macro",
-    "dep:proxmox-base64",
     "dep:proxmox-match-expression",
     "dep:regex",
     "dep:serde_regex",
@@ -68,7 +67,6 @@ send-impl = [
     "dep:proxmox-config-digest",
     "dep:mail-parser",
     "dep:percent-encoding",
-    "dep:proxmox-base64",
     "dep:proxmox-http",
     "proxmox-http/client-sync",
     "dep:proxmox-human-byte",
diff --git a/proxmox-notify/src/notification.rs b/proxmox-notify/src/notification.rs
index 45ab08b87..06b6e67e7 100644
--- a/proxmox-notify/src/notification.rs
+++ b/proxmox-notify/src/notification.rs
@@ -65,6 +65,10 @@ pub enum Content {
     #[serde(rename_all = "kebab-case")]
     ForwardedMail {
         /// Raw mail contents
+        #[serde(
+            serialize_with = "proxmox_base64::serialize_as_base64",
+            deserialize_with = "proxmox_base64::deserialize_from_base64"
+        )]
         raw: Vec<u8>,
     },
 }
@@ -248,6 +252,39 @@ mod tests {
         );
     }
 
+    #[test]
+    fn forwarded_mail_base64_roundtrip() {
+        // Not valid UTF-8 and not a multiple of three bytes, so the encoding needs padding.
+        let content = Content::ForwardedMail {
+            raw: b"\xff\x00mai".to_vec(),
+        };
+
+        let serialized = serde_json::to_value(&content).unwrap();
+        assert_eq!(serialized, json!({"forwarded-mail": {"raw": "/wBtYWk="}}));
+
+        let deserialized: Content = serde_json::from_value(serialized).unwrap();
+        assert_eq!(deserialized, content);
+
+        let content = Content::ForwardedMail {
+            raw: TEST_MAIL.to_vec(),
+        };
+
+        let serialized = serde_json::to_value(&content).unwrap();
+        assert_eq!(
+            serialized["forwarded-mail"]["raw"],
+            proxmox_base64::encode(TEST_MAIL)
+        );
+
+        let deserialized: Content = serde_json::from_value(serialized).unwrap();
+        assert_eq!(deserialized, content);
+    }
+
+    #[test]
+    fn forwarded_mail_base64_invalid() {
+        let result = serde_json::from_value::<Content>(json!({"forwarded-mail": {"raw": "$$$"}}));
+        assert!(result.is_err());
+    }
+
     #[test]
     fn serde_roundtrip() {
         let fields = HashMap::from([("hostname".to_string(), "pve".to_string())]);
-- 
2.47.3





  parent reply	other threads:[~2026-11-13 15:13 UTC|newest]

Thread overview: 58+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-13 15:07 [PATCH many 00/55] notify: maintenance, PDM preparations, queue + worker Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 01/55] notify: disallow `unsafe` code Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 02/55] notify: refactor error type Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 03/55] notify: move `Endpoint` trait to endpoint module as pub(crate) Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 04/55] notify: use ConfigDigest instead of raw byte arrays Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 05/55] notify: remove static context and pass context as parameter Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 06/55] notify: matcher: drop unused field matcher verifiers Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 07/55] notify: introduce 'send-impl', 'api-impl' and 'api-types' features Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 08/55] notify: matcher: make matcher validation methods public Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 09/55] notify: make SendmailEndpoint and SmtpEndpoint private types Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 10/55] notify: move `send`, `test_target` and `Bus` to new `send` module Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 11/55] notify: move Origin type to api_types module Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 12/55] notify: move Target " Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 13/55] notify: move Config to config module Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 14/55] notify: move Notification and sub-types to `notification` module Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 15/55] notify: move schema module to api_types Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 16/55] notify: move check_matches from api_types::matcher to send::Bus Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 17/55] notify: move TemplateSource to context module Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 18/55] notify: config: cleanup helpers names and visibility Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 19/55] notify: config: rename section config parser/writer methods Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 20/55] notify: config: pass builtins explicitly when parsing section config Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 21/55] notify: implement PartialEq for all Config types Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 22/55] notify: use typed SectionConfigData Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 23/55] notify: api: api types: rename Deleteable* -> Deletable* Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 24/55] notify: context: parse builtin config only once Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 25/55] notify: context: parse proxy config in trait impl, not at the callsite Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 26/55] notify: gate sub-modules with the appropriate feature flags Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 27/55] notify: fold mail-forwarder feature into send-impl Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 28/55] notify: forwarded mails: don't hardcode severity and metadata fields Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 29/55] notify: don't pass uid when instantiating forwarded mail notifications Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 30/55] notify: derive PartialEq for Notification type Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 31/55] notify: add getters for severity and fields on " Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 32/55] notify: add test suite for " Lukas Wagner
2026-11-13 15:08 ` Lukas Wagner [this message]
2026-11-13 15:08 ` [PATCH proxmox 34/55] notify: unify code paths for legacy-sendmail and regular notifications Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 35/55] notify: add backwards-compat note to Notification type Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 36/55] notify: remove 'test_target' API Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 37/55] notify: replace notification instantiation with builder pattern Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 38/55] notify: add send receipts Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 39/55] notify: implement notification queue submitter Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 40/55] notify: context: add read_config Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 41/55] notify: queue: add queue consumer Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 42/55] notify: queue: add worker task Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-backup 43/55] notifications: don't ignore verification errors in legacy-sendmail mode Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-backup 44/55] notifications: adapt to proxmox-notify API changes Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-backup 45/55] api: notification: send test notification via proxmox_notify::send Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-backup 46/55] notifications: send all notifications via proxmox-notify's queue Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-perl-rs 47/55] pve: notify: adapt to changes in the public API of proxmox-notify Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-perl-rs 48/55] pve: notify: add `PVE::RS::Notify::enqueue` method Lukas Wagner
2026-11-13 15:08 ` [PATCH pve-cluster 49/55] notify: send notifications via notification queue Lukas Wagner
2026-11-13 15:08 ` [PATCH manager 50/55] vzdump: send legacy-sendmail notifications via simplified API Lukas Wagner
2026-11-13 15:08 ` [PATCH manager 51/55] pve-notify-worker: add new rust-based notification worker daemon Lukas Wagner
2026-11-13 15:08 ` [PATCH manager 52/55] debian: add 'pve-notify-worker' as a dependency Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-mail-forward 53/55] forward mails via notification queue Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-mail-forward 54/55] use return code to signal permanent or temporary failure Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-mail-forward 55/55] limit size of mail read from stdin Lukas Wagner
2026-11-17  9:35 ` [PATCH many 00/55] notify: maintenance, PDM preparations, queue + worker Lukas Wagner
2026-11-18  9:55 ` superseded: " Lukas Wagner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261113150828.756836-34-l.wagner@proxmox.com \
    --to=l.wagner@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal