all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Lukas Wagner <l.wagner@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH proxmox 33/55] notify: serialize raw, forwarded mail content as base64
Date: Fri, 13 Nov 2026 16:08:06 +0100	[thread overview]
Message-ID: <20261113150828.756836-34-l.wagner@proxmox.com> (raw)
In-Reply-To: <20261113150828.756836-1-l.wagner@proxmox.com>

base64 is much more space-efficient than a JSON number array for
storing binary data. In an array of numbers, each byte consumes 2-4
bytes of space (1-3 digits plus a comma), while base64 has a constant
1.33x overhead.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
 proxmox-notify/Cargo.toml          |  4 +---
 proxmox-notify/src/notification.rs | 37 ++++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+), 3 deletions(-)

diff --git a/proxmox-notify/Cargo.toml b/proxmox-notify/Cargo.toml
index 1971f5b60..e47912d11 100644
--- a/proxmox-notify/Cargo.toml
+++ b/proxmox-notify/Cargo.toml
@@ -26,7 +26,7 @@ serde_regex = { workspace = true, optional = true }
 thiserror.workspace = true
 
 proxmox-api-macro = { workspace = true, optional = true }
-proxmox-base64 = { workspace = true, optional = true }
+proxmox-base64 = { workspace = true, features = ["serde"]}
 proxmox-config-digest = { workspace = true, features = ["openssl"], optional = true }
 proxmox-http = { workspace = true, optional = true }
 proxmox-http-error = { workspace = true, optional = true }
@@ -55,7 +55,6 @@ api-impl = [
 ]
 api-types = [
     "dep:proxmox-api-macro",
-    "dep:proxmox-base64",
     "dep:proxmox-match-expression",
     "dep:regex",
     "dep:serde_regex",
@@ -68,7 +67,6 @@ send-impl = [
     "dep:proxmox-config-digest",
     "dep:mail-parser",
     "dep:percent-encoding",
-    "dep:proxmox-base64",
     "dep:proxmox-http",
     "proxmox-http/client-sync",
     "dep:proxmox-human-byte",
diff --git a/proxmox-notify/src/notification.rs b/proxmox-notify/src/notification.rs
index 45ab08b87..06b6e67e7 100644
--- a/proxmox-notify/src/notification.rs
+++ b/proxmox-notify/src/notification.rs
@@ -65,6 +65,10 @@ pub enum Content {
     #[serde(rename_all = "kebab-case")]
     ForwardedMail {
         /// Raw mail contents
+        #[serde(
+            serialize_with = "proxmox_base64::serialize_as_base64",
+            deserialize_with = "proxmox_base64::deserialize_from_base64"
+        )]
         raw: Vec<u8>,
     },
 }
@@ -248,6 +252,39 @@ mod tests {
         );
     }
 
+    #[test]
+    fn forwarded_mail_base64_roundtrip() {
+        // Not valid UTF-8 and not a multiple of three bytes, so the encoding needs padding.
+        let content = Content::ForwardedMail {
+            raw: b"\xff\x00mai".to_vec(),
+        };
+
+        let serialized = serde_json::to_value(&content).unwrap();
+        assert_eq!(serialized, json!({"forwarded-mail": {"raw": "/wBtYWk="}}));
+
+        let deserialized: Content = serde_json::from_value(serialized).unwrap();
+        assert_eq!(deserialized, content);
+
+        let content = Content::ForwardedMail {
+            raw: TEST_MAIL.to_vec(),
+        };
+
+        let serialized = serde_json::to_value(&content).unwrap();
+        assert_eq!(
+            serialized["forwarded-mail"]["raw"],
+            proxmox_base64::encode(TEST_MAIL)
+        );
+
+        let deserialized: Content = serde_json::from_value(serialized).unwrap();
+        assert_eq!(deserialized, content);
+    }
+
+    #[test]
+    fn forwarded_mail_base64_invalid() {
+        let result = serde_json::from_value::<Content>(json!({"forwarded-mail": {"raw": "$$$"}}));
+        assert!(result.is_err());
+    }
+
     #[test]
     fn serde_roundtrip() {
         let fields = HashMap::from([("hostname".to_string(), "pve".to_string())]);
-- 
2.47.3





  parent reply	other threads:[~2026-11-13 15:13 UTC|newest]

Thread overview: 58+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-13 15:07 [PATCH many 00/55] notify: maintenance, PDM preparations, queue + worker Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 01/55] notify: disallow `unsafe` code Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 02/55] notify: refactor error type Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 03/55] notify: move `Endpoint` trait to endpoint module as pub(crate) Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 04/55] notify: use ConfigDigest instead of raw byte arrays Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 05/55] notify: remove static context and pass context as parameter Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 06/55] notify: matcher: drop unused field matcher verifiers Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 07/55] notify: introduce 'send-impl', 'api-impl' and 'api-types' features Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 08/55] notify: matcher: make matcher validation methods public Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 09/55] notify: make SendmailEndpoint and SmtpEndpoint private types Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 10/55] notify: move `send`, `test_target` and `Bus` to new `send` module Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 11/55] notify: move Origin type to api_types module Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 12/55] notify: move Target " Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 13/55] notify: move Config to config module Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 14/55] notify: move Notification and sub-types to `notification` module Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 15/55] notify: move schema module to api_types Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 16/55] notify: move check_matches from api_types::matcher to send::Bus Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 17/55] notify: move TemplateSource to context module Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 18/55] notify: config: cleanup helpers names and visibility Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 19/55] notify: config: rename section config parser/writer methods Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 20/55] notify: config: pass builtins explicitly when parsing section config Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 21/55] notify: implement PartialEq for all Config types Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 22/55] notify: use typed SectionConfigData Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 23/55] notify: api: api types: rename Deleteable* -> Deletable* Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 24/55] notify: context: parse builtin config only once Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 25/55] notify: context: parse proxy config in trait impl, not at the callsite Lukas Wagner
2026-11-13 15:07 ` [PATCH proxmox 26/55] notify: gate sub-modules with the appropriate feature flags Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 27/55] notify: fold mail-forwarder feature into send-impl Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 28/55] notify: forwarded mails: don't hardcode severity and metadata fields Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 29/55] notify: don't pass uid when instantiating forwarded mail notifications Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 30/55] notify: derive PartialEq for Notification type Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 31/55] notify: add getters for severity and fields on " Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 32/55] notify: add test suite for " Lukas Wagner
2026-11-13 15:08 ` Lukas Wagner [this message]
2026-11-13 15:08 ` [PATCH proxmox 34/55] notify: unify code paths for legacy-sendmail and regular notifications Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 35/55] notify: add backwards-compat note to Notification type Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 36/55] notify: remove 'test_target' API Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 37/55] notify: replace notification instantiation with builder pattern Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 38/55] notify: add send receipts Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 39/55] notify: implement notification queue submitter Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 40/55] notify: context: add read_config Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 41/55] notify: queue: add queue consumer Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox 42/55] notify: queue: add worker task Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-backup 43/55] notifications: don't ignore verification errors in legacy-sendmail mode Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-backup 44/55] notifications: adapt to proxmox-notify API changes Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-backup 45/55] api: notification: send test notification via proxmox_notify::send Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-backup 46/55] notifications: send all notifications via proxmox-notify's queue Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-perl-rs 47/55] pve: notify: adapt to changes in the public API of proxmox-notify Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-perl-rs 48/55] pve: notify: add `PVE::RS::Notify::enqueue` method Lukas Wagner
2026-11-13 15:08 ` [PATCH pve-cluster 49/55] notify: send notifications via notification queue Lukas Wagner
2026-11-13 15:08 ` [PATCH manager 50/55] vzdump: send legacy-sendmail notifications via simplified API Lukas Wagner
2026-11-13 15:08 ` [PATCH manager 51/55] pve-notify-worker: add new rust-based notification worker daemon Lukas Wagner
2026-11-13 15:08 ` [PATCH manager 52/55] debian: add 'pve-notify-worker' as a dependency Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-mail-forward 53/55] forward mails via notification queue Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-mail-forward 54/55] use return code to signal permanent or temporary failure Lukas Wagner
2026-11-13 15:08 ` [PATCH proxmox-mail-forward 55/55] limit size of mail read from stdin Lukas Wagner
2026-11-17  9:35 ` [PATCH many 00/55] notify: maintenance, PDM preparations, queue + worker Lukas Wagner
2026-11-18  9:55 ` superseded: " Lukas Wagner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261113150828.756836-34-l.wagner@proxmox.com \
    --to=l.wagner@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal