From: Christian Ebner <c.ebner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox v8 5/9] s3 client: add helper to parse http date headers
Date: Tue, 15 Jul 2025 14:52:43 +0200 [thread overview]
Message-ID: <20250715125332.954494-6-c.ebner@proxmox.com> (raw)
In-Reply-To: <20250715125332.954494-1-c.ebner@proxmox.com>
Add a helper to parse the preferred date/time format for http `Date`
headers as specified in RFC 2616 [0], which is a fixed-length subset
of the format specified in RFC 1123 [1], itself being a followup to
RFC 822 [2]. Does not implement the format as described in the
obsolete RFC 850 [3].
This allows to parse the `Date` and `Last-Modified` headers of S3 API
responses.
[0] https://datatracker.ietf.org/doc/html/rfc2616#section-3.3
[1] https://datatracker.ietf.org/doc/html/rfc1123#section-5.2.14
[2] https://datatracker.ietf.org/doc/html/rfc822#section-5
[3] https://datatracker.ietf.org/doc/html/rfc850
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
changes since version 7:
- no changes
proxmox-s3-client/src/timestamps.rs | 90 ++++++++++++++++++++++++++++-
1 file changed, 89 insertions(+), 1 deletion(-)
diff --git a/proxmox-s3-client/src/timestamps.rs b/proxmox-s3-client/src/timestamps.rs
index 3713b6d0..22330966 100644
--- a/proxmox-s3-client/src/timestamps.rs
+++ b/proxmox-s3-client/src/timestamps.rs
@@ -1,4 +1,9 @@
-use anyhow::{anyhow, Error};
+use anyhow::{anyhow, bail, Context, Error};
+
+const VALID_DAYS_OF_WEEK: [&str; 7] = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"];
+const VALID_MONTHS: [&str; 12] = [
+ "Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec",
+];
#[derive(Debug)]
/// Last modified timestamp as obtained from API response http headers.
@@ -16,3 +21,86 @@ impl std::str::FromStr for LastModifiedTimestamp {
}
serde_plain::derive_deserialize_from_fromstr!(LastModifiedTimestamp, "last modified timestamp");
+
+/// Preferred date format specified by RFC2616, given as fixed-length
+/// subset of RFC1123, which itself is a followup to RFC822.
+///
+/// https://datatracker.ietf.org/doc/html/rfc2616#section-3.3
+/// https://datatracker.ietf.org/doc/html/rfc1123#section-5.2.14
+/// https://datatracker.ietf.org/doc/html/rfc822#section-5
+#[derive(Debug)]
+pub struct HttpDate {
+ _epoch: i64,
+}
+
+impl std::str::FromStr for HttpDate {
+ type Err = Error;
+
+ fn from_str(timestamp: &str) -> Result<Self, Self::Err> {
+ let input = timestamp.as_bytes();
+ if input.len() != 29 {
+ bail!("unexpected length: got {}", input.len());
+ }
+
+ let expect = |pos: usize, c: u8| {
+ if input[pos] != c {
+ bail!("unexpected char at pos {pos}");
+ }
+ Ok(())
+ };
+
+ let digit = |pos: usize| -> Result<i32, Error> {
+ let digit = input[pos] as i32;
+ if !(48..=57).contains(&digit) {
+ bail!("unexpected char at pos {pos}");
+ }
+ Ok(digit - 48)
+ };
+
+ fn check_max(i: i32, max: i32) -> Result<i32, Error> {
+ if i > max {
+ bail!("value too large ({i} > {max})");
+ }
+ Ok(i)
+ }
+
+ let mut tm = proxmox_time::TmEditor::new(true);
+
+ if !VALID_DAYS_OF_WEEK
+ .iter()
+ .any(|valid| valid.as_bytes() == &input[0..3])
+ {
+ bail!("unexpected day of week, got {:?}", &input[0..3]);
+ }
+
+ expect(3, b',').context("unexpected separator after day of week")?;
+ expect(4, b' ').context("missing space after day of week separator")?;
+ tm.set_mday(check_max(digit(5)? * 10 + digit(6)?, 31)?)?;
+ expect(7, b' ').context("unexpected separator after day")?;
+ if let Some(month) = VALID_MONTHS
+ .iter()
+ .position(|month| month.as_bytes() == &input[8..11])
+ {
+ // valid conversion to i32, position stems from fixed size array of 12 months.
+ tm.set_mon(check_max(month as i32 + 1, 12)?)?;
+ } else {
+ bail!("invalid month");
+ }
+ expect(11, b' ').context("unexpected separator after month")?;
+ tm.set_year(digit(12)? * 1000 + digit(13)? * 100 + digit(14)? * 10 + digit(15)?)?;
+ expect(16, b' ').context("unexpected separator after year")?;
+ tm.set_hour(check_max(digit(17)? * 10 + digit(18)?, 23)?)?;
+ expect(19, b':').context("unexpected separator after hour")?;
+ tm.set_min(check_max(digit(20)? * 10 + digit(21)?, 59)?)?;
+ expect(22, b':').context("unexpected separator after minute")?;
+ tm.set_sec(check_max(digit(23)? * 10 + digit(24)?, 60)?)?;
+ expect(25, b' ').context("unexpected separator after second")?;
+ if !input.ends_with(b"GMT") {
+ bail!("unexpected timezone");
+ }
+
+ let _epoch = tm.into_epoch()?;
+
+ Ok(Self { _epoch })
+ }
+}
--
2.47.2
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
next prev parent reply other threads:[~2025-07-15 12:52 UTC|newest]
Thread overview: 108+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-07-15 12:52 [pbs-devel] [PATCH proxmox{, -backup} v8 00/54] fix #2943: S3 storage backend for datastores Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 1/9] s3 client: add crate for AWS s3 compatible object store client Christian Ebner
2025-07-15 21:13 ` [pbs-devel] partially-applied-series: " Thomas Lamprecht
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 2/9] s3 client: implement AWS signature v4 request authentication Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 3/9] s3 client: add dedicated type for s3 object keys Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 4/9] s3 client: add type for last modified timestamp in responses Christian Ebner
2025-07-15 12:52 ` Christian Ebner [this message]
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 6/9] s3 client: implement methods to operate on s3 objects in bucket Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 7/9] s3 client: add example usage for basic operations Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 8/9] pbs-api-types: extend datastore config by backend config enum Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 9/9] pbs-api-types: maintenance: add new maintenance mode S3 refresh Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 01/45] datastore: add helpers for path/digest to s3 object key conversion Christian Ebner
2025-07-18 7:24 ` Lukas Wagner
2025-07-18 8:34 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 02/45] config: introduce s3 object store client configuration Christian Ebner
2025-07-18 7:22 ` Lukas Wagner
2025-07-18 8:37 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 03/45] api: config: implement endpoints to manipulate and list s3 configs Christian Ebner
2025-07-18 7:32 ` Lukas Wagner
2025-07-18 8:40 ` Christian Ebner
2025-07-18 9:07 ` Lukas Wagner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 04/45] api: datastore: check s3 backend bucket access on datastore create Christian Ebner
2025-07-18 7:40 ` Lukas Wagner
2025-07-18 8:55 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 05/45] api/cli: add endpoint and command to check s3 client connection Christian Ebner
2025-07-18 7:43 ` Lukas Wagner
2025-07-18 9:04 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 06/45] datastore: allow to get the backend for a datastore Christian Ebner
2025-07-18 7:52 ` Lukas Wagner
2025-07-18 9:10 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 07/45] api: backup: store datastore backend in runtime environment Christian Ebner
2025-07-18 7:54 ` Lukas Wagner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 08/45] api: backup: conditionally upload chunks to s3 object store backend Christian Ebner
2025-07-18 8:11 ` Lukas Wagner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 09/45] api: backup: conditionally upload blobs " Christian Ebner
2025-07-18 8:13 ` Lukas Wagner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 10/45] api: backup: conditionally upload indices " Christian Ebner
2025-07-18 8:20 ` Lukas Wagner
2025-07-18 9:24 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 11/45] api: backup: conditionally upload manifest " Christian Ebner
2025-07-18 8:26 ` Lukas Wagner
2025-07-18 9:33 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 12/45] api: datastore: conditionally upload client log to s3 backend Christian Ebner
2025-07-18 8:28 ` Lukas Wagner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 13/45] sync: pull: conditionally upload content " Christian Ebner
2025-07-18 8:35 ` Lukas Wagner
2025-07-18 9:43 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 14/45] api: reader: fetch chunks based on datastore backend Christian Ebner
2025-07-18 8:38 ` Lukas Wagner
2025-07-18 9:58 ` Christian Ebner
2025-07-18 10:03 ` Lukas Wagner
2025-07-18 10:10 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 15/45] datastore: local chunk reader: read chunks based on backend Christian Ebner
2025-07-18 8:45 ` Lukas Wagner
2025-07-18 10:11 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 16/45] verify worker: add datastore backed to verify worker Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 17/45] verify: implement chunk verification for stores with s3 backend Christian Ebner
2025-07-18 8:56 ` Lukas Wagner
2025-07-18 11:45 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 18/45] datastore: create namespace marker in " Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 19/45] datastore: create/delete protected marker file on s3 storage backend Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 20/45] datastore: prune groups/snapshots from s3 object store backend Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 21/45] datastore: get and set owner for s3 " Christian Ebner
2025-07-18 9:25 ` Lukas Wagner
2025-07-18 12:12 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 22/45] datastore: implement garbage collection for s3 backend Christian Ebner
2025-07-18 9:47 ` Lukas Wagner
2025-07-18 14:31 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 23/45] ui: add datastore type selector and reorganize component layout Christian Ebner
2025-07-18 9:55 ` Lukas Wagner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 24/45] ui: add s3 client edit window for configuration create/edit Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 25/45] ui: add s3 client view for configuration Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 26/45] ui: expose the s3 client view in the navigation tree Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 27/45] ui: add s3 client selector and bucket field for s3 backend setup Christian Ebner
2025-07-18 10:02 ` Lukas Wagner
2025-07-19 12:28 ` Christian Ebner
2025-07-22 9:25 ` Lukas Wagner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 28/45] tools: lru cache: add removed callback for evicted cache nodes Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 29/45] tools: async lru cache: implement insert, remove and contains methods Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 30/45] datastore: add local datastore cache for network attached storages Christian Ebner
2025-07-18 11:24 ` Lukas Wagner
2025-07-18 14:59 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 31/45] api: backup: use local datastore cache on s3 backend chunk upload Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 32/45] api: reader: use local datastore cache on s3 backend chunk fetching Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 33/45] datastore: local chunk reader: get cached chunk from local cache store Christian Ebner
2025-07-18 11:36 ` Lukas Wagner
2025-07-18 15:04 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 34/45] api: backup: add no-cache flag to bypass local datastore cache Christian Ebner
2025-07-18 11:41 ` Lukas Wagner
2025-07-18 15:37 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 35/45] api/datastore: implement refresh endpoint for stores with s3 backend Christian Ebner
2025-07-18 12:01 ` Lukas Wagner
2025-07-18 15:51 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 36/45] cli: add dedicated subcommand for datastore s3 refresh Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 37/45] ui: render s3 refresh as valid maintenance type and task description Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 38/45] ui: expose s3 refresh button for datastores backed by object store Christian Ebner
2025-07-18 12:46 ` Lukas Wagner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 39/45] datastore: conditionally upload atime marker chunk to s3 backend Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 40/45] bin: implement client subcommands for s3 configuration manipulation Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 41/45] bin: expose reuse-datastore flag for proxmox-backup-manager Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 42/45] datastore: mark store as in-use by setting marker on s3 backend Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 43/45] datastore: run s3-refresh when reusing a datastore with " Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 44/45] api/ui: add flag to allow overwriting in-use marker for " Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 45/45] docs: Add section describing how to setup s3 backed datastore Christian Ebner
2025-07-18 13:14 ` Maximiliano Sandoval
2025-07-18 14:38 ` Christian Ebner
2025-07-18 13:16 ` [pbs-devel] [PATCH proxmox{, -backup} v8 00/54] fix #2943: S3 storage backend for datastores Lukas Wagner
2025-07-19 12:52 ` [pbs-devel] superseded: " Christian Ebner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250715125332.954494-6-c.ebner@proxmox.com \
--to=c.ebner@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox