[pbs-devel] [PATCH proxmox 0/6] followups and cleanups for s3 client

[pbs-devel] [PATCH proxmox 0/6] followups and cleanups for s3 client

[Christian Ebner]

These patches are not release critical, but are intended to improve
code quality and add basic testing.

This currently covers formatting issues, refactoring of code to make
it more concise, adding missing doc comments and adding basic
regression tests for parsers reading the http response body.

Christian Ebner (6):
  s3 client: fix formatting issues by `cargo fmt`
  s3 client: refactor list buckets response status code matching
  s3 client: refactor list buckets result list fetching
  s3 client: add doc comments for response parser helper methods
  s3 client: add and expand doc comments for response parsing objects
  s3 client: add basic regression tests for response parsing

 proxmox-s3-client/src/client.rs          |   6 +-
 proxmox-s3-client/src/object_key.rs      |   2 +-
 proxmox-s3-client/src/response_reader.rs | 274 ++++++++++++++++++++---
 proxmox-s3-client/src/timestamps.rs      |   2 +-
 4 files changed, 241 insertions(+), 43 deletions(-)

-- 
2.47.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox 1/6] s3 client: fix formatting issues by `cargo fmt`

[Christian Ebner]

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 proxmox-s3-client/src/client.rs | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/proxmox-s3-client/src/client.rs b/proxmox-s3-client/src/client.rs
index a2c62811..ae61c590 100644
--- a/proxmox-s3-client/src/client.rs
+++ b/proxmox-s3-client/src/client.rs
@@ -155,8 +155,7 @@ impl S3Client {
             options.endpoint.clone()
         };
 
-        let authority = authority_template
-            .replace("{{region}}", &options.region);
+        let authority = authority_template.replace("{{region}}", &options.region);
 
         let authority = if let Some(bucket) = &options.bucket {
             authority.replace("{{bucket}}", bucket)
@@ -475,8 +474,7 @@ impl S3Client {
             Some(bucket) => bucket,
             None => bail!("missing bucket name for copy source"),
         };
-        let copy_source =
-            source_key.to_copy_source_key(bucket, &self.options.common_prefix);
+        let copy_source = source_key.to_copy_source_key(bucket, &self.options.common_prefix);
         let copy_source = aws_sign_v4_uri_encode(&copy_source, true);
         let destination_key = destination_key.to_full_key(&self.options.common_prefix);
         let destination_key = aws_sign_v4_uri_encode(&destination_key, true);
-- 
2.47.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox 2/6] s3 client: refactor list buckets response status code matching

[Christian Ebner]

Makes the match for status code 200 more concise by using the
matches macro.

No functional changes intended.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 proxmox-s3-client/src/response_reader.rs | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/proxmox-s3-client/src/response_reader.rs b/proxmox-s3-client/src/response_reader.rs
index f9d7f1ac..0172805c 100644
--- a/proxmox-s3-client/src/response_reader.rs
+++ b/proxmox-s3-client/src/response_reader.rs
@@ -375,13 +375,10 @@ impl ResponseReader {
         let (parts, body) = self.response.into_parts();
         let body = body.collect().await?.to_bytes();
 
-        match parts.status {
-            StatusCode::OK => (),
-            status_code => {
-                Self::log_error_response_utf8(body);
-                bail!("unexpected status code {status_code}")
-            }
-        };
+        if !matches!(parts.status, StatusCode::OK) {
+            Self::log_error_response_utf8(body);
+            bail!("unexpected status code {}", parts.status);
+        }
 
         let body = String::from_utf8(body.to_vec())?;
 
-- 
2.47.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox 3/6] s3 client: refactor list buckets result list fetching

[Christian Ebner]

Instead of the match statement, use the more concise map and
unwrap_or_default.

No fuctional changes intended.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 proxmox-s3-client/src/response_reader.rs | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/proxmox-s3-client/src/response_reader.rs b/proxmox-s3-client/src/response_reader.rs
index 0172805c..fc9519ad 100644
--- a/proxmox-s3-client/src/response_reader.rs
+++ b/proxmox-s3-client/src/response_reader.rs
@@ -385,10 +385,10 @@ impl ResponseReader {
         let list_buckets_result: ListAllMyBucketsResult =
             serde_xml_rs::from_str(&body).context("failed to parse response body")?;
 
-        let buckets = match list_buckets_result.buckets {
-            Some(buckets) => buckets.bucket,
-            None => Vec::new(),
-        };
+        let buckets = list_buckets_result
+            .buckets
+            .map(|b| b.bucket)
+            .unwrap_or_default();
         Ok(ListBucketsResponse { buckets })
     }
 
-- 
2.47.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox 4/6] s3 client: add doc comments for response parser helper methods

[Christian Ebner]

Document the response reader's helper methods to parse the various
responses.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 proxmox-s3-client/src/response_reader.rs | 32 ++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/proxmox-s3-client/src/response_reader.rs b/proxmox-s3-client/src/response_reader.rs
index fc9519ad..cd3ee98a 100644
--- a/proxmox-s3-client/src/response_reader.rs
+++ b/proxmox-s3-client/src/response_reader.rs
@@ -186,10 +186,15 @@ pub struct Bucket {
 }
 
 impl ResponseReader {
+    /// Create a new response reader to parse given response.
     pub(crate) fn new(response: Response<Incoming>) -> Self {
         Self { response }
     }
 
+    /// Read and parse the list object v2 response.
+    ///
+    /// Returns with error if the bucket cannot be found, an unexpected status code is encountered
+    /// or the response body cannot be parsed.
     pub(crate) async fn list_objects_v2_response(self) -> Result<ListObjectsV2Response, Error> {
         let (parts, body) = self.response.into_parts();
         let body = body.collect().await?.to_bytes();
@@ -213,6 +218,10 @@ impl ResponseReader {
         Ok(response.with_date(date))
     }
 
+    /// Read and parse the head object response.
+    ///
+    /// Returns with error if an unexpected status code is encountered or the response headers or
+    /// body cannot be parsed.
     pub(crate) async fn head_object_response(self) -> Result<Option<HeadObjectResponse>, Error> {
         let (parts, body) = self.response.into_parts();
         let body = body.collect().await?.to_bytes();
@@ -244,6 +253,10 @@ impl ResponseReader {
         }))
     }
 
+    /// Read and parse the get object response.
+    ///
+    /// Returns with error if the object is not accessible, an unexpected status code is encountered
+    /// or the response headers or body cannot be parsed.
     pub(crate) async fn get_object_response(self) -> Result<Option<GetObjectResponse>, Error> {
         let (parts, content) = self.response.into_parts();
 
@@ -274,6 +287,10 @@ impl ResponseReader {
         }))
     }
 
+    /// Read and parse the put object response.
+    ///
+    /// Returns with error on bad request, an unexpected status code is encountered or the response
+    /// headers or body cannot be parsed.
     pub(crate) async fn put_object_response(self) -> Result<PutObjectResponse, Error> {
         let (parts, body) = self.response.into_parts();
         let body = body.collect().await?.to_bytes();
@@ -301,6 +318,9 @@ impl ResponseReader {
         Ok(PutObjectResponse::Success(e_tag))
     }
 
+    /// Read and parse the delete object response.
+    ///
+    /// Returns with error if an unexpected status code is encountered.
     pub(crate) async fn delete_object_response(self) -> Result<(), Error> {
         let (parts, _body) = self.response.into_parts();
 
@@ -312,6 +332,10 @@ impl ResponseReader {
         Ok(())
     }
 
+    /// Read and parse the delete objects response.
+    ///
+    /// Returns with error on bad request, an unexpected status code is encountered or the response
+    /// body cannot be parsed.
     pub(crate) async fn delete_objects_response(self) -> Result<DeleteObjectsResponse, Error> {
         let (parts, body) = self.response.into_parts();
         let body = body.collect().await?.to_bytes();
@@ -336,6 +360,10 @@ impl ResponseReader {
         Ok(delete_objects_response)
     }
 
+    /// Read and parse the copy object response.
+    ///
+    /// Returns with error if the source object cannot be found or is in-accessible, an unexpected
+    /// status code is encountered or the response headers or body cannot be parsed.
     pub(crate) async fn copy_object_response(self) -> Result<CopyObjectResponse, Error> {
         let (parts, body) = self.response.into_parts();
         let body = body.collect().await?.to_bytes();
@@ -371,6 +399,10 @@ impl ResponseReader {
         })
     }
 
+    /// Read and parse the list buckets response.
+    ///
+    /// Returns with error if an unexpected status code is encountered or the response body cannot
+    /// be parsed.
     pub(crate) async fn list_buckets_response(self) -> Result<ListBucketsResponse, Error> {
         let (parts, body) = self.response.into_parts();
         let body = body.collect().await?.to_bytes();
-- 
2.47.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox 5/6] s3 client: add and expand doc comments for response parsing objects

[Christian Ebner]

Most of the object defined for the response reader are used for xml
parsing of the corresponding response body's.
Extend or fix the already present doc comments and add new ones for
the ones not documented yet.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 proxmox-s3-client/src/response_reader.rs | 110 ++++++++++++++++++-----
 1 file changed, 86 insertions(+), 24 deletions(-)

diff --git a/proxmox-s3-client/src/response_reader.rs b/proxmox-s3-client/src/response_reader.rs
index cd3ee98a..da76ec3f 100644
--- a/proxmox-s3-client/src/response_reader.rs
+++ b/proxmox-s3-client/src/response_reader.rs
@@ -12,37 +12,54 @@ use serde::Deserialize;
 use crate::S3ObjectKey;
 use crate::{HttpDate, LastModifiedTimestamp};
 
+/// Response reader to check S3 api response status codes and parse response body, if any.
 pub(crate) struct ResponseReader {
     response: Response<Incoming>,
 }
 
 #[derive(Debug)]
-/// Subset of the list object v2 response including some header values
-/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html#API_CopyObject_ResponseSyntax
+/// Response contents of list objects v2 api calls.
 pub struct ListObjectsV2Response {
+    /// Parsed http date header from response.
     pub date: HttpDate,
+    /// Bucket name.
     pub name: String,
+    /// Requested key prefix.
     pub prefix: String,
+    /// Number of keys returned in this response.
     pub key_count: u64,
+    /// Number of max keys the response can contain.
     pub max_keys: u64,
+    /// Flag indication if response was truncated because of key limits.
     pub is_truncated: bool,
+    /// Token used for this request to get further keys in truncated responses.
     pub continuation_token: Option<String>,
+    /// Allows to fetch the next set of keys for truncated responses.
     pub next_continuation_token: Option<String>,
+    /// List of response objects, including their object key.
     pub contents: Vec<ListObjectsV2Contents>,
 }
 
 #[derive(Deserialize, Debug)]
 #[serde(rename_all = "PascalCase")]
-/// Subset of items used to deserialize a list objects v2 respsonse
-/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html#API_CopyObject_ResponseSyntax
+/// Subset of items used to deserialize a list objects v2 respsonse.
+/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html#API_ListObjectsV2_ResponseSyntax
 struct ListObjectsV2ResponseBody {
+    /// Bucket name.
     pub name: String,
+    /// Requested key prefix.
     pub prefix: String,
+    /// Number of keys returned in this response.
     pub key_count: u64,
+    /// Number of max keys the response can contain.
     pub max_keys: u64,
+    /// Flag indication if response was truncated because of key limits.
     pub is_truncated: bool,
+    /// Token used for this request to get further keys in truncated responses.
     pub continuation_token: Option<String>,
+    /// Allows to fetch the next set of keys for truncated responses.
     pub next_continuation_token: Option<String>,
+    /// List of response objects, including their object key.
     pub contents: Option<Vec<ListObjectsV2Contents>>,
 }
 
@@ -64,124 +81,169 @@ impl ListObjectsV2ResponseBody {
 
 #[derive(Deserialize, Debug)]
 #[serde(rename_all = "PascalCase")]
-/// Subset used to deserialize the contents of a list objects v2 respsonse
-/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html#API_CopyObject_ResponseSyntax
+/// Subset of contents used to deserialize the listed object contents of a list objects v2 respsonse.
+/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html#API_ListObjectsV2_ResponseSyntax
 pub struct ListObjectsV2Contents {
+    /// Object key.
     pub key: S3ObjectKey,
+    /// Object last modified timestamp.
     pub last_modified: LastModifiedTimestamp,
+    /// Entity tag for object.
     pub e_tag: String,
+    /// Content size of object.
     pub size: u64,
+    /// Storage class the object is stored on.
     pub storage_class: String,
 }
 
 #[derive(Debug)]
-/// Subset of the head object response (headers only, there is no body)
+/// Subset of contents for the head object response (headers only, there is no body).
 /// See https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html#API_HeadObject_ResponseSyntax
 pub struct HeadObjectResponse {
+    /// Content length header.
     pub content_length: u64,
+    /// Content type header.
     pub content_type: String,
+    /// Http date header.
     pub date: HttpDate,
+    /// Entity tag header.
     pub e_tag: String,
+    /// Last modified http header.
     pub last_modified: HttpDate,
 }
 
-/// Subset of the get object response including some headers
+/// Response contents of the get object api call.
 /// https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html#API_GetObject_ResponseSyntax
 pub struct GetObjectResponse {
+    /// Content length header.
     pub content_length: u64,
+    /// Content type header.
     pub content_type: String,
+    /// Http date header.
     pub date: HttpDate,
+    /// Entity tag header.
     pub e_tag: String,
+    /// Last modified http header.
     pub last_modified: HttpDate,
+    /// Object content in http response body.
     pub content: Incoming,
 }
 
-/// Subset of the put object response
-/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html#API_PutObject_ResponseSyntax
 #[derive(Debug)]
+/// Variants to distinguish object upload response states.
+/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html#API_PutObject_ResponseSyntax
 pub enum PutObjectResponse {
+    /// Object upload failed because of conflicting operation, upload should be retried.
     NeedsRetry,
+    /// Object was not uploaded because the provided pre-condition
+    /// (e.g. If-None-Match header) failed.
     PreconditionFailed,
+    /// Object was uploaded with success with the contained entity tag.
     Success(String),
 }
 
-/// Subset of the delete objects response
-/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html#API_DeleteObjects_ResponseElements
 #[derive(Deserialize, Debug, Default)]
 #[serde(rename_all = "PascalCase")]
+/// Response contents of the delete objects api call.
+/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html#API_DeleteObjects_ResponseElements
 pub struct DeleteObjectsResponse {
+    /// List of deleted objects, if any.
     pub deleted: Option<Vec<DeletedObject>>,
+    /// List of errors, if deletion failed for some objects.
     pub error: Option<Vec<DeleteObjectError>>,
 }
 
-/// Subset used to deserialize the deleted objects of a delete objects v2 respsonse
-/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeletedObject.html
 #[derive(Deserialize, Debug)]
 #[serde(rename_all = "PascalCase")]
+/// Subset of contents used to deserialize the deleted objects of a delete objects v2 respsonse.
+/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeletedObject.html
 pub struct DeletedObject {
+    /// Indicates whether the version of the object was a delete marker before deletion.
     pub delete_marker: Option<bool>,
+    /// Version ID of the delete marker created as result of the delete operation.
     pub delete_marker_version_id: Option<String>,
+    /// Key of the deleted object.
     pub key: Option<S3ObjectKey>,
+    /// Version ID of the deleted object.
     pub version_id: Option<String>,
 }
 
-/// Subset used to deserialize the deleted object errors of a delete objects v2 respsonse
-/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_Error.html
 #[derive(Deserialize, Debug)]
 #[serde(rename_all = "PascalCase")]
+/// Subset of contents used to deserialize the deleted object errors of a delete objects v2 respsonse
+/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_Error.html
 pub struct DeleteObjectError {
+    /// Error code identifying the error condition.
     pub code: Option<String>,
+    /// Object key for which the error occurred.
     pub key: Option<S3ObjectKey>,
+    /// Generic error description.
     pub message: Option<String>,
+    /// Version ID of error.
     pub version_id: Option<String>,
 }
 
 #[derive(Debug)]
-/// Subset used to deserialize the copy object response
+/// Response contents of the copy object api calls.
 /// https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html#API_CopyObject_ResponseSyntax
 pub struct CopyObjectResponse {
+    /// Result contents of the copy object operation.
     pub copy_object_result: CopyObjectResult,
+    /// Version ID of the newly created object copy.
     pub x_amz_version_id: Option<String>,
 }
 
 #[derive(Deserialize, Debug)]
 #[serde(rename_all = "PascalCase")]
-/// Subset used to deserialize the copy object result of a copy object respsonse
+/// Subset of contents used to deserialize the copy object result of a copy object respsonse.
 /// https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html#API_CopyObject_ResponseSyntax
 pub struct CopyObjectResult {
+    /// Entity tag.
     pub e_tag: String,
+    /// Last modified timestamp.
     pub last_modified: LastModifiedTimestamp,
 }
 
-/// Subset of the list buckets response
-/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListBuckets.html#API_ListBuckets_ResponseElements
 #[derive(Deserialize, Debug)]
 #[serde(rename_all = "PascalCase")]
+/// Response contents of the list buckets api calls.
+/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListBuckets.html#API_ListBuckets_ResponseElements
 pub struct ListBucketsResponse {
+    /// List of buckets accessible given caller's access key.
     pub buckets: Vec<Bucket>,
 }
-/// Subset of the list buckets response
-/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListBuckets.html#API_ListBuckets_ResponseElements
 #[derive(Deserialize, Debug)]
 #[serde(rename_all = "PascalCase")]
+/// Subset of contents used to deserialize the response of a list buckets api call.
+/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListBuckets.html#API_ListBuckets_ResponseElements
 pub struct ListAllMyBucketsResult {
+    /// List Bucket contents.
     pub buckets: Option<Buckets>,
 }
 
-/// Subset used to deserialize the list buckets response
+/// Subset of contents used to deserialize the list of buckets for response of a list buckets api
+/// call.
+/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListBuckets.html#API_ListBuckets_ResponseElements
 #[derive(Deserialize, Debug)]
 #[serde(rename_all = "PascalCase")]
 pub struct Buckets {
+    /// List of individual bucket contents.
     bucket: Vec<Bucket>,
 }
 
-/// Subset used to deserialize the list buckets response
 #[derive(Deserialize, Debug)]
 #[serde(rename_all = "PascalCase")]
+/// Subset of contents used to deserialize individual buckets for response of a list buckets api
+/// call.
+/// https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListBuckets.html#API_ListBuckets_ResponseElements
 pub struct Bucket {
+    /// Bucket name.
     pub name: String,
+    // Bucket ARN.
     pub bucket_arn: Option<String>,
+    /// Bucket region.
     pub bucket_region: Option<String>,
+    /// Bucket creation timestamp.
     pub creation_date: LastModifiedTimestamp,
 }
 
-- 
2.47.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox 6/6] s3 client: add basic regression tests for response parsing

[Christian Ebner]

Adds regression tests for methods parsing the http response body
by using responses as found in the AWS documentation for the
respective api method.

Requires to derive PartialEq on S3ObjectKey and LastModifiedTimestamp
structs in order to be able to easily compare the resulting contents.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 proxmox-s3-client/src/object_key.rs      |   2 +-
 proxmox-s3-client/src/response_reader.rs | 113 ++++++++++++++++++++++-
 proxmox-s3-client/src/timestamps.rs      |   2 +-
 3 files changed, 113 insertions(+), 4 deletions(-)

diff --git a/proxmox-s3-client/src/object_key.rs b/proxmox-s3-client/src/object_key.rs
index 327e8ac7..a68a588f 100644
--- a/proxmox-s3-client/src/object_key.rs
+++ b/proxmox-s3-client/src/object_key.rs
@@ -4,7 +4,7 @@ use anyhow::{bail, Error};
 /// See https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html
 const S3_OBJECT_KEY_MAX_LENGTH: usize = 1024;
 
-#[derive(Clone, Debug)]
+#[derive(Clone, Debug, PartialEq)]
 /// S3 Object Key
 pub enum S3ObjectKey {
     /// Object key which will not be prefixed any further by the client
diff --git a/proxmox-s3-client/src/response_reader.rs b/proxmox-s3-client/src/response_reader.rs
index da76ec3f..f895db19 100644
--- a/proxmox-s3-client/src/response_reader.rs
+++ b/proxmox-s3-client/src/response_reader.rs
@@ -79,7 +79,7 @@ impl ListObjectsV2ResponseBody {
     }
 }
 
-#[derive(Deserialize, Debug)]
+#[derive(Deserialize, Debug, PartialEq)]
 #[serde(rename_all = "PascalCase")]
 /// Subset of contents used to deserialize the listed object contents of a list objects v2 respsonse.
 /// https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html#API_ListObjectsV2_ResponseSyntax
@@ -231,7 +231,7 @@ pub struct Buckets {
     bucket: Vec<Bucket>,
 }
 
-#[derive(Deserialize, Debug)]
+#[derive(Deserialize, Debug, PartialEq)]
 #[serde(rename_all = "PascalCase")]
 /// Subset of contents used to deserialize individual buckets for response of a list buckets api
 /// call.
@@ -511,3 +511,112 @@ impl ResponseReader {
         Ok(value)
     }
 }
+
+#[test]
+fn parse_list_objects_v2_response_test() {
+    let response_body = r#"<?xml version="1.0" encoding="UTF-8"?>
+        <ListBucketResult>
+            <Name>bucket0</Name>
+            <Prefix>.cnt</Prefix>
+            <KeyCount>2</KeyCount>
+            <MaxKeys>1000</MaxKeys>
+            <IsTruncated>false</IsTruncated>
+            <Contents>
+                <Key>.cnt/key0</Key>
+                <LastModified>2011-02-26T01:56:20.000Z</LastModified>
+                <ETag>"bf1d737a4d46a19f3bced6905cc8b902"</ETag>
+                <Size>10</Size>
+                <StorageClass>STANDARD</StorageClass>
+            </Contents>
+            <Contents>
+                <Key>.cnt/key1</Key>
+                <LastModified>2011-02-26T01:56:20.000Z</LastModified>
+                <ETag>"9b2cf535f27731c974343645a3985328"</ETag>
+                <Size>20</Size>
+                <StorageClass>STANDARD</StorageClass>
+            </Contents>
+        </ListBucketResult>
+    "#;
+    let result: ListObjectsV2ResponseBody = serde_xml_rs::from_str(&response_body).unwrap();
+    assert_eq!(result.name, "bucket0");
+    assert_eq!(result.prefix, ".cnt");
+    assert_eq!(result.key_count, 2);
+    assert_eq!(result.max_keys, 1000);
+    assert_eq!(result.is_truncated, false);
+    assert_eq!(
+        result.contents.unwrap(),
+        vec![
+            ListObjectsV2Contents {
+                key: S3ObjectKey::try_from("/.cnt/key0").unwrap(),
+                last_modified: LastModifiedTimestamp::from_str("2011-02-26T01:56:20.000Z").unwrap(),
+                e_tag: "\"bf1d737a4d46a19f3bced6905cc8b902\"".to_string(),
+                size: 10,
+                storage_class: "STANDARD".to_string(),
+            },
+            ListObjectsV2Contents {
+                key: S3ObjectKey::try_from("/.cnt/key1").unwrap(),
+                last_modified: LastModifiedTimestamp::from_str("2011-02-26T01:56:20.000Z").unwrap(),
+                e_tag: "\"9b2cf535f27731c974343645a3985328\"".to_string(),
+                size: 20,
+                storage_class: "STANDARD".to_string(),
+            },
+        ]
+    );
+}
+
+#[test]
+fn parse_copy_object_response_test() {
+    let response_body = r#"<?xml version="1.0" encoding="UTF-8"?>
+        <CopyObjectResult>
+            <LastModified>2009-10-12T17:50:30.000Z</LastModified>
+            <ETag>"9b2cf535f27731c974343645a3985328"</ETag>
+        </CopyObjectResult>
+    "#;
+    let result: CopyObjectResult = serde_xml_rs::from_str(&response_body).unwrap();
+    assert_eq!(
+        result.last_modified,
+        LastModifiedTimestamp::from_str("2009-10-12T17:50:30.000Z").unwrap()
+    );
+    assert_eq!(
+        result.e_tag,
+        "\"9b2cf535f27731c974343645a3985328\"".to_string()
+    );
+}
+
+#[test]
+fn parse_list_buckets_response_test() {
+    let response_body = r#"<?xml version="1.0" encoding="UTF-8"?>
+        <ListAllMyBucketsResult>
+            <Buckets>
+                <Bucket>
+                    <CreationDate>2019-12-11T23:32:47+00:00</CreationDate>
+                    <Name>bucket0</Name>
+                </Bucket>
+                <Bucket>
+                    <CreationDate>2019-11-10T23:32:13+00:00</CreationDate>
+                    <Name>bucket1</Name>
+                </Bucket>
+            </Buckets>
+        </ListAllMyBucketsResult>
+    "#;
+    let result: ListAllMyBucketsResult = serde_xml_rs::from_str(&response_body).unwrap();
+    assert_eq!(
+        result.buckets.unwrap().bucket,
+        vec![
+            Bucket {
+                name: "bucket0".to_string(),
+                creation_date: LastModifiedTimestamp::from_str("2019-12-11T23:32:47+00:00")
+                    .unwrap(),
+                bucket_arn: None,
+                bucket_region: None,
+            },
+            Bucket {
+                name: "bucket1".to_string(),
+                creation_date: LastModifiedTimestamp::from_str("2019-11-10T23:32:13+00:00")
+                    .unwrap(),
+                bucket_arn: None,
+                bucket_region: None,
+            },
+        ]
+    );
+}
diff --git a/proxmox-s3-client/src/timestamps.rs b/proxmox-s3-client/src/timestamps.rs
index 22330966..661e1fdf 100644
--- a/proxmox-s3-client/src/timestamps.rs
+++ b/proxmox-s3-client/src/timestamps.rs
@@ -5,7 +5,7 @@ const VALID_MONTHS: [&str; 12] = [
     "Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec",
 ];
 
-#[derive(Debug)]
+#[derive(Debug, PartialEq)]
 /// Last modified timestamp as obtained from API response http headers.
 pub struct LastModifiedTimestamp {
     _datetime: iso8601::DateTime,
-- 
2.47.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

Re: [pbs-devel] [PATCH proxmox 0/6] followups and cleanups for s3 client

[Thomas Lamprecht]

On Mon, 04 Aug 2025 18:09:31 +0200, Christian Ebner wrote:
> These patches are not release critical, but are intended to improve
> code quality and add basic testing.
> 
> This currently covers formatting issues, refactoring of code to make
> it more concise, adding missing doc comments and adding basic
> regression tests for parsers reading the http response body.
> 
> [...]

Applied, thanks!

[1/6] s3 client: fix formatting issues by `cargo fmt`
      commit: 4a74b59ba60c06eb40ae720c16c757ef966e937b
[2/6] s3 client: refactor list buckets response status code matching
      commit: 674fc39d7a1e81db21d01cac966aca39c7995cf8
[3/6] s3 client: refactor list buckets result list fetching
      commit: c849445afe94f951a710eb7463542a659cfa353d
[4/6] s3 client: add doc comments for response parser helper methods
      commit: 3a45f2e7ae4a6312b3cd4572f4ed32a5b42eef03
[5/6] s3 client: add and expand doc comments for response parsing objects
      commit: dcec6ba0b88389458e43c8b12d96c00242d2af1d
[6/6] s3 client: add basic regression tests for response parsing
      commit: a02d890f01f212663ab75989a79c7fc36d479ba3


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

Re: [pve-devel] [pbs-devel] [PATCH proxmox 0/6] followups and cleanups for s3 client

[Thomas Lamprecht]

On Mon, 04 Aug 2025 18:09:31 +0200, Christian Ebner wrote:
> These patches are not release critical, but are intended to improve
> code quality and add basic testing.
> 
> This currently covers formatting issues, refactoring of code to make
> it more concise, adding missing doc comments and adding basic
> regression tests for parsers reading the http response body.
> 
> [...]

Applied, thanks!

[1/6] s3 client: fix formatting issues by `cargo fmt`
      commit: 4a74b59ba60c06eb40ae720c16c757ef966e937b
[2/6] s3 client: refactor list buckets response status code matching
      commit: 674fc39d7a1e81db21d01cac966aca39c7995cf8
[3/6] s3 client: refactor list buckets result list fetching
      commit: c849445afe94f951a710eb7463542a659cfa353d
[4/6] s3 client: add doc comments for response parser helper methods
      commit: 3a45f2e7ae4a6312b3cd4572f4ed32a5b42eef03
[5/6] s3 client: add and expand doc comments for response parsing objects
      commit: dcec6ba0b88389458e43c8b12d96c00242d2af1d
[6/6] s3 client: add basic regression tests for response parsing
      commit: a02d890f01f212663ab75989a79c7fc36d479ba3


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel