[PATCH proxmox] s3-client: copy_object: always set Content-Type header

[PATCH proxmox] s3-client: copy_object: always set Content-Type header

[Hannes Laimer]

Some S3 providers drop all source metadata, including Content-Type,
when using x-amz-metadata-directive REPLACE unless it is explicitly
provided in the copy request. This caused s3_refresh to fail with
"missing header 'content-type'" on such providers when fetching
objects moved via copy_object.

Setting Content-Type explicitly is a no-op for providers that already
preserve it, so this is safe regardless of the provider.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
did notice this with RustFS, MinIO did not have that problem

 proxmox-s3-client/src/client.rs | 1 +
 1 file changed, 1 insertion(+)

diff --git a/proxmox-s3-client/src/client.rs b/proxmox-s3-client/src/client.rs
index 81645716..baa8345e 100644
--- a/proxmox-s3-client/src/client.rs
+++ b/proxmox-s3-client/src/client.rs
@@ -602,6 +602,7 @@ impl S3Client {
             .method(Method::PUT)
             .uri(self.build_uri(&destination_key, &[])?)
             .header("x-amz-copy-source", HeaderValue::from_str(&copy_source)?)
+            .header(header::CONTENT_TYPE, "binary/octet-stream")
             .header(
                 "x-amz-metadata-directive",
                 HeaderValue::from_str("REPLACE")?,
-- 
2.47.3

Re: [PATCH proxmox] s3-client: copy_object: always set Content-Type header

[Christian Ebner]

On 3/11/26 10:56 AM, Hannes Laimer wrote:
> Some S3 providers drop all source metadata, including Content-Type,
> when using x-amz-metadata-directive REPLACE unless it is explicitly
> provided in the copy request. This caused s3_refresh to fail with
> "missing header 'content-type'" on such providers when fetching
> objects moved via copy_object.
> 
> Setting Content-Type explicitly is a no-op for providers that already
> preserve it, so this is safe regardless of the provider.

This should further state that it only acceptable  to set the header 
unconditionally here, since the current s3 client implementation does 
set it unconditionally as well. Which however raises the inconsistency 
as mentioned below.

> 
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> did notice this with RustFS, MinIO did not have that problem
> 
>   proxmox-s3-client/src/client.rs | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/proxmox-s3-client/src/client.rs b/proxmox-s3-client/src/client.rs
> index 81645716..baa8345e 100644
> --- a/proxmox-s3-client/src/client.rs
> +++ b/proxmox-s3-client/src/client.rs
> @@ -602,6 +602,7 @@ impl S3Client {
>               .method(Method::PUT)
>               .uri(self.build_uri(&destination_key, &[])?)
>               .header("x-amz-copy-source", HeaderValue::from_str(&copy_source)?)
> +            .header(header::CONTENT_TYPE, "binary/octet-stream")

This should be `application/octet-stream` [0], there is no 
`binary/octet-stream` according to [1]. While checking this, I did 
notice the original Content-Type in put_object() is set to 
`binary/octet` which is also wrong. So both should be fixed and aligned.

[0] https://www.iana.org/assignments/media-types/application/octet-stream
[1] https://www.iana.org/assignments/media-types/media-types.xhtml

>               .header(
>                   "x-amz-metadata-directive",
>                   HeaderValue::from_str("REPLACE")?,

Re: [PATCH proxmox] s3-client: copy_object: always set Content-Type header

[Hannes Laimer]

On 2026-03-12 13:39, Christian Ebner wrote:
> On 3/11/26 10:56 AM, Hannes Laimer wrote:
>> Some S3 providers drop all source metadata, including Content-Type,
>> when using x-amz-metadata-directive REPLACE unless it is explicitly
>> provided in the copy request. This caused s3_refresh to fail with
>> "missing header 'content-type'" on such providers when fetching
>> objects moved via copy_object.
>>
>> Setting Content-Type explicitly is a no-op for providers that already
>> preserve it, so this is safe regardless of the provider.
> 
> This should further state that it only acceptable  to set the header
> unconditionally here, since the current s3 client implementation does
> set it unconditionally as well. Which however raises the inconsistency
> as mentioned below.
> 
>>
>> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
>> ---
>> did notice this with RustFS, MinIO did not have that problem
>>
>>   proxmox-s3-client/src/client.rs | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/proxmox-s3-client/src/client.rs b/proxmox-s3-client/src/
>> client.rs
>> index 81645716..baa8345e 100644
>> --- a/proxmox-s3-client/src/client.rs
>> +++ b/proxmox-s3-client/src/client.rs
>> @@ -602,6 +602,7 @@ impl S3Client {
>>               .method(Method::PUT)
>>               .uri(self.build_uri(&destination_key, &[])?)
>>               .header("x-amz-copy-source",
>> HeaderValue::from_str(&copy_source)?)
>> +            .header(header::CONTENT_TYPE, "binary/octet-stream")
> 
> This should be `application/octet-stream` [0], there is no `binary/
> octet-stream` according to [1]. While checking this, I did notice the
> original Content-Type in put_object() is set to `binary/octet` which is
> also wrong. So both should be fixed and aligned.
> 

thanks for taking a look, makes sense! Funnily enough, even this did fix
the problem, I guess it being set at all was enough :P

I'll send a v2, with this and the one you mentioned updated


> [0] https://www.iana.org/assignments/media-types/application/octet-stream
> [1] https://www.iana.org/assignments/media-types/media-types.xhtml
> 
>>               .header(
>>                   "x-amz-metadata-directive",
>>                   HeaderValue::from_str("REPLACE")?,
> 

Re: [PATCH proxmox] s3-client: copy_object: always set Content-Type header

[Christian Ebner]

On 3/12/26 1:45 PM, Hannes Laimer wrote:
> On 2026-03-12 13:39, Christian Ebner wrote:
>> On 3/11/26 10:56 AM, Hannes Laimer wrote:
>>> Some S3 providers drop all source metadata, including Content-Type,
>>> when using x-amz-metadata-directive REPLACE unless it is explicitly
>>> provided in the copy request. This caused s3_refresh to fail with
>>> "missing header 'content-type'" on such providers when fetching
>>> objects moved via copy_object.
>>>
>>> Setting Content-Type explicitly is a no-op for providers that already
>>> preserve it, so this is safe regardless of the provider.
>>
>> This should further state that it only acceptable  to set the header
>> unconditionally here, since the current s3 client implementation does
>> set it unconditionally as well. Which however raises the inconsistency
>> as mentioned below.
>>
>>>
>>> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
>>> ---
>>> did notice this with RustFS, MinIO did not have that problem
>>>
>>>    proxmox-s3-client/src/client.rs | 1 +
>>>    1 file changed, 1 insertion(+)
>>>
>>> diff --git a/proxmox-s3-client/src/client.rs b/proxmox-s3-client/src/
>>> client.rs
>>> index 81645716..baa8345e 100644
>>> --- a/proxmox-s3-client/src/client.rs
>>> +++ b/proxmox-s3-client/src/client.rs
>>> @@ -602,6 +602,7 @@ impl S3Client {
>>>                .method(Method::PUT)
>>>                .uri(self.build_uri(&destination_key, &[])?)
>>>                .header("x-amz-copy-source",
>>> HeaderValue::from_str(&copy_source)?)
>>> +            .header(header::CONTENT_TYPE, "binary/octet-stream")
>>
>> This should be `application/octet-stream` [0], there is no `binary/
>> octet-stream` according to [1]. While checking this, I did notice the
>> original Content-Type in put_object() is set to `binary/octet` which is
>> also wrong. So both should be fixed and aligned.
>>
> 
> thanks for taking a look, makes sense! Funnily enough, even this did fix
> the problem, I guess it being set at all was enough :P
> 
> I'll send a v2, with this and the one you mentioned updated

Yes, some S3 clients seem to default to `binary/octet-stream` though, 
according to https://github.com/aws/aws-sdk-java/issues/1143

But using the IANA registered content-type is preferable i guess.

superseded: [PATCH proxmox] s3-client: copy_object: always set Content-Type header

[Hannes Laimer]

superseded-by:
https://lore.proxmox.com/pbs-devel/20260316141940.3576-1-h.laimer@proxmox.com/T/#u

On 2026-03-11 10:56, Hannes Laimer wrote:
> Some S3 providers drop all source metadata, including Content-Type,
> when using x-amz-metadata-directive REPLACE unless it is explicitly
> provided in the copy request. This caused s3_refresh to fail with
> "missing header 'content-type'" on such providers when fetching
> objects moved via copy_object.
> 
> Setting Content-Type explicitly is a no-op for providers that already
> preserve it, so this is safe regardless of the provider.
> 
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> did notice this with RustFS, MinIO did not have that problem
> 
>  proxmox-s3-client/src/client.rs | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/proxmox-s3-client/src/client.rs b/proxmox-s3-client/src/client.rs
> index 81645716..baa8345e 100644
> --- a/proxmox-s3-client/src/client.rs
> +++ b/proxmox-s3-client/src/client.rs
> @@ -602,6 +602,7 @@ impl S3Client {
>              .method(Method::PUT)
>              .uri(self.build_uri(&destination_key, &[])?)
>              .header("x-amz-copy-source", HeaderValue::from_str(&copy_source)?)
> +            .header(header::CONTENT_TYPE, "binary/octet-stream")
>              .header(
>                  "x-amz-metadata-directive",
>                  HeaderValue::from_str("REPLACE")?,