[pbs-devel] [PATCH proxmox-backup 0/2] docs: extend S3 related documentation

[pbs-devel] [PATCH proxmox-backup 0/2] docs: extend S3 related documentation

[Christian Ebner]

Two additional patches to improve the documentation for the S3 backend.

The first adds a warning for the costs which might arise from operating
the S3 backed datastore, the second shows some concrete, provider
specific configuration examples.

Christian Ebner (2):
  docs: s3: add warning of potential costs when using s3 backends
  docs: s3: add object store provider specific configuration examples

 docs/storage.rst | 75 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)

-- 
2.47.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox-backup 1/2] docs: s3: add warning of potential costs when using s3 backends

[Christian Ebner]

S3 object store providers typically charge not only for storage
usage, but also for API requests. Explicitley warn the user about
this in the docs.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 docs/storage.rst | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/docs/storage.rst b/docs/storage.rst
index 197bbd6c2..2b2db05c6 100644
--- a/docs/storage.rst
+++ b/docs/storage.rst
@@ -241,6 +241,11 @@ this, an S3 endpoint needs to be set-up under "Configuration" > "Remotes" > "S3
 
 .. important:: The S3 datastore backend is currently a technology preview.
 
+.. important:: Keep in mind that operating as S3 backed object store might cause additional costs.
+   Providers might charge you for storage space and API requests performed to the buckets, egress
+   and bandwidth fees might be charged as well. Therefore, monitoring of these values and eventual
+   costs is highly recommended.
+
 In the endpoint configuration, provide the REST API endpoint for the object store. The endpoint
 is provider dependent and allows for the bucket and region templating. For example, configuring
 the endpoint as e.g. ``{{bucket}}.s3.{{region}}.amazonaws.com`` will be expanded to
-- 
2.47.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox-backup 2/2] docs: s3: add object store provider specific configuration examples

[Christian Ebner]

To reduce friction, provide several provider specific example
configurations as reference.

With vhost style vs. path style bucket addressing, templating and all
the other provider specific configuration options, it can be rather
confusing on how to actually configure an S3 endpoint to be used as
PBS datastore backend. So having some concrete examples to lookup or
point to can help.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 docs/storage.rst | 70 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)

diff --git a/docs/storage.rst b/docs/storage.rst
index 2b2db05c6..7a4e703b1 100644
--- a/docs/storage.rst
+++ b/docs/storage.rst
@@ -314,6 +314,76 @@ same datastore name must be used.
    on the S3 object store manually and refresh the contents via an ``S3 refresh``, either via the
    CLI or UI.
 
+
+S3 Datastore Backend Configuration Examples
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The following shows example configurations for some typical S3 object store providers as excerpts
+(data relevant to S3 config only) from ``/etc/proxmox-backup/s3.cfg`` and
+``/etc/proxmox-backup/datastore.cfg``:
+
+Self hosted S3 object store with Ceph Rados Gateway using plain IP address, custom port, self-signed
+certificate and path-style bucket:
+
+.. code-block:: console
+
+   # cat /etc/proxmox-backup/s3.cfg
+
+   s3client: ceph-s3-rados-gw
+        access-key XXXXXXXXXXXXXXXXXXXX
+        endpoint 172.16.0.200
+        fingerprint XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX
+        path-style true
+        port 7480
+        secret-key XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
+
+
+   # cat /etc/proxmox-backup/datastore.cfg
+
+   datastore: ceph-s3-rgw-store
+        backend bucket=pbs-ceph-bucket,client=ceph-s3-rados-gw,type=s3
+        path /mnt/datastore/ceph-s3-rgw-store-local-cache
+
+AWS S3 with vhost style bucket addressing, using bucket name and region templating for the endpoint
+url:
+
+.. code-block:: console
+
+   # cat /etc/proxmox-backup/s3.cfg
+
+   s3client: aws-s3
+        access-key XXXXXXXXXXXXXXXXXXXX
+        endpoint {{bucket}}.s3.{{region}}.amazonaws.com
+        region eu-central-1
+        secret-key XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
+
+   # cat /etc/proxmox-backup/datastore.cfg
+
+   datastore: aws-s3-store
+        backend bucket=pbs-s3-bucket,client=aws-s3,type=s3
+        path /mnt/datastore/aws-s3-store-local-cache
+
+Cloudflare R2 with path style bucket addressing, note that region must be set to ``auto`` as
+otherwise request authentication might fail:
+
+.. code-block:: console
+
+   # cat /etc/proxmox-backup/s3.cfg
+
+   s3client: cloudflare-r2
+        access-key XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
+        endpoint XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.r2.cloudflarestorage.com
+        path-style true
+        region auto
+        secret-key XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
+
+    # cat /etc/proxmox-backup/datastore.cfg
+
+    datastore: r2-s3-store
+        backend bucket=pbs-r2-bucket,client=cloudflare-r2,type=s3
+        path /mnt/datastore/r2-s3-store-local-cache
+
+
 Managing Datastores
 ^^^^^^^^^^^^^^^^^^^
 
-- 
2.47.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

Re: [pbs-devel] [PATCH proxmox-backup 0/2] docs: extend S3 related documentation

[Lukas Wagner]

On Wed Jul 23, 2025 at 10:02 AM CEST, Christian Ebner wrote:
> Two additional patches to improve the documentation for the S3 backend.
>
> The first adds a warning for the costs which might arise from operating
> the S3 backed datastore, the second shows some concrete, provider
> specific configuration examples.
>
> Christian Ebner (2):
>   docs: s3: add warning of potential costs when using s3 backends
>   docs: s3: add object store provider specific configuration examples
>
>  docs/storage.rst | 75 ++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 75 insertions(+)

Looks good to me. Also built the docs and checked if anything in the new
examples section looked off - nothing did!

Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] applied: [PATCH proxmox-backup 0/2] docs: extend S3 related documentation

[Thomas Lamprecht]

On Wed, 23 Jul 2025 10:02:29 +0200, Christian Ebner wrote:
> Two additional patches to improve the documentation for the S3 backend.
> 
> The first adds a warning for the costs which might arise from operating
> the S3 backed datastore, the second shows some concrete, provider
> specific configuration examples.
> 
> Christian Ebner (2):
>   docs: s3: add warning of potential costs when using s3 backends
>   docs: s3: add object store provider specific configuration examples
> 
> [...]

Applied, thanks!

[1/2] docs: s3: add warning of potential costs when using s3 backends
      (no commit info)
[2/2] docs: s3: add object store provider specific configuration examples
      (no commit info)


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pve-devel] applied: [pbs-devel] [PATCH proxmox-backup 0/2] docs: extend S3 related documentation

[Thomas Lamprecht]

On Wed, 23 Jul 2025 10:02:29 +0200, Christian Ebner wrote:
> Two additional patches to improve the documentation for the S3 backend.
> 
> The first adds a warning for the costs which might arise from operating
> the S3 backed datastore, the second shows some concrete, provider
> specific configuration examples.
> 
> Christian Ebner (2):
>   docs: s3: add warning of potential costs when using s3 backends
>   docs: s3: add object store provider specific configuration examples
> 
> [...]

Applied, thanks!

[1/2] docs: s3: add warning of potential costs when using s3 backends
      (no commit info)
[2/2] docs: s3: add object store provider specific configuration examples
      (no commit info)


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel