From: Christian Ebner <c.ebner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: Re: [pbs-devel] [PATCH proxmox{, -backup} v4 00/48] fix #2943: S3 storage backend for datastores
Date: Mon, 30 Jun 2025 11:21:40 +0200 [thread overview]
Message-ID: <7e24b63f-fc37-4a9b-86c5-6b61a6b01f67@proxmox.com> (raw)
In-Reply-To: <20250623094106.299251-1-c.ebner@proxmox.com>
Some further, real-world testing using a Cloudflare R2 object store as
backend for the datastore reveals that request rate limiting can still
be an issue. These were not an issue with locally hosted object stores,
e.g. via Ceph RADOS gateway.
While the Cloudflare documentation specifies limits for some bucket
management operations [0], these should not apply to object write
operations. However, in my test case requests are returning with an 502
bad gateway response, reduced in occurrence if the upload to the PBS is
limited via e.g. traffic control rate limits.
Will look into how to best implement request limits in the s3 client
itself, ideally with a mechanism to dynamically adapt the rate in case
of such object upload errors, including a retry logic.
[0] https://developers.cloudflare.com/r2/platform/limits/
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
next prev parent reply other threads:[~2025-06-30 9:21 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-06-23 9:40 Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox v4 1/3] pbs-api-types: add types for S3 client configs and secrets Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox v4 2/3] pbs-api-types: extend datastore config by backend config enum Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox v4 3/3] pbs-api-types: maintenance: add new maintenance mode S3 refresh Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 01/45] api: fix minor formatting issues Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 02/45] bin: sort submodules alphabetically Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 03/45] datastore: ignore missing owner file when removing group directory Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 04/45] verify: refactor verify related functions to be methods of worker Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 05/45] s3 client: add crate for AWS s3 compatible object store client Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 06/45] s3 client: implement AWS signature v4 request authentication Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 07/45] s3 client: add dedicated type for s3 object keys Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 08/45] s3 client: add type for last modified timestamp in responses Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 09/45] s3 client: add helper to parse http date headers Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 10/45] s3 client: implement methods to operate on s3 objects in bucket Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 11/45] config: introduce s3 object store client configuration Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 12/45] api: config: implement endpoints to manipulate and list s3 configs Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 13/45] api: datastore: check s3 backend bucket access on datastore create Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 14/45] api/cli: add endpoint and command to check s3 client connection Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 15/45] datastore: allow to get the backend for a datastore Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 16/45] api: backup: store datastore backend in runtime environment Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 17/45] api: backup: conditionally upload chunks to s3 object store backend Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 18/45] api: backup: conditionally upload blobs " Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 19/45] api: backup: conditionally upload indices " Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 20/45] api: backup: conditionally upload manifest " Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 21/45] sync: pull: conditionally upload content to s3 backend Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 22/45] api: reader: fetch chunks based on datastore backend Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 23/45] datastore: local chunk reader: read chunks based on backend Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 24/45] verify worker: add datastore backed to verify worker Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 25/45] verify: implement chunk verification for stores with s3 backend Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 26/45] datastore: create namespace marker in S3 backend Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 27/45] datastore: create/delete protected marker file on s3 storage backend Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 28/45] datastore: prune groups/snapshots from s3 object store backend Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 29/45] datastore: get and set owner for S3 " Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 30/45] datastore: implement garbage collection for s3 backend Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 31/45] ui: add datastore type selector and reorganize component layout Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 32/45] ui: add S3 client edit window for configuration create/edit Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 33/45] ui: add S3 client view for configuration Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 34/45] ui: expose the S3 client view in the navigation tree Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 35/45] ui: add s3 client selector and bucket field for s3 backend setup Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 36/45] tools: lru cache: add removed callback for evicted cache nodes Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 37/45] tools: async lru cache: implement insert, remove and contains methods Christian Ebner
2025-06-23 9:40 ` [pbs-devel] [PATCH proxmox-backup v4 38/45] datastore: add local datastore cache for network attached storages Christian Ebner
2025-06-23 9:41 ` [pbs-devel] [PATCH proxmox-backup v4 39/45] api: backup: use local datastore cache on s3 backend chunk upload Christian Ebner
2025-06-23 9:41 ` [pbs-devel] [PATCH proxmox-backup v4 40/45] api: reader: use local datastore cache on s3 backend chunk fetching Christian Ebner
2025-06-23 9:41 ` [pbs-devel] [PATCH proxmox-backup v4 41/45] api: backup: add no-cache flag to bypass local datastore cache Christian Ebner
2025-06-23 9:41 ` [pbs-devel] [PATCH proxmox-backup v4 42/45] api/datastore: implement refresh endpoint for stores with s3 backend Christian Ebner
2025-06-23 9:41 ` [pbs-devel] [PATCH proxmox-backup v4 43/45] cli: add dedicated subcommand for datastore s3 refresh Christian Ebner
2025-06-23 9:41 ` [pbs-devel] [PATCH proxmox-backup v4 44/45] ui: render s3 refresh as valid maintenance type and task description Christian Ebner
2025-06-23 9:41 ` [pbs-devel] [PATCH proxmox-backup v4 45/45] ui: expose s3 refresh button for datastores backed by object store Christian Ebner
2025-06-30 9:21 ` Christian Ebner [this message]
2025-07-01 14:30 ` [pbs-devel] [PATCH proxmox{, -backup} v4 00/48] fix #2943: S3 storage backend for datastores Hannes Duerr
2025-07-01 14:53 ` Christian Ebner
2025-07-03 13:20 ` [pbs-devel] superseded: " Christian Ebner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7e24b63f-fc37-4a9b-86c5-6b61a6b01f67@proxmox.com \
--to=c.ebner@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal