From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pbs-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id 052921FF187
for <inbox@lore.proxmox.com>; Mon, 30 Jun 2025 11:21:06 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 0AF3F17899;
Mon, 30 Jun 2025 11:21:44 +0200 (CEST)
Message-ID: <7e24b63f-fc37-4a9b-86c5-6b61a6b01f67@proxmox.com>
Date: Mon, 30 Jun 2025 11:21:40 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
To: pbs-devel@lists.proxmox.com
References: <20250623094106.299251-1-c.ebner@proxmox.com>
Content-Language: en-US, de-DE
From: Christian Ebner <c.ebner@proxmox.com>
In-Reply-To: <20250623094106.299251-1-c.ebner@proxmox.com>
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.038 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: Re: [pbs-devel] [PATCH proxmox{,
-backup} v4 00/48] fix #2943: S3 storage backend for datastores
X-BeenThere: pbs-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Backup Server development discussion
<pbs-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>,
<mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/>
List-Post: <mailto:pbs-devel@lists.proxmox.com>
List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>,
<mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox Backup Server development discussion
<pbs-devel@lists.proxmox.com>
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Errors-To: pbs-devel-bounces@lists.proxmox.com
Sender: "pbs-devel" <pbs-devel-bounces@lists.proxmox.com>
Some further, real-world testing using a Cloudflare R2 object store as
backend for the datastore reveals that request rate limiting can still
be an issue. These were not an issue with locally hosted object stores,
e.g. via Ceph RADOS gateway.
While the Cloudflare documentation specifies limits for some bucket
management operations [0], these should not apply to object write
operations. However, in my test case requests are returning with an 502
bad gateway response, reduced in occurrence if the upload to the PBS is
limited via e.g. traffic control rate limits.
Will look into how to best implement request limits in the s3 client
itself, ideally with a mechanism to dynamically adapt the rate in case
of such object upload errors, including a retry logic.
[0] https://developers.cloudflare.com/r2/platform/limits/
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel