From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 5F8801FF0E5 for ; Wed, 29 Jul 2026 14:16:56 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id A4E1220AF5; Wed, 29 Jul 2026 14:16:55 +0200 (CEST) Date: Wed, 29 Jul 2026 14:16:46 +0200 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= Subject: applied-series: [RFC proxmox{,-backup} 0/4] fix #6841: implement request rate limits for all methods To: Christian Ebner , pbs-devel@lists.proxmox.com References: <20260630142828.660821-1-c.ebner@proxmox.com> In-Reply-To: <20260630142828.660821-1-c.ebner@proxmox.com> MIME-Version: 1.0 User-Agent: astroid/0.17.0 (https://github.com/astroidmail/astroid) Message-Id: <1785327335.vv0r4zawwj.astroid@yuna.none> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1785327402205 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.135 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_LOW -0.7 Sender listed at https://www.dnswl.org/, low trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: LI6ICF7YZI7IZRVZ7JNE3B6ATREBU7QP X-Message-ID-Hash: LI6ICF7YZI7IZRVZ7JNE3B6ATREBU7QP X-MailFrom: f.gruenbichler@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On June 30, 2026 4:28 pm, Christian Ebner wrote: > Currently request performed by the s3 client to the backend can only > be limited by bandwidth limits or on a per-instance based PUT rate > limit. Providers might however enforce limits on the number of requests > per second, non of which can currently be enforced. >=20 > Therefore this patch series implements request limits, using a memory > mapped shared request limiter, similar to what is used to enforce > bandwidth limits. Since providers might have different limits as e.g. > the soft limits by AWS S3 mentioned in [0], allow to set different limits > between `active` and `passive` requests methods. >=20 > Sending as RFC as a bit unsure if we should make this configurable > as currently the case or rather trying to dynamically adapt the request > rate in case of requests failing with the respective HTTP status codes. > The latter has the downside that different providers may not clearly > signal when a rate limit is being enforced and/or have different status > codes to do so. Therefore the current approach allows to hard code the > limits. >=20 > Currently the rate limit is not yet updated on config changes via API, > only on s3 client instantiation. >=20 > [1] https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-perf= ormance.html >=20 > Reference to the issue in bugzilla: https://bugzilla.proxmox.com/show_bug= .cgi?id=3D6841 >=20 >=20 > proxmox: >=20 > Christian Ebner (2): > fix #6841: allow to set active/passive request rate limits > s3-client: add 429 HTTP status code as retryable >=20 > proxmox-s3-client/src/api_types.rs | 8 +++- > proxmox-s3-client/src/client.rs | 66 ++++++++++++++++++++++++++---- > 2 files changed, 66 insertions(+), 8 deletions(-) >=20 >=20 > proxmox-backup: >=20 > Christian Ebner (2): > s3: config: update or delete active/passive request limit options > ui: expose request rate limit config options for S3 endpoints >=20 > src/api2/config/s3.rs | 16 ++++++++++++++++ > www/window/S3ClientEdit.js | 14 ++++++++++++++ > 2 files changed, 30 insertions(+) >=20 >=20 > Summary over all repositories: > 4 files changed, 96 insertions(+), 8 deletions(-) >=20 > --=20 > Generated by murpp 0.11.0 >=20 >=20 >=20 >=20 >=20