From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id 747AE1FF16B
for <inbox@lore.proxmox.com>; Thu, 3 Apr 2025 10:28:46 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 76D5C39FA6;
Thu, 3 Apr 2025 10:28:33 +0200 (CEST)
From: Dominik Csapak <d.csapak@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Thu, 3 Apr 2025 10:27:59 +0200
Message-Id: <20250403082759.2506153-1-d.csapak@proxmox.com>
X-Mailer: git-send-email 2.39.5
MIME-Version: 1.0
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.022 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: [pve-devel] [PATCH http-server] fix #6230: increase allowed post
size
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
In some situations, e.g. having a large resource mapping, the UI can
generate a request that is bigger than the current limit of 64KiB.
Our files in pmxcfs can grow up to 1 MiB, so theoretically, a single
mapping can grow to that size. In practice, a single entry will have
much less. In #6230, a user has a mapping with about ~130KiB.
Increase the limit to 512KiB so we have a bit of headroom left.
We have to also increase the 'rbuf_max' size here, otherwise the request
will fail (since the buffer is too small for the request). Since the
post limit and the rbuf_max are tightly coupled, let it reflect that in
the code. To do that sum the post size + max header size there.
A short benchmark shows that it only slightly impacts performance for
the same amount of data (but that could be runtime variance too):
I used a 4 node virtualized cluster, benchmarked with oha[0] with these
options:
oha --insecure -H $COOKIE -H $CSRFTOKEN -D bodyfile -m "PUT" -T
"application/x-www-form-urlencoded" -n 3000 -c 50 --disable-keepalive
--latency-correction https://<IP>:8006/api2/json/cluster/mapping/pci/test
So 3000 requests with 50 parallel. I also restarted pveproxy and daemon
in between runs, and took the rss values around the 50% runtime of the
benchmark.
average time requests/s pvedaemon rss pveproxy rss
old with 60k body 3.0067s 16.3487 140M-155M 141M-170M
new with 60k body 3.0865s 15.7623 140M-155M 141M-171M
new with 180k body 8.3834s 5.8934 140M-158M 141M-181M
Using a bigger body size had a large impact on the time, but that's IMHO
expected. Also, RSS is not that much impacted, only when using many
requests with larger request size, but this should also be expected.
0: https://github.com/hatoo/oha
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from rfc:
* s/buffer/headroom
* added benchmark data to the commit message
* corrected rbuf_max calculation
src/PVE/APIServer/AnyEvent.pm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/PVE/APIServer/AnyEvent.pm b/src/PVE/APIServer/AnyEvent.pm
index 8a52836..7499474 100644
--- a/src/PVE/APIServer/AnyEvent.pm
+++ b/src/PVE/APIServer/AnyEvent.pm
@@ -52,7 +52,7 @@ use PVE::APIServer::Utils;
my $limit_max_headers = 64;
my $limit_max_header_size = 8*1024;
-my $limit_max_post = 64*1024;
+my $limit_max_post = 512*1024;
my $known_methods = {
GET => 1,
@@ -1891,7 +1891,7 @@ sub accept_connections {
$self->{conn_count}++;
$reqstate->{hdl} = AnyEvent::Handle->new(
fh => $clientfh,
- rbuf_max => 64*1024,
+ rbuf_max => $limit_max_post + $limit_max_header_size,
timeout => $self->{timeout},
linger => 0, # avoid problems with ssh - really needed ?
on_eof => sub {
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel