From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id 2D4A91FF165
for <inbox@lore.proxmox.com>; Wed, 12 Mar 2025 14:27:51 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id ECA055A9E;
Wed, 12 Mar 2025 14:27:41 +0100 (CET)
From: Dominik Csapak <d.csapak@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Wed, 12 Mar 2025 14:27:38 +0100
Message-Id: <20250312132738.2268305-1-d.csapak@proxmox.com>
X-Mailer: git-send-email 2.39.5
MIME-Version: 1.0
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.023 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
information. [anyevent.pm]
Subject: [pve-devel] [RFC PATCH http-server] fix #6230: increase allowed
post size
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
In some situations, e.g. having a large resource mapping, the UI can
generate a request that is bigger than the current limit of 64KiB.
Our files in pmxcfs can grow up to 1 MiB, so theoretically, a single
mapping can grow to that size. In practice, a single entry will have
much less. In #6230, a user has a mapping with about ~130KiB.
Increase the limit to 512KiB so we have a bit of buffer left.
We have to also increase the 'rbuf_max' size here, otherwise the request
will fail (since the buffer is too small for the request).
Since the post limit and the rbuf_max are tightly coupled, let it
reflect that in the code. To do that sum the post size + max header
size there.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
sending as RFC because:
* not sure about the rbuf_max calculation, but we have to increase it
when we increase $limit_max_post. (not sure how much is needed exactly)
* ther are alternative ways to deal with that, but some of those are vastly
more work:
- optimize the pci mapping to reduce the number of bytes we have to
send (e.g. by reducing the property names, or somehow magically
detect devices that belong together)
- add a new api for the mappings that can update the entries without
sending the whole mapping again (not sure if we can make this
backwards compatible)
- ignore the problem and simply tell the users to edit the file
manually (I don't like this one...)
also, I tried to benchmark this, but did not find a tool that does this
in a good way (e.g. apachebench complained about ssl, and i couldn't get
it to work right). @Thomas you did such benchmarks laft according to git
log, do you remember what you used then?
src/PVE/APIServer/AnyEvent.pm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/PVE/APIServer/AnyEvent.pm b/src/PVE/APIServer/AnyEvent.pm
index 8a52836..43ced75 100644
--- a/src/PVE/APIServer/AnyEvent.pm
+++ b/src/PVE/APIServer/AnyEvent.pm
@@ -52,7 +52,7 @@ use PVE::APIServer::Utils;
my $limit_max_headers = 64;
my $limit_max_header_size = 8*1024;
-my $limit_max_post = 64*1024;
+my $limit_max_post = 512*1024;
my $known_methods = {
GET => 1,
@@ -1891,7 +1891,7 @@ sub accept_connections {
$self->{conn_count}++;
$reqstate->{hdl} = AnyEvent::Handle->new(
fh => $clientfh,
- rbuf_max => 64*1024,
+ rbuf_max => $limit_max_post + ($limit_max_headers * $limit_max_header_size),
timeout => $self->{timeout},
linger => 0, # avoid problems with ssh - really needed ?
on_eof => sub {
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel