From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <pve-devel-bounces@lists.proxmox.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 6CFEC1FF16B for <inbox@lore.proxmox.com>; Thu, 3 Apr 2025 08:46:29 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 78B3338594; Thu, 3 Apr 2025 08:46:16 +0200 (CEST) Message-ID: <ddf1f40c-cfb9-4000-802b-3bc39042f057@proxmox.com> Date: Thu, 3 Apr 2025 08:46:12 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta To: Thomas Lamprecht <t.lamprecht@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com> References: <20250312132738.2268305-1-d.csapak@proxmox.com> <beaba25c-d7b0-40bb-865a-db8a9cd06790@proxmox.com> Content-Language: en-US From: Dominik Csapak <d.csapak@proxmox.com> In-Reply-To: <beaba25c-d7b0-40bb-865a-db8a9cd06790@proxmox.com> X-SPAM-LEVEL: Spam detection results: 0 AWL 0.021 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] [RFC PATCH http-server] fix #6230: increase allowed post size X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/> List-Post: <mailto:pve-devel@lists.proxmox.com> List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe> Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com> On 4/2/25 22:09, Thomas Lamprecht wrote: > Am 12.03.25 um 14:27 schrieb Dominik Csapak: >> In some situations, e.g. having a large resource mapping, the UI can >> generate a request that is bigger than the current limit of 64KiB. >> >> Our files in pmxcfs can grow up to 1 MiB, so theoretically, a single >> mapping can grow to that size. In practice, a single entry will have >> much less. In #6230, a user has a mapping with about ~130KiB. >> >> Increase the limit to 512KiB so we have a bit of buffer left. > > s/buffer/headroom/ ? > yes, makes more sense^^ >> >> We have to also increase the 'rbuf_max' size here, otherwise the request >> will fail (since the buffer is too small for the request). >> Since the post limit and the rbuf_max are tightly coupled, let it >> reflect that in the code. To do that sum the post size + max header >> size there. >> >> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> >> --- >> sending as RFC because: >> * not sure about the rbuf_max calculation, but we have to increase it >> when we increase $limit_max_post. (not sure how much is needed exactly) >> * ther are alternative ways to deal with that, but some of those are vastly >> more work: >> - optimize the pci mapping to reduce the number of bytes we have to >> send (e.g. by reducing the property names, or somehow magically >> detect devices that belong together) >> - add a new api for the mappings that can update the entries without >> sending the whole mapping again (not sure if we can make this >> backwards compatible) >> - ignore the problem and simply tell the users to edit the file >> manually (I don't like this one...) >> >> also, I tried to benchmark this, but did not find a tool that does this >> in a good way (e.g. apachebench complained about ssl, and i couldn't get >> it to work right). @Thomas you did such benchmarks laft according to git >> log, do you remember what you used then? > > argh, my commit message back then looks like I tried to write what I used > but then fubmled (or got knocked on the head) and sent it out unfinished. > To my defence, Wolfgang applied it ;P > > I'm not totally sure what I used back then, might have been something > custom-made too. FWIW, recently I used oha [0] and found it quite OK, albeit > I did not try it with POST data, but one can define the method and pass a > request body from CLI argument directly or a file, and it has a flag to > allow "insecure" TLS certs. > > [0]: https://github.com/hatoo/oha thanks, i'll try to do some benchmarks with it > >> @@ -1891,7 +1891,7 @@ sub accept_connections { >> $self->{conn_count}++; >> $reqstate->{hdl} = AnyEvent::Handle->new( >> fh => $clientfh, >> - rbuf_max => 64*1024, >> + rbuf_max => $limit_max_post + ($limit_max_headers * $limit_max_header_size), > > The header part is wrong as the header limits are independent, i.e., the > request must have less than $limit_max_headers separate headers and all > those together must be smaller than $limit_max_header_size. > > So just adding $limit_max_header_size is enough, no multiplication required. > ah yes, seems i read those wrong >> timeout => $self->{timeout}, >> linger => 0, # avoid problems with ssh - really needed ? >> on_eof => sub { _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel