From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id A1FC01FF15C for ; Fri, 5 Sep 2025 15:49:18 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 0D87F15E6B; Fri, 5 Sep 2025 15:49:33 +0200 (CEST) Message-ID: <7fcdb7f7-9a30-4e3b-abcb-1971c2004f05@proxmox.com> Date: Fri, 5 Sep 2025 15:49:29 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Proxmox VE development discussion , Dominik Csapak References: <20250905120627.2585826-1-d.csapak@proxmox.com> <20250905120627.2585826-2-d.csapak@proxmox.com> From: Aaron Lauterer In-Reply-To: <20250905120627.2585826-2-d.csapak@proxmox.com> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1757080151912 X-SPAM-LEVEL: Spam detection results: 0 AWL -1.164 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_MAILER 2 Automated Mailer Tag Left in Email POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [api2tools.pm] Subject: Re: [pve-devel] [PATCH manager 1/4] API2Tools: rrd: remove O(n^2) lookup for keys X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" On 2025-09-05 14:06, Dominik Csapak wrote: > the idea was that we get any of the 'new' versions on lookup, but that > lead to iterating through possibly all keys. Since that was called for > each resource in e.g. /cluster/resources api call, the runtime was > O(n^2) for the number of resources. > > To avoid that, simply look up the currently only valid key here which > makes this lookup much cheaper. > > In my test setup with ~10000 guests, it reduces the time for a call > to /cluster/resources from ~22s to ~400ms > > Signed-off-by: Dominik Csapak > --- > NOTE: this is only a workaround and Aaron is working to making this > whole part unnecessary, but for now i think this is a good stop-gap > > PVE/API2Tools.pm | 7 +++---- > 1 file changed, 3 insertions(+), 4 deletions(-) > > diff --git a/PVE/API2Tools.pm b/PVE/API2Tools.pm > index abe73fa2..863f5f55 100644 > --- a/PVE/API2Tools.pm > +++ b/PVE/API2Tools.pm > @@ -52,10 +52,9 @@ sub get_rrd_key { > return "pve2.3-${type}/${id}"; > } > > - # if no old key has been found, we expect on in the newer format: pve-{type}-{version}/{id} > - # We accept all new versions, as the expectation is that they are only allowed to add new colums as non-breaking change > - for my $k (keys %$rrd) { > - return $k if $k =~ m/^pve-\Q${type}\E-\d\d?.\d\/\Q${id}\E$/; > + my $key = "pve-${type}-9.0/${id}"; > + if (defined($rrd->{$key})) { > + return $key; Have you tested this in a mixed PVE8 + PVE9 cluster? This might break it and we might have to keep the dynamic checking around. I don't have too much time right now. But I think this would break a mixed version situation, e.g. during a cluster upgrade, where we receive both keys, pve2-... and pve-...-9.0 ones. > } > } > _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel