From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 2CE7E1FF0C0 for ; Wed, 18 Nov 2026 15:28:22 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 95994218EB; Wed, 18 Nov 2026 15:26:33 +0100 (CET) From: Daniel Kral To: pve-devel@lists.proxmox.com Subject: [PATCH ha-manager v3 14/17] lrm: read service config once per work iteration Date: Wed, 18 Nov 2026 15:25:53 +0100 Message-ID: <20261118142556.218589-15-d.kral@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261118142556.218589-1-d.kral@proxmox.com> References: <20261118142556.218589-1-d.kral@proxmox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1795011970018 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.503 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: HNKMBZLO2TMJETZQ3AATF3BKWQNWAUVR X-Message-ID-Hash: HNKMBZLO2TMJETZQ3AATF3BKWQNWAUVR X-MailFrom: d.kral@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: The amount of times the LRM reads and parses the HA resource config primarily scales with the amount of HA resource command exit codes that need to be handled. This can become quite expensive especially with many HA resources in 'started' state, where the HA Manager issues an HA resource command on every HA Manager round. In the PVE2 environment, the HA resource config and vmlist is cached between PVE::Cluster::cfs_update() calls, and in the Test environment Sim Hardware commands can only change the HA resource config outside of the LRM round window. Therefore, the return value of read_service_config() can be cached inbetween and must only be invalidated again once a new LRM round starts. This significantly reduces the time spent reading and parsing the HA resource config every time run_workers() is called and every time a LRM worker's result is collected with handle_service_exitcode(). A test startup of 768 HA resources on a single node at the same time showed ~99.2% fewer calls to checked_resources_config() (from 10,333 to 82 calls) and ~99.3% fewer calls to parse_sid() (from 15,639,537 to 109,198 calls). Additionally, this reduces the inclusive average execution time per call for several subroutines where the HA resource config was read within: +---------------------------+---------------+--------------+ | Subroutine | Before Change | After Change | +---------------------------+---------------+--------------+ | check_active_workers | 327ms | 11.6ms | | resource_command_finished | 22.7ms | 2.81ms | | handle_service_exitcode | 21.1ms | 10µs | +---------------------------+---------------+--------------+ The profiling was done on a physical host with Devel::NYTProf 6.14 [0]. [0] https://metacpan.org/pod/Devel::NYTProf Signed-off-by: Daniel Kral Reviewed-by: Michael Köppl Tested-by: Michael Köppl --- src/PVE/HA/LRM.pm | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm index 6d7be883..ad3bbbb6 100644 --- a/src/PVE/HA/LRM.pm +++ b/src/PVE/HA/LRM.pm @@ -41,6 +41,9 @@ sub new { mode => 'active', cluster_state_update => 0, active_idle_rounds => 0, + # service config cache in current work() iteration + # does contain all HA resources, users must filter for the assigned node + service_config => undef, # service_status from manager_status in current work() iteration # does contain all HA resources, users must filter for the assigned node service_status => {}, @@ -247,6 +250,22 @@ sub update_lrm_status { } } +sub read_service_config { + my ($self) = @_; + + return $self->{service_config} if defined($self->{service_config}); + + $self->{service_config} = $self->{haenv}->read_service_config(); + + return $self->{service_config}; +} + +sub invalidate_service_config_cache { + my ($self) = @_; + + undef $self->{service_config}; +} + sub get_protected_ha_agent_lock { my ($self) = @_; @@ -349,6 +368,8 @@ sub do_one_iteration { $self->{cluster_state_update} = $haenv->cluster_state_update(); + $self->invalidate_service_config_cache(); + my $res = $self->work(); $haenv->loop_end_hook(); @@ -668,7 +689,7 @@ sub run_workers { # number of workers to start, if 0 we exec the command directly without forking my $max_workers = $haenv->get_max_workers(); - my $sc = $haenv->read_service_config(); + my $sc = $self->read_service_config(); my $worker = $self->{workers}; # we only got limited time but want to ensure that every queued worker is scheduled @@ -916,7 +937,7 @@ sub handle_service_exitcode { my $haenv = $self->{haenv}; my $tries = $self->{restart_tries}; - my $sc = $haenv->read_service_config(); + my $sc = $self->read_service_config(); my $max_restart = 0; -- 2.47.3