* [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements
@ 2026-11-18 14:25 Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 01/17] resources: remove commented ipaddr resource type Daniel Kral
` (17 more replies)
0 siblings, 18 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
Changes since v2 (thanks a lot @Michael!):
- add R-b and T-b from @Michael to patches except #2 and #9
- drop s/$name/$vmid/ from PVE::HA::LRM and PVE::HA::Resources in #2
- new patch #9: use an eval around the maintenance work branch, similar
to what is catched in the active work branch
Before it gets unnoticed: A user reported some results for v1 of this
patch series, which shows a test on another larger setup [0].
Changes since v1:
- rebase on master
- cleanup the patch messages a bit
- cache the read_service_config() instead of putting it in the 'new'
update_lrm_status() to not change the current behavior as well
unnecessarily reading the HA resource config in idle rounds
- do not prune LRM results in flush_lrm_status() but properly prune
these in update_lrm_status() and resource_command_finished(), because
flush_lrm_status() could be called by a signal handler calling
shutdown_request()
v1: https://lore.proxmox.com/pve-devel/20260720123705.216089-1-d.kral@proxmox.com/
v2: https://lore.proxmox.com/pve-devel/20261112163113.294302-3-d.kral@proxmox.com/
It was reported in the enterprise support, that nodes with high counts
of HA resources (250+ HA resources) cause the pve-ha-lrm daemon to
consumes quite a lot of CPU time while being idle. Furthermore,
migration commands for HA resources might take quite a long time to be
actually committed when the max_workers value is not high enough.
This smaller patch series tries to mitigate the former issue first,
which consists of making some performance improvements for the hot LRM
code paths, which consumed the most resources:
- the service config is read and validated very often
- because of that, parse_sid() is called many times as well
- other smaller loops are redundant and are moved out of the hot paths
I started out with making parse_sid() a little lighter, though because
of the latter changes the performance bottleneck of parse_sid() became
rather meaningless. I left the refactoring and changes in though as
having a stricter separation where both VMIDs and SIDs can be used and
where only SIDs can be used is a little nicer now.
The patch series consists of:
PATCH 1-2 small cleanups
PATCH 3-8 refactoring and splitting up parse_sid() and its users
PATCH 9 eval block around maintenance work branch
PATCH 10-12 prep work to add documentation and rename some methods
PATCH 13 reading the manager status only once per work iteration
PATCH 14 reading the service config only once per work iteration
PATCH 15-17 factoring out loops which are invariant per work iteration
A test startup of 768 HA resources on a single physical host at the same
time, profiling with Devel::NYTProf 6.14 [0] showed the following:
+---------------------------+---------------+--------------+
| Subroutine | Before Change | After Change |
+---------------------------+---------------+--------------+
| check_active_workers | 327ms | 998µs |
| resource_command_finished | 22.7ms | 50µs |
| handle_service_exitcode | 21.1ms | 12µs |
+---------------------------+---------------+--------------+
Considering that these are called quite often per LRM cycle, these drops
in execution time per call are significant and reduce how much CPU% the
pve-ha-lrm main thread uses.
[0] https://bugzilla.proxmox.com/show_bug.cgi?id=8233
Daniel Kral (17):
resources: remove commented ipaddr resource type
tree-wide: use the term vmid instead of name when referring to VMs/CTs
tree-wide: remove unused return values at parse_sid callsites
config: make update_single_resource_config_inplace only allow sids
make parse_sid always return an array
config: use early returns in parse_sid
config: introduce separate parse_vmid_or_sid helper subroutine
drop the unmodified sid return array entry value from parse_sid
lrm: catch and report dies while the LRM is in maintenance state
lrm: rename update_lrm_status to flush_lrm_status
lrm: rename update_service_status to update_lrm_status
lrm: document and initialize LRM instance properties
lrm: update the service status once per work iteration
lrm: read service config once per work iteration
lrm: compute valid service uids hash set once per work iteration
lrm: resource_command_finished: avoid writing stale LRM results
lrm: prune stale LRM results once per work iteration
src/PVE/API2/HA/Resources.pm | 17 +--
src/PVE/CLI/ha_manager.pm | 6 +-
src/PVE/HA/Config.pm | 42 +++----
src/PVE/HA/Env/PVE2.pm | 6 +-
src/PVE/HA/LRM.pm | 128 ++++++++++++++--------
src/PVE/HA/Manager.pm | 4 +-
src/PVE/HA/Resources.pm | 26 -----
src/PVE/HA/Resources/PVECT.pm | 4 +-
src/PVE/HA/Resources/PVEVM.pm | 4 +-
src/PVE/HA/Sim/Env.pm | 4 +-
src/test/test-cfs-unavailable1/log.expect | 5 -
src/test/test-group-migrate4/log.expect | 10 --
12 files changed, 130 insertions(+), 126 deletions(-)
--
2.47.3
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 01/17] resources: remove commented ipaddr resource type
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 02/17] tree-wide: use the term vmid instead of name when referring to VMs/CTs Daniel Kral
` (16 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
Since 85d36c36 ("remove ipaddr resource type") the ipaddr resource type
has been commented out, which indicates that it is not intended to be
used and should be removed.
Even if it might be used in the future, it would need more rework to fit
in the current HA stack.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/HA/Resources.pm | 26 --------------------------
1 file changed, 26 deletions(-)
diff --git a/src/PVE/HA/Resources.pm b/src/PVE/HA/Resources.pm
index df7c1ff3..e63d78d8 100644
--- a/src/PVE/HA/Resources.pm
+++ b/src/PVE/HA/Resources.pm
@@ -189,30 +189,4 @@ sub get_static_stats_from_config {
die "implement in subclass";
}
-# package PVE::HA::Resources::IPAddr;
-
-# use strict;
-# use warnings;
-# use PVE::Tools qw($IPV4RE $IPV6RE);
-
-# use base qw(PVE::HA::Resources);
-
-# sub type {
-# return 'ipaddr';
-# }
-
-# sub verify_name {
-# my ($class, $name) = @_;
-
-# die "invalid IP address\n" if $name !~ m!^$IPV6RE|$IPV4RE$!;
-# }
-
-# sub options {
-# return {
-# state => { optional => 1 },
-# group => { optional => 1 },
-# comment => { optional => 1 },
-# };
-# }
-
1;
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 02/17] tree-wide: use the term vmid instead of name when referring to VMs/CTs
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 01/17] resources: remove commented ipaddr resource type Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 03/17] tree-wide: remove unused return values at parse_sid callsites Daniel Kral
` (15 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
The variable name $name is a bit generic when referring to the unique
identifier for VMs/CTs in code paths where only VMIDs can be expected,
so actually name it after its identifier VMID.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
src/PVE/API2/HA/Resources.pm | 14 +++++++-------
src/PVE/CLI/ha_manager.pm | 4 ++--
src/PVE/HA/Config.pm | 20 ++++++++++----------
src/PVE/HA/Env/PVE2.pm | 6 +++---
src/PVE/HA/Resources/PVECT.pm | 4 ++--
src/PVE/HA/Resources/PVEVM.pm | 4 ++--
src/PVE/HA/Sim/Env.pm | 4 ++--
7 files changed, 28 insertions(+), 28 deletions(-)
diff --git a/src/PVE/API2/HA/Resources.pm b/src/PVE/API2/HA/Resources.pm
index b96572b9..f786e435 100644
--- a/src/PVE/API2/HA/Resources.pm
+++ b/src/PVE/API2/HA/Resources.pm
@@ -198,7 +198,7 @@ __PACKAGE__->register_method({
PVE::Cluster::check_cfs_quorum();
mkdir("/etc/pve/ha");
- my ($sid, $type, $name) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid, $type, $vmid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
if (my $param_type = extract_param($param, 'type')) {
# useless, but do it anyway
@@ -209,9 +209,9 @@ __PACKAGE__->register_method({
if defined($param->{group}) && PVE::HA::Config::have_groups_been_migrated();
my $plugin = PVE::HA::Resources->lookup($type);
- $plugin->verify_name($name);
+ $plugin->verify_name($vmid);
- $plugin->exists($name);
+ $plugin->exists($vmid);
my $opts = $plugin->check_config($sid, $param, 1, 1);
@@ -253,7 +253,7 @@ __PACKAGE__->register_method({
my $digest = extract_param($param, 'digest');
my $delete = extract_param($param, 'delete');
- my ($sid, $type, $name) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid, $type, $vmid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
if (my $param_type = extract_param($param, 'type')) {
# useless, but do it anyway
@@ -313,7 +313,7 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- my ($sid, $type, $name) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid, $type, $vmid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
my $purge = extract_param($param, 'purge') // 1;
if (!PVE::HA::Config::service_is_configured($sid)) {
@@ -401,7 +401,7 @@ __PACKAGE__->register_method({
my $result = {};
- my ($sid, $type, $name) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid, $type, $vmid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
my $req_node = extract_param($param, 'node');
PVE::HA::Config::service_is_ha_managed($sid);
@@ -503,7 +503,7 @@ __PACKAGE__->register_method({
my $result = {};
- my ($sid, $type, $name) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid, $type, $vmid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
my $req_node = extract_param($param, 'node');
PVE::HA::Config::service_is_ha_managed($sid);
diff --git a/src/PVE/CLI/ha_manager.pm b/src/PVE/CLI/ha_manager.pm
index 6625de68..2f89887f 100644
--- a/src/PVE/CLI/ha_manager.pm
+++ b/src/PVE/CLI/ha_manager.pm
@@ -193,8 +193,8 @@ our $cmddef = {
sub {
my $res = shift;
foreach my $rec (sort { $a->{sid} cmp $b->{sid} } @$res) {
- my ($type, $name) = split(':', $rec->{sid}, 2);
- print "$type:$name\n";
+ my ($type, $vmid) = split(':', $rec->{sid}, 2);
+ print "$type:$vmid\n";
foreach my $k (sort keys %$rec) {
next
if $k eq 'digest'
diff --git a/src/PVE/HA/Config.pm b/src/PVE/HA/Config.pm
index bee1ac7d..ec0551e9 100644
--- a/src/PVE/HA/Config.pm
+++ b/src/PVE/HA/Config.pm
@@ -114,7 +114,7 @@ my sub checked_resources_config {
foreach my $sid (keys %{ $cfg->{ids} }) {
my $d = $cfg->{ids}->{$sid};
- my (undef, undef, $name) = parse_sid($sid);
+ my (undef, undef, $vmid) = parse_sid($sid);
$d->{state} = 'started' if !defined($d->{state});
$d->{state} = 'started' if $d->{state} eq 'enabled'; # backward compatibility
$d->{failback} = 1 if !defined($d->{failback});
@@ -123,7 +123,7 @@ my sub checked_resources_config {
$d->{max_relocate} = 1 if !defined($d->{max_relocate});
if (PVE::HA::Resources->lookup($d->{type})) {
- if (my $vmd = $vmlist->{ids}->{$name}) {
+ if (my $vmd = $vmlist->{ids}->{$vmid}) {
$d->{node} = $vmd->{node};
$resources->{$sid} = $d;
} else {
@@ -148,7 +148,7 @@ sub read_and_check_resources_config {
my sub update_single_resource_config_inplace {
my ($cfg, $sid, $param, $delete) = @_;
- ($sid, my $type, my $name) = parse_sid($sid);
+ ($sid, my $type, my $vmid) = parse_sid($sid);
my $scfg = $cfg->{ids}->{$sid}
|| die "no such resource '$sid'\n";
@@ -198,27 +198,27 @@ sub update_resources_config {
sub parse_sid {
my ($sid) = @_;
- my ($type, $name);
+ my ($type, $vmid);
if ($sid =~ m/^(\d+)$/) {
- $name = $1;
+ $vmid = $1;
my $vmlist = PVE::Cluster::get_vmlist();
- if (defined($vmlist->{ids}->{$name})) {
- my $vm_type = $vmlist->{ids}->{$name}->{type};
+ if (defined($vmlist->{ids}->{$vmid})) {
+ my $vm_type = $vmlist->{ids}->{$vmid}->{type};
$type = PVE::HA::Tools::get_ha_resource_type($vm_type);
- $sid = "$type:$name";
+ $sid = "$type:$vmid";
} else {
die "unable to detect SID from VMID - VM/CT $1 does not exist\n";
}
} elsif ($sid =~ m/^(\S+):(\S+)$/) {
- $name = $2;
+ $vmid = $2;
$type = $1;
} else {
die "unable to parse service id '$sid'\n";
}
- return wantarray ? ($sid, $type, $name) : $sid;
+ return wantarray ? ($sid, $type, $vmid) : $sid;
}
sub read_rules_config {
diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index 5f195daf..f4d2b752 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -209,11 +209,11 @@ sub exec_fence_agent {
sub steal_service {
my ($self, $sid, $current_node, $new_node) = @_;
- my (undef, $type, $name) = PVE::HA::Config::parse_sid($sid);
+ my (undef, $type, $vmid) = PVE::HA::Config::parse_sid($sid);
if (my $plugin = PVE::HA::Resources->lookup($type)) {
- my $old = $plugin->config_file($name, $current_node);
- my $new = $plugin->config_file($name, $new_node);
+ my $old = $plugin->config_file($vmid, $current_node);
+ my $new = $plugin->config_file($vmid, $new_node);
rename($old, $new)
|| die "rename '$old' to '$new' failed - $!\n";
} else {
diff --git a/src/PVE/HA/Resources/PVECT.pm b/src/PVE/HA/Resources/PVECT.pm
index 177b9070..5121bbe1 100644
--- a/src/PVE/HA/Resources/PVECT.pm
+++ b/src/PVE/HA/Resources/PVECT.pm
@@ -27,9 +27,9 @@ sub type {
}
sub verify_name {
- my ($class, $name) = @_;
+ my ($class, $vmid) = @_;
- die "invalid VMID\n" if $name !~ m/^[1-9][0-9]+$/;
+ die "invalid VMID\n" if $vmid !~ m/^[1-9][0-9]+$/;
}
sub options {
diff --git a/src/PVE/HA/Resources/PVEVM.pm b/src/PVE/HA/Resources/PVEVM.pm
index 87532716..d6595da4 100644
--- a/src/PVE/HA/Resources/PVEVM.pm
+++ b/src/PVE/HA/Resources/PVEVM.pm
@@ -27,9 +27,9 @@ sub type {
}
sub verify_name {
- my ($class, $name) = @_;
+ my ($class, $vmid) = @_;
- die "invalid VMID\n" if $name !~ m/^[1-9][0-9]+$/;
+ die "invalid VMID\n" if $vmid !~ m/^[1-9][0-9]+$/;
}
sub options {
diff --git a/src/PVE/HA/Sim/Env.pm b/src/PVE/HA/Sim/Env.pm
index 38ac1d03..d3c4a11d 100644
--- a/src/PVE/HA/Sim/Env.pm
+++ b/src/PVE/HA/Sim/Env.pm
@@ -231,10 +231,10 @@ sub parse_sid {
die "unable to parse service id '$sid'\n"
if !($sid =~ m/^(\S+):(\S+)$/);
- my $name = $2;
+ my $vmid = $2;
my $type = $1;
- return wantarray ? ($sid, $type, $name) : $sid;
+ return wantarray ? ($sid, $type, $vmid) : $sid;
}
sub read_fence_config {
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 03/17] tree-wide: remove unused return values at parse_sid callsites
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 01/17] resources: remove commented ipaddr resource type Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 02/17] tree-wide: use the term vmid instead of name when referring to VMs/CTs Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 04/17] config: make update_single_resource_config_inplace only allow sids Daniel Kral
` (14 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
Remove the unused return values so it's easier to see which return
values are actually needed by the specific call sites.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/API2/HA/Resources.pm | 8 ++++----
src/PVE/HA/Config.pm | 2 +-
src/PVE/HA/Manager.pm | 2 +-
3 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/src/PVE/API2/HA/Resources.pm b/src/PVE/API2/HA/Resources.pm
index f786e435..5c2b4551 100644
--- a/src/PVE/API2/HA/Resources.pm
+++ b/src/PVE/API2/HA/Resources.pm
@@ -253,7 +253,7 @@ __PACKAGE__->register_method({
my $digest = extract_param($param, 'digest');
my $delete = extract_param($param, 'delete');
- my ($sid, $type, $vmid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid, $type) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
if (my $param_type = extract_param($param, 'type')) {
# useless, but do it anyway
@@ -313,7 +313,7 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- my ($sid, $type, $vmid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
my $purge = extract_param($param, 'purge') // 1;
if (!PVE::HA::Config::service_is_configured($sid)) {
@@ -401,7 +401,7 @@ __PACKAGE__->register_method({
my $result = {};
- my ($sid, $type, $vmid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
my $req_node = extract_param($param, 'node');
PVE::HA::Config::service_is_ha_managed($sid);
@@ -503,7 +503,7 @@ __PACKAGE__->register_method({
my $result = {};
- my ($sid, $type, $vmid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
my $req_node = extract_param($param, 'node');
PVE::HA::Config::service_is_ha_managed($sid);
diff --git a/src/PVE/HA/Config.pm b/src/PVE/HA/Config.pm
index ec0551e9..84102592 100644
--- a/src/PVE/HA/Config.pm
+++ b/src/PVE/HA/Config.pm
@@ -148,7 +148,7 @@ sub read_and_check_resources_config {
my sub update_single_resource_config_inplace {
my ($cfg, $sid, $param, $delete) = @_;
- ($sid, my $type, my $vmid) = parse_sid($sid);
+ ($sid, my $type) = parse_sid($sid);
my $scfg = $cfg->{ids}->{$sid}
|| die "no such resource '$sid'\n";
diff --git a/src/PVE/HA/Manager.pm b/src/PVE/HA/Manager.pm
index 5840a76e..4510dd44 100644
--- a/src/PVE/HA/Manager.pm
+++ b/src/PVE/HA/Manager.pm
@@ -283,7 +283,7 @@ sub load_balance {
my ($sid, $source, $target) = $migration->@{qw(sid source-node target-node)};
- my (undef, $type, $id) = $haenv->parse_sid($sid);
+ my (undef, $type) = $haenv->parse_sid($sid);
my $task = $type eq 'vm' ? "migrate" : "relocate";
my $cmd = "$task $sid $target";
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 04/17] config: make update_single_resource_config_inplace only allow sids
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (2 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 03/17] tree-wide: remove unused return values at parse_sid callsites Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 05/17] make parse_sid always return an array Daniel Kral
` (13 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
update_single_resource_config_inplace() has used the $sid return value
from parse_sid(), though all callers already provide the SID format and
do not provide the external VMID at any time.
The subroutine is mainly called internally to persist the HA resource
config state, e.g. when stopping an HA resource or migrating the HA
groups, and is only called externally in
PVE::API2::HA::Resources->update(), where the SID of a provided VMID is
already determined in advance.
Therefore, using the $sid return value is not needed here.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/HA/Config.pm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/PVE/HA/Config.pm b/src/PVE/HA/Config.pm
index 84102592..07290eee 100644
--- a/src/PVE/HA/Config.pm
+++ b/src/PVE/HA/Config.pm
@@ -148,7 +148,7 @@ sub read_and_check_resources_config {
my sub update_single_resource_config_inplace {
my ($cfg, $sid, $param, $delete) = @_;
- ($sid, my $type) = parse_sid($sid);
+ (undef, my $type) = parse_sid($sid);
my $scfg = $cfg->{ids}->{$sid}
|| die "no such resource '$sid'\n";
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 05/17] make parse_sid always return an array
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (3 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 04/17] config: make update_single_resource_config_inplace only allow sids Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 06/17] config: use early returns in parse_sid Daniel Kral
` (12 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
parse_sid() is only called within the pve-ha-manager package and there
are only two callers, which only need the $sid return value.
In preparation of the changes in the following patches and reduce the
clutter introduced by a wantarray, always return an array and adapt its
callers as $sid is already the first value in the returned array.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/API2/HA/Resources.pm | 2 +-
src/PVE/CLI/ha_manager.pm | 2 +-
src/PVE/HA/Config.pm | 2 +-
src/PVE/HA/Sim/Env.pm | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/PVE/API2/HA/Resources.pm b/src/PVE/API2/HA/Resources.pm
index 5c2b4551..a1fa8856 100644
--- a/src/PVE/API2/HA/Resources.pm
+++ b/src/PVE/API2/HA/Resources.pm
@@ -174,7 +174,7 @@ __PACKAGE__->register_method({
my $cfg = PVE::HA::Config::read_resources_config();
my $exclude_group_property = PVE::HA::Config::have_groups_been_migrated();
- my $sid = PVE::HA::Config::parse_sid($param->{sid});
+ my ($sid) = PVE::HA::Config::parse_sid($param->{sid});
return &$api_copy_config($cfg, $sid, $exclude_group_property);
},
diff --git a/src/PVE/CLI/ha_manager.pm b/src/PVE/CLI/ha_manager.pm
index 2f89887f..2bec33cb 100644
--- a/src/PVE/CLI/ha_manager.pm
+++ b/src/PVE/CLI/ha_manager.pm
@@ -104,7 +104,7 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- my $sid = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
PVE::HA::Config::service_is_ha_managed($sid);
diff --git a/src/PVE/HA/Config.pm b/src/PVE/HA/Config.pm
index 07290eee..be535329 100644
--- a/src/PVE/HA/Config.pm
+++ b/src/PVE/HA/Config.pm
@@ -218,7 +218,7 @@ sub parse_sid {
die "unable to parse service id '$sid'\n";
}
- return wantarray ? ($sid, $type, $vmid) : $sid;
+ return ($sid, $type, $vmid);
}
sub read_rules_config {
diff --git a/src/PVE/HA/Sim/Env.pm b/src/PVE/HA/Sim/Env.pm
index d3c4a11d..311eaad1 100644
--- a/src/PVE/HA/Sim/Env.pm
+++ b/src/PVE/HA/Sim/Env.pm
@@ -234,7 +234,7 @@ sub parse_sid {
my $vmid = $2;
my $type = $1;
- return wantarray ? ($sid, $type, $vmid) : $sid;
+ return ($sid, $type, $vmid);
}
sub read_fence_config {
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 06/17] config: use early returns in parse_sid
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (4 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 05/17] make parse_sid always return an array Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 07/17] config: introduce separate parse_vmid_or_sid helper subroutine Daniel Kral
` (11 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
While at it, also make use of the returned matches in list context by
assigning them directly to the variables instead of the special $1, $2,
... variables.
This is in preparation of an upcoming patch, which moves the conditional
vmid parsing and info-fetching branch to its separate helper subroutine.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/HA/Config.pm | 14 +++++---------
1 file changed, 5 insertions(+), 9 deletions(-)
diff --git a/src/PVE/HA/Config.pm b/src/PVE/HA/Config.pm
index be535329..25465486 100644
--- a/src/PVE/HA/Config.pm
+++ b/src/PVE/HA/Config.pm
@@ -200,25 +200,21 @@ sub parse_sid {
my ($type, $vmid);
- if ($sid =~ m/^(\d+)$/) {
- $vmid = $1;
-
+ if (($vmid) = $sid =~ m/^(\d+)$/) {
my $vmlist = PVE::Cluster::get_vmlist();
if (defined($vmlist->{ids}->{$vmid})) {
my $vm_type = $vmlist->{ids}->{$vmid}->{type};
$type = PVE::HA::Tools::get_ha_resource_type($vm_type);
$sid = "$type:$vmid";
+ return ($sid, $type, $vmid);
} else {
- die "unable to detect SID from VMID - VM/CT $1 does not exist\n";
+ die "unable to detect SID from VMID - VM/CT $vmid does not exist\n";
}
- } elsif ($sid =~ m/^(\S+):(\S+)$/) {
- $vmid = $2;
- $type = $1;
+ } elsif (($type, $vmid) = $sid =~ m/^(\S+):(\S+)$/) {
+ return ($sid, $type, $vmid);
} else {
die "unable to parse service id '$sid'\n";
}
-
- return ($sid, $type, $vmid);
}
sub read_rules_config {
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 07/17] config: introduce separate parse_vmid_or_sid helper subroutine
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (5 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 06/17] config: use early returns in parse_sid Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 08/17] drop the unmodified sid return array entry value from parse_sid Daniel Kral
` (10 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
The separate parse_vmid_or_sid() subroutine helps in stricter checking
that vmid references are only allowed in the API/CLI endpoints, while
the HA stack should always use the proper HA resource id ('type:vmid')
format internally.
This makes the parse_sid() subroutine slightly faster (by about 10%) as
it doesn't unnecessarily need to go through the vmid pattern matching.
The use of qr// was considered but increased the runtime again.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/API2/HA/Resources.pm | 13 +++++++------
src/PVE/CLI/ha_manager.pm | 2 +-
src/PVE/HA/Config.pm | 18 ++++++++++++------
3 files changed, 20 insertions(+), 13 deletions(-)
diff --git a/src/PVE/API2/HA/Resources.pm b/src/PVE/API2/HA/Resources.pm
index a1fa8856..7f910818 100644
--- a/src/PVE/API2/HA/Resources.pm
+++ b/src/PVE/API2/HA/Resources.pm
@@ -174,7 +174,7 @@ __PACKAGE__->register_method({
my $cfg = PVE::HA::Config::read_resources_config();
my $exclude_group_property = PVE::HA::Config::have_groups_been_migrated();
- my ($sid) = PVE::HA::Config::parse_sid($param->{sid});
+ my ($sid) = PVE::HA::Config::parse_vmid_or_sid($param->{sid});
return &$api_copy_config($cfg, $sid, $exclude_group_property);
},
@@ -198,7 +198,8 @@ __PACKAGE__->register_method({
PVE::Cluster::check_cfs_quorum();
mkdir("/etc/pve/ha");
- my ($sid, $type, $vmid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid, $type, $vmid) =
+ PVE::HA::Config::parse_vmid_or_sid(extract_param($param, 'sid'));
if (my $param_type = extract_param($param, 'type')) {
# useless, but do it anyway
@@ -253,7 +254,7 @@ __PACKAGE__->register_method({
my $digest = extract_param($param, 'digest');
my $delete = extract_param($param, 'delete');
- my ($sid, $type) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid, $type) = PVE::HA::Config::parse_vmid_or_sid(extract_param($param, 'sid'));
if (my $param_type = extract_param($param, 'type')) {
# useless, but do it anyway
@@ -313,7 +314,7 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- my ($sid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid) = PVE::HA::Config::parse_vmid_or_sid(extract_param($param, 'sid'));
my $purge = extract_param($param, 'purge') // 1;
if (!PVE::HA::Config::service_is_configured($sid)) {
@@ -401,7 +402,7 @@ __PACKAGE__->register_method({
my $result = {};
- my ($sid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid) = PVE::HA::Config::parse_vmid_or_sid(extract_param($param, 'sid'));
my $req_node = extract_param($param, 'node');
PVE::HA::Config::service_is_ha_managed($sid);
@@ -503,7 +504,7 @@ __PACKAGE__->register_method({
my $result = {};
- my ($sid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid) = PVE::HA::Config::parse_vmid_or_sid(extract_param($param, 'sid'));
my $req_node = extract_param($param, 'node');
PVE::HA::Config::service_is_ha_managed($sid);
diff --git a/src/PVE/CLI/ha_manager.pm b/src/PVE/CLI/ha_manager.pm
index 2bec33cb..013742e0 100644
--- a/src/PVE/CLI/ha_manager.pm
+++ b/src/PVE/CLI/ha_manager.pm
@@ -104,7 +104,7 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- my ($sid) = PVE::HA::Config::parse_sid(extract_param($param, 'sid'));
+ my ($sid) = PVE::HA::Config::parse_vmid_or_sid(extract_param($param, 'sid'));
PVE::HA::Config::service_is_ha_managed($sid);
diff --git a/src/PVE/HA/Config.pm b/src/PVE/HA/Config.pm
index 25465486..2b6f33f8 100644
--- a/src/PVE/HA/Config.pm
+++ b/src/PVE/HA/Config.pm
@@ -198,22 +198,28 @@ sub update_resources_config {
sub parse_sid {
my ($sid) = @_;
- my ($type, $vmid);
+ if (my ($type, $vmid) = $sid =~ m/^(\S+):(\S+)$/) {
+ return ($sid, $type, $vmid);
+ } else {
+ die "unable to parse service id '$sid'\n";
+ }
+}
- if (($vmid) = $sid =~ m/^(\d+)$/) {
+sub parse_vmid_or_sid {
+ my ($sid) = @_;
+
+ if (my ($vmid) = $sid =~ m/^(\d+)$/) {
my $vmlist = PVE::Cluster::get_vmlist();
if (defined($vmlist->{ids}->{$vmid})) {
my $vm_type = $vmlist->{ids}->{$vmid}->{type};
- $type = PVE::HA::Tools::get_ha_resource_type($vm_type);
+ my $type = PVE::HA::Tools::get_ha_resource_type($vm_type);
$sid = "$type:$vmid";
return ($sid, $type, $vmid);
} else {
die "unable to detect SID from VMID - VM/CT $vmid does not exist\n";
}
- } elsif (($type, $vmid) = $sid =~ m/^(\S+):(\S+)$/) {
- return ($sid, $type, $vmid);
} else {
- die "unable to parse service id '$sid'\n";
+ return parse_sid($sid);
}
}
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 08/17] drop the unmodified sid return array entry value from parse_sid
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (6 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 07/17] config: introduce separate parse_vmid_or_sid helper subroutine Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 09/17] lrm: catch and report dies while the LRM is in maintenance state Daniel Kral
` (9 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
As no caller of parse_sid() uses the now unmodified $sid return value,
remove it from the return array. It is only prepended in the case where
parse_vmid_or_sid() is given a correct SID instead of a VMID.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/HA/Config.pm | 8 ++++----
src/PVE/HA/Env/PVE2.pm | 2 +-
src/PVE/HA/LRM.pm | 2 +-
src/PVE/HA/Manager.pm | 4 ++--
src/PVE/HA/Sim/Env.pm | 2 +-
5 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/src/PVE/HA/Config.pm b/src/PVE/HA/Config.pm
index 2b6f33f8..f34c8c3a 100644
--- a/src/PVE/HA/Config.pm
+++ b/src/PVE/HA/Config.pm
@@ -114,7 +114,7 @@ my sub checked_resources_config {
foreach my $sid (keys %{ $cfg->{ids} }) {
my $d = $cfg->{ids}->{$sid};
- my (undef, undef, $vmid) = parse_sid($sid);
+ my (undef, $vmid) = parse_sid($sid);
$d->{state} = 'started' if !defined($d->{state});
$d->{state} = 'started' if $d->{state} eq 'enabled'; # backward compatibility
$d->{failback} = 1 if !defined($d->{failback});
@@ -148,7 +148,7 @@ sub read_and_check_resources_config {
my sub update_single_resource_config_inplace {
my ($cfg, $sid, $param, $delete) = @_;
- (undef, my $type) = parse_sid($sid);
+ my ($type) = parse_sid($sid);
my $scfg = $cfg->{ids}->{$sid}
|| die "no such resource '$sid'\n";
@@ -199,7 +199,7 @@ sub parse_sid {
my ($sid) = @_;
if (my ($type, $vmid) = $sid =~ m/^(\S+):(\S+)$/) {
- return ($sid, $type, $vmid);
+ return ($type, $vmid);
} else {
die "unable to parse service id '$sid'\n";
}
@@ -219,7 +219,7 @@ sub parse_vmid_or_sid {
die "unable to detect SID from VMID - VM/CT $vmid does not exist\n";
}
} else {
- return parse_sid($sid);
+ return ($sid, parse_sid($sid));
}
}
diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index f4d2b752..162c0249 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -209,7 +209,7 @@ sub exec_fence_agent {
sub steal_service {
my ($self, $sid, $current_node, $new_node) = @_;
- my (undef, $type, $vmid) = PVE::HA::Config::parse_sid($sid);
+ my ($type, $vmid) = PVE::HA::Config::parse_sid($sid);
if (my $plugin = PVE::HA::Resources->lookup($type)) {
my $old = $plugin->config_file($vmid, $current_node);
diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index 21055934..5ecd41cf 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -950,7 +950,7 @@ sub exec_resource_agent {
my $nodename = $haenv->nodename();
- my (undef, $service_type, $service_name) = $haenv->parse_sid($sid);
+ my ($service_type, $service_name) = $haenv->parse_sid($sid);
my $plugin = PVE::HA::Resources->lookup($service_type);
if (!$plugin) {
diff --git a/src/PVE/HA/Manager.pm b/src/PVE/HA/Manager.pm
index 4510dd44..26f66aef 100644
--- a/src/PVE/HA/Manager.pm
+++ b/src/PVE/HA/Manager.pm
@@ -283,7 +283,7 @@ sub load_balance {
my ($sid, $source, $target) = $migration->@{qw(sid source-node target-node)};
- my (undef, $type) = $haenv->parse_sid($sid);
+ my ($type) = $haenv->parse_sid($sid);
my $task = $type eq 'vm' ? "migrate" : "relocate";
my $cmd = "$task $sid $target";
@@ -555,7 +555,7 @@ my $fence_recovery_cleanup = sub {
my $haenv = $self->{haenv};
- my (undef, $type, $id) = $haenv->parse_sid($sid);
+ my ($type, $id) = $haenv->parse_sid($sid);
my $plugin = PVE::HA::Resources->lookup($type);
# should not happen
diff --git a/src/PVE/HA/Sim/Env.pm b/src/PVE/HA/Sim/Env.pm
index 311eaad1..72f494b8 100644
--- a/src/PVE/HA/Sim/Env.pm
+++ b/src/PVE/HA/Sim/Env.pm
@@ -234,7 +234,7 @@ sub parse_sid {
my $vmid = $2;
my $type = $1;
- return ($sid, $type, $vmid);
+ return ($type, $vmid);
}
sub read_fence_config {
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 09/17] lrm: catch and report dies while the LRM is in maintenance state
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (7 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 08/17] drop the unmodified sid return array entry value from parse_sid Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 10/17] lrm: rename update_lrm_status to flush_lrm_status Daniel Kral
` (8 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
This is similar in scope to the eval block in the active work branch,
where dies in watchdog releases, lock releases and manage_resources()
are caught and logged instead of making the LRM itself exit because of
these.
Furthermore, this prevents the LRM from exiting if a transient
manager_status read error in update_service_status() causes the whole
LRM to shut down with the watchdog armed and its HA agent lock held.
Best viewed with --ignore-all-space.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
src/PVE/HA/LRM.pm | 34 +++++++++++++++++++++-------------
1 file changed, 21 insertions(+), 13 deletions(-)
diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index 5ecd41cf..d09316dd 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -601,27 +601,35 @@ sub work {
} elsif ($state eq 'maintenance') {
my $startime = $haenv->get_time();
- return if !$self->update_service_status();
-
- # wait until all active services moved away
- my $service_count = $self->active_service_count();
my $exit_lrm = 0;
- if ($service_count == 0 && $self->run_workers() == 0) {
- # safety: going into maintenance and all active services got moved -> OK
- give_up_watchdog_protection($self);
+ eval {
+ # if we could not get the current service status there's no point
+ # in doing anything, try again next round.
+ return if !$self->update_service_status();
- if ($self->{shutdown_request}) {
- $exit_lrm = 1;
+ # wait until all active services moved away
+ my $service_count = $self->active_service_count();
+
+ if ($service_count == 0 && $self->run_workers() == 0) {
+ # safety: going into maintenance and all active services got moved -> OK
+ give_up_watchdog_protection($self);
+
+ if ($self->{shutdown_request}) {
+ $exit_lrm = 1;
+ }
+
+ # maintenance mode with no or only frozen services anymore, release the lock
+ $haenv->release_ha_agent_lock();
}
- # maintenance mode with no or only frozen services anymore, release the lock
- $haenv->release_ha_agent_lock();
+ $self->manage_resources() if !$exit_lrm;
+ };
+ if (my $err = $@) {
+ $haenv->log('err', "got unexpected error - $err");
}
- $self->manage_resources() if !$exit_lrm;
-
$self->update_lrm_status();
return 0 if $exit_lrm;
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 10/17] lrm: rename update_lrm_status to flush_lrm_status
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (8 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 09/17] lrm: catch and report dies while the LRM is in maintenance state Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 11/17] lrm: rename update_service_status to update_lrm_status Daniel Kral
` (7 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
Make the method name similar to PVE::HA::Manager::flush_manager_status
as the term "flush" is more telling about the method writing something
out than the term "update" which could be an update in any direction.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/HA/LRM.pm | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index d09316dd..36e41ff4 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -149,7 +149,7 @@ sub shutdown_request {
$self->{shutdown_request} = $haenv->get_time();
- eval { $self->update_lrm_status() or die "not quorate?\n"; };
+ eval { $self->flush_lrm_status() or die "not quorate?\n"; };
if (my $err = $@) {
$haenv->log('err', "unable to update lrm status file - $err");
}
@@ -180,7 +180,7 @@ sub set_local_status {
$self->{status} = $new;
}
-sub update_lrm_status {
+sub flush_lrm_status {
my ($self) = @_;
my $haenv = $self->{haenv};
@@ -359,7 +359,7 @@ sub work {
my $haenv = $self->{haenv};
if (!$wrote_lrm_status_at_startup) {
- if ($self->update_lrm_status()) {
+ if ($self->flush_lrm_status()) {
$wrote_lrm_status_at_startup = 1;
} else {
# do nothing
@@ -484,7 +484,7 @@ sub work {
return 0 if $self->{shutdown_request};
- $self->update_lrm_status();
+ $self->flush_lrm_status();
$haenv->sleep(5);
@@ -552,7 +552,7 @@ sub work {
$haenv->log('err', "got unexpected error - $err");
}
- $self->update_lrm_status();
+ $self->flush_lrm_status();
return 0 if $shutdown;
@@ -630,7 +630,7 @@ sub work {
$haenv->log('err', "got unexpected error - $err");
}
- $self->update_lrm_status();
+ $self->flush_lrm_status();
return 0 if $exit_lrm;
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 11/17] lrm: rename update_service_status to update_lrm_status
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (9 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 10/17] lrm: rename update_lrm_status to flush_lrm_status Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 12/17] lrm: document and initialize LRM instance properties Daniel Kral
` (6 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
update_lrm_status() is more fitting than update_service_status() here as
the internal LRM state that is updated by the information provided by
manager status does affect the LRM's behavior (mode and node_status) and
not only the LRM's behavior for its HA resources (service_status).
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/HA/LRM.pm | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index 36e41ff4..e1ee08c2 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -203,7 +203,7 @@ sub flush_lrm_status {
return 1;
}
-sub update_service_status {
+sub update_lrm_status {
my ($self) = @_;
my $haenv = $self->{haenv};
@@ -217,7 +217,6 @@ sub update_service_status {
my $nodename = $haenv->nodename();
$self->{node_status} = $ms->{node_status}->{$nodename} || 'unknown';
- # FIXME: method name is a bit confusing for doing this, either rename or move
if (!$self->{shutdown_request}) {
my $request = $ms->{node_request}->{$nodename} // {};
if ($request->{maintenance}) {
@@ -371,7 +370,7 @@ sub work {
my $status = $self->get_local_status();
my $state = $status->{state};
- $self->update_service_status();
+ $self->update_lrm_status();
my $fence_request = $self->is_fence_requested();
@@ -502,7 +501,7 @@ sub work {
# if we could not get the current service status there's no point
# in doing anything, try again next round.
- return if !$self->update_service_status();
+ return if !$self->update_lrm_status();
if ($self->{shutdown_request}) {
@@ -607,7 +606,7 @@ sub work {
eval {
# if we could not get the current service status there's no point
# in doing anything, try again next round.
- return if !$self->update_service_status();
+ return if !$self->update_lrm_status();
# wait until all active services moved away
my $service_count = $self->active_service_count();
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 12/17] lrm: document and initialize LRM instance properties
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (10 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 11/17] lrm: rename update_service_status to update_lrm_status Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 13/17] lrm: update the service status once per work iteration Daniel Kral
` (5 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
The properties of service_status and node_status in the LRM blessed
object reference are updated in the update_lrm_status() method in each
work() iteration.
No functional changes intended.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/HA/LRM.pm | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index e1ee08c2..c3bee12f 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -41,6 +41,13 @@ sub new {
mode => 'active',
cluster_state_update => 0,
active_idle_rounds => 0,
+ # service_status from manager_status in current work() iteration
+ # does contain all HA resources, users must filter for the assigned node
+ service_status => {},
+ # node_status from manager_status in current work() iteration
+ # this is only used to check whether a fence for the node is requested
+ # see $PVE::HA::NodeStatus::valid_node_states for possible values
+ node_status => 'unknown',
}, $class;
$self->set_local_status({ state => 'wait_for_agent_lock' });
@@ -203,6 +210,14 @@ sub flush_lrm_status {
return 1;
}
+=head3 $self->update_lrm_status()
+
+Updates the internal LRM properties from the current C<manager_status>.
+
+Returns 1 if the update was successful, otherwise returns undef.
+
+=cut
+
sub update_lrm_status {
my ($self) = @_;
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 13/17] lrm: update the service status once per work iteration
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (11 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 12/17] lrm: document and initialize LRM instance properties Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 14/17] lrm: read service config " Daniel Kral
` (4 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
Each invocation of update_lrm_status() does read the current
manager_status and update several internal LRM state properties.
Reduce it to be run only once at the beginning of the work iteration to
have consistent state across the whole work iteration.
The changes in the test cases reflect the change as test cases with
failing pmxcfs read/write do only output one log message as opposed to
two log messages when read_manager_status() fails.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/HA/LRM.pm | 6 +++---
src/test/test-cfs-unavailable1/log.expect | 5 -----
src/test/test-group-migrate4/log.expect | 10 ----------
3 files changed, 3 insertions(+), 18 deletions(-)
diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index c3bee12f..6d7be883 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -385,7 +385,7 @@ sub work {
my $status = $self->get_local_status();
my $state = $status->{state};
- $self->update_lrm_status();
+ my $status_update_successful = $self->update_lrm_status();
my $fence_request = $self->is_fence_requested();
@@ -516,7 +516,7 @@ sub work {
# if we could not get the current service status there's no point
# in doing anything, try again next round.
- return if !$self->update_lrm_status();
+ return if !$status_update_successful;
if ($self->{shutdown_request}) {
@@ -621,7 +621,7 @@ sub work {
eval {
# if we could not get the current service status there's no point
# in doing anything, try again next round.
- return if !$self->update_lrm_status();
+ return if !$status_update_successful;
# wait until all active services moved away
my $service_count = $self->active_service_count();
diff --git a/src/test/test-cfs-unavailable1/log.expect b/src/test/test-cfs-unavailable1/log.expect
index a9bb10f2..4d796c50 100644
--- a/src/test/test-cfs-unavailable1/log.expect
+++ b/src/test/test-cfs-unavailable1/log.expect
@@ -36,27 +36,22 @@ info 120 cmdlist: execute service vm:101 stopped
err 120 node1/crm: could not read manager status: cfs connection refused - not mounted?
err 120 node1/crm: got unexpected error - cfs connection refused - not mounted?
err 121 node1/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 121 node1/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 121 node1/lrm: unable to write lrm status file - cfs connection refused - not mounted?
err 140 node1/crm: could not read manager status: cfs connection refused - not mounted?
err 140 node1/crm: got unexpected error - cfs connection refused - not mounted?
err 141 node1/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 141 node1/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 141 node1/lrm: unable to write lrm status file - cfs connection refused - not mounted?
err 160 node1/crm: could not read manager status: cfs connection refused - not mounted?
err 160 node1/crm: got unexpected error - cfs connection refused - not mounted?
err 161 node1/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 161 node1/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 161 node1/lrm: unable to write lrm status file - cfs connection refused - not mounted?
err 180 node1/crm: could not read manager status: cfs connection refused - not mounted?
err 180 node1/crm: got unexpected error - cfs connection refused - not mounted?
err 181 node1/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 181 node1/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 181 node1/lrm: unable to write lrm status file - cfs connection refused - not mounted?
err 200 node1/crm: could not read manager status: cfs connection refused - not mounted?
err 200 node1/crm: got unexpected error - cfs connection refused - not mounted?
err 201 node1/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 201 node1/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 201 node1/lrm: unable to write lrm status file - cfs connection refused - not mounted?
info 220 cmdlist: execute cfs node1 rw work
info 220 node1/crm: service 'vm:101': state changed from 'started' to 'request_stop'
diff --git a/src/test/test-group-migrate4/log.expect b/src/test/test-group-migrate4/log.expect
index 52b79a3c..2bb3b5a5 100644
--- a/src/test/test-group-migrate4/log.expect
+++ b/src/test/test-group-migrate4/log.expect
@@ -96,53 +96,43 @@ info 420 cmdlist: execute cfs node2 rw fail
err 422 node2/crm: could not read manager status: cfs connection refused - not mounted?
err 422 node2/crm: got unexpected error - cfs connection refused - not mounted?
err 423 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 423 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 423 node2/lrm: unable to write lrm status file - cfs connection refused - not mounted?
err 442 node2/crm: could not read manager status: cfs connection refused - not mounted?
err 442 node2/crm: got unexpected error - cfs connection refused - not mounted?
err 443 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 443 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 443 node2/lrm: unable to write lrm status file - cfs connection refused - not mounted?
err 462 node2/crm: could not read manager status: cfs connection refused - not mounted?
err 462 node2/crm: got unexpected error - cfs connection refused - not mounted?
err 463 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 463 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 463 node2/lrm: unable to write lrm status file - cfs connection refused - not mounted?
err 482 node2/crm: could not read manager status: cfs connection refused - not mounted?
err 482 node2/crm: got unexpected error - cfs connection refused - not mounted?
err 483 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 483 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 483 node2/lrm: unable to write lrm status file - cfs connection refused - not mounted?
err 502 node2/crm: could not read manager status: cfs connection refused - not mounted?
err 502 node2/crm: got unexpected error - cfs connection refused - not mounted?
err 503 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 503 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 503 node2/lrm: unable to write lrm status file - cfs connection refused - not mounted?
info 520 cmdlist: execute pve-manager-version node2 set 9.0.0
err 522 node2/crm: could not read manager status: cfs connection refused - not mounted?
err 522 node2/crm: got unexpected error - cfs connection refused - not mounted?
err 523 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 523 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 523 node2/lrm: unable to write lrm status file - cfs connection refused - not mounted?
err 542 node2/crm: could not read manager status: cfs connection refused - not mounted?
err 542 node2/crm: got unexpected error - cfs connection refused - not mounted?
err 543 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 543 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 543 node2/lrm: unable to write lrm status file - cfs connection refused - not mounted?
err 562 node2/crm: could not read manager status: cfs connection refused - not mounted?
err 562 node2/crm: got unexpected error - cfs connection refused - not mounted?
err 563 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 563 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 563 node2/lrm: unable to write lrm status file - cfs connection refused - not mounted?
err 582 node2/crm: could not read manager status: cfs connection refused - not mounted?
err 582 node2/crm: got unexpected error - cfs connection refused - not mounted?
err 583 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 583 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 583 node2/lrm: unable to write lrm status file - cfs connection refused - not mounted?
err 602 node2/crm: could not read manager status: cfs connection refused - not mounted?
err 602 node2/crm: got unexpected error - cfs connection refused - not mounted?
err 603 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
-err 603 node2/lrm: updating service status from manager failed: cfs connection refused - not mounted?
err 603 node2/lrm: unable to write lrm status file - cfs connection refused - not mounted?
info 620 cmdlist: execute cfs node2 rw work
noti 702 node2/crm: start ha group migration...
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 14/17] lrm: read service config once per work iteration
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (12 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 13/17] lrm: update the service status once per work iteration Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 15/17] lrm: compute valid service uids hash set " Daniel Kral
` (3 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
The amount of times the LRM reads and parses the HA resource config
primarily scales with the amount of HA resource command exit codes that
need to be handled. This can become quite expensive especially with many
HA resources in 'started' state, where the HA Manager issues an HA
resource command on every HA Manager round.
In the PVE2 environment, the HA resource config and vmlist is cached
between PVE::Cluster::cfs_update() calls, and in the Test environment
Sim Hardware commands can only change the HA resource config outside of
the LRM round window.
Therefore, the return value of read_service_config() can be cached
inbetween and must only be invalidated again once a new LRM round
starts.
This significantly reduces the time spent reading and parsing the HA
resource config every time run_workers() is called and every time a LRM
worker's result is collected with handle_service_exitcode().
A test startup of 768 HA resources on a single node at the same time
showed ~99.2% fewer calls to checked_resources_config() (from 10,333 to
82 calls) and ~99.3% fewer calls to parse_sid() (from 15,639,537 to
109,198 calls).
Additionally, this reduces the inclusive average execution time per call
for several subroutines where the HA resource config was read within:
+---------------------------+---------------+--------------+
| Subroutine | Before Change | After Change |
+---------------------------+---------------+--------------+
| check_active_workers | 327ms | 11.6ms |
| resource_command_finished | 22.7ms | 2.81ms |
| handle_service_exitcode | 21.1ms | 10µs |
+---------------------------+---------------+--------------+
The profiling was done on a physical host with Devel::NYTProf 6.14 [0].
[0] https://metacpan.org/pod/Devel::NYTProf
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/HA/LRM.pm | 25 +++++++++++++++++++++++--
1 file changed, 23 insertions(+), 2 deletions(-)
diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index 6d7be883..ad3bbbb6 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -41,6 +41,9 @@ sub new {
mode => 'active',
cluster_state_update => 0,
active_idle_rounds => 0,
+ # service config cache in current work() iteration
+ # does contain all HA resources, users must filter for the assigned node
+ service_config => undef,
# service_status from manager_status in current work() iteration
# does contain all HA resources, users must filter for the assigned node
service_status => {},
@@ -247,6 +250,22 @@ sub update_lrm_status {
}
}
+sub read_service_config {
+ my ($self) = @_;
+
+ return $self->{service_config} if defined($self->{service_config});
+
+ $self->{service_config} = $self->{haenv}->read_service_config();
+
+ return $self->{service_config};
+}
+
+sub invalidate_service_config_cache {
+ my ($self) = @_;
+
+ undef $self->{service_config};
+}
+
sub get_protected_ha_agent_lock {
my ($self) = @_;
@@ -349,6 +368,8 @@ sub do_one_iteration {
$self->{cluster_state_update} = $haenv->cluster_state_update();
+ $self->invalidate_service_config_cache();
+
my $res = $self->work();
$haenv->loop_end_hook();
@@ -668,7 +689,7 @@ sub run_workers {
# number of workers to start, if 0 we exec the command directly without forking
my $max_workers = $haenv->get_max_workers();
- my $sc = $haenv->read_service_config();
+ my $sc = $self->read_service_config();
my $worker = $self->{workers};
# we only got limited time but want to ensure that every queued worker is scheduled
@@ -916,7 +937,7 @@ sub handle_service_exitcode {
my $haenv = $self->{haenv};
my $tries = $self->{restart_tries};
- my $sc = $haenv->read_service_config();
+ my $sc = $self->read_service_config();
my $max_restart = 0;
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 15/17] lrm: compute valid service uids hash set once per work iteration
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (13 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 14/17] lrm: read service config " Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 16/17] lrm: resource_command_finished: avoid writing stale LRM results Daniel Kral
` (2 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
$self->{service_status} is only updated in update_lrm_status(), which
makes the recomputation of valid HA resource command uids in every
invocation of resource_command_finished() unnecessary.
A test startup of 768 HA resources on a single node at the same time
showed a ~2.5x speedup for resource_command_finished().
+---------------------------+---------------+--------------+
| Subroutine | Before Change | After Change |
+---------------------------+---------------+--------------+
| check_active_workers | 11.6ms | 4.09ms |
| resource_command_finished | 2.81ms | 1.09ms |
+---------------------------+---------------+--------------+
The profiling was done on a physical host with Devel::NYTProf 6.14 [0].
[0] https://metacpan.org/pod/Devel::NYTProf
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/HA/LRM.pm | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index ad3bbbb6..89760fe9 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -47,6 +47,8 @@ sub new {
# service_status from manager_status in current work() iteration
# does contain all HA resources, users must filter for the assigned node
service_status => {},
+ # Valid service_status UID cache in current work() iteration
+ service_status_uids => {},
# node_status from manager_status in current work() iteration
# this is only used to check whether a fence for the node is requested
# see $PVE::HA::NodeStatus::valid_node_states for possible values
@@ -232,6 +234,12 @@ sub update_lrm_status {
return undef;
} else {
$self->{service_status} = $ms->{service_status} || {};
+ $self->{service_status_uids} = {};
+ for my $sd (values $self->{service_status}->%*) {
+ my $uid = $sd->{uid} or next;
+ $self->{service_status_uids}->{$uid} = 1;
+ }
+
my $nodename = $haenv->nodename();
$self->{node_status} = $ms->{node_status}->{$nodename} || 'unknown';
@@ -910,19 +918,9 @@ sub resource_command_finished {
exit_code => $exit_code,
};
- my $ss = $self->{service_status};
-
- # compute hash of valid/existing uids
- my $valid_uids = {};
- foreach my $sid (keys %$ss) {
- my $sd = $ss->{$sid};
- next if !$sd->{uid};
- $valid_uids->{ $sd->{uid} } = 1;
- }
-
my $results = {};
foreach my $id (keys %{ $self->{results} }) {
- next if !$valid_uids->{$id};
+ next if !$self->{service_status_uids}->{$id};
$results->{$id} = $self->{results}->{$id};
}
$self->{results} = $results;
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 16/17] lrm: resource_command_finished: avoid writing stale LRM results
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (14 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 15/17] lrm: compute valid service uids hash set " Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 17/17] lrm: prune stale LRM results once per work iteration Daniel Kral
2026-11-18 14:46 ` superseded: [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
resource_command_finished() is called whenever an HA resource command
finishes, which was previously queued by queue_resource_command().
However, if the HA resource command uid became stale in the meantime,
i.e. the HA Manager dropped or renewed an HA resource's uid, the LRM
results would be written to $self->{results} and pruned right after.
Therefore, do not write it to $self->{results} at all.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/HA/LRM.pm | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index 89760fe9..90c9be44 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -912,6 +912,9 @@ sub resource_command_finished {
return if $exit_code == ETRY_AGAIN; # tell nobody, simply retry
+ # do not report the result as the resource command uid is already stale
+ return if !$self->{service_status_uids}->{$uid};
+
$self->{results}->{$uid} = {
sid => $w->{sid},
state => $w->{state},
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH ha-manager v3 17/17] lrm: prune stale LRM results once per work iteration
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (15 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 16/17] lrm: resource_command_finished: avoid writing stale LRM results Daniel Kral
@ 2026-11-18 14:25 ` Daniel Kral
2026-11-18 14:46 ` superseded: [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:25 UTC (permalink / raw)
To: pve-devel
resource_command_finished() writes only LRM results to $self->{results},
whose HA resource command uid is still active in the current LRM round,
i.e., are in $self->{service_status_uids}.
Therefore, the only other reason for stale LRM results in
$self->{results} is because updates to $self->{service_status} and
therefore $self->{service_status_uids} made these become stale.
As both $self->{service_status} and $self->{service_status_uids} are
only updated in update_lrm_status(), pruning these stale LRM results is
only necessary whenever these change.
This makes the LRM results exposed in the lrm_status file be pruned more
often, which only happened before when the LRM was restarted or
resource_command_finished() was called.
A test startup of 768 HA resources on a single node at the same time
showed a ~21.8x speedup for resource_command_finished().
+---------------------------+---------------+--------------+
| Subroutine | Before Change | After Change |
+---------------------------+---------------+--------------+
| check_active_workers | 4.09ms | 998µs |
| resource_command_finished | 1.09ms | 50µs |
+---------------------------+---------------+--------------+
The profiling was done on a physical host with Devel::NYTProf 6.14 [0].
[0] https://metacpan.org/pod/Devel::NYTProf
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Reviewed-by: Michael Köppl <m.koeppl@proxmox.com>
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/PVE/HA/LRM.pm | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index 90c9be44..98d67691 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -240,6 +240,11 @@ sub update_lrm_status {
$self->{service_status_uids}->{$uid} = 1;
}
+ # prune results which became stale since the last update
+ for my $uid (keys $self->{results}->%*) {
+ delete $self->{results}->{$uid} if !$self->{service_status_uids}->{$uid};
+ }
+
my $nodename = $haenv->nodename();
$self->{node_status} = $ms->{node_status}->{$nodename} || 'unknown';
@@ -920,13 +925,6 @@ sub resource_command_finished {
state => $w->{state},
exit_code => $exit_code,
};
-
- my $results = {};
- foreach my $id (keys %{ $self->{results} }) {
- next if !$self->{service_status_uids}->{$id};
- $results->{$id} = $self->{results}->{$id};
- }
- $self->{results} = $results;
}
# processes the exit code from a finished resource agent, so that the CRM knows
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* superseded: [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
` (16 preceding siblings ...)
2026-11-18 14:25 ` [PATCH ha-manager v3 17/17] lrm: prune stale LRM results once per work iteration Daniel Kral
@ 2026-11-18 14:46 ` Daniel Kral
17 siblings, 0 replies; 19+ messages in thread
From: Daniel Kral @ 2026-11-18 14:46 UTC (permalink / raw)
To: Daniel Kral, pve-devel
Superseded-by: https://lore.proxmox.com/pve-devel/20261118144502.231134-1-d.kral@proxmox.com/
On Wed Nov 18, 2026 at 3:25 PM CET, Daniel Kral wrote:
> Changes since v2 (thanks a lot @Michael!):
> - add R-b and T-b from @Michael to patches except #2 and #9
> - drop s/$name/$vmid/ from PVE::HA::LRM and PVE::HA::Resources in #2
> - new patch #9: use an eval around the maintenance work branch, similar
> to what is catched in the active work branch
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2026-11-18 14:47 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 01/17] resources: remove commented ipaddr resource type Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 02/17] tree-wide: use the term vmid instead of name when referring to VMs/CTs Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 03/17] tree-wide: remove unused return values at parse_sid callsites Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 04/17] config: make update_single_resource_config_inplace only allow sids Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 05/17] make parse_sid always return an array Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 06/17] config: use early returns in parse_sid Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 07/17] config: introduce separate parse_vmid_or_sid helper subroutine Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 08/17] drop the unmodified sid return array entry value from parse_sid Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 09/17] lrm: catch and report dies while the LRM is in maintenance state Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 10/17] lrm: rename update_lrm_status to flush_lrm_status Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 11/17] lrm: rename update_service_status to update_lrm_status Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 12/17] lrm: document and initialize LRM instance properties Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 13/17] lrm: update the service status once per work iteration Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 14/17] lrm: read service config " Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 15/17] lrm: compute valid service uids hash set " Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 16/17] lrm: resource_command_finished: avoid writing stale LRM results Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 17/17] lrm: prune stale LRM results once per work iteration Daniel Kral
2026-11-18 14:46 ` superseded: [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox