From: Daniel Kral <d.kral@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH ha-manager v4 09/17] lrm: catch and report dies while the LRM is in maintenance state
Date: Wed, 18 Nov 2026 15:44:54 +0100 [thread overview]
Message-ID: <20261118144502.231134-10-d.kral@proxmox.com> (raw)
In-Reply-To: <20261118144502.231134-1-d.kral@proxmox.com>
This is similar in scope to the eval block in the active work branch,
where dies in watchdog releases, lock releases and manage_resources()
are caught and logged instead of making the LRM itself exit because of
these.
Furthermore, this prevents the LRM from exiting if a transient
manager_status read error in update_service_status() causes the whole
LRM to shut down with the watchdog armed and its HA agent lock held.
Best viewed with --ignore-all-space.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
src/PVE/HA/LRM.pm | 34 +++++++++++++++++++++-------------
1 file changed, 21 insertions(+), 13 deletions(-)
diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index ff3400e4..d52b95a1 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -601,27 +601,35 @@ sub work {
} elsif ($state eq 'maintenance') {
my $startime = $haenv->get_time();
- return if !$self->update_service_status();
-
- # wait until all active services moved away
- my $service_count = $self->active_service_count();
my $exit_lrm = 0;
- if ($service_count == 0 && $self->run_workers() == 0) {
- # safety: going into maintenance and all active services got moved -> OK
- give_up_watchdog_protection($self);
+ eval {
+ # if we could not get the current service status there's no point
+ # in doing anything, try again next round.
+ return if !$self->update_service_status();
- if ($self->{shutdown_request}) {
- $exit_lrm = 1;
+ # wait until all active services moved away
+ my $service_count = $self->active_service_count();
+
+ if ($service_count == 0 && $self->run_workers() == 0) {
+ # safety: going into maintenance and all active services got moved -> OK
+ give_up_watchdog_protection($self);
+
+ if ($self->{shutdown_request}) {
+ $exit_lrm = 1;
+ }
+
+ # maintenance mode with no or only frozen services anymore, release the lock
+ $haenv->release_ha_agent_lock();
}
- # maintenance mode with no or only frozen services anymore, release the lock
- $haenv->release_ha_agent_lock();
+ $self->manage_resources() if !$exit_lrm;
+ };
+ if (my $err = $@) {
+ $haenv->log('err', "got unexpected error - $err");
}
- $self->manage_resources() if !$exit_lrm;
-
$self->update_lrm_status();
return 0 if $exit_lrm;
--
2.47.3
next prev parent reply other threads:[~2026-11-18 14:46 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 14:44 [PATCH-SERIES ha-manager v4 00/17] some sid parsing and LRM speedup improvements Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 01/17] resources: remove commented ipaddr resource type Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 02/17] tree-wide: use the term vmid instead of name when referring to VMs/CTs Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 03/17] tree-wide: remove unused return values at parse_sid callsites Daniel Kral
2026-11-18 15:20 ` Michael Köppl
2026-11-18 14:44 ` [PATCH ha-manager v4 04/17] config: make update_single_resource_config_inplace only allow sids Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 05/17] make parse_sid always return an array Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 06/17] config: use early returns in parse_sid Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 07/17] config: introduce separate parse_vmid_or_sid helper subroutine Daniel Kral
2026-11-18 16:02 ` Michael Köppl
2026-11-18 14:44 ` [PATCH ha-manager v4 08/17] drop the unmodified sid return array entry value from parse_sid Daniel Kral
2026-11-18 14:44 ` Daniel Kral [this message]
2026-11-18 16:12 ` [PATCH ha-manager v4 09/17] lrm: catch and report dies while the LRM is in maintenance state Michael Köppl
2026-11-18 14:44 ` [PATCH ha-manager v4 10/17] lrm: rename update_lrm_status to flush_lrm_status Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 11/17] lrm: rename update_service_status to update_lrm_status Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 12/17] lrm: document and initialize LRM instance properties Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 13/17] lrm: update the service status once per work iteration Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 14/17] lrm: read service config " Daniel Kral
2026-11-18 14:45 ` [PATCH ha-manager v4 15/17] lrm: compute valid service uids hash set " Daniel Kral
2026-11-18 14:45 ` [PATCH ha-manager v4 16/17] lrm: resource_command_finished: avoid writing stale LRM results Daniel Kral
2026-11-18 14:45 ` [PATCH ha-manager v4 17/17] lrm: prune stale LRM results once per work iteration Daniel Kral
2026-11-18 20:39 ` applied: [PATCH-SERIES ha-manager v4 00/17] some sid parsing and LRM speedup improvements Thomas Lamprecht
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261118144502.231134-10-d.kral@proxmox.com \
--to=d.kral@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox