public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Daniel Kral <d.kral@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH ha-manager v3 09/17] lrm: catch and report dies while the LRM is in maintenance state
Date: Wed, 18 Nov 2026 15:25:48 +0100	[thread overview]
Message-ID: <20261118142556.218589-10-d.kral@proxmox.com> (raw)
In-Reply-To: <20261118142556.218589-1-d.kral@proxmox.com>

This is similar in scope to the eval block in the active work branch,
where dies in watchdog releases, lock releases and manage_resources()
are caught and logged instead of making the LRM itself exit because of
these.

Furthermore, this prevents the LRM from exiting if a transient
manager_status read error in update_service_status() causes the whole
LRM to shut down with the watchdog armed and its HA agent lock held.

Best viewed with --ignore-all-space.

Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
 src/PVE/HA/LRM.pm | 34 +++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 13 deletions(-)

diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index 5ecd41cf..d09316dd 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -601,27 +601,35 @@ sub work {
     } elsif ($state eq 'maintenance') {
 
         my $startime = $haenv->get_time();
-        return if !$self->update_service_status();
-
-        # wait until all active services moved away
-        my $service_count = $self->active_service_count();
 
         my $exit_lrm = 0;
 
-        if ($service_count == 0 && $self->run_workers() == 0) {
-            # safety: going into maintenance and all active services got moved -> OK
-            give_up_watchdog_protection($self);
+        eval {
+            # if we could not get the current service status there's no point
+            # in doing anything, try again next round.
+            return if !$self->update_service_status();
 
-            if ($self->{shutdown_request}) {
-                $exit_lrm = 1;
+            # wait until all active services moved away
+            my $service_count = $self->active_service_count();
+
+            if ($service_count == 0 && $self->run_workers() == 0) {
+                # safety: going into maintenance and all active services got moved -> OK
+                give_up_watchdog_protection($self);
+
+                if ($self->{shutdown_request}) {
+                    $exit_lrm = 1;
+                }
+
+                # maintenance mode with no or only frozen services anymore, release the lock
+                $haenv->release_ha_agent_lock();
             }
 
-            # maintenance mode with no or only frozen services anymore, release the lock
-            $haenv->release_ha_agent_lock();
+            $self->manage_resources() if !$exit_lrm;
+        };
+        if (my $err = $@) {
+            $haenv->log('err', "got unexpected error - $err");
         }
 
-        $self->manage_resources() if !$exit_lrm;
-
         $self->update_lrm_status();
 
         return 0 if $exit_lrm;
-- 
2.47.3





  parent reply	other threads:[~2026-11-18 14:27 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 14:25 [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 01/17] resources: remove commented ipaddr resource type Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 02/17] tree-wide: use the term vmid instead of name when referring to VMs/CTs Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 03/17] tree-wide: remove unused return values at parse_sid callsites Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 04/17] config: make update_single_resource_config_inplace only allow sids Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 05/17] make parse_sid always return an array Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 06/17] config: use early returns in parse_sid Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 07/17] config: introduce separate parse_vmid_or_sid helper subroutine Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 08/17] drop the unmodified sid return array entry value from parse_sid Daniel Kral
2026-11-18 14:25 ` Daniel Kral [this message]
2026-11-18 14:25 ` [PATCH ha-manager v3 10/17] lrm: rename update_lrm_status to flush_lrm_status Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 11/17] lrm: rename update_service_status to update_lrm_status Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 12/17] lrm: document and initialize LRM instance properties Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 13/17] lrm: update the service status once per work iteration Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 14/17] lrm: read service config " Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 15/17] lrm: compute valid service uids hash set " Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 16/17] lrm: resource_command_finished: avoid writing stale LRM results Daniel Kral
2026-11-18 14:25 ` [PATCH ha-manager v3 17/17] lrm: prune stale LRM results once per work iteration Daniel Kral
2026-11-18 14:46 ` superseded: [PATCH-SERIES ha-manager v3 00/17] some sid parsing and LRM speedup improvements Daniel Kral

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118142556.218589-10-d.kral@proxmox.com \
    --to=d.kral@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal