all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Daniel Kral <d.kral@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH ha-manager v4 09/17] lrm: catch and report dies while the LRM is in maintenance state
Date: Wed, 18 Nov 2026 15:44:54 +0100	[thread overview]
Message-ID: <20261118144502.231134-10-d.kral@proxmox.com> (raw)
In-Reply-To: <20261118144502.231134-1-d.kral@proxmox.com>

This is similar in scope to the eval block in the active work branch,
where dies in watchdog releases, lock releases and manage_resources()
are caught and logged instead of making the LRM itself exit because of
these.

Furthermore, this prevents the LRM from exiting if a transient
manager_status read error in update_service_status() causes the whole
LRM to shut down with the watchdog armed and its HA agent lock held.

Best viewed with --ignore-all-space.

Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
 src/PVE/HA/LRM.pm | 34 +++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 13 deletions(-)

diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index ff3400e4..d52b95a1 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -601,27 +601,35 @@ sub work {
     } elsif ($state eq 'maintenance') {
 
         my $startime = $haenv->get_time();
-        return if !$self->update_service_status();
-
-        # wait until all active services moved away
-        my $service_count = $self->active_service_count();
 
         my $exit_lrm = 0;
 
-        if ($service_count == 0 && $self->run_workers() == 0) {
-            # safety: going into maintenance and all active services got moved -> OK
-            give_up_watchdog_protection($self);
+        eval {
+            # if we could not get the current service status there's no point
+            # in doing anything, try again next round.
+            return if !$self->update_service_status();
 
-            if ($self->{shutdown_request}) {
-                $exit_lrm = 1;
+            # wait until all active services moved away
+            my $service_count = $self->active_service_count();
+
+            if ($service_count == 0 && $self->run_workers() == 0) {
+                # safety: going into maintenance and all active services got moved -> OK
+                give_up_watchdog_protection($self);
+
+                if ($self->{shutdown_request}) {
+                    $exit_lrm = 1;
+                }
+
+                # maintenance mode with no or only frozen services anymore, release the lock
+                $haenv->release_ha_agent_lock();
             }
 
-            # maintenance mode with no or only frozen services anymore, release the lock
-            $haenv->release_ha_agent_lock();
+            $self->manage_resources() if !$exit_lrm;
+        };
+        if (my $err = $@) {
+            $haenv->log('err', "got unexpected error - $err");
         }
 
-        $self->manage_resources() if !$exit_lrm;
-
         $self->update_lrm_status();
 
         return 0 if $exit_lrm;
-- 
2.47.3





  parent reply	other threads:[~2026-11-18 14:46 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 14:44 [PATCH-SERIES ha-manager v4 00/17] some sid parsing and LRM speedup improvements Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 01/17] resources: remove commented ipaddr resource type Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 02/17] tree-wide: use the term vmid instead of name when referring to VMs/CTs Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 03/17] tree-wide: remove unused return values at parse_sid callsites Daniel Kral
2026-11-18 15:20   ` Michael Köppl
2026-11-18 14:44 ` [PATCH ha-manager v4 04/17] config: make update_single_resource_config_inplace only allow sids Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 05/17] make parse_sid always return an array Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 06/17] config: use early returns in parse_sid Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 07/17] config: introduce separate parse_vmid_or_sid helper subroutine Daniel Kral
2026-11-18 16:02   ` Michael Köppl
2026-11-18 14:44 ` [PATCH ha-manager v4 08/17] drop the unmodified sid return array entry value from parse_sid Daniel Kral
2026-11-18 14:44 ` Daniel Kral [this message]
2026-11-18 16:12   ` [PATCH ha-manager v4 09/17] lrm: catch and report dies while the LRM is in maintenance state Michael Köppl
2026-11-18 14:44 ` [PATCH ha-manager v4 10/17] lrm: rename update_lrm_status to flush_lrm_status Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 11/17] lrm: rename update_service_status to update_lrm_status Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 12/17] lrm: document and initialize LRM instance properties Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 13/17] lrm: update the service status once per work iteration Daniel Kral
2026-11-18 14:44 ` [PATCH ha-manager v4 14/17] lrm: read service config " Daniel Kral
2026-11-18 14:45 ` [PATCH ha-manager v4 15/17] lrm: compute valid service uids hash set " Daniel Kral
2026-11-18 14:45 ` [PATCH ha-manager v4 16/17] lrm: resource_command_finished: avoid writing stale LRM results Daniel Kral
2026-11-18 14:45 ` [PATCH ha-manager v4 17/17] lrm: prune stale LRM results once per work iteration Daniel Kral
2026-11-18 20:39 ` applied: [PATCH-SERIES ha-manager v4 00/17] some sid parsing and LRM speedup improvements Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118144502.231134-10-d.kral@proxmox.com \
    --to=d.kral@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal