public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode
@ 2026-11-19  9:59 Dominik Rusovac
  2026-11-19  9:59 ` [PATCH pve-manager v5 01/10] fix #6144: api: nodes: add endpoint to set " Dominik Rusovac
                   ` (10 more replies)
  0 siblings, 11 replies; 12+ messages in thread
From: Dominik Rusovac @ 2026-11-19  9:59 UTC (permalink / raw)
  To: pve-devel

this picks up Thomas' v3 [0], rebased onto current master and reworked
in a few places; thx for the nice work Thomas! I added `Originally-by`
trailers

node maintenance mode can currently only be enabled/disabled with the HA
manager's CRM command in the CLI. this series adds an API endpoint and a
'Node Maintenance' menu in the HA status panel for it

the endpoint lives at PUT /nodes/{node}/maintenance in pve-manager, as
agreed on in the v1/v2 reviews, so that the maintenance concept can be
extended beyond HA managed resources later on. for now, it runs the HA
manager's CLI command and forwards its errors. the CLI now rejects nodes
unknown to the HA stack as well as enabling maintenance mode while HA is
disarmed or if the node is not online from the CRM's point of view,
since the CRM would ignore such requests. the validation of a request is
factored out of the CRM, so that the CRM and the CLI share the same
logic

to decide which action to offer, the UI uses the maintenance request
recorded by the CRM, which the HA status API now reports, instead of the
mode the LRM reports. the LRM also enters maintenance mode on shutdown
with the 'migrate' shutdown policy, which a disable request cannot undo

on top of that, the HA status API also reports whether a maintenance
request is still queued for the CRM, so that the UI can show a spinner
until the CRM processed the request, even if it ignored it. as discussed
offline, these two patches are optional and can be left out together,
the rest of the series works without them

basic functionality was tested in a virtualized 3-node cluster:
* enabling/disabling worked as expected using the UI
* possible actions were offered as expected
* errors are forwarded in both the CLI and UI

changes since v4 [1]:
* ha-manager: add a test for ignored node maintenance requests
* ha-manager: standardize the log messages for ignored maintenance
  mode requests
* ha-manager: factor out the reasons why the CRM ignores a request to
  enable maintenance mode
* ha-manager: cli: reject node maintenance requests that the CRM would
  ignore
* ha-manager: api: status: also report whether a node maintenance
  request is still queued for the CRM
* manager: api: run the HA manager's CLI command instead of reading the
  HA manager status and queuing the CRM command directly
* manager: ui: rename the selected node in the view model, throw on
  unknown maintenance modes
* manager: ui: split out the spinner into a separate, optional patch,
  and show it while the request is queued, so that it also ends if the
  CRM ignores the request

changes since v3:
* rebase onto current master, including the arm/disarm changes, which also
  added a toolbar to the HA status panel
* ha-manager: report the CRM maintenance request instead of the LRM mode
* manager: api: check Sys.Modify on /nodes/{node} instead of /, reject nodes
  unknown to HA, and reject enabling maintenance while HA is disarmed or if the
  node is not online
* manager: ui: move the menu next to Arm/Disarm, use the view model and
  controller of the HA status panel, fix selection handling, use complete
  sentences for translatable strings, show a spinner while a maintenance request
  is processed
* docs: condense the UI instructions and adapt them to the menu

[0] https://lore.proxmox.com/all/20251025183814.698627-1-thomas@atskinner.net/
[1] https://lore.proxmox.com/all/20261113152620.2662417-1-d.rusovac@proxmox.com/


pve-manager:

Dominik Rusovac (3):
  fix #6144: api: nodes: add endpoint to set maintenance mode
  fix #6144: ui: ha: status: allow toggling maintenance mode
  ui: ha: status: show spinner while maintenance mode request is queued

 PVE/API2/Nodes.pm             |  45 ++++++++++++++
 www/manager6/ha/Status.js     | 107 ++++++++++++++++++++++++++++++++++
 www/manager6/ha/StatusView.js |  45 +++++++++++++-
 3 files changed, 196 insertions(+), 1 deletion(-)


pve-ha-manager:

Dominik Rusovac (6):
  test: add test for ignoring maintenance mode request
  manager: standardize log messages for maintenance mode request
  manager: factor out check for maintenance mode request
  cli: reject maintenance mode requests rejected by crm
  api: status: report maintenance mode request for lrm entries
  api: status: report queued maintenance mode request for lrm entries

 src/PVE/API2/HA/Status.pm                     | 26 ++++++
 src/PVE/CLI/ha_manager.pm                     | 22 ++++-
 src/PVE/HA/Config.pm                          | 13 +++
 src/PVE/HA/Helpers.pm                         | 21 +++++
 src/PVE/HA/Manager.pm                         | 19 ++---
 .../test-crm-maintenance-request2/log.expect  |  2 +-
 src/test/test-crm-maintenance-request6/README |  3 +
 .../test-crm-maintenance-request6/cmdlist     |  9 ++
 .../hardware_status                           |  5 ++
 .../test-crm-maintenance-request6/log.expect  | 82 +++++++++++++++++++
 .../manager_status                            |  1 +
 .../service_config                            |  5 ++
 12 files changed, 191 insertions(+), 17 deletions(-)
 create mode 100644 src/test/test-crm-maintenance-request6/README
 create mode 100644 src/test/test-crm-maintenance-request6/cmdlist
 create mode 100644 src/test/test-crm-maintenance-request6/hardware_status
 create mode 100644 src/test/test-crm-maintenance-request6/log.expect
 create mode 100644 src/test/test-crm-maintenance-request6/manager_status
 create mode 100644 src/test/test-crm-maintenance-request6/service_config


pve-docs:

Dominik Rusovac (1):
  ha-manager: maintenance mode: mention web ui menu

 ha-manager.adoc | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)


Summary over all repositories:
  16 files changed, 396 insertions(+), 24 deletions(-)

-- 
Generated by murpp 0.11.0




^ permalink raw reply	[flat|nested] 12+ messages in thread

* [PATCH pve-manager v5 01/10] fix #6144: api: nodes: add endpoint to set maintenance mode
  2026-11-19  9:59 [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Dominik Rusovac
@ 2026-11-19  9:59 ` Dominik Rusovac
  2026-11-19  9:59 ` [PATCH pve-manager v5 02/10] fix #6144: ui: ha: status: allow toggling " Dominik Rusovac
                   ` (9 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Dominik Rusovac @ 2026-11-19  9:59 UTC (permalink / raw)
  To: pve-devel; +Cc: Thomas Skinner

node maintenance mode can currently only be requested with the HA
manager's CLI, which can't be used from the web UI

add an endpoint to enable/disable maintenance mode for a node, which
lives below the node path rather than the HA API, so that the
maintenance concept can be extended to more than HA managed resources
later on

for now, it runs the HA manager's CLI command to request the HA node
maintenance mode. the CLI rejects requests that the CRM would ignore,
e.g. for nodes unknown to HA, while HA is disarmed or if the node is not
online, and its error message is forwarded

Originally-by: Thomas Skinner <thomas@atskinner.net>
Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---

Notes:
    changes since v4:
    * reword commit subject and body
    * reorder name, path, method, ... etc.
    * reword 'maintenance-mode' description
    * run the HA manager's CLI command instead of reading the HA
      manager status and queuing the CRM command directly, as
      suggested in the v4 review, and forward its error message

 PVE/API2/Nodes.pm | 45 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
index 07b3ce1f..117daa45 100644
--- a/PVE/API2/Nodes.pm
+++ b/PVE/API2/Nodes.pm
@@ -249,6 +249,7 @@ __PACKAGE__->register_method({
             { name => 'hosts' },
             { name => 'journal' },
             { name => 'lxc' },
+            { name => 'maintenance' },
             { name => 'migrateall' },
             { name => 'netstat' },
             { name => 'network' },
@@ -2908,6 +2909,50 @@ __PACKAGE__->register_method({
     },
 });
 
+__PACKAGE__->register_method({
+    name => 'maintenance_set',
+    path => 'maintenance',
+    method => 'PUT',
+    description => "Enable or disable maintenance mode for the node. Currently, this only"
+        . " requests the HA node maintenance mode.",
+    protected => 1,
+    permissions => {
+        check => ['perm', '/nodes/{node}', ['Sys.Modify']],
+    },
+    parameters => {
+        additionalProperties => 0,
+        properties => {
+            node => get_standard_option('pve-node'),
+            'maintenance-mode' => {
+                description => "Specifies whether the maintenance mode for the specified"
+                    . " node should be enabled or disabled.",
+                type => 'string',
+                enum => ['enable', 'disable'],
+            },
+        },
+    },
+    returns => { type => 'null' },
+    code => sub {
+        my ($param) = @_;
+
+        my $node = $param->{node};
+        my $mode = $param->{'maintenance-mode'};
+
+        my $err = '';
+        eval {
+            PVE::Tools::run_command(
+                [
+                    'ha-manager', 'crm-command', 'node-maintenance', $mode, $node,
+                ],
+                errfunc => sub { $err .= "$_[0]\n" },
+            );
+        };
+        die($err || $@) if $@;
+
+        return undef;
+    },
+});
+
 # bash completion helper
 
 sub complete_templet_repo {
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH pve-manager v5 02/10] fix #6144: ui: ha: status: allow toggling maintenance mode
  2026-11-19  9:59 [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Dominik Rusovac
  2026-11-19  9:59 ` [PATCH pve-manager v5 01/10] fix #6144: api: nodes: add endpoint to set " Dominik Rusovac
@ 2026-11-19  9:59 ` Dominik Rusovac
  2026-11-19  9:59 ` [PATCH pve-manager v5 03/10] ui: ha: status: show spinner while maintenance mode request is queued Dominik Rusovac
                   ` (8 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Dominik Rusovac @ 2026-11-19  9:59 UTC (permalink / raw)
  To: pve-devel; +Cc: Thomas Skinner

maintenance mode could only be enabled or disabled through the HA
manager's CLI

add a 'Node Maintenance' menu to the HA status panel, which requests
enabling/disabling maintenance mode for the node of the selected lrm
entry

enabling is not offered while HA is disarmed, as the CRM ignores such
requests in that state. menu is disabled for users lacking the
Sys.Modify privilege on nodes

the LRM also enters maintenance mode on shutdown with the 'migrate'
shutdown policy, which a disable request would not undo

'fa fa-check' icon was chosen for 'Disable' option to be consistent
with which icon a node gets when coming back from maintenance mode

Originally-by: Thomas Skinner <thomas@atskinner.net>
Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---

Notes:
    changes since v4:
    * rename maintenanceNode to selectedLrmNode and comment which node
      the other maintenance field of the view model refers to
    * throw on unknown maintenance modes instead of defaulting to enable
    * split out the spinner into a separate patch
    * reword commit body accordingly

 www/manager6/ha/Status.js     | 98 +++++++++++++++++++++++++++++++++++
 www/manager6/ha/StatusView.js |  4 ++
 2 files changed, 102 insertions(+)

diff --git a/www/manager6/ha/Status.js b/www/manager6/ha/Status.js
index c01bd737..42e9aefb 100644
--- a/www/manager6/ha/Status.js
+++ b/www/manager6/ha/Status.js
@@ -11,6 +11,15 @@ Ext.define('PVE.ha.Status', {
     viewModel: {
         data: {
             haDisarmed: false,
+            selectedLrmNode: null,
+            // maintenance state of the selected LRM node
+            maintenanceRequested: false,
+        },
+        formulas: {
+            noMaintenanceAction: (get) =>
+                !get('selectedLrmNode') || !Ext.state.Manager.get('GuiCap').nodes['Sys.Modify'],
+            // the CRM ignores requests to enable maintenance mode while HA is disarmed
+            noMaintenanceEnable: (get) => get('maintenanceRequested') || get('haDisarmed'),
         },
     },
 
@@ -52,6 +61,65 @@ Ext.define('PVE.ha.Status', {
             );
         },
 
+        updateMaintenanceState: function () {
+            let me = this;
+            let vm = me.getViewModel();
+            let sv = me.getView().down('pveHAStatusView');
+            let rec = sv.getSelection()[0];
+            let isLrm = rec?.get('type') === 'lrm';
+
+            vm.set('selectedLrmNode', isLrm ? rec.get('node') : null);
+            vm.set('maintenanceRequested', isLrm && rec.get('maintenance-requested'));
+        },
+
+        handleMaintenanceButton: function (menuItem) {
+            let me = this;
+            let node = me.getViewModel().get('selectedLrmNode');
+            if (!node) {
+                return;
+            }
+
+            let msg;
+            if (menuItem.maintenanceMode === 'disable') {
+                msg = Ext.String.format(
+                    gettext("Are you sure you want to disable maintenance mode for node '{0}'?"),
+                    node,
+                );
+                if (me.getViewModel().get('haDisarmed')) {
+                    msg +=
+                        '<br><br>' +
+                        Ext.String.format(
+                            gettext(
+                                "HA resources will be migrated back to node '{0}' once HA is re-armed.",
+                            ),
+                            node,
+                        );
+                }
+            } else if (menuItem.maintenanceMode === 'enable') {
+                msg = Ext.String.format(
+                    gettext("Are you sure you want to enable maintenance mode for node '{0}'?"),
+                    node,
+                );
+                msg +=
+                    '<br><br>' +
+                    gettext('HA resources on the node will be migrated to other nodes.');
+            } else {
+                throw `unknown maintenance mode '${menuItem.maintenanceMode}'`;
+            }
+
+            Ext.Msg.confirm(gettext('Confirm'), msg, function (btn) {
+                if (btn !== 'yes') {
+                    return;
+                }
+                Proxmox.Utils.API2Request({
+                    url: `/nodes/${node}/maintenance`,
+                    params: { 'maintenance-mode': menuItem.maintenanceMode },
+                    method: 'PUT',
+                    failure: (response) => Ext.Msg.alert(gettext('Error'), response.htmlStatus),
+                });
+            });
+        },
+
         openCRSOptions: function () {
             Ext.create('PVE.form.CRSOptions', {
                 autoShow: true,
@@ -121,6 +189,34 @@ Ext.define('PVE.ha.Status', {
                 },
             ],
         },
+        '-',
+        {
+            text: gettext('Node Maintenance'),
+            iconCls: 'fa fa-wrench',
+            bind: {
+                disabled: '{noMaintenanceAction}',
+            },
+            menu: [
+                {
+                    text: gettext('Enable'),
+                    iconCls: 'fa fa-wrench',
+                    maintenanceMode: 'enable',
+                    bind: {
+                        disabled: '{noMaintenanceEnable}',
+                    },
+                    handler: 'handleMaintenanceButton',
+                },
+                {
+                    text: gettext('Disable'),
+                    iconCls: 'fa fa-check',
+                    maintenanceMode: 'disable',
+                    bind: {
+                        disabled: '{!maintenanceRequested}',
+                    },
+                    handler: 'handleMaintenanceButton',
+                },
+            ],
+        },
         '->',
         {
             text: gettext('CRS Settings'),
@@ -153,11 +249,13 @@ Ext.define('PVE.ha.Status', {
                 collapsible: true,
                 padding: '0 0 20 0',
                 listeners: {
+                    selectionchange: () => me.getController().updateMaintenanceState(),
                     hastatuschange: function (isDisarmed) {
                         let vm = me.getViewModel();
                         let sv = me.query('pveHAStatusView')[0];
 
                         vm.set('haDisarmed', isDisarmed);
+                        me.getController().updateMaintenanceState();
 
                         if (sv.isDisarmedPendingState === null) {
                             return;
diff --git a/www/manager6/ha/StatusView.js b/www/manager6/ha/StatusView.js
index f260bc0d..d08534b4 100644
--- a/www/manager6/ha/StatusView.js
+++ b/www/manager6/ha/StatusView.js
@@ -132,6 +132,10 @@ Ext.define(
                     },
                 },
                 'armed-state',
+                {
+                    name: 'maintenance-requested',
+                    type: 'boolean',
+                },
             ],
             idProperty: 'id',
         });
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH pve-manager v5 03/10] ui: ha: status: show spinner while maintenance mode request is queued
  2026-11-19  9:59 [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Dominik Rusovac
  2026-11-19  9:59 ` [PATCH pve-manager v5 01/10] fix #6144: api: nodes: add endpoint to set " Dominik Rusovac
  2026-11-19  9:59 ` [PATCH pve-manager v5 02/10] fix #6144: ui: ha: status: allow toggling " Dominik Rusovac
@ 2026-11-19  9:59 ` Dominik Rusovac
  2026-11-19  9:59 ` [PATCH pve-ha-manager v5 04/10] test: add test for ignoring maintenance mode request Dominik Rusovac
                   ` (7 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Dominik Rusovac @ 2026-11-19  9:59 UTC (permalink / raw)
  To: pve-devel

the CRM only processes a node maintenance request on its next round, so
until then, there is no feedback whether the request is still being
processed, and the menu still offers the same action again

show a spinner on the lrm entry of the node and keep the menu disabled
for it while the request is queued, like it is done when disarming HA.
this also ends the spinner if the CRM ignores the request, e.g. if the
node went offline in the meantime

as the request might not be reported as queued by a status load that was
already running, the node is also considered pending from the successful
request until a status load that started afterwards

Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---

Notes:
    new in v5, changes w.r.t. v4:
    * show the spinner while the request is queued for the CRM, so that
      it also ends if the CRM ignores the request, as pointed out in the
      v4 review
    * only initialize the per-node pending state in initComponent

 www/manager6/ha/Status.js     | 11 +++++++++-
 www/manager6/ha/StatusView.js | 41 ++++++++++++++++++++++++++++++++++-
 2 files changed, 50 insertions(+), 2 deletions(-)

diff --git a/www/manager6/ha/Status.js b/www/manager6/ha/Status.js
index 42e9aefb..44f95d61 100644
--- a/www/manager6/ha/Status.js
+++ b/www/manager6/ha/Status.js
@@ -14,10 +14,13 @@ Ext.define('PVE.ha.Status', {
             selectedLrmNode: null,
             // maintenance state of the selected LRM node
             maintenanceRequested: false,
+            maintenancePending: false,
         },
         formulas: {
             noMaintenanceAction: (get) =>
-                !get('selectedLrmNode') || !Ext.state.Manager.get('GuiCap').nodes['Sys.Modify'],
+                !get('selectedLrmNode') ||
+                get('maintenancePending') ||
+                !Ext.state.Manager.get('GuiCap').nodes['Sys.Modify'],
             // the CRM ignores requests to enable maintenance mode while HA is disarmed
             noMaintenanceEnable: (get) => get('maintenanceRequested') || get('haDisarmed'),
         },
@@ -70,6 +73,7 @@ Ext.define('PVE.ha.Status', {
 
             vm.set('selectedLrmNode', isLrm ? rec.get('node') : null);
             vm.set('maintenanceRequested', isLrm && rec.get('maintenance-requested'));
+            vm.set('maintenancePending', isLrm && sv.isMaintenancePending(rec));
         },
 
         handleMaintenanceButton: function (menuItem) {
@@ -115,6 +119,11 @@ Ext.define('PVE.ha.Status', {
                     url: `/nodes/${node}/maintenance`,
                     params: { 'maintenance-mode': menuItem.maintenanceMode },
                     method: 'PUT',
+                    success: function () {
+                        let sv = me.getView().down('pveHAStatusView');
+                        sv.setMaintenancePending(node);
+                        me.updateMaintenanceState();
+                    },
                     failure: (response) => Ext.Msg.alert(gettext('Error'), response.htmlStatus),
                 });
             });
diff --git a/www/manager6/ha/StatusView.js b/www/manager6/ha/StatusView.js
index d08534b4..cf8677e4 100644
--- a/www/manager6/ha/StatusView.js
+++ b/www/manager6/ha/StatusView.js
@@ -23,6 +23,19 @@ Ext.define(
             me.getView().refresh();
         },
 
+        setMaintenancePending: function (node) {
+            let me = this;
+            me.maintenanceRequestTime[node] = Date.now();
+            me.getView().refresh();
+        },
+
+        isMaintenancePending: function (rec) {
+            return (
+                rec.get('maintenance-request-queued') ||
+                this.maintenanceRequestTime[rec.get('node')] !== undefined
+            );
+        },
+
         initComponent: function () {
             var me = this;
 
@@ -30,6 +43,10 @@ Ext.define(
                 throw 'no rstore given';
             }
 
+            // maps node names to the time of their last maintenance request, until a status load
+            // that started afterwards reports whether the request is still queued for the CRM
+            me.maintenanceRequestTime = {};
+
             Proxmox.Utils.monStoreErrors(me, me.rstore);
 
             var store = Ext.create('Proxmox.data.DiffStore', {
@@ -51,7 +68,22 @@ Ext.define(
                 },
             });
 
+            me.rstore.on('beforeload', () => {
+                me.lastLoadStart = Date.now();
+            });
+
             me.rstore.on('load', function () {
+                let pendingCleared = false;
+                for (const [node, requestTime] of Object.entries(me.maintenanceRequestTime)) {
+                    if (requestTime < me.lastLoadStart) {
+                        delete me.maintenanceRequestTime[node];
+                        pendingCleared = true;
+                    }
+                }
+                if (pendingCleared) {
+                    me.getView().refresh();
+                }
+
                 let fencing = store.findRecord('type', 'fencing');
                 let disarmed = fencing && fencing.get('armed-state') === 'disarmed';
 
@@ -76,7 +108,10 @@ Ext.define(
                         flex: 1,
                         dataIndex: 'status',
                         renderer: function (value, _metaData, rec) {
-                            if (rec.data.type === 'fencing' && me.pending) {
+                            if (
+                                (rec.data.type === 'fencing' && me.pending) ||
+                                (rec.data.type === 'lrm' && me.isMaintenancePending(rec))
+                            ) {
                                 return value + ' <i class="fa fa-spinner fa-spin"></i>';
                             }
                             return value;
@@ -136,6 +171,10 @@ Ext.define(
                     name: 'maintenance-requested',
                     type: 'boolean',
                 },
+                {
+                    name: 'maintenance-request-queued',
+                    type: 'boolean',
+                },
             ],
             idProperty: 'id',
         });
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH pve-ha-manager v5 04/10] test: add test for ignoring maintenance mode request
  2026-11-19  9:59 [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Dominik Rusovac
                   ` (2 preceding siblings ...)
  2026-11-19  9:59 ` [PATCH pve-manager v5 03/10] ui: ha: status: show spinner while maintenance mode request is queued Dominik Rusovac
@ 2026-11-19  9:59 ` Dominik Rusovac
  2026-11-19  9:59 ` [PATCH pve-ha-manager v5 05/10] manager: standardize log messages for " Dominik Rusovac
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Dominik Rusovac @ 2026-11-19  9:59 UTC (permalink / raw)
  To: pve-devel

none of the existing tests covers the cases where the CRM ignores a
request to enable maintenance mode for a node that is already in
maintenance mode or while the HA stack is disarmed

Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---

Notes:
    new in v5

 src/test/test-crm-maintenance-request6/README |  3 +
 .../test-crm-maintenance-request6/cmdlist     |  9 ++
 .../hardware_status                           |  5 ++
 .../test-crm-maintenance-request6/log.expect  | 82 +++++++++++++++++++
 .../manager_status                            |  1 +
 .../service_config                            |  5 ++
 6 files changed, 105 insertions(+)
 create mode 100644 src/test/test-crm-maintenance-request6/README
 create mode 100644 src/test/test-crm-maintenance-request6/cmdlist
 create mode 100644 src/test/test-crm-maintenance-request6/hardware_status
 create mode 100644 src/test/test-crm-maintenance-request6/log.expect
 create mode 100644 src/test/test-crm-maintenance-request6/manager_status
 create mode 100644 src/test/test-crm-maintenance-request6/service_config

diff --git a/src/test/test-crm-maintenance-request6/README b/src/test/test-crm-maintenance-request6/README
new file mode 100644
index 0000000..a7a510f
--- /dev/null
+++ b/src/test/test-crm-maintenance-request6/README
@@ -0,0 +1,3 @@
+Test that the CRM ignores requests to enable the maintenance mode for a node
+that is already in maintenance mode, and while the HA stack is disarmed,
+respectively.
diff --git a/src/test/test-crm-maintenance-request6/cmdlist b/src/test/test-crm-maintenance-request6/cmdlist
new file mode 100644
index 0000000..574ee6a
--- /dev/null
+++ b/src/test/test-crm-maintenance-request6/cmdlist
@@ -0,0 +1,9 @@
+[
+    [ "power node1 on", "power node2 on", "power node3 on"],
+    [ "crm node3 enable-node-maintenance" ],
+    [ "crm node3 enable-node-maintenance" ],
+    [ "crm node1 disarm-ha freeze" ],
+    [ "crm node2 enable-node-maintenance" ],
+    [ "crm node1 arm-ha" ],
+    [ "crm node3 disable-node-maintenance" ]
+]
diff --git a/src/test/test-crm-maintenance-request6/hardware_status b/src/test/test-crm-maintenance-request6/hardware_status
new file mode 100644
index 0000000..451beb1
--- /dev/null
+++ b/src/test/test-crm-maintenance-request6/hardware_status
@@ -0,0 +1,5 @@
+{
+  "node1": { "power": "off", "network": "off" },
+  "node2": { "power": "off", "network": "off" },
+  "node3": { "power": "off", "network": "off" }
+}
diff --git a/src/test/test-crm-maintenance-request6/log.expect b/src/test/test-crm-maintenance-request6/log.expect
new file mode 100644
index 0000000..1db9138
--- /dev/null
+++ b/src/test/test-crm-maintenance-request6/log.expect
@@ -0,0 +1,82 @@
+info      0     hardware: starting simulation
+info     20      cmdlist: execute power node1 on
+info     20    node1/crm: status change startup => wait_for_quorum
+info     20    node1/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node2 on
+info     20    node2/crm: status change startup => wait_for_quorum
+info     20    node2/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node3 on
+info     20    node3/crm: status change startup => wait_for_quorum
+info     20    node3/lrm: status change startup => wait_for_agent_lock
+info     20    node1/crm: got lock 'ha_manager_lock'
+info     20    node1/crm: status change wait_for_quorum => master
+info     20    node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node3': state changed from 'unknown' => 'online'
+info     20    node1/crm: adding new service 'vm:101' on node 'node1'
+info     20    node1/crm: adding new service 'vm:102' on node 'node2'
+info     20    node1/crm: adding new service 'vm:103' on node 'node3'
+info     20    node1/crm: service 'vm:101': state changed from 'request_start' to 'started'  (node = node1)
+info     20    node1/crm: service 'vm:103': state changed from 'request_start' to 'started'  (node = node3)
+info     21    node1/lrm: got lock 'ha_agent_node1_lock'
+info     21    node1/lrm: status change wait_for_agent_lock => active
+info     21    node1/lrm: starting service vm:101
+info     21    node1/lrm: service status vm:101 started
+info     22    node2/crm: status change wait_for_quorum => slave
+info     23    node2/lrm: got lock 'ha_agent_node2_lock'
+info     23    node2/lrm: status change wait_for_agent_lock => active
+info     24    node3/crm: status change wait_for_quorum => slave
+info     25    node3/lrm: got lock 'ha_agent_node3_lock'
+info     25    node3/lrm: status change wait_for_agent_lock => active
+info     25    node3/lrm: starting service vm:103
+info     25    node3/lrm: service status vm:103 started
+info     40    node1/crm: service 'vm:102': state changed from 'request_stop' to 'stopped'
+info    120      cmdlist: execute crm node3 enable-node-maintenance
+info    125    node3/lrm: status change active => maintenance
+info    140    node1/crm: node 'node3': state changed from 'online' => 'maintenance'
+info    140    node1/crm: migrate service 'vm:103' to node 'node2' (running)
+info    140    node1/crm: service 'vm:103': state changed from 'started' to 'migrate'  (node = node3, target = node2)
+info    145    node3/lrm: service vm:103 - start migrate to node 'node2'
+info    145    node3/lrm: service vm:103 - end migrate to node 'node2'
+info    160    node1/crm: service 'vm:103': state changed from 'migrate' to 'started'  (node = node2)
+info    163    node2/lrm: starting service vm:103
+info    163    node2/lrm: service status vm:103 started
+info    220      cmdlist: execute crm node3 enable-node-maintenance
+info    220    node1/crm: ignoring crm command - node node3 is already in maintenance state
+info    320      cmdlist: execute crm node1 disarm-ha freeze
+info    320    node1/crm: got crm command: disarm-ha freeze
+info    320    node1/crm: disarm: freezing service 'vm:101' (was 'started')
+info    320    node1/crm: disarm: freezing service 'vm:102' (was 'stopped')
+info    320    node1/crm: disarm: freezing service 'vm:103' (was 'started')
+info    321    node1/lrm: HA disarm requested, releasing agent lock and watchdog
+info    321    node1/lrm: status change active => wait_for_agent_lock
+info    323    node2/lrm: HA disarm requested, releasing agent lock and watchdog
+info    323    node2/lrm: status change active => wait_for_agent_lock
+info    340    node1/crm: all LRMs disarmed, HA stack is now fully disarmed
+info    340    node1/crm: HA stack fully disarmed, releasing CRM watchdog
+info    420      cmdlist: execute crm node2 enable-node-maintenance
+warn    420    node1/crm: ignoring maintenance command for node node2 - HA stack is disarmed
+info    520      cmdlist: execute crm node1 arm-ha
+info    520    node1/crm: got crm command: arm-ha
+info    520    node1/crm: re-arming HA stack
+info    540    node1/crm: service 'vm:101': state changed from 'freeze' to 'started'
+info    540    node1/crm: service 'vm:102': state changed from 'freeze' to 'request_stop'
+info    540    node1/crm: service 'vm:103': state changed from 'freeze' to 'started'
+info    541    node1/lrm: got lock 'ha_agent_node1_lock'
+info    541    node1/lrm: status change wait_for_agent_lock => active
+info    543    node2/lrm: got lock 'ha_agent_node2_lock'
+info    543    node2/lrm: status change wait_for_agent_lock => active
+info    560    node1/crm: service 'vm:102': state changed from 'request_stop' to 'stopped'
+info    620      cmdlist: execute crm node3 disable-node-maintenance
+info    625    node3/lrm: got lock 'ha_agent_node3_lock'
+info    625    node3/lrm: status change maintenance => active
+info    640    node1/crm: node 'node3': state changed from 'maintenance' => 'online'
+info    640    node1/crm: moving service 'vm:103' back to 'node3', node came back from maintenance.
+info    640    node1/crm: migrate service 'vm:103' to node 'node3' (running)
+info    640    node1/crm: service 'vm:103': state changed from 'started' to 'migrate'  (node = node2, target = node3)
+info    643    node2/lrm: service vm:103 - start migrate to node 'node3'
+info    643    node2/lrm: service vm:103 - end migrate to node 'node3'
+info    660    node1/crm: service 'vm:103': state changed from 'migrate' to 'started'  (node = node3)
+info    665    node3/lrm: starting service vm:103
+info    665    node3/lrm: service status vm:103 started
+info   1220     hardware: exit simulation - done
diff --git a/src/test/test-crm-maintenance-request6/manager_status b/src/test/test-crm-maintenance-request6/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-crm-maintenance-request6/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-crm-maintenance-request6/service_config b/src/test/test-crm-maintenance-request6/service_config
new file mode 100644
index 0000000..c943ba6
--- /dev/null
+++ b/src/test/test-crm-maintenance-request6/service_config
@@ -0,0 +1,5 @@
+{
+    "vm:101": { "node": "node1", "state": "started" },
+    "vm:102": { "node": "node2" },
+    "vm:103": { "node": "node3", "state": "started" }
+}
\ No newline at end of file
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH pve-ha-manager v5 05/10] manager: standardize log messages for maintenance mode request
  2026-11-19  9:59 [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Dominik Rusovac
                   ` (3 preceding siblings ...)
  2026-11-19  9:59 ` [PATCH pve-ha-manager v5 04/10] test: add test for ignoring maintenance mode request Dominik Rusovac
@ 2026-11-19  9:59 ` Dominik Rusovac
  2026-11-19  9:59 ` [PATCH pve-ha-manager v5 06/10] manager: factor out check " Dominik Rusovac
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Dominik Rusovac @ 2026-11-19  9:59 UTC (permalink / raw)
  To: pve-devel

log messages for maintenance mode request now share a format describing
what the command does, and no longer depend on cmd variable

adapt the affected test logs accordingly

Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---

Notes:
    new in v5

 src/PVE/HA/Manager.pm                             | 6 +++---
 src/test/test-crm-maintenance-request2/log.expect | 2 +-
 src/test/test-crm-maintenance-request6/log.expect | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/src/PVE/HA/Manager.pm b/src/PVE/HA/Manager.pm
index 5c21fbe..90ef89d 100644
--- a/src/PVE/HA/Manager.pm
+++ b/src/PVE/HA/Manager.pm
@@ -794,17 +794,17 @@ sub update_crm_commands {
             if ($ms->{disarm}) {
                 $haenv->log(
                     'warning',
-                    "ignoring maintenance command for node $node - HA stack is disarmed",
+                    "ignoring maintenance mode request for node '$node' - HA stack is disarmed",
                 );
             } elsif ($state eq 'online') {
                 $ms->{node_request}->{$node}->{maintenance} = 1;
             } elsif ($state eq 'maintenance') {
                 $haenv->log(
                     'info',
-                    "ignoring crm command - node $node is already in maintenance state",
+                    "ignoring maintenance mode request for node '$node' - node is already in maintenance state",
                 );
             } else {
-                $haenv->log('err', "crm command error - node not online: $cmd");
+                $haenv->log('err', "maintenance mode request error - node '$node' not online");
             }
         } elsif ($cmd =~ m/^disable-node-maintenance\s+(\S+)$/) {
             my $node = $1;
diff --git a/src/test/test-crm-maintenance-request2/log.expect b/src/test/test-crm-maintenance-request2/log.expect
index c8d2563..abf86f8 100644
--- a/src/test/test-crm-maintenance-request2/log.expect
+++ b/src/test/test-crm-maintenance-request2/log.expect
@@ -43,7 +43,7 @@ info    165    node3/crm: killed by poweroff
 info    166    node3/lrm: killed by poweroff
 info    166     hardware: server 'node3' stopped by poweroff (watchdog)
 info    220      cmdlist: execute crm node3 enable-node-maintenance
-err     220    node1/crm: crm command error - node not online: enable-node-maintenance node3
+err     220    node1/crm: maintenance mode request error - node 'node3' not online
 info    240    node1/crm: got lock 'ha_agent_node3_lock'
 info    240    node1/crm: fencing: acknowledged - got agent lock for node 'node3'
 info    240    node1/crm: node 'node3': state changed from 'fence' => 'unknown'
diff --git a/src/test/test-crm-maintenance-request6/log.expect b/src/test/test-crm-maintenance-request6/log.expect
index 1db9138..7ca0d61 100644
--- a/src/test/test-crm-maintenance-request6/log.expect
+++ b/src/test/test-crm-maintenance-request6/log.expect
@@ -42,7 +42,7 @@ info    160    node1/crm: service 'vm:103': state changed from 'migrate' to 'sta
 info    163    node2/lrm: starting service vm:103
 info    163    node2/lrm: service status vm:103 started
 info    220      cmdlist: execute crm node3 enable-node-maintenance
-info    220    node1/crm: ignoring crm command - node node3 is already in maintenance state
+info    220    node1/crm: ignoring maintenance mode request for node 'node3' - node is already in maintenance state
 info    320      cmdlist: execute crm node1 disarm-ha freeze
 info    320    node1/crm: got crm command: disarm-ha freeze
 info    320    node1/crm: disarm: freezing service 'vm:101' (was 'started')
@@ -55,7 +55,7 @@ info    323    node2/lrm: status change active => wait_for_agent_lock
 info    340    node1/crm: all LRMs disarmed, HA stack is now fully disarmed
 info    340    node1/crm: HA stack fully disarmed, releasing CRM watchdog
 info    420      cmdlist: execute crm node2 enable-node-maintenance
-warn    420    node1/crm: ignoring maintenance command for node node2 - HA stack is disarmed
+warn    420    node1/crm: ignoring maintenance mode request for node 'node2' - HA stack is disarmed
 info    520      cmdlist: execute crm node1 arm-ha
 info    520    node1/crm: got crm command: arm-ha
 info    520    node1/crm: re-arming HA stack
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH pve-ha-manager v5 06/10] manager: factor out check for maintenance mode request
  2026-11-19  9:59 [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Dominik Rusovac
                   ` (4 preceding siblings ...)
  2026-11-19  9:59 ` [PATCH pve-ha-manager v5 05/10] manager: standardize log messages for " Dominik Rusovac
@ 2026-11-19  9:59 ` Dominik Rusovac
  2026-11-19  9:59 ` [PATCH pve-ha-manager v5 07/10] cli: reject maintenance mode requests rejected by crm Dominik Rusovac
                   ` (4 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Dominik Rusovac @ 2026-11-19  9:59 UTC (permalink / raw)
  To: pve-devel

move the reasons why the crm ignores a maintenance mode request into a
helper, so that the CLI can use the same logic to reject such requests
in advance, instead of queuing a crm command that is only ignored with a
log message

Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---

Notes:
    new in v5:
    * uses signature for function in Helpers to be consistent with rest of
      file

 src/PVE/HA/Config.pm  |  9 +++++++++
 src/PVE/HA/Helpers.pm | 21 +++++++++++++++++++++
 src/PVE/HA/Manager.pm | 19 ++++++-------------
 3 files changed, 36 insertions(+), 13 deletions(-)

diff --git a/src/PVE/HA/Config.pm b/src/PVE/HA/Config.pm
index bee1ac7..b3ab77b 100644
--- a/src/PVE/HA/Config.pm
+++ b/src/PVE/HA/Config.pm
@@ -423,6 +423,15 @@ sub get_resource_motion_info {
     return ($dependent_resources, $blocking_resources_by_node);
 }
 
+sub check_enable_maintenance {
+    my ($node, $log_fn) = @_;
+
+    my $ms = read_manager_status();
+    my $ns = ($ms->{node_status} // {})->{$node} // 'unknown';
+
+    return PVE::HA::Helpers::check_enable_maintenance($log_fn, $node, $ns, $ms);
+}
+
 # graceful, as long as locking + cfs_write works
 sub delete_service_from_config {
     my ($sid, $purge) = @_;
diff --git a/src/PVE/HA/Helpers.pm b/src/PVE/HA/Helpers.pm
index a58b1e1..ba69093 100644
--- a/src/PVE/HA/Helpers.pm
+++ b/src/PVE/HA/Helpers.pm
@@ -60,4 +60,25 @@ sub get_resource_motion_info($ss, $sid, $cd, $online_nodes, $compiled_rules) {
     return ($dependent_resources, $blocking_resources_by_node);
 }
 
+sub check_enable_maintenance($log_fn, $node, $node_state, $ms) {
+    if ($ms->{disarm}) {
+        $log_fn->(
+            'warning',
+            "ignoring maintenance mode request for node '$node' - HA stack is disarmed",
+        );
+        return 0;
+    } elsif ($node_state eq 'maintenance') {
+        $log_fn->(
+            'info',
+            "ignoring maintenance mode request for node '$node' - node is already in maintenance state",
+        );
+        return 0;
+    } elsif ($node_state ne 'online') {
+        $log_fn->('err', "maintenance mode request error - node '$node' not online");
+        return 0;
+    }
+
+    return 1;
+}
+
 1;
diff --git a/src/PVE/HA/Manager.pm b/src/PVE/HA/Manager.pm
index 90ef89d..c012750 100644
--- a/src/PVE/HA/Manager.pm
+++ b/src/PVE/HA/Manager.pm
@@ -770,6 +770,11 @@ sub update_crm_commands {
 
     my $cmdlist = $haenv->read_crm_commands();
 
+    my $log_fn = sub {
+        my ($level, $msg) = @_;
+        $haenv->log($level, $msg);
+    };
+
     foreach my $cmd (split(/\n/, $cmdlist)) {
         chomp $cmd;
 
@@ -791,20 +796,8 @@ sub update_crm_commands {
             my $node = $1;
 
             my $state = $ns->get_node_state($node);
-            if ($ms->{disarm}) {
-                $haenv->log(
-                    'warning',
-                    "ignoring maintenance mode request for node '$node' - HA stack is disarmed",
-                );
-            } elsif ($state eq 'online') {
+            if (PVE::HA::Helpers::check_enable_maintenance($log_fn, $node, $state, $ms)) {
                 $ms->{node_request}->{$node}->{maintenance} = 1;
-            } elsif ($state eq 'maintenance') {
-                $haenv->log(
-                    'info',
-                    "ignoring maintenance mode request for node '$node' - node is already in maintenance state",
-                );
-            } else {
-                $haenv->log('err', "maintenance mode request error - node '$node' not online");
             }
         } elsif ($cmd =~ m/^disable-node-maintenance\s+(\S+)$/) {
             my $node = $1;
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH pve-ha-manager v5 07/10] cli: reject maintenance mode requests rejected by crm
  2026-11-19  9:59 [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Dominik Rusovac
                   ` (5 preceding siblings ...)
  2026-11-19  9:59 ` [PATCH pve-ha-manager v5 06/10] manager: factor out check " Dominik Rusovac
@ 2026-11-19  9:59 ` Dominik Rusovac
  2026-11-19  9:59 ` [PATCH pve-ha-manager v5 08/10] api: status: report maintenance mode request for lrm entries Dominik Rusovac
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Dominik Rusovac @ 2026-11-19  9:59 UTC (permalink / raw)
  To: pve-devel

the CLI queued maintenance mode requests with no checks, so users only
learned from the CRM's log that a request was rejected

reject those before queuing the CRM command using the same logic as the
CRM. this only concerns requests to enable maintenance mode, as the CRM
always processes requests to disable it

requests for nodes unknown to HA are rejected as well, since maintenance
mode only affects HA managed resources and the queued command would make
an idle CRM become active

dying in the log function is supposed to forward the reason for
rejection to callers

deref node parameter once

Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---

Notes:
    new in v5

 src/PVE/CLI/ha_manager.pm | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/src/PVE/CLI/ha_manager.pm b/src/PVE/CLI/ha_manager.pm
index 6625de6..74f18d9 100644
--- a/src/PVE/CLI/ha_manager.pm
+++ b/src/PVE/CLI/ha_manager.pm
@@ -138,10 +138,26 @@ __PACKAGE__->register_method({
     code => sub {
         my ($param) = @_;
 
-        PVE::Cluster::check_node_exists($param->{node});
+        my $node = $param->{node};
+        PVE::Cluster::check_node_exists($node);
+
+        # maintenance mode only affects HA managed resources and queuing the CRM command for a
+        # node unknown to HA would make an idle CRM become active for nothing
+        my $node_status = PVE::HA::Config::read_manager_status()->{node_status} // {};
+        die "node '$node' is not part of HA\n" if !defined($node_status->{$node});
+
+        my $disable = $param->{disable};
+        if (!$disable) {
+            my $log_fn = sub {
+                my ($level, $msg) = @_;
+                die "$msg\n";
+            };
+
+            PVE::HA::Config::check_enable_maintenance($node, $log_fn);
+        }
 
-        my $cmd = $param->{disable} ? 'disable-node-maintenance' : 'enable-node-maintenance';
-        PVE::HA::Config::queue_crm_commands("$cmd $param->{node}");
+        my $cmd = $disable ? 'disable-node-maintenance' : 'enable-node-maintenance';
+        PVE::HA::Config::queue_crm_commands("$cmd $node");
 
         return undef;
     },
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH pve-ha-manager v5 08/10] api: status: report maintenance mode request for lrm entries
  2026-11-19  9:59 [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Dominik Rusovac
                   ` (6 preceding siblings ...)
  2026-11-19  9:59 ` [PATCH pve-ha-manager v5 07/10] cli: reject maintenance mode requests rejected by crm Dominik Rusovac
@ 2026-11-19  9:59 ` Dominik Rusovac
  2026-11-19  9:59 ` [PATCH pve-ha-manager v5 09/10] api: status: report queued " Dominik Rusovac
                   ` (2 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Dominik Rusovac @ 2026-11-19  9:59 UTC (permalink / raw)
  To: pve-devel; +Cc: Thomas Skinner

expose whether maintenance mode was manually requested for a node, as
recorded by the CRM

Originally-by: Thomas Skinner <thomas@atskinner.net>
Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
Reviewed-by: Daniel Kral <d.kral@proxmox.com>
---

Notes:
    changes since v4:
    * reword commit subject

 src/PVE/API2/HA/Status.pm | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/src/PVE/API2/HA/Status.pm b/src/PVE/API2/HA/Status.pm
index aecef0f..2622ec1 100644
--- a/src/PVE/API2/HA/Status.pm
+++ b/src/PVE/API2/HA/Status.pm
@@ -107,6 +107,12 @@ __PACKAGE__->register_method({
                     type => "integer",
                     optional => 1,
                 },
+                'maintenance-requested' => {
+                    description => "For type 'lrm'. Whether maintenance mode was manually"
+                        . " requested for the node.",
+                    type => "boolean",
+                    optional => 1,
+                },
                 crm_state => {
                     description => "For type 'service'. Service state as seen by the CRM.",
                     type => "string",
@@ -265,6 +271,8 @@ __PACKAGE__->register_method({
                 PVE::HA::Tools::count_active_services($status->{service_status}, $node);
             my $lrm_status = PVE::HA::Config::read_lrm_status($node);
             my $id = "lrm:$node";
+            my $node_request = $status->{node_request} // {};
+            my $maintenance_requested = $node_request->{$node}->{maintenance} ? 1 : 0;
             if (!$lrm_status->{timestamp}) {
                 push @$res,
                     {
@@ -272,6 +280,7 @@ __PACKAGE__->register_method({
                         type => 'lrm',
                         node => $node,
                         status => "$node (unable to read lrm status)",
+                        'maintenance-requested' => $maintenance_requested,
                     };
             } else {
                 my $status_str = &$timestamp_to_status($ctime, $lrm_status->{timestamp});
@@ -315,6 +324,7 @@ __PACKAGE__->register_method({
                         node => $node,
                         status => $status_text,
                         timestamp => $lrm_status->{timestamp},
+                        'maintenance-requested' => $maintenance_requested,
                     };
             }
         }
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH pve-ha-manager v5 09/10] api: status: report queued maintenance mode request for lrm entries
  2026-11-19  9:59 [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Dominik Rusovac
                   ` (7 preceding siblings ...)
  2026-11-19  9:59 ` [PATCH pve-ha-manager v5 08/10] api: status: report maintenance mode request for lrm entries Dominik Rusovac
@ 2026-11-19  9:59 ` Dominik Rusovac
  2026-11-19  9:59 ` [PATCH pve-docs v5 10/10] ha-manager: maintenance mode: mention web ui menu Dominik Rusovac
  2026-11-19 22:28 ` applied: [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Thomas Lamprecht
  10 siblings, 0 replies; 12+ messages in thread
From: Dominik Rusovac @ 2026-11-19  9:59 UTC (permalink / raw)
  To: pve-devel

expose whether a request to enable/disable maintenance mode for a node
is still queued, so that clients can tell when the CRM processed it; no
matter if CRM accepted or ignored the request

peeking into the queued CRM commands reads them without consuming them

Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---

Notes:
    new in v5:
    * towards fixing the bug mentioned in the review of v4
    * only needed for the spinner, can be left out together with it

 src/PVE/API2/HA/Status.pm | 16 ++++++++++++++++
 src/PVE/HA/Config.pm      |  4 ++++
 2 files changed, 20 insertions(+)

diff --git a/src/PVE/API2/HA/Status.pm b/src/PVE/API2/HA/Status.pm
index 2622ec1..e3a4e16 100644
--- a/src/PVE/API2/HA/Status.pm
+++ b/src/PVE/API2/HA/Status.pm
@@ -113,6 +113,13 @@ __PACKAGE__->register_method({
                     type => "boolean",
                     optional => 1,
                 },
+                'maintenance-request-queued' => {
+                    description => "For type 'lrm'. Whether a request to enable or disable"
+                        . " maintenance mode for the node is queued and not yet processed by"
+                        . " the CRM.",
+                    type => "boolean",
+                    optional => 1,
+                },
                 crm_state => {
                     description => "For type 'service'. Service state as seen by the CRM.",
                     type => "string",
@@ -266,6 +273,12 @@ __PACKAGE__->register_method({
                 };
         }
 
+        my $maintenance_queued_nodes = {};
+        for my $cmd (split(/\n/, PVE::HA::Config::peek_crm_commands())) {
+            next if $cmd !~ m/^(?:enable|disable)-node-maintenance\s+(\S+)$/;
+            $maintenance_queued_nodes->{$1} = 1;
+        }
+
         foreach my $node (sort keys %{ $status->{node_status} }) {
             my $active_count =
                 PVE::HA::Tools::count_active_services($status->{service_status}, $node);
@@ -273,6 +286,7 @@ __PACKAGE__->register_method({
             my $id = "lrm:$node";
             my $node_request = $status->{node_request} // {};
             my $maintenance_requested = $node_request->{$node}->{maintenance} ? 1 : 0;
+            my $maintenance_queued = $maintenance_queued_nodes->{$node} ? 1 : 0;
             if (!$lrm_status->{timestamp}) {
                 push @$res,
                     {
@@ -281,6 +295,7 @@ __PACKAGE__->register_method({
                         node => $node,
                         status => "$node (unable to read lrm status)",
                         'maintenance-requested' => $maintenance_requested,
+                        'maintenance-request-queued' => $maintenance_queued,
                     };
             } else {
                 my $status_str = &$timestamp_to_status($ctime, $lrm_status->{timestamp});
@@ -325,6 +340,7 @@ __PACKAGE__->register_method({
                         status => $status_text,
                         timestamp => $lrm_status->{timestamp},
                         'maintenance-requested' => $maintenance_requested,
+                        'maintenance-request-queued' => $maintenance_queued,
                     };
             }
         }
diff --git a/src/PVE/HA/Config.pm b/src/PVE/HA/Config.pm
index b3ab77b..4f02d5f 100644
--- a/src/PVE/HA/Config.pm
+++ b/src/PVE/HA/Config.pm
@@ -341,6 +341,10 @@ sub any_pending_crm_command {
     return defined($data) && length $data;
 }
 
+sub peek_crm_commands {
+    return cfs_read_file($crm_commands_filename) // '';
+}
+
 sub read_crm_commands {
 
     my $code = sub {
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH pve-docs v5 10/10] ha-manager: maintenance mode: mention web ui menu
  2026-11-19  9:59 [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Dominik Rusovac
                   ` (8 preceding siblings ...)
  2026-11-19  9:59 ` [PATCH pve-ha-manager v5 09/10] api: status: report queued " Dominik Rusovac
@ 2026-11-19  9:59 ` Dominik Rusovac
  2026-11-19 22:28 ` applied: [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Thomas Lamprecht
  10 siblings, 0 replies; 12+ messages in thread
From: Dominik Rusovac @ 2026-11-19  9:59 UTC (permalink / raw)
  To: pve-devel; +Cc: Thomas Skinner

maintenance mode can now also be enabled/disabled from the HA status
panel

describe where to find it and that it cannot be enabled while HA is
disarmed

fix the "enabled or disable" typo in the rewritten sentence

Originally-by: Thomas Skinner <thomas@atskinner.net>
Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---

Notes:
    no changes since v4

 ha-manager.adoc | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/ha-manager.adoc b/ha-manager.adoc
index 95ab799..4962bb9 100644
--- a/ha-manager.adoc
+++ b/ha-manager.adoc
@@ -1199,8 +1199,10 @@ During each migration, the original node will be recorded in the HA managers'
 state, so that the service can be moved back again automatically once the
 maintenance mode is disabled and the node is back online.
 
-Currently you can enabled or disable the maintenance mode using the ha-manager
-CLI tool.
+You can enable or disable the maintenance mode using the web interface or the
+`ha-manager` CLI tool. In the web interface, select the `lrm` entry of the node
+in the status panel under __Datacenter -> HA__ and use the *Node Maintenance*
+menu. Maintenance mode cannot be enabled while HA is disarmed.
 
 .Enabling maintenance mode for a node
 ----
@@ -1225,8 +1227,8 @@ out for a log line like: `pve-ha-lrm[PID]: watchdog closed (disabled)` to know
 when the node finished its transition into the maintenance mode.
 
 NOTE: The manual maintenance mode is not automatically deleted on node reboot,
-but only if it is either manually deactivated using the `ha-manager` CLI or if
-the manager-status is manually cleared.
+but only if it is either manually deactivated or if the manager-status is
+manually cleared.
 
 .Disabling maintenance mode for a node
 ----
@@ -1234,8 +1236,9 @@ the manager-status is manually cleared.
 ----
 
 The process of disabling the manual maintenance mode is similar to enabling it.
-Using the `ha-manager` CLI command shown above will queue a CRM command that,
-once processed, marks the respective LRM node as available again.
+Using the web interface or the `ha-manager` CLI command shown above will queue a
+CRM command that, once processed, marks the respective LRM node as available
+again.
 
 If you deactivate the maintenance mode, all services that were on the node when
 the maintenance mode was activated will be moved back.
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 12+ messages in thread

* applied: [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode
  2026-11-19  9:59 [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Dominik Rusovac
                   ` (9 preceding siblings ...)
  2026-11-19  9:59 ` [PATCH pve-docs v5 10/10] ha-manager: maintenance mode: mention web ui menu Dominik Rusovac
@ 2026-11-19 22:28 ` Thomas Lamprecht
  10 siblings, 0 replies; 12+ messages in thread
From: Thomas Lamprecht @ 2026-11-19 22:28 UTC (permalink / raw)
  To: pve-devel, Dominik Rusovac

On Thu, 19 Nov 2026 10:59:21 +0100, Dominik Rusovac wrote:
> this picks up Thomas' v3 [0], rebased onto current master and reworked
> in a few places; thx for the nice work Thomas! I added `Originally-by`
> trailers
> 
> node maintenance mode can currently only be enabled/disabled with the HA
> manager's CRM command in the CLI. this series adds an API endpoint and a
> 'Node Maintenance' menu in the HA status panel for it
> 
> [...]

Applied, thanks!

I folded some changes into the patches, and added two follow-ups:

ha-manager:
- the shared check returns the log level and reason instead of taking a
  log callback, so that the CRM and the CLI each phrase the message; the
  'not online' message uses the common format now too
- cli: requesting maintenance mode again for a node it is already
  requested for succeeds, instead of failing with the CRM's "already in
  maintenance state" log message. The early check only runs while the
  recorded state is current, i.e. with an active CRM and no other queued
  command such as arm-ha, otherwise the CRM decides
- status API: parse the queued commands in PVE::HA::Config
- follow-up: the CRM now records an enable request for a node that is in
  maintenance without one, e.g. while it shuts down with the 'migrate'
  shutdown policy, so that it stays in maintenance once back online.
  This also keeps an enable queued right after a disable

manager:
- the endpoint requires Sys.Modify on / instead of /nodes/{node}, as
  maintenance moves HA resources to other nodes; the UI checks the same
- ui: Enable is also disabled while HA is still disarming. The spinner
  tracks pending requests in a Map, as a node named 'constructor' was
  always pending, keeps them on failed loads and compares them to the
  start of the load that finished

docs:
- mention that requests for nodes that are not online are rejected,
  except during a 'migrate' policy shutdown, which a follow-up documents

[pve-docs] applied to pve-docs.git (master)
[1/1] ha-manager: maintenance mode: mention web ui menu
      https://git.proxmox.com/?p=pve-docs.git;a=commitdiff;h=744af141b66235061a729ba226bd2b0644f6cf25

[pve-ha-manager] applied to pve-ha-manager.git (master)
[1/6] test: add test for ignoring maintenance mode request
      https://git.proxmox.com/?p=pve-ha-manager.git;a=commitdiff;h=122e20ce41c19bd62506413b91b39b0e13e4f5fe
[2/6] manager: standardize log messages for maintenance mode request
      https://git.proxmox.com/?p=pve-ha-manager.git;a=commitdiff;h=9aeb853a02b80596c346501d76e6fec1f6b11c12
[3/6] manager: factor out check for maintenance mode request
      https://git.proxmox.com/?p=pve-ha-manager.git;a=commitdiff;h=b7787287ec1565ed517d7f61f1b175062dc4d808
[4/6] cli: reject maintenance mode requests rejected by crm
      https://git.proxmox.com/?p=pve-ha-manager.git;a=commitdiff;h=669933296ffd2854b96e7cf64014d62aaf737a11
[5/6] api: status: report maintenance mode request for lrm entries
      https://git.proxmox.com/?p=pve-ha-manager.git;a=commitdiff;h=a885666e11be7365209cd189b9a253e044aa5091
[6/6] api: status: report queued maintenance mode request for lrm entries
      https://git.proxmox.com/?p=pve-ha-manager.git;a=commitdiff;h=ef9c125a0277a80289e045a917eaa342fb17199f

[pve-manager] applied to pve-manager.git (master)
[1/3] fix #6144: api: nodes: add endpoint to set maintenance mode
      https://git.proxmox.com/?p=pve-manager.git;a=commitdiff;h=b65fde432d9a6a5d529b54a10e8bd6126bf3e978
[2/3] fix #6144: ui: ha: status: allow toggling maintenance mode
      https://git.proxmox.com/?p=pve-manager.git;a=commitdiff;h=96a8910070db987bb063f21f834aa97298e0d215
[3/3] ui: ha: status: show spinner while maintenance mode request is queued
      https://git.proxmox.com/?p=pve-manager.git;a=commitdiff;h=62c6654ee8eaeddec412dd8b4d053c946cb9b29b




^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2026-11-19 22:29 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-19  9:59 [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Dominik Rusovac
2026-11-19  9:59 ` [PATCH pve-manager v5 01/10] fix #6144: api: nodes: add endpoint to set " Dominik Rusovac
2026-11-19  9:59 ` [PATCH pve-manager v5 02/10] fix #6144: ui: ha: status: allow toggling " Dominik Rusovac
2026-11-19  9:59 ` [PATCH pve-manager v5 03/10] ui: ha: status: show spinner while maintenance mode request is queued Dominik Rusovac
2026-11-19  9:59 ` [PATCH pve-ha-manager v5 04/10] test: add test for ignoring maintenance mode request Dominik Rusovac
2026-11-19  9:59 ` [PATCH pve-ha-manager v5 05/10] manager: standardize log messages for " Dominik Rusovac
2026-11-19  9:59 ` [PATCH pve-ha-manager v5 06/10] manager: factor out check " Dominik Rusovac
2026-11-19  9:59 ` [PATCH pve-ha-manager v5 07/10] cli: reject maintenance mode requests rejected by crm Dominik Rusovac
2026-11-19  9:59 ` [PATCH pve-ha-manager v5 08/10] api: status: report maintenance mode request for lrm entries Dominik Rusovac
2026-11-19  9:59 ` [PATCH pve-ha-manager v5 09/10] api: status: report queued " Dominik Rusovac
2026-11-19  9:59 ` [PATCH pve-docs v5 10/10] ha-manager: maintenance mode: mention web ui menu Dominik Rusovac
2026-11-19 22:28 ` applied: [PATCH-SERIES docs/ha-manager/manager v5 00/10] fix #6144: add api and ui for node maintenance mode Thomas Lamprecht

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal