[pve-devel] [PATCH-SERIES v2 (ha-)manager] allow node HA shutdown policy override

[pve-devel] [PATCH-SERIES v2 (ha-)manager] allow node HA shutdown policy override

[Fiona Ebner]

Make it possible to specify the HA shutdown policy for the
/nodes/{node}/status POST API enpoint for user flexibilty and
convenience.

The override is written to the LRM service's (new dedicated) runtime
directory. The LRM will check and honor the override when it receives
a shutdown/reboot request.


(Build-)depedency bump pve-manager -> libpve-cluster-perl needed.

If patch ha-manager 5/7 is applied: (build-)depedency bump
pve-ha-manager -> libpve-cluster-perl needed.

Dependency bump pve-manager -> pve-ha-manager needed (to have the
runtime directory exist and LRM honor the new param).

Changes in v2:
    * Also check for Sys.Modify permisson when shutdown policy
      parameter is used.
    * Add UI patch.
    * Drop already applied cluster patch.


ha-manager:

Fiona Ebner (7):
  lrm service: add runtime directory
  lrm service: move PID file to service's runtime directory
  pve2 env: get shutdown policy override set by node's shutdown API
    endpoint
  lrm: honor shutdown policy override set by node's shutdown API
    endpoint
  pve2 env: validate shutdown policy from override file
  sim env: add support for datacenter config overrides
  tests: add test for shutdown policy override

 debian/pve-ha-lrm.service                     |  3 +-
 src/PVE/HA/Env/PVE2.pm                        | 19 +++++++
 src/PVE/HA/LRM.pm                             |  5 +-
 src/PVE/HA/Sim/Env.pm                         |  1 +
 src/PVE/Service/pve_ha_lrm.pm                 |  5 +-
 src/test/test-shutdown-policy-override/README |  3 +
 .../test-shutdown-policy-override/cmdlist     |  4 ++
 .../datacenter.cfg                            | 10 ++++
 .../hardware_status                           |  5 ++
 .../test-shutdown-policy-override/log.expect  | 57 +++++++++++++++++++
 .../manager_status                            |  1 +
 .../service_config                            |  6 ++
 12 files changed, 116 insertions(+), 3 deletions(-)
 create mode 100644 src/test/test-shutdown-policy-override/README
 create mode 100644 src/test/test-shutdown-policy-override/cmdlist
 create mode 100644 src/test/test-shutdown-policy-override/datacenter.cfg
 create mode 100644 src/test/test-shutdown-policy-override/hardware_status
 create mode 100644 src/test/test-shutdown-policy-override/log.expect
 create mode 100644 src/test/test-shutdown-policy-override/manager_status
 create mode 100644 src/test/test-shutdown-policy-override/service_config


manager:

Fiona Ebner (2):
  api: nodes: allow setting HA shutdown policy during shutdown/reboot
  ui: allow specifying shutdown policy during node shutdown/reboot

 PVE/API2/Nodes.pm                   |  18 +++-
 www/manager6/Makefile               |   1 +
 www/manager6/node/Config.js         |  32 +++----
 www/manager6/window/NodeShutdown.js | 126 ++++++++++++++++++++++++++++
 4 files changed, 156 insertions(+), 21 deletions(-)
 create mode 100644 www/manager6/window/NodeShutdown.js

-- 
2.39.2

[pve-devel] [PATCH v2 ha-manager 1/7] lrm service: add runtime directory

[Fiona Ebner]

in preparation to allow overriding the shutdown policy with a file
there.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

No changes in v2.

 debian/pve-ha-lrm.service | 1 +
 1 file changed, 1 insertion(+)

diff --git a/debian/pve-ha-lrm.service b/debian/pve-ha-lrm.service
index fb1b5db..9f3f73d 100644
--- a/debian/pve-ha-lrm.service
+++ b/debian/pve-ha-lrm.service
@@ -22,6 +22,7 @@ After=watchdog-mux.service
 ExecStart=/usr/sbin/pve-ha-lrm start
 ExecStop=/usr/sbin/pve-ha-lrm stop
 PIDFile=/run/pve-ha-lrm.pid
+RuntimeDirectory=pve-ha-lrm
 TimeoutStopSec=infinity
 KillMode=process
 Type=forking
-- 
2.39.2

[pve-devel] [RFC v2 ha-manager 2/7] lrm service: move PID file to service's runtime directory

[Fiona Ebner]

it's arguably cleaner to put it there, now that there is one.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

No changes in v2.

 debian/pve-ha-lrm.service     | 2 +-
 src/PVE/Service/pve_ha_lrm.pm | 5 ++++-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/debian/pve-ha-lrm.service b/debian/pve-ha-lrm.service
index 9f3f73d..d9fcf87 100644
--- a/debian/pve-ha-lrm.service
+++ b/debian/pve-ha-lrm.service
@@ -21,7 +21,7 @@ After=watchdog-mux.service
 [Service]
 ExecStart=/usr/sbin/pve-ha-lrm start
 ExecStop=/usr/sbin/pve-ha-lrm stop
-PIDFile=/run/pve-ha-lrm.pid
+PIDFile=/run/pve-ha-lrm/pve-ha-lrm.pid
 RuntimeDirectory=pve-ha-lrm
 TimeoutStopSec=infinity
 KillMode=process
diff --git a/src/PVE/Service/pve_ha_lrm.pm b/src/PVE/Service/pve_ha_lrm.pm
index 91a9409..0790215 100644
--- a/src/PVE/Service/pve_ha_lrm.pm
+++ b/src/PVE/Service/pve_ha_lrm.pm
@@ -13,7 +13,10 @@ use base qw(PVE::Daemon);
 
 my $cmdline = [$0, @ARGV];
 
-my %daemon_options = (stop_wait_time => 60*60);
+my %daemon_options = (
+    stop_wait_time => 60*60,
+    pidfile => '/run/pve-ha-lrm/pve-ha-lrm.pid',
+);
 
 my $daemon = __PACKAGE__->new('pve-ha-lrm', $cmdline, %daemon_options);
 
-- 
2.39.2

[pve-devel] [PATCH v2 ha-manager 3/7] pve2 env: get shutdown policy override set by node's shutdown API endpoint

[Fiona Ebner]

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

No changes in v2.

 src/PVE/HA/Env/PVE2.pm | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index f6ebfeb..6a75bd4 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -458,9 +458,17 @@ sub get_datacenter_settings {
     my $datacenterconfig = eval { cfs_read_file('datacenter.cfg') };
     $self->log('err', "unable to get HA settings from datacenter.cfg - $@") if $@;
 
+    my $sp_override_fn = '/run/pve-ha-lrm/shutdown-policy.local-override';
+    my $shutdown_policy_override = eval { PVE::Tools::file_read_firstline($sp_override_fn); };
+    $self->log('warning', "error reading shutdown policy override from $sp_override_fn - $@") if $@;
+
+    my $overrides = {};
+    $overrides->{ha}->{shutdown_policy} = $shutdown_policy_override if $shutdown_policy_override;
+
     return {
 	ha => $datacenterconfig->{ha} // {},
 	crs => $datacenterconfig->{crs} // {},
+	'local-overrides' => $overrides,
     };
 }
 
-- 
2.39.2

[pve-devel] [PATCH v2 ha-manager 4/7] lrm: honor shutdown policy override set by node's shutdown API endpoint

[Fiona Ebner]

The /nodes/<node>/status API endpoint in pve-manager allows to specify
the shutdown policy now. This change is required for it to have any
effect of course.

Being able to choose the shutdown policy on a per-node/per-shutdown
is more flexible and user-friendly.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

No changes in v2.

 src/PVE/HA/LRM.pm | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/src/PVE/HA/LRM.pm b/src/PVE/HA/LRM.pm
index b6ac0fe..b8b842f 100644
--- a/src/PVE/HA/LRM.pm
+++ b/src/PVE/HA/LRM.pm
@@ -60,7 +60,10 @@ sub shutdown_request {
     my ($shutdown, $reboot) = $haenv->is_node_shutdown();
 
     my $dc_cfg = $haenv->get_datacenter_settings();
-    my $shutdown_policy = $dc_cfg->{ha}->{shutdown_policy} // 'conditional';
+    my $dc_ha_overrides = $dc_cfg->{'local-overrides'}->{ha} // {};
+
+    my $shutdown_policy =
+	$dc_ha_overrides->{shutdown_policy} // $dc_cfg->{ha}->{shutdown_policy} // 'conditional';
 
     if ($shutdown) { # don't log this on service restart, only on node shutdown
 	$haenv->log('info', "got shutdown request with shutdown policy '$shutdown_policy'");
-- 
2.39.2

[pve-devel] [RFC v2 ha-manager 5/7] pve2 env: validate shutdown policy from override file

[Fiona Ebner]

for future-proofing.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

Not sure if this is worth it.

(Build-)dependency bump for libpve-cluster-perl needed

No changes in v2.

 src/PVE/HA/Env/PVE2.pm | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index 6a75bd4..6eb4aa7 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -12,6 +12,7 @@ use PVE::Tools;
 use PVE::Cluster qw(cfs_register_file cfs_read_file cfs_write_file cfs_lock_file);
 use PVE::DataCenterConfig;
 use PVE::INotify;
+use PVE::JSONSchema;
 use PVE::RPCEnvironment;
 
 use PVE::HA::Tools ':exit_codes';
@@ -463,7 +464,17 @@ sub get_datacenter_settings {
     $self->log('warning', "error reading shutdown policy override from $sp_override_fn - $@") if $@;
 
     my $overrides = {};
-    $overrides->{ha}->{shutdown_policy} = $shutdown_policy_override if $shutdown_policy_override;
+
+    if ($shutdown_policy_override) {
+	eval {
+	    PVE::JSONSchema::validate(
+		$shutdown_policy_override,
+		PVE::JSONSchema::get_standard_option('pve-ha-shutdown-policy'),
+	    );
+	    $overrides->{ha}->{shutdown_policy} = $shutdown_policy_override;
+	};
+	$self->log('warning', "error validating shutdown policy override - $@") if $@;
+    }
 
     return {
 	ha => $datacenterconfig->{ha} // {},
-- 
2.39.2

[pve-devel] [PATCH v2 ha-manager 6/7] sim env: add support for datacenter config overrides

[Fiona Ebner]

to model the recent change in the pve2 environment.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

No changes in v2.

 src/PVE/HA/Sim/Env.pm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/PVE/HA/Sim/Env.pm b/src/PVE/HA/Sim/Env.pm
index c6ea73c..1b8399a 100644
--- a/src/PVE/HA/Sim/Env.pm
+++ b/src/PVE/HA/Sim/Env.pm
@@ -433,6 +433,7 @@ sub get_datacenter_settings {
     return {
 	ha => $datacenterconfig->{ha} // {},
 	crs => $datacenterconfig->{crs} // {},
+	'local-overrides' => $datacenterconfig->{'local-overrides'} // {},
     };
 }
 
-- 
2.39.2

[pve-devel] [PATCH v2 ha-manager 7/7] tests: add test for shutdown policy override

[Fiona Ebner]

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

No changes in v2.

 src/test/test-shutdown-policy-override/README |  3 +
 .../test-shutdown-policy-override/cmdlist     |  4 ++
 .../datacenter.cfg                            | 10 ++++
 .../hardware_status                           |  5 ++
 .../test-shutdown-policy-override/log.expect  | 57 +++++++++++++++++++
 .../manager_status                            |  1 +
 .../service_config                            |  6 ++
 7 files changed, 86 insertions(+)
 create mode 100644 src/test/test-shutdown-policy-override/README
 create mode 100644 src/test/test-shutdown-policy-override/cmdlist
 create mode 100644 src/test/test-shutdown-policy-override/datacenter.cfg
 create mode 100644 src/test/test-shutdown-policy-override/hardware_status
 create mode 100644 src/test/test-shutdown-policy-override/log.expect
 create mode 100644 src/test/test-shutdown-policy-override/manager_status
 create mode 100644 src/test/test-shutdown-policy-override/service_config

diff --git a/src/test/test-shutdown-policy-override/README b/src/test/test-shutdown-policy-override/README
new file mode 100644
index 0000000..b9c71ce
--- /dev/null
+++ b/src/test/test-shutdown-policy-override/README
@@ -0,0 +1,3 @@
+Test shutdown policy override.
+
+Expect that the policy from the override is used.
diff --git a/src/test/test-shutdown-policy-override/cmdlist b/src/test/test-shutdown-policy-override/cmdlist
new file mode 100644
index 0000000..a86b9e2
--- /dev/null
+++ b/src/test/test-shutdown-policy-override/cmdlist
@@ -0,0 +1,4 @@
+[
+    [ "power node1 on", "power node2 on", "power node3 on"],
+    [ "shutdown node3" ]
+]
diff --git a/src/test/test-shutdown-policy-override/datacenter.cfg b/src/test/test-shutdown-policy-override/datacenter.cfg
new file mode 100644
index 0000000..0a188bd
--- /dev/null
+++ b/src/test/test-shutdown-policy-override/datacenter.cfg
@@ -0,0 +1,10 @@
+{
+    "ha": {
+        "shutdown_policy": "migrate"
+    },
+    "local-overrides": {
+        "ha": {
+            "shutdown_policy": "freeze"
+        }
+    }
+}
diff --git a/src/test/test-shutdown-policy-override/hardware_status b/src/test/test-shutdown-policy-override/hardware_status
new file mode 100644
index 0000000..451beb1
--- /dev/null
+++ b/src/test/test-shutdown-policy-override/hardware_status
@@ -0,0 +1,5 @@
+{
+  "node1": { "power": "off", "network": "off" },
+  "node2": { "power": "off", "network": "off" },
+  "node3": { "power": "off", "network": "off" }
+}
diff --git a/src/test/test-shutdown-policy-override/log.expect b/src/test/test-shutdown-policy-override/log.expect
new file mode 100644
index 0000000..4aecc4f
--- /dev/null
+++ b/src/test/test-shutdown-policy-override/log.expect
@@ -0,0 +1,57 @@
+info      0     hardware: starting simulation
+info     20      cmdlist: execute power node1 on
+info     20    node1/crm: status change startup => wait_for_quorum
+info     20    node1/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node2 on
+info     20    node2/crm: status change startup => wait_for_quorum
+info     20    node2/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node3 on
+info     20    node3/crm: status change startup => wait_for_quorum
+info     20    node3/lrm: status change startup => wait_for_agent_lock
+info     20    node1/crm: got lock 'ha_manager_lock'
+info     20    node1/crm: status change wait_for_quorum => master
+info     20    node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node3': state changed from 'unknown' => 'online'
+info     20    node1/crm: adding new service 'vm:103' on node 'node3'
+info     20    node1/crm: adding new service 'vm:104' on node 'node3'
+info     20    node1/crm: adding new service 'vm:105' on node 'node3'
+info     20    node1/crm: adding new service 'vm:106' on node 'node3'
+info     20    node1/crm: service 'vm:103': state changed from 'request_start' to 'started'  (node = node3)
+info     20    node1/crm: service 'vm:104': state changed from 'request_start' to 'started'  (node = node3)
+info     20    node1/crm: service 'vm:105': state changed from 'request_start' to 'started'  (node = node3)
+info     20    node1/crm: service 'vm:106': state changed from 'request_start' to 'started'  (node = node3)
+info     22    node2/crm: status change wait_for_quorum => slave
+info     24    node3/crm: status change wait_for_quorum => slave
+info     25    node3/lrm: got lock 'ha_agent_node3_lock'
+info     25    node3/lrm: status change wait_for_agent_lock => active
+info     25    node3/lrm: starting service vm:103
+info     25    node3/lrm: service status vm:103 started
+info     25    node3/lrm: starting service vm:104
+info     25    node3/lrm: service status vm:104 started
+info     25    node3/lrm: starting service vm:105
+info     25    node3/lrm: service status vm:105 started
+info     25    node3/lrm: starting service vm:106
+info     25    node3/lrm: service status vm:106 started
+info    120      cmdlist: execute shutdown node3
+info    120    node3/lrm: got shutdown request with shutdown policy 'freeze'
+info    120    node3/lrm: shutdown LRM, stop and freeze all services
+info    120    node1/crm: service 'vm:103': state changed from 'started' to 'freeze'
+info    120    node1/crm: service 'vm:104': state changed from 'started' to 'freeze'
+info    120    node1/crm: service 'vm:105': state changed from 'started' to 'freeze'
+info    120    node1/crm: service 'vm:106': state changed from 'started' to 'freeze'
+info    125    node3/lrm: stopping service vm:103
+info    125    node3/lrm: service status vm:103 stopped
+info    125    node3/lrm: stopping service vm:104
+info    125    node3/lrm: service status vm:104 stopped
+info    125    node3/lrm: stopping service vm:105
+info    125    node3/lrm: service status vm:105 stopped
+info    125    node3/lrm: stopping service vm:106
+info    125    node3/lrm: service status vm:106 stopped
+info    126    node3/lrm: exit (loop end)
+info    126     shutdown: execute crm node3 stop
+info    125    node3/crm: server received shutdown request
+info    145    node3/crm: exit (loop end)
+info    145     shutdown: execute power node3 off
+info    160    node1/crm: node 'node3': state changed from 'online' => 'unknown'
+info    720     hardware: exit simulation - done
diff --git a/src/test/test-shutdown-policy-override/manager_status b/src/test/test-shutdown-policy-override/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-shutdown-policy-override/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-shutdown-policy-override/service_config b/src/test/test-shutdown-policy-override/service_config
new file mode 100644
index 0000000..d6f4fd9
--- /dev/null
+++ b/src/test/test-shutdown-policy-override/service_config
@@ -0,0 +1,6 @@
+{
+    "vm:103": { "node": "node3", "state": "enabled" },
+    "vm:104": { "node": "node3", "state": "enabled" },
+    "vm:105": { "node": "node3", "state": "enabled" },
+    "vm:106": { "node": "node3", "state": "enabled" }
+}
-- 
2.39.2

[pve-devel] [PATCH v2 manager 1/2] api: nodes: allow setting HA shutdown policy during shutdown/reboot

[Fiona Ebner]

Increases flexibility/user-friendliness.

In the edge case that an override is already present, but the user
doesn't have Sys.Modify privilege, just proceed with the existing
override. Could in principle happen when the requests from a
privileged user with a policy and an unprivileged user without a
policy arrive at the same time.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

(Build-)dependency bump for libpve-cluster-perl needed.

Changes in v2:
    * Also check for Sys.Modify privilege when parameter is specified.

 PVE/API2/Nodes.pm | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
index 9269694d..b8f0c6ce 100644
--- a/PVE/API2/Nodes.pm
+++ b/PVE/API2/Nodes.pm
@@ -8,7 +8,7 @@ use Digest::SHA;
 use Filesys::Df;
 use HTTP::Status qw(:constants);
 use JSON;
-use POSIX qw(LONG_MAX);
+use POSIX qw(ENOENT LONG_MAX);
 use Time::Local qw(timegm_nocheck);
 use Socket;
 use IO::Socket::SSL;
@@ -544,6 +544,7 @@ __PACKAGE__->register_method({
     method => 'POST',
     permissions => {
 	check => ['perm', '/nodes/{node}', [ 'Sys.PowerMgmt' ]],
+	description => "The 'shutdown-policy' parameter additionally requires 'Sys.Modify'.",
     },
     protected => 1,
     description => "Reboot or shutdown a node.",
@@ -557,12 +558,27 @@ __PACKAGE__->register_method({
 		type => 'string',
 		enum => [qw(reboot shutdown)],
 	    },
+	    'shutdown-policy' => get_standard_option('pve-ha-shutdown-policy', { optional => 1 }),
 	},
     },
     returns => { type => "null" },
     code => sub {
 	my ($param) = @_;
 
+	my $rpcenv = PVE::RPCEnvironment::get();
+	my $user = $rpcenv->get_user();
+	my $node = $param->{node};
+
+	my $sp_override_fn = '/run/pve-ha-lrm/shutdown-policy.local-override';
+
+	if ($param->{'shutdown-policy'}) {
+	    $rpcenv->check($user, "/nodes/$node", ['Sys.Modify']);
+	    eval { PVE::Tools::file_set_contents($sp_override_fn, $param->{'shutdown-policy'}); };
+	    die "could not write shutdown policy override to $sp_override_fn - $@" if $@;
+	} elsif (-e $sp_override_fn && $rpcenv->check($user, "/nodes/$node", ['Sys.Modify'], 1)) {
+	    unlink $sp_override_fn or die "unable to remove $sp_override_fn - $!";
+	}
+
 	if ($param->{command} eq 'reboot') {
 	    system ("(sleep 2;/sbin/reboot)&");
 	} elsif ($param->{command} eq 'shutdown') {
-- 
2.39.2

[pve-devel] [PATCH v2 manager 2/2] ui: allow specifying shutdown policy during node shutdown/reboot

[Fiona Ebner]

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

New in v2, was sent separately before (no changes to that).

Used a new window, because it's small and couldn't find a good
fit with the existing ones. Maybe SafeDestroy, but not in name and
would require a few modifications too.

Not sure about using gettext() for the option names either.

 www/manager6/Makefile               |   1 +
 www/manager6/node/Config.js         |  32 +++----
 www/manager6/window/NodeShutdown.js | 126 ++++++++++++++++++++++++++++
 3 files changed, 139 insertions(+), 20 deletions(-)
 create mode 100644 www/manager6/window/NodeShutdown.js

diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 9b6dd13b..94c8c05e 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -107,6 +107,7 @@ JSSRC= 							\
 	window/FirewallLograteEdit.js			\
 	window/LoginWindow.js				\
 	window/Migrate.js				\
+	window/NodeShutdown.js				\
 	window/Prune.js					\
 	window/Restore.js				\
 	window/SafeDestroyGuest.js			\
diff --git a/www/manager6/node/Config.js b/www/manager6/node/Config.js
index 6ed2172a..7a074879 100644
--- a/www/manager6/node/Config.js
+++ b/www/manager6/node/Config.js
@@ -19,18 +19,6 @@ Ext.define('PVE.node.Config', {
 	    interval: 5000,
 	});
 
-	var node_command = function(cmd) {
-	    Proxmox.Utils.API2Request({
-		params: { command: cmd },
-		url: '/nodes/' + nodename + '/status',
-		method: 'POST',
-		waitMsgTarget: me,
-		failure: function(response, opts) {
-		    Ext.Msg.alert(gettext('Error'), response.htmlStatus);
-		},
-	    });
-	};
-
 	var actionBtn = Ext.create('Ext.Button', {
 	    text: gettext('Bulk Actions'),
 	    iconCls: 'fa fa-fw fa-ellipsis-v',
@@ -83,24 +71,28 @@ Ext.define('PVE.node.Config', {
 	    }),
 	});
 
-	let restartBtn = Ext.create('Proxmox.button.Button', {
+	let restartBtn = Ext.create('Ext.button.Button', {
 	    text: gettext('Reboot'),
 	    disabled: !caps.nodes['Sys.PowerMgmt'],
-	    dangerous: true,
-	    confirmMsg: Ext.String.format(gettext("Reboot node '{0}'?"), nodename),
 	    handler: function() {
-		node_command('reboot');
+		Ext.create('PVE.window.NodeShutdown', {
+		    confirmMsg: Ext.String.format(gettext("Reboot node '{0}'?"), nodename),
+		    url: '/nodes/' + nodename + '/status',
+		    command: 'reboot',
+		}).show();
 	    },
 	    iconCls: 'fa fa-undo',
 	});
 
-	var shutdownBtn = Ext.create('Proxmox.button.Button', {
+	let shutdownBtn = Ext.create('Ext.button.Button', {
 	    text: gettext('Shutdown'),
 	    disabled: !caps.nodes['Sys.PowerMgmt'],
-	    dangerous: true,
-	    confirmMsg: Ext.String.format(gettext("Shutdown node '{0}'?"), nodename),
 	    handler: function() {
-		node_command('shutdown');
+		Ext.create('PVE.window.NodeShutdown', {
+		    confirmMsg: Ext.String.format(gettext("Shutdown node '{0}'?"), nodename),
+		    url: '/nodes/' + nodename + '/status',
+		    command: 'shutdown',
+		}).show();
 	    },
 	    iconCls: 'fa fa-power-off',
 	});
diff --git a/www/manager6/window/NodeShutdown.js b/www/manager6/window/NodeShutdown.js
new file mode 100644
index 00000000..4cdc541c
--- /dev/null
+++ b/www/manager6/window/NodeShutdown.js
@@ -0,0 +1,126 @@
+Ext.define('PVE.window.NodeShutdown', {
+    extend: 'Ext.window.Window',
+    alias: 'widget.pveNodeShutdown',
+    mixins: ['Proxmox.Mixin.CBind'],
+
+    title: gettext('Confirm'),
+    modal: true,
+    buttonAlign: 'center',
+    bodyPadding: 10,
+    width: 450,
+    layout: { type: 'hbox' },
+    defaultFocus: 'nodeShutdownNoButton',
+
+    config: {
+	url: undefined,
+	confirmMsg: undefined,
+	command: undefined,
+    },
+
+    getParams: function() {
+	let me = this;
+	let params = { command: me.getCommand(), };
+
+	let shutdownPolicy = me.lookup('shutdownPolicy').getValue();
+	if (shutdownPolicy && shutdownPolicy !== '__default__') {
+	    params['shutdown-policy'] = shutdownPolicy;
+	}
+
+	return params;
+    },
+
+    controller: {
+	xclass: 'Ext.app.ViewController',
+
+	control: {
+	    'button[reference=yesButton]': {
+		click: function() {
+		    const view = this.getView();
+		    Proxmox.Utils.API2Request({
+			params: view.getParams(),
+			url: view.getUrl(),
+			method: 'POST',
+			waitMsgTarget: view,
+			failure: function(response, opts) {
+			    Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+			},
+			success: function(response, options) {
+			    view.close();
+			},
+		    });
+		},
+	    },
+	    'button[reference=noButton]': {
+		click: function() {
+		    this.getView().close();
+		},
+	    },
+	},
+    },
+
+    buttons: [
+	{
+	    xtype: 'proxmoxHelpButton',
+	    listeners: {
+		beforerender: () => {
+		    Ext.GlobalEvents.fireEvent('proxmoxShowHelp', 'ha_manager_shutdown_policy');
+		},
+	    },
+	},
+	'->',
+	{
+	    reference: 'yesButton',
+	    text: gettext('Yes'),
+	},
+	{
+	    id: 'nodeShutdownNoButton',
+	    reference: 'noButton',
+	    text: gettext('No'),
+	},
+    ],
+
+    items: [
+	{
+	    xtype: 'component',
+	    cls: [
+		Ext.baseCSSPrefix + 'message-box-icon',
+		Ext.baseCSSPrefix + 'message-box-warning',
+		Ext.baseCSSPrefix + 'dlg-icon',
+	    ],
+	},
+	{
+	    xtype: 'container',
+	    flex: 1,
+	    layout: {
+		type: 'vbox',
+		align: 'stretch',
+	    },
+	    items: [
+		{
+		    xtype: 'displayfield',
+		    reference: 'messageCmp',
+		    cbind: {
+			value: '{confirmMsg}',
+		    },
+		},
+		{
+		    reference: 'shutdownPolicy',
+		    fieldLabel: gettext('HA Shutdown Policy'),
+		    labelWidth: 130,
+		    xtype: 'proxmoxKVComboBox',
+		    comboItems: [
+			[
+			    '__default__',
+			    Ext.String.format(gettext("Fallback from {0}"), "datacenter.cfg")
+			],
+			['migrate', gettext("Migrate")],
+			['conditional', gettext("Conditional")],
+			['freeze', gettext("Freeze")],
+			['failover', gettext("Failover")],
+		    ],
+		    value: '__default__',
+		},
+	    ],
+	},
+    ],
+});
-- 
2.39.2