Re: [pve-devel] [RFC container 2/4] fix #4474: lxc api: add overrule-shutdown parameter to stop endpoint

[Wolfgang Bumiller <w.bumiller@proxmox.com>]

On Thu, Jan 26, 2023 at 09:32:12AM +0100, Friedrich Weber wrote:
> The new `overrule-shutdown` parameter is boolean and defaults to 0. If
> it is 1, all active `vzshutdown` tasks by the current user for the same
> CT are aborted before attempting to stop the CT.
> 
> Passing `overrule-shutdown=1` is forbidden for HA resources.
> 
> Signed-off-by: Friedrich Weber <f.weber@proxmox.com>
> ---
>  src/PVE/API2/LXC/Status.pm | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
> 
> diff --git a/src/PVE/API2/LXC/Status.pm b/src/PVE/API2/LXC/Status.pm
> index f7e3128..d1d67f4 100644
> --- a/src/PVE/API2/LXC/Status.pm
> +++ b/src/PVE/API2/LXC/Status.pm
> @@ -221,6 +221,12 @@ __PACKAGE__->register_method({
>  	    node => get_standard_option('pve-node'),
>  	    vmid => get_standard_option('pve-vmid', { completion => \&PVE::LXC::complete_ctid_running }),
>  	    skiplock => get_standard_option('skiplock'),
> +	    'overrule-shutdown' => {
> +		description => "Abort any active 'vzshutdown' task by the current user for this CT before stopping",
> +		optional => 1,
> +		type => 'boolean',
> +		default => 0,
> +	    }
>  	},
>      },
>      returns => {
> @@ -238,10 +244,15 @@ __PACKAGE__->register_method({
>  	raise_param_exc({ skiplock => "Only root may use this option." })
>  	    if $skiplock && $authuser ne 'root@pam';
>  
> +	my $overrule_shutdown = extract_param($param, 'overrule-shutdown');
> +
>  	die "CT $vmid not running\n" if !PVE::LXC::check_running($vmid);
>  
>  	if (PVE::HA::Config::vm_is_ha_managed($vmid) && $rpcenv->{type} ne 'ha') {
>  
> +	    raise_param_exc({ 'overrule-shutdown' => "Not applicable for HA resources." })
> +		if $overrule_shutdown;
> +
>  	    my $hacmd = sub {
>  		my $upid = shift;
>  
> @@ -272,6 +283,11 @@ __PACKAGE__->register_method({
>  		return $rpcenv->fork_worker('vzstop', $vmid, $authuser, $realcmd);
>  	    };
>  
> +	    if ($overrule_shutdown) {
> +		my $overruled_tasks = PVE::GuestHelpers::overrule_tasks('vzshutdown', $authuser, $vmid);
> +		syslog('info', "overruled vzshutdown tasks: " . join(", ", $overruled_tasks->@*) . "\n");
> +	    };
> +

^ So this part is fine (mostly¹)

>  	    return PVE::LXC::Config->lock_config($vmid, $lockcmd);

^ Here we lock first, then fork the worker, then do `vm_stop` with the
config lock inherited.

This means that creating multiple shutdown tasks before using one with
override=true could cause the override task to cancel the *first* ongoing
shutdown task, then move on to the `lock_config` call - in the meantime
a second shutdown task acquires this very lock and performs another
long-running shutdown, causing the `override` parameter to be
ineffective.

We should switch the ordering here: first fork the worker, then lock.
(¹ And your new chunk would go into the worker as well)

Unless I'm missing something, but AFAICT the current ordering there is
rather ... bad :-)