Re: [pve-devel] [RFC container] Improve feedback for startup

[Thomas Lamprecht <t.lamprecht@proxmox.com>]

On 19.08.20 12:30, Fabian Ebner wrote:
> Since it was necessary to switch to 'Type=Simple' in the systemd
> service (see 545d6f0a13ac2bf3a8d3f224c19c0e0def12116d ),
> 'systemctl start pve-container@ID' would not wait for the 'lxc-start'
> command anymore. Thus every container start was reported as a success
> and the 'post-start' hook would trigger immediately after the
> 'systemctl start' command.
> 
> Use 'lxc-monitor' to get the necessary information and detect
> startup failure and only run the 'post-start' hookscript after
> the container is effectively running. If something goes wrong
> with the monitor, fall back to the old behavior.
> 
> Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
> ---
>  src/PVE/LXC.pm | 36 +++++++++++++++++++++++++++++++++++-
>  1 file changed, 35 insertions(+), 1 deletion(-)
> 

appreciate the effort!
We could also directly connect to /run/lxc/var/lib/lxc/monitor-fifo (or the abstract
unix socket, but not much gained/difference here) of the lxc-monitord which publishes
all state changes and unpack the new state [0] directly.

[0] https://github.com/lxc/lxc/blob/8bdacc22a48f9c09902a1d2febd71439cb38c082/src/lxc/state.h#L10

@Wolfgang, what do you think?

> diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
> index db5b8ca..35dc54c 100644
> --- a/src/PVE/LXC.pm
> +++ b/src/PVE/LXC.pm
> @@ -2191,10 +2191,44 @@ sub vm_start {
>  
>      PVE::Storage::activate_volumes($storage_cfg, $vollist);
>  
> +    my $monitor_pid = open(my $monitor_fh, '-|', "/usr/bin/lxc-monitor -n $vmid")
> +	or warn "could not open pipe to lxc-monitor\n";
> +
>      my $cmd = ['systemctl', 'start', "pve-container\@$vmid"];
>  
>      PVE::GuestHelpers::exec_hookscript($conf, $vmid, 'pre-start', 1);
> -    eval { PVE::Tools::run_command($cmd); };
> +    eval {
> +	PVE::Tools::run_command($cmd);
> +
> +	my $success;
> +	if ($monitor_pid) {
> +	    eval {
> +		local $SIG{ALRM} = sub { die "got timeout\n" };
> +		alarm(10); # 'STARTING' should appear quickly
> +
> +		while (my $line = <$monitor_fh>) {
> +		    if ($line =~ m/^'$vmid' changed state to \[([A-Z]*)\]$/) {
> +			my $status = $1;
> +			alarm(0);
> +			$success = 1 if $status eq 'RUNNING';
> +			$success = 0 if $status eq 'ABORTING'
> +				     || $status eq 'STOPPING'
> +				     || $status eq 'STOPPED';
> +			if (defined($success)) {
> +			    kill('KILL', $monitor_pid);
> +			    waitpid($monitor_pid, 0);
> +			}
> +		    } else {
> +			die "unexpected output from lxc-monitor: $line\n";
> +		    }
> +		}
> +	    };
> +	    warn "Problem with lxc-monitor: $@" if $@;
> +	    alarm(0);
> +	}
> +	die "'lxc-start' failed for container '$vmid'\n"
> +	    if defined($success) && !$success;
> +    };
>      if (my $err = $@) {
>  	unlink $skiplock_flag_fn;
>  	die $err;
>