From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 54B8E61C81 for ; Mon, 7 Sep 2020 18:32:36 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 46950DBA5 for ; Mon, 7 Sep 2020 18:32:36 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 29F98DB98 for ; Mon, 7 Sep 2020 18:32:34 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id EC0D644A81 for ; Mon, 7 Sep 2020 18:32:33 +0200 (CEST) To: Proxmox VE development discussion , Wolfgang Bumiller References: <20200819103037.15143-1-f.ebner@proxmox.com> <1ef68f7f-437a-a160-05e2-f3b111ece024@proxmox.com> <20200827084404.o2aizoasd6r7xl4d@olga.proxmox.com> From: Thomas Lamprecht Message-ID: Date: Mon, 7 Sep 2020 18:32:32 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:81.0) Gecko/20100101 Thunderbird/81.0 MIME-Version: 1.0 In-Reply-To: <20200827084404.o2aizoasd6r7xl4d@olga.proxmox.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL 1.180 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -2.69 Looks like a legit reply (A) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] [RFC container] Improve feedback for startup X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Sep 2020 16:32:36 -0000 On 27.08.20 10:44, Wolfgang Bumiller wrote: > On Thu, Aug 20, 2020 at 11:36:39AM +0200, Thomas Lamprecht wrote: >> On 19.08.20 12:30, Fabian Ebner wrote: >>> Since it was necessary to switch to 'Type=3DSimple' in the systemd >>> service (see 545d6f0a13ac2bf3a8d3f224c19c0e0def12116d ), >>> 'systemctl start pve-container@ID' would not wait for the 'lxc-start'= >>> command anymore. Thus every container start was reported as a success= >>> and the 'post-start' hook would trigger immediately after the >>> 'systemctl start' command. >>> >>> Use 'lxc-monitor' to get the necessary information and detect >>> startup failure and only run the 'post-start' hookscript after >>> the container is effectively running. If something goes wrong >>> with the monitor, fall back to the old behavior. >>> >>> Signed-off-by: Fabian Ebner >>> --- >>> src/PVE/LXC.pm | 36 +++++++++++++++++++++++++++++++++++- >>> 1 file changed, 35 insertions(+), 1 deletion(-) >>> >> >> appreciate the effort! >> We could also directly connect to /run/lxc/var/lib/lxc/monitor-fifo (o= r the abstract >> unix socket, but not much gained/difference here) of the lxc-monitord = which publishes >> all state changes and unpack the new state [0] directly. >> >> [0] https://github.com/lxc/lxc/blob/8bdacc22a48f9c09902a1d2febd71439cb= 38c082/src/lxc/state.h#L10 >> >> @Wolfgang, what do you think? >=20 > Just tested adding a state client to our Command.pm directly, seems to > work, so we would depend neither on lxc-monitor nor lxc-monitord. >=20 > Example & code follow below. The only issue with it is that we'd need t= o > retry connecting to the command socket a few times since we don't know > when it becomes available, but that shouldn't be too bad IMO. >=20 > [..snip..] With below I never get the initial stopped -> running edge, though. I can monitor the CT getting stopped, but not the other way around. Adding extra code to check if the CTs running to abort the recv would feel like this missing the point a bit... >=20 > Usage example: >=20 > use PVE::LXC::Command; >=20 > my $sock =3D PVE::LXC::Command::get_state_client(404); > die "not running\n" if !defined($sock); >=20 > while (1) { > my ($type, $name, $value) =3D PVE::LXC::Command::read_lxc_messa= ge($sock); > last if !defined($type); > print("$name: $type =3D> $value\n"); > } >=20 > Patch for Command.pm: >=20 > ---8<--- > From 6ac578ef889a3a9c8aefc4f05215b4ec66049546 Mon Sep 17 00:00:00 2001 > From: Wolfgang Bumiller > Date: Thu, 27 Aug 2020 10:31:06 +0200 > Subject: [PATCH container] command: add state client functions >=20 > Signed-off-by: Wolfgang Bumiller > --- > src/PVE/LXC/Command.pm | 91 ++++++++++++++++++++++++++++++++++++++++++= > 1 file changed, 91 insertions(+) >=20 > diff --git a/src/PVE/LXC/Command.pm b/src/PVE/LXC/Command.pm > index beed890..6df767d 100644 > --- a/src/PVE/LXC/Command.pm > +++ b/src/PVE/LXC/Command.pm > @@ -11,20 +11,36 @@ use warnings; > =20 > use IO::Socket::UNIX; > use Socket qw(SOCK_STREAM SOL_SOCKET SO_PASSCRED); > +use POSIX qw(NAME_MAX); > =20 > use base 'Exporter'; > =20 > use constant { > + LXC_CMD_GET_STATE =3D> 3, > LXC_CMD_GET_CGROUP =3D> 6, > + LXC_CMD_ADD_STATE_CLIENT =3D> 10, > LXC_CMD_FREEZE =3D> 15, > LXC_CMD_UNFREEZE =3D> 16, > LXC_CMD_GET_LIMITING_CGROUP =3D> 19, > }; > =20 > +use constant { > + STATE_STOPPED =3D> 0, > + STATE_STARTING =3D> 1, > + STATE_RUNNING =3D> 2, > + STATE_STOPPING =3D> 3, > + STATE_ABORTING =3D> 4, > + STATE_FREEZING =3D> 5, > + STATE_FROZEN =3D> 6, > + STATE_THAWED =3D> 7, > + MAX_STATE =3D> 8, > +}; > + > our @EXPORT_OK =3D qw( > raw_command_transaction > simple_command > get_cgroup_path > + get_state_client > ); > =20 > # Get the command socket for a container. > @@ -81,6 +97,33 @@ my sub _unpack_lxc_cmd_rsp($) { > return ($ret, $len); > } > =20 > +my $LXC_MSG_SIZE =3D length(pack('I! Z'.(NAME_MAX+1).' x![I] I', 0, ""= , 0)); > +# Unpack an lxc_msg struct. > +my sub _unpack_lxc_msg($) { > + my ($packet) =3D @_; > + > + # struct lxc_msg { > + # lxc_msg_type_t type; > + # char name[NAME_MAX+1]; > + # int value; > + # }; > + > + my ($type, $name, $value) =3D unpack('I!Z'.(NAME_MAX+1).'I!', $pac= ket); > + > + if ($type =3D=3D 0) { > + $type =3D 'STATE'; > + } elsif ($type =3D=3D 1) { > + $type =3D 'PRIORITY'; > + } elsif ($type =3D=3D 2) { > + $type =3D 'EXITCODE'; > + } else { > + warn "unsupported lxc message type $type received\n"; > + $type =3D undef; > + } > + > + return ($type, $name, $value); > +} > + > # Send a complete packet: > my sub _do_send($$) { > my ($sock, $data) =3D @_; > @@ -206,4 +249,52 @@ sub unfreeze($$) { > return $res; > } > =20 > +# Add this command socket as a state client. > +# > +# Currently all states are observed. > +# > +# Returns undef if the container is not running, dies on errors. > +sub get_state_client($) { > + my ($vmid) =3D @_; > + > + my $socket =3D _get_command_socket($vmid) > + or return undef; > + > + # For now we want all states (except 'reboots', since we would nev= er see those, reboots would > + # use a value of '2' for STATE_RUNNING) > + my $states =3D pack('I!', 2) x MAX_STATE; > + > + my ($res, undef) =3D raw_command_transaction($socket, LXC_CMD_ADD_= STATE_CLIENT, $states); > + if ($res !=3D MAX_STATE) { > + die "container is currently in unexpected state $res\n"; > + } > + > + return $socket; > +} > + > +# Read an lxc message from a socket. > +# > +# Returns undef on EOF (if lxc exits). > +# Otherwise returns a (type, vmid, value) tuple. > +# > +# The returned 'type' currently can be 'STATE', 'PRIORITY' or 'EXITSTA= TUS'. > +sub read_lxc_message($) { > + my ($socket) =3D @_; > + > + my $msg; > + my $got =3D recv($socket, $msg, $LXC_MSG_SIZE, 0) > + // die "failed to read from state socket: $!\n"; > + > + if (length($msg) =3D=3D 0) { > + return undef; > + } > + > + die "short read on state socket ($LXC_MSG_SIZE !=3D ".length($msg)= =2E")\n" > + if length($msg) !=3D $LXC_MSG_SIZE; > + > + my ($type, $name, $value) =3D _unpack_lxc_msg($msg); > + > + return ($type, $name, $value); > +} > + > 1; >=20