From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <t.lamprecht@proxmox.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id ACEE76BB99 for <pve-devel@lists.proxmox.com>; Fri, 19 Feb 2021 15:24:36 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 940391990E for <pve-devel@lists.proxmox.com>; Fri, 19 Feb 2021 15:24:06 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 10D9D19901 for <pve-devel@lists.proxmox.com>; Fri, 19 Feb 2021 15:24:06 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id C9A064497C for <pve-devel@lists.proxmox.com>; Fri, 19 Feb 2021 15:24:05 +0100 (CET) Message-ID: <9d6fcfe8-fc16-d6e4-e4fb-61c2c5c0d466@proxmox.com> Date: Fri, 19 Feb 2021 15:24:05 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:86.0) Gecko/20100101 Thunderbird/86.0 Content-Language: en-US To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>, Stoiko Ivanov <s.ivanov@proxmox.com> References: <20210219124544.12593-1-s.ivanov@proxmox.com> <20210219124544.12593-3-s.ivanov@proxmox.com> From: Thomas Lamprecht <t.lamprecht@proxmox.com> In-Reply-To: <20210219124544.12593-3-s.ivanov@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.059 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [zfspoolplugin.pm] Subject: [pve-devel] applied: [PATCH storage v2 2/3] zfspoolplugin: check if mounted instead of imported X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/> List-Post: <mailto:pve-devel@lists.proxmox.com> List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe> X-List-Received-Date: Fri, 19 Feb 2021 14:24:36 -0000 On 19.02.21 13:45, Stoiko Ivanov wrote: > This patch addresses an issue we recently saw on a production machine: > * after booting a ZFS pool failed to get imported (due to an empty > /etc/zfs/zpool.cache) > * pvestatd/guest-startall eventually tried to import the pool > * the pool was imported, yet the datasets of the pool remained > not mounted > > A bit of debugging showed that `zpool import <poolname>` is not > atomic, in fact it does fork+exec `mount` with appropriate parameters. > If an import ran longer than the hardcoded timeout of 15s, it could > happen that the pool got imported, but the zpool command (and its > forks) got terminated due to timing out. > > reproducing this is straight-forward by setting (drastic) bw+iops > limits on a guests' disk (which contains a zpool) - e.g.: > `qm set 100 -scsi1 wd:vm-100-disk-1,iops_rd=10,iops_rd_max=20,\ > iops_wr=15,iops_wr_max=20,mbps_rd=10,mbps_rd_max=15,mbps_wr=10,\ > mbps_wr_max=15` > afterwards running `timeout 15 zpool import <poolname>` resulted in > that situation in the guest on my machine > > The patch changes the check in activate_storage for the ZFSPoolPlugin, > to check if any dataset below the 'pool' (which can also be a sub-dataset) > is mounted by parsing /proc/mounts: > * this is cheaper than running `zfs get` or `zpool list` > * it catches a properly imported and mounted pool in case the > root-dataset has 'canmount' set to off (or noauto), as long > as any dataset below is mounted > > After trying to import the pool, we also run `zfs mount -a` (in case > another check of /proc/mounts fails). > > Potential for regression: > * running `zfs mount -a` is problematic, if a dataset is manually > umounted after booting (without setting 'canmount') > * a pool without any mounted dataset (no mountpoint property set and > only zvols) - will result in repeated calls to `zfs mount -a` > > both of the above seem unlikely and should not occur, if using our > tooling. > > Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com> > --- > PVE/Storage/ZFSPoolPlugin.pm | 27 ++++++++++++++++++++++++--- > 1 file changed, 24 insertions(+), 3 deletions(-) > > applied, thanks!