all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>,
	Stoiko Ivanov <s.ivanov@proxmox.com>
Subject: [pve-devel] applied: [PATCH storage v2 2/3] zfspoolplugin: check if mounted instead of imported
Date: Fri, 19 Feb 2021 15:24:05 +0100	[thread overview]
Message-ID: <9d6fcfe8-fc16-d6e4-e4fb-61c2c5c0d466@proxmox.com> (raw)
In-Reply-To: <20210219124544.12593-3-s.ivanov@proxmox.com>

On 19.02.21 13:45, Stoiko Ivanov wrote:
> This patch addresses an issue we recently saw on a production machine:
> * after booting a ZFS pool failed to get imported (due to an empty
>   /etc/zfs/zpool.cache)
> * pvestatd/guest-startall eventually tried to import the pool
> * the pool was imported, yet the datasets of the pool remained
>   not mounted
> 
> A bit of debugging showed that `zpool import <poolname>` is not
> atomic, in fact it does fork+exec `mount` with appropriate parameters.
> If an import ran longer than the hardcoded timeout of 15s, it could
> happen that the pool got imported, but the zpool command (and its
> forks) got terminated due to timing out.
> 
> reproducing this is straight-forward by setting (drastic) bw+iops
> limits on a guests' disk (which contains a zpool) - e.g.:
> `qm set 100 -scsi1 wd:vm-100-disk-1,iops_rd=10,iops_rd_max=20,\
> iops_wr=15,iops_wr_max=20,mbps_rd=10,mbps_rd_max=15,mbps_wr=10,\
> mbps_wr_max=15`
> afterwards running `timeout 15 zpool import <poolname>` resulted in
> that situation in the guest on my machine
> 
> The patch changes the check in activate_storage for the ZFSPoolPlugin,
> to check if any dataset below the 'pool' (which can also be a sub-dataset)
> is mounted by parsing /proc/mounts:
> * this is cheaper than running `zfs get` or `zpool list`
> * it catches a properly imported and mounted pool in case the
>   root-dataset has 'canmount' set to off (or noauto), as long
>   as any dataset below is mounted
> 
> After trying to import the pool, we also run `zfs mount -a` (in case
> another check of /proc/mounts fails).
> 
> Potential for regression:
> * running `zfs mount -a` is problematic, if a dataset is manually
>   umounted after booting (without setting 'canmount')
> * a pool without any mounted dataset (no mountpoint property set and
>   only zvols) - will result in repeated calls to `zfs mount -a`
> 
> both of the above seem unlikely and should not occur, if using our
> tooling.
> 
> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> ---
>  PVE/Storage/ZFSPoolPlugin.pm | 27 ++++++++++++++++++++++++---
>  1 file changed, 24 insertions(+), 3 deletions(-)
> 
>

applied, thanks!




  reply	other threads:[~2021-02-19 14:24 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-19 12:45 [pve-devel] [PATCH storage v2 0/3] " Stoiko Ivanov
2021-02-19 12:45 ` [pve-devel] [PATCH storage v2 1/3] zfspoolplugin: activate_storage: minor cleanup Stoiko Ivanov
2021-02-19 13:18   ` [pve-devel] applied: " Thomas Lamprecht
2021-02-19 12:45 ` [pve-devel] [PATCH storage v2 2/3] zfspoolplugin: check if mounted instead of imported Stoiko Ivanov
2021-02-19 14:24   ` Thomas Lamprecht [this message]
2021-02-19 12:45 ` [pve-devel] [PATCH storage v2 3/3] zfspoolplugin: check if imported before importing Stoiko Ivanov
2021-02-19 15:28   ` [pve-devel] applied: " Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9d6fcfe8-fc16-d6e4-e4fb-61c2c5c0d466@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    --cc=s.ivanov@proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal