From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>,
Stoiko Ivanov <s.ivanov@proxmox.com>
Subject: [pve-devel] applied: [PATCH storage v2 2/3] zfspoolplugin: check if mounted instead of imported
Date: Fri, 19 Feb 2021 15:24:05 +0100 [thread overview]
Message-ID: <9d6fcfe8-fc16-d6e4-e4fb-61c2c5c0d466@proxmox.com> (raw)
In-Reply-To: <20210219124544.12593-3-s.ivanov@proxmox.com>
On 19.02.21 13:45, Stoiko Ivanov wrote:
> This patch addresses an issue we recently saw on a production machine:
> * after booting a ZFS pool failed to get imported (due to an empty
> /etc/zfs/zpool.cache)
> * pvestatd/guest-startall eventually tried to import the pool
> * the pool was imported, yet the datasets of the pool remained
> not mounted
>
> A bit of debugging showed that `zpool import <poolname>` is not
> atomic, in fact it does fork+exec `mount` with appropriate parameters.
> If an import ran longer than the hardcoded timeout of 15s, it could
> happen that the pool got imported, but the zpool command (and its
> forks) got terminated due to timing out.
>
> reproducing this is straight-forward by setting (drastic) bw+iops
> limits on a guests' disk (which contains a zpool) - e.g.:
> `qm set 100 -scsi1 wd:vm-100-disk-1,iops_rd=10,iops_rd_max=20,\
> iops_wr=15,iops_wr_max=20,mbps_rd=10,mbps_rd_max=15,mbps_wr=10,\
> mbps_wr_max=15`
> afterwards running `timeout 15 zpool import <poolname>` resulted in
> that situation in the guest on my machine
>
> The patch changes the check in activate_storage for the ZFSPoolPlugin,
> to check if any dataset below the 'pool' (which can also be a sub-dataset)
> is mounted by parsing /proc/mounts:
> * this is cheaper than running `zfs get` or `zpool list`
> * it catches a properly imported and mounted pool in case the
> root-dataset has 'canmount' set to off (or noauto), as long
> as any dataset below is mounted
>
> After trying to import the pool, we also run `zfs mount -a` (in case
> another check of /proc/mounts fails).
>
> Potential for regression:
> * running `zfs mount -a` is problematic, if a dataset is manually
> umounted after booting (without setting 'canmount')
> * a pool without any mounted dataset (no mountpoint property set and
> only zvols) - will result in repeated calls to `zfs mount -a`
>
> both of the above seem unlikely and should not occur, if using our
> tooling.
>
> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> ---
> PVE/Storage/ZFSPoolPlugin.pm | 27 ++++++++++++++++++++++++---
> 1 file changed, 24 insertions(+), 3 deletions(-)
>
>
applied, thanks!
next prev parent reply other threads:[~2021-02-19 14:24 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-19 12:45 [pve-devel] [PATCH storage v2 0/3] " Stoiko Ivanov
2021-02-19 12:45 ` [pve-devel] [PATCH storage v2 1/3] zfspoolplugin: activate_storage: minor cleanup Stoiko Ivanov
2021-02-19 13:18 ` [pve-devel] applied: " Thomas Lamprecht
2021-02-19 12:45 ` [pve-devel] [PATCH storage v2 2/3] zfspoolplugin: check if mounted instead of imported Stoiko Ivanov
2021-02-19 14:24 ` Thomas Lamprecht [this message]
2021-02-19 12:45 ` [pve-devel] [PATCH storage v2 3/3] zfspoolplugin: check if imported before importing Stoiko Ivanov
2021-02-19 15:28 ` [pve-devel] applied: " Thomas Lamprecht
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9d6fcfe8-fc16-d6e4-e4fb-61c2c5c0d466@proxmox.com \
--to=t.lamprecht@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
--cc=s.ivanov@proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox