all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Stoiko Ivanov <s.ivanov@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH zfsonlinux 11/16] sync scrub and trim cronjobs with debian upstream
Date: Thu, 17 Jun 2021 15:52:18 +0200	[thread overview]
Message-ID: <20210617135223.23472-12-s.ivanov@proxmox.com> (raw)
In-Reply-To: <20210617135223.23472-1-s.ivanov@proxmox.com>

* Pull in the changes to the automatic scrub cronjob from
  debian-upstream [0] commit
  f6d45405df0a2ed2748975667e8ea50714034d13
* Add a script and cronjob for regular trimming of pools
* Change the logic to scrub/trim pools based on a per pool('s root
  dataset property)

pulled the changes in as one commit instead of cherry-picking, since
the planned and reverted debconf questions would have caused
unneccessary churn.

commits identified by running:
`git log --full-diff -- debian/tree/zfsutils-linux/usr/lib/zfs-linux`
in a worktree from [0].

[0] https://salsa.debian.org/zfsonlinux-team/zfs/
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 .../zfsutils-linux/usr/lib/zfs-linux/scrub    | 43 +++++++++++--
 .../zfsutils-linux/usr/lib/zfs-linux/trim     | 60 +++++++++++++++++++
 debian/zfsutils-linux.cron.d                  |  5 +-
 3 files changed, 102 insertions(+), 6 deletions(-)
 create mode 100755 debian/tree/zfsutils-linux/usr/lib/zfs-linux/trim

diff --git a/debian/tree/zfsutils-linux/usr/lib/zfs-linux/scrub b/debian/tree/zfsutils-linux/usr/lib/zfs-linux/scrub
index 38f071af..f18c6e83 100755
--- a/debian/tree/zfsutils-linux/usr/lib/zfs-linux/scrub
+++ b/debian/tree/zfsutils-linux/usr/lib/zfs-linux/scrub
@@ -1,12 +1,45 @@
 #!/bin/sh -eu
 
-# Scrub all healthy pools that are not already scrubbing.
+# directly exit successfully when zfs module is not loaded
+if ! [ -d /sys/module/zfs ]; then
+	exit 0
+fi
+
+# [auto] / enable / disable
+PROPERTY_NAME="org.debian:periodic-scrub"
+
+get_property () {
+	# Detect the ${PROPERTY_NAME} property on a given pool.
+	# We are abusing user-defined properties on the root dataset,
+	# since they're not available on pools https://github.com/openzfs/zfs/pull/11680
+	# TODO: use zpool user-defined property when such feature is available.
+	pool="$1"
+	zfs get -H -o value "${PROPERTY_NAME}" "${pool}" 2>/dev/null || return 1
+}
+
+scrub_if_not_scrub_in_progress () {
+	pool="$1"
+	if ! zpool status "${pool}" | grep -q "scrub in progress"; then
+		# Ignore errors and continue with scrubbing other pools.
+		zpool scrub "${pool}" || true
+	fi
+}
+
+# Scrub all healthy pools that are not already scrubbing as per their configs.
 zpool list -H -o health,name 2>&1 | \
-	awk 'BEGIN {FS="\t"} {if ($1 ~ /^ONLINE/) print $2}' | \
+	awk -F'\t' '$1 == "ONLINE" {print $2}' | \
 while read pool
 do
-	if ! zpool status "$pool" | grep -q "scrub in progress"
-	then
-		zpool scrub "$pool"
+	# read user-defined config
+	ret=$(get_property "${pool}")
+	if [ $? -ne 0 ] || [ "disable" = "${ret}" ]; then
+		:
+	elif [ "-" = "${ret}" ] || [ "auto" = "${ret}" ] || [ "enable" = "${ret}" ]; then
+		scrub_if_not_scrub_in_progress "${pool}"
+	else
+		cat > /dev/stderr <<EOF
+$0: [WARNING] illegal value "${ret}" for property "${PROPERTY_NAME}" of ZFS dataset "${pool}".
+$0: Acceptable choices for this property are: auto, enable, disable. The default is auto.
+EOF
 	fi
 done
diff --git a/debian/tree/zfsutils-linux/usr/lib/zfs-linux/trim b/debian/tree/zfsutils-linux/usr/lib/zfs-linux/trim
new file mode 100755
index 00000000..5b6305b0
--- /dev/null
+++ b/debian/tree/zfsutils-linux/usr/lib/zfs-linux/trim
@@ -0,0 +1,60 @@
+#!/bin/sh -eu
+
+# directly exit successfully when zfs module is not loaded
+if ! [ -d /sys/module/zfs ]; then
+	exit 0
+fi
+
+# [auto] / enable / disable
+PROPERTY_NAME="org.debian:periodic-trim"
+
+get_property () {
+	# Detect the ${PROPERTY_NAME} property on a given pool.
+	# We are abusing user-defined properties on the root dataset,
+	# since they're not available on pools https://github.com/openzfs/zfs/pull/11680
+	# TODO: use zpool user-defined property when such feature is available.
+	pool="$1"
+	zfs get -H -o value "${PROPERTY_NAME}" "${pool}" 2>/dev/null || return 1
+}
+
+trim_if_not_already_trimming () {
+	pool="$1"
+	if ! zpool status "${pool}" | grep -q "trimming"; then
+		# Ignore errors (i.e. HDD pools),
+		# and continue with trimming other pools.
+		zpool trim "${pool}" || true
+	fi
+}
+
+zpool_is_nvme_only () {
+	zpool=$1
+	# get a list of devices attached to the specified zpool
+	zpool list -vHPL "${zpool}" |
+		awk -F'\t' '$2 ~ /^\/dev\// {
+			if($2 !~ /^\/dev\/nvme/)
+				exit 1
+		}'
+}
+
+# TRIM all healthy pools that are not already trimming as per their configs.
+zpool list -H -o health,name 2>&1 | \
+	awk -F'\t' '$1 == "ONLINE" {print $2}' | \
+while read pool
+do
+	# read user-defined config
+	ret=$(get_property "${pool}")
+	if [ $? -ne 0 ] || [ "disable" = "${ret}" ]; then
+		:
+	elif [ "enable" = "${ret}" ]; then
+		trim_if_not_already_trimming "${pool}"
+	elif [ "-" = "${ret}" ] || [ "auto" = "${ret}" ]; then
+		if zpool_is_nvme_only "${pool}"; then
+			trim_if_not_already_trimming "${pool}"
+		fi
+	else
+		cat > /dev/stderr <<EOF
+$0: [WARNING] illegal value "${ret}" for property "${PROPERTY_NAME}" of ZFS dataset "${pool}".
+$0: Acceptable choices for this property are: auto, enable, disable. The default is auto.
+EOF
+	fi
+done
diff --git a/debian/zfsutils-linux.cron.d b/debian/zfsutils-linux.cron.d
index 2f3a8cad..4c0d373a 100644
--- a/debian/zfsutils-linux.cron.d
+++ b/debian/zfsutils-linux.cron.d
@@ -1,4 +1,7 @@
 PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
 
+# TRIM the first Sunday of every month.
+24 0 1-7 * * root if [ $(date +\%w) -eq 0 ] && [ -x /usr/lib/zfs-linux/trim ]; then /usr/lib/zfs-linux/trim; fi
+
 # Scrub the second Sunday of every month.
-24 0 8-14 * * root [ $(date +\%w) -eq 0 ] && [ -x /usr/lib/zfs-linux/scrub ] && /usr/lib/zfs-linux/scrub
+24 0 8-14 * * root if [ $(date +\%w) -eq 0 ] && [ -x /usr/lib/zfs-linux/scrub ]; then /usr/lib/zfs-linux/scrub; fi
-- 
2.20.1





  parent reply	other threads:[~2021-06-17 13:53 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-17 13:52 [pve-devel] [PATCH zfsonlinux 00/16] sync buildsys with debian-upstream Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 01/16] buildsys: unify '{' and '(' in Makefile Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 02/16] buildsys: add all packages to make variable and sort Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 03/16] Reflect libssl requirement of libzfs Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 04/16] Elaborate the functionality of ZED in long description. (Closes: #979414) Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 05/16] buildsys: switch to automatic debug packages Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 06/16] Bump debhelper compat level to 12 Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 07/16] compat 12 automatically adds initramfs trigger Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 08/16] Bump Standards-Version to 4.5.1 (no changes) Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 09/16] Adjust zed.d symlink-preservation Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 10/16] Remove /etc/zfs/zed.d on purge Stoiko Ivanov
2021-06-17 13:52 ` Stoiko Ivanov [this message]
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 12/16] update lintian-overrides Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 13/16] drop unused lintian overrides Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 14/16] add symbol files for library packages Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 15/16] update libzpool symbols Stoiko Ivanov
2021-06-17 13:52 ` [pve-devel] [PATCH zfsonlinux 16/16] do not restart zfs-volume-wait.service Stoiko Ivanov
2021-06-18 13:44 ` [pve-devel] applied: [PATCH zfsonlinux 00/16] sync buildsys with debian-upstream Fabian Grünbichler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210617135223.23472-12-s.ivanov@proxmox.com \
    --to=s.ivanov@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal