[pve-devel] [PATCH zfsonlinux 0/6] update to ZFS 2.1.6

[pve-devel] [PATCH zfsonlinux 0/6] update to ZFS 2.1.6

[Stoiko Ivanov]

Upstream tagged 2.1.6 yesterday afternoon.
The release contains quite a few improvments w.r.t. code-safety and many
optimizations which imho might have a potential of improving our workloads

additionally I took a peak at debian-upstream and pulled in some relevant
changes to keep us in sync.

tested the lintian changes with the current versions in bullseye and
bullseye-backports.

minimally tested by building a 5.19 kernel and running ztest for 20
minutes


Stoiko Ivanov (6):
  update zfs submodule to 2.1.6 and refresh patches
  trim: determine if a device uses nvme transport with lsblk (Closes:
    #990745)
  Symlink zfs,zpool to /bin/ for non-root usage. (Closes: #979437)
  Symlink zpool_influxdb to /bin. (Closes: #1001446)
  debian/lintian-overrides: future-proof
  debian/zfs-initramfs.lintian-overrides add overrides file

 debian/libzfsbootenv1linux.lintian-overrides  |  2 +-
 ...at-Fix-integer-division-with-python3.patch |  2 +-
 ...-guard-access-to-l2arc-MFU-MRU-stats.patch |  2 +-
 .../zfsutils-linux/usr/lib/zfs-linux/trim     | 29 +++++++++++++++----
 debian/zfs-initramfs.lintian-overrides        |  1 +
 debian/zfs-test.lintian-overrides             | 18 ++++++------
 debian/zfsutils-linux.links                   |  3 ++
 debian/zfsutils-linux.lintian-overrides       | 15 +++++-----
 upstream                                      |  2 +-
 9 files changed, 48 insertions(+), 26 deletions(-)
 create mode 100644 debian/zfs-initramfs.lintian-overrides
 create mode 100644 debian/zfsutils-linux.links

-- 
2.30.2

[pve-devel] [PATCH zfsonlinux 1/6] update zfs submodule to 2.1.6 and refresh patches

[Stoiko Ivanov]

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 .../0010-arcstat-Fix-integer-division-with-python3.patch        | 2 +-
 ...1-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch | 2 +-
 upstream                                                        | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/debian/patches/0010-arcstat-Fix-integer-division-with-python3.patch b/debian/patches/0010-arcstat-Fix-integer-division-with-python3.patch
index 086347f8..33633e5d 100644
--- a/debian/patches/0010-arcstat-Fix-integer-division-with-python3.patch
+++ b/debian/patches/0010-arcstat-Fix-integer-division-with-python3.patch
@@ -22,7 +22,7 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
  1 file changed, 33 insertions(+), 33 deletions(-)
 
 diff --git a/cmd/arcstat/arcstat.in b/cmd/arcstat/arcstat.in
-index 9e7c52a6c..cd9a803a2 100755
+index 425e52d1f..010f69ec0 100755
 --- a/cmd/arcstat/arcstat.in
 +++ b/cmd/arcstat/arcstat.in
 @@ -441,73 +441,73 @@ def calculate():
diff --git a/debian/patches/0011-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch b/debian/patches/0011-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch
index 6a704848..0747a1e0 100644
--- a/debian/patches/0011-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch
+++ b/debian/patches/0011-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch
@@ -79,7 +79,7 @@ index 7b28012ed..fe6a6d9e2 100755
      print()
      prt_1('L2ARC breakdown:', f_hits(l2_access_total))
 diff --git a/cmd/arcstat/arcstat.in b/cmd/arcstat/arcstat.in
-index cd9a803a2..ea45dc602 100755
+index 010f69ec0..50e5a7150 100755
 --- a/cmd/arcstat/arcstat.in
 +++ b/cmd/arcstat/arcstat.in
 @@ -482,8 +482,8 @@ def calculate():
diff --git a/upstream b/upstream
index 6c3c5fcf..6a6bd493 160000
--- a/upstream
+++ b/upstream
@@ -1 +1 @@
-Subproject commit 6c3c5fcfbe27d9193cd131753cc7e47ee2784621
+Subproject commit 6a6bd493988c75331deab06e5352a9bed035a87d
-- 
2.30.2

[pve-devel] [PATCH zfsonlinux 2/6] trim: determine if a device uses nvme transport with lsblk (Closes: #990745)

[Stoiko Ivanov]

Thanks to Miao Wang for get_transp() function

(cherry picked from debian-upstream[0]
commit 1f5433b15e4f7c823983bb90390c4b023f3d1948)

[0] https://salsa.debian.org/zfsonlinux-team/zfs
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 .../zfsutils-linux/usr/lib/zfs-linux/trim     | 29 +++++++++++++++----
 1 file changed, 24 insertions(+), 5 deletions(-)

diff --git a/debian/tree/zfsutils-linux/usr/lib/zfs-linux/trim b/debian/tree/zfsutils-linux/usr/lib/zfs-linux/trim
index 5b6305b0..91d00bb0 100755
--- a/debian/tree/zfsutils-linux/usr/lib/zfs-linux/trim
+++ b/debian/tree/zfsutils-linux/usr/lib/zfs-linux/trim
@@ -26,14 +26,33 @@ trim_if_not_already_trimming () {
 	fi
 }
 
+get_transp () {
+        local dev="$1"
+        local par_dev="$dev"
+        local pd
+        while true; do
+                pd=$(lsblk -dnr -o PKNAME "$par_dev")
+                if [ "$?" -ne 0 ]; then
+                        return $?
+                fi
+                if [ -z "$pd" ]; then
+                        break
+                else
+                        par_dev="/dev/$pd"
+                fi
+        done
+        lsblk -dnr -o TRAN "$par_dev"
+}
+
 zpool_is_nvme_only () {
 	zpool=$1
 	# get a list of devices attached to the specified zpool
-	zpool list -vHPL "${zpool}" |
-		awk -F'\t' '$2 ~ /^\/dev\// {
-			if($2 !~ /^\/dev\/nvme/)
-				exit 1
-		}'
+        for x in $(zpool list -vHPL "${zpool}" |\
+            awk -F'\t' '{if($2 ~ /^\/dev\//) print $2}'); do
+                if [ "$(get_transp $x)" != "nvme" ]; then
+                        return 1
+                fi
+        done
 }
 
 # TRIM all healthy pools that are not already trimming as per their configs.
-- 
2.30.2

[pve-devel] [PATCH zfsonlinux 3/6] Symlink zfs, zpool to /bin/ for non-root usage. (Closes: #979437)

[Stoiko Ivanov]

(cherry picked from debian-upstream[0]
commit 9fe812af0c9fb0382b05b300244f120179e201e7)

[0] https://salsa.debian.org/zfsonlinux-team/zfs
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 debian/zfsutils-linux.links | 2 ++
 1 file changed, 2 insertions(+)
 create mode 100644 debian/zfsutils-linux.links

diff --git a/debian/zfsutils-linux.links b/debian/zfsutils-linux.links
new file mode 100644
index 00000000..79bb9717
--- /dev/null
+++ b/debian/zfsutils-linux.links
@@ -0,0 +1,2 @@
+sbin/zfs      bin/zfs
+sbin/zpool    bin/zpool
-- 
2.30.2

[pve-devel] [PATCH zfsonlinux 4/6] Symlink zpool_influxdb to /bin. (Closes: #1001446)

[Stoiko Ivanov]

(cherry picked from debian-upstream[0]
commit 4ce9e9b7a221ba1a84a921a34db582a2ef68b950)

[0] https://salsa.debian.org/zfsonlinux-team/zfs
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 debian/zfsutils-linux.links | 1 +
 1 file changed, 1 insertion(+)

diff --git a/debian/zfsutils-linux.links b/debian/zfsutils-linux.links
index 79bb9717..adb63c18 100644
--- a/debian/zfsutils-linux.links
+++ b/debian/zfsutils-linux.links
@@ -1,2 +1,3 @@
 sbin/zfs      bin/zfs
 sbin/zpool    bin/zpool
+usr/lib/zfs-linux/zpool_influxdb  bin/zpool_influxdb
-- 
2.30.2

[pve-devel] [PATCH zfsonlinux 5/6] debian/lintian-overrides: future-proof

[Stoiko Ivanov]

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 debian/libzfsbootenv1linux.lintian-overrides |  2 +-
 debian/zfs-test.lintian-overrides            | 18 +++++++++---------
 debian/zfsutils-linux.lintian-overrides      | 15 +++++++--------
 3 files changed, 17 insertions(+), 18 deletions(-)

diff --git a/debian/libzfsbootenv1linux.lintian-overrides b/debian/libzfsbootenv1linux.lintian-overrides
index b48c31d9..c5d06d89 100644
--- a/debian/libzfsbootenv1linux.lintian-overrides
+++ b/debian/libzfsbootenv1linux.lintian-overrides
@@ -1,2 +1,2 @@
 package-name-doesnt-match-sonames
-extra-license-file usr/share/doc/libzfsbootenv1linux/LICENSE.gz
+extra-license-file *usr/share/doc/libzfsbootenv1linux/LICENSE.gz*
diff --git a/debian/zfs-test.lintian-overrides b/debian/zfs-test.lintian-overrides
index f7a0bcc1..fd4b2ab0 100644
--- a/debian/zfs-test.lintian-overrides
+++ b/debian/zfs-test.lintian-overrides
@@ -1,13 +1,13 @@
 arch-dependent-file-in-usr-share
-script-not-executable usr/share/zfs/common.sh
 command-in-sbin-has-manpage-in-incorrect-section
 arch-dep-package-has-big-usr-share
 manpage-without-executable
-national-encoding usr/share/zfs/zfs-tests/tests/functional/channel_program/lua_core/tst.lib_table.lua
-executable-not-elf-or-script usr/share/zfs/zfs-tests/tests/functional/cli_root/zfs_jail/jail.conf
-script-not-executable usr/share/zfs/zfs-tests/include/default.cfg
-script-not-executable usr/share/zfs/zfs-tests/tests/functional/cli_root/zfs_wait/zfs_wait.kshlib
-script-not-executable usr/share/zfs/zfs-tests/tests/functional/cli_root/zpool_wait/zpool_wait.kshlib
-script-not-executable usr/share/zfs/zfs-tests/tests/functional/persist_l2arc/persist_l2arc.cfg
-script-not-executable usr/share/zfs/zfs-tests/tests/functional/redacted_send/redacted.kshlib
-package-contains-documentation-outside-usr-share-doc usr/share/zfs/zfs-tests/*
+national-encoding *usr/share/zfs/zfs-tests/tests/functional/channel_program/lua_core/tst.lib_table.lua*
+executable-not-elf-or-script *usr/share/zfs/zfs-tests/tests/functional/cli_root/zfs_jail/jail.conf*
+package-contains-documentation-outside-usr-share-doc *usr/share/zfs/zfs-tests/*
+script-not-executable *usr/share/zfs/common.sh*
+script-not-executable *usr/share/zfs/zfs-tests/include/default.cfg*
+script-not-executable *usr/share/zfs/zfs-tests/tests/functional/cli_root/zfs_wait/zfs_wait.kshlib*
+script-not-executable *usr/share/zfs/zfs-tests/tests/functional/cli_root/zpool_wait/zpool_wait.kshlib*
+script-not-executable *usr/share/zfs/zfs-tests/tests/functional/l2arc/l2arc.cfg*
+script-not-executable *usr/share/zfs/zfs-tests/tests/functional/redacted_send/redacted.kshlib*
diff --git a/debian/zfsutils-linux.lintian-overrides b/debian/zfsutils-linux.lintian-overrides
index 67254c84..2f21993c 100644
--- a/debian/zfsutils-linux.lintian-overrides
+++ b/debian/zfsutils-linux.lintian-overrides
@@ -1,14 +1,13 @@
 spare-manual-page
 systemd-service-file-refers-to-unusual-wantedby-target
-binary-without-manpage usr/sbin/dbufstat
-binary-without-manpage usr/sbin/arc_summary
+binary-without-manpage *usr/sbin/dbufstat*
+binary-without-manpage *usr/sbin/arc_summary*
 manpage-has-errors-from-man
 appstream-metadata-missing-modalias-provide
 command-in-sbin-has-manpage-in-incorrect-section
-package-supports-alternative-init-but-no-init.d-script lib/systemd/system/zfs-import-cache.service
-package-supports-alternative-init-but-no-init.d-script lib/systemd/system/zfs-import-scan.service
+package-supports-alternative-init-but-no-init.d-script *lib/systemd/system/zfs-import-cache.service*
+package-supports-alternative-init-but-no-init.d-script *lib/systemd/system/zfs-import-scan.service*
 spelling-error-in-manpage
-package-supports-alternative-init-but-no-init.d-script lib/systemd/system/zfs-volume-wait.service
-systemd-service-file-missing-documentation-key lib/systemd/system/zfs-volume-wait.service
-extra-license-file usr/share/doc/zfsutils-linux/LICENSE.gz
-package-supports-alternative-init-but-no-init.d-script lib/systemd/system/zfs-load-module.service
+package-supports-alternative-init-but-no-init.d-script *lib/systemd/system/zfs-volume-wait.service*
+systemd-service-file-missing-documentation-key *lib/systemd/system/zfs-volume-wait.service*
+extra-license-file *usr/share/doc/zfsutils-linux/LICENSE.gz*
-- 
2.30.2

[pve-devel] [PATCH zfsonlinux 6/6] debian/zfs-initramfs.lintian-overrides add overrides file

[Stoiko Ivanov]

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 debian/zfs-initramfs.lintian-overrides | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 debian/zfs-initramfs.lintian-overrides

diff --git a/debian/zfs-initramfs.lintian-overrides b/debian/zfs-initramfs.lintian-overrides
new file mode 100644
index 00000000..80c57b15
--- /dev/null
+++ b/debian/zfs-initramfs.lintian-overrides
@@ -0,0 +1 @@
+executable-not-elf-or-script *usr/share/initramfs-tools/scripts/zfs*
-- 
2.30.2

[pve-devel] applied-series: [PATCH zfsonlinux 0/6] update to ZFS 2.1.6

[Thomas Lamprecht]

Am 04/10/2022 um 15:47 schrieb Stoiko Ivanov:
> Upstream tagged 2.1.6 yesterday afternoon.
> The release contains quite a few improvments w.r.t. code-safety and many
> optimizations which imho might have a potential of improving our workloads
> 
> additionally I took a peak at debian-upstream and pulled in some relevant
> changes to keep us in sync.
> 
> tested the lintian changes with the current versions in bullseye and
> bullseye-backports.
> 
> minimally tested by building a 5.19 kernel and running ztest for 20
> minutes
> 
> 
> Stoiko Ivanov (6):
>   update zfs submodule to 2.1.6 and refresh patches
>   trim: determine if a device uses nvme transport with lsblk (Closes:
>     #990745)
>   Symlink zfs,zpool to /bin/ for non-root usage. (Closes: #979437)
>   Symlink zpool_influxdb to /bin. (Closes: #1001446)
>   debian/lintian-overrides: future-proof
>   debian/zfs-initramfs.lintian-overrides add overrides file
> 
>  debian/libzfsbootenv1linux.lintian-overrides  |  2 +-
>  ...at-Fix-integer-division-with-python3.patch |  2 +-
>  ...-guard-access-to-l2arc-MFU-MRU-stats.patch |  2 +-
>  .../zfsutils-linux/usr/lib/zfs-linux/trim     | 29 +++++++++++++++----
>  debian/zfs-initramfs.lintian-overrides        |  1 +
>  debian/zfs-test.lintian-overrides             | 18 ++++++------
>  debian/zfsutils-linux.links                   |  3 ++
>  debian/zfsutils-linux.lintian-overrides       | 15 +++++-----
>  upstream                                      |  2 +-
>  9 files changed, 48 insertions(+), 26 deletions(-)
>  create mode 100644 debian/zfs-initramfs.lintian-overrides
>  create mode 100644 debian/zfsutils-linux.links
> 

applied, thanks!