[pve-devel] [PATCH zfsonlinux 0/3] update to 2.1.1

[pve-devel] [PATCH zfsonlinux 0/3] update to 2.1.1

[Stoiko Ivanov]

Changes from the RFC sent 20 days ago:
* ZFS 2.1.1 was tagged upstream
* the abi-tracking changes were ported to 2.1-release, rendering the ported
  patches superfluous

Tested with the upcoming 5.13 kernel on 2 VMs with storage-replication:
* created a snapshot before the upgrade on one machine,
* upgraded both nodes
* rebooted
* migrated container
* rolled back to the snapshot

all working as expected

Original cover-letter for the RFC:
This patchset updates our packaging for the changes of ZFS 2.1

Sending as RFC because:
* I'm still not 100% sure my solution to the ABI-changes and updated
  package names is correct
* It's based on the current tip of upstream's zfs-2.1.1-staging branch,
  which yet needs to be tagged

I'm intentionally omitting most of patch 4/5 - since it's 2MB of xml-abi
definition diffs - I'll gladly resent if somebody wants this - else the
commit is on my staff repository.

The upgrade was straight-forward - only small issues were regarding the
incompatibile abigail-tools versions (1.8 (bullseye) produces different
output to before, thus `make checkabi` failed initially).

Additionally the bump of the libzfs and libzpool library versions in
ec311430e2fd66492498a1559f56ef25e1192266 should not result in the libzfs
package name to be changed as well (libzfs increased it's current and age
parameter - which still leaves the soname at libzfs.so.4 when building on
my machine.

Did some minimal tests in a virtual 2-node test-cluster of mine with
storage replication for one VM (snapshotting, migrating, upgrading the
zpool, migrating back) - all seemed ok.



Stoiko Ivanov (3):
  update submodule and patches to ZFS 2.1.1-staging
  buildsys: adapt install paths for zfs 2.1.1
  bump libzpool soname

 Makefile                                      |  2 +-
 debian/control                                |  8 +--
 ...ibzpool4linux.docs => libzpool5linux.docs} |  0
 ...x.install.in => libzpool5linux.install.in} |  0
 ...rides => libzpool5linux.lintian-overrides} |  0
 ...ith-d-dev-disk-by-id-in-scan-service.patch |  6 +-
 .../0006-dont-symlink-zed-scripts.patch       |  5 +-
 .../patches/0007-Use-installed-python3.patch  |  4 +-
 ...-move-manpage-arcstat-1-to-arcstat-8.patch | 61 +++++++++----------
 debian/zfsutils-linux.install                 | 14 +++--
 upstream                                      |  2 +-
 11 files changed, 53 insertions(+), 49 deletions(-)
 rename debian/{libzpool4linux.docs => libzpool5linux.docs} (100%)
 rename debian/{libzpool4linux.install.in => libzpool5linux.install.in} (100%)
 rename debian/{libzpool4linux.lintian-overrides => libzpool5linux.lintian-overrides} (100%)

-- 
2.30.2

[pve-devel] [PATCH zfsonlinux 1/3] update submodule and patches to ZFS 2.1.1-staging

[Stoiko Ivanov]

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 ...ith-d-dev-disk-by-id-in-scan-service.patch |  6 +-
 .../0006-dont-symlink-zed-scripts.patch       |  5 +-
 .../patches/0007-Use-installed-python3.patch  |  4 +-
 ...-move-manpage-arcstat-1-to-arcstat-8.patch | 61 +++++++++----------
 upstream                                      |  2 +-
 5 files changed, 38 insertions(+), 40 deletions(-)

diff --git a/debian/patches/0004-import-with-d-dev-disk-by-id-in-scan-service.patch b/debian/patches/0004-import-with-d-dev-disk-by-id-in-scan-service.patch
index 46b03fd4..822409d6 100644
--- a/debian/patches/0004-import-with-d-dev-disk-by-id-in-scan-service.patch
+++ b/debian/patches/0004-import-with-d-dev-disk-by-id-in-scan-service.patch
@@ -14,15 +14,15 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
  1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/etc/systemd/system/zfs-import-scan.service.in b/etc/systemd/system/zfs-import-scan.service.in
-index f0317e23e..9a5e9cb17 100644
+index d3c083f7e..b1c8c7b9d 100644
 --- a/etc/systemd/system/zfs-import-scan.service.in
 +++ b/etc/systemd/system/zfs-import-scan.service.in
 @@ -13,7 +13,7 @@ ConditionPathIsDirectory=/sys/module/zfs
  [Service]
  Type=oneshot
  RemainAfterExit=yes
--ExecStart=@sbindir@/zpool import -aN -o cachefile=none
-+ExecStart=@sbindir@/zpool import -aN -d /dev/disk/by-id -o cachefile=none
+-ExecStart=@sbindir@/zpool import -aN -o cachefile=none $ZPOOL_IMPORT_OPTS
++ExecStart=@sbindir@/zpool import -aN -d /dev/disk/by-id -o cachefile=none $ZPOOL_IMPORT_OPTS
  
  [Install]
  WantedBy=zfs-import.target
diff --git a/debian/patches/0006-dont-symlink-zed-scripts.patch b/debian/patches/0006-dont-symlink-zed-scripts.patch
index 078f6d6d..60fe5507 100644
--- a/debian/patches/0006-dont-symlink-zed-scripts.patch
+++ b/debian/patches/0006-dont-symlink-zed-scripts.patch
@@ -16,10 +16,10 @@ Forwarded: no need
  1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/cmd/zed/zed.d/Makefile.am b/cmd/zed/zed.d/Makefile.am
-index 8b2d0c200..118c96547 100644
+index 2c8173b3e..ad39292e4 100644
 --- a/cmd/zed/zed.d/Makefile.am
 +++ b/cmd/zed/zed.d/Makefile.am
-@@ -48,6 +48,6 @@ install-data-hook:
+@@ -49,7 +49,7 @@ install-data-hook:
  	for f in $(zedconfdefaults); do \
  	  test -f "$(DESTDIR)$(zedconfdir)/$${f}" -o \
  	       -L "$(DESTDIR)$(zedconfdir)/$${f}" || \
@@ -27,3 +27,4 @@ index 8b2d0c200..118c96547 100644
 +	    echo "$${f}" >> "$(DESTDIR)$(zedexecdir)/DEFAULT-ENABLED" ; \
  	done
  	chmod 0600 "$(DESTDIR)$(zedconfdir)/zed.rc"
+ 
diff --git a/debian/patches/0007-Use-installed-python3.patch b/debian/patches/0007-Use-installed-python3.patch
index 789ce2ad..33188d49 100644
--- a/debian/patches/0007-Use-installed-python3.patch
+++ b/debian/patches/0007-Use-installed-python3.patch
@@ -28,11 +28,11 @@ index 3788543b0..c7ee4ae9a 100755
  
  typeset -i cnt=0
 diff --git a/tests/zfs-tests/tests/functional/rsend/send_encrypted_files.ksh b/tests/zfs-tests/tests/functional/rsend/send_encrypted_files.ksh
-index f89cb3b31..375d483f7 100755
+index 1fd21cbf7..791dd25d5 100755
 --- a/tests/zfs-tests/tests/functional/rsend/send_encrypted_files.ksh
 +++ b/tests/zfs-tests/tests/functional/rsend/send_encrypted_files.ksh
 @@ -87,7 +87,7 @@ log_must xattrtest -f 10 -x 3 -s 32768 -r -k -p /$TESTPOOL/$TESTFS2/xattrsadir
- # ZoL issue #7432
+ # OpenZFS issue #7432
  log_must zfs set compression=on xattr=sa $TESTPOOL/$TESTFS2
  log_must touch /$TESTPOOL/$TESTFS2/attrs
 -log_must eval "python -c 'print \"a\" * 4096' | \
diff --git a/debian/patches/0009-Patch-move-manpage-arcstat-1-to-arcstat-8.patch b/debian/patches/0009-Patch-move-manpage-arcstat-1-to-arcstat-8.patch
index 917a1815..6b0453b6 100644
--- a/debian/patches/0009-Patch-move-manpage-arcstat-1-to-arcstat-8.patch
+++ b/debian/patches/0009-Patch-move-manpage-arcstat-1-to-arcstat-8.patch
@@ -8,47 +8,44 @@ Originally-By: Antonio Russo <aerusso@aerusso.net>
 
 Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
 ---
- man/man1/Makefile.am                   | 2 +-
- man/man8/Makefile.am                   | 1 +
+ man/Makefile.am                        | 2 +-
  man/{man1/arcstat.1 => man8/arcstat.8} | 2 +-
- 3 files changed, 3 insertions(+), 2 deletions(-)
+ 2 files changed, 2 insertions(+), 2 deletions(-)
  rename man/{man1/arcstat.1 => man8/arcstat.8} (99%)
 
-diff --git a/man/man1/Makefile.am b/man/man1/Makefile.am
-index 8d7457a3e..101af7b6c 100644
---- a/man/man1/Makefile.am
-+++ b/man/man1/Makefile.am
-@@ -1,4 +1,4 @@
--dist_man_MANS = zhack.1 ztest.1 raidz_test.1 zvol_wait.1 arcstat.1
-+dist_man_MANS = zhack.1 ztest.1 raidz_test.1 zvol_wait.1
- EXTRA_DIST = cstyle.1
- 
- if BUILD_LINUX
-diff --git a/man/man8/Makefile.am b/man/man8/Makefile.am
-index 07f6aefa6..a757b1c62 100644
---- a/man/man8/Makefile.am
-+++ b/man/man8/Makefile.am
-@@ -1,6 +1,7 @@
- include $(top_srcdir)/config/Substfiles.am
- 
- dist_man_MANS = \
-+	arcstat.8 \
- 	fsck.zfs.8 \
- 	mount.zfs.8 \
- 	vdev_id.8 \
+diff --git a/man/Makefile.am b/man/Makefile.am
+index 8ab1b7572..8b3dbe633 100644
+--- a/man/Makefile.am
++++ b/man/Makefile.am
+@@ -8,7 +8,6 @@ dist_man_MANS = \
+ 	man1/ztest.1 \
+ 	man1/raidz_test.1 \
+ 	man1/zvol_wait.1 \
+-	man1/arcstat.1 \
+ 	\
+ 	man5/vdev_id.conf.5 \
+ 	\
+@@ -21,6 +20,7 @@ dist_man_MANS = \
+ 	man7/zpoolconcepts.7 \
+ 	man7/zpoolprops.7 \
+ 	\
++	man8/arcstat.8 \
+ 	man8/fsck.zfs.8 \
+ 	man8/mount.zfs.8 \
+ 	man8/vdev_id.8 \
 diff --git a/man/man1/arcstat.1 b/man/man8/arcstat.8
 similarity index 99%
 rename from man/man1/arcstat.1
 rename to man/man8/arcstat.8
-index ca508b49c..0aa81849a 100644
+index a69cd8937..dfe9c971b 100644
 --- a/man/man1/arcstat.1
 +++ b/man/man8/arcstat.8
 @@ -13,7 +13,7 @@
- .\" Copyright (c) 2015 by Delphix. All rights reserved.
  .\" Copyright (c) 2020 by AJ Jordan. All rights reserved.
  .\"
--.TH ARCSTAT 1 "Oct 20, 2020" OpenZFS
-+.TH ARCSTAT 8 "Oct 20, 2020" OpenZFS
- .SH NAME
- arcstat \- report ZFS ARC and L2ARC statistics
- .SH SYNOPSIS
+ .Dd May 26, 2021
+-.Dt ARCSTAT 1
++.Dt ARCSTAT 8
+ .Os
+ .
+ .Sh NAME
diff --git a/upstream b/upstream
index e9353bc2..71c60985 160000
--- a/upstream
+++ b/upstream
@@ -1 +1 @@
-Subproject commit e9353bc2ef27c300281574453e0cf5189a375680
+Subproject commit 71c6098526c6d5fbfa84a58cefe6cdc403488d8c
-- 
2.30.2

[pve-devel] [PATCH zfsonlinux 2/3] buildsys: adapt install paths for zfs 2.1.1

[Stoiko Ivanov]

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 debian/zfsutils-linux.install | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/debian/zfsutils-linux.install b/debian/zfsutils-linux.install
index 4f93aa70..b7501b30 100644
--- a/debian/zfsutils-linux.install
+++ b/debian/zfsutils-linux.install
@@ -28,6 +28,7 @@ sbin/zstreamdump
 usr/bin/zvol_wait
 usr/lib/modules-load.d/ lib/
 usr/lib/zfs-linux/zpool.d/
+usr/lib/zfs-linux/zpool_influxdb
 usr/sbin/arc_summary
 usr/sbin/arcstat
 usr/sbin/dbufstat
@@ -35,7 +36,14 @@ usr/share/bash-completion/completions
 usr/share/man/man8/arcstat.8
 usr/share/man/man1/zhack.1
 usr/share/man/man1/zvol_wait.1
+usr/share/man/man4/zfs.4
+usr/share/man/man4/spl.4
 usr/share/man/man5/
+usr/share/man/man7/zfsconcepts.7
+usr/share/man/man7/zfsprops.7
+usr/share/man/man7/zpoolconcepts.7
+usr/share/man/man7/zpoolprops.7
+usr/share/man/man7/zpool-features.7
 usr/share/man/man8/fsck.zfs.8
 usr/share/man/man8/mount.zfs.8
 usr/share/man/man8/vdev_id.8
@@ -79,8 +87,6 @@ usr/share/man/man8/zfs-userspace.8
 usr/share/man/man8/zfs-wait.8
 usr/share/man/man8/zfs.8
 usr/share/man/man8/zfs_ids_to_path.8
-usr/share/man/man8/zfsconcepts.8
-usr/share/man/man8/zfsprops.8
 usr/share/man/man8/zgenhostid.8
 usr/share/man/man8/zpool-add.8
 usr/share/man/man8/zpool-attach.8
@@ -94,6 +100,7 @@ usr/share/man/man8/zpool-export.8
 usr/share/man/man8/zpool-get.8
 usr/share/man/man8/zpool-history.8
 usr/share/man/man8/zpool-import.8
+usr/share/man/man8/zpool_influxdb.8
 usr/share/man/man8/zpool-initialize.8
 usr/share/man/man8/zpool-iostat.8
 usr/share/man/man8/zpool-labelclear.8
@@ -114,7 +121,6 @@ usr/share/man/man8/zpool-trim.8
 usr/share/man/man8/zpool-upgrade.8
 usr/share/man/man8/zpool-wait.8
 usr/share/man/man8/zpool.8
-usr/share/man/man8/zpoolconcepts.8
-usr/share/man/man8/zpoolprops.8
 usr/share/man/man8/zstream.8
 usr/share/man/man8/zstreamdump.8
+usr/share/zfs/compatibility.d/
-- 
2.30.2

[pve-devel] [PATCH zfsonlinux 3/3] bump libzpool soname

[Stoiko Ivanov]

following commit ec311430e2fd66492498a1559f56ef25e1192266 upstream

However since libzfs increased both the current as well as the age
the soname of the library remained at libzfs4.so - following
https://www.debian.org/doc/debian-policy/ch-sharedlibs.html
I left the package name at libzfs4linux

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 Makefile                                                  | 2 +-
 debian/control                                            | 8 ++++----
 debian/{libzpool4linux.docs => libzpool5linux.docs}       | 0
 ...ibzpool4linux.install.in => libzpool5linux.install.in} | 0
 ...lintian-overrides => libzpool5linux.lintian-overrides} | 0
 5 files changed, 5 insertions(+), 5 deletions(-)
 rename debian/{libzpool4linux.docs => libzpool5linux.docs} (100%)
 rename debian/{libzpool4linux.install.in => libzpool5linux.install.in} (100%)
 rename debian/{libzpool4linux.lintian-overrides => libzpool5linux.lintian-overrides} (100%)

diff --git a/Makefile b/Makefile
index a6eeca09..3113001a 100644
--- a/Makefile
+++ b/Makefile
@@ -13,7 +13,7 @@ libpam-zfs_${ZFSPKGVER}_amd64.deb		\
 libuutil3linux_${ZFSPKGVER}_amd64.deb		\
 libzfs4linux_${ZFSPKGVER}_amd64.deb		\
 libzfsbootenv1linux_${ZFSPKGVER}_amd64.deb	\
-libzpool4linux_${ZFSPKGVER}_amd64.deb		\
+libzpool5linux_${ZFSPKGVER}_amd64.deb		\
 zfs-test_${ZFSPKGVER}_amd64.deb			\
 zfsutils-linux_${ZFSPKGVER}_amd64.deb		\
 zfs-zed_${ZFSPKGVER}_amd64.deb
diff --git a/debian/control b/debian/control
index 89f51393..06ec27b7 100644
--- a/debian/control
+++ b/debian/control
@@ -70,7 +70,7 @@ Depends: libssl-dev | libssl1.0-dev,
          libuutil3linux (= ${binary:Version}),
          libzfs4linux (= ${binary:Version}),
          libzfsbootenv1linux (= ${binary:Version}),
-         libzpool4linux (= ${binary:Version}),
+         libzpool5linux (= ${binary:Version}),
          ${misc:Depends}
 Provides: libnvpair-dev, libuutil-dev
 Description: OpenZFS filesystem development files for Linux
@@ -78,7 +78,7 @@ Description: OpenZFS filesystem development files for Linux
  libraries of OpenZFS filesystem.
  .
  This package includes the development files of libnvpair3, libuutil3,
- libzpool4 and libzfs4.
+ libzpool5 and libzfs4, libzfsbootenv1.
 
 Package: libzfs4linux
 Section: contrib/libs
@@ -106,7 +106,7 @@ Description: OpenZFS filesystem library for Linux
  .
  The zfsbootenv library provides support for modifying ZFS label information.
 
-Package: libzpool4linux
+Package: libzpool5linux
 Section: contrib/libs
 Architecture: linux-any
 Depends: ${misc:Depends}, ${shlibs:Depends}
@@ -274,7 +274,7 @@ Suggests: libnvpair3linux-dbgsym,
          libuutil3linux-dbgsym,
          libzfs4linux-dbgsym,
          libzfsbootenv1linux-dbgsym,
-         libzpool4linux-dbgsym,
+         libzpool5linux-dbgsym,
          zfs-test-dbgsym,
          zfsutils-linux-dbgsym,
          zfs-zed-dbgsym,
diff --git a/debian/libzpool4linux.docs b/debian/libzpool5linux.docs
similarity index 100%
rename from debian/libzpool4linux.docs
rename to debian/libzpool5linux.docs
diff --git a/debian/libzpool4linux.install.in b/debian/libzpool5linux.install.in
similarity index 100%
rename from debian/libzpool4linux.install.in
rename to debian/libzpool5linux.install.in
diff --git a/debian/libzpool4linux.lintian-overrides b/debian/libzpool5linux.lintian-overrides
similarity index 100%
rename from debian/libzpool4linux.lintian-overrides
rename to debian/libzpool5linux.lintian-overrides
-- 
2.30.2

[pve-devel] applied-series: [PATCH zfsonlinux 0/3] update to 2.1.1

[Thomas Lamprecht]

On 27.09.21 20:07, Stoiko Ivanov wrote:
> Changes from the RFC sent 20 days ago:
> * ZFS 2.1.1 was tagged upstream
> * the abi-tracking changes were ported to 2.1-release, rendering the ported
>   patches superfluous
> 
> Tested with the upcoming 5.13 kernel on 2 VMs with storage-replication:
> * created a snapshot before the upgrade on one machine,
> * upgraded both nodes
> * rebooted
> * migrated container
> * rolled back to the snapshot
> 
> all working as expected
> 

> 
> Stoiko Ivanov (3):
>   update submodule and patches to ZFS 2.1.1-staging
>   buildsys: adapt install paths for zfs 2.1.1
>   bump libzpool soname
> 
>  Makefile                                      |  2 +-
>  debian/control                                |  8 +--
>  ...ibzpool4linux.docs => libzpool5linux.docs} |  0
>  ...x.install.in => libzpool5linux.install.in} |  0
>  ...rides => libzpool5linux.lintian-overrides} |  0
>  ...ith-d-dev-disk-by-id-in-scan-service.patch |  6 +-
>  .../0006-dont-symlink-zed-scripts.patch       |  5 +-
>  .../patches/0007-Use-installed-python3.patch  |  4 +-
>  ...-move-manpage-arcstat-1-to-arcstat-8.patch | 61 +++++++++----------
>  debian/zfsutils-linux.install                 | 14 +++--
>  upstream                                      |  2 +-
>  11 files changed, 53 insertions(+), 49 deletions(-)
>  rename debian/{libzpool4linux.docs => libzpool5linux.docs} (100%)
>  rename debian/{libzpool4linux.install.in => libzpool5linux.install.in} (100%)
>  rename debian/{libzpool4linux.lintian-overrides => libzpool5linux.lintian-overrides} (100%)
> 



applied, thanks!