[pve-devel] [PATCH bullseye kernel 0/6] debug package and packaging/build cleanups

[pve-devel] [PATCH bullseye kernel 0/6] debug package and packaging/build cleanups

[Fabian Grünbichler]

patches 3,4,6 can easily be backported to buster as well.

patch 5 is just an invocation of 'wrap-and-sort -f debian/control.in',
so trivially backportable or adaptable in case only some of this series
gets applied/context has changed.

patch 2 is needed for building on a clean bullseye system which does not
have (EOL) python2.7 around by default.

patch 1 only makes sense to backport if we also backport the related
tools, since the ones from buster seem to be too old for our kernel
versions (see #3464).

Fabian Grünbichler (6):
  build: keep unstripped kernel and module files
  d/rules: build perf with python3
  d/control: provide linux-libc-dev with version
  d/control: remove references to 2.6 kernel
  d/control: wrap-and-sort
  d/rules: close race between 'cp' and module handling

 debian/control.in | 37 ++++++++++++++++++++-----------------
 debian/rules      | 19 +++++++++++++++----
 2 files changed, 35 insertions(+), 21 deletions(-)

-- 
2.30.2

[pve-devel] [PATCH kernel 1/6] build: keep unstripped kernel and module files

[Fabian Grünbichler]

and put them into a new -dbgsym package for usage with
crash/kdump-tools/...

fixes #3465, and now allows to do the following (after installing
and configuring kdump-tools to collect kernel crash dumps) when the
system crashes:

  $ apt install pve-kernel-5.11.21-1-dbgsym
  $ crash /usr/lib/debug/boot/vmlinux-5.11.21-1-pve /var/crash/202106151236/dump.202106151236

  crash 7.2.9
  Copyright (C) 2002-2020  Red Hat, Inc.
  Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
  Copyright (C) 1999-2006  Hewlett-Packard Co
  Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
  Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
  Copyright (C) 2005, 2011  NEC Corporation
  Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
  Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
  This program is free software, covered by the GNU General Public License,
  and you are welcome to change it and/or distribute copies of it under
  certain conditions.  Enter "help copying" to see the conditions.
  This program has absolutely no warranty.  Enter "help warranty" for details.

  GNU gdb (GDB) 7.6
  Copyright (C) 2013 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
  This is free software: you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
  and "show warranty" for details.
  This GDB was configured as "x86_64-unknown-linux-gnu"...

  WARNING: kernel relocated [812MB]: patching 136336 gdb minimal_symbol values

        KERNEL: /usr/lib/debug/boot/vmlinux-5.11.21-1-pve
      DUMPFILE: /var/crash/202106151236/dump.202106151236  [PARTIAL DUMP]
          CPUS: 4
          DATE: Tue Jun 15 12:36:38 CEST 2021
        UPTIME: 00:06:21
  LOAD AVERAGE: 0.04, 0.11, 0.08
         TASKS: 272
      NODENAME: test
       RELEASE: 5.11.21-1-pve
       VERSION: #1 SMP PVE 5.11.21-1 (Tue, 01 Jun 2021 16:38:57 +0200)
       MACHINE: x86_64  (3696 Mhz)
        MEMORY: 8 GB
         PANIC: "Kernel panic - not syncing: sysrq triggered crash"
           PID: 3167
       COMMAND: "bash"
          TASK: ffff9220c8f5be00  [THREAD_INFO: ffff9220c8f5be00]
           CPU: 3
         STATE: TASK_RUNNING (PANIC)

  crash> bt
  PID: 3167   TASK: ffff9220c8f5be00  CPU: 3   COMMAND: "bash"
   #0 [ffffa24ec0bfbc80] machine_kexec at ffffffffb3c751f3
   #1 [ffffa24ec0bfbce0] __crash_kexec at ffffffffb3d61092
   #2 [ffffa24ec0bfbdb0] panic at ffffffffb47b769d
   #3 [ffffa24ec0bfbe30] sysrq_handle_crash at ffffffffb434da4a
   #4 [ffffa24ec0bfbe40] __handle_sysrq.cold at ffffffffb47e2cdc
   #5 [ffffa24ec0bfbe78] write_sysrq_trigger at ffffffffb434e3f8
   #6 [ffffa24ec0bfbe90] proc_reg_write at ffffffffb3fc09ea
   #7 [ffffa24ec0bfbeb0] vfs_write at ffffffffb3f143b6
   #8 [ffffa24ec0bfbee8] ksys_write at ffffffffb3f16b97
   #9 [ffffa24ec0bfbf28] __x64_sys_write at ffffffffb3f16c2a
  #10 [ffffa24ec0bfbf38] do_syscall_64 at ffffffffb480e868
  #11 [ffffa24ec0bfbf50] entry_SYSCALL_64_after_hwframe at ffffffffb4a0008c
      RIP: 00007f367f7baf33  RSP: 00007ffe6175dc98  RFLAGS: 00000246
      RAX: ffffffffffffffda  RBX: 0000000000000002  RCX: 00007f367f7baf33
      RDX: 0000000000000002  RSI: 0000560510e640b0  RDI: 0000000000000001
      RBP: 0000560510e640b0   R8: 000000000000000a   R9: 0000000000000001
      R10: 0000560510e5f800  R11: 0000000000000246  R12: 0000000000000002
      R13: 00007f367f88b6a0  R14: 0000000000000002  R15: 00007f367f88b8a0
      ORIG_RAX: 0000000000000001  CS: 0033  SS: 002b

as well as lots of other fun things (see 'help' after opening a crash dump).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---

Notes:
    requires a recent enough version of kdump-tools/makedumpfile, so bullseye+
    unless we provide our own backport for buster..
    
    the resulting package is obviously quite large, so we don't want to install it
    automatically

 debian/control.in |  8 ++++++++
 debian/rules      | 11 +++++++++++
 2 files changed, 19 insertions(+)

diff --git a/debian/control.in b/debian/control.in
index 1f235f0..a9c8dd8 100644
--- a/debian/control.in
+++ b/debian/control.in
@@ -71,6 +71,14 @@ Recommends: grub-pc | grub-efi-amd64 | grub-efi-ia32 | grub-efi-arm64,
 Description: The Proxmox PVE Kernel Image
  This package contains the linux kernel and initial ramdisk used for booting
 
+Package: pve-kernel-@KVNAME@-dbgsym
+Architecture: any
+Provides: linux-debug
+Section: devel
+Priority: optional
+Description: The Proxmox PVE Kernel debug image
+ This package provides the kernel debug image for version @KVNAME@. The debug kernel image contained in this package is NOT meant to boot from - it is uncompressed, and unstripped, and suitable for use with crash/kdump-tools/.. to analyze kernel crashes. This package also contains the pve-kernel modules in their unstripped version.
+
 Package: pve-kernel-libc-dev
 Section: devel
 Priority: optional
diff --git a/debian/rules b/debian/rules
index 7b1b251..ec22136 100755
--- a/debian/rules
+++ b/debian/rules
@@ -14,6 +14,7 @@ include debian/rules.d/${DEB_BUILD_ARCH}.mk
 CHANGELOG_DATE:=$(shell dpkg-parsechangelog -SDate)
 
 PVE_KERNEL_PKG=pve-kernel-${KVNAME}
+PVE_DEBUG_KERNEL_PKG=pve-kernel-${KVNAME}-dbgsym
 PVE_HEADER_PKG=pve-headers-${KVNAME}
 PVE_USR_HEADER_PKG=pve-kernel-libc-dev
 LINUX_TOOLS_PKG=linux-tools-${KERNEL_MAJMIN}
@@ -132,6 +133,16 @@ binary: install
 	install -m 644 $(addprefix ${MODULES}/,zfs.ko zavl.ko znvpair.ko zunicode.ko zcommon.ko icp.ko zlua.ko spl.ko zzstd.ko) debian/${PVE_KERNEL_PKG}/lib/modules/${KVNAME}/zfs
 	# remove firmware
 	rm -rf debian/${PVE_KERNEL_PKG}/lib/firmware
+
+	# debug package
+	mkdir -p debian/${PVE_DEBUG_KERNEL_PKG}/usr/lib/debug/lib/modules/${KVNAME}
+	mkdir debian/${PVE_DEBUG_KERNEL_PKG}/usr/lib/debug/boot
+	install -m 644 ${KERNEL_SRC}/vmlinux debian/${PVE_DEBUG_KERNEL_PKG}/usr/lib/debug/boot/vmlinux-${KVNAME}
+	cp -r debian/${PVE_KERNEL_PKG}/lib/modules/${KVNAME} debian/${PVE_DEBUG_KERNEL_PKG}/usr/lib/debug/lib/modules/
+	rm -f debian/${PVE_DEBUG_KERNEL_PKG}/usr/lib/debug/lib/modules/${KVNAME}/source
+	rm -f debian/${PVE_DEBUG_KERNEL_PKG}/usr/lib/debug/lib/modules/${KVNAME}/build
+	rm -f debian/${PVE_DEBUG_KERNEL_PKG}/usr/lib/debug/lib/modules/${KVNAME}/modules.*
+
 	# strip debug info
 	find debian/${PVE_KERNEL_PKG}/lib/modules -name \*.ko -print | while read f ; do strip --strip-debug "$$f"; done
 	# finalize
-- 
2.30.2

[pve-devel] [PATCH kernel 2/6] d/rules: build perf with python3

[Fabian Grünbichler]

python2 is gone with bullseye

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 debian/rules | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/rules b/debian/rules
index ec22136..0bd8eb5 100755
--- a/debian/rules
+++ b/debian/rules
@@ -157,7 +157,7 @@ binary: install
 	touch $@
 
 .tools_compile_mark: .compile_mark
-	${MAKE} -C ${KERNEL_SRC}/tools/perf prefix=/usr HAVE_NO_LIBBFD=1 HAVE_CPLUS_DEMANGLE_SUPPORT=1 NO_LIBPYTHON=1 NO_LIBPERL=1 NO_LIBCRYPTO=1 PYTHON=python2.7
+	${MAKE} -C ${KERNEL_SRC}/tools/perf prefix=/usr HAVE_NO_LIBBFD=1 HAVE_CPLUS_DEMANGLE_SUPPORT=1 NO_LIBPYTHON=1 NO_LIBPERL=1 NO_LIBCRYPTO=1 PYTHON=python3
 	echo "checking GPL-2 only perf binary for library linkage with incompatible licenses.."
 	! ldd ${KERNEL_SRC}/tools/perf/perf | grep -q -E '\blibbfd'
 	! ldd ${KERNEL_SRC}/tools/perf/perf | grep -q -E '\blibcrypto'
-- 
2.30.2

[pve-devel] [PATCH kernel 3/6] d/control: provide linux-libc-dev with version

[Fabian Grünbichler]

needed for it to be a proper replacement for linux-libc-dev when
resolving dependencies, such as for liburing-dev

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 debian/control.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/control.in b/debian/control.in
index a9c8dd8..6cdc9df 100644
--- a/debian/control.in
+++ b/debian/control.in
@@ -83,7 +83,7 @@ Package: pve-kernel-libc-dev
 Section: devel
 Priority: optional
 Architecture: any
-Provides: linux-libc-dev,
+Provides: linux-libc-dev (=${binary:Version}),
 Conflicts: linux-libc-dev,
 Replaces: linux-libc-dev,
 Depends: ${misc:Depends}
-- 
2.30.2

[pve-devel] [PATCH kernel 4/6] d/control: remove references to 2.6 kernel

[Fabian Grünbichler]

those days are long gone by now ;)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 debian/control.in | 2 --
 1 file changed, 2 deletions(-)

diff --git a/debian/control.in b/debian/control.in
index 6cdc9df..81be6ac 100644
--- a/debian/control.in
+++ b/debian/control.in
@@ -53,7 +53,6 @@ Section: devel
 Priority: optional
 Architecture: any
 Provides: linux-headers,
-          linux-headers-2.6,
 Depends: coreutils | fileutils (>= 4.0),
 Description: The Proxmox PVE Kernel Headers
  This package contains the linux kernel headers
@@ -63,7 +62,6 @@ Section: admin
 Priority: optional
 Architecture: any
 Provides: linux-image,
-          linux-image-2.6,
 Suggests: pve-firmware,
 Depends: busybox,
          initramfs-tools,
-- 
2.30.2

[pve-devel] [PATCH kernel 5/6] d/control: wrap-and-sort

[Fabian Grünbichler]

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 debian/control.in | 27 ++++++++++++---------------
 1 file changed, 12 insertions(+), 15 deletions(-)

diff --git a/debian/control.in b/debian/control.in
index 81be6ac..4bae66f 100644
--- a/debian/control.in
+++ b/debian/control.in
@@ -32,8 +32,8 @@ Build-Depends: asciidoc-base,
                sphinx-common,
                tar,
                xmlto,
-               zlib1g-dev,
-Build-Conflicts: pve-headers-@KVNAME@,
+               zlib1g-dev
+Build-Conflicts: pve-headers-@KVNAME@
 Vcs-Git: git://git.proxmox.com/git/pve-kernel
 Vcs-Browser: https://git.proxmox.com/?p=pve-kernel.git
 
@@ -41,9 +41,7 @@ Package: linux-tools-@KVMAJMIN@
 Architecture: any
 Section: devel
 Priority: optional
-Depends: linux-base,
-         ${misc:Depends},
-         ${shlibs:Depends},
+Depends: linux-base, ${misc:Depends}, ${shlibs:Depends}
 Description: Linux kernel version specific tools for version @KVMAJMIN@
  This package provides the architecture dependent parts for kernel
  version locked tools (such as perf and x86_energy_perf_policy)
@@ -52,8 +50,8 @@ Package: pve-headers-@KVNAME@
 Section: devel
 Priority: optional
 Architecture: any
-Provides: linux-headers,
-Depends: coreutils | fileutils (>= 4.0),
+Provides: linux-headers
+Depends: coreutils | fileutils (>= 4.0)
 Description: The Proxmox PVE Kernel Headers
  This package contains the linux kernel headers
 
@@ -61,11 +59,10 @@ Package: pve-kernel-@KVNAME@
 Section: admin
 Priority: optional
 Architecture: any
-Provides: linux-image,
-Suggests: pve-firmware,
-Depends: busybox,
-         initramfs-tools,
-Recommends: grub-pc | grub-efi-amd64 | grub-efi-ia32 | grub-efi-arm64,
+Provides: linux-image
+Suggests: pve-firmware
+Depends: busybox, initramfs-tools
+Recommends: grub-pc | grub-efi-amd64 | grub-efi-ia32 | grub-efi-arm64
 Description: The Proxmox PVE Kernel Image
  This package contains the linux kernel and initial ramdisk used for booting
 
@@ -81,9 +78,9 @@ Package: pve-kernel-libc-dev
 Section: devel
 Priority: optional
 Architecture: any
-Provides: linux-libc-dev (=${binary:Version}),
-Conflicts: linux-libc-dev,
-Replaces: linux-libc-dev,
+Provides: linux-libc-dev (=${binary:Version})
+Conflicts: linux-libc-dev
+Replaces: linux-libc-dev
 Depends: ${misc:Depends}
 Description: Linux support headers for userspace development
  This package provides userspaces headers from the Linux kernel.  These headers
-- 
2.30.2

[pve-devel] [RFC kernel 6/6] d/rules: close race between 'cp' and module handling

[Fabian Grünbichler]

sometimes the build would fail with

cp: cannot stat 'ubuntu-hirsute/.tmp_1987275': No such file or directory
make[1]: *** [debian/rules:181: .headers_prepare_mark] Error 1
make[1]: Leaving directory '/home/fgruenbichler/pve-kernel/build'
dpkg-buildpackage: error: fakeroot debian/rules binary subprocess returned exit status 2
make: *** [Makefile:58: pve-kernel-5.11.21-1-pve_5.11.21-1_amd64.deb] Error 2

if copying was slow enough.

so let's do the copying first, then do the rest in parallel without
needing to worry about side-effects.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---

Notes:
    resulting debs show the same contents over a few rebuilds, but a
    second pair of eyes does not hurt!

 debian/rules | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/debian/rules b/debian/rules
index 0bd8eb5..6be3f18 100755
--- a/debian/rules
+++ b/debian/rules
@@ -114,6 +114,9 @@ binary: install
 .config_mark:
 	cd ${KERNEL_SRC}; scripts/config ${PVE_CONFIG_OPTS}
 	${MAKE} -C ${KERNEL_SRC} oldconfig
+	# copy to allow building in parallel to kernel/module compilation without interference
+	rm -rf ${KERNEL_SRC_COPY}
+	cp -ar ${KERNEL_SRC} ${KERNEL_SRC_COPY}
 	touch $@
 
 .compile_mark: .config_mark
@@ -179,9 +182,6 @@ binary: install
 	rm -rf debian/${PVE_HEADER_PKG}
 	mkdir -p debian/${PVE_HEADER_PKG}/usr/src/linux-headers-${KVNAME}
 	install -m 0644 ${KERNEL_SRC}/.config debian/${PVE_HEADER_PKG}/usr/src/linux-headers-${KVNAME}
-	# copy to allow building in parallel to kernel/module compilation without interference
-	rm -rf ${KERNEL_SRC_COPY}
-	cp -ar ${KERNEL_SRC} ${KERNEL_SRC_COPY}
 	make -C ${KERNEL_SRC_COPY} mrproper
 	cd ${KERNEL_SRC_COPY}; find . -path './debian/*' -prune \
 	    -o -path './include/*' -prune \
-- 
2.30.2

[pve-devel] applied: [PATCH bullseye kernel 0/6] debug package and packaging/build cleanups

[Thomas Lamprecht]

On 15.06.21 13:27, Fabian Grünbichler wrote:
> patches 3,4,6 can easily be backported to buster as well.
> 
> patch 5 is just an invocation of 'wrap-and-sort -f debian/control.in',
> so trivially backportable or adaptable in case only some of this series
> gets applied/context has changed.
> 
> patch 2 is needed for building on a clean bullseye system which does not
> have (EOL) python2.7 around by default.
> 
> patch 1 only makes sense to backport if we also backport the related
> tools, since the ones from buster seem to be too old for our kernel
> versions (see #3464).
> 
> Fabian Grünbichler (6):
>   build: keep unstripped kernel and module files
>   d/rules: build perf with python3
>   d/control: provide linux-libc-dev with version
>   d/control: remove references to 2.6 kernel
>   d/control: wrap-and-sort
>   d/rules: close race between 'cp' and module handling
> 
>  debian/control.in | 37 ++++++++++++++++++++-----------------
>  debian/rules      | 19 +++++++++++++++----
>  2 files changed, 35 insertions(+), 21 deletions(-)
> 


applied series, much thanks! No hard feelings either way on backporting...