[pve-devel] [PATCH pve-kernel-meta/pve-installer] improve boot-mode switching

[pve-devel] [PATCH pve-kernel-meta/pve-installer] improve boot-mode switching

[Stoiko Ivanov]

This patchset is the result from the discussion on the rfc for using grub as
only boot loader [0].

The patches for pve-kernel-meta are based on top of the v3 of the
`pin/unpin` addition [1].

The patches for pve-installer are independent from the pve-kernel-meta ones
(my tests used the current PVE 7.1 ISO w/o updating the pve-kernel-helper
package) - but I still consider them linked.

changes from the rfc:
* instead of using grub everywhere - we now use systemd-boot always for efi
  systems and leave grub only for legacy booted systems
* instead of unconditionally installing the boot-loaders for both modes on
  all ESPs users now can opt-in to installing the boot-loader for the other
  bootmode (both are updated if their configs are present, by default we
  only install the boot-loader for the current boot mode)

Tested the pve-kernel-meta with 3 VMs (ZFS+uefi, ZFS+legacy, ext4+uefi) and
switching boot-modes (after the necessary preparations).
Tested the pve-installer by scping proxinstall into a VM and installing
on btrfs+uefi, ext4+uefi, btrfs+legacy

[0] https://lists.proxmox.com/pipermail/pve-devel/2022-February/051562.html
[1] https://lists.proxmox.com/pipermail/pve-devel/2022-February/051732.html


pve-kernel-meta:
Stoiko Ivanov (5):
  rename pve-efiboot-manual-kernels to proxmox-boot-manual-kernels
  proxmox-boot: add reinit subcommand
  proxmox-boot: refresh based on bootloader config instead of bootmode
  proxmox-boot: init bootloaders based on esp contents
  proxmox-boot: allow to manually specify bootloader

 bin/grub-install-wrapper          |  19 +-----
 bin/proxmox-boot-tool             | 100 +++++++++++++++++++++++++-----
 debian/pve-kernel-helper.postinst |   4 ++
 proxmox-boot/functions            |   2 +-
 proxmox-boot/zz-proxmox-boot      |  78 ++++++++---------------
 5 files changed, 116 insertions(+), 87 deletions(-)

pve-installer:
Stoiko Ivanov (2):
  remove /mnt/hostrun after install
  use proxmox-boot-tool for all uefi installs

 proxinstall | 81 ++++++++++-------------------------------------------
 1 file changed, 15 insertions(+), 66 deletions(-)

-- 
2.30.2

[pve-devel] [PATCH pve-kernel-meta 1/5] rename pve-efiboot-manual-kernels to proxmox-boot-manual-kernels

[Stoiko Ivanov]

was forgotten during the general renaming of pve-efiboot ->
proxmox-boot.

follows commit 8c0a22adfe15dc00cf2194647bb254201d8d187b

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 debian/pve-kernel-helper.postinst | 4 ++++
 proxmox-boot/functions            | 2 +-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/debian/pve-kernel-helper.postinst b/debian/pve-kernel-helper.postinst
index 891396c..4c78766 100644
--- a/debian/pve-kernel-helper.postinst
+++ b/debian/pve-kernel-helper.postinst
@@ -8,6 +8,10 @@ case "$1" in
             echo "$0: legacy ESP list /etc/kernel/pve-efiboot-uuids found moving to /etc/kernel/proxmox-boot-uuids" 1>&2
             mv /etc/kernel/pve-efiboot-uuids /etc/kernel/proxmox-boot-uuids
         fi
+        if [ -e /etc/kernel/pve-efiboot-manual-kernels ]; then
+            echo "$0: legacy manual kernel list /etc/kernel/pve-efiboot-manual-kernels found moving to /etc/kernel/proxmox-boot-manual-kernels" 1>&2
+            mv /etc/kernel/pve-efiboot-manual-kernels /etc/kernel/proxmox-boot-manual-kernels
+        fi
         ;;
 esac
 
diff --git a/proxmox-boot/functions b/proxmox-boot/functions
index b46d198..bdab987 100755
--- a/proxmox-boot/functions
+++ b/proxmox-boot/functions
@@ -4,7 +4,7 @@ set -e
 ESP_LIST="/etc/kernel/proxmox-boot-uuids"
 ESPTYPE='c12a7328-f81f-11d2-ba4b-00a0c93ec93b'
 
-MANUAL_KERNEL_LIST="/etc/kernel/pve-efiboot-manual-kernels"
+MANUAL_KERNEL_LIST="/etc/kernel/proxmox-boot-boot-manual-kernels"
 PINNED_KERNEL_CONF="/etc/kernel/proxmox-boot-pin"
 NEXT_BOOT_PIN="/etc/kernel/next-boot-pin"
 
-- 
2.30.2

[pve-devel] [PATCH pve-kernel-meta 2/5] proxmox-boot: add reinit subcommand

[Stoiko Ivanov]

to iterate over all configured ESPs and refresh the boot-loader
installations.

the init function was changed to not run refresh directly - to prevent
refresh from running once for each ESP

currently reinit does not imply refresh

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 bin/grub-install-wrapper | 19 ++-----------------
 bin/proxmox-boot-tool    | 39 +++++++++++++++++++++++++++++++++++----
 2 files changed, 37 insertions(+), 21 deletions(-)

diff --git a/bin/grub-install-wrapper b/bin/grub-install-wrapper
index 9facd04..90c144e 100755
--- a/bin/grub-install-wrapper
+++ b/bin/grub-install-wrapper
@@ -2,22 +2,6 @@
 
 set -e
 
-. /usr/share/pve-kernel-helper/scripts/functions
-
-init_boot_disks() {
-    if ! (echo "${curr_uuid}" | grep -qE '[0-9a-fA-F]{4}-[0-9a-fA-F]{4}'); then
-        warn "WARN: ${curr_uuid} read from ${ESP_LIST} does not look like a VFAT-UUID - skipping"
-        return
-    fi
-
-    path="/dev/disk/by-uuid/$curr_uuid"
-    if [ ! -e "${path}" ]; then
-        warn "WARN: ${path} does not exist - clean '${ESP_LIST}'! - skipping"
-        return
-    fi
-    proxmox-boot-tool init "$path"
-}
-
 if proxmox-boot-tool status --quiet; then
     # detect when being called by dpkg (e.g. grub-pc.postinst
     if [ -n "$DPKG_RUNNING_VERSION" ] && echo "$DPKG_MAINTSCRIPT_PACKAGE" | grep -sq "^grub-"; then
@@ -28,7 +12,8 @@ if proxmox-boot-tool status --quiet; then
             MARKER_FILE="/tmp/proxmox-boot-tool.dpkg.marker"
             if [ ! -e "$MARKER_FILE" ]; then
                 warn "This system is booted via proxmox-boot-tool, running proxmox-boot-tool init for all configured bootdisks"
-                loop_esp_list init_boot_disks
+                proxmox-boot-tool reinit
+                proxmox-boot-tool refresh
                 touch "$MARKER_FILE"
                 exit 0
             else
diff --git a/bin/proxmox-boot-tool b/bin/proxmox-boot-tool
index 178b6fb..5197f5b 100755
--- a/bin/proxmox-boot-tool
+++ b/bin/proxmox-boot-tool
@@ -124,7 +124,7 @@ format() {
 	exit 0
 }
 
-init() {
+init_bootloader() {
 	part="$1"
 
 	_get_partition_info "$part"
@@ -174,8 +174,20 @@ init() {
 	echo "Adding '$part' to list of synced ESPs.."
 	_add_entry_to_list_file "$ESP_LIST" "$UUID"
 
-	echo "Refreshing kernels and initrds.."
-	refresh
+}
+
+reinit() {
+	if ! (echo "${curr_uuid}" | grep -qE '[0-9a-fA-F]{4}-[0-9a-fA-F]{4}'); then
+		warn "WARN: ${curr_uuid} read from ${ESP_LIST} does not look like a VFAT-UUID - skipping"
+		return
+	fi
+
+	path="/dev/disk/by-uuid/$curr_uuid"
+	if [ ! -e "${path}" ]; then
+		warn "WARN: ${path} does not exist - clean '${ESP_LIST}'! - skipping"
+		return
+	fi
+	init_bootloader "$path"
 }
 
 _clean_impl() {
@@ -306,6 +318,7 @@ usage() {
 	warn ""
 	warn "       $0 format <partition> [--force]"
 	warn "       $0 init <partition>"
+	warn "       $0 reinit"
 	warn "       $0 clean [--dry-run]"
 	warn "       $0 refresh [--hook <name>]"
 	warn "       $0 kernel <add|remove> <kernel-version>"
@@ -325,6 +338,10 @@ help() {
 	echo ""
 	echo "    initialize EFI system partition at <partition> for automatic synchronization of pve-kernels and their associated initrds."
 	echo ""
+	echo "USAGE: $0 reinit"
+	echo ""
+	echo "    reinitialize all configured EFI system partitions from $ESP_LIST."
+	echo ""
 	echo "USAGE: $0 clean [--dry-run]"
 	echo ""
 	echo "    remove no longer existing EFI system partition UUIDs from $ESP_LIST. Use --dry-run to only print outdated entries instead of removing them."
@@ -493,7 +510,21 @@ case "$1" in
 			usage
 			exit 1
 		fi
-		init "$@"
+		init_bootloader "$@"
+		echo "Refreshing kernels and initrds.."
+		refresh
+		exit 0
+	;;
+	'reinit')
+		reexec_in_mountns "$@"
+		shift
+		if [ "$#" -eq 1 ]; then
+			warn "E: no arguments allowed."
+			warn ""
+			usage
+			exit 1
+		fi
+                loop_esp_list reinit "$@"
 		exit 0
 	;;
 	'clean')
-- 
2.30.2

[pve-devel] [PATCH pve-kernel-meta 3/5] proxmox-boot: refresh based on bootloader config instead of bootmode

[Stoiko Ivanov]

ignore the current bootmode (uefi/legacy) when deciding which configs
to generate - make this decision based on the found boot loader
configs on the ESP.

Change systemd-boot to load the kernel+initrd from the ESPs root where
grub needs them. This prevents the double use of space for systems
having both boot-loaders present (since FAT does not support
symlinks). While this is against the recommendations of the
boot-loader-specification [0]) it works fine (and we deviate from [0]
by not having the machine-id in the kernel subdirs anyways).
The UEFI spec on the directory structure [1], also should be fine with
it.

Additionally adapt the output of `p-b-t status` to the new paths

[0] https://systemd.io/BOOT_LOADER_SPECIFICATION/
[1] section 13.3.1.3 of
https://uefi.org/sites/default/files/resources/UEFI_Spec_2_9_2021_03_18.pdf

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 bin/proxmox-boot-tool        | 14 +++----
 proxmox-boot/zz-proxmox-boot | 78 ++++++++++++------------------------
 2 files changed, 32 insertions(+), 60 deletions(-)

diff --git a/bin/proxmox-boot-tool b/bin/proxmox-boot-tool
index 5197f5b..320f49b 100755
--- a/bin/proxmox-boot-tool
+++ b/bin/proxmox-boot-tool
@@ -396,20 +396,18 @@ _status_detail() {
 
 	result=""
 	if [ -f "${mountpoint}/$PMX_LOADER_CONF" ]; then
-		if [ ! -d "${mountpoint}/$PMX_ESP_DIR" ]; then
-			warn "${path}/$PMX_ESP_DIR does not exist"
-		fi
-		versions_uefi=$(ls -1 ${mountpoint}/$PMX_ESP_DIR | awk '{printf (NR>1?", ":"") $0}')
-		result="uefi (versions: ${versions_uefi})"
+		result="uefi"
 	fi
 	if [ -d "${mountpoint}/grub" ]; then
-		versions_grub=$(ls -1 ${mountpoint}/vmlinuz-* | awk '{ gsub(/.*\/vmlinuz-/, ""); printf (NR>1?", ":"") $0 }')
 		if [ -n "$result" ]; then
-		    result="${result}, grub (versions: ${versions_grub})"
+			result="${result}, grub"
 		else
-		    result="grub (versions: ${versions_grub})"
+			result="grub"
 		fi
 	fi
+	versions=$(find "${mountpoint}" -maxdepth 1 -name 'vmlinuz-*' | awk '{ gsub(/.*\/vmlinuz-/, ""); printf (NR>1?", ":"") $0 }')
+	result="${result} (versions: ${versions})"
+
 	echo "$curr_uuid is configured with: $result"
 	umount "${mountpoint}" || \
 		{ warn "umount of ${path} failed - failure"; exit 0; }
diff --git a/proxmox-boot/zz-proxmox-boot b/proxmox-boot/zz-proxmox-boot
index 5fe16a6..244373d 100755
--- a/proxmox-boot/zz-proxmox-boot
+++ b/proxmox-boot/zz-proxmox-boot
@@ -75,33 +75,33 @@ update_esp_func() {
 		{ warn "creation of mountpoint ${mountpoint} failed - skipping"; return; }
 	mount "${path}" "${mountpoint}" || \
 		{ warn "mount of ${path} failed - skipping"; return; }
-	if [ -d /sys/firmware/efi ]; then
-		if [ ! -f "${mountpoint}/$PMX_LOADER_CONF" ]; then
-			warn "${path} contains no loader.conf - skipping"
-			return
-		fi
-		if [ ! -d "${mountpoint}/$PMX_ESP_DIR" ]; then
-			warn "${path}/$PMX_ESP_DIR does not exist- skipping"
-			return
-		fi
-	elif [ ! -d "${mountpoint}/grub" ]; then
-		warn "${path} contains no grub directory - skipping"
+	sd_boot=""
+	grub=""
+	if [ -f "${mountpoint}/$PMX_LOADER_CONF" ]; then
+		sd_boot=1
+	fi
+	if [ -d "${mountpoint}/grub" ]; then
+		grub=1
+	fi
+	if [ -z "$sd_boot" ] && [ -z "$grub" ]; then
+		warn "${path} contains no bootloader config - skipping!"
 		return
 	fi
 	warn "Copying and configuring kernels on ${path}"
-	copy_and_config_kernels "${mountpoint}"
+	copy_and_config_kernels "${mountpoint}" "${sd_boot}"
 
 	pinned_kernel=$(get_first_line "${PINNED_KERNEL_CONF}")
 
 	if [ -e "${NEXT_BOOT_PIN}" ]; then
 	    pinned_kernel=$(get_first_line "${NEXT_BOOT_PIN}")
 	fi
-	if [ -d /sys/firmware/efi ]; then
+	if [ -n "$sd_boot" ]; then
 		set_systemd_boot_default "${mountpoint}" "${pinned_kernel}"
-		remove_old_kernels_efi "${mountpoint}"
-	else
+		remove_old_kernels "${mountpoint}"
+	fi
+	if [ -n "$grub" ]; then
 		set_grub_default "${pinned_kernel}"
-		remove_old_kernels_legacy "${mountpoint}"
+		remove_old_kernels "${mountpoint}"
 		mount --bind "${mountpoint}" "/boot"
 		update-grub
 		umount /boot
@@ -116,7 +116,7 @@ update_esp_func() {
 
 copy_and_config_kernels() {
 	esp="$1"
-
+	sd_boot="$2"
 
 	for kver in ${BOOT_KVERS}; do
 
@@ -132,52 +132,25 @@ copy_and_config_kernels() {
 			continue
 		fi
 
-		if [ -d /sys/firmware/efi ]; then
-
-			warn "	Copying kernel and creating boot-entry for ${kver}"
-			KERNEL_ESP_DIR="${PMX_ESP_DIR}/${kver}"
-			KERNEL_LIVE_DIR="${esp}/${KERNEL_ESP_DIR}"
-			mkdir -p "${KERNEL_LIVE_DIR}"
-			cp --preserve=timestamps "${linux_image}" "${KERNEL_LIVE_DIR}/"
-			cp --preserve=timestamps "${initrd}" "${KERNEL_LIVE_DIR}/"
+		warn "	Copying kernel and creating boot-entry for ${kver}"
+		cp --preserve=timestamps "${linux_image}" "${esp}/"
+		cp --preserve=timestamps "${initrd}" "${esp}/"
 
-			# create loader entry
+		if [ -n "$sd_boot" ]; then
+		# create loader entry
 			cat > "${esp}/loader/entries/proxmox-${kver}.conf" <<- EOF
 				title    ${LOADER_TITLE}
 				version  ${kver}
 				options  ${CMDLINE}
-				linux    /${KERNEL_ESP_DIR}/vmlinuz-${kver}
-				initrd   /${KERNEL_ESP_DIR}/initrd.img-${kver}
+				linux    /vmlinuz-${kver}
+				initrd   /initrd.img-${kver}
 			EOF
-		else
-			warn "	Copying kernel ${kver}"
-			cp --preserve=timestamps "${linux_image}" "${esp}/"
-			cp --preserve=timestamps "${initrd}" "${esp}/"
-		fi
-	done
-
-}
-
-remove_old_kernels_efi() {
-	esp="$1"
-
-	for kerneldir in "${esp}/${PMX_ESP_DIR}"/*; do
-		if [ ! -d "${kerneldir}" ]; then
-			warn "	${kerneldir} is not a directory - skipping"
-			continue
 		fi
-
-		kver="$(echo "${kerneldir}" | sed -r "s#^${esp}/${PMX_ESP_DIR}/(.+)\$#\\1#")"
-
-		echo "${BOOT_KVERS}" | grep -q "${kver}" && continue;
-		warn "	Removing old version ${kver}"
-		rm -rf "${kerneldir}"
-		rm -f "${esp}/loader/entries/proxmox-${kver}.conf"
 	done
 
 }
 
-remove_old_kernels_legacy() {
+remove_old_kernels() {
 	esp="$1"
 
 	for kernel in "${esp}/"vmlinuz-*; do
@@ -187,6 +160,7 @@ remove_old_kernels_legacy() {
 		warn "	Removing old version ${kver}"
 		rm -rf "${esp}/vmlinuz-${kver}"
 		rm -rf "${esp}/initrd.img-${kver}"
+		rm -rf "${esp}/loader/entries/proxmox-${kver}.conf"
 	done
 
 }
-- 
2.30.2

[pve-devel] [PATCH pve-kernel-meta 4/5] proxmox-boot: init bootloaders based on esp contents

[Stoiko Ivanov]

in addition to installing the boot loader for the current bootmode
(grub for legacy, systemd-boot for uefi) also install the bootloader
if its config is already present on the ESP.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 bin/proxmox-boot-tool | 26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)

diff --git a/bin/proxmox-boot-tool b/bin/proxmox-boot-tool
index 320f49b..c697f64 100755
--- a/bin/proxmox-boot-tool
+++ b/bin/proxmox-boot-tool
@@ -127,6 +127,9 @@ format() {
 init_bootloader() {
 	part="$1"
 
+	install_sd_boot=""
+	install_grub=""
+
 	_get_partition_info "$part"
 
 	if [ -z "$PARTTYPE" ] || [ "$PARTTYPE" != "$ESPTYPE" ]; then
@@ -150,16 +153,35 @@ init_bootloader() {
 	echo "Mounting '$part' on '$esp_mp'."
 	mount -t vfat "$part" "$esp_mp"
 
+	if [ -e "${esp_mp}/$PMX_LOADER_CONF" ]; then
+		install_sd_boot=1
+	fi
+	if [ -d "${esp_mp}/grub" ]; then
+		install_grub=1
+	fi
+	# always init for the current boot-mode
 	if [ -d /sys/firmware/efi ]; then
+	    install_sd_boot=1
+	else
+	    install_grub=1
+	fi
+
+	if [ -n "$install_sd_boot" ]; then
 		echo "Installing systemd-boot.."
 		mkdir -p "$esp_mp/$PMX_ESP_DIR"
-		bootctl --path "$esp_mp" install
+		bootctl_args="--path ${esp_mp}"
+		if [ ! -d /sys/firmware/efi ]; then
+			bootctl_args="${bootctl_args} --no-variables"
+		fi
 
+		bootctl ${bootctl_args} install
 		echo "Configuring systemd-boot.."
 		echo "timeout 3" > "$esp_mp/$PMX_LOADER_CONF.tmp"
 		echo "default proxmox-*" >> "$esp_mp/$PMX_LOADER_CONF.tmp"
 		mv "$esp_mp/$PMX_LOADER_CONF.tmp" "$esp_mp/$PMX_LOADER_CONF"
-	else
+	fi
+
+	if [ -n "$install_grub" ]; then
 		echo "Installing grub i386-pc target.."
 		grub-install.real \
 			--boot-directory $esp_mp \
-- 
2.30.2

[pve-devel] [PATCH pve-kernel-meta 5/5] proxmox-boot: allow to manually specify bootloader

[Stoiko Ivanov]

this commit adds the optional [--legacy|--uefi] arguments to
`p-b-t init`. If provided the respective boot-loader is installed
(regardless of what is found on the esp, and of the boot-mode)

This should make switching the boot-mode possible without the need
to boot into a live CD, chrooting and manually running the necessary
p-b-t commands.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 bin/proxmox-boot-tool | 47 ++++++++++++++++++++++++++++---------------
 1 file changed, 31 insertions(+), 16 deletions(-)

diff --git a/bin/proxmox-boot-tool b/bin/proxmox-boot-tool
index c697f64..68401ee 100755
--- a/bin/proxmox-boot-tool
+++ b/bin/proxmox-boot-tool
@@ -126,6 +126,7 @@ format() {
 
 init_bootloader() {
 	part="$1"
+	mode="$2"
 
 	install_sd_boot=""
 	install_grub=""
@@ -153,17 +154,25 @@ init_bootloader() {
 	echo "Mounting '$part' on '$esp_mp'."
 	mount -t vfat "$part" "$esp_mp"
 
-	if [ -e "${esp_mp}/$PMX_LOADER_CONF" ]; then
-		install_sd_boot=1
-	fi
-	if [ -d "${esp_mp}/grub" ]; then
-		install_grub=1
-	fi
-	# always init for the current boot-mode
-	if [ -d /sys/firmware/efi ]; then
-	    install_sd_boot=1
+	if [ -n "$mode" ]; then
+		if [ "$mode" = "--uefi" ]; then
+			install_sd_boot=1
+		elif [ "$mode" = "--legacy" ]; then
+			install_grub=1
+		fi
 	else
-	    install_grub=1
+		if [ -e "${esp_mp}/$PMX_LOADER_CONF" ]; then
+			install_sd_boot=1
+		fi
+		if [ -d "${esp_mp}/grub" ]; then
+			install_grub=1
+		fi
+		# always init for the current boot-mode
+		if [ -d /sys/firmware/efi ]; then
+		    install_sd_boot=1
+		else
+		    install_grub=1
+		fi
 	fi
 
 	if [ -n "$install_sd_boot" ]; then
@@ -192,7 +201,6 @@ init_bootloader() {
 	fi
 	echo "Unmounting '$part'."
 	umount "$part"
-
 	echo "Adding '$part' to list of synced ESPs.."
 	_add_entry_to_list_file "$ESP_LIST" "$UUID"
 
@@ -339,7 +347,7 @@ usage() {
 	warn "USAGE: $0 <commands> [ARGS]"
 	warn ""
 	warn "       $0 format <partition> [--force]"
-	warn "       $0 init <partition>"
+	warn "       $0 init <partition> [--legacy|--uefi]"
 	warn "       $0 reinit"
 	warn "       $0 clean [--dry-run]"
 	warn "       $0 refresh [--hook <name>]"
@@ -530,10 +538,17 @@ case "$1" in
 			usage
 			exit 1
 		fi
-		init_bootloader "$@"
-		echo "Refreshing kernels and initrds.."
-		refresh
-		exit 0
+		if [ -z "$2" ] || [ "$2" = "--legacy" ] || [ "$2" = "--uefi" ]; then
+			init_bootloader "$@"
+			echo "Refreshing kernels and initrds.."
+			refresh
+			exit 0
+		else
+			warn "E: invalid 'init' mode '$2'."
+			warn ""
+			usage
+			exit 1
+		fi
 	;;
 	'reinit')
 		reexec_in_mountns "$@"
-- 
2.30.2

[pve-devel] [PATCH installer 1/2] remove /mnt/hostrun after install

[Stoiko Ivanov]

else the directory lingers around on fresh systems.
follows 8d7ddbde84b3ebde29108b2080977b03853b2b69

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
noticed in my last tests - not really related to the series
 proxinstall | 1 +
 1 file changed, 1 insertion(+)

diff --git a/proxinstall b/proxinstall
index a39d405..bddcc5d 100755
--- a/proxinstall
+++ b/proxinstall
@@ -1937,6 +1937,7 @@ _EOD
     syscmd("umount $targetdir/proc");
     syscmd("umount $targetdir/sys/firmware/efi/efivars");
     syscmd("umount $targetdir/sys");
+    rmdir("$targetdir/mnt/hostrun");
 
     if ($use_zfs) {
 	syscmd("zfs umount -a") == 0 ||
-- 
2.30.2

[pve-devel] [PATCH installer 2/2] use proxmox-boot-tool for all uefi installs

[Stoiko Ivanov]

grub by itself is only used for btrfs/ext4/xfs installs on legacy
systems
for zfs proxmox-boot-tool is used in legacy and uefi mode

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 proxinstall | 80 ++++++++++-------------------------------------------
 1 file changed, 14 insertions(+), 66 deletions(-)

diff --git a/proxinstall b/proxinstall
index bddcc5d..df65725 100755
--- a/proxinstall
+++ b/proxinstall
@@ -1220,37 +1220,6 @@ sub prepare_proxmox_boot_esp {
 	die "unable to init ESP and install proxmox-boot loader on '$espdev'\n";
 }
 
-sub prepare_grub_efi_boot_esp {
-    my ($dev, $espdev, $targetdir) = @_;
-
-    syscmd("mount -n $espdev -t vfat $targetdir/boot/efi") == 0 ||
-	die "unable to mount $espdev\n";
-
-    eval {
-	my $rc = syscmd("chroot $targetdir /usr/sbin/grub-install --target x86_64-efi --no-floppy --bootloader-id='proxmox' $dev");
-	if ($rc != 0) {
-	    if ($boot_type eq 'efi') {
-		die "unable to install the EFI boot loader on '$dev'\n";
-	    } else {
-		warn "unable to install the EFI boot loader on '$dev', ignoring (not booted using UEFI)\n";
-	    }
-	}
-	# also install fallback boot file (OVMF does not boot without)
-	mkdir("$targetdir/boot/efi/EFI/BOOT");
-	syscmd("cp $targetdir/boot/efi/EFI/proxmox/grubx64.efi $targetdir/boot/efi/EFI/BOOT/BOOTx64.EFI") == 0 ||
-	    die "unable to copy efi boot loader\n";
-    };
-    my $err = $@;
-
-    eval {
-	syscmd("umount $targetdir/boot/efi") == 0 ||
-	    die "unable to umount $targetdir/boot/efi\n";
-    };
-    warn $@ if $@;
-
-    die "failed to prepare EFI boot using Grub on '$espdev': $err" if $err;
-}
-
 sub extract_data {
     my ($basefile, $targetdir) = @_;
 
@@ -1266,6 +1235,7 @@ sub extract_data {
 
     my $use_zfs = 0;
     my $use_btrfs = 0;
+    my $btrfs_uuid;
 
     my $filesys = $config_options->{filesys};
 
@@ -1492,7 +1462,6 @@ sub extract_data {
 	}
 
 	mkdir "$targetdir/boot";
-	mkdir "$targetdir/boot/efi";
 
 	mkdir "$targetdir/var";
 	mkdir "$targetdir/var/lib";
@@ -1616,43 +1585,23 @@ sub extract_data {
 	if ($use_zfs) {
 	    # do nothing
 	} elsif ($use_btrfs) {
-	    my $fsuuid;
 	    my $cmd = "blkid -u filesystem -t TYPE=btrfs -o export $rootdev";
 	    run_command($cmd, sub {
 		my $line = shift;
 
 		if ($line =~ m/^UUID=([A-Fa-f0-9\-]+)$/) {
-		    $fsuuid = $1;
+		    $btrfs_uuid = $1;
 		}
 	    });
 
-	    die "unable to detect FS UUID" if !defined($fsuuid);
+	    die "unable to detect FS UUID" if !defined($btrfs_uuid);
 
-	    $fstab .= "UUID=$fsuuid / btrfs defaults 0 1\n";
+	    $fstab .= "UUID=$btrfs_uuid / btrfs defaults 0 1\n";
 	} else {
 	    my $root_mountopt = $fssetup->{$filesys}->{root_mountopt} || 'defaults';
 	    $fstab .= "$rootdev / $filesys ${root_mountopt} 0 1\n";
 	}
 
-	# mount /boot/efi
-	# Note: this is required by current grub, but really dangerous, because
-	# vfat does not have journaling, so it triggers manual fsck after each crash
-	# so we only mount /boot/efi if really required (efi systems).
-	if ($boot_type eq 'efi' && !$use_zfs) {
-	    if (scalar(@$bootdevinfo)) {
-		my $di = @$bootdevinfo[0]; # simply use first disk
-
-		if ($di->{esp}) {
-		    my $efi_boot_uuid = $di->{esp};
-		    if (my $uuid = find_dev_by_uuid ($di->{esp})) {
-			$efi_boot_uuid = "UUID=$uuid";
-		    }
-
-		    $fstab .= "${efi_boot_uuid} /boot/efi vfat defaults 0 1\n";
-		}
-	    }
-	}
-
 
 	$fstab .= "$swapfile none swap sw 0 0\n" if $swapfile;
 
@@ -1791,6 +1740,12 @@ _EOD
 
 	    write_config("root=ZFS=$zfspoolname/ROOT/$zfsrootvolname boot=zfs\n", "$targetdir/etc/kernel/cmdline");
 
+	} elsif ($boot_type eq 'efi') {
+	    if ($use_btrfs){
+		write_config("root=UUID=$btrfs_uuid ro quiet\n", "$targetdir/etc/kernel/cmdline");
+	    } else {
+		write_config("root=/dev/mapper/pve-root ro quiet\n", "$targetdir/etc/kernel/cmdline");
+	    }
 	}
 
 	diversion_remove($targetdir, "/usr/sbin/update-grub");
@@ -1823,7 +1778,7 @@ _EOD
 
 		foreach my $di (@$bootdevinfo) {
 		    my $dev = $di->{devname};
-		    if ($use_zfs) {
+		    if ($use_zfs || ($boot_type eq 'efi')) {
 			prepare_proxmox_boot_esp($di->{esp}, $targetdir);
 		    } else {
 			if (!$native_4k_disk_bootable) {
@@ -1833,18 +1788,11 @@ _EOD
 			    };
 			    push @$bootloader_err_list, $@ if $@;
 			}
-
-		    eval {
-			if (my $esp = $di->{esp}) {
-				prepare_grub_efi_boot_esp($dev, $esp, $targetdir);
-			    }
-			}
-		    };
-		    push @$bootloader_err_list, $@ if $@;
+			syscmd("chroot $targetdir /usr/sbin/update-grub") == 0 ||
+			    die "unable to update boot loader config\n";
+		    }
 		}
 
-		syscmd("chroot $targetdir /usr/sbin/update-grub") == 0 ||
-		    die "unable to update boot loader config\n";
 	    };
 	    push @$bootloader_err_list, $@ if $@;
 
-- 
2.30.2

Re: [pve-devel] [PATCH pve-kernel-meta 1/5] rename pve-efiboot-manual-kernels to proxmox-boot-manual-kernels

[Fabian Grünbichler]

needs a (trivial) rebase ;)

On February 14, 2022 4:57 pm, Stoiko Ivanov wrote:
> was forgotten during the general renaming of pve-efiboot ->
> proxmox-boot.
> 
> follows commit 8c0a22adfe15dc00cf2194647bb254201d8d187b
> 
> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> ---
>  debian/pve-kernel-helper.postinst | 4 ++++
>  proxmox-boot/functions            | 2 +-
>  2 files changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/debian/pve-kernel-helper.postinst b/debian/pve-kernel-helper.postinst
> index 891396c..4c78766 100644
> --- a/debian/pve-kernel-helper.postinst
> +++ b/debian/pve-kernel-helper.postinst
> @@ -8,6 +8,10 @@ case "$1" in
>              echo "$0: legacy ESP list /etc/kernel/pve-efiboot-uuids found moving to /etc/kernel/proxmox-boot-uuids" 1>&2
>              mv /etc/kernel/pve-efiboot-uuids /etc/kernel/proxmox-boot-uuids
>          fi
> +        if [ -e /etc/kernel/pve-efiboot-manual-kernels ]; then
> +            echo "$0: legacy manual kernel list /etc/kernel/pve-efiboot-manual-kernels found moving to /etc/kernel/proxmox-boot-manual-kernels" 1>&2
> +            mv /etc/kernel/pve-efiboot-manual-kernels /etc/kernel/proxmox-boot-manual-kernels
> +        fi
>          ;;
>  esac
>  
> diff --git a/proxmox-boot/functions b/proxmox-boot/functions
> index b46d198..bdab987 100755
> --- a/proxmox-boot/functions
> +++ b/proxmox-boot/functions
> @@ -4,7 +4,7 @@ set -e
>  ESP_LIST="/etc/kernel/proxmox-boot-uuids"
>  ESPTYPE='c12a7328-f81f-11d2-ba4b-00a0c93ec93b'
>  
> -MANUAL_KERNEL_LIST="/etc/kernel/pve-efiboot-manual-kernels"
> +MANUAL_KERNEL_LIST="/etc/kernel/proxmox-boot-boot-manual-kernels"
>  PINNED_KERNEL_CONF="/etc/kernel/proxmox-boot-pin"
>  NEXT_BOOT_PIN="/etc/kernel/next-boot-pin"
>  
> -- 
> 2.30.2
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
> 

Re: [pve-devel] [PATCH pve-kernel-meta 3/5] proxmox-boot: refresh based on bootloader config instead of bootmode

[Fabian Grünbichler]

On February 14, 2022 4:57 pm, Stoiko Ivanov wrote:
> ignore the current bootmode (uefi/legacy) when deciding which configs
> to generate - make this decision based on the found boot loader
> configs on the ESP.
> 
> Change systemd-boot to load the kernel+initrd from the ESPs root where
> grub needs them. This prevents the double use of space for systems
> having both boot-loaders present (since FAT does not support
> symlinks). While this is against the recommendations of the
> boot-loader-specification [0]) it works fine (and we deviate from [0]
> by not having the machine-id in the kernel subdirs anyways).
> The UEFI spec on the directory structure [1], also should be fine with
> it.
> 
> Additionally adapt the output of `p-b-t status` to the new paths
> 
> [0] https://systemd.io/BOOT_LOADER_SPECIFICATION/
> [1] section 13.3.1.3 of
> https://uefi.org/sites/default/files/resources/UEFI_Spec_2_9_2021_03_18.pdf
> 
> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> ---
>  bin/proxmox-boot-tool        | 14 +++----
>  proxmox-boot/zz-proxmox-boot | 78 ++++++++++++------------------------
>  2 files changed, 32 insertions(+), 60 deletions(-)
> 
> diff --git a/bin/proxmox-boot-tool b/bin/proxmox-boot-tool
> index 5197f5b..320f49b 100755
> --- a/bin/proxmox-boot-tool
> +++ b/bin/proxmox-boot-tool
> @@ -396,20 +396,18 @@ _status_detail() {
>  
>  	result=""
>  	if [ -f "${mountpoint}/$PMX_LOADER_CONF" ]; then
> -		if [ ! -d "${mountpoint}/$PMX_ESP_DIR" ]; then
> -			warn "${path}/$PMX_ESP_DIR does not exist"
> -		fi
> -		versions_uefi=$(ls -1 ${mountpoint}/$PMX_ESP_DIR | awk '{printf (NR>1?", ":"") $0}')
> -		result="uefi (versions: ${versions_uefi})"
> +		result="uefi"
>  	fi
>  	if [ -d "${mountpoint}/grub" ]; then
> -		versions_grub=$(ls -1 ${mountpoint}/vmlinuz-* | awk '{ gsub(/.*\/vmlinuz-/, ""); printf (NR>1?", ":"") $0 }')
>  		if [ -n "$result" ]; then
> -		    result="${result}, grub (versions: ${versions_grub})"
> +			result="${result}, grub"
>  		else
> -		    result="grub (versions: ${versions_grub})"
> +			result="grub"

this here

>  		fi
>  	fi
> +	versions=$(find "${mountpoint}" -maxdepth 1 -name 'vmlinuz-*' | awk '{ gsub(/.*\/vmlinuz-/, ""); printf (NR>1?", ":"") $0 }')
> +	result="${result} (versions: ${versions})"
> +
>  	echo "$curr_uuid is configured with: $result"
>  	umount "${mountpoint}" || \
>  		{ warn "umount of ${path} failed - failure"; exit 0; }
> diff --git a/proxmox-boot/zz-proxmox-boot b/proxmox-boot/zz-proxmox-boot
> index 5fe16a6..244373d 100755
> --- a/proxmox-boot/zz-proxmox-boot
> +++ b/proxmox-boot/zz-proxmox-boot
> @@ -75,33 +75,33 @@ update_esp_func() {
>  		{ warn "creation of mountpoint ${mountpoint} failed - skipping"; return; }
>  	mount "${path}" "${mountpoint}" || \
>  		{ warn "mount of ${path} failed - skipping"; return; }
> -	if [ -d /sys/firmware/efi ]; then
> -		if [ ! -f "${mountpoint}/$PMX_LOADER_CONF" ]; then
> -			warn "${path} contains no loader.conf - skipping"
> -			return
> -		fi
> -		if [ ! -d "${mountpoint}/$PMX_ESP_DIR" ]; then
> -			warn "${path}/$PMX_ESP_DIR does not exist- skipping"
> -			return
> -		fi
> -	elif [ ! -d "${mountpoint}/grub" ]; then
> -		warn "${path} contains no grub directory - skipping"
> +	sd_boot=""
> +	grub=""
> +	if [ -f "${mountpoint}/$PMX_LOADER_CONF" ]; then
> +		sd_boot=1
> +	fi
> +	if [ -d "${mountpoint}/grub" ]; then
> +		grub=1
> +	fi

and this here looks rather similar - maybe it makes sense to have 
helpers for 'ESP contains grub' and 'ESP contains sd-boot' before we add 
more copies? ;)

> +	if [ -z "$sd_boot" ] && [ -z "$grub" ]; then
> +		warn "${path} contains no bootloader config - skipping!"
>  		return
>  	fi
>  	warn "Copying and configuring kernels on ${path}"
> -	copy_and_config_kernels "${mountpoint}"
> +	copy_and_config_kernels "${mountpoint}" "${sd_boot}"
>  
>  	pinned_kernel=$(get_first_line "${PINNED_KERNEL_CONF}")
>  
>  	if [ -e "${NEXT_BOOT_PIN}" ]; then
>  	    pinned_kernel=$(get_first_line "${NEXT_BOOT_PIN}")
>  	fi
> -	if [ -d /sys/firmware/efi ]; then
> +	if [ -n "$sd_boot" ]; then
>  		set_systemd_boot_default "${mountpoint}" "${pinned_kernel}"
> -		remove_old_kernels_efi "${mountpoint}"
> -	else
> +		remove_old_kernels "${mountpoint}"
> +	fi
> +	if [ -n "$grub" ]; then
>  		set_grub_default "${pinned_kernel}"
> -		remove_old_kernels_legacy "${mountpoint}"
> +		remove_old_kernels "${mountpoint}"
>  		mount --bind "${mountpoint}" "/boot"
>  		update-grub
>  		umount /boot
> @@ -116,7 +116,7 @@ update_esp_func() {
>  
>  copy_and_config_kernels() {
>  	esp="$1"
> -
> +	sd_boot="$2"
>  
>  	for kver in ${BOOT_KVERS}; do
>  
> @@ -132,52 +132,25 @@ copy_and_config_kernels() {
>  			continue
>  		fi
>  
> -		if [ -d /sys/firmware/efi ]; then
> -
> -			warn "	Copying kernel and creating boot-entry for ${kver}"
> -			KERNEL_ESP_DIR="${PMX_ESP_DIR}/${kver}"
> -			KERNEL_LIVE_DIR="${esp}/${KERNEL_ESP_DIR}"
> -			mkdir -p "${KERNEL_LIVE_DIR}"
> -			cp --preserve=timestamps "${linux_image}" "${KERNEL_LIVE_DIR}/"
> -			cp --preserve=timestamps "${initrd}" "${KERNEL_LIVE_DIR}/"
> +		warn "	Copying kernel and creating boot-entry for ${kver}"

nit: technically only true for the uefi/sd_boot if below

> +		cp --preserve=timestamps "${linux_image}" "${esp}/"
> +		cp --preserve=timestamps "${initrd}" "${esp}/"
>  
> -			# create loader entry
> +		if [ -n "$sd_boot" ]; then
> +		# create loader entry
>  			cat > "${esp}/loader/entries/proxmox-${kver}.conf" <<- EOF
>  				title    ${LOADER_TITLE}
>  				version  ${kver}
>  				options  ${CMDLINE}
> -				linux    /${KERNEL_ESP_DIR}/vmlinuz-${kver}
> -				initrd   /${KERNEL_ESP_DIR}/initrd.img-${kver}
> +				linux    /vmlinuz-${kver}
> +				initrd   /initrd.img-${kver}
>  			EOF
> -		else
> -			warn "	Copying kernel ${kver}"
> -			cp --preserve=timestamps "${linux_image}" "${esp}/"
> -			cp --preserve=timestamps "${initrd}" "${esp}/"
> -		fi
> -	done
> -
> -}
> -
> -remove_old_kernels_efi() {
> -	esp="$1"
> -
> -	for kerneldir in "${esp}/${PMX_ESP_DIR}"/*; do
> -		if [ ! -d "${kerneldir}" ]; then
> -			warn "	${kerneldir} is not a directory - skipping"
> -			continue
>  		fi
> -
> -		kver="$(echo "${kerneldir}" | sed -r "s#^${esp}/${PMX_ESP_DIR}/(.+)\$#\\1#")"
> -
> -		echo "${BOOT_KVERS}" | grep -q "${kver}" && continue;
> -		warn "	Removing old version ${kver}"
> -		rm -rf "${kerneldir}"
> -		rm -f "${esp}/loader/entries/proxmox-${kver}.conf"
>  	done

this still needs to be done at least once per ESP post-upgrade, else all 
the (now unused) kernels+initrds in the old efi location are kept around 
forever, taking up precious space on our small ESPs ;)

maybe replace it with a simple

if dir_exists; then rm -rf ...; fi

?

>  
>  }
>  
> -remove_old_kernels_legacy() {
> +remove_old_kernels() {
>  	esp="$1"
>  
>  	for kernel in "${esp}/"vmlinuz-*; do
> @@ -187,6 +160,7 @@ remove_old_kernels_legacy() {
>  		warn "	Removing old version ${kver}"
>  		rm -rf "${esp}/vmlinuz-${kver}"
>  		rm -rf "${esp}/initrd.img-${kver}"
> +		rm -rf "${esp}/loader/entries/proxmox-${kver}.conf"
>  	done
>  
>  }
> -- 
> 2.30.2
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
> 

Re: [pve-devel] [PATCH pve-kernel-meta 5/5] proxmox-boot: allow to manually specify bootloader

[Fabian Grünbichler]

On February 14, 2022 4:57 pm, Stoiko Ivanov wrote:
> this commit adds the optional [--legacy|--uefi] arguments to
> `p-b-t init`. If provided the respective boot-loader is installed
> (regardless of what is found on the esp, and of the boot-mode)

would IMHO also make sense for `reinit` to do it for all configured 
ESPs? granted, it's likely that people will switch from 'not managed by 
p-b-t' to 'managed by p-b-t', and not so likely that they have to switch 
from 'managed by p-b-t, UEFI' to 'managed by p-b-t, legacy' or 
vice-versa, but it's basically free and you never know ;)

> 
> This should make switching the boot-mode possible without the need
> to boot into a live CD, chrooting and manually running the necessary
> p-b-t commands.
> 
> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> ---
>  bin/proxmox-boot-tool | 47 ++++++++++++++++++++++++++++---------------
>  1 file changed, 31 insertions(+), 16 deletions(-)
> 
> diff --git a/bin/proxmox-boot-tool b/bin/proxmox-boot-tool
> index c697f64..68401ee 100755
> --- a/bin/proxmox-boot-tool
> +++ b/bin/proxmox-boot-tool
> @@ -126,6 +126,7 @@ format() {
>  
>  init_bootloader() {
>  	part="$1"
> +	mode="$2"
>  
>  	install_sd_boot=""
>  	install_grub=""
> @@ -153,17 +154,25 @@ init_bootloader() {
>  	echo "Mounting '$part' on '$esp_mp'."
>  	mount -t vfat "$part" "$esp_mp"
>  
> -	if [ -e "${esp_mp}/$PMX_LOADER_CONF" ]; then
> -		install_sd_boot=1
> -	fi
> -	if [ -d "${esp_mp}/grub" ]; then
> -		install_grub=1
> -	fi
> -	# always init for the current boot-mode
> -	if [ -d /sys/firmware/efi ]; then
> -	    install_sd_boot=1
> +	if [ -n "$mode" ]; then
> +		if [ "$mode" = "--uefi" ]; then
> +			install_sd_boot=1
> +		elif [ "$mode" = "--legacy" ]; then
> +			install_grub=1
> +		fi
>  	else
> -	    install_grub=1
> +		if [ -e "${esp_mp}/$PMX_LOADER_CONF" ]; then
> +			install_sd_boot=1
> +		fi
> +		if [ -d "${esp_mp}/grub" ]; then
> +			install_grub=1
> +		fi
> +		# always init for the current boot-mode
> +		if [ -d /sys/firmware/efi ]; then
> +		    install_sd_boot=1
> +		else
> +		    install_grub=1
> +		fi
>  	fi
>  
>  	if [ -n "$install_sd_boot" ]; then
> @@ -192,7 +201,6 @@ init_bootloader() {
>  	fi
>  	echo "Unmounting '$part'."
>  	umount "$part"
> -
>  	echo "Adding '$part' to list of synced ESPs.."
>  	_add_entry_to_list_file "$ESP_LIST" "$UUID"
>  
> @@ -339,7 +347,7 @@ usage() {
>  	warn "USAGE: $0 <commands> [ARGS]"
>  	warn ""
>  	warn "       $0 format <partition> [--force]"
> -	warn "       $0 init <partition>"
> +	warn "       $0 init <partition> [--legacy|--uefi]"
>  	warn "       $0 reinit"
>  	warn "       $0 clean [--dry-run]"
>  	warn "       $0 refresh [--hook <name>]"
> @@ -530,10 +538,17 @@ case "$1" in
>  			usage
>  			exit 1
>  		fi
> -		init_bootloader "$@"
> -		echo "Refreshing kernels and initrds.."
> -		refresh
> -		exit 0
> +		if [ -z "$2" ] || [ "$2" = "--legacy" ] || [ "$2" = "--uefi" ]; then
> +			init_bootloader "$@"
> +			echo "Refreshing kernels and initrds.."
> +			refresh
> +			exit 0
> +		else
> +			warn "E: invalid 'init' mode '$2'."
> +			warn ""
> +			usage
> +			exit 1
> +		fi
>  	;;
>  	'reinit')
>  		reexec_in_mountns "$@"
> -- 
> 2.30.2
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
>