[pve-devel] [RFC pve-kernel-meta 0/5] unify boot-mode config

[pve-devel] [RFC pve-kernel-meta 0/5] unify boot-mode config

[Stoiko Ivanov]

This patchset is based on top of the pin/unpin/next-boot series (in it's v1)

It contains changes I would consider breaking (in the sense that as a user I
would expect to read about them in some Upgrade-notes, e.g. for a minor
release)

The changes should make it possible to boot a system irrespective of the
current boot-mode (UEFI/legacy) in the bios. Users should be able to change
their boot-mode in the bios, and the system should come up cleanly (and with
the same kernel-version as with the other boot-mode)

(currently changing boot-modes with p-b-t requrires booting from a live-CD)

patch 3 drops systemd-boot and uses grub for both boot-modes, hopefully
unifying the boot-experience and causing less confusion (currently I suggest
to look at the screen while booting to find out which boot-loader is used)

(Sadly systemd-boot (which I would prefer, justifiably)
won't get support for legacy boot)

* patch 1/5 renames a config-file (forgotten during the
  pve-efiboot->proxmox-boot rename)
* sending as RFC - since it's really just and idea (and I'm not sure it
  won't cause even more confusion/breakage in practice)
* very roughly tested on 2 VMs (manually reformatting the ESP to clean out
  systemd-boot) - both booted fine when changing their BIOS mode

as is it's definitely missing:
* a plan for transitioning current systems
* cleanup of the code (the changes are minimal for a POC)
* updated documentation
* quite a bit more testing (especially on real hardware)

Stoiko Ivanov (5):
  rename pve-efiboot-manual-kernels to proxmox-boot-manual-kernels
  proxmox-boot: add reinit subcommand
  proxmox-boot: keep EFI and legacy bootloaders in sync
  proxmox-boot: use grub for UEFI boot
  proxmox-boot: install grub in esp/EFI/BOOT/BOOTX64.EFI

 bin/grub-install-wrapper          | 19 +-------
 bin/proxmox-boot-tool             | 68 +++++++++++++++++++---------
 debian/pve-kernel-helper.postinst |  4 ++
 proxmox-boot/functions            |  2 +-
 proxmox-boot/zz-proxmox-boot      | 73 +++++--------------------------
 5 files changed, 64 insertions(+), 102 deletions(-)

-- 
2.30.2

[pve-devel] [RFC pve-kernel-meta 1/5] rename pve-efiboot-manual-kernels to proxmox-boot-manual-kernels

[Stoiko Ivanov]

was forgotten during the general renaming of pve-efiboot ->
proxmox-boot.

follows commit 8c0a22adfe15dc00cf2194647bb254201d8d187b

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 debian/pve-kernel-helper.postinst | 4 ++++
 proxmox-boot/functions            | 2 +-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/debian/pve-kernel-helper.postinst b/debian/pve-kernel-helper.postinst
index 891396c..4c78766 100644
--- a/debian/pve-kernel-helper.postinst
+++ b/debian/pve-kernel-helper.postinst
@@ -8,6 +8,10 @@ case "$1" in
             echo "$0: legacy ESP list /etc/kernel/pve-efiboot-uuids found moving to /etc/kernel/proxmox-boot-uuids" 1>&2
             mv /etc/kernel/pve-efiboot-uuids /etc/kernel/proxmox-boot-uuids
         fi
+        if [ -e /etc/kernel/pve-efiboot-manual-kernels ]; then
+            echo "$0: legacy manual kernel list /etc/kernel/pve-efiboot-manual-kernels found moving to /etc/kernel/proxmox-boot-manual-kernels" 1>&2
+            mv /etc/kernel/pve-efiboot-manual-kernels /etc/kernel/proxmox-boot-manual-kernels
+        fi
         ;;
 esac
 
diff --git a/proxmox-boot/functions b/proxmox-boot/functions
index 3bea421..869345a 100755
--- a/proxmox-boot/functions
+++ b/proxmox-boot/functions
@@ -4,7 +4,7 @@ set -e
 ESP_LIST="/etc/kernel/proxmox-boot-uuids"
 ESPTYPE='c12a7328-f81f-11d2-ba4b-00a0c93ec93b'
 
-MANUAL_KERNEL_LIST="/etc/kernel/pve-efiboot-manual-kernels"
+MANUAL_KERNEL_LIST="/etc/kernel/proxmox-boot-boot-manual-kernels"
 PINNED_KERNEL_CONF="/etc/kernel/proxmox-boot-pin"
 PMX_NEXT_BOOT_FILE="/etc/kernel/next-boot-active"
 
-- 
2.30.2

[pve-devel] [RFC pve-kernel-meta 2/5] proxmox-boot: add reinit subcommand

[Stoiko Ivanov]

to iterate over all configured ESPs and refresh the boot-loader
installations.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 bin/grub-install-wrapper | 19 ++-----------------
 bin/proxmox-boot-tool    | 39 +++++++++++++++++++++++++++++++++++----
 2 files changed, 37 insertions(+), 21 deletions(-)

diff --git a/bin/grub-install-wrapper b/bin/grub-install-wrapper
index 9facd04..90c144e 100755
--- a/bin/grub-install-wrapper
+++ b/bin/grub-install-wrapper
@@ -2,22 +2,6 @@
 
 set -e
 
-. /usr/share/pve-kernel-helper/scripts/functions
-
-init_boot_disks() {
-    if ! (echo "${curr_uuid}" | grep -qE '[0-9a-fA-F]{4}-[0-9a-fA-F]{4}'); then
-        warn "WARN: ${curr_uuid} read from ${ESP_LIST} does not look like a VFAT-UUID - skipping"
-        return
-    fi
-
-    path="/dev/disk/by-uuid/$curr_uuid"
-    if [ ! -e "${path}" ]; then
-        warn "WARN: ${path} does not exist - clean '${ESP_LIST}'! - skipping"
-        return
-    fi
-    proxmox-boot-tool init "$path"
-}
-
 if proxmox-boot-tool status --quiet; then
     # detect when being called by dpkg (e.g. grub-pc.postinst
     if [ -n "$DPKG_RUNNING_VERSION" ] && echo "$DPKG_MAINTSCRIPT_PACKAGE" | grep -sq "^grub-"; then
@@ -28,7 +12,8 @@ if proxmox-boot-tool status --quiet; then
             MARKER_FILE="/tmp/proxmox-boot-tool.dpkg.marker"
             if [ ! -e "$MARKER_FILE" ]; then
                 warn "This system is booted via proxmox-boot-tool, running proxmox-boot-tool init for all configured bootdisks"
-                loop_esp_list init_boot_disks
+                proxmox-boot-tool reinit
+                proxmox-boot-tool refresh
                 touch "$MARKER_FILE"
                 exit 0
             else
diff --git a/bin/proxmox-boot-tool b/bin/proxmox-boot-tool
index 90ebfb2..af473a5 100755
--- a/bin/proxmox-boot-tool
+++ b/bin/proxmox-boot-tool
@@ -124,7 +124,7 @@ format() {
 	exit 0
 }
 
-init() {
+init_bootloader() {
 	part="$1"
 
 	_get_partition_info "$part"
@@ -174,8 +174,20 @@ init() {
 	echo "Adding '$part' to list of synced ESPs.."
 	_add_entry_to_list_file "$ESP_LIST" "$UUID"
 
-	echo "Refreshing kernels and initrds.."
-	refresh
+}
+
+reinit() {
+	if ! (echo "${curr_uuid}" | grep -qE '[0-9a-fA-F]{4}-[0-9a-fA-F]{4}'); then
+		warn "WARN: ${curr_uuid} read from ${ESP_LIST} does not look like a VFAT-UUID - skipping"
+		return
+	fi
+
+	path="/dev/disk/by-uuid/$curr_uuid"
+	if [ ! -e "${path}" ]; then
+		warn "WARN: ${path} does not exist - clean '${ESP_LIST}'! - skipping"
+		return
+	fi
+	init_bootloader "$path"
 }
 
 _clean_impl() {
@@ -300,6 +312,7 @@ usage() {
 	warn ""
 	warn "       $0 format <partition> [--force]"
 	warn "       $0 init <partition>"
+	warn "       $0 reinit"
 	warn "       $0 clean [--dry-run]"
 	warn "       $0 refresh [--hook <name>]"
 	warn "       $0 kernel <add|remove|pin|next-boot> <kernel-version>"
@@ -317,6 +330,10 @@ help() {
 	echo ""
 	echo "    initialize EFI system partition at <partition> for automatic synchronization of pve-kernels and their associated initrds."
 	echo ""
+	echo "USAGE: $0 reinit"
+	echo ""
+	echo "    reinitialize all configured EFI system partitions from $ESP_LIST."
+	echo ""
 	echo "USAGE: $0 clean [--dry-run]"
 	echo ""
 	echo "    remove no longer existing EFI system partition UUIDs from $ESP_LIST. Use --dry-run to only print outdated entries instead of removing them."
@@ -462,7 +479,21 @@ case "$1" in
 			usage
 			exit 1
 		fi
-		init "$@"
+		init_bootloader "$@"
+		echo "Refreshing kernels and initrds.."
+		refresh
+		exit 0
+	;;
+	'reinit')
+		reexec_in_mountns "$@"
+		shift
+		if [ "$#" -eq 1 ]; then
+			warn "E: no arguments allowed."
+			warn ""
+			usage
+			exit 1
+		fi
+                loop_esp_list reinit "$@"
 		exit 0
 	;;
 	'clean')
-- 
2.30.2

[pve-devel] [RFC pve-kernel-meta 3/5] proxmox-boot: keep EFI and legacy bootloaders in sync

[Stoiko Ivanov]

The current boot-scheme with grub for legacy boot and systemd-boot for
UEFI makes it quite cumbersome to switch from one boot mode to the
other (by booting a live-cd, chrooting into the system with the
new boot-mode and running p-b-t init for the configured ESPs).
Additionally it causes old, unused kernel+initrd versions to take up
space on the ESPs (if a switch was performed - the old boot-mode files
are never cleaned up)

Sadly FAT does not support symlinks so it is not possible to copy the
kernel+image once and symlink it for both boot-loaders.
However systemd-boot does not need to have the files in a
vendor/version specific subdirectory (although this is against the
recommendations of the boot-loader-specification [0]) - so we can
simply point the entries to boot the kernel+initrd image from the ESPs
root (where grub needs them).

The UEFI spec on the directory structure [1], also should be fine with
it.

With the following patch p-b-t simply installs both boot-loaders in
parallel - if booted in legacy mode bootctl cannot update the efivars
accordingly - but bootctl installs systemd-boot as default boot-loader
($esp_root/EFI/BOOT/BOOTX64.EFI) in any case - so it should at least
be reachable from most UEFI menus.

Quickly tested on two VMs (one with UEFI one with legacy) - and
switching their BIOS setting did not change their ability to boot.

[0] https://systemd.io/BOOT_LOADER_SPECIFICATION/
[1] section 13.3.1.3 of
https://uefi.org/sites/default/files/resources/UEFI_Spec_2_9_2021_03_18.pdf

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 bin/proxmox-boot-tool        | 38 ++++++++--------
 proxmox-boot/zz-proxmox-boot | 88 ++++++++++--------------------------
 2 files changed, 45 insertions(+), 81 deletions(-)

diff --git a/bin/proxmox-boot-tool b/bin/proxmox-boot-tool
index af473a5..5397694 100755
--- a/bin/proxmox-boot-tool
+++ b/bin/proxmox-boot-tool
@@ -150,24 +150,26 @@ init_bootloader() {
 	echo "Mounting '$part' on '$esp_mp'."
 	mount -t vfat "$part" "$esp_mp"
 
-	if [ -d /sys/firmware/efi ]; then
-		echo "Installing systemd-boot.."
-		mkdir -p "$esp_mp/$PMX_ESP_DIR"
-		bootctl --path "$esp_mp" install
-
-		echo "Configuring systemd-boot.."
-		echo "timeout 3" > "$esp_mp/$PMX_LOADER_CONF.tmp"
-		echo "default proxmox-*" >> "$esp_mp/$PMX_LOADER_CONF.tmp"
-		mv "$esp_mp/$PMX_LOADER_CONF.tmp" "$esp_mp/$PMX_LOADER_CONF"
-	else
-		echo "Installing grub i386-pc target.."
-		grub-install.real \
-			--boot-directory $esp_mp \
-			--target i386-pc \
-			--no-floppy \
-			--bootloader-id='proxmox' \
-			"/dev/$PKNAME"
-	fi
+	echo "Installing systemd-boot.."
+	mkdir -p "$esp_mp/$PMX_ESP_DIR"
+	bootctl_args="--path ${esp_mp}"
+	if [ ! -d /sys/firmware/efi ]; then
+		bootctl_args="${bootctl_args} --no-variables"
+	fi
+
+	bootctl ${bootctl_args} install
+	echo "Configuring systemd-boot.."
+	echo "timeout 3" > "$esp_mp/$PMX_LOADER_CONF.tmp"
+	echo "default proxmox-*" >> "$esp_mp/$PMX_LOADER_CONF.tmp"
+	mv "$esp_mp/$PMX_LOADER_CONF.tmp" "$esp_mp/$PMX_LOADER_CONF"
+
+	echo "Installing grub i386-pc target.."
+	grub-install.real \
+		--boot-directory $esp_mp \
+		--target i386-pc \
+		--no-floppy \
+		--bootloader-id='proxmox' \
+		"/dev/$PKNAME"
 	echo "Unmounting '$part'."
 	umount "$part"
 
diff --git a/proxmox-boot/zz-proxmox-boot b/proxmox-boot/zz-proxmox-boot
index 27448b2..5609af2 100755
--- a/proxmox-boot/zz-proxmox-boot
+++ b/proxmox-boot/zz-proxmox-boot
@@ -75,16 +75,11 @@ update_esp_func() {
 		{ warn "creation of mountpoint ${mountpoint} failed - skipping"; return; }
 	mount "${path}" "${mountpoint}" || \
 		{ warn "mount of ${path} failed - skipping"; return; }
-	if [ -d /sys/firmware/efi ]; then
-		if [ ! -f "${mountpoint}/$PMX_LOADER_CONF" ]; then
-			warn "${path} contains no loader.conf - skipping"
-			return
-		fi
-		if [ ! -d "${mountpoint}/$PMX_ESP_DIR" ]; then
-			warn "${path}/$PMX_ESP_DIR does not exist- skipping"
-			return
-		fi
-	elif [ ! -d "${mountpoint}/grub" ]; then
+	if [ ! -f "${mountpoint}/$PMX_LOADER_CONF" ]; then
+		warn "${path} contains no loader.conf - skipping"
+		return
+	fi
+	if [ ! -d "${mountpoint}/grub" ]; then
 		warn "${path} contains no grub directory - skipping"
 		return
 	fi
@@ -94,17 +89,12 @@ update_esp_func() {
 	if [ -e "${PINNED_KERNEL_CONF}" ]; then
 	    pinned_kernel=$(get_first_line "${PINNED_KERNEL_CONF}")
 	fi
-	if [ -d /sys/firmware/efi ]; then
-		set_systemd_boot_default "${mountpoint}" "${pinned_kernel}"
-		remove_old_kernels_efi "${mountpoint}"
-	else
-		set_grub_default "${pinned_kernel}"
-		remove_old_kernels_legacy "${mountpoint}"
-		mount --bind "${mountpoint}" "/boot"
-		update-grub
-		umount /boot
-
-	fi
+	set_systemd_boot_default "${mountpoint}" "${pinned_kernel}"
+	set_grub_default "${pinned_kernel}"
+	remove_old_kernels "${mountpoint}"
+	mount --bind "${mountpoint}" "/boot"
+	update-grub
+	umount /boot
 
 	umount "${mountpoint}" || \
 		{ warn "umount of ${path} failed - failure"; exit 0; }
@@ -130,52 +120,23 @@ copy_and_config_kernels() {
 			continue
 		fi
 
-		if [ -d /sys/firmware/efi ]; then
-
-			warn "	Copying kernel and creating boot-entry for ${kver}"
-			KERNEL_ESP_DIR="${PMX_ESP_DIR}/${kver}"
-			KERNEL_LIVE_DIR="${esp}/${KERNEL_ESP_DIR}"
-			mkdir -p "${KERNEL_LIVE_DIR}"
-			cp --preserve=timestamps "${linux_image}" "${KERNEL_LIVE_DIR}/"
-			cp --preserve=timestamps "${initrd}" "${KERNEL_LIVE_DIR}/"
-
-			# create loader entry
-			cat > "${esp}/loader/entries/proxmox-${kver}.conf" <<- EOF
-				title    ${LOADER_TITLE}
-				version  ${kver}
-				options  ${CMDLINE}
-				linux    /${KERNEL_ESP_DIR}/vmlinuz-${kver}
-				initrd   /${KERNEL_ESP_DIR}/initrd.img-${kver}
-			EOF
-		else
-			warn "	Copying kernel ${kver}"
-			cp --preserve=timestamps "${linux_image}" "${esp}/"
-			cp --preserve=timestamps "${initrd}" "${esp}/"
-		fi
-	done
-
-}
-
-remove_old_kernels_efi() {
-	esp="$1"
-
-	for kerneldir in "${esp}/${PMX_ESP_DIR}"/*; do
-		if [ ! -d "${kerneldir}" ]; then
-			warn "	${kerneldir} is not a directory - skipping"
-			continue
-		fi
-
-		kver="$(echo "${kerneldir}" | sed -r "s#^${esp}/${PMX_ESP_DIR}/(.+)\$#\\1#")"
-
-		echo "${BOOT_KVERS}" | grep -q "${kver}" && continue;
-		warn "	Removing old version ${kver}"
-		rm -rf "${kerneldir}"
-		rm -f "${esp}/loader/entries/proxmox-${kver}.conf"
+		warn "	Copying kernel and creating boot-entry for ${kver}"
+		cp --preserve=timestamps "${linux_image}" "${esp}/"
+		cp --preserve=timestamps "${initrd}" "${esp}/"
+
+		# create loader entry
+		cat > "${esp}/loader/entries/proxmox-${kver}.conf" <<- EOF
+			title    ${LOADER_TITLE}
+			version  ${kver}
+			options  ${CMDLINE}
+			linux    /vmlinuz-${kver}
+			initrd   /initrd.img-${kver}
+		EOF
 	done
 
 }
 
-remove_old_kernels_legacy() {
+remove_old_kernels() {
 	esp="$1"
 
 	for kernel in "${esp}/"vmlinuz-*; do
@@ -185,6 +146,7 @@ remove_old_kernels_legacy() {
 		warn "	Removing old version ${kver}"
 		rm -rf "${esp}/vmlinuz-${kver}"
 		rm -rf "${esp}/initrd.img-${kver}"
+		rm -f "${esp}/loader/entries/proxmox-${kver}.conf"
 	done
 
 }
-- 
2.30.2

[pve-devel] [RFC pve-kernel-meta 4/5] proxmox-boot: use grub for UEFI boot

[Stoiko Ivanov]

originally systemd-boot was chosen for UEFI installs because of it's
less invasive and economical configuration (add a few short files,
instead of having update-grub run through /dev a few times identifying
potentially interesting devices)

However with the change to proxmox-boot-tool and also handling
non-uefi installations we can deal with grub installed on an ESP
anyways.

This changeset changes proxmox-boot-tool to use grub for both
boot-modes, unifying the experience (e.g. not needing to check the
boot mode in order to set kernel commandline arguments)

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 bin/proxmox-boot-tool        | 23 ++++++++---------------
 proxmox-boot/zz-proxmox-boot | 15 ---------------
 2 files changed, 8 insertions(+), 30 deletions(-)

diff --git a/bin/proxmox-boot-tool b/bin/proxmox-boot-tool
index 5397694..d8caa6b 100755
--- a/bin/proxmox-boot-tool
+++ b/bin/proxmox-boot-tool
@@ -150,26 +150,19 @@ init_bootloader() {
 	echo "Mounting '$part' on '$esp_mp'."
 	mount -t vfat "$part" "$esp_mp"
 
-	echo "Installing systemd-boot.."
-	mkdir -p "$esp_mp/$PMX_ESP_DIR"
-	bootctl_args="--path ${esp_mp}"
+	echo "Installing grub x86_64 target.."
+	grub_common_args="--boot-directory ${esp_mp} --no-floppy --bootloader-id=proxmox"
+	grub_efi_args="$grub_common_args --target=x86_64-efi --efi-directory=$esp_mp"
 	if [ ! -d /sys/firmware/efi ]; then
-		bootctl_args="${bootctl_args} --no-variables"
+		grub_efi_args="$grub_efi_args --no-nvram"
 	fi
 
-	bootctl ${bootctl_args} install
-	echo "Configuring systemd-boot.."
-	echo "timeout 3" > "$esp_mp/$PMX_LOADER_CONF.tmp"
-	echo "default proxmox-*" >> "$esp_mp/$PMX_LOADER_CONF.tmp"
-	mv "$esp_mp/$PMX_LOADER_CONF.tmp" "$esp_mp/$PMX_LOADER_CONF"
+	grub-install.real ${grub_efi_args} "/dev/$PKNAME"
 
 	echo "Installing grub i386-pc target.."
-	grub-install.real \
-		--boot-directory $esp_mp \
-		--target i386-pc \
-		--no-floppy \
-		--bootloader-id='proxmox' \
-		"/dev/$PKNAME"
+	grub_legacy_args="$grub_common_args --target=i386-pc"
+	grub-install.real ${grub_legacy_args} "/dev/$PKNAME"
+
 	echo "Unmounting '$part'."
 	umount "$part"
 
diff --git a/proxmox-boot/zz-proxmox-boot b/proxmox-boot/zz-proxmox-boot
index 5609af2..60948ac 100755
--- a/proxmox-boot/zz-proxmox-boot
+++ b/proxmox-boot/zz-proxmox-boot
@@ -75,10 +75,6 @@ update_esp_func() {
 		{ warn "creation of mountpoint ${mountpoint} failed - skipping"; return; }
 	mount "${path}" "${mountpoint}" || \
 		{ warn "mount of ${path} failed - skipping"; return; }
-	if [ ! -f "${mountpoint}/$PMX_LOADER_CONF" ]; then
-		warn "${path} contains no loader.conf - skipping"
-		return
-	fi
 	if [ ! -d "${mountpoint}/grub" ]; then
 		warn "${path} contains no grub directory - skipping"
 		return
@@ -89,7 +85,6 @@ update_esp_func() {
 	if [ -e "${PINNED_KERNEL_CONF}" ]; then
 	    pinned_kernel=$(get_first_line "${PINNED_KERNEL_CONF}")
 	fi
-	set_systemd_boot_default "${mountpoint}" "${pinned_kernel}"
 	set_grub_default "${pinned_kernel}"
 	remove_old_kernels "${mountpoint}"
 	mount --bind "${mountpoint}" "/boot"
@@ -123,15 +118,6 @@ copy_and_config_kernels() {
 		warn "	Copying kernel and creating boot-entry for ${kver}"
 		cp --preserve=timestamps "${linux_image}" "${esp}/"
 		cp --preserve=timestamps "${initrd}" "${esp}/"
-
-		# create loader entry
-		cat > "${esp}/loader/entries/proxmox-${kver}.conf" <<- EOF
-			title    ${LOADER_TITLE}
-			version  ${kver}
-			options  ${CMDLINE}
-			linux    /vmlinuz-${kver}
-			initrd   /initrd.img-${kver}
-		EOF
 	done
 
 }
@@ -146,7 +132,6 @@ remove_old_kernels() {
 		warn "	Removing old version ${kver}"
 		rm -rf "${esp}/vmlinuz-${kver}"
 		rm -rf "${esp}/initrd.img-${kver}"
-		rm -f "${esp}/loader/entries/proxmox-${kver}.conf"
 	done
 
 }
-- 
2.30.2

[pve-devel] [RFC pve-kernel-meta 5/5] proxmox-boot: install grub in esp/EFI/BOOT/BOOTX64.EFI

[Stoiko Ivanov]

by passing the --removable option

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 bin/proxmox-boot-tool | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bin/proxmox-boot-tool b/bin/proxmox-boot-tool
index d8caa6b..72f77ed 100755
--- a/bin/proxmox-boot-tool
+++ b/bin/proxmox-boot-tool
@@ -152,7 +152,7 @@ init_bootloader() {
 
 	echo "Installing grub x86_64 target.."
 	grub_common_args="--boot-directory ${esp_mp} --no-floppy --bootloader-id=proxmox"
-	grub_efi_args="$grub_common_args --target=x86_64-efi --efi-directory=$esp_mp"
+	grub_efi_args="$grub_common_args --target=x86_64-efi --efi-directory=$esp_mp --removable"
 	if [ ! -d /sys/firmware/efi ]; then
 		grub_efi_args="$grub_efi_args --no-nvram"
 	fi
-- 
2.30.2

Re: [pve-devel] [RFC pve-kernel-meta 0/5] unify boot-mode config

[Thomas Lamprecht]

On 01.02.22 23:03, Stoiko Ivanov wrote:
> patch 3 drops systemd-boot and uses grub for both boot-modes, hopefully
> unifying the boot-experience and causing less confusion (currently I suggest
> to look at the screen while booting to find out which boot-loader is used)
> 
> (Sadly systemd-boot (which I would prefer, justifiably)
> won't get support for legacy boot)
> 
The thing is, non-uefi systems will become more rare anyhow, so why
bother with that? The simplicity of systemd-boot is worth the few (?)
confusion - I mean what exactly is there confusing anyway, if most relevant
actions can be handled through our tool anyway?

I'm not definitive yet, but currently rather tending to NACK that.

Re: [pve-devel] [RFC pve-kernel-meta 0/5] unify boot-mode config

[Stoiko Ivanov]

On Wed, 2 Feb 2022 10:03:05 +0100
Thomas Lamprecht <t.lamprecht@proxmox.com> wrote:

> On 01.02.22 23:03, Stoiko Ivanov wrote:
> > patch 3 drops systemd-boot and uses grub for both boot-modes, hopefully
> > unifying the boot-experience and causing less confusion (currently I suggest
> > to look at the screen while booting to find out which boot-loader is used)
> > 
> > (Sadly systemd-boot (which I would prefer, justifiably)
> > won't get support for legacy boot)
> >   
> The thing is, non-uefi systems will become more rare anyhow, so why
> bother with that? The simplicity of systemd-boot is worth the few (?)
> confusion - I mean what exactly is there confusing anyway, if most relevant
> actions can be handled through our tool anyway?
> 
> I'm not definitive yet, but currently rather tending to NACK that.
Thanks for the feedback!

Hmm - I do see your point - motivation was that it felt like a logical
next step after adapting the config of systemd-boot to get the images
from where grub needs them to have the system bootable in both modes -
and having a single place to configure the boot-loader (kernel cmdline,
pinning) seemed sensible (and less code in p-b-t should correlate with
less bugs in p-b-t).

Add to that my biased view (a few forum-threads where people did not know
which boot loader they used - e.g. [0,1,2] vs. the silent majority, who
either does not need it, or knows it) that most users expect
/etc/default/grub to be the place for editing the kernel commandline
(following the blog/forum/website posts only mentioning this)

But OTOH I was a bit hesitant as well (since it would mean that the
blog/forum/website posts of the past 2 years would now become 'wrong' and
cause even more confusion). Also (quite biased as well) - there are quite
a few threads with grub failing to boot vs. none that I'm aware of, where
systemd-boot fails.

So - no hard feelings from my side either - I was curios if it would work
as a POC

On a side-note - I just learned that grub in efi-mode works fine (without
explicit configuration) over a serial terminal (a use-case I need for
myself ;)


[0]
https://forum.proxmox.com/threads/amd-core-display-pass-through-install-win10-does-not-start.103651/post-447007
[1]
https://forum.proxmox.com/threads/unable-to-boot-after-pve-efiboot-tool-refresh.103149/
[2]
https://forum.proxmox.com/threads/no-iommu-detected-please-activate-it.102119/

Re: [pve-devel] [RFC pve-kernel-meta 0/5] unify boot-mode config

[Thomas Lamprecht]

On 02.02.22 15:28, Stoiko Ivanov wrote:
> On Wed, 2 Feb 2022 10:03:05 +0100
> Thomas Lamprecht <t.lamprecht@proxmox.com> wrote:
> 
>> On 01.02.22 23:03, Stoiko Ivanov wrote:
>>> patch 3 drops systemd-boot and uses grub for both boot-modes, hopefully
>>> unifying the boot-experience and causing less confusion (currently I suggest
>>> to look at the screen while booting to find out which boot-loader is used)
>>>
>>> (Sadly systemd-boot (which I would prefer, justifiably)
>>> won't get support for legacy boot)
>>>   
>> The thing is, non-uefi systems will become more rare anyhow, so why
>> bother with that? The simplicity of systemd-boot is worth the few (?)
>> confusion - I mean what exactly is there confusing anyway, if most relevant
>> actions can be handled through our tool anyway?
>>
>> I'm not definitive yet, but currently rather tending to NACK that.
> Thanks for the feedback!
> 
> Hmm - I do see your point - motivation was that it felt like a logical
> next step after adapting the config of systemd-boot to get the images
> from where grub needs them to have the system bootable in both modes -
> and having a single place to configure the boot-loader (kernel cmdline,
> pinning) seemed sensible (and less code in p-b-t should correlate with
> less bugs in p-b-t).

No, IMO the next logical step is to rather use systemd-boot for all
EFI-booted setups, not only ZFS - makes not much sense to have switched
explicitly to systemd-boot only to switch back again, grub has a real
complexity cost that only can be argued for in non-UEFI systems.

Also, we already have the code and it already works well for all setups we
know of, further the scope of proxmox-boot-tool is rather small anyway, with
that in mind I'd really hope not to expect much bugs from that part, nor
would I think that there'll be much dropped - efi and legacy boot is
different enough anyway.

> 
> Add to that my biased view (a few forum-threads where people did not know
> which boot loader they used - e.g. [0,1,2] vs. the silent majority, who
> either does not need it, or knows it) that most users expect
> /etc/default/grub to be the place for editing the kernel commandline
> (following the blog/forum/website posts only mentioning this)

Yeah I do not see that in a relevant amount of times, iff it's a documentation
problem, and if you want to make it more simpler the answer is a p-b-t command
that allows to set and reset it centrally. Some people will always ask or have
trouble either way.

> 
> But OTOH I was a bit hesitant as well (since it would mean that the
> blog/forum/website posts of the past 2 years would now become 'wrong' and
> cause even more confusion). Also (quite biased as well) - there are quite
> a few threads with grub failing to boot vs. none that I'm aware of, where
> systemd-boot fails.

Yeah switching around will definitively produce much more confusion than it
solves, there are no technical advantages either.

> 
> So - no hard feelings from my side either - I was curios if it would work
> as a POC
> 

I mean, why wouldn't it? ^^

> On a side-note - I just learned that grub in efi-mode works fine (without
> explicit configuration) over a serial terminal (a use-case I need for
> myself ;)

systemd's does too here, at least qm terminal worked with both since I can
remember.