[PATCH] kernel config: don't automatically enable kexec handover (KHO)

[PATCH] kernel config: don't automatically enable kexec handover (KHO)

[Dominik Csapak]

CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT is enabled by default in the
upstream ubuntu kernel config, but has a (probably) unintended
side-effect: when KHO is automatically enabled, the kernel reserves
scratch memory for the kernel handoff and marks it with MIGRATE_CMA to
prevent non-movable allocations there.

This causes unusual stats in /proc/meminfo, for example:

CmaTotal:              0 kB
CmaFree:         4894720 kB

which looks nonsensical on the surface. It also reduces the available
memory for e.g. guests with PCI passthrough, as pinner memory can't live
in these areas.

Since we don't use kexec handover by default, and it can still be
enabled on the kernel commandline (kho=on), disable it by default to
increase the available pinnable memory for guests.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 debian/rules.d/config-common.opts | 1 +
 1 file changed, 1 insertion(+)

diff --git a/debian/rules.d/config-common.opts b/debian/rules.d/config-common.opts
index 64505b4..204bab4 100644
--- a/debian/rules.d/config-common.opts
+++ b/debian/rules.d/config-common.opts
@@ -68,3 +68,4 @@
 -d CONFIG_N_GSM
 -d UBSAN_BOUNDS
 -e CONFIG_RUST
+-d CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT
-- 
2.47.3

Re: [PATCH] kernel config: don't automatically enable kexec handover (KHO)

[Dominik Csapak]

just a note for this (maybe can be squashed into the commit message
on applying):

this does not disable the kexec handoff completely, it just
does not enable it by default, so users that would need it
can still enable it via 'kho=on' on the commandline.

conversely, users running into problem with the current configuration
can mitigate the issue by adding 'kho=off' to their commandline
to disable it.

On 7/14/26 2:44 PM, Dominik Csapak wrote:
> CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT is enabled by default in the
> upstream ubuntu kernel config, but has a (probably) unintended
> side-effect: when KHO is automatically enabled, the kernel reserves
> scratch memory for the kernel handoff and marks it with MIGRATE_CMA to
> prevent non-movable allocations there.
> 
> This causes unusual stats in /proc/meminfo, for example:
> 
> CmaTotal:              0 kB
> CmaFree:         4894720 kB
> 
> which looks nonsensical on the surface. It also reduces the available
> memory for e.g. guests with PCI passthrough, as pinner memory can't live
> in these areas.
> 
> Since we don't use kexec handover by default, and it can still be
> enabled on the kernel commandline (kho=on), disable it by default to
> increase the available pinnable memory for guests.
> 
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
>   debian/rules.d/config-common.opts | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/debian/rules.d/config-common.opts b/debian/rules.d/config-common.opts
> index 64505b4..204bab4 100644
> --- a/debian/rules.d/config-common.opts
> +++ b/debian/rules.d/config-common.opts
> @@ -68,3 +68,4 @@
>   -d CONFIG_N_GSM
>   -d UBSAN_BOUNDS
>   -e CONFIG_RUST
> +-d CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT

Re: [PATCH] kernel config: don't automatically enable kexec handover (KHO)

[Thomas Lamprecht]

Am 14.07.26 um 14:44 schrieb Dominik Csapak:
> CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT is enabled by default in the
> upstream ubuntu kernel config, but has a (probably) unintended
> side-effect: when KHO is automatically enabled, the kernel reserves
> scratch memory for the kernel handoff and marks it with MIGRATE_CMA to
> prevent non-movable allocations there.
> 
> This causes unusual stats in /proc/meminfo, for example:
> 
> CmaTotal:              0 kB
> CmaFree:         -
> 
> which looks nonsensical on the surface. It also reduces the available
> memory for e.g. guests with PCI passthrough, as pinner memory can't live
> in these areas.
> 
> Since we don't use kexec handover by default, and it can still be
> enabled on the kernel commandline (kho=on), disable it by default to
> increase the available pinnable memory for guests.

We really should evaluate that though, IMO this [0] is the dream, and
if it works, it might be used more compared to pcie passthrough.

Also, is this actually reserved memory, or just mappings to have a
linear address space available, when needed?

[0]: https://lwn.net/Articles/1033364/

> 
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
>  debian/rules.d/config-common.opts | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/debian/rules.d/config-common.opts b/debian/rules.d/config-common.opts
> index 64505b4..204bab4 100644
> --- a/debian/rules.d/config-common.opts
> +++ b/debian/rules.d/config-common.opts
> @@ -68,3 +68,4 @@
>  -d CONFIG_N_GSM
>  -d UBSAN_BOUNDS
>  -e CONFIG_RUST
> +-d CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT

Re: [PATCH] kernel config: don't automatically enable kexec handover (KHO)

[Dominik Csapak]

On 7/15/26 12:25 PM, Thomas Lamprecht wrote:
> Am 14.07.26 um 14:44 schrieb Dominik Csapak:
>> CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT is enabled by default in the
>> upstream ubuntu kernel config, but has a (probably) unintended
>> side-effect: when KHO is automatically enabled, the kernel reserves
>> scratch memory for the kernel handoff and marks it with MIGRATE_CMA to
>> prevent non-movable allocations there.
>>
>> This causes unusual stats in /proc/meminfo, for example:
>>
>> CmaTotal:              0 kB
>> CmaFree:         -
>>
>> which looks nonsensical on the surface. It also reduces the available
>> memory for e.g. guests with PCI passthrough, as pinner memory can't live
>> in these areas.
>>
>> Since we don't use kexec handover by default, and it can still be
>> enabled on the kernel commandline (kho=on), disable it by default to
>> increase the available pinnable memory for guests.
> 
> We really should evaluate that though, IMO this [0] is the dream, and
> if it works, it might be used more compared to pcie passthrough.

Yes, this sounds nice, and we should definitely look into that.
Do you have any idea how far along that is? (is it already usable?)

> 
> Also, is this actually reserved memory, or just mappings to have a
> linear address space available, when needed?

Ok my wording was a bit imprecise: it is memory marked such that
anything that lives there can be moved away by the kernel.

So with respect to 'pinned' memory, this is essentially unusable,
which includes memory used for guests when using pci passthrough.
(probably other use cases too)

so it's not strictly 'reserved', but can't be used by some processes
at all.

hope that clears it up
> 
> [0]: https://lwn.net/Articles/1033364/
> 
>>
>> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
>> ---
>>   debian/rules.d/config-common.opts | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/debian/rules.d/config-common.opts b/debian/rules.d/config-common.opts
>> index 64505b4..204bab4 100644
>> --- a/debian/rules.d/config-common.opts
>> +++ b/debian/rules.d/config-common.opts
>> @@ -68,3 +68,4 @@
>>   -d CONFIG_N_GSM
>>   -d UBSAN_BOUNDS
>>   -e CONFIG_RUST
>> +-d CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT
>