Re: [PATCH] kho: fix global scratch size calculation

[Pratyush Yadav]

On Sat, Oct 17 2026, BernardoGV@xxxxxxxxx wrote:

> On Mon, Oct 12, 2026, Mike Rapoport wrote:
>> I mean more fundamental code refactoring that takes into the account all
>> cases (and I could miss some)
>>
>> - NUMA vs no-NUMA
>> - systems that have global scratch containing lowmem scratch vs those that
>>   don't
>> - and of course fix global scratch over-allocation
>
> One more case for the list: reservations made before KHO sizes scratch
> that are never served from it. HugeTLB is already excluded. crashkernel
> is the other big one. Sourabh mentioned it in the powerpc thread. Here
> are x86 numbers from a machine that never asked for KHO.
>
> Ubuntu 26.04's generic kernel (7.0.0-38-generic) ships
> CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT=y, so scratch is reserved on every
> boot. (26.10 has since turned the default off. I've asked for the same
> in 26.04 [1].) The machine is a single-node Ryzen AI Max+ 395 with
> 128 GiB and no hugetlb. kdump is set up the stock Ubuntu way, which gives
> crashkernel 2048M high + 256M low.
>
> Three cold boots of the same kernel, KHO on by default:
>
>   kho/out/scratch_len   crashkernel   no crashkernel   delta
>   lowmem                   760 MiB         248 MiB      -512 = 2 x 256
>   global                 8,510 MiB       4,414 MiB    -4,096 = 2 x 2,048
>   per-node                4,338 MiB       4,338 MiB         0
>   total                 13,608 MiB       9,000 MiB    -4,608 = 2 x 2,304
>
> "Memory: ... reserved" goes from 18,849,832K to 11,771,924K. The
> difference, 6,912 MiB, is exactly the crashkernel plus 2x crashkernel of
> scratch. With kho=off it is 4,915,232K.
>
> Both problems show up here. Without crashkernel, lowmem + global is
> 2 x 2,331 MiB and per-node is 2 x 2,169 MiB, so on a single node about
> 2,169 MiB of reservations get counted twice. That is the over-allocation,
> and the total comes to roughly 4x what was reserved. crashkernel is
> allocated with NUMA_NO_NODE, so it lands only in lowmem/global, at 2x.
> If I read Sourabh's patch right, it would count NUMA_NO_NODE only for
> global. On this machine that leaves crashkernel as nearly all of the
> global scratch (about 2 x (162 + 2,304) MiB).

To remove crashkernel from scratch calculation, you'd have to allocate
it from outside of scratch, like the hugetlb pages. Crashkernel is
usually allocated earlier in boot, so we'd have to call
kho_extend_scratch() before crashkernel allocations, and I am not sure
if something will break if we do so. I suspect we at least need to check
efi_init().

Not saying it can't be done, but it isn't trivial.

>
> The cost is not only boot memory. Scratch ends up MIGRATE_CMA, so
> anything that pins can't use it. Here that is amdgpu GTT on an APU. A
> single process tops out at 107 GiB of a 120 GiB GTT limit with KHO on,
> and at 117 GiB with kho=off.

Well, my pipe dream is to have KHO entirely without scratch. You would
use the same mechanism that kho_extend_scratch() uses, to discover all
usable early boot memory, and not pass over any scratch at all (or, a
very small amount of scratch). The only problem is that it might fall
prey to fragmentation. If preserved regions are spread apart enough, you
might end up with _no_ memory to boot from. I am not sure how to make
this idea reliable yet.

>
> Happy to test patches on this box.
>
> [1] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2171297
>
>

-- 
Regards,
Pratyush Yadav