Re: [PATCH] kho: fix global scratch size calculation

[BernardoGV]

On Mon, Oct 12, 2026, Mike Rapoport wrote:
> I mean more fundamental code refactoring that takes into the account all
> cases (and I could miss some)
>
> - NUMA vs no-NUMA
> - systems that have global scratch containing lowmem scratch vs those that
>   don't
> - and of course fix global scratch over-allocation

One more case for the list: reservations made before KHO sizes scratch
that are never served from it. HugeTLB is already excluded. crashkernel
is the other big one. Sourabh mentioned it in the powerpc thread. Here
are x86 numbers from a machine that never asked for KHO.

Ubuntu 26.04's generic kernel (7.0.0-38-generic) ships
CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT=y, so scratch is reserved on every
boot. (26.10 has since turned the default off. I've asked for the same
in 26.04 [1].) The machine is a single-node Ryzen AI Max+ 395 with
128 GiB and no hugetlb. kdump is set up the stock Ubuntu way, which gives
crashkernel 2048M high + 256M low.

Three cold boots of the same kernel, KHO on by default:

  kho/out/scratch_len   crashkernel   no crashkernel   delta
  lowmem                   760 MiB         248 MiB      -512 = 2 x 256
  global                 8,510 MiB       4,414 MiB    -4,096 = 2 x 2,048
  per-node                4,338 MiB       4,338 MiB         0
  total                 13,608 MiB       9,000 MiB    -4,608 = 2 x 2,304

"Memory: ... reserved" goes from 18,849,832K to 11,771,924K. The
difference, 6,912 MiB, is exactly the crashkernel plus 2x crashkernel of
scratch. With kho=off it is 4,915,232K.

Both problems show up here. Without crashkernel, lowmem + global is
2 x 2,331 MiB and per-node is 2 x 2,169 MiB, so on a single node about
2,169 MiB of reservations get counted twice. That is the over-allocation,
and the total comes to roughly 4x what was reserved. crashkernel is
allocated with NUMA_NO_NODE, so it lands only in lowmem/global, at 2x.
If I read Sourabh's patch right, it would count NUMA_NO_NODE only for
global. On this machine that leaves crashkernel as nearly all of the
global scratch (about 2 x (162 + 2,304) MiB).

The cost is not only boot memory. Scratch ends up MIGRATE_CMA, so
anything that pins can't use it. Here that is amdgpu GTT on an APU. A
single process tops out at 107 GiB of a 120 GiB GTT limit with KHO on,
and at 117 GiB with kho=off.

Happy to test patches on this box.

[1] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2171297