Re: CXL Boot to Bash - Section 3: Memory (block) Hotplug
From: David Hildenbrand
Date: Tue Feb 18 2025 - 11:30:16 EST
---------------------
Step 9: Memory Zones.
---------------------
We've alluded to "Memory Zones" in prior sections, with really the only
detail about these concepts being that there are "Kernel-allocation
compatible" and "Movable" zones, as well as some relationship between
memory blocks and memory zones.
The two zones we really care about are `ZONE_NORMAL` and `ZONE_MOVABLE`.
For the purpose of this reading we'll consider two basic use-cases:
- memory block hot-unplug
- kernel resource allocation
You can (for the most part) consider these cases incompatible. If the
kernel allocates `struct page` memory from a block, then that block cannot
be hot-unplugged. This memory is typically unmovable (cannot be migrated),
and its pages unlikely to be removed from the memory map.
There are other scenarios, such as page pinning, that can block hot-unplug.
The individual mechanisms preventing hot-unplug are less important than
their relationship to memory zones.
ZONE_NORMAL basically allows any allocations, including things like page
tables, struct pages, and pinned memory.
ZONE_MOVABLE, under normal conditions, disallows most kernel allocations.
In essence, only movable allocations (some kernel allcoations are movable).
ZONE_MOVABLE does NOT make a *strong* guarantee of hut-unplug-ability.
The kernel and privileged users can cause long-term pinning to occur -
even in ZONE_MOVABLE. It should be seen as a best-attempt at providing
hot-unplug-ability under normal conditions.
Yes and no; actual long-term pinning is disallowed (FOLL_LONGTERM), but
we have a bunch of cases that need fixing. [1]
Of course, new cases keep popping up. It's a constant fight to make
hot-unplug as reliable as possible. So yes, we cannot give "strong"
guarantees, but make it as reliable as possible in sane configurations.
[1]
https://lkml.kernel.org/r/882b566c-34d6-4e68-9447-6c74a0693f18@xxxxxxxxxx
--
Cheers,
David / dhildenb