Re: [RFC PATCH 0/5] KHO map

From: Pratyush Yadav

Date: Tue Nov 17 2026 - 07:19:47 EST


Hi Chris,

On Tue, Nov 17 2026, Chris Down wrote:

> Hi Pratyush,
>
> Pratyush Yadav writes:
>>The main user of the map is memfd. Currently memfd uses a vmalloc buffer
>>that has a pair of {index,pfn} to track the folios in the file. That
>>uses 16 bytes per page. The KHO map uses only 8 bytes per page with some
>>overhead. So for a 1 GiB memfd, it uses roughly half the memory.
>
> That only holds for order-0 folios though. A leaf covers 512 indices, so
> a populated memfd needs one leaf page for every 2M of file whatever the
> folio size, while the array needs 16 bytes per folio. Per 2M of file,
> that works out as:
>
> folio size array map map/array
> 4K (o0) 8K 4K 0.5x
> 8K (o1) 4K 4K 1x
> 16K (o2) 2K 4K 2x
> 64K (o4) 512 4K 8x
> 2M (o9) 16 4K 256x

Right.

>
>>- Multi-index entry support: currently, while the map is very efficient
>> with 4k folios, it is horribly inefficient with 2M folios. Each folio
>> in the memfd takes up a full page to track. Multi-index entry support
>> can fix this by tracking entries that cover a whole page in the
>> previous level, similar to page tables.
>
> Since patch 5 makes things worse for memfds with large folios, I agree
> with Pasha that this needs fixing before memfd switches over. Storing

Right, I agree as well.

> PMD-sized entries at L1 fixes the 2M case, but mTHP sizes between 4K and
> 2M would still take a full leaf for every 2M of file. Do you expect
> mTHP-backed memfds to be preserved? If so, it's probably worth handling
> them in the format before it becomes ABI.

mTHP will still work, it will just be less space efficient than 2M THPs.
Even then, the overhead is around 2M for every 1G, which is roughly
0.2%. Even then, I don't think that is too bad. If you want to be more
space efficient, use 2M THPs, or hugetlb[*]. I think that is a fair
tradeoff. Using larger page sizes also makes KHO more efficient.

We _can_ make mTHPs fairly efficient, but that comes at the cost of a
more complicated ABI. For example, we can have a per-order KHO map in
the memfd, and shift the keys by the order. That should improve the
density. We could also do some other tricks like the shifted key
encoding scheme that KHO preserved memory map uses (see
kho_encode_radix_key()) to improve density.

So unless someone can explain why mTHP memfds are important to optimize
for, I'd say mTHP users can either take the 0.2% preservation overhead,
or use 2M THPs.

[*] I sent an RFC for that some time ago, but it's been blocked on
pending infrastructure rework like extended bootmem. I plan to send
out a refreshed version soon.

--
Regards,
Pratyush Yadav