Re: [LSF/MM/BPF TOPIC] 64k (or 16k) base page size on x86

[David Hildenbrand (Arm)]

> > When discussing per-process page sizes with Ryan and Dev, I mentioned that
> > having a larger emulated page size could be interesting for other
> > architectures as well.
> >
> > That is, we would emulate a 64K page size on Intel for user space as well,
> > but let the OS work with 4K pages.
>
> Just to clarify, do you want it to be enforced on userspace ABI.
> Like, all mappings are 64k aligned?

Right, see the proposal from Dev on the list.

From user-space POV, the pagesize would be 64K for these emulated 
processes. That is, VMAs must be suitable aligned etc.

One key thing I think is that you could run such emulated-64k process 
(that actually support it!) with 4k processes on the same machine, like 
Arm is considering.

You would have no weird "vma crosses base pages" handling, which is just 
rather nasty and makes my head hurt.

> > We'd only allocate+map large folios into user space + pagecache, but still
> > allow for page tables etc. to not waste memory.
>
> Waste of memory for page table is solvable and pretty straight forward.
> Most of such cases can be solve mechanically by switching to slab.

Well, yes, like Willy says, there are already similar custom solutions 
for s390x and ppc.

Pasha talked recently about the memory waste of 16k kernel stacks and 
how we would want to reduce that to 4k. In your proposal, it would be 
64k, unless you somehow manage to allocate multiple kernel stacks from 
the same 64k page. My head hurts thinking about whether that could work, 
maybe it could (no idea about guard pages in there, though).


Let's take a look at the history of page size usage on Arm (people can 
feel free to correct me):

(1) Most distros were using 64k on Arm.

(2) People realized that 64k was suboptimal many use cases (memory
    waste for stacks, pagecache, etc) and started to switch to 4k. I
    remember that mostly HPC-centric users sticked to 64k, but there was
    also demand from others to be able to stay on 64k.

(3) Arm improved performance on a 4k kernel by adding cont-pte support,
    trying to get closer to 64k native performance.

(4) Achieving 64k native performance is hard, which is why per-process
    page sizes are being explored to get the best out of both worlds
    (use 64k page size only where it really matters for performance).

Arm clearly has the added benefit of actually benefiting from hardware 
support for 64k.

IIUC, what you are proposing feels a bit like traveling back in time 
when it comes to the memory waste problem that Arm users encountered.

Where do you see the big difference to 64k on Arm in your proposal? 
Would you currently also be running 64k Arm in production and the memory 
waste etc is acceptable?

--
Cheers,

David