Re: Use higher-order pages in vmalloc

[Andy Lutomirski]

On Thu, Feb 22, 2018 at 1:36 PM, Michal Hocko <mhocko@xxxxxxxxxx> wrote:
> On Thu 22-02-18 04:22:54, Matthew Wilcox wrote:
>> On Thu, Feb 22, 2018 at 07:59:43AM +0100, Michal Hocko wrote:
>> > On Wed 21-02-18 09:01:29, Matthew Wilcox wrote:
>> > > Right.  It helps with fragmentation if we can keep higher-order
>> > > allocations together.
>> >
>> > Hmm, wouldn't it help if we made vmalloc pages migrateable instead? That
>> > would help the compaction and get us to a lower fragmentation longterm
>> > without playing tricks in the allocation path.
>>
>> I was wondering about that possibility.  If we want to migrate a page
>> then we have to shoot down the PTE across all CPUs, copy the data to the
>> new page, and insert the new PTE.  Copying 4kB doesn't take long; if you
>> have 12GB/s (current example on Wikipedia: dual-channel memory and one
>> DDR2-800 module per channel gives a theoretical bandwidth of 12.8GB/s)
>> then we should be able to copy a page in 666ns).  So there's no problem
>> holding a spinlock for it.
>>
>> But we can't handle a fault in vmalloc space today.  It's handled in
>> arch-specific code, see vmalloc_fault() in arch/x86/mm/fault.c
>> If we're going to do this, it'll have to be something arches opt into
>> because I'm not taking on the job of fixing every architecture!
>
> yes.

On x86, if you shoot down the PTE for the current stack, you're dead.
vmalloc_fault() might not even be called.  Instead we hit
do_double_fault(), and the manual warns extremely strongly against
trying to recover, and, in this case, I agree with the SDM.  If you
actually want this to work, there needs to be a special IPI broadcast
to the task in question (with appropriate synchronization) that calls
magic arch code that does the switcheroo.

Didn't someone (Christoph?) have a patch to teach the page allocator
to give high-order allocations if available and otherwise fall back to
low order?