Re: Handling NUMA page migration

[Mel Gorman]

On Tue, Jun 04, 2013 at 06:58:07AM -0500, Robin Holt wrote:
> > B) 1. allocate memory with alloc_pages()
> >    2. SetPageReserved()
> >    3. vm_mmap() to allocate a userspace mapping
> >    4. vm_insert_page()
> >    5. vm_flags |= (VM_DONTEXPAND | VM_DONTDUMP)
> >       (resulting flags are VM_MIXEDMAP | VM_DONTDUMP | VM_DONTEXPAND | 0xff)
> > 
> > At least the memory allocated like B) is affected by automatic NUMA page
> > migration. I'm not sure about A).
> > 
> > 1. How can I prevent automatic NUMA page migration on this memory?
> > 2. Can NUMA page migration also be handled on such kind of memory without
> >    preventing migration?
> > 

Page migration does not expect a PageReserved && PageLRU page. The only
reserved check that is made by migration is for the zero page and that
happens in the syscall path for move_pages() which is not used by either
compaction or automatic balancing.

At some point you must have a driver that is setting PageReserved on
anonymous pages that is later encountered by automatic numa balancing
during a NUMA hinting fault.  I expect this is an out-of-tree driver or
a custom kernel of some sort. Memory should be pinned by elevating the
reference count of the page, not setting PageReserved.

It's not particularly clear how you avoid hitting the same bug due to THP and
memory compaction to be honest but maybe your setup hits a steady state that
simply never hit the problem or it happens rarely and it was not identified.

-- 
Mel Gorman
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/