Re: [PATCH] mem-hotplug: use GFP_HIGHUSER_MOVABLE and alloc from next node in alloc_migrate_target()

[David Rientjes]

On Thu, 14 Jul 2016, Xishi Qiu wrote:

> alloc_migrate_target() is called from migrate_pages(), and the page
> is always from user space, so we can add __GFP_HIGHMEM directly.
> 
> Second, when we offline a node, the new page should alloced from other
> nodes instead of the current node, because re-migrate is a waste of
> time.
> 

alloc_migrate_target() is not only used from memory hotplug, it is also 
used for CMA: we won't be isolating PageHuge() pages in 
isolate_migratepages_range(), so this would cause a regression where we'd 
be migrating memory to a remote NUMA node rather than preferring to 
allocate locally.

You may find it useful to use the 'private' field of the migrate_pages() 
callback to specify the node the page should preferably be migrated to.

> Signed-off-by: Xishi Qiu <qiuxishi@xxxxxxxxxx>
> ---
>  mm/page_isolation.c | 16 ++++++----------
>  1 file changed, 6 insertions(+), 10 deletions(-)
> 
> diff --git a/mm/page_isolation.c b/mm/page_isolation.c
> index 612122b..83848dc 100644
> --- a/mm/page_isolation.c
> +++ b/mm/page_isolation.c
> @@ -282,20 +282,16 @@ int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn,
>  struct page *alloc_migrate_target(struct page *page, unsigned long private,
>  				  int **resultp)
>  {
> -	gfp_t gfp_mask = GFP_USER | __GFP_MOVABLE;
> -
>  	/*
> -	 * TODO: allocate a destination hugepage from a nearest neighbor node,
> +	 * TODO: allocate a destination page from a nearest neighbor node,
>  	 * accordance with memory policy of the user process if possible. For
>  	 * now as a simple work-around, we use the next node for destination.
>  	 */
> +	int nid = next_node_in(page_to_nid(page), node_online_map);
> +
>  	if (PageHuge(page))
>  		return alloc_huge_page_node(page_hstate(compound_head(page)),
> -					    next_node_in(page_to_nid(page),
> -							 node_online_map));
> -
> -	if (PageHighMem(page))
> -		gfp_mask |= __GFP_HIGHMEM;
> -
> -	return alloc_page(gfp_mask);
> +						 nid);
> +	else
> +		return __alloc_pages_node(nid, GFP_HIGHUSER_MOVABLE, 0);

I don't think this __alloc_pages_node() does what you think it does, it 
only prefers nid here and will readily fallback to other nodes if 
necessary.  That is different than alloc_huge_page_node() which does no 
fallback.  So there's two issues with this change: (1) inconsistency 
between PageHuge() and !PageHuge() behavior, and (2) the use of 
__alloc_pages_node() does not match the commit description which states 
"re-migrate is a waste of time."