Re: hugetlb page migration vs. overcommit

[Mike Kravetz]

On 11/22/2017 07:28 AM, Michal Hocko wrote:
> Hi,
> is there any reason why we enforce the overcommit limit during hugetlb
> pages migration? It's in alloc_huge_page_node->__alloc_buddy_huge_page
> path. I am wondering whether this is really an intentional behavior.

I do not think it was intentional.  But, I was not around when that
code was added.

> The page migration allocates a page just temporarily so we should be
> able to go over the overcommit limit for the migration duration. The
> reason I am asking is that hugetlb pages tend to be utilized usually
> (otherwise the memory would be just wasted and pool shrunk) but then
> the migration simply fails which breaks memory hotplug and other
> migration dependent functionality which is quite suboptimal. You can
> workaround that by increasing the overcommit limit.

Yes.  In an environment making optimal use of huge pages, you are unlikely
to have 'spare pages' set aside for a potential migration operation.  So
I agree that it would make sense to try and allocate overcommit pages for
this purpose.

> Why don't we simply migrate as long as we are able to allocate the
> target hugetlb page? I have a half baked patch to remove this
> restriction, would there be an opposition to do something like that?

I would not be opposed and would help with this effort.  My concern would
be any subtle hugetlb accounting issues once you start messing with
additional overcommit pages.

Since Naoya was originally involved in huge page migration, I would welcome
his comments.
-- 
Mike Kravetz