Re: [v2 PATCH 1/2] mm: thp: register mm for khugepaged when merging vma for shmem

[Kirill A. Shutemov]

On Thu, Jun 21, 2018 at 11:15:48PM +0000, yang.shi@xxxxxxxxxxxxxxxxx wrote:
> When merging anonymous page vma, if the size of vma can fit in at least
> one hugepage, the mm will be registered for khugepaged for collapsing
> THP in the future.
> 
> But, it skips shmem vma. Doing so for shmem too, but not file-private
> mapping, when merging vma in order to increase the odd to collapse
> hugepage by khugepaged.
> 
> Signed-off-by: Yang Shi <yang.shi@xxxxxxxxxxxxxxxxx>
> Cc: Hugh Dickins <hughd@xxxxxxxxxx>
> Cc: Kirill A. Shutemov <kirill.shutemov@xxxxxxxxxxxxxxx>
> Cc: Vlastimil Babka <vbabka@xxxxxxx>
> ---
> v1 --> 2:
> * Exclude file-private mapping per Kirill's comment
> 
>  mm/khugepaged.c | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/khugepaged.c b/mm/khugepaged.c
> index d7b2a4b..9b0ec30 100644
> --- a/mm/khugepaged.c
> +++ b/mm/khugepaged.c
> @@ -440,8 +440,12 @@ int khugepaged_enter_vma_merge(struct vm_area_struct *vma,
>  		 * page fault if needed.
>  		 */
>  		return 0;
> -	if (vma->vm_ops || (vm_flags & VM_NO_KHUGEPAGED))
> -		/* khugepaged not yet working on file or special mappings */
> +	if ((vma->vm_ops && (!shmem_file(vma->vm_file) || vma->anon_vma)) ||
> +	    (vm_flags & VM_NO_KHUGEPAGED))
> +		/*
> +		 * khugepaged not yet working on non-shmem file or special
> +		 * mappings. And, file-private shmem THP is not supported.
> +		 */
>  		return 0;

My point was that vma->anon_vma check above this one should not prevent
collapse for shmem.

Looking into this more, I think we should just replace all these checks
with hugepage_vma_check() call.

-- 
 Kirill A. Shutemov