Re: [PATCH] mm: Fix mmap MAP_POPULATE for DAX pmd mapping

[Toshi Kani]

On Wed, 2015-12-02 at 11:57 -0800, Dan Williams wrote:
> On Wed, Dec 2, 2015 at 12:12 PM, Toshi Kani <toshi.kani@xxxxxxx> wrote:
> > On Wed, 2015-12-02 at 13:02 -0700, Toshi Kani wrote:
> > > On Wed, 2015-12-02 at 11:00 -0800, Dan Williams wrote:
> > > > On Wed, Dec 2, 2015 at 11:26 AM, Toshi Kani <toshi.kani@xxxxxxx> wrote:
> > > > > On Wed, 2015-12-02 at 10:06 -0800, Dan Williams wrote:
> > > > > > On Wed, Dec 2, 2015 at 9:01 AM, Dan Williams <
> > > > > > dan.j.williams@xxxxxxxxx>
> > > > > > wrote:
> > > > > > > On Wed, Dec 2, 2015 at 9:43 AM, Toshi Kani <toshi.kani@xxxxxxx>
> > > > > > > wrote:
> > > > > > > > Oh, I see.  I will setup the memmap array and run the tests
> > > > > > > > again.
> > > > > > > > 
> > > > > > > > But, why does the PMD mapping depend on the memmap array?  We 
> > > > > > > > have observed major performance improvement with PMD.  This 
> > > > > > > > feature should always be enabled with DAX regardless of the 
> > > > > > > > option to allocate the memmap array.
> > > > > > > > 
> > > > > > > 
> > > > > > > Several factors drove this decision, I'm open to considering
> > > > > > > alternatives but here's the reasoning:
> > > > > > > 
> > > > > > > 1/ DAX pmd mappings caused crashes in the get_user_pages path 
> > > > > > > leading to commit e82c9ed41e8 "dax: disable pmd mappings".  The 
> > > > > > > reason pte mappings don't crash and instead trigger -EFAULT is due 
> > > > > > > to the _PAGE_SPECIAL pte bit.
> > > > > > > 
> > > > > > > 2/ To enable get_user_pages for DAX, in both the page and huge
> > > > > > > -page case, we need a new pte bit _PAGE_DEVMAP.
> > > > > > > 
> > > > > > > 3/ Given the pte bits are hard to come I'm assuming we won't get 
> > > > > > > two, i.e. both _PAGE_DEVMAP and a new _PAGE_SPECIAL for pmds. 
> > > > > > >  Even if we could get a _PAGE_SPECIAL for pmds I'm not in favor of 
> > > > > > > pursuing it.
> > > > > > 
> > > > > > Actually, Dave says they aren't that hard to come by for pmds, so we
> > > > > > could go add _PMD_SPECIAL if we really wanted to support the limited
> > > > > > page-less DAX-pmd case.
> > > > > > 
> > > > > > But I'm still of the opinion that we run away from the page-less 
> > > > > > case until it can be made a full class citizen with O_DIRECT for pfn
> > > > > > support.
> > > > > 
> > > > > I may be missing something, but per vm_normal_page(), I think
> > > > > _PAGE_SPECIAL can be substituted by the following check when we do not
> > > > > have the memmap.
> > > > > 
> > > > >         if ((vma->vm_flags & VM_PFNMAP) ||
> > > > >             ((vma->vm_flags & VM_MIXEDMAP) && (!pfn_valid(pfn)))) {
> > > > > 
> > > > > This is what I did in this patch for follow_trans_huge_pmd(), although 
> > > > > I missed the pfn_valid() check.
> > > > 
> > > > That works for __get_user_pages but not __get_user_pages_fast where we
> > > > don't have access to the vma.
> > > 
> > > __get_user_page_fast already refers current->mm, so we should be able to 
> > > get the vma, and pass it down to gup_pud_range().
> > 
> > Alternatively, we can obtain the vma from current->mm in gup_huge_pmd() when 
> > the !pfn_valid() condition is met, so that we do not add the code to the 
> > main path of __get_user_pages_fast.
> 
> The whole point of __get_user_page_fast() is to avoid the overhead of
> taking the mm semaphore to access the vma.  _PAGE_SPECIAL simply tells
> __get_user_pages_fast that it needs to fallback to the
> __get_user_pages slow path.

I see.  Then, I think gup_huge_pmd() can simply return 0 when !pfn_valid(),
instead of VM_BUG_ON.

Thanks,
-Toshi
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/