[PATCH] mm: Fix MAP_POPULATE and mlock() for DAX

[Toshi Kani]

DAX has the following issues in a shared or read-only private
mmap'd file.
 - mmap(MAP_POPULATE) does not pre-fault
 - mlock() fails with -ENOMEM

DAX uses VM_MIXEDMAP for mmap'd files, which do not have struct
page associated with the ranges.  Both MAP_POPULATE and mlock()
call __mm_populate(), which in turn calls __get_user_pages().
Because __get_user_pages() requires a valid page returned from
follow_page_mask(), MAP_POPULATE and mlock(), i.e. FOLL_POPULATE,
fail in the first page.

Change __get_user_pages() to proceed FOLL_POPULATE when the
translation is set but its page does not exist (-EFAULT), and
@pages is not requested.  With that, MAP_POPULATE and mlock()
set translations to the requested range and complete successfully.

MAP_POPULATE still provides a major performance improvement to
DAX as it will avoid page faults during initial access to the
pages.

mlock() continues to set VM_LOCKED to vma and populate the range.
Since there is no struct page, the range is pinned without marking
pages mlocked.

Note, MAP_POPULATE and mlock() already work for a write-able
private mmap'd file on DAX since populate_vma_page_range() breaks
COW, which allocates page caches.

Signed-off-by: Toshi Kani <toshi.kani@xxxxxx>
---
 mm/gup.c |   14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/mm/gup.c b/mm/gup.c
index 6297f6b..16d536f 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -490,8 +490,20 @@ retry:
 			}
 			BUG();
 		}
-		if (IS_ERR(page))
+		if (IS_ERR(page)) {
+			/*
+			 * No page may be associated with VM_MIXEDMAP. Proceed
+			 * FOLL_POPULATE when the translation is set but its
+			 * page does not exist (-EFAULT), and @pages is not
+			 * requested by the caller.
+			 */
+			if ((PTR_ERR(page) == -EFAULT) && (!pages) &&
+			    (gup_flags & FOLL_POPULATE) &&
+			    (vma->vm_flags & VM_MIXEDMAP))
+				goto next_page;
+
 			return i ? i : PTR_ERR(page);
+		}
 		if (pages) {
 			pages[i] = page;
 			flush_anon_page(vma, page, start);
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/