[PATCH 3/3] sparc64: Fix bugs in get_user_pages_fast() wrt. THP.

[David Miller]

First of all, we were blocking the wrong kind of interrupts
over the page table scan.

%pil based interrupts (the ones blocked by local_irq_disable) are not
the type which are used to deliver remote TLB flushes.  Rather,
interrupt vectors are used for that.

And interrupt vector delivery can only be blocked by turning off
PSTATE_IE in the %pstate register.

So use that instead of local_irq_{disable,enable}().

Secondly, the large PMD path needs to check _PAGE_VALID not
_PAGE_PRESENT, to decide if it needs to bail and return 0.

pmd_large() should therefore just check _PAGE_PMD_HUGE.

Calls to gup_huge_pmd() are guarded with a check of pmd_large(), so we
just need to add a valid bit check.

Signed-off-by: David S. Miller <davem@xxxxxxxxxxxxx>
---
 arch/sparc/include/asm/pgtable_64.h |  2 +-
 arch/sparc/mm/gup.c                 | 19 +++++++++++++++----
 2 files changed, 16 insertions(+), 5 deletions(-)

diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/pgtable_64.h
index e3db4b8..1f28103 100644
--- a/arch/sparc/include/asm/pgtable_64.h
+++ b/arch/sparc/include/asm/pgtable_64.h
@@ -633,7 +633,7 @@ static inline unsigned long pmd_large(pmd_t pmd)
 {
 	pte_t pte = __pte(pmd_val(pmd));
 
-	return (pte_val(pte) & _PAGE_PMD_HUGE) && pte_present(pte);
+	return pte_val(pte) & _PAGE_PMD_HUGE;
 }
 
 #ifdef CONFIG_TRANSPARENT_HUGEPAGE
diff --git a/arch/sparc/mm/gup.c b/arch/sparc/mm/gup.c
index c4d3da6..97fb4fc 100644
--- a/arch/sparc/mm/gup.c
+++ b/arch/sparc/mm/gup.c
@@ -73,7 +73,7 @@ static int gup_huge_pmd(pmd_t *pmdp, pmd_t pmd, unsigned long addr,
 	struct page *head, *page, *tail;
 	int refs;
 
-	if (!pmd_large(pmd))
+	if (!(pmd_val(pmd) & _PAGE_VALID))
 		return 0;
 
 	if (write && !pmd_write(pmd))
@@ -165,7 +165,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write,
 {
 	struct mm_struct *mm = current->mm;
 	unsigned long addr, len, end;
-	unsigned long next;
+	unsigned long next, pstate;
 	pgd_t *pgdp;
 	int nr = 0;
 
@@ -191,7 +191,12 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write,
 	 * So long as we atomically load page table pointers versus teardown,
 	 * we can follow the address down to the the page and take a ref on it.
 	 */
-	local_irq_disable();
+	__asm__ __volatile__(
+		"rdpr %%pstate, %0\n\t"
+		"andn %0, %1, %0\n\t"
+		"wrpr %0, %%g0, %%pstate"
+		: "=&r" (pstate)
+		: "i" (PSTATE_IE));
 
 	pgdp = pgd_offset(mm, addr);
 	do {
@@ -204,7 +209,13 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write,
 			goto slow;
 	} while (pgdp++, addr = next, addr != end);
 
-	local_irq_enable();
+	/* Re-enable interrupts. */
+	__asm__ __volatile__(
+		"rdpr %%pstate, %0\n\t"
+		"or %0, %1, %0\n\t"
+		"wrpr %0, %%g0, %%pstate"
+		: "=&r" (pstate)
+		: "i" (PSTATE_IE));
 
 	VM_BUG_ON(nr != (end - start) >> PAGE_SHIFT);
 	return nr;
-- 
1.9.0

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/