[PATCH for 7.4-rc2+ 4/7] s390/mm: always clear _SEGMENT_ENTRY_READ in pmdp_invalidate()

[Zi Yan]

pmdp_invalidate() sets _SEGMENT_ENTRY_INVALID, but on s390 that bit
alone also marks an old (not young) PMD, which is still a valid mapping
that GUP-fast follows. Without CONFIG_PAGE_TABLE_CHECK, an invalidated
PMD is therefore not pmd_protnone(), so GUP-fast can grab a reference on
the folio and, as the PMD value doesn't change until the caller
finishes, the recheck after taking the reference passes too.

Code that invalidates a PMD and then decides, from the folio refcount,
whether it may drop the mapping relies on GUP-fast backing off from an
invalidated PMD, like it does from a cleared one. x86, arm64 and powerpc
already make GUP-fast reject invalidated PMDs.

With CONFIG_PAGE_TABLE_CHECK, pmdp_invalidate() already clears
_SEGMENT_ENTRY_READ, which makes the entry pmd_protnone(), so GUP-fast
does not follow it. Do that unconditionally. As noted in commit
7b4dde5e40ad6 ("s390/pgtable: Add s390 support for page table check"),
an invalidated PMD that is pmd_protnone() is fine for the remaining
pmd_protnone() users, which run under the page table lock or are
lockless walkers that should not follow it anyway.

Assisted-by: LLM
Signed-off-by: Zi Yan <ziy@xxxxxxxxxx>
To: Alexander Gordeev <agordeev@xxxxxxxxxxxxx>
To: Gerald Schaefer <gerald.schaefer@xxxxxxxxxxxxx>
To: Heiko Carstens <hca@xxxxxxxxxxxxx>
To: Vasily Gorbik <gor@xxxxxxxxxxxxx>
Cc: Christian Borntraeger <borntraeger@xxxxxxxxxxxxx>
Cc: Sven Schnelle <svens@xxxxxxxxxxxxx>
Cc: Jason Gunthorpe <jgg@xxxxxxxxxx>
Cc: John Hubbard <jhubbard@xxxxxxxxxx>
Cc: Peter Xu <peterx@xxxxxxxxxx>
Cc: linux-s390@xxxxxxxxxxxxxxx
Cc: linux-kernel@xxxxxxxxxxxxxxx
---
 arch/s390/include/asm/pgtable.h | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h
index 2d5c2ab06de98..fe1ba746203ab 100644
--- a/arch/s390/include/asm/pgtable.h
+++ b/arch/s390/include/asm/pgtable.h
@@ -1781,10 +1781,14 @@ static inline pmd_t pmdp_invalidate(struct vm_area_struct *vma,
 	pmd_t pmd = *pmdp;
 
 	VM_WARN_ON_ONCE(!pmd_present(pmd));
+	/*
+	 * _SEGMENT_ENTRY_INVALID alone also marks an old (not young) entry,
+	 * which lockless walkers like GUP-fast may follow. Clear
+	 * _SEGMENT_ENTRY_READ too, so the invalidated entry is pmd_protnone()
+	 * and is not followed, and page table check does not count it.
+	 */
 	pmd = set_pmd_bit(pmd, __pgprot(_SEGMENT_ENTRY_INVALID));
-#ifdef CONFIG_PAGE_TABLE_CHECK
 	pmd = clear_pmd_bit(pmd, __pgprot(_SEGMENT_ENTRY_READ));
-#endif
 	page_table_check_pmd_set(vma->vm_mm, addr, pmdp, pmd);
 	pmd = pmdp_xchg_direct(vma->vm_mm, addr, pmdp, pmd);
 	return pmd;

-- 
2.53.0