[PATCH] mm: fix the race on huge alloc failed

[Guilherme Giacomo Simoes]

The race occurs because the reader (__vmf_anon_prepare()) checks
`vma->anon->vma` without holding the mmap_lock and withou the
READ_ONCE() macro. Since the writer (__anon_vma_prepare()) is holding the
mmap_lock and updating the pointer, it creates a data race as the two
access are not properly synchronized.

Use READ_ONCE() on the reader side and WRITE_ONCE() on the writer side
to tell to compiler treat these memory access carefully and not to
optimize them leading to inconsistent read.

Reported-by: syzbot+395b7abe9696862fc188@xxxxxxxxxxxxxxxxxxxxxxxxx
Closes: https://syzkaller.appspot.com/bug?extid=395b7abe9696862fc188
Fixes: 164b06f238b9 ("mm: call wp_page_copy() under the VMA lock")
Signed-off-by: Guilherme Giacomo Simoes <trintaeoitogc@xxxxxxxxx>
---
 mm/memory.c | 2 +-
 mm/rmap.c   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index 6b8280cfc1db..33c1fbd30cdd 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -3820,7 +3820,7 @@ vm_fault_t __vmf_anon_prepare(struct vm_fault *vmf)
 	struct vm_area_struct *vma = vmf->vma;
 	vm_fault_t ret = 0;
 
-	if (likely(vma->anon_vma))
+	if (likely(READ_ONCE(vma->anon_vma)))
 		return 0;
 	if (vmf->flags & FAULT_FLAG_VMA_LOCK) {
 		if (!mmap_read_trylock(vma->vm_mm))
diff --git a/mm/rmap.c b/mm/rmap.c
index 1c77d5dc06e9..9d64d776b8c5 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -209,7 +209,7 @@ int __anon_vma_prepare(struct vm_area_struct *vma)
 	/* page_table_lock to protect against threads */
 	spin_lock(&mm->page_table_lock);
 	if (likely(!vma->anon_vma)) {
-		vma->anon_vma = anon_vma;
+		WRITE_ONCE(vma->anon_vma, anon_vma);
 		anon_vma_chain_assign(vma, avc, anon_vma);
 		anon_vma_interval_tree_insert(avc, &anon_vma->rb_root);
 		anon_vma->num_active_vmas++;
-- 
2.52.0