[PATCH] mm/huge_memory: fix the memory leak due to the race

From: zhongjiang
Date: Tue Jun 21 2016 - 10:58:58 EST


From: zhong jiang <zhongjiang@xxxxxxxxxx>

with great pressure, I run some test cases. As a result, I found
that the THP is not freed, it is detected by check_mm().

BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512

Consider the following race :

CPU0 CPU1
__handle_mm_fault()
wp_huge_pmd()
do_huge_pmd_wp_page()
pmdp_huge_clear_flush_notify()
(pmd_none = true)
exit_mmap()
unmap_vmas()
zap_pmd_range()
pmd_none_or_trans_huge_or_clear_bad()
(result in memory leak)
set_pmd_at()

because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
and it make the pmd entry to be null. Therefore, The memory leak can occur.

The patch fix the scenario that the pmd entry can lead to be null.

Signed-off-by: zhong jiang <zhongjiang@xxxxxxxxxx>
---
mm/huge_memory.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index e10a4fe..95c7dfe 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1340,7 +1340,7 @@ alloc:
pmd_t entry;
entry = mk_huge_pmd(new_page, vma->vm_page_prot);
entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma);
- pmdp_huge_clear_flush_notify(vma, haddr, pmd);
+ pmdp_invalidate(vma, haddr, pmd);
page_add_new_anon_rmap(new_page, vma, haddr, true);
mem_cgroup_commit_charge(new_page, memcg, false, true);
lru_cache_add_active_or_unevictable(new_page, vma);
--
1.8.3.1