[RFC PATCH 2/2] x86/ibpb: Prevent missed IBPB flush

[Tim Chen]

It is possible that the last uesr mm that we recorded for a cpu was
released, and a new mm with identical address was allocated when we
check it again.  We could skip IBPB flush here for the process with
the new mm.

It is a difficult to exploit case as we have to exit() a process on a
cpu, free the mm, and fork() the victim to use the mm pointer on that
cpu. The exploiter needs the old mm to get recycled to the
newly forked process and no other processes run on the target cpu.

Nevertheless, the patch below is one way to close this hole by
adding a ref count to prevent the last user mm from being released.
It does add ref counting overhead, and extra memory cost of keeping an mm
(though not the VMAs and most of page tables) around longer than we will
otherwise need to. Any better solutions are welcomed.

Suggested-by: Dave Hansen <dave.hansen@xxxxxxxxx>
Signed-off-by: Tim Chen <tim.c.chen@xxxxxxxxxxxxxxx>
---
 arch/x86/mm/tlb.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 86ed07f..3bdaa10 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -7,6 +7,7 @@
 #include <linux/export.h>
 #include <linux/cpu.h>
 #include <linux/debugfs.h>
+#include <linux/sched/mm.h>
 
 #include <asm/tlbflush.h>
 #include <asm/mmu_context.h>
@@ -291,8 +292,17 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
 			trace_tlb_flush_rcuidle(TLB_FLUSH_ON_TASK_SWITCH, 0);
 		}
 
-		if (next != &init_mm && next != last)
+		if (next != &init_mm && next != last) {
+			if (last != NULL)
+				mmdrop(last);
+			/*
+			 * Keep 'next' allocated until we switch to another mm.
+			 * This keeps us from missing a flush of the branch predictors
+			 * if 'next' gets freed and reallocated.
+			 */
+			mmgrab(next);
 			this_cpu_write(cpu_tlbstate.last_usr_mm, next);
+		}
 		this_cpu_write(cpu_tlbstate.loaded_mm, next);
 		this_cpu_write(cpu_tlbstate.loaded_mm_asid, new_asid);
 	}
@@ -370,6 +380,7 @@ void initialize_tlbstate_and_flush(void)
 	write_cr3(build_cr3(mm->pgd, 0));
 
 	/* Reinitialize tlbstate. */
+	this_cpu_write(cpu_tlbstate.last_usr_mm, NULL);
 	this_cpu_write(cpu_tlbstate.loaded_mm_asid, 0);
 	this_cpu_write(cpu_tlbstate.next_asid, 1);
 	this_cpu_write(cpu_tlbstate.ctxs[0].ctx_id, mm->context.ctx_id);
-- 
2.9.4