Re: [PATCH v2 5/7] x86/mm, tracing: Fix CR2 corruption

[Eiichi Tsukata]

On 2019/07/05 11:18, Linus Torvalds wrote:
> On Fri, Jul 5, 2019 at 5:03 AM Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:
>>
>> Despire the current efforts to read CR2 before tracing happens there
>> still exist a number of possible holes:
> 
> So this whole series disturbs me for the simple reason that I thought
> tracing was supposed to save/restore cr2 and make it unnecessary to
> worry about this in non-tracing code.
> 
> That is very much what the NMI code explicitly does. Why shouldn't all
> the other tracing code do the same thing in case they can take page
> faults?
> 
> So I don't think the patches are wrong per se, but this seems to solve
> it at the wrong level.
> 
>                  Linus
> 

Steven previously tried to fix it by saving CR2 in TRACE_IRQS_OFF:
https://lore.kernel.org/lkml/20190320221534.165ab87b@xxxxxxxxxxxxxxxx/

But hit the following WARNING:
https://lore.kernel.org/lkml/20190321095502.47b51356@xxxxxxxxxxxxxxxxxx/

I tried to find out the root cause of the WARNING, and found that it is
caused by touching trace point(TRACE_IRQS_OFF) before search_binary_handler()
at exeve.

To prevent userstack trace code from reading user stack before it becomes ready,
checking current->in_execve in stack_trace_save_user() can help Steven's approach,
though trace_sched_process_exec() is called before current->in_execve = 0 so it changes
current behavior.

The PoC code is as follows:

diff --git a/arch/x86/kernel/stacktrace.c b/arch/x86/kernel/stacktrace.c
index 2abf27d7df6b..30fa6e1b7a87 100644
--- a/arch/x86/kernel/stacktrace.c
+++ b/arch/x86/kernel/stacktrace.c
@@ -116,10 +116,12 @@ void arch_stack_walk_user(stack_trace_consume_fn consume_entry, void *cookie,
                          const struct pt_regs *regs)
 {
        const void __user *fp = (const void __user *)regs->bp;
+       unsigned long address;
 
        if (!consume_entry(cookie, regs->ip, false))
                return;
 
+       address = read_cr2();
        while (1) {
                struct stack_frame_user frame;
 
@@ -131,11 +133,14 @@ void arch_stack_walk_user(stack_trace_consume_fn consume_entry, void *cookie,
                        break;
                if (frame.ret_addr) {
                        if (!consume_entry(cookie, frame.ret_addr, false))
-                               return;
+                               break;
                }
                if (fp == frame.next_fp)
                        break;
                fp = frame.next_fp;
        }
+
+       if (address != read_cr2())
+               write_cr2(address);
 }
 
diff --git a/kernel/stacktrace.c b/kernel/stacktrace.c
index 36139de0a3c4..489d33bb5d28 100644
--- a/kernel/stacktrace.c
+++ b/kernel/stacktrace.c
@@ -230,6 +230,9 @@ unsigned int stack_trace_save_user(unsigned long *store, unsigned int size)
        /* Trace user stack if not a kernel thread */
        if (!current->mm)
                return 0;
+       /* current can reach some trace points before its stack is ready */
+       if (current->in_execve)
+               return 0;
 
        arch_stack_walk_user(consume_entry, &c, task_pt_regs(current));
        return c.len;