Re: [PATCH 2/5] context_tracking: Don't implement exception_enter/exit() on CONFIG_HAVE_CONTEXT_TRACKING_OFFSTACK

[Frederic Weisbecker]

On Wed, Nov 11, 2020 at 03:32:18PM +0100, Peter Zijlstra wrote:
> On Tue, Oct 27, 2020 at 04:08:24PM +0100, Frederic Weisbecker wrote:
> > An architecture that provides this Kconfig feature doesn't need to
> > store the context tracking state on the task stack because its entry
> > code has been sanitized such that fragile path aren't preemptible
> > and special use of tracing and RCU read side critical sections in these
> > areas have been explicitly annotated.
> > 
> > Hence the exception_enter()/exception_exit() couple doesn't need to be
> > implemented in this case.
> 
> Could you please explain what exception_{enter,exit}() actually do, then
> explain what is required to make it superfluous? Because as is, I don't
> have enough information to verify the claims made.

So, the typical steps with context tracking are:

1) Task runs in userspace
2) Task enters the kernel (syscall/exception/IRQ)
3) Task switches from context tracking state CONTEXT_USER to CONTEXT_KERNEL (user_exit())
4) Task does stuff in kernel
5) Task switches from context tracking state CONTEXT_KERNEL to CONTEXT_USER (user_enter())
6) Task exits the kernel

If an exception fires between 5) and 6), the pt_regs and the context tracking disagree
on the context of the origin. We must set CONTEXT_KERNEL before the exception handler,
that's unconditional for those that want to call into schedule, but we must restore
CONTEXT_USER when we exit the exception while pt_regs tells us that we are resuming to
kernel space.

We can't fix that with storing the context tracking state in a per-cpu or per-task variable
since another exception may fire onto the current one and overwrite the saved state. Also
the task can schedule. So it has to be stored in a per task stack.

This is how exception_enter()/exception_exit() papers over the problem:

5) Task switches from context tracking state CONTEXT_KERNEL to CONTEXT_USER (user_enter())
5.1) Exception fires
5.2) prev_state = exception_enter() // save CONTEXT_USER to prev_state and set CONTEXT_KERNEL
5.3) Exception handler
5.4) exception_enter(prev_state) // restore CONTEXT_USER
5.5) Exception resumes
6) Task exits the kernel

What is required to make it superfluous is to forbid exceptions and IRQs between 2) and 3)
and between 5) and 6), or if we allow any, they won't call into context tracking, eg: NMIs,
and they won't schedule.

I can add that in the changelog if that makes it any clearer.

Thanks.