Re: [tip:x86/asm] x86/asm/entry/64: Migrate error and IRQ exit work to C and remove old assembly code

[Andy Lutomirski]

On Tue, Aug 11, 2015 at 4:22 PM, Frederic Weisbecker <fweisbec@xxxxxxxxx> wrote:
>
> On Tue, Aug 11, 2015 at 03:51:26PM -0700, Andy Lutomirski wrote:
>> On Tue, Aug 11, 2015 at 3:38 PM, Frederic Weisbecker <fweisbec@xxxxxxxxx> wrote:
>> >
>> > This makes me very nervous as well!
>> >
>> > It means that instead of using the context tracking save/restore model that we had
>> > with exception_enter/exception_exit(), now we rely on the CS register.
>> >
>> > I don't think we can do that because our "context tracking" is a soft tracking whereas
>> > CS is hard tracking and both are not atomically synchronized together.
>> >
>> > Imagine this situation: we are running in userspace. Context tracking knows it, everything
>> > is fine. Now we do a syscall, we enter in kernel entry code but we trigger an exception
>> > (DEBUG for example) before we got a chance to call user_exit(), which means that the context
>> > tracking code still thinks we are in userspace, so we look at CS from the exception entry code
>> > and it says the exception happened in the kernel. Hence we don't call user_exit() before calling
>> > the exception handler. There is the bug because the exception handler may use RCU which still
>> > thinks we run in userspace.
>>
>> #DB doesn't go through this patch -- it uses the paranoid entry path
>> and ist_enter.  But I see your point.  I think that, if we have a
>> problem like this in practice, then we should fix it.
>
> Whatever hack we do to prevent from exceptions happening in between real kernel entry
> to tracked kernel entry is going to be far less robust than relying strictly on soft
> context tracking.
>

Why?

Any exception that doesn't leave the context tracking state exactly
the way it found it is buggy.  That means that we need to make sure
that context tracking itself is safe wrt exceptions and that we need
to make sure that any exception that can happen early in entry is
itself safe.

The latter is annoying, but the entry code needs to deal with it
anyway.  For example, any exception early in NMI is currently really
bad.  Non-IST exceptions very early in SYSCALL are fatal.
Non-paranoid exceptions outside swapgs are fatal.  Etc.

> The resulting bugs are rare and very hard to reproduce and diagnose.

That's why I stuck assertions all over the place.  I know of exactly
one case that will trip the assertion, and it's a false positive and I
plan on fixing it soon.

>
>>
>> But the old code had the same issue.  If we got an exception (the most
>> likely one is probably a vmalloc fault) during user_exit and we then
>> hit exception_enter, the result would probably be bad.
>
> We have a recursion protection in context tracking that should protect against
> exceptions triggering in the middle of half-set states.

I sure hope so.  It would be nice to mark it with with nokprobes, etc
if needed, too.

>
>>
>> >
>> > In early context tracking days we have relied on CS. But I changed that because of such
>> > issue. The only reliable source for soft context tracking is the soft context tracking itself.
>>
>> I don't see why the soft state is more reliable.  The only bad case is
>> where the entry itself (HW entry up to user_exit) is not atomic
>> enough, but that path should be at least as atomic as user_exit itself
>> is.
>
> Note it's not only about entry code up to user_exit() but also about
> user_enter() up to iret.
>

We already need to block interrupts there, and the code for exit back
to userspace is very clean in -tip.

> Also as long as there is at least one instruction between entry to the kernel
> and context tracking noting it, there is a risk for an exception. Hence entry
> code will never be atomic enough to avoid this kind of bugs.

By that argument, we're doomed.  Non-IST exceptions outside swapgs are fatal.

>
> Heh if only we had something like local_exception_save()!

What would that mean?

Exceptions aren't magic asynchronous things.  They happen only when
you do something that can trigger an exception.

--Andy
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/