Re: WARNING: kernel stack frame pointer has bad value

[Steven Rostedt]

On Wed, 19 Apr 2017 08:44:57 -0500
Josh Poimboeuf <jpoimboe@xxxxxxxxxx> wrote:

> On Tue, Apr 18, 2017 at 11:37:14PM -0400, Steven Rostedt wrote:
> > Josh,
> > 
> > I'm starting to get a bunch of these warnings, and I'm thinking they
> > are false positives. The stack frame error is recorded at a call from
> > entry_SYSCALL_64_fastpath, where I would expect the bp to not be valid.
> > 
> > To trigger this, I only need to go into /sys/kernel/debug/tracing and
> > echo function > current_tracer then cat trace. Maybe function tracer
> > stack frames is messing it up some how, but it always fails at the
> > entry call.
> > 
> > Here's the dump;
> > 
> >  WARNING: kernel stack frame pointer at ffff8800bda0ff30 in sshd:1090 has bad value 000055b32abf1fa8  
> ...
> >  ffff8800bda0ff20: ffff8800bda0ff30 (0xffff8800bda0ff30)
> >  ffff8800bda0ff28: ffffffff810dc945 (SyS_rt_sigprocmask+0x5/0x1a0)
> >  ffff8800bda0ff30: 000055b32abf1fa8 (0x55b32abf1fa8)
> >  ffff8800bda0ff38: ffffffff81cf502a (entry_SYSCALL_64_fastpath+0x18/0xad)
> >  ffff8800bda0ff40: 000055b32abf1fa8 (0x55b32abf1fa8)
> >  ffff8800bda0ff48: ffffffff810dc945 (SyS_rt_sigprocmask+0x5/0x1a0)
> >  ffff8800bda0ff50: ffffffff81cf502a (entry_SYSCALL_64_fastpath+0x18/0xad)  
> 
> Thanks for reporting, I hadn't seen this one yet.
> 
> The problem is that the unwinder expects the last frame pointer to be at
> a certain address (0xffff8800bda0ff48 in this case), so it can know that
> it reached the end.  It's confused by the save_mcount_regs macro, which
> builds some fake frames -- which is good -- but then the last frame is
> at a different offset than what the unwinder expects.
> 
> Would it be possible for ftrace to rewrite the stack so that it looks
> like this instead?
> 
> >  ffff8800bda0ff38: ffff8800bda0ff48 (0xffff8800bda0ff48)
> >  ffff8800bda0ff40: ffffffff810dc945 (SyS_rt_sigprocmask+0x5/0x1a0)
> >  ffff8800bda0ff48: 000055b32abf1fa8 (0x55b32abf1fa8)
> >  ffff8800bda0ff50: ffffffff81cf502a (entry_SYSCALL_64_fastpath+0x18/0xad)  
> 
> In other words it would overwrite the "SyS_rt_sigprocmask+0x5/0x1a0"
> value on the stack at ffff8800bda0ff48 with the original bp, instead of
> appending to the existing stack.  If you would be ok with such an
> approach, I could take a stab at it.

This is because we have to handle each different config differently.
This is the case with FENTRY and FRAME_POINTERS. As I like to keep this
as efficient as possible. To do the above, we need to modify the return
address and then restore it. And handle that for each config type.

> 
> The alternative would be to change the unwinder, but I would rather
> avoid having to detect another special case if possible.

I'm not sure what's worse. Modifying all the special cases of ftrace,
or adding a new one to the undwinder.

You can take a crack at it if you like, but it needs to be negligible
in the performance of FENTRY and no frame pointers.

-- Steve