Re: [PATCH v4 0/3] Compile-time stack frame pointer validation

[Ingo Molnar]

* Josh Poimboeuf <jpoimboe@xxxxxxxxxx> wrote:

> On Wed, May 20, 2015 at 09:59:18AM -0700, Linus Torvalds wrote:
> > On Wed, May 20, 2015 at 9:25 AM, Josh Poimboeuf <jpoimboe@xxxxxxxxxx> wrote:
> > > On Wed, May 20, 2015 at 09:03:37AM -0700, Andy Lutomirski wrote:
> > >>
> > >> I've never quite understood what the '?' means.
> > >
> > > It basically means "here's a function address we found on the 
> > > stack, which may or may not have been called."  It's needed 
> > > because stack walking isn't currently 100% reliable.
> > 
> > It is often quite interesting and helpful, because it shows stale 
> > data on the stack, giving clues about what happened just before.
> > 
> > Now, I'd like gcc to generally be better about not wasting so much 
> > stack frame, so in that sense I'd like to see fewer '?" entries 
> > just from a code quality standpoint, but when debugging those 
> > things, the downside of "noise" is often cancelled by the upside 
> > of "ahh, it happens after calling X".
> > 
> > So the "perfect stack frames" is actually not as great a thing as 
> > some people want to make it seem.
> 
> Ok, I can see how looking at stale stack data could be useful for 
> some of the really tough problems.

And note that the tough problems are actually the ones where we need 
that information the most. So any stack backtrace printing method must 
be biased towards helping the difficult scenarios - not the trivial 
crashes. That is one of the reasons why we are always printing the 
question marks.

> But right now, the meaning of '?' is ambiguous.  It could be stale 
> data, or it could be part of a frame for the current stack which was 
> skipped due to missing frame pointers or an exception.

Yes, of course. That's not a big problem as the actual symbolic 
information will tell us a lot, which allows us to reconstruct the 
real call chain, plus allows us to see any 'recent execution activity' 
that might be on the stack as stale entries.

> If we can somehow make the stack unwinder reliable, then it would at 
> least allow us to remove the ambiguity of the '?' entries.  And it 
> would reduce the "noise" for the majority of issues where we don't 
> care about stale stack data, and can simply ignore it.

Yes, but note the above consideration - the probability distribution 
of kernel bugs tends to have a _very_ long tail, with bugs that 
sometimes take years to trigger and fix. Kernel developers upstream 
and at distros tend to spend a disproportionately large amount of time 
staring at difficult to decode bugs.

For that reason it is far more important to still stay maintainable 
with those kinds of difficult bugs, than to make the resolution of 
trivial, unambiguous crashes a tiny bit easier by printing fewer 
'distractions'...

Also, note that the '?' entries have another role: they cross-check 
the unwinder.

If you think we'll be able to do a perfect unwinder then think again: 
debug info _will_ be messed up periodically, either by us or by 
tooling, because right now no kernel code or other functionality 
relies on perfect unwinding.

So this is not like C++ exception handling where broken unwinding will 
break the code. This is something that is literally only visible in 
kernel logs currently, as a slight anomaly.

So any x86 stack unwinder code must be fundamentally based on the idea 
and expectation that stack unwinding is always going to be somewhat 
imperfect and somewhat statistical.

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/