Re: perf is unable to read dward from go programs

[Jiri Olsa]

On Mon, Dec 02, 2019 at 11:49:55AM -0800, Ivan Babrou wrote:
> I've tried building with libdw with mixed results:
> 
> 1. I can see stacks from some Go programs that were invisible before (yay!)
> 
> 2. Warnings like below still appear in great numbers for a system-wide
> flamegraph:
> 
> BFD: Dwarf Error: found dwarf version '18345', this reader only
> handles version 2, 3 and 4 information.
> 
> I'm not sure how to pinpoint this to a particular binary and would
> appreciate some help with this.

I'd need some way of reproducing this, could you please
paste me command lines you used?

> 
> 3. It takes minutes to produce a flamegraph of a running system
> whereas before it only took seconds. See this flamegraph of "perf
> script" itself:
> 
> * https://gist.github.com/bobrik/a9c46cffe9daa5840abd137443d8bab0#file-flamegraph-perf-svg
> 
> Seems like there is no caching and debug info is getting reparsed
> continuously for every stack. It's possible that it was not an issue
> before, because we spent no time decompressing dwarf.

possibly, if we have some clear reproducer we can hand it
to the libdw guy that helped us develop this

> 
> 4. Pretty much all luajit frames stacks that were marked as unknown
> before are now gone.
> 
> See before and after here: https://imgur.com/a/1LNfqAk
> 
> Before:
> 
> nginx-cache 94572 446642.722028:   10101010 cpu-clock:
>             5607d8d56718 ngx_http_lua_shdict_lookup+0x48 (inlined)
>             5607d8d5a09d ngx_http_lua_ffi_shdict_incr+0xcd
> (/usr/local/nginx-cache/sbin/nginx-cache)
>             560802fe58e4 [unknown] (/tmp/perf-94572.map)
> 
> After:
> 
> nginx-cache 94572 446543.008703:   10101010 cpu-clock:
>             5607d8d56718 ngx_http_lua_shdict_lookup+0x48 (inlined)
>             5607d8d59da7 ngx_http_lua_ffi_shdict_get+0x197
> (/usr/local/nginx-cache/sbin/nginx-cache)
> 
> The key is /tmp/perf-*.map frame at the bottom. I don't know if it's
> expected, but we grew dependent on knowing this.
> 
> 5. Special [[stack]], [[heap]] and [anon] frames are also gone, and
> you can see the following during "perf script" run:
> 
> open("[stack]", O_RDONLY)               = -1 ENOENT (No such file or directory)
> open("[heap]", O_RDONLY)                = -1 ENOENT (No such file or directory)
> open("//anon", O_RDONLY)                = -1 ENOENT (No such file or directory)

strange, let's start with the reproducer and I'll check
on this if I see it

thanks,
jirka