Re: Symbols not found for some files

From: Paul Menzel
Date: Wed Oct 03 2018 - 23:19:44 EST


Dear Arnaldo,


Am 03.10.2018 um 22:57 schrieb Arnaldo Carvalho de Melo:
Em Wed, Oct 03, 2018 at 10:42:39PM +0200, Paul Menzel escreveu:

For profiling the boot on my Debian Sid/unstable system (32-bit user space),
the following service unit is used.

You forgot to mention what is the version of the perf tool in that
system, so that I could check if this got fixed somehow after that
version.

Can you please provide it?

Sorry for not mentioning it. I built it from Linusâ Linux master branch.

$ perf --version
perf version 4.19.rc5.gd7887c


Kind regards,

Paul


```
$ systemctl cat perf
# /etc/systemd/system/perf.service

[Unit]
Description=Perf 10 s
DefaultDependencies=no

[Service]
ExecStart=/usr/local/bin/perf record -F 999 -g -a -o /dev/shm/perf.data
sleep 10

[Install]
WantedBy=sysinit.target
```

Running `perf.data` through `perf script`, the messages below are shown.

```
[vdso] with build id 69dd0f0c522f0d3e2a87722d70245ab9725c8b42 not found,
continuing without symbols
/lib/systemd/libsystemd-shared-239.so with build id
196c7922a15fa971d68775522dd16e82da46ebb3 not found, continuing without
symbols
/lib/systemd/systemd-journald with build id
9e31591e8c9bb88de06e96a251f34d8a7e201f6c not found, continuing without
symbols
/lib/systemd/systemd with build id 1b92f342a959b6ff370b82092bfadaf1d470a7b9
not found, continuing without symbols
/lib/systemd/systemd-networkd with build id
9fa8f6abc1f2e5c1018d0a33be058c22e25efaff not found, continuing without
symbols
/lib/systemd/systemd-logind with build id
63d664b5f8f8e85274aff6730ec9e924d74c8fed not found, continuing without
symbols
```

But, the debug symbol packages are installed (`systemd-dbgsym`), and GDB
loads the debug symbols just fine.

```
$ gdb --quiet /lib/systemd/libsystemd-shared-239.so
Reading symbols from /lib/systemd/libsystemd-shared-239.so...Reading symbols
from /usr/lib/debug/.build-id/19/6c7922a15fa971d68775522dd16e82da46ebb3.debug...done.
done.
(gdb)
```

Tracing `perf`, it also seems to find them, but does not use them.

```
stat64("/home/kodi/.debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f",
0xbfbd54ac) = -1 ENOENT (No such file or directory)
stat64("/home/kodi/.debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f/elf",
0xbfbd75bc) = -1 ENOENT (No such file or directory)
stat64("03af6935d5ab00f79fe12b4098d27d3f70c19f.debug", 0xbfbd1bfc) = -1
ENOENT (No such file or directory)
stat64("/lib/i386-linux-gnu/03af6935d5ab00f79fe12b4098d27d3f70c19f.debug",
0xbfbd1bfc) = -1 ENOENT (No such file or directory)
stat64("/lib/i386-linux-gnu/.debug/03af6935d5ab00f79fe12b4098d27d3f70c19f.debug",
0xbfbd1bfc) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/debug/lib/i386-linux-gnu/03af6935d5ab00f79fe12b4098d27d3f70c19f.debug",
0xbfbd1bfc) = -1 ENOENT (No such file or directory)
stat64("/home/kodi/.debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f",
0xbfbd1b5c) = -1 ENOENT (No such file or directory)
stat64("/home/kodi/.debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f/elf",
0xbfbd3c6c) = -1 ENOENT (No such file or directory)
stat64("/home/kodi/.debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f",
0xbfbd1b5c) = -1 ENOENT (No such file or directory)
stat64("/home/kodi/.debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f/debug",
0xbfbd3c6c) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f.debug",
{st_mode=S_IFREG|0644, st_size=4524336, ...}) = 0
openat(AT_FDCWD,
"/usr/lib/debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f.debug",
O_RDONLY|O_LARGEFILE) = 91
```

Looking into `perf` with GDB, the code in `util/symbol.c` below is not
setting `runtime_ss`, because `symsrc__possibly_runtime(ss)` returns false.
Then the for loop is not quit, and continues.

```
if (!syms_ss && symsrc__has_symtab(ss)) {
syms_ss = ss;
next_slot = true;
if (!dso->symsrc_filename)
dso->symsrc_filename = strdup(name);
}

if (!runtime_ss && symsrc__possibly_runtime(ss)) {
runtime_ss = ss;
next_slot = true;
}

if (next_slot) {
ss_pos++;

if (syms_ss && runtime_ss)
break;
} else {
symsrc__destroy(ss);
}

```

Do you have an idea, why the files from the systemd Debian package cause
problems for `perf` but not GDB? The Debian maintainers mentioned, that LTO
is enabled for the Debian package.

Please tell me, if you need other information to look into the problem.


Kind regards,

Paul