Symbols not found for some files

From: Paul Menzel
Date: Wed Oct 03 2018 - 16:43:22 EST


Dear Linux folks,


For profiling the boot on my Debian Sid/unstable system (32-bit user space), the following service unit is used.

```
$ systemctl cat perf
# /etc/systemd/system/perf.service

[Unit]
Description=Perf 10 s
DefaultDependencies=no

[Service]
ExecStart=/usr/local/bin/perf record -F 999 -g -a -o /dev/shm/perf.data sleep 10

[Install]
WantedBy=sysinit.target
```

Running `perf.data` through `perf script`, the messages below are shown.

```
[vdso] with build id 69dd0f0c522f0d3e2a87722d70245ab9725c8b42 not found, continuing without symbols
/lib/systemd/libsystemd-shared-239.so with build id 196c7922a15fa971d68775522dd16e82da46ebb3 not found, continuing without symbols
/lib/systemd/systemd-journald with build id 9e31591e8c9bb88de06e96a251f34d8a7e201f6c not found, continuing without symbols
/lib/systemd/systemd with build id 1b92f342a959b6ff370b82092bfadaf1d470a7b9 not found, continuing without symbols
/lib/systemd/systemd-networkd with build id 9fa8f6abc1f2e5c1018d0a33be058c22e25efaff not found, continuing without symbols
/lib/systemd/systemd-logind with build id 63d664b5f8f8e85274aff6730ec9e924d74c8fed not found, continuing without symbols
```

But, the debug symbol packages are installed (`systemd-dbgsym`), and GDB loads the debug symbols just fine.

```
$ gdb --quiet /lib/systemd/libsystemd-shared-239.so
Reading symbols from /lib/systemd/libsystemd-shared-239.so...Reading symbols from /usr/lib/debug/.build-id/19/6c7922a15fa971d68775522dd16e82da46ebb3.debug...done.
done.
(gdb)
```

Tracing `perf`, it also seems to find them, but does not use them.

```
stat64("/home/kodi/.debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f", 0xbfbd54ac) = -1 ENOENT (No such file or directory)
stat64("/home/kodi/.debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f/elf", 0xbfbd75bc) = -1 ENOENT (No such file or directory)
stat64("03af6935d5ab00f79fe12b4098d27d3f70c19f.debug", 0xbfbd1bfc) = -1 ENOENT (No such file or directory)
stat64("/lib/i386-linux-gnu/03af6935d5ab00f79fe12b4098d27d3f70c19f.debug", 0xbfbd1bfc) = -1 ENOENT (No such file or directory)
stat64("/lib/i386-linux-gnu/.debug/03af6935d5ab00f79fe12b4098d27d3f70c19f.debug", 0xbfbd1bfc) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/debug/lib/i386-linux-gnu/03af6935d5ab00f79fe12b4098d27d3f70c19f.debug", 0xbfbd1bfc) = -1 ENOENT (No such file or directory)
stat64("/home/kodi/.debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f", 0xbfbd1b5c) = -1 ENOENT (No such file or directory)
stat64("/home/kodi/.debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f/elf", 0xbfbd3c6c) = -1 ENOENT (No such file or directory)
stat64("/home/kodi/.debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f", 0xbfbd1b5c) = -1 ENOENT (No such file or directory)
stat64("/home/kodi/.debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f/debug", 0xbfbd3c6c) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f.debug", {st_mode=S_IFREG|0644, st_size=4524336, ...}) = 0
openat(AT_FDCWD, "/usr/lib/debug/.build-id/a2/03af6935d5ab00f79fe12b4098d27d3f70c19f.debug", O_RDONLY|O_LARGEFILE) = 91
```

Looking into `perf` with GDB, the code in `util/symbol.c` below is not setting `runtime_ss`, because `symsrc__possibly_runtime(ss)` returns false. Then the for loop is not quit, and continues.

```
if (!syms_ss && symsrc__has_symtab(ss)) {
syms_ss = ss;
next_slot = true;
if (!dso->symsrc_filename)
dso->symsrc_filename = strdup(name);
}

if (!runtime_ss && symsrc__possibly_runtime(ss)) {
runtime_ss = ss;
next_slot = true;
}

if (next_slot) {
ss_pos++;

if (syms_ss && runtime_ss)
break;
} else {
symsrc__destroy(ss);
}

```

Do you have an idea, why the files from the systemd Debian package cause problems for `perf` but not GDB? The Debian maintainers mentioned, that LTO is enabled for the Debian package.

Please tell me, if you need other information to look into the problem.


Kind regards,

Paul