Re: [PATCH] perf unwind-libdw: Fix unwinding of multi-threaded processes

[Ian Rogers]

On Wed, Jul 22, 2026 at 3:25 AM Alessio Podda <aleph.pi.gh@xxxxxxxxx> wrote:
>
> The libdw callback API has two levels: dwfl_getthread_frames() first finds
> the requested thread using the next_thread() or get_thread() callback and
> then walks its stack.
>
> Since perf only has a snapshot of the stack of a single thread, it
> provides a stubbed-out API that always returns the pid the Dwfl was
> attached with (i.e. whatever was passed to dwfl_attach_state()), rather
> than the actual sampled thread's TID.
>
> Commit 6b2658b3f36a ("perf unwind-libdw: Don't discard loaded ELF/DWARF
> after every unwind") changed libdw unwinding from recreating the Dwfl
> object for each sample to caching it in struct maps, which is shared by
> every thread in the process. It left next_thread() unchanged.
>
> Since the pid passed to dwfl_attach_state() is only set at creation, only
> the thread of the first sample is ever found. As a result,
> dwfl_getthread_frames() fails with ESRCH when asked to unwind a sample
> from another thread.
>
> Make next_thread() return the current sample's TID, provide get_thread()
> so libdw can find it directly, and pass the process PID expected by
> dwfl_attach_state(). This allows libdw to unwind samples from every thread
> in a multi-threaded process.
>
> Fixes: 6b2658b3f36a ("perf unwind-libdw: Don't discard loaded ELF/DWARF after every unwind")
> Assisted-by: Codex:gpt-5.6-sol
> Signed-off-by: Alessio Podda <aleph.pi.gh@xxxxxxxxx>

This is great, thank you! Did you have a test that would reproduce
this problem? Could we add it to tools/perf/tests ?

Thanks,
Ian

> ---
>
> A standalone reproducer is available at:
>
>   https://github.com/kryggird/perf-bug-repro/tree/019c8ae97f9e4370ef2265d7badc3269e2a5044f
>
> Run it with the perf binary being tested:
>
>   ./reproduce.sh /path/to/perf
>
> Results:
>
>   perf 7.1.3-201.fc44.x86_64:
>   1517 of 1522 samples have no userspace callchain
>
>   patched perf 7.2.rc4.g1590cf032971:
>   15 of 1449 samples have no userspace callchain
>
>  tools/perf/util/unwind-libdw.c | 23 ++++++++++++++++++++---
>  1 file changed, 20 insertions(+), 3 deletions(-)
>
> diff --git a/tools/perf/util/unwind-libdw.c b/tools/perf/util/unwind-libdw.c
> index 7f35042be567..3735ef83c8c4 100644
> --- a/tools/perf/util/unwind-libdw.c
> +++ b/tools/perf/util/unwind-libdw.c
> @@ -173,14 +173,30 @@ static int entry(u64 ip, struct unwind_info *ui)
>         return 0;
>  }
>
> -static pid_t next_thread(Dwfl *dwfl, void *arg, void **thread_argp)
> +static pid_t next_thread(Dwfl *dwfl __maybe_unused, void *arg, void **thread_argp)
>  {
> +       struct dwfl_ui_thread_info *dwfl_ui_ti = arg;
> +
>         /* We want only single thread to be processed. */
>         if (*thread_argp != NULL)
>                 return 0;
>
> +       assert(dwfl_ui_ti->ui != NULL);
>         *thread_argp = arg;
> -       return dwfl_pid(dwfl);
> +       return thread__tid(dwfl_ui_ti->ui->thread);
> +}
> +
> +static bool get_thread(Dwfl *dwfl __maybe_unused, pid_t tid, void *arg,
> +                      void **thread_argp)
> +{
> +       struct dwfl_ui_thread_info *dwfl_ui_ti = arg;
> +
> +       assert(dwfl_ui_ti->ui != NULL);
> +       if (tid != thread__tid(dwfl_ui_ti->ui->thread))
> +               return false;
> +
> +       *thread_argp = arg;
> +       return true;
>  }
>
>  static int access_dso_mem(struct unwind_info *ui, Dwarf_Addr addr,
> @@ -306,6 +322,7 @@ static bool libdw_set_initial_registers(Dwfl_Thread *thread, void *arg)
>
>  static const Dwfl_Thread_Callbacks callbacks = {
>         .next_thread           = next_thread,
> +       .get_thread            = get_thread,
>         .memory_read           = memory_read,
>         .set_initial_registers = libdw_set_initial_registers,
>  };
> @@ -400,7 +417,7 @@ int libdw__get_entries(unwind_entry_cb_t cb, void *arg,
>         if (err)
>                 goto out;
>
> -       dwfl_attach_state(dwfl, /*elf=*/NULL, thread__tid(thread), &callbacks,
> +       dwfl_attach_state(dwfl, /*elf=*/NULL, thread__pid(thread), &callbacks,
>                           /* Dwfl thread function argument*/dwfl_ui_ti);
>         // Ignore thread already attached error.
>
>
> base-commit: b95f03f04d475aa6719d15a636ddf32222d55657
> --
> 2.55.0