Re: [PATCH] perf callchain: Handle multiple address spaces

[Namhyung Kim]

On Mon, May 04, 2026 at 11:56:15AM +0200, Thomas Richter wrote:
> On 4/30/26 04:57, Namhyung Kim wrote:
> > On Wed, Apr 29, 2026 at 09:14:38AM +0200, Thomas Richter wrote:
> >> On 4/21/26 18:30, Namhyung Kim wrote:
> >>> Hello,
> >>>
> >>> On Tue, Apr 14, 2026 at 02:42:41PM +0200, Thomas Richter wrote:
> >>>> perf test 'perf inject to convert DWARF callchains to regular ones'
> >>>> fails on s390. It was introduced with
> >>>> commit 92ea788d2af4 ("perf inject: Add --convert-callchain option")
> >>>>
> >>>> The failure comes the difference in output. Without the inject script to
> >>>> convert DWARF the callchains is:
> >>>>  # ./perf record -F 999 --call-graph dwarf -- perf test -w noploop
> >>>>  # ./perf report -i perf.data --stdio --no-children -q \
> >>>> 					 --percent-limit=1 > /tmp/111
> >>>>  # cat /tmp/111
> >>>>     99.30%  perf-noploop  perf               [.] noploop
> >>>>             |
> >>>>             ---noploop
> >>>>                run_workload (inlined)
> >>>>                cmd_test
> >>>>                run_builtin (inlined)
> >>>>                handle_internal_command
> >>>>                run_argv (inlined)
> >>>>                main
> >>>>                __libc_start_call_main
> >>>>                __libc_start_main_impl (inlined)
> >>>>                _start
> >>>>  #
> >>>>
> >>>> With the inject script step the output is:
> >>>>  # ./perf inject -i perf.data --convert-callchain -o /tmp/perf-inject-1.out
> >>>>  # ./perf report -i /tmp/perf-inject-1.out --stdio --no-children -q \
> >>>> 		--percent-limit=1 > /tmp/222
> >>>>  # cat /tmp/222
> >>>>     99.40%  perf-noploop  perf               [.] noploop
> >>>>             |
> >>>>             ---noploop
> >>>>                run_workload (inlined)
> >>>>                cmd_test
> >>>>                run_builtin (inlined)
> >>>>                handle_internal_command
> >>>>                run_argv (inlined)
> >>>>                main
> >>>>                _start
> >>>>  # diff /tmp/111 /tmp/222
> >>>>  1c1
> >>>>  <     99.30%  perf-noploop  perf               [.] noploop
> >>>>  ---
> >>>>  >     99.40%  perf-noploop  perf               [.] noploop
> >>>>  10,11d9
> >>>>  <                __libc_start_call_main
> >>>>  <                __libc_start_main_impl (inlined)
> >>>>  #
> >>>>
> >>>> The difference are the symbols __libc_start_call_main and
> >>>> __libc_start_main_impl.
> >>>>
> >>>> On x86_64, kernel and user space share a single virtual address space,
> >>>> with the kernel mapped to the upper end of memory. The instruction
> >>>> pointer value alone is sufficient to distinguish between user space
> >>>> and kernel space addresses. This is not true for s390, which uses
> >>>> separate address spaces for user and kernel. The same virtual address
> >>>> can be valid in both address spaces, so the instruction pointer value
> >>>> alone cannot determine whether an address belongs to the kernel or
> >>>> user space. Instead, perf must rely on the cpumode metadata derived
> >>>> from the processor status word (PSW) at sample time.
> >>>>
> >>>> In function perf_event__convert_sample_callchain() the first part
> >>>> copies a kernel callchain and context entries, if any.
> >>>> It then appends additional entries ignoring the address space
> >>>> architecture. Taking that into account, the symbols at addresses
> >>>>
> >>>>    0x3ff970348cb __libc_start_call_main
> >>>>    0x3ff970349c5 __libc_start_main_impl
> >>>>
> >>>> (located after the kernel address space on s390) are now included.
> >>>>
> >>>> Output before:
> >>>>  # ./perf test 83
> >>>>  83: perf inject to convert DWARF callchains to regular ones : FAILED!
> >>>>
> >>>> Output after:
> >>>>  # ./perf test 83
> >>>>  83: perf inject to convert DWARF callchains to regular ones : Ok
> >>>>
> >>>> Question to Namhyung:
> >>>> In function perf_event__convert_sample_callchain() just before the
> >>>> for() loop this patch modifies, the kernel callchain is copied,
> >>>> see this comment and the next 5 lines:
> >>>>    /* copy kernel callchain and context entries */ 
> >>>> Then why is machine__kernel_ip() needed in the for() loop, when
> >>>> the kernel entries have been copied just before the loop?
> >>>
> >>> IIRC I wanted to make sure to have PERF_CONTEXT_* part in the raw
> >>> callchains.
> >>>
> >>>>
> >>>> Note: This patch was tested on x86_64 virtual machine and succeeded.
> >>>>
> >>>> Fixes: 92ea788d2af4 ("perf inject: Add --convert-callchain option")
> >>>> Cc: Namhyung Kim <namhyung@xxxxxxxxxx>
> >>>>
> >>>> Signed-off-by: Thomas Richter <tmricht@xxxxxxxxxxxxx>
> >>>
> >>> Acked-by: Namhyung Kim <namhyung@xxxxxxxxxx>
> >>>
> >>> Thanks,
> >>> Namhyung
> >>>
> >>
> >> Hi Namhyung,
> >>
> >> is there something which needs to be done on my side to get this patch picked?
> >> It still fails on s390.
> >>
> >> Please adviseon how to proceed.
> > 
> > Arnaldo will take care of 7.2 development cycle.
> > 
> 
> Today we are at v7.1-rc2. Does this mean the fix won't make it into v7.1 at all
> and we have to wait roughly 2 month?

Just out of curiousity, do you plan to use v7.1 for this feature
specifically or just for test failures?  Probably better to use v7.2
series (i.e. perf-tools-next) instead?

Anyway, I can merge this to v7.1 if you really want as it's from v7.0
which may be close enough for urgent care. :)  But it seems there would
be conflicts with Ian's e_machine patchset.

Thanks,
Namhyung