Re: [PATCH] perf callchain: Handle multiple address spaces

[Thomas Richter]

On 5/5/26 01:48, Namhyung Kim wrote:
> On Mon, May 04, 2026 at 11:56:15AM +0200, Thomas Richter wrote:
>> On 4/30/26 04:57, Namhyung Kim wrote:
>>> On Wed, Apr 29, 2026 at 09:14:38AM +0200, Thomas Richter wrote:
>>>> On 4/21/26 18:30, Namhyung Kim wrote:
>>>>> Hello,
>>>>>
>>>>> On Tue, Apr 14, 2026 at 02:42:41PM +0200, Thomas Richter wrote:
>>>>>> perf test 'perf inject to convert DWARF callchains to regular ones'
>>>>>> fails on s390. It was introduced with
>>>>>> commit 92ea788d2af4 ("perf inject: Add --convert-callchain option")
>>>>>>
>>>>>> The failure comes the difference in output. Without the inject script to
>>>>>> convert DWARF the callchains is:
>>>>>>  # ./perf record -F 999 --call-graph dwarf -- perf test -w noploop
>>>>>>  # ./perf report -i perf.data --stdio --no-children -q \
>>>>>> 					 --percent-limit=1 > /tmp/111
>>>>>>  # cat /tmp/111
>>>>>>     99.30%  perf-noploop  perf               [.] noploop
>>>>>>             |
>>>>>>             ---noploop
>>>>>>                run_workload (inlined)
>>>>>>                cmd_test
>>>>>>                run_builtin (inlined)
>>>>>>                handle_internal_command
>>>>>>                run_argv (inlined)
>>>>>>                main
>>>>>>                __libc_start_call_main
>>>>>>                __libc_start_main_impl (inlined)
>>>>>>                _start
>>>>>>  #
>>>>>>
>>>>>> With the inject script step the output is:
>>>>>>  # ./perf inject -i perf.data --convert-callchain -o /tmp/perf-inject-1.out
>>>>>>  # ./perf report -i /tmp/perf-inject-1.out --stdio --no-children -q \
>>>>>> 		--percent-limit=1 > /tmp/222
>>>>>>  # cat /tmp/222
>>>>>>     99.40%  perf-noploop  perf               [.] noploop
>>>>>>             |
>>>>>>             ---noploop
>>>>>>                run_workload (inlined)
>>>>>>                cmd_test
>>>>>>                run_builtin (inlined)
>>>>>>                handle_internal_command
>>>>>>                run_argv (inlined)
>>>>>>                main
>>>>>>                _start
>>>>>>  # diff /tmp/111 /tmp/222
>>>>>>  1c1
>>>>>>  <     99.30%  perf-noploop  perf               [.] noploop
>>>>>>  ---
>>>>>>  >     99.40%  perf-noploop  perf               [.] noploop
>>>>>>  10,11d9
>>>>>>  <                __libc_start_call_main
>>>>>>  <                __libc_start_main_impl (inlined)
>>>>>>  #
>>>>>>
>>>>>> The difference are the symbols __libc_start_call_main and
>>>>>> __libc_start_main_impl.
>>>>>>
>>>>>> On x86_64, kernel and user space share a single virtual address space,
>>>>>> with the kernel mapped to the upper end of memory. The instruction
>>>>>> pointer value alone is sufficient to distinguish between user space
>>>>>> and kernel space addresses. This is not true for s390, which uses
>>>>>> separate address spaces for user and kernel. The same virtual address
>>>>>> can be valid in both address spaces, so the instruction pointer value
>>>>>> alone cannot determine whether an address belongs to the kernel or
>>>>>> user space. Instead, perf must rely on the cpumode metadata derived
>>>>>> from the processor status word (PSW) at sample time.
>>>>>>
>>>>>> In function perf_event__convert_sample_callchain() the first part
>>>>>> copies a kernel callchain and context entries, if any.
>>>>>> It then appends additional entries ignoring the address space
>>>>>> architecture. Taking that into account, the symbols at addresses
>>>>>>
>>>>>>    0x3ff970348cb __libc_start_call_main
>>>>>>    0x3ff970349c5 __libc_start_main_impl
>>>>>>
>>>>>> (located after the kernel address space on s390) are now included.
>>>>>>
>>>>>> Output before:
>>>>>>  # ./perf test 83
>>>>>>  83: perf inject to convert DWARF callchains to regular ones : FAILED!
>>>>>>
>>>>>> Output after:
>>>>>>  # ./perf test 83
>>>>>>  83: perf inject to convert DWARF callchains to regular ones : Ok
>>>>>>
>>>>>> Question to Namhyung:
>>>>>> In function perf_event__convert_sample_callchain() just before the
>>>>>> for() loop this patch modifies, the kernel callchain is copied,
>>>>>> see this comment and the next 5 lines:
>>>>>>    /* copy kernel callchain and context entries */ 
>>>>>> Then why is machine__kernel_ip() needed in the for() loop, when
>>>>>> the kernel entries have been copied just before the loop?
>>>>>
>>>>> IIRC I wanted to make sure to have PERF_CONTEXT_* part in the raw
>>>>> callchains.
>>>>>
>>>>>>
>>>>>> Note: This patch was tested on x86_64 virtual machine and succeeded.
>>>>>>
>>>>>> Fixes: 92ea788d2af4 ("perf inject: Add --convert-callchain option")
>>>>>> Cc: Namhyung Kim <namhyung@xxxxxxxxxx>
>>>>>>
>>>>>> Signed-off-by: Thomas Richter <tmricht@xxxxxxxxxxxxx>
>>>>>
>>>>> Acked-by: Namhyung Kim <namhyung@xxxxxxxxxx>
>>>>>
>>>>> Thanks,
>>>>> Namhyung
>>>>>
>>>>
>>>> Hi Namhyung,
>>>>
>>>> is there something which needs to be done on my side to get this patch picked?
>>>> It still fails on s390.
>>>>
>>>> Please adviseon how to proceed.
>>>
>>> Arnaldo will take care of 7.2 development cycle.
>>>
>>
>> Today we are at v7.1-rc2. Does this mean the fix won't make it into v7.1 at all
>> and we have to wait roughly 2 month?
> 
> Just out of curiousity, do you plan to use v7.1 for this feature
> specifically or just for test failures?  Probably better to use v7.2
> series (i.e. perf-tools-next) instead?

Right now I just care for failing test features. I am fine with 7.2
as long as it gets picked.

Thanks.
-- 
Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany
--
IBM Deutschland Research & Development GmbH

Vorsitzender des Aufsichtsrats: Wolfgang Wendt

Geschäftsführung: David Faller

Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294