Re: [PATCH 1/2] perf tools: Fix seg fault with Intel PT

[Adrian Hunter]

On 26/01/16 15:54, Arnaldo Carvalho de Melo wrote:
> Em Tue, Jan 26, 2016 at 03:34:15PM +0200, Adrian Hunter escreveu:
>> On 26/01/16 15:23, Arnaldo Carvalho de Melo wrote:
>>> Em Tue, Jan 26, 2016 at 02:05:20PM +0200, Adrian Hunter escreveu:
>>>> Fix segmentation fault using:
>>>>
>>>> 	perf record -e intel_pt//u uname
>>>>
>>>> Back trace:
>>>>
>>>>   0  tracepoint_error (err=13, err@entry=-13, sys=sys@entry=0x18706c0 "sched", name=name@entry=0x1871c70 "sched_switch", e=<optimized out>, e=<optimized out>) at util/parse-events.c:416
>>>>   1  0x000000000049791c in add_tracepoint (head_config=0x0, err=0x0, evt_name=0x1871c70 "sched_switch", sys_name=0x18706c0 "sched", idx=<optimized out>, list=<optimized out>)
>>>>      at util/parse-events.c:433
>>>>   2  add_tracepoint_event (list=<optimized out>, idx=<optimized out>, sys_name=0x18706c0 "sched", evt_name=0x1871c70 "sched_switch", err=0x0, head_config=0x0) at util/parse-events.c:498
>>>>   3  0x0000000000498ec4 in parse_events_add_tracepoint (list=list@entry=0x1871c90, idx=idx@entry=0x7fffffffba30, sys=0x18706c0 "sched", event=0x1871c70 "sched_switch", err=err@entry=0x0,
>>>>      head_config=head_config@entry=0x0) at util/parse-events.c:936
>>>>   4  0x00000000004ccff2 in parse_events_parse (_data=_data@entry=0x7fffffffba20, scanner=0x18705f0) at util/parse-events.y:391
>>>>   5  0x000000000049a567 in parse_events__scanner (start_token=258, data=0x7fffffffba20, str=0x0) at util/parse-events.c:1361
>>>>   6  parse_events (evlist=evlist@entry=0x1871150, str=str@entry=0x55ef6c "sched:sched_switch", err=err@entry=0x0) at util/parse-events.c:1401
>>>>   7  0x00000000004eb4b6 in perf_evlist__can_select_event (evlist=evlist@entry=0x186fb90, str=str@entry=0x55ef6c "sched:sched_switch") at util/record.c:253
>>>>   8  0x000000000051e1d3 in intel_pt_track_switches (evlist=0x186fb90) at arch/x86/util/intel-pt.c:362
>>>>   9  intel_pt_recording_options (itr=0x186ef70, evlist=0x186fb90, opts=0x7c3e08 <record+232>) at arch/x86/util/intel-pt.c:662
>>>>  10  0x000000000042e244 in cmd_record (argc=1, argv=0x7fffffffe270, prefix=<optimized out>) at builtin-record.c:1260
>>>>  11  0x000000000047cbd3 in run_builtin (p=p@entry=0x7cf2e8 <commands+168>, argc=argc@entry=4, argv=argv@entry=0x7fffffffe270) at perf.c:390
>>>>  12  0x00000000004216a7 in handle_internal_command (argv=0x7fffffffe270, argc=4) at perf.c:451
>>>>  13  run_argv (argv=0x7fffffffdff0, argcp=0x7fffffffdffc) at perf.c:495
>>>>  14  main (argc=4, argv=0x7fffffffe270) at perf.c:618
>>>>
>>>> Intel PT attempts to find the sched:sched_switch tracepoint but that
>>>> seg faults if tracefs is not readable, because the error reporting
>>>
>>> How can I reproduce this problem? I tried your command but that doesn't cause
>>> any problem:
>>>
>>> [acme@jouet linux]$ ls -la /sys/kernel/debug/tracing/
>>> ls: cannot access /sys/kernel/debug/tracing/: Permission denied
>>> [acme@jouet linux]$ ls -la /sys/kernel/debug/
>>> ls: cannot open directory /sys/kernel/debug/: Permission denied
>>> [acme@jouet linux]$ perf record -e intel_pt//u uname
>>> Linux
>>> [ perf record: Woken up 1 times to write data ]
>>> [ perf record: Captured and wrote 0.041 MB perf.data ]
>>> [acme@jouet linux]$ perf list intel
>>>
>>> List of pre-defined events (to be used in -e):
>>>
>>>   intel_bts//                                        [Kernel PMU event]
>>>   intel_pt//                                         [Kernel PMU event]
>>>
>>> [acme@jouet linux]$ perf record -e intel_pt//u uname
>>> Linux
>>> [ perf record: Woken up 1 times to write data ]
>>> [ perf record: Captured and wrote 0.041 MB perf.data ]
>>> [acme@jouet linux]$ 
>>>
>>> [acme@jouet linux]$ perf evlist
>>> intel_pt//u
>>> dummy:u
>>> [acme@jouet linux]$ 
>>>
>>> [acme@jouet linux]$ perf evlist -v
>>> intel_pt//u: type: 6, size: 112, config: 0x400, { sample_period, sample_freq }: 1, sample_type: IP|TID|TIME|CPU|IDENTIFIER, read_format: ID, disabled: 1, inherit: 1, exclude_kernel: 1, exclude_hv: 1, enable_on_exec: 1, sample_id_all: 1
>>> dummy:u: type: 1, size: 112, config: 0x9, { sample_period, sample_freq }: 1, sample_type: IP|TID|TIME|CPU|IDENTIFIER, read_format: ID, disabled: 1, inherit: 1, exclude_kernel: 1, exclude_hv: 1, mmap: 1, comm: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, mmap2: 1, comm_exec: 1, context_switch: 1
>>> [acme@jouet linux]$ 
>>>
>>> Humm, ok, so this happens only if tracefs is not readable _and_ this is an
>>> older kernel where perf_event_attr.context_switch is not available?
>>
>> Yes, I should have thought of that, sorry.
> 
> So, I just tried on another machine, with an older kernel
>  
> [acme@ssdandy linux]$ uname -r
> 3.10.0-229.14.1.el7.x86_64
> [acme@ssdandy linux]$ ls -la /sys/kernel/debug/
> ls: cannot open directory /sys/kernel/debug/: Permission denied
> [acme@ssdandy linux]$ 
> [acme@ssdandy linux]$ grep tracefs /proc/filesystems 
> [acme@ssdandy linux]$ grep debugfs /proc/filesystems 
> nodev	debugfs
> [acme@ssdandy linux]$ ls -la /sys/kernel/debug/
> ls: cannot open directory /sys/kernel/debug/: Permission denied
> [acme@ssdandy linux]$ perf record -e intel_pt//u uname
> invalid or unsupported event: 'intel_pt//u'
> Run 'perf list' for a list of valid events
> 
>  Usage: perf record [<options>] [<command>]
>     or: perf record [<options>] -- <command> [<options>]
> 
>     -e, --event <event>   event selector. use 'perf list' to list available events
> [acme@ssdandy linux]$ 
> 
> So, this needs:
> 
> 1) A machine that supports Intel PT
> 2) A kernel that supports Intel PT
> 3) A kernel that doesn't have perf_event_attr.context_switch, so that it tries
>    to use sched:sched_switch
> 4) an unreadable tracefs
> 
> What was the kernel where you stumbled on this problem? What machine?

Kernel was v4.2, machine was Broadwell.

context_switch was added in v4.3 I think, and PT was v4.1
so I guess v4.1 and v4.2 are affected.

> 
> - Arnaldo
> 
>>> /me tries booting another machine, with an older kernel...
>>>
>>>> structure is null, as errors are not reported when automatically
>>>> adding tracepoints.  Fix by checking before using.
>>>>
>>>> Signed-off-by: Adrian Hunter <adrian.hunter@xxxxxxxxx>
>>>> Fixes: 196581717d85 ("perf tools: Enhance parsing events tracepoint error output")
>>>> Cc: stable@xxxxxxxxxxxxxxx # v4.4+
>>>> ---
>>>>  tools/perf/util/parse-events.c | 3 +++
>>>>  1 file changed, 3 insertions(+)
>>>>
>>>> diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
>>>> index 4f7b0efdde2f..813d9b272c81 100644
>>>> --- a/tools/perf/util/parse-events.c
>>>> +++ b/tools/perf/util/parse-events.c
>>>> @@ -399,6 +399,9 @@ static void tracepoint_error(struct parse_events_error *e, int err,
>>>>  {
>>>>  	char help[BUFSIZ];
>>>>  
>>>> +	if (!e)
>>>> +		return;
>>>> +
>>>>  	/*
>>>>  	 * We get error directly from syscall errno ( > 0),
>>>>  	 * or from encoded pointer's error ( < 0).
>>>> -- 
>>>> 1.9.1
>>>
>