RE: [RFC] perf: probe_finder: continue if atleast one probe point found

[平松雅巳 / HIRAMATU，MASAMI]

BTW,

./perf probe  \
  -k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc $params'

might help your case. $params is expanded to function parameters automatically
and if there is no parameters, it is just ignored :)

Thank you,

>From: 平松雅巳 / HIRAMATU，MASAMI [mailto:masami.hiramatsu.pt@xxxxxxxxxxx]
>
>Hi,
>
>>From: Arnaldo Carvalho de Melo [mailto:acme@xxxxxxxxxx]
>>
>>Em Sun, Feb 28, 2016 at 03:49:44AM -0800, Joel Fernandes escreveu:
>>> Taeung,
>>>
>>> Thanks for that. As this is an RFC, I have added them now to this email
>>> thread and when I post the final patch, I'll include them in CC.
>>>
>>> All,
>>> Please provide your comments on my patch.
>>
>>Masami, are you ok with this patch?
>
>Hmm, I think this is not enough. I concider that the case if users
>expect probing on "__malloc_get_state+11", just ignoring it may
>confuse or mislead them that perf-probe has a bug not to find it.
>
>IOW, this was by design that perf probe refuses all probe point
>if it hits any error.
>
>I think we have 2 options, one is adding an option (or reuse --force)
>to ignore error on finding probe points (but this MUST show the error
>message by pr_error.) The other is extending perf probe's argument
>format so that it can accept local-vars which may not exist, e.g.
>'malloc *bytes' or 'malloc {bytes,}' :)
>
>Thank you,
>
>
>>
>>- Arnaldo
>>
>>> Thanks,
>>> Joel
>>>
>>> On Sun, Feb 28, 2016 at 3:35 AM, Taeung Song <treeze.taeung@xxxxxxxxx>
>>> wrote:
>>>
>>> > Hi, Joel
>>> >
>>> > perf subsystem's maintainers are
>>> > Arnaldo Carvalho de Melo <acme@xxxxxxxxxx>
>>> > Peter Zijlstra <peterz@xxxxxxxxxxxxx>
>>> > Ingo Molnar <mingo@xxxxxxxxxx>
>>> >
>>> > (or Namhyung Kim <namhyung@xxxxxxxxxx>, Jiri Olsa <jolsa@xxxxxxxxxx>)
>>> >
>>> > AFAIK,
>>> > When sending some patches, it would be better to add them into To: or Cc:
>>> >
>>> >
>>> > Thanks,
>>> > Taeung
>>> >
>>> >
>>> > On 02/28/2016 08:14 PM, Joel Fernandes wrote:
>>> >
>>> >> Sometimes for inline functions, perf probe can fail such as if an
>>> >> arguments are
>>> >> requested at probe points. This is probably because when the compiler
>>> >> inlines,
>>> >> for some instances it optimizes away arguments. Either way, the DWARF has
>>> >> missing arguments for certain probe points of inline functions causing
>>> >> 'perf
>>> >> probe' to fail. I noticed this when probing the C library that ships with
>>> >> my
>>> >> distribution. With the following patch I am successfully able to record
>>> >> kprobe
>>> >> events with arguments.
>>> >>
>>> >> Test
>>> >> ----
>>> >>   ./perf probe \
>>> >>     -k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc
>>> >> bytes' -v
>>> >>
>>> >> Without the patch
>>> >> -----------------
>>> >> Matched function: __libc_malloc
>>> >> found inline addr: 0x831a6
>>> >> Probe point found: malloc_atfork+150
>>> >> Searching 'bytes' variable in context.
>>> >> Converting variable bytes into trace event.
>>> >> bytes type is long unsigned int.
>>> >> found inline addr: 0x844a0
>>> >> Probe point found: __libc_malloc+0
>>> >> Searching 'bytes' variable in context.
>>> >> Converting variable bytes into trace event.
>>> >> bytes type is long unsigned int.
>>> >> found inline addr: 0x8463b
>>> >> Probe point found: __malloc_get_state+11
>>> >> Searching 'bytes' variable in context.
>>> >> Failed to find 'bytes' in this function.
>>> >> An error occurred in debuginfo analysis (-2).
>>> >>    Error: Failed to add events. Reason: No such file or directory (Code:
>>> >> -2)
>>> >>
>>> >> With the patch
>>> >> --------------
>>> >> Open Debuginfo file: /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.21.so
>>> >> Try to find probe point from debuginfo.
>>> >> Symbol malloc address found : 844a0
>>> >> Matched function: __libc_malloc
>>> >> found inline addr: 0x831a6
>>> >> Probe point found: malloc_atfork+150
>>> >> Searching 'bytes' variable in context.
>>> >> Converting variable bytes into trace event.
>>> >> bytes type is long unsigned int.
>>> >> found inline addr: 0x844a0
>>> >> Probe point found: __libc_malloc+0
>>> >> Searching 'bytes' variable in context.
>>> >> Converting variable bytes into trace event.
>>> >> bytes type is long unsigned int.
>>> >> found inline addr: 0x8463b
>>> >> Probe point found: __malloc_get_state+11
>>> >> Searching 'bytes' variable in context.
>>> >> Failed to find 'bytes' in this function.
>>> >> Probe point error, ignoring. Atleast one probe point found
>>> >> found inline addr: 0x84bba
>>> >> Probe point found: __libc_realloc+410
>>> >> Searching 'bytes' variable in context.
>>> >> Failed to find 'bytes' in this function.
>>> >> Probe point error, ignoring. Atleast one probe point found
>>> >> found inline addr: 0x84d08
>>> >> ...
>>> >> Probe point error, ignoring. Atleast one probe point found
>>> >> Found 3 probe_trace_events.
>>> >> Opening /sys/kernel/debug/tracing//uprobe_events write=1
>>> >> Writing event: p:probe_libc/malloc /lib/x86_64-linux-gnu/libc-2.21.so:0x831a6
>>> >> bytes=%bp:u64
>>> >> Writing event: p:probe_libc/malloc_1 /lib/x86_64-linux-gnu/libc-2.21.so:0x844a0
>>> >> bytes=%di:u64
>>> >> Writing event: p:probe_libc/malloc_2 /lib/x86_64-linux-gnu/libc-2.21.so:0x85af7
>>> >> bytes=%di:u64
>>> >> Added new events:
>>> >>    probe_libc:malloc    (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>>> >> with bytes)
>>> >>    probe_libc:malloc_1  (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>>> >> with bytes)
>>> >>    probe_libc:malloc_2  (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>>> >> with bytes)
>>> >>
>>> >> Signed-off-by: Joel Fernandes <agnel.joel@xxxxxxxxx>
>>> >> ---
>>> >>   tools/perf/util/probe-finder.c | 4 ++++
>>> >>   1 file changed, 4 insertions(+)
>>> >>
>>> >> diff --git a/tools/perf/util/probe-finder.c
>>> >> b/tools/perf/util/probe-finder.c
>>> >> index 4ce5c5e..3ac9481 100644
>>> >> --- a/tools/perf/util/probe-finder.c
>>> >> +++ b/tools/perf/util/probe-finder.c
>>> >> @@ -1255,6 +1255,10 @@ end:
>>> >>         if (ret) {
>>> >>                 clear_probe_trace_event(tev);
>>> >>                 tf->ntevs--;
>>> >> +               if (tf->ntevs != 0) {
>>> >> +                       pr_debug("Ignoring error as atleast one probe
>>> >> point found.\n");
>>> >> +                       ret = 0;
>>> >> +               }
>>> >>         }
>>> >>         free(args);
>>> >>         return ret;
>>> >>
>>> >>