RE: [RFC] perf: probe_finder: continue if atleast one probe point found

[平松雅巳 / HIRAMATU，MASAMI]

Hi,

>From: Arnaldo Carvalho de Melo [mailto:acme@xxxxxxxxxx]
>
>Em Sun, Feb 28, 2016 at 03:49:44AM -0800, Joel Fernandes escreveu:
>> Taeung,
>>
>> Thanks for that. As this is an RFC, I have added them now to this email
>> thread and when I post the final patch, I'll include them in CC.
>>
>> All,
>> Please provide your comments on my patch.
>
>Masami, are you ok with this patch?

Hmm, I think this is not enough. I concider that the case if users
expect probing on "__malloc_get_state+11", just ignoring it may
confuse or mislead them that perf-probe has a bug not to find it.

IOW, this was by design that perf probe refuses all probe point
if it hits any error.

I think we have 2 options, one is adding an option (or reuse --force)
to ignore error on finding probe points (but this MUST show the error
message by pr_error.) The other is extending perf probe's argument
format so that it can accept local-vars which may not exist, e.g.
'malloc *bytes' or 'malloc {bytes,}' :)

Thank you,


>
>- Arnaldo
>
>> Thanks,
>> Joel
>>
>> On Sun, Feb 28, 2016 at 3:35 AM, Taeung Song <treeze.taeung@xxxxxxxxx>
>> wrote:
>>
>> > Hi, Joel
>> >
>> > perf subsystem's maintainers are
>> > Arnaldo Carvalho de Melo <acme@xxxxxxxxxx>
>> > Peter Zijlstra <peterz@xxxxxxxxxxxxx>
>> > Ingo Molnar <mingo@xxxxxxxxxx>
>> >
>> > (or Namhyung Kim <namhyung@xxxxxxxxxx>, Jiri Olsa <jolsa@xxxxxxxxxx>)
>> >
>> > AFAIK,
>> > When sending some patches, it would be better to add them into To: or Cc:
>> >
>> >
>> > Thanks,
>> > Taeung
>> >
>> >
>> > On 02/28/2016 08:14 PM, Joel Fernandes wrote:
>> >
>> >> Sometimes for inline functions, perf probe can fail such as if an
>> >> arguments are
>> >> requested at probe points. This is probably because when the compiler
>> >> inlines,
>> >> for some instances it optimizes away arguments. Either way, the DWARF has
>> >> missing arguments for certain probe points of inline functions causing
>> >> 'perf
>> >> probe' to fail. I noticed this when probing the C library that ships with
>> >> my
>> >> distribution. With the following patch I am successfully able to record
>> >> kprobe
>> >> events with arguments.
>> >>
>> >> Test
>> >> ----
>> >>   ./perf probe \
>> >>     -k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc
>> >> bytes' -v
>> >>
>> >> Without the patch
>> >> -----------------
>> >> Matched function: __libc_malloc
>> >> found inline addr: 0x831a6
>> >> Probe point found: malloc_atfork+150
>> >> Searching 'bytes' variable in context.
>> >> Converting variable bytes into trace event.
>> >> bytes type is long unsigned int.
>> >> found inline addr: 0x844a0
>> >> Probe point found: __libc_malloc+0
>> >> Searching 'bytes' variable in context.
>> >> Converting variable bytes into trace event.
>> >> bytes type is long unsigned int.
>> >> found inline addr: 0x8463b
>> >> Probe point found: __malloc_get_state+11
>> >> Searching 'bytes' variable in context.
>> >> Failed to find 'bytes' in this function.
>> >> An error occurred in debuginfo analysis (-2).
>> >>    Error: Failed to add events. Reason: No such file or directory (Code:
>> >> -2)
>> >>
>> >> With the patch
>> >> --------------
>> >> Open Debuginfo file: /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.21.so
>> >> Try to find probe point from debuginfo.
>> >> Symbol malloc address found : 844a0
>> >> Matched function: __libc_malloc
>> >> found inline addr: 0x831a6
>> >> Probe point found: malloc_atfork+150
>> >> Searching 'bytes' variable in context.
>> >> Converting variable bytes into trace event.
>> >> bytes type is long unsigned int.
>> >> found inline addr: 0x844a0
>> >> Probe point found: __libc_malloc+0
>> >> Searching 'bytes' variable in context.
>> >> Converting variable bytes into trace event.
>> >> bytes type is long unsigned int.
>> >> found inline addr: 0x8463b
>> >> Probe point found: __malloc_get_state+11
>> >> Searching 'bytes' variable in context.
>> >> Failed to find 'bytes' in this function.
>> >> Probe point error, ignoring. Atleast one probe point found
>> >> found inline addr: 0x84bba
>> >> Probe point found: __libc_realloc+410
>> >> Searching 'bytes' variable in context.
>> >> Failed to find 'bytes' in this function.
>> >> Probe point error, ignoring. Atleast one probe point found
>> >> found inline addr: 0x84d08
>> >> ...
>> >> Probe point error, ignoring. Atleast one probe point found
>> >> Found 3 probe_trace_events.
>> >> Opening /sys/kernel/debug/tracing//uprobe_events write=1
>> >> Writing event: p:probe_libc/malloc /lib/x86_64-linux-gnu/libc-2.21.so:0x831a6
>> >> bytes=%bp:u64
>> >> Writing event: p:probe_libc/malloc_1 /lib/x86_64-linux-gnu/libc-2.21.so:0x844a0
>> >> bytes=%di:u64
>> >> Writing event: p:probe_libc/malloc_2 /lib/x86_64-linux-gnu/libc-2.21.so:0x85af7
>> >> bytes=%di:u64
>> >> Added new events:
>> >>    probe_libc:malloc    (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>> >> with bytes)
>> >>    probe_libc:malloc_1  (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>> >> with bytes)
>> >>    probe_libc:malloc_2  (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>> >> with bytes)
>> >>
>> >> Signed-off-by: Joel Fernandes <agnel.joel@xxxxxxxxx>
>> >> ---
>> >>   tools/perf/util/probe-finder.c | 4 ++++
>> >>   1 file changed, 4 insertions(+)
>> >>
>> >> diff --git a/tools/perf/util/probe-finder.c
>> >> b/tools/perf/util/probe-finder.c
>> >> index 4ce5c5e..3ac9481 100644
>> >> --- a/tools/perf/util/probe-finder.c
>> >> +++ b/tools/perf/util/probe-finder.c
>> >> @@ -1255,6 +1255,10 @@ end:
>> >>         if (ret) {
>> >>                 clear_probe_trace_event(tev);
>> >>                 tf->ntevs--;
>> >> +               if (tf->ntevs != 0) {
>> >> +                       pr_debug("Ignoring error as atleast one probe
>> >> point found.\n");
>> >> +                       ret = 0;
>> >> +               }
>> >>         }
>> >>         free(args);
>> >>         return ret;
>> >>
>> >>