Re: Re: [PATCH perf/core v2 0/4] perf-probe code cleanup and support wildcard for probe points

From: Masami Hiramatsu
Date: Sat May 09 2015 - 23:03:01 EST


On 2015/05/09 4:38, Arnaldo Carvalho de Melo wrote:
> Em Fri, May 08, 2015 at 10:03:26AM +0900, Masami Hiramatsu escreveu:
>> Hi,
>>
>> Here is a patches for wild card support. I've added two patches to
>> reduce API parameters by introducing probe_conf. This simplifies
>> --no-inlines option. (Thanks Arnaldo!)
>>
>> Changes from v1:
>> - Update for the latest perf/core
>> - Use perf_probe_event.target instead of passing it as an independent
>> parameter [1/4]
>> - Introduce probe_conf for configuring parameters of probe-finder and
>> probe-event [2/4]
>>
>> Thank you,
>>
>> ---
>>
>> Masami Hiramatsu (4):
>> perf probe: Use perf_probe_event.target instead of passing as an argument
>> perf probe: Introduce probe_conf global configs
>> perf-probe: Add --no-inlines option to avoid searching inline functions
>> perf-probe: Support glob wildcards for function name
>
> Excellent stuff! Thanks, all tested.

Thank you for this good example! :)
Using this with perf-trace is certainly nice usage.

Thank you!

>
> And here is the example I used to test it plus other goodies:
>
> System wide syscall tracing + kmalloc* kernel functions with its parameters,
> filtering the pid of the sshd session to the test machine, to avoid a feedback
> loop:
>
> [root@ssdandy ~]# perf probe --no-inlines --add 'kmalloc* $params'
> Added new events:
> probe:kmalloc_order (on kmalloc* with $params)
> probe:kmalloc_order_trace (on kmalloc* with $params)
> probe:kmalloc_slab (on kmalloc* with $params)
> probe:kmalloc_large_node (on kmalloc* with $params)
>
> You can now use it in all perf tools, such as:
>
> perf record -e probe:kmalloc_large_node -aR sleep 1
>
> [root@ssdandy ~]# perf trace --ev probe:kmalloc* --filter-pids 27206
> 0.000 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 0.010 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 0.343 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x98 flags=0x10)
> 73.254 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 73.259 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 89.510 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 192.295 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 205.520 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 205.524 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 260.034 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 260.039 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220)
> 307.779 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 307.784 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 362.450 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 362.455 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220)
> 380.507 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 380.511 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 409.663 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 409.668 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 464.868 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 464.874 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220)
> 499.303 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 512.083 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 512.088 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 585.349 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 585.354 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 614.513 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 614.518 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 669.706 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 669.710 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220)
> 716.927 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 716.932 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 790.181 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 790.185 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 818.857 ( 0.000 ms): tuned/837 ... [continued]: select()) = 0 Timeout
> 819.568 ( 0.651 ms): tuned/837 select(tvp: 0x7fb3bb1f9e50 ) ...
> 819.568 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 819.573 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 879.453 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 879.458 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220)
> 922.236 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 924.408 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 924.412 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1024.172 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1024.177 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1079.381 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1079.386 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220)
> 1097.435 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1097.439 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1115.571 ( 0.000 ms): avahi-daemon/745 ... [continued]: poll()) = 0 Timeout
> 1115.592 ( 0.006 ms): avahi-daemon/745 write(fd: 7<pipe:[10912]>, buf: 0x7ffc066e35bf, count: 1) = 1
> 1115.596 ( 0.002 ms): avahi-daemon/745 write(fd: 7<pipe:[10912]>, buf: 0x7ffc066e35bf, count: 1) = 1
> 1115.602 ( 0.003 ms): avahi-daemon/745 read(fd: 6<pipe:[10912]>, buf: 0x7ffc066e35e0, count: 10) = 2
> 1115.607 ( 0.003 ms): avahi-daemon/745 poll(ufds: 0xaffa40, nfds: 8 ) = 0 Timeout
> 1115.612 ( 0.002 ms): avahi-daemon/745 write(fd: 7<pipe:[10912]>, buf: 0x7ffc066e35bf, count: 1) = 1
> 1115.616 ( 0.002 ms): avahi-daemon/745 write(fd: 7<pipe:[10912]>, buf: 0x7ffc066e35bf, count: 1) = 1
> 1115.620 ( 0.002 ms): avahi-daemon/745 read(fd: 6<pipe:[10912]>, buf: 0x7ffc066e35e0, count: 10) = 2
> 1126.606 (10.985 ms): avahi-daemon/745 poll(ufds: 0xaffa40, nfds: 8, timeout_msecs: 647 ) ...
> 1126.606 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1126.612 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1199.853 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1199.858 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1216.042 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1229.103 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1229.107 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1302.275 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1302.279 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1318.900 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1331.434 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1331.438 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1360.755 ( 0.000 ms): lsmd/746 ... [continued]: select()) = 0 Timeout
> 1360.764 ( 0.003 ms): lsmd/746 waitid(infop: 0x7ffeee2cdc90, options: 5 ) = -10
> 1404.691 (43.920 ms): lsmd/746 select(n: 6, inp: 0x7ffeee2cdd50, tvp: 0x7ffeee2cdd40 ) ...
> 1404.691 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1404.696 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1434.623 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1434.628 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1523.298 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1536.268 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1536.273 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1609.524 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1609.528 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1638.684 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1638.688 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1711.943 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1711.948 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1741.098 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1741.102 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1763.390 (647.769 ms): avahi-daemon/745 ... [continued]: poll()) = 0 Timeout
> 1763.401 ( 0.003 ms): avahi-daemon/745 write(fd: 7<pipe:[10912]>, buf: 0x7ffc066e35bf, count: 1) = 1
> 1763.404 ( 0.002 ms): avahi-daemon/745 write(fd: 7<pipe:[10912]>, buf: 0x7ffc066e35bf, count: 1) = 1
> 1763.408 ( 0.002 ms): avahi-daemon/745 read(fd: 6<pipe:[10912]>, buf: 0x7ffc066e35e0, count: 10) = 2
> 1814.362 (50.952 ms): avahi-daemon/745 poll(ufds: 0xaffa40, nfds: 8, timeout_msecs: 204419 ) ...
> 1814.362 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1814.367 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1820.100 (1001.183 ms): tuned/837 ... [continued]: select()) = 0 Timeout
> 1843.529 (23.407 ms): tuned/837 select(tvp: 0x7fb3bb1f9e50 ) ...
> 1843.529 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> 1843.533 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220)
> 1945.941 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220)
> ^C
> [root@ssdandy ~]# perf probe -d '*'
> Removed event: probe:kmalloc_large_node
> Removed event: probe:kmalloc_order
> Removed event: probe:kmalloc_order_trace
> Removed event: probe:kmalloc_slab
> [root@ssdandy ~]#
>
>
> :-)
>
> - Arnaldo
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>


--
Masami HIRAMATSU
Linux Technology Research Center, System Productivity Research Dept.
Center for Technology Innovation - Systems Engineering
Hitachi, Ltd., Research & Development Group
E-mail: masami.hiramatsu.pt@xxxxxxxxxxx
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/