Re: [RFC/PATCHSET 0/5] perf ftrace: Implement function latency histogram (v1)
From: Arnaldo Carvalho de Melo
Date: Tue Nov 30 2021 - 09:37:33 EST
Em Mon, Nov 29, 2021 at 03:18:25PM -0800, Namhyung Kim escreveu:
> Hello,
>
> I've implemented 'latency' subcommand in the perf ftrace command to
> show a histogram of function latency.
>
> To handle new subcommands, the existing functionality is moved to
> 'trace' subcommand while preserving backward compatibility of not
> having a subcommand at all (defaults to 'trace').
>
> The latency subcommand accepts a target (kernel, for now) function
> with -T option and shows a histogram like below:
Humm, wouldn't be interesting to shorten this by having a new 'perf
flat' (function latency) tool, on the same level as 'perf ftrace' and
leave 'perf ftrace' to just being a convenient perf interface to what
ftrace provides?
But all around, nice work, cool new toyz! :-)
- Arnaldo
> $ sudo ./perf ftrace latency -a -T mutex_lock sleep 1
> # DURATION | COUNT | GRAPH |
> 0 - 1 us | 2686 | ###################### |
> 1 - 2 us | 976 | ######## |
> 2 - 4 us | 879 | ####### |
> 4 - 8 us | 481 | #### |
> 8 - 16 us | 445 | ### |
> 16 - 32 us | 1 | |
> 32 - 64 us | 0 | |
> 64 - 128 us | 0 | |
> 128 - 256 us | 0 | |
> 256 - 512 us | 0 | |
> 512 - 1024 us | 0 | |
> 1 - 2 ms | 0 | |
> 2 - 4 ms | 0 | |
> 4 - 8 ms | 0 | |
> 8 - 16 ms | 0 | |
> 16 - 32 ms | 0 | |
> 32 - 64 ms | 0 | |
> 64 - 128 ms | 0 | |
> 128 - 256 ms | 0 | |
> 256 - 512 ms | 0 | |
> 512 - 1024 ms | 0 | |
> 1 - ... s | 0 | |
>
> It basically use the function graph tracer to extract the duration of
> the function. But with -b/--use-bpf option, it can use BPF to save
> the histogram in the kernel. For the same function, it gets:
>
> $ sudo ./perf ftrace latency -a -b -T mutex_lock sleep 1
> # DURATION | COUNT | GRAPH |
> 0 - 1 us | 4682 | ############################################# |
> 1 - 2 us | 11 | |
> 2 - 4 us | 0 | |
> 4 - 8 us | 0 | |
> 8 - 16 us | 7 | |
> 16 - 32 us | 6 | |
> 32 - 64 us | 0 | |
> 64 - 128 us | 0 | |
> 128 - 256 us | 0 | |
> 256 - 512 us | 0 | |
> 512 - 1024 us | 0 | |
> 1 - 2 ms | 0 | |
> 2 - 4 ms | 0 | |
> 4 - 8 ms | 0 | |
> 8 - 16 ms | 0 | |
> 16 - 32 ms | 0 | |
> 32 - 64 ms | 0 | |
> 64 - 128 ms | 0 | |
> 128 - 256 ms | 0 | |
> 256 - 512 ms | 0 | |
> 512 - 1024 ms | 0 | |
> 1 - ... s | 0 | |
>
>
> You can get the patches at 'perf/ftrace-latency-v1' branch on
>
> https://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
>
>
> Thanks,
> Namhyung
>
>
> Namhyung Kim (5):
> perf ftrace: Add 'trace' subcommand
> perf ftrace: Move out common code from __cmd_ftrace
> perf ftrace: Add 'latency' subcommand
> perf ftrace: Add -b/--use-bpf option for latency subcommand
> perf ftrace: Implement cpu and task filters in BPF
>
> tools/perf/Makefile.perf | 2 +-
> tools/perf/builtin-ftrace.c | 443 +++++++++++++++++---
> tools/perf/util/Build | 1 +
> tools/perf/util/bpf_ftrace.c | 154 +++++++
> tools/perf/util/bpf_skel/func_latency.bpf.c | 113 +++++
> tools/perf/util/ftrace.h | 81 ++++
> 6 files changed, 724 insertions(+), 70 deletions(-)
> create mode 100644 tools/perf/util/bpf_ftrace.c
> create mode 100644 tools/perf/util/bpf_skel/func_latency.bpf.c
> create mode 100644 tools/perf/util/ftrace.h
>
>
> base-commit: 8ab774587903771821b59471cc723bba6d893942
> --
> 2.34.0.rc2.393.gf8c9666880-goog
--
- Arnaldo