[PATCHv5 00/11] perf tools: Add perf data CTF conversion

From: Jiri Olsa
Date: Fri Feb 20 2015 - 17:21:04 EST

Next message: Alexey Khoroshilov: "[PATCH] i40e: Fix memory leak at failure path in i40e_dbg_command_write()"
Previous message: Jiri Olsa: "[PATCH 04/11] perf data: Add a 'perf' prefix to the generic fields"
Next in thread: Jiri Olsa: "[PATCH 01/11] perf tools: Add feature check for libbabeltrace"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

hi,
this is follow up on original RFC patchset:
http://marc.info/?t=140732735600004&r=1&w=2

Basically we are adding 'perf data convert' command to
allow conversion of perf data file into CTF [1] data.

v5 changes:
- added struct field_format::alias instead of priv void*
- rebased to latest acme's perf/core

v4 changes:
- sorted out babeltrace new code regressions
- rebased to latest acme's perf/core
- includes fixes from Wang Nan, patches 9/10/11
- patch 1/11 changelog updated with babeltrace
compilation instructions [Arnaldo]

v3 changes:
- rebased to latest acme's perf/core

v2 changes:
- addressed comments from Namhyung
- rebased to latest acme's perf/core

Changes from RFC:
- able to generate CTF data, that are possible to be displayed under
tracecompas GUI [3], please check several screenshots in here [4]
- storing CTF data streams per cpu
- several cleanups

Examples:
- Catch default perf data (cycles event):
$ perf record ls
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.012 MB perf.data (~546 samples) ]

- To display converted CTF data run [2]:
$ babeltrace ./ctf-data/
[03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
[03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
[03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 }
[03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 }
[03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 }
[03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 }
[03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 }
[03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 }
[03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }

- To get some nice output in tracecompas GUI [3], please capture sched:*
and syscall tracepoints like:
# perf record -e 'sched:*,raw_syscalls:*' -a
^C[ perf record: Woken up 0 times to write data ]
[ perf record: Captured and wrote 412.347 MB perf.data (~18015721 samples) ]

- To convert perf data file run:
# perf data convert --to-ctf=./ctf
[ perf data convert: Converted 'perf.data' into CTF data './ctf' ]
[ perf data convert: Converted and wrote 408.421 MB (3964792 samples) ]

- To display converted CTF data run [2]:
# babeltrace ./ctf-data/
[23:32:20.165354855] (+0.000000507) sched:sched_wakeup: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810BCA72, perf_tid = 0, perf_pid = 0, perf_id = 462554, perf_period = 1, common_type = 265, ...
[23:32:20.165359078] (+0.000001181) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 0, perf_pid = 0, perf_id = 462562, perf_period = 1, common_type = 263, ...
[23:32:20.165364686] (+0.000000328) sched:sched_stat_runtime: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 5326, perf_pid = 5326, perf_id = 462610, perf_period = 11380, ...
[23:32:20.165366067] (+0.000001205) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 5326, perf_pid = 5326, perf_id = 462562, perf_period = 1, common_type ...
[23:32:20.165723312] (+0.000001479) sched:sched_stat_runtime: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 11821, perf_pid = 11821, perf_id = 462612, perf_period = 1000265, ...
[23:32:20.065282391] (+?.?????????) raw_syscalls:sys_enter: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810230AF, perf_tid = 26155, perf_pid = 26155, perf_id = 462635, perf_period = 1, ...
[23:32:20.065286422] (+0.000004031) raw_syscalls:sys_exit: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810231D8, perf_tid = 26155, perf_pid = 26155, perf_id = 462639, perf_period = 1, ...

- Or run tracecompass and open the CTF data ;-)

Changes are also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/core_ctf_convert

thanks,
jirka

[1] Common Trace Format - http://www.efficios.com/ctf
[2] babeltrace - http://www.efficios.com/babeltrace
[3] Trace compass - http://projects.eclipse.org/projects/tools.tracecompass
[4] screenshots - http://people.redhat.com/~jolsa/tracecompass-perf/

Cc: Arnaldo Carvalho de Melo <acme@xxxxxxxxxx>
Cc: David Ahern <dsahern@xxxxxxxxx>
Cc: Dominique Toupin <dominique.toupin@xxxxxxxxxxxx>
Cc: Frederic Weisbecker <fweisbec@xxxxxxxxx>
Cc: Jeremie Galarneau <jgalar@xxxxxxxxxxxx>
Cc: Jiri Olsa <jolsa@xxxxxxxxxx>
Cc: Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx>
Cc: Namhyung Kim <namhyung@xxxxxxxxx>
Cc: Paul Mackerras <paulus@xxxxxxxxx>
Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Cc: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>
Cc: Tom Zanussi <tzanussi@xxxxxxxxx>
Cc: Wang Nan <wangnan0@xxxxxxxxxx>
---
Jiri Olsa (6):
perf tools: Add feature check for libbabeltrace
perf tools: Add new perf data command
perf data: Add perf data to CTF conversion support
perf data: Enable stream flush within processing
perf data: Add support for setting ordered_events queue size
tools lib traceevent: Add alias field to struct format_field

Sebastian Andrzej Siewior (3):
perf data: Add a 'perf' prefix to the generic fields
perf data: Add tracepoint events fields CTF conversion support
perf data: Switch to multiple cpu stream files

Wang Nan (2):
perf data: Fix duplicate field names and avoid reserved keywords
perf data: Fix signess of value

tools/lib/traceevent/event-parse.c | 1 +
tools/lib/traceevent/event-parse.h | 1 +
tools/perf/Build | 1 +
tools/perf/Documentation/perf-data.txt | 40 +
tools/perf/Documentation/perf.txt | 7 +-
tools/perf/Makefile.perf | 4 +-
tools/perf/builtin-data.c | 119 ++
tools/perf/builtin.h | 1 +
tools/perf/command-list.txt | 1 +
tools/perf/config/Makefile | 25 +
tools/perf/config/feature-checks/Makefile | 8 +-
tools/perf/config/feature-checks/test-all.c | 5 +
.../config/feature-checks/test-libbabeltrace.c | 8 +
tools/perf/perf.c | 1 +
tools/perf/util/Build | 2 +
tools/perf/util/data-convert-bt.c | 1174 ++++++++++++++++++++
tools/perf/util/data-convert-bt.h | 8 +
tools/perf/util/debug.c | 2 +
tools/perf/util/debug.h | 1 +
19 files changed, 1405 insertions(+), 4 deletions(-)
create mode 100644 tools/perf/Documentation/perf-data.txt
create mode 100644 tools/perf/builtin-data.c
create mode 100644 tools/perf/config/feature-checks/test-libbabeltrace.c
create mode 100644 tools/perf/util/data-convert-bt.c
create mode 100644 tools/perf/util/data-convert-bt.h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Next message: Alexey Khoroshilov: "[PATCH] i40e: Fix memory leak at failure path in i40e_dbg_command_write()"
Previous message: Jiri Olsa: "[PATCH 04/11] perf data: Add a 'perf' prefix to the generic fields"
Next in thread: Jiri Olsa: "[PATCH 01/11] perf tools: Add feature check for libbabeltrace"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]