[GIT PULL 00/43] perf/core new feature: 'perf stat record/report'

From: Arnaldo Carvalho de Melo
Date: Thu Dec 17 2015 - 14:58:29 EST

From: Arnaldo Carvalho de Melo <acme@xxxxxxxxxx>

Hi Ingo,

Please consider pulling, cool new feature! This is on top of the
perf-core-for-mingo-2.1 tag, with that RHEL6.7 bugfix, I had also to go
over this one fixing stuff in many spots :-\

- Arnaldo

The following changes since commit 1843b4e057b7717db21a3ad96fa16d6b4ee8f6c4:

tools subcmd: Rename subcmd header include guards (2015-12-17 14:27:14 -0300)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-3

for you to fetch changes up to 89af4e05c21d68f22e07fe66940ea675615a49ed:

perf stat report: Allow to override aggr_mode (2015-12-17 16:30:30 -0300)

----------------------------------------------------------------
perf/core improvement.

User visible:

- Generate perf.data files from 'perf stat', to tap into the scripting
capabilities perf has instead of defining a 'perf stat' specific scripting
support to calculate event ratios, etc. Simple example:

$ perf stat record -e cycles usleep 1

Performance counter stats for 'usleep 1':

1,134,996 cycles

0.000670644 seconds time elapsed

$ perf stat report

Performance counter stats for '/home/acme/bin/perf stat record -e cycles usleep 1':

1,134,996 cycles

0.000670644 seconds time elapsed

$

It generates PERF_RECORD_ userspace records to store the details:

$ perf report -D | grep PERF_RECORD
0xf0 [0x28]: PERF_RECORD_THREAD_MAP nr: 1 thread: 27637
0x118 [0x12]: PERF_RECORD_CPU_MAP nr: 1 cpu: 65535
0x12a [0x40]: PERF_RECORD_STAT_CONFIG
0x16a [0x30]: PERF_RECORD_STAT
-1 -1 0x19a [0x40]: PERF_RECORD_MMAP -1/0: [0xffffffff81000000(0x1f000000) @ 0xffffffff81000000]: x [kernel.kallsyms]_text
0x1da [0x18]: PERF_RECORD_STAT_ROUND
[acme@ssdandy linux]$

An effort was made to make perf.data files generated like this to not
generate cryptic messages when processed by older tools.

The 'perf script' bits need rebasing, will go up later.

Jiri's cover letter for this series:

The initial attempt defined its own formula lang and allowed triggering user's
script on the end of the stat command:

http://marc.info/?l=linux-kernel&m=136742146322273&w=2

This patchset abandons the idea of new formula language and rather adds support
to:

- store stat data into perf.data file
- add python support to process stat events

Basically it allows to store stat data into perf.data and post process it with
python scripts in a similar way we do for sampling data.

The stat data are stored in new stat, stat-round, stat-config user events.
stat - stored for each read syscall of the counter
stat round - stored for each interval or end of the command invocation
stat config - stores all the config information needed to process data
so report tool could restore the same output as record

The python script can now define 'stat__<eventname>_<modifier>' functions
to get stat events data and 'stat__interval' to get stat-round data.

See CPI script example in scripts/python/stat-cpi.py.

Signed-off-by: Arnaldo Carvalho de Melo <acme@xxxxxxxxxx>

----------------------------------------------------------------
Jiri Olsa (43):
perf thread_map: Add thread_map user level event
perf thread_map: Add thread_map event sythesize function
perf thread_map: Add thread_map__new_event function
perf thread_map: Add perf_event__fprintf_thread_map function
perf cpu_map: Add cpu_map user level event
perf cpu_map: Add cpu_map event synthesize function
perf cpu_map: Add cpu_map__new_event function
perf cpu_map: Add perf_event__fprintf_cpu_map function
perf tools: Add stat config user level event
perf tools: Add stat config event synthesize function
perf tools: Add stat config event read function
perf tools: Add stat user level event
perf tools: Add stat event synthesize function
perf tools: Add stat event read function
perf tools: Add stat round user level event
perf tools: Add stat round event synthesize function
perf tools: Add stat events fprintf functions
perf tools: Add event_update user level event
perf tools: Add event_update event unit type
perf tools: Add event_update event scale type
perf tools: Add event_update event name type
perf tools: Add event_update event cpus type
perf tools: Add perf_event__fprintf_event_update function
perf report: Display newly added events in raw dump
perf tools: Introduce stat perf.data header feature
perf stat record: Add record command
perf stat record: Initialize record features
perf stat record: Synthesize stat record data
perf evlist: Export id_add_fd()
perf stat record: Store events IDs in perf data file
perf stat record: Add pipe support for record command
perf stat record: Write stat events on record
perf stat record: Write stat round events on record
perf stat record: Do not allow record with multiple runs mode
perf stat record: Synthesize event update events
perf stat report: Add report command
perf stat report: Process cpu/threads maps
perf stat report: Process stat config event
perf stat report: Add support to initialize aggr_map from file
perf stat report: Move csv_sep initialization before report command
perf stat report: Process stat and stat round events
perf stat report: Process event update events
perf stat report: Allow to override aggr_mode

tools/perf/Documentation/perf-stat.txt | 34 ++
tools/perf/builtin-record.c | 2 +
tools/perf/builtin-stat.c | 614 ++++++++++++++++++++++++++++++++-
tools/perf/tests/Build | 3 +
tools/perf/tests/builtin-test.c | 24 ++
tools/perf/tests/cpumap.c | 88 +++++
tools/perf/tests/event_update.c | 117 +++++++
tools/perf/tests/stat.c | 111 ++++++
tools/perf/tests/tests.h | 6 +
tools/perf/tests/thread-map.c | 43 +++
tools/perf/util/cpumap.c | 42 +++
tools/perf/util/cpumap.h | 1 +
tools/perf/util/event.c | 308 +++++++++++++++++
tools/perf/util/event.h | 150 +++++++-
tools/perf/util/evlist.c | 6 +-
tools/perf/util/evlist.h | 3 +
tools/perf/util/header.c | 205 +++++++++++
tools/perf/util/header.h | 17 +
tools/perf/util/session.c | 189 ++++++++++
tools/perf/util/stat.c | 62 ++++
tools/perf/util/stat.h | 10 +
tools/perf/util/thread_map.c | 27 ++
tools/perf/util/thread_map.h | 3 +
tools/perf/util/tool.h | 8 +-
24 files changed, 2054 insertions(+), 19 deletions(-)
create mode 100644 tools/perf/tests/cpumap.c
create mode 100644 tools/perf/tests/event_update.c
create mode 100644 tools/perf/tests/stat.c
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/