[PATCH 00/14] export perf overheads information
From: kan . liang
Date: Wed Nov 23 2016 - 12:45:02 EST
From: Kan Liang <kan.liang@xxxxxxxxx>
Profiling brings additional overhead. High overhead may impacts the
behavior of the profiling object, impacts the accuracy of the
profiling result, and even hang the system.
Currently, perf has dynamic interrupt throttle mechanism to lower the
sample rate and overhead. But it has limitations.
- The mechanism only focus in the overhead from NMI. However, there
are other parts which bring big overhead. E.g, multiplexing.
- The hint from the mechanism doesn't work on fixed period.
- The system changes which caused by the mechanism are not recorded
in the perf.data. Users have no idea about the overhead and its
Acctually, any passive ways like dynamic interrupt throttle mechanism
are only palliative. The best way is to export overheads information,
provide more hints, and help the users design more proper perf command.
According to our test, there are four parts which can bring big overhead.
They include NMI handler, multiplexing handler, iterate side-band events,
and write data in file. Two new perf record type PERF_RECORD_OVERHEAD and
PERF_RECORD_USER_OVERHEAD are introduced to record the overhead
information in kernel and user space respectively.
The overhead information is the system per-CPU overhead, not per-event
overhead. The implementation takes advantage of the existing event log
To reduce the additional overhead from logging overhead information, the
overhead information only be output when the event is going to be
disabled or task is scheduling out.
In perf report, the overhead will be checked automatically. If the
overhead rate is larger than 10%. A warning will be displayed.
A new option is also introduced to display detial per-CPU overhead
Current implementation only include four overhead sources. There could be
more in other parts. The new overhead source can be easily added as a
Kan Liang (14):
perf/x86: Introduce PERF_RECORD_OVERHEAD
perf/x86: output NMI overhead
perf/x86: output multiplexing overhead
perf/x86: output side-band events overhead
perf tools: handle PERF_RECORD_OVERHEAD record type
perf tools: show NMI overhead
perf tools: show multiplexing overhead
perf tools: show side-band events overhead
perf tools: make get_nsecs visible for buildin files
perf tools: introduce PERF_RECORD_USER_OVERHEAD
perf tools: record write data overhead
perf tools: record elapsed time
perf tools: warn on high overhead
perf script: show overhead events
arch/x86/events/core.c | 19 ++++-
arch/x86/events/perf_event.h | 2 +
include/linux/perf_event.h | 9 ++
include/uapi/linux/perf_event.h | 33 ++++++++
kernel/events/core.c | 139 +++++++++++++++++++++++++------
tools/include/uapi/linux/perf_event.h | 33 ++++++++
tools/perf/Documentation/perf-report.txt | 3 +
tools/perf/builtin-record.c | 71 +++++++++++++++-
tools/perf/builtin-report.c | 25 ++++++
tools/perf/builtin-sched.c | 2 +-
tools/perf/builtin-script.c | 36 ++++++++
tools/perf/builtin.h | 1 +
tools/perf/util/event.c | 47 +++++++++++
tools/perf/util/event.h | 37 ++++++++
tools/perf/util/machine.c | 35 ++++++++
tools/perf/util/machine.h | 2 +
tools/perf/util/session.c | 68 +++++++++++++++
tools/perf/util/symbol.h | 3 +-
tools/perf/util/tool.h | 1 +
19 files changed, 535 insertions(+), 31 deletions(-)