[PATCH v8 0/7] powerpc/powernv: Nest Instrumentation support
From: Madhavan Srinivasan
Date: Mon Feb 22 2016 - 22:57:54 EST
This patchset enables Nest Instrumentation support on powerpc.
POWER8 has per-chip Nest Intrumentation which provides various
per-chip metrics like memory, powerbus, Xlink and Alink
bandwidth.
Nest Instrumentation provides an interface (via PORE Engine)
to configure and move the nest counter data to memory. From
kernel side, OPAL Call interface is used to activate/deactivate
PORE Engine for nest data collection.
OPAL at boot, detects the feature, initializes it and pass on
the nest units and other related information such as memory
region, events supported so on, to kernel via device-tree.
Kernel code then, parses the device-tree for nest pmu support
and registers nest pmu with the events available. PORE Engine collects
and accumulate nest counter data in per-chip reserved memory region, hence
device-tree also exports per-chip nest accumulation memory region.
And individual event offset are used as event configuration values.
Here is sample perf usage to explain the interface.
#./perf list
....
iTLB-load-misses [Hardware cache event]
Nest_Alink_BW/Alink0/ [Kernel PMU event]
Nest_Alink_BW/Alink1/ [Kernel PMU event]
Nest_Alink_BW/Alink2/ [Kernel PMU event]
Nest_MCS_Read_BW/MCS_00/ [Kernel PMU event]
Nest_MCS_Read_BW/MCS_01/ [Kernel PMU event]
Nest_MCS_Read_BW/MCS_02/ [Kernel PMU event]
Nest_MCS_Read_BW/MCS_03/ [Kernel PMU event]
Nest_MCS_Write_BW/MCS_00/ [Kernel PMU event]
Nest_MCS_Write_BW/MCS_01/ [Kernel PMU event]
Nest_MCS_Write_BW/MCS_02/ [Kernel PMU event]
Nest_MCS_Write_BW/MCS_03/ [Kernel PMU event]
Nest_PowerBus_BW/External/ [Kernel PMU event]
Nest_PowerBus_BW/Internal/ [Kernel PMU event]
Nest_Xlink_BW/Xlink0/ [Kernel PMU event]
Nest_Xlink_BW/Xlink1/ [Kernel PMU event]
Nest_Xlink_BW/Xlink2/ [Kernel PMU event]
rNNN [Raw hardware event descriptor]
cpu/t1=v1[,t2=v2,t3 ...]/modifier [Raw hardware event descriptor]
.....
# ./perf stat -e 'Nest_Xlink_BW/Xlink1/' -a -A sleep 1
Performance counter stats for 'system wide':
CPU0 15,913.18 MiB Nest_Xlink_BW/Xlink1/
CPU32 11,955.88 MiB Nest_Xlink_BW/Xlink1/
CPU64 11,042.43 MiB Nest_Xlink_BW/Xlink1/
CPU96 14,065.27 MiB Nest_Xlink_BW/Xlink1/
1.001062038 seconds time elapsed
# ./perf stat -e 'Nest_Alink_BW/Alink0/,Nest_Alink_BW/Alink1/,Nest_Alink_BW/Alink2/' -a -A -I 1000 sleep 5
Performance counter stats for 'system wide':
CPU0 0.00 MiB Nest_Alink_BW/Alink0/ (100.00%)
CPU32 0.00 MiB Nest_Alink_BW/Alink0/ (100.00%)
CPU64 0.00 MiB Nest_Alink_BW/Alink0/ (100.00%)
CPU96 0.00 MiB Nest_Alink_BW/Alink0/ (100.00%)
CPU0 1,430.43 MiB Nest_Alink_BW/Alink1/ (100.00%)
CPU32 320.99 MiB Nest_Alink_BW/Alink1/ (100.00%)
CPU64 3,443.83 MiB Nest_Alink_BW/Alink1/ (100.00%)
CPU96 1,904.41 MiB Nest_Alink_BW/Alink1/ (100.00%)
CPU0 2,856.85 MiB Nest_Alink_BW/Alink2/
CPU32 7.50 MiB Nest_Alink_BW/Alink2/
CPU64 4,034.29 MiB Nest_Alink_BW/Alink2/
CPU96 288.49 MiB Nest_Alink_BW/Alink2/
.....
OPAL side patches (v6) are posted in the skiboot mailing list.
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002824.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002825.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002826.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002833.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002832.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002831.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002829.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002827.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002828.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002830.html
Changelog from v7:
1)Rebased to latest upstream code.
2)Separated the device tree parser code and have created a new file "opal-nest.c"
in powernv/ folder. Have added a autoload function using a compatible string as match id.
3)Renamed variables and function names to skip "P8", since the nest pmu code is not
specific to POWER8 processor.
4)Updated comments and commit messages
Changelog from v6:
1) Modified the OPAL call number to sync with OPAL side changes.
2) Rebased to latest upstream.
Changelog from v5:
1) Replaced be32_to_cpup with of_property_read_u32 when reading from device tree
2) Replaced uintxx_t to u[32/64] variable types
3) Broken the memory location for event attribute group and events into
two alloc functions to have it simple.
4) Added separate functions for event string and event value as suggested.
5) Added P8_NEST_MAX_PMUS check for array overflow
6) Made changes to commit messages.
Changelog from v4:
1) Variable name changes for consistency and added more comments
2) Added sysfs_att_init to have lockdep happy
3) Updated OPAL Call interface changes and added code to handle
failure case.
4) Added new macro "P8_NEST_MODE_PRODUCTION" to specify PORE Engine mode
5) Modified nest_pmu_cpumask_init function to return value to
nest pmu init function incase of OPAL call failure.
Changelog from v3:
No logic change, just a rebase to latest upstream kernel.
Changelog from v2:
1) Changed variable and macro names to be consistent.
2) Made changes to commit message and code comment messages
3) Moved "format attribute" related code from patch 6 to 5
4) Added check for pmu register function
5) Changed cpu_init and cpu_exit functions to use first online
cpu of the chip, there by making code lot simplier.
Changelog from v1:
1) No logic changes, re-ordered patches make each patch compile
without errors
2) Added comments based on the review feedback.
3) removed perf_event_del function and replaced it with perf_event_stop.
4) Moved Nest feature detection code out of parser function.
5) Optimized functions and removed some variables.
6) squashed the makefile changes, instead of the separate patch
7) squashed the cpumask and hotplug patches as single patch
8) Added cpu checks in nest_change_cpu_context and nest_exit_cpu functions
9) Made changes to commit messages.
Changelog from RFC:
1) Removed "uncore" code and made each Nest Unit a separate PMU.
2) Removed uncore type abstraction and uncore related functions.
3) Added simple cpumask function since these are per-chip counters
4) Redesigned device-tree parser based on the latest platform enablement code
5) Made changes to commit message
Thanks to input from Sukadev Bhattiprolu, Preeti Murthy, Daniel Axtens,
Suzuki Poulose and Michael Ellerman
Kindly let me know you comments and feedback.
Cc: Michael Ellerman <mpe@xxxxxxxxxxxxxx>
Cc: Benjamin Herrenschmidt <benh@xxxxxxxxxxxxxxxxxxx>
Cc: Paul Mackerras <paulus@xxxxxxxxx>
Cc: Anton Blanchard <anton@xxxxxxxxx>
Cc: Daniel Axtens <dja@xxxxxxxxxx>
Cc: Stephane Eranian <eranian@xxxxxxxxxx>
Cc: Sukadev Bhattiprolu <sukadev@xxxxxxxxxxxxxxxxxx>
Signed-off-by: Madhavan Srinivasan <maddy@xxxxxxxxxxxxxxxxxx>
Madhavan Srinivasan (7):
powerpc/powernv: Data structure and macros definition
powerpc/powernv: Add OPAL support for Nest PMU
powerpc/powernv: autoload nest unit driver module
powerpc/powernv: detect supported nest units and its events
powerpc/perf: Add event attribute and group to nest pmu
powerpc/perf: generic nest pmu event functions
powerpc/perf: nest pmu cpumask and cpu hotplug support
arch/powerpc/include/asm/nest-pmu.h | 55 ++++
arch/powerpc/include/asm/opal-api.h | 3 +-
arch/powerpc/include/asm/opal.h | 3 +
arch/powerpc/perf/Makefile | 2 +-
arch/powerpc/perf/nest-pmu.c | 354 +++++++++++++++++++++++++
arch/powerpc/platforms/powernv/Makefile | 2 +-
arch/powerpc/platforms/powernv/opal-nest.c | 293 ++++++++++++++++++++
arch/powerpc/platforms/powernv/opal-wrappers.S | 1 +
arch/powerpc/platforms/powernv/opal.c | 12 +
9 files changed, 722 insertions(+), 3 deletions(-)
create mode 100644 arch/powerpc/include/asm/nest-pmu.h
create mode 100644 arch/powerpc/perf/nest-pmu.c
create mode 100644 arch/powerpc/platforms/powernv/opal-nest.c
--
1.9.1