[PATCH v1 00/51] shadow metric clean up and improvements

From: Ian Rogers
Date: Sun Feb 19 2023 - 04:29:07 EST

Next message: Ian Rogers: "[PATCH v1 01/51] perf tools: Ensure evsel name is initialized"
Previous message: Olliver Schinagl: "Re: [PATCH] mips: Realtek RTL: select NO_EXCEPT_FILL"
Next in thread: Ian Rogers: "[PATCH v1 01/51] perf tools: Ensure evsel name is initialized"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Recently the shadow stat metrics broke due to repeated aggregation and
a quick fix was applied:
https://lore.kernel.org/lkml/20230209064447.83733-1-irogers@xxxxxxxxxx/
This is the longer fix but one that comes with some extras. To avoid
fixing issues for hard coded metrics, the topdown, SMI cost and
transaction flags are moved into json metrics. A side effect of this
is that TopdownL1 metrics will now be displayed when supported, if no
"perf stat" events are specified.

Another fix included here is for event grouping as raised in:
https://lore.kernel.org/lkml/CA+icZUU_ew7pzWJJZLbj1xsU6MQTPrj8tkFfDhNdTDRQfGUBMQ@xxxxxxxxxxxxxx/
Metrics are now tagged with NMI and SMT flags, meaning that the events
shouldn't be grouped if the NMI watchdog is enabled or SMT is enabled.

Given the two issues, the metrics are re-generated and the patches
also include the latest Intel vendor events. The changes to the metric
generation code can be seen in:
https://github.com/intel/perfmon/pull/56

Hard coded metrics support thresholds, the patches add this ability to
json metrics so that the hard coded metrics can be removed. Migrate
remaining hard coded metrics to looking up counters from the
evlist/aggregation count. Finally, get rid of the saved_value logic
and thereby look to fix the aggregation issues.

Some related fix ups and code clean ups are included in the changes,
in particular to aid with the code's readability and to keep topdown
documentation in sync.

Ian Rogers (51):
perf tools: Ensure evsel name is initialized
perf metrics: Improve variable names
perf pmu-events: Remove aggr_mode from pmu_event
perf pmu-events: Change aggr_mode to be an enum
perf pmu-events: Change deprecated to be a bool
perf pmu-events: Change perpkg to be a bool
perf expr: Make the online topology accessible globally
perf pmu-events: Make the metric_constraint an enum
perf pmu-events: Don't '\0' terminate enum values
perf vendor events intel: Refresh alderlake events
perf vendor events intel: Refresh alderlake-n metrics
perf vendor events intel: Refresh broadwell metrics
perf vendor events intel: Refresh broadwellde metrics
perf vendor events intel: Refresh broadwellx metrics
perf vendor events intel: Refresh cascadelakex events
perf vendor events intel: Add graniterapids events
perf vendor events intel: Refresh haswell metrics
perf vendor events intel: Refresh haswellx metrics
perf vendor events intel: Refresh icelake events
perf vendor events intel: Refresh icelakex metrics
perf vendor events intel: Refresh ivybridge metrics
perf vendor events intel: Refresh ivytown metrics
perf vendor events intel: Refresh jaketown events
perf vendor events intel: Refresh knightslanding events
perf vendor events intel: Refresh sandybridge events
perf vendor events intel: Refresh sapphirerapids events
perf vendor events intel: Refresh silvermont events
perf vendor events intel: Refresh skylake events
perf vendor events intel: Refresh skylakex metrics
perf vendor events intel: Refresh tigerlake events
perf vendor events intel: Refresh westmereep-dp events
perf jevents: Add rand support to metrics
perf jevent: Parse metric thresholds
perf pmu-events: Test parsing metric thresholds with the fake PMU
perf list: Support for printing metric thresholds
perf metric: Compute and print threshold values
perf expr: More explicit NAN handling
perf metric: Add --metric-no-threshold option
perf stat: Add TopdownL1 metric as a default if present
perf stat: Implement --topdown using json metrics
perf stat: Remove topdown event special handling
perf doc: Refresh topdown documentation
perf stat: Remove hard coded transaction events
perf stat: Use metrics for --smi-cost
perf stat: Remove perf_stat_evsel_id
perf stat: Move enums from header
perf stat: Hide runtime_stat
perf stat: Add cpu_aggr_map for loop
perf metric: Directly use counts rather than saved_value
perf stat: Use counts rather than saved_value
perf stat: Remove saved_value/runtime_stat

tools/perf/Documentation/perf-stat.txt | 27 +-
tools/perf/Documentation/topdown.txt | 70 +-
tools/perf/arch/powerpc/util/header.c | 2 +-
tools/perf/arch/x86/util/evlist.c | 6 +-
tools/perf/arch/x86/util/topdown.c | 78 +-
tools/perf/arch/x86/util/topdown.h | 1 -
tools/perf/builtin-list.c | 13 +-
tools/perf/builtin-script.c | 9 +-
tools/perf/builtin-stat.c | 233 +-
.../arch/x86/alderlake/adl-metrics.json | 3190 ++++++++++-------
.../pmu-events/arch/x86/alderlake/cache.json | 36 +-
.../arch/x86/alderlake/floating-point.json | 27 +
.../arch/x86/alderlake/frontend.json | 9 +
.../pmu-events/arch/x86/alderlake/memory.json | 3 +-
.../arch/x86/alderlake/pipeline.json | 14 +-
.../arch/x86/alderlake/uncore-other.json | 28 +-
.../arch/x86/alderlaken/adln-metrics.json | 811 +++--
.../arch/x86/broadwell/bdw-metrics.json | 1439 ++++----
.../arch/x86/broadwellde/bdwde-metrics.json | 1405 ++++----
.../arch/x86/broadwellx/bdx-metrics.json | 1626 +++++----
.../arch/x86/broadwellx/uncore-cache.json | 74 +-
.../x86/broadwellx/uncore-interconnect.json | 64 +-
.../arch/x86/broadwellx/uncore-other.json | 4 +-
.../arch/x86/cascadelakex/cache.json | 24 +-
.../arch/x86/cascadelakex/clx-metrics.json | 2198 ++++++------
.../arch/x86/cascadelakex/frontend.json | 8 +-
.../arch/x86/cascadelakex/pipeline.json | 16 +
.../arch/x86/cascadelakex/uncore-memory.json | 18 +-
.../arch/x86/cascadelakex/uncore-other.json | 120 +-
.../arch/x86/cascadelakex/uncore-power.json | 8 +-
.../arch/x86/graniterapids/cache.json | 54 +
.../arch/x86/graniterapids/frontend.json | 10 +
.../arch/x86/graniterapids/memory.json | 174 +
.../arch/x86/graniterapids/other.json | 29 +
.../arch/x86/graniterapids/pipeline.json | 102 +
.../x86/graniterapids/virtual-memory.json | 26 +
.../arch/x86/haswell/hsw-metrics.json | 1220 ++++---
.../arch/x86/haswellx/hsx-metrics.json | 1397 ++++----
.../pmu-events/arch/x86/icelake/cache.json | 16 +
.../arch/x86/icelake/floating-point.json | 31 +
.../arch/x86/icelake/icl-metrics.json | 1932 +++++-----
.../pmu-events/arch/x86/icelake/pipeline.json | 23 +-
.../arch/x86/icelake/uncore-other.json | 56 +
.../arch/x86/icelakex/icx-metrics.json | 2153 +++++------
.../arch/x86/icelakex/uncore-memory.json | 2 +-
.../arch/x86/icelakex/uncore-other.json | 4 +-
.../arch/x86/ivybridge/ivb-metrics.json | 1270 ++++---
.../arch/x86/ivytown/ivt-metrics.json | 1311 ++++---
.../pmu-events/arch/x86/jaketown/cache.json | 6 +-
.../arch/x86/jaketown/floating-point.json | 2 +-
.../arch/x86/jaketown/frontend.json | 12 +-
.../arch/x86/jaketown/jkt-metrics.json | 602 ++--
.../arch/x86/jaketown/pipeline.json | 2 +-
.../arch/x86/jaketown/uncore-cache.json | 22 +-
.../x86/jaketown/uncore-interconnect.json | 74 +-
.../arch/x86/jaketown/uncore-memory.json | 4 +-
.../arch/x86/jaketown/uncore-other.json | 22 +-
.../arch/x86/jaketown/uncore-power.json | 8 +-
.../arch/x86/knightslanding/cache.json | 94 +-
.../arch/x86/knightslanding/pipeline.json | 8 +-
.../arch/x86/knightslanding/uncore-other.json | 8 +-
tools/perf/pmu-events/arch/x86/mapfile.csv | 29 +-
.../arch/x86/sandybridge/cache.json | 8 +-
.../arch/x86/sandybridge/floating-point.json | 2 +-
.../arch/x86/sandybridge/frontend.json | 12 +-
.../arch/x86/sandybridge/pipeline.json | 2 +-
.../arch/x86/sandybridge/snb-metrics.json | 601 ++--
.../arch/x86/sapphirerapids/cache.json | 24 +-
.../x86/sapphirerapids/floating-point.json | 32 +
.../arch/x86/sapphirerapids/frontend.json | 8 +
.../arch/x86/sapphirerapids/pipeline.json | 19 +-
.../arch/x86/sapphirerapids/spr-metrics.json | 2283 ++++++------
.../arch/x86/sapphirerapids/uncore-other.json | 60 +
.../arch/x86/silvermont/frontend.json | 2 +-
.../arch/x86/silvermont/pipeline.json | 2 +-
.../pmu-events/arch/x86/skylake/cache.json | 25 +-
.../pmu-events/arch/x86/skylake/frontend.json | 8 +-
.../pmu-events/arch/x86/skylake/other.json | 1 +
.../pmu-events/arch/x86/skylake/pipeline.json | 16 +
.../arch/x86/skylake/skl-metrics.json | 1877 ++++++----
.../arch/x86/skylake/uncore-other.json | 1 +
.../pmu-events/arch/x86/skylakex/cache.json | 8 +-
.../arch/x86/skylakex/frontend.json | 8 +-
.../arch/x86/skylakex/pipeline.json | 16 +
.../arch/x86/skylakex/skx-metrics.json | 2097 +++++------
.../arch/x86/skylakex/uncore-memory.json | 2 +-
.../arch/x86/skylakex/uncore-other.json | 96 +-
.../arch/x86/skylakex/uncore-power.json | 6 +-
.../arch/x86/tigerlake/floating-point.json | 31 +
.../arch/x86/tigerlake/pipeline.json | 18 +
.../arch/x86/tigerlake/tgl-metrics.json | 1942 +++++-----
.../arch/x86/tigerlake/uncore-other.json | 28 +-
.../arch/x86/westmereep-dp/cache.json | 2 +-
.../x86/westmereep-dp/virtual-memory.json | 2 +-
tools/perf/pmu-events/jevents.py | 58 +-
tools/perf/pmu-events/metric.py | 8 +-
tools/perf/pmu-events/pmu-events.h | 35 +-
tools/perf/tests/expand-cgroup.c | 3 +-
tools/perf/tests/expr.c | 7 +-
tools/perf/tests/parse-metric.c | 21 +-
tools/perf/tests/pmu-events.c | 49 +-
tools/perf/util/cpumap.h | 3 +
tools/perf/util/cputopo.c | 14 +
tools/perf/util/cputopo.h | 5 +
tools/perf/util/evsel.h | 2 +-
tools/perf/util/expr.c | 16 +-
tools/perf/util/expr.y | 12 +-
tools/perf/util/metricgroup.c | 178 +-
tools/perf/util/metricgroup.h | 5 +-
tools/perf/util/pmu.c | 17 +-
tools/perf/util/print-events.h | 1 +
tools/perf/util/smt.c | 11 +-
tools/perf/util/smt.h | 12 +-
tools/perf/util/stat-display.c | 117 +-
tools/perf/util/stat-shadow.c | 1287 ++-----
tools/perf/util/stat.c | 74 -
tools/perf/util/stat.h | 96 +-
tools/perf/util/synthetic-events.c | 2 +-
tools/perf/util/topdown.c | 68 +-
tools/perf/util/topdown.h | 11 +-
120 files changed, 18025 insertions(+), 15590 deletions(-)
create mode 100644 tools/perf/pmu-events/arch/x86/graniterapids/cache.json
create mode 100644 tools/perf/pmu-events/arch/x86/graniterapids/frontend.json
create mode 100644 tools/perf/pmu-events/arch/x86/graniterapids/memory.json
create mode 100644 tools/perf/pmu-events/arch/x86/graniterapids/other.json
create mode 100644 tools/perf/pmu-events/arch/x86/graniterapids/pipeline.json
create mode 100644 tools/perf/pmu-events/arch/x86/graniterapids/virtual-memory.json

--
2.39.2.637.g21b0678d19-goog

Next message: Ian Rogers: "[PATCH v1 01/51] perf tools: Ensure evsel name is initialized"
Previous message: Olliver Schinagl: "Re: [PATCH] mips: Realtek RTL: select NO_EXCEPT_FILL"
Next in thread: Ian Rogers: "[PATCH v1 01/51] perf tools: Ensure evsel name is initialized"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]