Re: [PATCH 04/13] perf tests: Add another metric parsing test

[Jiri Olsa]

On Tue, Jun 02, 2020 at 10:58:32AM -0700, Ian Rogers wrote:
> On Tue, Jun 2, 2020 at 4:51 AM Jiri Olsa <jolsa@xxxxxxxxxx> wrote:
> >
> > The test goes through all metrics compiled for arch
> > within pmu events and try to parse them.
> >
> > This test is different from 'test_parsing' in that
> > we go through all the events in the current arch,
> > not just one defined for current CPU model. Using
> > 'fake_pmu' to parse events which do not have PMUs
> > defined in the system.
> >
> > Say there's bad change in ivybridge metrics file, like:
> >
> >   --- a/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
> >   +++ b/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
> >   @@ -8,7 +8,7 @@
> >   -        "MetricExpr": "IDQ_UOPS_NOT_DELIVERED.CORE / (4 * ((
> >   +        "MetricExpr": "IDQ_UOPS_NOT_DELIVERED.CORE / / (4 *
> >
> > the test fails with (on my kabylake laptop):
> >
> >   $ perf test 'Parsing of PMU event table metrics with fake PMUs' -v
> >   parsing 'idq_uops_not_delivered.core / / (4 * (( ( cpu_clk_unh...
> >   syntax error, line 1
> >   expr__parse failed
> >   test child finished with -1
> >   ...
> 
> For this example as the problem is the expression, presumably this was
> "passing" with test_parsing due to returning TEST_SKIP? I did this
> initially so that we could get the test merged and then the metrics
> fixed. I'd prefer if test_parsing were returning TEST_FAIL.

it will fail:

	[jolsa@krava perf]$ git diff
	diff --git a/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json b/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
	index 28e25447d3ef..0cad6b709f96 100644
	--- a/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
	+++ b/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
	@@ -1,7 +1,7 @@
	 [
	     {
		 "BriefDescription": "This category represents fraction of slots where the processor's Frontend undersupplies its Backend",
	-        "MetricExpr": "IDQ_UOPS_NOT_DELIVERED.CORE / (4 * cycles)",
	+        "MetricExpr": "IDQ_UOPS_NOT_DELIVERED.CORE / / (4 * cycles)",
		 "MetricGroup": "TopdownL1",
		 "MetricName": "Frontend_Bound",

	[jolsa@krava perf]$ ./perf test  'Parsing of PMU event table metrics with fake PMUs' 
	10: PMU events                                            :
	10.4: Parsing of PMU event table metrics with fake PMUs   : FAILED!

because the metric is malformed

jirka