Re: [PATCH v1] perf tests: Harden branch stack sampling test

[Leo Yan]

On Mon, Mar 17, 2025 at 08:38:44AM -0700, Ian Rogers wrote:

[...]

> On our testing skipped == failed, I can change 2 to 1 above but I'd
> made it 2 as it wasn't clear to me all branch filter types would be
> supported by perf record and skipping/2 was a less terrible error
> message.

My concern is changing from returned val from 1 to 2 will lead to CI
to never report errors.

> I'm keen to land the pulling apart of the perf command from the
> tr/grep as if we hit say an asan error currently that is hidden by
> code like:
> ```
> perf record -o $TMPDIR/perf.data --branch-filter any,save_type,u --
> ${TESTPROG} > /dev/null 2>&1
> ```
> where all the output is sent to /dev/null but the asan error code will
> cause the "set -e" to fail. If this code fails with asan then
> currently the first thing to do is start pulling apart the
> expressions.

Agreed the most part in this patch, except returns err=2 in
test_user_branches() and test_filter().

> Code like:
> ```
> perf script -i $TMPDIR/perf.data --fields brstack | tr -s ' ' '\n' |
> grep '.' > $TMPDIR/perf.script
> ```
> is problematic as again we lose the asan like errors. Running the previous:
> ```
>  if grep -E -vm1 "^[^ ]*/($test_filter_expect|-|( *))/.*$"
> $TMPDIR/perf.script; then
> ```
> could fail because of an unexpected branch filter type, but was
> failing for me just because there were blank or similar lines in the
> output.

I saw you changed the command as:

  tr -s ' ' '\n' < "$TMPDIR/perf.script" | grep '.' | \
      grep -E -vm1 "^[^ ]*/($test_filter_expect|-|( *))/.*$" \
      > "$TMPDIR/perf.script-filtered" || true

This is a good change for me.  After we removed the noises caused by
spaces and empty lines, for an empty script output, I still think we
should report error, as the test fails to capture any branch stack.

> The new code doesn't change this but allows the output to be
> dumped for later diagnostics. The '|| true' in the expression means we
> get to dumping the diagnostics and dump just fail because some
> sub-command mismatched its input.

Yeah, agreed '|| true' is a good improvement.

Thanks,
Leo