Re: [PATCH v1] perf tests: Harden branch stack sampling test

From: Leo Yan
Date: Fri Mar 14 2025 - 05:13:40 EST


Hi Namhyung,

On Thu, Mar 13, 2025 at 01:18:08PM -0700, Namhyung Kim wrote:

[...]

> > > test_user_branches() {
> > > echo "Testing user branch stack sampling"
> > >
> > > - perf record -o $TMPDIR/perf.data --branch-filter any,save_type,u -- ${TESTPROG} > /dev/null 2>&1
> > > - perf script -i $TMPDIR/perf.data --fields brstacksym | tr -s ' ' '\n' > $TMPDIR/perf.script
> > > + perf record -o "$TMPDIR/perf.data" --branch-filter any,save_type,u -- ${TESTPROG} > "$TMPDIR/record.txt" 2>&1
> > > + perf script -i "$TMPDIR/perf.data" --fields brstacksym > "$TMPDIR/perf.script"
> > >
> > > # example of branch entries:
> > > # brstack_foo+0x14/brstack_bar+0x40/P/-/-/0/CALL
> > >
> > > - set -x
> > > - grep -E -m1 "^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/IND_CALL/.*$" $TMPDIR/perf.script
> > > - grep -E -m1 "^brstack_foo\+[^ ]*/brstack_bar\+[^ ]*/CALL/.*$" $TMPDIR/perf.script
> > > - grep -E -m1 "^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/CALL/.*$" $TMPDIR/perf.script
> > > - grep -E -m1 "^brstack_bench\+[^ ]*/brstack_bar\+[^ ]*/CALL/.*$" $TMPDIR/perf.script
> > > - grep -E -m1 "^brstack_bar\+[^ ]*/brstack_foo\+[^ ]*/RET/.*$" $TMPDIR/perf.script
> > > - grep -E -m1 "^brstack_foo\+[^ ]*/brstack_bench\+[^ ]*/RET/.*$" $TMPDIR/perf.script
> > > - grep -E -m1 "^brstack_bench\+[^ ]*/brstack_bench\+[^ ]*/COND/.*$" $TMPDIR/perf.script
> > > - grep -E -m1 "^brstack\+[^ ]*/brstack\+[^ ]*/UNCOND/.*$" $TMPDIR/perf.script
> > > - set +x
> > > -
> > > + expected=(
> > > + "^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/IND_CALL/.*$"
> > > + "^brstack_foo\+[^ ]*/brstack_bar\+[^ ]*/CALL/.*$"
> > > + "^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/CALL/.*$"
> > > + "^brstack_bench\+[^ ]*/brstack_bar\+[^ ]*/CALL/.*$"
> > > + "^brstack_bar\+[^ ]*/brstack_foo\+[^ ]*/RET/.*$"
> > > + "^brstack_foo\+[^ ]*/brstack_bench\+[^ ]*/RET/.*$"
> > > + "^brstack_bench\+[^ ]*/brstack_bench\+[^ ]*/COND/.*$"
> > > + "^brstack\+[^ ]*/brstack\+[^ ]*/UNCOND/.*$"
> > > + )
> > > + for x in "${expected[@]}"
> > > + do
> > > + if ! tr -s ' ' '\n' < "$TMPDIR/perf.script" | grep -E -m1 -q "$x"
> > > + then
> > > + echo "Branches missing $x"
> > > + if [ "x$err" == "x0" ]
> > > + then
> > > + err=2
> >
> > Here it sets "err=2", as a result, if any grep command fails, the script
> > exits while reporting to skip the test. This seems incorrect to me.
> >
> > My understanding is the regular expressions above are mandatory to be
> > matched, otherwise, it must be something is wrong. We should not skip
> > the test in this case.
> >
> > I can understand that 'perf record' cannot record all branch types, if
> > this is the case, maybe we can improve the recording quality rather
> > than reporting skip? E.g.,
> >
> > cat <<EOF > "$TMPDIR/loop.sh"
> > for run in {1..5}; do perf test -w brstack; done
> > EOF
> >
> > perf record -o "$TMPDIR/perf.data" --branch-filter any,save_type,u
> > -- sh $TMPDIR/loop.sh
> >
> > If we run the test for 5 times, should this can allow us to ensure the
> > branch samples are recorded?
>
> The brstack (and other workload programs) can take an argument to
> control its duration. For brstack, it's the number of loop iteration
> and default is 999999.

Sorry I did not dig into the brstack workload program.

If the workload has run for a large number of loops, the question is:
why isn't the test capturing the expected branch stacks?

Thanks,
Leo