Re: [PATCH v1] perf trace: Fix inconsistent failures in perf trace's tests

[Namhyung Kim]

On Thu, Apr 03, 2025 at 09:16:52PM -0700, Howard Chu wrote:
> There are two failures that frequently occur in perf trace's tests. One
> is the failure of 'perf trace BTF general tests'; The other one is the
> failure of 'perf trace record and replay', which, when run independently,
> always succeeds.
> 
> The root cause of the first failure, is that perf trace may give two types
> of output, depending on whether the comm of a process can be parsed, for
> example:
> 
> mv/312705 renameat2(CWD, "/tmp/file1_VJOT", CWD, "/tmp/file2_VJOT", NOREPLACE) = 0
> :312774/312774 renameat2(CWD, "/tmp/file1_5YcE", CWD, "/tmp/file2_5YcE", NOREPLACE) = 0
> 
> In the test, however, grep is always looking for the comm 'mv', which
> sometimes may not be present.
> 
> The cause of the second failure is that 'perf trace BTF general tests'
> modifies the perf config, and because tests are run concurrently,
> subsequent tests use the modified perf config before the BTF general
> test can restore the original config. Mark the BTF general tests as
> exclusive will solve the failure.

I'm not sure if the config is the cause of the failure.  Also I don't
see it restored.

IIUC the export only affects child processes from the current shell.
So other tests running in parallel won't see the config change.

But still, there should be something to affect the behavior.  It's
strange to miss the task name in COMM record.

I also confirm that running the test serially fixes it.

Thanks,
Namhyung

> 
> Signed-off-by: Howard Chu <howardchu95@xxxxxxxxx>
> Fixes: 0255338d6975 ("perf trace: Add tests for BTF general augmentation")
> ---
>  tools/perf/tests/shell/trace_btf_general.sh | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/tools/perf/tests/shell/trace_btf_general.sh b/tools/perf/tests/shell/trace_btf_general.sh
> index a25d8744695e..4833618a116b 100755
> --- a/tools/perf/tests/shell/trace_btf_general.sh
> +++ b/tools/perf/tests/shell/trace_btf_general.sh
> @@ -1,5 +1,5 @@
>  #!/bin/bash
> -# perf trace BTF general tests
> +# perf trace BTF general tests (exclusive)
>  # SPDX-License-Identifier: GPL-2.0
>  
>  err=0
> @@ -29,7 +29,7 @@ check_vmlinux() {
>  trace_test_string() {
>    echo "Testing perf trace's string augmentation"
>    if ! perf trace -e renameat* --max-events=1 -- mv ${file1} ${file2} 2>&1 | \
> -    grep -q -E "^mv/[0-9]+ renameat(2)?\(.*, \"${file1}\", .*, \"${file2}\", .*\) += +[0-9]+$"
> +    grep -q -E "^.*/[0-9]+ renameat(2)?\(.*, \"${file1}\", .*, \"${file2}\", .*\) += +[0-9]+$"
>    then
>      echo "String augmentation test failed"
>      err=1
> @@ -40,7 +40,7 @@ trace_test_buffer() {
>    echo "Testing perf trace's buffer augmentation"
>    # echo will insert a newline (\10) at the end of the buffer
>    if ! perf trace -e write --max-events=1 -- echo "${buffer}" 2>&1 | \
> -    grep -q -E "^echo/[0-9]+ write\([0-9]+, ${buffer}.*, [0-9]+\) += +[0-9]+$"
> +    grep -q -E "^.*/[0-9]+ write\([0-9]+, ${buffer}.*, [0-9]+\) += +[0-9]+$"
>    then
>      echo "Buffer augmentation test failed"
>      err=1
> @@ -50,7 +50,7 @@ trace_test_buffer() {
>  trace_test_struct_btf() {
>    echo "Testing perf trace's struct augmentation"
>    if ! perf trace -e clock_nanosleep --force-btf --max-events=1 -- sleep 1 2>&1 | \
> -    grep -q -E "^sleep/[0-9]+ clock_nanosleep\(0, 0, \{1,\}, 0x[0-9a-f]+\) += +[0-9]+$"
> +    grep -q -E "^.*/[0-9]+ clock_nanosleep\(0, 0, \{1,\}, 0x[0-9a-f]+\) += +[0-9]+$"
>    then
>      echo "BTF struct augmentation test failed"
>      err=1
> -- 
> 2.45.2
>