[PATCH v2 00/12] perf tests: Enhance robustness, speed up execution, and fix flakiness

[Ian Rogers]

This patch series introduces several robustness and performance improvements
to the perf test suite, alongside fixes for long-standing flakiness.

Key changes across the series:
- Introduces robust retry logic (`perf_record_with_retry`) to prevent
  spurious test failures due to transient `EPERM` issues during
  system-wide background profiling.
- Significantly speeds up test execution across multiple scripts (such as
  kvm, record, trace, off-cpu, and lock contention) by supporting
  sub-second durations in the `noploop` and `thloop` workloads, reducing
  unnecessary wait times.
- Fixes flakiness in the BPF counters hybrid test by parsing `taskset` to
  determine valid CPUs rather than relying on `shuf`, which can fail under
  cgroups or missing binaries.
- Fixes JIT dump file leaks by actively recording the injected Python PID
  to guarantee cleanup when zero samples are generated.
- Decouples format alignments in `builtin-test.c` to gracefully truncate
  test descriptions without losing alignment or failing under non-TTY
  environments.
- Restricts uncore PMU bypass in event parsing strictly to `--cputype` to
  prevent unintended metric evaluation impacts.

v2 changes:
- Plumbed `bool cputype_filter` in metric parsing (patch 1) to explicitly
  restrict the uncore bypass to `--cputype`.
- Fixed output truncation on file redirection in `builtin-test.c` (patch 2)
  by guarding `get_term_dimensions()` with `isatty(1)`, and decoupled the
  format minimum padding from maximum precision limits to avoid hard-truncating
  long descriptions unnecessarily. Restored the 35-character headroom comment.
- Clamped `ualarm()` microseconds calculation to `> 0 ? usecs : 1` (patch 3)
  to prevent an infinite loop DoS if the float parses to 0.
- Hardened `perf_record.sh` (patch 4): used secure `mktemp` pathing, replaced
  unsafe `rm -f ${perfdata}` with `.old` rotation, and fixed bash globbing
  in the cleanup trap.
- Handled the JIT dump Python PID leak (patch 6) by injecting `os.getpid()`
  output to a `${PERF_DATA}.pid` tracker file so the trap can robustly clean
  it up even if `perf report` is empty.
- Avoided 32-bit `expr` math overflow in `stat_bpf_counters.sh` (patch 8)
  by ensuring division occurs before multiplication (`x / 20 * 3`).
- Lowered off-cpu `sleep 0.1` thresholds (patch 10) to 50ms (above) and
  500ms (below) for much higher robustness against jitter.
- Fixed a severe performance regression in `stat_all_metrics.sh` (patch 5)
  caught by the sashiko reviewer, where missing structural events
  ("Cannot resolve IDs", "<not supported>") were unconditionally retrying
  with a 0.1s penalty. The script now correctly parses these errors and skips
  before the retry.

Ian Rogers (12):
  perf parse-events: Restrict core PMU bypass to --cputype option
  perf test: Truncate test description to fit terminal width
  perf tests workloads: Support sub-second durations in noploop and
    thloop
  perf tests: Add robust record retry helper and use subsecond workloads
  perf tests: Skip metrics validation if system-wide recording lacks
    permission
  perf tests: Fix Python JIT dump profiling test failure
  perf tests: Fix flakiness in trace record and replay test
  perf tests: Fix flakiness in BPF counters test on hybrid systems
  perf tests: Fix flakiness in branch stack sampling tests
  perf tests: Speed up off-cpu profiling tests
  perf tests: Speed up lock contention analysis shell test
  perf tests: Speed up metrics checking shell tests

 tools/perf/builtin-script.c                   |   1 +
 tools/perf/builtin-stat.c                     |  12 +-
 tools/perf/tests/builtin-test.c               | 110 ++++++++---
 tools/perf/tests/expand-cgroup.c              |   2 +-
 tools/perf/tests/parse-events.c               |  11 +-
 tools/perf/tests/parse-metric.c               |   2 +-
 tools/perf/tests/pmu-events.c                 |   8 +-
 tools/perf/tests/shell/jitdump-python.sh      |  76 +++++---
 tools/perf/tests/shell/kvm.sh                 |  61 ++++---
 .../tests/shell/lib/perf_metric_validation.py |  11 +-
 tools/perf/tests/shell/lib/perf_record.sh     |  52 ++++++
 tools/perf/tests/shell/lock_contention.sh     |  30 +--
 tools/perf/tests/shell/pipe_test.sh           |   4 +-
 tools/perf/tests/shell/record.sh              | 172 +++++++++---------
 tools/perf/tests/shell/record_lbr.sh          |  50 +++--
 tools/perf/tests/shell/record_offcpu.sh       |  12 +-
 tools/perf/tests/shell/stat_all_metrics.sh    |  77 +++++---
 tools/perf/tests/shell/stat_all_pfm.sh        |   2 +-
 tools/perf/tests/shell/stat_bpf_counters.sh   |  28 ++-
 tools/perf/tests/shell/stat_metrics_values.sh |   9 +-
 tools/perf/tests/shell/test_brstack.sh        |   6 +-
 tools/perf/tests/shell/trace_record_replay.sh |  18 +-
 tools/perf/tests/workloads/noploop.c          |  17 +-
 tools/perf/tests/workloads/thloop.c           |  16 +-
 tools/perf/util/metricgroup.c                 |  23 ++-
 tools/perf/util/metricgroup.h                 |   4 +-
 tools/perf/util/parse-events.c                |  22 ++-
 tools/perf/util/parse-events.h                |  17 +-
 tools/perf/util/python.c                      |   2 +-
 29 files changed, 560 insertions(+), 295 deletions(-)
 create mode 100644 tools/perf/tests/shell/lib/perf_record.sh

-- 
2.54.0.1136.gdb2ca164c4-goog