[PATCH v4 0/3] perf sched latency: Refine outputs, unit scaling, and histogram support
From: Aaron Tomlin
Date: Wed Jul 29 2026 - 11:21:10 EST
This patch series improves 'perf sched latency' by fixing misleading empty
table output, introducing dynamic unit auto-scaling for latency and runtime
statistics, and adding latency histogram visualisation along with time-span
filtering.
Patch 1 fixes an issue where 'perf sched latency' would fall through and
return success (0) when perf_session__has_traces() failed due to missing
tracepoint events in perf.data. This resulted in empty header tables and
zeroed summary statistics being printed. To support pipe mode
(i.e., 'perf sched latency -i -'), missing callbacks (i.e., .attr,
.tracing_data, .build_id) are registered in cmd_sched(), and trace checks
are performed post-processing when handling pipe streams. Additionally,
map_switch_event() is guarded against potential NULL pointer dereferences
under memory allocation failures.
Patch 2 introduces dynamic auto-scaling for latency and runtime display
columns (Runtime, Avg delay, Max delay). Previously, all values were
unconditionally formatted in milliseconds (ms), making microsecond or
second-scale latencies hard to read. Columns are now scaled dynamically to
the most appropriate unit (ns, us, ms, s), column headers are updated, and
format specifiers (%14s for Runtime, %11s for Avg/Max delay) are aligned
character-for-character with the table headers.
Patch 3 adds three new command-line options to 'perf sched latency':
--histogram (-H):
Displays an ASCII bar chart of CPU wait latencies between
snapshots
--hist-mode:
Configures the bucketing scheme to either logarithmic (log) or
100 us equal-width linear (linear) mode
--time:
Filters event processing to a specified [start,stop] time span
Changes since v3:
- Registered Missing Callbacks. In perf_tool__init configuration inside
cmd_sched(), added the .attr, .tracing_data, and .build_id callbacks.
Without these callbacks, pipe mode drops header attributes entirely,
preventing tracepoints from being populated in session->evlist
- Introduced an explicit post-processing pipe check. This ensures that
pipe mode aborts correctly and does not produce superfluous empty
latency tables when no trace samples are available
- Prevented potential NULL pointer dereference. Added a NULL check for
thread__get_runtime() in map_switch_event() under memory allocation
failures
- Fixed column alignment. Modified format specifiers to align
character-for-character with header column widths across all rows
- Adjusted header label padding. Updated header label padding and column
width delimiters in perf_sched__lat() to match the exact field format
specifiers printed in output_lat_thread()
- Fixed swapper histogram inclusion. Excluded "swapper" (i.e.,
CPU-specific idle thread) from global_hist buckets
- Preserved task state machine across --time bounds. Moved time-window
filtering inside add_sched_in_event() to gate latency statistics
recording without breaking wakeup state tracking across time interval
boundaries
- Link to v3: https://lore.kernel.org/lkml/20260726032533.712462-1-atomlin@xxxxxxxxxxx/
Changes since v2:
- Ensured vertical pipe separators align across all latency table columns
- Excluded "swapper" (idle thread) latency samples from global_hist
- Preserved task state machine transitions across --time boundaries
- Suppressed empty table headers, total lines, and histogram graphs when
no matching trace samples exist (e.g., when --time, --CPU, or --pids
exclude all samples), outputting "No matching trace samples found."
instead
- Linked to v2: https://lore.kernel.org/lkml/20260725173341.679782-1-atomlin@xxxxxxxxxxx/
Changes since v1:
- Fixed integer overflow in latency_bucket() (Ian Rogers)
- Linked to v1: https://lore.kernel.org/lkml/20260724142901.634761-1-atomlin@xxxxxxxxxxx/
Aaron Tomlin (3):
perf sched: Suppress latency table output when trace samples are
missing
perf sched latency: Auto-scale latency and runtime display units
perf sched latency: Add histogram and time interval options
tools/perf/Documentation/perf-sched.txt | 6 +
tools/perf/builtin-sched.c | 263 +++++++++++++++++++++---
2 files changed, 244 insertions(+), 25 deletions(-)
--
2.55.0