Re: [PATCH v2] rtla: clean up tracefs instances on SIGTERM, and test the abnormal exits
From: Tomas Glozar
Date: Mon Nov 16 2026 - 08:23:42 EST
On Wed, Nov 11, 2026 at 7:29 AM Chaosheng Qu
<chaosheng.qu.linux@xxxxxxxxx> wrote:
>
> rtla: clean up tracefs instances on SIGTERM, and test the abnormal exits
>
The commit summary is taken from the email's subject (without the
"[PATCH]" prefix), thus, it should not be repeated in the body. If I
apply this using "git am", I will get:
rtla: clean up tracefs instances on SIGTERM, and test the abnormal exits
rtla: clean up tracefs instances on SIGTERM, and test the abnormal exits
rtla creates a tracefs instance for each run and removes it on the normal
exit path. If the process is interrupted, killed by its own -d timer, or if
an --on-threshold shell action fails, the instance is left behind and the
next invocation finds it in use.
...
i.e. the commit summary duplicated.
> rtla creates a tracefs instance for each run and removes it on the normal
> exit path. If the process is interrupted, killed by its own -d timer, or if
> an --on-threshold shell action fails, the instance is left behind and the
> next invocation finds it in use.
>
> Install a SIGTERM handler next to the existing SIGINT one, so the same
> cleanup runs for both, and add a test that walks each of those paths and
> checks that the instance is gone afterwards.
>
I am confused. Interrupting the process and the "-d" timer send SIGINT
or SIGALRM respectively. As far as I am aware, those signals are
already handled correctly:
[root@cs9 tmp]# rtla timerlat top -q -c 0 -d 1s
Timer Latency
0 00:00:03 | IRQ Timer Latency (us) |
Thread Timer Latency (us) | Ret user Timer Latency (us)
CPU COUNT | cur min avg max | cur
min avg max | cur min avg max
0 #1000 | 0 0 0 18 | 2
2 3 22 | 3 2 4 30
---------------|----------------------------------------|----------------------------------------|---------------------------------------
ALL #1000 e0 | 0 0 18 |
2 3 22 | 2 4 30
[root@cs9 tmp]# ls /sys/kernel/tracing/instances/
[root@cs9 tmp]#
[root@cs9 tmp]# rtla timerlat top -q -c 0 -d 1s -T 1 --on-threshold
shell,command=false
Timer Latency
0 00:00:01 | IRQ Timer Latency (us) |
Thread Timer Latency (us) | Ret user Timer Latency (us)
CPU COUNT | cur min avg max | cur
min avg max | cur min avg max
0 #1 | 1 1 1 1 | 7
7 7 7 | - - - -
---------------|----------------------------------------|----------------------------------------|---------------------------------------
ALL #1 e0 | 1 1 1 |
7 7 7 | - - -
|[root@cs9 tmp]#
[root@cs9 tmp]# ls /sys/kernel/tracing/instances/
[root@cs9 tmp]#
[root@cs9 tmp]# rtla timerlat top -q -c 0
^C Timer Latency
0 00:00:02 | IRQ Timer Latency (us) |
Thread Timer Latency (us) | Ret user Timer Latency (us)
CPU COUNT | cur min avg max | cur
min avg max | cur min avg max
0 #605 | 1 0 5 2148 | 5
2 8 2159 | 6 2 9 2179
---------------|----------------------------------------|----------------------------------------|---------------------------------------
ALL #605 e0 | 0 5 2148 |
2 8 2159 | 2 9 2179
[root@cs9 tmp]# ls /sys/kernel/tracing/instances/
[root@cs9 tmp]#
SIGTERM is not related to either (but see my comment about SIGTERM down below).
> The test harness had two bugs, both reported by sashiko:
>
> - The availability probe ran "$RTLA" --version. rtla has no --version:
> main() only understands -h/--help and the three subcommands, and
> anything else falls through to rtla_usage(), which exits 129. The probe
> therefore always failed and the whole suite reported SKIP, so none of
> these cases ever ran. Probe --help instead, require exit 129 and the
> version banner, which is what the other rtla tests already do.
>
> - The TAP wrapper treated any line starting with "SKIP: " from the helper
> as a group-level skip. A subtest that could not cross the threshold on
> an idle machine printed one of those, which marked the entire test as
> skipped and left the loop below it unreachable. Per-case skips now say
> "note:", and only real group-level skips keep the SKIP: prefix.
>
Does this refer to fixes of bugs Sashiko found in the previous version
of the patch? If so, it shouldn't go into the commit description, but
below the "---" line, so that it won't be picked into the final commit
message.
> Assisted-by: LLM
> Signed-off-by: Chaosheng Qu <chaosheng.qu.linux@xxxxxxxxx>
>
> ---
Please add a link to the previous version of your patch (see here [1]
for an example), especially in cases like this one where the follow-up
patch has a different subject. I assume this supersedes both patches
in this thread [2]. Is that correct?
It appears to me that a portion of the information got dropped from
the commit message of [2] and it got replaced by a change log
referencing bugs found in v2 by Sashiko. A patch submission should
include all necessary information for review, including clear
references to older versions of itself.
[1] https://lore.kernel.org/linux-trace-kernel/20260929142944.1841462-1-tglozar@xxxxxxxxxx/
[2] https://lore.kernel.org/linux-trace-kernel/20261017122359.402696-1-chaosheng.qu.linux@xxxxxxxxx/
> diff --git a/tools/tracing/rtla/src/common.c b/tools/tracing/rtla/src/common.c
> index 8c7f5e7..b86a600 100644
> --- a/tools/tracing/rtla/src/common.c
> +++ b/tools/tracing/rtla/src/common.c
> @@ -39,6 +39,7 @@ static void stop_trace(int sig)
> static void set_signals(struct common_params *params)
> {
> signal(SIGINT, stop_trace);
> + signal(SIGTERM, stop_trace);
> if (params->duration) {
> signal(SIGALRM, stop_trace);
> alarm(params->duration);
> @@ -51,6 +52,7 @@ static void set_signals(struct common_params *params)
> static void unset_signals(struct common_params *params)
> {
> signal(SIGINT, SIG_DFL);
> + signal(SIGTERM, SIG_DFL);
> if (params->duration) {
> alarm(0);
> signal(SIGALRM, SIG_DFL);
I agree with also handling SIGTERM, which is sent for example by the
operating system using shutdown; for debugging purposes, SIGQUIT can
be used.
> diff --git a/tools/tracing/rtla/tests/cleanup.t b/tools/tracing/rtla/tests/cleanup.t
> new file mode 100644
> index 0000000..176b1a9
> --- /dev/null
> +++ b/tools/tracing/rtla/tests/cleanup.t
> @@ -0,0 +1,70 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Verify that rtla removes the tracefs instance it created on the abnormal
> +# exit paths users actually hit:
> +#
> +# - SIGINT during a run (interactive Ctrl-C)
> +# - SIGTERM during a run
> +# - rtla's own -d/--duration timer
> +# - a failing --on-threshold shell action
> +# - several interrupted runs in a row, followed by a normal run
> +#
> +# A leaked instance pollutes the next invocation.
> +source tests/engine.sh
> +test_begin
> +
> +set_timeout 2m
> +
> +#
> +# These cases need to run rtla in the background and signal it, which
> +# engine.sh's check() (one foreground command) cannot express.
> +#
> +run_cleanup_cases() {
> + local helper="$testdir/scripts/check-cleanup.sh"
> + local out rc timeout_cmd skip
> +
> + ctr=$((ctr + 1))
> + [ -z "$TEST_COUNT" ] && return
> +
> + if [ ! -x "$helper" ]; then
> + echo "not ok $ctr - cleanup helper is not executable"
> + return
> + fi
> +
> + # check() normally does this before every test. Do the same here, both to
> + # leave a clean tracer state and to make a hung helper terminate.
> + reset_osnoise || {
> + echo "ok $ctr - instances cleaned up on SIGINT, timeout and action failure # SKIP tracefs is not writable"
> + return
> + }
> + timeout_cmd=${TIMEOUT:-timeout -v -k 15s 2m}
> +
> + out=$($timeout_cmd "$helper" "$RTLA" 2>&1)
> + rc=$?
> + if [ $rc -eq 0 ]; then
> + # "SKIP:" is what the helper prints when it could not run at all
> + # (no writable tracefs, or the binary is unusable); a per-case skip
> + # says "note:" instead. So the first SKIP: line means nothing ran.
> + skip=$(printf '%s\n' "$out" | sed -n 's/^SKIP: //p' | head -n 1)
> + if [ -n "$skip" ]; then
> + echo "ok $ctr - instances cleaned up on SIGINT, timeout and action failure # SKIP $skip"
> + return
> + fi
> + printf '%s\n' "$out" | sed -n 's/^note: /# note: /p'
> + echo "ok $ctr - instances cleaned up on SIGINT, timeout and action failure"
> + elif [ $rc -eq 124 ] || [ $rc -eq 137 ]; then
> + echo "not ok $ctr - instances cleaned up on SIGINT, timeout and action failure"
> + echo "# helper timed out after 2m"
> + else
> + echo "not ok $ctr - instances cleaned up on SIGINT, timeout and action failure"
> + echo "# helper exited $rc"
> + printf '%s\n' "$out" | while read -r line; do
> + echo "# $line"
> + done
> + fi
> +}
> +
> +run_cleanup_cases
> +
> +test_end
> diff --git a/tools/tracing/rtla/tests/scripts/check-cleanup.sh b/tools/tracing/rtla/tests/scripts/check-cleanup.sh
> new file mode 100755
> index 0000000..c27db96
> --- /dev/null
> +++ b/tools/tracing/rtla/tests/scripts/check-cleanup.sh
> @@ -0,0 +1,211 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Verify that rtla removes the tracefs instances it created, even when it is
> +# interrupted (SIGINT/SIGTERM), killed by its own timeout, or when a shell
> +# action fails.
> +#
> +# This is a *test* for tools/tracing/rtla; it does not change rtla behaviour.
> +#
> +# Requires: root, tracefs mounted at /sys/kernel/tracing, CONFIG_OSNOISE_TRACER.
> +# Usage: rtla_cleanup_test.sh /path/to/rtla
> +
> +RTLA=${1:-rtla}
> +TRACEFS=${TRACEFS:-/sys/kernel/tracing}
> +INSTANCES=$TRACEFS/instances
> +FAIL=0
> +PASS=0
> +
> +log() { printf '%s\n' "$*"; }
> +pass() { PASS=$((PASS+1)); log "PASS: $*"; }
> +fail() { FAIL=$((FAIL+1)); log "FAIL: $*"; }
> +cleanup_all() {
> + for i in osnoise_top osnoise_hist timerlat_top timerlat_hist hwnoise_top; do
> + [ -d "$INSTANCES/$i" ] && rmdir "$INSTANCES/$i" 2>/dev/null
> + done
> +}
> +
> +# Wait up to $2 ms for $1 to disappear.
> +wait_gone() {
> + _path=$1
> + _ms=${2:-5000}
> + _i=0
> + while [ -d "$_path" ] && [ $_i -lt $_ms ]; do
> + sleep 0.05
> + _i=$((_i+50))
> + done
> + [ ! -d "$_path" ]
> +}
> +
> +# Wait until $1 exists (up to $2 ms).
> +wait_present() {
> + _path=$1
> + _ms=${2:-10000}
> + _i=0
> + while [ ! -d "$_path" ] && [ $_i -lt $_ms ]; do
> + sleep 0.05
> + _i=$((_i+50))
> + done
> + [ -d "$_path" ]
> +}
> +
> +if [ ! -d "$INSTANCES" ] || [ ! -w "$INSTANCES" ] ||
> + [ ! -d "$TRACEFS/osnoise" ]; then
> + log "SKIP: tracefs, writable instances or CONFIG_OSNOISE_TRACER unavailable"
> + exit 0
> +fi
> +
> +# Leave nothing behind from the test before it starts, and make sure the
> +# command under test is actually usable.
> +#
> +# rtla has no --version: main() only knows -h/--help and -usage, and anything
> +# else falls through to rtla_usage(), which exits 129 after printing the
> +# version banner. Probing --version therefore always failed and the whole test
> +# suite silently reported SKIP. Accept 129 as "this binary works".
> +cleanup_all
> +if [ ! -x "$RTLA" ]; then
> + log "SKIP: $RTLA is not executable"
> + exit 0
> +fi
> +help_out=$("$RTLA" --help 2>&1)
> +rtla_rc=$?
> +if [ "$rtla_rc" -ne 129 ] ||
> + ! printf '%s\n' "$help_out" | grep -q 'rtla version'; then
> + log "SKIP: $RTLA is not usable (--help exited $rtla_rc, no version banner)"
> + exit 0
> +fi
> +
> +# Kill a background rtla if a case fails before reaping it.
> +stop_pid() {
> + [ -n "$1" ] || return
> + kill -KILL "$1" 2>/dev/null
> + wait "$1" 2>/dev/null
> +}
> +
> +# Wait for an rtla instance and its signal handler. The instance appears
> +# before set_signals() runs, so wait for rtla to start its workload (or at
> +# least for it to survive a short grace period) before sending SIGINT.
> +wait_running() {
> + _path=$1
> + _pid=$2
> + wait_present "$_path" 10000 || return 1
> + sleep 1
> + kill -0 "$_pid" 2>/dev/null
> +}
> +
> +# ---------------------------------------------------------------------------
> +# 1. SIGINT during a long run
> +# ---------------------------------------------------------------------------
> +cleanup_all
> +$RTLA osnoise top -q -d 30s -P F:1 >/dev/null 2>&1 &
> +pid=$!
> +if ! wait_running "$INSTANCES/osnoise_top" "$pid"; then
> + fail "SIGINT: instance was never created (rtla did not start?)"
> + stop_pid "$pid"
> +else
> + kill -INT "$pid" 2>/dev/null
> + wait "$pid" 2>/dev/null
> + if wait_gone "$INSTANCES/osnoise_top"; then
> + pass "SIGINT: instance removed after interrupt"
> + else
> + fail "SIGINT: instance still present after interrupt"
> + fi
> +fi
> +
> +# ---------------------------------------------------------------------------
> +# 2. SIGTERM during a long run
> +# ---------------------------------------------------------------------------
> +cleanup_all
> +$RTLA osnoise top -q -d 30s -P F:1 >/dev/null 2>&1 &
> +pid=$!
> +if ! wait_running "$INSTANCES/osnoise_top" "$pid"; then
> + fail "SIGTERM: instance was never created (rtla did not start)"
> + stop_pid "$pid"
> +else
> + kill -TERM "$pid" 2>/dev/null
> + wait "$pid" 2>/dev/null
> + if wait_gone "$INSTANCES/osnoise_top"; then
> + pass "SIGTERM: instance removed after terminate"
> + else
> + fail "SIGTERM: instance still present after terminate"
> + fi
> +fi
> +
> +# ---------------------------------------------------------------------------
> +# 3. rtla's own -d timeout path
> +# ---------------------------------------------------------------------------
> +cleanup_all
> +$RTLA osnoise top -q -d 2s -P F:1 >/dev/null 2>&1
> +rc=$?
> +if [ "$rc" -ne 0 ]; then
> + fail "self-timeout: rtla failed to start or run (exit=$rc)"
> +elif wait_gone "$INSTANCES/osnoise_top"; then
> + pass "self-timeout: instance removed (exit=$rc)"
> +else
> + fail "self-timeout: instance still present (exit=$rc)"
> +fi
> +
> +# ---------------------------------------------------------------------------
> +# 4. shell action failure (-a / --on-threshold shell,command=...)
> +# ---------------------------------------------------------------------------
> +# -S is the stop-total threshold and yields exit 2 when the action fails.
> +# A shell action only runs if that threshold is crossed; an idle machine may
> +# never do so. Do not call the no-threshold case a pass.
> +cleanup_all
> +action_rc=0
> +for n in 1 2 3; do
> + $RTLA osnoise top -q -d 5s -P F:1 -r 900000 -S 1 \
> + --on-threshold 'shell,command="/bin/false"' >/dev/null 2>&1
> + rc=$?
> + action_rc=$rc
> + [ "$rc" -eq 2 ] && break
> +done
> +if [ "$action_rc" -eq 0 ]; then
> + # Not a group-level skip: the other cases still ran. Prefix it so the
> + # TAP wrapper does not report the whole test as skipped.
> + log "note: action-failure: stop-total threshold was not crossed"
> +elif [ "$action_rc" -ne 2 ]; then
> + fail "action-failure: unexpected exit=$action_rc (wanted 2)"
> +elif wait_gone "$INSTANCES/osnoise_top"; then
> + pass "action-failure: instance removed (exit=$action_rc)"
> +else
> + fail "action-failure: instance still present (exit=$action_rc)"
> +fi
> +
> +# ---------------------------------------------------------------------------
> +# 5. repeated interrupted runs must not poison the next run
> +# ---------------------------------------------------------------------------
> +cleanup_all
> +for n in 1 2 3; do
> + $RTLA osnoise top -q -d 20s -P F:1 >/dev/null 2>&1 &
> + pid=$!
> + if ! wait_running "$INSTANCES/osnoise_top" "$pid"; then
> + fail "repeated interrupts: run $n never created an instance"
> + stop_pid "$pid"
> + continue
> + fi
> + kill -INT "$pid" 2>/dev/null
> + wait "$pid" 2>/dev/null
> + if ! wait_gone "$INSTANCES/osnoise_top"; then
> + fail "repeated interrupts: run $n left an instance behind"
> + cleanup_all
> + fi
> +done
> +
> +$RTLA osnoise top -q -d 2s -P F:1 >/dev/null 2>&1
> +rc=$?
> +if [ "$rc" -eq 0 ]; then
> + pass "repeated interrupts: following run still succeeds"
> +else
> + fail "repeated interrupts: following run failed (exit=$rc)"
> +fi
> +if wait_gone "$INSTANCES/osnoise_top"; then
> + pass "repeated interrupts: no leftover instance"
> +else
> + fail "repeated interrupts: leftover instance"
> +fi
> +
> +cleanup_all
> +log ""
> +log "SUMMARY: $PASS passed, $FAIL failed"
> +[ "$FAIL" -eq 0 ]
>
Please send the test changes in their own commit so I can review them
separately.
Thanks,
Tomas