[RFC PATCH 3/3] NOT_FOR_INCLUSION: selftests/resctrl: Tasks keep using removed RMID after fork() reproducer

From: Reinette Chatre

Date: Fri Nov 20 2026 - 13:40:00 EST


From: Jedrzej Wasiukiewicz <jedrzej.wasiukiewicz@xxxxxxxxx>

Introduce stray_tasks_repro.sh that demonstrates how a task forked while
its monitor group is being removed may continue to use the removed RMID.

What this script does:
1. Starts 8 processes inside a new monitor group "m" that fork sleeping
children as fast as they can.
2. Removes "m" while they are forking.
3. Creates new monitor groups until no RMID is free, so the RMID freed in
step 2 gets a new owner. Nothing is ever moved into these groups.
4. Reports every task found in them. Each one carries the removed group's
RMID and is a stray. The expected result is none.

To avoid the freed RMID landing in limbo, the script requires that LLC
occupancy monitoring is disabled (on x86, boot with "rdt=!cmt").

Example of issue encountered:

monitor groups created: 566, stopped on: No space left on device
tasks in new monitor groups (expect none): 1
/sys/fs/resctrl/c/mon_groups/n565/tasks
RESULT: 1 stray task(s) kept the removed group's RMID

See script header for more detail.

Signed-off-by: Jedrzej Wasiukiewicz <jedrzej.wasiukiewicz@xxxxxxxxx>
Signed-off-by: Reinette Chatre <reinette.chatre@xxxxxxxxx>
---
.../selftests/resctrl/stray_tasks_repro.sh | 122 ++++++++++++++++++
1 file changed, 122 insertions(+)
create mode 100755 tools/testing/selftests/resctrl/stray_tasks_repro.sh

diff --git a/tools/testing/selftests/resctrl/stray_tasks_repro.sh b/tools/testing/selftests/resctrl/stray_tasks_repro.sh
new file mode 100755
index 000000000000..42f65d7d7bb4
--- /dev/null
+++ b/tools/testing/selftests/resctrl/stray_tasks_repro.sh
@@ -0,0 +1,122 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# resctrl: a task forked while its monitor group is being removed keeps the
+# removed group's RMID.
+#
+# rdt_move_group_tasks() moves a removed group's tasks to the parent group
+# under read_lock(&tasklist_lock). That lock only delays a new child from
+# joining the task list. The child copies its parent's closid/rmid earlier,
+# in dup_task_struct(), without the lock. A child whose fork() is in flight
+# when the walk starts therefore joins the list afterwards with the removed
+# group's RMID, and is never moved.
+#
+# What this script does:
+# 1. Starts 8 processes inside a new monitor group "m" that fork sleeping
+# children as fast as they can.
+# 2. Removes "m" while they are forking.
+# 3. Creates new monitor groups until no RMID is free, so the RMID freed in
+# step 2 gets a new owner. Nothing is ever moved into these groups.
+# 4. Reports every task found in them. Each one carries the removed group's
+# RMID and is a stray. The expected result is none.
+#
+# Requirements: bash, root, resctrl mounted with monitoring, and the L3
+# occupancy event (llc_occupancy) off. Memory bandwidth events (mbm_*) are
+# enough.
+# mount -t resctrl resctrl /sys/fs/resctrl
+# With llc_occupancy a freed RMID waits in "limbo" until the cache lines tagged
+# with it are gone, so it may not be reused and the race cannot be seen. Without
+# that event a freed RMID is reusable at once. To turn the event off, boot with
+# the kernel parameter rdt=!cmt. The script refuses to run when llc_occupancy is
+# listed in info/L3_MON/mon_features.
+#
+# Output (example of a hit):
+# monitor groups created: 566, stopped on: No space left on device
+# tasks in new monitor groups (expect none): 1
+# /sys/fs/resctrl/c/mon_groups/n565/tasks
+# RESULT: 1 stray task(s) kept the removed group's RMID
+#
+# The race is probabilistic: run it several times. "no stray this run" is not
+# a pass. If the run stops on anything but "No space left on device", the
+# freed RMID was never reused, so strays cannot be seen. The script then
+# reports RESULT: INCONCLUSIVE and the run counts as no data.
+#
+# Side effects: creates and removes the control group "c" and its monitor
+# groups (it refuses to run if "c" exists) and starts thousands of short-lived
+# sleeping processes. Other resctrl groups are not touched, but while the RMID
+# pool is full anyone else creating a monitor group gets ENOSPC.
+
+R=/sys/fs/resctrl
+C=$R/c # control group used for the test
+M=$C/mon_groups/m # monitor group that gets removed
+MF=$R/info/L3_MON/mon_features
+
+[ "$(id -u)" = 0 ] || { echo "run as root"; exit 1; }
+[ -f $MF ] || { echo "resctrl with monitoring is not mounted at $R"; exit 1; }
+if grep -q llc_occupancy $MF; then
+ echo "llc_occupancy is enabled: freed RMIDs can sit in limbo."
+ echo "Boot with the kernel parameter rdt=!cmt."
+ exit 1
+fi
+[ ! -e $C ] || { echo "$C already exists, remove it first"; exit 1; }
+
+FORKERS=() # PIDs of the forker processes
+KIDS="" # PIDs of their children
+
+# Runs on every exit, including Ctrl-C: stop all test processes and remove the
+# test groups.
+cleanup() {
+ trap - EXIT INT TERM
+ [ -n "$KIDS" ] || [ ${#FORKERS[@]} -eq 0 ] ||
+ KIDS=$(pgrep -P "$(IFS=,; echo "${FORKERS[*]}")")
+ [ ${#FORKERS[@]} -eq 0 ] || kill "${FORKERS[@]}" 2>/dev/null
+ [ -z "$KIDS" ] || kill $KIDS 2>/dev/null
+ sleep 1
+ rmdir $C/mon_groups/* 2>/dev/null
+ rmdir $C 2>/dev/null
+}
+trap cleanup EXIT
+trap 'exit 130' INT TERM
+
+mkdir $C $M || { echo "cannot create $C and $M"; exit 1; }
+
+# 8 forkers. Each joins c, then m, and then forks in a loop: a sleeping child
+# that lives 20 s, plus a very short one that keeps the fork rate high.
+for i in 1 2 3 4 5 6 7 8; do
+ ( echo $BASHPID > $C/tasks; echo $BASHPID > $M/tasks
+ while :; do sleep 20 & sleep 0.0003; done ) &
+ FORKERS+=($!)
+done
+sleep 0.3 # let them start forking
+
+# The race window is while this walks the task list.
+rmdir $M
+
+# Remember the children for cleanup, then stop the forkers. The children stay.
+KIDS=$(pgrep -P "$(IFS=,; echo "${FORKERS[*]}")")
+kill "${FORKERS[@]}" 2>/dev/null
+sleep 2 # let forks that are still in flight finish
+
+# Take every free RMID. Stop on the first error, normally ENOSPC (pool full).
+n=0
+while err=$(mkdir $C/mon_groups/n$n 2>&1); do n=$((n+1)); done
+echo "monitor groups created: $n, stopped on: ${err##*: }"
+
+# Nothing was ever put into n*, so any task listed here is a stray.
+strays=$(cat $C/mon_groups/n*/tasks 2>/dev/null | wc -l)
+echo "tasks in new monitor groups (expect none): $strays"
+if [ "$strays" -gt 0 ]; then
+ grep -l . $C/mon_groups/n*/tasks # which group holds them
+ echo "RESULT: $strays stray task(s) kept the removed group's RMID"
+ exit 0
+fi
+
+case $err in
+*"No space left"*)
+ echo "RESULT: no stray this run, run again"
+ exit 0 ;;
+esac
+
+# Not ENOSPC: the pool was never full, so the freed RMID was not reused and a
+# stray cannot be seen in this run.
+echo "RESULT: INCONCLUSIVE, the freed RMID was never reused (${err##*: })"
--
2.56.0