[PATCH 04/17] sched/mmcid: Let a running joiner take its CID after the fixups

From: Mathieu Desnoyers

Date: Tue Nov 17 2026 - 11:17:50 EST


The switch to per CPU mode relies on the new user not running yet. The
comment above the code says so: the new user is not visible in the
system, and the task that forks it runs the fixup.

A failed execve() is missed. It rejoins its mm in
sched_mm_cid_after_execve(): the new user is the running task, and it
runs the fixup itself. If it is preempted before the fixup is done,
mm_get_cid() can loop forever at its schedule in, with the runqueue
lock held.

In the diagram below, the machine has two possible CPUs. A and B are
threads of one process, which is in per task mode with two CIDs. C is
the vfork() child of B. A holds mm_cid.mutex from sched_mm_cid_fork(A)
on.

CPU 0 CPU 1

A: execve()
sched_mm_cid_before_execve()
A leaves, users = 1
B: vfork()
sched_mm_cid_fork(C)
users = 2
C takes the last CID
A: execve() fails
sched_mm_cid_after_execve()
sched_mm_cid_fork(A)
users = 3, switch to per CPU mode
A is active and has no CID
A is preempted
A is scheduled in
mm_cid_schedin(A)
mm_get_cid() loops:
B and C hold both CIDs
C: exit()
sched_mm_cid_exit()
waits for mm_cid.mutex,
which A holds

Nothing ends the loop. Only the fixup could free a CID, and A is the
task that runs it.

A guest with two CPUs hangs on this sequence when a SCHED_FIFO task
wakes every 50us on CPU 0 to preempt A. CPU 0 stays in __schedule() and
CPU 1 in queued_spin_lock_slowpath(). With this change the same guest
completes 20000 attempts.

Fix this by keeping a running joiner inactive until the fixups are done.
A schedule in gives it no CID before that. sched_mm_cid_fork() then
marks it active, and it takes its CID under the runqueue lock.

Fixes: 653fda7ae73d ("sched/mmcid: Switch over to the new mechanism")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx>
Cc: stable@xxxxxxxxxxxxxxx
---
kernel/sched/core.c | 20 +++++++++++++-------
1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index ab68e8605eed..510cd775723b 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -11054,7 +11054,16 @@ static void sched_mm_cid_fork(struct task_struct *t)
return;
}

- if (!sched_mm_cid_add_user(t, mm)) {
+ flip = sched_mm_cid_add_user(t, mm);
+ /*
+ * A running joiner takes no CID at a schedule in before the
+ * fixups are done. The users it joins can hold every CID
+ * until then, and the fixups are its own work.
+ */
+ if (t == current)
+ t->mm_cid.active = 0;
+
+ if (!flip) {
if (!cid_on_cpu(mm->mm_cid.mode))
t->mm_cid.cid = mm_get_cid(mm);
break;
@@ -11064,7 +11073,6 @@ static void sched_mm_cid_fork(struct task_struct *t)
* Handle the mode change and transfer current's CID. A
* running joiner has no CID yet. It takes one below.
*/
- flip = true;
percpu = cid_on_cpu(mm->mm_cid.mode);
if (!percpu)
mm_cid_transit_to_task(current, pcp);
@@ -11077,18 +11085,16 @@ static void sched_mm_cid_fork(struct task_struct *t)
mm_cid_fixup_tasks_to_cpus();
} else {
mm_cid_fixup_cpus_to_tasks(mm);
- /*
- * A running joiner may have taken a CID at a
- * schedule in. If not, it takes one below.
- */
+ /* A running joiner takes its CID below. */
if (t != current)
t->mm_cid.cid = mm_get_cid(mm);
}
}

- /* A running joiner with no CID takes one now, see above. */
+ /* The running joiner takes part from here on, see above. */
if (t == current) {
guard(task_rq_lock)(t);
+ t->mm_cid.active = 1;
if (t->mm_cid.cid == MM_CID_UNSET)
mm_cid_schedin(t);
}
--
2.47.3