[PATCH 2/2] sched/core: Don't skip a pending core-wide pick on idle re-entry

From: Ibrahim Elshahed

Date: Tue Nov 17 2026 - 09:27:35 EST


Since v6.12, with commit 3dcac251b066 ("sched/core: Introduce SM_IDLE
and an idle re-entry fast-path in __schedule()"), an idle CPU with an
empty runqueue skips pick_next_task() when it is rescheduled. Under
core scheduling that pick also consumes the pick a sibling's core-wide
selection left in rq->core_pick and ends or re-examines the core's
forced idle.

A core-wide selection chooses a task for an idle sibling and reschedules
it. If the chosen task is migrated away before the sibling schedules
and it was the sibling's only task, the sibling goes back to idle
through the fast path without consuming its pick. When that selection
also forced the other sibling idle, because its runnable tasks did not
match the chosen task's cookie, the core stays forced idle with nothing
running on it, and two things go wrong:

- Without the previous patch, the idle sibling's rq->core_pick keeps
pointing at the migrated task, and with schedstats enabled the forced
idle accounting keeps writing through it, even after the task and its
cgroup are freed:

BUG: unable to handle page fault for address: ffff8d6b205f7018
#PF: supervisor write access in kernel mode
#PF: error_code(0x0002) - not-present page
RIP: 0010:__cgroup_account_cputime_field+0x3c/0x70
Call Trace:
__sched_core_account_forceidle+0xd8/0x140
__schedule+0x1390/0x18d0
schedule_idle+0x1a/0x30
cpu_startup_entry+0x29/0x30
start_secondary+0x11e/0x140

- The forced-idle sibling keeps waiting with runnable tasks until some
unrelated event reschedules the core. Likewise, when the forced-idle
CPU itself loses its last task, the reschedule that
sched_core_dequeue() sends to end the forced idle also ends in the
fast path.

Take the SM_IDLE fast path only when core scheduling has neither a
pick pending for this CPU nor forced idle to settle.

Fixes: 3dcac251b066 ("sched/core: Introduce SM_IDLE and an idle re-entry fast-path in __schedule()")
Cc: stable@xxxxxxxxxxxxxxx
Co-developed-by: Roman Kagan <rkagan@xxxxxxxxx>
Signed-off-by: Roman Kagan <rkagan@xxxxxxxxx>
Signed-off-by: Ibrahim Elshahed <lshahed@xxxxxxxxx>
---
kernel/sched/core.c | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index b1718ba5a4..58b429bbe3 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -347,6 +347,17 @@ void sched_core_dequeue(struct rq *rq, struct task_struct *p, int flags)
resched_curr(rq);
}

+/*
+ * An idle CPU with an empty runqueue must still call pick_next_task() while
+ * it has a pending core-wide pick or the core is forced idle. Only that pick
+ * consumes rq->core_pick and ends the forced idle.
+ */
+static inline bool sched_core_idle_needs_pick(struct rq *rq)
+{
+ return sched_core_enabled(rq) &&
+ (rq->core_pick || rq->core->core_forceidle_count);
+}
+
static int sched_task_is_throttled(struct task_struct *p, int cpu)
{
if (p->sched_class->task_is_throttled)
@@ -550,6 +561,7 @@ void sched_core_put(void)
static inline void sched_core_enqueue(struct rq *rq, struct task_struct *p) { }
static inline void
sched_core_dequeue(struct rq *rq, struct task_struct *p, int flags) { }
+static inline bool sched_core_idle_needs_pick(struct rq *rq) { return false; }

#endif /* !CONFIG_SCHED_CORE */

@@ -7285,8 +7297,13 @@ static void __sched notrace __schedule(int sched_mode)
*/
prev_state = READ_ONCE(prev->__state);
if (sched_mode == SM_IDLE) {
- /* SCX must consult the BPF scheduler to tell if rq is empty */
- if (!rq->nr_running && !scx_enabled()) {
+ /*
+ * SCX must consult the BPF scheduler to tell if rq is empty,
+ * and core scheduling may need pick_next_task() to consume
+ * rq->core_pick or to end forced idle.
+ */
+ if (!rq->nr_running && !scx_enabled() &&
+ !sched_core_idle_needs_pick(rq)) {
next = prev;
rq->next_class = &idle_sched_class;
goto picked;
--
2.47.3