[PATCH v5 23/29] sched/deadline: Move a task's bandwidth when it must leave its CPU

From: Mathieu Desnoyers

Date: Sun Nov 08 2026 - 14:03:09 EST


When a CPU goes offline, a deadline task asleep on it moves at its
wakeup. select_fallback_rq() picks the new CPU. balance_push() moves a
queued task the same way. The new CPU can be in another root domain.
With isolcpus="domain", the isolated CPUs are in the default root
domain, and the housekeeping CPUs in another one. A cpuset that is not
a partition can hold both.

Commit 59d06cea1198 ("sched/deadline: Fix bandwidth accounting at all
levels after offline migration") fixed this for a throttled task.
dl_task_offline_migration() moves its bandwidth with it. A task that
select_fallback_rq() places still moves no bandwidth.

The rebuild of the accounting at the offline runs before the wakeup.
So the task runs in a root domain that does not count its bandwidth,
until the next rebuild. That root domain can admit deadline tasks
beyond its capacity.

Fix this in migrate_task_rq_dl(). When a task leaves a CPU that is
inactive or not in its mask, move its bandwidth to the root domain of
the new CPU. Record that CPU. As dl_task_offline_migration() does, add
the bandwidth even without room, since the task runs there. Add it also
if a rebuild left the task out.

This also covers a change of the mask that made no room for the task,
such as sched_setaffinity() without admission control. The bandwidth
now moves when the task leaves its CPU. Before, it stayed in the old
root domain until the next rebuild.

migrate_task_rq_dl() runs under the task's pi_lock for a wakeup, and
under its rq lock otherwise. With dl_departed_lock, either keeps the
record stable.

The hotplug_fallback cases of deadline_bw_test, added later in this
series, fail without this patch. A task woken on an isolated CPU stays
counted in the housekeeping root domain. This follows the Sashiko AI
review of v4.

Fixes: 59d06cea1198 ("sched/deadline: Fix bandwidth accounting at all levels after offline migration")
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx>
---
kernel/sched/deadline.c | 58 ++++++++++++++++++++++++++++++++++++-----
1 file changed, 51 insertions(+), 7 deletions(-)

diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index bc31045e4686..a35609a10ec5 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -2873,11 +2873,54 @@ select_task_rq_dl(struct task_struct *p, int cpu, int flags)
return cpu;
}

-static void migrate_task_rq_dl(struct task_struct *p, int new_cpu __maybe_unused)
+/*
+ * Move the bandwidth of @p to the root domain of @new_cpu. Count it there
+ * also if a rebuild left @p out. Add it even without room, since @p runs
+ * there. The caller holds the rq lock of @p, or its pi_lock for a wakeup.
+ */
+static void dl_task_move_bw(struct task_struct *p, int new_cpu)
+{
+ struct dl_bw *dl_b;
+ int cpu;
+
+ if (dl_entity_is_special(&p->dl) || !p->dl.dl_bw)
+ return;
+
+ guard(raw_spinlock)(&dl_departed_lock);
+ cpu = p->dl.dl_bw_cpu;
+ if (cpu < 0)
+ return;
+ if (p->dl.dl_bw_gen == dl_bw_gen) {
+ if (cpu_rq(cpu)->rd == cpu_rq(new_cpu)->rd)
+ return;
+ dl_b = dl_bw_of(cpu);
+ raw_spin_lock(&dl_b->lock);
+ __dl_sub(dl_b, p->dl.dl_bw, dl_bw_cpus(cpu));
+ raw_spin_unlock(&dl_b->lock);
+ }
+ dl_b = dl_bw_of(new_cpu);
+ raw_spin_lock(&dl_b->lock);
+ __dl_add(dl_b, p->dl.dl_bw, dl_bw_cpus(new_cpu));
+ raw_spin_unlock(&dl_b->lock);
+ p->dl.dl_bw_cpu = new_cpu;
+ p->dl.dl_bw_gen = dl_bw_gen;
+}
+
+static void migrate_task_rq_dl(struct task_struct *p, int new_cpu)
{
+ int cpu = task_cpu(p);
struct rq_flags rf;
struct rq *rq;

+ /*
+ * The task must leave a CPU that is no longer active, or that its
+ * mask no longer holds. The new CPU can then be in another root
+ * domain. Move the bandwidth there, as dl_task_offline_migration()
+ * does for a throttled task.
+ */
+ if (unlikely(!cpu_active(cpu) || !cpumask_test_cpu(cpu, p->cpus_ptr)))
+ dl_task_move_bw(p, new_cpu);
+
if (READ_ONCE(p->__state) != TASK_WAKING)
return;

@@ -3577,9 +3620,10 @@ static void set_cpus_allowed_dl(struct task_struct *p,
* (different root_domains) entails a bandwidth update. Move the
* bandwidth only if the caller made room for it in the destination,
* as SCA_DL_BW_RESERVED says. Only cpuset_attach_task() sets it.
- * Otherwise, the bandwidth stays where it is counted, until the next
- * rebuild of the accounting. Moving it without room would leave no
- * root domain counting it.
+ * Otherwise, the bandwidth stays where it is counted, until the task
+ * leaves a CPU that the new mask does not hold (migrate_task_rq_dl()),
+ * or the next rebuild of the accounting. Freeing it here without room
+ * would leave no root domain counting it.
*
* The caller made room in the root domain of ctx->dl_bw_cpu, the CPU
* it expects the move to record. Move the bandwidth only if the move
@@ -3734,7 +3778,7 @@ static void dl_get_task_effective_cpus(struct task_struct *p, struct cpumask *cp
* task runs, or wakes up, in the root domain of that CPU. A mask can hold
* CPUs of two root domains: housekeeping CPUs and CPUs isolated by
* isolcpus="domain". When every CPU of a mask goes offline,
- * select_fallback_rq() can widen it to both. It moves no bandwidth.
+ * select_fallback_rq() can widen it to both.
*
* Else prefer the CPU recorded with the bandwidth, if it is active and in
* the mask. It is in the root domain that admitted the bandwidth, or that
@@ -3801,8 +3845,8 @@ void dl_add_task_root_domain(struct task_struct *p)
/*
* No active CPU can count the bandwidth of this task. Leave it
* out of this rebuild. dl_bw_gen then keeps its release from
- * removing anything. A later rebuild, or
- * dl_task_offline_migration(), counts it again.
+ * removing anything. A later rebuild counts it again. So does
+ * the move of the task to an active CPU.
*/
raw_spin_unlock_irqrestore(&p->pi_lock, rf.flags);
return;
--
2.47.3