[PATCH v10 35/40] sched/deadline: Check the runtime of a task whose parameters change
From: Mathieu Desnoyers
Date: Fri Nov 20 2026 - 15:48:16 EST
Commit 40767b0dc768 ("sched/deadline: Fix deadline parameter
modification handling") lets new parameters take effect at the next
period. So sched_setattr() cannot give a task a fresh runtime.
But admission control counts the new bandwidth at once. When a task
shrinks its reservation, the runtime left from the old one is not
checked against the new one. A comment in sched_dl_overflow() names
the gap: the old bandwidth should stay counted until the 0-lag time.
So a running task can shrink its reservation, and still run on the old
one. Other tasks can meanwhile be admitted into the bandwidth it gave
back. On v7.4-rc2, a task of 50ms every 100ms changed to 10ms every
100ms. It ran 39ms of the next 100ms.
Fix this by applying the wakeup rule after a change of parameters, as
the previous patch does for a task that enters the class.
__setparam_dl() marks a change of the runtime, the deadline or the
period in dl_new_params. The next enqueue then calls update_dl_entity().
It starts a new period, or reduces the runtime of a constrained task,
if the runtime left does not fit. The task gets no more than a wakeup
would give it. A larger reservation still takes effect at the next
period.
The enqueue flags cannot tell a change of parameters: a cgroup move
also uses ENQUEUE_MOVE. Without the mark, a task that waited behind
others could fail the check, and get a later deadline for no reason.
A boosted task runs on the parameters of its donor. So it keeps the mark
until its deboost. dl_deboost() checks it there, before
rt_mutex_setprio() compares the task with its top waiter. Until then,
the task has the deadline of its old parameters. So rt_mutex_setprio()
deboosts a marked task first, even if its top waiter has the earlier
deadline. It then compares the waiter with the deadline from the check,
and boosts the task again if needed.
dl_deboost() leaves the check to the next enqueue in two cases:
- a constrained task past its deadline and before its next period: the
enqueue throttles it first.
- a task asleep on anything but an rt_mutex: a period started at the
deboost would run while it sleeps.
A task blocked on an rt_mutex is not enqueued before its wakeup, but its
waiter node holds its deadline. So sched_setattr() checks the runtime of
such a task at once, in dl_check_blocked_params(), also when the task
comes back to the class. rt_mutex_adjust_pi() then requeues its waiter.
prio_changed_dl() does not check it: a cgroup move or an affinity change
gets there too, and does not requeue the waiter.
Until the check, the 0-lag time of a task that blocks mixes its old
runtime and deadline with its new parameters. When the runtime left
does not fit them, that 0-lag time has passed. The bandwidth is then
released at once, as the check would do. A comment in
dl_time_to_zerolag() now says so.
The change_running_new_params case, added to deadline_bw_test later in
this series, tests this. change_boosted_new_params tests a task that
changes its parameters while boosted. Both fail on v7.4-rc2 and pass
with this patch. deboost_new_params_reboost and boost_new_params_stale
test the check at the deboost. setattr_blocked_new_period tests a task
blocked on an rt_mutex. Without dl_check_blocked_params(), it took the
lock after a waiter with a later deadline, in every run.
Fixes: 40767b0dc768 ("sched/deadline: Fix deadline parameter modification handling")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx>
Cc: stable@xxxxxxxxxxxxxxx
---
include/linux/sched.h | 5 ++
kernel/sched/core.c | 6 +++
kernel/sched/deadline.c | 106 ++++++++++++++++++++++++++++++++++++----
kernel/sched/sched.h | 1 +
kernel/sched/syscalls.c | 3 ++
5 files changed, 112 insertions(+), 9 deletions(-)
diff --git a/include/linux/sched.h b/include/linux/sched.h
index cbd387993607..491ddeea7d65 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -715,6 +715,10 @@ struct sched_dl_entity {
* @dl_bw_oncpu tells if the this_bw of a runqueue counts the task's
* @dl_bw: add_rq_bw() and sub_rq_bw() set and clear it, under that
* runqueue's lock. Not for servers, which keep @dl_bw_attached.
+ *
+ * @dl_new_params tells if sched_setattr() changed the runtime, the
+ * deadline or the period. The next enqueue that is not boosted, or
+ * the deboost, checks the runtime left against them, and clears it.
*/
unsigned int dl_throttled : 1;
unsigned int dl_yielded : 1;
@@ -729,6 +733,7 @@ struct sched_dl_entity {
unsigned int dl_defer_idle : 1;
unsigned int dl_bw_attached : 1;
unsigned int dl_bw_oncpu : 1;
+ unsigned int dl_new_params : 1;
/*
* Bandwidth enforcement timer. Each -deadline task has its
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 75bfde7046fb..0dce129b4146 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7966,8 +7966,14 @@ void rt_mutex_setprio(struct task_struct *p, struct task_struct *pi_task)
* running task
*/
if (dl_prio(prio)) {
+ /*
+ * A boosted task that changed its parameters still has
+ * the deadline of its old ones. Deboost it first, so
+ * that its top waiter is compared with its new one.
+ */
if (!dl_prio(p->normal_prio) ||
(pi_task && dl_prio(pi_task->prio) &&
+ !(p->dl.dl_new_params && p->dl.pi_se != &p->dl) &&
dl_entity_preempt(&pi_task->dl, &p->dl))) {
struct sched_dl_entity *old_donor = p->dl.pi_se;
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 83aefe5ab671..fe48137509ad 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -93,6 +93,19 @@ static inline void dl_start_own_period(struct sched_dl_entity *dl_se)
dl_se->runtime = dl_se->dl_runtime;
}
+/*
+ * Return true if @dl_se has a constrained deadline, and the rq clock is
+ * past its deadline and before its next period. An enqueue then throttles
+ * it until that period.
+ */
+static inline bool dl_in_constrained_gap(struct sched_dl_entity *dl_se)
+{
+ u64 now = rq_clock(rq_of_dl_se(dl_se));
+
+ return !dl_is_implicit(dl_se) && dl_time_before(dl_se->deadline, now) &&
+ dl_time_before(now, dl_se->deadline - dl_se->dl_deadline + dl_se->dl_period);
+}
+
#ifdef CONFIG_RT_MUTEXES
static inline struct sched_dl_entity *pi_of(struct sched_dl_entity *dl_se)
{
@@ -116,6 +129,14 @@ static inline void dl_note_donor_refill(struct sched_dl_entity *dl_se)
static void task_non_contending(struct sched_dl_entity *dl_se, bool dl_task);
+static void update_dl_entity(struct sched_dl_entity *dl_se);
+
+/* Return true if @p waits on an rt_mutex. The caller holds its pi_lock. */
+static inline bool dl_task_pi_blocked(struct task_struct *p)
+{
+ return p->pi_blocked_on;
+}
+
/*
* End the boost of @dl_se, whose task stays in the class. If the boost
* refilled its runtime from the donor's parameters, start a new period
@@ -124,11 +145,22 @@ static void task_non_contending(struct sched_dl_entity *dl_se, bool dl_task);
* first. Without a refill, the task keeps the runtime and the deadline
* of its own reservation.
*
+ * If sched_setattr() changed the parameters during the boost, check the
+ * runtime left against them now, for the same reason. rt_mutex_setprio()
+ * then compares the result with the top waiter. Leave that check to the
+ * next enqueue for a task with a constrained deadline, past its deadline
+ * and before its next period. The enqueue first throttles it until that
+ * period: see dl_check_constrained_dl(). Leave it to the wakeup too, for
+ * a task asleep on anything but an rt_mutex: a period started now would
+ * run while it sleeps.
+ *
* The caller holds the rq lock and the pi_lock of the task, and has
* updated the rq clock.
*/
void dl_deboost(struct sched_dl_entity *dl_se)
{
+ struct task_struct *p = dl_task_of(dl_se);
+
if (dl_se->dl_donor_refill) {
dl_start_own_period(dl_se);
/*
@@ -145,12 +177,41 @@ void dl_deboost(struct sched_dl_entity *dl_se)
}
}
dl_drop_boost(dl_se);
+
+ if (!task_on_rq_queued(p) && !dl_task_pi_blocked(p))
+ return;
+ if (dl_se->dl_new_params && !dl_se->dl_throttled && dl_in_constrained_gap(dl_se))
+ return;
+ if (dl_se->dl_new_params) {
+ dl_se->dl_new_params = 0;
+ if (!dl_se->dl_throttled)
+ update_dl_entity(dl_se);
+ }
}
-/* Return true if @p waits on an rt_mutex. The caller holds its pi_lock. */
-static inline bool dl_task_pi_blocked(struct task_struct *p)
+/*
+ * Check the runtime of @p against the parameters that sched_setattr() just
+ * gave it, if @p waits on an rt_mutex. Before any wakeup, its waiter node
+ * holds its deadline. The caller then calls rt_mutex_adjust_pi(), which
+ * requeues the waiter with the result. Other changes of @p, such as a
+ * cgroup move, do not requeue it: leave the check to sched_setattr().
+ *
+ * A boosted task, a throttled one, or one past its deadline and before
+ * its next period keeps the mark. Its deboost or its wakeup checks it.
+ * The caller holds the rq lock and the pi_lock of @p, and has updated the
+ * rq clock.
+ */
+void dl_check_blocked_params(struct task_struct *p)
{
- return p->pi_blocked_on;
+ struct sched_dl_entity *dl_se = &p->dl;
+
+ if (!dl_task(p) || task_on_rq_queued(p) || !dl_task_pi_blocked(p))
+ return;
+ if (dl_se->dl_new_params && !is_dl_boosted(dl_se) &&
+ !dl_se->dl_throttled && !dl_in_constrained_gap(dl_se)) {
+ dl_se->dl_new_params = 0;
+ update_dl_entity(dl_se);
+ }
}
#else /* !CONFIG_RT_MUTEXES: */
static inline struct sched_dl_entity *pi_of(struct sched_dl_entity *dl_se)
@@ -707,6 +768,15 @@ static unsigned int dl_task_root_domain_cpu(struct task_struct *p);
*
* Each check compares against the bound scaled by dl_runtime / dl_period.
* That quotient cannot overflow, as dl_runtime <= dl_period.
+ *
+ * After sched_setattr(), the runtime and the deadline can still be those
+ * of the old parameters: a boosted task, or one past its deadline and
+ * before its next period, keeps them until its deboost or its enqueue.
+ * The 0-lag time then mixes them with the new dl_runtime and dl_period.
+ * If the runtime left does not fit the new parameters, the 0-lag time
+ * has passed, and the bandwidth is released at once. The check of the
+ * runtime would start a new period there, with a 0-lag time of now or
+ * before.
*/
static s64 dl_time_to_zerolag(struct sched_dl_entity *dl_se, struct rq *rq)
{
@@ -2965,7 +3035,16 @@ static void __dequeue_dl_entity(struct sched_dl_entity *dl_se)
static void
enqueue_dl_entity(struct sched_dl_entity *dl_se, int flags)
{
+ bool new_params = dl_se->dl_new_params;
+
WARN_ON_ONCE(on_dl_rq(dl_se));
+ /*
+ * A boosted task is refilled from its donor. Keep the mark until the
+ * runtime is checked against its own parameters. dl_deboost() checks
+ * it at the deboost, and an enqueue that is not boosted checks it.
+ */
+ if (!is_dl_boosted(dl_se))
+ dl_se->dl_new_params = 0;
update_stats_enqueue_dl(dl_rq_of_se(dl_se), dl_se, flags);
@@ -3014,12 +3093,13 @@ enqueue_dl_entity(struct sched_dl_entity *dl_se, int flags)
update_dl_entity(dl_se);
} else if (flags & ENQUEUE_REPLENISH) {
replenish_dl_entity(dl_se);
- } else if ((flags & ENQUEUE_CLASS) && !is_dl_boosted(dl_se)) {
+ } else if (((flags & ENQUEUE_CLASS) || new_params) &&
+ !is_dl_boosted(dl_se)) {
/*
- * The task enters the class runnable. It may come back before
- * its deadline, with the runtime left from older parameters.
- * Check that runtime against the new parameters, as a wakeup
- * does.
+ * The task enters the class, or its parameters change, while
+ * it is runnable. It may have runtime left from older
+ * parameters. Check that runtime against the new parameters,
+ * as a wakeup does.
*/
update_dl_entity(dl_se);
} else if ((flags & ENQUEUE_MOVE) &&
@@ -4859,7 +4939,9 @@ static void switched_to_dl(struct rq *rq, struct task_struct *p)
* The chain walk sorts its waiter by its deadline first. A task
* new to the class has a deadline of 0, which goes before every
* real one. Start a new period now, as setup_new_dl_entity()
- * does at an enqueue.
+ * does at an enqueue. dl_check_blocked_params() has checked the
+ * runtime of a task that sched_setattr() brings back before its
+ * 0-lag time.
*/
if (dl_task_pi_blocked(p) && !p->dl.dl_throttled &&
dl_time_before(p->dl.deadline, rq_clock(rq)))
@@ -5179,6 +5261,11 @@ void __setparam_dl(struct task_struct *p, const struct sched_attr *attr)
u64 period = attr->sched_period ?: attr->sched_deadline;
bool held;
+ if (attr->sched_runtime != dl_se->dl_runtime ||
+ attr->sched_deadline != dl_se->dl_deadline ||
+ period != dl_se->dl_period)
+ dl_se->dl_new_params = 1;
+
/*
* Change the bandwidth that the runqueue counts here, with the
* parameters. A task that priority inheritance boosted into the
@@ -5312,6 +5399,7 @@ static void __dl_clear_params(struct sched_dl_entity *dl_se)
dl_se->dl_defer_running = 0;
dl_se->dl_defer_armed = 0;
dl_se->dl_bw_oncpu = 0;
+ dl_se->dl_new_params = 0;
}
void init_dl_entity(struct sched_dl_entity *dl_se)
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index e512091ce5ed..4151364ca4ff 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -414,6 +414,7 @@ static inline void dl_clear_donor(struct sched_dl_entity *dl_se)
extern void dl_deboost(struct sched_dl_entity *dl_se);
extern void dl_boost_blocked(struct task_struct *p);
+extern void dl_check_blocked_params(struct task_struct *p);
/* End the boost of @dl_se. The caller holds the rq lock of its task. */
static inline void dl_drop_boost(struct sched_dl_entity *dl_se)
diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index 9884fa55538a..b25504e8748b 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -311,6 +311,9 @@ static inline void __setscheduler_dl_pi(int newprio, int policy,
*/
if (!dl_prio(newprio))
dl_drop_boost(&p->dl);
+
+ /* rt_mutex_adjust_pi() then requeues the waiter of a blocked task. */
+ dl_check_blocked_params(p);
}
#else /* !CONFIG_RT_MUTEXES */
static inline void __setscheduler_dl_pi(int newprio, int policy,
--
2.47.3