[PATCH v4 06/26] sched: Take cpuset_mutex for a deadline policy change only with pi

[Mathieu Desnoyers]

__sched_setscheduler() takes cpuset_mutex for a change of or into
SCHED_DEADLINE, and releases it on success only where pi is set, with
rt_mutex_adjust_pi(). normalize_rt_tasks(), the one caller with pi
unset, then takes a mutex under read_lock(&tasklist_lock) for every
deadline task it finds, and never releases it: sysrq-n on a system with
a deadline task sleeps in atomic context, and every later change of or
into SCHED_DEADLINE blocks on cpuset_mutex for good.

Commit 111cd11bbc54 ("sched/cpuset: Bring back cpuset_mutex") made the
lock depend on the policy where it depended on pi before, and left the
release where it was. Take it where pi is set, as before:
normalize_rt_tasks() takes tasks out of the class, which admits no
bandwidth and needs no stable cpusets.

The policy the decision reads is the task's before the runqueue lock, and
only a caller that gave none has it read again under the lock. Given
another policy than SCHED_DEADLINE, a change into the class by another
caller in between makes the call a change out of it without the mutex.
Decide again under the lock, where the policy is settled: a change out
of the class needs no mutex, but the decision follows the policy the
call acts on.

sysrq-n on v7.4-rc1 with a deadline task, lockdep on, in 8-CPU VMs:

  sysrq: Nice All RT Tasks
  BUG: sleeping function called from invalid context at kernel/locking/mutex.c:623
  in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 4356, name: deadline_switch
  locks held by deadline_switch/4356: 4, last CPU#3:
   ...
   #3: ffffffff83006098 (tasklist_lock){.+.+}-{3:3}, at: normalize_rt_tasks+0x34/0x150

and a sched_setattr(SCHED_DEADLINE) after it never returns, in 2 VMs of
2. The deadline_switch_test selftest added later in this series has the
case, sysrq_normalize, which passes with this change.

Fixes: 111cd11bbc54 ("sched/cpuset: Bring back cpuset_mutex")
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx>
---
 kernel/sched/syscalls.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index bdffe5452dc8..c3f404cddb9e 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -554,9 +554,10 @@ int __sched_setscheduler(struct task_struct *p,
 
 	/*
 	 * SCHED_DEADLINE bandwidth accounting relies on stable cpusets
-	 * information.
+	 * information. Not where pi is unset: normalize_rt_tasks() takes
+	 * tasks out of the class under tasklist_lock, which admits none.
 	 */
-	if (dl_policy(policy) || dl_policy(p->policy)) {
+	if (pi && (dl_policy(policy) || dl_policy(p->policy))) {
 		cpuset_locked = true;
 		cpuset_lock();
 	}
@@ -647,6 +648,16 @@ int __sched_setscheduler(struct task_struct *p,
 		goto recheck;
 	}
 
+	/*
+	 * The policy read before this lock decided against cpuset_mutex: a
+	 * change into the class since makes this one a change out of it.
+	 * Decide again with the lock held.
+	 */
+	if (pi && !cpuset_locked && task_has_dl_policy(p)) {
+		task_rq_unlock(rq, p, &rf);
+		goto recheck;
+	}
+
 	oldprio = p->prio;
 
 	newprio = __normal_prio(policy, attr->sched_priority, attr->sched_nice);
-- 
2.47.3