[PATCH v3 16/24] cgroup/cpuset: Wait for an attach before rebuilding the sched domains

[Mathieu Desnoyers]

cpuset_can_attach() sets aside, in the root domain of the destination,
the bandwidth of the deadline tasks it is about to move, and drops
cpuset_mutex until cpuset_attach() moves them. A rebuild of the sched
domains in between, rebuild_sched_domains() from a write of
sched_rt_runtime_us or from the energy model, takes cpuset_mutex and
runs: dl_rebuild_rd_accounting() clears the root domains and counts
each task where its cpuset says, and the reservation is gone. The
attach then moves the bandwidth of each task out of the root domain it
left, on the strength of a reservation that is no longer there, and the
destination's total_bw ends short of what its tasks hold: it admits
deadline tasks beyond its capacity, and underflows when they leave.

The writes of a cpuset's files wait for an attach in progress before
taking cpuset_mutex, wait_attach_done_lock(), since
commit 75f7a25ec6bb ("cgroup/cpuset: Prevent race between task attach
and cpuset state change"), and CPU hotplug holds cpus_write_lock, which
an attach holds for reading throughout. Wait in
rebuild_sched_domains_cpuslocked() as well: a rebuild from the hotplug
path finds no attach in progress, and one from the sysctl or the energy
model lets the attach complete first.

The deadline_bw_test selftest added later in this series has the case,
move_vs_rebuild: a process of 2049 threads, one of them a deadline task
asleep, moved in and out of a one-CPU partition root while a writer of
sched_rt_runtime_us rebuilds the sched domains 20 times, six rounds.
With the patches of this series before this one, a round leaves a root
domain short of the task's bandwidth, the root domain of the other CPUs
admitting 3 more tasks of 10% in three 8-CPU VMs and the partition root
none at all in a fourth, in 4 VMs of 4; with this change the six rounds
pass, in 2 VMs of 2.

Found by the Sashiko AI review of the second posting of this series.

Fixes: 2ef269ef1ac0 ("cgroup/cpuset: Free DL BW in case can_attach() fails")
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx>
---
 kernel/cgroup/cpuset.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 5b8f1d9552a6..aa946baeb4b9 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1067,7 +1067,13 @@ void rebuild_sched_domains_locked(void)
 
 static void rebuild_sched_domains_cpuslocked(void)
 {
-	mutex_lock(&cpuset_mutex);
+	/*
+	 * An attach in progress has set aside, in the root domain its tasks
+	 * move to, the deadline bandwidth it moves once it takes
+	 * cpuset_mutex again: the rebuild of the accounting would drop that
+	 * reservation, and the move would go through on its strength.
+	 */
+	wait_attach_done_lock();
 	rebuild_sched_domains_locked();
 	mutex_unlock(&cpuset_mutex);
 }
-- 
2.47.3