[PATCH v4 18/26] cgroup/cpuset: Wait for an attach before rebuilding the sched domains

[Mathieu Desnoyers]

cpuset_can_attach() sets aside, in the destination's root domain, the
bandwidth of the deadline tasks it is about to move, and drops
cpuset_mutex until cpuset_attach() moves them. A rebuild of the sched
domains in between, from a write of sched_rt_runtime_us or from the
energy model, takes cpuset_mutex, clears the root domains and counts
each task where its cpuset says: the reservation is gone. The attach
then moves each task's bandwidth on the strength of it, and the
destination's total_bw ends short of what its tasks hold: it admits
deadline tasks beyond its capacity, and underflows when they leave.

The writes of a cpuset's files wait for an attach in progress,
wait_attach_done_lock(), since commit 75f7a25ec6bb ("cgroup/cpuset:
Prevent race between task attach and cpuset state change"), and CPU
hotplug holds cpus_write_lock, which an attach holds for reading
throughout. Wait in rebuild_sched_domains_cpuslocked() as well, where a
rebuild from hotplug finds no attach in progress, and in
cpuset_full_lock(), which the cgroup v1 writes of sched_load_balance and
sched_relax_domain_level and hk_sd_workfn() rebuild under, in the order
the writes of cpus and cpus.partition take it. The other rebuilds, at
the online, offline and kill of a cpuset, run under cgroup_mutex, which
the attach holds throughout.

The move_vs_rebuild case of deadline_bw_test, added later in this
series, moves a process of 2049 threads, the last a deadline task
asleep, in and out of a one-CPU partition root while sched_rt_runtime_us
is written 20 times, 5ms apart, six rounds. Without this patch a round
fails in 5 VMs of 8, the partition root left admitting no task at all,
or the other CPUs admitting 3 deadline tasks of 10% more than they have
room for; with it, it passes in 8 of 8. A rebuild later in the attach
does no harm until the next patch gives back what an attach did not
move: it would then give back a reservation already gone.

Found by the Sashiko AI review of the second posting of this series, the
cgroup v1 writes by that of the third.

Fixes: 2ef269ef1ac0 ("cgroup/cpuset: Free DL BW in case can_attach() fails")
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx>
---
 kernel/cgroup/cpuset.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index bf9a7f88c0ed..2ee58d46da0e 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -296,17 +296,19 @@ void lockdep_assert_cpuset_lock_held(void)
 	lockdep_assert_held(&cpuset_mutex);
 }
 
+static void wait_attach_done_lock(void) __acquires(&cpuset_mutex);
+
 /**
  * cpuset_full_lock - Acquire full protection for cpuset modification
  *
  * Takes both CPU hotplug read lock (cpus_read_lock()) and cpuset mutex
- * to safely modify cpuset data.
+ * to safely modify cpuset data, the latter once no attach is in progress.
  */
 void cpuset_full_lock(void)
 {
 	mutex_lock(&cpuset_top_mutex);
 	cpus_read_lock();
-	mutex_lock(&cpuset_mutex);
+	wait_attach_done_lock();
 }
 
 void cpuset_full_unlock(void)
@@ -1005,7 +1007,13 @@ void rebuild_sched_domains_locked(void)
 
 static void rebuild_sched_domains_cpuslocked(void)
 {
-	mutex_lock(&cpuset_mutex);
+	/*
+	 * An attach in progress has set aside, in the root domain its tasks
+	 * move to, the deadline bandwidth it moves once it takes
+	 * cpuset_mutex again: the rebuild of the accounting would drop that
+	 * reservation, and the move would go through on its strength.
+	 */
+	wait_attach_done_lock();
 	rebuild_sched_domains_locked();
 	mutex_unlock(&cpuset_mutex);
 }
-- 
2.47.3