[PATCH v3 0/3] sched/fair: Cut per-group work from sched domain rebuilds
From: Qiliang Yuan
Date: Fri Nov 20 2026 - 21:57:49 EST
A sched domain rebuild, from CPU hotplug or a cpuset partition change,
moves every active CPU to a new root domain through rq_attach_root().
It does so by taking the CPU offline and back online, and
rq_offline_fair() and rq_online_fair() then walk all task groups with
the rq lock held and interrupts disabled, so a rebuild costs
O(nr_cpus * nr_task_groups). Dietmar pointed at the offline side when
f60a631ab9ed was reviewed [1]. Hosts running AI agent harnesses
commonly carry over a thousand cpu cgroups, and their VMs hot-add vCPUs
at run time.
Based on Prateek's patch [2], patch 1 gives root domain moves their
own sched_class callbacks, which only rt and dl implement.
Patch 2 keeps CPUs that stay active online across a rebuild and moves
only their root domain state, so fair is no longer called for them.
Patch 3 drops a check in fair that patch 2 leaves dead.
With 4000 cpu cgroups on a 32-CPU VM booted with rt_group_sched=0
(median of 20 partition switches and 16 hotplug operations, both
kernels measured in the same run):
operation base (ms) patch 1-3 (ms) change
partition root 15.97 1.19 -93%
cpu offline 49.66 15.74 -68%
cpu online 37.39 13.33 -64%
Each rq_attach_root() call, which runs with interrupts disabled, takes
(1280 calls over the 20 partition switches, change of the median):
kernel median (us) p90 (us) max (us) change
base 229 265 1335
patch 1-3 1 1 10 -99.6%
Partition switches and onlining no longer grow with the number of
groups; offlining still walks them once for the CPU that goes down.
With RT_GROUP_SCHED enabled, the rt.c runtime walks remain.
[1] https://lore.kernel.org/r/dbd36b16-4acb-415a-9bbd-258b2e022475@xxxxxxx
[2] https://lore.kernel.org/r/e1a63a3c-3975-4b05-8ab4-518c37d867d6@xxxxxxx
Signed-off-by: Qiliang Yuan <qiliang.yuan@xxxxxxxxx>
---
V2 -> V3:
- Give root domain moves their own rq_attach_root() and rq_detach_root()
callbacks, implemented by rt and dl, and keep CPUs that stay active
online across rebuilds instead of passing a hotplug argument to
rq_online() and rq_offline() (K Prateek Nayak)
- Keep putting a CPU that comes up online from the rebuild in
sched_cpu_activate()
- Move the removal of the active CPU check in
unthrottle_offline_cfs_rqs() into patch 3
- Update the numbers from a new run
V1 -> V2:
- Pass a hotplug argument to rq_online() and rq_offline() instead of
adding a member to struct rq (Peter Zijlstra)
- Skip both fair task group walks on rebuilds, which leaves no second
loop to merge
- Rebase onto tip/sched/core, where sched_ext no longer implements
rq_online() and rq_offline(), and update the numbers from a new run
CPU hotplug and partition flips were also run with SCHED_FIFO tasks
(rt_group_sched=1, RT_RUNTIME_SHARE on and off), SCHED_DEADLINE tasks
and scx_simple loaded, plus the sched_ext selftests, with the same
results as on tip/sched/core (b251360cb2e6c).
v2: https://lore.kernel.org/r/20261118-bug-sched-fair-offline-active-v2-0-b6ede596c6a6@xxxxxxxxx
v1: https://lore.kernel.org/r/20261111-bug-sched-fair-offline-active-v1-0-4f54523c5a70@xxxxxxxxx
---
Qiliang Yuan (3):
sched: Give root domain moves their own sched_class callbacks
sched: Keep CPUs that stay active online across root domain moves
sched/fair: Drop the active CPU check from unthrottle_offline_cfs_rqs()
kernel/sched/core.c | 31 +++++++++++++++++++++++++++++++
kernel/sched/cpudeadline.c | 5 +++--
kernel/sched/deadline.c | 8 ++++----
kernel/sched/fair.c | 4 ----
kernel/sched/rt.c | 8 ++++----
kernel/sched/sched.h | 8 ++++++++
kernel/sched/topology.c | 20 +++++++++++++++++---
7 files changed, 67 insertions(+), 17 deletions(-)
---
base-commit: b251360cb2e6c7b74d962831f5797a4520511b36
change-id: 20261111-bug-sched-fair-offline-active-8d063e735d3f
Best regards,
--
Qiliang Yuan <qiliang.yuan@xxxxxxxxx>