Re: [PATCH 2/2] sched/fair: Avoid re-syncing runtime_enabled on root domain changes

[Peter Zijlstra]

On Wed, Nov 11, 2026 at 08:24:21PM -0500, Qiliang Yuan wrote:
> rq_online_fair() calls update_runtime_enabled(), which walks every task
> group and takes each group's cfs_bandwidth lock to copy its quota state
> into the CPU's cfs_rq. A sched domain rebuild reaches it for every
> active CPU through rq_attach_root(), with the rq lock held and
> interrupts disabled.
> 
> The walk only matters when the cached state may be stale. While a CPU
> is online, tg_set_cfs_bandwidth() updates runtime_enabled on it under
> the rq lock, and it holds cpus_read_lock() so it cannot miss a CPU that
> is coming up. Only unthrottle_offline_cfs_rqs() clears runtime_enabled
> behind its back, and a CPU that has never been online has not been
> visited at all.
> 
> Track this in a per-rq flag that update_runtime_enabled() sets after
> the walk and unthrottle_offline_cfs_rqs() clears, and skip the walk
> while it is set.
> 
> With 4000 cpu cgroups on a 32-CPU VM booted with rt_group_sched=0,
> switching a 4-CPU cpuset to a partition root and offlining or onlining
> one CPU take (median of 20 partition switches and 16 hotplug operations):
> 
>   operation         before (ms)  after (ms)  change
>   partition root           7.68        1.21    -84%
>   cpu offline             30.96       13.44    -57%
>   cpu online              28.48       12.82    -55%

How do you end up with 4k cgroups ?!? That's wild.

Also, why do you care about the rebuild performance? I'm not entirely
sure it makes sense to add data for something that 'never' happens. Do
we really need a new member variable for this?

> Signed-off-by: Qiliang Yuan <qiliang.yuan@xxxxxxxxx>
> ---
>  kernel/sched/fair.c  | 11 +++++++++++
>  kernel/sched/sched.h |  2 ++
>  2 files changed, 13 insertions(+)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index e245a31b0544e..e2f67bdc3031e 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -7798,6 +7798,15 @@ static void __maybe_unused update_runtime_enabled(struct rq *rq)
>  
>  	lockdep_assert_rq_held(rq);
>  
> +	/*
> +	 * tg_set_cfs_bandwidth() keeps runtime_enabled up to date while the
> +	 * CPU is online, so only the first activation and the one following
> +	 * unthrottle_offline_cfs_rqs() need the walk.
> +	 */
> +	if (rq->cfsb_runtime_valid)
> +		return;
> +	rq->cfsb_runtime_valid = true;
> +
>  	guard(rcu)();
>  
>  	list_for_each_entry_rcu(tg, &task_groups, list) {
> @@ -7816,6 +7825,8 @@ static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq)
>  
>  	lockdep_assert_rq_held(rq);
>  
> +	rq->cfsb_runtime_valid = false;
> +
>  	/*
>  	 * The rq clock has already been updated in the
>  	 * set_rq_offline(), so we should skip updating
> diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
> index 1d76950dbb0a1..1a73383204384 100644
> --- a/kernel/sched/sched.h
> +++ b/kernel/sched/sched.h
> @@ -1412,6 +1412,8 @@ struct rq {
>  #ifdef CONFIG_CFS_BANDWIDTH
>  	call_single_data_t	cfsb_csd;
>  	struct list_head	cfsb_csd_list;
> +	/* cfs_rq->runtime_enabled is in sync with the quota of every group */
> +	bool			cfsb_runtime_valid;
>  #endif
>  
>  	atomic_t		nr_iowait;
> 
> -- 
> 2.43.0
>