Re: [PATCH v2] sched/topology: Add a cpus_read_lock to rebuild_sched_domains()

[Dietmar Eggemann]

On 13.08.26 09:38, Sebastian Andrzej Siewior wrote:
> A read from /proc/sys/kernel/sched_rt_runtime_us leads to backtrace due
> to missing cpu_hotplug_lock with CONFIG_CPUSETS=n. The callchain is
> sched_rt_handler() -> partition_sched_domains() -> sched_cache_set() ->
> static_key_enable_cpuslocked(&sched_cache_present).
> 
> sched_cache_set() itself is also invoked from sched_init_domains() which
> is early during the boot, holding just the sched_domains_mutex_lock().
> Here is no warning because it happens before user space is running (and
> hotplug operations are not possible).
> 
> There is also sched_cache_active_set() which acquires the hotplug lock
> before invoking any of the _cpuslocked() functions.
> 
> This is only a problem with CONFIG_CPUSETS=n because in the =y case the
> other implementation of rebuild_sched_domains acquires the CPU-hotplug
> lock.
> 
> Acquire CPU hotplug lock before in rebuild_sched_domains(), before
> partition_sched_domains() is invoked for the CONFIG_CPUSETS=n case.
> 
> Fixes: a7660ce1590fc ("sched/cache: Fix has_multi_llcs iff at least one partition has multiple LLCs")
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>
> ---
> v1…v2: https://lore.kernel.org/all/20260812095800.gl06ANul@xxxxxxxxxxxxx/
>   - Move the lock from partition_sched_domains() to
>     rebuild_sched_domains() in the CONFIG_CPUSETS=n since this is the
>     only affected case. Noticed by Yu C Chen and Tim Chen.
> 
>  include/linux/cpuset.h | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h
> index 65d76a38974ba..bf3999daa080a 100644
> --- a/include/linux/cpuset.h
> +++ b/include/linux/cpuset.h
> @@ -273,6 +273,7 @@ static inline void dl_rebuild_rd_accounting(void)
>  
>  static inline void rebuild_sched_domains(void)
>  {
> +	guard(cpus_read_lock)();
>  	partition_sched_domains(1, NULL, NULL);
>  }
>  

This also fixes the EAS specific rebuild_sched_domains_energy() case
which gets invoked when we start/stop EAS e.g. while doing a CPUfreq
governor change: schedutil to any other governor (e.g. ondemand).

grep -h . /sys/devices/system/cpu/cpu[0-9]*/cpufreq/scaling_governor
schedutil
schedutil
schedutil
schedutil
schedutil
schedutil

for f in /sys/devices/system/cpu/cpu[0-9]*/cpufreq/scaling_governor; do echo ondemand > "$f"; done

[  142.900976] ------------[ cut here ]------------
[  142.900990] WARNING: kernel/cpu.c:527 at lockdep_assert_cpus_held+0x44/0x60, CPU#5: kworker/5:1/90
[  142.918755] Modules linked in: 
[  142.921841] CPU: 5 UID: 0 PID: 90 Comm: kworker/5:1 Not tainted 7.2.0-rc7-00035-g68e37487810a-dirty #87 PREEMPT 
[  142.932059] Hardware name: ARM Juno development board (r0) (DT)
[  142.938003] Workqueue: events rebuild_sd_workfn
[  142.942572] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  142.949563] pc : lockdep_assert_cpus_held+0x44/0x60
[  142.954473] lr : lockdep_assert_cpus_held+0x40/0x60
...
[  143.034566] Call trace:
[  143.037025]  lockdep_assert_cpus_held+0x44/0x60 (P) 
[  143.041939]  static_key_enable_cpuslocked+0x24/0xc8
[  143.046857]  partition_sched_domains+0x36c/0xb60
[  143.051509]  rebuild_sched_domains_energy+0x5c/0x78
...
[  143.110230] ---[ end trace 0000000000000000 ]---
[  143.115374] ------------[ cut here ]------------
[  143.115384] WARNING: kernel/cpu.c:527 at lockdep_assert_cpus_held+0x44/0x60, CPU#5: kworker/5:1/90
[  143.133300] Modules linked in: 
[  143.136383] CPU: 5 UID: 0 PID: 90 Comm: kworker/5:1 Tainted: G W 7.2.0-rc7-00035-g68e37487810a-dirty #87 PREEMPT 
[  143.148167] Tainted: [W]=WARN
[  143.151147] Hardware name: ARM Juno development board (r0) (DT)
[  143.157088] Workqueue: events rebuild_sd_workfn
[  143.161654] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  143.168644] pc : lockdep_assert_cpus_held+0x44/0x60
[  143.173554] lr : lockdep_assert_cpus_held+0x40/0x60
[  143.253643] Call trace:
[  143.256100]  lockdep_assert_cpus_held+0x44/0x60 (P) 
[  143.261013]  _sched_cache_active_set+0x2c/0xec
[  143.265490]  partition_sched_domains+0x370/0xb60
[  143.270141]  rebuild_sched_domains_energy+0x5c/0x78
...

Tested-by: Dietmar Eggemann <dietmar.eggemann@xxxxxxx>