Re: [PATCH 2/5] sched/fair: Attach sched_domain_shared to sd_asym_cpucapacity
From: Dietmar Eggemann
Date: Tue May 05 2026 - 08:51:23 EST
On 28.04.26 16:41, Andrea Righi wrote:
> From: K Prateek Nayak <kprateek.nayak@xxxxxxx>
>
> On asymmetric CPU capacity systems, the wakeup path uses
> select_idle_capacity(), which scans the span of sd_asym_cpucapacity
> rather than sd_llc.
>
> The has_idle_cores hint however lives on sd_llc->shared, so the
> wakeup-time read of has_idle_cores operates on an LLC-scoped blob while
> the actual scan/decision spans the asym domain; nr_busy_cpus also lives
> in the same shared sched_domain data, but it's never used in the asym
> CPU capacity scenario.
>
> Therefore, move the sched_domain_shared object to sd_asym_cpucapacity
> whenever the CPU has a SD_ASYM_CPUCAPACITY_FULL ancestor and that
> ancestor is non-overlapping (i.e., not built from SD_NUMA). In that case
> the scope of has_idle_cores matches the scope of the wakeup scan.
>
> Fall back to attaching the shared object to sd_llc in three cases:
>
> 1) plain symmetric systems (no SD_ASYM_CPUCAPACITY_FULL anywhere);
>
> 2) CPUs in an exclusive cpuset that carves out a symmetric capacity
> island: has_asym is system-wide but those CPUs have no
> SD_ASYM_CPUCAPACITY_FULL ancestor in their hierarchy and follow
> the symmetric LLC path in select_idle_sibling();
>
> 3) exotic topologies where SD_ASYM_CPUCAPACITY_FULL lands on an
> SD_NUMA-built domain. init_sched_domain_shared() keys the shared
> blob off cpumask_first(span), which on overlapping NUMA domains
> would alias unrelated spans onto the same blob. Keep the shared
> object on the LLC there; select_idle_capacity() gracefully skips
> the has_idle_cores preference when sd->shared is NULL.
Tested it with a coule of real & exotic topolgies, seems to work nicely.
$ cat /sys/devices/system/cpu/cpu*/cpu_capacity
160
160
160
160
498
498
1024
1024
(1) grouping CPUs with same CPU capacities
$ cat /sys/kernel/debug/sched/domains/cpu[0-7]/domain*/name
MC
PKG
$ cat /sys/kernel/debug/sched/domains/cpu[0-7]/domain*/flags
... SD_SHARE_LLC
... SD_ASYM_CPUCAPACITY SD_ASYM_CPUCAPACITY_FULL ...
PKG { 0-7 }
MC {0-3} {4,5} {6,7}
(2) flat
$ cat /sys/kernel/debug/sched/domains/cpu[0-7]/domain*/name
MC
$ cat /sys/kernel/debug/sched/domains/cpu[0-7]/domain*/flags
... SD_ASYM_CPUCAPACITY SD_ASYM_CPUCAPACITY_FULL ...
MC { 0-7 }
(3) flat, exotic, since w/ SMT
$ cat /sys/kernel/debug/sched/domains/cpu[0-7]/domain*/name
SMT
MC
... SD_SHARE_CPUCAPACITY SD_SHARE_LLC ...
... SD_ASYM_CPUCAPACITY SD_ASYM_CPUCAPACITY_FULL SD_SHARE_LLC ...
MC { 0-7 }
SMT {0-1} {2-3} {4-5} {6-7}
(4) exotic, since asymmetric and w/ SMT
$ cat /sys/kernel/debug/sched/domains/cpu[0-3]/domain*/name
SMT
MC
PKG
$ cat /sys/kernel/debug/sched/domains/cpu[0-3]/domain*/flags
... SD_SHARE_CPUCAPACITY SD_SHARE_LLC ...
... SD_SHARE_LLC
... SD_ASYM_CPUCAPACITY SD_ASYM_CPUCAPACITY_FULL ...
$ cat /sys/kernel/debug/sched/domains/cpu[4-7]/domain*/name
SMT
PKG
$ cat /sys/kernel/debug/sched/domains/cpu[4-7]/domain*/flags
... SD_SHARE_CPUCAPACITY SD_SHARE_LLC ...
... SD_ASYM_CPUCAPACITY SD_ASYM_CPUCAPACITY_FULL ...
PKG { 0-7 }
MC { 0-3 }
SMT {0-1} {2-3} {4-5} {6-7}
(5) same as (4) but partial CPU capacity asymmetry in MC { 0-3 }
cat /sys/devices/system/cpu/cpu*/cpu_capacity
160
160
498
498
160
160
1024
1024
$ cat /sys/kernel/debug/sched/domains/cpu[0-3]/domain*/flags
... SD_SHARE_CPUCAPACITY SD_SHARE_LLC ...
... SD_ASYM_CPUCAPACITY SD_SHARE_LLC ...
^^^^^^^^^^^^^^^^^^^
... SD_ASYM_CPUCAPACITY SD_ASYM_CPUCAPACITY_FULL ...
(6) (5) w/ exclusive cpusets with one symmetric island
cd /sys/fs/cgroup
echo +cpuset > cgroup.subtree_control
mkdir cs1
echo "threaded" > cs1/cgroup.type
echo 0-1,4-5 > cs1/cpuset.cpus
echo 0 > cs1/cpuset.mems
echo root > cs1/cpuset.cpus.partition
mkdir cs2
echo "threaded" > cs2/cgroup.type
echo 0 > cs2/cpuset.mems
echo 2-3,6-7 > cs2/cpuset.cpus
echo root > cs2/cpuset.cpus.partition
[ 0.006866] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=0
[ 0.006868] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=1
[ 0.006869] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=2
[ 0.006869] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=3
[ 0.006869] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=4
[ 0.006869] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=5
[ 0.006870] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=6
[ 0.006870] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=7
...
[ 222.767275] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=2
[ 222.767324] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=3
[ 222.767710] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=6
[ 222.767789] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=7
[ 222.781015] build_sched_domains() (3) sd=MC cpu=0
[ 222.781017] build_sched_domains() (3) sd=MC cpu=1
[ 222.781017] build_sched_domains() (3) sd=MC cpu=4
[ 222.781018] build_sched_domains() (3) sd=MC cpu=5
[...]
> @@ -2650,6 +2665,49 @@ static void adjust_numa_imbalance(struct sched_domain *sd_llc)
> }
> }
>
> +static void init_sched_domain_shared(struct s_data *d, struct sched_domain *sd)
> +{
> + int sd_id = cpumask_first(sched_domain_span(sd));
> +
> + sd->shared = *per_cpu_ptr(d->sds, sd_id);
> + atomic_set(&sd->shared->nr_busy_cpus, sd->span_weight);
Will be used only for sd_llc->shared, not for sd_asym, right?
> + atomic_inc(&sd->shared->ref);
> +}
> +
> +/*
> + * For asymmetric CPU capacity, attach sched_domain_shared on the innermost
> + * SD_ASYM_CPUCAPACITY_FULL ancestor of @cpu's base domain when that ancestor is
> + * not an overlapping NUMA-built domain (then LLC should claim shared).
> + *
> + * A CPU may lack any FULL ancestor (e.g., exclusive cpuset symmetric island),
> + * then LLC must claim shared instead.
> + *
> + * Note: SD_ASYM_CPUCAPACITY_FULL is only set when multiple distinct capacities
s/multiple/all ? We want to see all possible CPU capacity values in wakeup.
> + * exist in the domain span, so the asym domain we attach to cannot degenerate
> + * into a single-capacity group. The relevant edge cases are instead covered by
> + * the caveats above.
[...]