Re: [PATCH v5 0/6] sched: Fix cluster scheduling in the presence of asymmetric capacity

[Ricardo Neri]

On Mon, Jun 29, 2026 at 03:10:49PM +0200, Andrea Righi wrote:
> Hi Ricardo,
> 
> On Mon, Jun 22, 2026 at 05:05:50PM -0700, Ricardo Neri wrote:
> > Hi,
> > 
> > This is v5 of the patch series. The only changes are optimizations to check
> > same-capacity cluster and SMT siblings only when needed. Please read the
> > changelog for details.
> > 
> > Cluster scheduling aims to maximize performance by spreading load across
> > clusters of CPUs that share mid-level resources [2]. It works well on
> > uniform systems, but it breaks down on topologies with big and small
> > cores arranged in clusters. As a result, it fails on several generations
> > of Intel processors already shipped and upcoming.
> > 
> > Consider the topology below of big (B) cores and clusters of small (s)
> > cores.
> >          ------   ------
> >          | B  |   | B  |   -----------------   -----------------
> >          |    |   |    |   | s | s | s | s |   | s | s | s | s |
> >          ------   ------   -----------------   -----------------
> >          | L2 |   | L2 |   |      L2       |   |       L2      |
> >          -------------------------------------------------------
> >          |                          L3                         |
> >          -------------------------------------------------------
> > 
> > On a partially busy system (one with idle CPUs; busy CPUs have one task
> > each), scheduling for asymmetric capacity ensures that misfit tasks land on
> > the big CPUs. The remaining tasks, misfit or not, run on the small CPUs.
> > When CONFIG_SCHED_CLUSTER is enabled, these remaining tasks are supposed to
> > be evenly spread among the small-CPU clusters. Today, this does not
> > happen.
> > 
> > Several issues in the load balancer prevent a small CPU in one cluster
> > from pulling tasks from another:
> > 
> >  a) update_sd_pick_busiest() may select a fully_busy group with higher
> >     per-CPU capacity as the busiest, preventing a subsequent fully_busy
> >     group of equal capacity from being correctly selected.
> >  b) Misfit-load statistics are used to identify tasks that would benefit
> >     from migrating to bigger CPUs. Accounting misfit load is pointless if
> >     the destination CPU is equally small, and it also blocks balancing
> >     between clusters.
> >  c) Due to b), groups that are truly has_spare or fully_busy get
> >     misclassified as misfit_task. update_sd_pick_busiest() then skips
> >     them, since a small destination CPU cannot help with misfit tasks.
> >  d) Once a busiest group has been identified, sched_balance_find_src_rq()
> >     will refuse to migrate tasks to CPUs of equal capacity, even when
> >     doing so is precisely what is required to balance small-CPU clusters.
> >  e) The SD_PREFER_SIBLING flag is missing from scheduling domains with
> >     asymmetric capacity, preventing the balancer from equalizing load
> >     across sibling small-core clusters.
> > 
> > Together, these issues prevent cluster-level balancing on systems with
> > asymmetric CPU capacity.
> > 
> > This series addresses each problem and restores the intended behavior.
> > Details, rationale, and code changes are explained in each patch.
> > 
> > I tested these patches on Alder Lake, which has both SMT Pcores and
> > clusters of Ecores. I tested with SMT both disabled and enabled. I also
> > tested on Lunar Lake and Panther Lake, which have an Ecore cluster not
> > connected to the L3 cache. I repeated the same experiment with
> > CONFIG_SCHED_CLUSTER disabled. The load balancer behaves as expected.
> > 
> > Christian also tested this patchset on a synthetic arm64 qemu topology and
> > the expected behavior [3].
> > 
> > Link: https://lore.kernel.org/all/20260509180955.1840064-1-arighi@xxxxxxxxxx/ [1]
> > Link: https://lore.kernel.org/r/20210924085104.44806-1-21cnbao@xxxxxxxxx/ [2]
> > Link: https://lore.kernel.org/all/e08492e0-d9f3-4574-8841-b633db008507@xxxxxxx/ [3]
> 
> I looked at this series and did some tests on Vera Rubin (arm64). The series
> does not directly target this topology, because the system has no usable
> SD_CLUSTER domain. Nevertheless, there are changes to the common fair
> load-balancing and SD_ASYM_CPUCAPACITY paths, so I tested it just to check for
> any potential regression.
> 
> I used DCPerf MediaWiki and a CPU-intensive SGEMM workload based on NVPL. The
> results showed no measurable performance differences, with all observed
> variations falling within normal run-to-run variability. Based on these results,
> the series looks good on this platform as well.
> 
> Tested-by: Andrea Righi <arighi@xxxxxxxxxx>

Thanks Andrea! I will add this details in the cover letter of the next
version.