Re: [PATCH] sched/cache: Honor asym packing over cache aware scheduling on hybrid system

[Tim Chen]

On Mon, 2026-09-28 at 22:52 +0300, Kayra Cizmeci wrote:
> Helloooo Tim,
> 
> > A regression was reported on an AMD Ryzen AI HX 370 running a cache
> > intensive Clang full-LTO link. The little cores run at a much lower
> > frequency (3.3 GHz vs 5.1 GHz) and have only half of the L3 cache
> > (8 MB vs 16 MB), so pinning such a task to the little-core LLC hurts
> > twice, and full-LTO builds slow down dramatically compared to
> > pre-cache-aware-scheduling kernels.
> 
> > Asym packing and cache aware scheduling express conflicting placement
> > strategy. Asym packing wants a task to run on the highest priority CPU,
> > whereas cache aware scheduling wants to co-locate the tasks of a process
> > on one LLC regardless of the priority of CPUs in that LLC.
> 
> > When asym packing tries to migrate task to an empty
> > core that has higher priority than source cpu, let asym packing win.
> > Moving tasks to a higher performing idle core will buy more
> > performance than cache co-location.
> 
> 
> > +static inline bool sched_asym(struct sched_domain *sd, int dst_cpu, int src_cpu);
> > +
> >  /*
> >   * Check if task p can migrate from source LLC to
> >   * destination LLC in terms of cache aware load balance.
> > @@ -10847,6 +10849,10 @@ static enum llc_mig can_migrate_llc_task(struct lb_env *env,
> >  	if (cpu < 0 || cpus_share_cache(src_cpu, dst_cpu))
> >  		return mig_unrestricted;
> >  
> > +	/* Prioritize asym packing over cache awareness */
> > +	if (sched_asym(env->sd, dst_cpu, src_cpu))
> > +		return mig_unrestricted;
> > +
> >  	/* skip cache aware load balance for too many threads */
> >  	if (invalid_llc_nr(grp, p, dst_cpu) ||
> >  	    exceed_llc_capacity(grp, dst_cpu)) {
> > @@ -12043,6 +12049,15 @@ static inline bool llc_balance(struct lb_env *env, struct sg_lb_stats *sgs,
> >  	    sgs->group_misfit_task_load)
> >  		return false;
> >  
> > +	/*
> > +	 * On asym packing domains, if the destination CPU
> > +	 * has higher priority than all CPUs in the source group,
> > +	 * prioritize asym packing.
> > +	 */
> > +	if ((env->sd->flags & SD_ASYM_PACKING) &&
> > +	    sgs->group_asym_packing)
> > +		return false;
> > +
> >  	/*
> >  	 * Skip cache aware tagging if nr_balanced_failed is sufficiently high.
> >  	 * Threshold of cache_nice_tries is set to 1 higher than nr_balance_failed
> > @@ -13458,12 +13473,12 @@ static int need_active_balance(struct lb_env *env)
> >  {
> >  	struct sched_domain *sd = env->sd;
> >  
> > -	if (alb_break_llc(env))
> > -		return 0;
> > -
> >  	if (asym_active_balance(env))
> >  		return 1;
> >  
> > +	if (alb_break_llc(env))
> > +		return 0;
> > +
> >  	if (imbalanced_active_balance(env))
> >  		return 1;
>  
> Hope I could test this. But I don't really have hardware for it :-(.
> 
> Regardless tho.
> 
> Let's say when entering can_migrate_llc_task(), dst_cpu is CPU0, while src_cpu is CPU1.
> And CPU0 has a bigger asym_prio than CPU1. No SMT. When entering can_migrate_llc_task() and sched_asym()
> from there sched_use_asym_prio() returns true without checking if the core is fully idle or not.

sched_asym() does check whether the destination core is idle in sched_use_asym_prio() for non SMT domain.

                  (false for non-SMT sd)           (check idle core)
       return sd->flags & SD_SHARE_CPUCAPACITY || is_core_idle(cpu);

That is also a pre-condition for setting group_asym_packing.

> And sched_asym_prefer() comes back true too, so sched_asym() returns true and, we just returned mig_unrestricted.
> 
> I could be missing something, If I'm not tho is that on purpose? If it is, the last paragraph needs to change
> since it says that "asym packing tries to migrate task to an empty core" and after that "higher performing idle core"

I think I did try to point out the idle core aspect in my commit log:

"When asym packing tries to migrate task to an empty
core that has higher priority than source cpu, let asym packing win.
Moving tasks to a higher performing idle core will buy more
performance than cache co-location."

I am not sure how you would like it changed.

Thanks.

Tim

> 
> Thanks,
> Kayra