Re: [PATCH] Revert "sched/fair: Add SIS_UTIL support to select_idle_capacity()"

From: Chen Yu

Date: Mon Nov 16 2026 - 07:59:13 EST


On Sat, Nov 14, 2026 at 10:11:03AM +0100, Andrea Righi wrote:
> Date: Sat, 14 Nov 2026 10:11:03 +0100
> From: Andrea Righi <arighi@xxxxxxxxxx>
> To: Vincent Guittot <vincent.guittot@xxxxxxxxxx>
> Cc: Rik van Riel <riel@xxxxxxxxxxx>, Usama Arif <usama.arif@xxxxxxxxx>,
> peterz@xxxxxxxxxxxxx, mingo@xxxxxxxxxx, kprateek.nayak@xxxxxxx,
> juri.lelli@xxxxxxxxxx, dietmar.eggemann@xxxxxxx, rostedt@xxxxxxxxxxx,
> bsegall@xxxxxxxxxx, mgorman@xxxxxxx, vschneid@xxxxxxxxxx,
> samuelcrossley@xxxxxxxxx, linux-kernel@xxxxxxxxxxxxxxx,
> stable@xxxxxxxxxxxxxxx, kas@xxxxxxxxxx, Arnav Aghav <aghav@xxxxxxxx>
> Subject: Re: [PATCH] Revert "sched/fair: Add SIS_UTIL support to
> select_idle_capacity()"
>
> On Sat, Nov 14, 2026 at 09:18:30AM +0100, Vincent Guittot wrote:
> > On Fri, 13 Nov 2026 at 20:32, Rik van Riel <riel@xxxxxxxxxxx> wrote:
> > >
> > > On Fri, 2026-11-13 at 07:31 -0800, Usama Arif wrote:
> > > >
> > > > On a 72-CPU NVIDIA grace arm64 server, small reported capacity
> > > > differences
> > > > (1013-1024) within the single LLC activate asymmetric selection. Both
> > > > the reference and affected kernels were observed with a zero scan
> > > > hint. The reference kernel ignored it and still searched the domain.
> > > > A latency-sensitive, wakeup-heavy production workload saw its
> > > > reported
> > > > minute p50 latencies average 129 ms on the reference kernel and 223
> > > > ms
> > > > on the affected kernel.
> > > >
> > > > Disabling SIS_UTIL reduced the aggregate involuntary context switch
> > > > rate of worker main threads from 71.6k/s to 13.3k/s; re-enabling it
> > > > raised the rate to 73.7k/s.
> > >
> > > Afaik, the asymmetrical selection is supposed
> > > to solve a very specific problem: when a task
> > > requires X amount of CPU capacity to run, we
> > > try to avoid scheduling it on a CPU that has
> > > only a fraction of X available.
> > >
> > > This creates the issue that a task that appears
> > > to "need" exactly X, because it's been running
> > > on a CPU with full available capacity, ends up
> > > not being scheduled at all when the available
> > > idle core has X-3% available capacity, because
> > > that CPU happens to spend 3% of its time handling
> > > interrupts, or running something else.
> >
> > The task will be scheduled on this idle cpu/core of (X-3% capacity) if
> > no other idle CPU with X capacity is available.
> > The selection in select_idle_capacity is:
> > - Idle core with enough capacity
> > - Idle core with enough capacity but doesn't fullfill uclamp_min
> > - Idle core that doesn't fit
> > - Idle CPU with enough capacity
> > - Idle CPU with enough capacity but doesn't fullfill uclamp_min
> > - Idle CPU that doesn't fit
>
> Right, and when the scan budget is exhausted select_idle_capacity()
> returns the best idle candidate found so far, even if it doesn't fit.
>
> However, in the zero-budget case, we return before examining any
> candidates in the scan domain. Relaxing the capacity-fit rule inside
> that scan wouldn't change the outcome, because we never reach that
> check.
>
> Maybe Rik's suggestion could instead help with the earlier checks in
> select_idle_sibling()? Accepting an idle target, previous or recently
> used CPU whose capacity is "close enough" could avoid reaching the
> zero-budget bailout.
>
> The tradeoff is that we'd accept a potentially weaker CPU immediately,
> without searching for a better one.
>

Do you something similar to cluster wakeup, give the wakeup a second chance to find an idle
CPU in the LLC. I suppose what you suggested is:


diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 470ba0dc6719..d1f3b179013e 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9141,6 +9141,24 @@ static inline bool asym_fits_cpu(unsigned long util,
return true;
}

+static int asym_idle_retry(struct task_struct *p, int prev, int target,
+ int recent)
+{
+ if (choose_idle_cpu(target, p))
+ return target;
+
+ if (prev != target && cpus_share_cache(prev, target) &&
+ choose_idle_cpu(prev, p))
+ return prev;
+
+ if (recent != prev && recent != target &&
+ cpus_share_cache(recent, target) &&
+ choose_idle_cpu(recent, p))
+ return recent;
+
+ return target;
+}
+
/*
* Try and locate an idle core/thread in the LLC cache domain.
*/
@@ -9149,7 +9167,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
bool has_idle_core = false;
struct sched_domain *sd;
unsigned long task_util, util_min, util_max;
- int i, recent_used_cpu, prev_aff = -1;
+ int i, recent_used_cpu, recent, prev_aff = -1;

/*
* On asymmetric system, update task utilization because we will check
@@ -9206,6 +9224,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)

/* Check a recently used CPU as a potential idle candidate: */
recent_used_cpu = p->recent_used_cpu;
+ recent = recent_used_cpu;
p->recent_used_cpu = prev;
if (recent_used_cpu != prev &&
recent_used_cpu != target &&
@@ -9244,7 +9263,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
target = i;
goto select_smt_priority;
}
- return target;
+ return asym_idle_retry(p, prev, target, recent);
}
}

--
2.43.0


> Thanks,
> -Andrea
>
> >
> > >
> > > In that situation, it will be better to run the
> > > task on the CPU with 97% capacity right now,
> > > than to not run the task until a CPU frees up
> > > later.
> > >
> > > Things like SMT and power-driven hardware
> > > scaling complicate the matter further, because
> > > we simply cannot see the full amount of capacity
> > > that is available on each CPU.
> > >
> > > That makes me wonder if we should simply accept
> > > any CPU that's "close enough to X" when dealing
> > > with asymmetrical placement.
> > >
> > > Say, anything that's above 3/4 of the capacity
> > > of X would be considered good enough to run the task.
> > >
> > > That still prevents us from placing a CPU intensive
> > > task on a little core in a big.LITTLE system, or
> > > from placing a task on a CPU that spends 80% of its
> > > time handling interrupts, but it will allow a task
> > > to run right now, when the capacity imbalance is small.
> > >
> > > Does that make sense, or am I forgetting something
> > > important?
> > >
> > > --
> > > All Rights Reversed.