Re: [PATCH RFC 0/2] sched: Introduce idle SMT priority for asymmetric capacity systems

[Vincent Guittot]

On Mon, 12 Oct 2026 at 13:04, Mete Durlu <meted@xxxxxxxxxxxxx> wrote:
>
> On 09/10/2026 17:32, Vincent Guittot wrote:
> > On Thu, 8 Oct 2026 at 17:25, Mete Durlu <meted@xxxxxxxxxxxxx> wrote:
> >>
> >> On 08/10/2026 16:58, Shrikanth Hegde wrote:
> >>> Hi Meter/Andrea.
> >>>
> >>> On 10/8/26 5:40 PM, Andrea Righi wrote:
> >>>> Hi Mete,
> >>>>
> >>>> On Thu, Oct 08, 2026 at 10:30:46AM +0200, Mete Durlu wrote:
> >>>>> Summary
> >>>>> ===========================================================================
> >>>>> On systems with asymmetric CPU capacities, the scheduler prefers
> >>>>> fully idle
> >>>>> cores over idle SMT siblings of busy cores. This works generally well
> >>>>> but
> >>>>> virtualized platforms where low capacity cores should be avoided are not
> >>>>> considered. Introduce a new config option and arch hook to prioritize
> >>>>> SMT utilization.
> >>>
> >>> That's true only under physical CPU contention right? or is it always?
> >>
> >> Right, because of that s390 treats all CPUs as equal and starts to
> >> assign lower capacities to CPUs with low entitlement once a certain
> >> steal time threshold is crossed (aka physical CPU contention).
>
> Hi,
> Sorry for causing a misunderstanding. I should have been more patient
> and detailed while writing my reply.
>
> > I'm not sure to get what your topology is because "if s390 treats all
> > CPUs as equal"  then you should never use select_idle_capacity()
> > because  their capacity should be fixed and equal and
> > SD_ASYM_CPUCAPACITY_FULL never set.
>
> On s390 the entitlement of the vCPUs are signified by an arch specific
> value we call polarization. We can tell which vCPUs are assigned high
> and low entitlement. This entitlement value tells us which vCPUs are
> prone to experience steal time first. Without any contention all
> vCPUs behave and perform *almost* the same(of course, we want
> some priority for CPUs with high entitlement).
>
> Prior to experiencing steal time(it's what we use to determine
> contention) to get the most out of the system we assign all vCPUs
> the same capacities, making the system symmetric like you say.
> While there is no contention s390's vCPUs also perform better if
> fully idle cores are selected for task placement.
>
> > If your hypervisor lowers the capacity once a certain threshold is
> > crossed, we already account for the steal time in the scheduler with
> > irq and remove it from the capacity available for tasks but the
> > original capacity doesn't change.
> s390's linux implementation has a mechanism called hiperdispatch,
> that checks topology changes along with system steal time (and soon
> the total free capacity of the pyhsical machine) periodically and
> makes vCPU capacity adjustments accordingly.
>
> After steal time starts ramping up and cross a certain threshold,
> we start assigning lower capacities to vCPUs with low entitlement to
> hint the scheduler that they should be avoided.

If all your vCPUs starts with the same capacity, you should not use
asym_capacity or enabled it . What you describe above is system
pressure (steal time or other) on your vCPU that change over time but
not a capacity change. The value returned by arch_scale_cpu_capacity
is not supposed to change at runtime or you need to rebuild the
sched_domain topology. As an example, max_allowed_capacity does assume
capacity fixed otherwise the misfit detection will sometimes fails.

What you describe really fit into the prefered core that Shrikanth
Hegde has developped

> In this asymmetric configuration, utilizing low capacity but fully idle
> cores hurt performance. Instead packing the tasks first to SMT siblings
> of high capacity cores would give s390 a lot of benefit.
>
> I hope that clears some of the fog that I unintentionally created.
>
> [..snip..]
>