Re: [RFC PATCH 04/16] sched/fair: Remove magic hardcoded margin in fits_capacity()

[Dietmar Eggemann]

On 22/08/2024 07:09, Sultan Alsawaf (unemployed) wrote:
> Hi Qais,
> 
> On Tue, Aug 20, 2024 at 05:35:00PM +0100, Qais Yousef wrote:
>> Replace hardcoded margin value in fits_capacity() with better dynamic
>> logic.
>>
>> 80% margin is a magic value that has served its purpose for now, but it
>> no longer fits the variety of systems that exist today. If a system is
>> over powered specifically, this 80% will mean we leave a lot of capacity
>> unused before we decide to upmigrate on HMP system.
>>
>> On many systems the little cores are under powered and ability to
>> migrate faster away from them is desired.
>>
>> Redefine misfit migration to mean the utilization threshold at which the
>> task would become misfit at the next load balance event assuming it
>> becomes an always running task.
>>
>> To calculate this threshold, we use the new approximate_util_avg()
>> function to find out the threshold, based on arch_scale_cpu_capacity()
>> the task will be misfit if it continues to run for a TICK_USEC which is
>> our worst case scenario for when misfit migration will kick in.

[...]

>> +	/*
>> +	 * Calculate the util at which the task must be considered a misfit.
>> +	 *
>> +	 * We must ensure that a task experiences the same ramp-up time to
>> +	 * reach max performance point of the system regardless of the CPU it
>> +	 * is running on (due to invariance, time will stretch and task will
>> +	 * take longer to achieve the same util value compared to a task
>> +	 * running on a big CPU) and a delay in misfit migration which depends
>> +	 * on TICK doesn't end up hurting it as it can happen after we would
>> +	 * have crossed this threshold.
>> +	 *
>> +	 * To ensure that invaraince is taken into account, we don't scale time
>> +	 * and use it as-is, approximate_util_avg() will then let us know the
>> +	 * our threshold.
>> +	 */
>> +	limit = approximate_runtime(arch_scale_cpu_capacity(cpu)) * USEC_PER_MSEC;
> 
> Perhaps it makes more sense to use `capacity` here instead of
> `arch_scale_cpu_capacity(cpu)`? Seems like reduced capacity due to HW pressure
> (and IRQs + RT util) should be considered, e.g. for a capacity inversion due to
> HW pressure on a mid core that results in a little core being faster.

If you want to keep it strictly 'uarch & freq-invariant' based, then it
wouldn't have to be called periodically in update_cpu_capacity(). Just
set rq->fits_capacity_threshold once after cpu_scale has been fully
(uArch & Freq) normalized.

[...]