Re: [RFC PATCH 0/2] Prioritize idle cpus over SCHED_IDLE ones
From: Abel Wu
Date: Mon Mar 10 2025 - 04:25:35 EST
loop Viresh Kumar
On 3/10/25 3:40 PM, Abel Wu wrote:
Since commit
17346452b25b ("sched/fair: Make sched-idle CPU selection consistent")
the SCHED_IDLE cpus are given equal or more preference over idle ones in
fast- and slow-path of select_task_rq_fair(). This works well in flattened
cgroup hierarchy, e.g. all tasks running inside root cgroup, but may not
ideal in a deeper hierarchy where SCHED_IDLE doesn't mean anything between
entities whose parents are different. Hence a SCHED_IDLE cpu does NOT
necessarily implies any of the following:
- It is a less loaded cpu (since the parent of its topmost idle
ancestor could be a 'giant' entity with large cpu.weight).
- It can be expected to be preempted by a newly woken task soon
enough (which actually depends on their ancestors who have
common parent).
As a less loaded cpu probably has better ability to serve the newly
woken task, which also applies to the SCHED_IDLE cpus that less loaded
SCHED_IDLE cpu might be easier and faster preempted, let's not special
case SCHED_IDLE cpus at least in slowpath when selecting.
[ Plan to try the same in SIS, that is selecting a less loaded cpu
and get rid of ambiguous sched_idle_cpu(). Comments and suggests
are really appreciated. ]
Some benchmarks are tested in a Intel(R) Xeon(R) CPU E5-2650 v4 machine
based on tip/master ec3106906cbf, and are tested in following two setup:
(t1) (t2)
root root
/ \\ / \
benchmark noise benchmark X
\\
noise
where noise comes from a SCHED_IDLE cgroup. I didn't see an obvious diff
in hackbench or tbench, but not schbench.
vanilla-t1 patched-t1
Lat 50.0th-qrtle-1 8.00 ( 0.00%) 8.00 ( 0.00%)
Lat 90.0th-qrtle-1 13.00 ( 0.00%) 13.00 ( 0.00%)
Lat 99.0th-qrtle-1 19.00 ( 0.00%) 18.00 ( 5.26%)
Lat 99.9th-qrtle-1 22.00 ( 0.00%) 22.00 ( 0.00%)
Lat 20.0th-qrtle-1 257.00 ( 0.00%) 259.00 ( -0.78%)
Lat 50.0th-qrtle-2 8.00 ( 0.00%) 9.00 ( -12.50%)
Lat 90.0th-qrtle-2 13.00 ( 0.00%) 13.00 ( 0.00%)
Lat 99.0th-qrtle-2 19.00 ( 0.00%) 19.00 ( 0.00%)
Lat 99.9th-qrtle-2 23.00 ( 0.00%) 24.00 ( -4.35%)
Lat 20.0th-qrtle-2 484.00 ( 0.00%) 434.00 ( 10.33%)
Lat 50.0th-qrtle-4 9.00 ( 0.00%) 9.00 ( 0.00%)
Lat 90.0th-qrtle-4 14.00 ( 0.00%) 14.00 ( 0.00%)
Lat 99.0th-qrtle-4 19.00 ( 0.00%) 19.00 ( 0.00%)
Lat 99.9th-qrtle-4 25.00 ( 0.00%) 24.00 ( 4.00%)
Lat 20.0th-qrtle-4 877.00 ( 0.00%) 857.00 ( 2.28%)
Lat 50.0th-qrtle-8 9.00 ( 0.00%) 9.00 ( 0.00%)
Lat 90.0th-qrtle-8 14.00 ( 0.00%) 15.00 ( -7.14%)
Lat 99.0th-qrtle-8 20.00 ( 0.00%) 21.00 ( -5.00%)
Lat 99.9th-qrtle-8 26.00 ( 0.00%) 26.00 ( 0.00%)
Lat 20.0th-qrtle-8 1698.00 ( 0.00%) 1598.00 ( 5.89%)
Lat 50.0th-qrtle-16 10.00 ( 0.00%) 10.00 ( 0.00%)
Lat 90.0th-qrtle-16 15.00 ( 0.00%) 15.00 ( 0.00%)
Lat 99.0th-qrtle-16 20.00 ( 0.00%) 20.00 ( 0.00%)
Lat 99.9th-qrtle-16 28.00 ( 0.00%) 27.00 ( 3.57%)
Lat 20.0th-qrtle-16 2900.00 ( 0.00%) 2860.00 ( 1.38%)
Lat 50.0th-qrtle-23 9.00 ( 0.00%) 9.00 ( 0.00%)
Lat 90.0th-qrtle-23 15.00 ( 0.00%) 15.00 ( 0.00%)
Lat 99.0th-qrtle-23 49.00 ( 0.00%) 28.00 ( 42.86%)
Lat 99.9th-qrtle-23 1602.00 ( 0.00%) 1458.00 ( 8.99%)
Lat 20.0th-qrtle-23 3796.00 ( 0.00%) 3804.00 ( -0.21%)
vanilla-t2 patched-t2
Lat 50.0th-qrtle-1 10.00 ( 0.00%) 9.00 ( 10.00%)
Lat 90.0th-qrtle-1 271.00 ( 0.00%) 272.00 ( -0.37%)
Lat 99.0th-qrtle-1 1013.00 ( 0.00%) 793.00 ( 21.72%)
Lat 99.9th-qrtle-1 1998.00 ( 0.00%) 1642.00 ( 17.82%)
Lat 20.0th-qrtle-1 262.00 ( 0.00%) 262.00 ( 0.00%)
Lat 50.0th-qrtle-2 10.00 ( 0.00%) 9.00 ( 10.00%)
Lat 90.0th-qrtle-2 531.00 ( 0.00%) 510.00 ( 3.95%)
Lat 99.0th-qrtle-2 1034.00 ( 0.00%) 933.00 ( 9.77%)
Lat 99.9th-qrtle-2 1846.00 ( 0.00%) 1614.00 ( 12.57%)
Lat 20.0th-qrtle-2 517.00 ( 0.00%) 517.00 ( 0.00%)
Lat 50.0th-qrtle-4 10.00 ( 0.00%) 10.00 ( 0.00%)
Lat 90.0th-qrtle-4 523.00 ( 0.00%) 494.00 ( 5.54%)
Lat 99.0th-qrtle-4 973.00 ( 0.00%) 901.00 ( 7.40%)
Lat 99.9th-qrtle-4 1618.00 ( 0.00%) 1606.00 ( 0.74%)
Lat 20.0th-qrtle-4 1015.00 ( 0.00%) 1001.00 ( 1.38%)
Lat 50.0th-qrtle-8 11.00 ( 0.00%) 10.00 ( 9.09%)
Lat 90.0th-qrtle-8 527.00 ( 0.00%) 454.00 ( 13.85%)
Lat 99.0th-qrtle-8 931.00 ( 0.00%) 901.00 ( 3.22%)
Lat 99.9th-qrtle-8 1598.00 ( 0.00%) 1582.00 ( 1.00%)
Lat 20.0th-qrtle-8 1738.00 ( 0.00%) 1666.00 ( 4.14%)
Lat 50.0th-qrtle-16 9.00 ( 0.00%) 9.00 ( 0.00%)
Lat 90.0th-qrtle-16 410.00 ( 0.00%) 243.00 ( 40.73%)
Lat 99.0th-qrtle-16 979.00 ( 0.00%) 891.00 ( 8.99%)
Lat 99.9th-qrtle-16 1486.00 ( 0.00%) 1470.00 ( 1.08%)
Lat 20.0th-qrtle-16 2708.00 ( 0.00%) 2708.00 ( 0.00%)
Lat 50.0th-qrtle-23 8.00 ( 0.00%) 9.00 ( -12.50%)
Lat 90.0th-qrtle-23 454.00 ( 0.00%) 303.00 ( 33.26%)
Lat 99.0th-qrtle-23 3228.00 ( 0.00%) 2564.00 ( 20.57%)
Lat 99.9th-qrtle-23 5496.00 ( 0.00%) 5304.00 ( 3.49%)
Lat 20.0th-qrtle-23 3132.00 ( 0.00%) 3156.00 ( -0.77%)
Abel Wu (2):
Revert "sched/fair: Make sched-idle CPU selection consistent throughout"
sched/fair: Do not specialcase SCHED_IDLE cpus in select slowpath
kernel/sched/fair.c | 40 ++++++++++++++++++++++++----------------
1 file changed, 24 insertions(+), 16 deletions(-)