[PATCH 0/2] powerpc: Scheduler optimization for POWER9 bigcores

[Gautham R. Shenoy]

From: "Gautham R. Shenoy" <ego@xxxxxxxxxxxxxxxxxx>

Hi,

A pair of IBM POWER9 SMT4 cores can be fused together to form a
big-core with 8 SMT threads. This can be discovered via the
"ibm,thread-groups" CPU property in the device tree which will
indicate which group of threads that share the L1 cache, translation
cache and instruction data flow.  If there are multiple such group of
threads, then the core is a big-core. Furthermore, the thread-ids of
such a big-core is obtained by interleaving the thread-ids of the
component SMT4 cores.

Eg: Threads in the pair of component SMT4 cores of an interleaved
big-core are numbered {0,2,4,6} and {1,3,5,7} respectively.
    
On such a big-core, when multiple tasks are scheduled to run on the
big-core, we get the best performance when the tasks are spread across
the pair of SMT4 cores.

The Linux scheduler has a feature called "ASYM_SMT" which will bias
the load-balancing of the tasks on the smaller numbered threads in the
core. On an big-core whose threads are interleavings of the threads of
the small cores, enabling ASYM_SMT automatically results in spreading
the tasks uniformly across the associated pair of SMT4 cores.

In this patchset, we detect if the cores are big-cores with interleaved threads.
If so, we enable ASYM_SMT feature at the SMT-sched domain.

Experimental results for ebizzy with 2 threads, bound to a single big-core
show a marked improvement with this patchset over the 4.17-rc3 vanilla
kernel.

The results of 100 such runs, for 4.17-rc3 vanilla kernel and
4.17-rc3 + this_patchset are as follows.

4.17-rc3 vanilla:
================================================
  ebizzy records/s   :# samples: Histogram 
=================================================
[1000000 - 2000000]  :      14    : ###
[2000000 - 3000000]  :      13    : ###
[3000000 - 4000000]  :      15    : ####
[4000000 - 5000000]  :       2    : #
[5000000 - 6000000]  :      56    : ############
=================================================
 
4.17-rc3 vanilla + this_patchset
=================================================
  ebizzy records/s   :# samples: Histogram 
=================================================
[1000000 - 2000000]  :       0    : #
[2000000 - 3000000]  :      14    : ###
[3000000 - 4000000]  :       0    : #
[4000000 - 5000000]  :       1    : #
[5000000 - 6000000]  :      85    : ##################
=================================================

This patchset contains two patchset.

The first patch detects the presence of big-cores with interleaved
threads.

The second patch adds the ASYM_SMT bit to the flags the SMT-sched
domain when interleaved big-cores are detected.

Gautham R. Shenoy (2):
  powerpc: Detect the presence of big-core with interleaved threads
  powerpc: Enable ASYM_SMT on interleaved big-core systems

 arch/powerpc/include/asm/cputhreads.h |  8 +++--
 arch/powerpc/kernel/setup-common.c    | 63 +++++++++++++++++++++++++++++++++--
 arch/powerpc/kernel/smp.c             |  2 +-
 3 files changed, 67 insertions(+), 6 deletions(-)

-- 
1.9.4