[PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag

[Joel Fernandes]

Binder (Android's IPC mechanism) which uses sync wake ups during synchronous
transactions to the scheduler to indicate that the waker is about to sleep
soon. The current wake up path can improved when the sync flag is passed
resulting in higher binder performance. In this patch we more strongly wake up
the wakee on the waker's CPU if sync is passed based on a few other conditions
such as wake_cap, cpus allowed.  wake_wide is checked only after the sync flag
check so that it doesn't mess up sync.  Binder throughput tests see good
improvement improvement when waking up wakee (calling thread) on the waker's
CPU (called thread) with this flag. Some tests results are below:

On an 8-core ARM64 system, following is data from running
hwbinderThroughputTest with variable number of workers and services (the
workers are clients calling into the service threads and sleeps till the
service replies to them).

2 workers calling into 4 services:
Without patch: iterations per sec: 62757
With patch:    iterations per sec: 75236 (+19.88%)

4 workers calling into 2 services:
Without patch: iterations per sec: 82379
With patch:    iterations per sec: 85829 (+4.18%)

Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Cc: Josef Bacik <jbacik@xxxxxx>
Cc: Juri Lelli <Juri.Lelli@xxxxxxx>
Cc: Brendan Jackman <brendan.jackman@xxxxxxx>
Cc: Dietmar Eggemann <dietmar.eggemann@xxxxxxx>
Cc: Matt Fleming <matt@xxxxxxxxxxxxxxxxxxx>
Cc: Rik van Riel <riel@xxxxxxxxxx>
Signed-off-by: Joel Fernandes <joelaf@xxxxxxxxxx>
---
Rik, as we discussed on IRC I am hoping that the lkp bot will also do its own
tests with this patch. I'm not sure if anything special needs to be in the
subject line to trigger the tests, if that's the case let me know and thanks!

 kernel/sched/fair.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index eca6a57527f9..808571bc8ebe 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6276,10 +6276,19 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
 	int want_affine = 0;
 	int sync = wake_flags & WF_SYNC;
 
-	if (sd_flag & SD_BALANCE_WAKE) {
+	if (sd_flag & SD_BALANCE_WAKE)
 		record_wakee(p);
-		want_affine = !wake_wide(p) && !wake_cap(p, cpu, prev_cpu)
-			      && cpumask_test_cpu(cpu, &p->cpus_allowed);
+
+	if ((sd_flag & SD_BALANCE_WAKE) && !wake_cap(p, cpu, prev_cpu)
+	    && cpumask_test_cpu(cpu, &p->cpus_allowed)) {
+		/*
+		 * WF_SYNC indicates waker will goto sleep, incase waker is
+		 * the only one running, select the waker's CPU to run wakee
+		 */
+		if (sync && cpu_rq(cpu)->nr_running < 2)
+			return cpu;
+
+		want_affine = !wake_wide(p);
 	}
 
 	rcu_read_lock();
-- 
2.14.1.342.g6490525c54-goog