[PATCH] percpu_rwsem: let percpu_rwsem writer get rwsem faster

[Cruz Zhao]

In the scenario where a large number of containers are created
at the same time, there will be a lot of tasks created in a
short time, and they will be written into cgroup.procs.

copy_process() will require the cgroup_threadgroup_rwsem read
lock, cgroup_procs_write will require the cgroup_threadgroup_rwsem
write lock. As the readers will pre-increase the read_count and
then check whether there is any writers, resulting that the
writer may be starving, especially when there is a steady stream
of readers.

To alleviate this problem, we add one more check whether there
are writers waiting before increasing the read_count, to make
writers getting lock faster.

Signed-off-by: Cruz Zhao <CruzZhao@xxxxxxxxxxxxxxxxx>
---
 kernel/locking/percpu-rwsem.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/kernel/locking/percpu-rwsem.c b/kernel/locking/percpu-rwsem.c
index 6083883c4fe0..66bf18c28b43 100644
--- a/kernel/locking/percpu-rwsem.c
+++ b/kernel/locking/percpu-rwsem.c
@@ -47,6 +47,11 @@ EXPORT_SYMBOL_GPL(percpu_free_rwsem);
 
 static bool __percpu_down_read_trylock(struct percpu_rw_semaphore *sem)
 {
+	if (unlikely(atomic_read_acquire(&sem->block))) {
+		rcuwait_wake_up(&sem->writer);
+		return false;
+	}
+
 	this_cpu_inc(*sem->read_count);
 
 	/*
-- 
2.43.5