[PATCH] rtmutex: Consider deadline tasks in try_to_take_rt_mutex()

[Xunlei Pang]

If mutliple tasks contest try_to_take_rt_mutex(), it should
let the high-priority task acquire the lock, but it misses
the deadline tasks in the following condition:
    if (task->prio >= rt_mutex_top_waiter(lock)->prio)
        return 0;

Deadline tasks all have "-1" prio, so above logic will always
return 0, this is wrong.

To solve this, we added extra deadline comparing, and let the
task with smaller deadline win, or the top waiter win if equal.

Signed-off-by: Xunlei Pang <xlpang@xxxxxxxxxx>
---
 kernel/locking/rtmutex.c | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c
index 244b9f3..45d2f36 100644
--- a/kernel/locking/rtmutex.c
+++ b/kernel/locking/rtmutex.c
@@ -778,11 +778,24 @@ static int try_to_take_rt_mutex(struct rt_mutex *lock, struct task_struct *task,
 		 */
 		if (rt_mutex_has_waiters(lock)) {
 			/*
-			 * If @task->prio is greater than or equal to
-			 * the top waiter priority (kernel view),
+			 * If !deadline @task->prio is greater than or
+			 * equal to the top waiter priority (kernel view),
 			 * @task lost.
 			 */
-			if (task->prio >= rt_mutex_top_waiter(lock)->prio)
+			if (task->prio >= rt_mutex_top_waiter(lock)->prio &&
+			    !dl_prio(task->prio))
+				return 0;
+
+			/*
+			 * If the top waiter is deadline, @task must be
+			 * deadline, otherwise did return 0 above.
+			 *
+			 * If @task's deadline is not smaller than the
+			 * top waiter's, @task lost.
+			 */
+			if (dl_prio(rt_mutex_top_waiter(lock)->prio) &&
+			    !dl_time_before(task->dl.deadline,
+				  rt_mutex_top_waiter(lock)->task->dl.deadline))
 				return 0;
 
 			/*
-- 
1.8.3.1