Re: [PATCH 05/10] block: remove per-queue plugging

[Shaohua Li]

2011/3/4 Mike Snitzer <snitzer@xxxxxxxxxx>:
> I'm now hitting a lockdep issue, while running a 'for-2.6.39/stack-plug'
> kernel, when I try an fsync heavy workload to a request-based mpath
> device (the kernel ultimately goes down in flames, I've yet to look at
> the crashdump I took)
>
>
> =======================================================
> [ INFO: possible circular locking dependency detected ]
> 2.6.38-rc6-snitm+ #2
> -------------------------------------------------------
> ffsb/3110 is trying to acquire lock:
>  (&(&q->__queue_lock)->rlock){..-...}, at: [<ffffffff811b4c4d>] flush_plug_list+0xbc/0x135
>
> but task is already holding lock:
>  (&rq->lock){-.-.-.}, at: [<ffffffff8137132f>] schedule+0x16a/0x725
>
> which lock already depends on the new lock.
I hit this too. Can you check if attached debug patch fixes it?

Thanks,
Shaohua
diff --git a/block/blk-core.c b/block/blk-core.c
index 4984b46..4924aa0 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -1100,6 +1100,7 @@ static bool attempt_plug_merge(struct task_struct *tsk, struct request_queue *q,
 	struct request *rq;
 	bool ret = false;
 
+	preempt_disable();
 	plug = tsk->plug;
 	if (!plug)
 		goto out;
@@ -1122,6 +1123,7 @@ static bool attempt_plug_merge(struct task_struct *tsk, struct request_queue *q,
 		}
 	}
 out:
+	preempt_enable();
 	return ret;
 }
 
diff --git a/kernel/sched.c b/kernel/sched.c
index e806446..7b4b2f9 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -3938,6 +3938,14 @@ asmlinkage void __sched schedule(void)
 	struct rq *rq;
 	int cpu;
 
+	/*
+	 * If this task has IO plugged, make sure it gets flushed out to the
+	 * devices before we go to sleep. Must be called before below lock,
+	 * otherwise there is deadlock
+	 */
+	blk_flush_plug(current);
+	BUG_ON(current->plug && !list_empty(&current->plug->list));
+
 need_resched:
 	preempt_disable();
 	cpu = smp_processor_id();
@@ -3973,14 +3981,6 @@ need_resched_nonpreemptible:
 				if (to_wakeup)
 					try_to_wake_up_local(to_wakeup);
 			}
-			/*
-			 * If this task has IO plugged, make sure it
-			 * gets flushed out to the devices before we go
-			 * to sleep
-			 */
-			blk_flush_plug(prev);
-			BUG_ON(prev->plug && !list_empty(&prev->plug->list));
-
 			deactivate_task(rq, prev, DEQUEUE_SLEEP);
 		}
 		switch_count = &prev->nvcsw;