Re: What am I doing wrong? submit_bio() suddenly stops working...

[Jens Axboe]

On 2010-10-21 18:55, Ted Ts'o wrote:
> On Thu, Oct 21, 2010 at 08:59:46AM +0200, Jens Axboe wrote:
>>
>> I don't see anything immediately wrong with your approach. I suspect
>> we'll need to see sysrq-t traces of the relevant processes to make a
>> more educated guess!
> 
> I've uploaded a trace output that includes the sysrq-t trace, but I
> don't think it shows anything interesting.  We're not hanging on any
> kind of loack as near as I can tell.  It looks like
> __generic_make_request() is calling q->make_request_fn(), and this is
> returning without actually doing anything.
> 
> http://userweb.kernel.org/~tytso/ext4-bio-patches/kvm-console-2
> 
> In this trace, I added a patch to prove that __generic_make_request()
> is calling __make_request (I wasn't sure what q->make_request_fn was
> indirecting to, so I added a brute force lookup to make sure I
> understood what was going on), but at one point, it just starts
> queuing the request, and it enters cfq, but the request never gets
> dispatched out.  Maybe this is a failure of the plugging/unplugging
> mechanisms?
> 
> I guess I can start putting in more brute-force printk's inside
> __make_request and inside the cfq scheduler to try to understand what
> is going on, but I'm really guessing at this point.
> 
> If you have any suggestions about more elegant ways of figuring what
> is happening, please do let me know....

I will take a look at the traces.

By the sound of things, if I were you I'd turn on the mem and slab
debugging to catch use-before-init and use-after-free. Mysterious hangs
in the IO sub system are usually caused by such bugs. And the regular
debugging aids, just to see if that produces anything of interest.

-- 
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/