Re: [PATCH v2] loop: Limit the number of requests in the bio list

[Lukáš Czerner]

On Fri, 9 Nov 2012, Jens Axboe wrote:

> Date: Fri, 09 Nov 2012 08:34:03 +0100
> From: Jens Axboe <axboe@xxxxxxxxx>
> To: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
> Cc: Lukas Czerner <lczerner@xxxxxxxxxx>, dchinner@xxxxxxxxxx,
>     jmoyer@xxxxxxxxxx, linux-kernel@xxxxxxxxxxxxxxx
> Subject: Re: [PATCH v2] loop: Limit the number of requests in the bio list
> 
> On 2012-11-08 20:14, Andrew Morton wrote:
> > On Tue, 16 Oct 2012 11:21:45 +0200
> > Lukas Czerner <lczerner@xxxxxxxxxx> wrote:
> > 
> >> Currently there is not limitation of number of requests in the loop bio
> >> list. This can lead into some nasty situations when the caller spawns
> >> tons of bio requests taking huge amount of memory. This is even more
> >> obvious with discard where blkdev_issue_discard() will submit all bios
> >> for the range and wait for them to finish afterwards. On really big loop
> >> devices and slow backing file system this can lead to OOM situation as
> >> reported by Dave Chinner.
> >>
> >> With this patch we will wait in loop_make_request() if the number of
> >> bios in the loop bio list would exceed 'nr_requests' number of requests.
> >> We'll wake up the process as we process the bios form the list. Some
> >> threshold hysteresis is in place to avoid high frequency oscillation.
> >>
> > 
> > What's happening with this?
> 
> Sorry I didn't reply to this yet. My initial thought is that we had
> something like this for loop back in the 2.4 days, and it was deadlock
> prone. Can't seem to remember all the details on that yet.
> 
> v2 is a nice improvement, though. With 1:1 bio and wakeups, you would
> get tons of context switches. The batched approach is much better.
> 
> Lukas, have you beaten on this with a file backed loop and heavy traffic
> on a file system on top?

Hi Jeff,

sorry for the delay. Yes, I've tested this with xfstests using loop
driver for both test and scratch disk.

I'll resend the updated version hopefully later today.

Thanks!
-Lukas
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/