Re: [PATCH] loop: Limit the number of requests in the bio list

[Lukáš Czerner]

On Wed, 3 Oct 2012, Jeff Moyer wrote:

> Date: Wed, 03 Oct 2012 10:30:54 -0400
> From: Jeff Moyer <jmoyer@xxxxxxxxxx>
> To: Dave Chinner <dchinner@xxxxxxxxxx>
> Cc: LukÃÅ Czerner <lczerner@xxxxxxxxxx>, Jens Axboe <axboe@xxxxxxxxx>,
>     linux-kernel@xxxxxxxxxxxxxxx
> Subject: Re: [PATCH] loop: Limit the number of requests in the bio list
> 
> Dave Chinner <dchinner@xxxxxxxxxx> writes:
> 
> > On Tue, Oct 02, 2012 at 10:52:05AM +0200, LukÃÅ Czerner wrote:
> >> On Mon, 1 Oct 2012, Jeff Moyer wrote:
> >> > Date: Mon, 01 Oct 2012 12:52:19 -0400
> >> > From: Jeff Moyer <jmoyer@xxxxxxxxxx>
> >> > To: Lukas Czerner <lczerner@xxxxxxxxxx>
> >> > Cc: Jens Axboe <axboe@xxxxxxxxx>, linux-kernel@xxxxxxxxxxxxxxx,
> >> >     Dave Chinner <dchinner@xxxxxxxxxx>
> >> > Subject: Re: [PATCH] loop: Limit the number of requests in the bio list
> >> > 
> >> > Lukas Czerner <lczerner@xxxxxxxxxx> writes:
> >> > 
> >> > > Currently there is not limitation of number of requests in the loop bio
> >> > > list. This can lead into some nasty situations when the caller spawns
> >> > > tons of bio requests taking huge amount of memory. This is even more
> >> > > obvious with discard where blkdev_issue_discard() will submit all bios
> >> > > for the range and wait for them to finish afterwards. On really big loop
> >> > > devices this can lead to OOM situation as reported by Dave Chinner.
> >> > >
> >> > > With this patch we will wait in loop_make_request() if the number of
> >> > > bios in the loop bio list would exceed 'nr_requests' number of requests.
> >> > > We'll wake up the process as we process the bios form the list.
> >> > 
> >> > I think you might want to do something similar to what is done for
> >> > request_queues by implementing a congestion on and off threshold.  As
> >> > Jens writes in this commit (predating the conversion to git):
> >> 
> >> Right, I've had the same idea. However my first proof-of-concept
> >> worked quite well without this and my simple performance testing did
> >> not show any regression.
> 
> Did you look at system time?
> 
> -Jeff

Hi, none of the times showed any significant difference, there was
not any pattern suggesting a problem. Also the system time is included
in the real time, so it would show anyway I guess.

-Lukas