Re: [PATCH 07/13] aio: enabled thread based async fsync

[Dave Chinner]

On Wed, Jan 20, 2016 at 03:44:49PM -0500, Benjamin LaHaise wrote:
> On Wed, Jan 20, 2016 at 12:29:32PM -0800, Linus Torvalds wrote:
> > On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner <david@xxxxxxxxxxxxx> wrote:
> > >>
> > >> Are there other users outside of Solace? It would be good to get comments..
> > >
> > > I know of quite a few storage/db products that use AIO. The most
> > > recent high profile project that have been reporting issues with AIO
> > > on XFS is http://www.scylladb.com/. That project is architected
> > > around non-blocking AIO for scalability reasons...
> > 
> > I was more wondering about the new interfaces, making sure that the
> > feature set actually matches what people want to do..
> 
> I suspect this will be an ongoing learning exercise as people start to use 
> the new functionality and find gaps in terms of what is needed.  Certainly 
> there is a bunch of stuff we need to add to cover the cases where disk i/o 
> is required.  getdents() is one example, but the ABI issues we have with it 
> are somewhat more complicated given the history associated with that 
> interface.
> 
> > That said, I also agree that it would be interesting to hear what the
> > performance impact is for existing performance-sensitive users. Could
> > we make that "aio_may_use_threads()" case be unconditional, making
> > things simpler?
> 
> Making it unconditional is a goal, but some work is required before that 
> can be the case.  The O_DIRECT issue is one such matter -- it requires some 
> changes to the filesystems to ensure that they adhere to the non-blocking 
> nature of the new interface (ie taking i_mutex is a Bad Thing that users 
> really do not want to be exposed to; if taking it blocks, the code should 
> punt to a helper thread).

Filesystems *must take locks* in the IO path. We have to serialise
against truncate and other operations at some point in the IO path
(e.g. block mapping vs concurrent allocation and/or removal), and
that can only be done sanely with sleeping locks.  There is no way
of knowing in advance if we are going to block, and so either we
always use threads for IO submission or we accept that occasionally
the AIO submission will block.

Cheers,

Dave.
-- 
Dave Chinner
david@xxxxxxxxxxxxx