Re: [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-)
From: Ira Weiny
Date: Wed Aug 21 2019 - 16:44:23 EST
On Wed, Aug 21, 2019 at 04:48:10PM -0300, Jason Gunthorpe wrote:
> On Wed, Aug 21, 2019 at 11:57:03AM -0700, Ira Weiny wrote:
> > > Oh, I didn't think we were talking about that. Hanging the close of
> > > the datafile fd contingent on some other FD's closure is a recipe for
> > > deadlock..
> > The discussion between Jan and Dave was concerning what happens when a user
> > calls
> > fd = open()
> > fnctl(...getlease...)
> > addr = mmap(fd...)
> > ib_reg_mr() <pin>
> > munmap(addr...)
> > close(fd)
> I don't see how blocking close(fd) could work.
Well Dave was saying this _could_ work. FWIW I'm not 100% sure it will but I
can't prove it won't.. Maybe we are all just touching a different part of this
elephant but the above scenario or one without munmap is very reasonably
something a user would do. So we can either allow the close to complete (my
current patches) or try to make it block like Dave is suggesting.
I don't disagree with Dave with the semantics being nice and clean for the
filesystem. But the fact that RDMA, and potentially others, can "pass the
pins" to other processes is something I spent a lot of time trying to work out.
> Write it like this:
> fd = open()
> uverbs = open(/dev/uverbs)
> addr = mmap(fd...)
> ib_reg_mr() <pin>
> The order FD's are closed during sigkill is not deterministic, so when
> all the fputs happen during a kill'd exit we could end up blocking in
> close(fd) as close(uverbs) will come after in the close
> list. close(uverbs) is the thing that does the dereg_mr and releases
> the pin.
Of course, that is a different scenario which needs to be fixed in my patch
set. Now that my servers are back up I can hopefully make progress. (Power
was down for them yesterday).
> We don't need complexity with dup to create problems.
No but that complexity _will_ come unless we "zombie" layout leases.