RE: [PATCH v2 2/9] ext2: tell DAX the size of allocation holes

[Matthew Wilcox]

I feel like we're not only building on shifting sands, but we haven't decided whether we're building a Pyramid or a Sphinx.

I thought after Storage Summit, we had broad agreement that we were moving to a primary DAX API that was not BH (nor indeed iomap) based.  We would still have DAX helpers for block based filesystems (because duplicating all that code between filesystems is pointless), but I now know of three filesystems which are not block based that are interested in using DAX.  Jared Hulbert's AXFS is a nice public example.

I posted a prototype of this here:

https://groups.google.com/d/msg/linux.kernel/xFFHVCQM7Go/ZQeDVYTnFgAJ

It is, of course, woefully out of date, but some of the principles in it are still good (and I'm working to split it into digestible chunks).

The essence:

1. VFS or VM calls filesystem (eg ->fault()) 
2. Filesystem calls DAX (eg dax_fault()) 
3. DAX looks in radix tree, finds no information. 
4. DAX calls (NEW!) mapping->a_ops->populate_pfns 
5a. Filesystem (if not block based) does its own thing to find out the PFNs corresponding to the requested range, then inserts them into the radix tree (possible helper in DAX code)
5b. Filesystem (if block based) looks up its internal data structure (eg extent tree) and 
   calls dax_create_pfns() (see giant patch from yesterday, only instead of 
   passing a get_block_t, the filesystem has already filled in a bh which 
   describes the entire extent that this access happens to land in). 
6b. DAX takes care of calling bdev_direct_access() from dax_create_pfns().

Now, notice that there's no interaction with the rest of the filesystem here.  We can swap out BHs and iomaps relatively trivially; there's no call for making grand changes, like converting ext2 over to iomap.  The BH or iomap is only used for communicating the extent from the filesystem to DAX.

Do we have agreement that this is the right way to go?

-----Original Message-----
From: Ross Zwisler [mailto:ross.zwisler@xxxxxxxxxxxxxxx] 
Sent: Friday, September 9, 2016 12:48 PM
To: Theodore Ts'o <tytso@xxxxxxx>; Christoph Hellwig <hch@xxxxxxxxxxxxx>; Ross Zwisler <ross.zwisler@xxxxxxxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx; Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>; linux-nvdimm@xxxxxxxxxxx; Matthew Wilcox <mawilcox@xxxxxxxxxxxxx>; Dave Chinner <david@xxxxxxxxxxxxx>; linux-mm@xxxxxxxxx; Andreas Dilger <adilger.kernel@xxxxxxxxx>; Alexander Viro <viro@xxxxxxxxxxxxxxxxxx>; Jan Kara <jack@xxxxxxxx>; linux-fsdevel@xxxxxxxxxxxxxxx; linux-ext4@xxxxxxxxxxxxxxx
Subject: Re: [PATCH v2 2/9] ext2: tell DAX the size of allocation holes

On Mon, Aug 29, 2016 at 08:57:41AM -0400, Theodore Ts'o wrote:
> On Mon, Aug 29, 2016 at 12:41:16AM -0700, Christoph Hellwig wrote:
> > 
> > We're going to move forward killing buffer_heads in XFS.  I think ext4
> > would dramatically benefit from this a well, as would ext2 (although I
> > think all that DAX work in ext2 is a horrible idea to start with).
> 
> It's been on my todo list.  The only reason why I haven't done it yet
> is because I knew you were working on a solution, and I didn't want to
> do things one way for buffered I/O, and a different way for Direct
> I/O, and disentangling the DIO code and the different assumptions of
> how different file systems interact with the DIO code is a *mess*.
> 
> It may have gotten better more recently, but a few years ago I took a
> look at it and backed slowly away.....

Ted, what do you think of the idea of moving to struct iomap in ext2?

If ext2 stays with the current struct buffer_head + get_block_t interface,
then it looks like DAX basically has three options:

1) Support two I/O paths and two versions of each of the fault paths (PTE,
PMD, etc).  One of each of these would be based on struct iomap and would be
used by xfs and potentially ext4, and the other would be based on struct
buffer_head + get_block_t and would be used by ext2.

2) Only have a single struct iomap based I/O path and fault path, and add
shim/support code so that ext2 can use it, leaving the rest of ext2 to be
struct buffer_head + get_block_t based.

3) Only have a single struct buffer_head + get_block_t based DAX I/O and fault
path, and have XFS and potentially ext4 do the translation from their native
struct iomap interface.

It seems ideal for ext2 to switch along with everyone else, if getting rid of
struct buffer_head is a global goal.  If not, I guess barring technical issues
#2 above seems cleanest - move DAX to the new structure, and provide backwards
compatibility to ext2.  Thoughts?