Re: [PATCH V9 3/8] dax: add fsdev.c driver for fs-dax on character dax

[John Groves]

On 26/03/26 05:46PM, Ira Weiny wrote:
> John Groves wrote:
> > On 26/03/25 11:04AM, Ira Weiny wrote:
> > > John Groves wrote:
> > > > On 26/03/24 02:39PM, Jonathan Cameron wrote:
> > > > > On Tue, 24 Mar 2026 00:38:31 +0000
> > > > > John Groves <john@xxxxxxxxxxxxxx> wrote:
> > > > > 
> > > > > > From: John Groves <john@xxxxxxxxxx>
> > > > > > 
> > > > > > The new fsdev driver provides pages/folios initialized compatibly with
> > > > > > fsdax - normal rather than devdax-style refcounting, and starting out
> > > > > > with order-0 folios.
> > > > > > 
> > > > > > When fsdev binds to a daxdev, it is usually (always?) switching from the
> > > > > > devdax mode (device.c), which pre-initializes compound folios according
> > > > > > to its alignment. Fsdev uses fsdev_clear_folio_state() to switch the
> > > > > > folios into a fsdax-compatible state.
> > > > > > 
> > > > > > A side effect of this is that raw mmap doesn't (can't?) work on an fsdev
> > > > > > dax instance. Accordingly, The fsdev driver does not provide raw mmap -
> > > > > > devices must be put in 'devdax' mode (drivers/dax/device.c) to get raw
> > > > > > mmap capability.
> > > > > > 
> > > > > > In this commit is just the framework, which remaps pages/folios compatibly
> > > > > > with fsdax.
> > > > > > 
> > > > > > Enabling dax changes:
> > > > > > 
> > > > > > - bus.h: add DAXDRV_FSDEV_TYPE driver type
> > > > > > - bus.c: allow DAXDRV_FSDEV_TYPE drivers to bind to daxdevs
> > > > > > - dax.h: prototype inode_dax(), which fsdev needs
> > > > > > 
> > > > > > Suggested-by: Dan Williams <dan.j.williams@xxxxxxxxx>
> > > > > > Suggested-by: Gregory Price <gourry@xxxxxxxxxx>
> > > > > > Signed-off-by: John Groves <john@xxxxxxxxxx>
> > > > > 
> > > > > I was kind of thinking you'd go with a hidden KCONFIG option with default
> > > > > magic to do the same build condition to you had in the Makefil, but one the
> > > > > user can opt in or out for is also fine.
> > > > > 
> > > > > Comments on that below. Meh, I think this is better anyway :)
> > > > > 
> > > > > Reviewed-by: Jonathan Cameron <jonathan.cameron@xxxxxxxxxx>
> > > > > 
> > > > > 
> > > > > 
> > > > > > diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig
> > > > > > index d656e4c0eb84..7051b70980d5 100644
> > > > > > --- a/drivers/dax/Kconfig
> > > > > > +++ b/drivers/dax/Kconfig
> > > > > > @@ -61,6 +61,17 @@ config DEV_DAX_HMEM_DEVICES
> > > > > >  	depends on DEV_DAX_HMEM && DAX
> > > > > >  	def_bool y
> > > > > >  
> > > > > > +config DEV_DAX_FSDEV
> > > > > > +	tristate "FSDEV DAX: fs-dax compatible devdax driver"
> > > > > > +	depends on DEV_DAX && FS_DAX
> > > > > > +	help
> > > > > > +	  Support fs-dax access to DAX devices via a character device
> > > > > > +	  interface. Unlike device_dax (which pre-initializes compound folios
> > > > > > +	  based on device alignment), this driver leaves folios at order-0 so
> > > > > > +	  that fs-dax filesystems can manage folio order dynamically.
> > > > > > +
> > > > > > +	  Say M if unsure.
> > > > > Fine like this, but if you wanted to hide it in interests of not
> > > > > confusing users...
> > > > > 
> > > > > config DEV_DAX_FSDEV
> > > > > 	tristate
> > > > > 	depends on DEV_DAX && FS_DAX
> > > > > 	default DEV_DAX
> > > > 
> > > > I like this better. I see no reason not to default to including fsdev.
> > > > It does nothing other than frustrating famfs users if it's off - since
> > > > building it still has no effect unless you put a daxdev in famfs mode.
> > > > 
> > > > Ira, it's kinda in your hands at the moment. Do you feel like making this
> > > > change?
> > > 
> > > I don't mind making this change.  But we have to deal with the breakage to
> > > current device dax users.
> > > 
> > > https://lore.kernel.org/all/69c36921255b6_e9d8d1009b@iweiny-mobl.notmuch/
> > > 
> > > What am I missing?
> > > 
> > > Ira
> > 
> > OK, I can reproduce that failure with kernel 7.0.0-rc5 and 
> > straight ndctl v84. So it's not famfs.
> 
> No it is the fsdev_dax driver which causes the issue.
> 
> I can reload the driver and effectively change the order the drivers are
> searched.
> 
> I can prove this with a simple print.  With my test system (where
> fsdev_dax _happens_ to be the first driver searched) the failure happens.
> 
> [  526.564232] IKW searching drv type 0 ; type 1
> [  526.564515] IKW searching drv type 2 ; type 1
> 
> If I remove your driver (modprobe -r fsdev_dax) prior to running the test
> I get.
> 
> [   59.748171] IKW searching drv type 0 ; type 1
> [   59.749127] IKW searching drv type 1 ; type 1
> 
> And it passes.  I can continue by loading fsdev_dax back and it will
> continue to work.  If you are getting this to pass it must be because in
> your system that driver gets loaded first...  not sure how.
> 
> This is with the same exact kernel just with your module removed at run
> time.
> 
> dax_match_type() needs some other way of matching when the fsdev_dax
> driver should be used.

I think the correct answer is that fsdev/famfs should never automatically 
match and bind. Weird that I haven't seen it do that (or maybe it did but
I didn't notice?)

If one does a mkfs.famfs or 'famfs mount', the famfs tools already try to 
bind fsdev/famfs mode if necessary and fail if they can't.

> 
> I'm not seeing a clear path ATM.

I do, but I need to test it out. If it works I'll send a v10 patch set
in a day or two.

Also, I am definitely seeing ndctl/dax test failures from the device-dax 
and dm.sh tests at rc5 with no famfs code (dax or otherwise) at all; I'm 
puzzled that you don't see any ndctl test failures in that situation. If 
I understood Allison correctly, she saw something similar to what I saw). 
But no worries, we'll get it sorted.

If my strategy works, the next version won't ever automatically bind fsdev,
but it will be explicitly bindable via daxctl or famfs tools. Famfs does not 
need fsdev to ever be automatically bound do dax mem...

> 
> > 
> > I also studied the verbose logs trying to figure out if famfs
> > could cause it (while running a famfs kernel and ndctl), but
> > I don't see it.
> > 
> > Then I tried non-famfs kernel and ndctl and it's the same with
> > or without famfs kernel and famfs ndctl.
> 
> :-/  I'm not seeing any failures with rc5.
> 
> Also I'm not running with famfs.  Just the dax changes.

Right - if fsdev ever gets automatically bound instead of 
drivers/dax/device.c, that's my bad. Weird that I haven't seen that happen, 
but that's why we review and test :D

> 
> Ira

Thank you,
John