Re: [PATCH V9 3/8] dax: add fsdev.c driver for fs-dax on character dax

[Ira Weiny]

John Groves wrote:
> On 26/03/26 05:46PM, Ira Weiny wrote:
> > John Groves wrote:
> > > On 26/03/25 11:04AM, Ira Weiny wrote:
> > > > John Groves wrote:
> > > > > On 26/03/24 02:39PM, Jonathan Cameron wrote:
> > > > > > On Tue, 24 Mar 2026 00:38:31 +0000
> > > > > > John Groves <john@xxxxxxxxxxxxxx> wrote:
> > > > > > 
> > > > > > > From: John Groves <john@xxxxxxxxxx>
> > > > > > > 
> > > > > > > The new fsdev driver provides pages/folios initialized compatibly with
> > > > > > > fsdax - normal rather than devdax-style refcounting, and starting out
> > > > > > > with order-0 folios.
> > > > > > > 
> > > > > > > When fsdev binds to a daxdev, it is usually (always?) switching from the
> > > > > > > devdax mode (device.c), which pre-initializes compound folios according
> > > > > > > to its alignment. Fsdev uses fsdev_clear_folio_state() to switch the
> > > > > > > folios into a fsdax-compatible state.
> > > > > > > 
> > > > > > > A side effect of this is that raw mmap doesn't (can't?) work on an fsdev
> > > > > > > dax instance. Accordingly, The fsdev driver does not provide raw mmap -
> > > > > > > devices must be put in 'devdax' mode (drivers/dax/device.c) to get raw
> > > > > > > mmap capability.
> > > > > > > 
> > > > > > > In this commit is just the framework, which remaps pages/folios compatibly
> > > > > > > with fsdax.
> > > > > > > 
> > > > > > > Enabling dax changes:
> > > > > > > 
> > > > > > > - bus.h: add DAXDRV_FSDEV_TYPE driver type
> > > > > > > - bus.c: allow DAXDRV_FSDEV_TYPE drivers to bind to daxdevs
> > > > > > > - dax.h: prototype inode_dax(), which fsdev needs
> > > > > > > 
> > > > > > > Suggested-by: Dan Williams <dan.j.williams@xxxxxxxxx>
> > > > > > > Suggested-by: Gregory Price <gourry@xxxxxxxxxx>
> > > > > > > Signed-off-by: John Groves <john@xxxxxxxxxx>
> > > > > > 
> > > > > > I was kind of thinking you'd go with a hidden KCONFIG option with default
> > > > > > magic to do the same build condition to you had in the Makefil, but one the
> > > > > > user can opt in or out for is also fine.
> > > > > > 
> > > > > > Comments on that below. Meh, I think this is better anyway :)
> > > > > > 
> > > > > > Reviewed-by: Jonathan Cameron <jonathan.cameron@xxxxxxxxxx>
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > > diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig
> > > > > > > index d656e4c0eb84..7051b70980d5 100644
> > > > > > > --- a/drivers/dax/Kconfig
> > > > > > > +++ b/drivers/dax/Kconfig
> > > > > > > @@ -61,6 +61,17 @@ config DEV_DAX_HMEM_DEVICES
> > > > > > >  	depends on DEV_DAX_HMEM && DAX
> > > > > > >  	def_bool y
> > > > > > >  
> > > > > > > +config DEV_DAX_FSDEV
> > > > > > > +	tristate "FSDEV DAX: fs-dax compatible devdax driver"
> > > > > > > +	depends on DEV_DAX && FS_DAX
> > > > > > > +	help
> > > > > > > +	  Support fs-dax access to DAX devices via a character device
> > > > > > > +	  interface. Unlike device_dax (which pre-initializes compound folios
> > > > > > > +	  based on device alignment), this driver leaves folios at order-0 so
> > > > > > > +	  that fs-dax filesystems can manage folio order dynamically.
> > > > > > > +
> > > > > > > +	  Say M if unsure.
> > > > > > Fine like this, but if you wanted to hide it in interests of not
> > > > > > confusing users...
> > > > > > 
> > > > > > config DEV_DAX_FSDEV
> > > > > > 	tristate
> > > > > > 	depends on DEV_DAX && FS_DAX
> > > > > > 	default DEV_DAX
> > > > > 
> > > > > I like this better. I see no reason not to default to including fsdev.
> > > > > It does nothing other than frustrating famfs users if it's off - since
> > > > > building it still has no effect unless you put a daxdev in famfs mode.
> > > > > 
> > > > > Ira, it's kinda in your hands at the moment. Do you feel like making this
> > > > > change?
> > > > 
> > > > I don't mind making this change.  But we have to deal with the breakage to
> > > > current device dax users.
> > > > 
> > > > https://lore.kernel.org/all/69c36921255b6_e9d8d1009b@iweiny-mobl.notmuch/
> > > > 
> > > > What am I missing?
> > > > 
> > > > Ira
> > > 
> > > OK, I can reproduce that failure with kernel 7.0.0-rc5 and 
> > > straight ndctl v84. So it's not famfs.
> > 
> > No it is the fsdev_dax driver which causes the issue.
> > 
> > I can reload the driver and effectively change the order the drivers are
> > searched.
> > 
> > I can prove this with a simple print.  With my test system (where
> > fsdev_dax _happens_ to be the first driver searched) the failure happens.
> > 
> > [  526.564232] IKW searching drv type 0 ; type 1
> > [  526.564515] IKW searching drv type 2 ; type 1
> > 
> > If I remove your driver (modprobe -r fsdev_dax) prior to running the test
> > I get.
> > 
> > [   59.748171] IKW searching drv type 0 ; type 1
> > [   59.749127] IKW searching drv type 1 ; type 1
> > 
> > And it passes.  I can continue by loading fsdev_dax back and it will
> > continue to work.  If you are getting this to pass it must be because in
> > your system that driver gets loaded first...  not sure how.
> > 
> > This is with the same exact kernel just with your module removed at run
> > time.
> > 
> > dax_match_type() needs some other way of matching when the fsdev_dax
> > driver should be used.
> 
> I think the correct answer is that fsdev/famfs should never automatically 
> match and bind. Weird that I haven't seen it do that (or maybe it did but
> I didn't notice?)

Agreed.

> 
> If one does a mkfs.famfs or 'famfs mount', the famfs tools already try to 
> bind fsdev/famfs mode if necessary and fail if they can't.

Yep.

> 
> > 
> > I'm not seeing a clear path ATM.
> 
> I do, but I need to test it out. If it works I'll send a v10 patch set
> in a day or two.
> 
> Also, I am definitely seeing ndctl/dax test failures from the device-dax 
> and dm.sh tests at rc5 with no famfs code (dax or otherwise) at all; I'm 
> puzzled that you don't see any ndctl test failures in that situation. If 
> I understood Allison correctly, she saw something similar to what I saw). 
> But no worries, we'll get it sorted.

:-/  Ok I can get dm.sh to fail with rc5.  But it is intermittent.  I'll
investigate that.

FWIW I'm not saying device-dax does not ever fail on rc5.  I've not seen
it though.

But I can definitely get it to fail with the procedure above.  If there is
another failure it would be good to send your log with a report and I'll
look at that separately.

> 
> If my strategy works, the next version won't ever automatically bind fsdev,
> but it will be explicitly bindable via daxctl or famfs tools. Famfs does not 
> need fsdev to ever be automatically bound do dax mem...
> 
> > 
> > > 
> > > I also studied the verbose logs trying to figure out if famfs
> > > could cause it (while running a famfs kernel and ndctl), but
> > > I don't see it.
> > > 
> > > Then I tried non-famfs kernel and ndctl and it's the same with
> > > or without famfs kernel and famfs ndctl.
> > 
> > :-/  I'm not seeing any failures with rc5.
> > 
> > Also I'm not running with famfs.  Just the dax changes.
> 
> Right - if fsdev ever gets automatically bound instead of 
> drivers/dax/device.c, that's my bad. Weird that I haven't seen that happen, 
> but that's why we review and test :D

Sounds good!

Thanks,
Ira