Re: [RFC V2 14/18] famfs_fuse: GET_DAXDEV message and daxdev_table

[Darrick J. Wong]

On Fri, Aug 15, 2025 at 11:38:02AM -0500, John Groves wrote:
> On 25/08/14 03:58PM, Miklos Szeredi wrote:
> > On Thu, 3 Jul 2025 at 20:54, John Groves <John@xxxxxxxxxx> wrote:
> > >
> > > * The new GET_DAXDEV message/response is enabled
> > > * The command it triggered by the update_daxdev_table() call, if there
> > >   are any daxdevs in the subject fmap that are not represented in the
> > >   daxdev_dable yet.
> > 
> > This is rather convoluted, the server *should know* which dax devices
> > it has registered, hence it shouldn't need to be explicitly asked.
> 
> That's not impossible, but it's also a bit harder than the current
> approach for the famfs user space - which I think would need to become
> stateful as to which daxdevs had been pushed into the kernel. The
> famfs user space is as unstateful as possible ;)
> 
> > 
> > And there's already an API for registering file descriptors:
> > FUSE_DEV_IOC_BACKING_OPEN.  Is there a reason that interface couldn't
> > be used by famfs?
> 
> FUSE_DEV_IOC_BACKING_OPEN looks pretty specific to passthrough. The
> procedure for opening a daxdev is stolen from the way xfs does it in 
> fs-dax mode. It calls fs_dax_get() rather then open(), and passes in 
> 'famfs_fuse_dax_holder_ops' to 1) effect exclusivity, and 2) receive
> callbacks in the event of memory errors. See famfs_fuse_get_daxdev()...

Yeah, that's about what I would do to wire up fsdax in fuse-iomap.

> A *similar* ioctl could be added to push in a daxdev, but that would
> still add statefulness that isn't required in the current implementation.
> I didn't go there because there are so few IOCTLs currently (the overall 
> model is more 'pull' than 'push').
> 
> Keep in mind that the baseline case with famfs will be files that are 
> interleaved across strips from many daxdevs. We commonly create files 
> that are striped across 16 daxdevs, selected at random from a
> significantly larger pool. Because interleaving is essential to memory 
> performance...
> 
> There is no "device mapper" analog for memory, so famfs really does 
> have to span daxdevs. As Darrick commented somewhere, famfs fmaps do 
> repeating patterns well (i.e. striping)...
> 
> I think there is a certain elegance to the current approach, but
> if you feel strongly I will change it.

I still kinda wonder if you actually want BPF for this sort of thing
(programmatically computed file IO mappings) since they'd give you more
flexibility than hardcoded C in the kernel.

--D

> Thanks!
> John
> 
>