Re: [PATCH V8 10/11] fs/xfs: Change xfs_ioctl_setattr_dax_invalidate()

[Ira Weiny]

On Mon, Apr 20, 2020 at 12:31:31PM +1000, Dave Chinner wrote:
> On Tue, Apr 14, 2020 at 11:45:22PM -0700, ira.weiny@xxxxxxxxx wrote:
> > From: Ira Weiny <ira.weiny@xxxxxxxxx>
> > 
> > We only support changing FS_XFLAG_DAX on directories.  Files get their
> > flag from the parent directory on creation only.  So no data
> > invalidation needs to happen.
> > 
> > Alter the xfs_ioctl_setattr_dax_invalidate() to be
> > xfs_ioctl_setattr_dax_validate().  xfs_ioctl_setattr_dax_validate() now
> > validates that any FS_XFLAG_DAX change is ok.
> > 
> > This also allows use to remove the join_flags logic.
> > 
> > Signed-off-by: Ira Weiny <ira.weiny@xxxxxxxxx>
> > 
> > ---
> > Changes from V7:
> > 	Use new flag_inode_dontcache()
> > 	Skip don't cache if mount over ride is active.
> > 
> > Changes from v6:
> > 	Fix completely broken implementation and update commit message.
> > 	Use the new VFS layer I_DONTCACHE to facilitate inode eviction
> > 	and S_DAX changing on drop_caches
> > 
> > Changes from v5:
> > 	New patch
> > ---
> >  fs/xfs/xfs_ioctl.c | 105 +++++++++------------------------------------
> >  1 file changed, 20 insertions(+), 85 deletions(-)
> > 
> > diff --git a/fs/xfs/xfs_ioctl.c b/fs/xfs/xfs_ioctl.c
> > index c6cd92ef4a05..75d4a830ef38 100644
> > --- a/fs/xfs/xfs_ioctl.c
> > +++ b/fs/xfs/xfs_ioctl.c
> > @@ -1145,63 +1145,28 @@ xfs_ioctl_setattr_xflags(
> >  }
> >  
> >  /*
> > - * If we are changing DAX flags, we have to ensure the file is clean and any
> > - * cached objects in the address space are invalidated and removed. This
> > - * requires us to lock out other IO and page faults similar to a truncate
> > - * operation. The locks need to be held until the transaction has been committed
> > - * so that the cache invalidation is atomic with respect to the DAX flag
> > - * manipulation.
> > + * Mark inodes with a changing FS_XFLAG_DAX, I_DONTCACHE
> 
> That describes what the code is doing, not why.

I'll remove the comment.

> 
> >   */
> > -static int
> > +static void
> >  xfs_ioctl_setattr_dax_invalidate(
> >  	struct xfs_inode	*ip,
> > -	struct fsxattr		*fa,
> > -	int			*join_flags)
> > +	struct fsxattr		*fa)
> 
> It's not an invalidation function anymore, so needs a name change.

Done.

> 
> > -	if (!(fa->fsx_xflags & FS_XFLAG_DAX) && !IS_DAX(inode))
> > -		return 0;
> > +	struct xfs_mount	*mp = ip->i_mount;
> > +	struct inode            *inode = VFS_I(ip);
> >  
> >  	if (S_ISDIR(inode->i_mode))
> > -		return 0;
> > -
> > -	/* lock, flush and invalidate mapping in preparation for flag change */
> > -	xfs_ilock(ip, XFS_MMAPLOCK_EXCL | XFS_IOLOCK_EXCL);
> > -	error = filemap_write_and_wait(inode->i_mapping);
> > -	if (error)
> > -		goto out_unlock;
> > -	error = invalidate_inode_pages2(inode->i_mapping);
> > -	if (error)
> > -		goto out_unlock;
> > +		return;
> >  
> > -	*join_flags = XFS_MMAPLOCK_EXCL | XFS_IOLOCK_EXCL;
> > -	return 0;
> > -
> > -out_unlock:
> > -	xfs_iunlock(ip, XFS_MMAPLOCK_EXCL | XFS_IOLOCK_EXCL);
> > -	return error;
> > +	if (mp->m_flags & XFS_MOUNT_DAX_ALWAYS ||
> > +	    mp->m_flags & XFS_MOUNT_DAX_NEVER)
> > +		return;
> 
> 	if (mp->m_flags & (XFS_MOUNT_DAX_ALWAYS | XFS_MOUNT_DAX_NEVER))
> 		return;
> > +	if (((fa->fsx_xflags & FS_XFLAG_DAX) &&
> > +	    !(ip->i_d.di_flags2 & XFS_DIFLAG2_DAX)) ||
> > +	    (!(fa->fsx_xflags & FS_XFLAG_DAX) &&
> > +	     (ip->i_d.di_flags2 & XFS_DIFLAG2_DAX)))
> > +		flag_inode_dontcache(inode);
> 
> This doesn't set the XFS inode's "don't cache" flag, despite it
> having one that serves exactly the same purpose.  IOWs, if the XFS_IDONTCACHE
> flag is now redundant, please replace it's current usage with this new flag
> and get rid of the XFS inode flag. i.e.  the only place we set XFS_IDONTCACHE
> can be replaced with a call to this mark_inode_dontcache() call...

I agree, and I would have removed XFS_IDONTCACHE, except I was not convinced
that XFS_IDONTCACHE was redundant.

Currently XFS_IDONTCACHE can be cleared if the inode is found in the cache and
I was unable to convince myself that it would be ok to remove it.  I mentioned
this to Darrick in V7.

https://lore.kernel.org/lkml/20200413194432.GD1649878@xxxxxxxxxxxxxxxxxxxxxxxxx/

What am I missing with this code?

xfs_iget_cache_hit():
...
        if (!(flags & XFS_IGET_INCORE))
		xfs_iflags_clear(ip, XFS_ISTALE | XFS_IDONTCACHE);
...

Why is XFS_IDONTCACHE not 'sticky'?  And why does xfs_iget_cache_hit() clear it
rather than fail when XFS_IDONTCACHE is set?

Ira

> 
> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@xxxxxxxxxxxxx