Re: [PATCH V11.1] Documentation/dax: Update Usage section

[Ira Weiny]

On Tue, Apr 28, 2020 at 07:21:18PM -0700, Randy Dunlap wrote:
> On 4/28/20 3:21 PM, ira.weiny@xxxxxxxxx wrote:
> > From: Ira Weiny <ira.weiny@xxxxxxxxx>
> > 
> > Update the Usage section to reflect the new individual dax selection
> > functionality.
> > 
> > Signed-off-by: Ira Weiny <ira.weiny@xxxxxxxxx>
> > 
> > ---
> > Changes from V11:
> > 	Minor changes from Darrick
> > 
> > Changes from V10:
> > 	Clarifications from Dave
> > 	Add '-c' to xfs_io examples
> > 
> > Changes from V9:
> > 	Fix missing ')'
> > 	Fix trialing '"'
> 
> trailing

Thanks

> 
> > 
> > Changes from V8:
> > 	Updates from Darrick
> > 
> > Changes from V7:
> > 	Cleanups/clarifications from Darrick and Dan
> > 
> > Changes from V6:
> > 	Update to allow setting FS_XFLAG_DAX any time.
> > 	Update with list of behaviors from Darrick
> > 	https://lore.kernel.org/lkml/20200409165927.GD6741@magnolia/
> > 
> > Changes from V5:
> > 	Update to reflect the agreed upon semantics
> > 	https://lore.kernel.org/lkml/20200405061945.GA94792@xxxxxxxxxxxxxxxxxxxxxxxxx/
> > ---
> >  Documentation/filesystems/dax.txt | 142 +++++++++++++++++++++++++++++-
> >  1 file changed, 139 insertions(+), 3 deletions(-)
> > 
> > diff --git a/Documentation/filesystems/dax.txt b/Documentation/filesystems/dax.txt
> > index 679729442fd2..dc1c1aa36cc2 100644
> > --- a/Documentation/filesystems/dax.txt
> > +++ b/Documentation/filesystems/dax.txt
> > @@ -17,11 +17,147 @@ For file mappings, the storage device is mapped directly into userspace.
> >  Usage
> >  -----
> >  
> > -If you have a block device which supports DAX, you can make a filesystem
> > +If you have a block device which supports DAX, you can make a file system
> >  on it as usual.  The DAX code currently only supports files with a block
> >  size equal to your kernel's PAGE_SIZE, so you may need to specify a block
> > -size when creating the filesystem.  When mounting it, use the "-o dax"
> > -option on the command line or add 'dax' to the options in /etc/fstab.
> > +size when creating the file system.
> > +
> > +Currently 3 filesystems support DAX: ext2, ext4 and xfs.  Enabling DAX on them
> 
> Why "file system" in the first paragraph when "filesystem" is used here and below?

AFAIK they are interchangeable.  https://en.wikipedia.org/wiki/File_system

But consistency is a reasonable request.

The rest of the doc (save 1 place) uses filesystem so I will adopt that here,
and clean up that 1 place.

> 
> > +is different.
> > +
> > +Enabling DAX on ext4 and ext2
> > +-----------------------------
> > +
> > +When mounting the filesystem, use the "-o dax" option on the command line or
> > +add 'dax' to the options in /etc/fstab.  This works to enable DAX on all files
> > +within the filesystem.  It is equivalent to the '-o dax=always' behavior below.
> > +
> > +
> > +Enabling DAX on xfs
> > +-------------------
> > +
> > +Summary
> > +-------
> > +
> > + 1. There exists an in-kernel file access mode flag S_DAX that corresponds to
> > +    the statx flag STATX_ATTR_DAX.  See the manpage for statx(2) for details
> > +    about this access mode.
> > +
> > + 2. There exists a persistent flag FS_XFLAG_DAX that can be applied to regular
> > +    files and directories. This advisory flag can be set or cleared at any
> > +    time, but doing so does not immediately affect the S_DAX state.
> > +
> > + 3. If the persistent FS_XFLAG_DAX flag is set on a directory, this flag will
> > +    be inherited by all regular files and subdirectories that are subsequently
> > +    created in this directory. Files and subdirectories that exist at the time
> > +    this flag is set or cleared on the parent directory are not modified by
> > +    this modification of the parent directory.
> > +
> > + 4. There exists dax mount options which can override FS_XFLAG_DAX in the
> 
>              exist

Ah yea,
Done.

> 
> > +    setting of the S_DAX flag.  Given underlying storage which supports DAX the
> > +    following hold:
> > +
> > +    "-o dax=inode"  means "follow FS_XFLAG_DAX" and is the default.
> > +
> > +    "-o dax=never"  means "never set S_DAX, ignore FS_XFLAG_DAX."
> > +
> > +    "-o dax=always" means "always set S_DAX ignore FS_XFLAG_DAX."
> > +
> > +    "-o dax"        is a legacy option which is an alias for "dax=always".
> > +		    This may be removed in the future so "-o dax=always" is
> > +		    the preferred method for specifying this behavior.
> > +
> > +    NOTE: Modifications to and the inheritance behavior of FS_XFLAG_DAX remain
> > +    the same even when the file system is mounted with a dax option.  However,
> > +    in-core inode state (S_DAX) will be overridden until the file system is
> 
>                                      "file system" (2 times above)

Done.

> 
> > +    remounted with dax=inode and the inode is evicted from kernel memory.
> > +
> > + 5. The S_DAX policy can be changed via:
> > +
> > +    a) Setting the parent directory FS_XFLAG_DAX as needed before files are
> > +       created
> > +
> > +    b) Setting the appropriate dax="foo" mount option
> > +
> > +    c) Changing the FS_XFLAG_DAX on existing regular files and directories.
> 
>                        FS_XFLAGS_DAX flag on

Done.

> 
> > +       This has runtime constraints and limitations that are described in 6)
> > +       below.
> > +
> > + 6. When changing the S_DAX policy via toggling the persistent FS_XFLAG_DAX flag,
> > +    the change in behaviour for existing regular files may not occur
> > +    immediately.  If the change must take effect immediately, the administrator
> > +    needs to:
> > +
> > +    a) stop the application so there are no active references to the data set
> > +       the policy change will affect
> > +
> > +    b) evict the data set from kernel caches so it will be re-instantiated when
> > +       the application is restarted. This can be achieved by:
> > +
> > +       i. drop-caches
> > +       ii. a filesystem unmount and mount cycle
> 
> filesystem

Done.

> 
> > +       iii. a system reboot
> > +
> > +
> > +Details
> > +-------
> > +
> > +There are 2 per-file dax flags.  One is a persistent inode setting (FS_XFLAG_DAX)
> > +and the other is a volatile flag indicating the active state of the feature
> > +(S_DAX).
> > +
> > +FS_XFLAG_DAX is preserved within the file system.  This persistent config
> 
> file system

Done.

> 
> > +setting can be set, cleared and/or queried using the FS_IOC_FS[GS]ETXATTR ioctl
> > +(see ioctl_xfs_fsgetxattr(2)) or an utility such as 'xfs_io'.
> > +
> > +New files and directories automatically inherit FS_XFLAG_DAX from
> > +their parent directory _when_ _created_.  Therefore, setting FS_XFLAG_DAX at
> > +directory creation time can be used to set a default behavior for an entire
> > +sub-tree.
> > +
> > +To clarify inheritance, here are 3 examples:
> > +
> > +Example A:
> > +
> > +mkdir -p a/b/c
> > +xfs_io -c 'chattr +x' a
> > +mkdir a/b/c/d
> > +mkdir a/e
> > +
> > +	dax: a,e
> > +	no dax: b,c,d
> > +
> > +Example B:
> > +
> > +mkdir a
> > +xfs_io -c 'chattr +x' a
> > +mkdir -p a/b/c/d
> > +
> > +	dax: a,b,c,d
> > +	no dax:
> > +
> > +Example C:
> > +
> > +mkdir -p a/b/c
> > +xfs_io -c 'chattr +x' c
> > +mkdir a/b/c/d
> > +
> > +	dax: c,d
> > +	no dax: a,b
> > +
> > +
> > +The current enabled state (S_DAX) is set when a file inode is instantiated in
> > +memory by the kernel.  It is set based on the underlying media support, the
> > +value of FS_XFLAG_DAX and the file system's dax mount option.
> > +
> > +statx can be used to query S_DAX.  NOTE that only regular files will ever have
> > +S_DAX set and therefore statx will never indicate that S_DAX is set on
> > +directories.
> > +
> > +Setting the FS_XFLAG_DAX (specifically or through inheritance) occurs even if
> 
>            the FS_XFLAG_DAX flag

Done.

> 
> > +the underlying media does not support dax and/or the file system is overridden
> 
> file system
> 
> Just be consistent, please.

Fair enough,
Done.

> 
> > +with a mount option.
> > +
> >  
> >  
> >  Implementation Tips for Block Driver Writers
> > 
> 
> thanks.

NP, Thank you!
Ira

> -- 
> ~Randy
>