[PATCH v4 0/7] VFS: prepare for changes to directory locking

[NeilBrown]

The only change in this v4 over v3 are to replace lock_sync with an
acquire/release pair which builds correctly when lockdep isn't enabled,
to add the 7th patch which I previously mentioned, and to include
linux-kernel so that sashiko gets to see the series.

Previous intro:

Hi,
 As you know I am working on changes to locking for directory operations
such as lookup/create/remove/rename.  The ultimate goal is for the VFS
to lock the dentry, not the parent directory, and to push the i_rwsem
parent locking down into the filesystems where it can be kept, removed,
or adjusted as best fits each filesystem.

The next step is to change the order of locking for d_alloc_parallel() -
which allocates a locked (in-lookup) dentry or waits for an existing
dentry to be unlocked.  Currently d_alloc_parallel() is ordered below
i_rwsem on parent, so you cannot wait on i_rwsem while holding a locked
(in-lookup) dentry.

To push i_rwsem into filesystems I need to push i_rwsem below d_alloc_parallel(),
so code can wait for i_rwsem while holding an in-lookup dentry, but won't be
able to wait in d_alloc_parallel() while holding i_rwsem.

There are two broad changes that are needed before that order can be
swapped.  This series sets the direction.  Subsequent patches which I
hope can also land this cycle spread those changes throughout
filesystems.

The two changes are:
1 - don't call d_alloc_parallel() while holding i_rwsem.
   This involves introducing d_alloc_trylock() and changing
   some places to drop i_rwsem before taking d_alloc_parallel()
   for this to work they will need to know if the lock is shared
   or exclusive so LOOKUP_SHARED is added.

2 - don't d_drop() a dentry while it being worked on.  This might
   not be entirely needed yet, but it will be needed soon and doing it
   now is a convenient time, and it helps make the end result clearer.
   Calling d_drop() effectively unlocks an in-lookup denty.
   It is OK for a filesystem to do this *after* an operation has
   completed, whether in success or failure.  Doing it before completion
   will allow another dentry to be allocated maybe too early.

   Enhancing d_splice_alias() to handle hashed dentries is key to
   removing the need to d_drop() in filesystems.  Though not strictly
   necessary, this allows d_add() to be removed as there will be nothing
   that d_add() does which cannot be done with d_splice_alias()

This set of 6 patches makes core-VFS changes. After this I have
 - 7 nfs patches
 - 6 afs patches
 - 4 smb/client patches
 - 3 cephfs patches
 - 2 fuse patches
 - one each for shmem, code, configfs, hostfs, procfs, ovl
and a patch to d_add_ci() which affects xfs and ntfs.

Once all of those have landed I have 4 patches to remove deprecated interfaces.
All of these can be found at
   https://github.com/neilbrown/linux/commits/pdirops


If all goes to plan, then in the next cycle a few patches will invert the
lock order and remove LOOKUP_SHARED.  Then we can get on to the really
fun stuff! (branch pdirops-next)

Thanks,
NeilBrown


 [PATCH v4 1/7] VFS: fix various typos in documentation for
 [PATCH v4 2/7] VFS: enhance d_splice_alias() to handle hashed
 [PATCH v4 3/7] VFS: introduce d_alloc_trylock()
 [PATCH v4 4/7] VFS: add d_duplicate()
 [PATCH v4 5/7] VFS: Add LOOKUP_SHARED flag.
 [PATCH v4 6/7] VFS: add lockdep monitoring of DCACHE_PAR_LOOKUP lock.
 [PATCH v4 7/7] VFS: reserve a d_flags bit for fs-specific usage