[PATCH v2 00/31] vfs: pass S_IFDIR mode to vfs_prepare_mode()

[Jori Koolstra]

There is a comment in vfs_prepare_mode() that says:

Note that it's currently valid for @type to be 0 if a directory is
created. Filesystems raise that flag individually and we need to check
whether each filesystem can deal with receiving S_IFDIR from the vfs
before we enforce a non-zero type.

It is useful to do this clean-up ahead of O_CREAT|O_DIRECTORY.
Specifically, in lookup_open() we need to replace the vfs_prepare_mode()
with something that also handles dirs. I don't really want to push the
odd

    mode = vfs_prepare_mode(idmap, dir, mode, S_IRWXUGO | S_ISVTX, 0);

further into that code, and neither do I want this to be different from
the regular vfs_mkdir() path. We can then also match on S_IFMT in
may_o_create(), instead of passing a bool to signal whether we are
creating a dir (and assuming 0 means a dir is really ugly).

It is a bit challenging to verify that passing S_IFDIR is safe, as there
are many filesystems. Claude Opus 4.8 was used to generate the context
for each mkdir implementation from which it can be judged whether
passing S_IFDIR is OK. The result was then verified by hand by looking
at how the mode argument is used in each case. To check whether all
mkdir implementations are covered, 'rg "\.mkdir" ' was used and checked
against the list of uses Claude found.

It is safe to do this clean-up except that three filesystems (fuse,
cifs, and coda) forward the mkdir @mode unchanged to something outside
the kernel. Mask S_IFDIR back out in coda_mkdir(), fuse_mkdir() and
cifs_mkdir() so that what is sent outside the kernel is unchanged.
Their maintainers can drop the mask once they have confirmed it is safe.
For the other filesystems redundant S_IFDIR OR'ing is dropped.

Jori Koolstra (31):
  vfs: pass S_IFDIR mode to vfs_prepare_mode()
  9p: drop redundant S_IFDIR from mkdir
  affs: drop redundant S_IFDIR from mkdir
  afs: drop redundant S_IFDIR from mkdir
  autofs: drop redundant S_IFDIR from mkdir
  btrfs: drop redundant S_IFDIR from mkdir
  ceph: drop redundant S_IFDIR from mkdir
  ext2: drop redundant S_IFDIR from mkdir
  ext4: drop redundant S_IFDIR from mkdir
  f2fs: drop redundant S_IFDIR from mkdir
  gfs2: drop redundant S_IFDIR from mkdir
  hfs: drop redundant S_IFDIR from mkdir
  hfsplus: drop redundant S_IFDIR from mkdir
  hpfs: drop redundant S_IFDIR from mkdir
  hugetlbfs: drop redundant S_IFDIR from mkdir
  jffs2: drop redundant S_IFDIR from mkdir
  jfs: drop redundant S_IFDIR from mkdir
  minix: drop redundant S_IFDIR from mkdir
  nilfs2: drop redundant S_IFDIR from mkdir
  ntfs3: drop redundant S_IFDIR from mkdir
  ocfs2: drop redundant S_IFDIR from mkdir
  ocfs2: dlmfs: drop redundant S_IFDIR from mkdir
  omfs: drop redundant S_IFDIR from mkdir
  orangefs: drop redundant S_IFDIR from mkdir
  ramfs: drop redundant S_IFDIR from mkdir
  udf: drop redundant S_IFDIR from mkdir
  ufs: drop redundant S_IFDIR from mkdir
  nfs: drop redundant S_IFDIR from mkdir
  ubifs: drop redundant S_IFDIR from mkdir
  xfs: drop redundant S_IFDIR from mkdir
  ntfs: drop redundant S_IFDIR from mkdir

 fs/9p/vfs_inode.c      | 2 +-
 fs/9p/vfs_inode_dotl.c | 1 -
 fs/affs/namei.c        | 2 +-
 fs/afs/dir.c           | 2 +-
 fs/autofs/root.c       | 2 +-
 fs/btrfs/inode.c       | 2 +-
 fs/ceph/dir.c          | 1 -
 fs/coda/dir.c          | 7 ++++++-
 fs/ext2/namei.c        | 2 +-
 fs/ext4/namei.c        | 2 +-
 fs/f2fs/namei.c        | 2 +-
 fs/fuse/dir.c          | 8 ++++++++
 fs/gfs2/inode.c        | 2 +-
 fs/hfs/dir.c           | 2 +-
 fs/hfsplus/dir.c       | 2 +-
 fs/hpfs/namei.c        | 4 ++--
 fs/hugetlbfs/inode.c   | 2 +-
 fs/jffs2/dir.c         | 2 --
 fs/jfs/namei.c         | 2 +-
 fs/minix/namei.c       | 2 +-
 fs/namei.c             | 7 +------
 fs/nfs/dir.c           | 2 +-
 fs/nilfs2/namei.c      | 2 +-
 fs/ntfs/namei.c        | 2 +-
 fs/ntfs3/namei.c       | 2 +-
 fs/ocfs2/dlmfs/dlmfs.c | 2 +-
 fs/ocfs2/namei.c       | 2 +-
 fs/omfs/dir.c          | 2 +-
 fs/orangefs/namei.c    | 2 +-
 fs/ramfs/inode.c       | 2 +-
 fs/smb/client/inode.c  | 7 +++++++
 fs/ubifs/dir.c         | 2 +-
 fs/udf/namei.c         | 2 +-
 fs/ufs/namei.c         | 2 +-
 fs/xfs/xfs_iops.c      | 2 +-
 35 files changed, 51 insertions(+), 40 deletions(-)


base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482
-- 
2.54.0