Re: [PATCH v3 09/14] vfs: add O_CREAT|O_DIRECTORY to open*(2)

[Christian Brauner]

On 2026-07-07 09:35:03+01:00, Pedro Falcato wrote:
> Hi,
> 
> Sorry for comenting so late...
> 
> On Sat, Jul 04, 2026 at 06:41:43PM +0200, Jori Koolstra wrote:
> 
> > Currently there is no way to race-freely create and open a directory.
> > For regular files we have open(O_CREAT) for creating a new file inode,
> > and returning a pinning fd to it. The lack of such functionality for
> > directories means that when populating a directory tree there's always
> > a race involved: the inodes first need to be created, and then opened
> > to adjust their permissions/ownership/labels/timestamps/acls/xattrs/...,
> > but in the time window between the creation and the opening they might
> > be replaced by something else.
> > 
> > Addressing this race without proper APIs is possible (by immediately
> > fstat()ing what was opened, to verify that it has the right inode type),
> > but difficult to get right. Hence, adding support for a new flag combo
> > O_CREAT|O_DIRECTORY to open*(2) that creates a directory (if it does not
> > exist already) and returns an O_DIRECTORY fd is very useful.
> > 
> > Historically, the O_CREAT|O_DIRECTORY behaviour was to return ENOTDIR if
> > a regular file exists at the open path; EISDIR if a directory exists at
> > the path; and to create a regular file if no file exists at the path.
> > This behaviour changed accidentally with 973d4b73fbaf ("do_last(): rejoin
> > the common path even earlier in FMODE_{OPENED,CREATED} case") causing
> > ENOTDIR to return in the last case while still creating the file. As
> > this change was not detected for a long time, Brauner proposed to adopt
> > the more consistent NetBSD behaviour, i.e. to return EINVAL on the the
> > O_CREAT|O_DIRECTORY combination. This change was applied in 43b450632676
> > ("open: return EINVAL for O_DIRECTORY | O_CREAT") in March, 2023. As
> > the EINVAL behaviour has been in the kernel for about 3 year now, no
> > rollback is expected as a result of userspace reliance on old
> > behaviour, leaving us free to reassign the O_CREAT|O_DIRECTORY semantics.
> 
> Ok so I genuinely don't see how users are supposed to use O_CREAT | O_DIRECTORY.
> Lets think about the portability here. This is undefined by POSIX (meaning you
> cannot rely on a O_CREAT | O_DIRECTORY having particular semantics). FreeBSD
> and NetBSD each have different semantics (my investigation stopped there
> ~3 years ago; I vaguely remember that macOS/Darwin does not/did not support
> O_DIRECTORY. I don't remember what's the deal with OpenBSD/Solaris/Illumos
> here).
> So, essentially you can't rely on _any_ common behavior between platforms.
> Ok. Now, you may say "it's 2026 lol who cares about BSD". So lets look at Linux:
> - Linux for most of its lifetime had "return open directory, or create
>   a regular file".
> - It later had the above, with the added caveat that it always returned
>   ENOTDIR.
> - In 2023 I then found the bug and Christian changed it to always return
>   EINVAL on that combination.
> 
> Ok. So Linux had 3 behaviors and 2 of them do not make any kind of sense. But
> that's fine, we return EINVAL now. Except that the other behaviors existed.
> 
> So, lets say this feature lands as-is. If you're writing something *for Linux*,
> you have to contend with the previous *3 behaviors* for a grand total of 4
> behaviors. Including bizarre runtime tests, lest you are running on something
> pre-6.4 (system calls can be similarly problematic, but at least there its a
> simple ENOSYS test).
> 
> If you're writing something that wants to be portable, or that intends to
> standardize on something, you have some 5 different behaviors all across
> the FOSS UNIX landscape, not considering everything else. It's also something
> that will, FWIW, probably never be included in POSIX because no one can agree
> on the semantics here.
> 
> TL;DR I don't see, given the reasons above, how users are supposed to use
> this without it being a total minefield.
> 
> If you're intent on the open() path, I would suggest that perhaps openat2()
> would be a nice alternative, although definitely not as beautiful as
> open(O_CREAT | O_DIRECTORY).
> 
> Just my 2c. And I feel horrible typing this out because you clearly spent
> a great amount of time and effort between v2 and v3 as you redid this :/

I don't think any of this is really an argument worth considering.

This is a lot of handwaving for the sake of portability and concerns
about past API mistakes. To both: so what. If an API is useful it's
useful, POSIX be damned. We're not going to be fenced in by some
standard.

90% of our interfaces blow way past POSIX already - in most subsystems.
If we'd back down ever time someone shows up with "what about POSIX"
Linux would be irrelevant.

It's been years since O_DIRECTORY | O_CREAT has been made consistent
which means all LTS kernels have consistent behavior. We often make flag
combinations work that didn't work or where inconsisent before. Nothing
here is special at all.

I appreciate the concern but even if this were to fall flat then
worst-case we'd add a new flag and we'd still make that work. But I see
no reason to not try the semantically obvious thing and make
O_DIRECTORY | O_CREAT work.