Re: [PATCH v3 09/14] vfs: add O_CREAT|O_DIRECTORY to open*(2)

[Jori Koolstra]

Hi Pedro,

> Op 07-07-2026 10:35 CEST schreef Pedro Falcato <pfalcato@xxxxxxx>:
> 
> Ok so I genuinely don't see how users are supposed to use O_CREAT | O_DIRECTORY.
> Lets think about the portability here. This is undefined by POSIX (meaning you
> cannot rely on a O_CREAT | O_DIRECTORY having particular semantics). FreeBSD
> and NetBSD each have different semantics (my investigation stopped there
> ~3 years ago; I vaguely remember that macOS/Darwin does not/did not support
> O_DIRECTORY. I don't remember what's the deal with OpenBSD/Solaris/Illumos
> here).
> 
> So, essentially you can't rely on _any_ common behavior between platforms.
> Ok. Now, you may say "it's 2026 lol who cares about BSD". So lets look at Linux:
> - Linux for most of its lifetime had "return open directory, or create
>   a regular file".
> - It later had the above, with the added caveat that it always returned
>   ENOTDIR.
> - In 2023 I then found the bug and Christian changed it to always return
>   EINVAL on that combination.
> 
> Ok. So Linux had 3 behaviors and 2 of them do not make any kind of sense. But
> that's fine, we return EINVAL now. Except that the other behaviors existed.
> 
> So, lets say this feature lands as-is. If you're writing something *for Linux*,
> you have to contend with the previous *3 behaviors* for a grand total of 4
> behaviors. Including bizarre runtime tests, lest you are running on something
> pre-6.4 (system calls can be similarly problematic, but at least there its a
> simple ENOSYS test).
> 

But isn't this a problem with all UAPI design? You introduce a new feature that
does not exist on old kernels, then userspace can use that feature but must
provide fallback behaviors for any older kernel that the userspace program wants
to support. Ideally, this is abstracted away by some library. Then, at some point
this library raises its minimal supported kernel version bar, and it can clean-up
the now obsolete fallback code.

I am fairly positive for instance that libpathrs would like to use O_CREAT | O_DIRECTORY,
but it is not going to demand kernel > 7.x anytime soon.


> If you're writing something that wants to be portable, or that intends to
> standardize on something, you have some 5 different behaviors all across
> the FOSS UNIX landscape, not considering everything else. It's also something
> that will, FWIW, probably never be included in POSIX because no one can agree
> on the semantics here.
> 

I have no idea how POSIX standardizes things. Does a committee get together and
revisit old syscalls every now and then?

But yeah, I think POSIX compliance isn't something we are really doing anymore.
Afaik there's nothing in POSIX about openat2(2) either, and we definitely wish
people would move to that instead of keep relying on open(2) (or even better,
use libpathrs!)

> TL;DR I don't see, given the reasons above, how users are supposed to use
> this without it being a total minefield.
> 
> If you're intent on the open() path, I would suggest that perhaps openat2()
> would be a nice alternative, although definitely not as beautiful as
> open(O_CREAT | O_DIRECTORY).
> 

Christian told me (somewhere on this list) that at LSFMMBPF all vfs maintainers
agreed on the O_CREAT | O_DIRECTORY direction. It allows to use all the neat
options from openat2(2) at the cost of a bit of multiplexing in lookup_open().
The question remains whether we also want the other things like mknodat_fd().

> Just my 2c. And I feel horrible typing this out because you clearly spent
> a great amount of time and effort between v2 and v3 as you redid this :/
> 

It's been a lot of work, yes :)

But luckily it has allowed me to study path resolution and the open()/mkdir()/etc.
syscalls in a fun and educative way.

> -- 
> Pedro

Best,
Jori.