Sealed memfd & no-fault mmap

[Simon Ser]

Hi all,

I'm a user-space developer working on Wayland compositors.

Back in 2014, David Herrmann has posted a patchset [1] to introduce memfd
and file sealing. The patchset reads:

>   1) Graphics Compositors
>      If a graphics client creates a memory-backed render-buffer and passes a
>      file-decsriptor to it to the graphics server for display, the server
>      _has_ to setup SIGBUS handlers whenever mapping the given file. Otherwise,
>      the client might run ftruncate() or O_TRUNC on the on file in parallel,
>      thus crashing the server.
>      With sealing, a compositor can reject any incoming file-descriptor that
>      does _not_ have SEAL_SHRINK set. This way, any memory-mappings are
>      guaranteed to stay accessible. Furthermore, we still allow clients to
>      increase the buffer-size in case they want to resize the render-buffer for
>      the next frame. We also allow parallel writes so the client can render new
>      frames into the same buffer (client is responsible of never rendering into
>      a front-buffer if you want to avoid artifacts).
>
>      Real use-case: Wayland wl_shm buffers can be transparently converted

Fast-forward to 7 years later, and notice that there doesn't exist a
single Wayland compositor that enforces file sealing for its clients.

The reason is that there will always exist clients which are either old
(and predate file sealing) or refuse to use Linux-only APIs (they don't
use memfd and file sealing, instead they use e.g. shm_open). Requiring
sealed memfds in compositors would break these clients.

I don't believe the situation is about to change.

Rather than requiring changes in all compositors *and* clients, can we
maybe only require changes in compositors? For instance, OpenBSD has a
__MAP_NOFAULT flag. When passed to mmap, it means that out-of-bound
accesses will read as zeroes instead of triggering SIGBUS. Such a flag
would be very helpful to unblock the annoying SIGBUS situation.

Would something among these lines be welcome in the Linux kernel?

Thanks,

Simon

[1]: https://lwn.net/Articles/591108/