Re: [PATCH 2/2] binfmt_misc: enable sandboxed mounts

[Christian Brauner]

On Thu, Nov 04, 2021 at 11:30:00PM -0500, Serge Hallyn wrote:
> On Thu, Oct 28, 2021 at 12:31:14PM +0200, Christian Brauner wrote:
> > From: Laurent Vivier <laurent@xxxxxxxxx>
> > 
> > Enable unprivileged sandboxes to create their own binfmt_misc mounts.
> > This is based on Laurent's work in [1] but has been significantly
> > reworked to fix various issues we identified in earlier versions.
> > 
> > While binfmt_misc can currently only be mounted in the initial user
> > namespace, binary types registered in this binfmt_misc instance are
> > available to all sandboxes (Either by having them installed in the
> > sandbox or by registering the binary type with the F flag causing the
> > interpreter to be opened right away). So binfmt_misc binary types are
> > already delegated to sandboxes implicitly.
> > 
> > However, while a sandbox has access to all registered binary types in
> > binfmt_misc a sandbox cannot currently register its own binary types
> > in binfmt_misc. This has prevented various use-cases some of which were
> > already outlined in [1] but we have a range of issues associated with
> > this (cf. [3]-[5] below which are just a small sample).
> > 
> > Extend binfmt_misc to be mountable in non-initial user namespaces.
> > Similar to other filesystem such as nfsd, mqueue, and sunrpc we use
> > keyed superblock management. The key determines whether we need to
> > create a new superblock or can reuse an already existing one. We use the
> > user namespace of the mount as key. This means a new binfmt_misc
> > superblock is created once per user namespace creation. Subsequent
> > mounts of binfmt_misc in the same user namespace will mount the same
> > binfmt_misc instance. We explicitly do not create a new binfmt_misc
> > superblock on every binfmt_misc mount as the semantics for
> > load_misc_binary() line up with the keying model. This also allows us to
> > retrieve the relevant binfmt_misc instance based on the caller's user
> > namespace which can be done in a simple (bounded to 32 levels) loop.
> > 
> > Similar to the current binfmt_misc semantics allowing access to the
> > binary types in the initial binfmt_misc instance we do allow sandboxes
> > access to their parent's binfmt_misc mounts if they do not have created
> > a separate binfmt_misc instance.
> > 
> > Overall, this will unblock the use-cases mentioned below and in general
> > will also allow to support and harden execution of another
> > architecture's binaries in tight sandboxes. For instance, using the
> > unshare binary it possible to start a chroot of another architecture and
> > configure the binfmt_misc interpreter without being root to run the
> > binaries in this chroot and without requiring the host to modify its
> > binary type handlers.
> > 
> > Henning had already posted a few experiments in the cover letter at [1].
> > But here's an additional example where an unprivileged container
> > registers qemu-user-static binary handlers for various binary types in
> > its separate binfmt_misc mount and is then seamlessly able to start
> > containers with a different architecture without affecting the host:
> > 
> > [lxc monitor] /var/lib/lxc imp2
> >  \_ /sbin/init
> >      \_ /lib/systemd/systemd-journald
> >      \_ /lib/systemd/systemd-udevd
> >      \_ /lib/systemd/systemd-networkd
> >      \_ /usr/sbin/cron -f -P
> >      \_ @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
> >      \_ /usr/bin/python3 /usr/bin/networkd-dispatcher --run-startup-triggers
> >      \_ /usr/sbin/rsyslogd -n -iNONE
> >      \_ /lib/systemd/systemd-logind
> >      \_ /lib/systemd/systemd-resolved
> >      \_ dnsmasq --conf-file=/dev/null -u lxc-dnsmasq --strict-order --bind-interfaces --pid-file=/run/lxc/dnsmasq.pid --liste
> >      \_ /sbin/agetty -o -p -- \u --noclear --keep-baud console 115200,38400,9600 vt220
> >      \_ /sbin/agetty -o -p -- \u --noclear --keep-baud pts/0 115200,38400,9600 vt220
> >      \_ /sbin/agetty -o -p -- \u --noclear --keep-baud pts/1 115200,38400,9600 vt220
> >      \_ /sbin/agetty -o -p -- \u --noclear --keep-baud pts/2 115200,38400,9600 vt220
> >      \_ /sbin/agetty -o -p -- \u --noclear --keep-baud pts/3 115200,38400,9600 vt220
> >      \_ [lxc monitor] /var/lib/lxc alp1
> >          \_ /usr/libexec/qemu-binfmt/ppc64le-binfmt-P /sbin/init /sbin/init
> >              \_ /usr/libexec/qemu-binfmt/ppc64le-binfmt-P /lib/systemd/systemd-journald /lib/systemd/systemd-journald
> >              \_ /usr/libexec/qemu-binfmt/ppc64le-binfmt-P /lib/systemd/systemd-udevd /lib/systemd/systemd-udevd
> >              \_ /usr/libexec/qemu-binfmt/ppc64le-binfmt-P /usr/sbin/cron /usr/sbin/cron -f -P
> >              \_ /usr/libexec/qemu-binfmt/ppc64le-binfmt-P /lib/systemd/systemd-resolved /lib/systemd/systemd-resolved
> >              \_ /usr/libexec/qemu-binfmt/ppc64le-binfmt-P /lib/systemd/systemd-logind /lib/systemd/systemd-logind
> > 
> > Link: https://lore.kernel.org/r/20191216091220.465626-2-laurent@xxxxxxxxx
> > [1]: https://lore.kernel.org/all/20191216091220.465626-1-laurent@xxxxxxxxx
> > [2]: https://discuss.linuxcontainers.org/t/binfmt-misc-permission-denied
> > [3]: https://discuss.linuxcontainers.org/t/lxd-binfmt-support-for-qemu-static-interpreters
> > [4]: https://discuss.linuxcontainers.org/t/3-1-0-binfmt-support-service-in-unprivileged-guest-requires-write-access-on-hosts-proc-sys-fs-binfmt-misc
> > [5]: https://discuss.linuxcontainers.org/t/qemu-user-static-not-working-4-11
> > Cc: Sargun Dhillon <sargun@xxxxxxxxx>
> > Cc: Serge Hallyn <serge@xxxxxxxxxx>
> 
> I *think* this is ok.  I'm still trying to convince myself that there is
> no way for evict_inode() to run after the kfree(ns->binfmt_misc), but
> it doesn't look like there is.
> 
> Does this memory (as the number of register entries grows) need to be
> accounted for and/or limited ?

Good point. We should pass GFP_KERNEL_ACCOUNT andor use a cache with
SLAB_ACCOUNT. I'll fix that up.

Christian