[RFC UKL 00/10] Unikernel Linux (UKL)

[Ali Raza]

Unikernel Linux (UKL) is a research project aimed at integrating
application specific optimizations to the Linux kernel. This RFC aims to
introduce this research to the community. Any feedback regarding the idea,
goals, implementation and research is highly appreciated.

Unikernels are specialized operating systems where an application is linked
directly with the kernel and runs in supervisor mode. This allows the
developers to implement application specific optimizations to the kernel,
which can be directly invoked by the application (without going through the
syscall path). An application can control scheduling and resource
management and directly access the hardware. Application and the kernel can
be co-optimized, e.g., through LTO, PGO, etc. All of these optimizations,
and others, provide applications with huge performance benefits over
general purpose operating systems.

Linux is the de-facto operating system of today. Applications depend on its
battle tested code base, large developer community, support for legacy
code, a huge ecosystem of tools and utilities, and a wide range of
compatible hardware and device drivers. Linux also allows some degree of
application specific optimizations through build time config options,
runtime configuration, and recently through eBPF. But still, there is a
need for even more fine-grained application specific optimizations, and
some developers resort to kernel bypass techniques.

Unikernel Linux (UKL) aims to get the best of both worlds by bringing
application specific optimizations to the Linux ecosystem. This way,
unmodified applications can keep getting the benefits of Linux while taking
advantage of the unikernel-style optimizations. Optionally, applications
can be modified to invoke deeper optimizations.

There are two steps to unikernel-izing Linux, i.e., first, equip Linux with
a unikernel model, and second, actually use that model to implement
application specific optimizations. This patch focuses on the first part.
Through this patch, unmodified applications can be built as Linux
unikernels, albeit with only modest performance advantages. Like
unikernels, UKL would allow an application to be statically linked into the
kernel and executed in supervisor mode. However, UKL preserves most of the
invariants and design of Linux, including a separate page-able application
portion of the address space and a pinned kernel portion, the ability to
run multiple processes, and distinct execution modes for application and
kernel code. Kernel execution mode and application execution mode are
different, e.g., the application execution mode allows application threads
to be scheduled, handle signals, etc., which do not apply to kernel
threads. Application built as a Linux unikernel will have its text and data
loaded with the kernel at boot time, while the rest of the address space
would remain unchanged. These applications invoke the system call
functionality through a function call into the kernel system call entry
point instead of through the syscall assembly instruction. UKL would
support a normal userspace so the UKL application can be started, managed,
profiled, etc., using normal command line utilities.

Once Linux has a unikernel model, different application specific
optimizations are possible. We have tried a few, e.g., fast system call
transitions, shared stacks to allow LTO, invoking kernel functions
directly, etc. We have seen huge performance benefits, details of which are
not relevant to this patch and can be found in our paper.
(https://arxiv.org/pdf/2206.00789.pdf)

UKL differs significantly from previous projects, e.g., UML, KML and LKL.
User Mode Linux (UML) is a virtual machine monitor implemented on syscall
interface, a very different goal from UKL. Kernel Mode Linux (KML) allows
applications to run in kernel mode and replaces syscalls with function
calls. While KML stops there, UKL goes further. UKL links applications and
kernel together which allows further optimizations e.g., fast system call
transitions, shared stacks to allow LTO, invoking kernel functions directly
etc. Details can be found in the paper linked above. Linux Kernel Library
(LKL) harvests arch independent code from Linux, takes it to userspace as a
library to be linked with applications. A host needs to provide arch
dependent functionality. This model is very different from UKL. A detailed
discussion of related work is present in the paper linked above.

See samples/ukl for a simple TCP echo server example which can be built as
a normal user space application and also as a UKL application. In the Linux
config options, a path to the compiled and partially linked application
binary can be specified. Kernel built with UKL enabled will search this
location for the binary and link with the kernel. Applications and required
libraries need to be compiled with -mno-red-zone -mcmodel=kernel flags
because kernel mode execution can trample on application red zones and in
order to link with the kernel and be loaded in the high end of the address
space, application should have the correct memory model. Examples of other
applications like Redis, Memcached etc along with glibc and libgcc etc.,
can be found at https://github.com/unikernelLinux/ukl

List of authors and contributors:
=================================

Ali Raza - aliraza@xxxxxx
Thomas Unger - tommyu@xxxxxx
Matthew Boyd - mboydmcse@xxxxxxxxx
Eric Munson - munsoner@xxxxxx
Parul Sohal - psohal@xxxxxx
Ulrich Drepper - drepper@xxxxxxxxxx
Richard W.M. Jones - rjones@xxxxxxxxxx
Daniel Bristot de Oliveira - bristot@xxxxxxxxxx
Larry Woodman - lwoodman@xxxxxxxxxx
Renato Mancuso - rmancuso@xxxxxx
Jonathan Appavoo - jappavoo@xxxxxx
Orran Krieger - okrieg@xxxxxx

Ali Raza (9):
  kbuild: Add sections and symbols to linker script for UKL support
  x86/boot: Load the PT_TLS segment for Unikernel configs
  sched: Add task_struct tracking of kernel or application execution
  x86/entry: Create alternate entry path for system calls
  x86/uaccess: Make access_ok UKL aware
  x86/fault: Skip checking kernel mode access to user address space for
    UKL
  x86/signal: Adjust signal handler register values and return frame
  exec: Make exec path for starting UKL application
  Kconfig: Add config option for enabling and sample for testing UKL

Eric B Munson (1):
  exec: Give userspace a method for starting UKL process

 Documentation/index.rst           |   1 +
 Documentation/ukl/ukl.rst         | 104 +++++++++++++++++++++++
 Kconfig                           |   2 +
 Makefile                          |   4 +
 arch/x86/boot/compressed/misc.c   |   3 +
 arch/x86/entry/entry_64.S         | 133 ++++++++++++++++++++++++++++++
 arch/x86/include/asm/elf.h        |   9 +-
 arch/x86/include/asm/uaccess.h    |   8 ++
 arch/x86/kernel/process.c         |  13 +++
 arch/x86/kernel/process_64.c      |  49 ++++++++---
 arch/x86/kernel/signal.c          |  22 +++--
 arch/x86/kernel/vmlinux.lds.S     |  98 ++++++++++++++++++++++
 arch/x86/mm/fault.c               |   7 +-
 fs/binfmt_elf.c                   |  28 +++++++
 fs/exec.c                         |  75 +++++++++++++----
 include/asm-generic/sections.h    |   4 +
 include/asm-generic/vmlinux.lds.h |  32 ++++++-
 include/linux/sched.h             |  26 ++++++
 kernel/Kconfig.ukl                |  41 +++++++++
 samples/ukl/Makefile              |  16 ++++
 samples/ukl/README                |  17 ++++
 samples/ukl/syscall.S             |  28 +++++++
 samples/ukl/tcp_server.c          |  99 ++++++++++++++++++++++
 scripts/mod/modpost.c             |   4 +
 24 files changed, 785 insertions(+), 38 deletions(-)
 create mode 100644 Documentation/ukl/ukl.rst
 create mode 100644 kernel/Kconfig.ukl
 create mode 100644 samples/ukl/Makefile
 create mode 100644 samples/ukl/README
 create mode 100644 samples/ukl/syscall.S
 create mode 100644 samples/ukl/tcp_server.c


base-commit: 4fe89d07dcc2804c8b562f6c7896a45643d34b2f
-- 
2.21.3