RFC : Hyplets - User Space Interrupts for ARM

[raz]

We would like to present ARM hyplets. Hyplets are a novel way to code 
interrupt service routines under ARM.
Hyplets provide high performance, security, running time predictability 
,an RPC mechanism and a possible solution for
the priority inversion problem. Hyplets uses special features of ARM 
hypervisor memory architecture.
We demonstrate Hyplets implementation in the C programming language on 
ARMv8 platform and under the Linux kernel.
We provide performance measurements, use cases and security scenarios.

Hyplets are based on the concept of a delicate separation within a 
running process. Instead of running multiple operating
systems kernels in order to segment and divide the system resources, the 
hyplet divides the Linux process into two execution
modes. One part of the process would execute in an isolated, 
non-interrupted privileged safe execution environment while other
parts of the process would execute in a regular user mode. However, both 
execution modes run in the same Application processors.

Background
ARMv8 has unique approach to privilege level that is crucial to the 
implementation of hyplets. The ARM platform normally has 4 exceptions 
(permission) levels:
EL0 refers to user space code. This is analogues to âring 3â in x86 
platform.
EL1 refers to operating system code. This is analogues to âring 0â in 
x86Â platform.
EL2 refers to HYP mode. This is analogues to âring -1â or âreal modeâ on 
the x86 platform.
EL3 refers to TrustZone as a special security mode that can monitor the 
ARM processor and may run a security real time OS. There is no
direct analogues modes but related concepts in x86 are intelâs ME or SMM.

Each of the exception levels provides its own state of registers and can 
access the registers of the lower levels but not higher levels.
This architecture dictates that the translation tables of the different 
exception levels are distinct.
This means, that EL2 (hypervisor mode) may point to any memory 
translation table while the generic operating system,
running in EL1 uses another translation table. This way, we can map an 
executing process (program) or part of it to the hypervisor,
and we automatically gain access to the program address space, without 
any need to perform context switches or relocations.

The trivial approach of migrating code to kernel space to improve 
performance requires a high level of kernel programming skills.
Alternatively, if we try to access a user space code from the ISR, it 
would probably require a context
switch because there is no guarantee that the current process in the 
processor is the one with hyplet code. We needed to make sure
that the hyplet code is always accessible and this could be achieved 
only by disabling evacuation of the hyplet
code and data from the current translation table. So, we we chose to use 
TTBR0_EL2 ( the translation table register) to constantly hold the 
hyplet code. In order to map user space tasks we modified the Linux 
ARM-KVM mappings infrastructure to map a user space code in tandem with 
some kernel space data.

Restrictions
It is not possible to access any memory which is not mapped prematurely 
to the hypervisor. Also, it is not possible to perform
any system calls.

Some Figures
All benchmarks were conducted on a RASPBERRY PI 3.

Interrupt lantency
Hardware Interrupt Latency -> user hyplet: 2.5us
hyplet timer: Avg: <1us

A nother variant of the hyplet are hyplets RPCs (or in short HypRPC) are 
Remote Procedure calls that are executed
in hyplet context.

Scope of code change

The hyplet patch does not intervene with the hardcore of the kernel code 
nor require any modification to any hardware. The
modification are in the generic ISR routine, in sys_exit and introduces 
a new system called sys_hyplet. For this reason it easy to apply it as 
it does not change the operating system heuristics.

Evaluate

The Kernel code and the full paper are available at github at: 
https://git.scipio.org/raziebe/pi3-public.git

Installtation instructions:
1. Get a raspberry PI3.
2. Download & Install 
openSUSE-Leap42.2-ARM-E20-raspberrypi3.aarch64-2017.02.02-Build1.165.raw. 
This a 64 bit distro.
ÂÂ see 
http://ftp.yz.yamagata-u.ac.jp/pub/Linux/openSUSE/ports/aarch64/distribution/leap/42.2/appliances/

3. compile the kernel (aarch64) and install it on the PI.
4. compile the examples in arch/arm64/kvm/hyplet/tests/
ÂÂÂ see the compile shell script.
ÂÂÂ Start the "simple_hyplet" example. Use it as follows:
ÂÂÂ check /proc/interrupts for an irq that generates enough interrupts.
ÂÂÂ run: taskset <cpu id> simple_hyplet <irq number>
ÂÂÂÂÂÂÂÂ wait for 20 seconds. if some_global output a number bigger 
than zero then it worked.
ÂÂÂ ÂA typical output on the serial console looks like this:

linux:/opt/hyplet # ./simple_hyplet 2
Waiting [Â 284.158547] hyplet [3]:Â address 7f7e472000 already mapped
for irq [Â 284.158593] hyplet [3]:Â address 400820 already mapped
2 for 20 seconds
some global 75