[RFC PATCH v3 1/4] x86/sgx: Add a per-mm ENCLU exception fixup handler

[Sean Christopherson]

Intel Software Guard Extensions (SGX) introduces a new CPL3-only
"enclave" mode that runs as a sort of black box shared object that is
hosted by an untrusted "normal" CPl3 process.

Entering an enclave can only be done through SGX-specific instructions,
EENTER and ERESUME, and is a non-trivial process.  Because of the
complexity of transitioning to/from an enclave, the vast majority of
enclaves are expected to utilize a library to handle the actual
transitions.  This is roughly analogous to how e.g. a libc and dynamic
linker are used by most applications.

Another crucial characteristic of SGX enclaves is that they can generate
exceptions as part of their normal (at least as "normal" as SGX can be)
operation that need to be handled *in* the enclave and/or are unique
to SGX.

And because they are essentially fancy shared objects, a process can
host any number of enclaves, each of which can execute multiple threads
simultaneously.

Putting everything together, userspace enclaves will utilize a library
that must be prepared to handle any and (almost) all exceptions any time
at least one thread may be executing in an enclave.  Leveraging signals
to handle the enclave exceptions is unpleasant, to put it mildly, e.g.
the SGX library must constantly (un)register its signal handler based
on whether or not at least one thread is executing in an enclave, and
filter and forward exceptions that aren't related to its enclaves.  This
becomes particularly nasty when using multiple levels of libraries that
register signal handlers, e.g. running an enclave via cgo inside of the
Go runtime.

Add per-mm, i.e. per-process, exception fixup on ENCLU to so that the
kernel can redirect unhandled exceptions, i.e. exceptions would otherwise
generate a signal, to a user-provided exception handler.  The exception
handler ABI roughly follows the System V 64-bit ABI for function calls:

  - %rdi: trap number
  - %rsi: error code
  - %rdx: address

Cc: Andy Lutomirski <luto@xxxxxxxxxxxxxx>
Cc: Jarkko Sakkinen <jarkko.sakkinen@xxxxxxxxxxxxxxx>
Cc: Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx>
Cc: Josh Triplett <josh@xxxxxxxxxxxxxxxx>
Signed-off-by: Sean Christopherson <sean.j.christopherson@xxxxxxxxx>
---
 arch/x86/include/asm/mmu.h     |  4 ++++
 arch/x86/include/asm/sgx.h     | 13 +++++++++++++
 arch/x86/kernel/cpu/sgx/main.c | 18 ++++++++++++++++++
 3 files changed, 35 insertions(+)

diff --git a/arch/x86/include/asm/mmu.h b/arch/x86/include/asm/mmu.h
index 5ff3e8af2c20..1665c84e5844 100644
--- a/arch/x86/include/asm/mmu.h
+++ b/arch/x86/include/asm/mmu.h
@@ -54,6 +54,10 @@ typedef struct {
 	/* address of the bounds directory */
 	void __user *bd_addr;
 #endif
+#ifdef CONFIG_INTEL_SGX_CORE
+	unsigned long enclu_address;
+	unsigned long enclu_exception_handler;
+#endif
 } mm_context_t;
 
 #define INIT_MM_CONTEXT(mm)						\
diff --git a/arch/x86/include/asm/sgx.h b/arch/x86/include/asm/sgx.h
index d4f61d1c5c2a..bbf808a0ca91 100644
--- a/arch/x86/include/asm/sgx.h
+++ b/arch/x86/include/asm/sgx.h
@@ -309,6 +309,19 @@ static inline int __emodt(struct sgx_secinfo *secinfo, void *addr)
 	return __encls_ret_2(SGX_EMODT, secinfo, addr);
 }
 
+#ifdef CONFIG_INTEL_SGX_CORE
+extern bool fixup_sgx_enclu_exception(struct pt_regs *regs, int trapnr,
+				      unsigned long error_code,
+				      unsigned long fault_addr);
+#else
+static inline bool fixup_sgx_enclu_exception(struct pt_regs *regs, int trapnr,
+					     unsigned long error_code,
+					     unsigned long fault_addr)
+{
+	return false;
+}
+#endif
+
 struct sgx_epc_page *sgx_alloc_page(void *owner, bool reclaim);
 int __sgx_free_page(struct sgx_epc_page *page);
 void sgx_free_page(struct sgx_epc_page *page);
diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index 30fd69f1fc07..1994e003581d 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -363,6 +363,24 @@ void sgx_page_reclaimable(struct sgx_epc_page *page)
 }
 EXPORT_SYMBOL_GPL(sgx_page_reclaimable);
 
+bool fixup_sgx_enclu_exception(struct pt_regs *regs, int trapnr,
+			       unsigned long error_code,
+			       unsigned long fault_addr)
+{
+	if (current->mm->context.enclu_address != regs->ip)
+		return false;
+
+	if (!current->mm->context.enclu_address &&
+	    !current->mm->context.enclu_exception_handler)
+		return false;
+
+	regs->ip = current->mm->context.enclu_exception_handler;
+	regs->di = trapnr;
+	regs->si = error_code;
+	regs->dx = fault_addr;
+	return true;
+}
+
 static __init void sgx_free_epc_section(struct sgx_epc_section *section)
 {
 	struct sgx_epc_page *page;
-- 
2.19.2