Re: [PATCH v36 21/24] x86/vdso: Implement a vDSO for Intel SGX enclave call

[Andy Lutomirski]

> On Aug 17, 2020, at 6:12 AM, Nathaniel McCallum <npmccallum@xxxxxxxxxx> wrote:
> 
> On Mon, Aug 10, 2020 at 7:09 PM Andy Lutomirski <luto@xxxxxxxxxx> wrote:
>> 
>>> On Thu, Aug 6, 2020 at 7:55 AM Nathaniel McCallum <npmccallum@xxxxxxxxxx> wrote:
>>> 
>>> In a past revision of this patch, I had requested a void *misc
>>> parameter that could be passed through vdso_sgx_enter_enclave_t into
>>> sgx_enclave_exit_handler_t. This request encountered some push back
>>> and I dropped the issue. However, I'd like to revisit it or something
>>> similar.
>> 
>> Why do you need an exit handler at all?  IIRC way back when I
>> suggested that we simply not support it at all.  If you want to
>> call__vdso_sgx_enter_enclave() in a loop, call it in a loop.  If you
>> want to wrap it intelligently in Rust, you don't want a callback
>> anyway -- that forces you have an FFI (or non-Rust, anyway) frame on
>> the stack, which interacts poorly with panic handling and prevents you
>> from using await in your Rust callback handler.  If, on the other
>> hand, you just call __vdso_sg_enter_enclave() in a loop, all these
>> problems go away and, if you really want, you can pass in a callback
>> in Rust and call the callback from Rust.
>> 
>> What am I missing?  I still don't really understand why we are
>> supporting this mechanism at all.  Just the asm code to invoke the
>> callback seems to be about half of the entire function.
> 
> There are three ways to pass state between the enclave and the outside world:
> 1. A pre-allocated memory block at enclave creation time.
> 2. A contract for pushing values onto the stack during entry/exit.
> 3. All registers and flags besides rax, rbx, and rcx.
> 
> Under the current vDSO function:
> 
> #1 is completely possible without a handler. The challenge is how to
> communicate the address of this memory to the enclave. This can be
> accomplished by a parameter in a measured block or by convention.
> Otherwise, it needs to use #2 or #3 to communicate the address of the
> block.
> 
> #2 requires a handler written in assembly. The semantics are well known.

No one seems particularly interested in my suggestion that the RSP exposed to the enclave be different from the actual untrusted stack. Oh well.

That being said, if I were writing a Rust wrapper for SGX (or Python or any language with a reasonable form of async/await), I would want the vDSO code to support swapping RSP before ENCLU because I would want to have first-class support for invoking an enclave from an async function and suspending inside the handler. Allocating a real stack (and shadow stack once CET shows up) for this is disgusting.  If the vDSO supported this natively, it could (in principle) interact with signal delivery such that signals would not see the alternative stack.

I do admit that the implementation would not be pretty. Honestly, I think Intel messed up by exposing USER_RSP to enclaves in the first place.

> 
> #3 is possible without a handler, but only for the subset of the
> registers allowed by the calling convention. However, with a handler
> written in assembly you can pass both in and out the full range of
> registers. To do this, the assembly handler needs a pointer to a
> buffer to save the registers into. How does it get said pointer?
> Currently: offsetof, which Rust doesn't support.

I find this justification a bit silly. Binutils asm (gas) doesn’t support offsetof for C structs either, and Linux works just fine. We’re talking about hardcoding one number along with an assertion somewhere that the number is correct, right?  Couldn’t sizeof be used, too?

To be clear, I think that passing around a misc pointer seems entirely reasonable, but I see it as a nice feature, not as a requirement for correct usage of the function.