Re: [PATCH V2 16/32] x86/sgx: Support restricting of enclave page permissions

[Jarkko Sakkinen]

On Fri, Mar 11, 2022 at 02:10:24PM +0200, Jarkko Sakkinen wrote:
> On Thu, Mar 10, 2022 at 12:33:20PM -0600, Haitao Huang wrote:
> > Hi Jarkko
> > 
> > I have some trouble understanding the sequences below.
> > 
> > On Thu, 10 Mar 2022 00:10:48 -0600, Jarkko Sakkinen <jarkko@xxxxxxxxxx>
> > wrote:
> > 
> > > On Wed, Feb 23, 2022 at 07:21:50PM +0000, Dhanraj, Vijay wrote:
> > > > Hi All,
> > > > 
> > > > Regarding the recent update of splitting the page permissions change
> > > > request into two IOCTLS (RELAX and RESTRICT), can we combine them into
> > > > one? That is, revert to how it was done in the v1 version?
> > > > 
> > > > Why? Currently in Gramine (a library OS for unmodified applications,
> > > > https://gramineproject.io/) with the new proposed change, one needs to
> > > > store the page permission for each page or range of pages. And for every
> > > > request of `mmap` or `mprotect`, Gramine would have to do a lookup
> > > > of the
> > > > page permissions for the request range and then call the respective
> > > > IOCTL
> > > > either RESTRICT or RELAX. This seems a little overwhelming.
> > > > 
> > > > Request: Instead, can we do `MODPE`,  call `RESTRICT` IOCTL, and then do
> > > > an `EACCEPT` irrespective of RELAX or RESTRICT page permission request?
> > > > With this approach, we can avoid storing  page permissions and simplify
> > > > the implementation.
> > > > 
> > > > I understand RESTRICT IOCTL would do a `MODPR` and trigger `ETRACK`
> > > > flows
> > > > to do TLB shootdowns which might not be needed for RELAX IOCTL but I am
> > > > not sure what will be the performance impact. Is there any data point to
> > > > see the performance impact?
> > > > 
> > > > Thanks,
> > > > -Vijay
> > > 
> > > This should get better in the next versuin. "relax" is gone. And for
> > > dynamic EAUG'd pages only VMA and EPCM permissions matter, i.e.
> > > internal vm_max_prot_bits is set to RWX.
> > > 
> > > I patched the existing series eno
> > > 
> > > For Enarx I'm using the following patterns.
> > > 
> > > Shim mmap() handler:
> > > 1. Ask host for mmap() syscall.
> > > 2. Construct secinfo matching the protection bits.
> > > 3. For each page in the address range: EACCEPTCOPY with a
> > >    zero page.
> > 
> > For EACCEPTCOPY to work, I believe PTE.RW is required for the target page.
> > So this only works for mmap(..., RW) or mmap(...,RWX).
> 
> I use it only with EAUG.
> 
> > So that gives you pages with RW/RWX.
> > 
> > To change permissions of any of those pages from RW/RWX to R/RX , you need
> > call ENCLAVE_RESTRICT_PERMISSIONS ioctl with R or with PROT_NONE. you can't
> > just do EMODPE.
> > 
> > so for RW->R, you either:
> > 
> > 1)EMODPR(EPCM.NONE)
> > 2)EACCEPT(EPCM.NONE)
> > 3)EMODPE(R) -- not sure this would work as spec says EMODPE requires "Read
> > access permitted by enclave"
> > 
> > or:
> > 
> > 1)EMODPR(EPCM.PROT_R)
> > 2)EACCEPT(EPCM.PROT_R)
> 
> I checked from SDM and you're correct.
> 
> Then the appropriate thing is to reset to R.
> 
> > > Shim mprotect() handler:
> > > 1. Ask host for mprotect() syscall.
> > > 2. For each page in the address range: EACCEPT with PROT_NONE
> > >    secinfo and EMODPE with the secinfo having the prot bits.
> > 
> > EACCEPT requires PTE.R. And EAUG'd pages will always initialized with
> > EPCM.RW,
> > so EACCEPT(EPCM.PROT_NONE) will fail with SGX_PAGE_ATTRIBUTES_MISMATCH.
> 
> Ditto.
> 
> > > Backend mprotect() handler:
> > > 1. Invoke ENCLAVE_RESTRICT_PERMISSIONS ioctl for the address
> > >    range with PROT_NONE.
> > > 2. Invoke real mprotect() syscall.
> > > 
> > Note #1 can only be done after EACCEPT. MODPR is not allowed for pending
> > pages.
> 
> Yes, and that's what I'm doing. After that shim does EACCEPT's in a loop.
> 
> Reinette, the ioctl should already check that either R or W is set in
> secinfo and return -EACCES.
> 
> I.e.
> 
> (* Check for misconfigured SECINFO flags*)
> IF ( (SCRATCH_SECINFO reserved fields are not zero ) or
> (SCRATCH_SECINFO.FLAGS.R is 0 and SCRATCH_SECINFO.FLAGS.W is not 0) )
> THEN #GP(0); FI;
> 
> I was testing this and wondering why my enclave #GP's, and then I checked
> SDM after reading Haitao's response. So clearly check in kernel side is
> needed.

I would consider also adding such check "add pages". It's our least common
denominator.

If we can assume that at least R is there for every enclave page, then it
gives invariant that enables EMODPR with R all the time.

BR, Jarkko