Re: [PATCH 2/2] x86: sgx_vepc: implement SGX_IOC_VEPC_REMOVE_ALL ioctl

[Dave Hansen]

On 9/20/21 3:17 PM, Kai Huang wrote:
>> +The ioctl will often not able to remove SECS pages, in case their child
>> +pages have not gone through ``EREMOVE`` yet; therefore, the ioctl returns the
>> +number of pages that failed to be removed.  ``SGX_IOC_VEPC_REMOVE_ALL`` should
>> +first be invoked on all the ``/dev/sgx_vepc`` file descriptors mapped
>> +into the guest; a second call to the ioctl will be able to remove all
>> +leftover pages and will return 0.  Any other return value on the second call
>> +would be a symptom of a bug in either Linux or the userspace client.
> Maybe also worth to mention userspace should guarantee there's no vcpu running
> inside guest enclave when resetting guest's virtual EPC.

Why, specifically?

Is it because EREMOVE will also fail if there is a CPU running in the
enclave?

Maybe we should say something like:

The ioctl() returns the number of pages where removal failed.  Callers
of the ioctl() need to handle two sources of failure:

1) Page removal will always fail when any thread is running in the
   enclave to which the page belongs.  ioctl() users can avoid these
   failures by preventing execution of any vcpu which maps the virtual
   EPC.

2) Page removal will also fail for SGX "SECS" metadata pages which still
   have child pages.  SECS pages can be removed by executing
   ``SGX_IOC_VEPC_REMOVE_ALL`` on all ``/dev/sgx_vepc`` file descriptors
   mapped into the guest.  Yhis means that the ioctl() must be called
   twice: an initial set of calls to remove child pages and a subsequent
   set of calls to remove SECS pages.

   It indicates a bug in the kernel or the userspace client if any of
   the second round of ``SGX_IOC_VEPC_REMOVE_ALL`` calls fails (has a
   return code other than 0).