Re: [RFC PATCH] mm: extend memfd with ability to create "secret" memory areas

[Andy Lutomirski]

On Thu, Jan 30, 2020 at 8:23 AM Mike Rapoport <rppt@xxxxxxxxxx> wrote:
>
> Hi,
>
> This is essentially a resend of my attempt to implement "secret" mappings
> using a file descriptor [1].
>
> I've done a couple of experiments with secret/exclusive/whatever
> memory backed by a file-descriptor using a chardev and memfd_create
> syscall. There is indeed no need for VM_ flag, but there are still places
> that would require special care, e.g vm_normal_page(), madvise(DO_FORK), so
> it won't be completely free of core mm modifications.
>
> Below is a POC that implements extension to memfd_create() that allows
> mapping of a "secret" memory. The "secrecy" mode should be explicitly set
> using ioctl(), for now I've implemented exclusive and uncached mappings.

Hi-

Sorry for the extremely delayed response.

I like the general concept, and I like the exclusive concept.  While
it is certainly annoying for the kernel to manage non-direct-mapped
pages, I think it's the future.  But I have serious concerns about the
uncached part.  Here are some concerns.

If it's done at all, I think it should be MFD_SECRET_X86_UNCACHED.  I
think that uncached memory is outside the scope of things that can
reasonably be considered to be architecture-neutral.  (For example, on
x86, UC and WC have very different semantics, and UC has quite
different properties than WB for things like atomics.  Also, the
performance of UC is interesting at best, and the ways to even
moderately efficiently read from UC memory or write to UC memory are
highly x86-specific.)

I'm a little unconvinced about the security benefits.  As far as I
know, UC memory will not end up in cache by any means (unless
aliased), but it's going to be tough to do much with UC data with
anything resembling reasonable performance without derived values
getting cached.  It's likely entirely impossible to do it reliably
without asm.  But even with plain WB memory, getting it into L1 really
should not be that bad unless major new vulnerabilities are
discovered.  And there are other approaches that could be more
arch-neutral and more performant.  For example, there could be an
option to flush a few cache lines on schedule out.  This way a task
could work on some (exclusive but WB) secret memory and have the cache
lines flushed if anything interrupts it.  Combined with turning SMT
off, this could offer comparable protection with much less overhead.

UC also doesn't seem reliable on x86, sadly.  From asking around,
there are at least a handful of scenarios under which the kernel can
ask the CPU for UC but get WB anyway.  Apparently Xen hypervisors will
do this unless the domain has privileged MMIO access, and ESXi will do
it under some set of common circumstances.  So unless we probe somehow
or have fancy enumeration or administrative configuration, I'm not
sure we can even get predictable behavior if we hand userspace a
supposedly UC mapping.  Giving user code WB when it thinks it has UC
could end badly.

--Andy