Re: [PATCH v6 0/6] mm: introduce memfd_secret system call to create "secret" memory areas

[David Hildenbrand]

On 03.11.20 10:52, Mike Rapoport wrote:
> On Mon, Nov 02, 2020 at 06:51:09PM +0100, David Hildenbrand wrote:
> > > > Assume you have a system with quite some ZONE_MOVABLE memory (esp. in
> > > > virtualized environments), eating up a significant amount of !ZONE_MOVABLE
> > > > memory dynamically at runtime can lead to non-obvious issues. It looks like
> > > > you have plenty of free memory, but the kernel might still OOM when trying
> > > > to do kernel allocations e.g., for pagetables. With CMA we at least know
> > > > what we're dealing with - it behaves like ZONE_MOVABLE except for the owner
> > > > that can place unmovable pages there. We can use it to compute statically
> > > > the amount of ZONE_MOVABLE memory we can have in the system without doing
> > > > harm to the system.
> > >
> > > Why would you say that secretmem allocates from !ZONE_MOVABLE?
> > > If we put boot time reservations aside, the memory allocation for
> > > secretmem follows the same rules as the memory allocations for any file
> > > descriptor. That means we allocate memory with GFP_HIGHUSER_MOVABLE.
> >
> > Oh, okay - I missed that! I had the impression that pages are unmovable and
> > allocating from ZONE_MOVABLE would be a violation of that?
> >
> > > After the allocation the memory indeed becomes unmovable but it's not
> > > like we are eating memory from other zones here.
> >
> > ... and here you have your problem. That's a no-no. We only allow it in very
> > special cases where it can't be avoided - e.g., vfio having to pin guest
> > memory when passing through memory to VMs.
> >
> > Hotplug memory, online it to ZONE_MOVABLE. Allocate secretmem. Try to unplug
> > the memory again -> endless loop in offline_pages().
> >
> > Or have a CMA area that gets used with GFP_HIGHUSER_MOVABLE. Allocate
> > secretmem. The owner of the area tries to allocate memory - always fails.
> > Purpose of CMA destroyed.
> >
> > > > Ideally, we would want to support page migration/compaction and allow for
> > > > allocation from ZONE_MOVABLE as well. Would involve temporarily mapping,
> > > > copying, unmapping. Sounds feasible, but not sure which roadblocks we would
> > > > find on the way.
> > >
> > > We can support migration/compaction with temporary mapping. The first
> > > roadblock I've hit there was that migration allocates 4K destination
> > > page and if we use it in secret map we are back to scrambling the direct
> > > map into 4K pieces. It still sounds feasible but not as trivial :)
> >
> > That sounds like the proper way for me to do it then.
>   
> Although migration of secretmem pages sounds feasible now, there maybe
> other issues I didn't see because I'm not very familiar with
> migration/compaction code.

Migration of PMDs might also be feasible -  and it would be even 
cleaner. But I agree that that might require more work and starting with 
something simpler (!movable) is the right way to move forward.

--
Thanks,

David / dhildenb