Re: [PATCH RFC 3/9] x86/mm/cpa: Add grouped page allocations

[Edgecombe, Rick P]

On Sun, 2021-05-09 at 12:39 +0300, Mike Rapoport wrote:
> On Wed, May 05, 2021 at 09:57:17PM +0000, Edgecombe, Rick P wrote:
> > On Wed, 2021-05-05 at 21:45 +0300, Mike Rapoport wrote:
> > > On Wed, May 05, 2021 at 03:09:12PM +0200, Peter Zijlstra wrote:
> > > > On Wed, May 05, 2021 at 03:08:27PM +0300, Mike Rapoport wrote:
> > > > > On Tue, May 04, 2021 at 05:30:26PM -0700, Rick Edgecombe
> > > > > wrote:
> > > > > > For x86, setting memory permissions on the direct map
> > > > > > results
> > > > > > in fracturing
> > > > > > large pages. Direct map fracturing can be reduced by
> > > > > > locating
> > > > > > pages that
> > > > > > will have their permissions set close together.
> > > > > > 
> > > > > > Create a simple page cache that allocates pages from huge
> > > > > > page
> > > > > > size
> > > > > > blocks. Don't guarantee that a page will come from a huge
> > > > > > page
> > > > > > grouping,
> > > > > > instead fallback to non-grouped pages to fulfill the
> > > > > > allocation
> > > > > > if
> > > > > > needed. Also, register a shrinker such that the system can
> > > > > > ask
> > > > > > for the
> > > > > > pages back if needed. Since this is only needed when there
> > > > > > is a
> > > > > > direct
> > > > > > map, compile it out on highmem systems.
> > > > > 
> > > > > I only had time to skim through the patches, I like the idea
> > > > > of
> > > > > having a
> > > > > simple cache that allocates larger pages with a fallback to
> > > > > basic
> > > > > page
> > > > > size.
> > > > > 
> > > > > I just think it should be more generic and closer to the page
> > > > > allocator.
> > > > > I was thinking about adding a GFP flag that will tell that
> > > > > the
> > > > > allocated
> > > > > pages should be removed from the direct map. Then
> > > > > alloc_pages()
> > > > > could use
> > > > > such cache whenever this GFP flag is specified with a
> > > > > fallback
> > > > > for lower
> > > > > order allocations.
> > > > 
> > > > That doesn't provide enough information I think. Removing from
> > > > direct
> > > > map isn't the only consideration, you also want to group them
> > > > by
> > > > the
> > > > target protection bits such that we don't get to use 4k pages
> > > > quite
> > > > so
> > > > much.
> > > 
> > > Unless I'm missing something we anyway hand out 4k pages from the
> > > cache and
> > > the neighbouring 4k may end up with different protections.
> > > 
> > > This is also similar to what happens in the set Rick posted a
> > > while
> > > ago to
> > > support grouped vmalloc allocations:
> > > 
> > 
> > One issue is with the shrinker callbacks. If you are just trying to
> > reset and free a single page because the system is low on memory,
> > it
> > could be problematic to have to break a large page, which would
> > require
> > another page.
> 
> I don't follow you here. Maybe I've misread the patches but AFAIU the
> large
> page is broken at allocation time and 4k pages remain 4k pages
> afterwards.

Yea that's right.

I thought Peter was saying if the page allocator grouped all of the
same permission together it could often leave the direct map as large
pages, and so the page allocator would have to know about permissions.

So I was just trying to say, to leave large pages on the direct map,
the shrinker has to handle breaking a page while freeing a single page.
So that would have to be addressed to get large pages with permissions
in the first place.

It doesn't seem impossible to solve I guess, so maybe not an important
point. It could maybe just hold a page in reserve.

Now that I think about it, since this PKS tables series holds all
potentially needed direct map page tables in reserve, it shouldn't
actually be a problem for this case. So this could leave the PKS tables
pages as large on the direct map.

> In my understanding the problem with a simple shrinker is that even
> if we
> have the entire 2M free it is not being reinstated as 2M page in the
> direct
> mapping.

Yea, that is a downside to this simple shrinker.