Re: Struct page proposal

[Kent Overstreet]

On Mon, Sep 27, 2021 at 07:12:19PM +0100, Matthew Wilcox wrote:
> On Mon, Sep 27, 2021 at 02:09:49PM -0400, Kent Overstreet wrote:
> > On Mon, Sep 27, 2021 at 07:05:26PM +0100, Matthew Wilcox wrote:
> > > On Mon, Sep 27, 2021 at 07:48:15PM +0200, Vlastimil Babka wrote:
> > > > On 9/23/21 03:21, Kent Overstreet wrote:
> > > > > So if we have this:
> > > > > 
> > > > > struct page {
> > > > > 	unsigned long	allocator;
> > > > > 	unsigned long	allocatee;
> > > > > };
> > > > > 
> > > > > The allocator field would be used for either a pointer to slab/slub's state, if
> > > > > it's a slab page, or if it's a buddy allocator page it'd encode the order of the
> > > > > allocation - like compound order today, and probably whether or not the
> > > > > (compound group of) pages is free.
> > > > 
> > > > The "free page in buddy allocator" case will be interesting to implement.
> > > > What the buddy allocator uses today is:
> > > > 
> > > > - PageBuddy - determine if page is free; a page_type (part of mapcount
> > > > field) today, could be a bit in "allocator" field that would have to be 0 in
> > > > all other "page is allocated" contexts.
> > > > - nid/zid - to prevent merging accross node/zone boundaries, now part of
> > > > page flags
> > > > - buddy order
> > > > - a list_head (reusing the "lru") to hold the struct page on the appropriate
> > > > free list, which has to be double-linked so page can be taken from the
> > > > middle of the list instantly
> > > > 
> > > > Won't be easy to cram all that into two unsigned long's, or even a single
> > > > one. We should avoid storing anything in the free page itself. Allocating
> > > > some external structures to track free pages is going to have funny
> > > > bootstrap problems. Probably a major redesign would be needed...
> > > 
> > > Wait, why do we want to avoid using the memory that we're allocating?
> > 
> > The issue is where to stick the state for free pages. If that doesn't fit in two
> > ulongs, then we'd need a separate allocation, which means slab needs to be up
> > and running before free pages are initialized.
> 
> But the thing we're allocating is at least PAGE_SIZE bytes in size.
> Why is "We should avoid storing anything in the free page itself" true?

Good point!

Highmem and dax do complicate things though - would they make it too much of a
hassle? You want to get rid of struct page for dax (what's the right term for
that kind of memory?), but we're not there yet, right?

Very curious why we need to be able to pull pages off the middle of a freelist.
If we can make do with singly linked freelists, then I think two ulongs would be
sufficient.