Re: [RFC 4/4] Sparse initialization of struct page array.

[Robin Holt]

On Tue, Jul 23, 2013 at 10:32:11AM +0200, Ingo Molnar wrote:
> 
> * H. Peter Anvin <hpa@xxxxxxxxx> wrote:
> 
> > On 07/15/2013 11:26 AM, Robin Holt wrote:
> >
> > > Is there a fairly cheap way to determine definitively that the struct 
> > > page is not initialized?
> > 
> > By definition I would assume no.  The only way I can think of would be 
> > to unmap the memory associated with the struct page in the TLB and 
> > initialize the struct pages at trap time.
> 
> But ... the only fastpath impact I can see of delayed initialization right 
> now is this piece of logic in prep_new_page():
> 
> @@ -903,6 +964,10 @@ static int prep_new_page(struct page *page, int order, gfp_t gfp_flags)
> 
>         for (i = 0; i < (1 << order); i++) {
>                 struct page *p = page + i;
> +
> +               if (PageUninitialized2Mib(p))
> +                       expand_page_initialization(page);
> +
>                 if (unlikely(check_new_page(p)))
>                         return 1;
> 
> That is where I think it can be made zero overhead in the 
> already-initialized case, because page-flags are already used in 
> check_new_page():

The problem I see here is that the page flags we need to check for the
uninitialized flag are in the "other" page for the page aligned at the
2MiB virtual address, not the page currently being referenced.

Let me try a version of the patch where we set the PG_unintialized_2m
flag on all pages, including the aligned pages and see what that does
to performance.

Robin

> 
> static inline int check_new_page(struct page *page)
> {
>         if (unlikely(page_mapcount(page) |
>                 (page->mapping != NULL)  |
>                 (atomic_read(&page->_count) != 0)  |
>                 (page->flags & PAGE_FLAGS_CHECK_AT_PREP) |
>                 (mem_cgroup_bad_page_check(page)))) {
>                 bad_page(page);
>                 return 1;
> 
> see that PAGE_FLAGS_CHECK_AT_PREP flag? That always gets checked for every 
> struct page on allocation.
> 
> We can micro-optimize that low overhead to zero-overhead, by integrating 
> the PageUninitialized2Mib() check into check_new_page(). This can be done 
> by adding PG_uninitialized2mib to PAGE_FLAGS_CHECK_AT_PREP and doing:
> 
> 
> 	if (unlikely(page->flags & PAGE_FLAGS_CHECK_AT_PREP)) {
> 		if (PageUninitialized2Mib(p))
> 			expand_page_initialization(page);
> 		...
> 	}
> 
>         if (unlikely(page_mapcount(page) |
>                 (page->mapping != NULL)  |
>                 (atomic_read(&page->_count) != 0)  |
>                 (mem_cgroup_bad_page_check(page)))) {
>                 bad_page(page);
> 
>                 return 1;
> 
> this will result in making it essentially zero-overhead, the 
> expand_page_initialization() logic is now in a slowpath.
> 
> Am I missing anything here?
> 
> Thanks,
> 
> 	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/