A logical error in arch/x86/mm/init.c

[Nikita Popov]

Hello!

It appears that there is a logical error in arch/x86/mm/init.c in the
master branch. Although it is unlikely to occur in practice. I
discovered it while studying source code in that file.

Here is the description:
1) The function 'init_range_memory_mapping' is responsible for direct
mapping of the normal memory range. Before proceeding to mapping of a
valid subrange containing usable RAM the function checks that it is
safe to allocate pages from PGT_BUF subarea of BRK area (the 'safe'
means we we are not mapping the page belonging to PGT_BUF subarea
which can incur MMU issues if that page is allocated as a page
directory). As a result the global flag 'can_use_brk_pgt' reflects
whether the current subrange overlaps with the free area from PGT_BUF.
The 'true' value means no overlapping.
2) The function 'alloc_low_pages' is used during page directories
construction. If it happens that we exhausted PGT_BUF completely (such
condition usually causes can_use_brk_pgt == true), there is a chance
that we use common BRK area on memblock allocation failure (memblock
allocator is restricted to use already mapped pages only). If the
'can_use_brk_pgt' flag is set we allocate a page from the common BRK
area. But the BRK area is not protected with the similar check meaning
that in theory we can allocate a page serving as page directory for
the very same page. The flag 'can_use_brk_pgt' protects only the
PGT_BUF subarea of the BRK area. Not the whole BRK area.

In my opinion one of the simplest fixes here is to completely remove
the following lines:
    if (!ret && can_use_brk_pgt)
            ret = __pa(extend_brk(PAGE_SIZE * num, PAGE_SIZE));

I look forward to seeing your comments on this.
Kind regards.

Nikita Popov
Senior C Developer