Re: [PATCH 1/2] arm64: mem-model: add flatmem model for arm64

[Ard Biesheuvel]

On 11 April 2016 at 04:49, Chen Feng <puck.chen@xxxxxxxxxxxxx> wrote:
> Hi will,
> Thanks for review.
>
> On 2016/4/7 22:21, Will Deacon wrote:
>> On Tue, Apr 05, 2016 at 04:22:51PM +0800, Chen Feng wrote:
>>> We can reduce the memory allocated at mem-map
>>> by flatmem.
>>>
>>> currently, the default memory-model in arm64 is
>>> sparse memory. The mem-map array is not freed in
>>> this scene. If the physical address is too long,
>>> it will reserved too much memory for the mem-map
>>> array.
>>
>> Can you elaborate a bit more on this, please? We use the vmemmap, so any
>> spaces between memory banks only burns up virtual space. What exactly is
>> the problem you're seeing that makes you want to use flatmem (which is
>> probably unsuitable for the majority of arm64 machines).
>>
> The root cause we want to use flat-mem is the mam_map alloced in sparse-mem
> is not freed.
>
> take a look at here:
> arm64/mm/init.c
> void __init mem_init(void)
> {
> #ifndef CONFIG_SPARSEMEM_VMEMMAP
>         free_unused_memmap();
> #endif
> }
>
> Memory layout (3GB)
>
>  0             1.5G    2G             3.5G            4G
>  |              |      |               |              |
>  +--------------+------+---------------+--------------+
>  |    MEM       | hole |     MEM       |   IO (regs)  |
>  +--------------+------+---------------+--------------+
>
>
> Memory layout (4GB)
>
>  0                                    3.5G            4G    4.5G
>  |                                     |              |       |
>  +-------------------------------------+--------------+-------+
>  |                   MEM               |   IO (regs)  |  MEM  |
>  +-------------------------------------+--------------+-------+
>
> Currently, the sparse memory section is 1GB.
>
> 3GB ddr: the 1.5 ~2G and 3.5 ~ 4G are holes.
> 3GB ddr: the 3.5 ~ 4G and 4.5 ~ 5G are holes.
>
> This will alloc 1G/4K * (struct page) memory for mem_map array.
>

No, this is incorrect. Sparsemem vmemmap only allocates struct pages
for memory regions that are actually populated.

For instance, on the Foundation model with 4 GB of memory, you may see
something like this in the boot log

[    0.000000]     vmemmap : 0xffffffbdc0000000 - 0xffffffbfc0000000
(     8 GB maximum)
[    0.000000]               0xffffffbdc0000000 - 0xffffffbde2000000
(   544 MB actual)

but in reality, only the following regions have been allocated

---[ vmemmap start ]---
0xffffffbdc0000000-0xffffffbdc2000000          32M       RW NX SHD AF
      BLK UXN MEM/NORMAL
0xffffffbde0000000-0xffffffbde2000000          32M       RW NX SHD AF
      BLK UXN MEM/NORMAL
---[ vmemmap end ]---

so only 64 MB is used to back 4 GB of RAM with struct pages, which is
minimal. Moving to flatmem will not reduce the memory footprint at
all.