Re: Correlation CMA size and FORCE_MAX_ZONEORDER

From: David Hildenbrand
Date: Mon Sep 19 2022 - 07:03:50 EST


On 19.09.22 11:57, Michael Nazzareno Trimarchi wrote:
Hi

On Mon, Sep 19, 2022 at 11:31 AM David Hildenbrand <david@xxxxxxxxxx> wrote:

On 19.09.22 11:17, Michael Nazzareno Trimarchi wrote:
Hi David

On Mon, Sep 19, 2022 at 10:38 AM David Hildenbrand <david@xxxxxxxxxx> wrote:

On 15.09.22 23:36, Michael Nazzareno Trimarchi wrote:
Hi all

Hi,


Working on a small device with 128MB of memory and using imx_v6_v7
defconfig I found that CMA_SIZE_MBYTES, CMA_SIZE_PERCENTAGE
are not respected. The calculation done does not allow the requested
size. I think that this should be somehow documented and described but
I did not
find the documentation. Does it work this way?

With CMA_SIZE of 8MB I need to have FORCE_MAX_ZONEORDER=12 if I have
the default FORCE_MAX_ZONEORDER=14 the min size is 32Mb

The underlying constraint is that CMA regions require a certain minimum
alignment+size. They cannot be arbitrarily in size.

CMA_MIN_ALIGNMENT_BYTES expresses that, and corresponds in upstream
kernels to the size of a single pageblock.

In previous kernels, it used to be the size of the largest buddy
allocation granularity (derived from MAX_ORDER, derived from
FORCE_MAX_ZONEORDER).

On upstream kernels, the FORCE_MAX_ZONEORDER constraint should no longer
apply. On most archs, the minimum alignment+size should be 2 MiB
(x86-64, aarch64 with 4k base pages) -- the size of a single pageblock.

So far the theory. Are you still running into this limitation on
upstream kernels?


I can run 6-rc2 on my board. I test again but according to it, if I
put 4M as CMA in cma=4M in boot
parameters, the result is 32Mb of CMA. Apart of that seems that
process lime tiny membench can not even start
to mblock memory


The CMA alignemnt change went into v5.19. If "cma=4M" still gives you >
4M, can you post /proc/meminfo and the early console output?


cat /proc/cmdline
cma=4M mtdparts=gpmi-nand:4m(nandboot),1m(env),24m(kernel),1m(nanddtb),-(rootfs)
root=ubi0:root rw ubi.mtd=ro
otfs rootfstype=ubifs rootwait=1
# cat /proc/meminfo
MemTotal: 109560 kB
MemFree: 56084 kB
MemAvailable: 56820 kB
Buffers: 0 kB
Cached: 39680 kB
SwapCached: 0 kB
Active: 44 kB
Inactive: 644 kB
Active(anon): 44 kB
Inactive(anon): 644 kB
Active(file): 0 kB
Inactive(file): 0 kB
Unevictable: 39596 kB
Mlocked: 0 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 109560 kB
LowFree: 56084 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 628 kB
Mapped: 1480 kB
Shmem: 84 kB
KReclaimable: 4268 kB
Slab: 8456 kB
SReclaimable: 4268 kB
SUnreclaim: 4188 kB
KernelStack: 392 kB
PageTables: 88 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 54780 kB
Committed_AS: 1876 kB
VmallocTotal: 901120 kB
VmallocUsed: 2776 kB
VmallocChunk: 0 kB
Percpu: 72 kB
CmaTotal: 32768 kB
CmaFree: 32484 kB
# uname -a
Linux buildroot 6.0.0-rc5 #20 SMP Mon Sep 19 11:51:26 CEST 2022 armv7l GNU/Linux
#

Then here https://pastebin.com/6MUB2VBM dmesg

CONFIG_ARM_MODULE_PLTS=y
CONFIG_FORCE_MAX_ZONEORDER=14
CONFIG_ALIGNMENT_TRAP=y
...
CONFIG_CMA
CONFIG_CMA_AREAS=7
...

CONFIG_CMA_SIZE_MBYTES=8
CONFIG_CMA_SIZE_SEL_MBYTES=y


Thanks!

I assume that in your setup, the pageblock size depends on MAX_ORDER and, therefore, FORCE_MAX_ZONEORDER.

This should be the case especially if CONFIG_HUGETLB_PAGE is not defined (include/linux/pageblock-flags.h).

In contrast to what I remember, the pageblock size does not seem to depend on the THP size (weird) as well.


So, yes, that limitation is still in effect for some kernel configs.

One could make the pageblock size configurable (similar to CONFIG_HUGETLB_PAGE_SIZE_VARIABLE) or simply default to a smaller pageblock size as default with CONFIG_CMA and !CONFIG_HUGETLB_PAGE.

I imagine something reasonable might be to set the pageblock size to 2MiB without CONFIG_HUGETLB_PAGE but with CONFIG_CMA.

--
Thanks,

David / dhildenb