Re: [PATCH] MIPS: fix memory reservation for non-usermem setups

[Ilya Lipnitskiy]

Hi Thomas,

On Tue, Apr 6, 2021 at 6:18 AM Thomas Bogendoerfer
<tsbogend@xxxxxxxxxxxxxxxx> wrote:
>
> On Sat, Apr 03, 2021 at 07:02:13PM -0700, Ilya Lipnitskiy wrote:
> > Hi Mike,
> >
> > On Tue, Mar 16, 2021 at 11:33 PM Mike Rapoport <rppt@xxxxxxxxxx> wrote:
> > >
> > > Hi Ilya,
> > >
> > > On Tue, Mar 16, 2021 at 10:10:09PM -0700, Ilya Lipnitskiy wrote:
> > > > Hi Thomas,
> > > >
> > > > On Fri, Mar 12, 2021 at 7:19 AM Thomas Bogendoerfer
> > > > <tsbogend@xxxxxxxxxxxxxxxx> wrote:
> > > > >
> > > > > On Sun, Mar 07, 2021 at 11:40:30AM -0800, Ilya Lipnitskiy wrote:
> > > > > > From: Tobias Wolf <dev-NTEO@xxxxxxxxx>
> > > > > >
> > > > > > Commit 67a3ba25aa95 ("MIPS: Fix incorrect mem=X@Y handling") introduced a new
> > > > > > issue for rt288x where "PHYS_OFFSET" is 0x0 but the calculated "ramstart" is
> > > > > > not. As the prerequisite of custom memory map has been removed, this results
> > > > > > in the full memory range of 0x0 - 0x8000000 to be marked as reserved for this
> > > > > > platform.
> > > > >
> > > > > and where is the problem here ?
> > > > Turns out this was already attempted to be upstreamed - not clear why
> > > > it wasn't merged. Context:
> > > > https://lore.kernel.org/linux-mips/6504517.U6H5IhoIOn@loki/
> > > >
> > > > I hope the thread above helps you understand the problem.
> > >
> > > The memory initialization was a bit different then. Do you still see the
> > > same problem?
> > Thanks for asking. I obtained a RT2880 device and gave it a try. It
> > hangs at boot without this patch, however selecting
>
> can you provide debug logs with memblock=debug for both good and bad
> kernels ? I'm curious what's the reason for failing allocation...
Sorry for taking a while to respond. See attached.
FWIW, it seems these are the lines that stand out in hang.log:
[    0.000000] memblock_reserve: [0x00000000-0x07ffffff] setup_arch+0x214/0x5d8
[    0.000000] Wasting 1048576 bytes for tracking 32768 unused pages
...
[    0.000000]  reserved[0x0]    [0x00000000-0x087137aa], 0x087137ab
bytes flags: 0x0

Ilya
[    0.000000] Linux version 5.12.0-rc2+ (builder@buildhost) (mipsel-openwrt-linux-musl-gcc (OpenWrt GCC 7.5.0 r11114-7145a72d3ce2) 7.5.0, GNU ld (GNU Binutils) 2.31.1) #4 Mon Apr 12 23:41:18 PDT 2021
[    0.000000] SoC Type: Ralink RT2880 id:2 rev:1
[    0.000000] printk: bootconsole [early0] enabled
[    0.000000] CPU0 revision is: 0001906c (MIPS 4KEc)
[    0.000000] MIPS: machine is Belkin F5D8235 v1
[    0.000000] memblock_reserve: [0x085d84a8-0x085d9f5e] setup_arch+0x14c/0x5c0
[    0.000000] memblock_reserve: [0x08000000-0x0871378f] setup_arch+0x220/0x5c0
[    0.000000] Initrd not found or empty - disabling initrd
[    0.000000] memblock_alloc_try_nid: 6839 bytes align=0x40 nid=-1 from=0x00000000 max_addr=0x00000000 early_init_dt_alloc_memory_arch+0x40/0x84
[    0.000000] memblock_reserve: [0x087137c0-0x08715276] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 21180 bytes align=0x4 nid=-1 from=0x00000000 max_addr=0x00000000 early_init_dt_alloc_memory_arch+0x40/0x84
[    0.000000] memblock_reserve: [0x08715278-0x0871a533] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 27 bytes align=0x4 nid=-1 from=0x00000000 max_addr=0x00000000 early_init_dt_alloc_memory_arch+0x40/0x84
[    0.000000] memblock_reserve: [0x08713790-0x087137aa] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_reserve: [0x08526000-0x08525fff] setup_arch+0x390/0x5c0
[    0.000000] memblock_alloc_try_nid: 32 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 setup_arch+0x4ec/0x5c0
[    0.000000] memblock_reserve: [0x0871a540-0x0871a55f] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] Primary instruction cache 16kB, VIPT, 4-way, linesize 16 bytes.
[    0.000000] Primary data cache 16kB, 4-way, VIPT, no aliases, linesize 16 bytes
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000008000000-0x0000000009ffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000008000000-0x0000000009ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000008000000-0x0000000009ffffff]
[    0.000000] memblock_alloc_try_nid: 262144 bytes align=0x10 nid=0 from=0x00000000 max_addr=0x00000000 alloc_node_mem_map.constprop.145+0x6c/0xd0
[    0.000000] memblock_reserve: [0x0871a560-0x0875a55f] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 4 bytes align=0x10 nid=0 from=0x00000000 max_addr=0x00000000 setup_usemap+0x64/0x98
[    0.000000] memblock_reserve: [0x087137b0-0x087137b3] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] MEMBLOCK configuration:
[    0.000000]  memory size = 0x02000000 reserved size = 0x0075b542
[    0.000000]  memory.cnt  = 0x1
[    0.000000]  memory[0x0]	[0x08000000-0x09ffffff], 0x02000000 bytes flags: 0x0
[    0.000000]  reserved.cnt  = 0x6
[    0.000000]  reserved[0x0]	[0x00000000-0x00000fff], 0x00001000 bytes flags: 0x0
[    0.000000]  reserved[0x1]	[0x08000000-0x087137aa], 0x007137ab bytes flags: 0x0
[    0.000000]  reserved[0x2]	[0x087137b0-0x087137b3], 0x00000004 bytes flags: 0x0
[    0.000000]  reserved[0x3]	[0x087137c0-0x08715276], 0x00001ab7 bytes flags: 0x0
[    0.000000]  reserved[0x4]	[0x08715278-0x0871a533], 0x000052bc bytes flags: 0x0
[    0.000000]  reserved[0x5]	[0x0871a540-0x0875a55f], 0x00040020 bytes flags: 0x0
[    0.000000] memblock_alloc_try_nid: 61 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 start_kernel+0xec/0x59c
[    0.000000] memblock_reserve: [0x0875a560-0x0875a59c] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 61 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 start_kernel+0x168/0x59c
[    0.000000] memblock_reserve: [0x0875a5a0-0x0875a5dc] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 4096 bytes align=0x1000 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_alloc_alloc_info+0x84/0xe4
[    0.000000] memblock_reserve: [0x0875b000-0x0875bfff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 32768 bytes align=0x1000 nid=-1 from=0x09000000 max_addr=0x00000000 setup_per_cpu_areas+0x4c/0xa8
[    0.000000] memblock_reserve: [0x09000000-0x09007fff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 4 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_setup_first_chunk+0x19c/0x53c
[    0.000000] memblock_reserve: [0x0875a5e0-0x0875a5e3] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 4 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_setup_first_chunk+0x1dc/0x53c
[    0.000000] memblock_reserve: [0x0875a5f0-0x0875a5f3] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 4 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_setup_first_chunk+0x204/0x53c
[    0.000000] memblock_reserve: [0x0875a600-0x0875a603] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 4 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_setup_first_chunk+0x22c/0x53c
[    0.000000] memblock_reserve: [0x0875a610-0x0875a613] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 120 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_setup_first_chunk+0x43c/0x53c
[    0.000000] memblock_reserve: [0x0875a620-0x0875a697] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 92 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_alloc_first_chunk+0x8c/0x274
[    0.000000] memblock_reserve: [0x0875a6a0-0x0875a6fb] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 1024 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_alloc_first_chunk+0xf0/0x274
[    0.000000] memblock_reserve: [0x0875a700-0x0875aaff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 1028 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_alloc_first_chunk+0x118/0x274
[    0.000000] memblock_reserve: [0x0875ab00-0x0875af03] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 256 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_alloc_first_chunk+0x144/0x274
[    0.000000] memblock_reserve: [0x0875c000-0x0875c0ff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_free: [0x0875b000-0x0875bfff] start_kernel+0x190/0x59c
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 8128
[    0.000000] Kernel command line: console=ttyS0,57600 rootfstype=squashfs,jffs2 memblock=debug
[    0.000000] memblock_alloc_try_nid: 16384 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 alloc_large_system_hash+0x1bc/0x334
[    0.000000] memblock_reserve: [0x0875c100-0x087600ff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] Dentry cache hash table entries: 4096 (order: 2, 16384 bytes, linear)
[    0.000000] memblock_alloc_try_nid: 8192 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 alloc_large_system_hash+0x1bc/0x334
[    0.000000] memblock_reserve: [0x08760100-0x087620ff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] Inode-cache hash table entries: 2048 (order: 1, 8192 bytes, linear)
[    0.000000] memblock_phys_alloc_range: 4096 bytes align=0x2000 from=0x00000000 max_addr=0x00000000 trap_init+0x44/0x608
[    0.000000] memblock_reserve: [0x08764000-0x08764fff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 25168K/32768K available (4302K kernel code, 572K rwdata, 952K rodata, 1204K init, 205K bss, 7600K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=16, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] NR_IRQS: 256
[    0.000000] random: get_random_bytes called from start_kernel+0x3a4/0x59c with crng_init=0
[    0.000000] CPU Clock: 266MHz
[    0.000000] clocksource: MIPS: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 14334453388 ns
[    0.000005] sched_clock: 32 bits at 133MHz, resolution 7ns, wraps every 16106127356ns
[    0.015770] Calibrating delay loop... 265.42 BogoMIPS (lpj=1327104)
---------------------------CONTINUTES-BOOTING-NORMALLY-----------------------
[    0.000000] Linux version 5.12.0-rc2+ (builder@buildhost) (mipsel-openwrt-linux-musl-gcc (OpenWrt GCC 7.5.0 r11114-7145a72d3ce2) 7.5.0, GNU ld (GNU Binutils) 2.31.1) #3 Mon Apr 12 23:36:33 PDT 2021
[    0.000000] SoC Type: Ralink RT2880 id:2 rev:1
[    0.000000] printk: bootconsole [early0] enabled
[    0.000000] CPU0 revision is: 0001906c (MIPS 4KEc)
[    0.000000] MIPS: machine is Belkin F5D8235 v1
[    0.000000] memblock_reserve: [0x085d8368-0x085d9e1e] setup_arch+0x14c/0x5d8
[    0.000000] memblock_reserve: [0x08000000-0x0871378f] setup_arch+0x200/0x5d8
[    0.000000] memblock_reserve: [0x00000000-0x07ffffff] setup_arch+0x214/0x5d8
[    0.000000] Wasting 1048576 bytes for tracking 32768 unused pages
[    0.000000] Initrd not found or empty - disabling initrd
[    0.000000] memblock_alloc_try_nid: 6839 bytes align=0x40 nid=-1 from=0x00000000 max_addr=0x00000000 early_init_dt_alloc_memory_arch+0x40/0x84
[    0.000000] memblock_reserve: [0x087137c0-0x08715276] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 21180 bytes align=0x4 nid=-1 from=0x00000000 max_addr=0x00000000 early_init_dt_alloc_memory_arch+0x40/0x84
[    0.000000] memblock_reserve: [0x08715278-0x0871a533] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 27 bytes align=0x4 nid=-1 from=0x00000000 max_addr=0x00000000 early_init_dt_alloc_memory_arch+0x40/0x84
[    0.000000] memblock_reserve: [0x08713790-0x087137aa] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_reserve: [0x08526000-0x08525fff] setup_arch+0x3ac/0x5d8
[    0.000000] memblock_alloc_try_nid: 32 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 setup_arch+0x504/0x5d8
[    0.000000] memblock_reserve: [0x0871a540-0x0871a55f] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] Primary instruction cache 16kB, VIPT, 4-way, linesize 16 bytes.
[    0.000000] Primary data cache 16kB, 4-way, VIPT, no aliases, linesize 16 bytes
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000008000000-0x0000000009ffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000008000000-0x0000000009ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000008000000-0x0000000009ffffff]
[    0.000000] memblock_alloc_try_nid: 262144 bytes align=0x10 nid=0 from=0x00000000 max_addr=0x00000000 alloc_node_mem_map.constprop.145+0x6c/0xc8
[    0.000000] memblock_reserve: [0x0871a560-0x0875a55f] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 4 bytes align=0x10 nid=0 from=0x00000000 max_addr=0x00000000 setup_usemap+0x64/0x98
[    0.000000] memblock_reserve: [0x087137b0-0x087137b3] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] MEMBLOCK configuration:
[    0.000000]  memory size = 0x02000000 reserved size = 0x0875a542
[    0.000000]  memory.cnt  = 0x1
[    0.000000]  memory[0x0]	[0x08000000-0x09ffffff], 0x02000000 bytes flags: 0x0
[    0.000000]  reserved.cnt  = 0x5
[    0.000000]  reserved[0x0]	[0x00000000-0x087137aa], 0x087137ab bytes flags: 0x0
[    0.000000]  reserved[0x1]	[0x087137b0-0x087137b3], 0x00000004 bytes flags: 0x0
[    0.000000]  reserved[0x2]	[0x087137c0-0x08715276], 0x00001ab7 bytes flags: 0x0
[    0.000000]  reserved[0x3]	[0x08715278-0x0871a533], 0x000052bc bytes flags: 0x0
[    0.000000]  reserved[0x4]	[0x0871a540-0x0875a55f], 0x00040020 bytes flags: 0x0
[    0.000000] memblock_alloc_try_nid: 61 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 start_kernel+0xec/0x584
[    0.000000] memblock_reserve: [0x0875a560-0x0875a59c] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 61 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 start_kernel+0x168/0x584
[    0.000000] memblock_reserve: [0x0875a5a0-0x0875a5dc] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 4096 bytes align=0x1000 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_alloc_alloc_info+0x84/0xe4
[    0.000000] memblock_reserve: [0x0875b000-0x0875bfff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 32768 bytes align=0x1000 nid=-1 from=0x01000000 max_addr=0x00000000 setup_per_cpu_areas+0x38/0x94
[    0.000000] memblock_reserve: [0x0875c000-0x08763fff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 4 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_setup_first_chunk+0x19c/0x53c
[    0.000000] memblock_reserve: [0x0875a5e0-0x0875a5e3] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 4 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_setup_first_chunk+0x1dc/0x53c
[    0.000000] memblock_reserve: [0x0875a5f0-0x0875a5f3] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 4 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_setup_first_chunk+0x204/0x53c
[    0.000000] memblock_reserve: [0x0875a600-0x0875a603] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 4 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_setup_first_chunk+0x22c/0x53c
[    0.000000] memblock_reserve: [0x0875a610-0x0875a613] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 120 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_setup_first_chunk+0x43c/0x53c
[    0.000000] memblock_reserve: [0x0875a620-0x0875a697] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 92 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_alloc_first_chunk+0x8c/0x274
[    0.000000] memblock_reserve: [0x0875a6a0-0x0875a6fb] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 1024 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_alloc_first_chunk+0xf0/0x274
[    0.000000] memblock_reserve: [0x0875a700-0x0875aaff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 1028 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_alloc_first_chunk+0x118/0x274
[    0.000000] memblock_reserve: [0x0875ab00-0x0875af03] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_alloc_try_nid: 256 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 pcpu_alloc_first_chunk+0x144/0x274
[    0.000000] memblock_reserve: [0x08764000-0x087640ff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] memblock_free: [0x0875b000-0x0875bfff] start_kernel+0x190/0x584
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 8128
[    0.000000] Kernel command line: console=ttyS0,57600 rootfstype=squashfs,jffs2 memblock=debug
[    0.000000] memblock_alloc_try_nid: 16384 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 alloc_large_system_hash+0x1bc/0x334
[    0.000000] memblock_reserve: [0x08764100-0x087680ff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] Dentry cache hash table entries: 4096 (order: 2, 16384 bytes, linear)
[    0.000000] memblock_alloc_try_nid: 8192 bytes align=0x10 nid=-1 from=0x00000000 max_addr=0x00000000 alloc_large_system_hash+0x1bc/0x334
[    0.000000] memblock_reserve: [0x08768100-0x0876a0ff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] Inode-cache hash table entries: 2048 (order: 1, 8192 bytes, linear)
[    0.000000] memblock_phys_alloc_range: 4096 bytes align=0x2000 from=0x00000000 max_addr=0x00000000 trap_init+0x44/0x608
[    0.000000] memblock_reserve: [0x0876c000-0x0876cfff] memblock_alloc_range_nid+0xf0/0x184
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 25168K/32768K available (4299K kernel code, 575K rwdata, 952K rodata, 1204K init, 205K bss, 7600K reserved, 0K cma-reserved)
----------------------------------------HANGS-FOREVER-HERE---------------------------------