Re: [PATCH] arm64: Increase the max granular size

[Geert Uytterhoeven]

On Wed, Oct 28, 2015 at 8:09 PM, Catalin Marinas
<catalin.marinas@xxxxxxx> wrote:
> On Tue, Sep 22, 2015 at 07:59:48PM +0200, Robert Richter wrote:
>> From: Tirumalesh Chalamarla <tchalamarla@xxxxxxxxxx>
>>
>> Increase the standard cacheline size to avoid having locks in the same
>> cacheline.
>>
>> Cavium's ThunderX core implements cache lines of 128 byte size. With
>> current granulare size of 64 bytes (L1_CACHE_SHIFT=6) two locks could
>> share the same cache line leading a performance degradation.
>> Increasing the size fixes that.
>>
>> Increasing the size has no negative impact to cache invalidation on
>> systems with a smaller cache line. There is an impact on memory usage,
>> but that's not too important for arm64 use cases.
>>
>> Signed-off-by: Tirumalesh Chalamarla <tchalamarla@xxxxxxxxxx>
>> Signed-off-by: Robert Richter <rrichter@xxxxxxxxxx>
>
> Applied. Thanks.

This patch causes a BUG() on r8a7795/salvator-x, for which support is not
yet upstream.

My config (attached) uses SLAB. If I switch to SLUB, it works.
The arm64 defconfig works, even if I switch from SLUB to SLAB.

Booting Linux on physical CPU 0x0
Linux version 4.3.0-salvator-x-08136-gce303d3c239f417e-dirty
(geert@ramsan) (gcc version 4.9.0 (GCC) ) #264 SMP Tue Nov 3 11:40:59
CET 2015
Boot CPU: AArch64 Processor [411fd073]
bootconsole [uart0] enabled
debug: ignoring loglevel setting.
efi: Getting EFI parameters from FDT:
efi: UEFI not found.
On node 0 totalpages: 262144
  DMA zone: 3584 pages used for memmap
  DMA zone: 0 pages reserved
  DMA zone: 262144 pages, LIFO batch:31
PERCPU: Embedded 16 pages/cpu @ffffffc03ffc4000 s27136 r8192 d30208 u65536
pcpu-alloc: s27136 r8192 d30208 u65536 alloc=16*4096
pcpu-alloc: [0] 0
Detected PIPT I-cache on CPU0
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 258560
Kernel command line: earlycon ignore_loglevel ip=dhcp root=/dev/nfs rw
nfsroot=192.168.97.21:/home/salvator-x/debian-arm64
PID hash table entries: 4096 (order: 3, 32768 bytes)
Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
software IO TLB [mem 0x7ae00000-0x7ee00000] (64MB) mapped at
[ffffffc03ae00000-ffffffc03edfffff]
Memory: 946232K/1048576K available (4159K kernel code, 395K rwdat
a, 1964K rodata, 244K init, 13383K bss, 102344K reserved, 0K cma-reserved)
Virtual kernel memory layout:
    vmalloc : 0xffffff8000000000 - 0xffffffbdffff0000   (   247 GB)
    vmemmap : 0xffffffbe00000000 - 0xffffffbfc0000000   (     7 GB maximum)
              0xffffffbe00e00000 - 0xffffffbe01c00000   (    14 MB actual)
    fixed   : 0xffffffbffa7fd000 - 0xffffffbffac00000   (  4108 KB)
    PCI I/O : 0xffffffbffae00000 - 0xffffffbffbe00000   (    16 MB)
    modules : 0xffffffbffc000000 - 0xffffffc000000000   (    64 MB)
    memory  : 0xffffffc000000000 - 0xffffffc040000000   (  1024 MB)
      .init : 0xffffffc00067d000 - 0xffffffc0006ba000   (   244 KB)
      .text : 0xffffffc000080000 - 0xffffffc00067cdb4   (  6132 KB)
      .data : 0xffffffc0006c7000 - 0xffffffc000729e00   (   396 KB)
------------[ cut here ]------------
kernel BUG at mm/slab.c:2283!
Internal error: Oops - BUG: 0 [#1] SMP
Modules linked in:
CPU: 0 PID: 0 Comm: swapper Not tainted
4.3.0-salvator-x-08136-gce303d3c239f417e-dirty #264
Hardware name: Renesas Salvator-X board based on r8a7795 (DT)
task: ffffffc0006d22f0 ti: ffffffc0006c8000 task.ti: ffffffc0006c8000
PC is at __kmem_cache_create+0x21c/0x280
LR is at __kmem_cache_create+0x210/0x280
pc : [<ffffffc00014f9b4>] lr : [<ffffffc00014f9a8>] pstate: 800002c5
sp : ffffffc0006cbe50
x29: ffffffc0006cbe50 x28: 0000000000000000
x27: ffffffc000081198 x26: 000000004143f000
x25: 000000004143c000 x24: ffffffc03ffd6200
x23: 0000000080000000 x22: 0000000000000000
x21: 0000000080002000 x20: ffffffc03a800180
x19: 0000000000000020 x18: 000000007b73533d
x17: ffffffc00048c000 x16: 0000000000000000
x15: 0000000080000000 x14: 0000000000000081
x13: 0000000000001000 x12: 0000000000000001
x11: 0000000000000080 x10: ffffffc0006a7d58
x9 : 000000000000007f x8 : ffffffffffffff80
x7 : 0000000000000080 x6 : 0000000000001000
x5 : 0000000000000000 x4 : 0000000000000000
x3 : 0000000000000000 x2 : 0000000000000000
x1 : 0000000000000000 x0 : 0000000000000000

Process swapper (pid: 0, stack limit = 0xffffffc0006c8020)
Stack: (0xffffffc0006cbe50 to 0xffffffc0006cc000)
be40:                                   ffffffc0006cbe90 ffffffc00068be50
be60: ffffffc03a800180 0000000000000080 ffffffc0005e90a4 0000000000002000
be80: ffffffc0013ed450 ffffffc0000d5890 ffffffc0006cbed0 ffffffc00068bed8
bea0: ffffffc03a800180 ffffffc0005e90a4 0000000000000080 ffffffc0005e90a4
bec0: 0000000000000080 0000000000002000 ffffffc0006cbf10 ffffffc00068bfc0
bee0: ffffffc0006ac170 ffffffc0013ed450 0000000000000000 ffffffc00014e8b4
bf00: ffffffc0006cbf30 0000000000002000 ffffffc0006cbf60 ffffffc00068db08
bf20: ffffffc0006a7c60 ffffffc0013ed000 ffffffc0006e3818 ffffffc0013ed450
bf40: ffffffc0013ed000 ffffffc03ffd6200 000000004143c000 ffffffc0013ed450
bf60: ffffffc0006cbfa0 ffffffc00067d7d8 ffffffc0006ce000 ffffffc00072d000
bf80: ffffffc00072d000 ffffffc0006ce000 ffffffc0006a6ed8 ffffffc00067d7d0
bfa0: 0000000000000000 0000000040488000 0000000000000400 0000000000000e11
bfc0: 0000000040079000 0000000000000003 000000007fe31b40 0000000040000000
bfe0: 0000000000000000 ffffffc0006a6ed8 0000000000000000 0000000000000000
Call trace:
[<ffffffc00014f9b4>] __kmem_cache_create+0x21c/0x280
[<ffffffc00068be50>] create_boot_cache+0x4c/0x80
[<ffffffc00068bed8>] create_kmalloc_cache+0x54/0x88
[<ffffffc00068bfc0>] create_kmalloc_caches+0x50/0xf4
[<ffffffc00068db08>] kmem_cache_init+0x104/0x118
[<ffffffc00067d7d8>] start_kernel+0x218/0x33c
[<0000000040488000>] 0x40488000
Code: 97ffa117 f9002280 f100401f 54000048 (d4210000)
---[ end trace cb88537fdc8fa200 ]---

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@xxxxxxxxxxxxxx

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Attachment: config-r8a7795-salvator-x
Description: Binary data