Re: Widespread crashes in -next, bisected to 'mm: drop HASH_ADAPT'

[Pasha Tatashin]

The problem is due to 32-bit integer overflow in:

ADAPT_SCALE_BASE and adapt

In dcache_init_early() that is causing the problem. It was not enabled 
before 'mm: drop HASH_ADAPT' but is enabled now, and it should follow 
right after: "PID hash table entries: 1024 (order: 0, 4096 bytes)"

main()
  pidhash_init();
  vfs_caches_init_early();
    dcache_init_early()
      alloc_large_system_hash("Dentry cache", ...)

for (adapt = ADAPT_SCALE_NPAGES; adapt < numentries;
			     adapt <<= ADAPT_SCALE_SHIFT)

numentries  is very small, so it should be always smaller than adapt, 
and algorithm should not kick in, but 32-bit causes adapt to be smaller 
than numentries.

I will send out an updated "mm: Adaptive hash table scaling", with "mm: 
drop HASH_ADAPT" integrated.

Pasha

On 05/20/2017 10:21 AM, Guenter Roeck wrote:
> On 05/20/2017 12:26 AM, Michal Hocko wrote:
> > On Fri 19-05-17 09:46:23, Guenter Roeck wrote:
> > > Hi,
> > >
> > > my qemu tests of next-20170519 show the following results:
> > >     total: 122 pass: 30 fail: 92
> > >
> > > I won't bother listing all of the failures; they are available at
> > > http://kerneltests.org/builders. I bisected one (openrisc, because
> > > it gives me some console output before dying). It points to
> > > 'mm: drop HASH_ADAPT' as the culprit. Bisect log is attached.
> > >
> > > A quick glance suggests that 64 bit kernels pass and 32 bit kernels 
> > > fail.
> > > 32-bit x86 images fail and should provide an easy test case.
> >
> > Hmm, this is quite unexpected as the patch is not supposed to change
> > things much. It just removes the flag and perform the new hash scaling
>
> It may well be that the problem is introduced with an earlier patch and 
> just
> enabled by this one.
>
> > automatically for all requeusts which do not have any high limit.
> > Some of those didn't have HASH_ADAPT before but that shouldn't change
> > the picture much. The only thing that I can imagine is that what
> > formerly failed for early memblock allocations is now suceeding and that
> > depletes the early memory. Do you have any serial console from the boot?
>
> They are all the same. Either nothing or the following. Picking a couple:
>
> metag:
>
> Linux version 4.12.0-rc1-next-20170519 (groeck@xxxxxxxxxxxxxxxxxxxx) 
> (gcc version 4.2.4 (IMG-1.4.0.300)) #1 Fri May 19 00:50:50 PDT 2017
> LNKGET/SET go through cache but CONFIG_METAG_LNKGET_AROUND_CACHE=y
> DA present
> console [ttyDA1] enabled
> OF: fdt: Machine model: toumaz,tz1090
> Machine name: Generic Meta
> Node 0: start_pfn = 0xb0000, low = 0xbfff7
> Zone ranges:
>    Normal   [mem 0x00000000b0000000-0x00000000bfff6fff]
> Movable zone start for each node
> Early memory node ranges
>    node   0: [mem 0x00000000b0000000-0x00000000bfff6fff]
> Initmem setup node 0 [mem 0x00000000b0000000-0x00000000bfff6fff]
> Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 65015
> Kernel command line: rdinit=/sbin/init doreboot
> PID hash table entries: 1024 (order: 0, 4096 bytes)
>
> crisv32:
>
> Linux version 4.12.0-rc1-next-20170519 (groeck@xxxxxxxxxxxxxxxxxxxx) 
> (gcc version 4.9.2 (Buildroot 2015.02-rc1-00005-gb13bd8e-dirty) ) #1 Fri 
> May 19 00:52:55 PDT 2017
> bootconsole [early0] enabled
> Setting up paging and the MMU.
> Linux/CRISv32 port on ETRAX FS (C) 2003, 2004 Axis Communications AB
> Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 4080
> Kernel command line: console=ttyS0,115200,N,8 rdinit=/sbin/init
> PID hash table entries: 128 (order: -4, 512 bytes)
>
> powerpc:mpc8548cds:
>
> Memory CAM mapping: 256 Mb, residual: 0Mb
> Linux version 4.12.0-rc1-next-20170519 (groeck@xxxxxxxxxxxxxxxxxxxx) 
> (gcc version 4.8.1 (GCC) ) #1 Fri May 19 01:17:29 PDT 2017
> Found initrd at 0xc4000000:0xc4200c00
> Using MPC85xx CDS machine description
> bootconsole [udbg0] enabled
> -----------------------------------------------------
> phys_mem_size     = 0x10000000
> dcache_bsize      = 0x20
> icache_bsize      = 0x20
> cpu_features      = 0x0000000012100460
>    possible        = 0x0000000012100460
>    always          = 0x0000000000100000
> cpu_user_features = 0x84e08000 0x08000000
> mmu_features      = 0x00020010
> -----------------------------------------------------
> mpc85xx_cds_setup_arch()
> Could not find FPGA node.
> Zone ranges:
>    DMA      [mem 0x0000000000000000-0x000000000fffffff]
>    Normal   empty
> Movable zone start for each node
> Early memory node ranges
>    node   0: [mem 0x0000000000000000-0x000000000fffffff]
> Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
> MMU: Allocated 1088 bytes of context maps for 255 contexts
> Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 65024
> Kernel command line: rdinit=/sbin/init console=ttyS0 console=tty doreboot
> PID hash table entries: 1024 (order: 0, 4096 bytes)
>
> Guenter