Re: [BUG] kernel BUG at mm/memcontrol.c:1074!

[KAMEZAWA Hiroyuki]

On Thu, 19 Jan 2012 07:10:26 +0200
Sasha Levin <levinsasha928@xxxxxxxxx> wrote:

> Hi all,
> 
> During testing, I have triggered the OOM killer by mmap()ing a large block of memory. The OOM kicked in and tried to kill the process:
> 

two questions.

1. What is the kernel version  ?
2. are you using memcg moutned ?

Thanks,
-Kame

> [  526.657446] trinity invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
> [  526.659083] trinity cpuset=/ mems_allowed=0
> [  526.659854] Pid: 2200, comm: trinity Not tainted 3.2.0-next-20120119-sasha #128
> [  526.661203] Call Trace:
> [  526.661703]  [<ffffffff82583260>] ? _raw_spin_unlock+0x30/0x60
> [  526.662839]  [<ffffffff8116aefe>] dump_header+0x7e/0x330
> [  526.663841]  [<ffffffff82583303>] ? _raw_spin_unlock_irqrestore+0x73/0xa0
> [  526.665104]  [<ffffffff81835b20>] ? ___ratelimit+0xd0/0x180
> [  526.666149]  [<ffffffff8116b5cd>] oom_kill_process+0x7d/0x2d0
> [  526.667224]  [<ffffffff8116bcc0>] out_of_memory+0x1d0/0x400
> [  526.668237]  [<ffffffff81171011>] __alloc_pages_nodemask+0x8f1/0x910
> [  526.669388]  [<ffffffff811a8870>] alloc_pages_current+0xa0/0x110
> [  526.670486]  [<ffffffff8116713f>] __page_cache_alloc+0x8f/0xa0
> [  526.671610]  [<ffffffff81167f3a>] filemap_fault+0x34a/0x4e0
> [  526.672666]  [<ffffffff8118779f>] __do_fault+0x7f/0x5c0
> [  526.673665]  [<ffffffff810de041>] ? get_parent_ip+0x11/0x50
> [  526.674744]  [<ffffffff81053900>] ? native_sched_clock+0x60/0x90
> [  526.675868]  [<ffffffff8118a6e1>] handle_pte_fault+0xa1/0xa20
> [  526.676941]  [<ffffffff81107cfe>] ? put_lock_stats.clone.18+0xe/0x40
> [  526.678118]  [<ffffffff81108012>] ? lock_release_holdtime+0xb2/0x160
> [  526.679300]  [<ffffffff8118c7ae>] handle_mm_fault+0x1ce/0x330
> [  526.680405]  [<ffffffff8107d94d>] do_page_fault+0x15d/0x4d0
> [  526.681464]  [<ffffffff810aaf53>] ? do_fork+0x73/0x340
> [  526.682440]  [<ffffffff811ebff5>] ? vfsmount_lock_local_unlock+0x55/0x80
> [  526.683645]  [<ffffffff811ec988>] ? mntput_no_expire+0x38/0x100
> [  526.684709]  [<ffffffff811ed46e>] ? mntput+0x1e/0x30
> [  526.685605]  [<ffffffff811ce463>] ? fput+0x1b3/0x2b0
> [  526.686514]  [<ffffffff81076d11>] do_async_page_fault+0x31/0x90
> [  526.687573]  [<ffffffff825843d5>] async_page_fault+0x25/0x30
> [  526.688585] Mem-Info:
> [  526.689000] Node 0 DMA per-cpu:
> [  526.689605] CPU    0: hi:    0, btch:   1 usd:   0
> [  526.690484] Node 0 DMA32 per-cpu:
> [  526.691171] CPU    0: hi:   90, btch:  15 usd:   0
> [  526.692085] active_anon:1218 inactive_anon:12 isolated_anon:0
> [  526.692087]  active_file:1 inactive_file:6 isolated_file:0
> [  526.692087]  immediate:0 unevictable:48358 dirty:6 writeback:0 unstable:0
> [  526.692088]  free:864 slab_reclaimable:1696 slab_unreclaimable:3992
> [  526.692089]  mapped:5 shmem:2 pagetables:141 bounce:0
> [  526.697504] Node 0 DMA free:1300kB min:108kB low:132kB high:160kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB immediate:0kB unevictable:14568kB isolated(anon):0kB isolated(file):0kB present:15656kB mlocked:14576kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:32kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
> [  526.704557] lowmem_reserve[]: 0 299 299 299
> [  526.705458] Node 0 DMA32 free:2156kB min:2156kB low:2692kB high:3232kB active_anon:4872kB inactive_anon:48kB active_file:4kB inactive_file:24kB immediate:0kB unevictable:178864kB isolated(anon):0kB isolated(file):0kB present:306432kB mlocked:178880kB dirty:24kB writeback:0kB mapped:20kB shmem:8kB slab_reclaimable:6784kB slab_unreclaimable:15968kB kernel_stack:1376kB pagetables:532kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:44 all_unreclaimable? yes
> [  526.712825] lowmem_reserve[]: 0 0 0 0
> [  526.713633] Node 0 DMA: 1*4kB 1*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1308kB
> [  526.715878] Node 0 DMA32: 10*4kB 6*8kB 4*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2168kB
> [  526.718169] 10 total pagecache pages
> [  526.718820] 0 pages in swap cache
> [  526.719449] Swap cache stats: add 0, delete 0, find 0/0
> [  526.720392] Free swap  = 0kB
> [  526.720947] Total swap = 0kB
> [  526.722927] 81904 pages RAM
> [  526.723470] 14810 pages reserved
> [  526.724094] 558 pages shared
> [  526.724611] 65388 pages non-shared
> [  526.725239] [ pid ]   uid  tgid total_vm      rss cpu oom_adj oom_score_adj name
> [  526.726586] [ 2193]     0  2193     4505       92   0       0             0 sh
> [  526.727884] [ 2200]     0  2200     3959      560   0       0             0 trinity
> [  526.729301] [ 2201]     0  2201     3959      561   0       0             0 trinity
> [  526.730804] [13370]     0 13370   528247    48921   0       0             0 trinity
> [  526.732207] Out of memory: Kill process 13370 (trinity) score 700 or sacrifice child
> [  526.733624] Killed process 13370 (trinity) total-vm:2112988kB, anon-rss:195680kB, file-rss:4kB
> 
> So far, everything went on as expected.
> 
> The problem is, that it looks like this has triggered a BUG() in the memory cgroup code:
> 
> [  526.737227] ------------[ cut here ]------------
> [  526.738032] 
> [  526.738032] invalid opcode: 0000 [#1] PREEMPT SMP 
> [  526.738032] CPU 0 
> [  526.738032] Pid: 1091, comm: kswapd0 Not tainted 3.2.0-next-20120119-sasha #128  
> [  526.738032] RIP: 0010:[<ffffffff811c4b4a>]  [<ffffffff811c4b4a>] mem_cgroup_lru_del_list+0xca/0xd0
> [  526.738032] RSP: 0018:ffff8800127139a0  EFLAGS: 00010046
> [  526.738032] RAX: 0000000000000001 RBX: ffffea0000358300 RCX: 0000000000000000
> [  526.738032] RDX: ffff880012c0b800 RSI: 0000000000000000 RDI: 0000000000000000
> [  526.738032] RBP: ffff8800127139b0 R08: ffff880012713ad0 R09: 0000000000000001
> [  526.738032] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000002
> [  526.738032] R13: ffffea0000358300 R14: ffffea0000358320 R15: 0000000000000001
> [  526.738032] FS:  0000000000000000(0000) GS:ffff880013a00000(0000) knlGS:0000000000000000
> [  526.738032] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  526.738032] CR2: 00007fea7fa42e66 CR3: 000000000c42a000 CR4: 00000000000406f0
> [  526.738032] DR0: ffffffff810aaee0 DR1: 0000000000000000 DR2: 0000000000000000
> [  526.738032] DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000600
> [  526.738032] Process kswapd0 (pid: 1091, threadinfo ffff880012712000, task ffff880012f7d840)
> [  526.738032] Stack:
> [  526.738032]  ffff880012c0b968 ffff880012c0b968 ffff8800127139c0 ffffffff811c4f0a
> [  526.738032]  ffff880012713a70 ffffffff81178c63 ffff8800127139e0 ffffea00000cbba0
> [  526.738032]  ffff880012713a40 ffff880012713b08 0000000000000001 ffffffffffffffff
> [  526.738032] Call Trace:
> [  526.738032]  [<ffffffff811c4f0a>] mem_cgroup_lru_del+0x3a/0x40
> [  526.738032]  [<ffffffff81178c63>] isolate_lru_pages+0xe3/0x330
> [  526.738032]  [<ffffffff8117a11e>] ? shrink_inactive_list+0xce/0x480
> [  526.738032]  [<ffffffff8117a153>] shrink_inactive_list+0x103/0x480
> [  526.738032]  [<ffffffff811c2a46>] ? mem_cgroup_iter+0x176/0x310
> [  526.738032]  [<ffffffff810e2c55>] ? sched_clock_local+0x25/0x90
> [  526.738032]  [<ffffffff8117ac04>] shrink_mem_cgroup_zone+0x3f4/0x580
> [  526.738032]  [<ffffffff81107cfe>] ? put_lock_stats.clone.18+0xe/0x40
> [  526.738032]  [<ffffffff8117adfe>] shrink_zone+0x6e/0xa0
> [  526.738032]  [<ffffffff8117be65>] balance_pgdat+0x545/0x750
> [  526.738032]  [<ffffffff810de1ed>] ? sub_preempt_count+0x9d/0xd0
> [  526.738032]  [<ffffffff8117c233>] kswapd+0x1c3/0x320
> [  526.738032]  [<ffffffff810cee30>] ? abort_exclusive_wait+0xb0/0xb0
> [  526.738032]  [<ffffffff8117c070>] ? balance_pgdat+0x750/0x750
> [  526.738032]  [<ffffffff810ce06e>] kthread+0xbe/0xd0
> [  526.738032]  [<ffffffff82585df4>] kernel_thread_helper+0x4/0x10
> [  526.738032]  [<ffffffff810d8c88>] ? finish_task_switch+0x78/0x100
> [  526.738032]  [<ffffffff825840f8>] ? retint_restore_args+0x13/0x13
> [  526.738032]  [<ffffffff810cdfb0>] ? kthread_flush_work_fn+0x10/0x10
> [  526.738032]  [<ffffffff82585df0>] ? gs_change+0x13/0x13
> [  526.738032] Code: 8b 1c 24 4c 8b 64 24 08 c9 c3 0f 1f 80 00 00 00 00 8b 4b 68 eb ba 0f 1f 00 0f b6 4b 68 bb 01 00 00 00 d3 e3 48 63 cb eb c2 0f 0b <0f> 0b 0f 1f 40 00 55 48 89 e5 48 83 ec 60 48 89 5d d8 4c 89 65 
> [  526.738032] RIP  [<ffffffff811c4b4a>] mem_cgroup_lru_del_list+0xca/0xd0
> [  526.738032]  RSP <ffff8800127139a0>
> [  526.738032] ---[ end trace 866f4f6c624b8d58 ]---
> 
> -- 
> 
> Sasha.
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@xxxxxxxxxx  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
> Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/