Re: BUG: 2.6.26-rc1-git8: NULL reference in drop_buffers

[Randy Dunlap]

--- Original Message ---
> On Monday 12 May 2008, Randy Dunlap wrote:
> > Andrew Morton wrote:
> > > On Sun, 11 May 2008 10:54:29 -0700 Randy Dunlap <randy.dunlap@xxxxxxxxxx> 
> wrote:
> > >> On x86_64, during testing using "stress" package:
> > >>
> > >> BUG: unable to handle kernel NULL pointer dereference at
> > >> 0000000000000000
> > >>
> > >>
> > >> IP: [<ffffffff802ad273>] drop_buffers+0x2f/0xfb
> > >> PGD 1ee8ad067 PUD 26f19a067 PMD 0
> > >> Oops: 0000 [1] SMP
> > >> CPU 3
> > >> Modules linked in: parport_pc lp parport tg3 cciss ehci_hcd ohci_hcd
> > >> uhci_hcd Pid: 16860, comm: stress Not tainted 2.6.26-rc1-git8 #1
> > >> RIP: 0010:[<ffffffff802ad273>]  [<ffffffff802ad273>]
> > >> drop_buffers+0x2f/0xfb RSP: 0000:ffff81026bc03a08  EFLAGS: 00010203
> > >> RAX: 0000000000000000 RBX: ffffe20008bae680 RCX: ffff81027f490f00
> > >> RDX: 0000000000000000 RSI: ffff81026bc03a58 RDI: ffffe20008bae680
> > >> RBP: ffff81026bc03a38 R08: ffff81026bc03b78 R09: ffff810001103780
> > >> R10: ffff81026bc03a08 R11: ffff81026bc03c88 R12: ffffe20008bae680
> > >> R13: ffff81027c412850 R14: ffff81026bc03d58 R15: ffff81026bc03a58
> > >> FS:  00007fa9e7e416f0(0000) GS:ffff81027f806980(0000)
> > >> knlGS:00000000f7f856c0 CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> > >> CR2: 0000000000000000 CR3: 000000027f973000 CR4: 00000000000006e0
> > >> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > >> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> > >> Process stress (pid: 16860, threadinfo ffff81026bc02000, task
> > >> ffff81027e424c50) Stack:  ffffe20008bafa68 ffffe20008bae680
> > >> ffff81027f490f00 ffff81026bc03a58 ffff81026bc03d58 ffff81026bc03c88
> > >> ffff81026bc03a78 ffffffff802ad39f ffff81027f490f00 ffffe20008b14060
> > >> 0000000000000000 ffff81027f490f00 Call Trace:
> > >>  [<ffffffff802ad39f>] try_to_free_buffers+0x60/0xa2
> > >>  [<ffffffff80267f98>] try_to_release_page+0x3b/0x41
> > >>  [<ffffffff802719bc>] shrink_page_list+0x457/0x562
> > >>  [<ffffffff80271bed>] shrink_inactive_list+0x126/0x361
> > >>  [<ffffffff80271f0d>] shrink_zone+0xe5/0x10a
> > >>  [<ffffffff8027227d>] try_to_free_pages+0x1ef/0x326
> > >>  [<ffffffff80270f4b>] ? isolate_pages_global+0x0/0x34
> > >>  [<ffffffff8026d843>] __alloc_pages_internal+0x25a/0x3ad
> > >>  [<ffffffff8026d9ac>] __alloc_pages+0xb/0xd
> > >>  [<ffffffff80277759>] handle_mm_fault+0x238/0x6d0
> > >>  [<ffffffff8053d9c4>] do_page_fault+0x438/0x7de
> > >>  [<ffffffff8053b999>] error_exit+0x0/0x51
> > >>
> > >>
> > >> Code: 41 57 49 89 f7 41 56 41 55 41 54 49 89 fc 53 48 83 ec 08 48 8b 07
> > >> 25 00 08 00 00 48 85 c0 75 04 0f 0b eb fe 4c 8b 6f 10 4c 89 ea <48> 8b
> > >> 02 25 00 08 00 00 48 85 c0 74 10 49 8b 44 24 18 48 85 c0 RIP 
> > >> [<ffffffff802ad273>] drop_buffers+0x2f/0xfb
> > >>  RSP <ffff81026bc03a08>
> > >> CR2: 0000000000000000
> > >> Kernel panic - not syncing: Fatal exception
> > >
> > > Seems that local variable `bh' is NULL.
> > >
> > > I wonder what the heck we did to cause that.  Which filesystems were in
> > > use?
> >
> > ext3, nfs, and the usual procfs, sysfs, and tmpfs.
> >
> > Also in the kernel:  debugfs, usbfs, inotifyfs, configfs, ramfs,
> > hugetlbfs, msdos, vfat, iso9660, and rootfs.
> 
> If you stand on your head, and race really really hard, 
> nfs_inode_remove_request() does this without locking the page:
> 
>        set_page_private(req->wb_page, 0);
>        ClearPagePrivate(req->wb_page);
> 
> That code has been around for a long time though.
> 
> Probably not the droids we're looking for, but it was the only
> one that jumped 
> out at me during a quick search of set_page_private(foo, 0)
> callers.  
> 
> It seems more likely that we got there by an invalidatepage
> call that left 
> PagePrivate set but didn't allow the page to be freed.
> 
> The page would turn into the funky anonymous zombie thing meant
> for buffers 
> that had to be written before the page could be freed (PagePrivate
> set but 
> page->mapping == NULL), and eventually find its way to
> try_to_free_buffers().
> 
> The problem with that theory is that I would expect page->private
> to be 
> non-null in such a case.  Randy, any chance this can be
> reproduced?

No idea.  I'm rerunning the test now.

~Randy

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/