Re: frequent softlockups with 3.10rc6.

[Steven Rostedt]

On Mon, 2013-06-24 at 12:00 -0400, Dave Jones wrote:
> On Mon, Jun 24, 2013 at 10:52:29AM -0400, Steven Rostedt wrote:
>  
>  > > >  check_list_nodes corruption. next->prev should be prev (ffff88023b8a1a08), but was 00ffff88023b8a1a. (next=ffff880243288001).
>  > > 
>  > > Can't find "check_list_nodes" in lib/list_debug.c or elsewhere...
>  > > 
>  > > >   [<ffffffff816e467d>] dump_stack+0x19/0x1b
>  > > >   [<ffffffff8104a0c1>] warn_slowpath_common+0x61/0x80
>  > > >   [<ffffffff8104a12c>] warn_slowpath_fmt+0x4c/0x50
>  > > >   [<ffffffff81112c61>] rb_head_page_deactivate.isra.39+0x61/0x80
>  > > 
>  > > How? rb_list_head_clear() just modifies list->next directly.
>  > > 
>  > > > hopefully despite that it'll actually function as intended.
>  > > 
>  > > Yes ;)
>  > 
>  > I'm curious to what happened.
> 
> Ah, this is the first victim of my new 'check sanity of nodes during list walks' patch.
> It's doing the same prev->next next->prev checking as list_add and friends.
> I'm looking at getting it into shape for a 3.12 merge after some other preparatory patches
> go into 3.11

OK, and you may need to make an exception for the ring buffer. To do a
lockless swap out of the reader page for one of the pages in the buffer,
it uses the 2 LSB as flags. Notice the "next=ffff880243288001", that "1"
is a flag that states the next page is the "header" page (next to be
read). We use cmpxchg to update the pages to handle races between the
reader and writer.

In fact, the above code is done on reset of the ring buffer, where we
clear those bits:

	list_for_each(hd, cpu_buffer->pages)
		rb_list_head_clear(hd);

The function that was flagged, was the one going through and clearing
the bits.

It seems that your new tool is failing due to this trick.

-- Steve


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/