Re: [IPC] INFO: suspicious RCU usage.

[Davidlohr Bueso]

Ccing Andrew.

On Thu, 2013-06-06 at 21:25 +0800, Fengguang Wu wrote:
> Greetings,
> 
> I got the below dmesg and the first bad commit is
> 
> commit 1f6587114a689a5d7fdfb0d4abc818117e3182a5
> Author: Davidlohr Bueso <davidlohr.bueso@xxxxxx>
> Date:   Thu Jun 6 10:41:56 2013 +1000
> 
>     ipc: move rcu lock out of ipc_addid
>     
>     This patchset continues the work that began in the sysv ipc semaphore
>     scaling series: https://lkml.org/lkml/2013/3/20/546
>     
>     Just like semaphores used to be, sysv shared memory and msg queues also
>     abuse the ipc lock, unnecessarily holding it for operations such as
>     permission and security checks.  This patchset mostly deals with mqueues,
>     and while shared mem can be done in a very similar way, I want to get
>     these patches out in the open first.  It also does some pending cleanups,
>     mostly focused on the two level locking we have in ipc code, taking care
>     of ipc_addid() and ipcctl_pre_down_nolock() - yes there are still
>     functions that need to be updated as well.
>     
>     This patch:
>     
>     Make all callers explicitly take and release the RCU read lock.
>     
>     This addresses the two level locking seen in newary(), newseg() and
>     newqueue().  For the last two, explicitly unlock the ipc object and the
>     rcu lock, instead of calling the custom shm_unlock and msg_unlock
>     functions.  The next patch will deal with the open coded locking for
>     ->perm.lock
>     
>     Signed-off-by: Davidlohr Bueso <davidlohr.bueso@xxxxxx>
>     Cc: Andi Kleen <andi@xxxxxxxxxxxxxx>
>     Cc: Rik van Riel <riel@xxxxxxxxxx>
>     Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
> 
> [   51.524946] 
> [   51.525983] ===============================
> [   51.532875] [ INFO: suspicious RCU usage. ]
> [   51.535385] 3.10.0-rc4-next-20130606 #6 Not tainted
> [   51.538304] -------------------------------
> [   51.540937] /c/kernel-tests/src/stable/include/linux/rcupdate.h:471 Illegal context switch in RCU read-side critical section!
> [   51.548110] 
> [   51.548110] other info that might help us debug this:
> [   51.548110] 
> [   51.553055] 
> [   51.553055] rcu_scheduler_active = 1, debug_locks = 1
> [   51.557199] 2 locks held by trinity/1107:
> [   51.560168]  #0:  (&ids->rw_mutex){+.+.+.}, at: [<ffffffff811e15ee>] ipcget+0x38/0x2b3
> [   51.566465]  #1:  (rcu_read_lock){.+.+..}, at: [<ffffffff811e7698>] newseg+0x19d/0x3fd
> [   51.572413] 
> [   51.572413] stack backtrace:
> [   51.574761] CPU: 0 PID: 1107 Comm: trinity Not tainted 3.10.0-rc4-next-20130606 #6
> [   51.579331] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
> [   51.583068]  0000000000000001 ffff880004a07d88 ffffffff817b1f5c ffff880004a07db8
> [   51.592119]  ffffffff810f2f1d ffffffff81b78569 00000000000001a8 0000000000000000
> [   51.596726]  0000000000000000 ffff880004a07de8 ffffffff810ded5e ffff880004a07fd8
> [   51.605189] Call Trace:
> [   51.606409]  [<ffffffff817b1f5c>] dump_stack+0x19/0x1b
> [   51.609632]  [<ffffffff810f2f1d>] lockdep_rcu_suspicious+0xeb/0xf4
> [   51.612905]  [<ffffffff810ded5e>] __might_sleep+0x59/0x1dc
> [   51.618614]  [<ffffffff81238623>] idr_preload+0x9b/0x142
> [   51.621939]  [<ffffffff811e0e56>] ipc_addid+0x3d/0x193
> [   51.624373]  [<ffffffff811e771c>] newseg+0x221/0x3fd
> [   51.626596]  [<ffffffff811e7698>] ? newseg+0x19d/0x3fd
> [   51.630177]  [<ffffffff811e1774>] ipcget+0x1be/0x2b3
> [   51.633174]  [<ffffffff817bc094>] ? retint_swapgs+0x13/0x1b
> [   51.636356]  [<ffffffff811e7a5a>] SyS_shmget+0x59/0x5d
> [   51.639576]  [<ffffffff811e74fb>] ? shm_try_destroy_orphaned+0xbf/0xbf
> [   51.643673]  [<ffffffff811e6ce5>] ? shm_get_unmapped_area+0x20/0x20
> [   51.647321]  [<ffffffff811e6cf0>] ? shm_security+0xb/0xb
> [   51.650831]  [<ffffffff817bcb27>] system_call_fastpath+0x16/0x1b

I suspect this is caused because now we call idr_preload() in ipc_addid
with the rcu lock held by the caller. So, we can either have a two level
rcu locking or a two level idr_preload/idr_preload_end.

Thanks,
Davidlohr

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/