Re: [Bug report] NULL pointer dereference in frwr_unmap_sync()

From: Chuck Lever
Date: Wed Mar 05 2025 - 09:04:00 EST


On 3/4/25 9:43 PM, Li Nan wrote:
> We found a following problem in kernel 5.10, and the same problem should
> exist in mainline:
>
> During NFS mount using 'soft' option over RoCE network, we observed kernel
> crash with below trace when network issues occur (congestion/disconnect):
>   nfs: server 10.10.253.211 not responding, timed out
>   BUG: kernel NULL pointer dereference, address: 00000000000000a0
>   RIP: 0010:frwr_unmap_sync+0x77/0x200 [rpcrdma]
>   Call Trace:
>    ? __die_body.cold+0x8/0xd
>    ? no_context+0x155/0x230
>    ? __bad_area_nosemaphore+0x52/0x1a0
>    ? exc_page_fault+0x2dc/0x550
>    ? asm_exc_page_fault+0x1e/0x30
>    ? frwr_unmap_sync+0x77/0x200 [rpcrdma]
>    xprt_release+0x9e/0x1a0 [sunrpc]
>    rpc_release_resources_task+0xe/0x50 [sunrpc]
>    rpc_release_task+0x19/0xa0 [sunrpc]
>    rpc_async_schedule+0x29/0x40 [sunrpc]
>    process_one_work+0x1b2/0x350
>    worker_thread+0x49/0x310
>    ? rescuer_thread+0x380/0x380
>    kthread+0xfb/0x140
>
> Problem analysis:
> The crash happens in frwr_unmap_sync() when accessing req->rl_registered
> list, caused by either NULL pointer or accessing freed MR resources.
> There's a race condition between:
> T1
> __ib_process_cq
>  wc->wr_cqe->done (frwr_wc_localinv)
>   rpcrdma_flush_disconnect
>    rpcrdma_force_disconnect
>     xprt_force_disconnect
>      xprt_autoclose
>       xprt_rdma_close
>        rpcrdma_xprt_disconnect
>         rpcrdma_reqs_reset
>          frwr_reset
>           rpcrdma_mr_pop(&req->rl_registered)
> T2                   
> rpc_async_schedule
>  rpc_release_task
>   rpc_release_resources_task
>    xprt_release
>     xprt_rdma_free
>      frwr_unmap_sync
>       rpcrdma_mr_pop(&req->rl_registered)
>                    
> This problem also exists in function rpcrdma_mrs_destroy().
>

Dai, is this the same as the system test problem you've been looking at?

--
Chuck Lever