Re: [PATCH 5/7] blk-mq: remove REQ_ATOM_COMPLETE usages from blk-mq

[jianchao.wang]

Hi tejun

Many thanks for you kindly response.

On 01/09/2018 01:27 AM, Tejun Heo wrote:
> Hello, Jianchao.
> 
> On Fri, Dec 22, 2017 at 12:02:20PM +0800, jianchao.wang wrote:
>>> On Thu, Dec 21, 2017 at 11:56:49AM +0800, jianchao.wang wrote:
>>>> It's worrying that even though the blk_mark_rq_complete() here is
>>>> intended to synchronize with timeout path, but it indeed give the
>>>> blk_mq_complete_request() the capability to exclude with
>>
>> There could be scenario where the driver itself stop a request
>> itself with blk_mq_complete_request() or some other interface that
>> will invoke it, races with the normal completion path where a same
>> request comes.
> 
> But what'd prevent the completion reinitializing the request and then
> the actual completion path coming in and completing the request again?
> 
blk_mark_rq_complete() will gate and ensure there will be only one 
__blk_mq_complete_request() to be invoked.

>> For example:
>> a reset could be triggered through sysfs on nvme-rdma
>> Then the driver will cancel all the reqs, including in-flight ones.
>> nvme_rdma_reset_ctrl_work()
>>     nvme_rdma_shutdown_ctrl()
>>     >>>>
>>         if (ctrl->ctrl.queue_count > 1) {
>>             nvme_stop_queues(&ctrl->ctrl); //quiesce the queue
>>             blk_mq_tagset_busy_iter(&ctrl->tag_set,
>>                         nvme_cancel_request, &ctrl->ctrl); //invoke blk_mq_complete_request()
>>             nvme_rdma_destroy_io_queues(ctrl, shutdown);
>>         }
>>     >>>>
>>
>> These operations could race with the normal completion path of in-flight ones.
>> It should drain all the in-flight ones first here. But there maybe some other
>> places similar with this.
> 
> If there are any such places, they should be using an interface which
> is propelry synchronized like blk_abort_request(), which btw is what
> libata already does.  Otherwise, it's racy with or without these
> patches.
Yes, it is that.

Thanks for you kindly response again.
Jianchao