Re: [PATCH 2/9] IB: add a proper completion queue abstraction

From: Bart Van Assche
Date: Fri Nov 13 2015 - 14:58:10 EST

On 11/13/2015 10:25 AM, Jason Gunthorpe wrote:
On Fri, Nov 13, 2015 at 02:46:43PM +0100, Christoph Hellwig wrote:
This adds an abstraction that allows ULP to simply pass a completion
object and completion callback with each submitted WR and let the RDMA
core handle the nitty gritty details of how to handle completion
interrupts and poll the CQ.

This looks pretty nice, I'd really like to look it over carefully
after SC|15..

I know Bart and others have attempted to have switching between event
and polling driven operation, but there were problems resolving the
races. Would be nice to review that conversation.. Do you remember the
details Bart?

Hello Jason,

I think this is the conversation you are referring to: "About a shortcoming of the verbs API" ( That conversation occurred five years ago, which means that you have an excellent memory :-)

I doesn't seem to me like Christoph wanted to support dynamic switching between the IB_POLL_DIRECT, IB_POLL_SOFTIRQ and IB_POLL_WORKQUEUE polling modes. I think this should have been mentioned in the patch description.

The implementation of this patch makes it clear that it is essential that all polling is serialized. The WC array that is used for polling is embedded in the CQ and is not protected against concurrent access. This means that it is essential that _ib_process_cq() calls are serialized. I need some more time to verify whether such serialization is always guaranteed by this patch.

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at