RE: [PATCH 06/16] ntb: add check and comment for link up to mw_count and mw_get_align

From: Hubbe, Allen
Date: Thu Jun 29 2017 - 17:28:22 EST


From: Logan Gunthorpe
> On 6/29/2017 2:13 PM, Allen Hubbe wrote:
> > Unfortunately, it is to work around hardware errata. That is not so trivial to fix.
>
> Can you describe more what the work around is doing? Can you share the
> code? It seems odd that a workaround is based on the alignment
> restrictions of the mws.

Sure, while not making any claim that this is ready for upstream.

It is not a workaround for alignment restrictions of the mws. It is a restriction to avoid the use of doorbells and scratchpads. Memory windows are used exclusively.

Read msi-x local <addr,data> and send that to the peer:
https://github.com/ntrdma/ntrdma/blob/master/drivers/ntc/ntc_ntb_msi.c#L603

Transform peer's addr to the memory window region:
https://github.com/ntrdma/ntrdma/blob/master/drivers/ntc/ntc_ntb_msi.c#L622

Append a dma immediate value operation after other operations, to write the data at addr:
https://github.com/ntrdma/ntrdma/blob/master/drivers/ntc/ntc_ntb_msi.c#L1208

Above describes the workaround.

Used in the context of a rdma-over-ntb driver here:
https://github.com/ntrdma/ntrdma/blob/master/drivers/infiniband/hw/ntrdma/ntrdma_qp.c#L1585

>
> Logan