Re: [PATCH] ntb_perf: pass correct struct device to dma_alloc_coherent

[Logan Gunthorpe]

On 2019-12-10 6:07 a.m., Sanjay R Mehta wrote:
> From: Sanjay R Mehta <sanju.mehta@xxxxxxx>
> 
> Currently, ntb->dev is passed to dma_alloc_coherent
> and dma_free_coherent calls. The returned dma_addr_t
> is the CPU physical address. This works fine as long
> as IOMMU is disabled. But when IOMMU is enabled, we
> need to make sure that IOVA is returned for dma_addr_t.
> So the correct way to achieve this is by changing the
> first parameter of dma_alloc_coherent() as ntb->pdev->dev
> instead.
> 
> Fixes: 5648e56 ("NTB: ntb_perf: Add full multi-port NTB API support")
> Signed-off-by: Sanjay R Mehta <sanju.mehta@xxxxxxx>

Yes, I did the same thing as one of the patches in my fix-up series that
never got merged. See [1].

Hopefully you can make better progress than I did.

While you're at it I think it's worth doing the same thing in ntb_tool
as well as removing the dma_coerce_mask_and_coherent() calls that are in
the NTB drivers which are meaningless once we get back to using the
correct DMA device.

Thanks,

Logan

[1] https://lore.kernel.org/lkml/20190109192233.5752-3-logang@xxxxxxxxxxxx/

> ---
>  drivers/ntb/test/ntb_perf.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/ntb/test/ntb_perf.c b/drivers/ntb/test/ntb_perf.c
> index f5df33a..8729838 100644
> --- a/drivers/ntb/test/ntb_perf.c
> +++ b/drivers/ntb/test/ntb_perf.c
> @@ -559,7 +559,7 @@ static void perf_free_inbuf(struct perf_peer *peer)
>  		return;
>  
>  	(void)ntb_mw_clear_trans(peer->perf->ntb, peer->pidx, peer->gidx);
> -	dma_free_coherent(&peer->perf->ntb->dev, peer->inbuf_size,
> +	dma_free_coherent(&peer->perf->ntb->pdev->dev, peer->inbuf_size,
>  			  peer->inbuf, peer->inbuf_xlat);
>  	peer->inbuf = NULL;
>  }
> @@ -588,8 +588,9 @@ static int perf_setup_inbuf(struct perf_peer *peer)
>  
>  	perf_free_inbuf(peer);
>  
> -	peer->inbuf = dma_alloc_coherent(&perf->ntb->dev, peer->inbuf_size,
> -					 &peer->inbuf_xlat, GFP_KERNEL);
> +	peer->inbuf = dma_alloc_coherent(&perf->ntb->pdev->dev,
> +					 peer->inbuf_size, &peer->inbuf_xlat,
> +					 GFP_KERNEL);
>  	if (!peer->inbuf) {
>  		dev_err(&perf->ntb->dev, "Failed to alloc inbuf of %pa\n",
>  			&peer->inbuf_size);
>