Re: [PATCH] ntb_perf: pass correct struct device to dma_alloc_coherent

From: Logan Gunthorpe
Date: Tue Dec 10 2019 - 12:32:06 EST



On 2019-12-10 6:07 a.m., Sanjay R Mehta wrote:
> From: Sanjay R Mehta <sanju.mehta@xxxxxxx>
>
> Currently, ntb->dev is passed to dma_alloc_coherent
> and dma_free_coherent calls. The returned dma_addr_t
> is the CPU physical address. This works fine as long
> as IOMMU is disabled. But when IOMMU is enabled, we
> need to make sure that IOVA is returned for dma_addr_t.
> So the correct way to achieve this is by changing the
> first parameter of dma_alloc_coherent() as ntb->pdev->dev
> instead.
>
> Fixes: 5648e56 ("NTB: ntb_perf: Add full multi-port NTB API support")
> Signed-off-by: Sanjay R Mehta <sanju.mehta@xxxxxxx>

Yes, I did the same thing as one of the patches in my fix-up series that
never got merged. See [1].

Hopefully you can make better progress than I did.

While you're at it I think it's worth doing the same thing in ntb_tool
as well as removing the dma_coerce_mask_and_coherent() calls that are in
the NTB drivers which are meaningless once we get back to using the
correct DMA device.

Thanks,

Logan

[1] https://lore.kernel.org/lkml/20190109192233.5752-3-logang@xxxxxxxxxxxx/

> ---
> drivers/ntb/test/ntb_perf.c | 7 ++++---
> 1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/ntb/test/ntb_perf.c b/drivers/ntb/test/ntb_perf.c
> index f5df33a..8729838 100644
> --- a/drivers/ntb/test/ntb_perf.c
> +++ b/drivers/ntb/test/ntb_perf.c
> @@ -559,7 +559,7 @@ static void perf_free_inbuf(struct perf_peer *peer)
> return;
>
> (void)ntb_mw_clear_trans(peer->perf->ntb, peer->pidx, peer->gidx);
> - dma_free_coherent(&peer->perf->ntb->dev, peer->inbuf_size,
> + dma_free_coherent(&peer->perf->ntb->pdev->dev, peer->inbuf_size,
> peer->inbuf, peer->inbuf_xlat);
> peer->inbuf = NULL;
> }
> @@ -588,8 +588,9 @@ static int perf_setup_inbuf(struct perf_peer *peer)
>
> perf_free_inbuf(peer);
>
> - peer->inbuf = dma_alloc_coherent(&perf->ntb->dev, peer->inbuf_size,
> - &peer->inbuf_xlat, GFP_KERNEL);
> + peer->inbuf = dma_alloc_coherent(&perf->ntb->pdev->dev,
> + peer->inbuf_size, &peer->inbuf_xlat,
> + GFP_KERNEL);
> if (!peer->inbuf) {
> dev_err(&perf->ntb->dev, "Failed to alloc inbuf of %pa\n",
> &peer->inbuf_size);
>