Re: [PATCH for-next 0/3] EFA dmabuf memory regions

[Gal Pressman]

On 12/10/2021 15:09, Gal Pressman wrote:
> Hey all,
>
> This is a followup to my previous RFCs [1][2], which now adds a new api
> to the RDMA subsystem that allows drivers to get a pinned dmabuf memory
> region without requiring an implementation of the move_notify callback.
> The new api makes use of the dynamic attachment api implemented in the
> RDMA subsystem, but calls dma_buf_pin() in order to make sure that the
> callback will not be called, as suggested by Christian.
>
> As explained in the previous RFC, move_notify requires the RDMA device
> to support on-demand-paging (ODP) which is not common on most devices
> (only supported by mlx5).
>
> While the dynamic requirement makes sense for certain GPUs, some devices
> (such as habanalabs) have device memory that is always "pinned" and do
> not need/use the move_notify operation.
>
> Patch #1 changes the dmabuf documentation to make it clear that pinning
> does not necessarily mean the memory must be moved to system memory, it
> is up to the exporter to decide.
> Patch #2 adds the RDMA api that allows drivers to get pinned dmabuf
> memory regions.
> Patch #3 adds the EFA implementation of the dmabuf importer.
>
> The motivation of this submission is to use habanalabs as the dmabuf
> exporter, and EFA as the importer to allow for peer2peer access through
> libibverbs.
>
> [1] https://lore.kernel.org/linux-rdma/20210818074352.29950-1-galpress@xxxxxxxxxx/
> [2] https://lore.kernel.org/linux-rdma/20211007104301.76693-1-galpress@xxxxxxxxxx/
>
> Thanks


Hey Jason, did you get a chance to take a look?