Re: [PATCH for-next 0/3] EFA dmabuf memory regions

[Jason Gunthorpe]

On Tue, Oct 12, 2021 at 03:09:00PM +0300, Gal Pressman wrote:
> Hey all,
> 
> This is a followup to my previous RFCs [1][2], which now adds a new api
> to the RDMA subsystem that allows drivers to get a pinned dmabuf memory
> region without requiring an implementation of the move_notify callback.
> The new api makes use of the dynamic attachment api implemented in the
> RDMA subsystem, but calls dma_buf_pin() in order to make sure that the
> callback will not be called, as suggested by Christian.
> 
> As explained in the previous RFC, move_notify requires the RDMA device
> to support on-demand-paging (ODP) which is not common on most devices
> (only supported by mlx5).
> 
> While the dynamic requirement makes sense for certain GPUs, some devices
> (such as habanalabs) have device memory that is always "pinned" and do
> not need/use the move_notify operation.
> 
> Patch #1 changes the dmabuf documentation to make it clear that pinning
> does not necessarily mean the memory must be moved to system memory, it
> is up to the exporter to decide.
> Patch #2 adds the RDMA api that allows drivers to get pinned dmabuf
> memory regions.
> Patch #3 adds the EFA implementation of the dmabuf importer.
> 
> The motivation of this submission is to use habanalabs as the dmabuf
> exporter, and EFA as the importer to allow for peer2peer access through
> libibverbs.
> 
> [1] https://lore.kernel.org/linux-rdma/20210818074352.29950-1-galpress@xxxxxxxxxx/
> [2] https://lore.kernel.org/linux-rdma/20211007104301.76693-1-galpress@xxxxxxxxxx/
> 
> Thanks
> 
> Gal Pressman (3):
>   dma-buf: Fix pin callback comment
>   RDMA/umem: Allow pinned dmabuf umem usage
>   RDMA/efa: Add support for dmabuf memory regions

Applied to for-next, thanks

Jason