Re: Re: Re: Re: Re: [PATCH] RDMA/siw: Fix compiler warnings on 32-bit due to u64/pointer abuse

[Jason Gunthorpe]

On Mon, Aug 19, 2019 at 05:39:04PM +0000, Bernard Metzler wrote:
> 
> >To: "Bernard Metzler" <BMT@xxxxxxxxxxxxxx>
> >From: "Jason Gunthorpe" <jgg@xxxxxxxx>
> >Date: 08/19/2019 06:35PM
> >Cc: "Geert Uytterhoeven" <geert@xxxxxxxxxxxxxx>, "Doug Ledford"
> ><dledford@xxxxxxxxxx>, linux-rdma@xxxxxxxxxxxxxxx,
> >linux-kernel@xxxxxxxxxxxxxxx
> >Subject: [EXTERNAL] Re: Re: Re: Re: Re: [PATCH] RDMA/siw: Fix
> >compiler warnings on 32-bit due to u64/pointer abuse
> >
> >On Mon, Aug 19, 2019 at 04:29:11PM +0000, Bernard Metzler wrote:
> >> 
> >> >To: "Bernard Metzler" <BMT@xxxxxxxxxxxxxx>
> >> >From: "Jason Gunthorpe" <jgg@xxxxxxxx>
> >> >Date: 08/19/2019 06:05PM
> >> >Cc: "Geert Uytterhoeven" <geert@xxxxxxxxxxxxxx>, "Doug Ledford"
> >> ><dledford@xxxxxxxxxx>, linux-rdma@xxxxxxxxxxxxxxx,
> >> >linux-kernel@xxxxxxxxxxxxxxx
> >> >Subject: [EXTERNAL] Re: Re: Re: Re: [PATCH] RDMA/siw: Fix compiler
> >> >warnings on 32-bit due to u64/pointer abuse
> >> >
> >> >On Mon, Aug 19, 2019 at 03:54:56PM +0000, Bernard Metzler wrote:
> >> >
> >> >> Absolutely. But these addresses are conveyed through the
> >> >> API as unsigned 64 during post_send(), and land in the siw
> >> >> send queue as is. During send queue processing, these addresses
> >> >> must be interpreted according to its context and transformed
> >> >> (casted) back to the callers intention. I frankly do not
> >> >> know what we can do differently... The representation of
> >> >> all addresses as unsigned 64 is given. Sorry for the confusion.
> >> >
> >> >send work does not have pointers in it, so I'm confused what this
> >is
> >> >about. Does siw allow userspace to stick an ordinary pointer for
> >the
> >> >SG list?
> >> 
> >> Right a user references a buffer by address and local key it
> >> got during reservation of that buffer. The user can provide any
> >> VA between start of that buffer and registered length. 
> >
> >Oh gross, it overloads the IOVA in the WR with a kernel void * ??
> 
> Oh no. The user library writes the buffer address into
> the 64bit address field of the WR. This is nothing siw
> has invented.

No HW provider sticks pointers into the WR ring.

It is either an iova & lkey pair, or SGE information is inlined into
the WR ring.

Never, ever, a user or kernel pointer.

The closest we get to a kernel pointer is with the local dma lkey &
iova == physical memory address.

> >Why does siw_pbl_get_buffer not return a void *??
>
> 
> I think, in fact, it should be dma_addr_t, since this is
> what PBL's are described with. Makes sense?

You mean because siw uses dma_virt_ops and can translate a dma_addr_t
back to a pfn? Yes, that would make alot more sense.

If all conversions went explicitly from a iova & lkey -> dma_addr_t -> void * in
the kmap then I'd be a lot happier

Jason