Re: [PATCH rdma-next] RDMA/rdmavt: Decouple QP and SGE lists allocations

[Jason Gunthorpe]

On Tue, May 25, 2021 at 10:10:47AM -0400, Dennis Dalessandro wrote:
> On 5/25/21 9:13 AM, Jason Gunthorpe wrote:
> > On Thu, May 20, 2021 at 06:02:09PM -0400, Dennis Dalessandro wrote:
> > 
> > > > I don't want to encourage other drivers to do the same thing.
> > > 
> > > I would imagine they would get the same push back we are getting here. I
> > > don't think this would encourage anyone honestly.
> > 
> > Then we are back to making infrastructure that is only useful for one,
> > arguably wrong, driver.
> 
> That's just it, you argue that it's wrong. We don't agree that it's wrong.
> In fact you said previously:

You haven't presented a single shred of anything to substantiate this
disagreement beyoned "we have ancient benchmarks we can't reproduce"

Not even a hand wavey logical argument why it could matter.

> "
> The *only* reason to override the node behavior in the kernel is if
> the kernel knows with high certainty that allocations are only going
> to be touched by certain CPUs, such as because it knows that the
> allocation is substantially for use in a CPU pinned irq/workqeueue or
> accessed via DMA from a node affine DMA device.
> "
> 
> Well, that's pretty much why we are doing this.

Huh?I don't see DMA from the qp struct and as I said any MSI affinity
should be driven by the comp_vector, so no, I don't think that is what
HFI is doing at all.

> We are already mid 5.13 cycle. So the earliest this could be queued up to go
> in is 5.14. Can this wait one more cycle? If we can't get it tested/proven
> to make a difference mid 5.14, we will drop the objection and Leon's patch
> can go ahead in for 5.15. Fair compromise?

Fine, but the main question is if you can use normal memory policy
settings, not this.

Jason