Re: [PATCHv1 0/6] rdma controller support

[Parav Pandit]

On Thu, Jan 7, 2016 at 8:37 PM, Tejun Heo <tj@xxxxxxxxxx> wrote:
> Hello, Parav.
>
> On Thu, Jan 07, 2016 at 04:43:20AM +0530, Parav Pandit wrote:
>> > If different controllers can't agree upon the
>> > same set of resources, which probably is a pretty good sign that this
>> > isn't too well thought out to begin with,
>>
>> When you said "different controller" you meant "different hw vendors", right?
>> Or you meant, rdma, mem, cpu as controller here?
>
> Different hw vendors.
>
>> > at least make all resource
>> > types defined by the controller itself and let the controllers enable
>> > them selectively.
>> >
>> In this V1 patch, resource is defined by the IB stack and rdma cgroup
>> is facilitator for same.
>> By doing so, IB stack modules can define new resource without really
>> making changes to cgroup.
>> This design also allows hw vendors to define their own resources which
>> will be reviewed in rdma mailing list anway.
>> The idea is different hw versions can have different resource support,
>> so the whole intention is not about defining different resource but
>> rather enabling it.
>> But yes, I equally agree that by doing so, different hw controller
>> vendors can define different hw resources.
>
> How many vendors and resources are we talking about?
To my knowledge Intel HFI driver is the only one.

>  What I was
> trying to say was that unless the number is extremely high, it'd be
> far simpler to hard code them in the rdma controller and let drivers
> enable the ones which apply to them.

Instead of in rdma controller, its hard coded in IB stack.
I see this as an advantage where resource definition ownership remains
with IB stack maintainers, rather than rdma cgroup maintainer.
rdma cgroup maintainer doesn't have to understand what SRQ vs QP or
ODP type MR or multicast group is.
IB stack maintainer is better placed to judge and define it.

I would like to hear from Jason, Doug, Liran and other RDMA experts
about their thoughts.

> It would require updating the
> rdma cgroup controller to add new resource types but I think that'd
> actually be an upside, not down.  There needs to be some checks and
> balances against adding new resource types; otherwise, it'll soon
> become a mess.

I think this checks for new resource type can be ensured by IB stack
maintainer to avoid the mess.

My preference is IB stack to define resource, but I am ok to let go
this feature if consensus is RDMA cgroup.

>
> Thanks.
>
> --
> tejun