Re: [PATCHv12 1/3] rdmacg: Added rdma cgroup controller

From: Matan Barak
Date: Sun Sep 11 2016 - 04:40:17 EST


On 10/09/2016 20:01, Jason Gunthorpe wrote:
On Sat, Sep 10, 2016 at 06:14:42PM +0200, Christoph Hellwig wrote:
OFVWG meetings have absolutely zero relevance for Linux development.

Well, to be fair there are a fair number of kernel developers on that
particular call..

More "flexibility" for drivers just means giving up on designing a
coherent API and leaving it to drivers authors to add crap to their
own drivers. That's a major step backwards.

Sadly, it isn't a step backwards, it is status quo - at least as far
as the uapi is concerned.

Every single user space driver has its own private abi file, carefully
hidden in their driver, and dutifully copied over to user space:

providers/cxgb3/iwch-abi.h
providers/cxgb4/cxgb4-abi.h
providers/hfi1verbs/hfi-abi.h
providers/i40iw/i40iw-abi.h
providers/ipathverbs/ipath-abi.h
providers/mlx4/mlx4-abi.h
providers/mlx5/mlx5-abi.h
providers/mthca/mthca-abi.h
providers/nes/nes-abi.h
providers/ocrdma/ocrdma_abi.h
providers/rxe/rxe-abi.h

Just to pick two random examples:

struct mlx5_create_cq {
struct ibv_create_cq ibv_cmd;
__u64 buf_addr;
__u64 db_addr;
__u32 cqe_size;
};

struct iwch_create_cq {
struct ibv_create_cq ibv_cmd;
uint64_t user_rptr_addr;
};

Love to hear ideas on a way forward that doesn't involve rewriting
everything :(


Yeah, unfortunately, the RDMA ABI is more driver specific ABI than a common user-kernel ABI. I guess this will become even worse, as the RDMA subsystem is evolving to serve more drivers with different object types. For example, I would like to hear how hfi1 are going to define their user-kernel ABI (once they leave the custom ioctls).

They should not be using the code in drivers/infiniband. usnic is such
an example of a driver that should never have been added in it's current
form.

+1

Jason


Matan