Re: [PATCH 00/15] RDS: connection scalability and performance improvements

[santosh shilimkar]

On 9/20/2015 1:37 AM, Sagi Grimberg wrote:
> On 9/20/2015 2:04 AM, Santosh Shilimkar wrote:
> > This series addresses RDS connection bottlenecks on massive workloads and
> > improve the RDMA performance almost by 3X. RDS TCP also gets a small gain
> > of about 12%.
> >
> > RDS is being used in massive systems with high scalability where several
> > hundred thousand end points and tens of thousands of local processes
> > are operating in tens of thousand sockets. Being RC(reliable connection),
> > socket bind and release happens very often and any inefficiencies in
> > bind hash look ups hurts the overall system performance. RDS bin
> > hash-table
> > uses global spin-lock which is the biggest bottleneck. To make matter
> > worst,
> > it uses rcu inside global lock for hash buckets.
> > This is being addressed by simply using per bucket rw lock which makes
> > the
> > locking simple and very efficient. The hash table size is also scaled up
> > accordingly.
> >
> > For RDS RDMA improvement, the completion handling is revamped so that we
> > can do batch completions. Both send and receive completion handlers are
> > split logically to achieve the same. RDS 8K messages being one of the
> > key usecase, mr pool is adapted to have the 8K mrs along with default 1M
> > mrs. And while doing this, few fixes and couple of bottlenecks seen with
> > rds_sendmsg() are addressed.
>
> Hi Santosh,
>
> I think that can get a more effective code review if you CC the
> Linux-rdma mailing list.
I will do that from next time. Thanks Sagi !!

Regards,
Santosh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/