Re: Performance issue with 2.6 md raid0

[Mattias Wadenstein]

On Fri, 6 Feb 2004, Neil Brown wrote:

> On Friday February 6, piggin@xxxxxxxxxxxxxxx wrote:
> > Mattias Wadenstein wrote:
> > >On Fri, 6 Feb 2004, Nick Piggin wrote:
> > >>Mattias Wadenstein wrote:
> > >>>
> > >>>While testing a file server to store a couple of TB in resonably large
> > >>>files (>1G), I noticed an odd performance behaviour with the md raid0 in a
> > >>>pristine 2.6.2 kernel as compared to a 2.4.24 kernel.
> > >>>
> > >>>When striping two md raid5:s, instead of going from about 160-200MB/s for
> > >>>a single raid5 to 300M/s for the raid0 in 2.4.24, the 2.6.2 kernel gave
> > >>>135M/s in single stream read performance.
> > >>>
> > >>Can you try booting with elevator=deadline please?
> > >
> > >Ok, then I get 253267 kB/s write and 153187 kB/s read from the raid0. A
> > >bit better, but still nowhere near the 2.4.24 numbers.
> > >
> > >For a single raid5, 158028 kB/s write and 162944 kB/s read.
> >
> > Any idea what is holding back performance? Is it IO or CPU bound?

The CPU usage is not significant for the lower numbers (<30%) and seems
linear to the delivered bandwidth for the faster configurations (up to
80% or so when approaching 250M/s write and 360M/s read).

> > Can you get a profile of each kernel while doing a read please?
>
> Possibly the read-ahead size isn't getting set correctly.
>
> What chunksize are you using on the raid0?

I was only using 32k, but when I tried changing this to 4 megs I got no
improvement, leading me to guess that this wasn't it. Unfortunately I
changed it to "4M", something which mkraid happily accepted but
interpreted as "4k".

I didn't verify this against /proc/mdstat then, just noticed this now.

> Are you free to rebuild the raid0 array?

Yeah, the raid5s too if needed, but those take a while to resync before
benchmarking.

> If so, please rebuild it with a chunksize that is 2 or 4 times the
> size of a raid5 stripe (i.e. raid5-chunksize * (raid5-drives - 1) ).

Yes, this is much better. I have a 64k raid5 chunksize. A chunksize of
4096k gives 265313kB/s write and 368228kB/s for a single stream, 512k
chunksize also gives decent performance (slightly better read and rewrite
performance, somewhat lower read performance).

I'll continue tweaking and testing some, please respond (and Cc: me) if
you want to know anything more or if I should try something special.

/Mattias Wadenstein
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/