RE: Mainline kernel OLTP performance update

From: Ma, Chinang
Date: Thu Jan 15 2009 - 11:48:45 EST




>-----Original Message-----
>From: Matthew Wilcox [mailto:matthew@xxxxxx]
>Sent: Wednesday, January 14, 2009 5:22 PM
>To: Andrew Morton
>Cc: Wilcox, Matthew R; Ma, Chinang; linux-kernel@xxxxxxxxxxxxxxx; Tripathi,
>Sharad C; arjan@xxxxxxxxxxxxxxx; Kleen, Andi; Siddha, Suresh B; Chilukuri,
>Harita; Styner, Douglas W; Wang, Peter Xihong; Nueckel, Hubert;
>chris.mason@xxxxxxxxxx; srostedt@xxxxxxxxxx; linux-scsi@xxxxxxxxxxxxxxx;
>Andrew Vasquez; Anirban Chakraborty
>Subject: Re: Mainline kernel OLTP performance update
>
>On Wed, Jan 14, 2009 at 04:35:57PM -0800, Andrew Morton wrote:
>> On Tue, 13 Jan 2009 15:44:17 -0700
>> "Wilcox, Matthew R" <matthew.r.wilcox@xxxxxxxxx> wrote:
>> >
>>
>> (top-posting repaired. That @intel.com address is a bad influence ;))
>
>Alas, that email address goes to an Outlook client. Not much to be done
>about that.
>
>> (cc linux-scsi)
>>
>> > > This is latest 2.6.29-rc1 kernel OLTP performance result. Compare to
>> > > 2.6.24.2 the regression is around 3.5%.
>> > >
>> > > Linux OLTP Performance summary
>> > > Kernel# Speedup(x) Intr/s CtxSw/s us% sys% idle%
>iowait%
>> > > 2.6.24.2 1.000 21969 43425 76 24 0 0
>> > > 2.6.27.2 0.973 30402 43523 74 25 0 1
>> > > 2.6.29-rc1 0.965 30331 41970 74 26 0 0
>
>> But the interrupt rate went through the roof.
>
>Yes. I forget why that was; I'll have to dig through my archives for
>that.

I took a quick look at the interrupts figure between 2.6.24 and 2.6.27. i/o interuputs is slightly down in 2.6.27 (due to reduce throughput). But both NMI and reschedule interrupt increased. Reschedule interrupts is 2x of 2.6.24.

>
>> A 3.5% slowdown in this workload is considered pretty serious, isn't it?
>
>Yes. Anything above 0.3% is statistically significant. 1% is a big
>deal. The fact that we've lost 3.5% in the last year doesn't make
>people happy. There's a few things we've identified that have a big
>effect:
>
> - Per-partition statistics. Putting in a sysctl to stop doing them gets
> some of that back, but not as much as taking them out (even when
> the sysctl'd variable is in a __read_mostly section). We tried a
> patch from Jens to speed up the search for a new partition, but it
> had no effect.
>
> - The RT scheduler changes. They're better for some RT tasks, but not
> the database benchmark workload. Chinang has posted about
> this before, but the thread didn't really go anywhere.
> http://marc.info/?t=122903815000001&r=1&w=2
>
>SLUB would have had a huge negative effect if we were using it -- on the
>order of 7% iirc. SLQB is at least performance-neutral with SLAB.
>
>--
>Matthew Wilcox Intel Open Source Technology Centre
>"Bill, look, we understand that you're interested in selling us this
>operating system, but compare it to ours. We can't possibly take such
>a retrograde step."

-Chinang
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/