Re: -mm seems significanty slower than mainline on kernbench

From: Peter Williams
Date: Tue Jan 24 2006 - 01:21:51 EST


Martin J. Bligh wrote:

Oops. I was looking at the graphs for Moe but <http://test.kernel.org/perf/dbench.elm3b132.png> doesn't appear to be demonstrating a problem either. Given the fluctuation in the 2.6.16-rc1 results (235, 234, 211, 228.5 and 237.5), the results for 2.6.16-rc1-mm1 (229) and 2.6.16-mm2 (219) aren't significantly different.


I disagree. Look at the graph. mm results are consistent and stable, and significantly worse than mainline.

Peter
PS I have a modification for kernbench that calculates and displays the standard deviations for the various averages if you're interested. This would enable you to display 95% (say) confidence bars on the graphed results which in turn makes it easier to spot significant differences.


Thanks, but I have that. What do you think those vertical bars on the graph are for? ;-) They're deviation of 5 runs. I throw away the best and worst first.

Not very good scientific practice :-)

If it was just random noise, then you'd get the same variance *between* runs inside the -mm train and inside mainline. You don't see that ...

I was still looking at the wrong graph this time the dbench instead of kernbench :-( (explains why I didn't see the error bars). Now I see it.

Looking at the other 6 kernbench graphs, I see that it also occurs for elm3b70 but no others (including elm3b6 and elm3b67). Are there any differences between the various elm3b systems that could explain this?

Use the visuals in the graph .. it's very telling. -mm is *broken*.
It may well not be the same issue as last time though, I shouldn't
have jumped to that conclusion.

It's very hard to understand how it could be an issue on a system that doesn't have a lot of abnormally niced (i.e. non zero) tasks that are fairly active as it's now mathematically equivalent to the original in the absence of such tasks. Do these two systems have many such tasks running?

Would it be possible to get a run with the following patches backed out:

+sched-modified-nice-support-for-smp-load-balancing-fix.patch
+sched-modified-nice-support-for-smp-load-balancing-fix-fix.patch

Thanks
Peter
--
Peter Williams pwil3058@xxxxxxxxxxxxxx

"Learning, n. The kind of ignorance distinguishing the studious."
-- Ambrose Bierce
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/