Re: Slow DOWN, please!!!

[Stefan Richter]

Tarkan Erimer wrote:
> To improve the quality of kernel releases, maybe we can create a special 
> kernel testing tool.

A variety of bugs cannot be caught by automated tests.  Notably those 
which happen with rare hardware, or due to very specific interaction 
with hardware, or with very special workloads.

An interesting thing to investigate would be to start at the regression 
meta bugs at bugzilla.kernel.org, go through all bugs on which are 
linked from there, and try to figure out
  - if these bugs could have been found by automated or at least
    semiautomatic tests on pre-merge code, and
  - how those tests had to have looked like, e.g. what equipment would
    have been necessary.

Let's look back at the posting at the thread start:
| On Wed, Apr 30, 2008 at 10:03 AM, David Miller <davem@xxxxxxxxxxxxx> 
wrote:
| >  Yesterday, I spent the whole day bisecting boot failures
| >  on my system due to the totally untested linux/bitops.h
| >  optimization, which I fully analyzed and debugged.
...
| >  Yet another bootup regression got added within the last 24
| >  hours.

Bootup regressions can be automatically caught if the necessary machines 
are available, and candidate code gets exposure to test parks of those 
machines.  I hear this is already being done, and increasingly so.  But 
those test parks will ever only cover a tiny fraction of existing 
hardware and cannot be subjected to all code iterations and all possible 
.config permutations, hence will have limited coverage of bugs.

And things like the bitops issue depend on review much more than on 
tests, AFAIU.
--
Stefan Richter
-=====-==--- -=-= ----=
http://arcgraph.de/sr/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/