Re: [LTP] Towards 4.14 LTS

[Tom Gall]

> On Nov 20, 2017, at 10:10 AM, Cyril Hrubis <chrubis@xxxxxxx> wrote:
> 
> Hi!
>> So why didn???t we report these? As mentioned we???ve been tossing out dodgy
>> test cases to get to a clean baseline. We don???t need or want noise. 
>> 
>> For LTS, I want the system when it detects a failure to enable a quick 
>> bisect involving the affected test bucket. Given the nature of kernel 
>> bugs tho, there is that class of bug which only happens occasionally.
> 
> From my experience debugging kernel bugs requires an actuall human
> interaction and there is only certain level of automation that can be
> achieved. Don't take me wrong, automatic bisection and other bells and
> whistles are a nice to have, but at the end of the day you usually need
> someone to reproduce/look at the problem, possibly check the source
> code, report a bug, etc. Hence it does not make much sense to have an
> automated system without dedicated engineers assigned to review the test
> results.

You are entirely right automation only gets so far. We have a few lines
of defense that probably are worth a mention.

1) infra - sometimes results/runs need to be re-run for whatever reason.
2) triage - Crappy test case or something that is real?
3) kernel - bisecting etc

We donât have huge dedicated teams for each category but likewise each 
has a team.

> -- 
> Cyril Hrubis
> chrubis@xxxxxxx