Re: [Freedreno] Adding CI results to the kernel tree was Re: [RFC v2] drm/msm: Add initial ci/ subdirectory

[Rob Clark]

On Wed, May 11, 2022 at 12:12 PM Linus Torvalds
<torvalds@xxxxxxxxxxxxxxxxxxxx> wrote:
>
> On Wed, May 11, 2022 at 12:08 PM Linus Torvalds
> <torvalds@xxxxxxxxxxxxxxxxxxxx> wrote:
> >
> > The kernel tree might have just the expected *failures* listed, if
> > there are any. Presumably the ci tree has to have the expected results
> > anyway, so what's the advantage of listing non-failures?
>
> .. put another way: I think a list of "we are aware that these
> currently fail" is quite reasonable for a development tree, maybe even
> with a comment in the commit that created them about why they
> currently fail.
>
> That also ends up being very nice if you fix a problem, and the fix
> commit might then remove the failure for the list, and that all makes
> perfect sense.
>
> But having just the raw output of "these are the expected CI results"
> that is being done and specified by some other tree entirely - that
> seems pointless and just noise to me. There's no actual reason to have
> that kind of noise - and update that kind of noise - that I really
> see.

Yeah, the only reason we have full results is that the current tool to
check for pass/fail of the entire CI job is 'diff' ;-)

It has the nice benefit of generating a patch for you to squash into
whatever commit to update the expectation files, I suppose.  But we
have something more clever on the mesa-ci side of things where we list
skips/flakes/expected-fails but not expected-passes.  To be fair, the
# of tests on the mesa side is something on the order of 750,000, I
don't expect to ever get close to that # on the kernel side.

BR,
-R

>
>                 Linus