Re: QA: Monitor Linux log messages as port of release (candidate) testing

[Guenter Roeck]

Hi Paul,

On 9/7/21 8:18 AM, Paul Menzel wrote:
> Dear Guenter,
>
>
> Am 07.09.21 um 16:47 schrieb Guenter Roeck:
>
> > On Tue, Sep 07, 2021 at 03:50:39PM +0200, Paul Menzel wrote:
>
> > > Am 07.09.21 um 14:53 schrieb Guenter Roeck:
> > > > On Tue, Sep 07, 2021 at 10:40:31AM +0200, Paul Menzel wrote:
> > >
> > > > > Thank you for testing release candidates and releases [1]. Is your test
> > > > > setup documented somewhere?
> > > > Not really, except its source is available at github:
> > > >     https://github.com/groeck/linux-build-test
> > >
> > > Thank you.
> > >
> > > > > If not happening already, could the Linux messages (at least up to log level
> > > > > warning) also be monitored? For example, in Linux 5.14, a new warning snuck
> > > > > in by cefc7ca462 (ACPI: PRM: implement OperationRegion handler for the
> > > > > PlatformRtMechanism subtype) [2], which could have been caught early on, and
> > > > > fixed before the release.
> > > > >
> > > > > The test summaries would then also notify about possible behavior change.
> > > >
> > > > Logs are available and can be examined at kerneltests.org/builders.
> > >
> > > Sorry for being blind. Under *qemu-tests*, looking at build #1831 [1],
> > > clicking on *stdio* [2] under *Steps and Logfiles*, I do not see any Linux
> > > logs.
> > >
> > > > Reports are generated manually, so it would be way too much effort to add
> > > > build warnings to those. Besides, logs are way too noisy to be useful in a
> > > > summary e-mail.
> > >
> > > Just to avoid misunderstandings, it’s about the Linux run-time logs.
> >
> > Run-time logs are only provided if there are errors or runtime issues
> > (crashes, warning tracebacks, or test failures).
>
> Could this be changed to always publish them? Or is that too demanding on storage?

It isn't a matter of storage, but of noise. It is mostly me looking at the data,
and I really don't want to see logs if nothing interesting is there. With 450+
builds per release I'd drown in noise otherwise. Sure, a large part of that
is a UI issue, but that is where systems like kernelci come in. If I ever
find the time to do it, I might report build and runtime results/logs to kernelci.
But that is a big IF.

> > > > Also, Geert's build reports already provide build warnings and errors.
> > > > The same applies to reports sent by 0-day. Indeed, I do see at least
> > > > one 0-day report against commit cefc7ca46235.
> > >
> > > How can I find that report?
> > I just searched for the SHA.
> >
> > https://www.spinics.net/lists/linux-acpi/msg101721.html
>
> Alright, that is a build time thing. I am looking for runtime things.

If there is nothing in my reports, you'd probably not find what you
are looking for (it would be reported if it results in a backtrace).

Guenter