Re: [PATCH] kunit/kunit_kernel: Rebuild .config if .kunitconfig is modified

[shuah]

On 2/5/20 3:09 PM, Russell Currey wrote:
> On Thu, Feb 6, 2020, at 7:00 AM, Brendan Higgins wrote:
> > On Wed, Feb 5, 2020 at 9:58 AM David Gow <davidgow@xxxxxxxxxx> wrote:
> > > One thing we'd like to do with kunit_tool is to make its functionality
> > > a bit more independent: in particular, allowing the configuration,
> > > running the kernel, and parsing the results to be done independently.
> > >
> > > If that's the case, it may make sense for "kunit.py run" or similar to
> > > not do anything with the .config, and to relegate that to a separate
> > > "configuration" step, which would allow someone to modify the
> > > configuration themselves (e.g., using make menuconfig) and re-run the
> > > tests, but also allow the config to be explicitly regenerated when
> > > helpful.
> > >
> > > Exactly what that'd end up looking like (and to what extent we'd still
> > > want to support a single command that'd do both) are still up in the
> > > air: but I think a general "separation of concerns" like this is
> > > probably the right path forward for kunit_tool.
> >
> > You and I have talked about splitting up kunit_tool's functionality
> > before. I agree with the idea.
> >
> > I imagine it that we would have
> >
> > - configuration
> > - running tests
> > - dmesg/TAP parsing
> >
> > as separate runnable scripts. I think that would make it a lot easier
> > for people with various test bed setups to reuse our code in their
> > test harness.
> >
> > Nevertheless, I think it would also be nice to have, as Ted has
> > previously suggested, a short easy to remember one line command that
> > just works; it is easily said, and much harder to do, but I think it
> > is at odds with the separation of functionality. I guess one solution
> > might just be to have these three separate tools, and then the classic
> > kunit.py script that combines the functionalities in a single step, or
> > as Ted suggested we could have some sort of default "make kunit"
> > command or something like that. I am not really sure what is best
> > here.
> >
> > It doesn't address the problem of separation of functionality in
> > anyway, but one way we could achieve the idea of having a command that
> > just works, is by putting a line in MAINTAINERS file entries that have
> > a command that a maintainer expects a submitter to run before sending
> > a patch to LKML. That might at least make it possible to hack together
> > a single line KUnit command for every relevant MAINTAINERS entry.
> > (Obviously there is no reason we have to do this particular idea just
> > for KUnit. We could do this for other tests as well.) Russel, I think
> > this was your idea at LCA?
>
> Hi Brendan, it wasn't me, it was someone in the audience during questions in my
> testing talk.  I don't recall who.
>
> They were suggesting a script like get_maintainers - i.e. get_tests - that for a
> given file/patch/commit it gives you a suggested set of tests, whether that's
> KUnit you can run there and then, or selftests you can run once it's booted,
> or maybe external test suites that are relevant.

I like this idea of get_tests type script that could be run separately
as well as part of check_patch or get_maintainers will serve as a
reminder or hint to patch submitter.

We have some pieces in the MAINTAINERS file now. Selftest files are
usually listed under subsystem entries. get_tests could leverage
that and we will definitely more information to for a complete set
of tests for a subsystem.

thanks,
-- Shuah