Re: Converting kselftest test modules to kunit

[Shuah Khan]

On 7/29/24 01:55, Muhammad Usama Anjum wrote:
> On 7/27/24 12:35 AM, Shuah Khan wrote:
> > On 7/15/24 04:09, Muhammad Usama Anjum wrote:
> > > Hi Kees and All,
> > >
> > > There are several tests in kselftest subsystem which load modules to tests
> > > the internals of the kernel. Most of these test modules are just loaded by
> > > the kselftest, their status isn't read and reported to the user logs. Hence
> > > they don't provide benefit of executing those tests.
> > >
> > > I've found patches from Kees where he has been converting such kselftests
> > > to kunit tests [1]. The probable motivation is to move tests output of
> > > kselftest subsystem which only triggers tests without correctly reporting
> > > the results. On the other hand, kunit is there to test the kernel's
> > > internal functions which can't be done by userspace.
> > >
> > > Kselftest:    Test user facing APIs from userspace
> > > Kunit:        Test kernel's internal functions from kernelspace
> > >
> > > This brings me to conclusion that kselftest which are loading modules to
> > > test kernelspace should be converted to kunit tests. I've noted several
> > > such kselftests.
> > >
> > > This is just my understanding. Please mention if I'm correct above or more
> > > reasons to support kselftest test modules transformation into kunit test.
> > >
> > > [1] https://lore.kernel.org/all/20221018082824.never.845-kees@xxxxxxxxxx/
> >
> > Please make sure you aren't taking away the ability to run these tests during
> > boot.
> The kunit tests are usually run at boot time. They can be run later as
> well. So I'm not trying to remove any functionality. Rather adding a way
> where failures would actually be detected.
>
> > It doesn't make sense to convert every single test especially when it
> > is intended to be run during boot without dependencies - not as a kunit test
> > but a regression test during boot.
> I started investigating when these lib kselftests were just loading the
> test module without checking if test actually passed/failed (which proves
> that this type of kselftests can never detect regression as it doesn't
> process the results). It was strange. Hence I found out the conversion of
> such kselftests to kunit tests done by Kees and started this thread for
> discussion before doing actual work and sending patches.
>
> >

I explained this on two other threads. If you don't know how this test
is used why are you making this change?

> > bitmap is one example - pay attention to the config help test - bitmap
> > one clearly states it runs regression testing during boot. Any test that
> > says that isn't a candidate for conversion.
> Please can you explain what do you mean by bitmap testing during boot? As
> far as I understand, the kselftests are usespace tests and are run after
> kernel has booted completely and userspace has started. It cannot be
> regarded as testing during boot.

I responded to David - let me know you would like me to repeat it.

thanks,
-- Shuah