Re: [PATCH 1/2] selftests/kmod: increase the kmod timeout from 45 to 165

[Shuah Khan]

On 2/27/23 15:42, Luis Chamberlain wrote:
> On Mon, Feb 27, 2023 at 03:32:50PM -0700, Shuah Khan wrote:
> > On 2/6/23 16:43, Luis Chamberlain wrote:
> > > The default sefltests timeout is 45 seconds. If you run the kmod
> > > selftests on your own with say:
> > >
> > > ./tools/testings/selftests/kmod.sh
> > >
> > > Then the default timeout won't be in effect.
> > >
> > > I've never ran kmod selftests using the generic make wrapper
> > > (./tools/testing/selftests/run_kselftest.sh -s) util now
> > > that I have support for it on kdevops [0]. And with that the
> > > test is limitted to the default timeout which we quickly run
> > > into. Bump this up to what I see is required on 8GiB / 8 vcpu
> > > libvirt q35 guest as can be easily created now with kdevops.
> > >
> > > To run selftests with kdevops:
> > >
> > > make menuconfig # enable dedicated selftests and kmod test
> > > make
> > > make bringup
> > > make linux
> > > make selftests-kmod
> > >
> > > This ends up taking about 280 seconds now, give or take add
> > > 50 seconds more more and we end up with 350. Document the
> > > rationale.
> > >
> > > [0] https://github.com/linux-kdevops/kdevops
> > > Signed-off-by: Luis Chamberlain <mcgrof@xxxxxxxxxx>
> > > ---
> > >    tools/testing/selftests/kmod/settings | 4 ++++
> > >    1 file changed, 4 insertions(+)
> > >    create mode 100644 tools/testing/selftests/kmod/settings
> > >
> > > diff --git a/tools/testing/selftests/kmod/settings b/tools/testing/selftests/kmod/settings
> > > new file mode 100644
> > > index 000000000000..6fca0f1a4594
> > > --- /dev/null
> > > +++ b/tools/testing/selftests/kmod/settings
> > > @@ -0,0 +1,4 @@
> > > +# measured from a manual run:
> > > +# time ./tools/testing/selftests/kmod/kmod.sh
> > > +# Then add ~50 seconds more gracetime.
> > > +timeout=350
> >
> > Adding timeouts like this for individual tests increases the overall kselftest
> > run-time. I am not in favor of adding timeouts.
> >
> > We have to find a better way to do this.
>
> Well if folks don't have this the test will fail, and so a false
> positive. If the goal is to have a low time timeout for "do not run
> tests past this time and do not fail if we stopped the test" then
> that seems to be likely one way to go and each test may need to be
> modified to not fail fatally in case of a special signal.

We are finding more and more that timeout values are requiring
tweaks. I am in favor of coming up a way to exit the test with
a timeout condition.

thanks,
-- Shuah