Re: [PATCH 1/6] kunit: add example test case showing off all the expect macros

[Daniel Latypov]

On Mon, Jan 10, 2022 at 10:51 PM David Gow <davidgow@xxxxxxxxxx> wrote:
>
> On Sat, Jan 8, 2022 at 9:23 AM Daniel Latypov <dlatypov@xxxxxxxxxx> wrote:
> >
> > Currently, these macros are only really documented near the bottom of
> > https://www.kernel.org/doc/html/latest/dev-tools/kunit/api/test.html#c.KUNIT_FAIL.
> >
> > E.g. it's likely someone might just not realize that
> > KUNIT_EXPECT_STREQ() exists and instead use KUNIT_EXPECT_FALSE(strcmp())
> > or similar.
> >
> > This can also serve as a basic smoketest that the KUnit assert machinery
> > still works for all the macros.
> >
> > Signed-off-by: Daniel Latypov <dlatypov@xxxxxxxxxx>
> > ---
>
> I think this is a great idea. I will note that this definitely isn't a
> full test _of_ the assertion macros (in that it only exercises the
> success case), so keeping it as an example is probably best.
>
> A few possible ideas below, but I'm happy enough with this as-is regardless.

Applied the ideas locally.
It led to this diffstat
 lib/kunit/kunit-example-test.c | 42 +++++++++++++++++++-----------------------
 1 file changed, 19 insertions(+), 23 deletions(-)

So it's now a bit shorter and 8 of the added lines are new comments.

>
> Reviewed-by: David Gow <davidgow@xxxxxxxxxx>
>
> >  lib/kunit/kunit-example-test.c | 46 ++++++++++++++++++++++++++++++++++
> >  1 file changed, 46 insertions(+)
> >
> > diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c
> > index 51099b0ca29c..182a64c12541 100644
> > --- a/lib/kunit/kunit-example-test.c
> > +++ b/lib/kunit/kunit-example-test.c
> > @@ -69,6 +69,51 @@ static void example_mark_skipped_test(struct kunit *test)
> >         /* This line should run */
> >         kunit_info(test, "You should see this line.");
> >  }
> > +
> > +/*
> > + * This test shows off all the KUNIT_EXPECT macros.
> > + */
> > +static void example_all_expect_macros_test(struct kunit *test)
> > +{
> > +       KUNIT_EXPECT_TRUE(test, true);
> > +       KUNIT_EXPECT_FALSE(test, false);
>
> _Maybe_ it's worth having a comment for each of these groups ('boolean
> assertions', 'integer assertions', 'pointer assertions', etc)?

Good idea, done.

>
> > +
> > +       KUNIT_EXPECT_EQ(test, 1, 1);
> > +       KUNIT_EXPECT_GE(test, 1, 1);
> > +       KUNIT_EXPECT_LE(test, 1, 1);
> > +       KUNIT_EXPECT_NE(test, 1, 0);
> > +       KUNIT_EXPECT_GT(test, 1, 0);
> > +       KUNIT_EXPECT_LT(test, 0, 1);
> > +
> > +       KUNIT_EXPECT_NOT_ERR_OR_NULL(test, test);
> > +       KUNIT_EXPECT_PTR_EQ(test, NULL, NULL);
> > +       KUNIT_EXPECT_PTR_NE(test, test, NULL);
> > +
> > +       KUNIT_EXPECT_STREQ(test, "hi", "hi");
> > +       KUNIT_EXPECT_STRNEQ(test, "hi", "bye");
> > +
> > +       /*
> > +        * There are also _MSG variants of all of the above that let you include
> > +        * additional text on failure.
> > +        */
>
> There are also the ASSERT vs EXPECT variations. While it may be
> excessive to also include all of these, particularly in an example, it
> might be worth mentioning them in a comment somewhere?

I've gone ahead and added a section with one example

+       /*
+        * There are also ASSERT variants of all of the above that abort test
+        * execution if they fail. Useful for memory allocations, etc.
+        */
+       KUNIT_ASSERT_GT(test, sizeof(char), 0);
+


>
> Alternatively, if this is bloating the example too much, we could have
> only one example each of the ASSERT and _MSG variants.
>
> > +       KUNIT_EXPECT_TRUE_MSG(test, true, "msg");
> > +       KUNIT_EXPECT_FALSE_MSG(test, false, "msg");
>
> Part of me feels that a better message than "msg" would be nice to
> have here, but I can't think of a good one. Maybe (particularly for
> the less obvious integer/string/pointer macros below), having a
> description of what's being asserted?


I've gone ahead and added truncated this down to one example

+       KUNIT_EXPECT_GT_MSG(test, sizeof(int), 0, "Your ints are 0-bit?!");
+       KUNIT_ASSERT_GT_MSG(test, sizeof(int), 0, "Your ints are 0-bit?!");

>
>
>
> > +
> > +       KUNIT_EXPECT_EQ_MSG(test, 1, 1, "msg");
> > +       KUNIT_EXPECT_GE_MSG(test, 1, 1, "msg");
> > +       KUNIT_EXPECT_LE_MSG(test, 1, 1, "msg");
> > +       KUNIT_EXPECT_NE_MSG(test, 1, 0, "msg");
> > +       KUNIT_EXPECT_GT_MSG(test, 1, 0, "msg");
> > +       KUNIT_EXPECT_LT_MSG(test, 0, 1, "msg");
> > +
> > +       KUNIT_EXPECT_NOT_ERR_OR_NULL_MSG(test, test, "msg");
> > +       KUNIT_EXPECT_PTR_EQ_MSG(test, NULL, NULL, "msg");
> > +       KUNIT_EXPECT_PTR_NE_MSG(test, test, NULL, "msg");
> > +
> > +       KUNIT_EXPECT_STREQ_MSG(test, "hi", "hi", "msg");
> > +       KUNIT_EXPECT_STRNEQ_MSG(test, "hi", "bye", "msg");
> > +}
> > +
> >  /*
> >   * Here we make a list of all the test cases we want to add to the test suite
> >   * below.
> > @@ -83,6 +128,7 @@ static struct kunit_case example_test_cases[] = {
> >         KUNIT_CASE(example_simple_test),
> >         KUNIT_CASE(example_skip_test),
> >         KUNIT_CASE(example_mark_skipped_test),
> > +       KUNIT_CASE(example_all_expect_macros_test),
> >         {}
> >  };
> >
> > --
> > 2.34.1.575.g55b058a8bb-goog
> >