Re: [RFC PATCH 2/2] KUnit: KASAN Integration

From: Alan Maguire
Date: Tue Mar 03 2020 - 11:40:40 EST

Next message: Dmitry Safonov: "Re: [PATCH] sys/sysinfo: Respect boottime inside time namespace"
Previous message: Arnd Bergmann: "Re: About commit "io: change inX() to have their own IO barrier overrides""
In reply to: Dmitry Vyukov: "Re: [RFC PATCH 2/2] KUnit: KASAN Integration"
Next in thread: Patricia Alfonso: "Re: [RFC PATCH 2/2] KUnit: KASAN Integration"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Fri, 28 Feb 2020, Patricia Alfonso wrote:

> On Thu, Feb 27, 2020 at 6:04 AM Alan Maguire <alan.maguire@xxxxxxxxxx> wrote:
> >
> > On Wed, 26 Feb 2020, Patricia Alfonso wrote:
> >
> > > Integrate KASAN into KUnit testing framework.
> >
> > This is a great idea! Some comments/suggestions below...
> >
>
> Thank you so much for your suggestions!
>

No problem! Extending KUnit to test things like KASAN
is really valuable, as it shows us ways we can improve
the framework. More below...

> > > - Fail tests when KASAN reports an error that is not expected
> > > - Use KUNIT_EXPECT_KASAN_FAIL to expect a KASAN error in KASAN tests
> > > - KUnit struct added to current task to keep track of the current test
> > > from KASAN code
> > > - Booleans representing if a KASAN report is expected and if a KASAN
> > > report is found added to kunit struct
> > > - This prints "line# has passed" or "line# has failed"
> > >
> > > Signed-off-by: Patricia Alfonso <trishalfonso@xxxxxxxxxx>
> > > ---
> > > If anyone has any suggestions on how best to print the failure
> > > messages, please share!
> > >
> > > One issue I have found while testing this is the allocation fails in
> > > kmalloc_pagealloc_oob_right() sometimes, but not consistently. This
> > > does cause the test to fail on the KUnit side, as expected, but it
> > > seems to skip all the tests before this one because the output starts
> > > with this failure instead of with the first test, kmalloc_oob_right().
> > >
> > > include/kunit/test.h | 24 ++++++++++++++++++++++++
> > > include/linux/sched.h | 7 ++++++-
> > > lib/kunit/test.c | 7 ++++++-
> > > mm/kasan/report.c | 19 +++++++++++++++++++
> > > tools/testing/kunit/kunit_kernel.py | 2 +-
> > > 5 files changed, 56 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/include/kunit/test.h b/include/kunit/test.h
> > > index 2dfb550c6723..2e388f8937f3 100644
> > > --- a/include/kunit/test.h
> > > +++ b/include/kunit/test.h
> > > @@ -21,6 +21,8 @@ struct kunit_resource;
> > > typedef int (*kunit_resource_init_t)(struct kunit_resource *, void *);
> > > typedef void (*kunit_resource_free_t)(struct kunit_resource *);
> > >
> > > +void kunit_set_failure(struct kunit *test);
> > > +
> > > /**
> > > * struct kunit_resource - represents a *test managed resource*
> > > * @allocation: for the user to store arbitrary data.
> > > @@ -191,6 +193,9 @@ struct kunit {
> > > * protect it with some type of lock.
> > > */
> > > struct list_head resources; /* Protected by lock. */
> > > +
> > > + bool kasan_report_expected;
> > > + bool kasan_report_found;
> > > };
> > >
> >
> > Is this needed here? You're testing something pretty
> > specific so it seems wrong to add to the generic
> > kunit resource unless there's a good reason. I see the
> > code around setting these values in mm/kasan/report.c,
> > but I wonder if we could do something more generic.
> >
> > How about the concept of a static resource (assuming a
> > dynamically allocated one is out because it messes
> > with memory allocation tests)? Something like this:
> >
> > #define kunit_add_static_resource(test, resource_ptr, resource_field) \
> > do { \
> > spin_lock(&test->lock); \
> > (resource_ptr)->resource_field.init = NULL; \
> > (resource_ptr)->resource_field.free = NULL; \
> > list_add_tail(&(resource_ptr)->resource_field, \
> > &test->resources); \
> > spin_unlock(&test->lock); \
> > } while (0)
> >
> >
> > Within your kasan code you could then create a kasan-specific
> > structure that embends a kunit_resource, and contains the
> > values you need:
> >
> > struct kasan_report_resource {
> > struct kunit_resource res;
> > bool kasan_report_expected;
> > bool kasan_report_found;
> > };
> >
> > (One thing we'd need to do for such static resources is fix
> > kunit_resource_free() to check if there's a free() function,
> > and if not assume a static resource)
> >
> > If you then create an init() function associated with your
> > kunit suite (which will be run for every case) it can do this:
> >
> > int kunit_kasan_test_init(struct kunit *test)
> > {
> > kunit_add_static_resource(test, &my_kasan_report_resource, res);
> > ...
> > }
> >
> > The above should also be used to initialize current->kasan_unit_test
> > instead of doing that in kunit_try_run_case(). With those
> > changes, you don't (I think) need to change anything in core
> > kunit (assuming support for static resources).
> >
> > To retrieve the resource during tests or in kasan context, the
> > method seems to be to use kunit_resource_find(). However, that
> > requires a match function which seems a bit heavyweight for the
> > static case. We should probably have a default "find by name"
> > or similar function here, and add an optional "name" field
> > to kunit resources to simplify things. Anyway here you'd
> > use something like:
> >
> > kasan_report_resource = kunit_resource_find(test, matchfn,
> > NULL, matchdata);
> >
> >
> > Are there any barriers to taking this sort of approach (apart
> > from the support for static resources not being there yet)?
> >
>
> I'm not sure. I don't have any experience with kunit resources so I
> would have to put some more effort into understanding how this would
> work for myself. I wonder if this might be a bit of an over
> complicated way of eliminating an extraneous boolean... maybe we can
> find a simpler solution for the first version of this patch and add
> the notion of a static resource for generic use later.
>

My personal preference would be to try and learn what's needed
by KASAN and improve the KUnit APIs so the next developer finds
life a bit easier. More hassle for you I know, but actual use cases
like this are invaluable for improving the API. I've sent
out an RFC patchset which has the functionality I _think_ you
need but I may be missing something:

https://lore.kernel.org/linux-kselftest/1583251361-12748-1-git-send-email-alan.maguire@xxxxxxxxxx/T/#t

The idea is your test can do something like this:

struct kasan_data {
bool report_expected;
bool report_found;
};

my_kasan_test(struct kunit *test)
{
struct kunit_resource resource;
struct kasan_data kasan_data;

...
// add our named resource using static resource/data
kunit_add_named_resource(test, NULL, NULL, &resource,
"kasan_data", &kasan_data);
...

}

(The NULLs in the function arguments above reflect the fact we
don't require initialization or cleanup for such static resources)

Then, in KASAN context you can look the above resource up like so:

struct kunit_resource *resource;
struct kasan_data *kasan_data;

resource = kunit_find_named_resource(test, "kasan_data");
kasan_data = resource->data;

// when finished, reduce reference count on resource
kunit_put_resource(resource);

Does that work for your use case?

> > > void kunit_init_test(struct kunit *test, const char *name);
> > > @@ -941,6 +946,25 @@ do { \
> > > ptr, \
> > > NULL)
> > >
> > > +/**
> > > + * KUNIT_EXPECT_KASAN_FAIL() - Causes a test failure when the expression does
> > > + * not cause a KASAN error.
> > > + *
> > > + */
> > > +#define KUNIT_EXPECT_KASAN_FAIL(test, condition) do { \
> > > + test->kasan_report_expected = true; \
> > > + test->kasan_report_found = false; \
> > > + condition; \
> > > + if (test->kasan_report_found == test->kasan_report_expected) { \
> > > + pr_info("%d has passed", __LINE__); \
> > > + } else { \
> > > + kunit_set_failure(test); \
> > > + pr_info("%d has failed", __LINE__); \
> > > + } \
> > > + test->kasan_report_expected = false; \
> > > + test->kasan_report_found = false; \
> > > +} while (0)
> > > +
> >
> > Feels like this belongs in test_kasan.c, and could be reworked
> > to avoid adding test->kasan_report_[expected|found] as described
> > above.
>
> You're right. Since I don't see any reason why any other tests should
> want to expect a KASAN error, it does make sense to move this logic
> inside test_kasan.c. If, in the future, there is a need for this
> elsewhere, we can always move it back then.
>
> > Instead of having your own pass/fail logic couldn't you
> > do this:
> >
> > KUNIT_EXPECT_EQ(test, expected, found);
> >
> > ? That will set the failure state too so no need to export
> > a separate function for that, and no need to log anything
> > as KUNIT_EXPECT_EQ() should do that for you.
> >
>
> This is a great idea - I feel a little silly that I didn't think of
> that myself! Do we think the failure message for the KUNIT_EXPECT_EQ()
> would be sufficient for KASAN developers?
> i.e. "Expected kasan_report_expected == kasan_report_found, but
> kasan_report_expected == true
> kasan_report_found == false"
>

I guess the missing piece above is the line number where
the test failure was encountered, is that the concern?

> > > /**
> > > * KUNIT_EXPECT_TRUE() - Causes a test failure when the expression is not true.
> > > * @test: The test context object.
> > > diff --git a/include/linux/sched.h b/include/linux/sched.h
> > > index 04278493bf15..db23d56061e7 100644
> > > --- a/include/linux/sched.h
> > > +++ b/include/linux/sched.h
> > > @@ -32,6 +32,8 @@
> > > #include <linux/posix-timers.h>
> > > #include <linux/rseq.h>
> > >
> > > +#include <kunit/test.h>
> > > +
> >
> > This feels like the wrong place to add this #include, and
> > when I attempted to build to test I ran into a bunch of
> > compilation errors; for example:
> >
> > CC kernel/sched/core.o
> > In file included from ./include/linux/uaccess.h:11,
> > from ./arch/x86/include/asm/fpu/xstate.h:5,
> > from ./arch/x86/include/asm/pgtable.h:26,
> > from ./include/linux/kasan.h:16,
> > from ./include/linux/slab.h:136,
> > from ./include/kunit/test.h:16,
> > from ./include/linux/sched.h:35,
> > from init/do_mounts.c:3:
> > ./arch/x86/include/asm/uaccess.h: In function 'set_fs':
> > ./arch/x86/include/asm/uaccess.h:32:9: error: dereferencing pointer to
> > incomplete type 'struct task_struct'
> > current->thread.addr_limit = fs;
> >
> > (I'm testing with CONFIG_SLUB). Removing this #include
> > resolves these errors, but then causes problems for
> > lib/test_kasan.c. I'll dig around a bit more.
> >
>
> Yes, I was only testing with UML. Removing that #include fixed the
> problem for me for both x86 and UML. Could you share more about the
> errors you have encountered in lib/test_kasan.c?
>

I'll try this again and send details.

I think broadly the issue is that if we #include kunit headers
in the kasan headers, we end up creating all kinds of problems
for ourselves, since the kasan headers are in turn included
in so many places (including the kunit headers themselves, since
kunit uses memory allocation APIs). I suspect the way forward is
to try and ensure that we don't utilize the kunit headers in any
of the kasan headers, but rather just include kunit headers
in test_kasan.c, and any other kasan .c files we need KUnit APIs
for. Not sure if that's possible, but it's likely the best way to
go if it is.

> > > /* task_struct member predeclarations (sorted alphabetically): */
> > > struct audit_context;
> > > struct backing_dev_info;
> > > @@ -1178,7 +1180,10 @@ struct task_struct {
> > >
> > > #ifdef CONFIG_KASAN
> > > unsigned int kasan_depth;
> > > -#endif
> > > +#ifdef CONFIG_KUNIT
> > > + struct kunit *kasan_kunit_test;
> > > +#endif /* CONFIG_KUNIT */
> > > +#endif /* CONFIG_KASAN */
> > >
> > > #ifdef CONFIG_FUNCTION_GRAPH_TRACER
> > > /* Index of current stored address in ret_stack: */
> > > diff --git a/lib/kunit/test.c b/lib/kunit/test.c
> > > index 9242f932896c..d266b9495c67 100644
> > > --- a/lib/kunit/test.c
> > > +++ b/lib/kunit/test.c
> > > @@ -9,11 +9,12 @@
> > > #include <kunit/test.h>
> > > #include <linux/kernel.h>
> > > #include <linux/sched/debug.h>
> > > +#include <linux/sched.h>
> > >
> > > #include "string-stream.h"
> > > #include "try-catch-impl.h"
> > >
> > > -static void kunit_set_failure(struct kunit *test)
> > > +void kunit_set_failure(struct kunit *test)
> > > {
> > > WRITE_ONCE(test->success, false);
> > > }
> > > @@ -236,6 +237,10 @@ static void kunit_try_run_case(void *data)
> > > struct kunit_suite *suite = ctx->suite;
> > > struct kunit_case *test_case = ctx->test_case;
> > >
> > > +#ifdef CONFIG_KASAN
> > > + current->kasan_kunit_test = test;
> > > +#endif
> > > +
> > > /*
> > > * kunit_run_case_internal may encounter a fatal error; if it does,
> > > * abort will be called, this thread will exit, and finally the parent
> > > diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> > > index 5ef9f24f566b..5554d23799a5 100644
> > > --- a/mm/kasan/report.c
> > > +++ b/mm/kasan/report.c
> > > @@ -32,6 +32,8 @@
> > >
> > > #include <asm/sections.h>
> > >
> > > +#include <kunit/test.h>
> > > +
> > > #include "kasan.h"
> > > #include "../slab.h"
> > >
> > > @@ -461,6 +463,15 @@ void kasan_report_invalid_free(void *object, unsigned long ip)
> > > u8 tag = get_tag(object);
> > >
> > > object = reset_tag(object);
> > > +
> > > + if (current->kasan_kunit_test) {
> > > + if (current->kasan_kunit_test->kasan_report_expected) {
> > > + current->kasan_kunit_test->kasan_report_found = true;
> > > + return;
> > > + }
> > > + kunit_set_failure(current->kasan_kunit_test);
> > > + }
> > > +
> > > start_report(&flags);
> > > pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip);
> > > print_tags(tag, object);
> > > @@ -481,6 +492,14 @@ void __kasan_report(unsigned long addr, size_t size, bool is_write, unsigned lon
> > > if (likely(!report_enabled()))
> > > return;
> > >
> > > + if (current->kasan_kunit_test) {
> > > + if (current->kasan_kunit_test->kasan_report_expected) {
> > > + current->kasan_kunit_test->kasan_report_found = true;
> > > + return;
> > > + }
> > > + kunit_set_failure(current->kasan_kunit_test);
> > > + }
> > > +
> > > disable_trace_on_warning();
> > >
> > > tagged_addr = (void *)addr;
> > > diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
> > > index cc5d844ecca1..63eab18a8c34 100644
> > > --- a/tools/testing/kunit/kunit_kernel.py
> > > +++ b/tools/testing/kunit/kunit_kernel.py
> > > @@ -141,7 +141,7 @@ class LinuxSourceTree(object):
> > > return True
> > >
> > > def run_kernel(self, args=[], timeout=None, build_dir=''):
> > > - args.extend(['mem=256M'])
> > > + args.extend(['mem=256M', 'kasan_multi_shot'])
> > > process = self._ops.linux_bin(args, timeout, build_dir)
> > > with open(os.path.join(build_dir, 'test.log'), 'w') as f:
> > > for line in process.stdout:
> >
> > I tried applying this to the "kunit" branch of linux-kselftest, and
> > the above failed. Which branch are you building with? Probably
> > best to use the kunit branch I think. Thanks!
> >
> I believe I am on Torvalds/master. There was some debate as to which
> branch I should be developing on when I started, but it probably makes
> sense for me to move to the "kunit" branch.
>

I think for this case - given that we may need some new KUnit
functionality - that would be best. Thanks!

Alan

> > Alan
> >
> > > --
> > > 2.25.0.265.gbab2e86ba0-goog
> > >
> > >
>
> --
> Thank you for all your comments!
> Patricia Alfonso
>

Next message: Dmitry Safonov: "Re: [PATCH] sys/sysinfo: Respect boottime inside time namespace"
Previous message: Arnd Bergmann: "Re: About commit "io: change inX() to have their own IO barrier overrides""
In reply to: Dmitry Vyukov: "Re: [RFC PATCH 2/2] KUnit: KASAN Integration"
Next in thread: Patricia Alfonso: "Re: [RFC PATCH 2/2] KUnit: KASAN Integration"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]