Re: [PATCH net-next v6 09/11] seccomp: Enhance test_harness with an assert step mechanism

[Kees Cook]

On Tue, Mar 28, 2017 at 4:46 PM, MickaÃl SalaÃn <mic@xxxxxxxxxxx> wrote:
> This is useful to return an information about the error without being
> able to write to TH_LOG_STREAM.
>
> Helpers from test_harness.h may be useful outside of the seccomp
> directory.
>
> Signed-off-by: MickaÃl SalaÃn <mic@xxxxxxxxxxx>
> Cc: Andy Lutomirski <luto@xxxxxxxxxxxxxx>
> Cc: Arnaldo Carvalho de Melo <acme@xxxxxxxxxx>
> Cc: Kees Cook <keescook@xxxxxxxxxxxx>
> Cc: Shuah Khan <shuah@xxxxxxxxxx>
> Cc: Will Drewry <wad@xxxxxxxxxxxx>
> ---
>  tools/testing/selftests/seccomp/test_harness.h | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/seccomp/test_harness.h b/tools/testing/selftests/seccomp/test_harness.h
> index a786c69c7584..77e407663e06 100644
> --- a/tools/testing/selftests/seccomp/test_harness.h
> +++ b/tools/testing/selftests/seccomp/test_harness.h
> @@ -397,7 +397,7 @@ struct __test_metadata {
>         const char *name;
>         void (*fn)(struct __test_metadata *);
>         int termsig;
> -       int passed;
> +       __s8 passed;

Why the reduction here? int is signed too?

>         int trigger; /* extra handler after the evaluation */
>         struct __test_metadata *prev, *next;
>  };
> @@ -476,6 +476,12 @@ void __run_test(struct __test_metadata *t)
>                                         "instead of by signal (code: %d)\n",
>                                         t->name,
>                                         WEXITSTATUS(status));
> +                       } else if (t->passed < 0) {
> +                               fprintf(TH_LOG_STREAM,
> +                                       "%s: Failed at step #%d\n",
> +                                       t->name,
> +                                       t->passed * -1);
> +                               t->passed = 0;
>                         }

Instead of creating an overloaded mechanism here, perhaps have an
option reporting mechanism that can be enabled. Like adding to
__test_metadata "bool no_stream; int test_number;" and adding
test_number++ to each ASSERT/EXCEPT call, and doing something like:

if (t->no_stream) {
                              fprintf(TH_LOG_STREAM,
                                      "%s: Failed at step #%d\n",
                                      t->name,
                                       t->test_number);
}

It'd be a cleaner approach, maybe?

-Kees

-- 
Kees Cook
Pixel Security