Re: [PATCH v1] rtla: Set distinctive exit value for failed tests

[Tomas Glozar]

Hi Costa,

Thanks for the patch, this is really useful, as we will be also able
to test whether rtla properly resets the threshold option (see below).

ne 30. 3. 2025 v 20:48 odesílatel Costa Shulyupin
<costa.shul@xxxxxxxxxx> napsal:
> Failed tests return the same exit value as passed tests,
> requiring test frameworks to validate results by parsing output.

Please specify in the commit message more precisely what you mean by
"failed tests". That is, when running rtla with threshold, the exit
code is the same, regardless of whether the threshold is violated or
not.

> Set a unique exit value for failed tests to
> allow integration with testing frameworks without parsing output.

Note that this is not only for the test framework, users might use this as well.

Also, users might be already relying on rtla exiting with 0 on
threshold violation. What do you think about adding a new command line
argument, `--fail-on-threshold`, that would allow the user (including
the test suite) to trigger the new behavior?

>@@ -20,6 +20,6 @@ check "verify the --entries/-E param" \
> # and stopping on threshold.
> # If default period is not set, this will time out.
> check_with_osnoise_options "apply default period" \
>-       "osnoise hist -s 1" period_us=600000000
>+       "osnoise hist -s 1" 2 period_us=600000000

In the future, we can add regression tests like this:

check_with_osnoise_options "verify setting default threshold" \
        "osnoise hist -d 10" 0  stop_tracing_us=1

This will detect failure (return code 2) if the fix in commit
0122938a7a ("rtla: Always set all tracer options") no longer works.

Tomas