Re: [PATCH v2] selftests: livepatch: test if ftrace can trace a livepatched function

From: Petr Mladek
Date: Wed Jan 15 2025 - 10:33:07 EST

Next message: Jens Axboe: "Re: futex+io_uring: futex_q::task can maybe be dangling (but is not actually accessed, so it's fine)"
Previous message: Frank Li: "Re: [PATCH v5 3/3] arm64: dts: s32g399a-rdb3: Add INA231 sensor entry over I2C4"
In reply to: Joe Lawrence: "Re: [PATCH v2] selftests: livepatch: test if ftrace can trace a livepatched function"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hi Filipe,

first, thanks for the test. It is good. I just suggest a lot of clean
up ;-)

On Sat 2025-01-11 15:42:27, Filipe Xavier wrote:
> This new test makes sure that ftrace can trace a
> function that was introduced by a livepatch.
> ---
> Changes in v2:
> - functions.sh: added reset tracing on push and pop_config.
> - test-ftrace.sh: enabled tracing_on before test init.
> - nitpick: added double quotations on filenames and fixed some wording.
> - Link to v1: https://lore.kernel.org/r/20250102-ftrace-selftest-livepatch-v1-1-84880baefc1b@xxxxxxxxx
> ---
> tools/testing/selftests/livepatch/functions.sh | 14 ++++++++++
> tools/testing/selftests/livepatch/test-ftrace.sh | 33 ++++++++++++++++++++++++
> 2 files changed, 47 insertions(+)
>
> diff --git a/tools/testing/selftests/livepatch/functions.sh b/tools/testing/selftests/livepatch/functions.sh
> index e5d06fb402335d85959bafe099087effc6ddce12..e6c13514002dae5f8d7461f90b8241ab43024ea4 100644
> --- a/tools/testing/selftests/livepatch/functions.sh
> +++ b/tools/testing/selftests/livepatch/functions.sh
> @@ -62,6 +62,9 @@ function push_config() {
> awk -F'[: ]' '{print "file " $1 " line " $2 " " $4}')
> FTRACE_ENABLED=$(sysctl --values kernel.ftrace_enabled)
> KPROBE_ENABLED=$(cat "$SYSFS_KPROBES_DIR/enabled")
> + TRACING_ON=$(cat "$SYSFS_DEBUG_DIR/tracing/tracing_on")

The $SYSFS_DEBUG_DIR/tracing directory is used many times. It would
make sense to define:

SYSFS_TRACING_DIR="$SYSFS_DEBUG_DIR/tracing"

and use:

TRACING_ON=$(cat "$SYSFS_TRACING_DIR/tracing_on")

> + CURRENT_TRACER=$(cat "$SYSFS_DEBUG_DIR/tracing/current_tracer")
> + FTRACE_FILTER=$(cat "$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter")
> }
>
> function pop_config() {
> @@ -74,6 +77,17 @@ function pop_config() {
> if [[ -n "$KPROBE_ENABLED" ]]; then
> echo "$KPROBE_ENABLED" > "$SYSFS_KPROBES_DIR/enabled"
> fi
> + if [[ -n "$TRACING_ON" ]]; then
> + echo "$TRACING_ON" > "$SYSFS_DEBUG_DIR/tracing/tracing_on"
> + fi
> + if [[ -n "$CURRENT_TRACER" ]]; then
> + echo "$CURRENT_TRACER" > "$SYSFS_DEBUG_DIR/tracing/current_tracer"
> + fi
> + if [[ "$FTRACE_FILTER" == *"#"* ]]; then
> + echo > "$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter"

My understanding is that this is handling the situation when the
filter is:

#### all functions enabled ####

I was curious about the pattern *"#"*. The quotation looked pretty
non-standard to me. It made me to read the bash documentation and
the rules for extressions in [[ ]] looked quite complicated.

I wonder if we could make this more for "dummies". The following
seems to work:

if [[ -n "$FTRACE_FILTER" ]]; then
echo "$FTRACE_FILTER"
| sed -e "/#### all functions enabled ####/d"
> "$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter"
fi

Note that the string "#### all functions enabled ####" is a very
stable thing. It has never been modified since it was introduced
by the commit 0c75a3ed633419d75d ("ftrace: state that all functions
are enabled in set_ftrace_filter") in Feb 2009.

The string is used also in ftrace selftests, see
tools/testing/selftests/ftrace/test.d/ftrace/func_set_ftrace_file.tc

> + elif [[ -n "$FTRACE_FILTER" ]]; then
> + echo "$FTRACE_FILTER" > "$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter"
> + fi
> }
>
> function set_dynamic_debug() {
> diff --git a/tools/testing/selftests/livepatch/test-ftrace.sh b/tools/testing/selftests/livepatch/test-ftrace.sh
> index fe14f248913acbec46fb6c0fec38a2fc84209d39..66af5d726c52e48e5177804e182b4ff31784d5ac 100755
> --- a/tools/testing/selftests/livepatch/test-ftrace.sh
> +++ b/tools/testing/selftests/livepatch/test-ftrace.sh
> @@ -61,4 +61,37 @@ livepatch: '$MOD_LIVEPATCH': unpatching complete
> % rmmod $MOD_LIVEPATCH"
>
>
> +# - verify livepatch can load
> +# - check if traces have a patched function
> +# - unload livepatch and reset trace
> +
> +start_test "trace livepatched function and check that the live patch remains in effect"
> +
> +TRACE_FILE="$SYSFS_DEBUG_DIR/tracing/trace"
> +FUNCTION_NAME="livepatch_cmdline_proc_show"
> +
> +load_lp $MOD_LIVEPATCH
> +
> +echo 1 > "$SYSFS_DEBUG_DIR/tracing/tracing_on"
> +echo $FUNCTION_NAME > "$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter"

The above mentioned ftrace selftest shows that the behavior gets
more complicated when "traceoff" is used:

1. The "traceoff" entries are not cleared by the above command.
2. The "traceoff" entries might cause that the tracing will
stop before we finish this test.

> +echo "function" > "$SYSFS_DEBUG_DIR/tracing/current_tracer"
> +echo "" > "$TRACE_FILE"

I suggest to change the ordering and make sure that nothing is running
before we start:

# Start with the clean up
echo 0 > "$SYSFS_TRACING_DIR/tracing_on"
echo '!*:traceoff' > $SYSFS_TRACING_DIR/set_ftrace_filter
echo "" > "$SYSFS_TRACING_DIR/current_tracer"
echo "" > $SYSFS_TRACING_DIR/trace
# Setup and start our tracing
echo "function" > "$SYSFS_TRACING_DIR/current_tracer"
echo $FUNCTION_NAME > "$SYSFS_TRACING_DIR/set_ftrace_filter"
echo 1 > "$SYSFS_TRACING_DIR/tracing_on"

I suggest to this into helper functions which might be used
by more tests:

# cleanup_tracing() - stop and clean up function tracing
function cleanup_tracing() {
echo 0 > "$SYSFS_TRACING_DIR/tracing_on"
echo '!*:traceoff' > $SYSFS_TRACING_DIR/set_ftrace_filter
echo "" > "$SYSFS_TRACING_DIR/current_tracer"
echo "" > $SYSFS_TRACING_DIR/trace
}

# trace_function(function) - start tracing of a function
# function - to be traced function
function trace_function() {
local function="$1"; shift

cleanup_tracing

echo "function" > "$SYSFS_TRACING_DIR/current_tracer"
echo "$functions" > $SYSFS_TRACING_DIR/set_ftrace_filter
echo 1 > "$SYSFS_TRACING_DIR/tracing_on"
}

# check_traced_function(function) - check whether function appeared in trace log
# function - to be traced function
function check_trace_function() {
local function="$1"; shift

if ! grep -q "$function" $SYSFS_TRACING_DIR/trace" ; then
die "Function ($function) did not appear in the trace"
fi
}

As Marcos suggested, it would make sense to add the above mentioned
helper functions in a separate patch.

> +
> +if [[ "$(cat /proc/cmdline)" != "$MOD_LIVEPATCH: this has been live patched" ]] ; then
> + echo -e "FAIL\n\n"
> + die "livepatch kselftest(s) failed"
> +fi
> +
> +grep -q $FUNCTION_NAME "$TRACE_FILE"
> +FOUND=$?

We could replace this with the above proposed

check_traced_function "$FUNCTION_NAME"

Also I would personally stop the tracing after this sub-test finishes.
We might use the cleanup_tracing function here.

cleanup_tracing

> +
> +disable_lp $MOD_LIVEPATCH
> +unload_lp $MOD_LIVEPATCH
> +
> +if [ "$FOUND" -eq 1 ]; then
> + echo -e "FAIL\n\n"
> + die "livepatch kselftest(s) failed"
> +fi

As mentioned by Marcos, we should also print "ok" on success.
I would personally use the "check_result" function here. It
could catch non-expected entries in dmesg.

> +
> exit 0

Best Regards,
Petr

Next message: Jens Axboe: "Re: futex+io_uring: futex_q::task can maybe be dangling (but is not actually accessed, so it's fine)"
Previous message: Frank Li: "Re: [PATCH v5 3/3] arm64: dts: s32g399a-rdb3: Add INA231 sensor entry over I2C4"
In reply to: Joe Lawrence: "Re: [PATCH v2] selftests: livepatch: test if ftrace can trace a livepatched function"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]