Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
From: Evgenii Shatokhin
Date: Tue Jan 17 2023 - 11:22:57 EST
On 17.01.2023 16:16, Evgenii Shatokhin wrote:
Hi, Song,
On 17.01.2023 12:32, Song Shuai wrote:
Hi, Evgenii:
Evgenii Shatokhin <e.shatokhin@xxxxxxxxx> 于2023年1月16日周一 14:30写道：
Hi,
On 12.01.2023 12:06, guoren@xxxxxxxxxx wrote:
From: Song Shuai <suagrfillet@xxxxxxxxx>
select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
the ftrace-direct*.c files in samples/ftrace/.
Signed-off-by: Song Shuai <suagrfillet@xxxxxxxxx>
Tested-by: Guo Ren <guoren@xxxxxxxxxx>
Signed-off-by: Guo Ren <guoren@xxxxxxxxxx>
---
arch/riscv/Kconfig | 2 ++
samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
samples/ftrace/ftrace-direct-multi-modify.c | 37
+++++++++++++++++++++
samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
samples/ftrace/ftrace-direct.c | 22 ++++++++++++
6 files changed, 142 insertions(+)
The samples were built OK now, but ftrace-direct-multi and
ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
the messages in the trace:
# TASK-PID CPU# ||||| TIMESTAMP FUNCTION
# | | | ||||| | |
migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
direct func1 ip 0
migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
direct func1 ip ff60000001ba9600
migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
direct func1 ip ff60000003334d00
migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
direct func1 ip 0
rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
direct func1 ip 0
insmod-415 [000] ..... 3858.532270: my_direct_func1: my
direct func1 ip 7fffffffffffffff
<idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
direct func1 ip ff60000001ba9600
<idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
direct func1 ip ff60000001ba9600
rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
direct func1 ip 0
[...]
If I understand it right, my_direct_func1() should print the address of
some location in the code, probably - at the beginning of the traced
functions.
The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
addresses.
The invalid code address is only printed by accessing the schedule()
function's first argument whose address stores in a0 register.
While schedule() actually has no parameter declared, so my_direct_func
just prints the a0 in the context of the schedule()'s caller and
the address maybe varies depending on the caller.
I can't really understand why tracing the first argument of the
schedule() function, but it seems nonsense at this point.
The question is, what should be passed as the argument(s) of
my_direct_func() in this particular sample module. The kernel docs and
commit logs seem to contain no info on that.
With direct functions, I suppose, the trampoline can pass anything it
wants to my_direct_func(), not just the arguments of the traced function.
I'd check what these sample modules do on x86 and would try to match
that behaviour on RISC-V.
I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
be passed to my_direct_func().
ftrace-direct-multi.ko:
# TASK-PID CPU# ||||| TIMESTAMP FUNCTION
# | | | ||||| | |
insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
ffffffff87332f45 // wake_up_process+0x5
rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
ffffffff8828d935 // schedule+0x5
insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
ffffffff8828d935
systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
ffffffff8828d935
systemd-1 [000] ..... 1719.519141: my_direct_func: ip
ffffffff8828d935
<idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
ffffffff87332f45
<idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
ffffffff87332f45
rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
ffffffff8828d935
[...]
The ip values are wake_up_process+0x5 and schedule+0x5, the locations
where the execution of the traced functions resumes after the Ftrace
trampoline has finished.
The results with ftrace-direct-multi-modify.ko are similar to that.
The samples look like a demonstration, that one can pass anything
necessary to the handler in case of "direct" functions.
I suppose, the RISC-V-specific asm code in these two sample modules
could be updated to pass the saved pc value to my_direct_func() in a0.
As for this patch, it just impls a simple mcount (direct_caller) to
trace kernel functions, and basically saves the necessary ABI,
call the tracing function, and restores the ABI, just like other
arches do.
so It shouldn't be blamed.
I started an independent patch to replace schedule with kick_process
to make these samples more reasonable. And It has no conflict with the
current patch, so we can go on.
Link:
https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@xxxxxxxxx/T/#u
The same issue is with ftrace-direct-multi-modify.ko.
Is anything missing here?
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 307a9f413edd..e944af44f681 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -112,6 +112,8 @@ config RISCV
select HAVE_POSIX_CPU_TIMERS_TASK_WORK
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_FUNCTION_ARG_ACCESS_API
+ select HAVE_SAMPLE_FTRACE_DIRECT
+ select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_RSEQ
diff --git a/samples/ftrace/ftrace-direct-modify.c
b/samples/ftrace/ftrace-direct-modify.c
index de5a0f67f320..be7bf472c3c7 100644
--- a/samples/ftrace/ftrace-direct-modify.c
+++ b/samples/ftrace/ftrace-direct-modify.c
@@ -23,6 +23,39 @@ extern void my_tramp2(void *);
static unsigned long my_ip = (unsigned long)schedule;
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp1, @function\n"
+" .globl my_tramp1\n"
+" my_tramp1:\n"
+" addi sp,sp,-16\n"
+" sd t0,0(sp)\n"
+" sd ra,8(sp)\n"
+" call my_direct_func1\n"
+" ld t0,0(sp)\n"
+" ld ra,8(sp)\n"
+" addi sp,sp,16\n"
+" jr t0\n"
+" .size my_tramp1, .-my_tramp1\n"
+
+" .type my_tramp2, @function\n"
+" .globl my_tramp2\n"
+" my_tramp2:\n"
+" addi sp,sp,-16\n"
+" sd t0,0(sp)\n"
+" sd ra,8(sp)\n"
+" call my_direct_func2\n"
+" ld t0,0(sp)\n"
+" ld ra,8(sp)\n"
+" addi sp,sp,16\n"
+" jr t0\n"
+" .size my_tramp2, .-my_tramp2\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
b/samples/ftrace/ftrace-direct-multi-modify.c
index d52370cad0b6..10884bf418f7 100644
--- a/samples/ftrace/ftrace-direct-multi-modify.c
+++ b/samples/ftrace/ftrace-direct-multi-modify.c
@@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
extern void my_tramp1(void *);
extern void my_tramp2(void *);
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp1, @function\n"
+" .globl my_tramp1\n"
+" my_tramp1:\n"
+" addi sp,sp,-24\n"
+" sd a0,0(sp)\n"
+" sd t0,8(sp)\n"
+" sd ra,16(sp)\n"
+" call my_direct_func1\n"
+" ld a0,0(sp)\n"
+" ld t0,8(sp)\n"
+" ld ra,16(sp)\n"
+" addi sp,sp,24\n"
+" jr t0\n"
+" .size my_tramp1, .-my_tramp1\n"
+
+" .type my_tramp2, @function\n"
+" .globl my_tramp2\n"
+" my_tramp2:\n"
+" addi sp,sp,-24\n"
+" sd a0,0(sp)\n"
+" sd t0,8(sp)\n"
+" sd ra,16(sp)\n"
+" call my_direct_func2\n"
+" ld a0,0(sp)\n"
+" ld t0,8(sp)\n"
+" ld ra,16(sp)\n"
+" addi sp,sp,24\n"
+" jr t0\n"
+" .size my_tramp2, .-my_tramp2\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct-multi.c
b/samples/ftrace/ftrace-direct-multi.c
index ec1088922517..a35bf43bf6d7 100644
--- a/samples/ftrace/ftrace-direct-multi.c
+++ b/samples/ftrace/ftrace-direct-multi.c
@@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
extern void my_tramp(void *);
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp, @function\n"
+" .globl my_tramp\n"
+" my_tramp:\n"
+" addi sp,sp,-24\n"
+" sd a0,0(sp)\n"
+" sd t0,8(sp)\n"
+" sd ra,16(sp)\n"
+" call my_direct_func\n"
+" ld a0,0(sp)\n"
+" ld t0,8(sp)\n"
+" ld ra,16(sp)\n"
+" addi sp,sp,24\n"
+" jr t0\n"
+" .size my_tramp, .-my_tramp\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct-too.c
b/samples/ftrace/ftrace-direct-too.c
index e13fb59a2b47..3b62e33c2e6d 100644
--- a/samples/ftrace/ftrace-direct-too.c
+++ b/samples/ftrace/ftrace-direct-too.c
@@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
extern void my_tramp(void *);
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp, @function\n"
+" .globl my_tramp\n"
+" my_tramp:\n"
+" addi sp,sp,-40\n"
+" sd a0,0(sp)\n"
+" sd a1,8(sp)\n"
+" sd a2,16(sp)\n"
+" sd t0,24(sp)\n"
+" sd ra,32(sp)\n"
+" call my_direct_func\n"
+" ld a0,0(sp)\n"
+" ld a1,8(sp)\n"
+" ld a2,16(sp)\n"
+" ld t0,24(sp)\n"
+" ld ra,32(sp)\n"
+" addi sp,sp,40\n"
+" jr t0\n"
+" .size my_tramp, .-my_tramp\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct.c
b/samples/ftrace/ftrace-direct.c
index 1f769d0db20f..2cfe5a7d2d70 100644
--- a/samples/ftrace/ftrace-direct.c
+++ b/samples/ftrace/ftrace-direct.c
@@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
extern void my_tramp(void *);
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp, @function\n"
+" .globl my_tramp\n"
+" my_tramp:\n"
+" addi sp,sp,-24\n"
+" sd a0,0(sp)\n"
+" sd t0,8(sp)\n"
+" sd ra,16(sp)\n"
+" call my_direct_func\n"
+" ld a0,0(sp)\n"
+" ld t0,8(sp)\n"
+" ld ra,16(sp)\n"
+" addi sp,sp,24\n"
+" jr t0\n"
+" .size my_tramp, .-my_tramp\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
--
2.36.1
--
Thanks,
Song
Regards,
Evgenii