Re: riscv syscall performance regression

From: Alexandre Ghiti
Date: Tue Aug 13 2024 - 08:51:26 EST


Hi Fei,

On 23/02/2024 06:28, Wu, Fei wrote:
Hi All,

I am doing some performance regression testing on a sophgo machine, the
unixbench syscall benchmark drops 14% from 6.1 to 6.6. This change
should be due to commit f0bddf50 riscv: entry: Convert to generic entry.
I know it's a tradeoff, just checking if it's been discussed already and
any improvement can be done.

The unixbench benchmark I used is:
$ ./syscall 10 getpid

The dynamic instruction count per syscall is increased from ~200 to
~250, this should be the key factor so I switch to test it on system
QEMU to avoid porting different versions on sophgo, and use plugin
libinsn.so to count the instructions. There are a few background noises
during test but the impact should be limited. This is dyninst count per
syscall I got:

* commit d0db02c6 (right before the change): ~200
* commit f0bddf50 (the change): ~250
* commit ffd2cb6b (latest upstream): ~250

Any comment?

Thanks,
Fei.

_______________________________________________
linux-riscv mailing list
linux-riscv@xxxxxxxxxxxxxxxxxxx
http://lists.infradead.org/mailman/listinfo/linux-riscv


So I finally took some time to look into this. Indeed the conversion to the generic entry introduced the overhead you observe.

The numbers I get are similar:

* commit d0db02c6 (right before the change): 185

*  6.11-rc3: 245

I dived a bit deeper and noticed that we could regain ~40 instructions by inlining syscall_exit_to_user_mode() and do_trap_ecall_u():

- we used to intercept the syscall trap but now it's dealt with in the exception vector, not sure if we can inline do_trap_ecall_u()
- I quickly tried to inline syscall_exit_to_user_mode() but it pulls quite a few functions and I failed to do so.

Note that a recent effort already inlined most of the common entry functions already https://lore.kernel.org/all/20231218074520.1998026-1-svens@xxxxxxxxxxxxx/

The remaining instructions are caused by:

* the vector extension handling. It won't improve the above numbers because the test does not use the vector extension, but we could improve __riscv_v_vstate_discard() as mentioned in commit 9657e9b7d253 ("riscv: Discard vector state on syscalls")
* the random kernel stack offset

I'll add some performance regressions in my CI in the near future :)

Thanks,

Alex