Re: [RFC PATCH v1 00/11] Create fast idle path for short idle periods

From: Li, Aubrey
Date: Mon Jul 17 2017 - 09:25:02 EST


On 2017/7/14 12:05, Paul E. McKenney wrote:
>
> More specifically: rcu_needs_cpu(), rcu_prepare_for_idle(),
> rcu_cleanup_after_idle(), rcu_eqs_enter(), rcu_eqs_enter_common(),
> rcu_dynticks_eqs_enter(), do_nocb_deferred_wakeup(),
> rcu_dynticks_task_enter(), rcu_eqs_exit(), rcu_eqs_exit_common(),
> rcu_dynticks_task_exit(), rcu_dynticks_eqs_exit().
>
> The first three (rcu_needs_cpu(), rcu_prepare_for_idle(), and
> rcu_cleanup_after_idle()) should not be significant unless you have
> CONFIG_RCU_FAST_NO_HZ=y. If you do, it would be interesting to learn
> how often invoke_rcu_core() is invoked from rcu_prepare_for_idle()
> and rcu_cleanup_after_idle(), as this can raise softirq. Also
> rcu_accelerate_cbs() and rcu_try_advance_all_cbs().
>
> Knowing which of these is causing the most trouble might help me
> reduce the overhead in the current idle path.

I measured two cases, nothing notable found.

The one is CONFIG_NO_HZ_IDLE=y, so the following function is just empty.

rcu_prepare_for_idle(): NULL
rcu_cleanup_after_idle(): NULL
do_nocb_deferred_wakeup(): NULL
rcu_dynticks_task_enter(): NULL
rcu_dynticks_task_exit(): NULL

And the following functions are traced separately, for each function I
traced 3 times by intel_PT, for each time the sampling period is 1-second.
num means the times the function is invoked in 1-second. (min, max, avg) is
the function duration, the unit is nano-second.

rcu_needs_cpu():
1) num: 6110 min: 3 max: 564 avg: 17.0
2) num: 16535 min: 3 max: 683 avg: 18.0
3) num: 8815 min: 3 max: 394 avg: 20.0

rcu_eqs_enter():
1) num: 7956 min: 17 max: 656 avg: 32.0
2) num: 9170 min: 17 max: 1075 avg: 35.0
3) num: 8604 min: 17 max: 859 avg: 29.0

rcu_eqs_enter_common():
1) num: 14676 min: 15 max: 620 avg: 28.0
2) num: 11180 min: 15 max: 795 avg: 30.0
3) num: 11484 min: 15 max: 725 avg: 29.0

rcu_dynticks_eqs_enter():
1) num: 11035 min: 10 max: 580 avg: 17.0
2) num: 15518 min: 10 max: 456 avg: 16.0
3) num: 15320 min: 10 max: 454 avg: 19.0

rcu_eqs_exit():
1) num: 11080 min: 14 max: 893 avg: 23.0
2) num: 13526 min: 14 max: 640 avg: 23.0
3) num: 12534 min: 14 max: 630 avg: 22.0

rcu_eqs_exit_common():
1) num: 18002 min: 12 max: 553 avg: 17.0
2) num: 10570 min: 11 max: 485 avg: 17.0
3) num: 13628 min: 11 max: 567 avg: 16.0

rcu_dynticks_eqs_exit():
1) num: 11195 min: 11 max: 436 avg: 16.0
2) num: 11808 min: 10 max: 506 avg: 16.0
3) num: 8132 min: 10 max: 546 avg: 15.0

==============================================================================

The other case is CONFIG_NO_HZ_FULL, I also enabled the required config to make
all the functions not empty.

rcu_needs_cpu():
1) num: 8530 min: 5 max: 770 avg: 13.0
2) num: 9965 min: 5 max: 518 avg: 12.0
3) num: 12503 min: 5 max: 755 avg: 16.0

rcu_prepare_for_idle():
1) num: 11662 min: 5 max: 684 avg: 9.0
2) num: 15294 min: 5 max: 676 avg: 9.0
3) num: 14332 min: 5 max: 524 avg: 9.0

rcu_cleanup_after_idle():
1) num: 13584 min: 4 max: 657 avg: 6.0
2) num: 9102 min: 4 max: 529 avg: 5.0
3) num: 10648 min: 4 max: 471 avg: 6.0

rcu_eqs_enter():
1) num: 14222 min: 26 max: 745 avg: 54.0
2) num: 12502 min: 26 max: 650 avg: 53.0
3) num: 11834 min: 26 max: 863 avg: 52.0

rcu_eqs_enter_common():
1) num: 16792 min: 24 max: 973 avg: 43.0
2) num: 19755 min: 24 max: 898 avg: 45.0
3) num: 8167 min: 24 max: 722 avg: 42.0

rcu_dynticks_eqs_enter():
1) num: 11605 min: 10 max: 532 avg: 14.0
2) num: 10438 min: 9 max: 554 avg: 14.0
3) num: 19816 min: 9 max: 701 avg: 14.0

do_nocb_deferred_wakeup():
1) num: 15348 min: 1 max: 459 avg: 3.0
2) num: 12822 min: 1 max: 564 avg: 4.0
3) num: 8272 min: 0 max: 458 avg: 3.0

rcu_dynticks_task_enter():
1) num: 6358 min: 1 max: 268 avg: 1.0
2) num: 11128 min: 1 max: 360 avg: 1.0
3) num: 20516 min: 1 max: 214 avg: 1.0

rcu_eqs_exit():
1) num: 16117 min: 20 max: 782 avg: 43.0
2) num: 11042 min: 20 max: 775 avg: 47.0
3) num: 16499 min: 20 max: 752 avg: 46.0

rcu_eqs_exit_common():
1) num: 12584 min: 17 max: 703 avg: 28.0
2) num: 17412 min: 17 max: 759 avg: 28.0
3) num: 16733 min: 17 max: 798 avg: 29.0

rcu_dynticks_task_exit():
1) num: 11730 min: 1 max: 528 avg: 4.0
2) num: 18840 min: 1 max: 581 avg: 5.0
3) num: 9815 min: 1 max: 381 avg: 4.0

rcu_dynticks_eqs_exit():
1) num: 10902 min: 9 max: 557 avg: 13.0
2) num: 19474 min: 9 max: 563 avg: 13.0
3) num: 11865 min: 9 max: 672 avg: 12.0

Please let me know if there is some data not reasonable, I can revisit again.

Thanks,
-Aubrey