Re: [PATCH rcu 13/14] workqueue: Make queue_rcu_work() use call_rcu_flush()

From: Uladzislau Rezki
Date: Mon Oct 24 2022 - 14:28:19 EST

On Mon, Oct 24, 2022 at 06:55:16PM +0200, Uladzislau Rezki wrote:
> On Mon, Oct 24, 2022 at 09:48:19AM -0700, Paul E. McKenney wrote:
> > On Mon, Oct 24, 2022 at 06:25:30PM +0200, Uladzislau Rezki wrote:
> > > >
> > > > You guys might need to agree on the definition of "good" here. Or maybe
> > > > understand the differences in your respective platforms' definitions of
> > > > "good". ;-)
> > > >
> > > Indeed. Bad is when once per-millisecond infinitely :) At least in such use
> > > workload a can detect a power delta and power gain. Anyway, below is a new
> > > trace where i do not use "flush" variant for the kvfree_rcu():
> > >
> > > <snip>
> > > 1. Home screen swipe:
> > > rcuop/0-15 [003] d..1 1792.767750: rcu_batch_start: rcu_preempt CBs=1003 bl=10
> > > rcuop/2-33 [002] d..1 1792.771717: rcu_batch_start: rcu_preempt CBs=934 bl=10
> > > rcuop/3-40 [001] d..1 1794.811816: rcu_batch_start: rcu_preempt CBs=1508 bl=11
> > > rcuop/1-26 [003] d..1 1797.116382: rcu_batch_start: rcu_preempt CBs=2127 bl=16
> > > rcuop/4-48 [001] d..1 1797.124422: rcu_batch_start: rcu_preempt CBs=95 bl=10
> > > rcuop/5-55 [002] d..1 1797.124731: rcu_batch_start: rcu_preempt CBs=143 bl=10
> > > rcuop/6-62 [005] d..1 1798.911719: rcu_batch_start: rcu_preempt CBs=132 bl=10
> > > rcuop/2-33 [002] d..1 1803.003966: rcu_batch_start: rcu_preempt CBs=3797 bl=29
> > > rcuop/0-15 [003] d..1 1803.004707: rcu_batch_start: rcu_preempt CBs=2969 bl=23
> > > 2. App launches:
> > > rcuop/4-48 [005] d..1 1831.087612: rcu_batch_start: rcu_preempt CBs=6141 bl=47
> > > rcuop/7-69 [007] d..1 1831.095578: rcu_batch_start: rcu_preempt CBs=5464 bl=42
> > > rcuop/5-55 [004] d..1 1832.703571: rcu_batch_start: rcu_preempt CBs=8461 bl=66
> > > rcuop/0-15 [004] d..1 1833.731603: rcu_batch_start: rcu_preempt CBs=2548 bl=19
> > > rcuop/1-26 [006] d..1 1833.743691: rcu_batch_start: rcu_preempt CBs=2567 bl=20
> > > rcuop/2-33 [006] d..1 1833.744005: rcu_batch_start: rcu_preempt CBs=2359 bl=18
> > > rcuop/3-40 [006] d..1 1833.744286: rcu_batch_start: rcu_preempt CBs=3681 bl=28
> > > rcuop/4-48 [002] d..1 1838.079777: rcu_batch_start: rcu_preempt CBs=10444 bl=81
> > > rcuop/7-69 [001] d..1 1838.080375: rcu_batch_start: rcu_preempt CBs=12572 bl=98
> > > <...>-62 [002] d..1 1838.080646: rcu_batch_start: rcu_preempt CBs=14135 bl=110
> > > rcuop/6-62 [000] d..1 1838.087722: rcu_batch_start: rcu_preempt CBs=10839 bl=84
> > > <...>-62 [003] d..1 1839.227022: rcu_batch_start: rcu_preempt CBs=1834 bl=14
> > > <...>-26 [001] d..1 1839.963315: rcu_batch_start: rcu_preempt CBs=5769 bl=45
> > > rcuop/2-33 [001] d..1 1839.966485: rcu_batch_start: rcu_preempt CBs=3789 bl=29
> > > <...>-40 [001] d..1 1839.966596: rcu_batch_start: rcu_preempt CBs=6425 bl=50
> > > rcuop/2-33 [005] d..1 1840.541272: rcu_batch_start: rcu_preempt CBs=825 bl=10
> > > rcuop/2-33 [005] d..1 1840.547724: rcu_batch_start: rcu_preempt CBs=44 bl=10
> > > rcuop/2-33 [005] d..1 1841.075759: rcu_batch_start: rcu_preempt CBs=516 bl=10
> > > rcuop/0-15 [002] d..1 1841.695716: rcu_batch_start: rcu_preempt CBs=6312 bl=49
> > > rcuop/0-15 [003] d..1 1841.709714: rcu_batch_start: rcu_preempt CBs=39 bl=10
> > > rcuop/5-55 [004] d..1 1843.112442: rcu_batch_start: rcu_preempt CBs=16007 bl=125
> > > rcuop/5-55 [004] d..1 1843.115444: rcu_batch_start: rcu_preempt CBs=7901 bl=61
> > > rcuop/6-62 [001] dn.1 1843.123983: rcu_batch_start: rcu_preempt CBs=8427 bl=65
> > > rcuop/6-62 [006] d..1 1843.412383: rcu_batch_start: rcu_preempt CBs=981 bl=10
> > > rcuop/0-15 [003] d..1 1844.659812: rcu_batch_start: rcu_preempt CBs=1851 bl=14
> > > rcuop/0-15 [003] d..1 1844.667790: rcu_batch_start: rcu_preempt CBs=135 bl=10
> > > <snip>
> > >
> > > it is much more better. But. As i wrote earlier there is a patch that i have submitted
> > > some time ago improving kvfree_rcu() batching:
> > >
> > > <snip>
> > > commit 51824b780b719c53113dc39e027fbf670dc66028
> > > Author: Uladzislau Rezki (Sony) <urezki@xxxxxxxxx>
> > > Date: Thu Jun 30 18:33:35 2022 +0200
> > >
> > > rcu/kvfree: Update KFREE_DRAIN_JIFFIES interval
> > >
> > > Currently the monitor work is scheduled with a fixed interval of HZ/20,
> > > which is roughly 50 milliseconds. The drawback of this approach is
> > > low utilization of the 512 page slots in scenarios with infrequence
> > > kvfree_rcu() calls. For example on an Android system:
> > > <snip>
> > >
> > > The trace that i posted was taken without it.
> >
> > And if I am not getting too confused, that patch is now in mainline.
> > So it does make sense to rely on it, then. ;-)
> >
> Right.
>
> urezki@pc638:~/data/raid0/coding/linux.git$ git tag --contains
> 51824b780b719c53113dc39e027fbf670dc66028
> v6.1-rc1
> v6.1-rc2
> urezki@pc638:~/data/raid0/coding/linux.git$
>
Just in case: 5.10 + "rcu/kvfree: Update KFREE_DRAIN_JIFFIES interval":

<snip>
1. Home screen swipe:
rcuop/3-40 [003] d..1 94.202849: rcu_batch_start: rcu_preempt CBs=664 bl=10
rcuop/4-48 [001] d..1 95.999352: rcu_batch_start: rcu_preempt CBs=252 bl=10
rcuop/6-62 [002] d..1 97.534875: rcu_batch_start: rcu_preempt CBs=152 bl=10
rcuop/5-55 [003] d..1 98.042912: rcu_batch_start: rcu_preempt CBs=189 bl=10
rcuop/0-15 [002] d..1 98.306769: rcu_batch_start: rcu_preempt CBs=1457 bl=11
rcuop/1-26 [000] d..1 99.582931: rcu_batch_start: rcu_preempt CBs=2115 bl=16
rcuop/2-33 [003] d..1 99.582935: rcu_batch_start: rcu_preempt CBs=2019 bl=15
rcuop/3-40 [001] d..1 99.838885: rcu_batch_start: rcu_preempt CBs=1168 bl=10
rcuop/1-26 [000] d..1 100.603496: rcu_batch_start: rcu_preempt CBs=168 bl=10
2. Apps launches:
rcuop/4-48 [007] d..1 102.910580: rcu_batch_start: rcu_preempt CBs=1150 bl=10
rcuop/6-62 [007] d..1 102.910682: rcu_batch_start: rcu_preempt CBs=1001 bl=10
rcuop/5-55 [007] d..1 103.166607: rcu_batch_start: rcu_preempt CBs=939 bl=10
rcuop/0-15 [007] d..1 104.450598: rcu_batch_start: rcu_preempt CBs=1694 bl=13
rcuop/5-55 [006] d..1 104.478640: rcu_batch_start: rcu_preempt CBs=3125 bl=24
rcuop/3-40 [007] d..1 104.958565: rcu_batch_start: rcu_preempt CBs=1108 bl=10
rcuop/7-69 [007] d..1 106.238634: rcu_batch_start: rcu_preempt CBs=10275 bl=80
rcuop/4-48 [007] d..1 107.258586: rcu_batch_start: rcu_preempt CBs=8142 bl=63
rcuop/7-69 [007] d..1 107.260769: rcu_batch_start: rcu_preempt CBs=1880 bl=14
rcuop/2-33 [007] d..1 107.526638: rcu_batch_start: rcu_preempt CBs=1968 bl=15
rcuop/1-26 [007] d..1 107.542612: rcu_batch_start: rcu_preempt CBs=1796 bl=14
rcuop/5-55 [007] d..1 108.286588: rcu_batch_start: rcu_preempt CBs=3547 bl=27
rcuop/6-62 [007] d..1 108.287639: rcu_batch_start: rcu_preempt CBs=5820 bl=45
rcuop/7-69 [007] d..1 108.290548: rcu_batch_start: rcu_preempt CBs=2430 bl=18
rcuop/0-15 [000] d..1 109.826843: rcu_batch_start: rcu_preempt CBs=2282 bl=17
rcuop/3-40 [002] d..1 110.595455: rcu_batch_start: rcu_preempt CBs=1960 bl=15
<...>-48 [005] d..1 112.390702: rcu_batch_start: rcu_preempt CBs=5143 bl=40
rcuop/7-69 [004] d..1 112.402607: rcu_batch_start: rcu_preempt CBs=3379 bl=26
rcuop/2-33 [005] d..1 112.638614: rcu_batch_start: rcu_preempt CBs=3223 bl=25
rcuop/1-26 [004] d..1 112.638617: rcu_batch_start: rcu_preempt CBs=3026 bl=23
rcuop/5-55 [007] d..1 113.402581: rcu_batch_start: rcu_preempt CBs=7251 bl=56
rcuop/6-62 [007] d..1 113.658582: rcu_batch_start: rcu_preempt CBs=7035 bl=54
<snip>

--
Uladzislau Rezki