Re: [sched,rcu] b84c4e08143: +3.1% will-it-scale.per_thread_ops

From: Paul E. McKenney
Date: Thu Apr 17 2014 - 09:55:23 EST


On Thu, Apr 17, 2014 at 12:03:53PM +0800, Fengguang Wu wrote:
> Hi Paul,
>
> FYI, this improves will-it-scale/open1 throughput.

Cool! Not a planned benefit, but I will take it. ;-)

> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git dev.2014.04.14a
> commit b84c4e08143c98dad4b4d139f08db0b98b0d3ec4 ("sched,rcu: Make cond_resched() report RCU quiescent states")

But how should I read the data below? I see lots of positive percentages
and lots of negative percentages for the delta, and all near zero for
standard deviation. Is the overall improvement an average of these or
some such? What is being measured?

> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 563496 ~ 0% +3.1% 581059 ~ 0% nhm4/micro/will-it-scale/open1
> 563496 ~ 0% +3.1% 581059 ~ 0% TOTAL will-it-scale.per_thread_ops
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 756894 ~ 0% +2.8% 778452 ~ 0% nhm4/micro/will-it-scale/open1
> 756894 ~ 0% +2.8% 778452 ~ 0% TOTAL will-it-scale.per_process_ops
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 0.57 ~ 0% -2.7% 0.55 ~ 0% nhm4/micro/will-it-scale/open1
> 0.57 ~ 0% -2.7% 0.55 ~ 0% TOTAL will-it-scale.scalability
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 346764 ~ 2% -74.0% 90164 ~ 1% nhm4/micro/will-it-scale/open1
> 346764 ~ 2% -74.0% 90164 ~ 1% TOTAL slabinfo.kmalloc-256.active_objs
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 10837 ~ 2% -73.9% 2824 ~ 1% nhm4/micro/will-it-scale/open1
> 10837 ~ 2% -73.9% 2824 ~ 1% TOTAL slabinfo.kmalloc-256.active_slabs
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 10837 ~ 2% -73.9% 2824 ~ 1% nhm4/micro/will-it-scale/open1
> 10837 ~ 2% -73.9% 2824 ~ 1% TOTAL slabinfo.kmalloc-256.num_slabs
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 346821 ~ 2% -73.9% 90393 ~ 1% nhm4/micro/will-it-scale/open1
> 346821 ~ 2% -73.9% 90393 ~ 1% TOTAL slabinfo.kmalloc-256.num_objs
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 105961 ~ 1% -63.0% 39153 ~ 1% nhm4/micro/will-it-scale/open1
> 105961 ~ 1% -63.0% 39153 ~ 1% TOTAL meminfo.SUnreclaim
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 26432 ~ 1% -62.9% 9814 ~ 1% nhm4/micro/will-it-scale/open1
> 26432 ~ 1% -62.9% 9814 ~ 1% TOTAL proc-vmstat.nr_slab_unreclaimable
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 50298 ~ 0% +194.3% 148011 ~ 0% nhm4/micro/will-it-scale/open1
> 37020 ~ 0% +42.6% 52798 ~ 1% nhm4/micro/will-it-scale/signal1
> 87318 ~ 0% +130.0% 200809 ~ 0% TOTAL softirqs.RCU
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 140354 ~ 1% -47.6% 73490 ~ 0% nhm4/micro/will-it-scale/open1
> 140354 ~ 1% -47.6% 73490 ~ 0% TOTAL meminfo.Slab
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 77391 ~ 1% -46.7% 41235 ~ 2% nhm4/micro/will-it-scale/signal1
> 77391 ~ 1% -46.7% 41235 ~ 2% TOTAL cpuidle.C6-NHM.usage
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 19871 ~ 2% -37.6% 12397 ~ 2% nhm4/micro/will-it-scale/open1
> 18497 ~ 1% -37.5% 11556 ~ 1% nhm4/micro/will-it-scale/signal1
> 38368 ~ 2% -37.6% 23954 ~ 2% TOTAL softirqs.SCHED
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 1.24 ~ 4% -35.4% 0.80 ~ 3% nhm4/micro/will-it-scale/open1
> 1.24 ~ 4% -35.4% 0.80 ~ 3% TOTAL perf-profile.cpu-cycles.do_notify_resume.int_signal.close
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 1.43 ~ 4% +41.9% 2.03 ~ 4% nhm4/micro/will-it-scale/open1
> 1.43 ~ 4% +41.9% 2.03 ~ 4% TOTAL perf-profile.cpu-cycles.rcu_process_callbacks.__do_softirq.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 1.27 ~ 3% -30.0% 0.89 ~ 6% nhm4/micro/will-it-scale/open1
> 1.27 ~ 3% -30.0% 0.89 ~ 6% TOTAL perf-profile.cpu-cycles.setup_object.isra.46.new_slab.__slab_alloc.kmem_cache_alloc.get_empty_filp
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 1.54 ~ 7% +35.6% 2.09 ~ 8% nhm4/micro/will-it-scale/open1
> 1.54 ~ 7% +35.6% 2.09 ~ 8% TOTAL perf-profile.cpu-cycles.kmem_cache_alloc.getname_flags.getname.do_sys_open.sys_open
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 4.21 ~ 2% -29.1% 2.98 ~ 3% nhm4/micro/will-it-scale/open1
> 4.21 ~ 2% -29.1% 2.98 ~ 3% TOTAL perf-profile.cpu-cycles.link_path_walk.path_openat.do_filp_open.do_sys_open.sys_open
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 1.37 ~ 4% -23.1% 1.05 ~ 7% nhm4/micro/will-it-scale/open1
> 1.37 ~ 4% -23.1% 1.05 ~ 7% TOTAL perf-profile.cpu-cycles.__d_lookup_rcu.lookup_fast.link_path_walk.path_openat.do_filp_open
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 0.88 ~17% +29.1% 1.14 ~ 9% nhm4/micro/will-it-scale/open1
> 0.88 ~17% +29.1% 1.14 ~ 9% TOTAL perf-profile.cpu-cycles.path_init.path_openat.do_filp_open.do_sys_open.sys_open
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 0.67 ~16% +33.6% 0.90 ~10% nhm4/micro/will-it-scale/signal1
> 0.67 ~16% +33.6% 0.90 ~10% TOTAL perf-profile.cpu-cycles.restore_sigcontext.sys_rt_sigreturn.stub_rt_sigreturn.raise
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 3.19 ~ 1% +17.4% 3.74 ~ 5% nhm4/micro/will-it-scale/open1
> 3.19 ~ 1% +17.4% 3.74 ~ 5% TOTAL perf-profile.cpu-cycles.file_free_rcu.rcu_process_callbacks.__do_softirq.irq_exit.smp_apic_timer_interrupt
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 4329 ~ 7% +15.2% 4986 ~ 5% nhm4/micro/will-it-scale/open1
> 4329 ~ 7% +15.2% 4986 ~ 5% TOTAL slabinfo.vm_area_struct.active_objs
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 2536 ~ 1% -75.8% 614 ~ 9% nhm4/micro/will-it-scale/open1
> 2536 ~ 1% -75.8% 614 ~ 9% TOTAL time.involuntary_context_switches
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 32593 ~ 1% -62.1% 12349 ~ 2% nhm4/micro/will-it-scale/signal1
> 32593 ~ 1% -62.1% 12349 ~ 2% TOTAL interrupts.0:IO-APIC-edge.timer
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 4374 ~14% +124.7% 9830 ~ 8% nhm4/micro/will-it-scale/open1
> 2559 ~ 2% +20.3% 3078 ~ 3% nhm4/micro/will-it-scale/signal1
> 6934 ~ 9% +86.2% 12908 ~ 7% TOTAL interrupts.RES
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 490 ~ 1% -37.3% 307 ~ 1% nhm4/micro/will-it-scale/signal1
> 490 ~ 1% -37.3% 307 ~ 1% TOTAL vmstat.system.cs
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 1639 ~ 0% -8.8% 1495 ~ 0% nhm4/micro/will-it-scale/signal1
> 1639 ~ 0% -8.8% 1495 ~ 0% TOTAL vmstat.system.in
>
> ad86a04266f9b49 b84c4e08143c98dad4b4d139f
> --------------- -------------------------
> 419046 ~ 0% -1.5% 412661 ~ 0% nhm4/micro/will-it-scale/open1
> 400635 ~ 0% -5.9% 376866 ~ 0% nhm4/micro/will-it-scale/signal1
> 819681 ~ 0% -3.7% 789527 ~ 0% TOTAL interrupts.LOC
>
>
> Legend:
> ~XX% - stddev percent
> [+-]XX% - change percent
>
>
> time.involuntary_context_switches
>
> 3500 ++------------------------------------------------------------------+
> | .*.. |
> 3000 ++ .*. *..*.. .*..*.. .*.. |
> *..*..*..*. *. * |
> | *..*.. .*.. .*..* |
> 2500 ++ *..*. *..*. |
> | |
> 2000 ++ |
> | |
> 1500 ++ |
> | |
> | |
> 1000 ++ |
> | O O O O O O O O O
> 500 O+-O-----------O--O--O--O--------O-O--O--O--O--O--------O-----O--O--+
>
>
> [*] bisect-good sample
> [O] bisect-bad sample

So the good case increases involuntary context switches, but helps something
else? Or does the benefit stem from increased involuntary context switches
and thus less time spinning or some such?

Thanx, Paul

> Thanks,
> Fengguang
>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/