Re: [PATCH v7 1/4] spinlock: A new lockref structure for locklessupdate of refcount

From: Sedat Dilek
Date: Tue Sep 03 2013 - 19:45:17 EST

On Wed, Sep 4, 2013 at 12:37 AM, Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> On Sun, Sep 1, 2013 at 5:32 PM, Linus Torvalds
> <torvalds@xxxxxxxxxxxxxxxxxxxx> wrote:
>> On Sun, Sep 1, 2013 at 3:01 AM, Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
>>>
>>> Looks like this is now 10x faster: ~2.66Mloops (debug) VS.
>>> ~26.60Mloops (no-debug).
>>
>> Ok, that's getting to be in the right ballpark.
>>
>> But your profile is still odd.
>>
>>> Samples: 159K of event 'cycles:pp', Event count (approx.): 76968896763
>>> 12,79% t_lockref_from- [kernel.kallsyms] [k] irq_return
>>> 4,36% t_lockref_from- [kernel.kallsyms] [k] __ticket_spin_lock
>>
>> If you do the profile with "-g", what are the top callers of this? You
>> shouldn't see any spinlock load from the path lookup, but you have all
>> these other things going on..
>>
>>> 4,36% t_lockref_from- [kernel.kallsyms] [k] __acct_update_integrals
>>> 4,07% t_lockref_from- [kernel.kallsyms] [k] user_exit
>>> 3,12% t_lockref_from- [kernel.kallsyms] [k] local_clock
>>> 2,83% t_lockref_from- [kernel.kallsyms] [k] lockref_get_or_lock
>>> 2,73% t_lockref_from- [kernel.kallsyms] [k] kmem_cache_alloc
>>> 2,62% t_lockref_from- [kernel.kallsyms] [k] __d_lookup_rcu
>>
>> You're spending more time on the task stats than on the actual lookup.
>> Maybe you should turn off CONFIG_TASKSTATS..But why that whole
>> irq_return thing? Odd.
>>
>
> [ init/Kconfig ]
> ...
> config TASKSTATS
> bool "Export task/process statistics through netlink"
> depends on NET <--- Difficult to disable it?!
> default n
> help
> Export selected statistics for tasks/processes through the
> generic netlink interface. Unlike BSD process accounting, the
> statistics are available during the lifetime of tasks/processes as
> responses to commands. Like BSD accounting, they are sent to user
> space on task exit.
>
> Say N if unsure.
> ...
>

So with Dave J.'s help I disabled CONFIG_TASKSTATS.

But I still see that odd irq_ret* thing.

My kernel-config and patch (on top of Linux v3.11) are attached.

- Sedat -

$ sudo ~/src/linux-kernel/linux/tools/perf/perf report -tui

Samples: 161K of event 'cycles:pp', Event count (approx.): 76595555357
13,30%uit_lockref_from-ui[kernel.kallsyms] ui[k] irq_ret.rn
5,25%uit_lockref_from-ui[kernel.kallsyms] ui[k] lockref_get_or_lock
4,82%uit_lockref_from-ui[kernel.kallsyms] ui[k] __ticket_spin_lock
4,23%uit_lockref_from-ui[kernel.kallsyms] ui[k] .ser_exit
3,17%uit_lockref_from-ui[kernel.kallsyms] ui[k] local_clock
2,98%uit_lockref_from-ui[kernel.kallsyms] ui[k] kmem_cache_alloc
2,61%uit_lockref_from-uilibc-2.15.so ui[.] __xstat64
2,55%uit_lockref_from-ui[kernel.kallsyms] ui[k] link_path_walk
2,49%uit_lockref_from-ui[kernel.kallsyms] ui[k] kmem_cache_free
2,03%uit_lockref_from-ui[kernel.kallsyms] ui[k] tracesys
2,01%uit_lockref_from-ui[kernel.kallsyms] ui[k] path_look.pat
1,99%uit_lockref_from-ui[kernel.kallsyms] ui[k] rc._eqs_exit_common.isra.43
1,94%uit_lockref_from-ui[kernel.kallsyms] ui[k] .ser_enter
1,92%uit_lockref_from-ui[kernel.kallsyms] ui[k] rc._eqs_enter_common.isra.45
1,86%uit_lockref_from-ui[kernel.kallsyms] ui[k] sched_clock_cp.
1,72%uit_lockref_from-ui[kernel.kallsyms] ui[k] __d_look.p_rc.
1,71%uit_lockref_from-ui[kernel.kallsyms] ui[k] native_read_tsc
1,68%uit_lockref_from-ui[kernel.kallsyms] ui[k] cp_new_stat
1,65%uit_lockref_from-ui[kernel.kallsyms] ui[k] lockref_p.t_or_lock
1,64%uit_lockref_from-ui[kernel.kallsyms] ui[k] look.p_fast
1,59%uit_lockref_from-ui[kernel.kallsyms] ui[k] path_init
1,58%uit_lockref_from-ui[kernel.kallsyms] ui[k] native_sched_clock
1,57%uit_lockref_from-ui[kernel.kallsyms] ui[k] copy_.ser_generic_.nrolled
1,56%uit_lockref_from-ui[kernel.kallsyms] ui[k] syscall_trace_enter
1,53%uit_lockref_from-ui[kernel.kallsyms] ui[k] acco.nt_system_time
1,34%uit_lockref_from-ui[kernel.kallsyms] ui[k] getname_flags
1,28%uit_lockref_from-ui[kernel.kallsyms] ui[k] get_vtime_delta
1,26%uit_lockref_from-ui[kernel.kallsyms] ui[k] vfs_getattr
1,14%uit_lockref_from-ui[kernel.kallsyms] ui[k] syscall_trace_leave
1,11%uit_lockref_from-ui[kernel.kallsyms] ui[k] system_call
1,06%uit_lockref_from-ui[kernel.kallsyms] ui[k] strncpy_from_.ser
1,01%uit_lockref_from-ui[kernel.kallsyms] ui[k] generic_fillattr
0,97%uit_lockref_from-ui[kernel.kallsyms] ui[k] acco.nt_.ser_time
0,96%uit_lockref_from-ui[kernel.kallsyms] ui[k] .ser_path_at_empty
0,92%uit_lockref_from-ui[kernel.kallsyms] ui[k] filename_look.p
0,88%uit_lockref_from-ui[kernel.kallsyms] ui[k] __ticket_spin_.nlock
0,87%uit_lockref_from-ui[kernel.kallsyms] ui[k] complete_walk
0,86%uit_lockref_from-ui[kernel.kallsyms] ui[k] generic_permission
0,86%uit_lockref_from-ui[kernel.kallsyms] ui[k] common_perm
0,82%uit_lockref_from-ui[kernel.kallsyms] ui[k] vfs_fstatat
0,77%uit_lockref_from-ui[kernel.kallsyms] ui[k] rc._eqs_enter
0,75%uit_lockref_from-ui[kernel.kallsyms] ui[k] vtime_acco.nt_.ser
0,72%uit_lockref_from-ui[kernel.kallsyms] ui[k] __inode_permission
0,65%uit_lockref_from-ui[kernel.kallsyms] ui[k] dp.t
0,62%uit_lockref_from-ui[kernel.kallsyms] ui[k] vtime_.ser_enter
0,59%uit_lockref_from-ui[kernel.kallsyms] ui[k] apparmor_inode_getattr
0,55%uit_lockref_from-ui[kernel.kallsyms] ui[k] lg_local_lock
0,53%uit_lockref_from-ui[kernel.kallsyms] ui[k] __vtime_acco.nt_system
0,51%uit_lockref_from-ui[kernel.kallsyms] ui[k] sec.rity_inode_permission
0,51%uit_lockref_from-ui[kernel.kallsyms] ui[k] mntp.t
0,47%uit_lockref_from-uit_lockref_from-lin.sui[.] start_ro.tine
0,45%uit_lockref_from-ui[kernel.kallsyms] ui[k] cp.acct_acco.nt_field
0,45%uit_lockref_from-ui[kernel.kallsyms] ui[k] int_with_check
0,44%uit_lockref_from-ui[kernel.kallsyms] ui[k] rc._eqs_exit
Press '?' for help on key bindings

Attachment: config-3.11.0-2-lockref-small
Description: Binary data

Attachment: 3.11.0-2-lockref-small.patch
Description: Binary data