Re: [RFC][PATCH 00/13] percpu rwsem -v2

[Ingo Molnar]

* Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:

> On Tue, Jun 23, 2015 at 07:50:12PM +0200, Peter Zijlstra wrote:
> > On Tue, Jun 23, 2015 at 04:56:39PM +0200, Daniel Wagner wrote:
> > > flock02
> > >                              mean   variance      sigma        max        min
> > >                     tip-1    11.8994     0.5874     0.7664    13.2022     8.6324
> > >                     tip-2    11.7394     0.5252     0.7247    13.2540     9.7513
> > >                     tip-3    11.8155     0.5288     0.7272    13.2700     9.9480
> > >        tip+percpu-rswem-1    15.3601     0.8981     0.9477    16.8116    12.6910
> > >        tip+percpu-rswem-2    15.2558     0.8442     0.9188    17.0199    12.9586
> > >        tip+percpu-rswem-3    15.5297     0.6386     0.7991    17.4392    12.7992
> > 
> > I did indeed manage to get flock02 down to a usable level and found:
> 
> Aside from the flock_lock_file function moving up, we also get an
> increase in _raw_spin_lock.
> 
> Before:
> 
>      5.17%     5.17%  flock02       [kernel.vmlinux]            [k] _raw_spin_lock
>                  |
>                  ---_raw_spin_lock
>                     |          
>                     |--99.75%-- flock_lock_file_wait
>                     |          sys_flock
>                     |          entry_SYSCALL_64_fastpath
>                     |          flock
>                      --0.25%-- [...]
> 
> 
> After:
> 
>      7.20%     7.20%  flock02       [kernel.vmlinux]            [k] _raw_spin_lock
>                  |
>                  ---_raw_spin_lock
>                     |          
>                     |--52.23%-- flock_lock_file_wait
>                     |          sys_flock
>                     |          entry_SYSCALL_64_fastpath
>                     |          flock
>                     |          
>                     |--25.92%-- flock_lock_file
>                     |          flock_lock_file_wait
>                     |          sys_flock
>                     |          entry_SYSCALL_64_fastpath
>                     |          flock
>                     |          
>                     |--21.42%-- locks_delete_lock_ctx
>                     |          flock_lock_file
>                     |          flock_lock_file_wait
>                     |          sys_flock
>                     |          entry_SYSCALL_64_fastpath
>                     |          flock
>                      --0.43%-- [...]
> 
> 
> And its not at all clear to me why this would be. It looks like
> FILE_LOCK_DEFERRED is happening, but I've not yet figured out why that
> would be.

So I'd suggest to first compare preemption behavior: does the workload 
context-switch heavily, and is it the exact same context switching rate and are 
the points of preemption the same as well between the two kernels?

[ Such high variance is often caused by (dynamically) unstable load balancing and 
  the workload never finding a good equilibrium. Any observable locking overhead 
  is usually just a second order concern or a symptom. Assuming the workload 
  context switches heavily. ]

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/