Re: [PATCH v2 11/11] mm/slab: lockless decision to grow cache

[Jesper Dangaard Brouer]

On Tue, 12 Apr 2016 13:51:06 +0900
js1304@xxxxxxxxx wrote:

> From: Joonsoo Kim <iamjoonsoo.kim@xxxxxxx>
> 
> To check whther free objects exist or not precisely, we need to grab a
           ^^^^^^    
(spelling)
> lock.  But, accuracy isn't that important because race window would be
> even small and if there is too much free object, cache reaper would reap
> it.  So, this patch makes the check for free object exisistence not to
                                                      ^^^^^^^^^^^
(spelling)

> hold a lock.  This will reduce lock contention in heavily allocation case.
> 
> Note that until now, n->shared can be freed during the processing by
> writing slabinfo, but, with some trick in this patch, we can access it
> freely within interrupt disabled period.
> 
> Below is the result of concurrent allocation/free in slab allocation
> benchmark made by Christoph a long time ago.  I make the output simpler.
> The number shows cycle count during alloc/free respectively so less is
> better.

I cannot figure out which if Christoph's tests you are using.  And I
even have a copy of his test here:
 https://github.com/netoptimizer/prototype-kernel/blob/master/kernel/mm/slab_test.c

I think you need to describe the test a bit better...

Looking a long time at the output on my own system, I guess you are
showing results from the "Concurrent allocs".  Then it would be
relevant how many CPUs your system have.

It would also be relevant to mention that N=10000.  And perhaps mention
that it means, e.g all CPUs do N=10000 alloc concurrently, synchronize
before doing N free concurrently.

> * Before
> Kmalloc N*alloc N*free(32): Average=248/966
> Kmalloc N*alloc N*free(64): Average=261/949
> Kmalloc N*alloc N*free(128): Average=314/1016
> Kmalloc N*alloc N*free(256): Average=741/1061
> Kmalloc N*alloc N*free(512): Average=1246/1152
> Kmalloc N*alloc N*free(1024): Average=2437/1259
> Kmalloc N*alloc N*free(2048): Average=4980/1800
> Kmalloc N*alloc N*free(4096): Average=9000/2078
> 
> * After
> Kmalloc N*alloc N*free(32): Average=344/792
> Kmalloc N*alloc N*free(64): Average=347/882
> Kmalloc N*alloc N*free(128): Average=390/959
> Kmalloc N*alloc N*free(256): Average=393/1067
> Kmalloc N*alloc N*free(512): Average=683/1229
> Kmalloc N*alloc N*free(1024): Average=1295/1325
> Kmalloc N*alloc N*free(2048): Average=2513/1664
> Kmalloc N*alloc N*free(4096): Average=4742/2172
> 
> It shows that allocation performance decreases for the object size up to
> 128 and it may be due to extra checks in cache_alloc_refill().  But, with
> considering improvement of free performance, net result looks the same.
> Result for other size class looks very promising, roughly, 50% performance
> improvement.

Super nice performance boost.  The numbers on my system are
significantly smaller, but this is a before/after test and the absolute
numbers are not that important.

Oh, maybe this was because I ran the test with SLUB... recompiling with
SLAB... and the results are comparable to your numbers (on my 8 core
i7-4790K CPU @ 4.00GHz)

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer