Re: [RFC] Simple Slab: A slab allocator with minimal meta information

[Christoph Lameter]

On Fri, 11 Aug 2006, Manfred Spraul wrote:

> Christoph Lameter wrote:
> 
> > I still do not get the role of the shared cache though.
> > 
> The shared cache is just for efficient object transfers:
> Think about two nics, both cpu bound, one does rx, the other does tx.
> Result: a few 100k kmalloc, kmem_cache_alloc(skb_head_cache) calls each second
> on cpu1.
> the same number of kfree, kmem_cache_free(skb_head_cache) calls each second on
> cpu 2.

Hmmm. In that case a faster free/alloc would help since cachelines are not 
shared.

> Initially, the slab allocator just had the cpu cache. Thus an object transfer
> was a free_block call: add the freed object to the bufctl linked list. Move
> the slab to the tail of the partial list. Probably the list_del()/list_add()
> calls caused cache line trashing, but I don't remember the details. IIRC
> Robert Olsson did the test. Pentium III Xeon system?
> Anyway: The solution was the shared array. It allows to move objects around
> with a simple memmove of the pointers, without the list_del()/list_add()
> calls.

But the shared array still needs the list_lock for access. So this is 
avoiding the list operations?

In the case of the simple slab we would have one task continually 
allocating from its slab until its full. If the other task is freeing 
objects in the same slab at the same time then there is lock contention on 
the per cpu slab.

If however there would be sufficient distance between the free and the 
alloc then one task would complete a slab put it back to the full list and 
get it back from the partial list.
The other task would be removing objects from the full slab and move it 
back to the partial list.

In an ideal case we would have two slab continually trading roles with 
separate slab locks. The list lock would only be taken if they switch 
roles.If we can get that going then its the same effect as the shared 
cache.

Hmmm... Hmmm...

Essentially if we discover that a process frees in the cpuslab of another 
then the cpuslab must be detached and the other process must be forced to 
pick another slab for allocation.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/