[PATCH RT 5/5] allow preemption in slab_alloc_node and slab_free

[Nicholas Mc Guire]

drop preempt_disable/enable in slab_alloc_node and slab_free

__slab_alloc is only called from slub.c:slab_alloc_node
it runs with local irqs disabled so it can't be pushed off this CPU
asynchronously, the preempt_disable/enable is thus not needed.
Aside from that the later this_cpu_cmpxchg_double would catch such a 
migration event anyay.

slab_free:
 slowpath: (if the allocation was on a different CPU) detected by 
  (page == c->page) c pointing to the per cpu slab, this does not need a 
  consistent ref to tid so the slow path is safe without the 
  preempt_disable/enable
 fastpath: if allocation was on the same cpu but we got migrated between
  fetching the cpu_slab and the actual push onto the free list then
  this_cpu_cmpxchg_double would catch this case and loop in redo. So the
  fast path is also safe without the preempt_disable/enable

Testing:
 while : ; do ./hackbench 120 thread 10000 ; done 
Time: 296.631
Time: 298.723
Time: 301.468
Time: 303.880
Time: 301.988
Time: 300.038
Time: 299.634
Time: 301.488
 which seems to be a good way to stress-test slub

Impact on performance:
 The change could negatively impact performance if the removal of the 
 preempt_disable/enable would result in a significant increase of the 
 slow path being taken or looping via goto redo - this was checked by:
 static instrumentation:
  an instrumentation was added to check how often the redo loop is taken
  the results showed that the redo loop is very rarely taken (< 1 in 10000)
  and is below the value with the preempt_disable/enable present. Further
  the slowpath to fastpath ration improves slightly (not sure if this is 
  statistically significant though)
 running slab_test.c:
  the slub-benchmark from Christoph Lameter and Mathieu Desnoyers was used
  the only change being that asm/system.h was droped from the list of
  includes. The results indicate that the removal of preempt_disable/enable
  reduces the cycles needed slightly (though quite a few testsystems would
  need to be checked before this can be confirmed).

Tested-by: Andreas Platschek <platschek@xxxxxxxxxxxxxxxx>
Tested-by: Carsten Emde <C.Emde@xxxxxxxxx>
Signed-off-by: Nicholas Mc Guire <der.herr@xxxxxxx>
---
 mm/slub.c |    4 ----
 1 files changed, 0 insertions(+), 4 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index 546bd9a..c422988 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2424,7 +2424,6 @@ redo:
 	 * on a different processor between the determination of the pointer
 	 * and the retrieval of the tid.
 	 */
-	preempt_disable();
 	c = __this_cpu_ptr(s->cpu_slab);
 
 	/*
@@ -2434,7 +2433,6 @@ redo:
 	 * linked list in between.
 	 */
 	tid = c->tid;
-	preempt_enable();
 
 	object = c->freelist;
 	page = c->page;
@@ -2683,11 +2681,9 @@ redo:
 	 * data is retrieved via this pointer. If we are on the same cpu
 	 * during the cmpxchg then the free will succedd.
 	 */
-	preempt_disable();
 	c = __this_cpu_ptr(s->cpu_slab);
 
 	tid = c->tid;
-	preempt_enable();
 
 	if (likely(page == c->page)) {
 		set_freepointer(s, object, c->freelist);
-- 
1.7.2.5

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/