Re: [PATCH] slub: Fix sysfs circular locking dependency

[David Rientjes]

On Wed, 5 Jan 2011, Pekka Enberg wrote:

> >> > Subject: slub: Avoid use of slub_lock in show_slab_objects()
> >> >
> >> > The purpose of the locking is to prevent removal and additions
> >> > of nodes when statistics are gathered for a slab cache. So we
> >> > need to avoid racing with memory hotplug functionality.
> >> >
> >> > It is enough to take the memory hotplug locks there instead
> >> > of the slub_lock.
> >> >
> >> >
> >> > Signed-off-by: Christoph Lameter <cl@xxxxxxxxx>
> >> >
> >> > ---
> >> >  mm/slub.c |    4 ++--
> >> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >> >
> >> > Index: linux-2.6/mm/slub.c
> >> > ===================================================================
> >> > --- linux-2.6.orig/mm/slub.c    2011-01-05 09:55:34.000000000 -0600
> >> > +++ linux-2.6/mm/slub.c 2011-01-05 09:56:27.000000000 -0600
> >> > @@ -3821,7 +3821,7 @@ static ssize_t show_slab_objects(struct
> >> >                }
> >> >        }
> >> >
> >> > -       down_read(&slub_lock);
> >> > +       lock_memory_hotplug();
> >> >  #ifdef CONFIG_SLUB_DEBUG
> >> >        if (flags & SO_ALL) {
> >> >                for_each_node_state(node, N_NORMAL_MEMORY) {
> >> > @@ -3862,7 +3862,7 @@ static ssize_t show_slab_objects(struct
> >> >                        x += sprintf(buf + x, " N%d=%lu",
> >> >                                        node, nodes[node]);
> >> >  #endif
> >> > -       up_read(&slub_lock);
> >> > +       unlock_memory_hotplug();
> >> >        kfree(nodes);
> >> >        return x + sprintf(buf + x, "\n");
> >> >  }
> >>
> >> Makes sense. Bart, does this fix the problem for you?
> >
> > The action sequence that had triggered this sequence (invoke
> > kmem_cache_create(); read all files in /sys/kernel/slab; invoke
> > kmem_cache_destroy()) does now pass without triggering lock inversion
> > complaints.
> 
> Thanks for testing. David, does Christoph's patch look OK to you?
> 

I think it certainly fixes the problem at hand, but I think we also need 
to do lock_memory_hotplug() for memory hotplug in 
slab_mem_going_online_callback() to make show_slab_objects() consistent 
when being printed during concurrent node hot-add since it sets bits in 
N_NORMAL_MEMORY.  The MEM_OFFLINE callback is already handled at a higher 
level by taking the lock in the hotplug layer, but we need to protect the 
MEM_GOING_ONLINE and MEM_CANCEL_ONLINE callbacks if slub_lock is no longer 
used to protect node arrays (which was admittedly always convenient since 
it's typically associated with an iteration through slab_caches).