Re: list corruption in deferred_split_scan()

[Yang Shi]

On 7/17/19 10:02 AM, Shakeel Butt wrote:
> On Tue, Jul 16, 2019 at 5:12 PM Yang Shi <yang.shi@xxxxxxxxxxxxxxxxx> wrote:
> > On 7/16/19 4:36 PM, Shakeel Butt wrote:
> > > Adding related people.
> > >
> > > The thread starts at:
> > > http://lkml.kernel.org/r/1562795006.8510.19.camel@xxxxxx
> > >
> > > On Mon, Jul 15, 2019 at 8:01 PM Yang Shi <yang.shi@xxxxxxxxxxxxxxxxx> wrote:
> > > > On 7/15/19 6:36 PM, Qian Cai wrote:
> > > > > > On Jul 15, 2019, at 8:22 PM, Yang Shi <yang.shi@xxxxxxxxxxxxxxxxx> wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 7/15/19 2:23 PM, Qian Cai wrote:
> > > > > > > On Fri, 2019-07-12 at 12:12 -0700, Yang Shi wrote:
> > > > > > > > > Another possible lead is that without reverting the those commits below,
> > > > > > > > > kdump
> > > > > > > > > kernel would always also crash in shrink_slab_memcg() at this line,
> > > > > > > > >
> > > > > > > > > map = rcu_dereference_protected(memcg->nodeinfo[nid]->shrinker_map, true);
> > > > > > > > This looks a little bit weird. It seems nodeinfo[nid] is NULL? I didn't
> > > > > > > > think of where nodeinfo was freed but memcg was still online. Maybe a
> > > > > > > > check is needed:
> > > > > > > Actually, "memcg" is NULL.
> > > > > > It sounds weird. shrink_slab() is called in mem_cgroup_iter which does pin the memcg. So, the memcg should not go away.
> > > > > Well, the commit âmm: shrinker: make shrinker not depend on memcg kmemâ changed this line in shrink_slab_memcg(),
> > > > >
> > > > > -     if (!memcg_kmem_enabled() || !mem_cgroup_online(memcg))
> > > > > +     if (!mem_cgroup_online(memcg))
> > > > >                 return 0;
> > > > >
> > > > > Since the kdump kernel has the parameter âcgroup_disable=memoryâ, shrink_slab_memcg() will no longer be able to handle NULL memcg from mem_cgroup_iter() as,
> > > > >
> > > > > if (mem_cgroup_disabled())
> > > > >         return NULL;
> > > > Aha, yes. memcg_kmem_enabled() implicitly checks !mem_cgroup_disabled().
> > > > Thanks for figuring this out. I think we need add mem_cgroup_dsiabled()
> > > > check before calling shrink_slab_memcg() as below:
> > > >
> > > > diff --git a/mm/vmscan.c b/mm/vmscan.c
> > > > index a0301ed..2f03c61 100644
> > > > --- a/mm/vmscan.c
> > > > +++ b/mm/vmscan.c
> > > > @@ -701,7 +701,7 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int
> > > > nid,
> > > >            unsigned long ret, freed = 0;
> > > >            struct shrinker *shrinker;
> > > >
> > > > -       if (!mem_cgroup_is_root(memcg))
> > > > +       if (!mem_cgroup_disabled() && !mem_cgroup_is_root(memcg))
> > > >                    return shrink_slab_memcg(gfp_mask, nid, memcg, priority);
> > > >
> > > >            if (!down_read_trylock(&shrinker_rwsem))
> > > We were seeing unneeded oom-kills on kernels with
> > > "cgroup_disabled=memory" and Yang's patch series basically expose the
> > > bug to crash. I think the commit aeed1d325d42 ("mm/vmscan.c:
> > > generalize shrink_slab() calls in shrink_node()") missed the case for
> > > "cgroup_disabled=memory". However I am surprised that root_mem_cgroup
> > > is allocated even for "cgroup_disabled=memory" and it seems like
> > > css_alloc() is called even before checking if the corresponding
> > > controller is disabled.
> > I'm surprised too. A quick test with drgn shows root memcg is definitely
> > allocated:
> >
> >   >>> prog['root_mem_cgroup']
> > *(struct mem_cgroup *)0xffff8902cf058000 = {
> > [snip]
> >
> > But, isn't this a bug?
> It can be treated as a bug as this is not expected but we can discuss
> and take care of it later. I think we need your patch urgently as
> memory reclaim and /proc/sys/vm/drop_caches is broken for
> "cgroup_disabled=memory" kernel. So, please send your patch asap.

Sure. I'm going to post the patch soon.

> thanks,
> Shakeel