Re: [bisected] "sched: Allow per-cpu kernel threads to run on online && !active" causes warning

[Heiko Carstens]

On Wed, Jul 27, 2016 at 05:23:05PM +0200, Thomas Gleixner wrote:
> On Wed, 27 Jul 2016, Heiko Carstens wrote:
> > [    3.162961] ([<0000000000176c30>] select_task_rq+0xc0/0x1a8)
> > [    3.162963] ([<0000000000177d64>] try_to_wake_up+0x2e4/0x478)
> > [    3.162968] ([<000000000015d46c>] create_worker+0x174/0x1c0)
> > [    3.162971] ([<0000000000161a98>] alloc_unbound_pwq+0x360/0x438)
> 
> > For some unknown reason select_task_rq() gets called with a task that has
> > nr_cpus_allowed == 0. Hence "cpu = cpumask_any(tsk_cpus_allowed(p));"
> > within select_task_rq() will set cpu to nr_cpu_ids which in turn causes the
> > warning later on.
> > 
> > It only happens with more than one node, otherwise it seems to work fine.
> > 
> > Any idea what could be wrong here?
> 
> create_worker()
>     tsk = kthread_create_on_node();
>     kthread_bind_mask(tsk, pool->attrs->cpumask);
>         do_set_cpus_allowed(tsk, mask);
>             set_cpus_allowed_common(tsk, mask);
>                 cpumask_copy(&tsk->cpus_allowed, mask);
>                 tsk->nr_cpus_allowed = cpumask_weight(mask);
>     wake_up_process(task);
> 
> So this looks like pool->attrs->cpumask is simply empty.....

Just had some time to look into this a bit more. Looks like we initialize
the cpu_to_node_masks (way) too late on s390 for fake numa. So Peter's
patch just revealed that problem.

I'll see if initializing the masks earlier will fix this, but I think it
will.