Re: [PATCH 3/4] workqueue: Update workqueue's possible cpumask when a new node, coming up.

[Kamezawa Hiroyuki]

(2014/12/16 16:49), Lai Jiangshan wrote:
> On 12/15/2014 07:18 PM, Kamezawa Hiroyuki wrote:
> > Workqueue keeps cpu<->node relationship including all possible cpus.
> > The original information was made at boot but it may change when
> > a new node is added.
> >
> > Update information if a new node is ready with using node-hotplug callback.
> >
> > Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx>
> > ---
> >   include/linux/memory_hotplug.h |  2 ++
> >   kernel/workqueue.c             | 28 +++++++++++++++++++++++++++-
> >   mm/memory_hotplug.c            |  4 ++++
> >   3 files changed, 33 insertions(+), 1 deletion(-)
> >
> > diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
> > index 8f1a419..cd3cb67 100644
> > --- a/include/linux/memory_hotplug.h
> > +++ b/include/linux/memory_hotplug.h
> > @@ -270,4 +270,6 @@ extern void sparse_remove_one_section(struct zone *zone, struct mem_section *ms)
> >   extern struct page *sparse_decode_mem_map(unsigned long coded_mem_map,
> >   					  unsigned long pnum);
> >
> > +/* update for workqueues */
> > +void workqueue_node_register(int node);
> >   #endif /* __LINUX_MEMORY_HOTPLUG_H */
> > diff --git a/kernel/workqueue.c b/kernel/workqueue.c
> > index 2fd0bd7..5499b76 100644
> > --- a/kernel/workqueue.c
> > +++ b/kernel/workqueue.c
> > @@ -266,7 +266,7 @@ struct workqueue_struct {
> >   static struct kmem_cache *pwq_cache;
> >
> >   static cpumask_var_t *wq_numa_possible_cpumask;
> > -					/* possible CPUs of each node */
> > +					/* PL: possible CPUs of each node */
> >
> >   static bool wq_disable_numa;
> >   module_param_named(disable_numa, wq_disable_numa, bool, 0444);
> > @@ -4559,6 +4559,32 @@ static void restore_unbound_workers_cpumask(struct worker_pool *pool, int cpu)
> >   		WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task,
> >   						  pool->attrs->cpumask) < 0);
> >   }
> > +#ifdef CONFIG_MEMORY_HOTPLUG
> > +
> > +static void reflesh_wq_possible_mask(int cpu, int node)
> > +{
> > +	int oldnode;
> > +	for_each_node_state(oldnode, N_POSSIBLE)
> > +		cpumask_clear_cpu(cpu, wq_numa_possible_cpumask[oldnode]);
>
> You need to check and update all the wq->numa_pwq_tbl[oldnode]

you mean : if I drop patch 1/4, it will be required. Right ?
I'll add that check.

Thanks,
-Kame


> > +	cpumask_set_cpu(cpu, wq_numa_possible_cpumask[node]);
> > +}
> > +/*
> > + * When a cpu is physically added, cpu<->node relationship is established.
> > + * We can catch the whole view when a new NODE_DATA() coming up.
> > + * Update cpumask used for sched affinity of workers.
> > + */
> > +void workqueue_node_register(int node)
> > +{
> > +	int cpu;
> > +	mutex_lock(&wq_pool_mutex);
> > +	for_each_possible_cpu(cpu) {
> > +		if (node == cpu_to_node(cpu))
> > +			reflesh_wq_possible_mask(cpu, node);
> > +	}
> > +	mutex_unlock(&wq_pool_mutex);
> > +}
> > +
> > +#endif
> >
> >   /*
> >    * Workqueues should be brought up before normal priority CPU notifiers.
> > diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
> > index 1bf4807..d0c1ebb 100644
> > --- a/mm/memory_hotplug.c
> > +++ b/mm/memory_hotplug.c
> > @@ -1121,6 +1121,10 @@ static pg_data_t __ref *hotadd_new_pgdat(int nid, u64 start)
> >   	 * online_pages() and offline_pages().
> >   	 */
> >   	reset_node_present_pages(pgdat);
> > +	/*
> > +	 * Update workqueue's cpu/node affinity info.
> > +	 */
> > +	workqueue_node_register(nid);
> >
> >   	return pgdat;
> >   }


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/