Re: [PATCH] sunrpc: use better NUMA affinities

From: Greg Banks
Date: Fri Jul 29 2011 - 16:24:21 EST




Sent from my iPhone

On 30/07/2011, at 2:45, "J. Bruce Fields" <bfields@xxxxxxxxxxxx> wrote:

On Fri, Jul 29, 2011 at 04:53:21PM +1000, Greg Banks wrote:
On 29/07/11 16:30, Eric Dumazet wrote:
Le vendredi 29 juillet 2011 Ã 16:05 +1000, Greg Banks a Ãcrit :
On 29/07/11 15:32, NeilBrown wrote:

I seem to remember coming to the conclusion that Jeff eventually
addressed this problem...am I misremembering or did something regress?

Currently, all nfsd kthreads use memory for their kernel stack and
various initial data from a _single_ node, even if you use
sunrpc.pool_mode=pernode (or percpu)

That's just plain broken and I'm very pleased to see you fix it.

Should I take that as a "Reviewed-by"?

If you like.


[...] In ToT
svc_pool_map_set_cpumask() is called *after* kthread_create() and
applies to the child thread, *after* it's stack has been allocated
on the wrong node. In the working SGI code,
svc_pool_map_set_cpumask() is called by the parent node on itself
*before* calling kernel_thread() or doing any of the data structure
allocations, thus ensuring that everything gets allocated using the
default memory allocation policy, which on SGI NFS servers was
globally tuned to be "node-local".

OK, so would it be enough to just move the svc_pool_map_set_cpumask()
back a few lines, or do we want Eric's approach, in order to have
something that will work better with other memory allocation policies?


Relying on global policy was easy but not a great idea, I prefer Eric's approach of doing it explicitly.

Greg.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/