Re: [PATCH 11/39] autonuma: CPU follow memory algorithm
From: Peter Zijlstra
Date: Wed Mar 28 2012 - 07:26:36 EST
On Tue, 2012-03-27 at 18:15 +0200, Andrea Arcangeli wrote:
> This is _purely_ a performance optimization so if my design is so bad,
> and you're also requiring all apps that spans over more than one NUMA
> node to be modified to use your new syscalls, you won't have problems
> to win against AutoNUMA in the benchmarks.
Right, so can we agree that the only case where they diverge is single
processes that have multiple threads and are bigger than a single node (either
in memory, cputime or both)?
I've asked you several times why you care about that one case so much, but
I'll grant you that unmodified such processes might do better with your
- your stuff assumes there is a fair amount of locality to exploit.
I'm not seeing how this is true in general, since data partitioning is hard
and for those problems where its possible people tend to already do so,
yielding natural points to add the syscalls.
- your stuff doesn't actually nest, since a guest kernel has no clue as to
what constitutes a node (or if there even is such a thing) it will randomly
move tasks around on the vcpus, with complete disrespect for whatever host
vcpu<->page mappings you set up.
guest kernels actively scramble whatever relations you're building by
scanning, destroying whatever (temporal) locality you think you might
- also, by not exposing NUMA to the guest kernel, the guest kernel/userspace
has no clue it needs to behave as if there's multiple nodes etc..
Furthermore, most applications that are really big tend to have already thought
about parallelism and have employed things like data-parallelism if at all
possible. If this is not possible (many problems fall in this category) there
really isn't much you can do.
Related to this is that all applications that currently use mbind() and
sched_setaffinity() are trivial to convert.
Also, really big threaded programs have a natural enemy, the shared state that
makes it a process, most dominantly the shared address space (mmap_sem etc..).
There's also the reason Avi mentioned, core count tends to go up, which means
nodes are getting bigger and bigger.
But most importantly, your solution is big, complex and costly specifically to
handle this case which, as per the above reasons, I think is not very
So why not do the simple thing first before going overboard for a case that
might be irrelevant?
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/