On Mon, Jul 31, 2000 at 03:04:15PM -0500, Oliver Xymoron wrote:
> I think you'll find that the really big machines (ASCI, etc) are
> non-coherent. Coherence is expensive hardware-wise and the programming
> model it's designed to support just doesn't scale well. People don't
> ridicule Beowulfs and it's only a matter of time before commodity systems
> have competitive memory bandwidth with the SGIs and DECs of the world.
The only ccNUMA ASCI system I know is actually a cluster of 48 128CPU ccNUMAs.
Why? The Origin 2000 uses directories to avoid broadcasting of invalidates.
For machines larger than 128 CPUs the directories don't have enough bits to
allow a per-node granularity of invalidates, so for systems beyond that size
the hw has to broadcast invalidates into a region of machines thereby
causing an increase of traffic on the internal network.
Directories btw. are fairly expensive but yet a proven way to reduce the
coherency traffic on the internal network. The extra price is why I don't
believe that we'll see wide offerings of ccNUMAs any time soon. In fact
even SGI offers two memory configurations (standard / premium) which
implement different size directories and therefore support different
system sizes before having to use coarse invalidates.
Ralf
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Mon Aug 07 2000 - 21:00:13 EST