> OK, let's look at some numbers and compare. 100 Mb/s EtherNet is now a
> commidity item: Gb/s is still experimental. So we have networks with
> 10 MB/s bandwidth and 2 millisecond latencies.
Hogwash! I'm the Network Eng. at the county offices in Martin County
Florida. We not only have gigabit ethernet in our (sun) servers, but use
it for connecting remote sites!
Yes, the workstations are still connected via switched 100Mb/s.. But
gigabit ethernet works and will be commidity someday soon (5 years sounds
good).. However, switched 100Mb/s has a bandwidth of 20MB/s (full duplex).
Btw- 100Mb/s has better latency then 2ms! I imagine that if you had a MPI
implimentation that banged the hardware correctly (or a speical network
API for doing zero copy for MPI and such) you could get 1/4 or less that
latency.
> Now let's look at the bandwidth of a modern, commidity computer, say
> PII. With 100 MHz SDRAM, a 4-1-1-1 burst gives you 8 bytes per step;
> with 4 steps that's 32 bytes for the burst. It takes 7 cycles. Let's
> say 8 to make the maths simpler. So 8 cycles gives 32 bytes. That's 4
> bytes per cycle: 400 MBytes/s. The latency is going to be well under
> 200 nanoseconds.
>
> So: we have a factor of 40 in bandwidth and 10000 in latency. And this
> comparison is only done with the boring 64 bit bus in your average PC.
> It gets worse if you look at a 256 bit bus: bandwidth goes to 1.6 GB/s,
> which is 160 times better than current EtherNet.
Okay, cant argue there.
> Right now, there's no competition: networks are so much slower than
> what you find in a computer.
[snip]
> Networking has not kept up with Moore's Law. It seems computers have,
> however. So the logical progression is that in the future, we're going
> to see more of the same. It's going to take years before gigabit
> EtherNet is affordable, and by then computers will be even further
> ahead.
So? You have failed to show that slow networking makes clustering
worthless.
Consider, the Avalon which is interconnected with switched 100Mb/s
achieved enough performance (after a memory upgrade) to reach the 50th
fastest computer in the world at a total price of around what? $280K?
Even on REAL (and network smashing) code (spasm and grav tree code) they
got better price/performance then out of real supercomputers.
The nodes were 21164a's with more memory bandwidth then your 100MHz
example (66Mhz SDRAM/ 128Bit bus). Yes, the network was slow, and even
though it slaughtered performance (1 node performance * n nodes, was
something like 10 times better for some codes), clustering was still worth
while.
> > Heck even if this view is off and skewed to heck, *it is possible*.
> > And things are getting faster, it's only a matter of time before
> > networking is fast enough (or integrated enough) that clustering
> > such as this would be possible.
>
> Let me make a bottom-line statement: networking will *never* be as
> fast as internal computer speeds. Technology trends point to computers
> increasing at a greater rate. And then there's physics. Unless you're
> planning on breaking the speed of light, it's always going to take a
> lot longer to send information across a room than across a chip.
And then there is reality: Faster chips and 'real supercomputers' will
always cost so much more then commodity supercomputers that clustering
will be worthwhile long into the future.
> [...]
> > Think.
>
> Count.
Think different.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/