Re: Remote fork() and Parallel programming

Richard Gooch (Richard.Gooch@atnf.CSIRO.AU)
Tue, 16 Jun 1998 17:43:59 +1000

mshar@vax.ipm.ac.ir writes:
> Hi,
>
> lm@bitmover.com (Larry McVoy) wrote:
>
> >: Wrong argument. One encounters the latency problem _each_ time one transfers
> >: data over a network, no matter what the programmer does to initiate this
> >: transfer. The DSM and message passing argument is more about programming
> >: interfaces. The latency will not decrease if the programmer uses messages (or
> >: _any_ other method) to transfer 4KB of data to another computer.
> >
> >OK, so how about this one: process A spins on memory, process B sets
> >the memory, process B spins on same memory, process A sets it ....
> >
> >Every one of those handshakes is a 4K page in DSM, right? It pretty
> >much has to be if 4K is what your system thinks a page is and you
> >are mapping this stuff into your address space.
>
> Right.
>
> >So explain to me how the latency of a 4K transfer is going to be lower
> >or the same as a 4 byte transfer (which is all that is required to do
> >the same thing in message passing)?
>
> That is the definition of latency. The times needed to establish a
> connection or close it do not depend on the _amount_ of data transfered.
>
> In _high_speed_ networks, the difference in time for sending 4K bytes and
> 4 bytes is negligible in comparison to the time needed to open and close
> a connection.

4 bytes or 4 kBytes, who cares? We're still talking about latencies of
microseconds or tens of microseconds. The most optimistic estimate of
networking latencies I've seen in this discussion is a few
microseconds. That's an order of magnitude greater than the time taken
for local test-and-set operations. For most clusters with available
networking technologies, it will be two orders of magnitude.

And don't bother telling me how in X years we will have
sub-microsecond network latencies. By that time we will have
nanosecond latencies for local test-and-set. The difference remains.

> >And if you want to just ignore this, it would be nice if you were to
> >explain to our audience exactly how common this sort of operation is
> >in those shared memory applications that you want to support without
> >modification (hint: it's extremely common, every spin lock, mutex,
> >semaphore does it).
>
> If I see someone doing busy waiting, I'd say he is not a very good
> programmer. Busy waiting should be avoided even in normal applications
> running on a single computer. BTW, please refer to Linux's kernel sources to
> see if it uses busy waiting to implement its semaphores (hint: it doesn't,
> processes sleep while waiting).

Let's have a look at a typical mutex. You busy-wait for several spins
before going to sleep, because going to sleep is really
expensive. Hopefully another thread will unlock the mutex before you
go to sleep. After several spins, you may as well sleep because the
other thread is probably going to take a while anyway.

And if you think going to sleep takes a lot of time, wait till you see
how long it takes to do a network mutex!

You'd be surprised how much time a typical threaded computation spends
in mutexes. We want to keep that fast. Maybe you're interested in
times between mutexes of seconds, but I care about small fractions of
a second in my applications. My most important threaded application is
a hybrid: threaded/SHM for each node and message passing between the
nodes. Coding as a single virtual machine would be a lose.

Regards,

Richard....

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu