Funnily enough, I'm implementing IP clustering in order to try out a
Beowulf-style cluster for a (highly) multi-user interactive service.
It's sort of like inverse masqerading, only different :-). Although
each cluster node has an IP address for communicating with the other
nodes (on the high-bandwidth interconnect), it also has an
alias-style device on a shared IP number (which doesn't respond to
arps). A front-end node catches traffic to that shared address and
routes new connections to "shared" ports to an appropriate cluster
node, keeping track of it for future traffic on that connection.
Each node also has its own "private" part of the port space (e.g.
for a 16-node cluster, node1 gets 4096-7936, node1 gets 7937-12033)
which the kernel uses for non-explicitly bound connections (e.g.
outgoing telnet/rlogin etc.) and the front-end node routes those
ports directly to the relevant node. From the outside world (and
for users logged in doing "ordinary" things), the cluster appears
to be a single machine, with the Beowulf-style crossmounting, shared
PID space and a few other bells and whistles making everything fit
together.
Implementation involves a new firewall chain plus code in ip_rcv()
(for 2.0.x) to decide which host to route to depending on ipfwadm
rules, a current connection hash table and dynamically created
"pool" and "node" devices which let you split the cluster up into
pools of nodes each with given costs/loads for distributing particular
types of connections. From userland, it's a combination of "ipfwadm -C",
"clconfig" (hacked ifconfig to add support for ioctls to add, delete or
alter pool/node devices) and a daemon to update the current loads for
each node in each node device in the front-end so that it can make good
decisions for how to distribute new connections. Nodes other than the
front-end don't need anything special other than a change to the
inet_autobind/good_socknum stuff so that you can configure each node to
auto-bind into its private port space for outgoing connections.
The design is all done, it's now just a SMOP when time pressure eases
up a little in December. So if anyone has an online picture of a mob
of "penguins with attitude", let me know.
--Malcolm
-- Malcolm Beattie <mbeattie@sable.ox.ac.uk> Unix Systems Programmer Oxford University Computing Services