[H-GEN] HUMBUG cluster

David Jericho david.jericho at bytecomm.com.au
Wed Feb 5 21:43:17 EST 2003


[ Humbug *General* list - semi-serious discussions about Humbug and     ]
[ Unix-related topics. Posts from non-subscribed addresses will vanish. ]

On Wed, Feb 05, 2003 at 09:22:26PM +1000, Andrew Pullin wrote:
> bit of a problem. Where as there _ARE_ clustering software suites that run
> on Macs, and Macs _DO_ run Linux, it seems that for our purposes, this
> solution is probably not suitable for a first attempt at building a Humbug
> Cluster, and it will more than likely cause more technical problems than we
> have limited expertise to solve.

Well, what are your purposes? PVM, MPI and quite a few other
clustering solutions will work well, all being based in userspace
rather than MOSIX style clustering. 

According to the Debian site, most Apple platforms are supported.

And according to the Yellow Dog Linux site, they have the Black Lab
Cluster Management software, which claims automated node creation in 3
minutes on Power PC hardware.

If commerical software can do it in three minutes, a room full of Unix
users could do it in a night.

> about setting up Linux on Macs and compiling the Beowulf source, but this
> again seems not to be a valid solution due to the architecture thing.

There seems to be a misunderstanding as to what a "Beowulf" cluster
is.

>From the FAQ, at http://www.canonical.org/~kragen/beowulf-faq.txt

1. What's a Beowulf? [1999-05-13]

It's a kind of high-performance massively parallel computer built
primarily out of commodity hardware components, running a free-software
operating system like Linux or FreeBSD, interconnected by a private
high-speed network.  It consists of a cluster of PCs or workstations
dedicated to running high-performance computing tasks.  The nodes in
the cluster don't sit on people's desks; they are dedicated to running
cluster jobs.  It is usually connected to the outside world through
only a single node.

Some Linux clusters are built for reliability instead of speed.  These
are not Beowulfs.

2. Where can I get the Beowulf software? [1999-05-13]

There isn't a software package called "Beowulf".  There are, however,
several pieces of software many people have found useful for building
Beowulfs.  None of them are essential.  They include MPICH, LAM, PVM,
the Linux kernel, the channel-bonding patch to the Linux kernel (which
lets you 'bond' multiple Ethernet interfaces into a faster 'virtual'
Ethernet interface) and the global pid space patch for the Linux kernel
(which, as I understand it, lets you see all the processes on your
Beowulf with ps, and maybe kill etc. them), DIPC (which lets you use
sysv shared memory and semaphores and message queues transparently
across a cluster).  [Additions?  URLs?]

>     So people, this seems to put us back at square 2 again (1 was getting
> enough people interested to try in the first place). What we are still

I don't agree. I say give it a shot. MOSIX and similar load sharing
systems aren't all there is to being a cluster.

> looking for seems to be 4 similar Pentium Is.

I have a AMD K-166, and a Celeron 300A that I'm willing
to donate to the cause.

> My friend recomends that in the future, when the group has some clustering 
> experience, that we should tackle a Mac cluster as a separate, advanced 
> project.

Since when has a true Unix geek ever declared something too hard?

I've never learnt anything by sitting back and just using the stuff
other people have provided in a nice wrapper.

My personal opinion is that the machines have been offered, and they
will let us install the various methods for communicating between
nodes[1]. I say we accept and attempt it. 

If we succeed, we can contribute back with our results, and possibly even 
document our methods so that other people can repeat the exercise with ease.

I myself have built various Beowulf clusters ranging from 5 machines,
to 20, using all sorts of software systems, from MOSIX, to PVM, and I
fail to see what being a Macintosh has to do with anything.

[1] Even straight netcat can let us do a parallel software task with
the right shell scripts.

-- 
David Jericho
Senior Systems Administrator, Bytecomm Pty Ltd


--
Scanned and found clear of viruses by EntireScan.  http://www.entirescan.com/

--
* This is list (humbug) general handled by majordomo at lists.humbug.org.au .
* Postings to this list are only accepted from subscribed addresses of
* lists 'general' or 'general-post'.  See http://www.humbug.org.au/



More information about the General mailing list