AppleSeed Development Page
Welcome! This page is for those who want to write their own parallel programs that run on multiple Macintoshes.
Running in parallel on one dual processor Macintosh requires a different technique: see our multiprocessing page. To run multiple dual processor Macintoshes requires combining both techniques.

What software you need to write your own Parallel Programs:
For Fortran: Absoft Fortran 77 compiler For C: Absoft or Metrowerks ANSI C compiler
MacMPI.f  a Fortran subset of MPI
                    with Visual Monitor  (11/9/00)
MacMPI.c  a C subset of MPI
                    with Visual Monitor  (11/8/00)
MacMPI_IP.f  a Fortran subset of MPI
         based on Open Transport TCP/IP  (5/5/00)
MacMPI_IP.c  a C subset of MPI
         based on Open Transport TCP/IP  (5/5/00)
mpif.h, include file for MacMPI.f and MacMPI_IP.f mpi.h, include file for MacMPI.c and MacMPI_IP.c



Topics on this Page:
(Click a link to jump down the page to that section)

What is MPI?
What is MacMPI?
Using MacMPI with:
KKKK~The Absoft Pro Fortran compiler
KKKK~The Metrowerks CodeWarrior Pro compiler
KKKK~The Absoft C compiler
Troubleshooting
Monitor Window
Parallel Fractal Demo GPL Source
 


What is MPI?

MPI is an industry standard Applications Programming Interface (API), described in a number of textbooks, for sending and receiving messages between nodes of a parallel computer. If you are new to MPI, we recommend Using MPI by William Gropp, Ewing Lusk, and Anthony Skjellum [MIT Press, 1999]. There are also a number of online tutorials available for learning MPI such as at Argonne National Laboratory and Cornell University. If you're new to parallel processing, we have made available a tutorial, How to Write (Nearly) Portable Fortran Programs for Parallel Computers.

What is MacMPI?

MacMPI is a subset of MPI, which contains the collective subroutines most commonly used by university students and researchers. The main restriction of MacMPI is that messages are not buffered, which means that messages from a particular node must be received in the order sent, and probing for messages is not supported.  For regular problems, most programmers naturally send and receive messages in order.  The few times when this does not happen, it is usually unintentional and easily fixed.  Not buffering messages can also give better performance.

MacMPI currently exists in two versions. The original version, called MacMPI.f and MacMPI.c is based on the Program to Program Communcations (PPC) Toolbox, and we recommend starting with this because it is very simple and reliable and fewer things can go wrong. For higher performance, a TCP/IP version is available, called MacMPI_IP, based on Open Transport, but its use is more complex.  The simpler MacMPI also does not support multiple communicators and cartesian topologies.

If you donít have an MPI program yourself, you can first try one of our sample codes:
Text Iconknock.f, a simple MPI Fortran program for beginners
Text Iconknock.c, a simple MPI C program for beginners
Text Iconpingpong.f, a Fortran test program for measuring communication speeds 
Text Iconpingpong.c, a C test program for measuring communication speeds 

Details on how to create an executable depend on the version of the compiler being used.

Using MacMPI with:

The Absoft Pro Fortran compiler
To compile and run your own Fortran source code, two files are needed, a communications library, either MacMPI.f or MacMPI_IP.f, and the include file mpif.h. Creating an executable with the Absoft compiler is straightforward. First make sure the include file mpif.h is present, then compile the appropriate MPI library with the f77 compiler:
f77 -O -Q92 -c MacMPI.f
f77 -O -Q92 -c MacMPI_IP.f
Linking depends on which MPI library is chosen and the language of the main program. If test.f is the main program, one links with the original MacMPI as follows:
f77 -O -Q92 test.f MacMPI.f.o
f90 -O -604 test.f MacMPI.f.o
For the TCP/IP version, additional libraries must be included, which depend on the version of the operating system being used. For MacOS prior to 9.0, the link command is:
f77 -O -Q92 test.f MacMPI_IP.f.o &
"{PPCLibraries}"OpenTransportAppPPC.o &
"{PPCLibraries}"OpenTptAtalkPPC.o &
"{SharedLibraries}"OpenTransportLib &
"{SharedLibraries}"OpenTptInternetLib &
"{SharedLibraries}"OpenTptAppleTalkLib
Unfortunately, the Shared Library OpenTransportLib included with the MPW Absoft compiler is out of date. To fix this, one should make a copy of the file OpenTransportLib from the Extensions folder in System folder and place it in the SharedLibraries folder in Libraries folder of MPW. Linking with Fortran90 is similar. If one has an older version of the Absoft compiler (version 6.0), make sure to download the latest patches from their web site, or Open Transport will not link.

For MacOS 9.0 and later, Apple has combined some of these Shared Libraries into a new library called Open Transport, and the link command is simplified:

f77 -O -Q92 test.f MacMPI_IP.f.o &
"{PPCLibraries}"OpenTransportAppPPC.o "{PPCLibraries}"OpenTptAtalkPPC.o &
"{SharedLibraries}""Open Transport"
As before, one needs to copy the file Open Transport from the Extensions folder in the System folder into the Shared Libraries folder. One can also run Fortran77 code with automatic double precision, by using the -N113 and -N2 options:
f77 -O -N113 -N2 -Q92 test.f MacMPI.f.o
This option is not available for Fortran 90. It is possible to create a makefile both manually as well as via a graphical interface, although the makefiles differ from Unix style makefiles.

Note: In Pro Fortran version 6.2, Absoft changed the default behavior of the Fortran stop to require a manual carriage return to end the program. This is inconvient when running on multiple machines. To reverse this change, use the following command once:

setprefs '{AbsoftLibraries}mrwe.o'
All subsequently compiled code will now quit when the program completes its normal execution.
 
      The Metrowerks CodeWarrior Pro compiler


With CW, you have four major options: Using the Standard C Console window versus creating a Macintosh application, and using MacMPI.c versus MacMPI_IP.c. We recommend you begin and debug with MacMPI.c, then use MacMPI_IP.c later.

Standard Console C

If your C source code is standard ANSI C (e.g., this code is multiplatform and uses ANSI C calls like fprintf and scanf), you should start with the Standard C console project stationery and add your source as you normally would.  Then add the MacMPI.c library to the project. In the Target Settings Panel named "PPC Target", edit the 'SIZE' Flags to use localAndRemoteHLEvents. This allows the Launch Den Mother to see the parallel application on a remote machine. Also, be sure to set the Preferred Heap Size so that your code will have enough available memory. In addition, your code will need to edit certain run-time console flags. Before your main() code, add #include <SIOUX.h>, and at the top of your main() code add:

SIOUXSettings.asktosaveonclose=0;
SIOUXSettings.autocloseonquit=1;
This will have the app quit when it falls out of main(), otherwise, the apps will wait for user input before quitting, locking up the remote machines forever.

In addition, if you don't already have one, you may wish to add a call to printf prior to calling MPI_Init(). It appears to be necessary for proper initialization of the app's runtime environment, without which a crash may occur.

Mac OS Toolbox C

If your C source code is a Macintosh C application (your C calls Macintosh Toolbox routines exclusively), then begin with Mac OS Toolbox project stationery as you normally would, and add your source. Then add MacMPI.c as well. Since MacMPI relies on ANSI C routines, be sure CodeWarrior's SIOUX libraries (for PowerPC, MSL C.PPC.Lib and MSL SIOUX.PPC.Lib) are also loaded in your project. Again, in the Target Settings Panel named "PPC Target", edit the 'SIZE' Flags to use localAndRemoteHLEvents. And, when you write your code, be sure to have your remote apps quit without direct human interaction. It would also be helpful to have your app's event loop respond to Quit AppleEvents, so the Launch Den Mother can kill them remotely, if necessary.

Also, you may wish to set the monitor flag to 0, which will turn off the MacMPI status window. If you find your event loop, your windows, and MacMPI conflicting, this may help.

Using MacMPI_IP.c

Finally, if you wish to use MacMPI_IP.c instead of MacMPI.c, you will need to add the Open Transport libraries to your project file: OpenTransportLib, OpenTptAppleTalkLib, OpenTptInternetLib, OpenTptATalkPPC.o, and OpenTransportAppPPC.o. These libraries are in the MacOS Support folder in your compiler's home folder.

Obviously, for benchmarking, you should explore the other compiler and optimization settings. But you should note that the highest level of the Global Optimization settings doesn't always produce the fastest code. Experiment.

The Absoft C compiler
To compile and run a C source program, one needs the library MacMPI.c or MacMPI_IP.c and the header file mpi.h. With the Absoft C compiler, one compiles the library with one of the following commands:
acc -O -A -N92 -c MacMPI.c
acc -O -A -N92 -c MacMPI_IP.c
Linking depends on which MPI library is chosen. If test.c is the main program, one links with the original MacMPI as follows:
acc -O -A -N92 test.c MacMPI.c.o
For the TCP/IP version, additional libraries must be included, which depend on the version of the operating system being used. For MacOS prior to 9.0, the link command is:
acc -O -A -N92 test.c MacMPI.c.o &
"{PPCLibraries}"OpenTransportAppPPC.o &
"{PPCLibraries}"OpenTptAtalkPPC.o &
"{SharedLibraries}"OpenTransportLib &
"{SharedLibraries}"OpenTptInternetLib &
"{SharedLibraries}"OpenTptAppleTalkLib
Unfortunately, the Shared Library OpenTransportLib included with the MPW Absoft compiler is out of date. To fix this, one should make a copy of the file OpenTransportLib from the Extensions folder in System folder and place it in the SharedLibraries folder in Libraries folder of MPW. If one has an older version of the Absoft compiler (version 6.0), make sure to download the latest patches from their web site,or Open Transport will not link.

For MacOS 9.0 and later, Apple has combined some of these Shared Libraries into a new library called Open Transport, and the link command is simplified:

acc -O -A -N92 test.c MacMPI_IP.c.o &
"{PPCLibraries}"OpenTransportAppPPC.o "{PPCLibraries}"OpenTptAtalkPPC.o &
"{SharedLibraries}""Open Transport"
As before, one needs to copy the file Open Transport from the Extensions folder in the System folder into the Shared Libraries folder. With the Absoft compiler, C and Fortran programs can link with one another.
 

Troubleshooting

To make sure your network is properly set up, you should first test one of our precompiled programs, using the Launch Den Mother utility. The Parallel Fractal Demo, uses AppleTalk and is compiled with MacMPI.c. The AltiVec Fractal Demo IP uses TCP/IP and is compiled with MacMPI_IP.c. It will run on Power Macs even without AltiVec.  If the Launch Den Mother does not see the other nodes, it is likely that either program linking has not been set or the network is not working or is unreliable.

When compiling your own program, we recommend that one first start with MacMPI, which is based on AppleTalk, then move to MacMPI_IP, if higher performance is required. We also recommend that you start with only two nodes, then add more later.

The Launch Den Mother starts the program on each node and waits until it detects that MacMPI is actually running. If it does not detect that MacMPI is running, it will quit with an error. MacMPI might not be running because the program on some node had a runtime error independent of MPI.  The most common cause of failure is not setting the Preferred Size in Get Info for the application causing it to run out of memory, or making it too large for those nodes which have less memory.

If MPI fails, MacMPI will write an error file (called FOR002.DAT for Fortran and MPIErrs for C). You should examine these files (one on each machine), to determine the reasons for the failure. We have a separate page describing how to read these error files.  If the computers do not appear to be connecting, you can hit the escape key (or control-c or cmd-.) to abort the connection. With TCP/IP, if a connection is not made within one minute, it will abort. If a message does not arrive within five minutes, it will abort and report the status of all pending messages to the error file.

If the Launch Den Mother fails to launch the parallel program, it is also possible to start it up manually. This can sometimes give further information about the reasons for failure. MacMPI requires that the master node have a file called nodelist present in the same directory as the executable. If the parallel job is started manually, this file must be created by the user. (The Launch Den Mother utility creates this file automatically.) This is a straight text file, with different versions for AppleTalk and TCP/IP.

For AppleTalk, the first line contains a port name. If the name ppc_link is used, then the slave nodes do not need to have a host file, so this is recommended for debugging. (If some other port name is used, then the slave nodes need to have a nodelist file which contains only the port name.) The second line contains the name self. Finally the remaining lines consist of a list of computer names and zones, one name per line, in the form:

computer_name@zone_name
If there is only one zone in the AppleTalk network, the zone names are omitted. The names cannot have trailing blanks. A sample nodelist file looks like:
ppc_link
self
uclapic1
BarbH2@Physics-Plasma Theory
fried@Physics-Plasma Theory
For TCP/IP, the first line contains a port number. If the number 5013 is used, then the slave nodes do not need to have a host file, so this is recommended for debugging. (If some other number is used, then the slave nodes need to have a nodelist file which contains only the port number.) The second line contains the name self. Finally the remaining lines consist of a list of IP addresses, one address per line, in dotted decimal form:
12.13.14.15
The addresses cannot have trailing blanks. A sample nodelist file for TCP/IP looks like:
5013
self
128.97.65.90
128.97.65.91
128.97.65.92
To start the parallel job manually, one has to copy the executable to each node (via floppy disk, AppleShare, or ftp), and start up (by double clicking) each executable. The master must be started last.

If you run into any other difficulties that should be publicized here, please email us.

Monitor Window

All current versions of MacMPI have a global parameter called monitor which gives various levels of diagnostic information. If monitor = 1, a small status window appears. In this window, status lights indicate whether the node whose screen is being examined is sending and/or receiving messages from any other node. Since messages normally are sent very fast, these lights blink rapidly. The status window also shows a histogram of the size of messages being
sent or received.

When MacMPI_IP is being used, two additional dials are shown in the status window. One of these shows the percent of time spent in communication. The second dials is a speedometer which shows the average and instantaneous speeds achieved during communication. These dials are only approximate. The speedometer indicating communication speed is a useful indicator of network problems if it suddenly starts to run slowly. Finally, if monitor = 2, a debug log of every message sent and received is written to the error file.

For maximum performance, monitor = 0 will shut off this window.

Debugging MPI programs

The monitor = 2 setting in MacMPI is intended to help programmers find causes of MPI errors in their programs.  When this variable is set and MacMPI is recompiled, a log of every message sent and received by MPI is written to an error file on each node.  By comparing the error files, one can usually determine the entire state of the messaging system when a failure occurs.  A subroutine (called SET_MON in Fortran and Set_Mon in C) is available to set the monitor variable.  One can also use this subroutine to log only sections of code.  Since a lot of data is generated, we also have an additional subroutine (called LOGNAME in Fortran, Logname in C), which will embed a string into the error files to help the programmer track log entries in the program.  For example, call LOGNAME('calling fft') in Fortran will embed the string "calling fft" into the error files at that point in the log.  We have a separate page describing further details how to read and interpret these error files.

A common problem when debugging MPI programs is deadlocked code.  MacMPI will timeout if no data arrives for a posted receive for 5 minutes or if a posted send cannot transmit data for 5 minutes.  When MacMPI times out, the current status of all the messages is recorded in each of the error files, regardless of the value of the monitor variable.  The information here is very useful in figuring out the causes of deadlocks. The timeout value (in milliseconds) is set in the local variable mstime in the subroutine MPI_TEST (or MPI_Test).  When debugging, it is useful to reduce this value to about 30000 (30 seconds).

Parallel Fractal Demo GPL Source

For those who wish to work with existing parallel Macintosh application code, we have provided a Gnu Public License release of the source code to the Parallel Fractal Demo. This archive contains a complete Metrowerks CodeWarrior Pro 5 project.


For additional technical details, see the AppleSeed Technical Report.


AppleSeed Development Web Page | AppleSeed Multiprocessing Web Page | Back to AppleSeed

Last Updated: May 7, 2001