Install OpenMPI on a CSIL machine to Run MPI

You can test your MPI program on the Expanse cluster, but it might take minutes to wait for an available machine to run your program. If you can install MPI in your own laptop or CSIL, you can quickly test your MPI programs.

Note that in the end, you still need run your MPI programs at the Expanse cluster for the final running and collect data for reporting such as the running time.

Use a pre-installed MPI package in CSIL

You may directly use the MPI package installed in the tyang_class account without installing this MPI package by youself in CSIL machines.

In the attached makefile with PA1 called Makefile-CSIL, modify the path location prefix of the MPI package as
MPI_PREFIX = ~tyang_class/local/bin/
Then the above makefile should work. For example, you can test
make -f Makefile-CSIL run-mv_mult_test_mpi
A valid running result can be:

Test 1: Wall clock time = 0.000035 at Proc 0 of 2 processes
Test 2: Wall clock time = 0.001825 at Proc 0 of 2 processes
Summary: Failed 0 out of 2 tests
The mpicc and mpirun commands in ~tyang_class/local/bin/ can be used to compile and run MPI programs. See an example listed below.

Installation steps for CSIL machines or your own machines

The OpenMPI package is from this website, and its installation in a CSIL user account below uses about 288MB in openmpi-4.1.2 directory and 15MB in $(HOME)/local directory. The instructions below are borrowed from here and a sentence started with "#" is a comment. These instructions should work for CSIL Linux machines. The instructions for MacOS laptops should be similar if all prerequisite packages are installed.

Open a terminal before you start. Go to your root directory ($HOME) and type the following commands:

mkdir -p local/src # You can choose another directory.

wget https://download.open-mpi.org/release/open-mpi/v4.1/openmpi-4.1.2.tar.gz

tar xvzf openmpi-4.1.2.tar.gz

cd openmpi-4.1.2/

./configure --prefix=$HOME/local # Also you can choose another prefix.

make all

make install

How to test after installation

ls $HOME/local/bin # You should find 'mpicc' and 'mpirun' here.

cat hello.c # Now let us test our installation.

#include <stdio.h> #include <mpi.h> int main(int argc, char** argv) { MPI_Init(NULL, NULL); int rank; int world; MPI_Comm_rank(MPI_COMM_WORLD, &rank); MPI_Comm_size(MPI_COMM_WORLD, &world); printf("Hello: rank %d, world: %d\n",rank, world); MPI_Finalize(); }
$HOME/local/bin/mpicc -o hello hello.c

$HOME/local/bin/mpirun -np 2 hello # You can have more than 2 processes, but sometime your laptop may not allow that.

If you use the MPI package installed in the tyang_class account, then you can use ~tyang_class/local/bin/mpicc and ~tyang_class/local/bin/mpirun to compile and run the above hello.c file.

How to use more processes than the available cores

The number of cores available on a machine can limit the number of processes allowed to run an MPI job in the install MPI package. For using more processes than the available number of physical cores, one could write hostfile and then add --hostfile option when running an mpi program. (e.g. localhost slots=16 in hostfile)

Here is a reference on this subject.

An example to run your code with 8 processes is:
Your path/mpiexec --allow-run-as-root --hostfile hostfile -np 8 prog32_pi_test_mpi

Issues on failed MPI processes and speedup measurement

When your code running on a CSIL machine has a bug, the possibility of leaving some active background processes exists, which could overload CSIL machines and affect other users. Please use no more than 8 processes to debug on CSIL (considering the environment is shared), and there is no need to use more than 16 (our grading script will not test it).

Please use "ps -a" to find background/failed MPI processes and "kill -9 " to kill these MPI processes.

Also please note using more processes than the available number of cores is for a functionality debugging on CSIL machines and you will not get a speedup of 8 with 8 processes when the number of cores available is 4 or less. For the performance report, please use Expanse.