2. Running MPICH-Madeleine Programs

The sub-directory example/basics of the MPICH-Madeleine source directory contains basic examples that can be used to test your MPICH-Madeleine installation. We will go through this section by using the program cpi.c.

2.1 Compiling the application

Compile your application with mpicc.

% cd mpich-mad/examples/basic
% $MPICH_MAD_ROOT/bin/mpicc -o cpi cpi.c

2.2 Location of the executable

MPI expects to locate the executable on the local machine, and at the same location on the remote machines. In our example, the location of the executable is $PWD/cpi e.g /home/mpich-user/mpich-mad/example/basics/cpi and this location must be valid on any machine running the application. See in the full guide at http://runtime.futurs.inria.fr/mpi/manual/Section3.html#Section3.1 to overcome this restriction.

2.3 Defining the machines to run the application on

You need to define the network configuration file localnet.cfg. This file should be sitting in the directory $MPICH_MAD_ROOT/etc and describes the different available networks.

networks : (
  {
    name  : myrinet;
    hosts : (node-40.bordeaux.grid5000.fr,
             node-41.bordeaux.grid5000.fr,
             node-42.bordeaux.grid5000.fr,
             node-43.bordeaux.grid5000.fr);
    dev   : mx;
  },
  {
    name  : tcp;
    hosts : (node-15.bordeaux.grid5000.fr,
             node-16.bordeaux.grid5000.fr,
             node-17.bordeaux.grid5000.fr,
             node-18.bordeaux.grid5000.fr);
    dev   : tcp;
  },
  {
    name  : gateway;
    hosts : (node-40.bordeaux.grid5000.fr,
             node-15.bordeaux.grid5000.fr);
    dev   : tcp;
  }
);

A network is defined with its name (tag name), a list of machines it includes (tag hosts), and the identifier of the device connecting these machines (tag dev, should be one of the predefined identifiers recognized by Madeleine).

Define the machines you wish to run your application on. When using a OAR reservation, the application will be launched on the machines reserved by OAR. You can also specify a list of machines by setting the environment variable MPI_HOST_FILE to a file containing the list of the machines you wish to use.
```
% cat $OAR_NODEFILE
node-16.bordeaux.grid5000.fr
node-41.bordeaux.grid5000.fr
node-47.bordeaux.grid5000.fr
% cat $MPI_HOST_FILE
node-16.bordeaux.grid5000.fr
node-41.bordeaux.grid5000.fr
%
```
You need to make sure you can connect to the machines -- you wish to use to run your application -- using the SSH protocol. MPI applications are launched using the PM2's mechanisms to remotely connect to the machines, this requires a SSH access to the machines without password or pass-phrase. You might need to change the definition of the environment variable LEO_RSH to match the configuration of your SSH connection. Here an example to test your configuration is working:
```
% echo $LEO_RSH
ssh -n -f
% $LEO_RSH jack uname -a
% Linux jack 2.6.4-fkt #14 SMP Thu Sep 23 2004 i686 GNU/Linux

%
```

2.4 Executing the application

Start your application using the command mpirun taking in parameter the number of processes to launch and the name of the application. The number of processes does not have to be the number of machines in the machine file (as specified above). MPICH-Madeleine will use a subset of these machines, or will start more than one process per machine accordingly.
```
% $MPICH_MAD_ROOT/bin/mpirun -np 2 cpi
```
It is also possible to use the parameter -machinefile to specify the file name containing the list of machines to use to run the application on.
```
% $MPICH_MAD_ROOT/bin/mpirun -machinefile $MPI_HOST_FILE -np 2 cpi
```

You should obtain an output similar to:

Process 0 of 2 on your_machine
pi is approximately 3.1415926544231318, Error is 0.0000000008333387
wall clock time = 1.094123
Process 1 of 2 on your_machine

Your network configuration file can define more than one network to connect a same set of nodes. Such a configuration allows to switch easily between different networks when executing an application.

networks : (
  {
    name  : tcp;
    hosts : (node-40.bordeaux.grid5000.fr,
             node-41.bordeaux.grid5000.fr,
             node-42.bordeaux.grid5000.fr,
             node-43.bordeaux.grid5000.fr);
    dev   : tcp;
  },
  {
    name  : myrinet;
    hosts : (node-40.bordeaux.grid5000.fr,
             node-41.bordeaux.grid5000.fr,
             node-42.bordeaux.grid5000.fr,
             node-43.bordeaux.grid5000.fr);
    dev   : mx;
  }
);

By default, MPICH-Madeleine will use the first network to connect the nodes. You can use the environment variable MPICH_MAD_PROTOCOL to use another defined network.

% $MPICH_MAD_ROOT/bin/mpirun -np 2 cpi
... Uses TCP
% export MPICH_MAD_PROTOCOL=mx
% $MPICH_MAD_ROOT/bin/mpirun -np 2 cpi
... Uses MX
%

If you wish to start your application under the debugger, you can use the option -gdb. All the processes of the MPI application will be launched through gdb, the GNU debugger.
```
% mpirun -gdb -np 2 cpi
```

1 Installing MPICH-Madeleine