MPI backend

There is an ongoing development effort to automatically build cluster-enabled simulators from Equelle.

The instructions below detail how to enable the prerequisites for this on Ubuntu 13.10.

Installing Zoltan

Zoltan is a domain-decomposition library that is part of the Trilinos-package from Sandia.

1. Ensure you have mpi and Fortran installed sudo apt-get install openmpi-bin libopenmpi-dev gfortran
2. Download Trilinos from the Trilinos homepage.
3. Create a source directory mkdir $HOME/src
4. Change to directory cd $HOME/src
5. Extract the source code tar xvjf <path-to-trilinos-source>
6. Create a build directory mkdir $HOME/src/build-trilinos
7. Change to directory cd $HOME/src/build-trilinos
8. Generate the makefile It is imporant to build the shared-library of Zoltan, with the static library you can get very hard-to-debug link or runtime errors cmake ../trilinos-11.4.3-Source/ -DCMAKE_INSTALL_PREFIX=<ZOLTAN_INSTALL_DIR> -DTrilinos_ENABLE_Zoltan=ON -DTPL_ENABLE_MPI:BOOL=ON -DMPI_BASE_DIR:PATH=/usr/lib/openmpi -DBUILD_SHARED_LIBS:BOOL=ON -DTrilinos_ENABLE_CXX11:BOOL=ON -DCMAKE_CXX_FLAGS:STRING=-std=c++11
9. make -j4
10. make install

Building Equelle with MPI support

It is assumed Equelle is checked out into $EQUELLE_SRC and you are building into $EQUELLE_BUILD. Zoltan is assumed to be installed in $ZOLTAN_INSTALL_DIR.

If you get strange runtime errors when running MPI programs, it is very important that the ${MPI_INCLUDE_PATH} is before ${Zoltan_INCLUDE_DIRS} in the include_directories() clause of any involved CMakeLists.txt file.

1. cd $EQUELLE_BUILD
2. cmake $EQUELLE_SRC -DCMAKE_PREFIX_PATH=$ZOLTAN_INSTALL_DIR -DEQUELLE_BUILD_MPI=ON
3. make