1. Download and install Intel Fortran Compiler 1. Make sure Intel Fortran compiler is available to PATH and it's libraries and is capable of compiling * I added this to my path: {{{ source /opt/intel/fc/10.1.015/bin/ifortvars.sh }}} * I also needed to install libstd5 {{{ #found this with apt-file search filename sudo apt-get install libstdc++5 }}} 1. Download openmpi 1. Configure openmpi; make sure it can build fortran code {{{ #http://support.intel.com/support/performancetools/sb/CS-026988.htm tar -xzvf openmpi-1.2.6.tar.gz cd openmpi-1.2.6 ./configure --prefix=/usr/local F77=/opt/intel/fc/10.1.015/bin/ifort FC=/opt/intel/fc/10.1.015/bin/ifort sudo make all install }}} 1. Download and extract SuperLU_DIST_2.2 1. Copy MAKE_INC/ to make.inc. For me it was make.i386_linux. 1. Edit make.inc * Change DSuperLUroot to point to wherever you've extracted the SuperLU source code * Change LOADER to mpicc. I'm not sure yet if this was a good thing. * Make kept failing because it was missing for_write_seq_fmt_xmit. I Added this {{{ LOADOPTS = -lblas -L/opt/intel/fc/10.1.015/lib/ -lifcore }}} 1. Make 1. Copy /lib to /opt/pydusa/pydusa/mympi/src 1. Change Options.mk so that LLIBS includes -L/opt/pydusa/pydusa/mympi/src/ -lsuperlu_dist_2.0 1. ??? 1. make 1. make test === On hexahedral === 1. attachment:make.inc === On Lys (32 bit linux Red Hat ) === I followed essentially the same directions as above, but did not install libstd5 1. Installed latest version of OpenMPI 1. Configure openmpi; make sure it can build fortran code {{{ #http://www.open-mpi.org/software/ompi/v1.4/ tar -xzvf openmpi-1.4.2.tar.gz cd openmpi-1.4.2 ./configure --prefix=/usr/local F77=/opt/intel/fc/10.0.023/bin/ifort FC=/opt/intel/fc/10.0.023/bin/ifort make all install <-- This needs to be done as root }}} 1. Compile the mpi module within Continuity, no changes where made to any Makefiles there. {{{ cd $CONT_ROOT/src/mpi make }}} 1. Compile the distributed version of SuperLU (version 2.4 was used here), as above I had to modify the make.inc file to conform to my platform. The main difference from above is I had to add '-lg2c' to the LOADOPTS parameter, and of course change the locations/names of the BLAS, METISLIB, and PARMETISLIB library files. attachment:lys_make.inc {{{ LOADOPTS = -lblas -L/opt/intel/fc/10.0.023/lib/ -lifcore -lg2c }}} 1. Compile the python interface to SuperLU_Dist, located at $CONT_ROOT/src/solvers/dist_superlu. I had to make the following changes to the default Makefile, in addition to the location of various libraries. {{{ PINC = -I$(PY_ROOT)/include/python2.5 -I$(PY_ROOT)/lib/python2.5/site-packages/numpy/core/include/numpy -I$(PY_ROOT)/lib/python2.5/site-packages/numpy/core/include -I$(CONT_ROOT)/src/solverLibraries/SuperLU_DIST_2.4/SRC/ LIBS = -lm -lg2c -L/opt/intel/fc/10.0.023/lib/ -lifcore $(SLU_LIB) $(BLAS_LIB) $(PARMETIS_LIB) $(METIS_LIB) }}} 1. Lastly, I was getting a python error when running the src/solvers/dist_superlu/make test: ''python2.5: relocation error: /usr/local/lib/openmpi/mca_paffinity_linux.so: undefined symbol: mca_base_param_reg_int'', so I had to set: {{{ export LD_PRELOAD=/usr/local/openmpi/lib/libmpi.so }}} and the test succeeded after setting LD_PRELOAD. === On oolite (64 bit linux cluster ) === The instructions on lys mostly hold for oolite. Not having root priviledges, I had to install things in my local directories. 1. I downloaded/installed openMPI and configured it as above (i.e., I included Intel Fortran compiler), the openmpi on oolite at /opt/openmpi seemed to have been built using gfortran. 1. Downloaded and installed Parmetis from http://glaros.dtc.umn.edu/gkhome/metis/parmetis/download, added -fPIC to OPTFLAGS 1. Compiled $CONT_ROOT/src/mpi 1. Compiled $CONT_ROOT/src/solverLibraries/SuperLU_DIST_2.4, after modifying the make.inc file, the major change from what was done on lys was: {{{ LOADOPTS = -fPIC -lblas -L/opt/intel/Compiler/11.1/072/lib/intel64/ -lifcore -lirc -lsvml }}} 1. Compiled $CONT_ROOT/src/solvers/dist_superlu, making the same changes to the Makefile as for lys, expect: {{{ LIBS = -lm -L/opt/intel/Compiler/11.1/072/lib/intel64 -lifcore -lsvml $(SLU_LIB) $(BLAS_LIB) $(PARMETIS_LIB) $(METIS_LIB) }}} 1. As with lys, I had to do the LD_PRELOAD setting to get the 'make test' to work.