[Pw_forum] running in paralell

[Axel Kohlmeyer]

On Mon, 8 Sep 2008, Eduardo Ariel Menendez Proupin wrote:

EAMP> Hi,
EAMP> I have problem running pw using OpenMPI. It runs terribly slow if run in
EAMP> different nodes (2 in this case). I think that the problem is in the OpenMP
EAMP> parallelism included in the Intel Compiler and the MKL libraries.
EAMP> The configure program sets the flag -openmp. If delete the flag,  get
EAMP> unresolved libraries. To run using MPI, the MKL need the shell variable
EAMP> OMP_NUMTHREADS=1 and MKL_NUM_THREADS=1.

apart from setting environment variables, you can also link
with the serial version of mkl. in this case neither -openmp
nor -lguide and -lpthread are needed. ...and while at it, you
can also hardcode the search path to MKL into the binary.
the BLAS_LIBS entry in make.sys becomes then:

BLAS_LIBS      = -L/opt/intel/mkl/10.0.1.014/lib/em64t \
                 -Wl,-rpath,/opt/intel/mkl/10.0.1.014/lib/em64t \
                 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core

and 

LAPACK_LIBS    = 

can be left empty. the usual -lmkl is just a wrapper to link

 -lmkl_intel_lp64 -lmkl_thread_intel -lmkl_core

and -lmkl_em64t will link the equivalent static libraries.

if you want to make your life (and that of other users) _really_ 
easy, you can edit both libmkl.so and libmkl_em64t.a in a text
editor and just change libmkl_intel_thread.so/.a to 
libmkl_sequential.so/.a

and you won't have to worry about this issue again (until you
install a new version of MKL).

cheers,
   axel.

EAMP> With this, MPI works fine in one node. The problem is that running in two
EAMP> nodes, the shell variables are not passed to the remote machine, and I have
EAMP> not found how to do it, but there shluld be an easy way, or maybe in the
EAMP> configuration of MPI.
EAMP> Thanks
EAMP> 

-- 
=======================================================================
Axel Kohlmeyer   akohlmey at cmm.chem.upenn.edu   http://www.cmm.upenn.edu
   Center for Molecular Modeling   --   University of Pennsylvania
Department of Chemistry, 231 S.34th Street, Philadelphia, PA 19104-6323
tel: 1-215-898-1582,  fax: 1-215-573-6233,  office-tel: 1-215-898-5425
=======================================================================
If you make something idiot-proof, the universe creates a better idiot.