[Pw_forum] QE-5.4.0 on hpc cluster w gfortran + MKL and other

[eugenia wetrowa]

Dear all, please help. I'm trying to build parallel QE-5.4.0 on hpc
cluster.  System parameters:
20 nodes with
2* E5-2680 Intel Xeon CPU (8 n), 128GiB RAM 300 GiB HDD
arch: x86_64
Gentoo/Linux 3.2.59
GCC compilers, gfortran 4.8
OpenMPI, mpich-2 mpi-mpich2-1_5-x86_64

Autoconfig detects compilers and parallel environment correctly but  links
libraries:

BLAS_LIBS      = -L/opt/intel/composerxe-2013.0.080/mkl/lib/intel64
-lmkl_gf_lp64  -lmkl_sequential -lmkl_core
BLAS_LIBS_SWITCH = external
LAPACK_LIBS    =
LAPACK_LIBS_SWITCH = external
ELPA_LIBS_SWITCH = disabled
SCALAPACK_LIBS = -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64
FFT_LIBS       = -lfftw3
MPI_LIBS       =
MASS_LIBS      =
AR             = ar
ARFLAGS        = ruv
RANLIB         = ranlib
FLIB_TARGETS   = all
LIBOBJS        = ../clib/clib.a ../iotk/src/libiotk.a
LIBS           = $(SCALAPACK_LIBS) $(LAPACK_LIBS) $(FFT_LIBS) $(BLAS_LIBS)
$(MPI_LIBS) $(MASS_LIBS) $(LD_LIBS)

In this configuration "make" fails:

_ _ _ _ _ _
test -d bin || mkdir bin
( cd FFTXlib ; make TLDEPS= all || exit 1 )
make[1]: Entering directory `/home/osvasilyev/QE/FFTXlib'
mpif90 -O3 -g -x f95-cpp-input -D__GFORTRAN -D__STD_F95 -D__DFTI -D__MPI
-D__PARA -D__SCALAPACK  -I../include -I/include  -c fft_types.f90
Warning: Nonexistent include directory "/include"
mpif90 -O3 -g -x f95-cpp-input -D__GFORTRAN -D__STD_F95 -D__DFTI -D__MPI
-D__PARA -D__SCALAPACK  -I../include -I/include  -c scatter_mod.f90
Warning: Nonexistent include directory "/include"
mpif90 -O3 -g -x f95-cpp-input -D__GFORTRAN -D__STD_F95 -D__DFTI -D__MPI
-D__PARA -D__SCALAPACK  -I../include -I/include  -c fft_scalar.f90
Warning: Nonexistent include directory "/include"
In file included from fft_scalar.f90:27:0:
fft_scalar.DFTI.f90:20:0: fatal error: mkl_dfti.f90: No such file or
directory
 #include "mkl_dfti.f90"
 ^
compilation terminated.
make[1]: *** [fft_scalar.o] Error 1
make[1]: Leaving directory `/home/osvasilyev/QE/FFTXlib'
make: *** [libfft] Error 1
_ _ _ _ _ _ _

Then i tryed to build QE with MKL libraries following the Intel's manual,
linking explicitlly ./configure --LIBDIRS="
-L/opt/intel/composerxe-2013.0.080/mkl/lib/intel64
-Wl, -lmkl_scalapack_ilp64 -lmkl_gf_ilp64 -lmkl_gnu_thread -lmkl_core
-lmkl_blacs_intelmpi_ilp64 -lgomp -lpthread -lm -ldl" .
Configure says
"The following libraries have been found:
  BLAS_LIBS=-L/opt/intel/composerxe-2013.0.080/mkl/lib/intel64
-lmkl_gf_lp64  -lmkl_sequential -lmkl_core
  LAPACK_LIBS=
  SCALAPACK_LIBS=-lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64
  FFT_LIBS=-lfftw3
parallel environments detected successfully"

But compilation crushed again with same log. It's not clear for me what the
errors occure. Directory "include" existed. When linking MKL breafly, the
same. And there was any unsuccessful cases of configuration, with internal
libs too.

QE was builded only in configuration: LAPACK_LIBS  = -latllapack,
SCALAPACK_LIBS = , FFT_LIBS  =  -lfftw3,  MPI_LIBS  =
-L/usr/lib/mpi/mpi-mpich2-1_5-x86_64/usr/lib64/ , but it works poorly.
Examples go normally on -np 2..8, but jobs (scf, md, relax that start
normally on local)  always fail due to SEGFAULT. tryed -np 2..128.

So I ask for advice, has anyone dealt sucesfully with QE on cluster with
GFORTRAN? I suspect that gfortran+mkl+QE is unlivable, who knows some
appropriate configurations?

PS. Sysadmin's advices didn't help that's why I'm here)

Sincerely,
Eugenia W.

Research engineer
NRNU, Moscow.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20161101/bea28cba/attachment.html>