[Pw_forum] compiling QE: fftw problems
James Johns
jjohns at umn.edu
Mon Jan 25 20:14:07 CET 2016
Hi Denis,
I was having similar troubles but was able to get it to compile over the
weekend, and link to the MKL FFTW3 libraries without building the special
interfaces. This was on a cluster composed of Haswell architecture chips.
I ran the configure script with ./install/configure
BLAS_LIBS="-lmkl_intel_lp64 -lmkl_sequential -lmkl_core"
SCALAPACK_LIBS="-lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64" and like
you I used the mvapich mpi implementation with the intel 2015 composer
suite of Intel compilers. With mvapich you need to tell it to use the
intelmpi blas libraries rather then the ompi blas libraries, because that
is what the configure script will automatically try and use.
I subsequently manually edited the make.sys file to include external fftw3
libraries, adding the flag -I$(MKLROOT)/include/fftw to IFLAG=-I../ so
that it could find the fftw3.h header, and then added FFT_LIBS =
-lmkl_intel_lp64 -lmkl_sequential -lmkl_core. This compiled fine without
linking to any of the specifically built mkl fftw3 interface wrappers.
I ran a test geometry optimization/sample job using 24 cores because I
wasn't sure if that was actually doing anything. The wall time for each
identical job was:
204 minutes using open-mpi with intel compilers and mkl BLAS/SCALAPACK,
158 minutes using mvapich with intel compilers and MKL BLAS/SCALAPACK
124 minutes using mvapich with intel compilers and MKL BLAS/SCALAPACK and
MKL FFTW3 as linked above.
I hope this is helpful.
Best,
James
On Mon, Jan 25, 2016 at 5:17 AM, Denis E. Zavelev <metalian at mail.ru> wrote:
> Hello James!
>
> No, I haven't yet compiled QE.
> Ddamn, it was really easy to compile QE on my netbook and my desktops (all
> under linux), but really hard on supercomputers.
> Filippo promised to help me but he seems to be too busy, I am still
> waiting an answer from him.
>
>
> Пятница, 22 января 2016, 15:14 -06:00 от James Johns <jjohns at umn.edu>:
>
>
> Hi Denis,
> Did you ever get it figured out? I'd be very interested if you did. I
> too have compiled the MKL wrappers for fftw3, but I can't get QE to
> recognize the library.
> Best Regards,
> James
>
> On Tue, Jan 19, 2016 at 1:04 PM, Paolo Giannozzi <p.giannozzi at gmail.com
> <https://e.mail.ru/compose/?mailto=mailto%3ap.giannozzi@gmail.com>> wrote:
>
> As far as I know, QE uses quite standard MPI calls that work on any
> reasonable version of MPI. The MPI release you have dates back to 2010.
> Maybe you should inquire with your system administrator.
>
> Paolo
>
> On Tue, Jan 19, 2016 at 6:03 PM, Denis E. Zavelev <metalian at mail.ru
> <https://e.mail.ru/compose/?mailto=mailto%3ametalian@mail.ru>> wrote:
>
> Hello Filippo!
>
> Thanks for your quick response.
> I use MVAPICH-1.2rc1, it's the only available MPI on JSC cluster and I am
> not allowed to use any other MPI implementations.
> I've sent you the files you asked.
>
>
> Среда, 20 января 2016, 0:45 +08:00 от Filippo SPIGA <
> filippo.spiga at quantum-espresso.org
> <https://e.mail.ru/compose/?mailto=mailto%3afilippo.spiga@quantum%2despresso.org>
> >:
>
>
> Dear Denis,
>
> are you using OpenMPI or Intel MPI?
>
> If Open MPI, try also
>
> ./configure MPIF90=mpiifort ...
>
>
> If Intel MPI (I assume this is your case), try instead
>
> ./configure MPIF90=mpiifort ... --with-scalapack=intel
>
>
> In the meanwhile please send me personally (not to the mailing-list!) to
> my email the "install/config.log" file and your make.sys.
>
> Regards
>
> --
> Mr. Filippo SPIGA, M.Sc.
> Quantum ESPRESSO Foundation
> http://www.quantum-espresso.org ~ skype: filippo.spiga
>
> *****
> Disclaimer: "Please note this message and any attachments are CONFIDENTIAL
> and may be privileged or otherwise protected from disclosure. The contents
> are not to be disclosed to anyone other than the addressee. Unauthorized
> recipients are requested to preserve this confidentiality and to advise the
> sender immediately of any error in transmission."
>
> On Jan 20, 2016, at 12:20 AM, Denis E. Zavelev <metalian at mail.ru
> <https://e.mail.ru/compose?To=metalian@mail.ru>> wrote:
> > Hello!
> >
> > I am trying to compile QE on JSC RAS cluster. As I have user
> permissions, I can install any programs only locally.
> > Cluster works under Linux. We have intel compilers (mpif90, icc, ifort)
> and MKL installed on cluster. No FFTW libraries are installed though even
> MKL ones (I have to build them locally).
> >
> > I've downloaded espresso 5.3.0 from the site.
> > Configure script finished successfully. But then I got 2 warnings and
> subsequent error message when compiling internal FFT.
> >
> > I decided to use some other FFT libs.
> > So I've downloaded FFTW3 from its site, successfully built, tested and
> installed it. But the same error message.
> >
> > Libraries found by configure script:
> > BLAS_LIBS= -lmkl_intel_lp64 -lmkl_sequential -lmkl_core
> > LAPACK_LIBS=
> > SCALAPACK_LIBS=-lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64
> > FFT_LIBS= -lfftw3
> >
> > Here's the output:
> >
> > ============================
> > bash-3.2$ make pw
> > make: Warning: File `make.sys' has modification time 0.47 s in the future
> > test -d bin || mkdir bin
> > ( cd FFTXlib ; make TLDEPS= all || exit 1 )
> > make[1]: Entering directory
> `/nethome/metalian/espresso/espresso-5.3.0/FFTXlib'
> > make[1]: Warning: File `../make.sys' has modification time 0.46 s in the
> future
> > mpif90 -O2 -assume byterecl -g -traceback -par-report0 -vec-report0
> -nomodule -fpp -D__INTEL -D__FFTW3 -D__MPI -D__PARA -D__SCALAPACK
> -I../include -I../iotk/src -I. -c fft_types.f90
> > mpif90 -O2 -assume byterecl -g -traceback -par-report0 -vec-report0
> -nomodule -fpp -D__INTEL -D__FFTW3 -D__MPI -D__PARA -D__SCALAPACK
> -I../include -I../iotk/src -I. -c scatter_mod.f90
> > icc -O3 -D__INTEL -D__FFTW3 -D__MPI -D__PARA -D__SCALAPACK -I../include
> -c fftw.c
> > fftw.c(27449): warning #188: enumerated type mixed with another type
> > EXPECT_INT(dir);
> > ^
> >
> > fftw.c(27450): warning #188: enumerated type mixed with another type
> > EXPECT_INT(type);
> > ^
> >
> > mpif90 -O2 -assume byterecl -g -traceback -par-report0 -vec-report0
> -nomodule -fpp -D__INTEL -D__FFTW3 -D__MPI -D__PARA -D__SCALAPACK
> -I../include -I../iotk/src -I. -c fft_scalar.f90
> > mpif90 -O2 -assume byterecl -g -traceback -par-report0 -vec-report0
> -nomodule -fpp -D__INTEL -D__FFTW3 -D__MPI -D__PARA -D__SCALAPACK
> -I../include -I../iotk/src -I. -c fft_parallel.f90
> > mpif90 -O2 -assume byterecl -g -traceback -par-report0 -vec-report0
> -nomodule -fpp -D__INTEL -D__FFTW3 -D__MPI -D__PARA -D__SCALAPACK
> -I../include -I../iotk/src -I. -c fft_smallbox.f90
> > mpif90 -O2 -assume byterecl -g -traceback -par-report0 -vec-report0
> -nomodule -fpp -D__INTEL -D__FFTW3 -D__MPI -D__PARA -D__SCALAPACK
> -I../include -I../iotk/src -I. -c fft_interfaces.f90
> > mpif90 -O2 -assume byterecl -g -traceback -par-report0 -vec-report0
> -nomodule -fpp -D__INTEL -D__FFTW3 -D__MPI -D__PARA -D__SCALAPACK
> -I../include -I../iotk/src -I. -c stick_base.f90
> > stick_base.f90(169): error #6404: This name does not have a type, and
> must have an explicit type. [MPI_IN_PLACE]
> > CALL MPI_ALLREDUCE(MPI_IN_PLACE, st, SIZE(st), MPI_INTEGER, MPI_SUM,
> comm, ierr)
> > -----------------------------^
> > compilation aborted for stick_base.f90 (code 1)
> > make[1]: *** [stick_base.o] Error 1
> > make[1]: Leaving directory
> `/nethome/metalian/espresso/espresso-5.3.0/FFTXlib'
> > make: *** [libfft] Error 1
> > ============================
> >
> > I've also tries espresso 5.2.1. Configure works the same way, but
> compilation also fails though not so fast, it ends on the following:
> >
> > fft_scalar.f90(69): #error: can't find include file: fftw3.f
> > make[1]: *** [fft_scalar.o] Error 1
> > make[1]: Leaving directory
> `/nethome/metalian/espresso/espresso-5.2.1/Modules'
> > make: *** [mods] Error 1
> >
> > This is strange because
> > -sh-3.2$ ls /nethome/metalian/soft/include/
> > fftw3.f fftw3.f03 fftw3.h fftw3l.f03 fftw3q.f03
> >
> > -sh-3.2$ export
> > export C_INCLUDE_PATH="/nethome/metalian/soft/include"
> > export
> INCLUDE="/nethome/metalian/soft/include:/opt/intel/composerxe-2011.3.174/mkl/include:/opt/intel/composerxe-2011.3.174/mkl/include"
> >
> > [I've mentioned only environment variables dealing with including]
> >
> >
> > So what am I doing wrong?
> > Are there any experienced users of Quantum Expresso on JSC cluster?
> >
> >
> >
> > Best regards,
> > Denis E. Zavelev
> >
> > _______________________________________________
> > Pw_forum mailing list
> > Pw_forum at pwscf.org <https://e.mail.ru/compose?To=Pw_forum@pwscf.org>
> > http://pwscf.org/mailman/listinfo/pw_forum
>
>
>
> Best regards,
> Denis E. Zavelev
>
> _______________________________________________
> Pw_forum mailing list
> Pw_forum at pwscf.org
> <https://e.mail.ru/compose/?mailto=mailto%3aPw_forum@pwscf.org>
> http://pwscf.org/mailman/listinfo/pw_forum
>
>
>
>
> --
> Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,
> Univ. Udine, via delle Scienze 208, 33100 Udine, Italy
> Phone +39-0432-558216, fax +39-0432-558222
>
>
> _______________________________________________
> Pw_forum mailing list
> Pw_forum at pwscf.org
> <https://e.mail.ru/compose/?mailto=mailto%3aPw_forum@pwscf.org>
> http://pwscf.org/mailman/listinfo/pw_forum
>
>
>
>
> --
> James E. Johns
> Assistant Professor
> Department of Chemistry
> University of Minnesota
> 207 Pleasant St.
> Minneapolis, MN 55455
> http://www.chem.umn.edu/groups/johns/index.html
>
>
>
>
>
--
James E. Johns
Assistant Professor
Department of Chemistry
University of Minnesota
207 Pleasant St.
Minneapolis, MN 55455
http://www.chem.umn.edu/groups/johns/index.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20160125/1264b456/attachment.html>
More information about the users
mailing list