[Pw_forum] Problem while compiling QE v5.3.0 with ELPA
Rolly Ng
rollyng at gmail.com
Tue Mar 7 10:48:16 CET 2017
Dear Ari,
Thanks and I was trying to follow the installation guide recommended by the
HPC advisory council.
http://www.hpcadvisorycouncil.com/pdf/Quantum_ESPRESSO_5.3.0_Installation_Be
st_Practices.pdf
But I tried that on a different version of ELPA, wget http://elpa.mpcdf.mpg.
de/html/Releases/2016.11.001.pre/elpa-2016.11.001.pre.tar.gz
Anyway, I did managed to overcome the problem simply using,
./configure --enable-parallel --with-scalapack=intel CFLAGS="-O3
-I$MKLROOT/include/fftw -xCORE-AVX2" CXXFLAGS="-O3 -I$MKLROOT/include/fftw
-xCORE-AVX2" FCFLAGS="-O3 -I$MKLROOT/include/fftw -xCORE-AVX2" F90FLAGS="-O3
-I$MKLROOT/include/fftw -xCORE-AVX2" CC=mpicc CXX=mpicxx FC=mpif90
F90=mpif90
Since I am testing on a single node, ELPA is not that important. If our
cluster has only 7 compute nodes, I think scalapack may be sufficient?
Please correct me if I am wrong. Thank you!
Regards,
Rolly
PhD, Research Fellow,
Department of Physics and Materials Science,
City University of Hong Kong
Tel: +852 3442 4000
Fax: +852 3442 0538
-----Original Message-----
From: pw_forum-bounces at pwscf.org [mailto:pw_forum-bounces at pwscf.org] On
Behalf Of Ari P Seitsonen
Sent: 2017年3月7日 16:51
To: PWSCF Forum
Subject: Re: [Pw_forum] Problem while compiling QE v5.3.0 with ELPA
Dear Rolly Ng,
First a question: Why do you try to compile v5.3.0, in particular with
pre-processor options that are meaningful only in newer versions of QE
("-D__EXX_ACE", probably also "-D__ELPA3" and "-D__ELPA_2016")?
Indeed the directory where the files 'fft_defs.h' is not in the options
with '-I'; if you want to enforce it, you can always add the options in
'make.sys' (nowadays 'make.inc') with
"-I/home/qeuser/QE530-CPU/espresso-5.3.0/include", but I would first
recommend you to try to get a consistent set of pre-processor flags (and,
preferably, v6.1 if you can).
I also do not know/remember if "fft_defs.h" has anything to do with ELPA;
which version of ELPA do you try to include, is it something that existed
when Quantum ESPRESSO v5.3.0 was developed? It could be that something has
changed in the interface to ELPA in the meantime too.
Greetings,
apsi
-=*=-=*=-=*=-=*=-=*=-=*=-=*=-=*=-=*=-=*=-=*=-=*=-=*=-=*=-=*=-=*=-=*=-=*=-=*=
-
Ari Paavo Seitsonen / Ari.P.Seitsonen at iki.fi / http://www.iki.fi/~apsi/
Ecole Normale Supérieure (ENS), Département de Chimie, Paris
Mobile (F) : +33 789 37 24 25 (CH) : +41 79 71 90 935
On Tue, 7 Mar 2017, Rolly Ng wrote:
> Dear QE users,
>
> I am trying to compile QE v5.3.0 with ELPA by following the steps
> instructed here,
>
> https://software.intel.com/en-us/articles/quantum-espresso-for-the-int
> el-xeon-phi-processor
>
> But I am having unexpected error that fft_scalar.f90 complains unable
> to find fft_def.h?
>
> It stops with:
> mpiifort -O2 -align array64byte -threads -heap-arrays 4096 -xCORE-AVX2
> -fp-model fast=2 -complex-limited-range -assume byterecl -nomodule
> -fpp -D__INTEL -D__FFTW3 -D__DFTI -D__NON_BLOCKING_SCATTER -D__EXX_ACE
> -D__MPI -D__PARA -D__SCALAPACK -D__ELPA3 -D__ELPA_2016
> -I../../iotk/src -I../../Modules -I../../FFTXlib -c
> print_ks_energies.f90 mpiifort -O2 -align array64byte -threads
> -heap-arrays 4096 -xCORE-AVX2 -fp-model fast=2 -complex-limited-range
> -assume byterecl -nomodule -fpp -D__INTEL -D__FFTW3 -D__DFTI
> -D__NON_BLOCKING_SCATTER -D__EXX_ACE -D__MPI -D__PARA -D__SCALAPACK
> -D__ELPA3 -D__ELPA_2016 -I../../iotk/src -I../../Modules
> -I../../FFTXlib -c punch.f90 mpiifort -O2 -align array64byte -threads
> -heap-arrays 4096 -xCORE-AVX2 -fp-model fast=2 -complex-limited-range
> -assume byterecl -nomodule -fpp -D__INTEL -D__FFTW3 -D__DFTI
> -D__NON_BLOCKING_SCATTER -D__EXX_ACE -D__MPI -D__PARA -D__SCALAPACK
> -D__ELPA3 -D__ELPA_2016 -I../../iotk/src -I../../Modules
> -I../../FFTXlib -c
> ../../FFTXlib/fft_scalar.f90
> fft_scalar.f90(18): #error: can't find include file: fft_defs.h
> make[2]: *** [../../FFTXlib/fft_scalar.o] Error 1
> make[2]: Leaving directory `/home/qeuser/QE530-CPU/espresso-5.3.0/PW/src'
> make[1]: *** [pw] Error 1
> make[1]: Leaving directory `/home/qeuser/QE530-CPU/espresso-5.3.0/PW'
> make: *** [pw] Error 1
>
> But I am sure fft_def.h is located at
> /home/qeuser/QE530-CPU/espresso-5.3.0/include, but what can't it be
> located by the fft_scalar.f90?
>
> Please have a look at my make.sys attached.
>
> # make.sys. Generated from make.sys.in by configure.
>
> # compilation rules
>
> .SUFFIXES :
> .SUFFIXES : .o .c .f .f90
>
> # most fortran compilers can directly preprocess c-like directives: use
> # $(MPIF90) $(F90FLAGS) -c $<
> # if explicit preprocessing by the C preprocessor is needed, use:
> # $(CPP) $(CPPFLAGS) $< -o $*.F90
> # $(MPIF90) $(F90FLAGS) -c $*.F90 -o $*.o
> # remember the tabulator in the first column !!!
>
> .f90.o:
> $(MPIF90) $(F90FLAGS) -c $<
>
> # .f.o and .c.o: do not modify
>
> .f.o:
> $(F77) $(FFLAGS) -c $<
>
> .c.o:
> $(CC) $(CFLAGS) -c $<
>
>
>
> # topdir for linking espresso libs with plugins TOPDIR =
> /home/qeuser/QE530-CPU/espresso-5.3.0
>
> # DFLAGS = precompilation options (possible arguments to -D and -U)
> # used by the C compiler and preprocessor
> # FDFLAGS = as DFLAGS, for the f90 compiler # See
> include/defs.h.README for a list of options and their meaning # With
> the exception of IBM xlf, FDFLAGS = $(DFLAGS) # For IBM xlf, FDFLAGS
> is the same as DFLAGS with separating commas
>
> # MANUAL_DFLAGS = additional precompilation option(s), if desired
> # You may use this instead of tweaking DFLAGS and FDFLAGS
> # BEWARE: will not work for IBM xlf! Manually edit
FDFLAGS
> MANUAL_DFLAGS =
> DFLAGS = -D__INTEL -D__FFTW3 -D__DFTI -D__NON_BLOCKING_SCATTER
> -D__EXX_ACE -D__MPI -D__PARA -D__SCALAPACK -D__ELPA3 -D__ELPA_2016
> $(MANUAL_DFLAGS)
> FDFLAGS = $(DFLAGS) $(MANUAL_DFLAGS)
>
> # IFLAGS = how to locate directories where files to be included are #
> In most cases, IFLAGS = -I../include
>
> IFLAGS = -I../include -I$(MKLROOT)/include/fftw
> -I/home/qeuser/QE530-CPU/espresso-5.3.0/../elpa/default-hsw/include/el
> pa/modules
>
>
> # MOD_FLAGS = flag used by f90 compiler to locate modules # Each
> Makefile defines the list of needed modules in MODFLAGS
>
> MOD_FLAG = -I
>
> # Compilers: fortran-90, fortran-77, C # If a parallel compilation is
> desired, MPIF90 should be a fortran-90 # compiler that produces
> executables for parallel execution using MPI # (such as for instance
> mpif90, mpf90, mpxlf90,...); # otherwise, an ordinary fortran-90
> compiler (f90, g95, xlf90, ifort,...) # If you have a parallel machine
> but no suitable candidate for MPIF90, # try to specify the directory
> containing "mpif.h" in IFLAGS # and to specify the location of MPI
> libraries in MPI_LIBS
>
> MPIF90 = mpiifort
> #F90 = ifort
> CC = mpiicc
> F77 = ifort
>
> # C preprocessor and preprocessing flags - for explicit preprocessing,
> # if needed (see the compilation rules above) # preprocessing flags
> must include DFLAGS and IFLAGS
>
> CPP = cpp
> CPPFLAGS = -P -C -traditional $(DFLAGS) $(IFLAGS)
>
> # compiler flags: C, F90, F77
> # C flags must include DFLAGS and IFLAGS # F90 flags must include
> MODFLAGS, IFLAGS, and FDFLAGS with appropriate syntax
>
> CFLAGS = -O3 -xCORE-AVX2 -fp-model fast=2
> -complex-limited-range -fno-alias -ansi-alias $(DFLAGS) $(IFLAGS)
> F90FLAGS = $(FFLAGS) -nomodule -fpp $(FDFLAGS) $(IFLAGS) $(MODFLAGS)
> FFLAGS = -O2 -align array64byte -threads -heap-arrays 4096
> -xCORE-AVX2 -fp-model fast=2 -complex-limited-range -assume byterecl
>
> # compiler flags without optimization for fortran-77 # the latter is
> NEEDED to properly compile dlamch.f, used by lapack
>
> FFLAGS_NOOPT = -O0 -assume byterecl -g -traceback
>
> # compiler flag needed by some compilers when the main is not fortran
> # Currently used for Yambo
>
> FFLAGS_NOMAIN = -nofor_main
>
> # Linker, linker-specific flags (if any) # Typically LD coincides with
> F90 or MPIF90, LD_LIBS is empty
>
> LD = mpiifort
> LDFLAGS = -static-intel -static-libgcc -static-libstdc++
> LD_LIBS = -Wl,--as-needed -liomp5 -Wl,--no-as-needed
>
> # External Libraries (if any) : blas, lapack, fft, MPI
>
> # If you have nothing better, use the local copy :
> # BLAS_LIBS = /your/path/to/espresso/BLAS/blas.a
> # BLAS_LIBS_SWITCH = internal
>
> BLAS_LIBS = -Wl,--start-group
> /opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/li
> bmkl_intel_lp64.a
> /opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/li
> bmkl_core.a
> /opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/li
> bmkl_sequential.a
> /opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/li
> bmkl_blacs_intelmpi_lp64.a
> -Wl,--end-group
> BLAS_LIBS_SWITCH = external
>
> # If you have nothing better, use the local copy :
> # LAPACK_LIBS = /your/path/to/espresso/lapack-3.2/lapack.a
> # LAPACK_LIBS_SWITCH = internal
> # For IBM machines with essl (-D__ESSL): load essl BEFORE lapack !
> # remember that LAPACK_LIBS precedes BLAS_LIBS in loading order
>
> LAPACK_LIBS = -Wl,--start-group
> /opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/li
> bmkl_intel_lp64.a
> /opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/li
> bmkl_core.a
> /opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/li
> bmkl_sequential.a
> /opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/li
> bmkl_blacs_intelmpi_lp64.a
> -Wl,--end-group
> LAPACK_LIBS_SWITCH = external
>
> ELPA_LIBS_SWITCH = disabled
> SCALAPACK_LIBS =
> /home/qeuser/QE530-CPU/espresso-5.3.0/../elpa/default-hsw/lib/libelpa.
> a
> /opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/li
> bmkl_scalapack_lp64.a
>
> # nothing needed here if the the internal copy of FFTW is compiled #
> (needs -D__DFTI -D__NON_BLOCKING_SCATTER -D__EXX_ACE in DFLAGS)
>
> FFT_LIBS = -Wl,--start-group
> /opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/li
> bmkl_intel_lp64.a
> /opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/li
> bmkl_core.a
> /opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/li
> bmkl_sequential.a
> /opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/li
> bmkl_blacs_intelmpi_lp64.a
> -Wl,--end-group
>
> # For parallel execution, the correct path to MPI libraries must # be
> specified in MPI_LIBS (except for IBM if you use mpxlf)
>
> MPI_LIBS =
>
> # IBM-specific: MASS libraries, if available and if -D__MASS is
> defined in FDFLAGS
>
> MASS_LIBS =
>
> # ar command and flags - for most architectures: AR = ar, ARFLAGS =
> ruv
>
> AR = xiar
> ARFLAGS = ruv
>
> # ranlib command. If ranlib is not needed (it isn't in most cases) use
> # RANLIB = echo
>
> RANLIB = ranlib
>
> # all internal and external libraries - do not modify
>
> FLIB_TARGETS = all
>
> LIBOBJS = ../flib/ptools.a ../flib/flib.a ../clib/clib.a
> ../iotk/src/libiotk.a
> LIBS = $(SCALAPACK_LIBS) $(LAPACK_LIBS) $(FFT_LIBS)
> $(BLAS_LIBS) $(MPI_LIBS) $(MASS_LIBS) $(LD_LIBS)
>
> # wget or curl - useful to download from network WGET = wget -O
>
> # Install directory
> PREFIX = /usr/local
>
> ######################################################################
> #######
> # Copyright (c) 2014-2017, Intel
> Corporation #
> # All rights
> reserved. #
> # #
> # Redistribution and use in source and binary forms, with or
> without #
> # modification, are permitted provided that the following
> conditions #
> # are met: #
> # 1. Redistributions of source code must retain the above
> copyright #
> # notice, this list of conditions and the following
> disclaimer. #
> # 2. Redistributions in binary form must reproduce the above
> copyright #
> # notice, this list of conditions and the following disclaimer in
> the #
> # documentation and/or other materials provided with the
> distribution. #
> # 3. Neither the name of the copyright holder nor the names of
> its #
> # contributors may be used to endorse or promote products
> derived #
> # from this software without specific prior written
> permission. #
> # #
> # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> CONTRIBUTORS #
> # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT
> NOT #
> # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> FOR #
> # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> COPYRIGHT #
> # HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> INCIDENTAL, #
> # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> LIMITED # # TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
> USE, DATA,
> OR #
> # PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
> OF #
> # LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> (INCLUDING #
> # NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
> THIS #
> # SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> DAMAGE. #
> ######################################################################
> #######
> # Hans Pabst (Intel Corp.)
> ######################################################################
> #######
>
> # TBB runtime made for oldest/latest supported GCC TBBGCC_OLD ?= 1 #
> ultimately disable/enable Intel TBB's malloc proxy TBBMALLOC ?= 0
>
> ifneq (0,$(TBBMALLOC))
> ifneq (,$(TBBROOT))
> GCC = $(notdir $(shell which gcc 2> /dev/null))
> ifneq (,$(GCC))
> GCC_VERSION_STRING = $(shell $(GCC) --version 2> /dev/null |
> head
> -n1 | sed "s/..* \([0-9][0-9]*\.[0-9][0-9]*\.*[0-9]*\)[ \S]*.*/\1/")
> GCC_VERSION_MAJOR = $(shell echo "$(GCC_VERSION_STRING)" | cut
> -d"." -f1)
> GCC_VERSION_MINOR = $(shell echo "$(GCC_VERSION_STRING)" | cut
> -d"." -f2)
> GCC_VERSION_PATCH = $(shell echo "$(GCC_VERSION_STRING)" | cut
> -d"." -f3)
> TBBLIBDIR =
> $(TBBROOT)/lib/intel64/gcc$(GCC_VERSION_MAJOR).$(GCC_VERSION_MINOR)
> TBBMALLOCLIB = $(wildcard $(TBBLIBDIR)/libtbbmalloc_proxy.so)
> endif
> ifeq (,$(TBBMALLOCLIB))
> ifneq (0,$(TBBGCC_OLD))
> TBBGCCDIR = $(shell ls -1 "$(TBBROOT)/lib/intel64" | tr "\n" "
> " | cut -d" " -f1)
> else
> TBBGCCDIR = $(shell ls -1 "$(TBBROOT)/lib/intel64" | tr "\n" "
> " | rev | cut -d" " -f2 | rev)
> endif
> TBBLIBDIR = $(TBBROOT)/lib/intel64/$(TBBGCCDIR)
> TBBMALLOCLIB = $(wildcard $(TBBLIBDIR)/libtbbmalloc_proxy.so)
> endif
> ifneq (,$(TBBMALLOCLIB))
> LIBS += $(TBBMALLOCLIB) $(TBBLIBDIR)/libtbbmalloc.so
> ifneq (1,$(TBBMALLOC)) # TBBMALLOC=2
> FCFLAGS += -heap-arrays
> endif
> endif
> endif
> endif
>
> default: all
>
> init_us_1.o: init_us_1.f90
> $(MPIF90) $(F90FLAGS) -O1 -c $<
>
> new_ns.o: new_ns.f90
> $(MPIF90) $(F90FLAGS) -O1 -c $<
>
> us_exx.o: us_exx.f90
> $(MPIF90) $(F90FLAGS) -qopenmp -qoverride_limits -c $<
>
> wypos.o: wypos.f90
> $(MPIF90) $(F90FLAGS) -qopenmp -qoverride_limits -O0 -c $<
>
> lr_apply_liouvillian.o: lr_apply_liouvillian.f90
> $(MPIF90) $(F90FLAGS) -qopenmp -qoverride_limits -O1 -c $<
>
> realus.o: realus.f90
> $(MPIF90) $(F90FLAGS) -qopenmp -qoverride_limits -O1 -c $<
>
>
> Thank you,
> Rolly
>
> --
> PhD. Research Fellow,
> Dept. of Physics & Materials Science,
> City University of Hong Kong
> Tel: +852 3442 4000
> Fax: +852 3442 0538
>
> _______________________________________________
> Pw_forum mailing list
> Pw_forum at pwscf.org
> http://pwscf.org/mailman/listinfo/pw_forum
>
More information about the users
mailing list