[Q-e-developers] Compilation on FERMI/BGQ

[Carlo Cavazzoni]

Dear all,

to compile QE on FERMI(BlueGene Q machine) with
optimal parameters, from what I've seen till now,
proceed as follow:

module load bgq-xl/1.0
module load essl/5.1
module load lapack/3.4.1--bgq-xl--1.0
module load scalapack/2.0.2--bgq-xl--1.0
module load mass/7.3--bgq-xl--1.0

and then use the make.sys you'll find in attachment.

During the compilation the compiler will complain with the error:
"functionals.f90", 1500-004 (U) INTERNAL COMPILER ERROR while compiling 
wpbe_analy_erfc_approx_grad.  Compilation ended.  Contact your Service 
Representative and provide the following information: GRARNN: fp885 is 
used before it is defined. For more information visit: 
http://www.ibm.com/support/docview.wss?uid=swg21110810
1586-346 (U) An error occurred during code generation.  The code 
generation return code was 1.
1501-511  Compilation failed for file functionals.f90.
make[1]: *** [functionals.o] Error 1
make[1]: Leaving directory `/fermi/home/userinternal/ccavazzo/espresso/flib'
make: *** [libs] Error 2

go to the directory "flib", and compile "by hand" functionals.f90 with 
the command:
[ccavazzo at fen01 flib]$ mpixlf90_r -qalias=noaryovrlp:nointptr -O2 
-qstrict -qdpc=e -qsuffix=cpp=f90 
-WF,-D__XLF,-D__FFTW,-D__MASS,-D__MPI,-D__PARA,-D__SCALAPACK,-D__OPENMP 
-I../include -I../iotk/src -I../Modules -I. -c functionals.f90

go back to the espresso home dir and restart with "make".

PS: I've also installed QE on FERMI among the general available software.
You can test it with the command:
module load qe/5.0bgq



-- 
Ph.D. Carlo Cavazzoni
SuperComputing Applications and Innovation Department
CINECA - Via Magnanelli 6/3, 40033 Casalecchio di Reno (Bologna)
Tel: +39 051 6171411  Fax: +39 051 6132198
www.cineca.it

-------------- next part --------------
# make.sys.  Generated from make.sys.in by configure.

# compilation rules

.SUFFIXES :
.SUFFIXES : .o .c .f .f90 .cu

# most fortran compilers can directly preprocess c-like directives: use
# 	$(MPIF90) $(F90FLAGS) -c $<
# if explicit preprocessing by the C preprocessor is needed, use:
# 	$(CPP) $(CPPFLAGS) $< -o $*.F90 
#	$(MPIF90) $(F90FLAGS) -c $*.F90 -o $*.o
# remember the tabulator in the first column !!!

.f90.o:
	$(MPIF90) $(F90FLAGS) -c $<

# .f.o and .c.o: do not modify

.f.o:
	$(F77) $(FFLAGS) -c $<

.c.o:
	$(CC) $(CFLAGS)  -c $<

# CUDA files
.cu.o:
	$(NVCC) $(NVCCFLAGS) -I../../include $(IFLAGS) $(DFLAGS)   -c $<


# DFLAGS  = precompilation options (possible arguments to -D and -U)
#           used by the C compiler and preprocessor
# FDFLAGS = as DFLAGS, for the f90 compiler
# See include/defs.h.README for a list of options and their meaning
# With the exception of IBM xlf, FDFLAGS = $(DFLAGS)
# For IBM xlf, FDFLAGS is the same as DFLAGS with separating commas 

# -D__BGQ is a work-around for IBM-BlueGene/Q specific getarg (flib/inpfile.f90)

MANUAL_DFLAGS  = 
DFLAGS         = -D__XLF -D__FFTW -D__MASS -D__MPI -D__PARA -D__SCALAPACK -D__OPENMP
FDFLAGS        = -D__XLF,-D__FFTW,-D__MASS,-D__MPI,-D__PARA,-D__SCALAPACK,-D__OPENMP


# IFLAGS = how to locate directories where files to be included are
# In most cases, IFLAGS = -I../include

IFLAGS         = -I../include

# MOD_FLAGS = flag used by f90 compiler to locate modules
# Each Makefile defines the list of needed modules in MODFLAGS

MOD_FLAG      = -I

# Compilers: fortran-90, fortran-77, C
# If a parallel compilation is desired, MPIF90 should be a fortran-90 
# compiler that produces executables for parallel execution using MPI
# (such as for instance mpif90, mpf90, mpxlf90,...);
# otherwise, an ordinary fortran-90 compiler (f90, g95, xlf90, ifort,...)
# If you have a parallel machine but no suitable candidate for MPIF90,
# try to specify the directory containing "mpif.h" in IFLAGS
# and to specify the location of MPI libraries in MPI_LIBS

MPIF90         = mpixlf90_r
#F90           = bgxlf90_r
CC             = bgxlc_r
F77            = bgxlf_r

# C preprocessor and preprocessing flags - for explicit preprocessing, 
# if needed (see the compilation rules above)
# preprocessing flags must include DFLAGS and IFLAGS

CPP            = cpp
CPPFLAGS       = -P -traditional $(DFLAGS) $(IFLAGS)

# compiler flags: C, F90, F77
# C flags must include DFLAGS and IFLAGS
# F90 flags must include MODFLAGS, IFLAGS, and FDFLAGS with appropriate syntax

CFLAGS         = -O3 -qsimd=auto -qarch=qp -qtune=qp $(DFLAGS) $(IFLAGS)
F90FLAGS       = $(FFLAGS) -qsuffix=cpp=f90 -WF,$(FDFLAGS) $(IFLAGS) $(MODFLAGS)
FFLAGS         = -qalias=noaryovrlp:nointptr -O3 -qstrict -qdpc=e -qsmp=noauto:omp -qthreaded -qarch=qp -qtune=qp -qtm -qsimd=auto

# compiler flags without optimization for fortran-77
# the latter is NEEDED to properly compile dlamch.f, used by lapack

FFLAGS_NOOPT   = -O0

# compiler flag needed by some compilers when the main is not fortran
# Currently used for Yambo

FFLAGS_NOMAIN   = 

# Linker, linker-specific flags (if any)
# Typically LD coincides with F90 or MPIF90, LD_LIBS is empty

LD             = mpixlf90_r
LDFLAGS        = -qstatic -qsmp=omp -qtm -qthreaded -Wl,--allow-multiple-definition 
#LDFLAGS        = -qstatic -qsmp=omp -qtm -qthreaded -Wl,--allow-multiple-definition -Wl,--dy -qnosave -qnostaticlink 
LD_LIBS        = 

# External Libraries (if any) : blas, lapack, fft, MPI

# If you have nothing better, use the local copy :
# BLAS_LIBS = /your/path/to/espresso/BLAS/blas.a
# BLAS_LIBS_SWITCH = internal

BLAS_LIBS      = -L$(ESSL_LIB) -lesslsmpbg 
BLAS_LIBS_SWITCH = external

# If you have nothing better, use the local copy :
# LAPACK_LIBS = /your/path/to/espresso/lapack-3.2/lapack.a
# LAPACK_LIBS_SWITCH = internal
# For IBM machines with essl (-D__ESSL): load essl BEFORE lapack !
# remember that LAPACK_LIBS precedes BLAS_LIBS in loading order

LAPACK_LIBS    = 
LAPACK_LIBS_SWITCH = external

SCALAPACK_LIBS = -L$(SCALAPACK_LIB) -lscalapack -L$(LAPACK_LIB) -llapack

# nothing needed here if the the internal copy of FFTW is compiled
# (needs -D__FFTW in DFLAGS)

FFT_LIBS       = 

# For parallel execution, the correct path to MPI libraries must
# be specified in MPI_LIBS (except for IBM if you use mpxlf)

MPI_LIBS       = 

# IBM-specific: MASS libraries, if available and if -D__MASS is defined in FDFLAGS

MASS_LIBS      = $(MASS_LIB)/libmassv.a $(MASS_LIB)/libmass_simd.a

# ar command and flags - for most architectures: AR = ar, ARFLAGS = ruv

AR             = ar
ARFLAGS        = ruv

# ranlib command. If ranlib is not needed (it isn't in most cases) use
# RANLIB = echo

RANLIB         = ranlib

# all internal and external libraries - do not modify

FLIB_TARGETS   = all

# CUDA section (flags temporary hard-coded)
NVCC            = 
NVCCFLAGS       =  

PHIGEMM_ENABLED = 
MAGMA_ENABLED   = 

LIBOBJS        = ../flib/ptools.a ../flib/flib.a ../clib/clib.a ../iotk/src/libiotk.a 
LIBS           = $(SCALAPACK_LIBS) $(LAPACK_LIBS) $(FFT_LIBS) $(BLAS_LIBS) $(MPI_LIBS) $(MASS_LIBS) $(LD_LIBS)

# wget or curl - useful to download from network
WGET = wget -O

# topdir for linking espresso libs with plugins
TOPDIR = /fermi/home/userinternal/ccavazzo/espresso