[QE-developers] Bug in non self-consistent calculation
Lorenzo Monacelli
mesonepigreco at gmail.com
Wed Jun 12 15:55:20 CEST 2019
Dear all,
Thank you for your replies.
I have two different versions on two different machines. The one I sent
you my results was compiled with gfortran and standard lapack/blas as
provided from the ubuntu-software library (16.04). The other is compiled
with the intel compiler and MKL and runs on a cluster. Both of them
experienced the issue quite randomly. I attach my make.inc of both
compilations.
The same issue, at least on a different input file, was experienced by
the pw.x (v 6.2) already pre-installed in the Spanish MARENOSTRUM
cluster, that I assume was correctly compiled.
I noticed that a way to reproduce the error is asking for many bands in
the nscf calculation in a system with many atoms (with few symmetries)
in the cell ( with 96 atoms it almost impossible for me to run a nscf
calculation).
It is possible that the different behavior on different machines is
actually suggesting that the bug could be located in some variable
ill-initialized (that its automatic initialization is maybe demanded to
the compiler)?
Another question: How does cdiaghg work? I assumed that the S matrix
should be the identity for local norm conserving pseudos and GGA xc
functionals, but if I enforce it to be the identity at the begining of
the subroutine the code is no more able to converge any calculation
(even in the scf, where now it works). I am a bit skeptical thinking
that this is just an error of LAPACK or MPI: why does SCF with the same
input (that should solve the same problem as the nscf but many times)
works very well (even with many atoms and even if I ask many bands)?
Bests,
Lorenzo
On 12/06/19 14:37, Paolo Giannozzi wrote:
> I was about to write the same, before noticing that the crash occurs
> randomly (one run completes, a subsequent one doesn't). Unless some
> regularity is found (that is: under conditions xyz, the code always
> crashes) it will be impossible to locate the origin of the problem.
> Note that the origin of the problem might well be in mathematical
> libraries, or in MPI. I am 100% sure that in at least some cases
> diagonalization failures were due to some misbehavior of mathematical
> libraries (but this was many years ago, on machines that do not exist
> any longer). Also: a frequent source of random crashes in parallel
> execution is explained in sec.7.3 of the developer manual,
> http://www.quantum-espresso.org/Doc/developer_man/developer_man.html#SECTION00080000000000000000
>
> Paolo
>
> On Wed, Jun 12, 2019 at 2:03 PM Davide Ceresoli
> <davide.ceresoli at cnr.it <mailto:davide.ceresoli at cnr.it>> wrote:
>
> Dear Lorenzo,
> is your QE compiled with a decent compiler and with decent
> libraries?
> Your inputs works perfectly for me, with no crashes.
>
> HTH.
> D.
>
>
>
> On 6/12/19 12:29 PM, Lorenzo Monacelli wrote:
> > Dear QE developers,
> >
> > I think I found a bad bug in the non self-consistent calculation
> of pw.x
> >
> > While the self consistent calculation ends properly, when
> running a non
> > self-consistent calculation results in a crash with the error:
> >
> >
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> > task # 0
> > from cdiaghg : error # 40
> > S matrix not positive definite
> >
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> >
> > I checked the cdiaghg subroutine, the S matrix should be the
> overlap matrix for
> > the eigenvalue problem Hv = eSv
> >
> > That, in case of local Norm Conserving pseudo of Hydrogen (my
> calculation) I
> > suppose it should be the identity, however, if I enforce it to
> be the indentity
> > at the beginning of cdiaghg the code says that it is not able to
> converge the
> > scf caclulation either.
> >
> > I attach the input of the scf calculation (that converges) and
> the one of the
> > non-self-consistent calculation (that produces this output).
> >
> > I also tried to switch the diagonalization method to cg as
> suggested as fix, but
> > nothing changes.
> >
> > I modified also the cdiaghg subroutine, to print the S matrix,
> that you find
> > attached (random numbers, seems to be uninitialized).
> >
> > In both the diagonalization methods if I enforce S to be the
> identity matrix the
> > code crashes by saying that it was not able to converge.
> >
> > The problem seems to arise especially if I request for more
> bands with the nbnd
> > flag in system (but sometimes it occurs even if no extra band is
> required).
> >
> > The QE version I used is the current version in the develop
> branch of gitlab,
> > but I noticed the same error occurring also with 6.3 and 6.2 in
> other cases.
> >
> > If I ask for exactly the same input file a scf calculation
> (instead of a nscf)
> > everything goes fine (same K points, same diagonalization, same
> number of
> > extrabands), but indeed, this is not what I would like to do...
> >
> > I I run the nscf calculation after a scf calculation with
> exactly the same input
> > (that works), the nscf calculation fails (this means that the
> crash is not
> > caused by a bad starting point for the density).
> >
> > All these make me really think of a bug in the nscf calculation,
> rather than a
> > wrong input.
> >
> > Best regards,
> >
> > Lorenzo Monacelli
> >
> >
> > P.S.
> >
> > In the attached file the pw_* are the nscf input and output, the
> scf* are the
> > scf input and output. I run
> >
> >
> >
> > _______________________________________________
> > developers mailing list
> > developers at lists.quantum-espresso.org
> <mailto:developers at lists.quantum-espresso.org>
> > https://lists.quantum-espresso.org/mailman/listinfo/developers
> >
>
> --
> +--------------------------------------------------------------+
> Davide Ceresoli
> CNR Institute of Molecular Science and Technology (CNR-ISTM)
> c/o University of Milan, via Golgi 19, 20133 Milan, Italy
> Email: davide.ceresoli at cnr.it <mailto:davide.ceresoli at cnr.it>
> Phone: +39-02-50314276, +39-347-1001570 (mobile)
> Skype: dceresoli
> Website: http://sites.google.com/site/dceresoli/
> +--------------------------------------------------------------+
> _______________________________________________
> developers mailing list
> developers at lists.quantum-espresso.org
> <mailto:developers at lists.quantum-espresso.org>
> https://lists.quantum-espresso.org/mailman/listinfo/developers
>
>
>
> --
> Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,
> Univ. Udine, via delle Scienze 208, 33100 Udine, Italy
> Phone +39-0432-558216, fax +39-0432-558222
>
>
> _______________________________________________
> developers mailing list
> developers at lists.quantum-espresso.org
> https://lists.quantum-espresso.org/mailman/listinfo/developers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20190612/30ab1c47/attachment-0001.html>
-------------- next part --------------
# make.inc. Generated from make.inc.in by configure.
# compilation rules
.SUFFIXES :
.SUFFIXES : .o .c .f .f90
# most fortran compilers can directly preprocess c-like directives: use
# $(MPIF90) $(F90FLAGS) -c $<
# if explicit preprocessing by the C preprocessor is needed, use:
# $(CPP) $(CPPFLAGS) $< -o $*.F90
# $(MPIF90) $(F90FLAGS) -c $*.F90 -o $*.o
# remember the tabulator in the first column !!!
.f90.o:
$(MPIF90) $(F90FLAGS) -c $<
# .f.o and .c.o: do not modify
.f.o:
$(F77) $(FFLAGS) -c $<
.c.o:
$(CC) $(CFLAGS) -c $<
# Top QE directory, useful for locating libraries, linking QE with plugins
# The following syntax should always point to TOPDIR:
TOPDIR = $(dir $(abspath $(filter %make.inc,$(MAKEFILE_LIST))))
# if it doesn't work, uncomment the following line (edit if needed):
# TOPDIR = /home/darth-vader/Documenti/Programmazione/q-e
# DFLAGS = precompilation options (possible arguments to -D and -U)
# used by the C compiler and preprocessor
# To use libxc (v>=3.0.1), add -D__LIBXC to DFLAGS
# See include/defs.h.README for a list of options and their meaning
# With the exception of IBM xlf, FDFLAGS = $(DFLAGS)
# For IBM xlf, FDFLAGS is the same as DFLAGS with separating commas
# MANUAL_DFLAGS = additional precompilation option(s), if desired
# BEWARE: it does not work for IBM xlf! Manually edit FDFLAGS
MANUAL_DFLAGS =
DFLAGS = -D__FFTW -D__MPI
FDFLAGS = $(DFLAGS) $(MANUAL_DFLAGS)
# IFLAGS = how to locate directories with *.h or *.f90 file to be included
# typically -I$(TOPDIR)/include -I/some/other/directory/
# the latter contains .e.g. files needed by FFT libraries
# for libxc add -I/path/to/libxc/include/
IFLAGS = -I$(TOPDIR)/include -I$(TOPDIR)/FoX/finclude -I$(TOPDIR)/S3DE/iotk/include/
# MOD_FLAG = flag used by f90 compiler to locate modules
MOD_FLAG = -I
# BASEMOD_FLAGS points to directories containing basic modules,
# while BASEMODS points to the corresponding module libraries
# Each Makefile can add directories to MODFLAGS and libraries to QEMODS
BASEMOD_FLAGS= $(MOD_FLAG)$(TOPDIR)/iotk/src \
$(MOD_FLAG)$(TOPDIR)/Modules \
$(MOD_FLAG)$(TOPDIR)/FFTXlib \
$(MOD_FLAG)$(TOPDIR)/LAXlib \
$(MOD_FLAG)$(TOPDIR)/UtilXlib \
$(MOD_FLAG)$(TOPDIR)/FoX/finclude
# Compilers: fortran-90, fortran-77, C
# If a parallel compilation is desired, MPIF90 should be a fortran-90
# compiler that produces executables for parallel execution using MPI
# (such as for instance mpif90, mpf90, mpxlf90,...);
# otherwise, an ordinary fortran-90 compiler (f90, g95, xlf90, ifort,...)
# If you have a parallel machine but no suitable candidate for MPIF90,
# try to specify the directory containing "mpif.h" in IFLAGS
# and to specify the location of MPI libraries in MPI_LIBS
MPIF90 = mpif90
F90 = gfortran
CC = gcc
F77 = gfortran
# GPU architecture (Kepler: 35, Pascal: 60, Volta: 70 )
GPU_ARCH=
# CUDA runtime (Pascal: 8.0, Volta: 9.0)
CUDA_RUNTIME=
# CUDA F90 Flags
CUDA_F90FLAGS=
# C preprocessor and preprocessing flags - for explicit preprocessing,
# if needed (see the compilation rules above)
# preprocessing flags must include DFLAGS and IFLAGS
CPP = cpp
CPPFLAGS = -P -traditional $(DFLAGS) $(IFLAGS)
# compiler flags: C, F90, F77
# C flags must include DFLAGS and IFLAGS
# F90 flags must include MODFLAGS, IFLAGS, and FDFLAGS with appropriate syntax
CFLAGS = -O3 $(DFLAGS) $(IFLAGS)
F90FLAGS = $(FFLAGS) -x f95-cpp-input $(FDFLAGS) $(CUDA_F90FLAGS) $(IFLAGS) $(MODFLAGS)
FFLAGS = -O3 -g
# compiler flags without optimization for fortran-77
# the latter is NEEDED to properly compile dlamch.f, used by lapack
FFLAGS_NOOPT = -O0 -g
# compiler flag needed by some compilers when the main program is not fortran
# Currently used for Yambo
FFLAGS_NOMAIN =
# Linker, linker-specific flags (if any)
# Typically LD coincides with F90 or MPIF90, LD_LIBS is empty
# for libxc, set LD_LIBS=-L/path/to/libxc/lib/ -lxcf90 -lxc
LD = mpif90
LDFLAGS = -g
LD_LIBS =
# External Libraries (if any) : blas, lapack, fft, MPI
# If you have nothing better, use the local copy via "--with-netlib" :
# BLAS_LIBS = /your/path/to/espresso/LAPACK/blas.a
# BLAS_LIBS_SWITCH = internal
BLAS_LIBS = -lblas
BLAS_LIBS_SWITCH = external
# If you have nothing better, use the local copy via "--with-netlib" :
# LAPACK_LIBS = /your/path/to/espresso/LAPACK/lapack.a
# LAPACK_LIBS_SWITCH = internal
# For IBM machines with essl (-D__ESSL): load essl BEFORE lapack !
# remember that LAPACK_LIBS precedes BLAS_LIBS in loading order
LAPACK_LIBS = -llapack -lblas
LAPACK_LIBS_SWITCH = external
SCALAPACK_LIBS =
# nothing needed here if the the internal copy of FFTW is compiled
# (needs -D__FFTW in DFLAGS)
FFT_LIBS =
# HDF5
HDF5_LIB =
FOX_LIB = -L$(TOPDIR)/FoX/lib -lFoX_dom -lFoX_sax -lFoX_wxml -lFoX_common\
-lFoX_utils -lFoX_fsys
FOX_FLAGS =
# For parallel execution, the correct path to MPI libraries must
# be specified in MPI_LIBS (except for IBM if you use mpxlf)
MPI_LIBS =
# IBM-specific: MASS libraries, if available and if -D__MASS is defined in FDFLAGS
MASS_LIBS =
# CUDA libraries
CUDA_LIBS=
CUDA_EXTLIBS =
# ar command and flags - for most architectures: AR = ar, ARFLAGS = ruv
AR = ar
ARFLAGS = ruv
# ranlib command. If ranlib is not needed (it isn't in most cases) use
# RANLIB = echo
RANLIB = ranlib
# all internal and external libraries - do not modify
FLIB_TARGETS = all
LIBOBJS = $(TOPDIR)/clib/clib.a $(TOPDIR)/iotk/src/libiotk.a
LIBXC_LIBS =
QELIBS = $(CUDA_LIBS) $(SCALAPACK_LIBS) $(LAPACK_LIBS) $(FOX_LIB) $(FFT_LIBS) $(BLAS_LIBS) $(MPI_LIBS) $(MASS_LIBS) $(HDF5_LIB) $(LIBXC_LIBS) $(LD_LIBS)
# wget or curl - useful to download from network
WGET = wget -O
# Install directory - "make install" copies *.x executables there
PREFIX = /usr/local
-------------- next part --------------
# make.inc. Generated from make.inc.in by configure.
# compilation rules
.SUFFIXES :
.SUFFIXES : .o .c .f .f90
# most fortran compilers can directly preprocess c-like directives: use
# $(MPIF90) $(F90FLAGS) -c $<
# if explicit preprocessing by the C preprocessor is needed, use:
# $(CPP) $(CPPFLAGS) $< -o $*.F90
# $(MPIF90) $(F90FLAGS) -c $*.F90 -o $*.o
# remember the tabulator in the first column !!!
.f90.o:
$(MPIF90) $(F90FLAGS) -c $<
# .f.o and .c.o: do not modify
.f.o:
$(F77) $(FFLAGS) -c $<
.c.o:
$(CC) $(CFLAGS) -c $<
# Top QE directory, useful for locating libraries, linking QE with plugins
# The following syntax should always point to TOPDIR:
TOPDIR = $(dir $(abspath $(filter %make.inc,$(MAKEFILE_LIST))))
# if it doesn't work, uncomment the following line (edit if needed):
# TOPDIR = /home/ehu50/ehu50977/q-e-develop
# DFLAGS = precompilation options (possible arguments to -D and -U)
# used by the C compiler and preprocessor
# To use libxc (v>=3.0.1), add -D__LIBXC to DFLAGS
# See include/defs.h.README for a list of options and their meaning
# With the exception of IBM xlf, FDFLAGS = $(DFLAGS)
# For IBM xlf, FDFLAGS is the same as DFLAGS with separating commas
# MANUAL_DFLAGS = additional precompilation option(s), if desired
# BEWARE: it does not work for IBM xlf! Manually edit FDFLAGS
MANUAL_DFLAGS =
DFLAGS = -D__DFTI -D__MPI
FDFLAGS = $(DFLAGS) $(MANUAL_DFLAGS)
# IFLAGS = how to locate directories with *.h or *.f90 file to be included
# typically -I$(TOPDIR)/include -I/some/other/directory/
# the latter contains .e.g. files needed by FFT libraries
# for libxc add -I/path/to/libxc/include/
IFLAGS = -I$(TOPDIR)/include -I$(TOPDIR)/FoX/finclude -I$(TOPDIR)/S3DE/iotk/include/ -I/apps/INTEL/2017.4/mkl/include
# MOD_FLAG = flag used by f90 compiler to locate modules
MOD_FLAG = -I
# BASEMOD_FLAGS points to directories containing basic modules,
# while BASEMODS points to the corresponding module libraries
# Each Makefile can add directories to MODFLAGS and libraries to QEMODS
BASEMOD_FLAGS= $(MOD_FLAG)$(TOPDIR)/iotk/src \
$(MOD_FLAG)$(TOPDIR)/Modules \
$(MOD_FLAG)$(TOPDIR)/FFTXlib \
$(MOD_FLAG)$(TOPDIR)/LAXlib \
$(MOD_FLAG)$(TOPDIR)/UtilXlib \
$(MOD_FLAG)$(TOPDIR)/FoX/finclude
# Compilers: fortran-90, fortran-77, C
# If a parallel compilation is desired, MPIF90 should be a fortran-90
# compiler that produces executables for parallel execution using MPI
# (such as for instance mpif90, mpf90, mpxlf90,...);
# otherwise, an ordinary fortran-90 compiler (f90, g95, xlf90, ifort,...)
# If you have a parallel machine but no suitable candidate for MPIF90,
# try to specify the directory containing "mpif.h" in IFLAGS
# and to specify the location of MPI libraries in MPI_LIBS
MPIF90 = mpif90
F90 = ifort
CC = icc
F77 = ifort
# GPU architecture (Kepler: 35, Pascal: 60, Volta: 70 )
GPU_ARCH=
# CUDA runtime (Pascal: 8.0, Volta: 9.0)
CUDA_RUNTIME=
# CUDA F90 Flags
CUDA_F90FLAGS=
# C preprocessor and preprocessing flags - for explicit preprocessing,
# if needed (see the compilation rules above)
# preprocessing flags must include DFLAGS and IFLAGS
CPP = cpp
CPPFLAGS = -P -traditional $(DFLAGS) $(IFLAGS)
# compiler flags: C, F90, F77
# C flags must include DFLAGS and IFLAGS
# F90 flags must include MODFLAGS, IFLAGS, and FDFLAGS with appropriate syntax
CFLAGS = -O3 $(DFLAGS) $(IFLAGS)
F90FLAGS = $(FFLAGS) -nomodule -fpp $(FDFLAGS) $(CUDA_F90FLAGS) $(IFLAGS) $(MODFLAGS)
FFLAGS = -O2 -assume byterecl -g -traceback
# compiler flags without optimization for fortran-77
# the latter is NEEDED to properly compile dlamch.f, used by lapack
FFLAGS_NOOPT = -O0 -assume byterecl -g -traceback
# compiler flag needed by some compilers when the main program is not fortran
# Currently used for Yambo
FFLAGS_NOMAIN = -nofor_main
# Linker, linker-specific flags (if any)
# Typically LD coincides with F90 or MPIF90, LD_LIBS is empty
# for libxc, set LD_LIBS=-L/path/to/libxc/lib/ -lxcf90 -lxc
LD = mpif90
LDFLAGS =
LD_LIBS =
# External Libraries (if any) : blas, lapack, fft, MPI
# If you have nothing better, use the local copy via "--with-netlib" :
# BLAS_LIBS = /your/path/to/espresso/LAPACK/blas.a
# BLAS_LIBS_SWITCH = internal
BLAS_LIBS = -L/apps/INTEL/2017.4/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core
BLAS_LIBS_SWITCH = external
# If you have nothing better, use the local copy via "--with-netlib" :
# LAPACK_LIBS = /your/path/to/espresso/LAPACK/lapack.a
# LAPACK_LIBS_SWITCH = internal
# For IBM machines with essl (-D__ESSL): load essl BEFORE lapack !
# remember that LAPACK_LIBS precedes BLAS_LIBS in loading order
LAPACK_LIBS =
LAPACK_LIBS_SWITCH = external
SCALAPACK_LIBS =
# nothing needed here if the the internal copy of FFTW is compiled
# (needs -D__FFTW in DFLAGS)
FFT_LIBS =
# HDF5
HDF5_LIB =
FOX_LIB = -L$(TOPDIR)/FoX/lib -lFoX_dom -lFoX_sax -lFoX_wxml -lFoX_common\
-lFoX_utils -lFoX_fsys
FOX_FLAGS =
# For parallel execution, the correct path to MPI libraries must
# be specified in MPI_LIBS (except for IBM if you use mpxlf)
MPI_LIBS =
# IBM-specific: MASS libraries, if available and if -D__MASS is defined in FDFLAGS
MASS_LIBS =
# CUDA libraries
CUDA_LIBS=
CUDA_EXTLIBS =
# ar command and flags - for most architectures: AR = ar, ARFLAGS = ruv
AR = ar
ARFLAGS = ruv
# ranlib command. If ranlib is not needed (it isn't in most cases) use
# RANLIB = echo
RANLIB = ranlib
# all internal and external libraries - do not modify
FLIB_TARGETS = all
LIBOBJS = $(TOPDIR)/clib/clib.a $(TOPDIR)/iotk/src/libiotk.a
LIBXC_LIBS =
QELIBS = $(CUDA_LIBS) $(SCALAPACK_LIBS) $(LAPACK_LIBS) $(FOX_LIB) $(FFT_LIBS) $(BLAS_LIBS) $(MPI_LIBS) $(MASS_LIBS) $(HDF5_LIB) $(LIBXC_LIBS) $(LD_LIBS)
# wget or curl - useful to download from network
WGET = wget -O
# Install directory - "make install" copies *.x executables there
PREFIX = /usr/local
More information about the developers
mailing list