<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body>
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p>Hi,</p>
<p><br>
</p>
<p>I have done more test:</p>
<p>1. intel mpi 2015 yields segment fault</p>
<p>2. intel mpi 2013 yields the same error here</p>
<p>Did I do something wrong with compiling? Here's my make.sys:</p>
<p><br>
</p>
<p></p>
<p class="p1"><span class="s1"># make.sys. Generated from make.sys.in by configure.</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># compilation rules</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">.SUFFIXES :</span></p>
<p class="p1"><span class="s1">.SUFFIXES : .o .c .f .f90</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># most fortran compilers can directly preprocess c-like directives: use</span></p>
<p class="p1"><span class="s1"># <span class="Apple-tab-span"></span>$(MPIF90) $(F90FLAGS) -c $<</span></p>
<p class="p1"><span class="s1"># if explicit preprocessing by the C preprocessor is needed, use:</span></p>
<p class="p1"><span class="s1"># <span class="Apple-tab-span"></span>$(CPP) $(CPPFLAGS) $< -o $*.F90</span></p>
<p class="p1"><span class="s1">#<span class="Apple-tab-span"> </span>$(MPIF90) $(F90FLAGS) -c $*.F90 -o $*.o</span></p>
<p class="p1"><span class="s1"># remember the tabulator in the first column !!!</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">.f90.o:</span></p>
<p class="p1"><span class="s1"><span class="Apple-tab-span"></span>$(MPIF90) $(F90FLAGS) -c $<</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># .f.o and .c.o: do not modify</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">.f.o:</span></p>
<p class="p1"><span class="s1"><span class="Apple-tab-span"></span>$(F77) $(FFLAGS) -c $<</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">.c.o:</span></p>
<p class="p1"><span class="s1"><span class="Apple-tab-span"></span>$(CC) $(CFLAGS) -c $<</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># Top QE directory, not used in QE but useful for linking QE libs with plugins</span></p>
<p class="p1"><span class="s1"># The following syntax should always point to TOPDIR:</span></p>
<p class="p1"><span class="s1"># $(dir $(abspath $(filter %make.sys,$(MAKEFILE_LIST))))</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">TOPDIR = /home/wangc/temp/espresso-5.4.0</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># DFLAGS = precompilation options (possible arguments to -D and -U)</span></p>
<p class="p1"><span class="s1"># used by the C compiler and preprocessor</span></p>
<p class="p1"><span class="s1"># FDFLAGS = as DFLAGS, for the f90 compiler</span></p>
<p class="p1"><span class="s1"># See include/defs.h.README for a list of options and their meaning</span></p>
<p class="p1"><span class="s1"># With the exception of IBM xlf, FDFLAGS = $(DFLAGS)</span></p>
<p class="p1"><span class="s1"># For IBM xlf, FDFLAGS is the same as DFLAGS with separating commas</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># MANUAL_DFLAGS = additional precompilation option(s), if desired</span></p>
<p class="p1"><span class="s1"># BEWARE: it does not work for IBM xlf! Manually edit FDFLAGS</span></p>
<p class="p1"><span class="s1">MANUAL_DFLAGS =</span></p>
<p class="p1"><span class="s1">DFLAGS = -D__GFORTRAN -D__STD_F95 -D__DFTI -D__MPI -D__PARA -D__SCALAPACK</span></p>
<p class="p1"><span class="s1">FDFLAGS = $(DFLAGS) $(MANUAL_DFLAGS)</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># IFLAGS = how to locate directories with *.h or *.f90 file to be included</span></p>
<p class="p1"><span class="s1"># typically -I../include -I/some/other/directory/</span></p>
<p class="p1"><span class="s1"># the latter contains .e.g. files needed by FFT libraries</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">IFLAGS = -I../include -I/opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/include</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># MOD_FLAGS = flag used by f90 compiler to locate modules</span></p>
<p class="p1"><span class="s1"># Each Makefile defines the list of needed modules in MODFLAGS</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">MOD_FLAG = -I</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># Compilers: fortran-90, fortran-77, C</span></p>
<p class="p1"><span class="s1"># If a parallel compilation is desired, MPIF90 should be a fortran-90</span></p>
<p class="p1"><span class="s1"># compiler that produces executables for parallel execution using MPI</span></p>
<p class="p1"><span class="s1"># (such as for instance mpif90, mpf90, mpxlf90,...);</span></p>
<p class="p1"><span class="s1"># otherwise, an ordinary fortran-90 compiler (f90, g95, xlf90, ifort,...)</span></p>
<p class="p1"><span class="s1"># If you have a parallel machine but no suitable candidate for MPIF90,</span></p>
<p class="p1"><span class="s1"># try to specify the directory containing "mpif.h" in IFLAGS</span></p>
<p class="p1"><span class="s1"># and to specify the location of MPI libraries in MPI_LIBS</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">MPIF90 = mpif90</span></p>
<p class="p1"><span class="s1">#F90 = gfortran</span></p>
<p class="p1"><span class="s1">CC = cc</span></p>
<p class="p1"><span class="s1">F77 = gfortran</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># C preprocessor and preprocessing flags - for explicit preprocessing,</span></p>
<p class="p1"><span class="s1"># if needed (see the compilation rules above)</span></p>
<p class="p1"><span class="s1"># preprocessing flags must include DFLAGS and IFLAGS</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">CPP = cpp</span></p>
<p class="p1"><span class="s1">CPPFLAGS = -P -C -traditional $(DFLAGS) $(IFLAGS)</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># compiler flags: C, F90, F77</span></p>
<p class="p1"><span class="s1"># C flags must include DFLAGS and IFLAGS</span></p>
<p class="p1"><span class="s1"># F90 flags must include MODFLAGS, IFLAGS, and FDFLAGS with appropriate syntax</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">CFLAGS = -O3 $(DFLAGS) $(IFLAGS)</span></p>
<p class="p1"><span class="s1">F90FLAGS = $(FFLAGS) -x f95-cpp-input $(FDFLAGS) $(IFLAGS) $(MODFLAGS)</span></p>
<p class="p1"><span class="s1">FFLAGS = -O3 -g</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># compiler flags without optimization for fortran-77</span></p>
<p class="p1"><span class="s1"># the latter is NEEDED to properly compile dlamch.f, used by lapack</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">FFLAGS_NOOPT = -O0 -g</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># compiler flag needed by some compilers when the main program is not fortran</span></p>
<p class="p1"><span class="s1"># Currently used for Yambo</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">FFLAGS_NOMAIN = </span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># Linker, linker-specific flags (if any)</span></p>
<p class="p1"><span class="s1"># Typically LD coincides with F90 or MPIF90, LD_LIBS is empty</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">LD = mpif90</span></p>
<p class="p1"><span class="s1">LDFLAGS = -g -pthread</span></p>
<p class="p1"><span class="s1">LD_LIBS = </span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># External Libraries (if any) : blas, lapack, fft, MPI</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># If you have nothing better, use the local copy :</span></p>
<p class="p1"><span class="s1"># BLAS_LIBS = /your/path/to/espresso/BLAS/blas.a</span></p>
<p class="p1"><span class="s1"># BLAS_LIBS_SWITCH = internal</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">BLAS_LIBS = -lmkl_gf_lp64 -lmkl_sequential -lmkl_core</span></p>
<p class="p1"><span class="s1">BLAS_LIBS_SWITCH = external</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># If you have nothing better, use the local copy :</span></p>
<p class="p1"><span class="s1"># LAPACK_LIBS = /your/path/to/espresso/lapack-3.2/lapack.a</span></p>
<p class="p1"><span class="s1"># LAPACK_LIBS_SWITCH = internal</span></p>
<p class="p1"><span class="s1"># For IBM machines with essl (-D__ESSL): load essl BEFORE lapack !</span></p>
<p class="p1"><span class="s1"># remember that LAPACK_LIBS precedes BLAS_LIBS in loading order</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">LAPACK_LIBS = </span></p>
<p class="p1"><span class="s1">LAPACK_LIBS_SWITCH = external</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">ELPA_LIBS_SWITCH = disabled</span></p>
<p class="p1"><span class="s1">SCALAPACK_LIBS = -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># nothing needed here if the the internal copy of FFTW is compiled</span></p>
<p class="p1"><span class="s1"># (needs -D__FFTW in DFLAGS)</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">FFT_LIBS = </span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># For parallel execution, the correct path to MPI libraries must</span></p>
<p class="p1"><span class="s1"># be specified in MPI_LIBS (except for IBM if you use mpxlf)</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">MPI_LIBS = </span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># IBM-specific: MASS libraries, if available and if -D__MASS is defined in FDFLAGS</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">MASS_LIBS = </span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># ar command and flags - for most architectures: AR = ar, ARFLAGS = ruv</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">AR = ar</span></p>
<p class="p1"><span class="s1">ARFLAGS = ruv</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># ranlib command. If ranlib is not needed (it isn't in most cases) use</span></p>
<p class="p1"><span class="s1"># RANLIB = echo</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">RANLIB = ranlib</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># all internal and external libraries - do not modify</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">FLIB_TARGETS = all</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1">LIBOBJS = ../clib/clib.a ../iotk/src/libiotk.a</span></p>
<p class="p1"><span class="s1">LIBS = $(SCALAPACK_LIBS) $(LAPACK_LIBS) $(FFT_LIBS) $(BLAS_LIBS) $(MPI_LIBS) $(MASS_LIBS) $(LD_LIBS)</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># wget or curl - useful to download from network</span></p>
<p class="p1"><span class="s1">WGET = wget -O</span></p>
<p class="p2"><span class="s1"></span><br>
</p>
<p class="p1"><span class="s1"># Install directory - not currently used</span></p>
<p class="p1"><span class="s1">PREFIX = /usr/local</span></p>
<br>
<p></p>
<p>Cheers!</p>
<p><br>
</p>
<p>Chong Wang</p>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> pw_forum-bounces@pwscf.org <pw_forum-bounces@pwscf.org> on behalf of Paolo Giannozzi <p.giannozzi@gmail.com><br>
<b>Sent:</b> Sunday, May 15, 2016 8:28:26 PM<br>
<b>To:</b> PWSCF Forum<br>
<b>Subject:</b> Re: [Pw_forum] mpi error using pw.x</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>It looks like a compiler/mpi bug, since there is nothing special in your input and in your execution, unless you find evidence that the problem is reproducible on other compiler/mpi versions.<br>
<br>
</div>
Paolo<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Sun, May 15, 2016 at 10:11 AM, Chong Wang <span dir="ltr">
<<a href="mailto:ch-wang@outlook.com" target="_blank">ch-wang@outlook.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Hi,</p>
<p><br>
</p>
<p>Thank you for replying.</p>
<p><br>
</p>
<p>More details:</p>
<p><br>
</p>
<p>1. input data:</p>
<p></p>
<div>&control</div>
<div> calculation='scf'</div>
<div> restart_mode='from_scratch',</div>
<div> pseudo_dir = '../pot/',</div>
<div> outdir='./out/'</div>
<div> prefix='BaTiO3'</div>
<div>/</div>
<div>&system</div>
<div> nbnd = 48</div>
<div> ibrav = 0, nat = 5, ntyp = 3</div>
<div> ecutwfc = 50</div>
<div> occupations='smearing', smearing='gaussian', degauss=0.02 </div>
<div>/</div>
<div>&electrons</div>
<div> conv_thr = 1.0e-8</div>
<div>/</div>
<div>ATOMIC_SPECIES</div>
<div> Ba 137.327 Ba.pbe-mt_fhi.UPF</div>
<div> Ti 204.380 Ti.pbe-mt_fhi.UPF</div>
<div> O 15.999 O.pbe-mt_fhi.UPF</div>
<div>ATOMIC_POSITIONS</div>
<div> Ba 0.0000000000000000 0.0000000000000000 0.0000000000000000</div>
<div> Ti 0.5000000000000000 0.5000000000000000 0.4819999933242795</div>
<div> O 0.5000000000000000 0.5000000000000000 0.0160000007599592</div>
<div> O 0.5000000000000000 -0.0000000000000000 0.5149999856948849</div>
<div> O 0.0000000000000000 0.5000000000000000 0.5149999856948849</div>
<div>K_POINTS (automatic)</div>
<div>11 11 11 0 0 0</div>
<div>CELL_PARAMETERS {angstrom}</div>
<div>3.999800000000001 0.000000000000000 0.000000000000000</div>
<div>0.000000000000000 3.999800000000001 0.000000000000000</div>
<div>0.000000000000000 0.000000000000000 4.018000000000000</div>
<div><br>
</div>
<p></p>
2. number of processors:</div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
I tested 24 cores and 8 cores, and both yield the same result.</div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<br>
</div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
3. <span style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px">type of parallelization:</span></div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
I don't know your meaning. I execute pw.x by:</div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<span>mpirun -np 24 pw.x < <a href="http://BTO.scf.in" target="_blank">BTO.scf.in</a> >> output</span></div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<span><br>
</span></div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<span>'which mpirun' output:</span></div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<span>
<div>/opt/intel/compilers_and_libraries_2016.3.210/linux/mpi/intel64/bin/mpirun</div>
</span></div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<br>
</div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<span></span>4. when the error occurs:</div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
in the middle of the run. The last a few lines of the output is</div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<div> total cpu time spent up to now is 32.9 secs</div>
<div><br>
</div>
<div> total energy = -105.97885119 Ry</div>
<div> Harris-Foulkes estimate = -105.99394457 Ry</div>
<div> estimated scf accuracy < 0.03479229 Ry</div>
<div><br>
</div>
<div> iteration # 7 ecut= 50.00 Ry beta=0.70</div>
<div> Davidson diagonalization with overlap</div>
<div> ethr = 1.45E-04, avg # of iterations = 2.7</div>
<div><br>
</div>
<div> total cpu time spent up to now is 37.3 secs</div>
<div><br>
</div>
<div> total energy = -105.99039982 Ry</div>
<div> Harris-Foulkes estimate = -105.99025175 Ry</div>
<div> estimated scf accuracy < 0.00927902 Ry</div>
<div><br>
</div>
<div> iteration # 8 ecut= 50.00 Ry beta=0.70</div>
<div> Davidson diagonalization with overlap</div>
<div><br>
</div>
</div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
5. Error message:</div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
Something like:</div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<div>Fatal error in PMPI_Cart_sub: Other MPI error, error stack:</div>
<div>PMPI_Cart_sub(242)...................: MPI_Cart_sub(comm=0xc400fcf3, remain_dims=0x7ffc03ae5f38, comm_new=0x7ffc03ae5e90) failed</div>
<div>PMPI_Cart_sub(178)...................: </div>
<div>MPIR_Comm_split_impl(270)............: </div>
<div>MPIR_Get_contextid_sparse_group(1330): Too many communicators (0/16384 free on this process; ignore_id=0)</div>
<div>Fatal error in PMPI_Cart_sub: Other MPI error, error stack:</div>
<div>PMPI_Cart_sub(242)...................: MPI_Cart_sub(comm=0xc400fcf3, remain_dims=0x7ffd10080408, comm_new=0x7ffd10080360) failed</div>
<div>PMPI_Cart_sub(178)...................: </div>
<div><br>
</div>
Cheers!</div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<br>
</div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
Chong</div>
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<div style="color:rgb(0,0,0)">
<hr style="display:inline-block;width:98%">
<div dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b>
<a href="mailto:pw_forum-bounces@pwscf.org" target="_blank">pw_forum-bounces@pwscf.org</a> <<a href="mailto:pw_forum-bounces@pwscf.org" target="_blank">pw_forum-bounces@pwscf.org</a>> on behalf of Paolo Giannozzi <<a href="mailto:p.giannozzi@gmail.com" target="_blank">p.giannozzi@gmail.com</a>><br>
<b>Sent:</b> Sunday, May 15, 2016 3:43 PM<br>
<b>To:</b> PWSCF Forum<br>
<b>Subject:</b> Re: [Pw_forum] mpi error using pw.x</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>
<div>
<div>Please tell us what is wrong and we will fix it.<br>
<br>
</div>
Seriously: nobody can answer your question unless you specify, as a strict minimum, input data, number of processors and type of parallelization that trigger the error, and where the error occurs (at startup, later, in the middle of the run, ...).<br>
<br>
</div>
<div>Paolo<br>
</div>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Sun, May 15, 2016 at 7:50 AM, Chong Wang <span dir="ltr">
<<a href="mailto:ch-wang@outlook.com" target="_blank">ch-wang@outlook.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<p></p>
<div>I compiled quantum espresso 5.4 with intel mpi and mkl 2016 update 3.</div>
<div><br>
</div>
<div>However, when I ran pw.x the following errors were reported:</div>
<div><br>
</div>
<div>...</div>
<div>MPIR_Get_contextid_sparse_group(1330): Too many communicators (0/16384 free on this process; ignore_id=0)</div>
<div>Fatal error in PMPI_Cart_sub: Other MPI error, error stack:</div>
<div>PMPI_Cart_sub(242)...................: MPI_Cart_sub(comm=0xc400fcf3, remain_dims=0x7ffde1391dd8, comm_new=0x7ffde1391d30) failed</div>
<div>PMPI_Cart_sub(178)...................: </div>
<div>MPIR_Comm_split_impl(270)............: </div>
<div>MPIR_Get_contextid_sparse_group(1330): Too many communicators (0/16384 free on this process; ignore_id=0)</div>
<div>Fatal error in PMPI_Cart_sub: Other MPI error, error stack:</div>
<div>PMPI_Cart_sub(242)...................: MPI_Cart_sub(comm=0xc400fcf3, remain_dims=0x7ffc02ad7eb8, comm_new=0x7ffc02ad7e10) failed</div>
<div>PMPI_Cart_sub(178)...................: </div>
<div>MPIR_Comm_split_impl(270)............: </div>
<div>MPIR_Get_contextid_sparse_group(1330): Too many communicators (0/16384 free on this process; ignore_id=0)</div>
<div>Fatal error in PMPI_Cart_sub: Other MPI error, error stack:</div>
<div>PMPI_Cart_sub(242)...................: MPI_Cart_sub(comm=0xc400fcf3, remain_dims=0x7fffb24e60f8, comm_new=0x7fffb24e6050) failed</div>
<div>PMPI_Cart_sub(178)...................: </div>
<div>MPIR_Comm_split_impl(270)............: </div>
<div>MPIR_Get_contextid_sparse_group(1330): Too many communicators (0/16384 free on this process; ignore_id=0)</div>
<br>
<p></p>
<p>I googled and found out this might be caused by hitting os limits of number of opened files. However, After I increased number of opened files per process from 1024 to 40960, the error persists.</p>
<p><br>
</p>
<p>What's wrong here?</p>
<p><br>
</p>
<p>Chong Wang</p>
<p>Ph. D. candidate</p>
<p>Institute for Advanced Study, Tsinghua University, Beijing, 100084</p>
</div>
</div>
<br>
_______________________________________________<br>
Pw_forum mailing list<br>
<a href="mailto:Pw_forum@pwscf.org" target="_blank">Pw_forum@pwscf.org</a><br>
<a href="http://pwscf.org/mailman/listinfo/pw_forum" rel="noreferrer" title="http://pwscf.org/mailman/listinfo/pw_forum
Ctrl+Click or tap to follow the link" target="_blank">http://pwscf.org/mailman/listinfo/pw_forum</a><span class="HOEnZb"><font color="#888888"><br>
</font></span></blockquote>
<span class="HOEnZb"><font color="#888888"></font></span></div>
<span class="HOEnZb"><font color="#888888"><br>
<br clear="all">
<br>
-- <br>
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,<br>
Univ. Udine, via delle Scienze 208, 33100 Udine, Italy<br>
Phone <a href="tel:%2B39-0432-558216" value="+390432558216" target="_blank">+39-0432-558216</a>, fax
<a href="tel:%2B39-0432-558222" value="+390432558222" target="_blank">+39-0432-558222</a><br>
<br>
</div>
</div>
</div>
</div>
</div>
</font></span></div>
</div>
</div>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Pw_forum mailing list<br>
<a href="mailto:Pw_forum@pwscf.org">Pw_forum@pwscf.org</a><br>
<a href="http://pwscf.org/mailman/listinfo/pw_forum" rel="noreferrer" target="_blank">http://pwscf.org/mailman/listinfo/pw_forum</a><br>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div class="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,<br>
Univ. Udine, via delle Scienze 208, 33100 Udine, Italy<br>
Phone +39-0432-558216, fax +39-0432-558222<br>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>