<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p>Hi,</p>
<p><br>
</p>
<p>Thank you for replying.</p>
<p><br>
</p>
<p>More details:</p>
<p><br>
</p>
<p>1. input data:</p>
<p></p>
<div>&control</div>
<div> calculation='scf'</div>
<div> restart_mode='from_scratch',</div>
<div> pseudo_dir = '../pot/',</div>
<div> outdir='./out/'</div>
<div> prefix='BaTiO3'</div>
<div>/</div>
<div>&system</div>
<div> nbnd = 48</div>
<div> ibrav = 0, nat = 5, ntyp = 3</div>
<div> ecutwfc = 50</div>
<div> occupations='smearing', smearing='gaussian', degauss=0.02 </div>
<div>/</div>
<div>&electrons</div>
<div> conv_thr = 1.0e-8</div>
<div>/</div>
<div>ATOMIC_SPECIES</div>
<div> Ba 137.327 Ba.pbe-mt_fhi.UPF</div>
<div> Ti 204.380 Ti.pbe-mt_fhi.UPF</div>
<div> O 15.999 O.pbe-mt_fhi.UPF</div>
<div>ATOMIC_POSITIONS</div>
<div> Ba 0.0000000000000000 0.0000000000000000 0.0000000000000000</div>
<div> Ti 0.5000000000000000 0.5000000000000000 0.4819999933242795</div>
<div> O 0.5000000000000000 0.5000000000000000 0.0160000007599592</div>
<div> O 0.5000000000000000 -0.0000000000000000 0.5149999856948849</div>
<div> O 0.0000000000000000 0.5000000000000000 0.5149999856948849</div>
<div>K_POINTS (automatic)</div>
<div>11 11 11 0 0 0</div>
<div>CELL_PARAMETERS {angstrom}</div>
<div>3.999800000000001 0.000000000000000 0.000000000000000</div>
<div>0.000000000000000 3.999800000000001 0.000000000000000</div>
<div>0.000000000000000 0.000000000000000 4.018000000000000</div>
<div><br>
</div>
<p></p>
2. number of processors:</div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
I tested 24 cores and 8 cores, and both yield the same result.</div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<br>
</div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
3. <span style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px;">
type of parallelization:</span></div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
I don't know your meaning. I execute pw.x by:</div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<span>mpirun -np 24 pw.x < BTO.scf.in >> output</span></div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<span><br>
</span></div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<span>'which mpirun' output:</span></div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<span>
<div>/opt/intel/compilers_and_libraries_2016.3.210/linux/mpi/intel64/bin/mpirun</div>
</span></div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<br>
</div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<span></span>4. when the error occurs:</div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
in the middle of the run. The last a few lines of the output is</div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<div> total cpu time spent up to now is 32.9 secs</div>
<div><br>
</div>
<div> total energy = -105.97885119 Ry</div>
<div> Harris-Foulkes estimate = -105.99394457 Ry</div>
<div> estimated scf accuracy < 0.03479229 Ry</div>
<div><br>
</div>
<div> iteration # 7 ecut= 50.00 Ry beta=0.70</div>
<div> Davidson diagonalization with overlap</div>
<div> ethr = 1.45E-04, avg # of iterations = 2.7</div>
<div><br>
</div>
<div> total cpu time spent up to now is 37.3 secs</div>
<div><br>
</div>
<div> total energy = -105.99039982 Ry</div>
<div> Harris-Foulkes estimate = -105.99025175 Ry</div>
<div> estimated scf accuracy < 0.00927902 Ry</div>
<div><br>
</div>
<div> iteration # 8 ecut= 50.00 Ry beta=0.70</div>
<div> Davidson diagonalization with overlap</div>
<div><br>
</div>
</div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
5. Error message:</div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
Something like:</div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<div>Fatal error in PMPI_Cart_sub: Other MPI error, error stack:</div>
<div>PMPI_Cart_sub(242)...................: MPI_Cart_sub(comm=0xc400fcf3, remain_dims=0x7ffc03ae5f38, comm_new=0x7ffc03ae5e90) failed</div>
<div>PMPI_Cart_sub(178)...................: </div>
<div>MPIR_Comm_split_impl(270)............: </div>
<div>MPIR_Get_contextid_sparse_group(1330): Too many communicators (0/16384 free on this process; ignore_id=0)</div>
<div>Fatal error in PMPI_Cart_sub: Other MPI error, error stack:</div>
<div>PMPI_Cart_sub(242)...................: MPI_Cart_sub(comm=0xc400fcf3, remain_dims=0x7ffd10080408, comm_new=0x7ffd10080360) failed</div>
<div>PMPI_Cart_sub(178)...................: </div>
<div><br>
</div>
Cheers!</div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<br>
</div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
Chong</div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<div style="color: rgb(0, 0, 0);">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> pw_forum-bounces@pwscf.org <pw_forum-bounces@pwscf.org> on behalf of Paolo Giannozzi <p.giannozzi@gmail.com><br>
<b>Sent:</b> Sunday, May 15, 2016 3:43 PM<br>
<b>To:</b> PWSCF Forum<br>
<b>Subject:</b> Re: [Pw_forum] mpi error using pw.x</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>
<div>
<div>Please tell us what is wrong and we will fix it.<br>
<br>
</div>
Seriously: nobody can answer your question unless you specify, as a strict minimum, input data, number of processors and type of parallelization that trigger the error, and where the error occurs (at startup, later, in the middle of the run, ...).<br>
<br>
</div>
<div>Paolo<br>
</div>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Sun, May 15, 2016 at 7:50 AM, Chong Wang <span dir="ltr">
<<a href="mailto:ch-wang@outlook.com" target="_blank">ch-wang@outlook.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div style="font-size:12pt; color:#000000; background-color:#ffffff; font-family:Calibri,Arial,Helvetica,sans-serif">
<p></p>
<div>I compiled quantum espresso 5.4 with intel mpi and mkl 2016 update 3.</div>
<div><br>
</div>
<div>However, when I ran pw.x the following errors were reported:</div>
<div><br>
</div>
<div>...</div>
<div>MPIR_Get_contextid_sparse_group(1330): Too many communicators (0/16384 free on this process; ignore_id=0)</div>
<div>Fatal error in PMPI_Cart_sub: Other MPI error, error stack:</div>
<div>PMPI_Cart_sub(242)...................: MPI_Cart_sub(comm=0xc400fcf3, remain_dims=0x7ffde1391dd8, comm_new=0x7ffde1391d30) failed</div>
<div>PMPI_Cart_sub(178)...................: </div>
<div>MPIR_Comm_split_impl(270)............: </div>
<div>MPIR_Get_contextid_sparse_group(1330): Too many communicators (0/16384 free on this process; ignore_id=0)</div>
<div>Fatal error in PMPI_Cart_sub: Other MPI error, error stack:</div>
<div>PMPI_Cart_sub(242)...................: MPI_Cart_sub(comm=0xc400fcf3, remain_dims=0x7ffc02ad7eb8, comm_new=0x7ffc02ad7e10) failed</div>
<div>PMPI_Cart_sub(178)...................: </div>
<div>MPIR_Comm_split_impl(270)............: </div>
<div>MPIR_Get_contextid_sparse_group(1330): Too many communicators (0/16384 free on this process; ignore_id=0)</div>
<div>Fatal error in PMPI_Cart_sub: Other MPI error, error stack:</div>
<div>PMPI_Cart_sub(242)...................: MPI_Cart_sub(comm=0xc400fcf3, remain_dims=0x7fffb24e60f8, comm_new=0x7fffb24e6050) failed</div>
<div>PMPI_Cart_sub(178)...................: </div>
<div>MPIR_Comm_split_impl(270)............: </div>
<div>MPIR_Get_contextid_sparse_group(1330): Too many communicators (0/16384 free on this process; ignore_id=0)</div>
<br>
<p></p>
<p>I googled and found out this might be caused by hitting os limits of number of opened files. However, After I increased number of opened files per process from 1024 to 40960, the error persists.</p>
<p><br>
</p>
<p>What's wrong here?</p>
<p><br>
</p>
<p>Chong Wang</p>
<p>Ph. D. candidate</p>
<p>Institute for Advanced Study, Tsinghua University, Beijing, 100084</p>
</div>
</div>
<br>
_______________________________________________<br>
Pw_forum mailing list<br>
<a href="mailto:Pw_forum@pwscf.org">Pw_forum@pwscf.org</a><br>
<a href="http://pwscf.org/mailman/listinfo/pw_forum" rel="noreferrer" target="_blank" title="http://pwscf.org/mailman/listinfo/pw_forum
Ctrl+Click or tap to follow the link">http://pwscf.org/mailman/listinfo/pw_forum</a><br>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div class="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,<br>
Univ. Udine, via delle Scienze 208, 33100 Udine, Italy<br>
Phone +39-0432-558216, fax +39-0432-558222<br>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>