<div dir="ltr"><div><div><div><div><div>A hybrid calculation with k-points requires, in addition to all other terms for ordinary DFT, the computation of the exact-exchange term costing O(Nk*Nq*Nb*Nb*Nw) floating-point operations, plus the storage of Kohn-Sham states, filling Nq*Nb*Nr double precision complex numbers. Legend:<br>Nk = number of k-points in the Irreducible Brillouin Zone (301 in your case)<br></div>Nq = number of k+q-points (1728)<br></div>Nb = number of Kohn-Sham states (53)<br></div>Nw = number of plane waves (20005)<br></div>Nr = number of points in the FFT grid = nr1*nr2*nr3 = 72*72*72<br>Arrays and computations are distributed so divide Nw and Nr by the number of processors. If I made the math correctly, it's 2.65 Gb RAM per MPI process on 192 processors. I am not surprised it crashes. Even if it doesn't, it will take approximately forever. Are you sure you need that many k- and q-points?<br><br></div><div>Paolo<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 6, 2016 at 9:32 PM, Dumett Torres, Daniel <span dir="ltr"><<a href="mailto:dumettt2@illinois.edu" target="_blank">dumettt2@illinois.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px;background-color:white">I have been trying to run a hybrid PBE0 calculation in parallel
and have run into a recurring problem in which the calculation stops after scf convergence is first acheived - the calculation stops and doesn't go on to refine the calculation with exx. I have tried to run the calculation with between 24 and 192 processors
because we previously thought memory requirements were to blame for the error, but this didn't seem to help.</span></font><span style="font-size:13.3333px"></span>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><br>
</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><i>''</i><i>TACC: Setting up parallel environment for MVAPICH2+mpispawn.</i></span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><i>TACC: Starting parallel tasks...</i></span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><i><br>
</i></span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><i> Program PWSCF v.5.2.0 </i>'</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><br>
</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px">
<div> </div>
<div><i>'Parallel version (MPI), running on 192 processors</i></div>
<div><i> R & G space division: proc/nbgrp/npool/nimage = 192</i>'</div>
</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><br>
</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><br>
</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><i>'highest occupied, lowest unoccupied level (ev): 6.3409 6.0501</i><i><br>
</i></span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px">
<div><i><br>
</i></div>
<div><i>convergence has been achieved in 16 iterations</i><i><br>
</i></div>
<div><i> </i></div>
<div><i>TACC: MPI job exited with code: 1</i></div>
<div><i> </i></div>
<div><i>TACC: Shutdown complete. Exiting.'</i></div>
</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><i><br>
</i></span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><i><br>
</i></span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px">this error produces a CRASH file that reports an error reading an xml-data file and upon inspection,the .save
folder produced contains the charge-density.dat file but is missing the .UPF files, the data-file.xml file, and data-file.xml.eig file. Furthermore, while the .igk files have content, all the .wfc files produced by the job are empty (0 bytes). My impression
is that this missing content is what is keeping the calculation from being able to proceed but I don't know how it might be addressed. </span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><br>
</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px">attached is the input file for my calculation. Attached are the input and output files resulting from one such
run. I did notice an additional message in the output </span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><i><br>
</i></span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><i>' Message from routine sym_rho_init:</i></span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><i> likely internal error: no G-vectors found'</i></span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><br>
</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px">but the calculation seems to proceed despite it...</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><br>
</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px">I appreciate any insights you might have regarding these complications I am facing </span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px"><br>
</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px">Thank you for your help,</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px">Daniel Dumett Torres</span></font></div>
<div style="font-size:13.3333px;background-color:white"><font face="arial,sans-serif" size="1" color="#222222"><span style="font-size:12.79px">Graduate Student at the University of Illinois at Urbana-Champaign </span></font></div>
</div>
</div>
<br>_______________________________________________<br>
Pw_forum mailing list<br>
<a href="mailto:Pw_forum@pwscf.org">Pw_forum@pwscf.org</a><br>
<a href="http://pwscf.org/mailman/listinfo/pw_forum" rel="noreferrer" target="_blank">http://pwscf.org/mailman/listinfo/pw_forum</a><br></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature"><div dir="ltr"><div>Paolo Giannozzi, Dip. Matematica Informatica e Fisica,<br>Univ. Udine, via delle Scienze 208, 33100 Udine, Italy<br>Phone +39-0432-558216, fax +39-0432-558222<br><br></div></div></div>
</div>