<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I think this summary is about right. At least I can run something now and have an idea of what it is. I will probably also be transitioning to cp for my runs.<div><br></div><div>I have been working with the folks at the supercomputing center, and will continue in hopes of finding out what is causing the trouble.</div><div><br></div><div>Thanks,</div><div><br></div><div>Dave</div><div><br><div><div>On Feb 13, 2009, at 3:33 AM, Paolo Giannozzi wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>In conclusion, and if I understand correctly:<br><br> PWscf runs only with 1 MPI process per core, but<br> it runs 800+ atoms in as little as 32 processes<br> or in as much as 512, as I was expecting.<br><br>This confirms my opinion that the problem is on the<br>BG side and not on the PWscf side, since there is<br>NOTHING in the fortran code that depends upon where<br>the MPI processes are running. Of course one can<br>never rule out the possibility that some obscure<br>bug is triggered only in that special cases, but<br>it seems to me highly unlikely.<br><br>Implementation of mixed MPI-openMPI parallelization<br>is under development, but it wil take some time. In<br>the meantime, if you can link openMPI-aware<br>mathematical libraries, you might get some speedup.<br><br>If you do not need k-points, and if you know how to<br>deal with metallic systems, you might try CP instead<br>of PWscf - it is better tested for large systems - but<br>I don't expect a different behavior, since the routines<br>performing parallel subspace diagonalization are the<br>same that perform iterative orthonormalization, so the<br>trouble is likely to move from "cholesky" to "ortho".<br><br>You might try to find out what is wrong, since you have<br>two cases that should yield exactly the same results<br>but don't. It may take a lot of time and lead to no<br>result, though. You may also try to raise this issue<br>with the technical staff of the computing center.<br><br>Paolo<br>-- <br>Paolo Giannozzi, Democritos and University of Udine, Italy<br></div></blockquote></div><br><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; font-size: 12px; ">David E. Farrell</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size: 12px; ">Post-Doctoral Fellow</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size: 12px; ">Department of Materials Science and Engineering</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; font-size: 12px; ">Northwestern University</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; font-size: 12px; ">email: <a href="mailto:d-farrell2@northwestern.edu">d-farrell2@northwestern.edu</a></font></div></div></div></span> </div><br></div></body></html>