<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Just to add to my last email, I tried to reproduce your results (and it looks like you emailed the list back before I had a chance to finish this)<div><br></div><div>I found when I took the 432 atom system I sent you, and ran it on 128 cores in smp mode (1 MPI process/node - 2 GB per process) it did work (-ntg 32 -ndiag 121 as well as -ntg 4 -ndiag 121) - the system didn't fit into memory in vn mode (4 mpi processes/node - 512 MB per process). </div><div><br></div><div>the error in the vn mode case was:</div><div>'"fft_parallel.f90", line 104: 1525-108 Error encountered while attempting to allocate a data object. The program will stop.'</div><div><br></div><div>I then tried the system in dual mode (2 mpi processes/node - 1 GB per process) using -ntg 4 and -ndiag 121. In this case, the cholesky error came up:</div><div><br></div><div><div> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%</div><div> task # 12</div><div> from pdpotf : error # 1</div><div> problems computing cholesky decomposition </div><div> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%</div><div><br></div></div><div>I then tried the dual mode case with -ntg 4 and -ndiag 100</div><div><br></div><div><div>%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%</div><div> task # 11</div><div> from pdpotf : error # 1</div><div> problems computing cholesky decomposition </div><div> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%</div><div><br></div></div><div><br></div><div>So now I am left wondering if I am not trying to spread the work out too much (sounds like this was the case) and that is what is leading to these cholesky errors. But the dual mode error seems to point to something else going on.</div><div><br></div><div><br></div><div>Dave</div><div><br><div><div>On Feb 12, 2009, at 6:53 AM, Paolo Giannozzi wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>On Feb 11, 2009, at 19:41 , David Farrell wrote:<br><br><blockquote type="cite">Let me know if the attachment doesn't make it through<br></blockquote><br>it didn't to the mailing list (max attachment size 40kB), but<br>I received it. Attached is what I got (for a scf calculation) on<br>a cray xt5. Apart from the bogus values of planes printed in<br>the output, everything else seems ok, including parallel<br>subspace diagonalization (scalapack) on 121 processors.<br><br>Paolo<br><br><span><test432.out.gz></span><br>---<br>Paolo Giannozzi, Democritos and University of Udine, Italy<br><br><br></div></blockquote></div><br><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; font-size: 12px; ">David E. Farrell</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size: 12px; ">Post-Doctoral Fellow</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size: 12px; ">Department of Materials Science and Engineering</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; font-size: 12px; ">Northwestern University</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; font-size: 12px; ">email: <a href="mailto:d-farrell2@northwestern.edu">d-farrell2@northwestern.edu</a></font></div></div></div></span> </div><br></div></body></html>