[Pw_forum] PW taskgroups and a large run on a BG/P

[David Farrell]

>> I found when I took the 432 atom system I sent you, and ran it on  
>> 128 cores in smp mode
>> (1 MPI process/node - 2 GB per process) it did work (-ntg 32 -ndiag  
>> 121
>
> 32 task groups? that's a lot

Yeah - I ran that by mistake. But I figured I'd mention it anyway


>
>
>> as well as -ntg 4 -ndiag 121)
>
> 4 looks more reasonable in my opinion
>

Right.


>> - the system didn't fit into memory  in vn mode (4 mpi processes/ 
>> node -
>> 512 MB per process)
>
> that job requires approx. 100Mb of dynamically allocated RAM per  
> process, plus
> a few tens of Mb of work space. Why it does not fit into 512Mb is a  
> mystery,
> unless each process comes with a copy of all libraries. If this is  
> the case, the
> maximum you can fit into 512Mb is a code printing "Hello world" in  
> parallel.
>

Yeah - I am very curious about this. I was actually *very* surprised  
when this happened.


> By the way: the default number of bands in metallic calculations can  
> be trimmed
> by a significant amount (e.g. 500 instead of 576)
>
>> I then tried the system in dual mode (2 mpi processes/node - 1 GB  
>> per process)
>> using -ntg 4 and -ndiag 121. In this case, the cholesky error came  
>> up:
>
>
> the code performs exactly the same operations, independently on how  
> the MPI
> processes are distributed. It looks like yet another BlueGene  
> weirdness, like this:
>  http://www.democritos.it:8888/O-sesame/chngview?cn=5777
>  http://www.democritos.it:8888/O-sesame/chngview?cn=5932
> that however affected only the internal parallel diagonalization,  
> not the new
> scalapack algorithm. I do not see any evidence that there is  
> anything wrong
> with the code itself.
>

OK - so perhaps I should try this same run with scalapack enabled  
instead just to be sure of what happens and that I haven't miscompiled  
anything. That is only available in the latest CVS versions right?

Dave


David E. Farrell
Post-Doctoral Fellow
Department of Materials Science and Engineering
Northwestern University
email: d-farrell2 at northwestern.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20090212/10e2af13/attachment.html>