[Pw_forum] question on parallelism options and scaling factor

[Giovanni Pizzi]

Dear Pascal,
since you have more than one k-point, you could try to have each pool 
within one node, so that only inter-pool communication occurs over 
infiniband; for instance if you have 4 k-points you may try to use 4 
pools on 4 nodes (or possibly 2 pools on 2 nodes, or 4 pools on 2 nodes, 
etc).
This kind of parallelization should scale pretty well, if your system 
allows it (i.e. you have  enough kpoints and your system fits in RAM). 
Then, you can try to optimize the parallelization using the other 
parallelization options.

If you manage to do some scaling tests using the pools, could you please 
report your results on this mailing list?

Thanks, and best regards,

Giovanni Pizzi


On 02/05/2013 10:12 PM, pascal boulet wrote:
> Dear all,
>
> I have a basic question about parallelism and scaling factor.
>
> First, I am running calculations on a cubic system with 58 atoms
> (alat=19.5652  a.u.), 540 electrons (324 KS states) and few k-points
> (4x4x4 grid=4 k-points), on 32 cores (4 nodes) but I can submit on many
> more.
>
> I guess the best thing to do is to parallelize the calculation on the
> bands but maybe also on the FFTs. We have an infiniband interconnection
> network between the nodes.
>
> What would you suggest as values for image/pools/ntg/bands ?
>
> I have made a SCF test calculation on 16 and 32 cores. For the SCF cycle
> (13 steps) I get the following timing:
> For 16 cores: total cpu time spent up to now is    22362.4 secs
> For 32 cores: total cpu time spent up to now is    17932.6 secs
>
> The speedup is "only" 25%. I would have expected a better speedup for
> such a small number of cores. Am i wrong? What is your experience?
>
> (For additional information, if helpful: QE 5.0.1 has been compiled with
> openMPI, intel 12.1 and FFTW 3.2.2.)
>
> thank you for your answers.
>
> Regards
> Pascal
>
>
>
> _______________________________________________
> Pw_forum mailing list
> Pw_forum at pwscf.org
> http://pwscf.org/mailman/listinfo/pw_forum


-- 
Giovanni Pizzi
Post-doctoral Research Scientist
EPFL STI IMX THEOS
MXC 340 (Bâtiment MXC)
Station 12
CH-1015 Lausanne (Switzerland)
Phone: +41 21 69 31124

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20130206/65b493f8/attachment.html>