<HTML>
<HEAD>
<TITLE>Re: [Pw_forum] Insufficient Virtual Memory</TITLE>
</HEAD>
<BODY>
<FONT SIZE="4"><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>Dear Axel,<BR>
<BR>
Thanks for clarification.<BR>
<BR>
About the benchmarks, I just simply to see how well is the performance of the cluster we bought in term of scaling with QE. I sent some plots to you, but the email did not go thru because of the restriction of the size (larger than 40K).<BR>
<BR>
Currently, I am not happy at the fact that the difference in CPU time and wall time is too large. </SPAN></FONT><FONT FACE="Consolas, Courier New, Courier"><SPAN STYLE='font-size:10pt'>When I run a longer job, which took ~2h CPU time long, the wall time was ~7h when I run from the head node, and ~4h when I run from another node, which is not the head node. The reason why I avoided running from the head node was that other users might use the head node to do their work, causing the long wall time. Even when I did the test runs (short runs), when no one used the cluster, the difference was still up to 25%, depending on how many nodes I used. I do not know what I should do to improve this difference. According to you, what should I look at to fix the problem. (I also need to read the forum discussion as Lorenzo said in his email). <BR>
<BR>
Thanks,<BR>
<BR>
Trinh<BR>
</SPAN></FONT><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><BR>
On 2/26/09 4:34 PM, "Axel Kohlmeyer" <akohlmey@cmm.chem.upenn.edu> wrote:<BR>
<BR>
</SPAN></FONT></FONT><BLOCKQUOTE><FONT SIZE="4"><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>On Thu, 26 Feb 2009, Vo, Trinh wrote:<BR>
<BR>
TV> Dear Axel,<BR>
TV><BR>
<BR>
TV> I am sorry. I may be out of date. Is the cp2k code included in QE<BR>
TV> package, or in some other codes? I saw the word "cp2k" in the title<BR>
<BR>
perhaps i was not clear enough. cp2k is a completely different project<BR>
and has not much else to do with Q-E outside of the fact that both<BR>
have a DFT module and that i and a few others are using both codes.<BR>
<BR>
TV> of the plot, but I thought that it might be the same cp code of QE. <BR>
<BR>
no. i already wrote that. in fact, "there is no CP in CP2k".<BR>
i suggested this phase as a subtitle to the project, but people<BR>
found it to negative. ;)<BR>
<BR>
TV> Actually, the benchmark plot that I obtained does not look as good<BR>
TV> as the posted plot. I will send it to you in the next email. I need<BR>
TV> to make a better plot before I send it.<BR>
<BR>
benchmark in terms of scaling or in terms of problem set size?<BR>
<BR>
there should be a post in the cp2k google group referring to the<BR>
benchmark graph and describing the exact system setup.<BR>
<BR>
cheers,<BR>
axel.<BR>
<BR>
TV><BR>
TV> Thanks,<BR>
TV><BR>
TV> Trinh<BR>
TV><BR>
TV> <BR>
TV> TV> tried to repeat all the calculations to compare, but I could not run<BR>
TV> TV> that case.<BR>
TV><BR>
TV> this graph was done with a _very_ different code, cp2k, that<BR>
TV> uses a method (quickstep) different from the CP in cp.x and has<BR>
TV> therefore different memory requirements and system size scaling.<BR>
TV> the README of example21 explicitly warns you about running out<BR>
TV> of memory. give the current difference in speed between hard<BR>
TV> drives and main memory, every calculation where you run into<BR>
TV> using swap is not worth the effort (unlike 10 years ago).<BR>
TV><BR>
TV> just for reference: because of the difference in methods cp.x<BR>
TV> should scale better with the number of processors, whereas<BR>
TV> cp2k should scale better with the problem set size than cp.x<BR>
TV> with the same number of processors.<BR>
TV><BR>
TV> that being said, cp2k demonstrates the problem of overloading<BR>
TV> memory and communication channels very nicely and thus the<BR>
TV> graphs are a nice point of reference to showcase how bad it<BR>
TV> can get and that the not so obvious choice of not using all<BR>
TV> cpus actually gives you the better performance.<BR>
TV><BR>
TV> cheers,<BR>
TV> axel.<BR>
TV><BR>
TV> TV><BR>
TV> TV> Thank you,<BR>
TV> TV><BR>
TV> TV> Trinh <BR>
TV> TV><BR>
TV> TV><BR>
TV> TV><BR>
TV> TV> -----Original Message-----<BR>
TV> TV> From: pw_forum-bounces@pwscf.org [<a href="mailto:pw_forum-bounces@pwscf.org]">mailto:pw_forum-bounces@pwscf.org]</a> On Behalf Of Axel Kohlmeyer<BR>
TV> TV> Sent: Thursday, February 26, 2009 1:04 PM<BR>
TV> TV> To: PWSCF Forum<BR>
TV> TV> Subject: Re: [Pw_forum] Insufficient Virtual Memory<BR>
TV> TV><BR>
TV> TV> On Thu, 26 Feb 2009, Vo, Trinh wrote:<BR>
TV> TV><BR>
TV> TV> TV> Dear PWSCF users,<BR>
TV> TV> TV><BR>
TV> TV> TV> I was running a benchmark (using example21 in QE) for my computer cluster (total 18 nodes) to compare the results with those posted on web link that was mentioned by Axel in some previous forum emails..<BR>
TV> TV> TV><BR>
TV> TV> TV> When I performed the calculation for the case of 512 H2O with 4cores/node, the job crashed with the error:<BR>
TV> TV> TV><BR>
TV> TV> TV> "forrtl: severe (41): insufficient virtual memory".<BR>
TV> TV> TV><BR>
TV> TV><BR>
TV> TV> TV> I do not know how to deal with this. Any help or suggestion is<BR>
TV> TV> TV> appreciated. For other cases, the runs were fine.<BR>
TV> TV><BR>
TV> TV> have you _read_ the README file in that directory???<BR>
TV> TV><BR>
TV> TV> axel.<BR>
TV> TV><BR>
TV> TV><BR>
TV> TV> TV><BR>
TV> TV> TV> Below are the specifications of my cluster:<BR>
TV> TV> TV><BR>
TV> TV> TV> Intel CPU Xeon X5450 Quad Core 3.0GHz with 12M cache 133MHz FSB<BR>
TV> TV> TV> DDRII667 4GB RAM FBD ECC (D-667D2D4F5/4GB)<BR>
TV> TV> TV><BR>
TV> TV> TV> For compilation, I used the following:<BR>
TV> TV> TV> QE ver. 4.0.3<BR>
TV> TV> TV> ifort 10.1015, MKL 10.0.3.020, impi 3.1<BR>
TV> TV> TV> internal fftw<BR>
TV> TV> TV><BR>
TV> TV> TV> I also attach here the output file.<BR>
TV> TV> TV><BR>
TV> TV> TV><BR>
TV> TV> TV> Thank you,<BR>
TV> TV> TV><BR>
TV> TV> TV> Trinh Vo<BR>
TV> TV> TV><BR>
TV> TV> TV><BR>
TV> TV> TV><BR>
TV> TV> TV><BR>
TV> TV> TV><BR>
TV> TV><BR>
TV> TV><BR>
TV><BR>
TV><BR>
<BR>
--<BR>
=======================================================================<BR>
Axel Kohlmeyer akohlmey@cmm.chem.upenn.edu <a href="http://www.cmm.upenn.edu">http://www.cmm.upenn.edu</a><BR>
Center for Molecular Modeling -- University of Pennsylvania<BR>
Department of Chemistry, 231 S.34th Street, Philadelphia, PA 19104-6323<BR>
tel: 1-215-898-1582, fax: 1-215-573-6233, office-tel: 1-215-898-5425<BR>
=======================================================================<BR>
If you make something idiot-proof, the universe creates a better idiot.<BR>
_______________________________________________<BR>
Pw_forum mailing list<BR>
Pw_forum@pwscf.org<BR>
<a href="http://www.democritos.it/mailman/listinfo/pw_forum">http://www.democritos.it/mailman/listinfo/pw_forum</a><BR>
<BR>
</SPAN></FONT></FONT></BLOCKQUOTE>
</BODY>
</HTML>