<html><head><base href="x-msg://37/"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>Il giorno 08/giu/2011, alle ore 20.20, Vi Vo ha scritto:</div><br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; ">Is it possible that I double the number of CPUs and use -npool 2? <span class="Apple-converted-space"> </span><br></div></div></span></blockquote><div><br></div>Yes, it is possible, and that's what you should to in order to speed up the calculation.</div><div><br></div><div><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "> I see the number of planes printed out in the beginning of the output is 90CPUs. Thus the max number of CPUs that I can use is 90CPUs, ie one CPU per plane. </div></div></span></blockquote><div><br></div></div><div>The number of cpus employed for the FFT parallelization is nproc_pool=nproc_tot/npool. Therefore, if you double the number of processors (let's say from 90 to 180) and double the number of pools (from 1 to 2), then you will still have the same parallelization scheme for the FFT grid (within each pool).</div><div><br></div><div>In principle nothing forbids you to use more cpus than the number of FFT planes, but this might be inefficient. Then you should employ task groups or threading via OpenMP (for more detail, please refer to the user guide or to the QE paper, linked in the bibliography section of the <a href="http://quantum-espresso.org">quantum-espresso.org</a> website)</div><div><br></div><div><br></div><div>HTH</div><div><br></div><div>GS<br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; ">Thanks,<br><br>Vi<br><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><font face="Tahoma" size="2"><hr size="1"><b><span style="font-weight: bold; ">From:</span></b><span class="Apple-converted-space"> </span>Gabriele Sclauzero <<a href="mailto:sclauzer@sissa.it">sclauzer@sissa.it</a>><br><b><span style="font-weight: bold; ">To:</span></b><span class="Apple-converted-space"> </span>PWSCF Forum <<a href="mailto:pw_forum@pwscf.org">pw_forum@pwscf.org</a>><br><b><span style="font-weight: bold; ">Sent:</span></b><span class="Apple-converted-space"> </span>Wed, June 8, 2011 12:18:49 AM<br><b><span style="font-weight: bold; ">Subject:</span></b><span class="Apple-converted-space"> </span>Re: [Pw_forum] nscf restart<br></font><br>Dear Vi,<div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Il giorno 07/giu/2011, alle ore 23.46, Vi Vo ha scritto:</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; font-size: medium; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font-family: times, serif; font-size: 12pt; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Dear All,<br><br>I need to run nscf with a kpt-grid 17x17x17. However, I can only have 24 hrs to run, so the job won't be finished in that short time slot. I will need to restart after every 24hrs. If I use the 'restart' option, one thing I am worried is that after the first run, the scf charge density file will be overwritten and replaced by nscf charge density file. </div></div></div></span></blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I don't think that the nscf run will overwrite the scf charge density file. I think it will just be read and used to compute the scf potential. What will be changed are the eigenfunctions in .wfc and the eigenvalues in the restart files inside .save</div><br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; font-size: medium; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font-family: times, serif; font-size: 12pt; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">When the job is restarted, the charge density file saved in previous run and the *.wfc files will be read. Is the continuing nscf calculation still correct?<br></div></div></div></span></blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I remember that this could be done it correctly if one specifies disk_io="high". Then some additional files should be written to keep track at which k-point and band the calculation is stopped. At that time the max_seconds option was not working in that case (because the check is outside the subroutine electrons), so the run is interrupted "brutally" (i.e. with kill by the queuing system, in your case). Anyway the restart should work fine anyway if you add this option. You can also add verbosity="high" to see how many k-points have been computed up to that point. </div><br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; font-size: medium; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font-family: times, serif; font-size: 12pt; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br>One other option is that I can look at how many kpts needed for the grid 17x17x17 by using the kpt list printed out in the output file if 17x17x17 is used, then run smaller jobs, each of which, for example, includes the nscf calculation of 20 kpts. However, when I started the job in this way, more kpts than those that I specified in the input file were calculated, eg 40kpts instead of 20kpts. I understand the code searched for other equivalent kpts and calculate them. Because of this, the job required longer time to finish all 40 kpts than the time I plan. In order to avoid this, I specified the option "nosym=.true.", so that only those kpts I specify in the input file are calculated. However, I am not sure if it is correct to do in this way. </div></div></div></span></blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Not sure either that this gives you exactly what you want and I don't know if there are other side effects.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br></div><br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; font-size: medium; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font-family: times, serif; font-size: 12pt; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Another point encountered is that by chopping into smaller jobs, the kpt weight in each 20kpts-job is not correct anymore due to the way the code compute the kpt weight. </div></div></div></span></blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Of course if you want to compute DOS or PDOS, and therefore need also correct weights, the above method is not the ideal.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">What about using more processors together with pools? You should be able to reduce by about a half the running time by doubling the number of processors with -npool 2, for instance.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br></div>HTH</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">GS</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; font-size: medium; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font-family: times, serif; font-size: 12pt; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Could you give an advise if there is anything wrong on way I described above?<br><br>Thank you very much,<br><br>Vi<br>University of Houston <span class="Apple-converted-space"> </span><br><br></div></div>_______________________________________________<br>Pw_forum mailing list<br><a rel="nofollow" ymailto="mailto:Pw_forum@pwscf.org" target="_blank" href="mailto:Pw_forum@pwscf.org">Pw_forum@pwscf.org</a><br><span><a target="_blank" href="http://www.democritos.it/mailman/listinfo/pw_forum">http://www.democritos.it/mailman/listinfo/pw_forum</a></span><br></div></span></blockquote></div><br><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><span class="Apple-style-span" style="color: rgb(126, 126, 126); font-size: 16px; font-style: italic; "><br class="Apple-interchange-newline">§ Gabriele Sclauzero, EPFL SB ITP CSEA</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font class="Apple-style-span" color="#7e7e7e"><i> PH H2 462, Station 3, CH-1015 Lausanne</i></font></div></span></div><br></div></div></div></div>_______________________________________________<br>Pw_forum mailing list<br><a href="mailto:Pw_forum@pwscf.org">Pw_forum@pwscf.org</a><br><a href="http://www.democritos.it/mailman/listinfo/pw_forum">http://www.democritos.it/mailman/listinfo/pw_forum</a><br></div></span></blockquote></div><br><div>
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div><span class="Apple-style-span" style="color: rgb(126, 126, 126); font-size: 16px; font-style: italic; "><br class="Apple-interchange-newline">§ Gabriele Sclauzero, EPFL SB ITP CSEA</span></div><div><font class="Apple-style-span" color="#7E7E7E"><i> PH H2 462, Station 3, CH-1015 Lausanne</i></font></div></span>
</div>
<br></body></html>