<html><head></head><body><div class="ydpc872b59cyahoo-style-wrap" style="font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:16px;"><div></div>
<div>Dear prof. Giannozzi</div><div><br></div><div>I am writing to the same thread as it may be relevant here.</div><div><br></div><div>I am using qe 6.5 on 3 node linux cluster.</div><div>When the calculation is performed everything runs normally. When saving wf something unusual happens.</div><div>1) calculation exits without time stamps and job done stamp</div><div>2) this happened due to mpi exit from one of the node which cannot write to out dir.</div><div>3) scf run only starts after copying the files to slave nodes without which it will terminate saying files cannot be read.</div><div>4) after pointing outdir to common paths (via NFS).</div><div>These errors have disappeared. </div><div><br></div><div>1) My question is if recent versions of QE is collecting all the wf to head node why slave nodes eco mpi abort while they have no access to head node.</div><div>2) is there any way that we can restart calculations without copying to slave nodes.</div><div><br></div><div>Thanks and regards </div><div>Janardhan </div><div><br></div><div><br></div><div><br></div><div><br></div>
<div id="ydpc872b59cyahoo_quoted_2680875141" class="ydpc872b59cyahoo_quoted">
<div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;">
<div>
On Thursday, 20 February, 2020, 11:15:53 pm IST, Paolo Giannozzi <p.giannozzi@gmail.com> wrote:
</div>
<div><br></div>
<div><br></div>
<div><div id="ydpc872b59cyiv1888137993"><div><div dir="ltr"><div>It's a long story. By default, recent versions of QE collect both the wavefunctions and the charge density into a single array on a single processor that writes them to file. Even if you do not have a parallel file system, your data is no longer spread on scratch directories that are not visible to the other processors. This means that in principle it is possible to restart, witj several potential caveats:<br clear="none"></div><div>- there is no guarantee that a batch queuing system will distribute processes across processors in the same way as in the previous run;</div><div>- pseudopotential files are in principle read from data file so they may still be a source of problems;<br clear="none"></div><div>- if you parallelize on k-points, with Nk pools, one process per pool will write wavefunctions, that will thus end up on Nk different processors.</div><div><br clear="none"></div><div>Paolo<br clear="none"></div></div><br clear="none"><div class="ydpc872b59cyiv1888137993gmail_quote"><div class="ydpc872b59cyiv1888137993yqt9991920915" id="ydpc872b59cyiv1888137993yqt27062"><div class="ydpc872b59cyiv1888137993gmail_attr" dir="ltr">On Thu, Feb 20, 2020 at 4:54 PM alberto <<a shape="rect" href="mailto:voodoo.bender@gmail.com" rel="nofollow" target="_blank">voodoo.bender@gmail.com</a>> wrote:<br clear="none"></div><blockquote class="ydpc872b59cyiv1888137993gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;"><div dir="ltr"><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;">Hi, <br clear="none"></div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;">I'm using QE in some single point simulations. <br clear="none"></div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;">In particular I'm running scf/nscf calculations <br clear="none"></div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;"><br clear="none"></div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;">In my block input <br clear="none"></div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;"><br clear="none"></div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;">calculation = 'nscf' ,<br clear="none"> restart_mode = 'from_scratch' ,<br clear="none"> outdir = './tmp_qe' ,<br clear="none"> pseudo_dir = '/home/alberto/QUANTUM_ESPRESSO/BASIS/upf_files/' ,<br clear="none"> prefix = 'BIS-IMID-PbI4_SR' ,<br clear="none"> verbosity = 'high' ,<br clear="none"> etot_conv_thr = 1.0D-8 ,<br clear="none"> forc_conv_thr = 1.0D-7 ,<br clear="none"> wf_collect = .true.</div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;"><br clear="none"></div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;">the out dir is located in /home/alberto/ and I notice that the writing/reading time is very long</div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;"><br clear="none"></div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;">I would use /tmp dir of one node where the jobs is running.</div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;">(my cluster has got some nodes xeon to 20 CPU every nodes)</div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;"><br clear="none"></div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;">This is my PBS script</div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;"><br clear="none"></div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;">## Script for parallel Quantum Espresso job by Alberto<br clear="none">## Run script with 3 arguments:<br clear="none">## $1 = Name of input-file, without extension<br clear="none">## $2 = Numbers of nodes to use (ncpus=nodes*20)<br clear="none">## $3 = Module to run<br clear="none"><br clear="none">if [ -z "$1" -o -z "$2" -o -z "$3" ]; then<br clear="none"> echo "Usage: $0 <input_file> <np> <module> "<br clear="none">fi<br clear="none"><br clear="none">if [ $2 -ge 8 ]; then<br clear="none"> NODES=$(($2/20))<br clear="none"> CPUS=20<br clear="none">else<br clear="none"> NODES=1<br clear="none"> CPUS=$2<br clear="none">fi<br clear="none"><br clear="none">cat<<EOF>$1.job<br clear="none">#!/bin/bash<br clear="none">#PBS -l nodes=xeon1:ppn=$CPUS:xeon20+xeon2:ppn=$CPUS:xeon20+xeon3:ppn=$CPUS:xeon20+xeon4:ppn=$CPUS:xeon20+xeon5:ppn=$CPUS:xeon20+xeon6:ppn=$CPUS:xeon20<br clear="none">#PBS -l walltime=9999:00:00<br clear="none">#PBS -N $1<br clear="none">#PBS -e $1.err<br clear="none">#PBS -o $1.sum<br clear="none">#PBS -j oe<br clear="none">job=$1 # Name of input file, no extension<br clear="none">project=\$PBS_O_WORKDIR<br clear="none">cd \$project<br clear="none">cat \$PBS_NODEFILE > \$PBS_O_WORKDIR/nodes.txt <br clear="none"><br clear="none">export OMP_NUM_THREADS=$(($2/40))<br clear="none">time /opt/openmpi-1.4.5/bin/mpirun -machinefile \$PBS_NODEFILE -np $2 /opt/qe-6.4.1/bin/$3 -ntg $(($2/60)) -npool $(($2/60)) < $1.inp > $1.out<br clear="none">EOF<br clear="none"><br clear="none">qsub $1.job</div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;"><br clear="none"></div><div class="ydpc872b59cyiv1888137993gmail_default" style="font-family:courier new, monospace;font-size:small;"><div><div><span lang="en"><span title="">how could I use the directory /tmp and avoid that the nscf calculation don't stop it because no files are found! </span><span lang="en"><span title="">really the files are present, but they are divided on different nodes</span></span><br clear="none"><br clear="none"><span title="">regards</span></span></div><div><br clear="none"></div><div>Alberto<br clear="none"></div><div><span lang="en"><span title=""></span></span><span></span></div></div></div></div>
_______________________________________________<br clear="none">
Quantum ESPRESSO is supported by MaX (<a shape="rect" href="http://www.max-centre.eu/quantum-espresso" rel="nofollow" target="_blank">www.max-centre.eu/quantum-espresso</a>)<br clear="none">
users mailing list <a shape="rect" href="mailto:users@lists.quantum-espresso.org" rel="nofollow" target="_blank">users@lists.quantum-espresso.org</a><br clear="none">
<a shape="rect" href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="nofollow" target="_blank">https://lists.quantum-espresso.org/mailman/listinfo/users</a></blockquote></div></div><br clear="all"><br clear="none">-- <br clear="none"><div class="ydpc872b59cyiv1888137993gmail_signature" dir="ltr"><div dir="ltr"><div><div dir="ltr"><div>Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,<br clear="none">Univ. Udine, via delle Scienze 208, 33100 Udine, Italy<br clear="none">Phone +39-0432-558216, fax +39-0432-558222<br clear="none"><br clear="none"></div></div></div></div></div></div></div><div class="ydpc872b59cyqt9991920915" id="ydpc872b59cyqt76276">_______________________________________________<br clear="none">Quantum ESPRESSO is supported by MaX (www.max-centre.eu/quantum-espresso)<br clear="none">users mailing list <a shape="rect" href="mailto:users@lists.quantum-espresso.org" rel="nofollow" target="_blank">users@lists.quantum-espresso.org</a><br clear="none"><a shape="rect" href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="nofollow" target="_blank">https://lists.quantum-espresso.org/mailman/listinfo/users</a></div></div>
</div>
</div></div></body></html>