Hi all!<div><br></div><div>I had some trouble implementing the grid calculation for phonon calculation, which splits the calculation over q and representations.</div><div><br></div><div><div style>I have spend last few days try to do the splitting following the run_example script in GRID_example folder.</div>
<div style><font color="#222222" face="arial, sans-serif">I am running the QE V4.3 on HPC machine </font></div>(IBM Blade Center Linux Cluster, henry2, 981 dual Xeon compute nodes with Intel Xeon Processors (mix of single-, dual-, quad-, and six- core), 2-4GB per core distributed memory, dual gigabit Ethernet interconnects) <div style>
<font color="#222222" face="arial, sans-serif">8-16 cpus are assigned to each splitted jobs </font></div><div style><br></div><div style>After some calculations, I found that the splitting is actually not feasible on HPC!</div>
<div style>Before you can split the job over q points and representations, you have to copy PREFIX.save (which is generated by scf calculation) to each sub directories where you want to do the ph calculation separately. If the system is slightly large (more 20 atoms in unit cell), the PREFIX.save will be several GBs, then let's say if you want to split over 3 q points and 48 representations, you have to copy the PREFIX.save files 3x48=144 times! (144x 3GB + 144 X 5GB > 1TB)</div>
<div style><br></div><div style>I had a script doing this job, but it quickly used up the disk space (it used more 1TB )! which made this approach unusable on HPC machine!</div><div style><br></div><div style>Is there anyway to do the splitting phonon calculation without copying the
PREFIX.save generated by scf to each sub directories? </div><div style>Can we let all the splitted phonon calculations look for the same input directory for PREFIX.save so that we do not need to copy files around? </div>
<div style><br></div><div style>Any comment on how to do the splitting job without using up the disk space will be greatly appreciated!</div><div style>Thank you,</div><div style>Rui Mao</div><div style><br></div><div style>
Part of the script that copy the PREFIX.save files generated by scf to each sub directories:</div><div style><br></div><div style><div><font color="#222222" face="arial, sans-serif">mkdir $TMP_DIR/$q.$irr</font></div><div>
<font color="#222222" face="arial, sans-serif">cp -r $TMP_DIR/$PREFIX.save $TMP_DIR/$q.$irr</font></div><div><font color="#222222" face="arial, sans-serif">mkdir -p $TMP_DIR/$q.$irr/_ph0/$PREFIX.phsave</font></div><div><font color="#222222" face="arial, sans-serif">cp -r $TMP_DIR/_ph0/$PREFIX.phsave/* $TMP_DIR/$q.$irr/_ph0/$PREFIX.phsave</font></div>
</div><div style><br></div><div style>Scf putfile:</div><div style><br></div><div><div>&control</div><div> calculation = 'scf',</div><div> restart_mode='from_scratch',</div><div> wf_collect=.true.,</div>
<div> prefix='graphite',</div><div> tstress = .true.,</div><div> tprnfor = .true.,</div><div> forc_conv_thr = 1.0D-3,</div><div> etot_conv_thr = 1.0D-4,</div><div> pseudo_dir = '/home/rmao/Pseudo/',</div>
<div> outdir='/share2/Ray/grid_graphite/scf',</div><div> /</div><div> &system</div><div> ibrav=4, </div><div> celldm(1) =9.2961,</div><div> celldm(3)=1.3579, </div><div> nat=16, </div><div> ntyp= 1,</div>
<div> ecutwfc =70,</div><div> ecutrho=700,</div><div> occupations='smearing', smearing='mp', degauss=0.003,</div><div> /</div><div> &electrons</div><div> diagonalization='cg',</div>
<div> mixing_mode = 'plain',</div><div> mixing_beta = 0.5,</div><div> conv_thr = 1.0d-8,</div><div>/</div><div>ATOMIC_SPECIES</div><div>C 12.01070 C.pz-rrkjus.UPF</div><div>ATOMIC_POSITIONS {angstrom}</div>
<div>C 1.183416184 0.709967199 -3.339999736</div><div>C 1.183397032 2.130056591 -3.339999831</div><div>C 2.413378375 2.839955863 -3.339999742</div><div>C -0.046321206 2.840033775 -3.339999742</div>
<div>C 2.413196667 4.260044397 -3.339999829</div><div>C -0.046512842 4.260108061 -3.339999831</div><div>C 1.183461168 4.970022576 -3.339999741</div><div>C 3.643012948 2.129829924 -3.339999823</div>
<div>C 0.000008073 -0.000036198 -0.000000172</div><div>C -0.000009822 1.420051269 -0.000000260</div><div>C 1.229970973 2.129951526 -0.000000171</div><div>C -1.229729485 2.130029208 -0.000000172</div>
<div>C 1.229791025 3.550039979 -0.000000260</div><div>C -1.229918926 3.550102386 -0.000000260</div><div>C 0.000053606 4.260017465 -0.000000171</div><div>C 2.459606230 1.419825978 -0.000000260</div>
<div>K_POINTS {automatic}</div><div>8 8 16 0 0 0</div></div><div><br></div><div><span style>Ph inputfile</span></div><div><span style><br></span></div><div><div>phonons of Graphite_2x2</div><div> &inputph</div><div> tr2_ph=1.0d-16,</div>
<div> alpha_mix(1)=0.5,</div><div> prefix='graphite',</div><div> ldisp=.true.,</div><div> epsil=.false.,</div><div> recover=.false.,</div><div> reduce_io=.false.,</div><div> amass(1)=12.01070,</div><div> start_irr=0,</div>
<div> last_irr=0,</div><div> nq1=1, nq2=1, nq3=4,</div><div> outdir='/share2/Ray/grid_graphite/scf',</div><div> fildyn='graphite.dyn'</div><div> /</div></div><div><br></div><div><br></div><div><br></div>
-- <br>Rui Mao<br>============================================================<br>Department of Electrical and Computer Engineering (ECE)<br>North Carolina State University (NCSU)<br>Raleigh, NC, 27606<br>Email: <a href="mailto:rmao@ncsu.edu">rmao@ncsu.edu</a><br>
Email: <a href="mailto:ruimao20@gmail.com">ruimao20@gmail.com</a><br>============================================================<br>
</div>