<p>
<span style="font-family:Arial;font-size:16px;"><span style="line-height:1.5;">Dear QE users and developers,</span></span>
</p>
<p>
<span style="font-family:Arial;font-size:16px;"><span style="line-height:1.5;">I have been using QE6.4.1 version quite normally. Few days ago, I tried to compile the latest QE6.7 with the same Intel compiler. The compilation went smoothly, but there were two issues in the calculation</span><span style="line-height:1.5;">.</span><br>
</span>
</p>
<p>
<span style="font-family:Arial;font-size:16px;"><span style="line-height:1.5;"><strong>First, the calculation of QE6.7 seems much slower than QE6.4.1</strong><strong></strong><br>
</span></span>
</p>
<p>
<span style="font-family:Arial;font-size:16px;"><span style="line-height:1.5;">Following is the second q point in phonon calculation of Graphene. As you can see, it 's about 10 times slower for every single step.(the input files I used are in attachment)</span></span>
</p>
<p>
<span style="font-family:Arial;font-size:16px;"><span style="line-height:1.5;">################################</span></span>
</p>
<p>
QE6.4.1
</p>
<p>
Representation # 1 mode # 1<br>
<br>
Self-consistent Calculation<br>
<br>
iter # 1 total cpu time : 236.4 secs av.it.: 6.5<br>
thresh= 1.000E-02 alpha_mix = 0.300 |ddv_scf|^2 = 2.305E-05<br>
<br>
iter # 2 total cpu time : 246.7 secs av.it.: 8.7<br>
thresh= 4.801E-04 alpha_mix = 0.300 |ddv_scf|^2 = 1.071E-05<br>
<br>
iter # 3 total cpu time : 256.4 secs av.it.: 8.0<br>
thresh= 3.273E-04 alpha_mix = 0.300 |ddv_scf|^2 = 2.762E-09<br>
<br>
iter # 4 total cpu time : 268.9 secs av.it.: 11.6<br>
thresh= 5.256E-06 alpha_mix = 0.300 |ddv_scf|^2 = 4.235E-10<br>
<br>
iter # 5 total cpu time : 281.8 secs av.it.: 12.0<br>
thresh= 2.058E-06 alpha_mix = 0.300 |ddv_scf|^2 = 7.233E-12<br>
<br>
iter # 6 total cpu time : 294.1 secs av.it.: 11.3<br>
thresh= 2.689E-07 alpha_mix = 0.300 |ddv_scf|^2 = 1.057E-12<br>
<br>
iter # 7 total cpu time : 306.7 secs av.it.: 11.6<br>
thresh= 1.028E-07 alpha_mix = 0.300 |ddv_scf|^2 = 1.011E-13<br>
<br>
iter # 8 total cpu time : 318.5 secs av.it.: 10.5<br>
thresh= 3.179E-08 alpha_mix = 0.300 |ddv_scf|^2 = 2.282E-14<br>
<br>
iter # 9 total cpu time : 330.9 secs av.it.: 11.4<br>
thresh= 1.511E-08 alpha_mix = 0.300 |ddv_scf|^2 = 8.720E-15
</p>
<div style="white-space:nowrap;">
<br>
</div>
QE6.7
<p>
<br>
</p>
<p>
Representation # 1 mode # 1<br>
<br>
Self-consistent Calculation<br>
<br>
iter # 1 total cpu time : 585.9 secs av.it.: 6.8<br>
thresh= 1.000E-02 alpha_mix = 0.300 |ddv_scf|^2 = 7.246E-06<br>
<br>
iter # 2 total cpu time : 684.9 secs av.it.: 9.5<br>
thresh= 2.692E-04 alpha_mix = 0.300 |ddv_scf|^2 = 1.917E-06<br>
<br>
iter # 3 total cpu time : 777.8 secs av.it.: 9.0<br>
thresh= 1.385E-04 alpha_mix = 0.300 |ddv_scf|^2 = 2.398E-09<br>
<br>
iter # 4 total cpu time : 890.5 secs av.it.: 11.4<br>
thresh= 4.897E-06 alpha_mix = 0.300 |ddv_scf|^2 = 1.623E-10<br>
<br>
iter # 5 total cpu time : 1006.1 secs av.it.: 11.6<br>
thresh= 1.274E-06 alpha_mix = 0.300 |ddv_scf|^2 = 7.910E-12<br>
<br>
iter # 6 total cpu time : 1116.9 secs av.it.: 11.4<br>
thresh= 2.812E-07 alpha_mix = 0.300 |ddv_scf|^2 = 1.176E-12<br>
<br>
iter # 7 total cpu time : 1231.4 secs av.it.: 11.5<br>
thresh= 1.084E-07 alpha_mix = 0.300 |ddv_scf|^2 = 4.460E-14<br>
<br>
iter # 8 total cpu time : 1341.1 secs av.it.: 11.2<br>
thresh= 2.112E-08 alpha_mix = 0.300 |ddv_scf|^2 = 1.019E-15<br>
<br>
iter # 9 total cpu time : 1452.8 secs av.it.: 11.3<br>
thresh= 3.193E-09 alpha_mix = 0.300 |ddv_scf|^2 = 1.677E-16<span style="white-space:nowrap;"></span>
</p>
<p>
<span style="font-size:16px;font-family:Arial;">###################################################################</span>
</p>
<p>
<span style="font-size:16px;font-family:Arial;"><br>
</span>
</p>
<p>
<span style="font-size:16px;font-family:Arial;line-height:1.5;"><strong>second, the phonon running crashed after the second q point calculation finished in QE6.7, while it ran to the end successfully in QE6.4.1</strong></span><span style="font-size:16px;font-family:Arial;line-height:1.5;"><strong></strong></span>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;"><span style="line-height:1.5;">If I restart the phonon calculation by recover=.true. , it can go on running, but crashed again after the third q point was finished</span><span style="line-height:1.5;">.</span></span></span>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;"><span style="line-height:1.5;">the error message is as following:</span><span style="line-height:1.5;"></span></span></span>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;">Fatal error in PMPI_Comm_split: Other MPI error, error stack:<br>
PMPI_Comm_split(532)................: MPI_Comm_split(comm=0xc400000d, color=2, key=2, new_comm=0x7ffe99ed9478) failed<br>
PMPI_Comm_split(508)................: fail failed<br>
MPIR_Comm_split_impl(260)...........: fail failed<br>
MPIR_Get_contextid_sparse_group(672): Too many communicators (16357/16384 free on this process; ignore_id=0)<br>
Fatal error in PMPI_Comm_split: Other MPI error, error stack:<br>
PMPI_Comm_split(532)................: MPI_Comm_split(comm=0xc400000d, color=3, key=1, new_comm=0x7ffe5fe7bc78) failed<br>
PMPI_Comm_split(508)................: fail failed<br>
MPIR_Comm_split_impl(260)...........: fail failed<br>
MPIR_Get_contextid_sparse_group(672): Too many communicators (16357/16384 free on this process; ignore_id=0)<br>
Fatal error in PMPI_Comm_split: Other MPI error, error stack:<br>
PMPI_Comm_split(532)................: MPI_Comm_split(comm=0xc400000d, color=3, key=3, new_comm=0x7ffc20a56278) failed<br>
PMPI_Comm_split(508)................: fail failed<br>
MPIR_Comm_split_impl(260)...........: fail failed<br>
MPIR_Get_contextid_sparse_group(672): Too many communicators (16357/16384 free on this process; ignore_id=0)<br>
Fatal error in PMPI_Comm_split: Other MPI error, error stack:<br>
PMPI_Comm_split(532)................: MPI_Comm_split(comm=0xc400000d, color=4, key=0, new_comm=0x7ffea22b5478) failed<br>
PMPI_Comm_split(508)................: fail failed<br>
MPIR_Comm_split_impl(260)...........: fail failed<br>
MPIR_Get_contextid_sparse_group(672): Too many communicators (16357/16384 free on this process; ignore_id=0)<br>
</span></span>
</p>
<div>
<br>
</div>
<p>
<br>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;"><span style="line-height:1.5;">For QE6.4.1 and 6.7, both are compiled with Intel2018 as following (I also put the make.inc in the attachment):</span><span style="line-height:1.5;"></span></span></span>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;"> </span></span>
</p>
<p style="margin-left:0in;text-align:left;">
<span style="line-height:1.5;font-size:16px;font-family:Arial;">source /THL7/software/intel2018.4/compilers_and_libraries_2018.5.274/linux/bin/compilervars.sh intel64</span>
</p>
<p style="margin-left:0in;text-align:left;">
<span style="font-size:16px;font-family:Arial;">./configure
--prefix=/THL7/home/soft/QuantumEspresso/qe-6.7</span>
</p>
<p style="margin-left:0in;text-align:left;">
<span style="font-size:16px;font-family:Arial;">--with-scalapack=intel
CC="icc" FC="ifort" F77="ifort"
MPICC="mpiicc" MPIF90="mpiifort"</span>
</p>
<p style="margin-left:0in;text-align:left;">
<span style="font-size:16px;font-family:Arial;">DFLAGS="-D__DFTI -D__MPI -D__SCALAPACK
-D__FFTW"</span>
</p>
<p style="margin-left:0in;text-align:left;">
<span style="font-size:16px;font-family:Arial;">LDFLAGS=-shared-intel</span>
</p>
<p style="margin-left:0in;text-align:left;">
<span style="font-size:16px;font-family:Arial;">FFT_LIBS="-L/THL7/software/intel2018.4/compilers_and_libraries_2018.5.274/linux/mkl/interfaces/fftw3xf
-lfftw3xf_intel"</span>
</p>
<p>
<br>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;">After configuration, it shows:</span></span>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;">The following libraries have been found:<br>
BLAS_LIBS= -lmkl_intel_lp64 -lmkl_sequential -lmkl_core<br>
LAPACK_LIBS=<br>
SCALAPACK_LIBS=-lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64<br>
FFT_LIBS=-L/THL7/software/intel2018.4/compilers_and_libraries_2018.5.274/linux/mkl/interfaces/fftw3xf -lfftw3xf_intel<br>
</span></span>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;">It's weird for me that the same compilation settings work well for QE6.4.1, but failed for QE6.7. I guess maybe some settings are no longer suitable for QE6.7.<br>
</span></span>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;">For the first issue, I have also tested it in QE6.5 and QE6.6, the calculation are as slow as QE6.7. Someone can tell me what caused the slower calculation after the 6.4.1 version?<br>
For the second issue, I have check the Intel communiny. <a href="https://community.intel.com/t5/Intel-oneAPI-HPC-Toolkit/MPI-error-while-running-SIESTA-code/td-p/1073134" title="" target="_blank">https://community.intel.com/t5/Intel-oneAPI-HPC-Toolkit/MPI-error-while-running-SIESTA-code/td-p/1073134</a> the staff said the same problem has been fixed in MKL 2018u3. well, one side, I'm using the Intel2018u4; the other side, it works well for QE6.4.1. Someone can give me some advice?</span></span>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;"><br>
</span></span>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;">Sorry for such long post and such terrible English.</span></span>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;"><br>
</span></span>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;">Best regards,</span></span>
</p>
<p>
<span style="font-family:Arial;"><span style="font-size:16px;">Jian-qi Huang<br>
<br>
</span></span>
</p>
<p>
<br>
</p>
<br>
<span class="spnEditorSign">
<hr class="signature-separator" align="left" style="margin:0.5em 0;width:10em;height:1px;background-color:#999;border:none;">
<p>
Jian-qi Huang
</p>
<p>
<span style="display:inline !important;float:none;background-color:transparent;color:#000000;font-family:" 宋体="">Magnetism and Magnetic Materials Division</span><br style="background-color:transparent;box-sizing:content-box;color:#000000;font-family:&font-size:14px;font-style:normal;font-variant:normal;font-weight:400;height:auto;letter-spacing:normal;line-height:16.8px;margin-bottom:0px;margin-left:0px;margin-right:0px;margin-top:0px;orphans:2;overflow:visible;padding-bottom:0px;padding-left:0px;padding-right:0px;padding-top:0px;text-align:left;text-decoration:none;text-indent:0px;text-transform:none;-webkit-text-stroke-width:0px;white-space:normal;width:auto;word-spacing:0px;word-wrap:break-word;">
<span style="display:inline !important;float:none;background-color:transparent;color:#000000;font-family:" 宋体="">Institute of Metal Research </span><br style="background-color:transparent;box-sizing:content-box;color:#000000;font-family:&font-size:14px;font-style:normal;font-variant:normal;font-weight:400;height:auto;letter-spacing:normal;line-height:16.8px;margin-bottom:0px;margin-left:0px;margin-right:0px;margin-top:0px;orphans:2;overflow:visible;padding-bottom:0px;padding-left:0px;padding-right:0px;padding-top:0px;text-align:left;text-decoration:none;text-indent:0px;text-transform:none;-webkit-text-stroke-width:0px;white-space:normal;width:auto;word-spacing:0px;word-wrap:break-word;">
<span style="display:inline !important;float:none;background-color:transparent;color:#000000;font-family:" 宋体="">Chinese Academy of Sciences</span><br style="background-color:transparent;box-sizing:content-box;color:#000000;font-family:&font-size:14px;font-style:normal;font-variant:normal;font-weight:400;height:auto;letter-spacing:normal;line-height:16.8px;margin-bottom:0px;margin-left:0px;margin-right:0px;margin-top:0px;orphans:2;overflow:visible;padding-bottom:0px;padding-left:0px;padding-right:0px;padding-top:0px;text-align:left;text-decoration:none;text-indent:0px;text-transform:none;-webkit-text-stroke-width:0px;white-space:normal;width:auto;word-spacing:0px;word-wrap:break-word;">
<span style="display:inline !important;float:none;background-color:transparent;color:#000000;font-family:" 宋体="">72 Wenhua Road, Shenyang 110016, China</span>
</p>
<p>
<span style="display:inline !important;float:none;background-color:transparent;color:#000000;font-family:" 宋体="">email:<a href="mailto:jqhuang16b@imr.ac.cn">jqhuang16b@imr.ac.cn</a></span>
</p>
<p>
<span style="display:inline !important;float:none;background-color:transparent;color:#000000;font-family:" 宋体=""><br>
</span>
</p>
</span>