<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"><base href="x-msg://18/"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Dear Miller,<div><br></div><div>I do not see a any reasonable reason to compile QE using Intel compiler on a CRAY, the PGI compiler (usually the default) works. Moreover it is weird that mpif90 is called because CRAY systems the correct FORTRAN compiler is always "ftn" (that is actually a wrapper to whatever compiler is selected through the modules loaded). Anyway, I am currently using the CRAY XE6 installed at CSCS in Switzerland (MonteRosa) and I simply use the PGI compiler and the default CRAY LibSci. Basically after every login in the system and I digit:</div><div><br></div><div>$ module load pgi</div><div><br></div><div>and the environment is perfectly set up. So by running the configure in this way:</div><div><br></div><div>$ ./configure --enable-openmp --enable-parallel --with-scalapack ARCH=crayxt</div><div><br></div><div>plus manually adding to DFLAGS (inside make.sys) "<span style="font-family: Arial; ">-D__IOTK_WORKAROUND1</span>" (very important step if you are using PGI version >= 12.x), I can compile QE-5.0 using PGI and CRAY LibSci (ScaLAPACK is automatically detected) without any problem.</div><div><br></div><div>If the configure fails for some weird reasons (even if you specified ARCH=crayxt) please send to us the config.log so we can fix the problem.</div><div><br></div><div>Cheers,</div><div>F.</div><div><br></div><div><br><div><div>On Aug 16, 2012, at 12:10 AM, "Miller, MJ ERDC-RDE-ITL-MS Contractor" <<a href="mailto:MJ.Miller@erdc.dren.mil">MJ.Miller@erdc.dren.mil</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div lang="EN-US" link="blue" vlink="purple" style="font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">Classification: UNCLASSIFIED<br>Caveats: FOUO<br><br><div class="WordSection1" style="page: WordSection1; "><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">QE:<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><o:p> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">I’m trying to troubleshoot a compilation problem on our Cray XE6. The code seems to compile following the instructions given in the README, but fails to link with the MPI libraries as far as I can see. We are getting undefined reference errors that seem to indicate at least a library path problem, but may be more involved. Our efforts thus far have included setting environment variables as described in the README file, and swapping compiler and open_mpi versions. Here is a excerpt from the compile log where things start to go wrong. Please take a look and reply if you can help us.<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><o:p> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">make[2]: Entering directory `/work/mjmiller/espresso-5.0--htakephi/PW/src'<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">test -n "" && ( cd ../.. ; make -w || exit 1) || :<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">mpif90 -static-intel -openmp -o pw.x \<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "> pwscf.o libpw.a ../../Modules/libqemod.a ../../flib/ptools.a ../../flib/flib.a ../../clib/clib.a ../../iotk/src/libiotk.a -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">/opt/intel/Compiler/11.1/056/mkl/lib/em64t/libmkl_blacs_openmpi_lp64.a(igesd2d_.o): In function `igesd2d_':<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">_igesd2d_.c:(.text+0x40): undefined reference to `ompi_mpi_int'<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">_igesd2d_.c:(.text+0x99): undefined reference to `ompi_mpi_byte'<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">/opt/intel/Compiler/11.1/056/mkl/lib/em64t/libmkl_blacs_openmpi_lp64.a(sgesd2d_.o): In function `sgesd2d_':<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">_sgesd2d_.c:(.text+0x40): undefined reference to `ompi_mpi_float'<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">_sgesd2d_.c:(.text+0x99): undefined reference to `ompi_mpi_byte'<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">/opt/intel/Compiler/11.1/056/mkl/lib/em64t/libmkl_blacs_openmpi_lp64.a(dgesd2d_.o): In function `dsendrecv_':<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">_dgesd2d_.c:(.text+0x8e): undefined reference to `ompi_mpi_double'<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">_dgesd2d_.c:(.text+0xab): undefined reference to `ompi_mpi_double'<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">_dgesd2d_.c:(.text+0x101): undefined reference to `ompi_mpi_double'<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">_dgesd2d_.c:(.text+0x125): undefined reference to `ompi_mpi_double'<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">/opt/intel/Compiler/11.1/056/mkl/lib/em64t/libmkl_blacs_openmpi_lp64.a(dgesd2d_.o): In function `dgesd2d_':<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">_dgesd2d_.c:(.text+0x21a): undefined reference to `ompi_mpi_double'<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">_dgesd2d_.c:(.text+0x273): undefined reference to `ompi_mpi_byte'<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">/opt/intel/Compiler/11.1/056/mkl/lib/em64t/libmkl_blacs_openmpi_lp64.a(cgesd2d_.o): In function `cgesd2d_':<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">_cgesd2d_.c:(.text+0x99): undefined reference to `ompi_mpi_byte'<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">/opt/intel/Compiler/11.1/056/mkl/lib/em64t/libmkl_blacs_openmpi_lp64.a(zgesd2d_.o): In function `zgesd2d_':<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">_zgesd2d_.c:(.text+0x99): undefined reference to `ompi_mpi_byte'<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">/opt/intel/Compiler/11.1/056/mkl/lib/em64t/libmkl_blacs_openmpi_lp64.a(strsd2d_.o):</div></div></div></blockquote><br></div><div apple-content-edited="true">
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-align: -webkit-auto; text-indent: 0px; widows: 2; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-align: -webkit-auto; text-indent: 0px; widows: 2; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-align: -webkit-auto; text-indent: 0px; widows: 2; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-indent: 0px; widows: 2; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-indent: 0px; widows: 2; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-indent: 0px; widows: 2; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-indent: 0px; widows: 2; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-indent: 0px; widows: 2; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-indent: 0px; widows: 2; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-indent: 0px; widows: 2; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-indent: 0px; widows: 2; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-indent: 0px; widows: 2; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-indent: 0px; widows: 2; border-spacing: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">--<br>Mr. Filippo SPIGA, M.Sc., Ph.D. Candidate <br>CADMOS - Chair of Numerical Algorithms and HPC (ANCHP)<br>École Polytechnique Fédérale de Lausanne (EPFL)<br><a href="http://anchp.epfl.ch">http://anchp.epfl.ch</a> ~ <a href="http://filippospiga.me">http://filippospiga.me</a> ~ skype: filippo.spiga<br><br>«Nobody will drive us out of Cantor's paradise.» ~ David Hilbert</div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></div>
</div>
<br></div></body></html>