<div dir="ltr">no I am using the same machine for the build and run, its a workstation with 32 cores</div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 7, 2018 at 8:22 PM, Ye Luo <span dir="ltr"><<a href="mailto:xw111luoye@gmail.com" target="_blank">xw111luoye@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Did you build the code and run it on different machines? Ye<br></div></div><div class="gmail_extra"><span class=""><br clear="all"><div><div class="m_3074225631177547600gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">===================<br>
Ye Luo, Ph.D.<br>
Leadership Computing Facility<br>
Argonne National Laboratory</div></div></div>
<br></span><div><div class="h5"><div class="gmail_quote">2018-08-07 19:18 GMT-05:00 Aziz Fall <span dir="ltr"><<a href="mailto:afall@umich.edu" target="_blank">afall@umich.edu</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">ok so regardless of whether I run pw.x with mpirun or not I get the same error saying pw.x: symbol lookup error: /usr/lib/libblacsCinit-openmpi<wbr>.so.1: undefined symbol: ompi_mpi_comm_world  <div><br></div><div>when I do which mpirun I get /opt/intel/compilers_and_l<wbr>ibraries_2018.3.222/linux/mpi/<wbr>intel64/bin/mpirun  so its very weird </div></div><div class="m_3074225631177547600HOEnZb"><div class="m_3074225631177547600h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 7, 2018 at 8:14 PM, Ye Luo <span dir="ltr"><<a href="mailto:xw111luoye@gmail.com" target="_blank">xw111luoye@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Everything seems good. I'm wondering if it is the problem of your mpirun.<br></div><div>Could you run pw.x directly without mpirun?</div><div>If you type "which mpirun" on the machine your are running, is it from the intel parallel studio folder since you said you are using Intel MPI.</div><span class="m_3074225631177547600m_7749504430770588510HOEnZb"><font color="#888888"><div></div><div>Ye<br></div></font></span></div><div class="gmail_extra"><span><br clear="all"><div><div class="m_3074225631177547600m_7749504430770588510m_-9070505818633134524gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">===================<br>
Ye Luo, Ph.D.<br>
Leadership Computing Facility<br>
Argonne National Laboratory</div></div></div>
<br></span><div><div class="m_3074225631177547600m_7749504430770588510h5"><div class="gmail_quote">2018-08-07 19:07 GMT-05:00 Aziz Fall <span dir="ltr"><<a href="mailto:afall@umich.edu" target="_blank">afall@umich.edu</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">yeah so when I type ldd pw.x I get the following<div><br></div><div><div>        linux-vdso.so.1 =>  (0x00007ffffcb9e000)</div><div>        libmkl_scalapack_lp64.so => /opt/intel/compilers_and_libra<wbr>ries_2018.3.222/linux/mkl/lib/<wbr>intel64_lin/libmkl_scalapack_l<wbr>p64.so (0x00007ffed7300000)</div><div>        libmkl_blacs_intelmpi_lp64.so => /opt/intel/compilers_and_libra<wbr>ries_2018.3.222/linux/mkl/lib/<wbr>intel64_lin/libmkl_blacs_intel<wbr>mpi_lp64.so (0x00007ffed70b0000)</div><div>        libmkl_gf_lp64.so => /opt/intel/compilers_and_libra<wbr>ries_2018.3.222/linux/mkl/lib/<wbr>intel64_lin/libmkl_gf_lp64.so (0x00007ffed65a0000)</div><div>        libmkl_sequential.so => /opt/intel/compilers_and_libra<wbr>ries_2018.3.222/linux/mkl/lib/<wbr>intel64_lin/libmkl_sequential.<wbr>so (0x00007ffed5130000)</div><div>        libmkl_core.so => /opt/intel/compilers_and_libra<wbr>ries_2018.3.222/linux/mkl/lib/<wbr>intel64_lin/libmkl_core.so (0x00007ffed10fe000)</div><div>        libmpifort.so.12 => /opt/intel/compilers_and_libra<wbr>ries_2018.3.222/linux/mpi/inte<wbr>l64/lib/libmpifort.so.12 (0x00007ffed0d40000)</div><div>        libmpi.so.12 => /opt/intel/compilers_and_libra<wbr>ries_2018.3.222/linux/mpi/inte<wbr>l64/lib/libmpi.so.12 (0x00007ffed00b0000)</div><div>        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthre<wbr>ad.so.0 (0x00007ffecfe70000)</div><div>        libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libg<wbr>fortran.so.3 (0x00007ffecfb40000)</div><div>        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.<wbr>6 (0x00007ffecf830000)</div><div>        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s<wbr>.so.1 (0x00007ffecf610000)</div><div>        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.<wbr>6 (0x00007ffecf240000)</div><div>        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so<wbr>.2 (0x00007ffecf020000)</div><div>        librt.so.1 => /lib/x86_64-linux-gnu/librt.so<wbr>.1 (0x00007ffecee10000)</div><div>        /lib64/ld-linux-x86-64.so.2 (0x00007ffed7e00000)</div><div>        libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libq<wbr>uadmath.so.0 (0x00007ffecebc0000)  </div></div><div><br></div><div>I am not sure I know what to make of this though, do you have an idea by any chance?</div><div><br></div><div>Sincerely Aziz Fall</div></div><div class="m_3074225631177547600m_7749504430770588510m_-9070505818633134524HOEnZb"><div class="m_3074225631177547600m_7749504430770588510m_-9070505818633134524h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 7, 2018 at 7:48 PM, Ye Luo <span dir="ltr"><<a href="mailto:xw111luoye@gmail.com" target="_blank">xw111luoye@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>$ldd pw.x.</div><div>What prints out?<br></div><div>It seems that /usr/lib/libblacsCinit-openmpi<wbr>.so.1 get into your binary at a certain point. But it should not.</div><div>This is the blacs from your OS used by scalapack but it is supposed to be the Intel one.</div><span class="m_3074225631177547600m_7749504430770588510m_-9070505818633134524m_2315919885853455854HOEnZb"><font color="#888888"><div>Ye<br></div></font></span></div><div class="gmail_extra"><span><br clear="all"><div><div class="m_3074225631177547600m_7749504430770588510m_-9070505818633134524m_2315919885853455854m_4393794887238904688gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">===================<br>
Ye Luo, Ph.D.<br>
Leadership Computing Facility<br>
Argonne National Laboratory</div></div></div>
<br></span><div><div class="m_3074225631177547600m_7749504430770588510m_-9070505818633134524m_2315919885853455854h5"><div class="gmail_quote">2018-08-07 18:21 GMT-05:00 Aziz Fall <span dir="ltr"><<a href="mailto:afall@umich.edu" target="_blank">afall@umich.edu</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">so I changed the options in configure like you suggested with the --with-scalapack=intel but I still get the same error after re-compiling. Do you have any further suggestions?</div><div class="m_3074225631177547600m_7749504430770588510m_-9070505818633134524m_2315919885853455854m_4393794887238904688HOEnZb"><div class="m_3074225631177547600m_7749504430770588510m_-9070505818633134524m_2315919885853455854m_4393794887238904688h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 7, 2018 at 6:22 PM, Aziz Fall <span dir="ltr"><<a href="mailto:afall@umich.edu" target="_blank">afall@umich.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">No, I did not add that configuration as part of configure but I did change the make.inc <div><br></div><div>to have SCALAPACK_LIBS = -L/opt/intel/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_tbb_thread -lmkl_core -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64 -lpthread -ltbb -lstdc++ -lm -ldl  and </div><div>MPI_LIBS       =  -L/opt/intel/impi/2018.3.222/l<wbr>ib64/ -lmpi, is that not enough or should I configure it again with the exact options --with-scalapack=intel? <br></div><div><br></div><div>Thank You,</div><div>Sincerely Aziz Fall </div></div><div class="m_3074225631177547600m_7749504430770588510m_-9070505818633134524m_2315919885853455854m_4393794887238904688m_-2738380387076933187HOEnZb"><div class="m_3074225631177547600m_7749504430770588510m_-9070505818633134524m_2315919885853455854m_4393794887238904688m_-2738380387076933187h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 7, 2018 at 6:04 PM, Ye Luo <span dir="ltr"><<a href="mailto:xw111luoye@gmail.com" target="_blank">xw111luoye@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>That seems for openmpi.<br></div><div>Did you put --with-scalapack=intel when you run configure?<br></div><div>Ye<br></div></div><div class="gmail_extra"><br clear="all"><div><div class="m_3074225631177547600m_7749504430770588510m_-9070505818633134524m_2315919885853455854m_4393794887238904688m_-2738380387076933187m_-7357646823492864187m_1840784576205178795gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">===================<br>
Ye Luo, Ph.D.<br>
Leadership Computing Facility<br>
Argonne National Laboratory</div></div></div>
<br><div class="gmail_quote"><div><div class="m_3074225631177547600m_7749504430770588510m_-9070505818633134524m_2315919885853455854m_4393794887238904688m_-2738380387076933187m_-7357646823492864187h5">2018-08-07 16:42 GMT-05:00 Aziz Fall <span dir="ltr"><<a href="mailto:afall@umich.edu" target="_blank">afall@umich.edu</a>></span>:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_3074225631177547600m_7749504430770588510m_-9070505818633134524m_2315919885853455854m_4393794887238904688m_-2738380387076933187m_-7357646823492864187h5"><div dir="ltr">Dear Quantum espresso team,<div><br></div><div>So recently I have been trying to run the executable pw.x on the bin directory. I pass in my input and output file appropriately, and I am using intel's Parallel studio compiler, with intel MPI. but when I run the program I get the following error </div><div><br></div><div>pw.x: symbol lookup error: /usr/lib/libblacsCinit-openmpi<wbr>.so.1: undefined symbol: ompi_mpi_comm_world   </div><div><br></div><div><br></div><div><br></div><div>my LD_LIBRARY_PATH is the following </div><div><br></div><div>LD_LIBRARY_PATH=/opt/intel/com<wbr>pilers_and_libraries_2018.3.22<wbr>2/linux/mpi/intel64/lib:/opt/i<wbr>ntel/compilers_and_libraries_2<wbr>018.3.222/linux/mpi/mic/lib:/o<wbr>pt/intel/compilers_and_librari<wbr>es_2018.3.222/linux/compiler/l<wbr>ib/intel64_lin:/opt/intel/comp<wbr>ilers_and_libraries_2018.3.222<wbr>/linux/mkl/lib/intel64_lin:/op<wbr>t/intel/compilers_and_librarie<wbr>s_2018.3.222/linux/compiler/li<wbr>b/intel64:/opt/intel/compilers<wbr>_and_libraries_2018.3.222/linu<wbr>x/compiler/lib/intel64_lin:/op<wbr>t/intel/compilers_and_librarie<wbr>s_2018.3.222/linux/mpi/intel64<wbr>/lib:/opt/intel/compilers_and_<wbr>libraries_2018.3.222/linux/mpi<wbr>/mic/lib:/opt/intel/compilers_<wbr>and_libraries_2018.3.222/linux<wbr>/ipp/lib/intel64:/opt/intel/co<wbr>mpilers_and_libraries_2018.3.<wbr>222/linux/compiler/lib/intel64<wbr>_lin:/opt/intel/compilers_and_<wbr>libraries_2018.3.222/linux/<wbr>mkl/lib/intel64_lin:/opt/<wbr>intel/compilers_and_libraries_<wbr>2018.3.222/linux/tbb/lib/intel<wbr>64/gcc4.7:/opt/intel/compilers<wbr>_and_libraries_2018.3.222/<wbr>linux/tbb/lib/intel64/gcc4.7:/<wbr>opt/intel/debugger_2018/iga/<wbr>lib:/opt/intel/debugger_2018/<wbr>libipt/intel64/lib:/opt/intel/<wbr>compilers_and_libraries_2018.<wbr>3.222/linux/daal/lib/intel64_<wbr>lin:/opt/intel/compilers_and_<wbr>libraries_2018.3.222/linux/<wbr>mpi/intel64/lib:/opt/intel/<wbr>compilers_and_libraries_2018.<wbr>3.222/linux/mpi/mic/lib:/opt/<wbr>intel/compilers_and_libraries_<wbr>2018.3.222/linux/compiler/lib/<wbr>intel64_lin:/opt/intel/<wbr>compilers_and_libraries_2018.<wbr>3.222/linux/mkl/lib/intel64_<wbr>lin:/opt/intel/compilers_and_<wbr>libraries_2018.3.222/linux/<wbr>compiler/lib/intel64:/opt/<wbr>intel/compilers_and_libraries_<wbr>2018.3.222/linux/compiler/lib/<wbr>intel64_lin:/opt/intel/<wbr>compilers_and_libraries_2018.<wbr>3.222/linux/mpi/intel64/lib:/<wbr>opt/intel/compilers_and_<wbr>libraries_2018.3.222/linux/<wbr>mpi/mic/lib:/opt/intel/<wbr>compilers_and_libraries_2018.<wbr>3.222/linux/ipp/lib/intel64:/<wbr>opt/intel/compilers_and_<wbr>libraries_2018.3.222/linux/<wbr>compiler/lib/intel64_lin:/opt/<wbr>intel/compilers_and_libraries_<wbr>2018.3.222/linux/mkl/lib/<wbr>intel64_lin:/opt/intel/<wbr>compilers_and_libraries_2018.<wbr>3.222/linux/tbb/lib/intel64/<wbr>gcc4.7:/opt/intel/compilers_<wbr>and_libraries_2018.3.222/<wbr>linux/tbb/lib/intel64/gcc4.7:/<wbr>opt/intel/debugger_2018/iga/<wbr>lib:/opt/intel/debugger_2018/<wbr>libipt/intel64/lib:/opt/intel/<wbr>compilers_and_libraries_2018.<wbr>3.222/linux/daal/lib/intel64_<wbr>lin:/opt/intel/compilers_and_<wbr>libraries_2018.3.222/linux/<wbr>mpi/intel64/lib:/opt/intel/<wbr>compilers_and_libraries_2018.<wbr>3.222/linux/mpi/mic/lib:/opt/<wbr>intel/compilers_and_libraries_<wbr>2018.3.222/linux/compiler/lib/<wbr>intel64_lin:/opt/intel/<wbr>compilers_and_libraries_2018.<wbr>3.222/linux/mkl/lib/intel64_<wbr>lin:/opt/intel/compilers_and_<wbr>libraries_2018.3.222/linux/<wbr>compiler/lib/intel64:/opt/<wbr>intel/compilers_and_libraries_<wbr>2018.3.222/linux/compiler/lib/<wbr>intel64_lin:/opt/intel/<wbr>compilers_and_libraries_2018.<wbr>3.222/linux/mpi/intel64/lib:/<wbr>opt/intel/compilers_and_<wbr>libraries_2018.3.222/linux/<wbr>mpi/mic/lib:/opt/intel/<wbr>compilers_and_libraries_2018.<wbr>3.222/linux/ipp/lib/intel64:/<wbr>opt/intel/compilers_and_<wbr>libraries_2018.3.222/linux/<wbr>compiler/lib/intel64_lin:/opt/<wbr>intel/compilers_and_libraries_<wbr>2018.3.222/linux/mkl/lib/<wbr>intel64_lin:/opt/intel/<wbr>compilers_and_libraries_2018.<wbr>3.222/linux/tbb/lib/intel64/<wbr>gcc4.7:/opt/intel/compilers_<wbr>and_libraries_2018.3.222/<wbr>linux/tbb/lib/intel64/gcc4.7:/<wbr>opt/intel/debugger_2018/iga/<wbr>lib:/opt/intel/debugger_2018/<wbr>libipt/intel64/lib:/opt/intel/<wbr>compilers_and_libraries_2018.<wbr>3.222/linux/daal/lib/intel64_<wbr>lin:/opt/intel/compilers_and_<wbr>libraries_2018.3.222/linux/<wbr>daal/../tbb/lib/intel64_lin/<wbr>gcc4.4 </div><div><br></div><div><br></div><div><br></div><div>any help will be highly appreciated. Thanks in advanced</div><div><br></div><div>Sincerely Aziz Fall </div></div>
<br></div></div>______________________________<wbr>_________________<br>
users mailing list<br>
<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.o<wbr>rg</a><br>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.quantum-espresso<wbr>.org/mailman/listinfo/users</a><br></blockquote></div><br></div>
<br>______________________________<wbr>_________________<br>
users mailing list<br>
<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.o<wbr>rg</a><br>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.quantum-espresso<wbr>.org/mailman/listinfo/users</a><br></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
users mailing list<br>
<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.o<wbr>rg</a><br>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.quantum-espresso<wbr>.org/mailman/listinfo/users</a><br></blockquote></div><br></div></div></div>
<br>______________________________<wbr>_________________<br>
users mailing list<br>
<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.o<wbr>rg</a><br>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.quantum-espresso<wbr>.org/mailman/listinfo/users</a><br></blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
users mailing list<br>
<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.o<wbr>rg</a><br>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.quantum-espresso<wbr>.org/mailman/listinfo/users</a><br></blockquote></div><br></div></div></div>
<br>______________________________<wbr>_________________<br>
users mailing list<br>
<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.o<wbr>rg</a><br>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.quantum-espresso<wbr>.org/mailman/listinfo/users</a><br></blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
users mailing list<br>
<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.o<wbr>rg</a><br>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.quantum-espresso<wbr>.org/mailman/listinfo/users</a><br></blockquote></div><br></div></div></div>
<br>______________________________<wbr>_________________<br>
users mailing list<br>
<a href="mailto:users@lists.quantum-espresso.org">users@lists.quantum-espresso.<wbr>org</a><br>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.quantum-<wbr>espresso.org/mailman/listinfo/<wbr>users</a><br></blockquote></div><br></div>