<div dir="ltr"><div class="gmail_default"><div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">Dear QE users and developers,</font></div><div class="gmail_default"><font face="garamond, serif" size="4" color="#000000"><br>
</font></div><div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">Recently I compiled QE-GPU on my system with GPU NVidia Tesla K20Xm.</font></div><div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">I have CUDA ver. 5.5.</font></div>
<div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">Intel MPI 4.1.3.048</font></div><div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">Intel Compiler 14.0</font></div>
<div class="gmail_default"><font face="garamond, serif" size="4" color="#000000"><br></font></div><div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">I compiled GPU-QE using the instruction on the website. I downloaded espresso-5.0.2, than downloaded QE-GPU-14.01.0 with patch file. After the patching I type a configuration command:</font></div>
<div class="gmail_default"><font face="garamond, serif" size="4" color="#000000"><br></font></div><div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">./configure LAPACK_LIBS=/opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64/libmkl_lapack95_lp64.a --enable-openmp --enable-parallel --enable-cuda --with-gpu-arch=30 --with-cuda-dir=/opt/cuda/5.5 --enable-phigemm --enable-magma</font></div>
<div class="gmail_default"><font face="garamond, serif" size="4" color="#000000"><br></font></div><div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">After successful configuration I compile pw-gpu.x and everything was fine. Then I tried to test the performance of this system compared with common HPC. </font></div>
<div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">But I have troubles. As I understand commonly 1 process goes to 1GPU, so I need to run 2 MPI processes on my host with 2 GPUs. If I am wrong, please, tell me where? After the start I have the error message:</font></div>
<div class="gmail_default"><font face="garamond, serif" size="4" color="#000000"><br></font></div><div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%</font></div>
<div class="gmail_default"><font face="garamond, serif" size="4" color="#000000"> task # 0</font></div><div class="gmail_default"><font face="garamond, serif" size="4" color="#000000"> from cdiaghg : error # 61</font></div>
<div class="gmail_default"><font face="garamond, serif" size="4" color="#000000"> diagonalization (ZHEGV*) failed</font></div><div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%</font></div>
<div class="gmail_default"><font face="garamond, serif" size="4" color="#000000"><br></font></div><div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">And it does no matter what file I ran. I double checked all the files I run. In any case I had the same error. Moreover when I ran the same file using QE without GPU I had successful running. Would you be so kind to tell me where I did the mistake? And what should I do to achieve successful running?</font></div>
<div class="gmail_default"><font face="garamond, serif" size="4" color="#000000"><br></font></div><div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">The output of ldd command for pw-gpu.x is shown below:</font></div>
<div class="gmail_default"><font face="garamond, serif" size="4" color="#000000"><div class="gmail_default"> linux-vdso.so.1 => (0x00007ffff1dbb000)</div><div class="gmail_default"> libmkl_scalapack_lp64.so => /opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64/libmkl_scalapack_lp64.so (0x00007f549bc79000)</div>
<div class="gmail_default"> libmkl_blacs_intelmpi_lp64.so => /opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.so (0x00007f549ba3d000)</div><div class="gmail_default"> libmkl_gf_lp64.so => /opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64/libmkl_gf_lp64.so (0x00007f549b2f7000)</div>
<div class="gmail_default"> libmkl_gnu_thread.so => /opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64/libmkl_gnu_thread.so (0x00007f549a811000)</div><div class="gmail_default"> libmkl_core.so => /opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64/libmkl_core.so (0x00007f5499133000)</div>
<div class="gmail_default"> libcudart.so.5.5 => /opt/cuda/5.5/lib64/libcudart.so.5.5 (0x00007f5498ee6000)</div><div class="gmail_default"> libcublas.so.5.5 => /opt/cuda/5.5/lib64/libcublas.so.5.5 (0x00007f5495a2e000)</div>
<div class="gmail_default"> libcufft.so.5.5 => /opt/cuda/5.5/lib64/libcufft.so.5.5 (0x00007f5490f0d000)</div><div class="gmail_default"> libmpigf.so.4 => /opt/intel//impi/<a href="http://4.1.3.048/intel64/lib/libmpigf.so.4">4.1.3.048/intel64/lib/libmpigf.so.4</a> (0x00007f5490cdd000)</div>
<div class="gmail_default"> libmpi_dbg_mt.so.4 => /opt/intel//impi/<a href="http://4.1.3.048/intel64/lib/libmpi_dbg_mt.so.4">4.1.3.048/intel64/lib/libmpi_dbg_mt.so.4</a> (0x00007f549044c000)</div><div class="gmail_default">
libdl.so.2 => /lib64/libdl.so.2 (0x0000003c8d800000)</div><div class="gmail_default"> librt.so.1 => /lib64/librt.so.1 (0x0000003c8e800000)</div><div class="gmail_default"> libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003c8e000000)</div>
<div class="gmail_default"> libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00007f549013c000)</div><div class="gmail_default"> libm.so.6 => /lib64/libm.so.6 (0x0000003c8dc00000)</div><div class="gmail_default">
libgomp.so.1 => /usr/lib64/libgomp.so.1 (0x0000003c8fa00000)</div><div class="gmail_default"> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003c91000000)</div><div class="gmail_default"> libc.so.6 => /lib64/libc.so.6 (0x0000003c8d400000)</div>
<div class="gmail_default"> libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x0000003c94000000)</div><div class="gmail_default"> /lib64/ld-linux-x86-64.so.2 (0x0000003c8d000000)</div><div><br></div></font></div>
<div class="gmail_default"><font face="garamond, serif" size="4" color="#000000">Thank you in advance!</font></div></div><div class="gmail_default"><br></div><div><div style="font-size:13px;font-family:Verdana,Arial,Helvetica,sans-serif">
<div style="margin:0px 0px 8px"><p style="margin:0px"><span style="font-size:small"><strong><span style="font-family:georgia,palatino">--</span></strong></span><br>
<span style="font-size:small"><strong><span style="font-family:georgia,palatino">Sincerely yours,</span></strong></span><br><span style="font-size:small"><strong><span style="font-family:georgia,palatino">Alexander G. Kvashnin </span></strong></span></p>
<table border="0">
<tbody>
<tr>
<td>
<div><span style="font-size:small"><strong><span style="font-family:georgia,palatino">=====================================================<br>PhD Student <br>Moscow Institute of Physics and Technology <a href="http://s.wisestamp.com/links?url=http%3A%2F%2Fmipt.ru%2F&sn=" target="_blank"><span>http://mipt.ru/</span></a></span></strong></span></div>
<div><span style="font-size:small"><strong><span style="font-family:georgia,palatino">141700, Institutsky lane 9, Dolgoprudny, Moscow Region, Russia</span></strong></span></div>
</td>
<td><span style="font-family:georgia,palatino;font-size:x-small"><strong><br></strong></span></td>
</tr>
<tr>
<td><span style="font-size:small"><strong><span style="font-family:georgia,palatino"><span>Junior</span> <span>research</span> <span>scientist<br></span>Technological Institute for Superhard <br>
and Novel Carbon Materials <a href="http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.tisnum.ru%2F&sn=" target="_blank">http://www.tisnum.ru/</a><a href="http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.ntcstm.troitsk.ru%2F&sn=" target="_blank"><span><br>
</span></a>142190, Central'naya St. 7a, Troitsk, Moscow Region, Russia<br>=====================================================</span></strong></span></td>
<td><span style="font-family:georgia,palatino;font-size:x-small"><strong><br></strong></span></td>
</tr>
</tbody>
</table>
<p style="margin:0px"><br></p></div></div></div></div>