<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Dear Chengyang Li,<div><br></div><div>I thought a bit and there is a way to workaround the ISO_C_BINDING dependency. The ISO_C_BINDING dependency is mainly related to pinned memory allocations. Most of the time I suggest to do not turn on pinned allocations because, despite better data transfer between GPU and CPU, the overall calculation usually slow down.</div><div><br></div><div>So, I managed to remove all the ISO_C_BINDING dependencies in the code where __CUDA_PINNED is not defined. This workaround also exclude MAGMA because in order to efficiently use it some data structure has to be pinned (so the code need ISO_C_BINDING so you need to upgrade GFORTRAN). Since you are using GPU with compute capability 1.3, in the README.GPU I already suggested to disable MAGMA on cc1.3.</div><div><br></div><div>I rebuild the package for you (build8_alpha). The new code is here:</div><div><a href="http://www.fislab.disco.unimib.it/~filippo/qe-builds/espresso-5.0-GPU-build8_alpha.tar.gz">http://www.fislab.disco.unimib.it/~filippo/qe-builds/espresso-5.0-GPU-build8_alpha.tar.gz</a></div><div><br></div><div>Please You can try to compile this build in this way</div><div>$ ./configure --enable-phigemm --disable-magma --prefix=/opt/qe432 --enable-parallel --enable-cuda --with-cuda-dir=/opt/cuda --with-gpu-arch=13 --enable-openmp</div><div><br></div><div>I did a test on a workstation and it failed. It failed because GFORTRAN 4.1 is not able to compile a file called sum_band.f90 (standard CPU code, no GPU code inside it). This is the version:</div><div><div><font class="Apple-style-span" face="'Courier New'">$ gfortran -v</font></div><div><font class="Apple-style-span" face="'Courier New'">Using built-in specs.</font></div><div><font class="Apple-style-span" face="'Courier New'">Target: x86_64-redhat-linux</font></div><div><font class="Apple-style-span" face="'Courier New'">Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-libgcj-multifile --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --disable-plugin --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre --with-cpu=generic --host=x86_64-redhat-linux</font></div><div><font class="Apple-style-span" face="'Courier New'">Thread model: posix</font></div><div><font class="Apple-style-span" face="'Courier New'">gcc version 4.1.2 20080704 (Red Hat 4.1.2-50)</font></div></div><div><br></div><div><br></div><div>I suggest you to upgrade GCC/GFORTRAn to 4.4 (not version 4.5 because GNU 4.5 are not compatible with CUDA 4.0 or 4.1). Since you do not have Intel MKL library, instead of using the internal BLAS/LAPACK you can use GotoBLAS. The compilation of GotoBLAS is very easy and it supports multi-threading (by define GOTO_NUM_THREADS=XXX). Moreover phiGEMM and MAGMA are compatible with it. You can find the source code here:</div><div><a href="http://www.fislab.disco.unimib.it/~filippo/qe-builds/GotoBLAS2-1.13.tar.gz">http://www.fislab.disco.unimib.it/~filippo/qe-builds/GotoBLAS2-1.13.tar.gz</a></div><div>or here:</div><div><a href="http://www.tacc.utexas.edu/tacc-projects/gotoblas2/">http://www.tacc.utexas.edu/tacc-projects/gotoblas2/</a></div><div><br></div><div>Hope this help. I look forward to receiving a feedback from you.</div><div><br></div><div>Cheers</div><div>F</div><div><br><div><div>On Jul 14, 2012, at 10:41 PM, Filippo Spiga wrote:</div><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Dear Chengyang Li,<div><br></div><div>the problem that I can see is the GFORTRAN version. GFORTRAN 4.1 does not have iso_c_binding extensions (as far as I know, I will double-check this). So you need a newer version of GCC/GFORTRAN and I suggest you the GCC 4.4. There is no way to workaround this issue, you should upgrade your GNU compilers. However, think about to compile the code using Intel compiler (if available). Moreover I do not understand why you want to link FFTW in that way (especially the library with MPI support). If OpenMP is enable, leave the code to use its internal FFTW library. </div><div><br></div><div>I see you want to target GPU with (only) compute capability 1.3. What kind of GPU system do you have?</div><div><br></div><div>There is a specific ML for the GPU stuff. I monitor constantly that one instead of this general one that is more related to the entire QE. The address is <a href="mailto:q-e-gpgpu@qe-forge.org">q-e-gpgpu@qe-forge.org</a> and you can subscribe directly here: <a href="http://qe-forge.org/mailman/listinfo/q-e-gpgpu">http://qe-forge.org/mailman/listinfo/q-e-gpgpu</a></div><div><br></div><div>Regards,</div><div>Filippo</div></div></blockquote><br></div><div apple-content-edited="true">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">-- <br>Mr. Filippo SPIGA (穗安駒), HPC and GPU Technologist <spiga.filippo_at_gmail.com></div><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">website: <a href="http://filippospiga.me">http://filippospiga.me</a> ~ skype: filippo.spiga<br><br>«Nobody will drive us out of Cantor's paradise.» ~ David Hilbert</div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></div>
</div>
<br></div></body></html>