<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi all,<div class=""><br class=""></div><div class="">I’m trying to compile the latest QE version with OpenMP and GPU support. The latest NVIDIA hpc package is in use (nvidia/hpc_sdk/Linux_x86_64/21.3). 4xTesla V100 SXM2 32GB are mounted on the computing node.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">There are some points that are not clear to me and I would be grateful if someone might help me in this respect:</div><div class=""><br class=""></div><div class="">i) configure command</div><div class="">export PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/21.3/comm_libs/openmpi/openmpi-3.1.5/bin/:/opt/nvidia/hpc_sdk/Linux_x86_64/21.3/compilers/bin/:$PATH</div><div class="">./install/configure FC=nvfortran CC=nvcc --enable-openmp --with-cuda=/usr/local/cuda --with-cuda-cc=70 --with-cuda-runtime=11.2 LIBDIRS=/opt/nvidia/hpc_sdk/Linux_x86_64/21.3/compilers/lib/ /opt/nvidia/hpc_sdk/Linux_x86_64/21.3/math_libs/11.2/targets/x86_64-linux/lib/stubs /opt/nvidia/hpc_sdk/Linux_x86_64/21.3/comm_libs/openmpi/openmpi-3.1.5/lib/ /opt/nvidia/hpc_sdk/Linux_x86_64/21.3/comm_libs/mpi/lib/</div><div class=""><br class=""></div><div class="">I’ve given the needed paths and libraries that should suffice to a proper installation with CUDA, lapack, blas and so on. The following compilation does work. However, I find something strange in</div><div class="">make.inc :</div><div class="">LAPACK_LIBS = -L/opt/nvidia/hpc_sdk/Linux_x86_64/21.3/comm_libs/openmpi/openmpi-3.1.5/lib/ -llapack -lblas</div><div class="">The -L…. directory, among those provided through LIBDIRS, is NOT the one containing lapack and blas libraries, so although the subsequent compilation seems to work, the link as above seems to be wrong (it shouldn’t work!). Is that normal? For example, it seems that a library containing dgeem is not found. However, /opt/nvidia/hpc_sdk/Linux_x86_64/21.3/compilers/lib/ contains both libblas.a and liblapack.a .</div><div class=""><br class=""></div><div class="">The install/config.log is attached for further checks.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">ii) CUDA support seems to have been correctly included. I tried to run pw.x and it seems to work. Also, using nvidia-smi, I see that the pw.x processes are running on the GPUs. However, while on Marconi100 the output file contains a section like</div><div class=""> GPU acceleration is ACTIVE.<br class=""><br class=""><br class=""> GPU used by master process:<br class=""><br class=""> Device Number: 0<br class=""> Device name: Tesla V100-SXM2-16GB<br class=""> Compute capability : 70<br class=""> Ratio of single to double precision performance : 2<br class=""> Memory Clock Rate (KHz): 877000<br class=""> Memory Bus Width (bits): 4096<br class=""> Peak Memory Bandwidth (GB/s): 898.05<br class="">, I do not find any message like this in my output. Does it mean that the GPU support has some troubles?</div><div class=""><br class=""></div><div class="">iii) last point, it seems that OpenMP support has not been included. I cannot find any indication of that either in config.log or in make.inc (also attached to this message), despite —enable-openmp was specified to the configure command. I also do not find any information in the output concerning the number of threads. Is there anything else I should specify at compiling / running time ?</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">I’m very sorry for overburdening you with so many questions, but any help would be very much appreciated.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class=""><br class=""></div><div class=""> Giovanni</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""></div></body></html>