[QE-users] QE GPU installation problems

[Steve Prior]

Hi all, I'm struggling to get GPU-accelerated QE to install. I'm using a Dell T7910 with a Quadro M6000 24GB running Linux Mint 19.3 Cinnamon.

$CUDA_HOME is empty - I have these folders - which one do I use?

/opt/nvidia/hpc_sdk/Linux_x86_64/22.3/cuda
/usr/local/cuda-11.4

the /usr/local option doesn't contain nvfortran, just nvcc - is this important? I've tried setting CUDA_HOME to every variation I can think of.

nvaccelinfo returns the following:

# /opt/nvidia/hpc_sdk/Linux_x86_64/22.3/compilers/bin/nvaccelinfo  | grep -e 'Target' -e 'Driver'
CUDA Driver Version:           11020
Default Target:                cc50

So according to the readme I should configure with the command:

./configure --enable-openmp --with-cuda=$CUDA_HOME --with-cuda-cc=35 --with-cuda-runtime=11.2 --enable-cuda-env-check=yes

When I run this, it kicks me out asking if I'm using a PGI compiler (it looks like it can't find nvfortran and has defaulted to mpif90). If I invoke using:

./configure --enable-openmp --with-cuda="/opt/nvidia/hpc_sdk/Linux_x86_64/22.3/cuda" --with-cuda-cc=35 --with-cuda-runtime=11.6 --enable-cuda-env-check=yes MPIF90="/opt/nvidia/hpc_sdk/Linux_x86_64/22.3/compilers/bin/nvfortran" F90="/opt/nvidia/hpc_sdk/Linux_x86_64/22.3/compilers/bin/nvfortran" F77="/opt/nvidia/hpc_sdk/Linux_x86_64/22.3/compilers/bin/nvfortran"

IE supplying the absolute path to nvfortran then it does complete the configuration process. However, when I do make all it does compile a fair number of objects but eventually crashes out before creating the executables. This is the last bit of the output:

lapack-3.6.1/BLAS/SRC/ztrsm.f
lapack-3.6.1/BLAS/SRC/ztrsv.f
if test ! -e ../LAPACK/liblapack.a && test -e make_lapack.inc; then \
  (cp make_lapack.inc ../LAPACK/make.inc; \
  cd ../LAPACK; make blaslib lapacklib); else \
        (echo "no configuration file found for lapack"; \
         echo "run configure from main QE dir"; exit); fi
make[2]: Entering directory '/usr/local/qe/quantum-espresso/q-e-gpu-qe-gpu-6.7/lapack-3.6.1'
( cd BLAS/SRC; make )
make[3]: Entering directory '/usr/local/qe/quantum-espresso/q-e-gpu-qe-gpu-6.7/lapack-3.6.1/BLAS/SRC'
nvfortran -O1  -c isamax.f -o isamax.o
make[3]: nvfortran: Command not found
Makefile:171: recipe for target 'isamax.o' failed
make[3]: *** [isamax.o] Error 127
make[3]: Leaving directory '/usr/local/qe/quantum-espresso/q-e-gpu-qe-gpu-6.7/lapack-3.6.1/BLAS/SRC'
Makefile:21: recipe for target 'blaslib' failed
make[2]: *** [blaslib] Error 2
make[2]: Leaving directory '/usr/local/qe/quantum-espresso/q-e-gpu-qe-gpu-6.7/lapack-3.6.1'
extlibs_makefile:42: recipe for target 'liblapack_internal' failed
make[1]: *** [liblapack_internal] Error 2
make[1]: Leaving directory '/usr/local/qe/quantum-espresso/q-e-gpu-qe-gpu-6.7/install'
Makefile:213: recipe for target 'liblapack' failed
make: *** [liblapack] Error 2

I think it's the line in bold that's the problem (make[3]: nvfortran: Command not found) - it's not including the full path to nvfortran so isn't able to find it. I've set up an environment variable directing nvfortran to the correct file, but it doesn't seem to work. Looking at the log files, it seems to be a problem with F77 - the calls to MPIF90 and F90 seem to be redirecting to nvfortran okay (I could be wrong about this though).

I'm guessing that if I can find the correct value to store in CUDA_HOME then it'll work, but I've tried everything I can think of and the results are always the same.

Any help would be gratefully received,
Steve

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20220426/7f44af8c/attachment.html>