[QE-users] [ge-GPU] compiling q-e-gpu-qe-gpu-6.7
Pietro Bonfa'
pietro.bonfa at unipr.it
Sun Dec 20 18:38:03 CET 2020
Dear Mohammad,
for some reason you are having trouble accessing gitlab. I uploaded a
package that includes all dependencies and can be compiled without
network access. You can find it here:
https://univpr-my.sharepoint.com/:u:/g/personal/pietro_bonfa_unipr_it/EV-nHENjf1lFkat0RvJypFIBap2o92v9BzG75po06z48WA?e=uiDjDD
Best wishes,
Pietro
--
Pietro Bonfà,
University of Parma
On 12/19/20 7:27 AM, Mohammad Moaddeli wrote:
> Dear Louis and Pietro,
>
> *With the config options as following:*
> *
> *
> *./configure FC=pgf90 F90=pgf90 CC=pgcc
> --with-cuda=/opt/nvidia/hpc_sdk/Linux_x86_64/20.11/cuda/11.0
> --with-cuda-runtime=11.0 --with-cuda-cc=70 --enable-openmp
> --with-scalapack=no*
> *
> *
> *results is:*
>
> checking build system type... x86_64-pc-linux-gnu
> checking ARCH... x86_64
> checking setting AR... ... ar
> checking setting ARFLAGS... ... ruv
> checking whether the Fortran compiler works... yes
> checking for Fortran compiler default output file name... a.out
> checking for suffix of executables...
> checking whether we are cross compiling... no
> checking for suffix of object files... o
> checking whether we are using the GNU Fortran compiler... no
> checking whether pgf90 accepts -g... yes
> configure: WARNING: F90 value is set to be consistent with value of MPIF90
> checking for mpiifort... no
> checking for mpif90... mpif90
> checking whether we are using the GNU Fortran compiler... no
> checking whether mpif90 accepts -g... yes
> checking version of mpif90... nvfortran 20.11-0
> checking for Fortran flag to compile .f90 files... none
> setting F90... nvfortran
> setting MPIF90... mpif90
> checking whether we are using the GNU C compiler... yes
> checking whether pgcc accepts -g... yes
> checking for pgcc option to accept ISO C89... none needed
> setting CC... pgcc
> setting CFLAGS... -fast -Mpreprocess
> using F90... nvfortran
> setting FFLAGS... -O1
> setting F90FLAGS... $(FFLAGS)
> setting FFLAGS_NOOPT... -O0
> setting CPP... cpp
> setting CPPFLAGS... -P -traditional -Uvector
> setting LD... mpif90
> setting LDFLAGS...
> checking for Fortran flag to compile .f90 files... (cached) none
> checking whether Fortran compiler accepts -Mcuda=cuda11.0... yes
> checking for nvcc...
> /opt/nvidia/hpc_sdk/Linux_x86_64/20.11/compilers/bin/nvcc
> checking whether nvcc works... yes
> checking for cuInit in -lcuda... no
> configure: error: in `/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7':
> configure: error: Couldn't find libcuda
> See `config.log' for more details
>
> *Although the option --enable-cuda-env-check=no resulted in the
> configuration finished:*
> *
> *
> *./configure FC=pgf90 F90=pgf90 CC=pgcc --with-cuda=yes
> --enable-cuda-env-check=no --with-cuda-runtime=11.0 --with-cuda-cc=70
> --enable-openmp --with-scalapack=no*
>
> checking build system type... x86_64-pc-linux-gnu
> checking ARCH... x86_64
> checking setting AR... ... ar
> checking setting ARFLAGS... ... ruv
> checking whether the Fortran compiler works... yes
> checking for Fortran compiler default output file name... a.out
> checking for suffix of executables...
> checking whether we are cross compiling... no
> checking for suffix of object files... o
> checking whether we are using the GNU Fortran compiler... no
> checking whether pgf90 accepts -g... yes
> configure: WARNING: F90 value is set to be consistent with value of MPIF90
> checking for mpiifort... no
> checking for mpif90... mpif90
> checking whether we are using the GNU Fortran compiler... no
> checking whether mpif90 accepts -g... yes
> checking version of mpif90... nvfortran 20.11-0
> checking for Fortran flag to compile .f90 files... none
> setting F90... nvfortran
> setting MPIF90... mpif90
> checking whether we are using the GNU C compiler... yes
> checking whether pgcc accepts -g... yes
> checking for pgcc option to accept ISO C89... none needed
> setting CC... pgcc
> setting CFLAGS... -fast -Mpreprocess
> using F90... nvfortran
> setting FFLAGS... -O1
> setting F90FLAGS... $(FFLAGS)
> setting FFLAGS_NOOPT... -O0
> setting CPP... cpp
> setting CPPFLAGS... -P -traditional -Uvector
> setting LD... mpif90
> setting LDFLAGS...
> checking for Fortran flag to compile .f90 files... (cached) none
> checking whether Fortran compiler accepts -Mcuda=cuda11.0... yes
> checking for /usr/local/cuda/... no
> checking for /usr/local/cuda/include... no
> checking for /usr/local/cuda/lib64... no
> checking for nvcc...
> /opt/nvidia/hpc_sdk/Linux_x86_64/20.11/compilers/bin/nvcc
> checking whether nvcc works... yes
> checking for cusolverDnZhegvdx_bufferSize in -lcusolver... no
> configure: WARNING: Using legacy custom solver.
> checking whether make sets $(MAKE)... yes
> checking whether Fortran files must be preprocessed... yes
> checking whether we are using the GNU Fortran 77 compiler... no
> checking whether nvfortran accepts -g... yes
> checking for library containing dgemm... no
> MKL not found
> in /opt/intel/mkl/lib/intel64: checking for library containing dgemm...
> -lmkl_intel_lp64
> setting BLAS_LIBS... -L/opt/intel/mkl/lib/intel64 -lmkl_intel_lp64
> -lmkl_intel_thread -lmkl_core
> checking FFT...
> checking MASS...
> checking for library containing mpi_init... none required
> checking ELPA...
> checking BEEF... -lbeef
> setting BEEF_LIBS... $(TOPDIR)/LIBBEEF/libbeef.a
> checking for ranlib... ranlib
> checking for wget... wget -O
> setting WGET... wget -O
> setting DFLAGS... -D__CUDA -D__DFTI -D__MPI
> setting IFLAGS... -I$(TOPDIR)/include -I$(TOPDIR)/FoX/finclude
> -I/opt/intel/mkl/include
> configure: creating ./config.status
> config.status: creating install/make_lapack.inc
> config.status: creating include/configure.h
> config.status: creating make.inc
> config.status: creating configure.msg
> config.status: creating install/make_wannier90.inc
> config.status: creating include/qe_cdefs.h
> --------------------------------------------------------------------
> ESPRESSO can take advantage of several optimized numerical libraries
> (essl, fftw, mkl...). This configure script attempts to find them,
> but may fail if they have been installed in non-standard locations.
> If a required library is not found, the local copy will be compiled.
>
> The following libraries have been found:
> BLAS_LIBS=-L/opt/intel/mkl/lib/intel64 -lmkl_intel_lp64
> -lmkl_intel_thread -lmkl_core
> LAPACK_LIBS=
> FFT_LIBS=
>
> Please check if this is what you expect.
>
> If any libraries are missing, you may specify a list of directories
> to search and retry, as follows:
> ./configure LIBDIRS="list of directories, separated by spaces"
>
> Parallel environment detected successfully.\
> Configured for compilation of parallel executables.
>
> For more info, read the ESPRESSO User's Guide (Doc/users-guide.tex).
> --------------------------------------------------------------------
> configure: success
>
> *make pw results in:*
>
> ....
> .....
> ....
> cpp -P -traditional -Uvector -D__CUDA -D__DFTI -D__MPI
> -I/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7//include
> -I/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7//FoX/finclude
> -I/opt/intel/mkl/include thread_util.f90 -o thread_util_tmp.f90 ; \
> mpif90 -O1 -D__CUDA -D__DFTI -D__MPI -Mcuda=cc70,cuda11.0
> -I/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7//external/devxlib/src
> -I/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7//external/devxlib/include
> -I/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7//EIGENSOLVER_GPU/lib_eigsolve
> -I/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7//external/devxlib/src
> -I/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7//include
> -I/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7//FoX/finclude
> -I/opt/intel/mkl/include -I. -c thread_util_tmp.f90 -o thread_util.o
> ar ruv libutil.a clocks_handler.o device_helper.o divide.o data_buffer.o
> error_handler.o export_gstart_2_solvers.o find_free_unit.o
> fletcher32_mod.o mem_counter.o mp.o mp_base.o mp_base_gpu.o
> mp_bands_util.o parallel_include.o set_mpi_comm_4_solvers.o util_param.o
> thread_util.o
> ar: creating libutil.a
> a - clocks_handler.o
> a - device_helper.o
> a - divide.o
> a - data_buffer.o
> a - error_handler.o
> a - export_gstart_2_solvers.o
> a - find_free_unit.o
> a - fletcher32_mod.o
> a - mem_counter.o
> a - mp.o
> a - mp_base.o
> a - mp_base_gpu.o
> a - mp_bands_util.o
> a - parallel_include.o
> a - set_mpi_comm_4_solvers.o
> a - util_param.o
> a - thread_util.o
> ranlib libutil.a
> make[1]: Leaving directory `/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7/UtilXlib'
> cd install ; make -f extlibs_makefile liblapack
> make[1]: Entering directory `/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7/install'
> make[1]: Nothing to be done for `liblapack'.
> make[1]: Leaving directory `/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7/install'
> cd install ; make -f extlibs_makefile libcuda
> make[1]: Entering directory `/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7/install'
> cd ../external/devxlib; \
> if test ! -e configure; then \
> wget
> "https://gitlab.com/max-centre/components/devicexlib/-/archive/master/devicexlib-master.tar.gz
> <https://gitlab.com/max-centre/components/devicexlib/-/archive/master/devicexlib-master.tar.gz>"
> -O devxlib.tar.gz || curl
> "https://gitlab.com/max-centre/components/devicexlib/-/archive/master/devicexlib-master.tar.gz
> <https://gitlab.com/max-centre/components/devicexlib/-/archive/master/devicexlib-master.tar.gz>"
> -o devxlib.tar.gz ; \
> tar xzf devxlib.tar.gz --strip-components=1 -C . ; \
> rm devxlib.tar.gz ; \
> fi; \
> touch make.inc; \
> make clean; \
> export F90FLAGS=""; \
> ./configure FC=nvfortran CC=pgcc \
> --with-cuda= \
> --with-cuda-cc=70 \
> --with-cuda-runtime=11.0 \
> --disable-parallel \
> --enable-cuda-env-check=no; \
> make all
> --2020-12-19 09:29:12--
> https://gitlab.com/max-centre/components/devicexlib/-/archive/master/devicexlib-master.tar.gz
> <https://gitlab.com/max-centre/components/devicexlib/-/archive/master/devicexlib-master.tar.gz>
> Resolving gitlab.com
> <http://gitlab.com/>
> (gitlab.com
> <http://gitlab.com/>)...
> 172.65.251.78, 2606:4700:90:0:f22e:fbec:5bed:a9b9
> Connecting to gitlab.com
> <http://gitlab.com/>
> (gitlab.com
> <http://gitlab.com/>)|172.65.251.78|:443...
> connected.
> HTTP request sent, awaiting response... 403 Forbidden
> 2020-12-19 09:29:23 ERROR 403: Forbidden.
>
> % Total % Received % Xferd Average Speed Time Time Time
> Current
> Dload Upload Total Spent Left
> Speed
> 100 16 100 16 0 0 27 0 --:--:-- --:--:-- --:--:--
> 27
>
> gzip: stdin: not in gzip format
> tar: Child returned status 1
> tar: Error is not recoverable: exiting now
> make[2]: Entering directory
> `/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7/external/devxlib'
> make[2]: *** No rule to make target `clean'. Stop.
> make[2]: Leaving directory
> `/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7/external/devxlib'
> /bin/sh: line 9: ./configure: No such file or directory
> make[2]: Entering directory
> `/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7/external/devxlib'
> make[2]: *** No rule to make target `all'. Stop.
> make[2]: Leaving directory
> `/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7/external/devxlib'
> make[1]: *** [libcuda_devxlib] Error 2
> make[1]: Leaving directory `/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7/install'
> make: *** [libcuda] Error 2
>
> *Moreover, "which libcuda.so" results in:*
> *
> *
> /opt/nvidia/hpc_sdk/Linux_x86_64/20.11/cuda/11.0/lib64/stubs/libcuda.so
> *
> *
> *Any help will be greatly appreciated.*
> *
> *
> *Best regards,*
> *
> *
> *Mohammad*
> *ShirazU*
>
> On Wed, Dec 16, 2020 at 5:08 PM Pietro Bonfa' <pietro.bonfa at unipr.it
> <mailto:pietro.bonfa at unipr.it>> wrote:
>
> Dear Mohammad,
>
> CUDA may be installed somewhere else, anyway, if you want to skip the
> environment check, you may configure QE with this command
>
> ./configure FC=pgf90 F90=pgf90 CC=pgcc --with-cuda=yes
> --enable-cuda-env-check=no --with-cuda-runtime=SETME --with-cuda-cc=70
> --enable-openmp
>
> Remember to set the cuda runtime according to what is provided by your
> setup.
>
> Hope this helps,
> best,
> Pietro
>
> ---
> Pietro Bonfà
> University of Parma
>
>
> On 12/16/20 9:10 AM, Mohammad Moaddeli wrote:
> > Dear all,
> > I am trying to compile the 6.7 version of the code using PGI 2020.
> > I followed these steps:
> >
> > *1) NVIDIA driver (NVIDIA-Linux-x86_64-450.80.02.rpm) is installed.*
> > *the output of nvidia-smi:*
> >
> > Wed Dec 16 09:07:11 2020
> >
> +-----------------------------------------------------------------------------+
> > | NVIDIA-SMI 450.80.02 Driver Version: 450.80.02 CUDA Version:
> > 11.0 |
> >
> |-------------------------------+----------------------+----------------------+
> > | GPU Name Persistence-M| Bus-Id Disp.A | Volatile
> > Uncorr. ECC |
> > | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util
> > Compute M. |
> > | | |
> > MIG M. |
> >
> |===============================+======================+======================|
> > | 0 TITAN V Off | 00000000:06:00.0 Off |
> > N/A |
> > | 27% 37C P0 32W / 250W | 0MiB / 12066MiB | 0%
> > Default |
> > | | |
> > N/A |
> >
> +-------------------------------+----------------------+----------------------+
> > | 1 TITAN V Off | 00000000:07:00.0 Off |
> > N/A |
> > | 25% 37C P0 35W / 250W | 0MiB / 12066MiB | 0%
> > Default |
> > | | |
> > N/A |
> >
> +-------------------------------+----------------------+----------------------+
> >
> >
> +-----------------------------------------------------------------------------+
> > | Processes:
> > |
> > | GPU GI CI PID Type Process name
> GPU
> > Memory |
> > | ID ID
> Usage
> > |
> >
> |=============================================================================|
> > | No running processes found
> > |
> >
> +-----------------------------------------------------------------------------+
> >
> > *The output of pgaccelinfo:*
> >
> > CUDA Driver Version: 11000
> > NVRM version: NVIDIA UNIX x86_64 Kernel Module
> > 450.80.02 Wed Sep 23 01:13:39 UTC 2020
> >
> > Device Number: 0
> > Device Name: TITAN V
> > Device Revision Number: 7.0
> > Global Memory Size: 12652838912
> > Number of Multiprocessors: 80
> > Concurrent Copy and Execution: Yes
> > Total Constant Memory: 65536
> > Total Shared Memory per Block: 49152
> > Registers per Block: 65536
> > Warp Size: 32
> > Maximum Threads per Block: 1024
> > Maximum Block Dimensions: 1024, 1024, 64
> > Maximum Grid Dimensions: 2147483647 x 65535 x 65535
> > Maximum Memory Pitch: 2147483647B
> > Texture Alignment: 512B
> > Clock Rate: 1455 MHz
> > Execution Timeout: No
> > Integrated Device: No
> > Can Map Host Memory: Yes
> > Compute Mode: default
> > Concurrent Kernels: Yes
> > ECC Enabled: No
> > Memory Clock Rate: 850 MHz
> > Memory Bus Width: 3072 bits
> > L2 Cache Size: 4718592 bytes
> > Max Threads Per SMP: 2048
> > Async Engines: 7
> > Unified Addressing: Yes
> > Managed Memory: Yes
> > Concurrent Managed Memory: Yes
> > Preemption Supported: Yes
> > Cooperative Launch: Yes
> > Multi-Device: Yes
> > Default Target: cc70
> >
> > Device Number: 1
> > Device Name: TITAN V
> > Device Revision Number: 7.0
> > Global Memory Size: 12652838912
> > Number of Multiprocessors: 80
> > Concurrent Copy and Execution: Yes
> > Total Constant Memory: 65536
> > Total Shared Memory per Block: 49152
> > Registers per Block: 65536
> > Warp Size: 32
> > Maximum Threads per Block: 1024
> > Maximum Block Dimensions: 1024, 1024, 64
> > Maximum Grid Dimensions: 2147483647 x 65535 x 65535
> > Maximum Memory Pitch: 2147483647B
> > Texture Alignment: 512B
> > Clock Rate: 1455 MHz
> > Execution Timeout: No
> > Integrated Device: No
> > Can Map Host Memory: Yes
> > Compute Mode: default
> > Concurrent Kernels: Yes
> > ECC Enabled: No
> > Memory Clock Rate: 850 MHz
> > Memory Bus Width: 3072 bits
> > L2 Cache Size: 4718592 bytes
> > Max Threads Per SMP: 2048
> > Async Engines: 7
> > Unified Addressing: Yes
> > Managed Memory: Yes
> > Concurrent Managed Memory: Yes
> > Preemption Supported: Yes
> > Cooperative Launch: Yes
> > Multi-Device: Yes
> > Default Target: cc70
> >
> > *2) PGI compiler is installed:*
> > *yum install nvhpc-20-11-20.11-1.x86_64.rpm
> nvhpc-2020-20.11-1.x86_64.rpm
> > *
> > *PATHs that are set in ~/.bashrc file:
> > *
> > *
> > *
> > export
> PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/20.11/cuda/11.1/bin:$PATH
> > export
> PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/20.11/cuda/11.1/include:$PATH
> > export
> >
> LD_LIBRARY_PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/20.11/cuda/11.1/lib64:$LD_LIBRARY_PATH
> > export
> >
> LD_LIBRARY_PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/20.11/cuda/11.1/extras/CUPTI/lib64:$LD_LIBRARY_PATH
> > export
> >
> LD_LIBRARY_PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/20.11/cuda/11.1/lib64/stubs:$LD_LIBRARY_PATH
> > NVARCH=`uname -s`_`uname -m`; export NVARCH
> > NVCOMPILERS=/opt/nvidia/hpc_sdk; export NVCOMPILERS
> > MANPATH=$MANPATH:$NVCOMPILERS/$NVARCH/20.11/compilers/man; export
> MANPATH
> > PATH=$NVCOMPILERS/$NVARCH/20.11/compilers/bin:$PATH; export PATH
> > PATH=$NVCOMPILERS/$NVARCH/20.11/compilers/include:$PATH; export PATH
> > LD_LIBRARY_PATH=$NVCOMPILERS/$NVARCH/20.11/compilers/lib:$PATH;
> export
> > LD_LIBRARY_PATH
> > export
> PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/20.11/comm_libs/mpi/bin:$PATH
> > export
> >
> PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/20.11/comm_libs/mpi/include:$PATH
> > export
> >
> LD_LIBRARY_PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/20.11/comm_libs/mpi/lib:$LD_LIBRARY_PATH
> > export
> >
> LD_LIBRARY_PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/20.11/math_libs/11.1/lib64:$LD_LIBRARY_PATH
> > export
> >
> LD_LIBRARY_PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/20.11/math_libs/11.1/lib64/stubs:$LD_LIBRARY_PATH*
> > *
> > *
> > *
> > *3) compiling the code using:*
> > *./configure FC=pgf90 F90=pgf90 F77=pgf90 CC=pgcc MPIF90=mpif90
> > --with-cuda=/opt/nvidia/hpc_sdk/Linux_x86_64/20.11/cuda
> > --with-cuda-runtime=11.1 --with-cuda-cc=70 --enable-openmp
> > --with-scalapack=no*
> >
> > checking build system type... x86_64-pc-linux-gnu
> > checking ARCH... x86_64
> > checking setting AR... ... ar
> > checking setting ARFLAGS... ... ruv
> > checking whether the Fortran compiler works... yes
> > checking for Fortran compiler default output file name... a.out
> > checking for suffix of executables...
> > checking whether we are cross compiling... no
> > checking for suffix of object files... o
> > checking whether we are using the GNU Fortran compiler... no
> > checking whether pgf90 accepts -g... yes
> > configure: WARNING: F90 value is set to be consistent with value
> of MPIF90
> > checking for mpif90... mpif90
> > checking whether we are using the GNU Fortran compiler... no
> > checking whether mpif90 accepts -g... yes
> > checking version of mpif90... nvfortran 20.11-0
> > checking for Fortran flag to compile .f90 files... none
> > setting F90... nvfortran
> > setting MPIF90... mpif90
> > checking whether we are using the GNU C compiler... yes
> > checking whether pgcc accepts -g... yes
> > checking for pgcc option to accept ISO C89... none needed
> > setting CC... pgcc
> > setting CFLAGS... -fast -Mpreprocess
> > using F90... nvfortran
> > setting FFLAGS... -O1
> > setting F90FLAGS... $(FFLAGS)
> > setting FFLAGS_NOOPT... -O0
> > setting CPP... cpp
> > setting CPPFLAGS... -P -traditional -Uvector
> > setting LD... mpif90
> > setting LDFLAGS...
> > checking for Fortran flag to compile .f90 files... (cached) none
> > checking whether Fortran compiler accepts -Mcuda=cuda11.1... yes
> > checking for nvcc...
> > /opt/nvidia/hpc_sdk/Linux_x86_64/20.11/compilers/bin/nvcc
> > checking whether nvcc works... no
> > configure: WARNING: CUDA compiler has problems.
> > checking for cuInit in -lcuda... no
> > configure: error: in `/codes/qe_6.7_GPU/q-e-gpu-qe-gpu-6.7':
> > configure: error: Couldn't find libcuda
> > See `config.log' for more details*
> > *
> > *
> > *
> > *
> > *
> > *
> > *
> > *Any Help will be greatly appreciated.*
> > *
> > *
> > *
> > *
> > *
> > *
> > *P.S.*
> > *When I run nvcc in terminal, the following error appears:*
> > $ which nvcc
> > /opt/nvidia/hpc_sdk/Linux_x86_64/20.11/compilers/bin/nvcc
> > **
> > $ nvcc
> > nvcc-Error-CUDA version 10.2 was not installed with this HPC SDK:
> > /opt/nvidia/hpc_sdk/Linux_x86_64/20.11/cuda/10.2/bin
> >
> >
> > *Best,*
> > *Mohammad Moaddeli*
> > *Shiraz University*
> > **
> > **
> >
> > _______________________________________________
> > Quantum ESPRESSO is supported by MaX (www.max-centre.eu
> <http://www.max-centre.eu/>)
> > users mailing list users at lists.quantum-espresso.org
> <mailto:users at lists.quantum-espresso.org>
> > https://lists.quantum-espresso.org/mailman/listinfo/users
> <https://lists.quantum-espresso.org/mailman/listinfo/users>
> >
> _______________________________________________
> Quantum ESPRESSO is supported by MaX (www.max-centre.eu
> <http://www.max-centre.eu/>)
> users mailing list users at lists.quantum-espresso.org
> <mailto:users at lists.quantum-espresso.org>
> https://lists.quantum-espresso.org/mailman/listinfo/users
> <https://lists.quantum-espresso.org/mailman/listinfo/users>
>
>
> _______________________________________________
> Quantum ESPRESSO is supported by MaX (www.max-centre.eu)
> users mailing list users at lists.quantum-espresso.org
> https://lists.quantum-espresso.org/mailman/listinfo/users
>
More information about the users
mailing list