[Pw_forum] QE-GPU: compilation error with Espresso 5.1

Filippo Spiga spiga.filippo at gmail.com
Sat Sep 6 10:43:12 CEST 2014 

Dear Mauro,

I understand you want to use OpenBLAS based on how you tried to run the configure the first time. After configure, before make, change both make.sys and GPU/install/make_phiGEMM.inc  to point to Open BLAS

What version of GCC are you using? My suggestion is 4.4 or 4.6. You can also download Intel MKL runtime library from Intel website and link them with GCC to QE and QE-GPU. Performance will be better than internal BLAS/LAPACK and the configure pick the libraries up automatically (not like OpenBLAS)

HTH
F


> On Sep 5, 2014, at 2:59 PM, Mauro Sgroi <maurofrancesco.sgroi at gmail.com> wrote:
> 
> Dear Filippo,
> I created a brand new directory of QE just for the GPU compilation and I'm trying now to use the configure command indicated in the README.
> The error now is the following, it seems that phiGEMM doesn't find libgfortran. 
> 
> make[3]: Entering directory `/usr/local/src/espresso-5.1_gpu/phiGEMM/src'
> /usr/local/cuda-6.0/bin/nvcc -ccbin gcc -O3 --compiler-options '-c -fPIC -fopenmp' -D__PHIGEMM_WEAK_INTERFACES -D__PHIGEMM_SELFTUNE -D__PHIGEMM_ENABLE_SPECIALK  -I/usr/local/src/espresso-5.1_gpu/GPU/..//phiGEMM/include -I/usr/local/cuda-6.0/include -I../include/ -c phigemm_auxiliary.c -o phigemm_auxiliary.o
> nvcc warning : The 'compute_10' and 'sm_10' architectures are deprecated, and may be removed in a future release.
> /usr/local/cuda-6.0/bin/nvcc -ccbin gcc -O3 --compiler-options '-c -fPIC -fopenmp' -D__PHIGEMM_WEAK_INTERFACES -D__PHIGEMM_SELFTUNE -D__PHIGEMM_ENABLE_SPECIALK  -I/usr/local/src/espresso-5.1_gpu/GPU/..//phiGEMM/include -I/usr/local/cuda-6.0/include -I../include/ -c phigemm_env.c -o phigemm_env.o
> nvcc warning : The 'compute_10' and 'sm_10' architectures are deprecated, and may be removed in a future release.
> /usr/local/cuda-6.0/bin/nvcc -ccbin gcc -O3 --compiler-options '-c -fPIC -fopenmp' -D__PHIGEMM_WEAK_INTERFACES -D__PHIGEMM_SELFTUNE -D__PHIGEMM_ENABLE_SPECIALK  -I/usr/local/src/espresso-5.1_gpu/GPU/..//phiGEMM/include -I/usr/local/cuda-6.0/include -I../include/ -c phigemm_dgemm.c -o phigemm_dgemm.o
> nvcc warning : The 'compute_10' and 'sm_10' architectures are deprecated, and may be removed in a future release.
> /usr/local/cuda-6.0/bin/nvcc -ccbin gcc -O3 --compiler-options '-c -fPIC -fopenmp' -D__PHIGEMM_WEAK_INTERFACES -D__PHIGEMM_SELFTUNE -D__PHIGEMM_ENABLE_SPECIALK  -I/usr/local/src/espresso-5.1_gpu/GPU/..//phiGEMM/include -I/usr/local/cuda-6.0/include -I../include/ -c phigemm_zgemm.c -o phigemm_zgemm.o
> nvcc warning : The 'compute_10' and 'sm_10' architectures are deprecated, and may be removed in a future release.
> /usr/local/cuda-6.0/bin/nvcc -ccbin gcc -O3 --compiler-options '-c -fPIC -fopenmp' -D__PHIGEMM_WEAK_INTERFACES -D__PHIGEMM_SELFTUNE -D__PHIGEMM_ENABLE_SPECIALK  -I/usr/local/src/espresso-5.1_gpu/GPU/..//phiGEMM/include -I/usr/local/cuda-6.0/include -I../include/ -c phigemm_dgemm_specialK.c -o phigemm_dgemm_specialK.o
> nvcc warning : The 'compute_10' and 'sm_10' architectures are deprecated, and may be removed in a future release.
> /usr/local/cuda-6.0/bin/nvcc -ccbin gcc -O3 --compiler-options '-c -fPIC -fopenmp' -D__PHIGEMM_WEAK_INTERFACES -D__PHIGEMM_SELFTUNE -D__PHIGEMM_ENABLE_SPECIALK  -I/usr/local/src/espresso-5.1_gpu/GPU/..//phiGEMM/include -I/usr/local/cuda-6.0/include -I../include/ -c phigemm_zgemm_specialK.c -o phigemm_zgemm_specialK.o
> nvcc warning : The 'compute_10' and 'sm_10' architectures are deprecated, and may be removed in a future release.
> /usr/local/cuda-6.0/bin/nvcc -ccbin gcc -O3 --compiler-options '-c -fPIC -fopenmp' -D__PHIGEMM_WEAK_INTERFACES -D__PHIGEMM_SELFTUNE -D__PHIGEMM_ENABLE_SPECIALK  -I/usr/local/src/espresso-5.1_gpu/GPU/..//phiGEMM/include -I/usr/local/cuda-6.0/include -I../include/ -c phigemm_cgemm.c -o phigemm_cgemm.o
> nvcc warning : The 'compute_10' and 'sm_10' architectures are deprecated, and may be removed in a future release.
> /usr/local/cuda-6.0/bin/nvcc -ccbin gcc -O3 --compiler-options '-c -fPIC -fopenmp' -D__PHIGEMM_WEAK_INTERFACES -D__PHIGEMM_SELFTUNE -D__PHIGEMM_ENABLE_SPECIALK  -I/usr/local/src/espresso-5.1_gpu/GPU/..//phiGEMM/include -I/usr/local/cuda-6.0/include -I../include/ -c phigemm_sgemm.c -o phigemm_sgemm.o
> nvcc warning : The 'compute_10' and 'sm_10' architectures are deprecated, and may be removed in a future release.
> mkdir -p ../bin ../lib
> ar ruv libphigemm.a phigemm_auxiliary.o phigemm_env.o phigemm_dgemm.o phigemm_zgemm.o phigemm_dgemm_specialK.o phigemm_zgemm_specialK.o phigemm_cgemm.o phigemm_sgemm.o
> ar: creating libphigemm.a
> a - phigemm_auxiliary.o
> a - phigemm_env.o
> a - phigemm_dgemm.o
> a - phigemm_zgemm.o
> a - phigemm_dgemm_specialK.o
> a - phigemm_zgemm_specialK.o
> a - phigemm_cgemm.o
> a - phigemm_sgemm.o
> mv libphigemm.a ../lib/.
> mkdir -p ../bin ../lib
> cc -O3 -fopenmp -fPIC -z muldefs  -o libphigemm.so.2.0 phigemm_auxiliary.o phigemm_env.o phigemm_dgemm.o phigemm_zgemm.o phigemm_dgemm_specialK.o phigemm_zgemm_specialK.o phigemm_cgemm.o phigemm_sgemm.o -L/usr/local/cuda-6.0/lib64 -lcublas  -lcufft -lcudart /usr/local/src/espresso-5.1_gpu/GPU/..//phiGEMM/lib/libphigemm.a  /usr/local/src/espresso-5.1_gpu/GPU/..//BLAS/blas.a -L/usr/local/cuda-6.0/lib64 -lcublas  -lcufft -lcudart
> /usr/lib/gcc/x86_64-redhat-linux/4.4.6/../../../../lib64/crt1.o: In function `_start':
> (.text+0x20): undefined reference to `main'
> /usr/local/src/espresso-5.1_gpu/GPU/..//BLAS/blas.a(xerbla.o): In function `xerbla_':
> xerbla.f:(.text+0x51): undefined reference to `_gfortran_st_write'
> xerbla.f:(.text+0x61): undefined reference to `_gfortran_transfer_character'
> xerbla.f:(.text+0x71): undefined reference to `_gfortran_transfer_integer'
> xerbla.f:(.text+0x79): undefined reference to `_gfortran_st_write_done'
> xerbla.f:(.text+0x83): undefined reference to `_gfortran_stop_numeric'
> collect2: ld returned 1 exit status
> make[3]: *** [shared] Error 1
> make[3]: Leaving directory `/usr/local/src/espresso-5.1_gpu/phiGEMM/src'
> make[2]: *** [phigemm] Error 2
> make[2]: Leaving directory `/usr/local/src/espresso-5.1_gpu/phiGEMM'
> make[1]: *** [libphiGEMM] Error 2
> make[1]: Leaving directory `/usr/local/src/espresso-5.1_gpu/GPU/install'
> 
> 
> Moreover I have some more questions about the integration of external BLAS, LAPACK and FFTW libraries. 
> Is it useful in terms of speed to insert manually openblas and fftw3 in the make.sys? 
> Moreover, having a single PC with a 8-cores processor and a single GPU, it does make sense to compile the parallel version? Or it's better to use the seria/OMP version? In this last case, how many threads should I use?
> 
> Sorry for the multiple (probably silly) questions..
> Best regards,
> Mauro Sgroi
> Centro Ricerche FIAT
> Italia
> 
> 
> 
> 
> 2014-09-05 9:47 GMT+02:00 Filippo Spiga <spiga.filippo at gmail.com>:
> Dear Mauro,
> 
> After configure, what is the text after DFLAGS = inside make.sys?
> 
> I suspect two things:
> - it looks like there are dirty object files around, please  "make -f Makefile.gpu distclean" first
> - by override FFT_LIBS, BLAS_LIBS and LAPACK_LIBS some of the internal checkings in the configure do not work properly
> 
> My strong suggestion is to edit make.sys after you run the configure, overriding FFT_LIBS, BLAS_LIBS and LAPACK_LIBS only if it is strictly necessary. In your particular case, by doing so you miss phiGEMM and MAGMA and other libraries to link to the final executable. I am improving over time the configure but it will take time to percolate these changes to the main distribution and into QE-GPU. I can add OpenBLAS support to the configure natively, it is not a process that take one day.
> 
> Speaking about CUDA 6.5, I has trouble running MAGMA with the latest CUDA toolkit recently. So my suggestion is to use a more stable CUDA 6.0 or temporary disable MAGMA. It is worh try but if the application has problems this is the first thing to remove.
> 
> HTH
> F
> 
> 
> On Sep 4, 2014, at 3:37 PM, Mauro Sgroi <maurofrancesco.sgroi at gmail.com> wrote:
> > Dear Developers.
> > I'm trying to install the GPU version of Quantum Espresso 5.1.
> > I'm using RedHat EL 6.3 with the gcc 4.4.6 compiler and Cuda Toolkit 6.5.
> > My configure command is as follows:
> >
> > configure FFT_LIBS='-L/usr/local/lib/ -lfftw3' LAPACK_LIBS='-L/usr/local/lib -lopenblas' --disable-parallel --enable-openmp --enable-cuda --with-gpu-arch=sm_30 --with-cuda-dir=/usr/local/cuda --with-magma --with-phigemm BLAS_LIBS='-L/usr/local/lib -lopenblas'
> >
> > and I get the following error:
> >
> > make[1]: Entering directory `/usr/local/src/espresso-5.1_GPU/GPU/Modules'
> > ( if test -x ../install/update_gpu_version ; then \
> >         ../install/update_gpu_version; \
> >         else if test ! -f ../include/gpu-version.h ; then \
> >         cat ../include/gpu-version.h.in > ../include/gpu-version.h ; fi ; fi )
> > nvcc -ccbin=/usr/bin  -I../../include -I../include -D__GFORTRAN -D__STD_F95 -D__FFTW -D__OPENMP    -c cuda_init.cu
> > cuda_init.cu(361): error: identifier "ngpus_per_process" is undefined
> >
> > cuda_init.cu(363): error: identifier "qe_gpu_bonded" is undefined
> >
> > cuda_init.cu(376): error: identifier "qe_gpu_kernel_launch" is undefined
> >
> > cuda_init.cu(376): error: identifier "update_gpu_kernel_specs" is undefined
> >
> > cuda_init.cu(413): warning: variable "lRankThisNode" was declared but never referenced
> >
> > cuda_init.cu(413): warning: variable "lSizeThisNode" was declared but never referenced
> >
> > 4 errors detected in the compilation of "/tmp/tmpxft_00004217_00000000-6_cuda_init.cpp1.ii".
> > make[1]: *** [cuda_init.o] Error 2
> > make[1]: Leaving directory `/usr/local/src/espresso-5.1_GPU/GPU/Modules'
> > make: *** [gpu-mods] Error 2
> >
> >
> > Could you help me with this problem?
> > Thanks a lot in advance and best regards,
> > Mauro Sgroi.
> > _______________________________________________
> > Pw_forum mailing list
> > Pw_forum at pwscf.org
> > http://pwscf.org/mailman/listinfo/pw_forum
> 
> --
> Mr. Filippo SPIGA, M.Sc.
> http://filippospiga.info ~ skype: filippo.spiga
> 
> «Nobody will drive us out of Cantor's paradise.» ~ David Hilbert
> 
> *****
> Disclaimer: "Please note this message and any attachments are CONFIDENTIAL and may be privileged or otherwise protected from disclosure. The contents are not to be disclosed to anyone other than the addressee. Unauthorized recipients are requested to preserve this confidentiality and to advise the sender immediately of any error in transmission."
> 
> 
> 
> _______________________________________________
> Pw_forum mailing list
> Pw_forum at pwscf.org
> http://pwscf.org/mailman/listinfo/pw_forum
> 
> _______________________________________________
> Pw_forum mailing list
> Pw_forum at pwscf.org
> http://pwscf.org/mailman/listinfo/pw_forum

--
Mr. Filippo SPIGA, M.Sc.
http://filippospiga.info ~ skype: filippo.spiga

«Nobody will drive us out of Cantor's paradise.» ~ David Hilbert

*****
Disclaimer: "Please note this message and any attachments are CONFIDENTIAL and may be privileged or otherwise protected from disclosure. The contents are not to be disclosed to anyone other than the addressee. Unauthorized recipients are requested to preserve this confidentiality and to advise the sender immediately of any error in transmission."

More information about the users mailing list