[QE-users] [QE 7.0/GPU] Build configured with Autotools fails

Paolo Giannozzi paolo.giannozzi at uniud.it
Mon May 23 21:45:47 CEST 2022 

Here some explanations:
https://www.mail-archive.com/users@lists.quantum-espresso.org/msg41494.html
https://www.mail-archive.com/users@lists.quantum-espresso.org/msg40215.html
The path to the fftw3.f03 file, needed by fftw3, must be specified in 
INCLUDE_FFTW or similar variables. The latest development version (soon 
to be released) has somewhat better FFTW3 auto-detection

Paolo

On 23/05/2022 17:55, Robert MIJAKOVIC wrote:
> # Summary
> 
> QE 7.0/GPU compilation with Autotools fails with "Can't find include 
> file fftw3.f03 (fft_scalar.FFTW3.f90: 40)"
> 
> # Version
> 
> qe-7.0-ReleasePack.tgz
> 
> # Environment
> 
> ## Hardware
> 
> 1. 2xAMD EPYC 7452
> 
> 2. 4xNVIDIA A100
> 
> 3. 512 GB RAM
> 
> ## Software
> 
> 1. OS: Rocky Linux release 8.5 (Green Obsidian)
> 
> 2. NVHPC 22.3
> 
> 3. OpenMPI 4.1.3 built with NVHPC 22.3
> 
> 4. CUDA 11.3.1 with Driver 470.82.01
> 
> 5. libxc 5.1.5
> 
> 6. Autotools 20210128
> 
> # Steps to reproduce
> 
> ## Configured with:
> 
> `./configure --prefix=$HOME/install/qe-7.0 --enable-openmp 
> --with-scalapack=no --enable-cuda-env-check=no 
> --with-cuda=$EBROOTNVHPC/Linux_x86_64/22.3/cuda/11.0 
> --with-cuda-runtime=11.0 --with-cuda-cc=80`
> 
> ## Prebuild options
> 
> `cp $EBROOTLIBXC/include/*.mod Modules/mod/qe_modules && export 
> FPP='nvfortran -Mpreprocess -E' && export CPP='cpp -E' && export 
> FCPP='cpp -E' && `
> 
> ## make options
> 
> `make all epw`
> 
> # Observed behavior
> 
> Compilation fails with:
> 
> ```
> 
> make[1]: Entering directory 
> '/mnt/tier2/users/apps/scratchpad/qe-7.0/FFTXlib'
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c fft_param.f90
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c fft_support.f90
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c stick_base.f90
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c fft_types.f90
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c scatter_mod.f90
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c 
> fft_scatter_2d.f90
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c fft_buffers.f90
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c 
> fft_scatter_gpu.f90
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c 
> fft_scatter_2d_gpu.f90
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c fft_ggen.f90
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c 
> fft_scalar.DFTI.f90
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c 
> fft_scalar.ESSL.f90
> 
> NVFORTRAN-W-0006-Input file empty (fft_scalar.ESSL.f90)
> 
> NVFORTRAN/x86-64 Linux 22.3-0: compilation completed with warnings
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c 
> fftw_interfaces.f90
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c 
> fft_scalar.FFTW.f90
> 
> NVFORTRAN-W-0006-Input file empty (fft_scalar.FFTW.f90)
> 
> NVFORTRAN/x86-64 Linux 22.3-0: compilation completed with warnings
> 
> mpif90 -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp -D__PGI 
> -D__CUDA -D__USE_CUSOLVER -D__FFTW3 -D__MPI  -cuda -gpu=cc80,cuda11.0 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/include -acc 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//external/devxlib/src 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//include 
> -I/mnt/tier2/users/apps/scratchpad/qe-7.0//FoX/finclude   -c 
> fft_scalar.FFTW3.f90
> 
> NVFORTRAN-F-0906-Can't find include file fftw3.f03 
> (fft_scalar.FFTW3.f90: 40)
> 
> NVFORTRAN/x86-64 Linux 22.3-0: compilation aborted
> 
> make[1]: *** [../make.inc:16: fft_scalar.FFTW3.o] Error 2
> 
> make[1]: Leaving directory '/mnt/tier2/users/apps/scratchpad/qe-7.0/FFTXlib'
> 
> make: *** [Makefile:196: libfft] Error 1
> 
> ```
> 
> # Questions
> 
> 1. What do I do wrong?
> 
> 2. Why configure didn't fail if FFTW is mandatory?
> 
> 3. Is it possible to configure the build system to use cuFFTinstead?
> 
> Dr. rer. nat. Robert Mijaković | HPC System Software Architect
> 
> 
> *Lux**Provide*
> 3, Op der Poukewiss | L-7795 Bissen
> 
> Grand-Duchy of Luxembourg
> M (+352) 691 396 474
> 
> robert.mijakovic at lxp.lu 
> <mailto:robert.mijakovic at lxp.lu> |www.luxprovide.lu 
> <http://www.luxprovide.lu/>
> 
> 
> _______________________________________________
> The Quantum ESPRESSO community stands by the Ukrainian
> people and expresses its concerns about the devastating
> effects that the Russian military offensive has on their
> country and on the free and peaceful scientific, cultural,
> and economic cooperation amongst peoples
> _______________________________________________
> Quantum ESPRESSO is supported by MaX (www.max-centre.eu)
> users mailing list users at lists.quantum-espresso.org
> https://lists.quantum-espresso.org/mailman/listinfo/users

-- 
Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,
Univ. Udine, via delle Scienze 208, 33100 Udine, Italy
Phone +39-0432-558216, fax +39-0432-558222

More information about the users mailing list