[QE-users] [QE 7.0/GPU] Run built with CMake fails with cuFFT error

[Paolo Giannozzi]

Actually NVHPC 22.3 seems to work so only versions > 21.9 and < 22.3 are 
not allowed

Paolo

On 23/05/2022 18:54, Louis Stuber via users wrote:
> Hi Robert,
> 
> I think it’s an issue with PGI in NVHPC 22.3.  Please use NVHPC 21.9 
> instead (PGI compiler).
> 
> Normally the configure script should catch it and prevent using NVHPC 
> 22.3, unfortunately the check has been introduced only after QE 7.0.
> 
> Best,
> 
> Louis
> 
> *From:* users <users-bounces at lists.quantum-espresso.org> *On Behalf Of 
> *Robert MIJAKOVIC
> *Sent:* Monday, May 23, 2022 5:23 PM
> *To:* users at lists.quantum-espresso.org
> *Subject:* [QE-users] [QE 7.0/GPU] Run built with CMake fails with cuFFT 
> error
> 
> 	
> 
> Some people who received this message don't often get email from 
> robert.mijakovic at lxp.lu <mailto:robert.mijakovic at lxp.lu>. Learn why this 
> is important <https://aka.ms/LearnAboutSenderIdentification>
> 
> 	
> 
> *External email: Use caution opening links or attachments*
> 
> # Summary
> 
> QE 7.0/GPU compiled with CMake fails on our system in "routine 
> fft_scalar_cuFFT: cft_1z_gpu (8)".
> 
> # Version
> 
> qe-7.0-ReleasePack.tgz
> 
> # Environment
> 
> ## Hardware
> 
> 1. 2xAMD EPYC 7452
> 
> 2. 4xNVIDIA A100
> 
> 3. 512 GB RAM
> 
> ## Software
> 
> 1. OS: Rocky Linux release 8.5 (Green Obsidian)
> 
> 2. NVHPC 22.3
> 
> 3. OpenMPI 4.1.3 built with NVHPC 22.3
> 
> 4. CUDA 11.3.1 with Driver 470.82.01
> 
> 5. libxc 5.1.5
> 
> 6. CMake 3.20.1
> 
> 7. M4 1.4.19
> 
> # Steps to reproduce
> 
> ## Configured with:
> 
> `-DQE_ENABLE_CUDA=1 -DQE_FFTW_VENDOR=Internal -DQE_ENABLE_LIBXC=1 
> -DQE_ENABLE_OPENMP=1 `
> 
> ## Prebuild options
> 
> `cp $EBROOTLIBXC/include/*.mod Modules/mod/qe_modules && export 
> FPP='nvfortran -Mpreprocess -E' && export CPP='cpp -E' && export 
> FCPP='cpp -E' && `
> 
> ## make options
> 
> `make all epw`
> 
> ## Execute
> 
> srun
> 
> ## Input files
> 
> QEF AUSURF112 benchmark
> 
> # Observed behaviour
> 
> When example is started it fails with:
> 
> ```
> 
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> 
> Error in routine fft_scalar_cuFFT: cft_1z_gpu (8):
> 
> cufftPlanMany failed
> 
> ```
> 
> # Questions
> 
> 1. What do I do wrong?
> 
> 2. Why is there no option to set FFTW_VENDOR to cuFFT?
> 
> 3. Why it got linked against cuFFT if FFTW_VENDOR is set to Internal?
> 
> Dr. rer. nat. Robert Mijaković | HPC System Software Architect
> 
> 
> *Lux**Provide*
> 3, Op der Poukewiss | L-7795 Bissen
> 
> Grand-Duchy of Luxembourg
> M (+352) 691 396 474
> 
> robert.mijakovic at lxp.lu 
> <mailto:robert.mijakovic at lxp.lu> |www.luxprovide.lu 
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.luxprovide.lu%2F&data=05%7C01%7Clstuber%40nvidia.com%7C92950777c584476ead5508da3cd03d33%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637889163050914322%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ofAyQvxQ6OinQiA5HgsBEO6nQfPosHKHdTnXt17ld4Y%3D&reserved=0>
> 
> 
> _______________________________________________
> The Quantum ESPRESSO community stands by the Ukrainian
> people and expresses its concerns about the devastating
> effects that the Russian military offensive has on their
> country and on the free and peaceful scientific, cultural,
> and economic cooperation amongst peoples
> _______________________________________________
> Quantum ESPRESSO is supported by MaX (www.max-centre.eu)
> users mailing list users at lists.quantum-espresso.org
> https://lists.quantum-espresso.org/mailman/listinfo/users

-- 
Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,
Univ. Udine, via delle Scienze 208, 33100 Udine, Italy
Phone +39-0432-558216, fax +39-0432-558222