[QE-users] [QE 7.0/GPU] Run built with CMake fails with cuFFT error

[Louis Stuber]

Hi Robert,

I think it's an issue with PGI in NVHPC 22.3.  Please use NVHPC 21.9 instead (PGI compiler).

Normally the configure script should catch it and prevent using NVHPC 22.3, unfortunately the check has been introduced only after QE 7.0.

Best,
Louis

From: users <users-bounces at lists.quantum-espresso.org> On Behalf Of Robert MIJAKOVIC
Sent: Monday, May 23, 2022 5:23 PM
To: users at lists.quantum-espresso.org
Subject: [QE-users] [QE 7.0/GPU] Run built with CMake fails with cuFFT error

Some people who received this message don't often get email from robert.mijakovic at lxp.lu<mailto:robert.mijakovic at lxp.lu>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
External email: Use caution opening links or attachments

# Summary
QE 7.0/GPU compiled with CMake fails on our system in "routine fft_scalar_cuFFT: cft_1z_gpu (8)".

# Version
qe-7.0-ReleasePack.tgz

# Environment
## Hardware
1. 2xAMD EPYC 7452
2. 4xNVIDIA A100
3. 512 GB RAM
## Software
1. OS: Rocky Linux release 8.5 (Green Obsidian)
2. NVHPC 22.3
3. OpenMPI 4.1.3 built with NVHPC 22.3
4. CUDA 11.3.1 with Driver 470.82.01
5. libxc 5.1.5
6. CMake 3.20.1
7. M4 1.4.19

# Steps to reproduce
## Configured with:
`-DQE_ENABLE_CUDA=1 -DQE_FFTW_VENDOR=Internal -DQE_ENABLE_LIBXC=1 -DQE_ENABLE_OPENMP=1 `
## Prebuild options
`cp $EBROOTLIBXC/include/*.mod Modules/mod/qe_modules && export FPP='nvfortran -Mpreprocess -E' && export CPP='cpp -E' && export FCPP='cpp -E' && `
## make options
`make all epw`
## Execute
srun

## Input files
QEF AUSURF112 benchmark

# Observed behaviour
When example is started it fails with:
```
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Error in routine fft_scalar_cuFFT: cft_1z_gpu (8):
cufftPlanMany failed
```

# Questions
1. What do I do wrong?
2. Why is there no option to set FFTW_VENDOR to cuFFT?
3. Why it got linked against cuFFT if FFTW_VENDOR is set to Internal?

Dr. rer. nat. Robert Mijaković | HPC System Software Architect

LuxProvide
3, Op der Poukewiss | L-7795 Bissen
Grand-Duchy of Luxembourg
M (+352) 691 396 474
robert.mijakovic at lxp.lu<mailto:robert.mijakovic at lxp.lu> | www.luxprovide.lu<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.luxprovide.lu%2F&data=05%7C01%7Clstuber%40nvidia.com%7C92950777c584476ead5508da3cd03d33%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637889163050914322%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ofAyQvxQ6OinQiA5HgsBEO6nQfPosHKHdTnXt17ld4Y%3D&reserved=0>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20220523/6ba82186/attachment.html>