[QE-users] QE7.2 with oneAPI IMPI and MKL+SCALAPACK

[Anderson, Sean]

Hi QE Users,

I have been having a lot of trouble with QE 7.2 using IMPI and MKL
SCALAPACK from the oneAPI (2021.2). This is a toolchain that I have used
many times before for other codes (ABINIT, VASP, etc.) with no issues, so I
am confident that the toolchain is working and good.

An example build process is as follows:

module load compilers/gcc/10.2.0
source /opt/intel/oneapi/setvars.sh

cd qe-7.2 && mkdir -p build && cd build

cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
      -DCMAKE_C_COMPILER=mpiicc \
      -DCMAKE_Fortran_COMPILER=mpiifort \
      -DQE_ENABLE_MPI=on \
      -DQE_ENABLE_SCALAPACK=on \
      -DENABLE_SCALAPACK_MPI=on \
      ..

make -j8 all

This procedure will generate the expected binaries that pass all of the
"system--pw" and "system--cp" tests. That said, all production calculations
hang at the Davidson diagonalization step. This does NOT occur if I disable
SCALAPACK, so it would seem that the problem stems from there. I have
confirmed that the generated linker lines are correct, that everything is
linked after building, and that all dependencies are present at run-time.

Switching over to OMPI (4.1.1) + MKL SCALPACK works fine and does not hang
at the aforementioned diagonalization.

Two questions:

1. Is CMake the preferred method for building? I have had far more success
with CMake over the traditional 'make/make.inc'.
2. Has anyone else encountered similar issues with the aforementioned
oneAPI toolchain? Any help would be greatly appreciated.

Thanks!

------------------
Sean Anderson, PhD
The HPC Team
Information Systems, WFU
anderss at wfu.edu
c 336.486.9338; w 336.758.6462
Book an Appointment <https://calendar.app.google/iWacKvukvAUbXLSf9>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20230503/8bf359d9/attachment.html>