[QE-users] QE7.2 with oneAPI IMPI and MKL+SCALAPACK

[pdelugas at sissa.it]

Hello Sean

I checked on galileo-100 on CINECA. Scalapack with intelmpi works fine. 
except for the -DCMAME_BULID_TYPE spec, that I overlooked, I used cmake 
with your same options.

both cmake and autoconf build systems are fully supported and 
functional, so use the one that suits you better without worries.

I have maybe used a slightly newer version of the tool chain (2021.5)

may I ask how many nodes and ranks have you used ?

regards

Pietro

On 03/05/23 18:14, Anderson, Sean wrote:
> Hi QE Users,
>
> I have been having a lot of trouble with QE 7.2 using IMPI and MKL 
> SCALAPACK from the oneAPI (2021.2). This is a toolchain that I have 
> used many times before for other codes (ABINIT, VASP, etc.) with no 
> issues, so I am confident that the toolchain is working and good.
>
> An example build process is as follows:
>
> module load compilers/gcc/10.2.0
> source /opt/intel/oneapi/setvars.sh
>
> cd qe-7.2 && mkdir -p build && cd build
>
> cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
>       -DCMAKE_C_COMPILER=mpiicc \
>       -DCMAKE_Fortran_COMPILER=mpiifort \
>       -DQE_ENABLE_MPI=on \
>       -DQE_ENABLE_SCALAPACK=on \
>       -DENABLE_SCALAPACK_MPI=on \
>       ..
>
> make -j8 all
>
> This procedure will generate the expected binaries that pass all of 
> the "system--pw" and "system--cp" tests. That said, all production 
> calculations hang at the Davidson diagonalization step. This does NOT 
> occur if I disable SCALAPACK, so it would seem that the problem stems 
> from there. I have confirmed that the generated linker lines are 
> correct, that everything is linked after building, and that all 
> dependencies are present at run-time.
>
> Switching over to OMPI (4.1.1) + MKL SCALPACK works fine and does not 
> hang at the aforementioned diagonalization.
>
> Two questions:
>
> 1. Is CMake the preferred method for building? I have had far more 
> success with CMake over the traditional 'make/make.inc'.
> 2. Has anyone else encountered similar issues with the aforementioned 
> oneAPI toolchain? Any help would be greatly appreciated.
>
> Thanks!
>
> ------------------
> Sean Anderson, PhD
> The HPC Team
> Information Systems, WFU
> anderss at wfu.edu
> c 336.486.9338; w 336.758.6462
> Book an Appointment <https://calendar.app.google/iWacKvukvAUbXLSf9>
>
> _______________________________________________
> The Quantum ESPRESSO community stands by the Ukrainian
> people and expresses its concerns about the devastating
> effects that the Russian military offensive has on their
> country and on the free and peaceful scientific, cultural,
> and economic cooperation amongst peoples
> _______________________________________________
> Quantum ESPRESSO is supported by MaX (www.max-centre.eu)
> users mailing listusers at lists.quantum-espresso.org
> https://lists.quantum-espresso.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20230505/2c3deeef/attachment.html>