[QE-users] qe-gpu 7.0 - compilation issue (MKL not detected)

Tue Mar 8 17:18:15 CET 2022

Thank you all for so many hints!

Paolo’s suggestion seems to work, but besides BLAS_LIBS it is necessary to set LAPACK_LIBS as well, the internal $(TOPDI$)…. lapack
lib is selected otherwise.

Now things seem to work (with exception of unresolved do to some link mismatch of Intel MKL and PGI OpenMPI, that I can hopefully solve by myself).

The remaining issue is that if I try to set up FCFLAGS, again they are checked against gfortran, so the configure fails if those flags are not recognised by gfortran.
In other words: with NO FCFLAGS set up, nvfortran is selected correctly. But if I set up FCFLAGS, then the selected compiler is gfortran. Any suggestion on how to override
this choice?

Thanks again to all.

Giovanni

> On 8 Mar 2022, at 16:39, Ye Luo <xw111luoye at gmail.com> wrote:
> 
> The error message indicates -lmkl_intel_lp64 failed to find libmkl_intel_lp64.so file on the default library search path known by the compiler.
> How to fix that? You can either add the directory to the default search path or specify the search path with -L${DIR}.
> 1. LDFLAGS=“-L${MKLROOT}/lib/intel64" whether it fixes the issue or not depends on if this flag is propagated to the configure internal test line. In addition its location on the linking line may also matter.
> 2. Add  "${MKLROOT}/lib/intel64" to the environment variable LIBRARY_PATH which may be honored by some compilers.
> Remember don't try to set LIBRARY_PATH persistent in your environment. It may cause surprise another day when you try to use a different MKL installation without deleting this line in your bashrc.
> 
> I would say both methods are not friendly to users. You could try CMake. https://gitlab.com/QEF/q-e/-/wikis/Developers/CMake-build-system <https://gitlab.com/QEF/q-e/-/wikis/Developers/CMake-build-system>
> What we do with CMake is first locate the exact library file and specify the full path on the link line.
> Hopefully it gives you a chance of successful compilation.
> 
> Best,
> Ye
> 
> ===================
> Ye Luo, Ph.D.
> Computational Science Division & Leadership Computing Facility
> Argonne National Laboratory
> 
> 
> On Tue, Mar 8, 2022 at 6:15 AM Giovanni Cantele <giovanni.cantele at spin.cnr.it <mailto:giovanni.cantele at spin.cnr.it>> wrote:
> Dear Pietro, I tried both, but it does not work.
> 
> My LD_LIBRARY_PATH after sourcing vars.sh is
> /nfsexports/intel/oneapi/mkl/2021.1.1/env/../lib/intel64:/nfsexports/nvidia/hpc_sdk/Linux_x86_64/20.11/cuda/lib64:/nfsexports/nvidia/hpc_sdk/Linux_x86_64/20.11/compilers/lib:/nfsexports/nvidia/hpc_sdk/Linux_x86_64/20.11/math_libs/lib64:/nfsexports/nvidia/hpc_sdk/Linux_x86_64/20.11/comm_libs/mpi/lib:/nfsexports/nvidia/hpc_sdk/Linux_x86_64/20.11/comm_libs/nccl/lib:/nfsexports/nvidia/hpc_sdk/Linux_x86_64/20.11/comm_libs/nvshmem/lib:/usr/mpi/gcc/openmpi-4.1.0rc5/lib64:
> that indeed contains the bat to MKL library.
> 
> In install/config.log I find, among the others, this message:
> configure:5252: nvfortran -o conftest -fast -Mcache_align -Mpreprocess -Mlarge_arrays -mp  -mp   conftest.f -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core   >&5
> /usr/bin/ld: cannot find -lmkl_intel_lp64
> /usr/bin/ld: cannot find -lmkl_intel_thread
> /usr/bin/ld: cannot find -lmkl_core
> 
> It is like although LD_LIBRARY_PATH contains the correct path to the library, ld cannot find it. On the other hand, if I pass the whole link command using LDFLAGS, then configure for some reason finds
> gfrotran in place of nvfortran and then exits with errors.
> 
> Giovanni
> 
>> On 8 Mar 2022, at 13:00, Pietro Davide Delugas <pdelugas at sissa.it <mailto:pdelugas at sissa.it>> wrote:
>> 
>> dear Giovanni 
>> you could  try just adding the /.../mkl/lib/intel64/ to the LD_LIBRARY_PATH or 
>> source of the /mkl/<version>/env/vars.sh script and then running the configure script 
>> 
>> Hope this solves your problem 
>> greetings 
>> Pietro 
>> Da: users <users-bounces at lists.quantum-espresso.org <mailto:users-bounces at lists.quantum-espresso.org>> per conto di Giovanni Cantele <giovanni.cantele at spin.cnr.it <mailto:giovanni.cantele at spin.cnr.it>>
>> Inviato: martedì 8 marzo 2022 12:46
>> A: Quantum ESPRESSO users Forum <users at lists.quantum-espresso.org <mailto:users at lists.quantum-espresso.org>>
>> Oggetto: [QE-users] qe-gpu 7.0 - compilation issue (MKL not detected)
>>  
>> Dear all,
>> 
>> I’m trying to configure the compilation of qe-7.0 on GPUs. I’m using the following script to launch configure:
>> -------------------------------------------------------------------------------------------------------------------
>> export VERSION=21.3
>> export VERSION=20.11
>> export CUDA_VERSION=11.1
>> 
>> export NVHPC=/nfsexports/nvidia/hpc_sdk
>> 
>> nvcudadir=$NVHPC/Linux_x86_64/$VERSION/cuda
>> nvcompdir=$NVHPC/Linux_x86_64/$VERSION/compilers
>> nvmathdir=$NVHPC/Linux_x86_64/$VERSION/math_libs
>> nvcommdir=$NVHPC/Linux_x86_64/$VERSION/comm_libs
>> 
>> export OPAL_PREFIX=$nvcommdir/mpi
>> 
>> export MANPATH=$MANPATH:$nvcompdir/man
>> 
>> PATH=$nvcudadir/bin:$nvcompdir/bin:$nvcommdir/mpi/bin:$PATH
>> export PATH
>> 
>> CPATH=$nvcudadir/include:$nvmathdir/include:$nvcommdir/mpi/include:$nvcommdir/nccl/include:$nvcommdir/nvshmem/include:$CPATH
>> export CPATH
>> 
>> LD_LIBRARY_PATH=$nvcudadir/lib64:$nvcompdir/lib:$nvmathdir/lib64:$nvcommdir/mpi/lib:$nvcommdir/nccl/lib:$nvcommdir/nvshmem/lib:$LD_LIBRARY_PATH
>> export LD_LIBRARY_PATH
>> . /nfsexports/intel/oneapi/mkl/latest/env/vars.sh
>> 
>> export OMPI_FC=nvfortran
>> export CUDA_HOME=$nvcudadir/$CUDA_VERSION
>> 
>> ./configure --enable-openmp  --with-cuda=$CUDA_HOME --with-cuda-cc=70 --with-cuda-runtime=11.1 --without-scalapack -with-cuda=yes --enable-cuda-env-check=no 
>> -------------------------------------------------------------------------------------------------------------------
>> 
>> 
>> Now, the problem is that I’m not able to make configure to recognise Intel MKL (2021.1.1 version).
>> 
>> The correct line to link to them is
>>  -L${MKLROOT}/lib/intel64 -lmkl_intel_ilp64 -lmkl_pgi_thread -lmkl_core -pgf90libs -mp -lpthread -lm -ldl
>> and adding 
>>  -i8  -I"${MKLROOT}/include”
>> to the compiler options.
>> 
>> However, if I try to run the above configure command adding
>> LDFLAGS=“-L${MKLROOT}/lib/intel64 -lmkl_intel_ilp64 -lmkl_pgi_thread -lmkl_core -pgf90libs -mp -lpthread -lm -ldl”
>> the configure stops finding the cuda enviroment, selects gfortran as compiler and exits with error. On the other hand, if I skip LDFLAGS (but
>> with MKLROOT correctly set up), the cuda environment is correctly detected but MKL are not automatically detected.
>> 
>> Could you please check the above script and, possibly, give any hint useful to solve the issue?
>> 
>> I thank you all in advance.
>> 
>> Giovanni
>> 
>> 
>> 
>> -- 
>> 
>> Giovanni Cantele, PhD
>> CNR-SPIN
>> c/o Dipartimento di Fisica
>> Universita' di Napoli "Federico II"
>> Complesso Universitario M. S. Angelo - Ed. 6
>> Via Cintia, I-80126, Napoli, Italy
>> e-mail: giovanni.cantele at spin.cnr.it <mailto:giovanni.cantele at spin.cnr.it>
>> Phone: +39 081 676910
>> Skype contact: giocan74
>> 
>> ResearcherID: http://www.researcherid.com/rid/A-1951-2009 <http://www.researcherid.com/rid/A-1951-2009>
>> Web page: https://sites.google.com/view/giovanni-cantele/home <https://sites.google.com/view/giovanni-cantele/home>
>> _______________________________________________
>> The Quantum ESPRESSO community stands by the Ukrainian
>> people and expresses its concerns about the devastating
>> effects that the Russian military offensive has on their
>> country and on the free and peaceful scientific, cultural,
>> and economic cooperation amongst peoples
>> _______________________________________________
>> Quantum ESPRESSO is supported by MaX (www.max-centre.eu <http://www.max-centre.eu/>)
>> users mailing list users at lists.quantum-espresso.org <mailto:users at lists.quantum-espresso.org>
>> https://lists.quantum-espresso.org/mailman/listinfo/users <https://lists.quantum-espresso.org/mailman/listinfo/users>
> -- 
> 
> Giovanni Cantele, PhD
> CNR-SPIN
> c/o Dipartimento di Fisica
> Universita' di Napoli "Federico II"
> Complesso Universitario M. S. Angelo - Ed. 6
> Via Cintia, I-80126, Napoli, Italy
> e-mail: giovanni.cantele at spin.cnr.it <mailto:giovanni.cantele at spin.cnr.it>
> Phone: +39 081 676910
> Skype contact: giocan74
> 
> ResearcherID: http://www.researcherid.com/rid/A-1951-2009 <http://www.researcherid.com/rid/A-1951-2009>
> Web page: https://sites.google.com/view/giovanni-cantele/home <https://sites.google.com/view/giovanni-cantele/home>
> _______________________________________________
> The Quantum ESPRESSO community stands by the Ukrainian
> people and expresses its concerns about the devastating
> effects that the Russian military offensive has on their
> country and on the free and peaceful scientific, cultural,
> and economic cooperation amongst peoples
> _______________________________________________
> Quantum ESPRESSO is supported by MaX (www.max-centre.eu <http://www.max-centre.eu/>)
> users mailing list users at lists.quantum-espresso.org <mailto:users at lists.quantum-espresso.org>
> https://lists.quantum-espresso.org/mailman/listinfo/users <https://lists.quantum-espresso.org/mailman/listinfo/users>_______________________________________________
> The Quantum ESPRESSO community stands by the Ukrainian
> people and expresses its concerns about the devastating
> effects that the Russian military offensive has on their
> country and on the free and peaceful scientific, cultural,
> and economic cooperation amongst peoples
> _______________________________________________
> Quantum ESPRESSO is supported by MaX (www.max-centre.eu)
> users mailing list users at lists.quantum-espresso.org
> https://lists.quantum-espresso.org/mailman/listinfo/users

-- 

Giovanni Cantele, PhD
CNR-SPIN
c/o Dipartimento di Fisica
Universita' di Napoli "Federico II"
Complesso Universitario M. S. Angelo - Ed. 6
Via Cintia, I-80126, Napoli, Italy
e-mail: giovanni.cantele at spin.cnr.it
Phone: +39 081 676910
Skype contact: giocan74

ResearcherID: http://www.researcherid.com/rid/A-1951-2009
Web page: https://sites.google.com/view/giovanni-cantele/home

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20220308/13cebec9/attachment.html>