<div dir="ltr">Dear Fabrizio,<div>thanks a lot for the explanation.</div><div>I was unsure about how to proceed and worried not to get the proper performance on the GPU.</div><div><br></div><div>May I ask for help regarding the way of running the code? Where can I find instructions on how to launch the executable?</div><div><br></div><div>For example, how can I control the number of GPUs used and the parallel processes? </div><div><br></div><div>I have 2 GPUs for each node.</div><div><br></div><div>Thanks a lot and best regards,</div><div>Mauro Sgroi.<br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font color="#000000" face="tahoma, sans-serif">___</font><font color="#000000" face="tahoma, sans-serif">____________________</font></p><p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font face="tahoma, sans-serif"><span style="color:rgb(0,0,0)">Dr.
</span><b style="color:rgb(0,0,0)">Mauro Francesco Sgroi</b><br></font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%">
<font face="tahoma, sans-serif"><font color="#000000"><font>Department
of Chemistry </font></font>
</font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%">
<font color="#000000" face="tahoma, sans-serif">University
of Turin</font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font color="#000000" face="tahoma, sans-serif">Via Quarello 15a</font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%">
<font color="#000000" face="tahoma, sans-serif">I-10135
TORINO (Italy)</font></p><p lang="zxx" style="margin-bottom:0cm;line-height:100%"><span style="color:rgb(0,0,0)"><font face="tahoma, sans-serif">Tel.</font></span></p><p lang="zxx" style="margin-bottom:0cm;line-height:100%"><span style="color:rgb(0,0,0)"><font face="tahoma, sans-serif">+39 011-670-8372 </font></span></p><p lang="zxx" style="margin-bottom:0cm;line-height:100%"><span style="color:rgb(0,0,0)"><font face="tahoma, sans-serif">+39 011-670-7364</font></span></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font face="tahoma, sans-serif"><span style="color:rgb(0,0,0)">e-mail:
</span><a href="mailto:maurofrancesco.sgroi@unito.it" target="_blank">maurofrancesco.sgroi@unito.it</a><br></font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%">
<font face="tahoma, sans-serif"><font color="#000000"><font>Web: </font></font></font></p><p lang="zxx" style="margin-bottom:0cm;line-height:100%"><a href="http://www.met.unito.it/" rel="noreferrer" style="color:rgb(17,85,204)" target="_blank">www.met.unito.it</a><br></p><p lang="zxx" style="margin-bottom:0cm;line-height:100%"><a href="http://www.chimica.unito.it" style="font-family:tahoma,sans-serif" target="_blank">www.chimica.unito.it</a><span style="color:rgb(0,0,0);font-family:tahoma,sans-serif"> </span><br></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%">
<font color="#000000"><font><font face="tahoma, sans-serif">Orcid:
<a href="https://orcid.org/0000-0002-0914-4217" target="_blank">https://orcid.org/0000-0002-0914-4217</a></font>
</font></font>
</p><p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font color="#000000"><font>Webex: </font><a href="https://unito.webex.com/webappng/sites/unito/dashboard/pmr/maurofrancesco.sgroi" target="_blank">https://unito.webex.com/webappng/sites/unito/dashboard/pmr/maurofrancesco.sgroi</a> </font></p></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Il giorno ven 2 ago 2024 alle ore 14:11 Fabrizio Ferrari Ruffino <<a href="mailto:faferrar@sissa.it">faferrar@sissa.it</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg-4131132194137127674">
<div dir="ltr">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Hi,</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
there are a few minor FFTXlib calls somewhere in QE which are still on CPU, therefore it is better to have a CPU fft backend enabled too. Whether to use the internal one or FFTW3 should not make much difference, since all the main stuff runs on gpu (therefore
calling cuFFT).<br>
In a CPU run the FFTW3 backend is faster than the internal one, but, as I said, in a GPU run it should be quite irrelevant.</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Cheers,</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Fabrizio</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
CNR IOM</div>
<div id="m_-4131132194137127674appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="m_-4131132194137127674divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> users <<a href="mailto:users-bounces@lists.quantum-espresso.org" target="_blank">users-bounces@lists.quantum-espresso.org</a>> on behalf of Mauro Francesco Sgroi via users <<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a>><br>
<b>Sent:</b> Friday, August 2, 2024 12:13 PM<br>
<b>To:</b> Quantum ESPRESSO users Forum <<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a>><br>
<b>Subject:</b> [QE-users] Help for compilation with Nvidia HPC SDK</font>
<div> </div>
</div>
<div>
<div dir="ltr">Dear all,
<div>
<div>I am trying to compile the 7.3.1 version of Quantum Espresso using the last Nvidia HPC SDK (24.7) on Ubuntu 24.04.</div>
<div><br>
</div>
<div>I am configuring as follows:</div>
<div><br>
</div>
<div>export BLAS_LIBS='-L/opt/nvidia/hpc_sdk/Linux_x86_64/2024/math_libs/lib64 -lcublas -lcublasLt -L/opt/nvidia/hpc_sdk/Linux_x86_64/2024/compilers/lib -lblas -L/opt/nvidia/hpc_sdk/Linux_x86_64/2024/cuda/lib64 -lcudart'<br>
<br>
</div>
<div>export LAPACK_LIBS='-L/opt/nvidia/hpc_sdk/Linux_x86_64/2024/math_libs/lib64 -lcusolver -lcurand -lcublas -lcublasLt -lcusparse -L/opt/nvidia/hpc_sdk/Linux_x86_64/2024/compilers/lib -llapack -L/opt/nvidia/hpc_sdk/Linux_x86_64/2024/cuda/lib64 -lcudart'<br>
<br>
</div>
<div>export SCALAPACK_LIBS='-/opt/nvidia/hpc_sdk/Linux_x86_64/2024/comm_libs/12.5/openmpi4/openmpi-4.1.5/lib -lscalapack -L/opt/nvidia/hpc_sdk/Linux_x86_64/2024/comm_libs/12.5/openmpi4/latest/lib -lmpi -lopen-pal'</div>
<div><br>
</div>
<div>./configure --with-cuda=/opt/nvidia/hpc_sdk/Linux_x86_64/2024/cuda/12.5 --with-cuda-cc=75 --with-cuda-runtime=12.5 --with-cuda-mpi=yes</div>
<div><br>
</div>
<div>In this way, the internal FFTW library is selected. Should I use the FFTW3 library together with cufft? <br>
</div>
<div><br>
</div>
<div>Can the two libraries work together? Is it normal that the internal FFTW library is used? Or should the cufft library be sufficient?</div>
<div><br>
</div>
<div>Or is it better to use the cufftw library supplied by NVIDIA? </div>
<div><br>
</div>
<div>Can I have some guidance on these aspects?</div>
<div><br>
</div>
<div>Thanks a lot and best regards,</div>
<div>Mauro Sgroi.</div>
<div><br clear="all">
<div>
<div dir="ltr">
<div dir="ltr">
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font color="#000000" face="tahoma, sans-serif">___</font><font color="#000000" face="tahoma, sans-serif">____________________</font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font face="tahoma, sans-serif"><span style="color:rgb(0,0,0)">Dr.
</span><b style="color:rgb(0,0,0)">Mauro Francesco Sgroi</b><br>
</font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font face="tahoma, sans-serif"><font color="#000000"><font>Department of Chemistry
</font></font></font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font color="#000000" face="tahoma, sans-serif">University of Turin</font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font color="#000000" face="tahoma, sans-serif">Via Quarello 15a</font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font color="#000000" face="tahoma, sans-serif">I-10135 TORINO (Italy)</font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><span style="color:rgb(0,0,0)"><font face="tahoma, sans-serif">Tel.</font></span></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><span style="color:rgb(0,0,0)"><font face="tahoma, sans-serif">+39 011-670-8372 </font></span></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><span style="color:rgb(0,0,0)"><font face="tahoma, sans-serif">+39 011-670-7364</font></span></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font face="tahoma, sans-serif"><span style="color:rgb(0,0,0)">e-mail:
</span><a href="mailto:maurofrancesco.sgroi@unito.it" target="_blank">maurofrancesco.sgroi@unito.it</a><br>
</font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font face="tahoma, sans-serif"><font color="#000000"><font>Web: </font></font></font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><a href="http://www.met.unito.it/" rel="noreferrer" style="color:rgb(17,85,204)" target="_blank">www.met.unito.it</a><br>
</p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><a href="http://www.chimica.unito.it" style="font-family:tahoma,sans-serif" target="_blank">www.chimica.unito.it</a><span style="color:rgb(0,0,0);font-family:tahoma,sans-serif"> </span><br>
</p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font color="#000000"><font><font face="tahoma, sans-serif">Orcid:
<a href="https://orcid.org/0000-0002-0914-4217" target="_blank">https://orcid.org/0000-0002-0914-4217</a></font>
</font></font></p>
<p lang="zxx" style="margin-bottom:0cm;line-height:100%"><font color="#000000"><font>Webex: </font><a href="https://unito.webex.com/webappng/sites/unito/dashboard/pmr/maurofrancesco.sgroi" target="_blank">https://unito.webex.com/webappng/sites/unito/dashboard/pmr/maurofrancesco.sgroi</a> </font></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div></blockquote></div>