<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
The "__GPU_MPI" flag is not strictly necessary to compile and run the MPI parallel versions with GPU. </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
It only enables more efficient communications when the MPI library has been compiled with the necessary support for GPU-aware communications. The only essential flag for MPI is the -D__MPI. </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
The failure's origin might be that your MPI library lacks such support. But you had better ask your system administrator about this. </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
You can remove the GPU_MPI flag and see if the code works. </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Hope it helps </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Pietro </div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>Da:</b> users <users-bounces@lists.quantum-espresso.org> per conto di Siwakorn Sukharom <siwakorn.suk@ncr.nstda.or.th><br>
<b>Inviato:</b> giovedì 25 maggio 2023 06:34<br>
<b>A:</b> Paolo Giannozzi <paolo.giannozzi@uniud.it>; Quantum ESPRESSO users Forum <users@lists.quantum-espresso.org><br>
<b>Oggetto:</b> Re: [QE-users] R: [QE-GPU] compilation issue : NVFORTRAN-F-1225-Unmatched directive</font>
<div> </div>
</div>
<style>
<!--
.x_EmailQuote
{margin-left:1pt;
padding-left:4pt;
border-left:#800000 2px solid}
-->
</style>
<div>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p>Now, I can compiled GPU version for QE 7.2. Thank you for the suggestion. It can run fine on 1 gpu. However, I still having a problem with running multiple gpus. I have added the <span>--with-cuda-mpi=yes during the configuration, and I can see that there
is <span>-D__GPU_MPI tag on DFLAGS in make.inc</span></span></p>
<p><span><span><br>
</span></span></p>
<p><span><span>When compiled and run on multiple gpus, I get the error message below:</span></span></p>
<p><span><span><br>
</span></span></p>
<p><span><span></p>
<div>%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%</div>
<div> Error in routine fft_buffers (4):</div>
<div> Allocation failed</div>
<div> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%</div>
<div><br>
</div>
<div> stopping ...</div>
<div>MPICH ERROR [Rank 0] [job id 77558.0] [Thu May 25 11:11:15 2023] [x1000c2s0b0n1] - Abort(4) (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, 4) - process 0</div>
<div><br>
</div>
<div>aborting job:</div>
<div>application called MPI_Abort(MPI_COMM_WORLD, 4) - process 0</div>
<div><br>
</div>
What is happening?</span></span>
<p></p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Paolo Giannozzi <paolo.giannozzi@uniud.it><br>
<b>Sent:</b> Wednesday, May 24, 2023 1:05:38 PM<br>
<b>To:</b> Siwakorn Sukharom; Quantum ESPRESSO users Forum<br>
<b>Subject:</b> Re: [QE-users] R: [QE-GPU] compilation issue : NVFORTRAN-F-1225-Unmatched directive</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt">
<div class="x_PlainText">On 24/05/2023 05:47, Siwakorn Sukharom wrote:<br>
> <br>
> I have added -D__OPENACC to DFLAGS for QE 7.2 and 7.1 <br>
<br>
"-D_OPENACC", with one underscore<br>
<br>
Paolo<br>
<br>
. The compilation<br>
> was failed for QE 7.2, but success for QE 7.1. The <a href="https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fmake.inc&c=E,1,RCjygUVrHz_y_vPiylqafXnFbjj3qIZgqer4PcMPTMAL1nWOb0F2WH7i2o4bEwXkmPY9pBKdip8024dPJMlrqTn00xrWzoEusar_ykVw&typo=1">
https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fmake.inc&c=E,1,RCjygUVrHz_y_vPiylqafXnFbjj3qIZgqer4PcMPTMAL1nWOb0F2WH7i2o4bEwXkmPY9pBKdip8024dPJMlrqTn00xrWzoEusar_ykVw&typo=1</a> is almost
<br>
> the same.<br>
> <br>
> ------------------------------------------------------------------------<br>
> *From:* Paolo Giannozzi <paolo.giannozzi@uniud.it><br>
> *Sent:* Tuesday, May 23, 2023 5:29:09 PM<br>
> *To:* Quantum ESPRESSO users Forum; Siwakorn Sukharom<br>
> *Subject:* Re: [QE-users] R: [QE-GPU] compilation issue : <br>
> NVFORTRAN-F-1225-Unmatched directive<br>
> On 5/23/23 12:13, Siwakorn Sukharom wrote:<br>
> <br>
>> I can successfully compile on QE version 7.1, but not 7.2. The error <br>
>> is still the same. Will focus on 7.1 instead.<br>
> <br>
> Not a good idea. You just need to add _OPENACC to DFLAGS (something that<br>
> the Cray compiler should take care of, but it doesn't).<br>
> <br>
> Paolo<br>
> -- <br>
> Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,<br>
> Univ. Udine, via delle Scienze 208, 33100 Udine Italy, +39-0432-558216<br>
> <br>
> ------------------------------------------------------------------------<br>
> <br>
> Disclaimer:<br>
> <br>
> This e-mail and any files transmitted with it may contain confidential <br>
> and proprietary information of the National Science and Technology <br>
> Development Agency (NSTDA), Thailand. They are intended solely for the <br>
> use of the addressed individuals or entities. If you are not the <br>
> intended recipient, you are required to immediately delete this e-mail <br>
> and its contents from your system. Any disclosure, distribution, or <br>
> action based upon the contents of this e-mail is strictly prohibited. <br>
> Any views or opinions presented in this e-mail are solely those of the <br>
> sender and do not necessarily represent those of NSTDA. NSTDA does not <br>
> accept any responsibility for the content of this message or the <br>
> consequences of any actions taken on the basis of the information <br>
> provided. NSTDA accepts no liability for any damage caused by any virus <br>
> or malware which may be inserted in this e-mail during transmission.<br>
<br>
-- <br>
Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,<br>
Univ. Udine, via delle Scienze 206, 33100 Udine Italy, +39-0432-558216<br>
</div>
</span></font><br>
<hr>
<font color="Gray" size="1"><br>
Disclaimer:<br>
<br>
This e-mail and any files transmitted with it may contain confidential and proprietary information of the National Science and Technology Development Agency (NSTDA), Thailand. They are intended solely for the use of the addressed individuals or entities. If
you are not the intended recipient, you are required to immediately delete this e-mail and its contents from your system. Any disclosure, distribution, or action based upon the contents of this e-mail is strictly prohibited. Any views or opinions presented
in this e-mail are solely those of the sender and do not necessarily represent those of NSTDA. NSTDA does not accept any responsibility for the content of this message or the consequences of any actions taken on the basis of the information provided. NSTDA
accepts no liability for any damage caused by any virus or malware which may be inserted in this e-mail during transmission.
<br>
</font></div>
</body>
</html>