<div dir="ltr"><div>This time OneAPI runs faster. The ifort in OneAPI should be very similar to the one in previous parallel studio releases.</div><div>I think the performance difference is from your machine. Neither QE nor the compiler plays anything here.<br></div><div>Ye<br></div><div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">===================<br>
Ye Luo, Ph.D.<br>Computational Science Division & Leadership Computing Facility<br>
Argonne National Laboratory</div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 3, 2021 at 4:46 PM Baer, Bradly <<a href="mailto:bradly.b.baer@vanderbilt.edu">bradly.b.baer@vanderbilt.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Hello,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
1) Ah, I did not notice that. I generally suppress I/O for test jobs but one I did a followup phonon calculation to test timings on that so I/O was turned on. The timing difference persists regardless of the I/O and also persisted into the phonon calculation
(1hr//1.5hr roughly)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
2) I have run it multiple times and timing is reproducible. Timing issue exists in ph.x as well as mentioned above. I have 16 physical cores and 32 hyperthreads.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
3) I have run with mpirun -np 1 which is what I think 1 MPI rank means. The cpu/wall timings are much more consistent, but I must confess that I am not experienced enough to understand what this result indicates is causing my issue.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Thanks,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Brad</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div>
<div id="gmail-m_-7887040961330492078Signature">
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
--------------------------------------------------------</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Bradly Baer</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif;background-color:rgb(255,255,255);display:inline">Graduate Research Assistant, Walker Lab</span><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif;background-color:rgb(255,255,255);display:inline">Interdisciplinary Materials Science</span><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Vanderbilt University</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
</div>
</div>
</div>
<div id="gmail-m_-7887040961330492078appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-7887040961330492078divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> users <<a href="mailto:users-bounces@lists.quantum-espresso.org" target="_blank">users-bounces@lists.quantum-espresso.org</a>> on behalf of Ye Luo <<a href="mailto:xw111luoye@gmail.com" target="_blank">xw111luoye@gmail.com</a>><br>
<b>Sent:</b> Thursday, June 3, 2021 4:22 PM<br>
<b>To:</b> Quantum ESPRESSO users Forum <<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a>><br>
<b>Subject:</b> Re: [QE-users] Compiling with Intel's OneAPI - Parallel Performance Issues</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>Hi Brad,<br>
</div>
<div>1. Your output files differ. One says 'Writing output data file ./pwscf.save' one doesn't. Does one have I/O and one doesn't?</div>
<div>2. Your simulation so small and also you are running 16 MPI ranks, so largely exercising MPI overhead. Run it a couple times and see if the timing is reproducible. Does your machine have 16 physical cores or 8 cores 16 hyperthreads?<br>
</div>
<div>3. To validate it is actually a compiler regression, run with 1 MPI rank and compare the timing.<br>
</div>
<div></div>
<div><br>
</div>
<div>Ye<br>
</div>
<div>
<div>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">===================<br>
Ye Luo, Ph.D.<br>
Computational Science Division & Leadership Computing Facility<br>
Argonne National Laboratory</div>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
<br>
<div>
<div dir="ltr">On Thu, Jun 3, 2021 at 3:36 PM Baer, Bradly <<a href="mailto:bradly.b.baer@vanderbilt.edu" target="_blank">bradly.b.baer@vanderbilt.edu</a>> wrote:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Hello Users,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
I have a working QE6.7 install built with Intel's Parallel Studios from 2020. I want to compile the d3q code but I have found that my parallel studios license is expired, and I must switch to Intel's new OneAPI distribution to continue using ifort, icc etc.
I have configured everything in the same way as with the parallel studios version, including using the same make.inc file, but my parallel performance is very poor when using the OneAPI version. </div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Attached are my make.inc file that I used for both compiles and an example output file using pw.x compiled with parallel studios and OneAPI. The parallel studios calculation had a CPU//wall time of 5.89s//5.97s but the OneAPI version has almost a 50% performance
loss and shows times of 5.92s//8.71s. Both were made using the same inputs.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Has anyone had experience compiling with the new OneAPI versions of things? Have I missed some small but important change in how the libraries are linked now?</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Thanks,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Brad</div>
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div id="gmail-m_-7887040961330492078x_gmail-m_-5915815349419976073Signature">
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
--------------------------------------------------------</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Bradly Baer</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif;background-color:rgb(255,255,255);display:inline">Graduate Research Assistant, Walker Lab</span><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif;background-color:rgb(255,255,255);display:inline">Interdisciplinary Materials Science</span><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Vanderbilt University</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
</div>
</div>
</div>
</div>
_______________________________________________<br>
Quantum ESPRESSO is supported by MaX (<a href="https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.max-centre.eu%2F&data=04%7C01%7Cbradly.b.baer%40vanderbilt.edu%7Cebb26f1903cc4b988a4c08d926d5b9a4%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C637583521651058549%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=VC%2B7hzIdNUfdKmK33zLvgMlJ054sVVnG8PMDknhuHxA%3D&reserved=0" rel="noreferrer" target="_blank">www.max-centre.eu</a>)<br>
users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank">
users@lists.quantum-espresso.org</a><br>
<a href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.quantum-espresso.org%2Fmailman%2Flistinfo%2Fusers&data=04%7C01%7Cbradly.b.baer%40vanderbilt.edu%7Cebb26f1903cc4b988a4c08d926d5b9a4%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C637583521651068545%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=itDrMarLfOmf7HCGKhnuMIPbOofSV6gPIys%2BqJxxE%2Bo%3D&reserved=0" rel="noreferrer" target="_blank">https://lists.quantum-espresso.org/mailman/listinfo/users</a></blockquote>
</div>
</div>
</div>
_______________________________________________<br>
Quantum ESPRESSO is supported by MaX (<a href="http://www.max-centre.eu" rel="noreferrer" target="_blank">www.max-centre.eu</a>)<br>
users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a><br>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.quantum-espresso.org/mailman/listinfo/users</a></blockquote></div>