<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body dir="auto">
<div><br>
</div>
Thanks! Would be great to have inputs/outputs of those benchmarks.
<div><br>
</div>
<div>Nicola<br>
<br>
<div id="AppleMailSignature" dir="ltr">Sent from a tiny keyboard... Contact info:
<div>
<div><a href="http://theossrv1.epfl.ch/Main/Contact">http://theossrv1.epfl.ch/Main/Contact</a></div>
</div>
</div>
<div dir="ltr"><br>
On 10 Jul 2019, at 09:28, Milos Maric <<a href="mailto:mmaric@nvidia.com">mmaric@nvidia.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">Dear Michal,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I run benchmarks on 2x Intel Xeon E5-2698 v4 and 2x Intel E5-2698 v4 + 4x Nvidia V100 16GB SMX2 (DGX-1V 16GB with only four GPUs enabled).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Depending on the benchmark I normally see 6x to 11x speedup.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best regards,<o:p></o:p></p>
<p class="MsoNormal">Miloš<span lang="SR-LATN-RS"><o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>From:</b> users <<a href="mailto:users-bounces@lists.quantum-espresso.org">users-bounces@lists.quantum-espresso.org</a>>
<b>On Behalf Of </b>Michal Krompiec<br>
<b>Sent:</b> Tuesday, July 9, 2019 9:12 PM<br>
<b>To:</b> Quantum ESPRESSO users Forum <<a href="mailto:users@lists.quantum-espresso.org">users@lists.quantum-espresso.org</a>><br>
<b>Subject:</b> Re: [QE-users] Bad scaling on GPU version QE<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Dear Pietro,<o:p></o:p></p>
<div>
<p class="MsoNormal">Thank you very much. Is this implementation restricted to Nvidia GPUs?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Best regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Michal<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Tue, 9 Jul 2019 at 19:14, Pietro BONFA' <<a href="mailto:pietro.bonfa@unipr.it">pietro.bonfa@unipr.it</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">Dear Michal,<br>
<br>
I will hopefully be able to share some new benchmarking results in the<br>
wiki soon, but for the time being the most accurate estimate is in this<br>
presentation:<br>
<a href="https://on-demand.gputechconf.com/gtc-eu/2018/pdf/e8340-porting-quantum-espresso-to-gpu-accelerated-systems.pdf" target="_blank">https://on-demand.gputechconf.com/gtc-eu/2018/pdf/e8340-porting-quantum-espresso-to-gpu-accelerated-systems.pdf</a><br>
.<br>
<br>
In the slides above you will find a comparison between v6.1 and v6.3.<br>
Depending on the problem size and on the type of simulation, v6.1 can be<br>
substantially faster.<br>
<br>
Our plan is to close the gap with the future releases, since the<br>
difference mainly comes from partially implemented features (for<br>
example, charge density mixing, exchange and correlation, forces and<br>
stress are accelerated in v6.1 but not in v6.4). This, however, must be<br>
done with care in order to keep the GPU development sustainable.<br>
<br>
Hope this helps,<br>
kind regards,<br>
Pietro Bonfa'<br>
<br>
<br>
<br>
On 7/8/19 10:27 PM, Michal Krompiec wrote:<br>
> Dear Pietro,<br>
> What is the typical speed up vs a cpu-only system? Is this<br>
> <a href="https://www.dcs.warwick.ac.uk/pmbs/pmbs17/PMBS17/pres/paper3.pdf" target="_blank">
https://www.dcs.warwick.ac.uk/pmbs/pmbs17/PMBS17/pres/paper3.pdf</a> still<br>
> valid? Can you share any benchmarks on V100?<br>
> Best,<br>
> Michal Krompiec<br>
> Merck KGaA, Darmstadt, Germany<br>
><br>
> On Mon, 8 Jul 2019 at 20:29, Pietro BONFA' <<a href="mailto:pietro.bonfa@unipr.it" target="_blank">pietro.bonfa@unipr.it</a><br>
> <mailto:<a href="mailto:pietro.bonfa@unipr.it" target="_blank">pietro.bonfa@unipr.it</a>>> wrote:<br>
><br>
> Dear Jing,<br>
><br>
> good point, I'll add the list of accelerated features in the wiki. In<br>
> the mean time you can check page 21 of the first presentation that you<br>
> can find here: <a href="https://gitlab.com/QEF/q-e-gpu/wikis/home#performance" target="_blank">
https://gitlab.com/QEF/q-e-gpu/wikis/home#performance</a><br>
><br>
> Exact exchange will come with the next release but you can already try<br>
> it by compiling this develop branch:<br>
> <a href="https://gitlab.com/QEF/q-e-gpu/tree/gpu-exx" target="_blank">https://gitlab.com/QEF/q-e-gpu/tree/gpu-exx</a><br>
><br>
> Kind regards,<br>
> Pietro Bonfa'<br>
><br>
> On 7/8/19 8:33 PM, JING YANG wrote:<br>
> > I am working on the latest version 6.4-a1. I have followed the<br>
> > instructions on the link you provided during the installation. I<br>
> would<br>
> > like to know which part of the calculation in pw being accelerated by<br>
> > GPU. For example, is it boosting the matrix diagonalization<br>
> during scf<br>
> > cycles, or boosting the hybrid functional calculations? I am<br>
> hoping to<br>
> > GPU acceleration can boost up the calculation on the exact<br>
> exchange in<br>
> > hybrid functional calculations. Is this the case?<br>
> ><br>
> > Thanks,<br>
> > Jing<br>
> ><br>
> > On Mon, Jul 8, 2019 at 1:57 PM Pietro BONFA'<br>
> <<a href="mailto:pietro.bonfa@unipr.it" target="_blank">pietro.bonfa@unipr.it</a> <mailto:<a href="mailto:pietro.bonfa@unipr.it" target="_blank">pietro.bonfa@unipr.it</a>><br>
> > <mailto:<a href="mailto:pietro.bonfa@unipr.it" target="_blank">pietro.bonfa@unipr.it</a> <mailto:<a href="mailto:pietro.bonfa@unipr.it" target="_blank">pietro.bonfa@unipr.it</a>>>> wrote:<br>
> ><br>
> > Dear Jing,<br>
> ><br>
> > which GPU-enabled version are you using?<br>
> > GPU support starting from v6.1 is only available when using PGI<br>
> > compilers (well, in principle, any compiler implementing CUDA<br>
> Fortran).<br>
> ><br>
> > You can find additional information here:<br>
> ><br>
> > <a href="https://gitlab.com/QEF/q-e-gpu/wikis/home" target="_blank">https://gitlab.com/QEF/q-e-gpu/wikis/home</a><br>
> ><br>
> > Kinds regards,<br>
> > Pietro Bonfa'<br>
> ><br>
> > On 7/8/19 5:01 PM, JING YANG wrote:<br>
> > > Hi,<br>
> > > I am trying to test the scaling of GPU version of QE<br>
> recently. I<br>
> > > found out the computational time scaling of the GPU version is<br>
> > very bad.<br>
> > > Is there any unique flags or keywords I don't know about<br>
> the GPU<br>
> > > version? I am using gcc-6.2.0, openmpi-1/8/4,<br>
> > craype-accel-nvidia35. I<br>
> > > know it is not the recommended set up, but I guess if I can<br>
> > compile it<br>
> > > successfully, then it should work normally.<br>
> > ><br>
> > > Thanks,<br>
> > > Jing<br>
> > ><br>
> > > _______________________________________________<br>
> > > Quantum ESPRESSO is supported by MaX<br>
> > (<a href="http://www.max-centre.eu/quantum-espresso" target="_blank">www.max-centre.eu/quantum-espresso</a><br>
> <<a href="http://www.max-centre.eu/quantum-espresso" target="_blank">http://www.max-centre.eu/quantum-espresso</a>><br>
> > <<a href="http://www.max-centre.eu/quantum-espresso" target="_blank">http://www.max-centre.eu/quantum-espresso</a>>)<br>
> > > users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank">
users@lists.quantum-espresso.org</a><br>
> <mailto:<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a>><br>
> > <mailto:<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a><br>
> <mailto:<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a>>><br>
> > > <a href="https://lists.quantum-espresso.org/mailman/listinfo/users" target="_blank">
https://lists.quantum-espresso.org/mailman/listinfo/users</a><br>
> > ><br>
> ><br>
> ><br>
> > --<br>
> > Pietro Bonfà<br>
> > Department of Mathematical, Physical and Computer Sciences,<br>
> > University of Parma,<br>
> > Italy<br>
> ><br>
> > Firma il tuo 5 per mille all’Università di Parma e aiuta così i<br>
> > nostri studenti che vogliono realizzare un’esperienza di studio<br>
> > all’estero - Indica 00308780345 nella tua denuncia dei redditi.<br>
> > _______________________________________________<br>
> > Quantum ESPRESSO is supported by MaX<br>
> > (<a href="http://www.max-centre.eu/quantum-espresso" target="_blank">www.max-centre.eu/quantum-espresso</a><br>
> <<a href="http://www.max-centre.eu/quantum-espresso" target="_blank">http://www.max-centre.eu/quantum-espresso</a>><br>
> > <<a href="http://www.max-centre.eu/quantum-espresso" target="_blank">http://www.max-centre.eu/quantum-espresso</a>>)<br>
> > users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank">
users@lists.quantum-espresso.org</a><br>
> <mailto:<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a>><br>
> > <mailto:<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a><br>
> <mailto:<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a>>><br>
> > <a href="https://lists.quantum-espresso.org/mailman/listinfo/users" target="_blank">
https://lists.quantum-espresso.org/mailman/listinfo/users</a><br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > Quantum ESPRESSO is supported by MaX<br>
> (<a href="http://www.max-centre.eu/quantum-espresso" target="_blank">www.max-centre.eu/quantum-espresso</a><br>
> <<a href="http://www.max-centre.eu/quantum-espresso" target="_blank">http://www.max-centre.eu/quantum-espresso</a>>)<br>
> > users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank">
users@lists.quantum-espresso.org</a><br>
> <mailto:<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a>><br>
> > <a href="https://lists.quantum-espresso.org/mailman/listinfo/users" target="_blank">
https://lists.quantum-espresso.org/mailman/listinfo/users</a><br>
> ><br>
><br>
><br>
> --<br>
> Pietro Bonfà<br>
> Department of Mathematical, Physical and Computer Sciences,<br>
> University of Parma,<br>
> Italy<br>
><br>
> Firma il tuo 5 per mille all’Università di Parma e aiuta così i<br>
> nostri studenti che vogliono realizzare un’esperienza di studio<br>
> all’estero - Indica 00308780345 nella tua denuncia dei redditi.<br>
> _______________________________________________<br>
> Quantum ESPRESSO is supported by MaX<br>
> (<a href="http://www.max-centre.eu/quantum-espresso" target="_blank">www.max-centre.eu/quantum-espresso</a><br>
> <<a href="http://www.max-centre.eu/quantum-espresso" target="_blank">http://www.max-centre.eu/quantum-espresso</a>>)<br>
> users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank">
users@lists.quantum-espresso.org</a><br>
> <mailto:<a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a>><br>
> <a href="https://lists.quantum-espresso.org/mailman/listinfo/users" target="_blank">https://lists.quantum-espresso.org/mailman/listinfo/users</a><br>
><br>
><br>
> _______________________________________________<br>
> Quantum ESPRESSO is supported by MaX (<a href="http://www.max-centre.eu/quantum-espresso" target="_blank">www.max-centre.eu/quantum-espresso</a>)<br>
> users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank">
users@lists.quantum-espresso.org</a><br>
> <a href="https://lists.quantum-espresso.org/mailman/listinfo/users" target="_blank">
https://lists.quantum-espresso.org/mailman/listinfo/users</a><br>
><br>
<br>
<br>
--<br>
Pietro Bonfà<br>
Department of Mathematical, Physical and Computer Sciences,<br>
University of Parma,<br>
Italy<br>
<br>
Firma il tuo 5 per mille all’Università di Parma e aiuta così i nostri studenti che vogliono realizzare un’esperienza di studio all’estero - Indica 00308780345 nella tua denuncia dei redditi.<br>
_______________________________________________<br>
Quantum ESPRESSO is supported by MaX (<a href="http://www.max-centre.eu/quantum-espresso" target="_blank">www.max-centre.eu/quantum-espresso</a>)<br>
users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank">
users@lists.quantum-espresso.org</a><br>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" target="_blank">https://lists.quantum-espresso.org/mailman/listinfo/users</a><o:p></o:p></p>
</blockquote>
</div>
</div>
<p>-----------------------------------------------------------------------------------<br>
NVIDIA GmbH<br>
Wuerselen<br>
Amtsgericht Aachen<br>
HRB 8361<br>
Managing Director: Karen Theresa Burns</p>
<p>-----------------------------------------------------------------------------------<br>
This email message is for the sole use of the intended recipient(s) and may contain<br>
confidential information. Any unauthorized review, use, disclosure or distribution<br>
is prohibited. If you are not the intended recipient, please contact the sender by<br>
reply email and destroy all copies of the original message.<br>
-----------------------------------------------------------------------------------</p>
</div>
</blockquote>
<blockquote type="cite">
<div dir="ltr"><span>_______________________________________________</span><br>
<span>Quantum ESPRESSO is supported by MaX (<a href="http://www.max-centre.eu/quantum-espresso">www.max-centre.eu/quantum-espresso</a>)</span><br>
<span>users mailing list <a href="mailto:users@lists.quantum-espresso.org">users@lists.quantum-espresso.org</a></span><br>
<span><a href="https://lists.quantum-espresso.org/mailman/listinfo/users">https://lists.quantum-espresso.org/mailman/listinfo/users</a></span></div>
</blockquote>
</div>
</body>
</html>