<div dir="ltr"><div>The GPU acceleration of DFT-D3, using openacc, as well as its MPI parallelization, was implemented no more than a few days ago and will appear in the next release (soon). Apparently DFT-D3 takes a non-negligible amount of time. Without MPI parallelization or GPU acceleration, it may easily become a bottleneck when running on many processors, or on GPUs.</div><div><br></div><div>Paolo<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jul 6, 2021 at 7:44 PM Jonathan D. Halverson <<a href="mailto:halverson@princeton.edu">halverson@princeton.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="margin:0px;font-size:12pt"><span style="margin:0px;font-size:15px;color:rgb(32,31,30)"><span style="margin:0px;font-size:12pt;color:rgb(0,0,0)">Hello (@Louis Stuber),</span></span></span></div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<div style="margin:0px;font-size:12pt">
<div style="margin:0px;font-size:15px;color:rgb(32,31,30)"><br>
</div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30)"><span style="margin:0px;font-size:12pt;color:rgb(0,0,0)">The </span><span style="margin:0px;font-size:12pt;color:rgb(0,0,0);display:inline">QE container on NGC (<a href="https://ngc.nvidia.com/catalog/containers/hpc:quantum_espresso" target="_blank">https://ngc.nvidia.com/catalog/containers/hpc:quantum_espresso</a>)</span><span style="margin:0px;display:inline"><span style="margin:0px;font-size:12pt;color:rgb(0,0,0)"> appears
to be running very well for us on a node with two A100's for the </span><span style="margin:0px"><span style="margin:0px;font-size:12pt;color:rgb(0,0,0);display:inline">"AUSURF112, Gold surface (112 atoms), DEISA pw" benchmark. We see a speed-up
of 8x in comparison to running on 80 Skylake CPU-cores (no GPUs) where the code was built from source.</span></span></span></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30)"><span style="margin:0px;display:inline"><span style="margin:0px"><span style="margin:0px;display:inline"><br>
</span></span></span></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30)"><span style="margin:0px;display:inline"><span style="margin:0px"><span style="margin:0px;font-size:12pt;color:rgb(0,0,0);display:inline">The procedure we used for the above
is here:</span><span style="margin:0px;display:inline"><br>
<div style="margin:0px"><a href="https://researchcomputing.princeton.edu/support/knowledge-base/quantum-espresso" rel="noopener noreferrer" style="margin:0px" target="_blank"><span style="margin:0px;font-size:12pt;color:rgb(0,0,0)">https://researchcomputing.princeton.edu/support/knowledge-base/quantum-espresso</span></a></div>
</span></span></span></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30)"><span style="margin:0px;display:inline"><span style="margin:0px"><span style="margin:0px;display:inline"><br>
</span></span></span></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30)"><span style="margin:0px;display:inline"><span style="margin:0px"><span style="margin:0px;font-size:12pt;color:rgb(0,0,0);display:inline">However, for one system we see a slow
down (i.e., the code runs faster using only CPU-cores). Can you tell if the system below should perform well using the container?</span></span></span></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30)"><br>
</div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30)"><span style="margin:0px;font-size:12pt;color:rgb(0,0,0);display:inline">"My system is basically just two carbon dioxide molecules and doing a</span><span style="margin:0px;font-size:12pt;color:rgb(0,0,0);display:inline"> </span><span style="margin:0px;font-size:12pt;color:rgb(0,0,0);display:inline">single point
calculation on them using the PBE-D3 functional and basically just </span><span style="margin:0px;font-size:12pt;color:rgb(0,0,0);display:inline">altering the distance between the two molecules in the atomic coordinates."</span></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30)"><br>
</div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30)"><span style="margin:0px;font-size:12pt;color:rgb(0,0,0)">Can someone comment in general on when one would expect the container running on GPUs to outperform a build-from-source executable running
on CPU-cores?</span></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30)"><br>
</div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30)"><span style="margin:0px;font-size:12pt;color:rgb(0,0,0)">CUDA-aware MPI is nice. It appears that the container is configured to use the MPI libraries in the container instead of those installed
for the local cluster. Is this true? Can users take advantage of their local CUDA-aware MPI libraries?</span></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30)"><br>
</div>
<span style="margin:0px">Jon</span></div>
</div>
</div>
_______________________________________________<br>
Quantum ESPRESSO is supported by MaX (<a href="http://www.max-centre.eu" rel="noreferrer" target="_blank">www.max-centre.eu</a>)<br>
users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a><br>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.quantum-espresso.org/mailman/listinfo/users</a></blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,<br>Univ. Udine, via delle Scienze 206, 33100 Udine, Italy<br>Phone +39-0432-558216, fax +39-0432-558222<br><br></div></div></div></div></div>