<div dir="ltr"><div><div>Hi Carlo,<br><br></div> Most of our profiling has been done using boxes of water of varying size. I have attached input files for boxes of 16, 32, and 64 water molecules. Running with systems of different sizes is important for the purpose of evaluating which parts of the calculation become important in the limit of large system size.<br> In addition to the water systems, it is probably wise to also run some profiling calculations using a metallic system. For this purpose, we have been working with the AUSURF112 benchmark (see <a href="http://qe-forge.org/gf/project/q-e/frs/?action=FrsReleaseView&release_id=45" target="_blank">http://qe-forge.org/gf/project/q-e/frs/?action=FrsReleaseView&release_id=45</a>). We have also run tests on a water-platinum interface, although we are still in the process of refining our geometries for this system.<br></div><div> I wouldn't necessarily say that this is comprehensive suite of tests, but it does encompass the regimes that most strongly interest us. Feel free to let me know if you have identified other systems that make for ideal benchmarking candidates.<br></div><div><br></div><div>Best,<br></div><div>Taylor<br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Apr 6, 2016 at 8:36 AM, Carlo Cavazzoni <span dir="ltr"><<a href="mailto:c.cavazzoni@cineca.it" target="_blank">c.cavazzoni@cineca.it</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>Dear Taylor,<br>
dear all,<br>
<br>
since in Cineca we are interested in optimizing EXX computation as
well,<br>
I would like to know the input set you have used, or in
alternative,<br>
to agree on a new, non trivial, input set to share among all those<br>
interested in profiling and benchmarking EXX computations.<br>
I'm not an expert myself in EXX so I would prefer to receive<br>
a meaningful non trivial input set from someone understanding<br>
this extension,<br>
<br>
best,<br>
carlo<div><div class="h5"><br>
<br>
Il 05/04/2016 06:05, Taylor Barnes ha scritto:<br>
</div></div></div>
<blockquote type="cite"><div><div class="h5">
<div dir="ltr">Dear All,<br>
<br>
I wanted to inform everyone about some improvements that we
have been making at LBNL to the implementation of exact exchange
in QE. These improvements have been made as part of NERSC's
Exascale Science Applications Program, which is an effort to
update codes for execution on next-generation architectures such
as NERSC's upcoming Cori Phase II. The following is a brief
overview of these changes, which we are currently in the process
of testing and debugging. Depending on our progress, we intend
to submit these changes as an addition to either QE 5.4 or 6.0.<br>
<br>
1. Parallelization Over Band Pairs<br>
We have extended the parallelization of subroutine vexx_k
such that both of the loops over bands (i.e.,
"LOOP_ON_PSI_BANDS" and "IBND_LOOP_K") are parallelized with
respect to band groups. This improves load balancing, and also
enables parallelization using larger numbers of band groups than
was previously possible<br>
<br>
2. Improved OMP Support<br>
We have added OMP threading to numerous vector operations
within exx.f90. In addition, we have given special priority to
enhancing the threaded performance of the FFTs.<br>
<br>
3. Implementation of Different and Interchangeable Data Layouts
for Local and EXX Portions of the Calculation<br>
One observation that we have made is that for calculations
that utilize many band groups, the local portion of the
calculation (i.e., everything outside of exx.f90) often
represents a non-negligible (or even dominant) contribution to
the total cost of the calculation. This is largely because the
local portion of the calculation is duplicated on each band
group. We have implemented changes to the code that allow the
local portion of the code to be parallelized in a manner that is
independent of the number of band groups, thus avoiding
duplication of work.<br>
This is the single most significant modification that we have
made, both in terms of increasing the efficiency of QE, as well
as the amount of coding work required. For several test
calculations we are finding that this change results in more
than a factor of two speedup.<br>
In terms of code development, the primary challenge of our
approach is that when the EXX part of the calculation is
performed (such as when vexx is called), we must change the data
structure from the one that is used by the local portion of the
code to a different data structure that is used by the EXX
portion of the code. This change of data structure requires a
great deal of bookkeeping in order to update arrays like igk,
ig_l2g, psi, and hpsi. As a result, we a still the process of
making our updated code compatible with gamma-point only
calculations and with calculations that employ multiple
k-points.<br>
<br>
<span><span>Sincerely,<br>
</span></span><span><span>Dr. Taylor Barnes<br>
</span></span>
<div><span><span>Postdoctoral Scholar,<br>
</span></span></div>
<span><span>Lawrence Berkeley National Laboratory</span></span><br>
</div>
<br>
<fieldset></fieldset>
<br>
</div></div><span class=""><pre>_______________________________________________
Q-e-developers mailing list
<a href="mailto:Q-e-developers@qe-forge.org" target="_blank">Q-e-developers@qe-forge.org</a>
<a href="http://qe-forge.org/mailman/listinfo/q-e-developers" target="_blank">http://qe-forge.org/mailman/listinfo/q-e-developers</a>
</pre>
</span></blockquote><span class="HOEnZb"><font color="#888888">
<br>
<br>
<pre cols="72">--
Ph.D. Carlo Cavazzoni
SuperComputing Applications and Innovation Department
CINECA - Via Magnanelli 6/3, 40033 Casalecchio di Reno (Bologna)
Tel: <a href="tel:%2B39%20051%206171411" value="+390516171411" target="_blank">+39 051 6171411</a> Fax: <a href="tel:%2B39%20051%206132198" value="+390516132198" target="_blank">+39 051 6132198</a>
<a href="http://www.cineca.it" target="_blank">www.cineca.it</a></pre>
</font></span></div>
<br>_______________________________________________<br>
Q-e-developers mailing list<br>
<a href="mailto:Q-e-developers@qe-forge.org">Q-e-developers@qe-forge.org</a><br>
<a href="http://qe-forge.org/mailman/listinfo/q-e-developers" rel="noreferrer" target="_blank">http://qe-forge.org/mailman/listinfo/q-e-developers</a><br>
<br></blockquote></div><br></div>