[Q-e-developers] Improvements to the EXX Implementation

[Taylor Barnes]

Hi Carlo,

   Most of our profiling has been done using boxes of water of varying
size.  I have attached input files for boxes of 16, 32, and 64 water
molecules.  Running with systems of different sizes is important for the
purpose of evaluating which parts of the calculation become important in
the limit of large system size.
   In addition to the water systems, it is probably wise to also run some
profiling calculations using a metallic system.  For this purpose, we have
been working with the AUSURF112 benchmark (see
http://qe-forge.org/gf/project/q-e/frs/?action=FrsReleaseView&release_id=45).
We have also run tests on a water-platinum interface, although we are still
in the process of refining our geometries for this system.
   I wouldn't necessarily say that this is comprehensive suite of tests,
but it does encompass the regimes that most strongly interest us.  Feel
free to let me know if you have identified other systems that make for
ideal benchmarking candidates.

Best,
Taylor


On Wed, Apr 6, 2016 at 8:36 AM, Carlo Cavazzoni <c.cavazzoni at cineca.it>
wrote:

> Dear Taylor,
> dear all,
>
> since in Cineca we are interested in optimizing EXX computation as well,
> I would like to know the input set you have used, or in alternative,
> to agree on a new, non trivial, input set to share among all those
> interested in profiling and benchmarking EXX computations.
> I'm not an expert myself in EXX so I would prefer to receive
> a meaningful non trivial input set from someone understanding
> this extension,
>
> best,
> carlo
>
>
> Il 05/04/2016 06:05, Taylor Barnes ha scritto:
>
> Dear All,
>
>    I wanted to inform everyone about some improvements that we have been
> making at LBNL to the implementation of exact exchange in QE.  These
> improvements have been made as part of NERSC's Exascale Science
> Applications Program, which is an effort to update codes for execution on
> next-generation architectures such as NERSC's upcoming Cori Phase II.  The
> following is a brief overview of these changes, which we are currently in
> the process of testing and debugging.  Depending on our progress, we intend
> to submit these changes as an addition to either QE 5.4 or 6.0.
>
> 1. Parallelization Over Band Pairs
>    We have extended the parallelization of subroutine vexx_k such that
> both of the loops over bands (i.e., "LOOP_ON_PSI_BANDS" and "IBND_LOOP_K")
> are parallelized with respect to band groups.  This improves load
> balancing, and also enables parallelization using larger numbers of band
> groups than was previously possible
>
> 2. Improved OMP Support
>    We have added OMP threading to numerous vector operations within
> exx.f90.  In addition, we have given special priority to enhancing the
> threaded performance of the FFTs.
>
> 3. Implementation of Different and Interchangeable Data Layouts for Local
> and EXX Portions of the Calculation
>    One observation that we have made is that for calculations that utilize
> many band groups, the local portion of the calculation (i.e., everything
> outside of exx.f90) often represents a non-negligible (or even dominant)
> contribution to the total cost of the calculation.  This is largely because
> the local portion of the calculation is duplicated on each band group.  We
> have implemented changes to the code that allow the local portion of the
> code to be parallelized in a manner that is independent of the number of
> band groups, thus avoiding duplication of work.
>    This is the single most significant modification that we have made,
> both in terms of increasing the efficiency of QE, as well as the amount of
> coding work required.  For several test calculations we are finding that
> this change results in more than a factor of two speedup.
>    In terms of code development, the primary challenge of our approach is
> that when the EXX part of the calculation is performed (such as when vexx
> is called), we must change the data structure from the one that is used by
> the local portion of the code to a different data structure that is used by
> the EXX portion of the code.  This change of data structure requires a
> great deal of bookkeeping in order to update arrays like igk, ig_l2g, psi,
> and hpsi.  As a result, we a still the process of making our updated code
> compatible with gamma-point only calculations and with calculations that
> employ multiple k-points.
>
> Sincerely,
> Dr. Taylor Barnes
> Postdoctoral Scholar,
> Lawrence Berkeley National Laboratory
>
>
> _______________________________________________
> Q-e-developers mailing listQ-e-developers at qe-forge.orghttp://qe-forge.org/mailman/listinfo/q-e-developers
>
>
>
> --
> Ph.D. Carlo Cavazzoni
> SuperComputing Applications and Innovation Department
> CINECA - Via Magnanelli 6/3, 40033 Casalecchio di Reno (Bologna)
> Tel: +39 051 6171411  Fax: +39 051 6132198www.cineca.it
>
>
> _______________________________________________
> Q-e-developers mailing list
> Q-e-developers at qe-forge.org
> http://qe-forge.org/mailman/listinfo/q-e-developers
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20160407/79f9e8bd/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: input16
Type: application/octet-stream
Size: 2356 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20160407/79f9e8bd/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: input32
Type: application/octet-stream
Size: 4084 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20160407/79f9e8bd/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: input64
Type: application/octet-stream
Size: 7541 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20160407/79f9e8bd/attachment-0002.obj>