[Q-e-developers] Improvements to the EXX Implementation

[Taylor Barnes]

Hi Nicola,

   We haven't really collaborated with Lin Lin yet, but that might change
in the future.  Up to this point, we have mostly restricted our efforts to
refactoring the code.  After we submit this patch, we will be interested in
making methodological improvements, and Lin Lin's work is definitely
something that we will be looking into.  Thanks for the article!

Best,
Taylor

On Tue, Apr 5, 2016 at 7:15 AM, Nicola Marzari <nicola.marzari at epfl.ch>
wrote:

>
>
>
> Dear Taylor,
>
>
> first of all, great about having active contributors at LBNL!
>
> One curiosity - did you have any interaction with Lin Lin? This
> work seem intriguing: http://arxiv.org/pdf/1601.07159.pdf
>
>                         nicola
>
>
>
>
> On 05/04/2016 06:05, Taylor Barnes wrote:
> > Dear All,
> >
> >     I wanted to inform everyone about some improvements that we have
> > been making at LBNL to the implementation of exact exchange in QE.
> > These improvements have been made as part of NERSC's Exascale Science
> > Applications Program, which is an effort to update codes for execution
> > on next-generation architectures such as NERSC's upcoming Cori Phase
> > II.  The following is a brief overview of these changes, which we are
> > currently in the process of testing and debugging.  Depending on our
> > progress, we intend to submit these changes as an addition to either QE
> > 5.4 or 6.0.
> >
> > 1. Parallelization Over Band Pairs
> >     We have extended the parallelization of subroutine vexx_k such that
> > both of the loops over bands (i.e., "LOOP_ON_PSI_BANDS" and
> > "IBND_LOOP_K") are parallelized with respect to band groups.  This
> > improves load balancing, and also enables parallelization using larger
> > numbers of band groups than was previously possible
> >
> > 2. Improved OMP Support
> >     We have added OMP threading to numerous vector operations within
> > exx.f90.  In addition, we have given special priority to enhancing the
> > threaded performance of the FFTs.
> >
> > 3. Implementation of Different and Interchangeable Data Layouts for
> > Local and EXX Portions of the Calculation
> >     One observation that we have made is that for calculations that
> > utilize many band groups, the local portion of the calculation (i.e.,
> > everything outside of exx.f90) often represents a non-negligible (or
> > even dominant) contribution to the total cost of the calculation.  This
> > is largely because the local portion of the calculation is duplicated on
> > each band group.  We have implemented changes to the code that allow the
> > local portion of the code to be parallelized in a manner that is
> > independent of the number of band groups, thus avoiding duplication of
> work.
> >     This is the single most significant modification that we have made,
> > both in terms of increasing the efficiency of QE, as well as the amount
> > of coding work required.  For several test calculations we are finding
> > that this change results in more than a factor of two speedup.
> >     In terms of code development, the primary challenge of our approach
> > is that when the EXX part of the calculation is performed (such as when
> > vexx is called), we must change the data structure from the one that is
> > used by the local portion of the code to a different data structure that
> > is used by the EXX portion of the code.  This change of data structure
> > requires a great deal of bookkeeping in order to update arrays like igk,
> > ig_l2g, psi, and hpsi.  As a result, we a still the process of making
> > our updated code compatible with gamma-point only calculations and with
> > calculations that employ multiple k-points.
> >
> > Sincerely,
> > Dr. Taylor Barnes
> > Postdoctoral Scholar,
> > Lawrence Berkeley National Laboratory
> >
> >
> > _______________________________________________
> > Q-e-developers mailing list
> > Q-e-developers at qe-forge.org
> > http://qe-forge.org/mailman/listinfo/q-e-developers
> >
>
> --
> ----------------------------------------------------------------------
> Prof Nicola Marzari, Chair of Theory and Simulation of Materials, EPFL
> Director, National Centre for Competence in Research NCCR MARVEL, EPFL
> http://theossrv1.epfl.ch/Main/Contact http://nccr-marvel.ch/en/project
> _______________________________________________
> Q-e-developers mailing list
> Q-e-developers at qe-forge.org
> http://qe-forge.org/mailman/listinfo/q-e-developers
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20160405/4a1e92d5/attachment.html>