[Q-e-developers] Behavior of band-parallelization in QE 6.1

[Ryan McAvoy]

Hello,

I am Ryan L. McAvoy, a PhD student in Giulia Galli's group. I am trying to
use the band parallelization for hybrids in QE 6.1 and I am finding
unexpected behavior. I have created a test case on a small test system (the
zinc dimer) to illustrate the following behavior. I have attached those
files.


   1. The output informing the user that there is band-parallelization for
   a hybrid functional appears to have been broken as changing the number of
   band groups with -nbgrp does not trigger the output that should occur from
   subroutine parallel_info() in environment.f90, which would indicate it
   believes nbgrp to be 1. This may be triggered by the statement
   "mp_start_bands(1 ,...." at line 94 of mp_global.f90 as that I have checked
   that "nband_" is the correct value after "CALL get_command_line()" in
   mp_global.f90
   2. The number of planewaves appears to be distributed over all of the
   processors even at large numbers of "nband_". I have checked that this is
   more than an output error by printing the lda(npw) at each run of h_psi and
   it exactly conforms to what one would expect by dividing the total number
   of planewaves  by the number of processors(plus a factor of 1/2 for gamma
   tricks).
   3. Behavior 2 prevents me from scaling to as large a number of
   processors as I could with QE 6.0. As using QE 6.0 hybrids on C60, I could
   run on 8000+ processors on the BGQ machine Cetus at Argonne National Lab
   but with QE 6.1 the output says that it has run out of planewaves even with
   a large number of band groups(I have demonstrated this behavior below on
   the zinc dimer on 640 Intel processors to aid reproducibility)

Is #2 the intended behavior for this new parallelization method?

Thank you for your time and attention to this matter,
Ryan L. McAvoy

...................................................................................................................................................................................


My run scripts are of the form

module load mkl/11.2
module load intelmpi/5.0+intel-15.0

QE_BIN_DIR=PUTPATHHERE/qe-6.1/bin

export MPI_TASKS=$SLURM_NTASKS

exe=${QE_BIN_DIR}/pw.x

export OMP_NUM_THREADS=1

nband=10
mpirun -n $MPI_TASKS ${exe} -nb $nband <  ${fileVal}.in >
 ${fileVal}_nband${nband}_nproc${MPI_TASKS}.out
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20170710/edeef210/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pw_scf.in
Type: application/octet-stream
Size: 521 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20170710/edeef210/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pw_scf_nband10_nproc640.out
Type: application/octet-stream
Size: 82204 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20170710/edeef210/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pw_scf_nband1_nproc320.out
Type: application/octet-stream
Size: 32291 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20170710/edeef210/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pw_scf_nband2_nproc320.out
Type: application/octet-stream
Size: 32644 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20170710/edeef210/attachment-0003.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pw_scf_nband5_nproc320.out
Type: application/octet-stream
Size: 32644 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20170710/edeef210/attachment-0004.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pw_scf_nband10_nproc320.out
Type: application/octet-stream
Size: 32644 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20170710/edeef210/attachment-0005.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pw_scf_nband1_nproc20.out
Type: application/octet-stream
Size: 29717 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20170710/edeef210/attachment-0006.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pw_scf_nband2_nproc20.out
Type: application/octet-stream
Size: 30069 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20170710/edeef210/attachment-0007.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pw_scf_nband5_nproc20.out
Type: application/octet-stream
Size: 30068 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20170710/edeef210/attachment-0008.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pw_scf_nband10_nproc20.out
Type: application/octet-stream
Size: 30068 bytes
Desc: not available
URL: <http://lists.quantum-espresso.org/pipermail/developers/attachments/20170710/edeef210/attachment-0009.obj>