[Pw_forum] 62 atom interface relaxation

Louis Fry-Bouriaux ellf at leeds.ac.uk
Wed Mar 15 17:42:37 CET 2017


Hi Cameron,


    I am no expert on QE at all, but I can maybe suggest a few things. It seems QE 5.1 is an old version, I have only used 6.0 so I am unfamiliar with some of the output, particularly the report on memory usage. It could be worth using QE 6.0 at least or 6.1 which is out now to be on par with other users?


As for 'exit signal 9', if this is the OS (linux) then this is SIGKILL which may have been issued due to running out of memory amongst possibly other reasons. Again I am unfamiliar with this version of QE, but ecutwfc seems particularly high compared to what I've seen on this forum, and perusing the 'recommended ecutwfc' in pseudopotential files, I don't know if you need this though? QE 6.0 pwscf tells you the total estimated memory per process, which has been very accurate on 4/8/16/32 cpu scf calculations I have done so far.


Kindest Regards,

Louis

PhD student, University of Leeds, IMP

________________________________
From: pw_forum-bounces at pwscf.org <pw_forum-bounces at pwscf.org> on behalf of Cameron Foss <cjfoss at umass.edu>
Sent: 14 March 2017 22:55:19
To: pw_forum at pwscf.org
Subject: [Pw_forum] 62 atom interface relaxation

Hello all,

I am trying to relax a supercell of an graphene/MoS2 interface and have been running into an exit signal 9 error. I have a 62 atom representation of the interface, which consists of a 5x2 supercell of graphene and a 4x2 supercell of MoS2. Below are the submission script, vc-relax input file, and output file.

>From what I understand, the exit signal 9 is a sort of load balancing error (?). I have worked with much smaller systems (generally unit cells with less than 10 atoms) with relatively good success using 16 MPI processes. However, I have no experience with a simulation of this scale.  I have tried increasing to 32 or 40 MPI processes but only to get the same exit 9 signal. I should note that, in the past I have only made use of PW parallelization and haven't used parallelization over images or k-points as I suspect may be necessary for this simulation.

My first question is that, is my above assessment accurate in that the exit signal 9 is (in this case) a load balancing issue and that it can be resolved with the appropriate resources for PW, images, and k-point parallelization?

And second, I only have access to say 200 or so cpu cores (equating to about 200 MPI processes), is that a reasonable range of resources that can handle a simulation of this size? I apologize for the rather naive question but I have very little experience with simulations of this size and am unfamiliar with how one should scale resources accordingly. Perhaps a better question would be how should one digest the 'Parallelization Info' in the output file in order to get a feel for the best distribution of resources? (perhaps a good resource material would suffice for this question but what I've found on the topic has been difficult to apply to this case).

% submission line : bsub -n 40 < ./myscript
% submission script
#!/bin/sh
#BSUB -J GrMoS2PHlda
#BSUB -o grmos2-lda.out
#BSUB -e grmos2-lda.err
#BSUB -q long
#BSUB -W 96:00
#BSUB -R select[ncpus=20]
#BSUB -R "span[ptile=10]"
#BSUB -R rusage[mem=10000]
#BSUB -L /bin/sh

export OMP_NUM_THREADS=1
module load gcc/4.7.4
module load mvapich2/2.0a

mpirun -n 32 ~/espresso-par/espresso-5.1.2/bin/pw.x <gr_mos2.vc-relax.reduced.in<http://gr_mos2.vc-relax.reduced.in>> gr_mos2.vc-relax.out

%%%%%%%%%%%
pw input file
  1 &control
  2  calculation='vc-relax',
  3  restart_mode='from_scratch',
  4  outdir='/project/uma_zlatan_aksamija/out_grmos2',
  5  prefix='GrMoS2-LDA3'
  6  pseudo_dir = '/home/cf79a/espresso-par/espresso-5.1.2/pseudo',
  7 /
  8 &system
  9  ibrav = 0,
 10  celldm(1) =4.6477,
 11  !celldm(3)=2,
 12  nat=  62,
 13  ntyp= 3,
 14  ecutwfc = 250.0
 15 /
 16 &electrons
 17  mixing_beta = 0.7
 18  conv_thr =  1.0d-8
 19 /
 20 &ions
 21   ion_dynamics='bfgs',
 22 /
 23 &cell
 24   cell_dynamics='bfgs',
 25 /
 26 ATOMIC_SPECIES
 27  C  12.01  C.pw-mt_fhi.UPF
 28  Mo 95.94  Mo.pw-mt_fhi.UPF
 29  S  32.065 S.pw-mt_fhi.UPF
 30 ATOMIC_POSITIONS {alat}
 31   C     0.00000000000   0.00000000000   0.63039440618
 32   C     0.50000000000   0.28867281200   0.63039440618
 33   C     1.00000000000   0.00000000000   0.63039440618
 34   C     1.50000000000   0.28867281200   0.63039440618
 35   C     0.50000000000   0.86602500000   0.63039440618
 36   C     2.00000000000   0.00000000000   0.63039440618
 37   C     2.50000000000   0.28867281200   0.63039440618
 38   C     1.50000000000   0.86602500000   0.63039440618
 39   C     0.00000000000   1.15469781200   0.63039440618
 40   C     1.00000000000   1.15469781200   0.63039440618
 41   C     2.00000000000   1.15469781200   0.63039440618
 42   C     0.00000000000   1.73205000000   0.63039440618
 43   C     1.00000000000   1.73205000000   0.63039440618
 44   C     1.50000000000   2.02072321400   0.63039440618
 45   C     1.00000000000   2.88674861800   0.63039440618
 46   C     0.50000000000   3.75277402100   0.63039440618
 47   C     0.50000000000   2.59807540400   0.63039440618
 48   C     0.00000000000   3.46410080800   0.63039440618
 49   C     0.50000000000   2.02072321400   0.63039440618
 50   C     0.00000000000   2.88674861800   0.63039440618
 51   C     -0.50000000000  3.75277402200   0.63039440618
 52   C     -0.50000000000  2.59807540400   0.63039440618
 53   C     -1.00000000000  3.46410080800   0.63039440618
 54   C     -0.50000000000  2.02072321400   0.63039440618
 55   C     -1.00000000000  2.88674861800   0.63039440618
 56   C     -1.50000000000  3.75277402100   0.63039440618
 57   C     -0.50000000000  0.86602500000   0.63039440618
 58   C     -1.00000000000  1.73205000000   0.63039440618
 59   C     -1.50000000000  2.59807540000   0.63039440618
 60   C     -2.00000000000  3.46410081000   0.63039440618
 61   S     2.00000000000   2.17500376000   0.00000000000
 62   S     2.00000000000   2.17500376000   1.26078881236
 63   S     1.37213049000   3.26250564000   0.00000000000
 64   S     1.37213049000   3.26250564000   1.26078881236
 65   S     3.25573901037   0.00000000000   0.00000000000
 66   S     3.25573901037   0.00000000000   1.26078881236
 67   S     2.62786950519   1.08750188350   0.00000000000
 68   S     2.62786950519   1.08750188350   1.26078881236
 69   S     4.51147802074   0.00000000000   0.00000000000
 70   S     4.51147802074   0.00000000000   1.26078881236
 71   S     3.88360851556   1.08750188350   0.00000000000
 72   S     3.88360851556   1.08750188350   1.26078881236
 73   S     5.76721702400   0.00000000000   0.00000000000
 74   S     5.76721702400   0.00000000000   1.26078881236
 75   S     5.13934752100   1.08750188350   0.00000000000
 76   S     5.13934752100   1.08750188350   1.26078881236
 77   S     3.25573901000   2.17500376000   0.00000000000
 78   S     3.25573901000   2.17500376000   1.26078881236
 79   S     2.62786950000   3.26250564000   0.00000000000
 80   S     2.62786950000   3.26250564000   1.26078881236
 81   S     4.51147801000   2.17500376000   0.00000000000
 82   S     4.51147801000   2.17500376000   1.26078881236
 83   S     3.88360850000   3.26250564000   0.00000000000
 84   S     3.88360850000   3.26250564000   1.26078881236
 85   Mo    3.88360851555   0.36249771126   0.63039440618
 86   Mo    5.13934752593   0.36249771126   0.63039440618
 87   Mo    3.25573901037   1.44999959476   0.63039440618
 88   Mo    4.51147802074   1.44999959476   0.63039440618
 89   Mo    2.62786950000   2.53750147400   0.63039440618
 90   Mo    3.88360850500   2.53750147400   0.63039440618
 91   Mo    2.00000000000   3.62500335400   0.63039440618
 92   Mo    3.25573899500   3.62500335400   0.63039440618
 93 K_POINTS AUTOMATIC
 94  27 27 1 1 1 1
 95 CELL_PARAMETERS alat
 96  6.5 0.00000000000 0.0
 97 -2.5 4.330127018900 0.0
 98  0.0 0.000000000000 6.0
%%%%%%%%%%%%%

And the output file:
  1
  2      Program PWSCF v.5.1.2 starts on 10Mar2017 at 18:29:19
  3
  4      This program is part of the open-source Quantum ESPRESSO suite
  5      for quantum simulation of materials; please cite
  6          "P. Giannozzi et al., J. Phys.:Condens. Matter 21 395502 (2009);
  7           URL http://www.quantum-espresso.org",
  8      in publications or presentations arising from this work. More details at
  9      http://www.quantum-espresso.org/quote
 10
 11      Parallel version (MPI & OpenMP), running on      32 processor cores
 12      Number of MPI processes:                32
 13      Threads/MPI process:                     1
 14      R & G space division:  proc/nbgrp/npool/nimage =      32
 15      Waiting for input...
 16      Reading input from standard input
 17
 18      Current dimensions of program PWSCF are:
 19      Max number of different atomic species (ntypx) = 10
 20      Max number of k-points (npk) = 400000
 21      Max angular momentum in pseudopotentials (lmaxx) =  3
 22                file C.pw-mt_fhi.UPF: wavefunction(s)  4f renormalized
 23                file Mo.pw-mt_fhi.UPF: wavefunction(s)  4f renormalized
 24                file S.pw-mt_fhi.UPF: wavefunction(s)  4f renormalized
 25
 26      Subspace diagonalization in iterative solution of the eigenvalue problem:
 27      a serial algorithm will be used
 28
 29
 30      Parallelization info
 31      --------------------
 32      sticks:   dense  smooth     PW     G-vecs:    dense   smooth      PW
 33      Min        1511    1511    391               282925   282925   37169
 34      Max        1512    1512    392               282930   282930   37173
 35      Sum       48359   48359  12513              9053647  9053647 1189491
 36
 37
 38
 39      bravais-lattice index     =            0
 40      lattice parameter (alat)  =       4.6477  a.u.
 41      unit-cell volume          =   16954.2860 (a.u.)^3
 42      number of atoms/cell      =           62
 43      number of atomic types    =            3
 44      number of electrons       =       312.00
 45      number of Kohn-Sham states=          156
 46      kinetic-energy cutoff     =     250.0000  Ry
 47      charge density cutoff     =    1000.0000  Ry
 48      convergence threshold     =      1.0E-08
 49      mixing beta               =       0.7000
 50      number of iterations used =            8  plain     mixing
 51      Exchange-correlation      = SLA-PW ( 1  4  0  0 0 0)
 52      nstep                     =           50
 53
 54
 55      celldm(1)=   4.647700  celldm(2)=   0.000000  celldm(3)=   0.000000
 56      celldm(4)=   0.000000  celldm(5)=   0.000000  celldm(6)=   0.000000
 57
 58      crystal axes: (cart. coord. in units of alat)
 59                a(1) = (   6.500000   0.000000   0.000000 )
 60                a(2) = (  -2.500000   4.330127   0.000000 )
 61                a(3) = (   0.000000   0.000000   6.000000 )
 62
 63      reciprocal axes: (cart. coord. in units 2 pi/alat)
 64                b(1) = (  0.153846  0.088823 -0.000000 )
 65                b(2) = (  0.000000  0.230940  0.000000 )
 66                b(3) = (  0.000000 -0.000000  0.166667 )
 67
 68
 69      PseudoPot. # 1 for  C read from file:
 70      /home/cf79a/espresso-par/espresso-5.1.2/pseudo/C.pw-mt_fhi.UPF
 71      MD5 check sum: 13ff76dfb29e90984447383659675a25
 72      Pseudo is Norm-conserving, Zval =  4.0
 73      Generated using FHI98PP, converted with fhi2upf.x v.5.0.1
 74      Using radial grid of  461 points,  3 beta functions with:
 75                 l(1) =   0
 76                 l(2) =   1
 77                 l(3) =   3
 78
 79      PseudoPot. # 2 for Mo read from file:
 80      /home/cf79a/espresso-par/espresso-5.1.2/pseudo/Mo.pw-mt_fhi.UPF
 81      MD5 check sum: a86585854cc5d78a2dd10559b26ab9a5
 82      Pseudo is Norm-conserving, Zval =  6.0
 83      Generated using FHI98PP, converted with fhi2upf.x v.5.0.1
 84      Using radial grid of  541 points,  3 beta functions with:
 85                 l(1) =   0
 86                 l(2) =   2
 87                 l(3) =   3
 88
 89      PseudoPot. # 3 for  S read from file:
 90      /home/cf79a/espresso-par/espresso-5.1.2/pseudo/S.pw-mt_fhi.UPF
 91      MD5 check sum: d54d28406084323b3c894f38c8f5e215
 92      Pseudo is Norm-conserving, Zval =  6.0
 93      Generated using FHI98PP, converted with fhi2upf.x v.5.0.1
 94      Using radial grid of  501 points,  3 beta functions with:
 95                 l(1) =   0
 96                 l(2) =   1
 97                 l(3) =   3
 98
 99      atomic species   valence    mass     pseudopotential
100         C              4.00    12.01000      C( 1.00)
101         Mo             6.00    95.94000     Mo( 1.00)
102         S              6.00    32.06500      S( 1.00)
103
104      No symmetry found
105           (note:  1 additional sym.ops. were found but ignored
106            their fractional translations are incommensurate with FFT grid)
107
108
109    Cartesian axes
110
111      site n.     atom                  positions (alat units)
112          1           C   tau(   1) = (   0.0000000   0.0000000   0.6303944  )
113          2           C   tau(   2) = (   0.5000000   0.2886728   0.6303944  )
114          3           C   tau(   3) = (   1.0000000   0.0000000   0.6303944  )
................skipped for sake of redundancy
172         61           Mo  tau(  61) = (   2.0000000   3.6250034   0.6303944  )
173         62           Mo  tau(  62) = (   3.2557390   3.6250034   0.6303944  )
174
175      number of k points=   365
176
177      Number of k-points >= 100: set verbosity='high' to print them.
178
179      Dense  grid:  9053647 G-vectors     FFT dimensions: ( 320, 240, 288)
180
181      Largest allocated arrays     est. size (Mb)     dimensions
182         Kohn-Sham Wavefunctions        84.31 Mb     (   35420,  156)
183         NL pseudopotentials           377.25 Mb     (   35420,  698)
184         Each V/rho on FFT grid         10.55 Mb     (  691200)
185         Each G-vector array             2.16 Mb     (  282930)
186         G-vector shells                 2.16 Mb     (  282930)
187      Largest temporary arrays     est. size (Mb)     dimensions
188         Auxiliary wavefunctions       337.25 Mb     (   35420,  624)
189         Each subspace H/S matrix        5.94 Mb     (     624,  624)
190         Each <psi_i|beta_j> matrix      1.66 Mb     (     698,  156)
191         Arrays for rho mixing          84.38 Mb     (  691200,    8)
192
193      Initial potential from superposition of free atoms
194      Check: negative starting charge=   -0.137438
195
196      starting charge  311.98520, renormalised to  312.00000
197
198      negative rho (up, down):  1.374E-01 0.000E+00
199      Starting wfc are  992 randomized atomic wfcs
200
201 ===================================================================================
202 =   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
203 =   EXIT CODE: 9
204 =   CLEANING UP REMAINING PROCESSES
205 =   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
206 ===================================================================================

Best regards,
Cameron


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20170315/b508871b/attachment.html>


More information about the users mailing list