[Pw_forum] 62 atom interface relaxation
Louis Fry-Bouriaux
ellf at leeds.ac.uk
Wed Mar 15 17:42:37 CET 2017
Hi Cameron,
I am no expert on QE at all, but I can maybe suggest a few things. It seems QE 5.1 is an old version, I have only used 6.0 so I am unfamiliar with some of the output, particularly the report on memory usage. It could be worth using QE 6.0 at least or 6.1 which is out now to be on par with other users?
As for 'exit signal 9', if this is the OS (linux) then this is SIGKILL which may have been issued due to running out of memory amongst possibly other reasons. Again I am unfamiliar with this version of QE, but ecutwfc seems particularly high compared to what I've seen on this forum, and perusing the 'recommended ecutwfc' in pseudopotential files, I don't know if you need this though? QE 6.0 pwscf tells you the total estimated memory per process, which has been very accurate on 4/8/16/32 cpu scf calculations I have done so far.
Kindest Regards,
Louis
PhD student, University of Leeds, IMP
________________________________
From: pw_forum-bounces at pwscf.org <pw_forum-bounces at pwscf.org> on behalf of Cameron Foss <cjfoss at umass.edu>
Sent: 14 March 2017 22:55:19
To: pw_forum at pwscf.org
Subject: [Pw_forum] 62 atom interface relaxation
Hello all,
I am trying to relax a supercell of an graphene/MoS2 interface and have been running into an exit signal 9 error. I have a 62 atom representation of the interface, which consists of a 5x2 supercell of graphene and a 4x2 supercell of MoS2. Below are the submission script, vc-relax input file, and output file.
>From what I understand, the exit signal 9 is a sort of load balancing error (?). I have worked with much smaller systems (generally unit cells with less than 10 atoms) with relatively good success using 16 MPI processes. However, I have no experience with a simulation of this scale. I have tried increasing to 32 or 40 MPI processes but only to get the same exit 9 signal. I should note that, in the past I have only made use of PW parallelization and haven't used parallelization over images or k-points as I suspect may be necessary for this simulation.
My first question is that, is my above assessment accurate in that the exit signal 9 is (in this case) a load balancing issue and that it can be resolved with the appropriate resources for PW, images, and k-point parallelization?
And second, I only have access to say 200 or so cpu cores (equating to about 200 MPI processes), is that a reasonable range of resources that can handle a simulation of this size? I apologize for the rather naive question but I have very little experience with simulations of this size and am unfamiliar with how one should scale resources accordingly. Perhaps a better question would be how should one digest the 'Parallelization Info' in the output file in order to get a feel for the best distribution of resources? (perhaps a good resource material would suffice for this question but what I've found on the topic has been difficult to apply to this case).
% submission line : bsub -n 40 < ./myscript
% submission script
#!/bin/sh
#BSUB -J GrMoS2PHlda
#BSUB -o grmos2-lda.out
#BSUB -e grmos2-lda.err
#BSUB -q long
#BSUB -W 96:00
#BSUB -R select[ncpus=20]
#BSUB -R "span[ptile=10]"
#BSUB -R rusage[mem=10000]
#BSUB -L /bin/sh
export OMP_NUM_THREADS=1
module load gcc/4.7.4
module load mvapich2/2.0a
mpirun -n 32 ~/espresso-par/espresso-5.1.2/bin/pw.x <gr_mos2.vc-relax.reduced.in<http://gr_mos2.vc-relax.reduced.in>> gr_mos2.vc-relax.out
%%%%%%%%%%%
pw input file
1 &control
2 calculation='vc-relax',
3 restart_mode='from_scratch',
4 outdir='/project/uma_zlatan_aksamija/out_grmos2',
5 prefix='GrMoS2-LDA3'
6 pseudo_dir = '/home/cf79a/espresso-par/espresso-5.1.2/pseudo',
7 /
8 &system
9 ibrav = 0,
10 celldm(1) =4.6477,
11 !celldm(3)=2,
12 nat= 62,
13 ntyp= 3,
14 ecutwfc = 250.0
15 /
16 &electrons
17 mixing_beta = 0.7
18 conv_thr = 1.0d-8
19 /
20 &ions
21 ion_dynamics='bfgs',
22 /
23 &cell
24 cell_dynamics='bfgs',
25 /
26 ATOMIC_SPECIES
27 C 12.01 C.pw-mt_fhi.UPF
28 Mo 95.94 Mo.pw-mt_fhi.UPF
29 S 32.065 S.pw-mt_fhi.UPF
30 ATOMIC_POSITIONS {alat}
31 C 0.00000000000 0.00000000000 0.63039440618
32 C 0.50000000000 0.28867281200 0.63039440618
33 C 1.00000000000 0.00000000000 0.63039440618
34 C 1.50000000000 0.28867281200 0.63039440618
35 C 0.50000000000 0.86602500000 0.63039440618
36 C 2.00000000000 0.00000000000 0.63039440618
37 C 2.50000000000 0.28867281200 0.63039440618
38 C 1.50000000000 0.86602500000 0.63039440618
39 C 0.00000000000 1.15469781200 0.63039440618
40 C 1.00000000000 1.15469781200 0.63039440618
41 C 2.00000000000 1.15469781200 0.63039440618
42 C 0.00000000000 1.73205000000 0.63039440618
43 C 1.00000000000 1.73205000000 0.63039440618
44 C 1.50000000000 2.02072321400 0.63039440618
45 C 1.00000000000 2.88674861800 0.63039440618
46 C 0.50000000000 3.75277402100 0.63039440618
47 C 0.50000000000 2.59807540400 0.63039440618
48 C 0.00000000000 3.46410080800 0.63039440618
49 C 0.50000000000 2.02072321400 0.63039440618
50 C 0.00000000000 2.88674861800 0.63039440618
51 C -0.50000000000 3.75277402200 0.63039440618
52 C -0.50000000000 2.59807540400 0.63039440618
53 C -1.00000000000 3.46410080800 0.63039440618
54 C -0.50000000000 2.02072321400 0.63039440618
55 C -1.00000000000 2.88674861800 0.63039440618
56 C -1.50000000000 3.75277402100 0.63039440618
57 C -0.50000000000 0.86602500000 0.63039440618
58 C -1.00000000000 1.73205000000 0.63039440618
59 C -1.50000000000 2.59807540000 0.63039440618
60 C -2.00000000000 3.46410081000 0.63039440618
61 S 2.00000000000 2.17500376000 0.00000000000
62 S 2.00000000000 2.17500376000 1.26078881236
63 S 1.37213049000 3.26250564000 0.00000000000
64 S 1.37213049000 3.26250564000 1.26078881236
65 S 3.25573901037 0.00000000000 0.00000000000
66 S 3.25573901037 0.00000000000 1.26078881236
67 S 2.62786950519 1.08750188350 0.00000000000
68 S 2.62786950519 1.08750188350 1.26078881236
69 S 4.51147802074 0.00000000000 0.00000000000
70 S 4.51147802074 0.00000000000 1.26078881236
71 S 3.88360851556 1.08750188350 0.00000000000
72 S 3.88360851556 1.08750188350 1.26078881236
73 S 5.76721702400 0.00000000000 0.00000000000
74 S 5.76721702400 0.00000000000 1.26078881236
75 S 5.13934752100 1.08750188350 0.00000000000
76 S 5.13934752100 1.08750188350 1.26078881236
77 S 3.25573901000 2.17500376000 0.00000000000
78 S 3.25573901000 2.17500376000 1.26078881236
79 S 2.62786950000 3.26250564000 0.00000000000
80 S 2.62786950000 3.26250564000 1.26078881236
81 S 4.51147801000 2.17500376000 0.00000000000
82 S 4.51147801000 2.17500376000 1.26078881236
83 S 3.88360850000 3.26250564000 0.00000000000
84 S 3.88360850000 3.26250564000 1.26078881236
85 Mo 3.88360851555 0.36249771126 0.63039440618
86 Mo 5.13934752593 0.36249771126 0.63039440618
87 Mo 3.25573901037 1.44999959476 0.63039440618
88 Mo 4.51147802074 1.44999959476 0.63039440618
89 Mo 2.62786950000 2.53750147400 0.63039440618
90 Mo 3.88360850500 2.53750147400 0.63039440618
91 Mo 2.00000000000 3.62500335400 0.63039440618
92 Mo 3.25573899500 3.62500335400 0.63039440618
93 K_POINTS AUTOMATIC
94 27 27 1 1 1 1
95 CELL_PARAMETERS alat
96 6.5 0.00000000000 0.0
97 -2.5 4.330127018900 0.0
98 0.0 0.000000000000 6.0
%%%%%%%%%%%%%
And the output file:
1
2 Program PWSCF v.5.1.2 starts on 10Mar2017 at 18:29:19
3
4 This program is part of the open-source Quantum ESPRESSO suite
5 for quantum simulation of materials; please cite
6 "P. Giannozzi et al., J. Phys.:Condens. Matter 21 395502 (2009);
7 URL http://www.quantum-espresso.org",
8 in publications or presentations arising from this work. More details at
9 http://www.quantum-espresso.org/quote
10
11 Parallel version (MPI & OpenMP), running on 32 processor cores
12 Number of MPI processes: 32
13 Threads/MPI process: 1
14 R & G space division: proc/nbgrp/npool/nimage = 32
15 Waiting for input...
16 Reading input from standard input
17
18 Current dimensions of program PWSCF are:
19 Max number of different atomic species (ntypx) = 10
20 Max number of k-points (npk) = 400000
21 Max angular momentum in pseudopotentials (lmaxx) = 3
22 file C.pw-mt_fhi.UPF: wavefunction(s) 4f renormalized
23 file Mo.pw-mt_fhi.UPF: wavefunction(s) 4f renormalized
24 file S.pw-mt_fhi.UPF: wavefunction(s) 4f renormalized
25
26 Subspace diagonalization in iterative solution of the eigenvalue problem:
27 a serial algorithm will be used
28
29
30 Parallelization info
31 --------------------
32 sticks: dense smooth PW G-vecs: dense smooth PW
33 Min 1511 1511 391 282925 282925 37169
34 Max 1512 1512 392 282930 282930 37173
35 Sum 48359 48359 12513 9053647 9053647 1189491
36
37
38
39 bravais-lattice index = 0
40 lattice parameter (alat) = 4.6477 a.u.
41 unit-cell volume = 16954.2860 (a.u.)^3
42 number of atoms/cell = 62
43 number of atomic types = 3
44 number of electrons = 312.00
45 number of Kohn-Sham states= 156
46 kinetic-energy cutoff = 250.0000 Ry
47 charge density cutoff = 1000.0000 Ry
48 convergence threshold = 1.0E-08
49 mixing beta = 0.7000
50 number of iterations used = 8 plain mixing
51 Exchange-correlation = SLA-PW ( 1 4 0 0 0 0)
52 nstep = 50
53
54
55 celldm(1)= 4.647700 celldm(2)= 0.000000 celldm(3)= 0.000000
56 celldm(4)= 0.000000 celldm(5)= 0.000000 celldm(6)= 0.000000
57
58 crystal axes: (cart. coord. in units of alat)
59 a(1) = ( 6.500000 0.000000 0.000000 )
60 a(2) = ( -2.500000 4.330127 0.000000 )
61 a(3) = ( 0.000000 0.000000 6.000000 )
62
63 reciprocal axes: (cart. coord. in units 2 pi/alat)
64 b(1) = ( 0.153846 0.088823 -0.000000 )
65 b(2) = ( 0.000000 0.230940 0.000000 )
66 b(3) = ( 0.000000 -0.000000 0.166667 )
67
68
69 PseudoPot. # 1 for C read from file:
70 /home/cf79a/espresso-par/espresso-5.1.2/pseudo/C.pw-mt_fhi.UPF
71 MD5 check sum: 13ff76dfb29e90984447383659675a25
72 Pseudo is Norm-conserving, Zval = 4.0
73 Generated using FHI98PP, converted with fhi2upf.x v.5.0.1
74 Using radial grid of 461 points, 3 beta functions with:
75 l(1) = 0
76 l(2) = 1
77 l(3) = 3
78
79 PseudoPot. # 2 for Mo read from file:
80 /home/cf79a/espresso-par/espresso-5.1.2/pseudo/Mo.pw-mt_fhi.UPF
81 MD5 check sum: a86585854cc5d78a2dd10559b26ab9a5
82 Pseudo is Norm-conserving, Zval = 6.0
83 Generated using FHI98PP, converted with fhi2upf.x v.5.0.1
84 Using radial grid of 541 points, 3 beta functions with:
85 l(1) = 0
86 l(2) = 2
87 l(3) = 3
88
89 PseudoPot. # 3 for S read from file:
90 /home/cf79a/espresso-par/espresso-5.1.2/pseudo/S.pw-mt_fhi.UPF
91 MD5 check sum: d54d28406084323b3c894f38c8f5e215
92 Pseudo is Norm-conserving, Zval = 6.0
93 Generated using FHI98PP, converted with fhi2upf.x v.5.0.1
94 Using radial grid of 501 points, 3 beta functions with:
95 l(1) = 0
96 l(2) = 1
97 l(3) = 3
98
99 atomic species valence mass pseudopotential
100 C 4.00 12.01000 C( 1.00)
101 Mo 6.00 95.94000 Mo( 1.00)
102 S 6.00 32.06500 S( 1.00)
103
104 No symmetry found
105 (note: 1 additional sym.ops. were found but ignored
106 their fractional translations are incommensurate with FFT grid)
107
108
109 Cartesian axes
110
111 site n. atom positions (alat units)
112 1 C tau( 1) = ( 0.0000000 0.0000000 0.6303944 )
113 2 C tau( 2) = ( 0.5000000 0.2886728 0.6303944 )
114 3 C tau( 3) = ( 1.0000000 0.0000000 0.6303944 )
................skipped for sake of redundancy
172 61 Mo tau( 61) = ( 2.0000000 3.6250034 0.6303944 )
173 62 Mo tau( 62) = ( 3.2557390 3.6250034 0.6303944 )
174
175 number of k points= 365
176
177 Number of k-points >= 100: set verbosity='high' to print them.
178
179 Dense grid: 9053647 G-vectors FFT dimensions: ( 320, 240, 288)
180
181 Largest allocated arrays est. size (Mb) dimensions
182 Kohn-Sham Wavefunctions 84.31 Mb ( 35420, 156)
183 NL pseudopotentials 377.25 Mb ( 35420, 698)
184 Each V/rho on FFT grid 10.55 Mb ( 691200)
185 Each G-vector array 2.16 Mb ( 282930)
186 G-vector shells 2.16 Mb ( 282930)
187 Largest temporary arrays est. size (Mb) dimensions
188 Auxiliary wavefunctions 337.25 Mb ( 35420, 624)
189 Each subspace H/S matrix 5.94 Mb ( 624, 624)
190 Each <psi_i|beta_j> matrix 1.66 Mb ( 698, 156)
191 Arrays for rho mixing 84.38 Mb ( 691200, 8)
192
193 Initial potential from superposition of free atoms
194 Check: negative starting charge= -0.137438
195
196 starting charge 311.98520, renormalised to 312.00000
197
198 negative rho (up, down): 1.374E-01 0.000E+00
199 Starting wfc are 992 randomized atomic wfcs
200
201 ===================================================================================
202 = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
203 = EXIT CODE: 9
204 = CLEANING UP REMAINING PROCESSES
205 = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
206 ===================================================================================
Best regards,
Cameron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20170315/b508871b/attachment.html>
More information about the users
mailing list