[QE-users] Runtime GPU memory issue of q-e-gpu- -6.4a1
xh14
xh14 at rice.edu
Thu Sep 5 21:55:01 CEST 2019
Hi, We have a runtime question of QE-6.4a1.
When compiling, we used the -D options as: -D_CUDA -D__PGI -D__FFTW -D__MKL
We loaded modules as GCCcore/8.3.0, CUDA/10.1.168, PGI/19.4 and Intel/2019a.
When running a typical example, it can pass the CPU calculations, but
meet a runtime memory issue, as the message like:
"line 176: cudaLaunchKernel returned status 700: an illegal memory
access was encountered".
We do not know where the "line 176" refers to, and how to fix it.
We used the slurm script, requesting 1 GPU (Volta 100), and 16 openmp
threads on 16 CPUs.
The attached file is the slurm output.
Please help us of how to do next about this issue.
Thanks a lot!
Xiaoqin huang
Rice University
Hosuton, Texas 77030
USA
[text/plain]
-------------- next part --------------
Program PWSCF v.6.4rc starts on 5Sep2019 at 12: 3: 2
This program is part of the open-source Quantum ESPRESSO suite
for quantum simulation of materials; please cite
"P. Giannozzi et al., J. Phys.:Condens. Matter 21 395502 (2009);
"P. Giannozzi et al., J. Phys.:Condens. Matter 29 465901 (2017);
URL http://www.quantum-espresso.org",
in publications or presentations arising from this work. More details at
http://www.quantum-espresso.org/quote
Serial multi-threaded version, running on 16 processor cores
Reading input from bench.in
Current dimensions of program PWSCF are:
Max number of different atomic species (ntypx) = 10
Max number of k-points (npk) = 40000
Max angular momentum in pseudopotentials (lmaxx) = 3
G-vector sticks info
--------------------
sticks: dense smooth PW G-vecs: dense smooth PW
Sum 7127 2845 765 1939571 490659 68121
Using Slab Decomposition
bravais-lattice index = 8
lattice parameter (alat) = 19.1980 a.u.
unit-cell volume = 24510.8429 (a.u.)^3
number of atoms/cell = 183
number of atomic types = 9
number of electrons = 900.00
number of Kohn-Sham states= 540
kinetic-energy cutoff = 28.0000 Ry
charge density cutoff = 280.0000 Ry
convergence threshold = 1.0E-06
mixing beta = 0.3000
number of iterations used = 8 plain mixing
Exchange-correlation = SLA PW PBE PBE ( 1 4 3 4 0 0)
GPU acceleration is ACTIVE.
celldm(1)= 19.197998 celldm(2)= 0.866025 celldm(3)= 4.000000
celldm(4)= 0.000000 celldm(5)= 0.000000 celldm(6)= 0.000000
crystal axes: (cart. coord. in units of alat)
a(1) = ( 1.000000 0.000000 0.000000 )
a(2) = ( 0.000000 0.866025 0.000000 )
a(3) = ( 0.000000 0.000000 4.000000 )
reciprocal axes: (cart. coord. in units 2 pi/alat)
b(1) = ( 1.000000 0.000000 0.000000 )
b(2) = ( 0.000000 1.154701 0.000000 )
b(3) = ( 0.000000 0.000000 0.250000 )
PseudoPot. # 1 for Au read from file:
./Au.pbe-van_ak.UPF
MD5 check sum: 9040a7dcfae4ad769d923573f5dfd312
Pseudo is Ultrasoft, Zval = 11.0
Generated by new atomic code, or converted to UPF format
Using radial grid of 895 points, 6 beta functions with:
l(1) = 0
l(2) = 0
l(3) = 1
l(4) = 1
l(5) = 2
l(6) = 2
Q(r) pseudized with 8 coefficients, rinner = 1.600 1.800 1.600
1.600 1.800
PseudoPot. # 2 for H read from file:
./H.pbe-van_bm.UPF
MD5 check sum: b5ee9d7bfd3a09ee7748110bda690ff7
Pseudo is Ultrasoft, Zval = 1.0
Generated by new atomic code, or converted to UPF format
Using radial grid of 615 points, 1 beta functions with:
l(1) = 0
Q(r) pseudized with 8 coefficients, rinner = 0.800
PseudoPot. # 3 for C read from file:
./C.pbe-van_bm.UPF
MD5 check sum: 1a69bf6b8db32088f5b2163dbdb77a27
Pseudo is Ultrasoft, Zval = 4.0
Generated by new atomic code, or converted to UPF format
Using radial grid of 721 points, 4 beta functions with:
l(1) = 0
l(2) = 0
l(3) = 1
l(4) = 1
Q(r) pseudized with 8 coefficients, rinner = 0.800 0.800 0.800
PseudoPot. # 4 for N read from file:
./N.pbe-van_ak.UPF
MD5 check sum: 1179dd96bccda41bd78de041b702c012
Pseudo is Ultrasoft, Zval = 5.0
Generated by new atomic code, or converted to UPF format
Using radial grid of 729 points, 4 beta functions with:
l(1) = 0
l(2) = 0
l(3) = 1
l(4) = 1
Q(r) pseudized with 8 coefficients, rinner = 0.800 0.800 0.800
PseudoPot. # 5 for O read from file:
./O.pbe-van_ak.UPF
MD5 check sum: 78c19812648e8988a0fe644b0f25305e
Pseudo is Ultrasoft, Zval = 6.0
Generated by new atomic code, or converted to UPF format
Using radial grid of 737 points, 4 beta functions with:
l(1) = 0
l(2) = 0
l(3) = 1
l(4) = 1
Q(r) pseudized with 8 coefficients, rinner = 0.800 0.800 0.800
PseudoPot. # 6 for S read from file:
./S.pbe-van_bm.UPF
MD5 check sum: 5981569f150ce26f656387b858196b6a
Pseudo is Ultrasoft, Zval = 6.0
Generated by new atomic code, or converted to UPF format
Using radial grid of 811 points, 6 beta functions with:
l(1) = 0
l(2) = 0
l(3) = 1
l(4) = 1
l(5) = 2
l(6) = 2
Q(r) pseudized with 8 coefficients, rinner = 1.100 1.100 1.100
1.100 1.100
PseudoPot. # 7 for Fe read from file:
./Fe.pbe-sp-van.UPF
MD5 check sum: 077d28d745de5d3d6a1b066eb4f7e31e
Pseudo is Ultrasoft, Zval = 16.0
Generated by new atomic code, or converted to UPF format
Using radial grid of 601 points, 6 beta functions with:
l(1) = 0
l(2) = 0
l(3) = 1
l(4) = 1
l(5) = 2
l(6) = 2
Q(r) pseudized with 8 coefficients, rinner = 1.000 1.000 1.000
1.000 1.000
PseudoPot. # 8 for Na read from file:
./Na.pbe-sp-van_ak.UPF
MD5 check sum: b3e39564eed18be2046c1643d42efc62
Pseudo is Ultrasoft, Zval = 9.0
Generated by new atomic code, or converted to UPF format
Using radial grid of 819 points, 6 beta functions with:
l(1) = 0
l(2) = 0
l(3) = 1
l(4) = 1
l(5) = 2
l(6) = 2
Q(r) pseudized with 8 coefficients, rinner = 0.900 0.900 0.900
0.900 0.900
PseudoPot. # 9 for Cl read from file:
./Cl.pbe-n-van.UPF
MD5 check sum: 37d17b79490292b7cf0a9c9387775579
Pseudo is Ultrasoft + core correction, Zval = 7.0
Generated by new atomic code, or converted to UPF format
Using radial grid of 909 points, 4 beta functions with:
l(1) = 0
l(2) = 0
l(3) = 1
l(4) = 1
Q(r) pseudized with 8 coefficients, rinner = 0.900 0.900 0.900
atomic species valence mass pseudopotential
Au 11.00 196.96654 Au( 1.00)
H 1.00 1.00000 H ( 1.00)
C 4.00 12.01000 C ( 1.00)
N 5.00 14.01000 N ( 1.00)
O 6.00 15.99000 O ( 1.00)
S 6.00 32.07000 S ( 1.00)
Fe 16.00 55.85000 Fe( 1.00)
Na 9.00 22.99000 Na( 1.00)
Cl 7.00 35.45000 Cl( 1.00)
No symmetry found
Cartesian axes
site n. atom positions (alat units)
1 Fe tau( 1) = ( 0.3399893 0.4783868 1.9044914 )
2 C tau( 2) = ( 0.1844644 0.6060551 1.9201423 )
3 C tau( 3) = ( 0.2797480 0.6563546 1.8288944 )
4 C tau( 4) = ( 0.2950052 0.5608741 1.7258346 )
5 C tau( 5) = ( 0.2091712 0.4521051 1.7535928 )
6 C tau( 6) = ( 0.1407599 0.4799617 1.8738785 )
7 C tau( 7) = ( 0.4048570 0.4422617 2.0926962 )
8 C tau( 8) = ( 0.3807408 0.3213854 2.0245802 )
9 C tau( 9) = ( 0.4622437 0.3190230 1.9085271 )
10 C tau( 10) = ( 0.5365610 0.4404899 1.9059679 )
11 C tau( 11) = ( 0.5012234 0.5154962 2.0199539 )
12 H tau( 12) = ( 0.3562308 0.4745479 2.1824676 )
13 H tau( 13) = ( 0.3119358 0.2442135 2.0522400 )
14 H tau( 14) = ( 0.6085159 0.4698231 1.8321427 )
15 H tau( 15) = ( 0.5385297 0.6128470 2.0448575 )
16 H tau( 16) = ( 0.1530641 0.6542875 2.0106027 )
17 H tau( 17) = ( 0.3325084 0.7492758 1.8376550 )
18 H tau( 18) = ( 0.3615462 0.5674691 1.6420676 )
19 H tau( 19) = ( 0.1999184 0.3627274 1.6952217 )
20 H tau( 20) = ( 0.0701831 0.4157831 1.9226031 )
21 Au tau( 21) = ( 0.0724471 0.0610288 0.8004612 )
22 Au tau( 22) = ( 0.5816436 0.0569930 0.7931771 )
23 Au tau( 23) = ( 0.0714627 0.3557387 0.7975082 )
24 Au tau( 24) = ( 0.5983772 0.3510139 0.8115842 )
25 Au tau( 25) = ( 0.0701831 0.6467081 0.8020362 )
26 Au tau( 26) = ( 0.5767219 0.6412943 0.7957364 )
27 Au tau( 27) = ( 0.3148888 0.1980482 0.8126670 )
28 Au tau( 28) = ( 0.8341255 0.2031667 0.7996738 )
29 Au tau( 29) = ( 0.3119358 0.5126417 0.7912085 )
30 Au tau( 30) = ( 0.8310740 0.5026999 0.7990832 )
31 Au tau( 31) = ( 0.3263071 0.7861884 0.7997722 )
32 Au tau( 32) = ( 0.8261524 0.7854993 0.8004612 )
33 Au tau( 33) = ( 0.2431307 0.0719549 0.5686503 )
34 Au tau( 34) = ( 0.7425823 0.0673285 0.5601850 )
35 Au tau( 35) = ( 0.2418511 0.3556402 0.5666817 )
36 Au tau( 36) = ( 0.7389403 0.3549512 0.5697331 )
37 Au tau( 37) = ( 0.2399809 0.6469050 0.5563461 )
38 Au tau( 38) = ( 0.7417949 0.6430661 0.5602835 )
39 Au tau( 39) = ( 0.4907894 0.2119273 0.5721939 )
40 Au tau( 40) = ( 0.9927018 0.2117305 0.5599882 )
41 Au tau( 41) = ( 0.4906910 0.4978766 0.5684535 )
42 Au tau( 42) = ( 0.9929971 0.5009281 0.5592007 )
43 Au tau( 43) = ( 0.4892145 0.7876649 0.5596929 )
44 Au tau( 44) = ( 0.9907332 0.7888461 0.5600866 )
45 Au tau( 45) = ( 0.1601513 0.2126164 0.3281773 )
46 Au tau( 46) = ( 0.6557640 0.2123211 0.3305397 )
47 Au tau( 47) = ( 0.1593638 0.4986641 0.3267008 )
48 Au tau( 48) = ( 0.6566499 0.4976798 0.3300475 )
49 Au tau( 49) = ( 0.1594623 0.7924881 0.3315240 )
50 Au tau( 50) = ( 0.6582248 0.7886492 0.3297522 )
51 Au tau( 51) = ( 0.4084990 0.0693956 0.3300475 )
52 Au tau( 52) = ( 0.9075569 0.0676238 0.3282757 )
53 Au tau( 53) = ( 0.4086959 0.3550496 0.3337880 )
54 Au tau( 54) = ( 0.9045054 0.3559355 0.3321146 )
55 Au tau( 55) = ( 0.4098771 0.6419833 0.3271930 )
56 Au tau( 56) = ( 0.9077537 0.6447394 0.3281773 )
57 Au tau( 57) = ( 0.0751048 0.0661473 0.0928228 )
58 Au tau( 58) = ( 0.5770172 0.0635880 0.0925275 )
59 Au tau( 59) = ( 0.0701831 0.3544590 0.0939056 )
60 Au tau( 60) = ( 0.5800686 0.3550496 0.0852434 )
61 Au tau( 61) = ( 0.0750063 0.6441488 0.0942009 )
62 Au tau( 62) = ( 0.5781984 0.6455269 0.0921338 )
63 Au tau( 63) = ( 0.3231572 0.2061197 0.0857356 )
64 Au tau( 64) = ( 0.8238884 0.2104508 0.0926259 )
65 Au tau( 65) = ( 0.3230587 0.5024046 0.0874089 )
66 Au tau( 66) = ( 0.8249712 0.4997469 0.0920353 )
67 Au tau( 67) = ( 0.3241415 0.7882555 0.0940040 )
68 Au tau( 68) = ( 0.8273336 0.7880586 0.0914447 )
69 C tau( 69) = ( 0.4620468 0.2103524 1.8105858 )
70 N tau( 70) = ( 0.5219928 0.2387012 1.6922687 )
71 C tau( 71) = ( 0.5071294 0.1525719 1.5797592 )
72 O tau( 72) = ( 0.4130270 0.1008943 1.8346036 )
73 H tau( 73) = ( 0.5332143 0.3354614 1.6668729 )
74 C tau( 74) = ( 0.4688388 0.2348623 1.4590798 )
75 N tau( 75) = ( 0.4315325 0.1645808 1.3508029 )
76 C tau( 76) = ( 0.3870405 0.2274798 1.2293360 )
77 C tau( 77) = ( 0.4896082 0.2210816 1.1187952 )
78 H tau( 78) = ( 0.5994600 0.0990241 1.5576117 )
79 O tau( 79) = ( 0.4731698 0.3567230 1.4615406 )
80 H tau( 80) = ( 0.4274967 0.0647692 1.3577917 )
81 S tau( 81) = ( 0.4311387 0.3250274 0.9804961 )
82 H tau( 82) = ( 0.2944146 0.1813145 1.1969514 )
83 H tau( 83) = ( 0.3665664 0.3309334 1.2540428 )
84 H tau( 84) = ( 0.5838091 0.2636049 1.1512782 )
85 H tau( 85) = ( 0.5064404 0.1202857 1.0832607 )
86 H tau( 86) = ( 0.4317293 0.0778609 1.6028911 )
87 H tau( 87) = ( 0.4088928 0.3549512 0.0136823 )
88 Cl tau( 88) = ( 0.4196220 0.3705037 3.0539978 )
89 O tau( 89) = ( 0.3456000 0.4625390 3.1326461 )
90 O tau( 90) = ( 0.5119526 0.2990410 3.1367803 )
91 O tau( 91) = ( 0.5032905 0.4411789 2.9619624 )
92 O tau( 92) = ( 0.3370363 0.2789605 2.9837162 )
93 Na tau( 93) = ( 0.7000591 0.3694209 3.0393312 )
94 C tau( 94) = ( 0.1795427 0.0701831 2.2986191 )
95 O tau( 95) = ( 0.1741289 0.2112383 2.3063953 )
96 H tau( 96) = ( 0.0969570 0.0260849 2.3516747 )
97 H tau( 97) = ( 0.2699047 0.0284473 2.3409455 )
98 H tau( 98) = ( 0.1711759 0.0358298 2.1968388 )
99 H tau( 99) = ( 0.2594707 0.2498242 2.2875945 )
100 C tau( 100) = ( 0.1109346 0.5059482 1.2559130 )
101 O tau( 101) = ( 0.1558202 0.5337064 1.1251933 )
102 H tau( 102) = ( 0.0591585 0.4115505 1.2575864 )
103 H tau( 103) = ( 0.0410468 0.5797733 1.2925303 )
104 H tau( 104) = ( 0.1920438 0.4974829 1.3264898 )
105 H tau( 105) = ( 0.2107461 0.6117642 1.1236184 )
106 C tau( 106) = ( 0.0915432 0.5628428 2.3799252 )
107 O tau( 107) = ( 0.0914447 0.6869674 2.3137778 )
108 H tau( 108) = ( 0.0851450 0.5733751 2.4869224 )
109 H tau( 109) = ( 0.1784599 0.5036842 2.3552184 )
110 H tau( 110) = ( 0.0049217 0.5070310 2.3492139 )
111 H tau( 111) = ( 0.1657620 0.7410074 2.3382878 )
112 C tau( 112) = ( 0.4599797 0.2936271 2.4541440 )
113 O tau( 113) = ( 0.4484630 0.2875243 2.3134825 )
114 H tau( 114) = ( 0.3760160 0.2452963 2.5013921 )
115 H tau( 115) = ( 0.4610625 0.3955058 2.4885957 )
116 H tau( 116) = ( 0.5472903 0.2438198 2.4935174 )
117 H tau( 117) = ( 0.5344939 0.2934303 2.2720420 )
118 C tau( 118) = ( 0.4091881 0.2600613 3.4807070 )
119 O tau( 119) = ( 0.3846781 0.3970807 3.4610203 )
120 H tau( 120) = ( 0.4428523 0.2170459 3.3881795 )
121 H tau( 121) = ( 0.3210901 0.2045448 3.5089574 )
122 H tau( 122) = ( 0.4868521 0.2441151 3.5537446 )
123 H tau( 123) = ( 0.3347723 0.4357651 3.5325814 )
124 C tau( 124) = ( 0.4113536 0.2609472 3.8442219 )
125 O tau( 125) = ( 0.2912647 0.3135107 3.8958011 )
126 H tau( 126) = ( 0.3949152 0.2051354 3.7535646 )
127 H tau( 127) = ( 0.4638187 0.1976545 3.9141097 )
128 H tau( 128) = ( 0.4786821 0.3408752 3.8192198 )
129 H tau( 129) = ( 0.2420480 0.2469697 3.9426554 )
130 C tau( 130) = ( 0.4567314 0.6891329 1.3385972 )
131 O tau( 131) = ( 0.4945299 0.6541890 1.2072869 )
132 H tau( 132) = ( 0.3783784 0.6236747 1.3727536 )
133 H tau( 133) = ( 0.4198189 0.7903226 1.3430267 )
134 H tau( 134) = ( 0.5369547 0.6801755 1.4100599 )
135 H tau( 135) = ( 0.5808561 0.6889361 1.1864190 )
136 C tau( 136) = ( 0.2461822 0.4264139 2.6890063 )
137 O tau( 137) = ( 0.2110414 0.5626459 2.6983575 )
138 H tau( 138) = ( 0.2064150 0.3724724 2.7733639 )
139 H tau( 139) = ( 0.3527857 0.4100740 2.6912703 )
140 H tau( 140) = ( 0.2066119 0.3809376 2.5998256 )
141 H tau( 141) = ( 0.2695109 0.6177686 2.6466799 )
142 C tau( 142) = ( 0.2975645 0.5812498 3.7557302 )
143 O tau( 143) = ( 0.2635065 0.5757375 3.6190061 )
144 H tau( 144) = ( 0.3852687 0.5226819 3.7756138 )
145 H tau( 145) = ( 0.2207863 0.5401046 3.8194167 )
146 H tau( 146) = ( 0.3208932 0.6817504 3.7866383 )
147 H tau( 147) = ( 0.1743257 0.6073347 3.6068987 )
148 C tau( 148) = ( 0.8132576 0.3237478 2.1446691 )
149 O tau( 149) = ( 0.7156115 0.3254212 2.2473352 )
150 H tau( 150) = ( 0.8181793 0.2273814 2.0972241 )
151 H tau( 151) = ( 0.9140535 0.3444188 2.1782349 )
152 H tau( 152) = ( 0.7891414 0.3946199 2.0674972 )
153 H tau( 153) = ( 0.7484883 0.3761144 2.3211604 )
154 C tau( 154) = ( 0.6806677 0.1157578 3.7269876 )
155 O tau( 155) = ( 0.6760413 0.2566161 3.7295469 )
156 H tau( 156) = ( 0.6330258 0.0763844 3.6386928 )
157 H tau( 157) = ( 0.6253480 0.0771719 3.8105576 )
158 H tau( 158) = ( 0.7803808 0.0750063 3.7338779 )
159 H tau( 159) = ( 0.7518351 0.2982535 3.6895829 )
160 C tau( 160) = ( 0.7855978 0.7380544 1.3365301 )
161 O tau( 161) = ( 0.7544928 0.7226003 1.1993138 )
162 H tau( 162) = ( 0.7316562 0.8181793 1.3858452 )
163 H tau( 163) = ( 0.8910200 0.7520319 1.3522794 )
164 H tau( 164) = ( 0.7615800 0.6480862 1.3904716 )
165 H tau( 165) = ( 0.7462244 0.8068594 1.1550187 )
166 C tau( 166) = ( 0.8902326 0.5984757 2.2991113 )
167 O tau( 167) = ( 0.8758613 0.4869505 2.3846500 )
168 H tau( 168) = ( 0.8327474 0.6824395 2.3346457 )
169 H tau( 169) = ( 0.8533200 0.5761313 2.2005793 )
170 H tau( 170) = ( 0.9932924 0.6286948 2.2858227 )
171 H tau( 171) = ( 0.9296060 0.4981719 2.4628062 )
172 C tau( 172) = ( 0.6702337 0.5932587 2.5716736 )
173 O tau( 173) = ( 0.7671907 0.6367663 2.6639058 )
174 H tau( 174) = ( 0.6880502 0.4925612 2.5379109 )
175 H tau( 175) = ( 0.6645246 0.6587170 2.4859380 )
176 H tau( 176) = ( 0.5734736 0.5924712 2.6182327 )
177 H tau( 177) = ( 0.8545996 0.6369632 2.6248277 )
178 C tau( 178) = ( 0.5993616 0.5525072 3.6476502 )
179 O tau( 179) = ( 0.6214107 0.5730798 3.7854571 )
180 H tau( 180) = ( 0.5228788 0.6192451 3.6116235 )
181 H tau( 181) = ( 0.5647130 0.4524988 3.6275698 )
182 H tau( 182) = ( 0.6878533 0.5701268 3.5885901 )
183 H tau( 183) = ( 0.6950389 0.5204179 3.8154793 )
number of k points= 8 gaussian smearing, width (Ry)= 0.0080
cart. coord. in units 2pi/alat
k( 1) = ( 0.1250000 0.1443376 0.0000000), wk = 0.2500000
k( 2) = ( 0.1250000 0.4330129 0.0000000), wk = 0.2500000
k( 3) = ( 0.3750000 0.1443376 0.0000000), wk = 0.2500000
k( 4) = ( 0.3750000 0.4330129 0.0000000), wk = 0.2500000
k( 5) = ( -0.1250000 0.1443376 0.0000000), wk = 0.2500000
k( 6) = ( -0.1250000 0.4330129 0.0000000), wk = 0.2500000
k( 7) = ( -0.3750000 0.1443376 0.0000000), wk = 0.2500000
k( 8) = ( -0.3750000 0.4330129 0.0000000), wk = 0.2500000
Dense grid: 1939571 G-vectors FFT dimensions: ( 108, 90, 432)
Smooth grid: 490659 G-vectors FFT dimensions: ( 72, 60, 270)
Estimated max dynamical RAM per process > 28.50 GB
Initial potential from superposition of free atoms
starting charge 898.97706, renormalised to 900.00000
negative rho (up, down): 1.647E-03 0.000E+00
Starting wfcs are 748 randomized atomic wfcs
total cpu time spent up to now is 33.7 secs
Self-consistent Calculation
iteration # 1 ecut= 28.00 Ry beta= 0.30
line 176: cudaLaunchKernel returned status 700: an illegal memory access was encountered
Davidson diagonalization with overlap
ethr = 1.00E-02, avg # of iterations = 4.6
srun: error: bc11u25n2: task 0: Exited with exit code 1
More information about the users
mailing list