<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Hi Ian,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I was able to get this crash when I compiled w/o scalapack (nor elpa, etc.). Assuming you’re also running w/o scalapack, try editing LAXlib/ptoolkit.f90. Starting at line 4212 you should have:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> ! split communicator is present and must be freed on all processors<o:p></o:p></p>
<p class="MsoNormal"> CALL mpi_comm_free( col_comm, ierr )<o:p></o:p></p>
<p class="MsoNormal"> IF( ierr /= 0 ) &<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Change it to:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> ! split communicator is present and must be freed on all processors<o:p></o:p></p>
<p class="MsoNormal"> IF( col_comm /= MPI_COMM_NULL ) THEN<o:p></o:p></p>
<p class="MsoNormal"> CALL mpi_comm_free( col_comm, ierr )<o:p></o:p></p>
<p class="MsoNormal"> IF( ierr /= 0 ) &<o:p></o:p></p>
<p class="MsoNormal"> CALL lax_error__( " pdtrtri ", " in mpi_comm_free 25 ", ABS( ierr ) )<o:p></o:p></p>
<p class="MsoNormal"> END IF<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">And see if that fixes it for you.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">John<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">users <users-bounces@lists.quantum-espresso.org> on behalf of Ian Shuttleworth <shuttleworth.ian@gmail.com><br>
<b>Reply-To: </b>Quantum ESPRESSO users Forum <users@lists.quantum-espresso.org><br>
<b>Date: </b>Friday, December 20, 2019 at 10:59 AM<br>
<b>To: </b>Quantum ESPRESSO users Forum <users@lists.quantum-espresso.org><br>
<b>Subject: </b>Re: [QE-users] MPI in qe-6.5<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">I've attached "<a href="https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftest.in&data=02%7C01%7Cjohn.vinson%40nist.gov%7Cc07d9fa036e5429e021908d785659d51%7C2ab5d82fd8fa4797a93e054655c61dec%7C1%7C1%7C637124543805045449&sdata=6MhdnfIg9tQ2YmKTC6bye%2BaAaGnYStkcSCZ3flJgRrc%3D&reserved=0">test.in</a>"
the input file - the pseudos just come directly from the QE web-site<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">With thanks<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Ian<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Fri, Dec 20, 2019 at 3:47 PM Pietro Delugas <<a href="mailto:pdelugas@sissa.it">pdelugas@sissa.it</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<p><span style="font-size:10.0pt">Dear Ian </span><o:p></o:p></p>
<p><span style="font-size:10.0pt">I just compiled pw with gcc-4.8.5 and openmpi 1.10.7 and the test-suite tests are passing, so either it is a problem in someway related to your input, to some issue related to the way you compiled the program or to some feature
of you system other than the compiler and the mpi library. </span><o:p></o:p></p>
<p><span style="font-size:10.0pt">kind regards - Pietro</span><o:p></o:p></p>
<p><span style="font-size:10.0pt">Could you send you input </span><o:p></o:p></p>
<div>
<p class="MsoNormal">On 20/12/19 16:23, Ian Shuttleworth wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<div>
<p class="MsoNormal">Dear all<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I encounter MPI communication errors when running 6.5 compiled both with gcc-4.8.5 and openMPI 1.10.7, and also - on another HPC - compiled with gcc 6.4.0 and open OpenMPI 2.1.2.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The code starts correctly with statements:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> This program is part of the open-source Quantum ESPRESSO suite<br>
for quantum simulation of materials; please cite<br>
"P. Giannozzi et al., J. Phys.:Condens. Matter 21 395502 (2009);<br>
"P. Giannozzi et al., J. Phys.:Condens. Matter 29 465901 (2017);<br>
URL <a href="https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.quantum-espresso.org&data=02%7C01%7Cjohn.vinson%40nist.gov%7Cc07d9fa036e5429e021908d785659d51%7C2ab5d82fd8fa4797a93e054655c61dec%7C1%7C1%7C637124543805055443&sdata=0gEwitsOMxujynLItOAJe4Q8n3AUD97ZuXiEzjrMvd0%3D&reserved=0" target="_blank">
http://www.quantum-espresso.org</a>", <br>
in publications or presentations arising from this work. More details at<br>
<a href="https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.quantum-espresso.org%2Fquote&data=02%7C01%7Cjohn.vinson%40nist.gov%7Cc07d9fa036e5429e021908d785659d51%7C2ab5d82fd8fa4797a93e054655c61dec%7C1%7C1%7C637124543805055443&sdata=Uxu67hdAKo%2Fh2TGa%2FXA9EWLXx7v0QDFTfV07marSZX8%3D&reserved=0" target="_blank">
http://www.quantum-espresso.org/quote</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> Parallel version (MPI), running on 16 processors<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The execution stops with the statement:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> Starting wfcs are 192 randomized atomic wfcs<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">and the following error messages then appear:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><br>
[node44:20988] *** An error occurred in MPI_Comm_free<br>
[node44:20988] *** reported by process [140653000785921,140724603453442]<br>
[node44:20988] *** on communicator MPI_COMM_WORLD<br>
[node44:20988] *** MPI_ERR_COMM: invalid communicator<br>
[node44:20988] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,<br>
[node44:20988] *** and potentially your MPI job)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Compiling 6.4.1 using exactly the same gcc/openmpi's doesn't produce the same problem, and the execution in fact completes without error. So my question is: what differences are there in the MPI between 6.5 and 6.4.1, and are there any
'tweaks' that could be applied to the compile script to remove the problems I'm seeing with 6.5?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">With thanks<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Ian Shuttleworth<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">(Nottingham Trent University)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>Quantum ESPRESSO is supported by MaX (<a href="https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.max-centre.eu%2Fquantum-espresso&data=02%7C01%7Cjohn.vinson%40nist.gov%7Cc07d9fa036e5429e021908d785659d51%7C2ab5d82fd8fa4797a93e054655c61dec%7C1%7C1%7C637124543805065437&sdata=1RmTscxQZflZOy0rA2mt1B%2FuDIIYt79fUX0ZN5AnOxs%3D&reserved=0" target="_blank">www.max-centre.eu/quantum-espresso</a>)<o:p></o:p></pre>
<pre>users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank">users@lists.quantum-espresso.org</a><o:p></o:p></pre>
<pre><a href="https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.quantum-espresso.org%2Fmailman%2Flistinfo%2Fusers&data=02%7C01%7Cjohn.vinson%40nist.gov%7Cc07d9fa036e5429e021908d785659d51%7C2ab5d82fd8fa4797a93e054655c61dec%7C1%7C1%7C637124543805065437&sdata=Byr38ph91oRMhfIUZkMGB0vosBDrKuqtZb%2Fc5CN%2F%2FXY%3D&reserved=0" target="_blank">https://lists.quantum-espresso.org/mailman/listinfo/users</a><o:p></o:p></pre>
</blockquote>
</div>
<p class="MsoNormal">_______________________________________________<br>
Quantum ESPRESSO is supported by MaX (<a href="https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.max-centre.eu%2Fquantum-espresso&data=02%7C01%7Cjohn.vinson%40nist.gov%7Cc07d9fa036e5429e021908d785659d51%7C2ab5d82fd8fa4797a93e054655c61dec%7C1%7C1%7C637124543805075434&sdata=%2BhVcfRsTvDVMzSRQwthUjUk7bTu1UGhA3WmxnnSJ%2Bh4%3D&reserved=0" target="_blank">www.max-centre.eu/quantum-espresso</a>)<br>
users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank">
users@lists.quantum-espresso.org</a><br>
<a href="https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.quantum-espresso.org%2Fmailman%2Flistinfo%2Fusers&data=02%7C01%7Cjohn.vinson%40nist.gov%7Cc07d9fa036e5429e021908d785659d51%7C2ab5d82fd8fa4797a93e054655c61dec%7C1%7C1%7C637124543805075434&sdata=L5bIozKD3Aqs8Z1Y%2B7y2hBlZxLBMDIqCxANwcJZaONI%3D&reserved=0" target="_blank">https://lists.quantum-espresso.org/mailman/listinfo/users</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</body>
</html>