<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Paolo,</p>
<p>I discovered that the file in question was the `-machinefile` argument I was passing into `mpirun`. This was being generated and stored in the non-shared /tmp file system. This is certainly not an error with QE. Thank you for your patience.<br>
</p>
<div class="moz-cite-prefix">On 11/6/19 11:10 AM, Ben Comer wrote:<br>
</div>
<blockquote type="cite" cite="mid:ded68d18-4d81-2c30-c973-9b8db9317153@gatech.edu">
<p>Paolo,</p>
<p>Thanks for your help, I'll let you know if I track down what file is causing the issue.<br>
</p>
<div class="moz-cite-prefix">On 11/6/19 4:25 AM, Paolo Giannozzi wrote:<br>
</div>
<blockquote type="cite" cite="mid:CAPMgbCtZZt+bff4RCCjYEQ2CBDfbkH-1J=uLRg+B7SZEbDrz4A@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">On Tue, Nov 5, 2019 at 10:36 PM Paolo Giannozzi <<a href="mailto:p.giannozzi@gmail.com" moz-do-not-send="true">p.giannozzi@gmail.com</a>> wrote:</div>
<div dir="ltr"><br>
</div>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div>Yes, I was referring to the "develop" branch of <a href="https://gitlab.com/QEF/q-e" target="_blank" moz-do-not-send="true">
https://gitlab.com/QEF/q-e</a></div>
</div>
</blockquote>
<div><br>
</div>
<div>Just a clarification: <a href="https://github.com/QEF/q-e" moz-do-not-send="true">
https://github.com/QEF/q-e</a> is a mirror that is automatically aligned to the above gitlab repository</div>
<div><br>
</div>
<div>Paolo<br>
</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div>. There is no need for the "-D__BEOWULF" option any more. There is nothing I can do unless there is some hint of why and for which file FoX fails</div>
<div><br>
</div>
<div>Paolo<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Nov 5, 2019 at 10:21 PM Ben Comer <<a href="mailto:bcomer3@gatech.edu" target="_blank" moz-do-not-send="true">bcomer3@gatech.edu</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Thanks for the help, <br>
</p>
<p>Paolo,</p>
<p>to confirm, the development version is the "develop" branch of the repo below? Compiling this code also yields the same error. I presume that I don't need the -D__BEOWULF flag that Pietro recommends. I also the error I have listed below thew "ERROR(FoX)
Cannot open file" errors.<br>
</p>
<p><a href="https://github.com/QEF/q-e" target="_blank" moz-do-not-send="true">https://github.com/QEF/q-e</a></p>
<p>[rich133-k35-19-l.pace.gatech.edu:mpi_rank_0][handle_cqe] Send desc error in msg to 28, wc_opcode=0<br>
[rich133-k35-19-l.pace.gatech.edu:mpi_rank_0][handle_cqe] Msg from 28: wc.status=12, wc.wr_id=0x2a60fb0, wc.opcode=0, vbuf->phead->type=0 = MPIDI_CH3_PKT_EAGER_SEND<br>
[rich133-k35-19-l.pace.gatech.edu:mpi_rank_0][handle_cqe] src/mpid/ch3/channels/mrail/src/gen2/ibv_channel_manager.c:547: [] Got completion with error 12, vendor code=0x81, dest rank=28<br>
: Inappropriate ioctl for device (25)<br>
<br>
</p>
<div>On 11/4/19 12:22 PM, Paolo Giannozzi wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Try the development version, that no longer assumes that all files are readable by all processors.</div>
<div><br>
</div>
<div>Paolo<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Nov 4, 2019 at 5:22 PM Ben Comer <<a href="mailto:bcomer3@gatech.edu" target="_blank" moz-do-not-send="true">bcomer3@gatech.edu</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
Pietro,<br>
<br>
I am running quantum espresso with 56 cores across 2 nodes (28 cores per <br>
node) when I receive this error. The error does not appear when I am <br>
running on a single node (I've tested 1,2,4,8,16, and 28 cores on a <br>
single node.) However, all runs that involve more than one node do have <br>
this issue. Previous messages implied that this was because the nodes <br>
did not share a common file system, which is the case for our cluster. <br>
Each node has it's own /tmp directory located on each node. Please let <br>
me know if I can provide more information.<br>
<br>
On 11/4/19 11:08 AM, Pietro Davide Delugas wrote:<br>
> Dear Ben Comer.<br>
> The flag is not valid any more. In fact It was not a fix just a <br>
> configure option to select the old file format rather than the new one.<br>
><br>
> The old format is no longer supported, that is why the option is not <br>
> recognized by the configure anymore.<br>
><br>
> Cannot open file means that the program is not finding some file. <br>
> Could you give more details on how you are running the code in order <br>
> to understand why the error issues?<br>
><br>
> best regards - Pietro<br>
><br>
><br>
><br>
> On 11/4/19 4:56 PM, Ben Comer wrote:<br>
>> Hello,<br>
>><br>
>> Since I've switched from espresso 5 to espresso 6.4.1 I've gotten the<br>
>> "ERROR(FoX) Cannot open file" error when running across nodes. I see in<br>
>> the mailing list that the prescribed fix for this is to recompile with<br>
>> the "--enable-xml=no" flag in the "./configure" step. However, when I do<br>
>> this I get a warning saying that the flag was not recognized and I<br>
>> continue to get the error after compilation. Is there any new fix for<br>
>> this issue?<br>
>><br>
>> Thanks,<br>
>><br>
>> Ben Comer<br>
>> Georgia Tech<br>
>><br>
>> _______________________________________________<br>
>> Quantum ESPRESSO is supported by MaX <br>
>> (<a href="http://www.max-centre.eu/quantum-espresso" rel="noreferrer" target="_blank" moz-do-not-send="true">www.max-centre.eu/quantum-espresso</a>)<br>
>> users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank" moz-do-not-send="true">
users@lists.quantum-espresso.org</a><br>
>> <a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank" moz-do-not-send="true">
https://lists.quantum-espresso.org/mailman/listinfo/users</a><br>
><br>
><br>
> _______________________________________________<br>
> Quantum ESPRESSO is supported by MaX (<a href="http://www.max-centre.eu/quantum-espresso" rel="noreferrer" target="_blank" moz-do-not-send="true">www.max-centre.eu/quantum-espresso</a>)<br>
> users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank" moz-do-not-send="true">
users@lists.quantum-espresso.org</a><br>
> <a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank" moz-do-not-send="true">
https://lists.quantum-espresso.org/mailman/listinfo/users</a><br>
_______________________________________________<br>
Quantum ESPRESSO is supported by MaX (<a href="http://www.max-centre.eu/quantum-espresso" rel="noreferrer" target="_blank" moz-do-not-send="true">www.max-centre.eu/quantum-espresso</a>)<br>
users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank" moz-do-not-send="true">
users@lists.quantum-espresso.org</a><br>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.quantum-espresso.org/mailman/listinfo/users</a></blockquote>
</div>
<br clear="all">
<br>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,<br>
Univ. Udine, via delle Scienze 208, 33100 Udine, Italy<br>
Phone +39-0432-558216, fax +39-0432-558222<br>
<br>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Quantum ESPRESSO is supported by MaX (<a href="http://www.max-centre.eu/quantum-espresso" target="_blank" moz-do-not-send="true">www.max-centre.eu/quantum-espresso</a>)
users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank" moz-do-not-send="true">users@lists.quantum-espresso.org</a>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" target="_blank" moz-do-not-send="true">https://lists.quantum-espresso.org/mailman/listinfo/users</a></pre>
</blockquote>
</div>
_______________________________________________<br>
Quantum ESPRESSO is supported by MaX (<a href="http://www.max-centre.eu/quantum-espresso" rel="noreferrer" target="_blank" moz-do-not-send="true">www.max-centre.eu/quantum-espresso</a>)<br>
users mailing list <a href="mailto:users@lists.quantum-espresso.org" target="_blank" moz-do-not-send="true">
users@lists.quantum-espresso.org</a><br>
<a href="https://lists.quantum-espresso.org/mailman/listinfo/users" rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.quantum-espresso.org/mailman/listinfo/users</a></blockquote>
</div>
<br clear="all">
<br>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,<br>
Univ. Udine, via delle Scienze 208, 33100 Udine, Italy<br>
Phone +39-0432-558216, fax +39-0432-558222<br>
<br>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br clear="all">
<br>
-- <br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>Paolo Giannozzi, Dip. Scienze Matematiche Informatiche e Fisiche,<br>
Univ. Udine, via delle Scienze 208, 33100 Udine, Italy<br>
Phone +39-0432-558216, fax +39-0432-558222<br>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Quantum ESPRESSO is supported by MaX (<a class="moz-txt-link-abbreviated" href="http://www.max-centre.eu/quantum-espresso" moz-do-not-send="true">www.max-centre.eu/quantum-espresso</a>)
users mailing list <a class="moz-txt-link-abbreviated" href="mailto:users@lists.quantum-espresso.org" moz-do-not-send="true">users@lists.quantum-espresso.org</a>
<a class="moz-txt-link-freetext" href="https://lists.quantum-espresso.org/mailman/listinfo/users" moz-do-not-send="true">https://lists.quantum-espresso.org/mailman/listinfo/users</a></pre>
</blockquote>
</blockquote>
</body>
</html>