<html>
  <head>
    <meta content="text/html; charset=windows-1252"
      http-equiv="Content-Type">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-cite-prefix">what was the conv_thr in this example ?<br>
      the default is 1.d-6 which is the range of variations you are
      reporting.<br>
      If so, could you try with a tighter threshold ?<br>
      stefano<br>
      <br>
      On 26/01/2016 17:55, Paolo Giannozzi wrote:<br>
    </div>
    <blockquote
cite="mid:CAPMgbCuHLxkM-5EFtSmf09V-7tS5_e-uZmRbCgKkOh6FkzBpzw@mail.gmail.com"
      type="cite">
      <div dir="ltr">
        <div>Very interesting, and excellent questions, for which
          unfortunately I have no clear answer (nor has anybody else, I
          am afraid). <br>
          <br>
          One should obtain the same numbers - within the errors due to
          roundoff, though -† in serial, OpenMP, MPI execution, and on
          different machines, and with different compilers and
          mathematical libraries. In practice, there are invariably
          small differences, that sometimes do not completely disappear
          even if one pushes convergence thresholds to very strict
          limits.† In addition to†<a moz-do-not-send="true"
            href="http://noncolin-constrain_total.in">noncolin-constrain_total.in</a>,
          another notable offender is <a moz-do-not-send="true"
            href="http://vdw-ts.in">vdw-ts.in</a>.† <br>
          <br>
          This could signal a small bug, but in my experience, most of
          those cases can be linked to specific optimized mathematical
          libraries or compiler versions. As long as we can blame
          somebody else, not† a big problem :-)<br>
          <br>
        </div>
        <div>Paolo<br>
        </div>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Tue, Jan 26, 2016 at 12:23 PM, Nick
          Wilson <span dir="ltr"><<a moz-do-not-send="true"
              href="mailto:nw.qeforge.5211@family-wilson.me.uk"
              target="_blank">nw.qeforge.5211@family-wilson.me.uk</a>></span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
            <br>
            Iíve been testing the OpenMP build of Quantum Espresso
            5.3.0† on our system using the Intel compiler and MKL and
            have a question about variation of energy with the number of
            OpenMP threads used.<br>
            <br>
            I ran all the plane wave tests in the test-suite directory
            using between 1 and 16 OpenMP threads and they all gave
            consistent results apart from pw_noncolin/<a
              moz-do-not-send="true"
              href="http://noncolin-constrain_total.in" rel="noreferrer"
              target="_blank">noncolin-constrain_total.in</a> which
            showed variation in† between -55.54478325 Ry and
            -55.54478414 Ry.<br>
            <br>
            I ran the test through the Intel Inspector tool but that
            didnít show up any threading deadlocks or data races.<br>
            <br>
            I dropped the compiler optimisation to -O0 and added the
            ď-fp-model strictĒ and ď-fp-model sourceĒ compiler flags but
            that had no effect.<br>
            <br>
            I tried using some of the relevant environment variables
            (KMP_DETERMINISTIC_REDUCTION=1 and MKL_CBWR=COMPATIBLE)
            which also had no effect.<br>
            <br>
            Changing to use the internal BLAS library resolved the issue
            so it looks to be MKL-related. Itís present with both the
            GNU and Intel compilers.<br>
            <br>
            I dropped back to an earlier version of MKL but the effect
            was still present.<br>
            <br>
            As it was thread-related I tried linking against the
            sequential version of MKL but that didnít help.<br>
            <br>
            So, I guess my questions are:<br>
            Should the results always be invariant of the number of
            OpenMP threads?<br>
            Is there anything unique about the <a
              moz-do-not-send="true"
              href="http://noncolin-constrain_total.in" rel="noreferrer"
              target="_blank">noncolin-constrain_total.in</a> test case
            which would cause it to behave differently to the rest of
            the tests?<br>
            <br>
            Best regards,<br>
            Nick Wilson<br>
            <br>
            <br>
            System details:<br>
            † Intel Sandy Bridge E5-2650 CPU<br>
            † CentOS Linux release 7.2.1511<br>
            † MKL from Intel compiler 16.0.0<br>
            †GNU compiler version 4.8.5<br>
            _______________________________________________<br>
            Pw_forum mailing list<br>
            <a moz-do-not-send="true" href="mailto:Pw_forum@pwscf.org">Pw_forum@pwscf.org</a><br>
            <a moz-do-not-send="true"
              href="http://pwscf.org/mailman/listinfo/pw_forum"
              rel="noreferrer" target="_blank">http://pwscf.org/mailman/listinfo/pw_forum</a></blockquote>
        </div>
        <br>
        <br clear="all">
        <br>
        -- <br>
        <div class="gmail_signature">
          <div dir="ltr">
            <div>
              <div dir="ltr">
                <div>Paolo Giannozzi, Dip. Scienze Matematiche
                  Informatiche e Fisiche,<br>
                  Univ. Udine, via delle Scienze 208, 33100 Udine, Italy<br>
                  Phone +39-0432-558216, fax +39-0432-558222<br>
                  <br>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Pw_forum mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Pw_forum@pwscf.org">Pw_forum@pwscf.org</a>
<a class="moz-txt-link-freetext" href="http://pwscf.org/mailman/listinfo/pw_forum">http://pwscf.org/mailman/listinfo/pw_forum</a></pre>
    </blockquote>
    <br>
  </body>
</html>