<div dir="ltr"><span style="font-family:arial,sans-serif;font-size:13px">Hi Everyone,</span><br style="font-family:arial,sans-serif;font-size:13px"><br>(apologies if this has been sent twice)<br><br style="font-family:arial,sans-serif;font-size:13px">
<span style="font-family:arial,sans-serif;font-size:13px">I have compiled QE 5.0.2 on a computer with AMD interlagos processors, using the acml, compiling with openmp enabled, and submitting jobs with PBS.  I've had a speed up using 2 openmp threads per mpi process.</span><br style="font-family:arial,sans-serif;font-size:13px">
<br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">I've been trying to do the same on another computer, that has MOAB as the scheduler, E5 series xeon processors (E5-2660) and uses the Intel MKL (E5-2660).  I'm pretty sure hyperthreading has been turned off, as each node has two sockets and 16 cores in total.</span><br style="font-family:arial,sans-serif;font-size:13px">
<br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">I've seen a slow down in performance using OpenMP and MPI, but have read that this might be the case in the documentation.  I'm waiting in the computer's queue to run the following:</span><br style="font-family:arial,sans-serif;font-size:13px">
<br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">#!/bin/bash</span><br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">#MOAB -l "nodes=2:ppn=16"</span><br style="font-family:arial,sans-serif;font-size:13px">
<span style="font-family:arial,sans-serif;font-size:13px">#MOAB -l "walltime=0:01:00"</span><br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">#MOAB -j oe</span><br style="font-family:arial,sans-serif;font-size:13px">
<span style="font-family:arial,sans-serif;font-size:13px">#MOAB -N pwscf_calc</span><br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">#MOAB -A readmsd02</span><br style="font-family:arial,sans-serif;font-size:13px">
<span style="font-family:arial,sans-serif;font-size:13px">#MOAB -q bbtest</span><br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">cd "$PBS_O_WORKDIR"</span><br style="font-family:arial,sans-serif;font-size:13px">
<span style="font-family:arial,sans-serif;font-size:13px">module load apps/openmpi/v1.6.3/intel-tm-</span><u style="font-family:arial,sans-serif;font-size:13px"></u><span style="font-family:arial,sans-serif;font-size:13px">i</span><span style="font-family:arial,sans-serif;font-size:13px">b/v2013.0.079</span><br style="font-family:arial,sans-serif;font-size:13px">
<span style="font-family:arial,sans-serif;font-size:13px">export PATH=$HOME/bin:$PATH</span><br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">export OMP_NUM_THREADS=2</span><br style="font-family:arial,sans-serif;font-size:13px">
<span style="font-family:arial,sans-serif;font-size:13px">mpiexec -np 16 -x OMP_NUM_THREADS=2 -npernode 8 -bind-to-socket -display-map -report-bindings pw_openmp_5.0.2.x -in </span><a href="http://benchmark2.in/" target="_blank" style="font-family:arial,sans-serif;font-size:13px">benchmark2.in</a><span style="font-family:arial,sans-serif;font-size:13px"> > benchmark2c.out</span><br style="font-family:arial,sans-serif;font-size:13px">
<br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">I just wondered if anyone had any tips on the settings or flags for hybrid MPI/OpenMP with the E5 Xeon processors?</span><br style="font-family:arial,sans-serif;font-size:13px">
<br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">All the best,</span><br style="font-family:arial,sans-serif;font-size:13px"><br style="font-family:arial,sans-serif;font-size:13px">
<span style="font-family:arial,sans-serif;font-size:13px">Ben Palmer</span><br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">Student @ University of Birmingham, UK</span><br>
</div>