<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div dir="auto">Hello
<div dir="auto"><br>
<div dir="auto"><br>
<div dir="auto">Usually the "<span style="color: rgb(0, 0, 0); font-family: aptos, aptos_embeddedfont, aptos_msfontservice, calibri, helvetica, sans-serif; font-size: 16px;">libgomp: TODO"</span> error message occurs because the program has been linked with
a cuda version that is not compatible with the gpu driver that is running in the nodes. </div>
<div dir="auto">You should inquire with the system managers of your cluster about the right toolchain to use. </div>
<div dir="auto">Or check the cuda and driver versions and report them in the forum to get more help. </div>
<div dir="auto">Pietro</div>
<div dir="auto"><br>
</div>
</div>
<br>
<div class="gmail_extra" dir="auto"><br>
<div class="gmail_quote">Il 24 set 2024 7:06 PM, "Hazra, Shilpa" <shazra3@uic.edu> ha scritto:<br type="attribution">
<blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<b>Hello,</b></div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<b><br>
</b></div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<b>I am using the cuda version of Quantum Espresso. The input and the job submission script I am using are written bellow.<br>
<br>
</b></div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<b><br>
</b></div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<b>INPUT:</b></div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
&control</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
calculation = 'vc-relax'</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
prefix = 'silicon'</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
outdir = './tmp/'</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
pseudo_dir = './'</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
etot_conv_thr = 1e-5</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
forc_conv_thr = 1e-4</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
/</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
&system</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
ibrav=2, celldm(1) =14,</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
nat=2, ntyp=1,</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
ecutwfc=30</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
/</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
&electrons</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
conv_thr=1e-8</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
/</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
&ions</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
/</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
&cell</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
cell_dofree='ibrav'</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
/</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
ATOMIC_SPECIES</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
Si 28.0855 Si.pbe-n-kjpaw_psl.1.0.0.UPF</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
ATOMIC_POSITIONS (alat)</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
Si 0.00 0.00 0.00 0 0 0</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
Si 0.25 0.25 0.25 0 0 0</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
K_POINTS (automatic)</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
6 6 6 1 1 1</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
~ </div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
~</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<b>SCRIPT:</b></div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#!/bin/bash</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH --job-name="test2"</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH --output="test2.out"</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH --partition=gpuA40x4</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH --mem=16G</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH --nodes=1</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH --ntasks-per-node=1 </div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH --cpus-per-task=16 # spread out to use 1 core per numa, set to 64 if tasks is 1</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH --constraint="scratch"</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH --gpus-per-node=4</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH --gpu-bind=closest # select a cpu close to gpu on pci bus topology</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH --account=bcox-delta-gpu # <- match to a "Project" returned by the "accounts" command</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH -t 08:00:00</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH -e slurm-%j.err</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
#SBATCH -o slurm-%j.out</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
module reset</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
module load nvhpc/22.11</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
module load openmpi/4.1.5+cuda</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
module load quantum-espresso/7.3.1+cuda</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
export OMP_NUM_THREADS=16 # if code is not multithreaded, otherwise set to 8 or 16</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
srun pw.x -N 1 -n 1 test2.in > test2.out </div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<b>Moreover, the job is running well. But I am not getting any output data printed in the output file even after running the job for 30 hours. Along with that, I am getting the following error massage in the .err file.</b></div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
libgomp: TODO</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
srun: error: gpub075: task 0: Exited with exit code 1 </div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<b>I am really not getting why this error is happening and the error is coming. I tried to adjust the script for parallel computing. However, all the time I am getting the same error. Please help me out form this problem. If you could help me with solving this
problem, it would be really beneficial for me. Thank you.</b></div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<b>Sincerely,</b></div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<b>Shilpa Hazra</b></div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<b>Ph.D., Chemistry, University of Illinois Chicago<br>
<br>
</b></div>
<div style="font-family:'aptos' , 'aptos_embeddedfont' , 'aptos_msfontservice' , 'calibri' , 'helvetica' , sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
</div>
</blockquote>
</div>
<br>
<br>
</div>
</div>
</div>
</body>
</html>