[Pw_forum] about MPI and OpenMP threads for QE-GPU

[Mutlu COLAKOGULLARI]

Dear All,
I am trying to do some calculations on simple silicon with GPU support as a test case before proceeding serious work. 
QE-GPU has been installed by intel cluster suite 13, cuda 5.5 and latest svn commits of QE and QE-GPU. 
	MANUAL_DFLAGS  = -D__MPICH_SKIP_MPICXX -D__ISO_C_BINDING -D__VERBOSE
	DFLAGS         =  -D__INTEL -D__FFTW3 -D__MPI -D__PARA -D__SCALAPACK -D__ELPA -D__CUDA -D__PHIGEMM -D__OPENMP $(MANUAL_DFLAGS)
	I have for 4 nodes. Each one has 24 cores and 2 GPU support so totally 96 cores + 8 GPU. 
	I haven't got any problem for 1 node. I did it. When I passed to bigger than 1 node I am getting warnings, for example,
	***WARNING: unbalanced configuration (1 MPI per node, 2 GPUs per node)
	***WARNING: unbalanced configuration (13 MPI per node, 2 GPUs per node)
	So, I am in trouble for sharing the cores between MPI and threads for OpenMP. 
	Question: How should I share the cores between MPI and OpenMP threads? 
	Here is my case:
	#SBATCH -N 4
	#SBATCH -n 4
	#SBATCH --gres=gpu:2
	module load centos6.4/lib/cuda/5.5
	 export OMP_NUM_THREADS=24
	 export MKL_NUM_THREADS=$OMP_NUM_THREADS
	 export PHI_DGEMM_SPLIT=0.975
	 export PHI_ZGEMM_SPLIT=0.975
	mpirun pw-gpu.x -nd 1 < si.in > out.GPU
	Sincerely Yours,
	     Mutlu.
------------------------------------------Mutlu ÇOLAKOĞULLARITrakya Universitesi Fen FakultesiFizik Bolumu22030 Merkez-EDIRNE/TURKEY
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/users/attachments/20150824/997740d3/attachment.html>