<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Then Intel compiler has a pre-processor macros that contains the compiler version called "__ICC". Just set it to be major of 1310 (so Intel 14.x minimum is required to use this compiler directive)<div class=""><div class=""><br class=""></div><div class=""><font face="Courier New" class="">#if defined(__ICC)<br class="">#if __ICC > 1310</font></div><div class=""><font face="Courier New" class="">!dir$ attributes align: 4096 :: yf, aux</font></div><div class=""><font face="Courier New" class="">#endif</font></div><div class=""><font face="Courier New" class="">#endif</font></div><div class=""><br class=""></div><div class="">It should fix the problem. I can apply and test it later today if you are ok. Carlo, opinions?</div><div class=""><br class=""></div><div class="">F</div><div class=""><br class=""></div><div class="">On Feb 24, 2015, at 6:42 AM, Paolo Giannozzi <<a href="mailto:paolo.giannozzi@uniud.it" class="">paolo.giannozzi@uniud.it</a>> wrote:</div><div class=""><blockquote type="cite" class="">It's line 76:<br class="">!dir$ attributes align: 4096 :: yf, aux<br class="">that makes my compiler (v.12) cras.<br class=""><br class="">P.<br class="">On Mon, 2015-02-23 at 22:12 +0000, Filippo Spiga wrote:<br class=""><blockquote type="cite" class="">Ciao Carlo,<br class=""><br class="">I spot that for a old (how old Paolo G?) version of Intel compiler the fft_parallel.f90 files with your latest changes generate a internal compiler error. It happens with and without MPI support.<br class=""><br class="">See: <br class="">- <a href="http://xiexie.syslab.disco.unimib.it:8010/builders/UDINE%20%28HYBRID%29" class="">http://xiexie.syslab.disco.unimib.it:8010/builders/UDINE%20%28HYBRID%29</a><br class="">- <a href="http://xiexie.syslab.disco.unimib.it:8010/builders/UDINE%20%28SERIAL%29/builds/4" class="">http://xiexie.syslab.disco.unimib.it:8010/builders/UDINE%20%28SERIAL%29/builds/4</a><br class="">- <a href="http://xiexie.syslab.disco.unimib.it:8010/builders/UDINE%20%28SERIAL%29/builds/4/steps/make%20all/logs/stdio" class="">http://xiexie.syslab.disco.unimib.it:8010/builders/UDINE%20%28SERIAL%29/builds/4/steps/make%20all/logs/stdio</a><br class=""><br class="">Did you test these changes using which version of Intel?<br class=""><br class="">Cheers,<br class="">Filippo<br class=""><br class="">Begin forwarded message:<br class=""><blockquote type="cite" class="">Date: February 23, 2015 at 11:14:16 AM GMT<br class="">From: ccavazzoni <<a href="mailto:ccavazzoni@qeforge.qe-forge.org" class="">ccavazzoni@qeforge.qe-forge.org</a>><br class="">To: <a href="mailto:q-e-commits@qe-forge.org" class="">q-e-commits@qe-forge.org</a><br class="">Subject: [Q-e-commits] r11407 - trunk/espresso/Modules<br class="">Reply-To: Quantum ESPRESSO svn commit messages <<a href="mailto:q-e-commits@qe-forge.org" class="">q-e-commits@qe-forge.org</a>><br class=""><br class="">Author: ccavazzoni<br class="">Date: 2015-02-23 12:14:15 +0100 (Mon, 23 Feb 2015)<br class="">New Revision: 11407<br class=""><br class="">Modified:<br class=""> trunk/espresso/Modules/fft_parallel.f90<br class=""> trunk/espresso/Modules/fft_scalar.f90<br class=""> trunk/espresso/Modules/wavefunctions.f90<br class="">Log:<br class="">- adding memory alignment directives to have a performance improvement on Intel architecure (CPU+Network),<br class="">only meaningful for intel compiler, shold be of no arm for all the other<br class="">- Intel DFTI MKL fft interface back in again (with __DFTI) in fft_scalar, some issues with the fftw3 interface<br class="">is prevending porting to Xeon PHI processor (no arm to all other procs/fft)<br class="">- A split in thress different subs of the general driver (tg_cft3s) is added to the fft_parallel module,<br class="">to support software pipelining optimizations, to mask communication and data transfer latency.<br class=""><br class=""><br class="">Modified: trunk/espresso/Modules/fft_parallel.f90<br class="">===================================================================<br class="">--- trunk/espresso/Modules/fft_parallel.f90<span class="Apple-tab-span" style="white-space:pre"> </span>2015-02-23 10:58:36 UTC (rev 11406)<br class="">+++ trunk/espresso/Modules/fft_parallel.f90<span class="Apple-tab-span" style="white-space:pre"> </span>2015-02-23 11:14:15 UTC (rev 11407)<br class="">@@ -73,6 +73,7 @@<br class=""> LOGICAL, OPTIONAL, INTENT(in) :: use_task_groups<br class=""> ! specify if you want to use task groups parallelization<br class=""> !<br class="">+!dir$ attributes align: 4096 :: yf, aux<br class=""> INTEGER :: me_p<br class=""> INTEGER :: n1, n2, n3, nx1, nx2, nx3<br class=""> COMPLEX(DP), ALLOCATABLE :: yf(:), aux (:)<br class="">@@ -329,4 +330,418 @@<br class=""> !<br class="">END SUBROUTINE tg_cft3s<br class="">!<br class="">+!<br class="">+!<br class="">+!----------------------------------------------------------------------------<br class="">+SUBROUTINE tg_cft3s_z( f, dfft, aux, isgn, use_task_groups )<br class="">+ !----------------------------------------------------------------------------<br class="">+ !<br class="">+ USE fft_scalar, ONLY : cft_1z, cft_2xy<br class="">+ USE fft_base, ONLY : fft_scatter<br class="">+ USE kinds, ONLY : DP<br class="">+ USE fft_types, ONLY : fft_dlay_descriptor<br class="">+ USE parallel_include<br class="">+<br class="">+ !<br class="">+ IMPLICIT NONE<br class="">+ !<br class="">+ COMPLEX(DP), INTENT(inout) :: f( : ) ! array containing data to be transformed<br class="">+ COMPLEX(DP), INTENT(inout) :: aux (:)<br class="">+ TYPE (fft_dlay_descriptor), INTENT(in) :: dfft<br class="">+ ! descriptor of fft data layout<br class="">+ INTEGER, INTENT(in) :: isgn ! fft direction<br class="">+ LOGICAL, OPTIONAL, INTENT(in) :: use_task_groups<br class="">+ ! specify if you want to use task groups parallelization<br class="">+ !<br class="">+ INTEGER :: me_p<br class="">+ INTEGER :: n1, n2, n3, nx1, nx2, nx3<br class="">+ COMPLEX(DP), ALLOCATABLE :: yf(:)<br class="">+ INTEGER :: planes( dfft%nr1x )<br class="">+ LOGICAL :: use_tg<br class="">+ !<br class="">+ !<br class="">+ IF( present( use_task_groups ) ) THEN<br class="">+ use_tg = use_task_groups<br class="">+ ELSE<br class="">+ use_tg = .false.<br class="">+ ENDIF<br class="">+ !<br class="">+ IF( use_tg .and. .not. dfft%have_task_groups ) &<br class="">+ CALL errore( ' tg_cft3s_x ', ' call requiring task groups for a descriptor without task groups ', 1 )<br class="">+ !<br class="">+ n1 = dfft%nr1<br class="">+ n2 = dfft%nr2<br class="">+ n3 = dfft%nr3<br class="">+ nx1 = dfft%nr1x<br class="">+ nx2 = dfft%nr2x<br class="">+ nx3 = dfft%nr3x<br class="">+ !<br class="">+ IF( use_tg ) THEN<br class="">+ ALLOCATE( YF ( dfft%nogrp * dfft%tg_nnr ) )<br class="">+ ENDIF<br class="">+ !<br class="">+ me_p = dfft%mype + 1<br class="">+ !<br class="">+ IF ( isgn > 0 ) THEN<br class="">+ !<br class="">+ IF ( isgn /= 2 ) THEN<br class="">+ !<br class="">+ IF( use_tg ) &<br class="">+ CALL errore( ' tg_cft3s ', ' task groups on large mesh not implemented ', 1 )<br class="">+ !<br class="">+ CALL cft_1z( f, dfft%nsp( me_p ), n3, nx3, isgn, aux )<br class="">+ !<br class="">+ ELSE<br class="">+ !<br class="">+ CALL pack_group_sticks()<br class="">+ !<br class="">+ IF( use_tg ) THEN<br class="">+ CALL cft_1z( yf, dfft%tg_nsw( me_p ), n3, nx3, isgn, aux )<br class="">+ ELSE<br class="">+ CALL cft_1z( f, dfft%nsw( me_p ), n3, nx3, isgn, aux )<br class="">+ ENDIF<br class="">+ !<br class="">+ ENDIF<br class="">+ !<br class="">+ ELSE<br class="">+ !<br class="">+ IF ( isgn /= -2 ) THEN<br class="">+ !<br class="">+ IF( use_tg ) &<br class="">+ CALL errore( ' tg_cft3s ', ' task groups on large mesh not implemented ', 1 )<br class="">+ !<br class="">+ ENDIF<br class="">+<br class="">+ IF ( isgn /= -2 ) THEN<br class="">+ !<br class="">+ CALL cft_1z( aux, dfft%nsp( me_p ), n3, nx3, isgn, f )<br class="">+ !<br class="">+ ELSE<br class="">+ !<br class="">+ IF( use_tg ) THEN<br class="">+ CALL cft_1z( aux, dfft%tg_nsw( me_p ), n3, nx3, isgn, yf )<br class="">+ ELSE<br class="">+ CALL cft_1z( aux, dfft%nsw( me_p ), n3, nx3, isgn, f )<br class="">+ ENDIF<br class="">+ !<br class="">+ CALL unpack_group_sticks()<br class="">+ !<br class="">+ ENDIF<br class="">+ !<br class="">+ ENDIF<br class="">+ !<br class="">+ IF( use_tg ) THEN<br class="">+ DEALLOCATE( yf )<br class="">+ ENDIF<br class="">+ !<br class="">+ RETURN<br class="">+ !<br class="">+CONTAINS<br class="">+ !<br class="">+ SUBROUTINE pack_group_sticks()<br class="">+<br class="">+ INTEGER :: ierr<br class="">+ !<br class="">+ IF( .not. use_tg ) RETURN<br class="">+ !<br class="">+ IF( dfft%tg_rdsp(dfft%nogrp) + dfft%tg_rcv(dfft%nogrp) > size( yf ) ) THEN<br class="">+ CALL errore( 'pack_group_sticks' , ' inconsistent size ', 1 )<br class="">+ ENDIF<br class="">+ IF( dfft%tg_psdsp(dfft%nogrp) + dfft%tg_snd(dfft%nogrp) > size( f ) ) THEN<br class="">+ CALL errore( 'pack_group_sticks', ' inconsistent size ', 2 )<br class="">+ ENDIF<br class="">+<br class="">+ CALL start_clock( 'ALLTOALL' )<br class="">+ !<br class="">+ ! Collect all the sticks of the different states,<br class="">+ ! in "yf" processors will have all the sticks of the OGRP<br class="">+<br class="">+#if defined __MPI<br class="">+<br class="">+ CALL MPI_ALLTOALLV( f(1), dfft%tg_snd, dfft%tg_psdsp, MPI_DOUBLE_COMPLEX, yf(1), dfft%tg_rcv, &<br class="">+ & dfft%tg_rdsp, MPI_DOUBLE_COMPLEX, dfft%ogrp_comm, IERR)<br class="">+ IF( ierr /= 0 ) THEN<br class="">+ CALL errore( 'pack_group_sticks', ' alltoall error 1 ', abs(ierr) )<br class="">+ ENDIF<br class="">+<br class="">+#endif<br class="">+<br class="">+ CALL stop_clock( 'ALLTOALL' )<br class="">+ !<br class="">+ !YF Contains all ( ~ NOGRP*dfft%nsw(me) ) Z-sticks<br class="">+ !<br class="">+ RETURN<br class="">+ END SUBROUTINE pack_group_sticks<br class="">+<br class="">+ !<br class="">+<br class="">+ SUBROUTINE unpack_group_sticks()<br class="">+ !<br class="">+ ! Bring pencils back to their original distribution<br class="">+ !<br class="">+ INTEGER :: ierr<br class="">+ !<br class="">+ IF( .not. use_tg ) RETURN<br class="">+ !<br class="">+ IF( dfft%tg_usdsp(dfft%nogrp) + dfft%tg_snd(dfft%nogrp) > size( f ) ) THEN<br class="">+ CALL errore( 'unpack_group_sticks', ' inconsistent size ', 3 )<br class="">+ ENDIF<br class="">+ IF( dfft%tg_rdsp(dfft%nogrp) + dfft%tg_rcv(dfft%nogrp) > size( yf ) ) THEN<br class="">+ CALL errore( 'unpack_group_sticks', ' inconsistent size ', 4 )<br class="">+ ENDIF<br class="">+<br class="">+ CALL start_clock( 'ALLTOALL' )<br class="">+<br class="">+#if defined __MPI<br class="">+ CALL MPI_Alltoallv( yf(1), &<br class="">+ dfft%tg_rcv, dfft%tg_rdsp, MPI_DOUBLE_COMPLEX, f(1), &<br class="">+ dfft%tg_snd, dfft%tg_usdsp, MPI_DOUBLE_COMPLEX, dfft%ogrp_comm, IERR)<br class="">+ IF( ierr /= 0 ) THEN<br class="">+ CALL errore( 'unpack_group_sticks', ' alltoall error 2 ', abs(ierr) )<br class="">+ ENDIF<br class="">+#endif<br class="">+<br class="">+ CALL stop_clock( 'ALLTOALL' )<br class="">+<br class="">+ RETURN<br class="">+ END SUBROUTINE unpack_group_sticks<br class="">+ !<br class="">+END SUBROUTINE tg_cft3s_z<br class="">+!<br class="">+!<br class="">+!----------------------------------------------------------------------------<br class="">+SUBROUTINE tg_cft3s_scatter( f, dfft, aux, isgn, use_task_groups )<br class="">+ !----------------------------------------------------------------------------<br class="">+ !<br class="">+ USE fft_scalar, ONLY : cft_1z, cft_2xy<br class="">+ USE fft_base, ONLY : fft_scatter<br class="">+ USE kinds, ONLY : DP<br class="">+ USE fft_types, ONLY : fft_dlay_descriptor<br class="">+ USE parallel_include<br class="">+<br class="">+ !<br class="">+ IMPLICIT NONE<br class="">+ !<br class="">+ COMPLEX(DP), INTENT(inout) :: f( : ), aux( : ) ! array containing data to be transformed<br class="">+ TYPE (fft_dlay_descriptor), INTENT(in) :: dfft<br class="">+ ! descriptor of fft data layout<br class="">+ INTEGER, INTENT(in) :: isgn ! fft direction<br class="">+ LOGICAL, OPTIONAL, INTENT(in) :: use_task_groups<br class="">+ ! specify if you want to use task groups parallelization<br class="">+ !<br class="">+ INTEGER :: me_p<br class="">+ INTEGER :: n1, n2, n3, nx1, nx2, nx3<br class="">+ INTEGER :: planes( dfft%nr1x )<br class="">+ LOGICAL :: use_tg<br class="">+ !<br class="">+ !<br class="">+ IF( present( use_task_groups ) ) THEN<br class="">+ use_tg = use_task_groups<br class="">+ ELSE<br class="">+ use_tg = .false.<br class="">+ ENDIF<br class="">+ !<br class="">+ IF( use_tg .and. .not. dfft%have_task_groups ) &<br class="">+ CALL errore( ' tg_cft3s ', ' call requiring task groups for a descriptor without task groups ', 1 )<br class="">+ !<br class="">+ n1 = dfft%nr1<br class="">+ n2 = dfft%nr2<br class="">+ n3 = dfft%nr3<br class="">+ nx1 = dfft%nr1x<br class="">+ nx2 = dfft%nr2x<br class="">+ nx3 = dfft%nr3x<br class="">+ !<br class="">+ me_p = dfft%mype + 1<br class="">+ !<br class="">+ IF ( isgn > 0 ) THEN<br class="">+ !<br class="">+ IF ( isgn /= 2 ) THEN<br class="">+ !<br class="">+ IF( use_tg ) &<br class="">+ CALL errore( ' tg_cft3s ', ' task groups on large mesh not implemented ', 1 )<br class="">+ !<br class="">+ ENDIF<br class="">+ !<br class="">+ CALL fw_scatter( isgn ) ! forwart scatter from stick to planes<br class="">+ !<br class="">+ ELSE<br class="">+ !<br class="">+ IF ( isgn /= -2 ) THEN<br class="">+ !<br class="">+ IF( use_tg ) &<br class="">+ CALL errore( ' tg_cft3s ', ' task groups on large mesh not implemented ', 1 )<br class="">+ !<br class="">+ ENDIF<br class="">+ !<br class="">+ CALL bw_scatter( isgn )<br class="">+ !<br class="">+ ENDIF<br class="">+ !<br class="">+ RETURN<br class="">+ !<br class="">+CONTAINS<br class="">+ !<br class="">+ SUBROUTINE fw_scatter( iopt )<br class="">+<br class="">+ !Transpose data for the 2-D FFT on the x-y plane<br class="">+ !<br class="">+ !NOGRP*dfft%nnr: The length of aux and f<br class="">+ !nr3x: The length of each Z-stick<br class="">+ !aux: input - output<br class="">+ !f: working space<br class="">+ !isgn: type of scatter<br class="">+ !dfft%nsw(me) holds the number of Z-sticks proc. me has.<br class="">+ !dfft%npp: number of planes per processor<br class="">+ !<br class="">+ !<br class="">+ USE fft_base, ONLY: fft_scatter<br class="">+ !<br class="">+ INTEGER, INTENT(in) :: iopt<br class="">+ !<br class="">+ IF( iopt == 2 ) THEN<br class="">+ !<br class="">+ IF( use_tg ) THEN<br class="">+ !<br class="">+ CALL fft_scatter( dfft, aux, nx3, dfft%nogrp*dfft%tg_nnr, f, dfft%tg_nsw, dfft%tg_npp, iopt, use_tg )<br class="">+ !<br class="">+ ELSE<br class="">+ !<br class="">+ CALL fft_scatter( dfft, aux, nx3, dfft%nnr, f, dfft%nsw, dfft%npp, iopt )<br class="">+ !<br class="">+ ENDIF<br class="">+ !<br class="">+ ELSEIF( iopt == 1 ) THEN<br class="">+ !<br class="">+ CALL fft_scatter( dfft, aux, nx3, dfft%nnr, f, dfft%nsp, dfft%npp, iopt )<br class="">+ !<br class="">+ ENDIF<br class="">+ !<br class="">+ RETURN<br class="">+ END SUBROUTINE fw_scatter<br class="">+<br class="">+ !<br class="">+<br class="">+ SUBROUTINE bw_scatter( iopt )<br class="">+ !<br class="">+ USE fft_base, ONLY: fft_scatter<br class="">+ !<br class="">+ INTEGER, INTENT(in) :: iopt<br class="">+ !<br class="">+ IF( iopt == -2 ) THEN<br class="">+ !<br class="">+ IF( use_tg ) THEN<br class="">+ !<br class="">+ CALL fft_scatter( dfft, aux, nx3, dfft%nogrp*dfft%tg_nnr, f, dfft%tg_nsw, dfft%tg_npp, iopt, use_tg )<br class="">+ !<br class="">+ ELSE<br class="">+ !<br class="">+ CALL fft_scatter( dfft, aux, nx3, dfft%nnr, f, dfft%nsw, dfft%npp, iopt )<br class="">+ !<br class="">+ ENDIF<br class="">+ !<br class="">+ ELSEIF( iopt == -1 ) THEN<br class="">+ !<br class="">+ CALL fft_scatter( dfft, aux, nx3, dfft%nnr, f, dfft%nsp, dfft%npp, iopt )<br class="">+ !<br class="">+ ENDIF<br class="">+ !<br class="">+ RETURN<br class="">+ END SUBROUTINE bw_scatter<br class="">+ !<br class="">+END SUBROUTINE tg_cft3s_scatter<br class="">+!<br class="">+!----------------------------------------------------------------------------<br class="">+SUBROUTINE tg_cft3s_xy( f, dfft, aux, isgn, use_task_groups )<br class="">+ !----------------------------------------------------------------------------<br class="">+ !<br class="">+ USE fft_scalar, ONLY : cft_1z, cft_2xy<br class="">+ USE fft_base, ONLY : fft_scatter<br class="">+ USE kinds, ONLY : DP<br class="">+ USE fft_types, ONLY : fft_dlay_descriptor<br class="">+ USE parallel_include<br class="">+<br class="">+ !<br class="">+ IMPLICIT NONE<br class="">+ !<br class="">+ COMPLEX(DP), INTENT(inout) :: f( : ), aux( : ) ! array containing data to be transformed<br class="">+ TYPE (fft_dlay_descriptor), INTENT(in) :: dfft<br class="">+ ! descriptor of fft data layout<br class="">+ INTEGER, INTENT(in) :: isgn ! fft direction<br class="">+ LOGICAL, OPTIONAL, INTENT(in) :: use_task_groups<br class="">+ ! specify if you want to use task groups parallelization<br class="">+ !<br class="">+ INTEGER :: me_p<br class="">+ INTEGER :: n1, n2, n3, nx1, nx2, nx3<br class="">+ COMPLEX(DP), ALLOCATABLE :: yf(:)<br class="">+ INTEGER :: planes( dfft%nr1x )<br class="">+ LOGICAL :: use_tg<br class="">+ !<br class="">+ !<br class="">+ IF( present( use_task_groups ) ) THEN<br class="">+ use_tg = use_task_groups<br class="">+ ELSE<br class="">+ use_tg = .false.<br class="">+ ENDIF<br class="">+ !<br class="">+ IF( use_tg .and. .not. dfft%have_task_groups ) &<br class="">+ CALL errore( ' tg_cft3s ', ' call requiring task groups for a descriptor without task groups ', 1 )<br class="">+ !<br class="">+ n1 = dfft%nr1<br class="">+ n2 = dfft%nr2<br class="">+ n3 = dfft%nr3<br class="">+ nx1 = dfft%nr1x<br class="">+ nx2 = dfft%nr2x<br class="">+ nx3 = dfft%nr3x<br class="">+ !<br class="">+ me_p = dfft%mype + 1<br class="">+ !<br class="">+ IF ( isgn > 0 ) THEN<br class="">+ !<br class="">+ IF ( isgn /= 2 ) THEN<br class="">+ !<br class="">+ IF( use_tg ) &<br class="">+ CALL errore( ' tg_cft3s ', ' task groups on large mesh not implemented ', 1 )<br class="">+ !<br class="">+ planes = dfft%iplp<br class="">+ !<br class="">+ ELSE<br class="">+ !<br class="">+ planes = dfft%iplw<br class="">+ !<br class="">+ ENDIF<br class="">+ !<br class="">+ IF( use_tg ) THEN<br class="">+ CALL cft_2xy( f, dfft%tg_npp( me_p ), n1, n2, nx1, nx2, isgn, planes )<br class="">+ ELSE<br class="">+ CALL cft_2xy( f, dfft%npp( me_p ), n1, n2, nx1, nx2, isgn, planes )<br class="">+ ENDIF<br class="">+ !<br class="">+ ELSE<br class="">+ !<br class="">+ IF ( isgn /= -2 ) THEN<br class="">+ !<br class="">+ IF( use_tg ) &<br class="">+ CALL errore( ' tg_cft3s ', ' task groups on large mesh not implemented ', 1 )<br class="">+ !<br class="">+ planes = dfft%iplp<br class="">+ !<br class="">+ ELSE<br class="">+ !<br class="">+ planes = dfft%iplw<br class="">+ !<br class="">+ ENDIF<br class="">+<br class="">+ IF( use_tg ) THEN<br class="">+ CALL cft_2xy( f, dfft%tg_npp( me_p ), n1, n2, nx1, nx2, isgn, planes )<br class="">+ ELSE<br class="">+ CALL cft_2xy( f, dfft%npp( me_p ), n1, n2, nx1, nx2, isgn, planes)<br class="">+ ENDIF<br class="">+ !<br class="">+ ENDIF<br class="">+ !<br class="">+ RETURN<br class="">+ !<br class="">+END SUBROUTINE tg_cft3s_xy<br class="">+!<br class="">+<br class="">END MODULE fft_parallel<br class=""><br class="">Modified: trunk/espresso/Modules/fft_scalar.f90<br class="">===================================================================<br class="">--- trunk/espresso/Modules/fft_scalar.f90<span class="Apple-tab-span" style="white-space:pre"> </span>2015-02-23 10:58:36 UTC (rev 11406)<br class="">+++ trunk/espresso/Modules/fft_scalar.f90<span class="Apple-tab-span" style="white-space:pre"> </span>2015-02-23 11:14:15 UTC (rev 11407)<br class="">@@ -16,11 +16,20 @@<br class="">!--------------------------------------------------------------------------!<br class=""><br class="">#include "fft_defs.h"<br class="">+<br class="">+#if defined __DFTI<br class="">+#include "mkl_dfti.f90"<br class="">+#endif<br class="">+<br class="">!=----------------------------------------------------------------------=!<br class=""> MODULE fft_scalar<br class="">!=----------------------------------------------------------------------=!<br class=""> USE kinds<br class=""><br class="">+#if defined __DFTI<br class="">+ USE MKL_DFTI ! -- this can be found int he MKL include directory<br class="">+#endif<br class="">+<br class=""> IMPLICIT NONE<br class=""> SAVE<br class=""><br class="">@@ -77,6 +86,12 @@<br class=""><br class="">#endif<br class=""><br class="">+#if defined __DFTI<br class="">+ TYPE dfti_descriptor_array<br class="">+ TYPE(DFTI_DESCRIPTOR), POINTER :: desc<br class="">+ END TYPE<br class="">+#endif<br class="">+<br class="">!=----------------------------------------------------------------------=!<br class=""> CONTAINS<br class="">!=----------------------------------------------------------------------=!<br class="">@@ -95,6 +110,10 @@<br class=""><br class=""> SUBROUTINE cft_1z(c, nsl, nz, ldz, isign, cout)<br class=""><br class="">+#if defined __DFTI<br class="">+ USE iso_c_binding<br class="">+#endif<br class="">+<br class="">! driver routine for nsl 1d complex fft's of length nz<br class="">! ldz >= nz is the distance between sequences to be transformed<br class="">! (ldz>nz is used on some architectures to reduce memory conflicts)<br class="">@@ -142,6 +161,21 @@<br class=""> C_POINTER, SAVE :: fw_planz( ndims ) = 0<br class=""> C_POINTER, SAVE :: bw_planz( ndims ) = 0<br class=""><br class="">+#elif defined __DFTI<br class="">+<br class="">+ ! Intel MKL native FFT driver<br class="">+<br class="">+ TYPE(DFTI_DESCRIPTOR_ARRAY), SAVE :: hand( ndims )<br class="">+ LOGICAL, SAVE :: dfti_first = .TRUE.<br class="">+ INTEGER :: dfti_status = 0<br class="">+ !<br class="">+ IF( dfti_first .EQ. .TRUE. ) THEN<br class="">+ DO ip = 1, ndims<br class="">+ hand(ip)%desc => NULL()<br class="">+ END DO<br class="">+ dfti_first = .FALSE.<br class="">+ END IF<br class="">+<br class="">#elif defined __ESSL || defined __LINUX_ESSL<br class=""><br class=""> ! ESSL IBM library: see the ESSL manual for DCFT<br class="">@@ -207,7 +241,6 @@<br class=""> ! initialize a new one<br class=""><br class=""> ! WRITE( stdout, fmt="('DEBUG cft_1z, reinitializing tables ', I3)" ) icurrent<br class="">-<br class="">#if defined __FFTW<br class=""><br class=""> IF( fw_planz( icurrent) /= 0 ) CALL DESTROY_PLAN_1D( fw_planz( icurrent) )<br class="">@@ -227,6 +260,53 @@<br class=""> CALL dfftw_plan_many_dft( bw_planz( icurrent), 1, nz, nsl, c, &<br class=""> (/SIZE(c)/), 1, ldz, cout, (/SIZE(cout)/), 1, ldz, idir, FFTW_ESTIMATE)<br class=""><br class="">+#elif defined __DFTI<br class="">+<br class="">+ if( ASSOCIATED( hand( icurrent )%desc ) ) THEN<br class="">+ dfti_status = DftiFreeDescriptor( hand( icurrent )%desc )<br class="">+ IF( dfti_status /= 0) THEN<br class="">+ WRITE(*,*) "stopped in DftiFreeDescriptor", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ END IF<br class="">+<br class="">+ dfti_status = DftiCreateDescriptor(hand( icurrent )%desc, DFTI_DOUBLE, DFTI_COMPLEX, 1,nz)<br class="">+ IF(dfti_status /= 0) THEN<br class="">+ WRITE(*,*) "stopped in DftiCreateDescriptor", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ dfti_status = DftiSetValue(hand( icurrent )%desc, DFTI_NUMBER_OF_TRANSFORMS,nsl)<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DFTI_NUMBER_OF_TRANSFORMS", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ dfti_status = DftiSetValue(hand( icurrent )%desc,DFTI_INPUT_DISTANCE, ldz )<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DFTI_INPUT_DISTANCE", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ dfti_status = DftiSetValue(hand( icurrent )%desc, DFTI_PLACEMENT, DFTI_INPLACE)<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DFTI_PLACEMENT", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ tscale = 1.0_DP/nz<br class="">+ dfti_status = DftiSetValue( hand( icurrent )%desc, DFTI_FORWARD_SCALE, tscale);<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DFTI_FORWARD_SCALE", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ dfti_status = DftiSetValue( hand( icurrent )%desc, DFTI_BACKWARD_SCALE, DBLE(1) );<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DFTI_BACKWARD_SCALE", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ dfti_status = DftiCommitDescriptor(hand( icurrent )%desc)<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DftiCommitDescriptor", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+<br class="">#elif defined __ESSL || defined __LINUX_ESSL<br class=""><br class=""> tscale = 1.0_DP / nz<br class="">@@ -286,11 +366,9 @@<br class=""> CALL FFT_Z_STICK_SINGLE(fw_planz( ip), c(offset), ldz_t)<br class=""> END DO<br class="">!$omp end do<br class="">+!$omp end parallel<br class=""> tscale = 1.0_DP / nz<br class="">-!$omp workshare<br class=""> cout( 1 : ldz * nsl ) = c( 1 : ldz * nsl ) * tscale<br class="">-!$omp end workshare<br class="">-!$omp end parallel<br class=""> ELSE IF (isign > 0) THEN<br class="">!$omp parallel default(none) private(tid,offset,i) shared(c,isign,nsl,bw_planz,ip,cout,ldz) &<br class="">!$omp & firstprivate(ldz_t)<br class="">@@ -330,6 +408,24 @@<br class=""> CALL dfftw_execute_dft( bw_planz( ip), c, cout)<br class=""> END IF<br class=""><br class="">+#elif defined __DFTI<br class="">+<br class="">+ IF (isign < 0) THEN<br class="">+ dfti_status = DftiComputeForward(hand(ip)%desc, c )<br class="">+ cout( 1 : ldz * nsl ) = c( 1 : ldz * nsl )<br class="">+ IF(dfti_status /= 0) THEN<br class="">+ WRITE(*,*) "stopped in DftiComputeForward", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ ELSE IF (isign > 0) THEN<br class="">+ dfti_status = DftiComputeBackward(hand(ip)%desc, c )<br class="">+ cout( 1 : ldz * nsl ) = c( 1 : ldz * nsl )<br class="">+ IF(dfti_status /= 0) THEN<br class="">+ WRITE(*,*) "stopped in DftiComputeBackward", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ END IF<br class="">+<br class="">#elif defined __SCSL<br class=""><br class=""> IF ( isign < 0 ) THEN<br class="">@@ -415,6 +511,10 @@<br class=""><br class=""> SUBROUTINE cft_2xy(r, nzl, nx, ny, ldx, ldy, isign, pl2ix)<br class=""><br class="">+#if defined __DFTI<br class="">+ USE iso_c_binding<br class="">+#endif<br class="">+<br class="">! driver routine for nzl 2d complex fft's of lengths nx and ny<br class="">! input : r(ldx*ldy) complex, transform is in-place<br class="">! ldx >= nx, ldy >= ny are the physical dimensions of the equivalent<br class="">@@ -448,11 +548,23 @@<br class=""> EXTERNAL :: omp_get_thread_num, omp_get_num_threads<br class="">#endif<br class=""><br class="">-#if defined __FFTW || defined __FFTW3<br class="">+#if defined __DFTI<br class=""><br class="">+ TYPE(DFTI_DESCRIPTOR_ARRAY), SAVE :: hand( ndims )<br class="">+ LOGICAL, SAVE :: dfti_first = .TRUE.<br class="">+ INTEGER :: dfti_status = 0<br class="">+<br class="">+#elif defined __FFTW || defined __FFTW3<br class="">+<br class="">+#if defined __FFTW && __FFTW_ALL_XY_PLANES<br class="">+ C_POINTER, SAVE :: fw_plan_2d( ndims ) = 0<br class="">+ C_POINTER, SAVE :: bw_plan_2d( ndims ) = 0<br class="">+#else<br class=""> C_POINTER, SAVE :: fw_plan( 2, ndims ) = 0<br class=""> C_POINTER, SAVE :: bw_plan( 2, ndims ) = 0<br class="">+#endif<br class=""><br class="">+<br class="">#elif defined __ESSL || defined __LINUX_ESSL<br class=""><br class=""> INTEGER, PARAMETER :: ltabl = 20000 + 3 * nfftx<br class="">@@ -503,6 +615,14 @@<br class=""> !<br class=""> ! Here initialize table only if necessary<br class=""> !<br class="">+#if defined __DFTI<br class="">+ IF( dfti_first .EQ. .TRUE. ) THEN<br class="">+ DO ip = 1, ndims<br class="">+ hand(ip)%desc => NULL()<br class="">+ END DO<br class="">+ dfti_first = .FALSE.<br class="">+ END IF<br class="">+#endif<br class=""><br class=""> DO ip = 1, ndims<br class=""><br class="">@@ -524,8 +644,62 @@<br class=""><br class=""> ! WRITE( stdout, fmt="('DEBUG cft_2xy, reinitializing tables ', I3)" ) icurrent<br class=""><br class="">-#if defined __FFTW<br class="">+#if defined __DFTI<br class=""><br class="">+ if( ASSOCIATED( hand( icurrent )%desc ) ) THEN<br class="">+ dfti_status = DftiFreeDescriptor( hand( icurrent )%desc )<br class="">+ IF( dfti_status /= 0) THEN<br class="">+ WRITE(*,*) "stopped in DftiFreeDescriptor", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ END IF<br class="">+<br class="">+ dfti_status = DftiCreateDescriptor(hand( icurrent )%desc, DFTI_DOUBLE, DFTI_COMPLEX, 2,(/nx,ny/))<br class="">+ IF(dfti_status /= 0) THEN<br class="">+ WRITE(*,*) "stopped in DftiCreateDescriptor", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ dfti_status = DftiSetValue(hand( icurrent )%desc, DFTI_NUMBER_OF_TRANSFORMS,nzl)<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DFTI_NUMBER_OF_TRANSFORMS", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ dfti_status = DftiSetValue(hand( icurrent )%desc,DFTI_INPUT_DISTANCE, ldx*ldy )<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DFTI_INPUT_DISTANCE", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ dfti_status = DftiSetValue(hand( icurrent )%desc, DFTI_PLACEMENT, DFTI_INPLACE)<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DFTI_PLACEMENT", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ tscale = 1.0_DP/ (nx * ny )<br class="">+ dfti_status = DftiSetValue( hand( icurrent )%desc, DFTI_FORWARD_SCALE, tscale);<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DFTI_FORWARD_SCALE", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ dfti_status = DftiSetValue( hand( icurrent )%desc, DFTI_BACKWARD_SCALE, DBLE(1) );<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DFTI_BACKWARD_SCALE", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ dfti_status = DftiCommitDescriptor(hand( icurrent )%desc)<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DftiCommitDescriptor", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+<br class="">+<br class="">+#elif defined __FFTW<br class="">+<br class="">+#if defined __FFTW_ALL_XY_PLANES<br class="">+ IF( fw_plan_2d( icurrent) /= 0 ) CALL DESTROY_PLAN_2D(fw_plan_2d(icurrent) )<br class="">+ IF( bw_plan_2d( icurrent) /= 0 ) CALL DESTROY_PLAN_2D(bw_plan_2d(icurrent) )<br class="">+ idir = -1; CALL CREATE_PLAN_2D( fw_plan_2d(icurrent), nx, ny, idir)<br class="">+ idir = 1; CALL CREATE_PLAN_2D( bw_plan_2d(icurrent), nx, ny, idir)<br class="">+#else<br class=""> IF( fw_plan( 2,icurrent) /= 0 ) CALL DESTROY_PLAN_1D( fw_plan( 2,icurrent) )<br class=""> IF( bw_plan( 2,icurrent) /= 0 ) CALL DESTROY_PLAN_1D( bw_plan( 2,icurrent) )<br class=""> idir = -1; CALL CREATE_PLAN_1D( fw_plan( 2,icurrent), ny, idir)<br class="">@@ -535,6 +709,7 @@<br class=""> IF( bw_plan( 1,icurrent) /= 0 ) CALL DESTROY_PLAN_1D( bw_plan( 1,icurrent) )<br class=""> idir = -1; CALL CREATE_PLAN_1D( fw_plan( 1,icurrent), nx, idir)<br class=""> idir = 1; CALL CREATE_PLAN_1D( bw_plan( 1,icurrent), nx, idir)<br class="">+#endif<br class=""><br class="">#elif defined __FFTW3<br class=""><br class="">@@ -645,10 +820,46 @@<br class="">#endif<br class=""><br class=""><br class="">-#if defined __FFTW<br class="">+#if defined __DFTI<br class=""><br class="">-#if defined __OPENMP<br class="">+ IF( isign < 0 ) THEN<br class="">+ !<br class="">+ dfti_status = DftiComputeForward(hand(ip)%desc, r(:))<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DftiComputeForward", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ !<br class="">+ ELSE IF( isign > 0 ) THEN<br class="">+ !<br class="">+ dfti_status = DftiComputeBackward(hand(ip)%desc, r(:))<br class="">+ IF(dfti_status /= 0)THEN<br class="">+ WRITE(*,*) "stopped in DftiComputeBackward", dfti_status<br class="">+ STOP<br class="">+ ENDIF<br class="">+ !<br class="">+ END IF<br class=""><br class="">+<br class="">+#elif defined __FFTW<br class="">+<br class="">+#if defined __FFTW_ALL_XY_PLANES<br class="">+<br class="">+ IF( isign < 0 ) THEN<br class="">+ !<br class="">+ tscale = 1.0_DP / ( nx * ny )<br class="">+ !<br class="">+ CALL fftw_inplace_drv_2d( fw_plan_2d(ip), nzl, r(1), 1, ldx*ldy )<br class="">+ CALL ZDSCAL( ldx * ldy * nzl, tscale, r(1), 1)<br class="">+ !<br class="">+ ELSE IF( isign > 0 ) THEN<br class="">+ !<br class="">+ CALL fftw_inplace_drv_2d( bw_plan_2d(ip), nzl, r(1), 1, ldx*ldy )<br class="">+ !<br class="">+ END IF<br class="">+<br class="">+#elif defined __OPENMP<br class="">+<br class=""> nx_t = nx<br class=""> ny_t = ny<br class=""> nzl_t = nzl<br class="">@@ -1980,6 +2191,7 @@<br class=""> !<br class=""> if (mod (n, 2) ==0) nx = n + 1<br class=""> ! for nec vector machines: if n is even increase dimension by 1<br class="">+ !<br class="">#endif<br class=""> !<br class=""> good_fft_dimension = nx<br class=""><br class="">Modified: trunk/espresso/Modules/wavefunctions.f90<br class="">===================================================================<br class="">--- trunk/espresso/Modules/wavefunctions.f90<span class="Apple-tab-span" style="white-space:pre"> </span>2015-02-23 10:58:36 UTC (rev 11406)<br class="">+++ trunk/espresso/Modules/wavefunctions.f90<span class="Apple-tab-span" style="white-space:pre"> </span>2015-02-23 11:14:15 UTC (rev 11407)<br class="">@@ -28,6 +28,7 @@<br class=""> ! electronic wave functions, CPV code<br class=""> ! distributed over gvector and bands<br class=""> !<br class="">+!dir$ attributes align: 4096 :: c0_bgrp, cm_bgrp, phi_bgrp<br class=""> COMPLEX(DP), ALLOCATABLE :: c0_bgrp(:,:) ! wave functions at time t<br class=""> COMPLEX(DP), ALLOCATABLE :: cm_bgrp(:,:) ! wave functions at time t-delta t<br class=""> COMPLEX(DP), ALLOCATABLE :: phi_bgrp(:,:) ! |phi> = s'|c0> = |c0> + sum q_ij |i><j|c0><br class=""><br class="">_______________________________________________<br class="">Q-e-commits mailing list<br class=""><a href="mailto:Q-e-commits@qe-forge.org" class="">Q-e-commits@qe-forge.org</a><br class="">http://qe-forge.org/mailman/listinfo/q-e-commits<br class=""></blockquote><br class="">--<br class="">Mr. Filippo SPIGA, M.Sc.<br class=""><a href="http://filippospiga.info" class="">http://filippospiga.info</a> ~ skype: filippo.spiga<br class=""><br class="">«Nobody will drive us out of Cantor's paradise.» ~ David Hilbert<br class=""><br class="">*****<br class="">Disclaimer: "Please note this message and any attachments are CONFIDENTIAL and may be privileged or otherwise protected from disclosure. The contents are not to be disclosed to anyone other than the addressee. Unauthorized recipients are requested to preserve this confidentiality and to advise the sender immediately of any error in transmission."<br class=""><br class=""><br class=""></blockquote><br class=""><br class=""></blockquote><br class=""><div class="">--<br class="">Mr. Filippo SPIGA, M.Sc.<br class=""><a href="http://filippospiga.info" class="">http://filippospiga.info</a> ~ skype: filippo.spiga<br class=""><br class="">«Nobody will drive us out of Cantor's paradise.» ~ David Hilbert<br class=""><br class="">*****<br class="">Disclaimer: "Please note this message and any attachments are CONFIDENTIAL and may be privileged or otherwise protected from disclosure. The contents are not to be disclosed to anyone other than the addressee. Unauthorized recipients are requested to preserve this confidentiality and to advise the sender immediately of any error in transmission."<br class=""><br class=""></div><br class=""></div></div></body></html>