<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Dear all, </div><div><br></div><div>After some trial and error, following advices given here, i come up with the solution on how to make a c++ library from PW with a controllable MPI communicator. Everything compiles fine with a small example, but i did not yet implement passing input file name. </div><div><br></div><div>In a hope that it will be useful / interesting for other users / developers, i write a quick list of how to achieve it below. </div><div>Things can definitely be made more pretty, i would be grateful for any advices. </div><div>Maybe a more clean version of this can make it to a release one day. I think such an interface makes it easier to combine different codes while having a full control over MPI. </div><div>.check_pw.x.j runs fine so far. I did not try anything else yet. </div><div><br></div><div>I hope the content below is appropriate for this mailing list, please, excuse me if it is not. </div><div><br></div><div>HOWTO: </div><div>1. pwscf.f90 -- i wrapped the main function in another function. </div><div><br></div><div><div style="margin: 0px; font-size: 11px; font-family: Monaco; ">PROGRAM pwscf</div></div><div><div style="margin: 0px; font-size: 11px; font-family: Monaco; "> USE parallel_include</div><div style="margin: 0px; font-size: 11px; font-family: Monaco; ">!</div><div style="margin: 0px; font-size: 11px; font-family: Monaco; "> INTEGER :: comm = mpi_comm_world</div><div style="margin: 0px; font-size: 11px; font-family: Monaco; "> INTEGER :: sz</div><div style="margin: 0px; font-size: 11px; font-family: Monaco; "> CHARACTER(len=256) ::name</div><div style="margin: 0px; font-size: 11px; font-family: Monaco; "> INTEGER :: do_init = 1</div><div style="margin: 0px; font-size: 11px; font-family: Monaco; "> WRITE (*,*) ' pwscf do_init= ', do_init</div><div style="margin: 0px; font-size: 11px; font-family: Monaco; min-height: 15px; "><br></div><div style="margin: 0px; font-size: 11px; font-family: Monaco; "> CALL pwscf2(comm,do_init,name,sz)</div><div style="margin: 0px; font-size: 11px; font-family: Monaco; min-height: 15px; "><br></div><div style="margin: 0px; font-size: 11px; font-family: Monaco; ">END PROGRAM pwscf</div></div><div style="margin: 0px; "><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">SUBROUTINE pwscf2 (input_comm,do_init, in_string,in_size)</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">…</div><div style="margin: 0px; "><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">USE parallel_include</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">… </div><div style="margin: 0px; "><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">INTEGER, INTENT (IN) :: input_comm</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">CHARACTER(len=256), INTENT (IN) :: in_string</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">INTEGER, INTENT (IN) :: in_size</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">INTEGER, INTENT (IN) :: do_init</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "><br></div><div style="margin: 0px; "><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">#ifdef __MPI</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "> !</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "> CALL set_mp_comm(input_comm, do_init)</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "> …</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "><br></div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">2. In parallel_include.f90 the last function in the above is defined as:</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">...</div><div style="margin: 0px; "><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">INTEGER, PUBLIC :: qe_mp_world = mpi_comm_world</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">LOGICAL, PUBLIC :: do_mpi_init = .TRUE.</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">…</div><div style="margin: 0px; "><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">CONTAINS</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; min-height: 15px; "><br></div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "> SUBROUTINE set_mp_comm(comm, do_init)</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "> IMPLICIT NONE</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "> INTEGER, INTENT(IN) :: comm</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "> INTEGER, INTENT(IN) :: do_init</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; min-height: 15px; "><br></div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "> do_mpi_init = (do_init.EQ.1)</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "> qe_mp_world = comm</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; min-height: 15px; "><br></div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "> RETURN</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "> END SUBROUTINE set_mp_comm</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "><br></div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">3. In mp.f90 and mp_global.f90 i replaced all the occurrence of mpi_comm_world with a new </div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">communicator qe_mp_world, which by default is equal to mpi_comm_world. </div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">In addition MPI_Init and MPI_Finalized are wrapped in IF statement, e.g. </div><div style="margin: 0px; "><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">IF (do_mpi_init) CALL mpi_init(ierr).</div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">We don't want to run those if MPI was initialized already externally. </div><div style="font-family: Monaco; font-size: 11px; margin: 0px; "><br></div><div style="font-family: Monaco; font-size: 11px; margin: 0px; ">4. a c++ wrapper looks like:</div><div style="margin: 0px; "><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;">#include "pw.h"</span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;"><br></span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;">extern"C" {</span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;"> void pwscf2_(int * comm,int *do_init , char *file, int * size);</span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;">}</span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;"><br></span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;">void qe_run(char *name, MPI_Comm my_comm ) </span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;">{</span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;"> MPI_Fint fcomm;</span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;"> fcomm = MPI_Comm_c2f(my_comm);</span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;"> int size = sizeof(name);</span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;"> int do_init = 0;</span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;"> pwscf2_(&fcomm, &do_init, name, &size);</span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;">}</span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;"><br></span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;">with a trivial header:</span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;"><div style="margin: 0px; ">#include "mpi.h"</div><div style="margin: 0px; ">void qe_run(char *name, MPI_Comm my_comm );</div></span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;"><br></span></font></div><div style="margin: 0px; "><font face="Monaco"><span style="font-size: 11px;">The whole thing can be run as:</span></font></div><div style="margin: 0px; "><font face="Monaco"><div style="font-size: 11px; margin: 0px; ">#include "pw.h"</div><div style="font-size: 11px; margin: 0px; ">using namespace std;</div><div style="font-size: 11px; margin: 0px; "><br></div><div style="font-size: 11px; margin: 0px; ">int</div><div style="font-size: 11px; margin: 0px; ">main(int argc, char* argv[])</div><div style="font-size: 11px; margin: 0px; ">{</div><div style="font-size: 11px; margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span>// Initialize the MPI library:</div><div style="font-size: 11px; margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span>MPI::Init(argc, argv);</div><div style="font-size: 11px; margin: 0px; "><br></div><div style="font-size: 11px; margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span>int me,nprocs;</div><div style="font-size: 11px; margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span>MPI_Comm_rank(MPI_COMM_WORLD,&me);</div><div style="font-size: 11px; margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span>MPI_Comm_size(MPI_COMM_WORLD,&nprocs);</div><div style="font-size: 11px; margin: 0px; "><br></div><div style="font-size: 11px; margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span>char name [] = "my_input.in";</div><div style="font-size: 11px; margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span>qe_run(name, MPI_COMM_WORLD);</div><div style="font-size: 11px; margin: 0px; "><br></div><div style="font-size: 11px; margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span>// Tell the MPI library to release all resources it is using:</div><div style="font-size: 11px; margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span>MPI::Finalize();</div><div style="font-size: 11px; margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span>return 0;</div><div style="font-size: 11px; margin: 0px; ">}</div><div style="font-size: 11px; margin: 0px; "><br></div><div style="font-size: 11px; margin: 0px; ">5. Compiling:</div><div style="font-size: 11px; margin: 0px; ">All the c++ files are compiled in a usual way with mpic++. </div><div style="font-size: 11px; margin: 0px; ">The wrapper-library is compiled within PW/Makefile using:</div><div style="font-size: 11px; margin: 0px; "><br></div><div style="margin: 0px; "><div style="margin: 0px; "><span style="font-size: 11px;">MPI_LINK_FLAGS = $(shell mpic++ --showme:link)</span></div><div style="margin: 0px; "><span style="font-size: 11px;">CPP_LINK_FLAGS = -L/opt/local/lib/gcc47/ -lstdc++</span></div></div><div style="margin: 0px; "><div style="margin: 0px; "><span style="font-size: 11px;">libpwc++.a : pw.x </span></div><div style="margin: 0px; "><span style="font-size: 11px;"><span class="Apple-tab-span" style="white-space: pre; "> </span>$(LD) -shared -fpic $(LDFLAGS) pw.o $(PWOBJS) libpw.a $(QEMODS) $(LIBOBJS) $(LIBS) $(MPI_LINK_FLAGS) $(CPP_LINK_FLAGS) -o $@</span></div></div><div style="font-size: 11px; margin: 0px; "><br></div><div style="font-size: 11px; margin: 0px; "><br></div><div style="font-size: 11px; margin: 0px; ">I did not yet look at how to exactly to pass the input file, </div><div style="font-size: 11px; margin: 0px; ">but that should not be a big deal anyway. </div><div style="font-size: 11px; margin: 0px; "><br></div></font></div></div></div></div></div></div></div></div></div><br><div>
<div>Kind regards,</div><div>Denis </div>
</div>
<br></body></html>