<div dir="ltr">Dear Denis,<div><br></div><div style>this is an interesting proof of concepts actually, good job. I am personally curious to understand more about how you are planning to use PWscf as a library by coupling it with other codes (C/C++ codes presumably or pre-/post-processing tools. Sharing your view might help to understand how to improve the user usage experience of PWscf and QE in general.</div>
<div style><br></div><div style>Regards,</div><div style>Filippo</div><div style><br></div><div><br></div></div><div class="gmail_extra"><br clear="all"><div><div>--</div><div>Mr. Filippo SPIGA, M.Sc.</div><div><a href="http://filippospiga.me" target="_blank">http://filippospiga.me</a> ~ skype: filippo.spiga</div>
<div><br></div><div>«Nobody will drive us out of Cantor's paradise.» ~ David Hilbert</div><div><br></div></div>
<br><br><div class="gmail_quote">On Mon, Apr 8, 2013 at 5:35 PM, Denis Davydov <span dir="ltr"><<a href="mailto:davydden@gmail.com" target="_blank">davydden@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word"><div>Dear all, </div><div><br></div><div>After some trial and error, following advices given here, i come up with the solution on how to make a c++ library from PW with a controllable MPI communicator. Everything compiles fine with a small example, but i did not yet implement passing input file name. </div>
<div><br></div><div>In a hope that it will be useful / interesting for other users / developers, i write a quick list of how to achieve it below. </div><div>Things can definitely be made more pretty, i would be grateful for any advices. </div>
<div>Maybe a more clean version of this can make it to a release one day. I think such an interface makes it easier to combine different codes while having a full control over MPI. </div><div>.check_pw.x.j runs fine so far. I did not try anything else yet. </div>
<div><br></div><div>I hope the content below is appropriate for this mailing list, please, excuse me if it is not. </div><div><br></div><div>HOWTO: </div><div>1. pwscf.f90 -- i wrapped the main function in another function. </div>
<div><br></div><div><div style="margin:0px;font-size:11px;font-family:Monaco">PROGRAM pwscf</div></div><div><div style="margin:0px;font-size:11px;font-family:Monaco"> USE parallel_include</div><div style="margin:0px;font-size:11px;font-family:Monaco">
!</div><div style="margin:0px;font-size:11px;font-family:Monaco"> INTEGER :: comm = mpi_comm_world</div><div style="margin:0px;font-size:11px;font-family:Monaco"> INTEGER :: sz</div><div style="margin:0px;font-size:11px;font-family:Monaco">
CHARACTER(len=256) ::name</div><div style="margin:0px;font-size:11px;font-family:Monaco"> INTEGER :: do_init = 1</div><div style="margin:0px;font-size:11px;font-family:Monaco"> WRITE (*,*) ' pwscf do_init= ', do_init</div>
<div style="margin:0px;font-size:11px;font-family:Monaco;min-height:15px"><br></div><div style="margin:0px;font-size:11px;font-family:Monaco"> CALL pwscf2(comm,do_init,name,sz)</div><div style="margin:0px;font-size:11px;font-family:Monaco;min-height:15px">
<br></div><div style="margin:0px;font-size:11px;font-family:Monaco">END PROGRAM pwscf</div></div><div style="margin:0px"><div style="font-family:Monaco;font-size:11px;margin:0px">SUBROUTINE pwscf2 (input_comm,do_init, in_string,in_size)</div>
<div style="font-family:Monaco;font-size:11px;margin:0px">…</div><div style="margin:0px"><div style="font-family:Monaco;font-size:11px;margin:0px">USE parallel_include</div><div style="font-family:Monaco;font-size:11px;margin:0px">
… </div><div style="margin:0px"><div style="font-family:Monaco;font-size:11px;margin:0px">INTEGER, INTENT (IN) :: input_comm</div><div style="font-family:Monaco;font-size:11px;margin:0px">CHARACTER(len=256), INTENT (IN) :: in_string</div>
<div style="font-family:Monaco;font-size:11px;margin:0px">INTEGER, INTENT (IN) :: in_size</div><div style="font-family:Monaco;font-size:11px;margin:0px">INTEGER, INTENT (IN) :: do_init</div><div style="font-family:Monaco;font-size:11px;margin:0px">
<br></div><div style="margin:0px"><div style="font-family:Monaco;font-size:11px;margin:0px">#ifdef __MPI</div><div style="font-family:Monaco;font-size:11px;margin:0px"> !</div><div style="font-family:Monaco;font-size:11px;margin:0px">
CALL set_mp_comm(input_comm, do_init)</div><div style="font-family:Monaco;font-size:11px;margin:0px"> …</div><div style="font-family:Monaco;font-size:11px;margin:0px"><br></div><div style="font-family:Monaco;font-size:11px;margin:0px">
2. In parallel_include.f90 the last function in the above is defined as:</div><div style="font-family:Monaco;font-size:11px;margin:0px">...</div><div style="margin:0px"><div style="font-family:Monaco;font-size:11px;margin:0px">
INTEGER, PUBLIC :: qe_mp_world = mpi_comm_world</div><div style="font-family:Monaco;font-size:11px;margin:0px">LOGICAL, PUBLIC :: do_mpi_init = .TRUE.</div><div style="font-family:Monaco;font-size:11px;margin:0px">…</div>
<div style="margin:0px"><div style="font-family:Monaco;font-size:11px;margin:0px">CONTAINS</div><div style="font-family:Monaco;font-size:11px;margin:0px;min-height:15px"><br></div><div style="font-family:Monaco;font-size:11px;margin:0px">
SUBROUTINE set_mp_comm(comm, do_init)</div><div style="font-family:Monaco;font-size:11px;margin:0px"> IMPLICIT NONE</div><div style="font-family:Monaco;font-size:11px;margin:0px"> INTEGER, INTENT(IN) :: comm</div>
<div style="font-family:Monaco;font-size:11px;margin:0px"> INTEGER, INTENT(IN) :: do_init</div><div style="font-family:Monaco;font-size:11px;margin:0px;min-height:15px"><br></div><div style="font-family:Monaco;font-size:11px;margin:0px">
do_mpi_init = (do_init.EQ.1)</div><div style="font-family:Monaco;font-size:11px;margin:0px"> qe_mp_world = comm</div><div style="font-family:Monaco;font-size:11px;margin:0px;min-height:15px"><br></div><div style="font-family:Monaco;font-size:11px;margin:0px">
RETURN</div><div style="font-family:Monaco;font-size:11px;margin:0px"> END SUBROUTINE set_mp_comm</div><div style="font-family:Monaco;font-size:11px;margin:0px"><br></div><div style="font-family:Monaco;font-size:11px;margin:0px">
3. In mp.f90 and mp_global.f90 i replaced all the occurrence of mpi_comm_world with a new </div><div style="font-family:Monaco;font-size:11px;margin:0px">communicator qe_mp_world, which by default is equal to mpi_comm_world. </div>
<div style="font-family:Monaco;font-size:11px;margin:0px">In addition MPI_Init and MPI_Finalized are wrapped in IF statement, e.g. </div><div style="margin:0px"><div style="font-family:Monaco;font-size:11px;margin:0px">IF (do_mpi_init) CALL mpi_init(ierr).</div>
<div style="font-family:Monaco;font-size:11px;margin:0px">We don't want to run those if MPI was initialized already externally. </div><div style="font-family:Monaco;font-size:11px;margin:0px"><br></div><div style="font-family:Monaco;font-size:11px;margin:0px">
4. a c++ wrapper looks like:</div><div style="margin:0px"><div style="margin:0px"><font face="Monaco"><span style="font-size:11px">#include "pw.h"</span></font></div><div style="margin:0px"><font face="Monaco"><span style="font-size:11px"><br>
</span></font></div><div style="margin:0px"><font face="Monaco"><span style="font-size:11px">extern"C" {</span></font></div><div style="margin:0px"><font face="Monaco"><span style="font-size:11px"> void pwscf2_(int * comm,int *do_init , char *file, int * size);</span></font></div>
<div style="margin:0px"><font face="Monaco"><span style="font-size:11px">}</span></font></div><div style="margin:0px"><font face="Monaco"><span style="font-size:11px"><br></span></font></div><div style="margin:0px"><font face="Monaco"><span style="font-size:11px">void qe_run(char *name, MPI_Comm my_comm ) </span></font></div>
<div style="margin:0px"><font face="Monaco"><span style="font-size:11px">{</span></font></div><div style="margin:0px"><font face="Monaco"><span style="font-size:11px"> MPI_Fint fcomm;</span></font></div><div style="margin:0px">
<font face="Monaco"><span style="font-size:11px"> fcomm = MPI_Comm_c2f(my_comm);</span></font></div><div style="margin:0px"><font face="Monaco"><span style="font-size:11px"> int size = sizeof(name);</span></font></div>
<div style="margin:0px"><font face="Monaco"><span style="font-size:11px"> int do_init = 0;</span></font></div><div style="margin:0px"><font face="Monaco"><span style="font-size:11px"> pwscf2_(&fcomm, &do_init, name, &size);</span></font></div>
<div style="margin:0px"><font face="Monaco"><span style="font-size:11px">}</span></font></div><div style="margin:0px"><font face="Monaco"><span style="font-size:11px"><br></span></font></div><div style="margin:0px"><font face="Monaco"><span style="font-size:11px">with a trivial header:</span></font></div>
<div style="margin:0px"><font face="Monaco"><span style="font-size:11px"><div style="margin:0px">#include "mpi.h"</div><div style="margin:0px">void qe_run(char *name, MPI_Comm my_comm );</div></span></font></div>
<div style="margin:0px"><font face="Monaco"><span style="font-size:11px"><br></span></font></div><div style="margin:0px"><font face="Monaco"><span style="font-size:11px">The whole thing can be run as:</span></font></div>
<div style="margin:0px">
<font face="Monaco"><div style="font-size:11px;margin:0px">#include "pw.h"</div><div style="font-size:11px;margin:0px">using namespace std;</div><div style="font-size:11px;margin:0px"><br></div><div style="font-size:11px;margin:0px">
int</div><div style="font-size:11px;margin:0px">main(int argc, char* argv[])</div><div style="font-size:11px;margin:0px">{</div><div style="font-size:11px;margin:0px"><span style="white-space:pre-wrap"> </span>// Initialize the MPI library:</div>
<div style="font-size:11px;margin:0px"><span style="white-space:pre-wrap"> </span>MPI::Init(argc, argv);</div><div style="font-size:11px;margin:0px"><br></div><div style="font-size:11px;margin:0px"><span style="white-space:pre-wrap"> </span>int me,nprocs;</div>
<div style="font-size:11px;margin:0px"><span style="white-space:pre-wrap"> </span>MPI_Comm_rank(MPI_COMM_WORLD,&me);</div><div style="font-size:11px;margin:0px"><span style="white-space:pre-wrap"> </span>MPI_Comm_size(MPI_COMM_WORLD,&nprocs);</div>
<div style="font-size:11px;margin:0px"><br></div><div style="font-size:11px;margin:0px"><span style="white-space:pre-wrap"> </span>char name [] = "<a href="http://my_input.in" target="_blank">my_input.in</a>";</div>
<div style="font-size:11px;margin:0px"><span style="white-space:pre-wrap"> </span>qe_run(name, MPI_COMM_WORLD);</div><div style="font-size:11px;margin:0px"><br></div><div style="font-size:11px;margin:0px"><span style="white-space:pre-wrap"> </span>// Tell the MPI library to release all resources it is using:</div>
<div style="font-size:11px;margin:0px"><span style="white-space:pre-wrap"> </span>MPI::Finalize();</div><div style="font-size:11px;margin:0px"><span style="white-space:pre-wrap"> </span>return 0;</div><div style="font-size:11px;margin:0px">
}</div><div style="font-size:11px;margin:0px"><br></div><div style="font-size:11px;margin:0px">5. Compiling:</div><div style="font-size:11px;margin:0px">All the c++ files are compiled in a usual way with mpic++. </div><div style="font-size:11px;margin:0px">
The wrapper-library is compiled within PW/Makefile using:</div><div style="font-size:11px;margin:0px"><br></div><div style="margin:0px"><div style="margin:0px"><span style="font-size:11px">MPI_LINK_FLAGS = $(shell mpic++ --showme:link)</span></div>
<div style="margin:0px"><span style="font-size:11px">CPP_LINK_FLAGS = -L/opt/local/lib/gcc47/ -lstdc++</span></div></div><div style="margin:0px"><div style="margin:0px"><span style="font-size:11px">libpwc++.a : pw.x </span></div>
<div style="margin:0px"><span style="font-size:11px"><span style="white-space:pre-wrap"> </span>$(LD) -shared -fpic $(LDFLAGS) pw.o $(PWOBJS) libpw.a $(QEMODS) $(LIBOBJS) $(LIBS) $(MPI_LINK_FLAGS) $(CPP_LINK_FLAGS) -o $@</span></div>
</div><div style="font-size:11px;margin:0px"><br></div><div style="font-size:11px;margin:0px"><br></div><div style="font-size:11px;margin:0px">I did not yet look at how to exactly to pass the input file, </div><div style="font-size:11px;margin:0px">
but that should not be a big deal anyway. </div><div style="font-size:11px;margin:0px"><br></div></font></div></div></div></div></div></div></div></div></div><br><div>
<div>Kind regards,</div><div>Denis </div>
</div>
<br></div><br>_______________________________________________<br>
Pw_forum mailing list<br>
<a href="mailto:Pw_forum@pwscf.org">Pw_forum@pwscf.org</a><br>
<a href="http://pwscf.org/mailman/listinfo/pw_forum" target="_blank">http://pwscf.org/mailman/listinfo/pw_forum</a><br></blockquote></div><br></div>