<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div class="">Hi Ye,</div>
<div class=""><br class="">
</div>
<div class="">The changes I am making are part of the EPW/QE code and so the parallelization depends on the QE MPI routines. I have no problem using mp_sum for a 2-D array such as</div>
<div class="">energy(iband,ik). I can use mp_sum to add the contributions from various nodes and print the total in agreement with the output from the serial run. As far as the allocation goes, I use the same 2-D approach to my 3-D array.</div>
<div class=""><br class="">
</div>
<div class="">The problem arises when I use the 3-D array with mp_sum. I noticed that in the Modules/mp.f90 file, there is a subroutine as follows:</div>
<div class=""><br class="">
</div>
<div class=""> SUBROUTINE mp_sum_rt( msg, gid )<br class="">
IMPLICIT NONE<br class="">
REAL (DP), INTENT (INOUT) :: msg(:,:,:)<br class="">
INTEGER, INTENT(IN) :: gid<br class="">
#if defined(__MPI)<br class="">
INTEGER :: msglen<br class="">
msglen = size(msg)<br class="">
CALL reduce_base_real( msglen, msg, gid, -1 )<br class="">
#endif<br class="">
END SUBROUTINE mp_sum_rt<br class="">
<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Here msg has three dimensions. There are other subroutines for 1, 2, and 4 dimensions. Somehow, if I can get EPW to use the above subroutine, it might work. Adding a statement like:</div>
<div class=""><br class="">
</div>
<div class="">USE mp, ONLY : mp_sum_rt</div>
<div class=""><br class="">
</div>
<div class="">at the beginning of the file does not work as the compilation fails because EPW does not see the subroutine. Interestingly, it does see mp_sum which is in the same mp.f90 file.</div>
<div class=""><br class="">
</div>
<div class="">So is there a way that I can get EPW to see mp_sum_rt? This might solve my problem. </div>
<div class=""><br class="">
</div>
<div class="">Since I know that mp_sum works with 2-D arrays, one option is to rewrite my files in 2-D, one array for each x,y,z direction. I am trying to avoid this.</div>
<div class=""><br class="">
</div>
<div class="">Thanks,</div>
<div class=""><br class="">
</div>
<div class="">Vahid</div>
<div class=""><br class="">
</div>
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Oct 30, 2016, at 3:59 PM, Ye Luo <<a href="mailto:xw111luoye@gmail.com" class="">xw111luoye@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<div class="">
<div class="">
<div class="">Hi Vahid,<br class="">
</div>
segfault in mp_sum doesn't necessarily mean a problem there. Probably you wrote something to output array but not in a valid place before mp_sum.<br class="">
</div>
Try to check your allocation of output and the copy make sure they are correct.<br class="">
<br class="">
</div>
Ye<br class="">
<div class="">
<div class=""><br class="">
</div>
</div>
</div>
<div class="gmail_extra" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br clear="all" class="">
<div class="">
<div class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr" class="">===================<br class="">
Ye Luo, Ph.D.<br class="">
Leadership Computing Facility<br class="">
Argonne National Laboratory</div>
</div>
</div>
<br class="">
<div class="gmail_quote">2016-10-28 14:33 GMT-05:00 Vahid Askarpour<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:vh261281@dal.ca" target="_blank" class="">vh261281@dal.ca</a>></span>:<br class="">
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;">
<div style="word-wrap: break-word;" class="">
<div class="">Hi Ye,</div>
<div class=""><br class="">
</div>
<div class="">Thank you for your suggestion. I tried it and when I ran the code, it seg-faulted. I put flags in the code to see where the segmentation faults occurs. It happens as the code calls mp_sum. It seems that mp_sum may not</div>
<div class="">be able to handle this reduction.</div>
<div class=""><br class="">
</div>
<div class="">Cheers,</div>
<div class="">Vahid</div>
<div class="">
<div class="h5"><br class="">
<div class="">
<blockquote type="cite" class="">
<div class="">On Oct 28, 2016, at 2:51 PM, Ye Luo <<a href="mailto:xw111luoye@gmail.com" target="_blank" class="">xw111luoye@gmail.com</a>> wrote:</div>
<br class="m_2808508241249506849Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">
<div class="">
<div class="">In Fortran, whatever-D array is 1-D array. mp_sum should be fine.<br class="">
</div>
I saw something strange in your code that you were not copying the right things as you expected.<br class="">
How about the following?<br class="">
output(1:3,1:nbnds,(k_pool*<wbr class="">pool_id+1:k_pool*pool_id+k_<wbr class="">pool))=input(1:3,1:nbnds,1:k_<wbr class="">pool)<br class="">
</div>
<br class="">
Ye<br class="">
<div class="">
<div class="gmail_extra"><br clear="all" class="">
<div class="">
<div class="m_2808508241249506849gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr" class="">===================<br class="">
Ye Luo, Ph.D.<br class="">
Leadership Computing Facility<br class="">
Argonne National Laboratory</div>
</div>
</div>
<br class="">
<div class="gmail_quote">2016-10-28 12:29 GMT-05:00 Vahid Askarpour<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:vh261281@dal.ca" target="_blank" class="">vh261281@dal.ca</a>></span>:<br class="">
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;">
Dear QE Users,<br class="">
<br class="">
I am working on some modifications to the QE-6.0 code using symmetry. When I try to combine a 3-D array scattered across nodes, I use the following:<br class="">
<br class="">
output(3,nbnds,(k_pool*pool_id<wbr class="">+1:k_pool*pool_id+k_pool))=inp<wbr class="">ut(3,nbnds,1:k_pool)<br class="">
<br class="">
Here, nbnds is the number of bands, k_pool is the number of k points/pool, and pool_id is the id of the pool. Here I am assuming the the number of k points is divisible by the number of pools.<br class="">
<br class="">
Then I call mp_sum(output,inter_pool_comm) to put all the segments of input across the nodes into one output file.<br class="">
<br class="">
When I run the modified QE code in parallel, the output file is different from the serial run.<br class="">
<br class="">
Does the QE's mp_sum allow the above operation for a three-D array?<br class="">
<br class="">
Any hints or suggestions would be greatly appreciated.<br class="">
<br class="">
Vahid<br class="">
<span class="m_2808508241249506849HOEnZb"><font color="#888888" class=""><br class="">
Vahid Askarpour<br class="">
Department of Physics and Atmospheric Science<br class="">
Dalhousie University,<br class="">
Halifax, NS, Canada<br class="">
______________________________<wbr class="">_________________<br class="">
Pw_forum mailing list<br class="">
<a href="mailto:Pw_forum@pwscf.org" target="_blank" class="">Pw_forum@pwscf.org</a><br class="">
<a href="http://pwscf.org/mailman/listinfo/pw_forum" rel="noreferrer" target="_blank" class="">http://pwscf.org/mailman/listi<wbr class="">nfo/pw_forum</a><br class="">
</font></span></blockquote>
</div>
<br class="">
</div>
</div>
</div>
______________________________<wbr class="">_________________<br class="">
Pw_forum mailing list<br class="">
<a href="mailto:Pw_forum@pwscf.org" target="_blank" class="">Pw_forum@pwscf.org</a><br class="">
<a href="http://pwscf.org/mailman/listinfo/pw_forum" target="_blank" class="">http://pwscf.org/mailman/<wbr class="">listinfo/pw_forum</a></div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
<br class="">
______________________________<wbr class="">_________________<br class="">
Pw_forum mailing list<br class="">
<a href="mailto:Pw_forum@pwscf.org" class="">Pw_forum@pwscf.org</a><br class="">
<a href="http://pwscf.org/mailman/listinfo/pw_forum" rel="noreferrer" target="_blank" class="">http://pwscf.org/mailman/<wbr class="">listinfo/pw_forum</a><br class="">
</blockquote>
</div>
<br class="">
</div>
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Pw_forum
mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<a href="mailto:Pw_forum@pwscf.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Pw_forum@pwscf.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<a href="http://pwscf.org/mailman/listinfo/pw_forum" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">http://pwscf.org/mailman/listinfo/pw_forum</a></div>
</blockquote>
</div>
<br class="">
</body>
</html>