[Q-e-developers] calbec() hungs if # of MPI process > 1

[Ilya Ryabinkin]

Not sure if I understood how to proceed from there...
I read Sec 8.3 but didn't find answers to my issue.

Could you direct me more?

--
I.

On Tue, Nov 3, 2015 at 4:48 PM, Paolo Giannozzi <p.giannozzi at gmail.com> wrote:
>
>
> On Tue, Nov 3, 2015 at 10:16 PM, Ilya Ryabinkin <igryabinkin at gmail.com>
> wrote:
>>
>>
>> I call get_nacs() subroutine from subroutine verlet(), in particular,
>> past the line 229 of dynamics_module.f90, if the  matters.
>
>
> it matters. Subroutine verlet is called only by processor 0 ("ionode"), so
> if you try to do anything in parallel inside it, it will hang for sure. What
> is not parallel is often done on a single processor, results are then
> broadcast to the other processors. Why, it is explained in Sec.8.3. of the
> developer manual
>
> Paolo
>>
>>
>> > There isn't anything obviously wrong in your piece of code. Is it inside
>> > a
>> > modified version of pw.x, or is it stand-alone? in the latter case,
>> > there
>> > might be some missing initialization.
>>
>> It sounds relevant. Indeed, in extrapolate_wfcs() the very first (as
>> well as the very last) line is
>>
>>   CALL mp_barrier( intra_image_comm ) ! debug
>>
>> I tried to add these lines to my get_nacs() as well, but it doesn't do
>> any good: code hungs immediately, without even going past mp_barrier()
>>
>> --
>> Ilya
>>
>>
>> In the former case...it shouldn't
>> > happen. Anyway: try first of all to figure out where your code hangs.
>> > "calbec" is an interface to several different routines. Maybe it goes
>> > into
>> > the wrong one. It should just do a call to DGEMM or ZGEMM and a call to
>> > mp_sum (wrapper to mpi_reduce)
>> >
>> > Paolo
>> >
>> > On Tue, Nov 3, 2015 at 7:47 PM, Ilya Ryabinkin <igryabinkin at gmail.com>
>> > wrote:
>> >>
>> >> Dear colleagues:
>> >> I'm developing a code to calculate the non-adiabatic couplings using
>> >> finite-difference scheme. The problem I currently stumbled upon is the
>> >> following:
>> >>
>> >> I need to calculate the overlap matrix between MOs at t-dt and t,
>> >> gamma_only. What  I'm doing:
>> >>
>> >> Code is borrowed from update_pot.f90, subroutine extrapolate_wfcs(),
>> >> lines  595-600
>> >>
>> >>       ALLOCATE( evcold( npwx, nbnd ), aux( npwx, nbnd ) )
>> >>       !! Retreive old WF at t-dt from *.oldwfc file
>> >>       CALL davcio( evcold, 2*nwordwfc, iunoldwfc, 1, -1 )
>> >>       !! Get current WF at t
>> >>       aux = evc
>> >>       !
>> >>      ALLOCATE( S( nbnd, nbnd ), O( nbnd, nbnd ) )
>> >>       CALL calbec( npw, aux, evcold, S ) ! Get overlap S(t-dt, t)
>> >>       !
>> >>       O = ANINT( S )    ! O is a rounded S matrix to perform phase and
>> >>                                 ! ordering matching
>> >>       IF ( ANY( diag( O ) == 0.0_dp ) ) THEN
>> >>          ...
>> >>
>> >> Essentially, my code hungs at calbec() call if I ran it in parallell
>> >> with # of MPI process >1. gdb shows that processes are in a polling
>> >> loop, which means that they are wait for something. I'm baffled at
>> >> this point, as there is nothing they should wait for.
>> >>
>> >> What I am missing?
>> >>
>> >> Thanks for the help,
>> >> Ilya
>> >> _______________________________________________
>> >> Q-e-developers mailing list
>> >> Q-e-developers at qe-forge.org
>> >> http://qe-forge.org/mailman/listinfo/q-e-developers
>> >
>> >
>> >
>> >
>> > --
>> > Paolo Giannozzi, Dept. Chemistry&Physics&Environment,
>> > Univ. Udine, via delle Scienze 208, 33100 Udine, Italy
>> > Phone +39-0432-558216, fax +39-0432-558222
>> >
>> > _______________________________________________
>> > Q-e-developers mailing list
>> > Q-e-developers at qe-forge.org
>> > http://qe-forge.org/mailman/listinfo/q-e-developers
>> >
>>
>> _______________________________________________
>> Q-e-developers mailing list
>> Q-e-developers at qe-forge.org
>> http://qe-forge.org/mailman/listinfo/q-e-developers
>>
>
>
>
> --
> Paolo Giannozzi, Dept. Chemistry&Physics&Environment,
> Univ. Udine, via delle Scienze 208, 33100 Udine, Italy
> Phone +39-0432-558216, fax +39-0432-558222
>
> _______________________________________________
> Q-e-developers mailing list
> Q-e-developers at qe-forge.org
> http://qe-forge.org/mailman/listinfo/q-e-developers
>