[Wannier] [EXTERNAL] Re: pw2wannier very slow on large system

[Daniel Marchand]

Hi Lee,

Thank you very much for your kind reply. Unfortunately it seems that the recent PR helps a bit, but the calculation is still quite slow. Our hope is perhaps we can take further advantage of FFW parallelization but its not clear which settings we would need to play with to do so.

Thanks again,

Daniel Marchand
Research Intern – Microsoft Station Q


From: H. Lee <hjunlee at gmail.com>
Sent: Thursday, November 19, 2020 5:10 PM
To: Daniel Marchand <t-dmarchand at microsoft.com>
Subject: Re: [Wannier] [EXTERNAL] Re: pw2wannier very slow on large system

Dear Daniel:

pw2wannier90.x doesn't support the task group parallelization; so the flag of -ntg doesn't work.
I still think that you can achieve some speedup from using the recent MR I mentioned.

Sincerely,

Hyungjun Lee
UT Austin

On Wed, Nov 18, 2020 at 4:14 PM Daniel Marchand <t-dmarchand at microsoft.com<mailto:t-dmarchand at microsoft.com>> wrote:
Hi Everyone,

Thanks for your responses. A quick question regarding the parallelization settings. Prof. Mostofi, you mention that parallelization is supported across the FFT grid, but I wanted to know if I am enabling its use correctly. So far I am not using any specific flags, as I would with pw.x. Am I correct in assuming that pw2wannier automatically adjust the settings appropriately with the number of cores available? For pw.x there is a -ntg flag to adjust FFW parallelization, yet pw2wannier does not seem to accept this as input. Is there something I’m doing wrong?

Best,

Daniel Marchand
Research Intern – Microsoft Station Q

From: H. Lee <hjunlee at gmail.com<mailto:hjunlee at gmail.com>>
Sent: Wednesday, November 18, 2020 9:18 AM
To: Daniel Marchand <t-dmarchand at microsoft.com<mailto:t-dmarchand at microsoft.com>>
Cc: wannier at lists.quantum-espresso.org<mailto:wannier at lists.quantum-espresso.org>
Subject: [EXTERNAL] Re: [Wannier] pw2wannier very slow on large system

Dear Daniel:

Regarding this issue, you could try the following recent merge request:

https://gitlab.com/QEF/q-e/-/merge_requests/1176

Sincerely,

Hyungjun Lee
UT Austin

On Mon, Nov 16, 2020 at 1:08 PM Daniel Marchand <t-dmarchand at microsoft.com<mailto:t-dmarchand at microsoft.com>> wrote:
Hi Everyone,

We’re having some trouble running pw2wannier on a very large system as it is running very slow. To give some comparison, we are able to finish the nscf calculation in ~4hours, while pw2wannier has not finished the first MMN: iknum step even after ~8 hours of runtime. It seems like in theory pw2wannier should be parallelizable but we found poor performance beyond a single node.

Is there anything we can do to help speed up the process? Parallelization steps to try out or common pitfalls that we could avoid?

Best,

Daniel
_______________________________________________
Wannier mailing list
Wannier at lists.quantum-espresso.org<mailto:Wannier at lists.quantum-espresso.org>
https://lists.quantum-espresso.org/mailman/listinfo/wannier
_______________________________________________
Wannier mailing list
Wannier at lists.quantum-espresso.org<mailto:Wannier at lists.quantum-espresso.org>
https://lists.quantum-espresso.org/mailman/listinfo/wannier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quantum-espresso.org/pipermail/wannier/attachments/20201119/78685566/attachment.html>