[Wannier] MPI version large systems

[Jonathan Backman]

Dear Jerome,

thanks for you answer.

When running using 27 MPI processes (one per k-point), 26 use about 6.6 
GB each, and 1 use 24 GB. So in total this does not even use half of the 
memory on the machine.  So, to me it does not look like a lack of memory 
is the problem.

Best regards,

Jonathan

On 26/02/2021 11:48, Jackson, Jerome (STFC,DL,SC) wrote:
> Dear Jonathan,
>
> I did some test calculation with 1685 bands and 1296 Wannier functions, so only half the size of yours, but "ballpark"-ish.  In that case the MMN and AMN were much smaller (intentionally), at 1.2G and 109M; with 1 k-point (but not using the gamma-only mode) it took about 18h in serial on some ordinary xeon machine.  I think wannierisation scales more or less like ~ n_wann^2 (and disentanglement n_wann^3).  So it should be quite a bit faster than you seem to be seeing.
>
> Do you know how much memory each MPI process is using?  How many MPI processes are you running on the 512G machine?  Could it be that memory is after all being exhausted?
> Yours,
> Jerome Jackson.
>