[QE-users] [neb.x] How to lessen I/O?

[Takahiro Chiba]

Dear users,

P.S. to the previous post:

1. To execute with multiple machines, outdir needs to be NFS-mounted
in order to be visible from all processes.

2. In nowf case, wavefunction files exist, but 0KB in size.

---Sender---
Mr. Takahiro Chiba
Undergraduate at Hokkaido University
Expected graduation date: Mar. 2021
Expected to be a graduate student at HU from Apr. 2021
takahiro_chiba at eis.hokudai.ac.jp
---送信者---
北海道大学理学部化学科四年
千葉 卓紘
takahiro_chiba at eis.hokudai.ac.jp
-----

2021年3月17日(水) 20:20 Takahiro Chiba <takahiro_chiba at eis.hokudai.ac.jp>:
>
> Dear QE users,
>
> To lessen I/O with neb.x, what should I do? The 'outdir' grows really
> fast. I want advice from experienced users or devs.
>
> Issue:
> Although the image parallelism (neb.x -ni $NumOfNodes) is scaling well
> with the number of nodes even with gigabit ethernet, massive I/O to
> 'outdir' is observed.
> 1. Regardless of the existence of the wfcdir option, wavefunctions
> from each MPI processes (pwscf.wfc1, pwscf.wfc2, pwscf.wfc3, ...
> pwscf.wfc$ProcPerNode) are written under 'outdir' except for the first
> and the last image. Is this a bug?
> 2. When the 'disk_io' option is set to 'nowf', the neb calculation
> fails at the beginning of the second iteration because wavefunction
> files cannot be opened for writing. To execute properly, 'low' or
> above is necessary. I think this is expected by the devs, but warning
> at the beginning would be better.
>
> These issues can be reproduced on:
> a. QE 6.7, intel 19.5.281 (both MPI & compilers), CentOS 7, with LUSTRE
> a. QE 6.5, intel 19.5.281 (both MPI & compilers), CentOS 7, with LUSTRE
> b. QE 6.7, intel 19.5.281 (both MPI & compilers), CentOS 6, without LUSTRE
> c. QE 6.5, intel 18.1.038 (both MPI & compilers), CentOS 6, without LUSTRE
> ( a = Subsystem A in this page:
> https://www.hucc.hokudai.ac.jp/en/supercomputer/sc-overview/ )
> According to the batch queue system on system A, it wrote 32.7GB/hour
> when 4 x 40 processes each. The command is "mpiexec.hydra -n 160 neb.x
> -ni 4 -i $INP >>$OUT".
>
> Again, my question is, what can I do to lessen I/O other than the
> 'disk_io' option and 'wfcdir' option?
>
> ---Sender---
> Takahiro Chiba
> Undergraduate at Hokkaido University
> takahiro_chiba at eis.hokudai.ac.jp
> -----